Skip to content

$dateToString aggregation with timezone outputs invalid ISO8601 string #1768

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

$dateToString aggregation with timezone outputs invalid ISO8601 string #1768

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
281c7a3
make format optional and create testing case about $dateToString aggr…
NathanQingyangXu Jul 17, 2025
9b6b7a3
Update driver-core/src/test/functional/com/mongodb/client/model/mql/T…
NathanQingyangXu Jul 17, 2025
f4f59bd
code change as per review comments
NathanQingyangXu Jul 17, 2025
0954ac5
Update driver-core/src/main/com/mongodb/client/model/mql/MqlExpressio…
NathanQingyangXu Jul 18, 2025
3a4b76b
follow up of the last suggestion commit
NathanQingyangXu Jul 18, 2025
8f68b79
remove unnecessary new method to MqlDate
NathanQingyangXu Jul 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions driver-core/src/main/com/mongodb/client/model/mql/MqlDate.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,20 @@ public interface MqlDate extends MqlValue {
*/
MqlString asString(MqlString timezone, MqlString format);

/**
* The string representation of {@code this} date as determined by the
* provided {@code timezone} with default format as below:
*
* <ul>
* <li>If {@code timezone} is UTC timezone, {@code "%Y-%m-%dT%H:%M:%S.%LZ"} will be the default
* <li>Otherwise the default format will be {@code "%Y-%m-%dT%H:%M:%S.%L"}
* </ul>
*
* @param timezone the UTC Offset or Olson Timezone Identifier.
* @return the resulting value.
*/
MqlString asString(MqlString timezone);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why add this to the external API? This would expose a method that would always fail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed. I misunderstood something.


/**
* The result of passing {@code this} value to the provided function.
* Equivalent to {@code f.apply(this)}, and allows lambdas and static,
Expand Down
23 changes: 19 additions & 4 deletions driver-core/src/main/com/mongodb/client/model/mql/MqlExpression.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
package com.mongodb.client.model.mql;

import com.mongodb.assertions.Assertions;
import com.mongodb.lang.Nullable;
import org.bson.BsonArray;
import org.bson.BsonDocument;
import org.bson.BsonInt32;
Expand Down Expand Up @@ -947,10 +948,24 @@ public MqlInteger millisecond(final MqlString timezone) {
public MqlString asString(final MqlString timezone, final MqlString format) {
Assertions.notNull("timezone", timezone);
Assertions.notNull("format", format);
return newMqlExpression((cr) -> astDoc("$dateToString", new BsonDocument()
.append("date", this.toBsonValue(cr))
.append("format", toBsonValue(cr, format))
.append("timezone", toBsonValue(cr, timezone))));
return doAsString(timezone, format);
}

@Override
public MqlString asString(final MqlString timezone) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that we do not need this method, and that it might actually be introducing the bug described in the Jira. That is, the API as it is now intentionally prevents the user from specifying a timezone without also forcing that user to explicitly specify a format, thus preventing the situation from arising.

For reference:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for sharing. Yeah, also saw you raising the point in our team slack channel. I have no objection of closing this PR.

Assertions.notNull("timezone", timezone);
return doAsString(timezone, null);
}

private MqlString doAsString(final MqlString timezone, @Nullable final MqlString format) {
return newMqlExpression((cr) -> {
BsonDocument document = new BsonDocument("date", this.toBsonValue(cr));
if (format != null) {
document.append("format", toBsonValue(cr, format));
}
document.append("timezone", toBsonValue(cr, timezone));
return astDoc("$dateToString", document);
});
}

@Override
Expand Down
30 changes: 30 additions & 0 deletions ...er-core/src/test/functional/com/mongodb/client/model/mql/TypeMqlValuesFunctionalTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
import static com.mongodb.client.model.mql.MqlValues.ofIntegerArray;
import static com.mongodb.client.model.mql.MqlValues.ofMap;
import static com.mongodb.client.model.mql.MqlValues.ofNull;
import static java.time.format.DateTimeFormatter.ISO_INSTANT;
import static java.time.format.DateTimeFormatter.ISO_LOCAL_DATE_TIME;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assumptions.assumeTrue;
Expand Down Expand Up @@ -225,6 +226,35 @@ public void dateAsStringTest() {
of(instant).asString(of("America/New_York"), of("%Y-%m-%dT%H:%M:%S.%L%z")));
}

/**
* Tests that with default format and non-UTC timezone, {@code $dateToString} won't output invalid ISO 8601 string ending with 'Z'.
* Ticket: JAVA-5044
*/
@Test
void dateAsStringWithDefaultFormat() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is testing a new API. But, since this is a bugfix, I would expect it to test an existing API, where the test would fail before the prod change, and pass afterward. Is that not the case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, correct. The testing would only succeed for v7.1..0+; it would fail for v6, for instance.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not referring to the difference in test outcome based on the server version. The server would write such a test.

I mean, the ticket suggests that we need to make changes in the driver. When we write a regression test (or, all tests actually), the idea is that the test should not work before the change, but should work after. If the test works both before and after the change, the test has no relation to the change and is not testing it. If it fails both times, the change did not fix the issue.

Before the change to the production API, it is not actually possible to write a test that exhibits the bug via our API. So, no error. After the change, we can now write a test that exhibits the bug. But, we block the test from running on the server version necessary for the bug to show up. So things are "green" at first for that version, but after the prod code change, they are "red". This means we introduced a bug.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I didn't misunderstand, you said the same testing case would fail or be red on v6, correct? That is true and I understand that is an issue.

// the bug was reported on v6.0.3, but was fixed in v7.1 without backporting
// due to the concern that it is a breaking change.
assumeTrue(serverVersionAtLeast(7, 1));

final Instant instant = Instant.parse("2025-07-17T18:00:36.052Z");
MqlDate date = of(instant);
ZonedDateTime utcDateTime = ZonedDateTime.ofInstant(instant, ZoneId.of(ZoneOffset.UTC.getId()));

// UTC timezone's default format should be "%Y-%m-%dT%H:%M:%S.%LZ"
assertExpression(
utcDateTime.format(ISO_INSTANT),
date.asString(of("UTC")),
"{'$dateToString': {'date': {'$date': '2025-07-17T18:00:36.052Z'}, "
+ "'timezone': 'UTC'}}");

// non-UTC timezone's default format should be "%Y-%m-%dT%H:%M:%S.%L"
assertExpression(
utcDateTime.withZoneSameInstant(ZoneId.of("America/New_York")).format(ISO_LOCAL_DATE_TIME),
date.asString(of("America/New_York")),
"{'$dateToString': {'date': {'$date': '2025-07-17T18:00:36.052Z'}, "
+ "'timezone': 'America/New_York'}}");
}

// parse string

@Test
Expand Down