-
Notifications
You must be signed in to change notification settings - Fork 25.6k
ESQL: Locale and timezone argument for date_parse #136548
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
ESQL: Locale and timezone argument for date_parse #136548
Conversation
...ql/src/main/java/org/elasticsearch/xpack/esql/expression/function/scalar/date/DateParse.java
Outdated
Show resolved
Hide resolved
ℹ️ Important: Docs version tagging👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version. We use applies_to tags to mark version-specific features and changes. Expand for a quick overviewWhen to use applies_to tags:✅ At the page level to indicate which products/deployments the content applies to (mandatory) What NOT to do:❌ Don't remove or replace information that applies to an older version 🤔 Need help?
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
Hi @flash1293, I've created a changelog YAML for you. |
assertThat(e.getMessage(), startsWith("invalid date pattern for []: Invalid format: [" + pattern + "]")); | ||
} | ||
|
||
public void testInvalidLocale() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add those tests as TestCaseSupplier
s in the parameters()
method? They automatically execute a bunch of cases.
I'm not sure of how map parameters work there really, but if it works like other params,you would be able to just test that it returns null
with a warning with the error, which is what the user would see
second = randomValueOtherThan(second, () -> randomBoolean() ? null : randomChild()); | ||
} | ||
return new DateParse(source, first, second); | ||
Expression options = instance.children().size() == 3 ? instance.children().get(2) : null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We usually use a switch here, to only change 1 field. For example:
Lines 26 to 38 in c40c5a6
@Override | |
protected Top mutateInstance(Top instance) throws IOException { | |
Source source = instance.source(); | |
Expression field = instance.field(); | |
Expression limit = instance.limitField(); | |
Expression order = instance.orderField(); | |
switch (between(0, 2)) { | |
case 0 -> field = randomValueOtherThan(field, AbstractExpressionSerializationTests::randomChild); | |
case 1 -> limit = randomValueOtherThan(limit, AbstractExpressionSerializationTests::randomChild); | |
case 2 -> order = randomValueOtherThan(order, AbstractExpressionSerializationTests::randomChild); | |
} | |
return new Top(source, field, limit, order); | |
} |
Expression first = randomChild(); | ||
Expression second = randomBoolean() ? null : randomChild(); | ||
return new DateParse(source, first, second); | ||
Expression options = second != null && randomBoolean() ? randomChild() : null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't we create the options even if "second" is null?
super(source, fields(first, second, options)); | ||
this.field = second != null ? second : first; | ||
this.format = second != null ? first : null; | ||
this.options = options; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This troubles me a bit. We're only allowing the options if the 2 previous parameters are present, even knowing that the format is optional too. That means, we're not allowing something like: DATE_PARSE(date, {})
However, the SVG (docs) shows it correctly.
Now, I don't know if we did something like this before, or if we should allow it. Our function overriding detection is quite manual right now, to begin with. I would like if somebody else from the team can review this first.
The worse that could happen if we ship this is that:
- We would have incorrect docs
- We would probably give meaningless errors, as users would expect the map parameter to "work"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good call... To me both allowing DATE_PARSE(date, {})
or enforcing three params for options make sense, if we clearly communicate this to the user.
Happy for someone else to make the call.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For reference, DATE_PARSE(date_string, {\"time_zone\": \"Europe/Paris\",\"locale\":\"fr\"})
yields
{
"error": {
"root_cause": [
{
"type": "verification_exception",
"reason": "Found 1 problem\nline 1:51: second argument of [DATE_PARSE(date_string, {\"time_zone\": \"Europe/Paris\",\"locale\":\"fr\"})] must be [string], found value [{\"time_zone\": \"Europe/Paris\",\"locale\":\"fr\"}] type [unsupported]"
}
],
"type": "verification_exception",
"reason": "Found 1 problem\nline 1:51: second argument of [DATE_PARSE(date_string, {\"time_zone\": \"Europe/Paris\",\"locale\":\"fr\"})] must be [string], found value [{\"time_zone\": \"Europe/Paris\",\"locale\":\"fr\"}] type [unsupported]"
},
"status": 400
}
right now.
Your call of course, but if you feel comfortable with this behavior, we can get it in, then follow up later with supporting a more flexible calling pattern - we are lucky that the existing behavior on this PR is a subset of what could realistically be supported.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm fine with closing this as-is. I would try to follow up soon though, as to not have the wrong docs there.
Btw, I was discussing this a bit, and I think we could make a working check here. Something like:
if (options == null) {
if (second == null) {
// 1 parameter, it's the date
} else {
if (second instanceof MapExpression) {
// Second and options params, no format
} else {
// First and second params, no options
}
}
} else {
// 3 params available, no doubt here
}
In general, having an optional param before a required is quite weird. But this is ""historical"" already, so here we are. For this special case, I think the logic to check it shouldn't be too complex, and we can manage to do it.
If there's some weird planning error after doing it, we can check it 👀
…3/elasticsearch into flash1293/date-parsing-settings
Fixes #132487
returns
In case the time zone or the locale can't be resolved, this throws an error.