Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,11 @@ public List<T> children() {
@SuppressWarnings("unchecked")
public void forEachDown(Consumer<? super T> action) {
action.accept((T) this);
children().forEach(c -> c.forEachDown(action));
// please do not refactor it to a for-each loop to avoid
// allocating iterator that performs concurrent modification checks and extra stack traces
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
// allocating iterator that performs concurrent modification checks and extra stack traces
// allocating iterator that performs concurrent modification checks and extra stack frames

for (int c = 0, size = children.size(); c < size; c++) {
children.get(c).forEachDown(action);
}
}

@SuppressWarnings("unchecked")
Expand All @@ -81,7 +85,11 @@ public <E extends T> void forEachDown(Class<E> typeToken, Consumer<? super E> ac

@SuppressWarnings("unchecked")
public void forEachUp(Consumer<? super T> action) {
children().forEach(c -> c.forEachUp(action));
// please do not refactor it to a for-each loop to avoid
// allocating iterator that performs concurrent modification checks and extra stack traces
for (int c = 0, size = children.size(); c < size; c++) {
children.get(c).forEachUp(action);
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

forEachDown and forEachUp do not seem to have visible impact on profile I reviewed, however replacing them to loops should make profiles outputs several stacks shorter, especially when those are nested.

Screenshot at 2025-03-19 15-21-23

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add this as a comment, please? The explanation in the comment helps, but I think the important point is that this reduces stack size rather than increasing performance, which the current comment kinda implies by mentioning the concurrent modification checks - which I understand didn't seem to be a problem?

action.accept((T) this);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,24 @@ private static Object doTransformExpression(Object arg, Function<Expression, ? e
// preserving the type information is hacky and weird (a lot of context needs to be passed around and the lambda itself
// has no type info so it's difficult to have automatic checking without having base classes).

if (arg instanceof Collection<?> c) {
if (arg instanceof List<?> c) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please confirm we do not expect sets in here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't however it's not enforced - add an else throwing an error for safety.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ this could easily become a bug in the future, as it makes total sense for a node property to be e.g. a set of expressions. We might also just not have any test that exercises this code path with a non-list collection. To really know, we'd have to go and check all the info() method implementations on Node subclasses manually. And enforce that the only collection that node properties can use are lists.

My suggestion: let's add a code path for List<?> in addition to the more general code path for Collection<?> - and let's add an assertion in the latter because that code path shouldn't be used, so that this only trips in tests but doesn't damage production code.

List<Object> transformed = null;
boolean hasChanged = false;
// please do not refactor it to a for-each loop to avoid
// allocating iterator that performs concurrent modification checks and extra stack traces
for (int i = 0, size = c.size(); i < size; i++) {
var e = c.get(i);
Copy link
Contributor Author

@idegtiarenko idegtiarenko Mar 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ArrayList.Itr is surprisingly visible in profile (those are blue rectangles on top left of the pink ones) when executing this code (maybe because it spends time checking for concurrent modifications?).
Replacing it with a for loop should make it a bit cheaper.

Screenshot at 2025-03-19 15-23-26

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're making more and more improvements to how we perform query plan transformations/traversals.

Could we please add at least a micro benchmark as one of the next steps? Without one, it's both hard to understand the impact of optimizations, and it's also unclear if we maybe introduce accidental regressions (in whichever PRs).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I will try to add one in a separate change.

Object next = doTransformExpression(e, traversal);
if (e.equals(next) == false) {
if (hasChanged == false) {
hasChanged = true;
transformed = new ArrayList<>(c);
}
transformed.set(i, next);
}
}
return hasChanged ? transformed : arg;
} else if (arg instanceof Collection<?> c) {
List<Object> transformed = null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine to mark this branch as deprecated/likely dead + we could still add an assert false here - before maybe later getting rid of this branch altogether and enforcing that the only collections used as properties are all lists (but in a more visible place, e.g. in the c'tor of QueryPlan/Node).

boolean hasChanged = false;
int i = 0;
Expand All @@ -149,13 +166,14 @@ private static Object doTransformExpression(Object arg, Function<Expression, ? e
if (e.equals(next) == false) {
if (hasChanged == false) {
hasChanged = true;
// TODO if collection is a set then this silently changes its semantics by allowing duplicates
// We should fix it or confirm this branch is never needed
transformed = new ArrayList<>(c);
}
transformed.set(i, next);
}
i++;
}

return hasChanged ? transformed : arg;
}

Expand Down