Correlated EXISTS #257

afs · 2025-08-13T16:14:17Z

TallTed

Incomplete, but better than no review... Some clear fixes; some less clear questions.

TallTed · 2025-08-13T19:17:18Z

spec/index.html

@@ -8849,16 +8849,27 @@ <h4>Variable Scope</h4>
                <td><code>VALUES varlist { values }</code></td>
                <td><code>v</code> is in-scope if <code>v</code> is in <code>varlist</code></td>
              </tr>
+              <tr>
+                <td>`EXISTS` and `NOT EXISTS` filters</td>


Suggested change

<td>`EXISTS` and `NOT EXISTS` filters</td>

<td>`FILTERs` using `EXISTS` and `NOT EXISTS`</td>

First, to @TallTed's edit suggestion: since FILTERs is not a keyword in SPARQL, I would not code-fence it. Additionally, for this rephrasing, it should be "or" instead of "and". So, better something like:

Suggested change

<td>`EXISTS` and `NOT EXISTS` filters</td>

<td>`FILTER` statements using `EXISTS` or `NOT EXISTS`</td>

or:

Suggested change

<td>`EXISTS` and `NOT EXISTS` filters</td>

<td>`FILTER` containing `EXISTS` or `NOT EXISTS`</td>

Now, more generally: Is this really only about filters? Shouldn't BIND with EXISTS or NOT EXISTS be considered as well?

In fact, I wonder why EXISTS and NOT EXISTS need to be mentioned at all here.

For every FILTER expr, it holds that no variable is in-scope, no matter whether expr contains EXISTS/NOT EXISTS or not. The notion of in-scope variables is about variables that may be in a solution mapping produced for a query construct and a FILTER itself does not result in solution mappings. So, no variable can be in-scope for a FILTER. Of course, there may be variables that are in-scope for a group pattern that contains a FILTER, but that case is covered by the Group-related row above in this table.

since `FILTERs is not a keyword in SPARQL

Is your thinking that FILTERs (plural) is not a SPARQL keyword, wile FILTER (singular) is? Yes, you're correct. However, codefencing the singular and trailing it with a non-codefenced s to produce the plural, "FILTERs" generally looks awful, and often (if not always), a search on filters will not hit on "FILTERs", which I find terrible as a reader.

That said, I think your "FILTER statements" is better. I might suggest "FILTER clauses" or "FILTER expressions", since these do not stand on their own.

(I note that "filter" and "FILTER" were treated inconsistently in SPARQL 1.1 Query Language. I think "filter" should be avoided as a noun, and should generally be replaced by "FILTER clauses", always including the codefence around FILTER.)

Shouldn't BIND with EXISTS or NOT EXISTS be considered as well?

Quite possibly. At first blush, I would guess that the answer is "yes".

I wonder why EXISTS and NOT EXISTS need to be mentioned at all here

I will be interested in the result of this question. It might deserve its own issue.

since `FILTERs is not a keyword in SPARQL

Is your thinking that FILTERs (plural) is not a SPARQL keyword, wile FILTER (singular) is? Yes, you're correct.

Indeed, that's my thinking.

However, codefencing the singular and trailing it with a non-codefenced s to produce the plural, "FILTERs" generally looks awful, and often (if not always), a search on filters will not hit on "FILTERs", which I find terrible as a reader.

I totally agree.

That said, I think your "FILTER statements" is better. I might suggest "FILTER clauses" or "FILTER expressions", since these do not stand on their own.

These would be fine for me as well.

(I note that "filter" and "FILTER" were treated inconsistently in SPARQL 1.1 Query Language. I think "filter" should be avoided as a noun, and should generally be replaced by "FILTER clauses", always including the codefence around FILTER.)

Yes! But that is something to be implemented in a separate PR.

Now in #273

TallTed · 2025-08-13T19:18:13Z

spec/index.html

+          <p>The variable <code>v</code> must not be in-scope at the point of the 
+            <code>(expr AS v)</code> form. The scoping for <code>(expr AS v)</code>
+            applies immediately in <code>SELECT</code> expressions.
+          </p>
          <p>In <code>BIND (expr AS v)</code> requires that the variable <code>v</code> is not


Suggested change

<p>In <code>BIND (expr AS v)</code> requires that the variable <code>v</code> is not

<p><code>BIND (expr AS v)</code> requires that the variable <code>v</code> is not

TallTed · 2025-08-13T19:19:55Z

spec/index.html

          <p>In <code>SELECT</code>, the variable <code>v</code> must not be in-scope in the graph
            pattern of the <code>SELECT</code> clause, nor used in another select expression earlier in


This appears to be non-normative text. I am therefore writing to replace the lowercase RFC2119 language.

Suggested change

<p>In <code>SELECT</code>, the variable <code>v</code> must not be in-scope in the graph

pattern of the <code>SELECT</code> clause, nor used in another select expression earlier in

<p>In <code>SELECT</code>, the variable <code>v</code> cannot be in-scope in the graph

pattern of the <code>SELECT</code> clause, nor have been used in another select expression earlier in

TallTed · 2025-08-13T19:22:45Z

spec/index.html

+                <div class="ednote" id="note-BindingInScope">
+                  <p>
+                    One way to provide `BindingInScope` is to change `eval` to also have the current row
+                    as an argument or a "null" token. . Normally this the "null" token. This would be set in


Suggested change

as an argument or a "null" token. . Normally this the "null" token. This would be set in

as an argument or a "null" token, and is normally the "null" token. This would be set in

or

Suggested change

as an argument or a "null" token. . Normally this the "null" token. This would be set in

as an argument or a "null" token, and is normally the latter. This would be set in

TallTed · 2025-08-13T19:26:55Z

spec/index.html

+                    Another way is to have a global (to the execution) variable. In eval-exists, the
+                    old value is recorded, the global set to the new value, and reset on exit - this forms a
+                    stack for nested EXISTS.


It's not clear what is reset on exit. I think, maybe, replace that phrase with the global reset to the old value on exit, as suggested below. The hyphen should become either a full stop (as suggested below) or an em-dash (which I think doesn't work as well, here).

Suggested change

Another way is to have a global (to the execution) variable. In eval-exists, the

old value is recorded, the global set to the new value, and reset on exit - this forms a

stack for nested EXISTS.

Another way is to have a global (to the execution) variable. In eval-exists, the

old value is recorded, the global set to the new value, and the global reset to the old value on exit. This forms a

stack for nested `EXISTS`.

TallTed · 2025-08-13T19:40:51Z

spec/index.html

+                <p>
+                  The outcome of `PrjMap` is independent of the order of replacement
+                  (e.g. bottom-up or top-down).
+                  Replacements may happen several times, depending on recursive order


Suggested change

Replacements may happen several times, depending on recursive order

Replacements may happen several times, depending on recursive order,

TallTed · 2025-08-13T19:42:15Z

spec/index.html

+                  The outcome of `PrjMap` is independent of the order of replacement
+                  (e.g. bottom-up or top-down).
+                  Replacements may happen several times, depending on recursive order
+                  but each time a replacement is made, the variable not used anywhere else.


I don't quite understand what's trying to be said here. It might just need the is I've added, but a larger change may be needed.

Suggested change

but each time a replacement is made, the variable not used anywhere else.

but each time a replacement is made, the variable is not used anywhere else.

TallTed · 2025-08-13T19:42:43Z

spec/index.html

+                <div class="note">
+                  <p>
+                    A variable inside a project expression that is not in the variables projected
+                    is not affected by the values insertion operation because it is renamed apart.


Suggested change

is not affected by the values insertion operation because it is renamed apart.

is not affected by the value insertion operation because it is renamed separately.

TallTed · 2025-08-13T19:43:23Z

spec/index.html

@@ -10502,6 +10623,12 @@ <h3>Evaluation Semantics</h3>
          </div>
          <div class="defn">
            <p><b>Definition: <span id="defn_evalFilter">Evaluation of Filter</span></b></p>
+
+            <p class="ednote">@@ Make current μ available. Or use current language in Exist.
+              Long term: change eval() to be arity three - 3rd argument is "current row".


Suggested change

Long term: change eval() to be arity three - 3rd argument is "current row".

Long term: change eval() to be arity three — 3rd argument is "current row".

TallTed · 2025-08-13T19:45:18Z

spec/index.html

@@ -10994,52 +11125,95 @@ <h3>Grammar</h3>
          section 6 <a data-cite="xml11#sec-notation">Notation</a>.</p>
        <p>Notes:</p>
        <ol>


Order doesn't seem to be important in this <ol>. If there is nothing that refers to these notes by number, then this <ol> should be changed to an <ul>

The numbering is essential so people can talk about the notes conveniently. It is a long list - saying the "8th bullet" is convenient. Counting manually is not convenient.

OK. An anticipated reader-need to refer to these notes by number is almost as compelling as a reference from somewhere in the documents we're publishing. It's certainly sufficient. I would mark this comment resolved, if I had such a button.

hartig · 2025-08-14T08:02:07Z

@afs do you already want feedback on this one? (I am asking because it still has WiP status in the GitHub UI and you didn't say anything in the description.)

afs · 2025-08-14T11:11:11Z

@hartig - I was going to bring it up at the next SPARQL TF.

I use marking as "draft" to indicate its still changing so it's more like "FYI" in this state and not a request for feedback but feedback on the design is welcome. (here - the areas are marked @@.)

On this PR, one area that might be of interest is whether "eval(pattern, D, G)" could be useful generalized to take a correlation
row as an extra argument. While it may not be necessary for EXISTS and handling the correlation row could be EXIST specific. But being mindful of future possibilities and the desire to not have too many algorithm variations, it might be a useful way to go while the spec is in WG (i.e pre features+maintenance). It isn't worse than EXIST-specific handling. The timing with regard to your current work might be quite good.

TallTed · 2025-08-25T17:52:59Z

spec/index.html

          <p>In <code>BIND (expr AS v)</code> requires that the variable <code>v</code> is not
-            in-scope from the preceeding elements in the group graph pattern in which it is used.</p>
+            in-scope from the preceeding elements in the group graph pattern in which it is used.


Suggested change

in-scope from the preceeding elements in the group graph pattern in which it is used.

in-scope from the preceding elements in the group graph pattern in which it is used.

TallTed · 2025-08-25T17:55:05Z

spec/index.html

+                  <li>A1 = translate(P) without applying simplification</li>
+                  <li>A2 = A1 rewritten to include access current binding</li>
+                  <li>A3 = A2 Remapped hidden-scope variables (if we do that)</li>


Suggested change

<li>A1 = translate(P) without applying simplification</li>

<li>A2 = A1 rewritten to include access current binding</li>

<li>A3 = A2 Remapped hidden-scope variables (if we do that)</li>

<li>`A1 = translate(P)` without applying simplification</li>

<li>`A2 = A1` rewritten to include access current binding</li>

<li>`A3 = A2` Remapped hidden-scope variables (if we do that)</li>

TallTed · 2025-08-25T17:55:51Z

spec/index.html

+                      During <a href="#sparqlQuery">translation to the SPARQL algebra</a>
+                    </p>
+                  <pre>
+Replace each occurence of `Y` in X where `Y` is one of


Suggested change

Replace each occurence of `Y` in X where `Y` is one of

Replace each occurrence of `Y` in `X` where `Y` is one of

TallTed · 2025-08-25T18:03:39Z

spec/index.html

+            There are two entry points into the grammar: <code>QueryUnit</code> for the SPARQL query language
+            and <code>UpdateUnit</code> for the SPARQL update language.


Suggested change

There are two entry points into the grammar: <code>QueryUnit</code> for the SPARQL query language

and <code>UpdateUnit</code> for the SPARQL update language.

There are two entry points into the grammar: <code>QueryUnit</code> for the SPARQL Query language

and <code>UpdateUnit</code> for the SPARQL Update language.

TallTed · 2025-08-25T18:04:56Z

spec/index.html

+            and variables in a `VALUES` clause.
+          </li>
+          <li>
+            Aggregate functions can be one of the 
            <a href="#rAggregate">built-in keywords for aggregates</a> 
            or a custom aggregate, which is syntactically a <a href="#rFunctionCall">function
              call</a>. Aggregate functions may only be used in 
            <a href="#rSelectClause">SELECT</a>, <a href="#rHavingClause">HAVING</a>


Suggested change

<a href="#rSelectClause">SELECT</a>, <a href="#rHavingClause">HAVING</a>

<a href="#rSelectClause">SELECT</a>, <a href="#rHavingClause">HAVING</a>,

TallTed · 2025-08-25T18:05:11Z

spec/index.html

+            and <a href="#rOrderClause">ORDER BY</a> clauses.
+          </li>
+          <li>
+            The expression argument of an aggregate function can not contain an aggregate function.


Suggested change

The expression argument of an aggregate function can not contain an aggregate function.

The expression argument of an aggregate function cannot contain an aggregate function.

afs · 2025-08-25T18:15:44Z

This PR is unlikely to proceed due to the alternative eval/expr based approach to definition.

TallTed suggested changes Aug 13, 2025

View reviewed changes

afs force-pushed the exists-correlated branch 3 times, most recently from 867c4bd to 1a0a79f Compare August 15, 2025 13:07

afs added 5 commits August 25, 2025 17:07

Initial values-insertion draft

f409261

Add 'row in scope' approach

eed6f4f

Updates from reviews

753e340

EXISTS: scoping restriction

2364961

WIP: Sketch correlated EXISTS

203391e

afs force-pushed the exists-correlated branch from 1a0a79f to 203391e Compare August 25, 2025 16:10

TallTed mentioned this pull request Aug 25, 2025

replace "filter(s)" in most running text with "FILTER clause(s)" including that codefence #273

Open

TallTed reviewed Aug 25, 2025

View reviewed changes

	<td>`EXISTS` and `NOT EXISTS` filters</td>
	<td>`FILTERs` using `EXISTS` and `NOT EXISTS`</td>

	<td>`EXISTS` and `NOT EXISTS` filters</td>
	<td>`FILTER` statements using `EXISTS` or `NOT EXISTS`</td>

	<td>`EXISTS` and `NOT EXISTS` filters</td>
	<td>`FILTER` containing `EXISTS` or `NOT EXISTS`</td>

	<p>In <code>BIND (expr AS v)</code> requires that the variable <code>v</code> is not
	<p><code>BIND (expr AS v)</code> requires that the variable <code>v</code> is not

		<p>In <code>SELECT</code>, the variable <code>v</code> must not be in-scope in the graph
		pattern of the <code>SELECT</code> clause, nor used in another select expression earlier in

	as an argument or a "null" token. . Normally this the "null" token. This would be set in
	as an argument or a "null" token, and is normally the "null" token. This would be set in

	Replacements may happen several times, depending on recursive order
	Replacements may happen several times, depending on recursive order,

	but each time a replacement is made, the variable not used anywhere else.
	but each time a replacement is made, the variable is not used anywhere else.

	is not affected by the values insertion operation because it is renamed apart.
	is not affected by the value insertion operation because it is renamed separately.

	Long term: change eval() to be arity three - 3rd argument is "current row".
	Long term: change eval() to be arity three — 3rd argument is "current row".

	in-scope from the preceeding elements in the group graph pattern in which it is used.
	in-scope from the preceding elements in the group graph pattern in which it is used.

	Replace each occurence of `Y` in X where `Y` is one of
	Replace each occurrence of `Y` in `X` where `Y` is one of

		There are two entry points into the grammar: <code>QueryUnit</code> for the SPARQL query language
		and <code>UpdateUnit</code> for the SPARQL update language.

	<a href="#rSelectClause">SELECT</a>, <a href="#rHavingClause">HAVING</a>
	<a href="#rSelectClause">SELECT</a>, <a href="#rHavingClause">HAVING</a>,

	The expression argument of an aggregate function can not contain an aggregate function.
	The expression argument of an aggregate function cannot contain an aggregate function.

Correlated EXISTS #257

Are you sure you want to change the base?

Correlated EXISTS #257

Uh oh!

Conversation

afs commented Aug 13, 2025 • edited by pr-preview bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TallTed left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TallTed Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TallTed Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hartig commented Aug 14, 2025

Uh oh!

afs commented Aug 14, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TallTed Aug 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

afs commented Aug 25, 2025

Uh oh!

Uh oh!

afs commented Aug 13, 2025 •

edited by pr-preview bot

Loading

TallTed Aug 15, 2025 •

edited

Loading

TallTed Aug 13, 2025 •

edited

Loading

TallTed Aug 25, 2025 •

edited

Loading