Skip to content
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions extensions/functions_comparison.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,7 @@ scalar_functions:
- value: any1
variadic:
min: 2
parameterConsistency: CONSISTENT
return: any1
-
name: "least"
Expand All @@ -277,6 +278,7 @@ scalar_functions:
- value: any1
variadic:
min: 2
parameterConsistency: CONSISTENT
return: any1
nullability: MIRROR
-
Expand All @@ -289,6 +291,7 @@ scalar_functions:
- value: any1
variadic:
min: 2
parameterConsistency: CONSISTENT
return: any1
# NOTE: The return type nullability as described above cannot be expressed currently
# See https://github.com/substrait-io/substrait/issues/601
Expand All @@ -304,6 +307,7 @@ scalar_functions:
- value: any1
variadic:
min: 2
parameterConsistency: CONSISTENT
return: any1
nullability: MIRROR
-
Expand All @@ -316,6 +320,7 @@ scalar_functions:
- value: any1
variadic:
min: 2
parameterConsistency: CONSISTENT
return: any1
# NOTE: The return type nullability as described above cannot be expressed currently
# See https://github.com/substrait-io/substrait/issues/601
Expand Down
27 changes: 26 additions & 1 deletion site/docs/expressions/scalar_functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,32 @@ When the same parameter name is used multiple times in a function definition, th

#### Type Parameter Resolution in Variadic Functions

When the last argument of a function is variadic and declares a type parameter e.g. `fn(A, B, C...)`, the C parameter can be marked as either consistent or inconsistent. If marked as consistent, the function can only be bound to arguments where all the C types are the same concrete type. If marked as inconsistent, each unique C can be bound to a different type within the constraints of what T allows.
When the last argument of a function is variadic and declares a type parameter e.g. `fn(A, B, C...)`, the C parameter can be marked as either consistent or inconsistent. If marked as consistent, the function can only be bound to arguments where all the C types are the same concrete type. If marked as inconsistent, each unique C can be bound to a different type within the type parameter's constraints. The default behavior is `CONSISTENT`.

##### Examples

```yaml
--8<-- "examples/extensions/variadic_consistent_example.yaml"
```
With `CONSISTENT`, all variadic arguments must resolve to the same concrete type. In the `all_equal` function above, valid invocations include `all_equal(1, 2, 3)` where all arguments are `i32`, or `all_equal('a', 'b')` where all arguments are `string`. However, `all_equal(1, 'hello')` would be invalid because the arguments have different types.

```yaml
--8<-- "examples/extensions/variadic_inconsistent_example.yaml"
```
With `INCONSISTENT`, each variadic argument can resolve to a different concrete type. In the `count_args` function above, an invocation like `count_args(1, 'hello', true)` is valid because arguments of type `i32`, `string`, and `boolean` can be freely mixed.

```yaml
--8<-- "examples/extensions/variadic_complex_type_example.yaml"
```
When the variadic argument contains a type parameter like `any1`, all occurrences of `any1` within each argument must still match (as described above). In the `all_pairs_equal` function above, each `struct<any1, any1>` must have both fields be the same type (e.g., `struct<i32, i32>` is valid, but `struct<i32, string>` is not). With `CONSISTENT`, all variadic arguments must also use the same `any1` binding. For example, `all_pairs_equal({1, 2}, {3, 4})` is valid because all pairs are `struct<i32, i32>`, but `all_pairs_equal({1, 2}, {'a', 'b'})` would be invalid because the pairs use different types.

```yaml
--8<-- "examples/extensions/variadic_complex_inconsistent_example.yaml"
```
With `INCONSISTENT`, each variadic argument can use a different `any1` binding, while still requiring `any1` to be consistent within each argument. In the `count_pairs` function above, `count_pairs({1, 2}, {'a', 'b'})` is valid because the first pair is `struct<i32, i32>` and the second is `struct<string, string>`. However, `count_pairs({1, 'a'})` would still be invalid because within a single pair, both fields must match.

!!! note
The `parameterConsistency` field only has an effect when the variadic argument contains a type parameter (like `any1`, `T`, etc.). For concrete types (like `boolean?`, `i32`, `string`), it has no effect. Additionally, if the type parameter appears in non-variadic positions (such as the return type), those occurrences pin the parameter and all variadic arguments must match that type regardless of the `parameterConsistency` setting. When the variadic argument does contain a type parameter, it is recommended that extension authors explicitly specify `parameterConsistency` to make the intended behavior clear.



Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
urn: extension:example:variadic_complex_inconsistent
scalar_functions:
- name: count_pairs
description: Returns the number of pairs. Within each pair, both elements must be the same type. Across pairs, different types are allowed.
impls:
- args:
- name: first
value: struct<any1, any1>
variadic:
min: 1
parameterConsistency: INCONSISTENT
return: i64
12 changes: 12 additions & 0 deletions site/examples/extensions/variadic_complex_type_example.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
urn: extension:example:variadic_complex_type
scalar_functions:
- name: all_pairs_equal
description: Returns true if all pairs are equal. Within each pair, both elements must be the same type. Across pairs, all must use the same type.
impls:
- args:
- name: first
value: struct<any1, any1>
variadic:
min: 1
parameterConsistency: CONSISTENT
return: boolean
12 changes: 12 additions & 0 deletions site/examples/extensions/variadic_consistent_example.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
urn: extension:example:variadic_consistent
scalar_functions:
- name: all_equal
description: Returns true if all arguments are equal. All arguments must be the same type.
impls:
- args:
- name: first
value: any
variadic:
min: 1
parameterConsistency: CONSISTENT
return: boolean
12 changes: 12 additions & 0 deletions site/examples/extensions/variadic_inconsistent_example.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
urn: extension:example:variadic_inconsistent
scalar_functions:
- name: count_args
description: Returns the number of arguments. Each argument can be a different type.
impls:
- args:
- name: first
value: any
variadic:
min: 1
parameterConsistency: INCONSISTENT
return: i64
Loading