[RFC FS-1151] Spread operator for F#

[brianrourkeboll]

Discussion thread for RFC FS-1151: #805

Language suggestion: fsharp/fslang-suggestions#1253
Proposed implementation: TODO

Subthreads

Links to subthreads below for focused discussion on subtopics:

• Type definitions
• Expressions
• Patterns (?)
• .. versus ...
• Alternatives
• Tooling

Not everything mentioned here is equally likely to be a good idea to implement.

I am personally in favor of:

1. Record type spreads.
2. Interface implementation spreads in type definitions and object expressions.
3. Spreading into nominal record expressions.
4. Spreading into anonymous record expressions.
5. Spreading in collection and computation expressions.
6. Maybe spreading in certain specific pattern scenarios.

[brianrourkeboll]

Type definitions

Subthread for discussing spreads in type declarations/definitions.

Record type spreads

In record type declarations, ...SrcTy in type DestTy = { ...SrcTy } implies the union of the fields of SrcTy with any additional fields declared in DestTy.

type R1 = { A : int; B : int }
type R2 = { C : int }
type R3 = { ...R1; ...R2; D : int }

Duplicate field names derived from spreads are allowed in type declarations. Duplicate field names arising from explicit field declarations remain disallowed.

type R1 = { A : int }
type R2 = { ...R1; A : string }   // Allowed.
type R3 = { A : int; A : string } // Still not allowed.

The rightmost instance of a field with a given name, whether from explicit declaration or type spread, wins out and shadows any leftward fields with the same name.

type R1 = { A : int }
type R2 = { ...R1; A : string } // `A : string` wins out over `A : int` from `...R1`.

type R1 = { A : int }
type R2 = { A : string }
type R3 = { ...R1; ...R2 } // `A : string` from `R2` wins out over `A : int` from `R1`.

A suppressible warning is emitted when the intent is less clear, e.g., when a field from a spread shadows a field with the same name that is explicitly declared to the left:

type R1 = { A : int }
type R2 = { A : string; ...R1 }
------------------------^^^^^
warning FSXXXX: Field 'A: int' from spread type 'R1' shadows the explicitly declared field 'A: string` with the same name.

Generic type parameters must be passed through to spreads:

type R1<'a> = { A : 'a; B : string }
type R2<'a> = { X : 'a; Y : string }
type R3<'a> = { ...R1<'a>; ...R2<'a> }

type R1<'a> = { A : 'a; B : string }
type R2<'a> = { X : 'a; Y : string }
type R3<'a, 'b> = { ...R1<'a>; ...R2<'b> }

Attributes on generic type parameters must match:

type R1<[<Measure>] 'a> = { A : int<'a> }
type R2<[<Measure>] 'a> = { ...R1<'a> }

Constraints on generic type parameters must match:

type R1<'a when 'a : comparison> = { A : 'a }
type R2<'a when 'a : comparison> = { ...R1<'a> }

Interface implementation spreads

Type definitions

type IA =
    abstract A: int -> int

type IA =
    abstract A: int -> int
    abstract B: int -> int

type A () =
    interface IA with
        member _.A x = x + 1

type B (a : IA) =
    interface IB with
        ...a
        member _.B x = x - 1 

This should also probably work with a self-alias (thanks for bringing the scenario up, @Happypig375):

type IA =
    abstract A: int -> int

type IB =
    abstract A: int -> int
    abstract B: int -> int

type B () as this =
    member _.A x = x + 1
    interface IA with ...this // uses the above implementation of A
    interface IB with
        ...this // uses the above implementation of A
        member _.B x = x - 1 

Object expressions

Spreads in object expressions would work similarly to how they would in type definitions:

type IA =
    abstract A: int -> int

type IA =
    abstract A: int -> int
    abstract B: int -> int

let a =
    { new IA with
        member _.A x = x + 1 }

let b =
    { new IB with
        ...a
        member _.B x = x - 1 }

[brianrourkeboll]

@T-Gro I assume we would still want to disallow duplicate explicit field declarations?

type R = { A : int; A : string } // This should still be disallowed, no?

Or do you think that should now become valid, but with a warning?

[T-Gro]

I use to think about it like this:

• For a type just being authored, we assume the user is in a full control and we can actively error or warn on obvious non-sense or useless constructs (like shadowing a binding which was never used, overwriting a field A which could never have been used etc.)

• For building upon other type (...R1; ...R2), it can be possible that the user has no way to change that type's definiton. error on that could prevent valid use cases.

=> type R = { A : int; A : string } should raise warning/error

[brianrourkeboll]

• For a type just being authored, we assume the user is in a full control and we can actively error or warn on obvious non-sense or useless constructs (like shadowing a binding which was never used, overwriting a field A which could never have been used etc.)

Presumably we should then also warn or error if the same source type is spread twice into the same target type?

type R1 = { A : int }
type R2 = { ...R1; ...R1 } // This should probably be a warning or error.

[Happypig375]

• For building upon other type (...R1; ...R2), it can be possible that the user has no way to change that type's definiton. error on that could prevent valid use cases.

or you can also support explicit omission in spreads like with a set subtraction operator \.

[T-Gro]

type R2 = { ...R1; ...R1 } // This should probably be a warning or error.

Would the definition "Warn if a syntactic construct for fields, either scalar or spread, becomes fully overridden by subsequent fields" cover all cases here?

e.g. this would be covered as well:

type R1 = { A : int }
type R2 = { B : int }
type R_Both = { ... R1; ... R2}


type This_Gives_Error =  { ...R_Both  ; ... R1 ; ... R2 }

[brianrourkeboll]

Expressions

Subthread for discussing expression-level spreads.

Spreads into nominal records

Spreads into nominal records are constrained by the set of fields defined in the target nominal record type. Missing fields in the source expression's/expressions' type(s) must be specified explicitly; extra fields in the source expression's/expressions' type(s) are ignored.

type A =
    { X : int
      Y : int }

type B =
    { Z : int }

type C =
    { X : int
      Y : int
      Z : int
      Extra : string }

let a = { X = 1; Y = 2 }
let b = { Z = 3 }
let c = { ...a; ...b; Extra = "four" }

From other nominal or anonymous records

From other object types

TODO

Spreads into anonymous records

From other nominal or anonymous records

type R1 =
    { A : int
      B : string }
    member _.Lol = 99

type R2 = { ...R1; C : string }

let r1 = { A = 3; B = "4"; Q = 99 }
let r2 = { A = 5; B = "6"; C = "haha" }

let r1AndR2 = {| ...r1; ...r2 |}

From other object types

TODO

Spreads in collection or computation expressions

In collection expressions and computation expressions, ... becomes a new synonym for yield!:

let xs = [10..20]
let ys = [1; 2; 3; ...xs]

≡

let xs = [10..20]
let ys = [1; 2; 3; yield! xs]

While yield! and return! currently partially share a representation in the AST, (SynExpr.YieldOrReturnFrom), I do not see a reason to use ... as an equivalent to return!.

More (TBD)

...

[T-Gro]

The r1AndR2 case show a scenario with inherited/spread duplicity.
The fields and duplicates are known at compile-time, values not.

I would considered "conflict resolution" to be a separate sub-topic, whose resolution would consistently apply to other occurrences of conflicts.

1. First wins?
2. Last wins? (i.e. spreading works like overwriting)
3. Turn a scalar into a tuple (since the duplicity is compile-time detected) ?
4. Disallow duplicates alltogether without any resolution mechanism (I would personally be against this - I would prefer a consistent ruleset with coding patterns allowing to express preference. let r1AndR2 = {| ...r1; ...r2 ; .. r1.A|} // somewhat overwrite A` field with the value from r1 ?)
5. Either of 1/2 with a special rule of "better wins" for optional containers like Option,Result,nulls - this is way to opinionated, I agree.

[brianrourkeboll]

Patterns (?)

Subthread for discussing pattern-level spreads.

Records

Record patterns already allow specifying only a subset of fields and ignoring those unuttered. Would we allow ... as well?

Anonymous records?

fsharp/fslang-suggestions#713 (comment)

Could this be implemented via SRTPs?

let f x =
    match x with
    | {| A = 3; B = 4; ... |} -> …

let f x =
    match x with
    | {| A = 3; B = 4; ...rest |} -> …

Tuples?

let tuple = 1, 2, 3, 4, 5, 6, 7
let x, y, ...rest = tuple

Arrays?

match xs with
| [|...; 3; 4|] -> true
| _ -> false

match xs with
| [|...rest; 3; 4|] -> f rest
| _ -> false

Lists?

We already have

match xs with
| 3 :: 4 :: _ -> true
| _ -> false

Would we add

match xs with
| [3; 4; ...] -> true
| _ -> false

?

What about

match xs with
| [...; 3; 4] -> true
| _ -> false

, which would be $O(n)$?

Or

match xs with
| [...rest; 3; 4] -> f rest
| _ -> false

?

More (TBD)

...

[T-Gro]

I do get standalone ... to indicate the rest is ignored.
The ... rest looks like it should be binding something and would indeed lead us to anti-spreads as well, for a possible very narrow use case?

[Happypig375]

Don't you find it inconsistent that this proposed anonymous record pattern must be complete by default and the spread is required to make it non-complete, whereas the nominal record pattern is non-complete by default?

[brianrourkeboll]

Don't you find it inconsistent that this proposed anonymous record pattern must be complete by default and the spread is required to make it non-complete, whereas the nominal record pattern is non-complete by default?

Yes, they would indeed be inconsistent.

As I mention in fsharp/fslang-suggestions#713 (comment), though, some form of fsharp/fslang-suggestions#968 could subsume both. I think I would probably lean towards that instead.

[Happypig375]

Yea, in that case just go directly for object patterns, the anonymous record pattern isn't needed.

[T-Gro]

There is a big difference in type checking and type inference though - even though the record pattern is syntactically incomplete, it does bind a single named type.

I am aware of SRTP proposals to desugar the anon pattern into. But I am not yet convinced that using SRTPs as a vehicle to power basic features is deserved (especially thinking about debuggability and troubleshooting).

[Happypig375]

Prefix .. operator is also distinguishable from infix .. operator (given that we allow 1 ..2 to change meaning). We can potentially save a dot and align with C#.

[Happypig375]

@T-Gro Any input from a backcompat perspective?

[T-Gro]

(..) is an user-definable operator which does not prohibit a single-argument declaration, I am not sure if we could guarantee not breaking anyone. (...) avoids that and is also aligned with TypeScript.

[brianrourkeboll]

(..) is an user-definable operator which does not prohibit a single-argument declaration, I am not sure if we could guarantee not breaking anyone.

Even though you can define it as a single-argument function, it won't actually parse when used in prefix form; you can only invoke it like (..) x, not like ..x.

The only place ..expr is currently syntactically valid is in indexers: xs[..i]. That would only cause compatibility issues with some of the more unlikely potential uses involving tuples or function/method arguments, like currying/uncurrying (fsharp/fslang-suggestions#1253 (comment)), spreading tuples into other tuples, spreading records or tuples as method args, spreading into param arrays, etc.

So I think that .. would be syntactically viable if we were confident that we never wanted to use spreading for anything tuple or arg-related.

[Happypig375]

or you can also just define spreading into an indexer with an integer argument as a ranged lookup?

[brianrourkeboll]

or you can also just define spreading into an indexer with an integer argument as a ranged lookup?

I'm not sure what you mean. Right now, you can define a GetSlice method with nearly any signature you want (or indeed multiple overloads with different signatures).

type T =
    | T
    member _.GetSlice _ = 99

// These are all currently syntactically valid.
T[..null]
T[..T]
T[..1000]
T[..()]
T[..3, ..99]
T[..(1, 2, 3)]
T[..[1, 2, 3]]

Changing the effect of .. on its argument(s) in indexers in any way would be a breaking change.

[brianrourkeboll]

Alternatives

Subthread for discussing alternatives: pros and cons.

with

type R1 = { A : int; B : int }
type R2 = { R1 with C : int; D : int }
type R3 = { R1 with R2 } // ?

let r1 = { A = 1; B = 2 }
let r2 = { r1 with r2 }

Pros

• The with keyword already exists and its usage overlaps with that proposed for ....

  let r1 = {| A = 1; B = 2 |}
  let r2 = {| ...r1; C = 3 |} // = {| r1 with C = 3 |}

Cons

• with is not as universally applicable as ..., e.g., it would not make sense in patterns...
• with is both more semantically and syntactically ambiguous and less composable in various scenarios, e.g.,

  let r1 = {| A = 3; B = 4 |}
  let r2 = {| C = 5; D = 6 |}
  let r3 = {| ...r1; ...r2 |}         // Would this be {| r1 with r2 |}?
  let r4 = {| ...r1; E = 99; ...r2 |} // How would this work with `with`?

Type providers

E.g., fsharp/fslang-suggestions#212 + fsharp/fslang-suggestions#154 for type spreads.

Pros

• Potentially even more powerful.

Cons

• Almost certainly harder to implement.
• Only solves type spreads, not expression spreads (or other proposed uses).

[kerams]

First thing that comes to mind

let r4 = {| r1 with r2; E = 99 |}
let r4 = {| r1 with r2 with E = 99 |}

Maybe another con - the proposed use of with signifies extending something, while the record update syntax replaces existing values. These are slightly different concepts.

[brianrourkeboll]

Yeah, ... in records/anonymous record expressions would imply a (compile-time) set union operation where the rightmost instance of a field would win out (as in, e.g., JavaScript and TypeScript).

[brianrourkeboll]

Tooling

Rendering & signatures of record types containing spreads

Rendering

F# tooling currently lists the fields (or a truncated subset thereof) of a record type definition on hover or when passed to F# interactive.

What should the tooling show for a record type that includes a spread? E.g., given

type R1 = { A : int; B : int }
type R2 = { ...R1; C : int }

should the tooling indicate that R2 has any relationship to R1?

Recall that the relationship of R2 to R1 implied by ...R1 is compile-time only and indicates only that R2 contains all of the fields of R1; it does not imply inheritance.

In such a case, should the tooling show:

1. The spread alone:

   type R2 =
       { ...R1; C : int }

2. An inlining of the spread without mention of the source type:

   type R2 = { ...{ A : int; B : int }; C : int }

3. A full expansion of the spread without mention of the source type:

   type R2 = { A : int; B : int; C : int }

4. Or some form of both:

   type R2 = { ...R1 (A : int; B : int); C : int }

?

Signatures

The signature of a record built from spreads would surely be the fully expanded version with no mention of the underlying spreads, i.e., given the definition

type R1 = { A : int; B : int }
type R2 = { ...R1; C : int }

the signature would be

type R1 = { A : int; B : int }
type R2 = { A : int; B : int; C : int }

Code fixes & refactorings

Would it be feasible and worthwhile to offer a refactoring to convert a spread to explicit redefinitions or delegations? This could be useful if the developer decides that there should no longer be a compile-time relationship between the source of the spread and the target, but wishes to keep all or most of the definitions, implementations, or values implied by the spread.

[T-Gro]

For all different perspectives ( public API, reflection, cross-compilation,..), should spreads be considered an implementation detail and be fully interchangeable with their fully written counterparts?

i.e. how does replacing a type composed from spreading to a handwritten one affect aspects of compatibility?

[brianrourkeboll]

should spreads be […] fully interchangeable with their fully written counterparts?

Yes.