[API Proposal]: linq "from" clause without "on"

[vmikhailov]

Background and motivation

The idea behind the new API is to improve readability and performance. Currently, the intuitive and compact way to write cross joins for multiple collections is to use multiple "from" clauses. The drawback of the existing approach is multiple enumerations of each nested "from" source.

from a in GetData1()
from b in GetData2()
from c in GetData3() 
select new SomeObject(a, b, c)

As far as the code above translates into nested SelectMany the GetData2 and GetData3 methods will be executed multiple times. GetData2() will be executed GetData1().Count times, while GetData3() will be executed GetData1().Count * GetData2().Count. The behaviour is not obvious and could cause unexpected performance drop.

Alternative option is to use join

from a in GetData1()
join b in GetData2() on 1 equals 1
join c in GetData3() on 1 equals 1
select new SomeObject(a, b, c)

In this case all the GetData* methods will be executed just once which is way more effective

API Proposal

One possible solution would be to make the "on " clause optional.

from a in GetData1()
join b in GetData2()
join c in GetData3()
select new SomeObject(a, b, c)

also corresponding CrossJoin/Join chain methods will be needed.

API Usage

see the API proposal

Alternative Designs

Alternatively one more keyword "cross" could be added.

from a in GetData1()
cross join b in GetData2()
cross join c in GetData3()
select new SomeObject(a, b, c)

Risks

No response

[HaloFour]

How does a different syntax address any of the other concerns? The behavior of the from clause to map to a SelectMany behaving like a "cross join" here is intentional.

[vmikhailov]

Could you please be more specific about the other concerns?

[HaloFour]

You mention the performance issues with cross joins. This doesn't seem to address that in any way.

[zergusik]

SQL defines syntaxes:

• ... FROM a, b
• ... FROM a CROSS JOIN b
• ... FROM a JOIN b ON 1=1
  as equivalent, while the implementation (nested loop, hash match, merge) depends on data and is up to DB engine.

but in LINQ from and join are implemented differently:

• from is de-facto a cross apply, which is implemented as a nested loop (leading to multiple enumerations of nested queries)
• join (with TRUE condition being a cross join) is implemented as a sort of hash match (Lookup), which does not lead to multiple enumerations of subqueries (however, it eagerly fetches the subquery).

So the proposal is to simplify join syntax to allow for the cross join-like behavior syntax to avoid multiple enumerations.

[vmikhailov]

You mention the performance issues with cross joins. This doesn't seem to address that in any way.

Not sure why do you think it does not solve the performance issue. The proposal describes syntax improvements for the join which solves the issues by itself

[HaloFour]

Forcing materialization is not necessarily a performance benefit, but I understand why you'd want to do so in some cases.

IMO, cross joins are a relatively uncommon use case, and there is already syntax to accomplish the desired behavior. The C# syntax would have to continue mapping down to the same BCL methods anyway, especially since adding a new pattern here would likely break existing queryable providers. Given the language team hasn't seemed terribly interested in expanding on the join syntax even to cover the much more common case of left outer joins I kinda doubt that dedicated cross join syntax would be considered, but that's just my opinion.

[vmikhailov]

It is not about forcing materialization, it happens anyway for the join behaviour. The proposal is about optional omiting of the redundant 1=1. Trivial syntax change with minimal impact on the underlying BCL classes. I don't think there will be an issue with implementation of Join function overload without fields selectors and contidition. Do you know why the langugage team is not open for that sort of improvement? Probably the comunity can help with some extra hands? I personally can contribute to this.

[HaloFour]

Do you know why the langugage team is not open for that sort of improvement?

If I had to guess it just doesn't bubble to a level where it demands attention. There is a working solution to the use case, even if it's not as ergonomic as it could be.

Here is the "left join" discussion for comparison: #361