Stuck because of the lack of composition support. How can I combine read, write, read-tree together?

[Nefcanto]

I have a read business, that only reads data.

public abstract class ReadBusiness<T>
    where T : class, IEntity, new()

And I have a business that both reads and writes data. To DRY, I inherit it from ReadBusiness<T> to reuse the read logic:

public abstract class Business<ReadModel, WriteModel> : ReadBusiness<ReadModel>
    where ReadModel : class, IEntity, new()
    where WriteModel : class, IEntity, new()

Now I want to support hierarchical structures and mix them with reading and writing. So, I need to have ReadTreeBusiness and TreeBusiness.

public abstract class ReadTreeBusiness<T> : ReadBusiness<T>
    where T : class, IEntity, IHierarchical, new()

This ReadTreeBusiness adds hierarchical methods for readings, like GetTree or GetAncestorsPath.

I inherited it from ReadBusiness.

The problem arises when I want to create the TreeBusiness. I need to have read, write, and read-tree all in it, and add some methods to it. How can I do that? I'm stuck at this point because of the lack of composability support in C#.

I have tried many times to persuade the .NET team that we need a clean composition syntax. I don't know where to go from here.

In fact the TreeBusiness needs:

1. Simple read methods which are encapsulated in the ReadBusiness
2. Simple write operations which are encapsulated in the Business
3. Simple tree-read methods which are encapsulated in the ReadTreeBusiness
4. And it then needs to add its own writing methods for hierarchical data, like CalculateNestedSets or RemoveChildrenRecursively

What should I do? What options do I have?

1. Default interfaces won't work. Because I need the same architecture for the API and I don't have access to the HttpContext there.
2. I can create a common class like ReadUtilities and include it in both ReadBusiness and ReadTreeBusiness. But then I have to write all of the wrapper methods to expose the methods inside the ReadUtilities.
3. Code generation is not an option here, because it's extremely overcomplicated for this use-case. It's a good option for tens or hundreds of controller files. But it's not good for a single TreeBusiness file.

[CyrusNajmabadi]

I don't really understand your use of inheritance here. Similar to how Channels and Pipes expose distinct Reader/Writer properties, i would consider doing the same here, modeling everything as composition, not a mix of composition and inheritance.

Note: to bridge wanting to treat the composed type as either of the composed members, Channels have implicit operators to the respective reading/writing forms. Seems like you could do the same here instead of trying to make this work through inheritance.

I have tried many times to persuade the .NET team that we need a clean composition syntax. I don't know where to go from here.

Model things differently?

Code generation is not an option here, because it's extremely overcomplicated for this use-case.

I don't see how. With a few minutes of investment you can have somethign generating all your bridge methods for whenever this comes up. It would keep things up to date for you automatically as well. Seems like a reasonable return on investment.

[Nefcanto]

@CyrusNajmabadi,

Similar to how Channels and Pipes expose distinct Reader/Writer properties, i would consider doing the same here, modeling everything as composition, not a mix of composition and inheritance.

That's what I crave. But how? C# does not have composition capabilities. Where are the keywords for that? What is the syntax for that?

I would love to be able to create ReadUtilities and WriteUtilities and ReadTreeUtilities and compose them all in the TreeBusiness. How?

[CyrusNajmabadi]

That's what I crave. But how?

As mentioned, look at what those bcl types do and model similarly.

[Nefcanto]

@CyrusNajmabadi, please at least give me a link. I don't know what to search for. You just said "channels and pipes". What namespace? Which DLL?

[CyrusNajmabadi]

please at least give me a link. I don't know what to search for. You just said "channels and pipes". What namespace? Which DLL?

look at the bcl types for Channel/ChannelReader/ChannelWriter, and Pipe/PipeReader/PipeWriter and model similarly.

Channel/ChannelReader/ChannelWriter can be found at:

1. https://learn.microsoft.com/en-us/dotnet/api/system.threading.channels.channel-1?view=net-8.0
2. https://learn.microsoft.com/en-us/dotnet/api/system.threading.channels.channelreader-1?view=net-8.0
3. https://learn.microsoft.com/en-us/dotnet/api/system.threading.channels.channelwriter-1?view=net-8.0

Pipe/PipeReader/PipeWriter can be found at:

1. https://learn.microsoft.com/en-us/dotnet/api/system.io.pipelines.pipe?view=net-8.0
2. https://learn.microsoft.com/en-us/dotnet/api/system.io.pipelines.pipereader?view=net-8.0
3. https://learn.microsoft.com/en-us/dotnet/api/system.io.pipelines.pipewriter?view=net-8.0

[Nefcanto]

I saw some samples of code generation. Can I create a placeholder like below and replace it with the source code? I don't want to create partial classes. I want to include a code inside the existing code. Is that possible?

[Nefcanto]

As I expected, source generators are a mess. The idea is great. But:

1. The fact that we should target netstandard2.0 and lose all of the new goodies, is terrible
2. The fact that we should define them separately from our source code means that we now need extra maintenance for MSBuild files
3. I got this error. The fact that the debugging is not easy (as I have not been able to do it on Ubuntu and inside VS Code with many failed efforts) sucks:

CSC : warning CS8785: Generator 'HierarchicalReadGenerator' failed to generate source. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'ArgumentNullException' with message 'Value cannot be null. (Parameter 'contents')'.

@CyrusNajmabadi, can you please add more details regarding the channels and pipes? I don't understand what you mean.

[CyrusNajmabadi]

The fact that we should target netstandard2.0 and lose all of the new goodies, is terrible

Why? What features post net2.0 do you need when generating source?

Exception was of type 'ArgumentNullException' with message 'Value cannot be null.

Sounds like your code is crashing with an argument exception. Either run under the debugger to find out where, or produce a binlog to see the full stack, or write a unit test you can debug through.

can you please add more details regarding the channels and pipes?

Literally look at the bcl types for Channel/ChannelReader/ChannelWriter, and Pipe/PipeReader/PipeWriter and model similarly.

[KennethHoff]

Why? What features post net2.0 do you need when generating source?

I can only speak for myself here (which is not OP, for reference), but init and required is huge for me. PolySharp helps tremendously, but it's odd I have to run a third party package to get basic functionality in the language I've used for years.

Additionally, it feels wrong writing code that will only ever run in a .Net 8 context, but need to write code that - for what I can only assume is due to some issue with Visual Studio - has to support .Net Framework (which, funnily enough, is not even supported on the hardware I'm using - a Mac).

[colejohnson66]

Why? What features post net2.0 do you need when generating source?

The pain points of source generators have been mentioned countless times on the internet, but here's a few I've run into:

• Records. Requires an undocumented workaround of polyfilling IsExternalInit.
• JSON. Requires referencing all dependencies in our csproj, then packing them into our nupkg because Roslyn can't use NuGet.
  • Strictly speaking, any NuGet reference is a pain.
• Spans. Good luck being performant with source generators when all string operations allocate.

[CyrusNajmabadi]

I can only speak for myself here (which is not OP, for reference), but init and required is huge for me.

You can use those. The attributes for those don't need to be provided by the BLC.

but it's odd I have to run a third party package to get basic functionality in the language I've used for years.

That's literally always been our design.

Additionally, it feels wrong writing code that will only ever run in a .Net 8 context,

That may be the case for you. But it's not hte case for our ecosystem. Tons of people run VS.

which, funnily enough, is not even supported on the hardware I'm using - a Mac).

The restrictions are based on the ecosystem as a whole, not on pockets of it.

we also def don't want people then requiring net8 and above featuers, and then having pain when they want to expand to the much larger set of users in the ecosystem using VS. Sure, if you only care about a small set, this restriction isn't helpful. But we put in requirements with a broad view of the entirety of the ecosystem.

[CyrusNajmabadi]

Requires referencing all dependencies in our csproj, then packing them into our nupkg because Roslyn can't use NuGet.

Being on net8 doesn't change this. this is an aspect simply around generating authoring.

Good luck being performant with source generators when all string operations allocate.

We have not seen any cases where SG perf would be impacted at all by having spans available. Note: You can use spans if you want (we use it in netstandard2.0 code), you may be missing some functions. But we haven't seen a single generator (literally from any of our first parties) where the absence of this mattered for the code-gen part.

That's why i ask. because people bring this up, but we haven't seen any demonstration of the actual impact, just theoretical ones. And the first parties doing large scale generator work aren't running into issues with this. So it feelsl ike a lot like people just speaking out from their gut, which is not at all reasonable for us to design around.

[Nefcanto]

Three hours of trying to add a simple line to my code. The most terrible experience a developer can get.

1. Support is very limited. Lots of questions on StackOverflow have no answers.
2. ChatGPT and Gemini both fail to give practical code, and if even one line of their suggested codes is erroneous, one stuck at debugging it
3. Multiply that with modern development inside containers and ...

But returning back to the main problem. How can I compose three sets of utility methods for reading flat data, writing flat data, and reading hierarchical data? What would you do to DRY your code?

[CyrusNajmabadi]

and if even one line of their suggested codes is erroneous, one stuck at debugging it

Just write a unit test and debug it. Takes a minute or so to get set up, and you can figure eveything out.

Multiply that with modern development inside containers

I see absolutely nothing beneficial about developing in a container here. Seems like it would just add a ton of pain.

[Nefcanto]

@CyrusNajmabadi, with respect you apparently don't intend to solve problems and from what I see, you prefer to deny the pain the community goes through. That's not a cooperative approach. If C# is alive, it's because of the feedback of the community. People like me use it in different ways to hit walls and provide suggestions on the new requirements.

Please note that when we want to write code, we don't do it in isolation. Isolated code provides no business value at all. We need to integrate new tech into our existing codebase. Please let me repeat myself:

We need to write integrated code, not in isolation

So, when you come back with a sentence like:

I see absolutely nothing beneficial about developing in a container here. Seems like it would just add a ton of pain.

apparently, you have not grasped that fundamental fact. A word is of no use unless it's part of a sentence. Likewise, a source generator in isolation is of no use unless it's part of our codebase.

We develop inside containers because:

1. We want to ensure development environment consistency across our team (valid reason)
2. We want to get closer to our production environment (DevOps, valid reason)
3. We need to be able to bring down containers and re-create them as part of our chaos engineering
4. We want consistency across our tech stack. We develop inside containers for the front end, for database, for API, for mobile development, for search, etc.
5. We dynamically create a docker-compose.yml file and this is part of our overall composition (composing modules to create the codebase of an app). That's one way of reusability.

And many more reasons. So, the isolated source generator, a hello-world source generator is of no use.

[CyrusNajmabadi]

with respect you apparently don't intend to solve problems and from what I see, you prefer to deny the pain the community goes through.

I'm presenting potential solutions to your issue, using technology that exists today that you can try out right now. Will that solution necessarily be perfectly suited to your needs? Hard to say. We'll have to find that out together.

apparently, you have not grasped that fundamental fact.

Correct. It's not something i grasp here. Writing a generator in a container doesn't seem to help out here, and it seems to be actively hindering you. So my recommendation is to not do that. Based on my own experience, things will be a lot simpler and easier. :)

So, the isolated source generator, a hello-world source generator is of no use.

It seems like your'e starting on hte hardest mode possible while examining a solution that you are unfamiliar with, discovering roadblocks because of that hard mode, and then stating the solution isn't viable.

My position is that you should make this as easy as possible, learn a lot about SGs and how they can effectively address your higher level lang problem, solve hte solution at the high level (esp. using things like unit tests). Once you're there, then try to take that solution and make it work in containers and whatnot.

it would be akin to me trying to learn how to use a simple caching library to help with an app i have, and then complaining that setting it up in kubernetes is too hard before i've even learned the library or figured out if it works for my app case.

Solve the initial problem first, then work on the secondary and tertiary concerns after that :)

[CyrusNajmabadi]

We need to integrate new tech into our existing codebase.

I'm constantly integrating new tech into our tech stack. But i don't start by trying to just integrate it into our massively complex codebase right from the start. Instead, i learn the tech first. Then i often create some small scale projects/experiments to try it out and get comfortable and familiar with it. Once i've shaken off the learning curve doing that, then i incorporate it into our tech. And i do that writing unit tests, using the tech, and then iterating on both until things are passing. :)

[HaloFour]

Learning how to do something always takes longer the first time. And source generators can be made to be general purpose, something that automatically applies based on the shape of your project or application of custom attributes, so that you only have to write it once. The community can share and reuse their generators: https://github.com/mrpmorris/Morris.Moxy

Beyond that, if you want to have a discussion about traits or mixins in the language, feel free to open such a discussion. The trait conversation did evolve into default implementations.

[HaloFour]

I do agree that default implementations don't quite get you to traits. The issue is largely visibility, since the members aren't publicly exposed by the implementing class. It's fine if you're always passing the instance around as one of the interfaces, but otherwise it's not exactly a great experience. Source generators help you bridge that gap, but I do think that there should be some first class considerations there.

I'd be interested to see how extensions might also be able to bridge this gap.

[Nefcanto]

@HaloFour, I'm stuck. I am literally stuck. I either have to repeat myself which reduces maintainability and increases bugs. Or I should find a way to write what I need.

I'm stuck because default implementations are extremely limited (no support for instance-level variables, no support for accessing HttpContext, no support for generics, etc etc etc). Because code generation using source generators has a bad integration design. If you need to hear it, then please hear it from me as a developer that it has a bad DX. Of course, bad DX is not a good excuse for not using something valuable, but I have not been able to make it work for me, to include a piece of code between two classes after almost a complete day of work. This is a failure. We can't afford to spend lots of resources to offset the limitations of composition in C#.

I'm stuck. I need to be able to compose read, write, hierarchical read with a clean code and a viable solution. I'm going to see the source of channels and pipes as @CyrusNajmabadi mentioned. Maybe that could help.

Please hear the pain of the community. The pain that we go through is one thing. The fact that you deny it makes it ten-fold. Please hear us.

[HaloFour]

I need to be able to compose read, write, hierarchical read with a clean code and a viable solution.

The issue is you have a very specific opinion as to what a "clean code and a viable solution" looks like informed by solutions you would apply in other languages. Many languages do not have mixins or multiple inheritance. Each language has its own idioms for solving these kinds of problems.

If you want to start a conversation around C# supporting some of these features I would suggest opening a discussion with a more concrete proposal as to what you had in mind. Otherwise, I would suggest visiting a more general purpose help site or the C# Discord.

[CyrusNajmabadi]

I either have to repeat myself

Sometimes that's just life. Our goal isn't to have people have no repetition whatsoever in any part of the solution. Your feedback that you are unhappy with that is def heard, but that doesn't mean it's a priority over the other thousands of things we hear for 100x that many customers.

We have to look holistically at all the feedback and decide where to invest. And this area simply is not that important. We've assessed that to be hte case since we really just do not hear much about this from the ecosystem, and the first, second, and third parties out there don't really find this to be a problem when they're producing their own solutions.

Sure, a novel lang feature here might help some people out, and some apis might get some amount nicer. But it's not substantive enough for us to invest in it at this time.

In the meantime though, we have built technologies and tools to help you take these situations into your own hands so you're not dependent on us and the slow pace of language and runtime design. It's very intentional that we have modes to allow compiler extension, and we will continually push for people to go that route since it means you can get something up and running in a weekend instead of waiting the literal years it will take for us to:

1. decide that thsi is worth working on.
2. designing such a feature.
3. revising the design N-fold over.
4. working with the runtime/bcl to even see if they'll use this.
5. going back to '2', and repeating over and over again.
6. finally solidifying that into a language feature.
7. finally shipping it.

The above is a minimum of 2-3 years if we were to believe this was truly important and were to drop everything else and shift focus to this. That's realistically not happening. So the practical result of that is that if you want a solution here you either wait and hope for us to maybe ever do something. Or you learn use the existing tools and extensions mechanisms to build it yourself.

[CyrusNajmabadi]

but I have not been able to make it work for me, to include a piece of code between two classes after almost a complete day of work.

It's very hard to respond to this as we have no idea what you tried, or hwat issues you ran into. As Halo mentioned, you likely want to just hop onto discord and work with the team and community to help you through any SG issues you're running into. :)

We can't afford to spend lots of resources to offset the limitations of composition in C#.

In a similar vein, we can't afford to spend time on things we don't think will have enough value to warrant the investment. We're both in the position of having to determine where to spend our time and energy to get the best results as we see it.

I need to be able to compose read, write, hierarchical read with a clean code and a viable solution.

I don't know how you define "clean code and a viable solution". Everyone is going to have a different opinion on that. From my own experience i haven't had problems with similar problem spaces, but i don't really grasp (at a visceral level) what your personal problem domain is, and why basic solutions are not viable for it. It's too abstract, and (to be honest) seems too over-architected, trying to model things with an overabundance of generics and inheritance.

Practically, just writing things out with composition (and potentially a little SG) seems like it would totally give "clean code and a viable solution" in terms of how i assess things. Alas, your assessment may differ though. :)

[TahirAhmadov]

I made an SG for composition/mixings/traits:

#5897 (reply in thread)

[CyrusNajmabadi]

Can I create a placeholder like below

I don't know what 'below' means here.

and replace it with the source code?

Yes.

I don't want to create partial classes.

You'll need partial. It's the feature we've had for almost 20 years to support code Gen.

I want to include a code inside the existing code. Is that possible?

Yes. Source generators allow you to automatically generate code you would write yourself, while keeping it up to date.

[CyrusNajmabadi]

Complaining that since it's a netstandard2.0 code, that feature is not supported.

Almost no language features complain about netstandard2.0 (esp. not file scoped namespaces). Source, i was the author of file scoped namespaces and they have no restrictions on TFMs in them :)

There are some features that do depend on having a more modern runtime. But there are very few of these. This would be, for example, if you tried to use Default-Interface-Methods. However, you really don't need to use those in your generator itself. Your generator can, of course, still generate code that uses net-core features. it's just the generator itself that needs to be netstandard2.0.

In terms of lang features, some of them do need polyfills (like required-members if you choose to use those). These can be obtained with polysharp (humorously enough, another generator).

However, for file-scoped-namespaces, there's def no tfm issue, and you won't have any issues on netstandard2.0.

As an example, most of roslyn IDE uses file-scoped-namespaces, and we cross target netstandard2.0, net7.0 and net8.0.

[Nefcanto]

@CyrusNajmabadi, but I get this error:

HierarchicalReadGenerator.cs(6,1): error CS8370: Feature 'file-scoped namespace' is not available in C# 7.3. Please use language version 10.0 or greater.

For this code:

using System;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.CSharp.Syntax;

namespace SourceGenerators;

[Generator]
public class HierarchicalReadGenerator : ISourceGenerator
{
    public void Execute(GeneratorExecutionContext context)
    {
        foreach (var syntaxTree in context.Compilation.SyntaxTrees)
        {
            var visitor = new HierarchicalClassVisitor();
            visitor.Visit(syntaxTree.GetRoot());
            var hierarchicalClasses = visitor.HierarchicalClassNames;
        }
    }

    public void Initialize(GeneratorInitializationContext context)
    {
#if DEBUG
if (!System.Diagnostics.Debugger.IsAttached)
    System.Diagnostics.Debugger.Launch();
#endif
    }
}

And if I put the class inside the namespace curly braces, the error goes away.

[colejohnson66]

Use <LangVersion> in your csproj

[chrisoverzero]

error CS8370: Feature 'file-scoped namespace' is not available in C# 7.3. Please use language version 10.0 or greater.

Have you considered setting the language version to 10.0 or greater? Different target frameworks have different default language versions. TFM netstandard2.0 maps to C# 7.3, which is exactly what the compiler is telling you.

[CyrusNajmabadi]

Feature 'file-scoped namespace' is not available in C# 7.3. Please use language version 10.0 or greater.

Yes. Newer language features will require newer language versions. That's how things have worked since we shipped 2.0. However, outside of a tiny number of features that require an updated runtime, nearly everything works with netstandard2 (though I do recommend polysharp to make things easier). :-)

[Nefcanto]

@CyrusNajmabadi, apparently the Source Generation curve is too high. After almost two days I could not do it. I'm not a junior developer. Thus I consider it to be fairly complex for a composition requirement.

This is what I ended up doing. I created a preprocessor (getting the idea from CSS) that runs before we get to MSBuild. I created a simple syntax for inclusion. And I combined it with the idea of partial classes (though a dirty solution IMO).

So, I have this file called Read.snippet. A sample code can contain this content:

    public List<T> GetList(ListParameters listParameters)
    {
        // real code here, but won't bother the compiler because of the extension
    }

Then in my ReadBusiness.cs I would create a placeholder for it:

public partial class ReadBusiness<T> : where T class, new()
{
    //Compose:Read
}

Then using my preprocessor which is written in Python and in my build pipeline before I call dotnet build, I would find all files that contain the //Compose: in them, find what they need to compose, find their class syntax, and based on that info I create generated files (basically source generation but easier):

public partial class ReadBusiness<T>
{
    public List<T> GetList(ListParameters listParameters)
    {
        // real code here, but won't bother the compiler because of the extension
    }
}

I tested it and it worked. And it took less than 3 hours to be built.

But I felt very bad being rejected by the .NET team and being presented with hard solutions for something that based on my 18 years of coding is absolutely necessary for the future of .NET and C#.

I hope your higher-ranking managers see this post. I suffer communicating with the .NET team. If we did not have 63183 lines of code in C# I would definitely consider moving away from it. C# needs to have better composition support. C# lacks composition support. Yes, it might have some workarounds to mimic composition and mixin. Yet those workarounds are expensive, hard to implement and maintain, and hard to learn and communicate with the rest of the team.

C# needs first-level keywords to support composition.

Thank you all.

[CyrusNajmabadi]

This is what I ended up doing. I created a preprocessor

Sounds like a great solution. Glad you were able to make it work so easily. Given this, it def seems like overkill to have a huge investment in a language feature that can so easily be solved with existing tech.

Thanks for the info and insights. It def helps feed into future designs! :-)

If you have more ideas, don't hesitate to add them here or in new discussions if they're in new topics!

[CyrusNajmabadi]

After almost two days I could not do it

Feel free to hop into discord if you get into this space again. Based on your solution description, this would be something we could pretty quickly and easily help you setup with source generators. Thanks!