Optional References

[Kukkimonsuta]

I'd like to open discussion about having ProjectReference / PackageReference option to be optional. I suspect this will be bit controversial as it might seem like I want to make the language weaker/more dynamic, but please bear with me and see the Why? section - I welcome any other alternative solutions reducing dependency noise.

What?

Allow referencing a library without importing it as reference.

• All types using types (and all types using those, etc.) from such library would be available only if given library was already loaded.
• Attributes that are not available would be ignored, but decorated types would still be available.

Why?

As library author I'd like to support many things out of the box - for instance I'd like my types to be properly serializable by Newtonsoft.Json, Protobuf.NET and MessagePack. To provide such support I need to docorate my types and/or implement converters both of which need to use types from given libraries. Currently there are two ways I can go:

1. Introduce dependency on Newtonsoft.Json, Protobuf.NET, MessagePack. This allows me both to decorate types properly and implement converters.
2. Introduce 3 new assemblies each providing support to given library (MyLibrary.Json, MyLibrary.Protobuf, MyLibrary.MessagePack..). This allows me to implement converters and hoping DataContract attributes would be enough to decorate my types in core library.

The second approach might seem like a good way to go, but it gets quickly out of hand when you have set of libraries instead of only one.

How?

I'm inexperienced in this area, so I might make some wrong assumptions here - please correct me if I do.

As far as I know there may be methods in a assembly that use missing types and it won't matter unless you try to do something with them, meaning it might be enough if compiler could track what type depends on what optional reference (using attributes probably?) and block me from using given types when not referencing required assemblies.
See reply from tannergooding below.

Possible issues

• there would have to be API that would allow identify types that currently can't be loaded due to missing optional reference so that enumerating on types in assembly using reflection won't fail (or hide those from reflection completely?)
• there would have to be special handling for attributes as using unavailable ones must not block usage of decorated type when optional reference is missing

[tannergooding]

I don't believe the IL metadata allows this.

The actual emitted metadata frequently requires a typeTok (type token). This is enforced to by a type definition, a type reference, or a type specification.

In all cases, the typeTok has to carry the appropriate metadata to tell the runtime how to load the type when it is first required.

There are other programming techniques that will help you resolve this problem, such as dependency injection.

[svick]

@tannergooding How does dependency injection help you make types from your library serializable by different serializers?

[tannergooding]

You create your own interface that defines the core serializable members, and implement a wrapper for each framework you want to support, you can then decide which serializer to load/use at runtime.

Dependency Injection means that you can say, I just want to add this Newtonsoft depdendency for this app or just Protobuf for this other app, or maybe both of them for this third app.

[svick]

@tannergooding That sounds like the "Introduce 3 new assemblies each providing support to given library" solution mentioned in the initial post. I was assuming you were talking about some other solution.

[tannergooding]

@svick, No. As far as I am aware, that is really the best choice today (assuming you want to avoid having some crazy number of method overloads or just loading everything at runtime, even if it might not be needed). You create an abstraction, and dynamically choose your dependencies at runtime (I generally just use MEF, since it is simple, easy, and can be quite powerful).

[yaakov-h]

This sounds like a runtime and compiler change, not a language change.

[erik-kallen]

Another use for this would be logging, so you could write something like:

if (typeof(log4net.LogManager) != null)
{
    // Log with log4net
}
else if (typeof(NLog.LogManager) != null)
{
    // Log with NLog
}

[Kukkimonsuta]

@erik-kallen Current proposal wouldn't make such thing possible because the whole type containing any reference to optional assembly would only be available if the assembly was present - meaning your method and type containing such method would run only if there was both log4net and NLog referenced.

@yaakov-h Is there a better place to discuss this? Should I move the discussion?

@tannergooding What would be the least painful way to implement this knowing these constraints? Would moving affected types into additional modules (and I suppose modifying runtime to allow optional modules?) help?

[erik-kallen]

@Kukkimonsuta Perhaps there is the possibility to amend the proposal?

[tannergooding]

@Kukkimonsuta, You have to have the metadata describing what you want somewhere.

You can access the types directly, which will cause the assembly to be loaded by whichever method uses said types.

or

You can create an abstraction over the types so that they are only loaded if needed or only used if loaded (depending on your requirements).

It sounds like you don't want the former because you want to prevent loads of unused or unneeded dependencies or because you want to follow "good practices" and have clean layers /separation of concerns/etc.

For creating an abstraction, you have several methods. My recommendation is:

• Create an abstract class or interface that defines a common API surface
  • This is probably the fastest
  • This requires you to create a sub-type from the interface for each "dependency"
  • You can load these yourself (ex: Assembly.Load and Activator.CreateInstance), based on some logic
  • You can use something like "dependency injection" (ex: MEF, Ninject, etc) so a consumer can tell you which dependencies they want

You can also use reflection/dynamic to achieve the same thing (albeit slower), without having to create the wrapper/contract type.

There are probably other mechanisms as well that I don't know about or aren't thinking of.

I don't think this is a new problem and I don't see a situation where the solutions we have for this today are insufficient enough to warrant language or runtime changes to support this.

[tannergooding]

Also to be clear, that is just my opinion on this. Others may think elsewise and maybe someone on the compiler or runtime team feels that it is an issue that is worth investigating.

[yaakov-h]

@Kukkimonsuta I’d suggest starting with dotnet/coreclr.

[svick]

@erik-kallen You kind of already can do that, though it's not very nice and the tooling won't help you at all.

Consider this code, that tries to configure and then log using both log4net and NLog:

ExecuteIfReferencesAvailable(() => log4net.Config.BasicConfigurator.Configure());
ExecuteIfReferencesAvailable(() => NLog.Config.SimpleConfigurator.ConfigureForConsoleLogging());

ExecuteIfReferencesAvailable(() => log4net.LogManager.GetLogger(typeof(Program)).Info("Logging from log4net."));
ExecuteIfReferencesAvailable(() => NLog.LogManager.GetLogger("Program").Info("Logging from NLog."));

If I compile this code and then delete one or both logger assemblies from the output directory, then the ones that weren't deleted do log and otherwise the code continues uninterrupted.

What's the magic?

static void ExecuteIfReferencesAvailable(Action action)
{
    try
    {
        action();
    }
    catch (FileNotFoundException ex) when (ex.FusionLog != null)
    {
    }
}

(The check for FusionLog is my attempt to differentiate FileNotFoundExceptions caused by assembly loading.)