[Ramblings] Default Interface Methods and the Fragile Substitution Problem

[DavidArno]

Let's say I have an interface and a couple of types that implements it:

interface IFoo
{
    int F1(int value);
}

class Bar : IFoo
{
    public int F1(int value) => value;
    public int F2(int value) => value - 1;
}
class Baz : IFoo
{
    public int F1(int value) => value;
}

Then, when the default interface methods feature is released, I modify IFoo:

interface IFoo
{
    int F1(int value);
    int F2(int value) => value + 1;
}

Now according to my understanding of the Liskov Substitution Principle (LSP), by modifying IFoo in this way, I've created a situation where F2 has a certain expected behaviour, which Baz respects. But Bar doesn't, through no fault of its own, thus breaking the principle.

This strikes me as being similar to the fragile base class problem. In that latter case, changes to a base class can cause sub classes to malfunction, as base classes are inherently tightly coupled to their derivatives. In the case of interfaces, the fragile base class problem shouldn't occur as interfaces will have no state. Instead, a change to the interface causes a method in an implementation to become a substitution, which can then break the LSP. The term "fragile substitution problem" seems a good name for this.

Clearly, F2 contains no useful semantic information and thus it's hard to argue that it has any promise of behaviour, so what Bar and Baz do here is undefined by the name. However, out there in the big wide world of millions of lines of code, I'm assuming this could be a real problem at some stage.

Now I'm not in anyway arguing that we should abandon default interface methods (that conversion has been well and truly had), more highlighting a potential problem it could bring for the unwary.

Is this really a problem? Should we care? I'm not sure, I just thought I'd share my thoughts on this matter.

[benaadams]

Making default interface methods only be overriden explicitly would solve the scenario.

However might make it annoying for new implementers of the interface and new interfaces that provided convince implementations would then be less convenient to override.

[HaloFour]

@DavidArno

I don't see where this is a problem. The implementation details of an interface is not a part of the contract.

The "fragility" also exists today in that you can decide to break and refactor an interface by adding new members and not realize that existing implementations already implicitly implement those members by matching the signature but not in the manner expected.

You'd probably be a lot more annoyed if C# worked like VB.NET and implementing members were required to be decorated as such:

Public Class C : Implements IDisposable
    Public Sub Dispose() Implements IDisposable.Dispose
    End Sub
End Class

[VisualMelon]

I certainly care. As @benaadams suggests, requiring an override modifier is absolutely necessarily, if implicit overriding of default implementations is allowed at all (I'm not sure that this is a given, reviewing the recent LDM notes).

@HaloFour the more general problem is that the proposal amounts to 'infecting' (for lack of a better word) the lovely and safe world of interfaces (a contract, no behaviour) with the potential for misery that is inherent in inheriting dynamic behaviour. Overriding a default implementation becomes a risky business if you don't control the interface itself.

You are quite right about the existing fragility, but at the moment people have to think hard about adding to APIs because they are always a breaking change (silent or otherwise). Default implementations (without explicit override for implicit methods) is a way that something which otherwise looks 'safe' can silently go wrong. Personally I would prefer that C# didn't allow implicit implementation (with a tidy mechanism to expose the methods when casted to the sub type), but I suspect this is a minority viewpoint.

[benaadams]

requiring an override modifier is...

No override required Just an explicit implementation:

class Bar : IFoo
{
    public int F1(int value) => value;
    public int F2(int value) => value - 1;

    int IFoo.F2(int value) => F2(value); // explicit override of interface default
}

If it went that way...

[VisualMelon]

@benaadams apologies, I was perhaps not clear. override would be necessary for implicit implementations (as per the OP) if they are allowed. We are in agreement, I think, that there is no issue with already explicit implementations.

[HaloFour]

@VisualMelon

the more general problem is that the proposal amounts to 'infecting' (for lack of a better word) the lovely and safe world of interfaces (a contract, no behaviour) with the potential for misery that is inherent in inheriting dynamic behaviour. Overriding a default implementation becomes a risky business if you don't control the interface itself.

Tell us how you really feel?

This is an academic issue. Default implementations were released with Java 3.5 years ago and this issue has not cropped up, and that language has fewer tools when it comes to how a class may implement an interface.

requiring an override modifier is absolutely necessarily, if implicit overriding of default implementations is allowed at all (I'm not sure that this is a given, reviewing the recent LDM notes).

I'm not aware of any LDM notes that touch on this. There was a question about requiring interfaces that extend other interfaces to provide a more specific implementation if the base interface had a default implementation but the team decided against that. It's up to the implementing class to resolve ambiguities arising from member collision.

[VisualMelon]

@HaloFour

Tell us how you really feel?

Sorry, I don't understand the question

This is an academic issue. Default implementations were released with Java 3.5 years ago and this issue has not cropped up, and that language has fewer tools when it comes to how a class may implement an interface.

I will concede that I have little experience with ('modern') Java, but all I'm saying is that currently, when you implement an interface, you know everything about it, because it is only what it says it is (it is just a contract, no behaviour). Adding dynamically overridable behaviour to interfaces means they cease to be a simple and predictable component to something that, well, isn't. The proposal undoubtedly makes interfaces more powerful, but having something less powerful is often key to reliable system design. Arguably, the proposed changes turn interfaces into a more powerful abstract class, and leaves the language without interfaces: C# is loosing a really great feature. Just because the new feature hasn't gone as wrong as it could doesn't mean the constraints of the old feature won't be missed (I wouldn't care nearly as much multi-inheritance of abstract classes was proposed!).

I'm not aware of any LDM notes that touch on this...

I looked through them again... and I can't find them either, which is a bit surprising (perhaps why I hedged at suggested they weren't a given!). There is discussion about 'non-virtual members', wherein it is implied that implicit implementation will be carried out. (I am terrible at finding things online, if you can find something that contradicts me, I would be very glad to be made aware of it!)

I'm aware that the implementing class has to resolve ambiguities, but if there is no ambiguity (as in the OP) then the behaviour can still change. My concern is that adding a new method to an interface with a default implementation will (as far as I can tell, given my understanding) never cause a compiler error. This creates the illusion that it is a 'safe' thing to do. By requiring override when there is behaviour to override, then the implementing class has to acknowledge this, solving the problem in the OP, and maintaining commonality with existing usage of override. It seems a small cost in typing to provide a little more clarity at the call-site and mitigate a possible problem.

There is so much that can go wrong when an interface changes, and adding a default method is just another way to change an interface. It is my opinion that (if implicit implementations are allowed) that they should require override. I'll qualify that by saying that much of my complaint can be levelled at existing implicit implementation, but given we already use override to override behaviour, I can see no good reason not to require it, given it helps to mitigate some of the many issues with implicit interface implementation in the context of the proposal. (Again, I'm not clear whether implicit implementation will even be allowed: I'd accept an argument that it is already used, so I would be opposed to it, so long as it requires override)

I must apologise for how long this is...

[alrz]

If I understand this correctly, this means that bar.F2 results in a different value than ((IFoo)bar).F2. There are other cases where this can happen like using new or explicit implementations but both of them are "explicit". I'm not sure if override could solve the problem either. Imagine if the interface and the implementation are declared in different assemblies, the compiler doesn't even know that Bar.F2 is virtual.

[VisualMelon]

@alrz My understanding was that bar.F2 overrides ((IFoo)bar).F2; however, on HaloFour's prompting I have looked for something to support this in the LDM notes and failed to find it... so I'm not so sure now! The behaviour you suggest is (in my book) preferable, because bar.F2 can't be 'accidently' called on the grounds it implicitly implements IFoo.F2. The implicit 'overriding' behaviour is how adding a non-default method to an interface already works (and is one of the many reasons I dislike implicit interface implementation in C#).

That last part (cross assembly stuff) is another kettle of fish, and has been discussed: here are some LDM notes. I think I have yet to fully absorbed the issues therewith (but I'm sure that being more explicit can't make them worse!)

[HaloFour]

@VisualMelon

when you implement an interface, you know everything about it, because it is only what it says it is (it is just a contract, no behaviour).

That doesn't change here. The interface is still a contract which enforces that the members are available to the consumer. How the interface is implemented has always been an implementation detail. The implementation could be satisfied by a public non-virtual member on the base class which just happens to match the signature. It doesn't even need to be explicit in IL, the CLR will wire up the implementation at runtime. You might not like that "dynamic" behavior, but that's the way it's always been. Even if C# went the VB.NET route and required an explicit override at compile-time it couldn't change the implicit implementation permitted by the CLR.

@alrz

The complaint is that Baz.F2 will implicitly override IFoo.F2, which is true. It's also always been true, even if IFoo and Bar exist in different assemblies.

I guess the complaint is that with default interface methods the problem might arise more often as people are going to be adding new members to their interfaces with wanton abandon.

[alrz]

The implementation could be satisfied by a public non-virtual member on the base class which just happens to match the signature. It doesn't even need to be explicit in IL, the CLR will wire up the implementation at runtime.

I don't think so. an implicit interface impl does emit virtual in the method signature.

[HaloFour]

@alrz

I don't think so. an implicit interface impl does emit virtual in the method signature.

Hah, so it does, if they are in the same project. It's amusing that the compiler will modify the signature of the parent type even if it doesn't directly implement the interface.

I swear that I've tried this before in IL with a non-virtual member and it worked. I'll give it another try. Either way, same problem, just more limited if the types are in different assemblies.

[alrz]

Just checked, if base member is not virtual by any means (and not in the same assembly), it will emit a private member (explicit impl) with .override.

[HaloFour]

@alrz

That it does. That's probably the behavior I saw before. It's interesting the length that the compiler will go to make implicit implementation seamless.

The CLR also supports it so some extent as well as long as the members are compatible, e.g. public, virtual with the exact name and signature. You only explicitly need the .override modifier if the method has a different name or accessibility.

[alrz]

Now, as I described, with such requirements in clr, runtime has no information to make bar.F2 and ((IFoo)bar).F2 call the same impl if the interface is declared and later changed outside assembly boundaries.

The presence of override in source would have no impact to the final IL because where we can detect such impls we will emit appropriate IL anyways so requiring it is just inconvenience while the root issue is unavoidable unless at runtime we walk through the hierarchy unconditionally (even in the absence of virtual marker in IL) which I think is an expensive (or impossible?) thing to do.

[VisualMelon]

@HaloFour all true, and be in no doubt that I love interfaces as they are in C# (including their constraints, the dynamic behaviour, etc. these things are all useful). Hopefully this will finally clarify my position!

I was referring more to the behaviour of an interface as seen from the author of an implementing class (as opposed to a consumer). The Fragility is that the default behaviour could (though it will surely be rare and a result of poor decisions) work at odds with the code in the implementing class (there is a crazy example in the Wikipedia article the OP linked to). It is certainly an edge case, but my point is that an absolute confidence has been lost, and the interface construct is now open to all manner of abuse. Of course, all of these horrible things are already possible with class inheritance; a miserable developer like me would perhaps rather than they weren't possible (if unlikely) with interfaces. Inheritance has its place, and interfaces have their (much more important) place, and they are not the same place.

The prospect of the exposed behaviours of my object as seen by colleague's code when 'projected' through a third party interface being changed by this third party does not appeal! If the interface changes then I want my compiler to tell me about it, and (though I know this isn't your question) I think requiring override will reduce the opportunity for misery while being consistent with the rest of the language.

[VisualMelon]

@alrz I didn't choose my words very well... I meant rather to suggest that life would be better if we require override; not for the compiler's benefit, but for the developer's benefit, as there would be one fewer situations were the implicit implementation can occur 'by accident' and it looks the same as a 'normal' base-class method overriding.

[HaloFour]

Considering that the vector for this issue has existed in the language since C# 1.0, is a known liability with implicit interface implementation and has not demonstrated an actual burden to developers I don't think that there is any value in changing the language to require an explicit override. Having interface implementation be sometimes implicit and sometimes explicit will only be confusing and can only "solve" the problem in a limited set of circumstances.

[VisualMelon]

@HaloFour fair enough,

I think we finally understand each other (I create confusion everywhere I go); I don't necessarily disagree with anything you've said, but I stand by my preference for the reasons given above, and I do think the OP presents a valid concern that should be considered (not that I should wish to imply that it hasn't already been taken into account).

[alrz]

@VisualMelon

fewer situations were the implicit implementation can occur 'by accident'

When we're talking about an implicit implementation in the same source as the interface is declared then there is a chance that it might be actually expected to work that way i.e. implicitly, it will help you to have source-compat when you want to group some existing members in an interface and have them silently conform to that interface.

I agree that the way default interface implementations interact with implicit implementations is unfortunate but if you amend either side you're nullifying the value they are providing in the first place, like requiring override on "implicit" impls or rejecting a class that does not implement a defaulted interface member.

[VisualMelon]

@alrz

there is a chance that it might be actually expected to work that way

Quite so, but there is also a chance that it isn't meant to work like that (or even, if the method 'just about' conforms, it could readily hide an edge case), and the compiler can't know, so the spec shouldn't assume intent that hasn't been stated. (As noted prior, this is a long standing problem with implicit implementation)

I would argue that the 'value' from implicit interfaces comes from the fact that the method is exposed by the class itself, not just when it is projected as an interface. Requiring override from the outset would not have detracted from this.

[alrz]

@VisualMelon

I would argue that the 'value' from implicit interfaces comes from the fact that the method is exposed by the class itself, not just when it is projected as an interface. Requiring override from the outset would not have detracted from this.

Yes, it could be the case at times but so far it's been a "feature" to have implicit implementations and as HaloFour said it hasn't been much of an inconvenience. Although, you could have an analyzer to require [Override] on such members. Java has had this problem and they've solved it the exact same way but supposedly it's been so painful that they've added this built-in in the language.

[HaloFour]

@alrz

In Java's case it wasn't interface implementation that was the problem. All implementation in Java is implicit.

Java added the @Override annotation specifically to double-check overriding methods from a base class. As Java lacks an override modifier the only way you override a method is by duplicating the method signature in the derived class. If you make any mistakes, e.g. misspell the name, rather than overriding the method you're defining a new method and leaving the original implementation intact. This was a fairly common cause of bugs. When you annotate a method with the @Override annotation what you are telling the compiler is that if that method isn't actually an override that it should cause a compiler error which should catch those mistakes.

[alrz]

@HaloFour

Yes, I think C# learned a lesson there and introduced override and new modifiers, but there's been enough merit to it that they kept it for interfaces.

[OJacot-Descombes]

If an automatic implicit implementation of an interface default implementation is desired in a class, we could write:

class Bar : implicit IFoo
{
}

This would allow us to access all the default implementation members through the class.