Write custom operators like Swift

[HamedFathi]

In Swift, there can be operators that operate on 1, 2, or 3 values. We call those operators unary, binary, or ternary (respectively).

Swift has 4 types of operators, 3 of which are available for us to use with custom operators they are:

prefix A unary (operating on one value) operator that is just before the value it operates on.
postfix A unary operator that is just after the value it operates on.
infix A binary (operating on two values) operator placed between the two values it operates on.

Reference : http://www.codingexplorer.com/custom-operators-swift/

It would be great if C# Next support for writing custom operators, can be a way to access DSLs in .NET.

The syntax can be like

public prefix operator double % (int percentage) {
    return ((double)percentage) / 100)
}

//Example Use
var test = %42 // = 0.42

//=====================================

public infix operator double ^^ (double value,double power) {
    return Math.Pow(value, power);
}

//Example Use
var test = 2^^3 // = 8

//=====================================

public postfix operator double % (int percentage) {
    return ((double)percentage) / 100)
}

//Example Use
var test = 42% // = 0.42

//=====================================

[casperOne]

1. What's the scope?
2. What happens if there are multiple conflicting operators in scope? At compile time? At run time?
3. What's the order of precedence (how does it interact with operators defined on a class of the target type)?
4. What are acceptable operator characters?

[Logerfo]

@HamedFathi This feature interests me, I hope you don't mind if I try to answer @casperOne questions, so we might later achieve a proper proposal. I'm also considering being able to define operators both in instance classes (like C++) and static classes (like extension methods).

1. The same as extension methods are today.
2. The same as extension methods are today.
3. Suggestions?
4. It depends on other questions, like: How many characters can an operator have?

May I also ask: Would the already existing operators be accepted as static ones?

[casperOne]

@Logerfo With your answer to 2, there's a problem; with extension methods, if there's a conflict, I can always use the fully qualified type name and method name to call the method.

With operators, how do we call this if there's a conflict? I see no way to do that right now.

Order of precedence should be the same as extension methods; if there's an operator on the target type then that takes precedence, otherwise, an "extension" operator (for lack of a better term) could be applied.

How many characters can an operator have? I can see where this would get unwieldy/confusing with just one character.

As for existing operators, they would have precedence, as they are defined on the target class already, by point 2.

[YaakovDavis]

It would be nice if operators could have named identifiers, eg.

var fraction = 5 by 7; //infix
var box = wrap value; //prefix
var x = 5 percent; //postfix

[yaakov-h]

@YaakovDavis I'm suddenly a huge fan of named postfix operators.

Temperature w = 5 celsius;
Temperature x = 5 farenheit;
Weight y = 20 kilograms;
TimeSpan z = 2 days;
process.WaitForExit(20 seconds);
// ...

[YaakovDavis]

@HamedFathi
Since you seem to like the naming suggestion, would you mind augmenting the proposal?

An example declaration:

public infix operator Fraction by (int nominator, int denominator) {
    return new Fraction(nominator, denominator);
}

[barld]

With the names it sounds like the units of measure that exist in for example F#
Is that what you mean @yaakov-h

Here is an article of the feature in F#: https://docs.microsoft.com/en-us/dotnet/articles/fsharp/language-reference/units-of-measure

[YaakovDavis]

Units of measure in F# are lost at compile time.
Named postfix operators can return specific types which are preserved at runtime. This makes them more useful in that context, IMO.

Note though that "units" is just one use case of postfix operators. Conversions are another.

[lachbaer]

I fear that somebody is going to write a weired operator %§ that nobody is going to understand what it actually did, not even the creator when coming back after some time.

The idea of named postfix operators is actually quite good. But IMO they should be declared and used like extension methods (with . punctuation).

// Extension operator on type int:
public shared explicit Temperature operator celsius(this int left) 
    => new Temperature(left);
public shared explicit Temperature operator farenheit(this int left)
    => new Temperature((left - 32) / 1.8);

// Usage
Temperature w = 5.celsius;
Temperature x = 5.farenheit;

[yaakov-h]

That would then come back to Extension Everything.

There is still value in custom prefix and infix operators, and non-named postfix operators too. You can go completely crazy and write horrible code, but you could already do that...

[lachbaer]

@yaakov-h of course you can already write completely crazy code and have op_add be a substraction operator 😁 But with the standard (symbolic) operators there it at least an expectation that I have on its operation. But with whatever operator is customly written I don't know the slightest meaning behind it, even when it looks okay, like e.g. ~>.

That would then come back to Extension Everything.

Yes, for the example above the extension then was a property celsius.

[yaakov-h]

Ah yes, the twiddle-wakka operator (as paket calls it)

Infix I would expect to mostly be used for making mathematical code look more like the formulae that it represents.

Prefix... only thing I can think of off the top of my head would be currency literals, e.g.

public prefix operator Currency AU$(this double value)
    => new Currency(Currencies.AustralianDollar, value);

public prefix operator Currency €(this double value)
    => new Currency(Currencies.Euro, value);

[YaakovDavis]

@yaakov-h
@lachbaer

Custom operators, named or symbolic, can be used to build mini embedded-DSLs.

Example 1 - a parser combinators (ala FParsec):

var parser = maybe stringLiteralParser^ then char ':' then (intParser or floatParser)^ then digit.* then eof;

^ consumes the parsed value (similar to .< in FParsec).
Of course, one could define such a lib with symbolic operators as well.

Example 2 - FRP:

var cell = lift value using transformer;

Symbolic:

var cell = value .> transformer;

Example 3 - UI:

var dockPanel = dock textBlock onTop then tree toLeft then fillBy mainPanel;

And with symbolic operators:

var dockPanel = dock textBlock.^ then tree.< then mainPanel.*;

Those are just ad-hoc examples. Using such DSLs, one might create safer and easier codebases.

[casperOne]

@yaakov-h I don't agree that currency literals would be referenced with prefix operators. I see those as a postfix operator (just like Celsius and Fahrenheit).

Of course that's a style thing, it's not here or there, but currency feels unnatural to me as a prefix.

[lachbaer]

@yaakov-h

Prefix... only thing I can think of off the top of my head would be currency literals

In Germany we put the currency as postfix "operator":

So even here the language will diverge just because of regional differences.

var dockPanel = dock textBlock onTop then tree toLeft then fillBy mainPanel;

Where the fu... nny bunny is the puke smiley????? 😷

[YaakovDavis]

@orthoxerox Those restrictions aren't too harsh. The user can provide this information to the compiler.
With an additional rule that the built-in ones always have higher priority, I think there's an opportunity here for something nice.

[louthy]

I fear that somebody is going to write a weired operator %§ that nobody is going to understand what it actually did, not even the creator when coming back after some time.

I really find this argument frustrating. Yes, people can do stupid things. I could write an API that looks like this:

    public class A
    {
        public void B();
        public void C();
        public void D();
        public void E();
    }

But I don't, because it's idiotic. This is akin to saying 'users can't be trusted'.

Defining operators shouldn't be any different to defining functions. They're just functions with symbols for their names rather than words (although I'm totally for words too). If the tooling allows you to Go To Definition then a lot of the issues with knowing what they're for go away. But their conciseness for certain situations is massively valuable.

It could open a door to a ton of useful operators for functional programming (for example). In my library I have an Apply function for Option, Lst, Either, Try, etc. (for applicatives). Apply is an operator, but I can't define an operator to do the application. For monads I have a Bind function, it is also an operator, yet I have to chain the Bind functions, which is pretty ugly. We could also have a 'pipe' operator for function composition for example.

@CyrusNajmabadi

One of those goals being that SyntaxTrees can be parsed in a completely context free manner. If parsing now needs to know about the operators you may have defined elsewhere (including in metadata, the same file, or other files), then that's just a non-starter.

But surely this is already an issue with overridden implementations of existing operators? If I change the + operator on a type, then the compiler needs to resolve that, and use the correct implementation. I understand that this resolution process would be more complicated, but how is it any more complicated than say extension method/concrete method resolution? (although I admit the precedence and associativity is an issue, but perhaps a system like F# where the first char of the operator defines the precedence --- although it's one thing I find frustrating about F#'s operators)

[svick]

@louthy

This is akin to saying 'users can't be trusted'.

That's not quite it. What code users write depends a lot on what tools you give them and what tools you don't give them. For example, if you give them the option to define custom operators, but not to overload them, then you'll end up with |>, ||> and |||>, all doing essentially the same thing, just like F# did. Or if you give them yield return, but not await, they will abuse the former for the latter, just like Unity did.

That's what "pit of quality" means: when you give your users some tools, you need to make sure you're giving them the right ones and especially that you're not missing something.

But surely this is already an issue with overridden implementations of existing operators?

It's not. On the syntactic level, the expression a + b always has the same meaning (BinaryExpression(SyntaxKind.AddExpression, IdentifierName("a"), IdentifierName("b"))), no matter what the types of a and b are or what operators the define. Only at the semantic level, types, overloaded operators, metadata, etc. start to matter.

how is it any more complicated than say extension method/concrete method resolution

It's more complicated, because the choice between an extension method, an instance method or even a delegate property invocation can happen at the semantic level. On the syntactic level, a.M() has always the same meaning (InvocationExpression(MemberAccessExpression(SyntaxKind.SimpleMemberAccessExpression, IdentifierName("a"), IdentifierName("M")))). If parsing depended on types and overloaded operators and referenced assemblies, you would basically have to throw away the whole SyntaxTree API.

[louthy]

That's not quite it. What code users write depends a lot on what tools you give them and what tools you don't give them. For example, if you give them the option to define custom operators, but not to overload them, then you'll end up with |>, ||> and |||>, all doing essentially the same thing, just like F# did. Or if you give them yield return, but not await, they will abuse the former for the latter, just like Unity did
That's what "pit of quality" means: when you give your users some tools, you need to make sure you're giving them the right ones and especially that you're not missing something.

That's a massive dose of hyperbole right there. You're essentially saying users can't be trusted, as I stated. And you're also implying that all other languages that allow you to define custom operators have fucked up. It's a ridiculous position to take.

If parsing depended on types and overloaded operators and referenced assemblies, you would basically have to throw away the whole SyntaxTree API.

Another massive dose of hyperbole. The expression doesn't have to be:

    BinaryExpression(SyntaxKind.AddExpression, IdentifierName("a"), IdentifierName("b"))

It could be:

    BinarySpecialOperatorExpression(OperatorName("==>"), IdentifierName("a"), IdentifierName("b"))

Anything can be deferred until the semantic analysis. The only issue I can see is what was raised by @CyrusNajmabadi:

One of those goals being that SyntaxTrees can be parsed in a completely context free manner. If parsing now needs to know about the operators you may have defined elsewhere (including in metadata, the same file, or other files), then that's just a non-starter.

Because of it being context free it makes it impossible to ascertain when building the SyntaxTree what the associativity and precedence is. I accept that would mean a significant change to the SyntaxTree system, and therefore would be a non-starter. But it would be possible to bake associativity and precedence into the operator name itself. This is the F# table:

• Any prefix operator - Left
• Infix operator starting with ** - Right
• Infix operator starting with *, / or % - Left
• Infix operator starting with - or + - Left
• Infix operator starting with ^ - Right
• Infix operator starting with !=, <, >, =, |, &, $, @ - Left
• Infix operator &, && (no subsequent characters) - Left
• Infix operator or or || - Left
• Infix operator := - Left

That's not an ideal solution, but it would give an implementation path that seems achievable.

[HaloFour]

@louthy

In short, yes. That's precisely why the C# team carefully chose a very limited list of overloadable operators, they watched users of other languages screw it up, whether out of inexperience or out of being "clever". I, for one, don't want C# to turn into something that looks like my cat walked on the keyboard. Nor do I want to deal with any languages trying to hack in all of the lunacy and subtlies if the English language into some kind of DSL that is incomprehensible in all but the simplest cases. If I wanted either of those I have any number of other languages I could choose from.

[svick]

@louthy

You're essentially saying users can't be trusted, as I stated.

I wouldn't phrase it that way, but okay, to some degree, yes.

And you're also implying that all other languages that allow you to define custom operators have fucked up. It's a ridiculous position to take.

I have never said anything like that. I'm saying that when adding new feature, you have to carefully evaluate it, consider how likely it is that it's going to be misused and think if it aligns well with the goals of your language.

Maybe other languages with custom operators have different goals. Maybe their custom operators are not misused because of some other feature or because of their target audience. Maybe custom operators are not likely to be misused, period. I don't know. What I do know is that concerns about misuse should not be dismissed.

Another massive dose of hyperbole. […] Anything can be deferred until the semantic analysis.

It's not hyperbole. It's an issue some implementations of custom operators have, including those that let you specify how an operator syntactically behaves (mostly associativity and precedence, as you noted), or those where any identifier can be operator.

And the quote by @CyrusNajmabadi you originally reacted to was talking about the latter, where It becomes non trivial to determine what actually is an operator was to be a legal C# code (though it might not have been clear, if you didn't follow the conversation closely).

[louthy]

@HaloFour

In short, yes. That's precisely why the C# team carefully chose a very limited list of overloadable operators, they watched users of other languages screw it up,

Can you link me to the discussion on this? I am interested what rationale the C# team used when deciding that other languages had screwed up.

I, for one, don't want C# to turn into something that looks like my cat walked on the keyboard. Nor do I want to deal with any languages trying to hack in all of the lunacy and subtlies if the English language into some kind of DSL that is incomprehensible in all but the simplest cases

I agree. The solution is to not use the libraries that abuse the power. Just like I wouldn't use a library with a type like this in it:

    public interface _
    {
        _ _(_ _);
        _ __(_ _);
        _ ___(_ _);
        _ ____(_ _);
        _ _____(_ _);
    }

(Yes, that's valid C#)

@svick

It becomes non trivial to determine what actually is an operator

Most language lexers have a set of valid operator characters [1]. Doesn't seem to be too difficult to decide if something is an operator or not. That may be the case when using infix identifiers, but I struggle to see how that's the case with punctuation characters. For identifier based operators then following a similar approach to Haskell and surrounding the identifier with backticks would work.

[1] Example

[orthoxerox]

@louthy when you bake fixness, associativity and precedence into the operator token, this system starts to restrict your operator choice almost just as much as having a handful overridable operators does. Let's say I want to implement a PEG DSL. / is the preferred operator for ordered choice, a single space would've been ideal for concatenation, but I still have to use a real infix operator. What's more, its precedence must be higher than /, so I can't use '+'. I'm still restricted.

What I wouldn't mind at all is getting a few more pre-baked operators for the functional stuff, like 'pipe', (which I'm prototyping for my own amusement), 'compose' and 'bind'.

[louthy]

@orthoxerox Yep, it's certainly not ideal. I was wondering if there could be an alternative approach (this isn't fully thought through so be nice!).

Some of the operators I'd like to see should be pre-baked as you say (forward and backward composition, bind, material implication (==>), bi-conditional (<==>), etc.); but a lot of what I want for types in lang-ext are operators that work on the outer-type (so the Option part of an Option<A>):

For example, if I have an Option<A> type and I overload the + operator, what would you expect it to do if I did this?

    var ma = Some("abc");
    var mb = Some("def");
    var mc = ma + mb;         // Would you expect Some("abcdef") ?

What about:

    var ma = Some(Guid.NewGuid());
    var mb = Some(Guid.NewGuid());
    var mc = ma + mb;       

What I want to be able to do is define the monadic plus operator mplus. Which works on the state of the monad not the bound value.

Something like:

    var ma = Some("abc");
    var mb = Some("def");
    var mc = ma <+> mb;       

Would be great. Because it looks like a + in a box. And so I would get:

    var ma = Some("abc");
    var mb = Some("def");
    var mc = ma <+> mb;       // Some("abc")

    var ma = None;
    var mb = Some("def");
    var mc = ma <+> mb;       // Some("def")

    var ma = Some("abc");
    var mb = None;
    var mc = ma <+> mb;       // Some("abc")

So I wonder if we could have a set of sanctioned operators that aren't used in the BCL, but can be overridden in the usual way that represent 'boxed' versions of current operators:

<+>, <->, <*>, etc.

They would then have a contextual meaning: operators that work on the container objects, and could even share the same precedence and associativity as their non-boxed counterparts. That obviously doesn't solve all of the problems you mentioned, but it may lead to a more controlled operator environment compared to the custom-operator free-for-all.

I'd really love to see generic operators too. That combined with the suggestions above would solve most of the problems I run into.

[svick]

@louthy Specifically for Option<T>, I would be confused by <+> and would prefer if the operator was called ?? (which currently isn't overloadable either). I think you can assume users of your C# library are familiar with C#, but not necessarily with Haskell, which means they know ??, but not mplus.

The <+> operator might make more sense for some other monads, like list.

[louthy]

@svick Why would ?? mean 'monad plus'?

I think you can assume users of your C# library are familiar with C#, but not necessarily with Haskell, which means they know ??, but not mplus.

I don't assume anything about anybody. I am looking for solutions to real world problems that I face.

The <+> operator might make more sense for some other monads, like list.

But that's the point. mplus for Lst<A> does concatenate, for Set<A> it performs a union, for Task<A> it takes the result of the first task to complete, for optional types it takes the first non-None value. Each monad has its own behaviour for mplus; they have behaviour which is not +, not ??, not ||. So abusing those operators for this type of behaviour leads to just as much confusion. Having well defined operators that have a specific meaning for whole sets of types (in my case 'for the set of all monadic types'), is useful.

I have already generalised monads using the 'concepts' approach. I'm hoping that if in the future we get the shapes feature that we'll also get shape operators, which means with this approach I would be able to simply define <+> to mean Monad<MA, A>.Plus(ma, mb). So choosing different operators for different types is travelling in the wrong direction; I'm trying to unify the types and operators, not trying to create bespoke API surfaces for each type.

I realise I'm fighting at the edges of C# here, I get it, but a more powerful operators story would really facilitate expression oriented dev greatly (imho). Which is my main driver when contributing to these discussions, I want to be able to make everything I do into an expression, it's so close and improving all the time, but stuff like mplus(mplus(mplus(ma, mb), mc), md) is painful compared to ma <+> mb <+> mc <+> md.

[svick]

@louthy

I did not realize that your Option was part of a larger monadic library. That's why I suggested ??, but I understand now that it is not a good option for your library.

[TheUnlocked]

Sorry for a bit of a necropost, but this is still a useful discussion.

I'm going to suggest a possibly controversial idea for this: Only allow operators to be given currently valid names. The suggested ^^ operator could just be replaced with expt. This could completely remove the discussion with what characters should be allowed, and it would also hopefully make the meaning more clear for strange operators like &** (I don't know what that would do either. That's the point.)

[AustinBryan]

This is something that I've always wanted to do, and loved working in Unreal Engine's visual scripting because you could make the functions look like the default operators. Power operators and percent ones were really useful, and then I made a lot of others.

The trouble is when we have stuff like this:

int x = y <^v> z   // what's going on?

But with named operators we can much better tell what they are going to do in a less subjective way. However, I find it's really more just of a method at that point.

double a = y%;
double b = y percent;
double c = y.percent();

So I would prefer having them be symbols and get the benefit of typing a one-to-two letter thing and it working. Besides, if you don't understand what it is, you can just open up the method with F12 and view it.

[theunrepentantgeek]

if you don't understand what it is, you can just open up the method with F12 and view it.

This is like saying that method names can be cryptic because anyone who doesn't understand what it means can just open up the method with F12 and view it.

I disagree. Readability of code is critical - it's read many many more times than it is written.