Rich Errors: Motivation and Rationale

[zarechenskiy]

This is a discussion of motivation and rationale behind rich errors. The current full text of the proposal can be found here:

fun load(): User | NotFound

when (val user = load()) {
    is User -> println("Hello, ${user.name}")
    is Notfound -> println("Not found!")
}

Please note that a detailed design document on rich errors will be submitted later, once we collect more feedback and figure out the missing pieces.

[francescotescari]

Question: is the synthetic Error supertype exposed in the API?

Meaning that I can write code like the following:

fun logError(error: Error /* Any error */) {
  println(error.toString())
}

error object NotFound

fun main() {
  logError(NotFound)
}

If so, isn't it clearer if instead of a error keyword we just use plain inheritance from the special Error supertype? Feels more aligned with the current langauge state and with other languages (e.g. java Throwable)

object NotFound: Error()

What is missing

What I really miss in the proposal is a better way to propagate the error to the parent caller: adding .ifError { return it } is just too boilerplate and distracting. What I would like is something like swift's try or Rust's ?

fn open_file() -> Result<(), Error> {
    let file = File::open("hello.txt")?;
    Ok(())
}

From this point of view, even the third-party library Arrow provides monad comprehension with a less-verbose syntax.

  val dean = either<NotFound, Dean> {
    val alice = student(Name("Alice")).bind()
    val uca = university(alice.universityId).bind()
    val james = dean(uca.deanName).bind()
    james
  }

In my personal experience, expected runtime IO errors such as FileNotFound, NetworkOffline, etc. are very often just propagated to the caller rather than being handled directly, having some syntax to do that properly is quite important.

Happy vs unhappy path

By simply making an error a return value, we break all the libraries that make use of the "unhappy path". For example, a DB transaction

fun <T> Database.transact(block: () -> T): T {
  return try {
    block()
  } catch(t: Throwable) {
    rollback()
    throw t
  }
}

fun doSomething(): String | MyError

fun main() {
   // doesn't rollback the transaction on error
   val result = db.transact {
     doSomething()
   }
}

or even kotlinx.coroutines

fun main() = coroutineScope {
  val deferred = async { readFile() }
  // Most likely I want the following to be cancelled if readFile returns an Error
  launch { 
   delay(1000)
   println("I was not cancelled!")
  }
}

potentially becoming a major pain for the migration-to-errors period. Libraries that want to handle error types will have to start adding the additional if (result is Error) case to check at runtime the path (super error prone), or leave the entire "act on error" logic to the user (e.g. manually rollback the transaction on error).

It's also unclear how an opaque Error should be treated in those cases, since based on this proposal, both an IO error and a "no such element" would end up being "Error" types, where currently IO errors are treated as "unhappy path", where "no such element" are often handled with the separate categories: throwable vs nullability:

// before
fun main() = coroutineScope {
  val deferred1: Deferred<Int?> = async { list.firstOrNull() } // Doesn't cancel the scope on missing element
  val deferred2: Deferred<ByteArray> = async { readFile() } // Cancels the scope (throws) on missing file
}

// after
fun main = coroutineScope {
  val deferred1: Deferred<Int | NoSuchElement> = async { list.first() } // Should it cancel the scope?
  val deferred2: Deferred<ByteArray | FileNotFound> = async { readFile() } // Should it cancel the scope?
}

So I'm not sure that having a flat hierarchy that puts together basic cases like NoSuchElement with actual unhappy paths like FileNotFound is enough for a library author to act on an opaque Error unhappy path.

From this point of view, the Arrow library provides a more pragmatic solution since the monad comprehension logic is based on Throwable so it works transparently as expected:

fun main() = either {
  // Creates the binding scope
  coroutineScope {
    /* ... */
    // propagates the error to the binding scope through a throwable, cancelling the coroutine scope as expected
    val deferred2: Deferred<ByteArray> = async { readFile().bind() } 
  }
}

I would love to see Kotlin having a solution that goes towards that syntax, even though it's more challenging on the implementation side (e.g. ensure throwable propagation, avoid escaping functions, etc.).
Example desired syntax

suspend fun main(): ByteArray | NotFoundError = coroutineScope {
   // if any file reading fails, it cancels the scope and returns the error 
   val file1 = async { readFile("1.txt")? }
   val file2 = async { readFile("2.txt")? }
   file1.await() + file2.await()
}

With the current proposal

suspend fun main(): ByteArray | NotFoundError = coroutineScope {
  // How to implement the example above correctly? Impossible
}

[francescotescari]

This assumes that we can always decide, for a given type, whether it's a symbol or an error, in all situations.

That is something that we do today already with values vs exceptions. We throw to indicate an unhappy path, knowing it will cancel the coroutine. We return a value (e.g. firstOrNull) to indicate a specific case but still on the happy path. The caller of the function can always convert from a specific case to an unhappy path (e.g. firstOrNull() ?: throw NoUserFound()) or recover from an unhappy path to a specific case.

Without the unhappy-path-specific semantic that is today guaranteed by throw+exceptions, I don't see how it is possible to use rich Error with coroutines (or any library dealing with unhappy paths) at all.

Challenge: rewrite the following (common) snippet with rich errors instead of exceptions. I cannot do it.

suspend fun main(): ByteArray = coroutineScope {
   // if any file reading fails, cancel the other reading, return error in the main function
   // bonus: include suppressed errors
   val file1 = async { readFile("1.txt") }
   val file2 = async { readFile("2.txt") }
   file1.await() + file2.await()
} 

[e2e4b6b7]

For your examples, I would not expect any of the library functions to process an arbitrary rich error in any way. Errors are values, and they are valid results of async computations or transactions. It is up to the caller to choose whether it is an unrecoverable case or not in the current context. Even FileNotFound could be part of the happy path in some cases. For example:

val file1 = async { readFile(path) }
val file1Synced = file1.await()

In that case, FileNotFound is treated as a valid value for the file. It means that this is the expected behavior. For example, the path was provided by the user, or there are other possible locations for this file. In that case, no computations should be cancelled; the code has to process this case and continue execution.

val file1 = async { readFile(path)!! }
val file1Synced = file1.await()

In that case, the program is expecting this file to exist - for example, it is the file that was downloaded some lines above, or it is a config file required for the program to operate. In that case, FileNotFound is considered an unrecoverable case which has to stop the execution not only of the current function but of the whole logic of this module or some logical unit, and be escalated to the boundaries provided by the framework or platform.

Regarding the proposed separation into "symbol" and "error", I would say that all errors should be considered as "symbols" in this logic. Whenever the state represented by this symbol is considered unrecoverable in the current context, it should be transformed into an exception and handled not by the current code, but by the code of something logically above it (framework, platform, supervisor).

Challenge: rewrite the following (common) snippet with rich errors instead of exceptions. I cannot do it.

If the absence of a file is an unrecoverable case, exceptions should be used:

suspend fun main(): ByteArray = coroutineScope {
   val file1 = async { readFile("1.txt")!! }
   val file2 = async { readFile("2.txt")!! }
   file1.await() + file2.await()
} 

If the absence of the file is the expected case, but for some reason the value of file2 is not needed in that specific case:

suspend fun main(): ByteArray = coroutineScope {
   val file1 = async { readFile("1.txt")!! }
   val file2 = async { readFile("2.txt")!! }
   val file1Synced = file1.await()
   if (file1Synced is FileNotFound) {
      file2.cancelAndJoin()
      // handle this case
   }
}

[francescotescari]

Regarding the error handling and coroutines, it's not true that there are only two possibilities (unrecoverable or recover it), there are three things a function can do:

• make it unrecoverable (swift's try!)
• recover it (swift's do-catch and try?)
• propagate it up the stack to the caller (swift's try)

Notice that these cases are semantically there regardless of how the error case is encoded (unchecked exceptions, swift's checked exceptions, or rich errors), and similarly the actions that should be taken are generally correlated:

• make it unrecoverable -> terminate ASAP (aka cancel the coroutine) and go to the framework exception handler (your first challenge solution)
• recover it -> decide what to do with the coroutines, recover it based on the local context (your second challenge solution)
• propagate it up the stack to the caller -> terminate ASAP (aka cancel the coroutine) and pass it up the stack -> no decent solution to the challenge with rich errors

The third case is probably the most common, has no error "handling"/"recovery" semantics, and should always cancel the coroutine and pass it up. I'm not saying that case 3 is the only one, nor that cannot be a mix (like partially handling it by converting the error type, or adding some middleware logic). I'm saying that with the current proposal, there is no idiomatic/fun/pragmatic way to implement case 3 properly. If you try, you will want to go back to exceptions (or Arrow's Raise).

Notice: swift has checked exceptions to materialize typed/excepted errors, and by default throwing an exception will cancel the structured concurrency task tree. This is the correct way of implementing structured concurrency.

[kyay10]

@francescotescari
I think Arrow is that sweet spot for implementing case 3 though.
I dont' know what more the language can do here (other than providing QoL features like capture checking, and maybe a multiplatform stacktrace-less exception for performance).

In fact, this is a feature commonly known as an error effect, under the idea of algebraic effects and handlers, and it functions in the same way as Arrow's Raise does (i.e. Arrow's Raise is an error effect, and error effects are exactly the solution to case 3). Structured concurrency is another example of effects and handlers (in my understanding of it), and that's why the 2 fit really well.

Rich errors will make Raise slightly nicer at times, so that's good.

Maybe you want Raise to be in the stdlib? I could get behind that.
I like that Arrow gets to evolve on its own and explore the design space for Raise slightly faster without needing a KEEP for every behaviour change.
A very minimal and extensible implementation can be added to the stdlib though, then Arrow can provide its own opinionated extensions on top of that.

Other than that, I don't see how the language can add this in any way other than the way Arrow is doing it, so other than improving it, I don't see what can be done.

[e2e4b6b7]

propagate it up the stack to the caller -> terminate ASAP (aka cancel the coroutine) and pass it up the stack -> no decent solution to the challenge with rich errors

Indeed, it also might be a desired behaviour in some cases, but for simple sequential functions it is just a simple .ifError { return it } or some operator if we are able to design one.

For asynchronous code, the other part of this pattern is to cancel remaining children. Unfortunately, (AFAIU) kotlinx.coroutines does not provide the possibility to configure a scope in such a way that it will cancel all the remaining children if the result is already computed; this has to be done explicitly by coroutineContext.cancelChildren(). But isn't this a feature of kotlinx.coroutines, not of the language itself? (As some children might be spawned for their effect, not for the result, so this could not be the default behaviour.)

[kyay10]

Error types cannot have superclasses, superinterfaces, or generic parameters.

Might that be relaxed in the future? Especially with generic parameters. I think Ross Tate's presentation (on type outference) implied that generics can be supported while still staying polynomial

[MMauro94]

I second this. As I understand it, error classes can't directly implement "normal" interfaces, or they wouldn't keep the separation between normal types and error types, required for the type system to be polynomial. However, couldn't also support for error interface be added? I think this might prove useful, for example in a case like this:

error interface HttpError {
  val statusCode: Int
  val statusMessage: String?
}

error object NotFound : HttpError {
  override val statusCode get() = 404
  override val statusMessage get() = "Not Found"
}

fun receiveApiCall() : ApiResponse | HttpError

In this example, imagine receiveApiCall is a function that's required by some HTTP server framework, that doesn't really care about the error type, as long as it's able to convert it to an HTTP response with a certain status.

Similarly, abstract error classes and normal inheritance between the error types could be added to make the error types as powerful as normal types.

[sam-coopercode]

I've often wanted to do something like that HttpError example, and I'd add that I'd also find it very useful to be able to put the interface in one module and the implementation in another.

At the moment I use a custom Result<T, E> type in all my projects. For anything non-trivial, I always end up with a whole hierarchy of error types, with base error interfaces in my shared API modules and more specific errors in individual implementation modules. I have a whole lot of higher-order functions and decorator classes that work with those Result objects (think caching, retries, etc). It would be great to be able to use rich errors for all those things, but if errors can't have supertypes, subtypes, or generics, I don't think I'd be able to do so.

The KEEP talks about errors being able to compose well with higher order functions. Being able to pass an error through a higher order function unchanged is one part of that, but another important consideration is the ability to write higher order functions that can handle or transform a particular error or class of errors.

[CLOVIS-AI]

IMO, type parameters are very important to support these kinds of use-cases:

error class CannotResolve<T : Entity>(val id: Id<T>)

fun <T : Entity> resolveAll(ids: Iterable<Id<T>>) : List<T | CannotResolve<T>>

Without type parameters, functions that error will be very difficult to compose together.

Similarly, error interface and especially interface delegation would be very useful to unify domain errors across a codebase.

error interface HttpConvertibleError {
    val statusCode: HttpStatusCode
}

fun <T> ApplicationCall<…>.respond(content: T | HttpConvertibleError) {
    // …
}

// In a completely different part of the application;
error class UserNotFound : HttpConvertibleError {
    override val statusCode get() = HttpStatusCode.NotFound
}

I am in favor of forbidding other types of error constructs, like abstract class, open class, sealed class, even if it was somehow possible, because I think they would be confusing since their union-like semantics would make them unclear.

[CLOVIS-AI]

In case that helps with error interface: IMO, when an error interface is part of an error union, all its subtypes should be forbidden (or at least do nothing).

That is, if we have:

error interface Foo

error object Bar : Foo
error object Baz : Foo

then:

typealias Other = Foo | Bar

is either forbidden, or strictly identical to just writing typealias Other = Foo. This should solve the type ambiguity, no?

[RossTate]

@kyay10 Sorry to be slow to respond, but it seems others have provided good discussion in my absence. Nonetheless, now that I'm available, I wanted to check if the discussion has addressed the item you raised.

[kyay10]

Typo in this
Should be:

fun foo(): String | Err1 | Err2

fun f() { 
    val x = foo()
    if (x == Err2) return
  
    // no need for `is Err2` or an `else` branch here
    when (x) {
        is String -> println("string")
        is Err1 -> println("Err1")
    }
}

Also a clarification that Err2 is an error object might be useful

[zarechenskiy]

Thanks, I've updated the example

[kyay10]

It's often justified to place preconditions closer to their signatures. We already have "contracts" that hold a special place in function bodies with a sort of similar meaning. We could consider abstracting these concepts so that tooling and the compiler are more aware of preconditions/contracts, enabling deeper analysis or simply rendering functions differently if needed.

I'd be in support of Restricted Types for this. They don't even need new contract support.

[kyay10]

Error forms a parallel hierarchy to Any/Any?; it is not related to Any, and null is not an error. The new global supertype becomes Any? | Error.

I'm very against this! If we want errors to be true values, then existing functions that take generic type parameters should support errors too. I think adding a special AnyNotError type would be better here.
Again, this is very important because existing Kotlin code that does e.g. this:

fun <T> requireNotNull(t: T): T & Any

Would still be perfectly functional with errors, but now it isn't.
Same with

fun <T: Any> requireNotNull(t: T?): T 

Similarly, apply, let, etc all take generic type parameters. Are we really gonna have to update them all to have a bound of Any | Error?

I think the situations where a generic function must take a non-Error value are very rare, and it's okay to make them need special syntax or an extra type bound or something for that.

Consider also all datatypes.
Are Lists not going to allow Errors inside of them?
Same with Pairs, Maps, etc.
What about List.first?
Would we need another overload of it?

Do you see what I mean?

Errors are values, so it's important to allow them everywhere by default, and only exclude them explicitly when it's necessary.

Maybe having negative type bounds can be useful too here (I believe this was discussed briefly in the YouTrack issue) because then we can do this:

error object NotFound
inline fun <T> Sequence<T>.last(predicate: (T) -> Boolean): T where NotFound ~: T {
    var last: T | NotFound = NotFound
    for (element in this) {
        if (predicate(element)) {
            last = element
        }
    }
    if (last == NotFound) throw NoSuchElementException()
    return last // smart-cast to T
}

so that this can still work with Errors other than NotFound.

[zarechenskiy]

I'm very against this! If we want errors to be true values, then existing functions that take generic type parameters should support errors too

You’re getting straight to the root of the problem! This part is actually one of the most non-trivial aspects of the design: whether we should, and if so, how to adapt existing generic functions to accept errors. I’ll comment on this in more detail a bit later and include it in the KEEP.

[kyay10]

That's fair. I'd love to hear the justification for this! I'm just worried about this fracturing where Errors are barely useful as values since most generic functions cannot take them. I cannot imagine having to define my own List<Error> for instance, that's just unacceptable.

Hopefully it'll be obvious in the prototype whether this is a good idea or not, so that changes may be made.

[MMauro94]

I agree. To me it looks like that if special bounds is needed for a generic function/class to accept also error types, we would basically have another checked-exceptions nightmare on our hands. Imagine if fun <T, R> List<T>.map(transform: (T) -> R): List<R> wasn't able to accept R being an error type: we would effectively artificially limit where errors are able to "propagate", unless we go and update each single function.

Wouldn't be easier to keep Any to really mean any (as any possible type, including errors), and simply add another super-type that means "values, but no errors"?

I think I would also like the duality of having:

• Error types -> supertype is Error
• Normal (could be called value) types -> supertype is Value/to be decided

Plus Any for any of the above.

[gpopides]

In general i like the idea, i have 2 questions regarding ergonomics

Firstly, assuming a function like

fun fetchBalance(): Double | Error

how our handling is supposed to look like? Something like below?

when (val balance = fetchBalance()) {
  is Double -> ...
  is Error -> ...
}

Also, how will we be able to deal with multiple errors? Will we need to nest them?

fun fetchBalance(): Balance | BalanceError

fun fetchAccount(): Account | AccountError


fun deposit() {

	when (val account = fetchAccount()) {
		is Account -> {
			when (val account = fetchBalance()) {
				is Balance -> ...
				is BalanceError -> ...
			}
		}
		is  AccountError -> ...
	}
}

With arrow for example, we can use bind, from the docs

fun foo(n: Int): Either<Error, String> = either {
  val s = f(n).bind()
  val t = g(s).bind()
  t.summarize()
}

[kyay10]

I believe it'll be like:

fun deposit() {
	when (val account = fetchAccount()?.fetchBalance()) {
		is Balance -> ...
		is BalanceError -> ...
		is AccountError -> ...
	}
}

[gpopides]

Maybe my example is not clear but for example fetchBalance is not part of an Account.

The question is towards 2 separate functions that both can return an error

[kyay10]

Then:

fun deposit() {
	when (val account = fetchAccount()?.let { fetchBalance() }) {
		is Balance -> ...
		is BalanceError -> ...
		is AccountError -> ...
	}
}

[rnett]

The elvis operator ?: is not extended to error unions. What stops us from doing this is that there's no syntactic place where we can provide an error value, and without one, errors could be silently swallowed. Instead, a standard function ifError { ... } is provided for explicit handling

I think ifError will end up being very awkward to use when you don't actually care about the details of the error. I would think that if A errors in any way, do B would be a pretty common pattern, where B doesn't care about the type of error. ?: expresses that well. When you want to reference the error, you could do something like .let{ it ?: errorHandling(it) } and rely on smart casting for it to the error type. Although that's pretty awkward too and I could see wanting to have ifError to make it easier. I just suspect that a common case is to not care about the error at all, and ?: makes that case much more pleasant.

I would also think that the risk of accidentally swallowing the error in ?: isn't that much greater than using ifError and ignoring it.

I'm curious, did you have some evidence that this isn't the case that pushed you towards not including ?:?

[edrd-f]

I would also think that the risk of accidentally swallowing the error in ?: isn't that much greater than using ifError and ignoring it.

The risk is greater with ?: than with ifError because elvis is used for nulls. If you add ?: to handle a null and later the functions in the call chain are changed to return errors, they would be swallowed without you noticing.

[rnett]

Ah good point.

[rnett]

Are there any plans to make the mentioned OrError variants of stdlib functions the default? It's going to add a huge amount of duplication and make it easy to accidentally introduce exceptions in what should be error-based code by calling the wrong color of function. It's very easy to turn an error-returning function into a throwing one with !!, but not the other way around.

[moddyfire]

It might be a performance issue if every call to first() will now need to check for an Error even if the user doesn't want it.

[CLOVIS-AI]

I think this is pretty much impossible for existing APIs, it would break everything.

[rnett]

Much of this proposal makes use of the difference between "recoverable" and "unrecoverable" errors. The intent is that "recoverable" errors can use error types and be part of the function signature. However, I think there's a problem with this: what is recoverable and what isn't is not up to the called function, but rather the caller. How can a function definition know what should be a recoverable error and what shouldn't without knowing how it's called? This is especially a problem for stdlib and library functions.

This is touched on in the "Errors as values section"

While unchecked exceptions in Spring are convenient and often exactly what we want for cross-cutting concerns, there are many cases where no higher-level component will (or should) handle an error, or where failures must be handled precisely at a particular API boundary.

But then isn't talked about again in the proposed solution.

Examples are not that hard to come by. The "username is empty" error may be unrecoverable to the user-add service, but recoverable in the UI where we show some error text. Or a "wrong format exception" may be unrecoverable for that format, but recoverable for a multi-format process, except if it matches no known formats. The broader point, I think, is that the recoverability of an error is context-dependent and may vary back and forth throughout the call tree.

Library functions have a relatively easy solution: treat everything as potentially recoverable (i.e. an error type) and let the caller decide that it can't recover from some errors. It could get very verbose very fast, though, and doesn't handle the case where some errors are recoverable very well.

I think there's a general need for more ability to easily transition errors between "recoverable" and "unrecoverable" (and back) throughout the call tree.

---

Making some errors unrecoverable can be done with if or when blocks, but they add a lot of verbosity which can obfuscate the "throwy-ness".

I think functions to treat some errors as unrecoverable (i.e. throw them) would help with this. For example, a (T | E).throwIf<T, E : Error>(): T function, although getting the type signature to work so that you can call it like .throwIf<NotFound>() is tricky. Another one that could help is a .narrow<T>() that's used like .narrow<String | NotFound>() and would throw any non-NotFound errors. This one would mostly be useful with type inference.

---

Making some unrecoverable errors (or exceptions) aka "widening" can be done using catch (I'm assuming ErrorException is the name of the exception type used when throwing errors via !!):

try {
  return foo()
} catch(e: ErrorException) {
  when(val error = e.error) {
    is FileNotFound -> return error
    is EmptyCollection -> return error
    else -> ...
  }
} catch(e: OtherRecoverableException) {
  return OtherError
}

But this is awkward and verbose, especially for multiple potentially erroneous calls, and it's easy to accidentally not handle an error that is in your return signature. I think it would be helpful to have a widen<T | FileNotFound | EmptyCollection> { block() } that, if block throws an ErrorException with an error that fits within the type, catches the exception and returns the error. Another alternative would be to extend runCatching to be able to do something like this, but given it's issues with cancelation I'm not sure you want to encourage more use of it.

[MMauro94]

The way I see it is that basically that everything that you would use IllegalStateException or IllegalArgumentException would today be the same. Those are ideally never caught, and indicate a programming error. In general, anything that should never ever happen, stays on exceptions.

Everything that is out of your control (IO, network, filesystem, user input, etc.) goes on the new error types.

For everything in between (e.g. stdlib functions) where you don't know the context, use error types and let the caller decide whether to propagate it/rewrap or throw an ErrorException with the !! operator.

[moddyfire]

I look at is a checked exceptions. I'd use Error whenever I used checked exceptions in Java.

[MMauro94]

The bang-bang operator !! throws an exception wrapping the error value for cases where users want to escalate an error to an exception.

This I really don't like. From your own example:

fun bar(v: Int? | MyError) {
    v!! // throws NPE if v is null; throws KotlinErrorException(MyError) if v is MyError
}

What if I want to throw if there's an error, but otherwise keep my Int? type?

Wouldn't it be better to have a simply orThrow() extension function and leave !! only for null handling?

[joffrey-bion]

I think they didn't like foo?!.bar?!.baz so they repurposed ?

Could you elaborate on this? I would actually like this combination of operators.

[lsafer-meemer]

@joffrey-bion I am just speculating. From my point of view, they want rich error to replace the old null for failure but source compatibility is an issue. So, just don't introduce new operators and stick with the ? operator and this would be perfect as a drop-in replacement everywhere. That is why I think they have gone with the ?. operator instead of !.. And yes, I too like ?!. since I don't want null handler be merged with error handler.

[e2e4b6b7]

We think it is going to be quite a common pattern to throw in case of any error, so it is worth an operator. There are several reasons why we decided to re-use the existing one:

• It is going to be used mostly in the same cases as !! for nulls (to transform a recoverable exceptional case into an unrecoverable one).
• There are cases where you would like to throw on errors but not on nulls, or on nulls but not on errors, or on both. With this new behavior of !! we are trying to target the most common and simple scenario. For all other cases you are free to use v.ifError { it!! } or if (v is Error) v!!, which is quite simple to understand.
• We do not want to provide many variations of the same operator for all possible cases, as it decreases readability. We are still considering the possibility of them for the safe call and elvis operators, but for bang-bang it seems that one simple operator is the best fit.

[MMauro94]

Thanks for the reply.

If a decision is made to allow error types in currently existing error functions, have you considered the ambiguity that using !! would bring? For example, take the currently existing function:

fun <T> processValue(value: T) : T {
  if (/*some case that requires value to be not null */) {
    processNotNullValue(value!!)
  } else {
    // do something else
  }
}

As of now, this function works perfectly fine. However, if T is allowed to assume, for example, the type User|NotFound, the function will change its behavior drastically once rich errors are introduced.

[e2e4b6b7]

You go straight to the heart of the matter. To keep the behaviour of this code, we are currently considering 3 options:

• Keep Any? as a default upper bound. Thus, errors could not be passed into this function. To opt in to errors, it is required to specify an explicit upper bound Any? | Error.
• Keep nullable operators for nullability only and invent new ones for errors.
• Slowly migrate the nullable operators by warning/deprecating usages that are going to change their behaviour in ambiguous cases.

Yes, with a good design of new operators, option 2 seems the best one, but the problem lies in the new operators. We already have ?:, ?., and !! for nulls, and there is a request for

• Elvis for errors
• BangBang for errors
• Two other flavours of safe call
• Early return on error operator (like ? in Rust)

And the goal is not to end up with a mess of all possible strings of ? and !, which would decrease the readability of the code.

[adrian-shape]

Is Nothing a valid subtype of Error?

Eg we have Result<T, E> that has a subtype Result.Success<T>: Result<T, Nothing> which can be used in functions that take Result<T, E> as argument

If Result.Success<T> translates to T instead of T | Nothing then it might be hard to use in functions that take T | Error as argument

[joffrey-bion]

Since Nothing is the type that holds no value, I don't see how it could not be a valid subtype of Error.

There is no value of that type that is not an Error, because there is no value of that type at all. It is the bottom type for this reason, so I guess there is no other way.

[e2e4b6b7]

Actually Error is equivalent to Nothing | Error for the type system (and ParsingError to Nothing | ParsingError), where Nothing stands for empty "Value" component. In this form it is quite straightforward that Nothing is a subtype of Nothing | Error.

[joffrey-bion]

Right, that's a more visual way to put it :)

[adrian-shape]

If eg Error expresses that the "Value" component is Nothing, what does T express in terms of the "Error" component?

I'm wondering whether a function that accepts the type Result<T | Error> could accept an instance of Result<T>

[joffrey-bion]

Could you explain why the arguments against repurposing ?: didn't apply to ?.?

I really dislike repurposing ?. for errors. Why not !. for errors and ?. for nulls? If we need both we can have ?!. too.

In the same vein, we could have !: for fallbacks from error values (when the exact error doesn't matter).

I have a gut feeling that mixing null-safe and error-safe is a bad idea, especially given the current state of the ecosystem.

If a function returns Int? | SomeError, I cannot chain with an extension on Int? and just let errors through (but not nulls).

For instance, I would like to be able to process nullable elements just like other elements, while still playing with the error dimension:

val list: List<Int?> = listOf(null, 0, 1)
list.firstOrError()!.let(::println) !: println("no element")

[kyay10]

?. doesn't lose information, only merely propagates it. It's like a map or flatMap. ?: loses information

[gpopides]

After going through the KEEP again once more, i would also oppose using ?. for both nulls and errors.

We should aim for clarity in our code and using the same symbols for 2 different things is not something that i would call clear.

Using for example:

getUser()?.getLikedPosts()?.getMostUpvoted()

We cannot say if any of these return an error or nulls without looking into the signature of the functions.

On the other hand,

getUser()?.getLikedPosts()!.getMostUpvoted()

The distinction is very clear that user is nullable but get liked posts can return an error because we fetch the posts from a different source for example

[CLOVIS-AI]

Especially because the KEEP clearly mentions that in this new world with rich errors, null is not an error, and an error can never be null. null is always part of the happy-path, so it's very different from errors.

[e2e4b6b7]

The problem with !: is that there is also no place for the value of the error. And encouraging this pattern is not a good idea because it hides not only the currently available errors but also any others that might be introduced in the future.

We tried to repurpose the ?. because in the majority of cases it is the expected behaviour. Extension functions defined on a nullable type do not seem to be a common scenario.

The let function is a specific case, which is used here not as a function but as a control flow operator. That is why for that kind of function all possible scenarios might be useful. But it is not a good approach to hide complex control flow logic in flavours of ?, !, and ?!. It makes the code more complex to read in general.

Currently we are re-evaluating the operators’ behaviour due to backward compatibility challenges and other input, so it is not yet a finalized resolution. The main problem here is that we already have ?:, ?., and !! for nulls, and there is a request for

• Elvis for errors
• BangBang for errors
• Two other flavours of safe call
• Early return on error operator (like ? in Rust)

And the goal is not to end up in a mess of all possible strings of ? and !, which would decrease the readability of the code.

[moddyfire]

1. I think saying that it doesn't introduce "checked exceptions" for Kotlin is not accurate. It is more like "we do it so we'll never need to introduce checked exception". They are solving the exact same problem, but Error does it much better.

2. I'd like a stdlib construct to deal with collection of errors. I find the following code too cumbersome

error class ParseErrorList(val errors: List<ParseError>)

fun convertToInt(list) {
  val result = list.map { it.toInt()}
   if ( result.any {it is ParseError} )
       return ParseErrorList( result.filterIsInstance<ParseError>())
  else
       return result.map {it!!}
}

3. What exactly does it mean that errors can't be generic?

I think it means we can't write

error class<E: Error> ErrorList(val errors: List<E>)

But what about

fun<T: Any | DatabaseError> T.dealWithDatabaseError()

4. can ifError be generic>? It would be useful to write:

val result: CalculationResult = calcSometing()
    .ifError<ParseError> { e -> dealWithParseError(e) }
    .ifError<UserNotFound> { e -> displayError(e) }

5. Which exception does bangbang throw? and can I control it?

error class InvalidUserName(val name: String, val reason: String) {
  operator fun bangbang() { throw IllegalargumentException("name $name is invalid since $reason") }
}

6. the link to [KEEP-412] is broken

[lsafer-meemer]

This is just my opinion, I think if you have an error handling system such as this (an error list) then this is not the kind of syntax you should be looking for. In my point of view, you could use a mapping function from error objects to data classes.

Additionally, as explained in the KEEP. Error objects doesn't support generics but can be used as generic argument and since it does not support inheritance in any shape or form, there is no confusion on how it behaves when used as a generic argument.

[e2e4b6b7]

What exactly does it mean that errors can't be generic?

It means that it is not possible to declare generic error classes. Generics in functions and other classes can be upper-bounded by an error union:

// both functions are ok
fun <T : Int | Err1 | Err2> foo(t: T): T { ... }
fun <T : Any, E : Err1 | Err2> bar(t: T | E): String | E { ... }

Can ifError be generic? It would be useful to write

ifError will be a generic function. Its signature is mentioned in the KEEP. However, it is not possible to declare the signature of a function in your example.

Which exception does bangbang throw? And can I control it?

By default, it is going to be KotlinErrorException with the error stored inside. And yes, we are going to allow specifying your own exception with a virtual function inside the error class, as in your example.

[mikehearn]

I'd like to get a response to the issues I raised in this YouTrack comment and which have also been raised above by @rnett - I'll summarize them quickly here.

1. It will introduce major ambiguity into the language, creating bike-shedding and making Kotlin less fun.
2. As Ryan and I argue, this proposal relies heavily on the claim that error conditions naturally separate into recoverable and unrecoverable. But this isn't the case. Recoverability is a property of the caller, not the callee. The examples in the justification sometimes feel forced as a consequence: who writes a function like fun fetch(): User? that returns null in case of database error? Nobody! The only places null is used as an indicator of an error condition is in cases where the "errory-ness" of something is highly debatable to begin with, e.g. firstOrNull is meant for cases where an empty list is often expected and thus by definition not an error. first() is for cases where the list is always believed to be populated, in which case non-local control flow to your web server framework or similar is highly desirable.
3. Languages like Kotlin should be guided by what is seen in real codebases and widely used libraries, but all the libraries I used over many years don't invent their own error types (meta-libraries like Arrow obviously don't count here unless libraries built on them become widely used). So whilst the proposal makes claims about "the prevailing view" being this or that, I don't see much evidence for this. If people wanted to express errors this way they'd be finding hacks to do it already, but generally don't.
4. Many of the stated benefits of error types aren't actually exclusive to them, e.g. nothing stops you creating an exception object and passing it around instead of throwing it, and nothing about exceptions makes retries/backoffs hard to implement (quite the opposite, it's another form of non-local control flow). The benefit of treating exceptions as error objects is that an exception carries a stack trace and already has many features related to that, and can always be thrown later if wanted. In practice treating exceptions as ordinary data carriers is rare, but because the pattern does works fine, I'd argue this just indicates that throw is a good fit for what people really want.
5. Changing error cases forces callers to update - this is what checked exceptions do, and yet Kotlin correctly doesn't use them. Errors are rare by definition (a common expected occurrence cannot be really called an error), and often handled non-locally, so we usually don't want to be forced to do busywork refactoring everything because a new edge case failure appeared!

If I was asked to choose between the precondition improvements or the so-called "rich" errors (which have fewer features than exceptions), I'd go for the precondition improvements. My codebases are always full of preconditions but I never felt the need to introduce custom Result-style error types.

[lsafer-meemer]

There is also another function that just merge the NotFound error with the insufficient funds error because there is no way of expressing it and sealed types are too much boilerplate for the function only used in one place:

        /**
         * - Returns false if the user does not have sufficient funds. ([delta] < current_amount)
         *
         * @return null if state is not the expected.
         *         Otherwise, return true, if document found and modified.
         */
        suspend fun atomicIncStrict(
            user: UserId,
            expectedState: State,
            delta: ULong,
        ): Boolean?

[zarechenskiy]

@mikehearn Thanks for your feedback, I’ll go through your questions over time.

Meanwhile, I wonder if you could also share your experience and thoughts on kotlin.Result. Have you used it, or run into cases where it was used? And what’s your general stance on it?

[mikehearn]

@zarechenskiy I just looked at usages in my main Kotlin codebase. There are 1500 files with only a couple of usages of Result, all in code written by other people. Of those they are:

1. Used to hold the result of user input validation, which is then converted to a type used in a third party UI library.
2. Used to effectively "capture" an object or exception type in an object field that's referenced much later in non-local ways.

They seem OK, although for (1) it seems that it's a bit overly complex for what it's doing and that could have just been a thrown exception. For (2) it seems like a decent fit and if there wasn't one in the standard library, we'd have rolled our own. However, it's only being used as a Pair<T, Exception?> in effect and none of the extension functions on Result are being used. So in practice the type saves us just a one liner data class here, it is not important.

So I guess my stance is that I don't use it myself and wouldn't care if it weren't in the standard library. It can be a little annoying when encountered in other people's APIs or code, but it's also not a big deal and not worth worrying about.

In contrast this particular codebase uses exceptions quite extensively and gets a lot of mileage out of them for things like crash reporting, generating pretty errors for the user, tearing down parallel computations etc.

[mikehearn]

        /**
         * @return null if state is not the expected.
         *         Otherwise, return true, if document found and modified.
         */
        suspend fun atomicInc(
            user: UserId,
            expectedState: State,
            delta: ULong,
        ): Boolean?

The caller is then use the error type to either do a complete recalculation or stop since the document no longer exists.

Data races are presumably expected in your system, so I'd hesitate to call them errors.

Usually it's important to restart a whole block of logic in case of a serializability conflict. For your API, if you get null you don't want to immediately retry the same call, there will be other calls that also need to retry like reading the value again and maybe deciding on a new delta. The best way to model that is to have a higher order function in which exceptions of a certain type from the lambda cause a re-execution from the start (and maybe incrementing a metric, etc). So exceptions seem a good fit for this to me because you want non-local control flow, without being able to forget to check a return value. If conflict rates are so high the overhead of an exception becomes non-negligable you can simply create one that doesn't have a stack trace, as that's where the vast bulk of the cost is.

Frameworks like Spring and Micronaut abstract this pattern behind retryable advice and it works well.

[lsafer-meemer]

@mikehearn the problem with exceptions is the unnecessary flow interruption and since exceptions are built that way, there is no actual way of specifying expectation of business-logic exceptions in the type system other than introducing the horror that is checked exceptions from java.

The difference between checked exceptions from java and error unions in this proposal is how they are treated. Checked exceptions are exceptions and always interrupt the flow of execution. On the other hand, errors in this proposal are values and can be passed around as values or intentionally ignored or even aggregated enabling the ability to use them in async and awaitAll etc.

[rnett]

What are the valid property types for error classes? I'd like to be able to do something like:

error class FieldError(val fieldName: String, val error: Error)

[kyay10]

I don't imagine there's any such restrictions, so you're fine.

[rnett]

Interesting, I would naively have thought that it was limited to primitives and other error types. But I suppose it doesn't actually need to be.

[kyay10]

The restrictions on error types mainly come from the type inference and checking algorithm, and properties don't really factor into that.

[e2e4b6b7]

Yes, error classes can have all the same properties as any other (non-generic) classes

[gdoenlen]

How are these unions going to be expressed in Java bytecode? Is interop a priority?

[kyay10]

I think it'll all just be Object.

[gdoenlen]

Unfortunately that sounds a lot like the road Scala has gone down; forcing people to write interop layers. It's a valid choice I guess, but something simpler like Result<T, E> could have been expressed in both type systems rather easily.

[kyay10]

It should be relatively easy to write a KSP plugin that'll generate such interop bridges. Add a @Deprecated(level = HIDDEN) for good measure so that Kotlin users are unaffected.

[e2e4b6b7]

Currently, we are thinking about plain Object as a runtime representation on the JVM for several reasons:

• To allow passing List<Int> where List<Int | Error> is expected. This is the only possible representation, as Int values have to be represented in the same way in both cases.
• Plain representation does not introduce overhead for boxing.

Yes, it is possible to provide an annotation to produce bridges with another representation for top-level values. It might be part of the plugin or even the standard one.

Note: kotlin.Result is also currently represented as an Object in the bytecode.

[CLOVIS-AI]

It's question whether we should allow nullable error types on their own though, since they could be viewed as Nothing? | Error.

I'm in favor of forbidding nullable error types at all for now.

From what is said in the KEEP, this would be forbidden:

error object Foo
typealias FooOrNull = Foo?

fun foo(): String | FooOrNull // ! Compile-time error, error object cannot be nullable

If that's the case, I think it is less confusing if Error? is forbidden. Users can create typealiases around Nothing? | Error if needed, and we can figure out something else later based on usage.

[e2e4b6b7]

Yes, currently we are not going to allow writing question mark after the error type, as it is a property of the "value" component

[CLOVIS-AI]

Overall, I really like this KEEP, and I think it will make many things better.

I particularly like the section "Local tags and in-place errors". I didn't realize this design could be used in such a way, and I think this will help a lot in many situations.

[CLOVIS-AI]

I would love a section with more precision on how users should use errors and exceptions together.

For example, it's common that something that is locally an error ("the thing you requested is not found") cannot be recovered locally and actually should become an exception to be handled by the framework elsewhere. The function itself should still use errors, because in some other cases it may be recoverable, but in some cases callers should make the decision that it's actually not recoverable there.

Do we except users to just use !!? Do we expect users to do this often? Then maybe something can be done on the other side. For example, what about allowing:

error class FailedA

try { 
    …
} catch (e: FailedA) {
     x()
}

which would desugar to:

try {
    …
} catch (e: KotlinErrorException) {
    if (e is FailedA) x()
    else throw e
}

Or alternatively, some kind of "catch guards":

try {
    …
} catch (e: KotlinErrorException if e is FailedA) {
    …
}

Otherwise, I fear that this will become very verbose, and thus people will be lazy and just write catch (Throwable) instead of being precise.

[CLOVIS-AI]

After reading the other comments, I think my sentiment above ties with the "recoverability is a property of the caller" comments elsewhere. Being both an application and library developer, I can see how rich errors fit extremely well the use-cases of library development but are confusing for application developers.

During library development, it's very important to always be clear with the failure conditions. It's very easy to decide between:

• IllegalArgumentException → you called my library incorrectly
• IllegalStateException / AssertionError → my library is broken
• Another exception → something that we can't anticipate (because there's too many possible reasons for it) happened, deal with it (e.g. in a database driver, "the connection suddenly failed while I was making that request")
• Rich errors → here's a reason why this function could fail that you should deal with (e.g. in a database driver, during the very first connection, "I couldn't reach the database at the coordinates you gave").

In library development, this system seems to work very well because we always expect the caller (which is probably only a single digit of layers under us in the callstack) to deal with the issues, and I think this explains why the most convincing examples in the KEEP are very local library-like functions, like firstOrNotFound. In library development, we can usually enumerate all the things that can go wrong.

However, during application development, the world is much more muddy, we know much less about the reasons things would fail, and failures should often be handled much further, for example multiple services lower in the callstack. For example, we should expect that some entity is found; if it isn't, we probably have no idea what can be done, and should just log/capture as much context as possible and fail the request. We don't want to propagate all the reasons why things that should never fail could fail, even if they may be recoverable. In this kind of world, I don't think rich errors are particularly useful. I expect these kinds of codebases will continue to use exceptions (or continue to use Arrow's Raise if they were functional).

Rich errors are still valuable because they make small algorithmic failures much easier to work with, but they probably shouldn't be seen by application developers as a major change.

[dkandalov]

Thank you for publishing the proposal 🙏

Here is some feedback even though it's been mostly covered above:

• The ? operator to propagate errors seems quite fundamental. I hope it makes it into the proposal at some point.
• Similarly, it would be good to have ?: operator, maybe with a syntactic place like foo ?: { error -> println(error) }. Functions like ifError exist in the libraries with Either/Result, and are quite verbose to use.
• Flat hierarchy and no generics for error types seem limiting. I'm sure I'm missing some important details, but why couldn't error be a "marker" to ensure disjoint union while keeping most of the existing functionality as it is (e.g., interface, data, enum errors)?
• As mentioned by others, "recoverable" and "local" are subjective terms. Even in the KEEP examples, there are both UserNotFoundException and error object NotFound used for the case when a user cannot be found. Without very clear communication, people are likely to come up with contradictory definitions of when to use rich errors (similar to checked exceptions). It's also not perfect and a subjective definition, but I prefer "logical error" as used in the Arrow library.
• It's not clear why these errors are "rich". They have less data than exceptions; the error classes are restricted 🤷

There is a typo in the link https://github.com/Kotlin/KEEP/blob/main/proposals/proposals/KEEP-0412-unused-return-value-checker.md.