Feature: Support error causes

[conartist6]

This is a proposal to support printing stacks of errors nested with error.cause. The JS spec allows error.cause to be any value, but this proposal is mainly concerned with what happens when cause is present as a string or another error.

I've written up the most descriptive form of this proposal as a proposed jest feature. If you don't mind I'll avoid copying it all out here. Jest uses stack-utils under the hood to reformat errors.

The TC39 committee has so far indicated that the language will not provide any official implementation of printing nested errors. Their hands are tied to some extent until they finish specifying the semantics of error.stack. But until then it will be very useful to have best-effort printing and formatting for errors printed in the most common way.

I think errors with causes should be printed more or less like this:

SomeError: The grumblecakes could not be found
    at grumblecake-locator.js:12:2
Caused by: QueryError: Invalid query `SELECT <grumblecakes>`
    at db.js:51:4
Caused by: ParseError: Unexpected character `<` at position 8
    at db.js:1233:10

Currently there is no standard for this. I think stack-utils would be an excellent library in which to create an implementation which formats, prints, and loosely parses errors with multiple messages and stack traces.

I'm willing to build it. Is there interest from the maintainers?

[conartist6]

I've adapted the project's current regex into a lexer that I think will work:

const moo = require("moo");

let lexer = moo.compile({
  Whitespace: { match: /[ \t\n]/, lineBreaks: true },
  CausedBy: /^\s*Cause(?:d by)?:/,
  // Any line ending with file:line:col is a stack frame
  // file:line:col may be wrapped in parens, or may be the literal 'native'
  StackFrame: /^.*?(?:native|:\d+:\d+|\((?:.+?:\d+:\d+|native)\))$/,
  FreeText: /.+$/,
});

Stack frame lines would be subject to further parsing. In particular the current big stack-frame-line regex will need to be broken down in order to support embedding URI syntax and fix #60.

[conartist6]

Here's a condensed summary of the tokens emitted from that lexer when run on the example input above:

[
  {
    type: 'FreeText',
    text: 'SomeError: The grumblecakes could not be found',
  },
  {
    type: 'Whitespace',
    text: '\n',
  },
  {
    type: 'StackFrame',
    text: 'at grumblecake-locator.js:12:2',
  },
  {
    type: 'Whitespace',
    text: '\n',
  },
  {
    type: 'CausedBy',
    text: 'Caused by:',
  },
  {
    type: 'Whitespace',
    text: ' ',
  },
  {
    type: 'FreeText',
    text: 'QueryError: Invalid query `SELECT <grumblecakes>`',
  },
  {
    type: 'Whitespace',
    text: '\n',
  },
  {
    type: 'StackFrame',
    text: 'at db.js:51:4',
  },
  {
    type: 'Whitespace',
    text: '\n',
  },
  {
    type: 'CausedBy',
    text: 'Caused by:',
  },
  {
    type: 'Whitespace',
    text: ' ',
  },
  {
    type: 'FreeText',
    text: 'ParseError: Unexpected character `<` at position 8',
  },
  {
    type: 'Whitespace',
    text: '\n',
  },
  {
    type: 'StackFrame',
    text: 'at db.js:1233:10',
  }
]

[conartist6]

Oh boy this project actually already supports "nesting" of error stacks through string concatenation as done by nested-error-stacks. Unfortunately the behavior of nested-error-stacks is fundamentally not interoperable with errors that have causes. After all, to print errors stacked using cause you need access to their unstacked stack traces, which are destroyed by nested-error-stacks.

In practice nested-error-stacks outputs the same the same Caused by: string that I'm now working on parsing, so the only real difference would be in printing. If err.cause is present, then correctness of the output relies on err.stack not containing the same nested traces. The good news is that nobody should be doing this. Thus it may be fine to just print everything and let it come out in the wash.

[conartist6]

Oh and I think that answers another question I had. I was wondering why the at prefix at the beginning of a stack trace line is not required, and that's the reason (edit: or part of it -- cleaning a trace with this tool can also have that result). Because the parsed format essentially mirrors the trickery that nested-error-stacks does that's how it keeps track of what's a nested cause, which must then be outdented.

It was my intention anyway, but I'd say my plan is to make breaking changes and publish my fork. Once I have something that works and I can actively maintain we can discuss whether the fork should merge back into this package.

[conartist6]

Hmm, clean is pretty explicit though:

    if (!(/^\s*at /.test(stack[0])) && (/^\s*at /.test(stack[1]))) {
      stack = stack.slice(1);
    }

If a line doesn't begin with at it isn't part of the trace. This looks a little off to me though -- the initial message could be more than one line long.

[conartist6]

I'm very tempted to rip out support for method and file names which support ( in my fork. I recognize that the code in this project handles quite a lot of cases, but also the way it does it is terribly convoluted. I think it's a heck of a lot easier to just explain: "hey, if you're going to parse your errors, the results aren't going to be good for a file named )(.js.

[conartist6]

Similarly I think it's probably safe to assume that most methods names won't have () in them. I've never seen that professionally. Of course it's trivial to create such a thing, but that doesn't mean it's not fine to have less than ideal behavior when it is encountered. Since file names are usually allowed to contain ( and ) and identifiers are not, I think it's likely better practice for the parser to give some mangled results when the function name contains parens. And we can soften the cases which fail: we're sure that the parens around a file name will be preceded by a space.

That would allow us to handle some pretty weird stuff without doing anything all that weird, e.g.:

const obj = {
  'weird fn name()': () => {
    throw new Error('message');
  },
};

obj['weird fn name()']();

which results in

Error: message
    at Object.weird fn name() (/home/conartist6/dev/repos/stack-utils/space file.js:3:11)
    at Object.<anonymous> (/home/conartist6/dev/repos/stack-utils/space file.js:7:23)

[conartist6]

I could also stipulate that results may be bad if file names contained mismatched parens. Then I could just go back from the end of the frame string until I find the nearest balanced paren. I'd slice that part out and feed it back into a path parser that could differentiate between URIs and different kinds of paths.

I see from testing that windows and unix path styles are sometimes mixed in a single trace, which is just a pain.

    at Object.<anonymous> (W:\home\conartist6\dev\repos\stack-utils\space file.js:7:23)
    at Module._compile (internal/modules/cjs/loader.js:689:30)

[conartist6]

It may be that the easiest way to deal with this is to always tokenize parens. This will sometimes split apart parts of a single token like Object.weird fn name() that really belong together, but I then the parser will be able to handle balancing of parens and make the best possible effort to match the basic structure of a line (at method (file:1:2)), which will just leave us to find any paren tokens that were not consumed and print them back out as literals. If file is defined as being acceptable when having balanced parens, it should work fine.

[conartist6]

My new lexer is:

let lexer = moo.states({
  error: {
    Newline: { match: "\n", lineBreaks: true },
    Whitespace: /[ \t]/,
    CausedBy: { match: /^\s*[cC]ause(?:d [bB]y)?:/, value: () => "Caused by:" },
    At: { match: /^[ \t]*at /, push: "frame" },
    // Any line ending with file:line:col is a stack frame
    // file:line:col may be wrapped in parens, or may be the literal 'native'
    FreeText: /.+$/,
  },
  frame: {
    Newline: { match: "\n", lineBreaks: true, pop: true },
    "(": "(",
    ")": ")",
    Whitespace: /[ \t]/,
    FrameFragment: /[^() \t\n]+/,
  },
});

[conartist6]

And here is a complete parser. It's still missing some of the more minor features, but it proves that everything works together. Try pasting it into the nearley playground

[conartist6]

Aight I'm out for the day. Signing off. No more work. Thanks for your help everyone! Errors are gonna be great!

[conartist6]

In the end everything I was afraid I might not support works great. This parser will be even more flexible and powerful than the current one.

[conartist6]

Ah ok now we get into more fun stuff. The more we add into these stack lines the more we're going to hit a very particular problem: the grammar is fundamentally ambiguous. foo [as bar] could conceivably be either a reference to a method called foo [as bar], or of course much more likely it is a function foo used as a method bar. Nearley will output both interpretations, which on one hand is great, but on the other hand there might be a lot of these lines, and allowing ambiguities to compound (exponentially) would be disastrous.

[conartist6]

I think what this means is that I need two parsers, one for errors, and one for lines. That way the line parser can be ambiguous by design but for each line we'll eliminate any ongoing ambiguity.

The only question is how much overlap should there be? If stack lines are any lines that begin with at , there need be little overlap. But if we aim to support stacks were the at prefixes have been stripped away then we need to keep enough of the frame parsing to identify (file:line:col) at the end -- something we already know we are able to do unambiguously.