RFE: Better extensibility

[arv]

We are building some features on top of Valita and with the .4 release a lot of the public APIs that we depend on are now marked as private. I'm going to outline the things we are using to see if we can figure out how to expose these in a clean way.

Command Line Parser

We have a library that takes a Valita schema and generates a command line parser.

The code is ~500 LOC and it will be open sourced this year. I can give access to it early if you are curious.

API Used

toTerminals: This is used for the visitor pattern. We use it to gather the literals and the types.

ArrayType.prefix, ArrayType.rest, ArrayType.suffix: toTerminals was not called on these so we had to do that manually. This is used with the above. I believe we can mostly create our own visitor except for these which are not exposed.

func: We used this to get the default value out of an optional. I think I can test for name === 'optiopnal' and in that case there is no default value. Otherwise I can call schema.parse(undefined) and see what I get out of it.

Customized Error Messages

No breaking changes here but it does reproduce some of the functionality (~100 LOC). Not a big deal at the moment.

Better Union Error Reporting

When a union fails to parse we find the longest partial match and show the error for that. This allows errors like

API Used

• UnionType.options: This is used to get the different types of the union and we try all of them to see which one gives use the longest path.

Sample Implementation

type FailedType = {type: v.Type; err: v.Err};

function getDeepestUnionParseError(
  value: unknown,
  schema: v.UnionType,
  mode: ParseOptionsMode,
): string {
  const failures: FailedType[] = [];
  for (const type of schema.options) {
    const r = type.try(value, {mode});
    if (!r.ok) {
      failures.push({type, err: r});
    }
  }
  if (failures.length) {
    // compare the first and second longest-path errors
    failures.sort(pathCmp);
    if (failures.length === 1 || pathCmp(failures[0], failures[1]) < 0) {
      return getMessage(failures[0].err, value, failures[0].type, mode);
    }
  }
  // paths are equivalent
  try {
    const str = JSON.stringify(value);
    return `Invalid union value: ${str}`;
  } catch (e) {
    // fallback if the value could not be stringified
    return `Invalid union value`;
  }
}


// Descending-order comparison of Issue paths.
// * [1, 'a'] sorts before [1]
// * [1] sorts before [0]  (i.e. errors later in the tuple sort before earlier errors)
function pathCmp(a: FailedType, b: FailedType) {
  const aPath = a.err.issues[0].path;
  const bPath = b.err.issues[0].path;
  if (aPath.length !== bPath.length) {
    return bPath.length - aPath.length;
  }
  for (let i = 0; i < aPath.length; i++) {
    if (bPath[i] > aPath[i]) {
      return -1;
    }
    if (bPath[i] < aPath[i]) {
      return 1;
    }
  }
  return 0;
}

[jviide]

Glad to hear that Valita has been of use. It's very interesting to hear about your use-case.

The original purpose of toTerminals is to break unions apart for further analysis. As such it is not that well suited for implementing visitors, as it doesn't recurse into object properties - or array items, as you've noted.

The lightest approach would be to have a minimal selection of public properties that's just enough to write custom type-safe visitors:

function visit(type: Type | Optional) {
  switch (type.name) {
    case "literal":
      return type.value;
    case "options":
      return type.options.map(visit);
    // and so on ...  
  }
}

Some of them are already public, like Literal#value, and it's something we may extend further when time allows. However, it's worth noting that we consider being a "data definition language" outside the scope of Valita. Therefore we probably won't include a generic visitor implementation in this package, but the type-level approach sound fine.

Regarding customized error messages and union reporting: We recommend using a similar visitor approach, e.g. a function that branches based on Issue#code and possibly recurses deeper when needed. To make this easier for "invalid_union" issues, commit c648586 adds the property .issues to them.

Combining that with the visitor suggestion allows writing e.g. a custom helper that finds the issue with the deepest path, and also works nicely with nested unions:

function* flattenIssues(
  issues: Issue[],
  path: (string | number)[] = [],
): Iterable<Issue> {
  for (const issue of issues) {
    if (issue.code === "invalid_union") {
      yield* flattenIssues(issue.issues, [...path, ...issue.path]);
    } else {
      yield { ...issue, path: [...path, ...issue.path] };
    }
  }
}

function findDeepestIssue(issues: readonly Issue[]): Issue | undefined {
  let deepest: Issue | undefined;

  for (const issue of flattenIssues(issues)) {
    if (!deepest || deepest.path.length < issue.path.length) {
      deepest = issue;
    }
  }

  return deepest;
}

[arv]

That is useful. I will see if I can find time to a PR that exposes enough for us to transition to .4

[arv]

One more issue popped up this week. We are creating a deepPartial that recursively applies partial. However _restType is private on ObjectType.

deepPartial implementation

/**
 * Similar to `ObjectType.partial()` except it recurses into nested objects.
 * Rest types are not supported.
 */
export function deepPartial<Shape extends ObjectShape>(
  s: v.ObjectType<Shape, undefined>,
) {
  const shape = {} as Record<string, unknown>;
  for (const [key, type] of Object.entries(s.shape)) {
    if (type.name === 'object') {
      shape[key] = deepPartial(type as v.ObjectType).optional();
    } else {
      shape[key] = type.optional();
    }
  }
  return v.object(shape as {[K in keyof Shape]: v.Optional<v.Infer<Shape[K]>>});
}