Condition recipes

[kylewlacy]

Let's say we have a build that ends up with a lib/ or a lib64/ directory in the output-- say, depending on the platform. For consistency, we want to add a symlink from lib to lib64 in the latter case. Today, the easiest way to do that is with a recipe like this:

std.runBash`
  if [ -e lib64 ] && ! [ -e lib ]; then
    ln -s lib64 lib
  fi
`
  .outputScaffold(recipe)
  .currentDir(std.outputPath)
  .toDirectory();

It'd feel more natural to write this with normal TypeScript features, like if (recipe.contains("lib64") && !recipe.contains("lib")) { ... }, but this has the downside that we'd realistically need to add an await in the method, which causes some challenges when used too liberally.

Here's another option: we represent this conditional as a recipe, and let the runtime handle it. It could look something like this:

std.when(
  // if recipe contains "lib64" but not "lib",
  recipe.contains("lib64").and(not(recipe.contains("lib"))),
  // ...then symlink lib to lib64
  recipe.insert("lib", std.symlink({ target: "lib64" })),
  // ...otherwise, use the recipe as-is
  recipe 
);

Under the hood, this would serialize to a JSON recipe that would look something like:

{
  "type": "when",
  "test": {
    "type": "and",
    "left": {"type": "contains", "recipe": {/* ... */}, "path": "lib64"},
    "recipe": {"type": "not", "value": {"type": "contains", "recipe": {/* ... */}, "path": "lib"}},
  },
  "then": {/* recipe if true */},
  "else": {/* recipe if false */},
}

---

So at a high level, the idea is to write recipe conditions with a DSL, which turns into a JSON value that the runtime can handle. It doesn't look nearly as nice as using normal JS features, so this is a trade-off between flexibility and readability.

I think this would fit well with my cross-compilation ideas too too, since this would provide a way to handle platform-specific features, since the runtime would be able to handle conditions based on dynamically-scoped variables (host, target, etc.).

Some open questions / challenges:

• What should the name of this std.when / std.if DSL function be? if is the most natural name, but it's a JS keyword and so isn't allowed as an identifier in some places.
• How should conditions be written? I went with the style if(test, then, else), but there are many different ways this could be written too. I think some Lisps use a (cond) form, which JS-ified would be more like cond([test, then], [test2, then2], ... ["else", else]), which is probably more readable if you have a lot of else-if conditions.
• Our runtime evaluation only handles Artifact values (File, Directory, or Symlink), while we would need booleans for this change. Are there other places where booleans could appear (e.g. process templates)? Do we want other types of values too, like generic numbers and strings?

[kylewlacy]

A bit of background context too...

Previously on Discord, @jaudiger and I were discussing a brioche plan command (Discord link), where it'd be nice to have a way to preview a build before queuing it for a CI build, e.g. to see how many packages would be affected by a change. The challenge there is around packages that await the build of another package, since for those packages, we can't see more of the build graph until after the awaited build finishes... which we wouldn't want to do when just previewing a build!

The idea I mentioned at the time was to get rid of all places that we use awaits in package builds (there were only 2, and I don't think we've added any new ones since then), instead pushing more work down from TypeScript into more recipes. I don't think doing that change would be trivial, so I decided to hold off on actually changing the existing packages.

So part of the motivation for this idea is to make recipes more expressive without needing to fall back to a Bash script. I thought to open this discussion after seeing this comment today too: https://github.com/brioche-dev/brioche-packages/pull/1603/files#r2481188583

I think the extreme path would be completely forbid awaits. Honestly, I'm kind of feeling like that might be the right path for us... but it's a pretty major change that could have long-term consequences!