Binary-Prolog interaction

[bakaq]

This expands on a possible design for #41.

Problem statement

We currently use POSIX shell for stuff that is impossible, hard or inconvenient to do in Prolog (at least portably), like dealing with files and subprocesses. I chose POSIX shell because you can assume it's preinstalled in every single Unix system, and therefore doesn't generate an extra dependency for users of Bakage to install and manage. It doesn't support Windows however, and it's really limited an annoying.

The solution to this that I'm most interested in pursuing in the future is doing all the things we would now do in POSIX shell in a dedicated binary instead (written in Rust for example). It would still be desirable to leave most stuff in Prolog, and leave the option for users to use any Prolog implementation to actually run the Prolog part of Bakage.

Proposal: Subprocess IPC

So to achieve these goals, I propose that we have the toplevel bakage executable be the binary, for it to immediately call the Prolog part as a subprocess¹, and for both to then communicate through the Prolog process stdio with a predefined protocol. This would lift the need for shell/1, setenv/2 and friends in the Prolog code, making it more portable. We would only need the ISO IO predicates like write_term/3 to communicate with the binary to actually do these hard parts.

I'm gonna use JSON here as the protocol format for examples, but that isn't required, and probably isn't the best option. Ideally we would want something that is easy to work from both the Rust and Prolog sides, and it's easier to write parsers in Prolog so being easier in Rust has more weight.

An example of a message from Prolog to Rust that we could eventually have:

{
  "sequence_number": 42,
  "command": "ensure_dependencies",
  "dependencies": [
    {"name": "testing", "git_repo": "https://github.com/bakaq/testing.pl.git"}
  ]
}

This would just be written to stdout with write/1 or something similar. The binary would have the Prolog stdout connected to pipe so that it can read this request. After receiving it, the binary acts on it and then sends a response:

{
  "sequence_number": 42,
  "success": true,
  "report": ["..."]
}

This is much more complicated that just using the Scryer embeding API (a big reason to have an embedding interface is so that you don't have to do stuff like this), but it does allow any Prolog implementation, not just Scryer.

Footnotes

1. If we eventually have Bakage running in Wasm (imagine having arbitrary packages available in the Scryer Playground!!) we can run a WASI "subprocess" for the implementations that support WASI (Scryer and Trealla do), so this idea still works there! ↩

[triska]

Regarding this syntax:

{
  "sequence_number": 42,
  "success": true,
  "report": ["..."]
}

I would like to add that this is completely valid Prolog syntax:

?- write_canonical({
  "sequence_number": 42,
  "success": true,
  "report": ["..."]
}).
{}(','(:('.'(s,'.'(e,'.'(q,'.'(u,'.'(e,'.'(n,'.'(c,'.'(e,'.'('_','.'(n,'.'(u,'.'(m,'.'(b,'.'(e,'.'(r,[]))))))))))))))),42),','(:('.'(s,'.'(u,'.'(c,'.'(c,'.'(e,'.'(s,'.'(s,[]))))))),true),:('.'(r,'.'(e,'.'(p,'.'(o,'.'(r,'.'(t,[])))))),'.'('.'('.','.'('.','.'('.',[]))),[])))))   true.

[bakaq]

Prolog being secretly JSON wasn't on my bingo card 🤣.

[aarroyoc]

A very similar approach was taken by a Logtalk + Tcl/Tk demo a few years ago (LogtalkDotOrg/logtalk3#110). I also experimented with it, although I did it on reverse (I parsed Prolog from Rust, using nom). It definitely has an overhead, but I think for this kind of app is perfectly reasonable.