Declarativ--uh--ization(?) of impure side-effecty-code a la PIO and format_//2

[jjtolton]

Let me give 3 examples:

1. Creating random UUIDs
2. Getting the system time
3. Writing a build pipeline

Regarding the build pipeline, it could look something like:

run_build_pipeline(Runner, Output) :-
    ticket_ok(Runner),
    initialize_metrics(Runner),
    pull_latest_version(Runner)
    unittests_ok(Runner),
    build_artifacts_ok(Runner),
    acceptance_tests_ok(Runner),
    update_vcs_ok(Runner),
    finalize_metrics(Runner),
    results(Runner, Output).

where Runner might be a module or perhaps just a symbol, depending on how the others were defined.

Now, all of these task predicates are heavily side-effecty. This is quite annoying, because it makes testing very difficult. And that is annoying because it makes testing-based design difficult. i.e., how do I write:

test("My build pipeline",
        run_build_pipeline(simulation_runner, "{\"TaskId\": 1234, \"start\": <timestamp0>, \"end\": <timestamp1>, \"status\": \"in progress\"}")
      ).

Simply mocking out the runner is... ok, but thing is that the output of format_//2 is actually useful. So it can be used for verification and it actually does the job.

It would be tempting to think that the output of run_build_pipeline/2 might product a series of commands? Part of me feels like that's not really solving the problem, it's just adding more indirection to the problem. Because why not just have run_build_pipeline/2 produce

    [ticket_ok(Runner),
    initialize_metrics(Runner),
    pull_latest_version(Runner)
    unittests_ok(Runner),
    build_artifacts_ok(Runner),
    acceptance_tests_ok(Runner),
    update_vcs_ok(Runner),
    finalize_metrics(Runner),
    results(Runner, Output)]

at that point, have I really saved myself anything?

Likewise with declarative timestamps and declarative UUIDs, it seems like "creating a command" doesn't really save much here. But if feels close to the right answer.

Somehow library(pio) and format_//2 do a nice job of converting normally side-effecting work into terms that can be reasoned about, but the generalization of this technique escapes me.

[jjtolton]

Wait, producing a series of commands might be more valuable than I thought, especially in Prolog.

Am I right in thinking that:

    [ticket_ok(Runner),
    initialize_metrics(Runner),
    pull_latest_version(Runner)
    unittests_ok(Runner),
    build_artifacts_ok(Runner),
    acceptance_tests_ok(Runner),
    update_vcs_ok(Runner),
    finalize_metrics(Runner),
    results(Runner, Output)]

(or something like it) would be a great target for a MI (metainterpreter) to consume? Is it "valid" to abuse an MI by making IT do the dirty work of the side-effects? Am I just shifting the complexity burden of the testing to testing the impure MI? 🤔

[jjtolton]

Well, there's also no reason why the interpretation of ticket_ok(Runner) couldn't also "expand" to yet another series of commands, and so on and so on until the commands are so small as to be easy to reason about even if they do side-effecting calls. 🤔

mi(ticket_ok(Runner)) :- 
   mi([vcs_details(Runner, VcsDetails), 
       ticketing_details(Runner, TicketingDetails), 
       latest_ticket(TicketingDetails, TicketId), 
       ticket_text(TicketId, TicketText), 
       latest_branch(VcsDetails, BranchInformation), 
       ticket_matches_branch(TicketText, BranchInformation), 
       other_required_info_present(TicketText)]).

🤯

[bakaq]

You could also then have multiple interpreters/runners that do the task in a different way. One could do it sequentially, the other parallelizing the parts that can be parallelized, one where no actual IO is done so that you can do deterministic simulation testing, etc... This here is kinda what a Monad is but without all the type level stuff, and may be the general way to solve "pure IO" in Prolog in an elegant way. We are doing exactly this in in my package manager: all the planning of what needs to be done is completely pure Prolog, and then we have a side-effectful runner that actually executes the plan.

[hurufu]

I had similar situation in the past, I wanted to mock I/O for tests. Maybe it doesn't solve exactly your problem, but let me share it nevertheless.

The idea is to run your program in a "world" where I/O predicates don't produce any side-effects. I got inspiration from Linear Logic which have very nice declarative reading. Here is the code:

:- initialization(main).
:- dynamic([out/1]).

main :- callenv(useful,  example).
test :- callenv(useless, example).

example :-
    complex_predicate(P),
    out(P).

complex_predicate(P) :- P = "Hello world!".

callenv(useless, G) :-
    setup_call_cleanup(asserta(out(_)), G, retract(out(_))).
callenv(usefull, G) :-
    setup_call_cleanup(asserta((out(A) :- write(A))), G, retract((out(_):-_))).

Your main program resides in example/0 predicate, and it does I/O only via dynamic out/1. That means that you can execute your program in an environment were out/1 has side-effect (here "useful" environment) or is a no-op (here "useless" environment).

P.S. You can read about algebraic effects, but I find it an overkill for my use-case.