Implementing goal timeouts?

[jjtolton]

I am curious if anyone has any thoughts on what it might take to achieve some semantics along the lines of time_out/3?

The purpose of this is I want to be able to use Scryer async mechanics for streaming over chunks of socket data so that we can handle multiple connections at the same time and get intermediate data without needing for the entire process to terminate for use with get_n_chars/3. The best way I can think to do this would be to do goal timeouts on blocking I/O. It's not perfect but it would be a good start.

It sounds like we might need to have some kind of watchdog timer on the rust side?

[triska]

Maybe get_n_chars/4 could be added, with an additional argument specifying the timeout?

[jjtolton]

This would be better -- especially if the semantics were such that Chars became bound to however many characters could be obtained in Seconds.

[triska]

There is an issue for this #1083 and even a PR thanks to @aarroyoc: #1152

Maybe sockets cannot be handled in the same way as pipes. If so, and they may need to be handled separately, then it would already be invaluable to have a timeout for reading from sockets, because other communication patterns can likely be mapped to sockets together with library(process).

[jjtolton]

@aarroyoc @Skgland @bakaq I'm happy to take a swing at implementing this on the rust side. It will be sub-optimal most likely but enough to get things moving. The best outcome would be "slicing I/O". What I mean by this is something like

?- get_n_chars(Stream, N, Chars, [timeout(Seconds), close(false)])

where N would be the number of Chars (or bytes) that were gathered during float Seconds (or would it be better to do the timeout in integer milliseconds?)

It would be better to apply this consistently across the back-end rather than ad-hoc, but this is where my rust knowledge is not strong enough to know how to apply this consistently for robust behavior.

Also @triska (and anyone else), I would be curious if you had any thoughts on the best Prolog semantics for this. Should this always be applied only to Streams? For instance, are sockets somehow uniquely different than Streams, or could this be applied consistently to Streams?

Motivation

I believe if we are able to achieve slicing I/O, we can make some extremely powerful tools (such as web servers and Erlang-style processes) without the need to implement OS-level threading, and this would dramatically increase the power and reach of the applications which Scryer is suited for. Because it does not rely on OS-level threading, it would also be applicable to WASM/javascript code, and therefor very useful as a web language!

[jjtolton]

Sockets are extremely important, but I think file descriptors will be very important too for long running processes.

i.e.

:- use_module(library(async)).
:- use_module(library(process)).
:- use_module(library(charsio)).
:- use_module(library(format)).

shell_program("sleep 1; echo 'a'; sleep 1; echo 'b'; sleep 3; echo 'c'").


write_proc_data(Proc, Stdout) :-
        process_wait(Proc, Status, [timeout(0)]),
        write_proc_data(Status, Proc, Stdout).
                                 

write_proc_data(exit(_), _Proc, _Stdout).
write_proc_data(killed(_), _Proc, _Stdout).
write_proc_data(0, Proc, Stdout) :-
        write_proc_data_(Proc, Stdout).
write_proc_data(infinite, Proc, Stdout) :-
        write_proc_data_(Proc, Stdout).

write_proc_data_(Proc, Stdout) :-
        get_n_chars(Stdout, N, Chars, [timeout(0.1), close(False)]),
        format("~s~n", [Chars]),
        await(write_proc_data(Proc, Stdout)).



run :-
        shell_program(Program),
        process_create("bash", ["-c", Program], [stdout(pipe(Stdout)), process(Proc)]),
        process_create("bash", ["-c", Program], [stdout(pipe(Stdout1)), process(Proc1)]),
        async_event_loop([write_proc_data(Proc, Stdout),
                          write_proc_data(Proc1, Stdout1)]).

I think it would be ideal in this case to not have to write a socket server.

[jjtolton]

However, if there is a way to universally write a socket server for a file descriptor stream in Prolog that we could use that transparently in such cases, that would be fine and very versatile. I do not know if this is possible or not, but it would allow us to consolidate effort.

[hurufu]

Is CPU time really good measure for timeouts? It will not cover time spent inside system call waiting for data to arrive, isn't it? What exactly is measured by '$cpu_now'/1?

[jjtolton]

Is CPU time really good measure for timeouts? It will not cover time spent inside system call waiting for data to arrive, isn't it? What exactly is measured by '$cpu_now'/1?

Hmm this is a good point. For the purposes of what I was thinking, it does not matter if it is CPU time or wall clock time or any other form of time as long no bytes are dropped.

However @hurufu something tells me that this might be an important choice, although I could not articulate why.

[triska]

Yes good point! The timeout means walltime!

[jjtolton]

@mthom or @Skgland do we need all of this B59 for alias? I'm trying to stick a timeout on the stream options

#[bitfield]
#[repr(u64)]
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub struct StreamOptions {
    pub stream_type: StreamType,
    pub reposition: bool,
    pub eof_action: EOFAction,
    pub has_alias: bool,
    pub alias: B59
    // pub stream_timeout: B16
}

[jjtolton]

Ok, I have a very crappy proof of concept for non-blocking I/O by focusing on peek_char in char_reader.rs. I hard-coded a timeout of 100ms.

The general technique works like this:

1. spawn a thread
2. try to read data within a certain period of time
3. cancel the thread if the timeout hasn't been completed.

This causes get_char/2 to fail if it times out before data is read, but it can be retried later (after a sleep/1, for instance)

?- process_create("bash", ["-c", "sleep 1; echo 'hey'"], [stdout(pipe(Stream))]),
   length(Cs, 4),
   (             maplist(get_char(Stream), Cs)
   ;             sleep(1)
   ),
   length(Cs1, 4),
   maplist(get_char(Stream), Cs1).
%@    Stream = '$stream'(0x55f9461332a0), Cs = [_A,_B,_C,_D], Cs1 = "hey\n".

Unfortunately, this broke something somewhere else:

   ?- process_create("bash", ["-c", "sleep 1; echo 'hey'"], [stdout(pipe(Stream))]),
   (             get_n_chars(Stream, N, Cs)
   ;             sleep(1)
   ),
   get_n_chars(Stream, N1, Cs1).
   %@    error(syntax_error(input_output_error),get_n_chars/2).

If you can happen to spot any obvious errors in here, I'd appreciate it: master...jjtolton:scryer-prolog:nonblocking-io-poc

(yes, most of this was tortured out of a carefully supervised LLM -- I claim no craftsman ship here. My rust is not nearly strong enough to do this task justice, but it needs to be done...)

[triska]

This holds for a specific form of OS; nothing prevents an OS to make files accessible as Prolog facts, for example.

[jjtolton]

Well now that I'm awake I think the reason why get_n_chars/3 is breaking is because it is gets an unexpected false somewhere. The fact that it is coming back as a error(syntax_error(input_output_error),get_n_chars/2) is somewhat baffling, though.

[jjtolton]

Hot damn, I think I got it.

:- use_module(library(process)).
:- use_module(library(time)).
:- use_module(library(lists)).
:- use_module(library(charsio)).

sleepy(S) :- '$sleep'(S).

?- time((process_create("bash", ["-c", "sleep 5; echo 'hey'"], [stdout(pipe(Stream))]),
   length(Cs, 4),
   maplist(get_char(Stream), Cs))).
%@    % CPU time: 0.012s, 574 inferences
%@    Stream = '$stream'(0x55c045eaa5e0), Cs = [end_of_file,end_of_file,end_of_file,end_of_file].

?- process_create("bash", ["-c", "sleep 1; echo 'hey'"], [stdout(pipe(Stream))]),
   sleepy(1.1),
   length(Cs, 4),
   maplist(get_char(Stream), Cs).
%@    Stream = '$stream'(0x562963ce3f80), Cs = "hey\n".

   ?- process_create("bash", ["-c", "sleep 1; echo 'hey'"], [stdout(pipe(Stream))]),
      length(Cs, 4),
      maplist(get_char(Stream), Cs).
   %@    Stream = '$stream'(0x55a1780c04a0), Cs = [end_of_file,end_of_file,end_of_file,end_of_file].

   ?- process_create("bash", ["-c", "sleep 1; echo 'hey'"], [stdout(pipe(Stream))]),
      get_n_chars(Stream, N, Cs),
      sleepy(1),
      get_n_chars(Stream, N1, Cs1).
   %@    Stream = '$stream'(0x559649e3a550), N = 0, Cs = [], N1 = 4, Cs1 = "hey\n".

   ?- time((process_create("bash", ["-c", "sleep 5; echo 'hey'"], [stdout(pipe(Stream))]),
            get_n_chars(Stream, N, Cs))).
   %@    % CPU time: 0.013s, 516 inferences
   %@    Stream = '$stream'(0x7f32c4000bb0), N = 0, Cs = [].

   ?- time((process_create("bash", ["-c", "sleep 1; echo 'hey'"], [stdout(pipe(Stream))]),
            get_n_chars(Stream, _, Cs),
            sleepy(1),
            get_n_chars(Stream, _, Cs1))).
   %@    % CPU time: 0.012s, 600 inferences
   %@    Stream = '$stream'(0x55cbc379d210), Cs = [], Cs1 = "hey\n".

[jjtolton]

The CPU time is not actually showing the wall clock time, unfortunately. I finally understand the difference 😆

Anyway this seems to be a working recipe for making byte reads timeout after 100ms: https://github.com/mthom/scryer-prolog/pull/1152/files

The next thing would be to make a timeout/1 stream option so that this behavior could be optional.

[jjtolton]

This is the most exciting usecase:

   ?- process_create("bash", ["-c", "echo 'hello!'; sleep 1; echo 'hey'"], [stdout(pipe(Stream))]),
      get_n_chars(Stream, N, Cs),
      sleepy(1),
      get_n_chars(Stream, N1, Cs1).
   %@    Stream = '$stream'(0x5586575c5ce0), N = 7, Cs = "hello!\n", N1 = 4, Cs1 = "hey\n".

This is a demonstration of streaming input from a process!

[jjtolton]

Update:, get_n_chars/3 is nonblocking (at least on TCP streams)!!!! This doesn't solve my library(process) issues, but in general this is huge!!!

% this works great!
request_response_loop(Stream) :-
        get_n_chars(Stream, N, Chars),
        if_(N=0,
            sleep(0.1),
            (   format("~s~n", [Chars]),
                format(Stream, "Received: ~s", [Chars]),
                handle_term(Stream, Chars),
                request_response_loop(Stream)
            )
           ),
        request_response_loop(Stream).

[triska]

get_n_chars/3 is supposed to read everything it can, could you please post an example where it does not, with current master?

For instance, I tried:

?- socket_client_open('metalevel.at':443, S, []), get_n_chars(S, N, Cs).
% waiting...

[jjtolton]

get_n_chars/3 is supposed to read everything it can, could you please post an example where it does not, with current master?

For instance, I tried:

?- socket_client_open('metalevel.at':443, S, []), get_n_chars(S, N, Cs).

% waiting...

Very odd, the example I posted I thought was from master. Maybe I was using the wrong build artifact from my nonblocking IO branch, will check soon.

[jjtolton]

ahhh.. my system scryer was pointing to my async IO branch 🤦‍♂️ Too bad!