show output

[gingerlime]

Trackman looks great! Thanks for creating it.

I'm looking at using it to parallelize parts of our CI/CD process however, and one thing that I'm missing is the ability to show output for each step.

Currently, I can use show_command but it won't show output. Setting --loglevel debug will show output, but it will be "mixed" between different steps...

I'm wondering if there's a way to "de-mux" the output of each step?

[khash]

Thank you for your suggestion @gingerlime

I think we can certainly relax the logging a bit so you don't need to have a debug log level just to see the process outputs. As for demuxing, we need more time to investigate the options (specially as most of those interfere with the normal Linux pipeline setup many use).

[khash]

@gingerlime a new release is on its way that allows each step to have it's own logging configuration. This means you can split the logs into different files. However splitting them onto different regions of the display is not supported in this release. I guess you can use tmux tail or something similar for that purpose.

[gingerlime]

@khash thank you. It sounds great! Looking forward to it.

[khash]

v1.0.2 with split logging is released https://github.com/cloud66-oss/trackman/releases/tag/1.0.2

[gingerlime]

Hi @khash sorry for the delay, but I only now had a chance to play around with the logs. Overall it works great! Thanks again.

A couple of small things however:

• with format=text and file output, I still get a formatted message rather than the raw output ... I guess I can output to json and then parse it though, but I thought a plaintext stdout/stderr of the command would be more intuitive?
• to get the output, I still need to set the log level to debug. Somehow feels a bit counter-intuitive to me. I don't really care about debug or info messages, I do care about my command output...
• I also noticed however that even when I set the log destination to a file, if I set the log level to debug, I still get DEBUG output to stdout when I launch trackman ... (I guess I can just pipe to /dev/null if I want to silence it, but also feels a bit counter-intuitive to me)

Nothing major, and definitely works great, but just some observations more about the "ergonomics" of it I guess :)

[khash]

@gingerlime thanks for the comments.

• As for the log-format=text the format will always include the log level, date etc. I suppose the issue is these being interlaced with the process output. As the process output can be received in chunks, there is no way for Trackman to dump all of them in a single log. Also, since you can run multiple steps in parallel, not having the metadata on each line can make the logs from different steps mixed and unusable.
• I agree that seeing the process logs shouldn't need setting the level to debug. Let me see what we can do about that.
• I'm less inclined to change this behaviour if we can remove the need to set the logging level to debug if the purpose for that is to see process output. Unless you have a different reasons for this?

[gingerlime]

Hi @khash sorry for the delay. We ended up using gnu parallel instead. It fits our simple use case of running multiple jobs, configurable behaviour on failure, and getting output from each job separately (although it swallows colors, but I guess that's some kind of a TTY limitation? I'm not sure).

trackman definitely looks like a more modern alternative, and I really like what you're doing with it. These limitations make it harder for us to use unfortunately.