Real time stdout handling

[Taitava]

@Troberg suggested this idea in #48:

One more request on this one, and to be honest, I don't know if it's doable, and it's not very important, but still useful: It would be nice if StdOut is added to the document on the fly, not waiting until the shelled program exits. That would allow programs which can remain open and insert text as needed, for example a "text soundboard" (a program which inserts standard texts at the push of a button), or when triggered by external events.

I think it's a good idea. Like you suspected, I'm not sure either if it's doable, though. I need to think about this, so at least this won't happen anytime soon.

If this is doable, then the next feature to think about might be, should a user be able to send messages to the already spawned process?

[Troberg]

I wouldn't need to send messages to the spawned process, at least not now. I'm sure it could be useful, though.

The next step would probably be able to send commands from the spawned process to control Obsidian, but that might be a security risk...

[Taitava]

The next step would probably be able to send commands from the spawned process to control Obsidian, but that might be a security risk...

Obsidian can be controlled by external processes already even without the Shell commands plugin, by using Obsidian URI, and possibly Advanced Obsidian URI.

Just out of curiosity, what kind of security risks can you see here?

[Taitava]

Just a small update on this topic: I started discussing about a change that would allow the reltime usage of stdout (= streaming) in another feature request's discussion topic: #89 (reply in thread)

This is not yet to say that any kind of progress would have made, but more to inform that this is still under planning. Even if the change gets implemented in the future, it will take even more time to use it for enabling the realtime handling of stdout.

[Taitava]

Trying to implement this

I have just some short time now and I decided I could at least start trying to do this. I'll see how far I get making the output handling realtime. I just wanted to write a short plan for how all the current output channels should alter their behavior in realtime mode.

• Ignore: No changes required.
• Status bar: Show the latest outputted text in the statusbar. Previous output from the same process should be shown when howering over the staturbar element. Clear the statusbar element every time a new shell process gives it's first output (= practically the same as what is the current behavior).
• Notification/error balloon: Two decisions to consider:
  • a) Display each outputted text part in a new notification balloon.
  • b) Reuse an old notification balloon (if still exists) and add new output text to there. When adding text, reset the notification's lifetime so it won't disappear too soon. I'm not sure if this idea B can be implemented with Obsidian API. Edit 2022-10-30: Will use this one, I got it working.
• Current file:
  • Caret position: always insert new text output to the caret position. Make sure caret is positioned to the end of the newly inserted output, so that next output will be inserted to the end of the previous output.
  • Bottom: Works very well without changes (I guess), new output will always go to the end.
  • Top: I don't know. Maybe insert new output always to the top, but then e.g. multiple echo outputs may go into upsidedown order. Perhaps a good temporary solution would be to prevent execution if output channel is Current file: top and realtime mode is turned on, and display a message that this problem is not yet solved. Edit 2022-10-30: I decided that output will go to "upsidedown" order, and that feels ok. If the order needs to be cahnged later, can try to come up with a better idea and create setting for it.
• Clipboard: Clear the clipboard when a shell process gives its first output. After that, always update the clipboard content by adding outputted text there, do not replace text.
• Ask after execution: Just add output text to the bottom of the big textarea element every time. If user has edited previous output, it does not matter, new output will just go to the end of the textarea, so there's no problem with a user editing old output at the same time.
• Open files: Just open files every time new output comes. Should not be any problems.

[Taitava]

Progress...

Here's a quick GIF showing how realtime output receiving works in the Ask after execution output channel. The test shell command outputs three lines of text, waiting one second between each line.

The modal indicates that the execution process is still going on by showing an "Executing..." text. When the execution finishes, the text is replaced with an exit code.

P.S. This is not shown on the GIF, but users can edit the output text manually even when the execution is still in process. All coming output will be inserted to the end of the textarea, so user's manual edits will not be overwritten. 👍

[Taitava]

stdout and stderr might go into wrong order

Quote from myself in #275:

I have one problem that I need to think about. If both stdout and stderr contain output, sometimes they will be outputted in a bit wrong order.

Example:

echo Line0
[Console]::Error.WriteLine("Error line1")
[Console]::Error.WriteLine("Error line2")
echo Line1
echo Line2
echo Line3
echo Line4
echo Line5
echo Line6
echo Line7
echo Line8
echo Line9

Sometimes Error line1 and Error line2 are processed after Line1, or even later. According to my quick research, it looks like there's no way to guarantee the output order:

* [Order of stdout and stderr events not maintained nodejs/node#26516](https://github.com/nodejs/node/issues/26516)

* https://stackoverflow.com/questions/57046930/node-js-spawn-keep-stdout-and-stderr-in-the-original-order

I have only tested this in PowerShell.

I posted this message also here so that there is a commenting place - in case someone will actually find a solution and want to discuss about it. So please, ideas are welcome from anybody! 🙂

[Taitava]

Released now in 0.17.0

[Taitava]

Instructions on how to make your shell commands output be handled in realtime: https://publish.obsidian.md/shellcommands/Output+handling/Realtime+output+handling
Contains also notes about how different output channels behave differently in realtime mode.

@Troberg I know it's been a year, but in case you happen to have time at some point, please try it out and let me know how you like it 🙂. Any feedback is welcome! 😎