Get output text from shell commands (output channels)

[Taitava]

Currently, most of the time any output text that shell commands might produce are just ignored, and not shown anywhere. The only exception to this is if a shell command fails to execute. Then the error message is shown for 20 seconds on the upper right corner. (The time is configurable in the plugin's settings).

Commands that produce useful output

There might be some beneficial use cases for regular output when a command executes successfully. For example:

• Get a list of files in the Obsidian vault's root: ls (Linux command) / dir (Windows command).
• Get a list of files in a sub directory: ls Folder (Linux command) / cd Folder & dir (Windows command).
• Check if a specific file exists: ls MyNote.md (Linux command). Outputs the file name if the file exists, or nothing if it does not exist. Could be finetuned to get a better understandable output: File exists / File does not exist.
• Check if today's file/folder exists: ls {{date:YYYY-MM-DD}} (Linux command). Outputs the folder name if the file/folder exists, or nothing if it does not exist. Could be finetuned too.
• Get a list of changed files in a Git repository: git status (Linux/Mac/Windows, if Git is installed). List files that are modified or deleted after the last Git commit, and files that are new (= files that are not stored in the Git repository).

There could be a lot more examples.

Different ways to use the output text

One reason why Shell commands does not yet have a possibility to get the output text is that I haven't been able to figure out all the possible forms how a user might want to receive the output. It's not enough to display the output on a balloon in the top right corner for a limited amount of time. There needs to be options.

Ideas so far (these could be called channels or something).

• A bullet point item means: Idea (some of these are not yet researched for if they can be implemented or not)
• An empty box means: Will be implemented next.
• A checked box means: Implemented, but not yet released.
• ✔️ means: Implemented and released.

---

• ✔️ A balloon on the top right corner. Yes, it can be one option for the user to choose. Suitable for multirow but short outputs, like a list of around five files in a folder.
• ✔️ A statusbar text. Suitable for a single line output, which is not too long. Can be visible for an extended amount of time without being in the user's way, unlike a balloon. Visible text in the statusbar should be truncated to a small length. Hover over with mouse to see the output text fully, if there's more.
• ✔️ Ask after execution in a modal (= popup window). Very prominent, and requires a click for it to go away. Can contain quite a lot of text: many lines, long lines. Not the fastest one to use, as - like I mentioned - it needs a click or a keypress to close the modal. Should have the ability to copy the output to the clipboard. Edit 2022-01-01: Will also support editing the output text and then selecting an output channel for it (to clipboard, to file, or to status bar).
• ✔️ Which brings to my mind... the clipboard. Need I say more? (Edit 2021-09-27: Maybe no clipboard option, because you can pipe the command output to other, natural commands that can put the output to clipboard. Edit 2021-10-06: On a third thought, this would be handy compared to piping.)
• ✔️ Insert into the current file at caret location. Can also replace text if something was selected at the time of executing a command. Good for supplying data where ever you want it in your notes. Permanent, output is saved on disk and does not disappear.
• ✔️ Insert into the top of the current file. Good for log type of note files, where the newest record will be on the top.
• ✔️ Insert into the bottom of the current file. Good for log type of note files, where the oldest record will allways be on the top, and the file can be read in natural chronological order.
• Insert output to another note in background (without opening the destination note) + defined positions: 1- caret position , 2-top , 3-bottom (option to apply line break \n after/before 1 2 3) - Thanks @FelipeRearden !
• ✔️ Ignore the output. Just like it's currently done.
• ✔️ Open a file in Obsidian. The output should be a path to a file, relative from the vault's root, or absolute. The file needs to reside in the vault. If the conditions are not met, an error message will be displayed.
• Anything else?

Customisable for each command

Not a single way to handle outputs suit all commands. Some commands are used very often, in which case the UI should not require the user to click or hit a key to close an output modal. Some commands produce more output than others. Therefore, it's beneficial that the user can configure a different channel for each command.

How to treat errors?

Would there be any case where the user would want possible error messages to be part of other output text? I need to think about this still. If there is a use case for this, I could offer the user two toggleable options:

• Direct possible error messages into the same channel that was chosen for normal output.
• Show possible error messages in a balloon at the top right corner (normal behavior).
  These two can be on at the same time: then errors can be shown both in a balloon and in another channel at the same time.
  Both of them can also be off at the same time: errors will be always ignored.

Edit 2021-10-07: The way I implemented error handling is quite different from my original thoughts. In short, the output channel for stderr can be chosen separately from the stdout channel. The same set of channels is available to stderr and stdout.

[FelipeRearden]

Different ways to use the output text:

• Insert output to another note (defined before in the shell command) in background (without opening the destination note) + defined positions: 1- caret position , 2-top , 3-bottom (option to apply line break \n after/before 1 2 3)

[Taitava]

Excellent! We just need to think about how the user can select the note where the output will go. I can see at least two ways to do it:

• Offer a dropdown selection menu which lists all notes from all folders (a tree structure grouped by folders). Easy for the user to pick up a note name if the amount of notes in the vault is quite small, 50 at max. But many people have vaults with hundreds of notes, so this does not serve them well. Does not support variables, which is a downside too.
• Offer a text field. The user needs to type the path to the desired note. Vault size is no problem. This is also used by some core plugins, such as Templates (for defining a folder) and Daily notes (for a folder and a file), so it could be in line with what users might already expect. This can support variables, so that the note name will be decided dynamically. A downside is that typing a file name is not convenient if the user does not rember the file name exactly - then they need to check it upfront by closing the settings panel and navigating to the file, and reopening the settings panel. But core plugins tend to ease this a little bit by offering an autocomplete menu that shows the available folders and files.

Btw if the file is not open, I'm not sure how caret position works, will Obsidian remember the last caret position when the file was open the last time? And more importantly, can the plugin access the caret position without opening the file first? But that can be checked during development.

I will add your idea to my first post.

[Taitava]

Oh, I didn't notice this:

(defined before in the shell command)

I'd like to have a separate field for defining where the output goes, as it's quite a different thing from the command.

And btw shell commands do have a natural support for directing output to files, e.g. ls >> MyNote.md lists the current directory's files and folders and writes the output to the end of MyNote.md. So >> appends to the end (on a new line), but you can also use a more dangerous one > which overwrites the whole file. Both of these operators create a new file if it does not exist. And 2> and 2>> can be used to direct error messages to a file.

That being said, I'd still like this plugin itself to have the ability to direct output to a file. Then we can direct also to top of the file or to caret position.

[FelipeRearden]

@Taitava !!!!!!

Thank you so much for >> works as you said on MacOS echo "Hello World" >> "/Users/username/vault_folder/MyNote.md"

• On MacOS you have to use the whole file name path and between " ".
• Another good variation echo {{clipboard}} >> "/Users/username/vault_folder/MyNote.md"

As I said before. Every time I learn something new with you my friend. Thank you very much :)

[FelipeRearden]

Just to let you know that I am deeply thinking about #17 and #16.

I'm gonna bring some ideas for us to talk about.

I wish you a fantastic Saturday :)

[Taitava]

Thanks, have a nice day too!

[Taitava]

I can't remember everything from the past discussions, and I'm in a bit of hurry to read all, but this output thing has been on my mind in the past few days, and now that it's also requested in #48, I feel that it's time to actually implement some of these ideas. However, there are some bugs that need to be dealt with first.

After fixing the bugs I'll start to implement at least this:

Insert into the current file at caret location. Can also replace text if something was selected at the time of executing a command. Good for supplying data where ever you want it in your notes. Permanent, output is saved on disk and does not disappear.

I'll try to implement it so that it's easy to later continue development with other ideas mentioned in this discussion. I'll edit #34 when I'll start this implementation, so it will contain details of what actually will be implemented in the first phase.

[Troberg]

I recently registered an issue on this feature request, without seeing this discussion (shame on me!). Either way, I have some thoughts on usecases and stuff, so I'll just past it here as well:

It would be very useful if one could launch an external program, and take the output from that program and enter it into the current note at the cursor.

Example: Write a phone number, select it, call the external program which looks up the phone number in a database, sends back formatted name, address, phone and so on on StdOut, which then replace the selected text.

It would also be great for writing "generators", where you use a GUI to enter information, do some stuff with that information, and then inserts it neatly formatted into the note.

[Taitava]

I recently registered an issue on this feature request, without seeing this discussion (shame on me!).

It can happen to all of us! 🙂 And I'm not even sure if the Discussions section is meantioned in README.md. Maybe I should check and put it there if it's not there.

It would also be great for writing "generators", where you use a GUI to enter information, do some stuff with that information, and then inserts it neatly formatted into the note.

External GUIs/generators is a nice usage example. A user could create tool programs using whatever language/framework they want, and invoke it from Obsidian and get the output back to Obsidian. 👍

[Troberg]

Yep, that's my immediate need. I open the GUI, enter some text, it does some formatting, fetches some additional data, does some calculations and validation, formats the output and returns it to the note.

[Taitava]

This is the first thing ever inserted by Shell commands into caret position in current, active file. It's Windows' dir command's output:

The file was originally empty, so next I'll test how it inserts in between text, and how replacing a selection works.

[FelipeRearden]

AMAZING !!!!!!!

[Taitava]

Hi @Troberg and @FelipeRearden !

I'd like to know your thoughts about what would you like to have as a default way to handle output (stdout), for those commands where a user does not explicitly define an output method? What would suit your use-cases the best? This default option will be applied to all your existing shell commands when you upgrade to 0.5.0 (not released yet).

Options:

• Ignore output (that's how it currently is)
• Show a notification balloon

This question concerns only the default value, so I left out the Current file, caret position option, as it would not make sense as a default.

[FelipeRearden]

Hello @Taitava and @Troberg !

My opinion is ignore output. Because I spend a lot of time testing my Shell Commands before using into my workflows.

BUT .....

When I'm testing, I can see an extreme value in having a notification ballon telling me that my Shell Command is finished.

This happened yesterday with me: I was building a QuickAdd Macro that Have the Shell Commands (my awk example that is in this Git)

• My first instinct was thinking that my Shell Command was not running
• But after a lot of thinking I realize that the Shell Command was running perfect and I made a mistake in QuickAdd.

---

Personally, I don't like this top-right balloons when I'm doing a workflow that I already know the result:

• Workbench generates 2 balloons every time I use one command
• I run this commands when writing books notes 100 times in 3 hours. It's something that I don't want to see in that moment.

---

All said my opinion is to ignore output. But maybe an toggle to define this per command might be a good feature when I'm on testing a Shell Command for a workflow to see the ballon telling that my shell command is done.

• After my tests, I get rid off the ballon.

Just to remember that I'm a newbie and you guys have more experience than me.

Forgive me if I did not understood correctly your question :)

Have a great day!

[Taitava]

Personally, I don't like this top-right balloons when I'm doing a workflow that I already know the result:

So I think I'll keep the Ignore option as a default. I can of course reconsider, if new ideas come up later.

All said my opinion is to ignore output. But maybe an toggle to define this per command might be a good feature when I'm on testing a Shell Command for a workflow to see the ballon telling that my shell command is done.

When you are testing, you can go to edit a shell command's settings and choose to display output in a notification:

(A similar dropdown selector element will be created for stderr too.)

Have a nice day, too! 🙂

[Taitava]

When I'm testing, I can see an extreme value in having a notification ballon telling me that my Shell Command is finished.

Btw, there will be no balloon if the command does not give any output, even if Notification was selected in the settings.

[FelipeRearden]

So I think I'll keep the Ignore option as a default. I can of course reconsider, if new ideas come up later.

Great idea @Taitava !

When you are testing, you can go to edit a shell command's settings and choose to display output in a notification:

Thank you very much!!!!!!!!!

Btw, there will be no balloon if the command does not give any output, even if Notification was selected in the settings.

Roger that! Let me see how the notification output will behave in my test workflow :)

This is fantastic @Taitava !!!!

Have a great day!

[Taitava]

Thanks, have a great day too! 🙂

[Taitava]

Just for the record in between this long discussion: These are now implemented in Part 1 (#34) and released in 0.5.0:

• A balloon on the top right corner. Yes, it can be one option for the user to choose. Suitable for multirow but short outputs, like a list of around five files in a folder.
• Insert into the current file at caret location. Can also replace text if something was selected at the time of executing a command. Good for supplying data where ever you want it in your notes. Permanent, output is saved on disk and does not disappear.
• Ignore the output. Just like it's currently done.

[FelipeRearden]

Thanks for the screenshots! Hmm, I need to think about it. What happens if you change an old command's setting to Ignore, then restart Obsidian and go to the settings again to see if the same command's output setting is still Ignore? I just want to make sure that saving changed settings work ok.

It is doing right. Other words: It saves my last change in the selection box !!!!!

Do you happen to have a backup of your old settings file (.obsidian/plugins/obsidian-shellcommands/data.json) from before upgrading to 0.5.0?

Sorry for this, I don't have :(

But I have an idea ....

Why we don't let things as is now and prepare for 0.5.1 ?

• I will change my settings to ignore;
• Save my data.json from 0.5.0 (as a backup in my desktop);
• Install the new version 0.5.1 and we see what is going on;

Since it's not a new field in the settings anymore, I think it will remember the last choice .

If it is not working in 0.5.1 we already have my backup for you to debug

[FelipeRearden]

⚠️UPDATE: New Information⚠️

After restarting Obsidian (close and open) the fields are ignore !!!!!

• I was going to start selecting ignore to prepare the data.json and all shell commands are ignore again.

What I did too before writing this to you @Taitava:

• I changed one shell command to Current File: caret position
• Restart Obsidian to see if it will save my change

Perfect behavior: ignore` for all shell commands except the one that I changed.

[FelipeRearden]

I think everything is Okay with 0.5.0 after a simple restart :)

Sorry for the false alarm @Taitava :(

[Taitava]

No problem 🙂 . I'd still like to test this more, because upgrades should go more smoothly. If there are incorrect setting values loaded after an upgrade, even if they change to correct values after a restart, I'd still like it to be so that the settings would be correct without a restart. There is a risk for dataloss if the settings are incorrect even temporarily. I started a new discussion #63 for continuing this possible bug discussion.

[FelipeRearden]

Okay @Taitava! I made a comment in #63 !

[Troberg]

Finally back home and able to test. Works beautifully, thanks!

One more request on this one, and to be honest, I don't know if it's doable, and it's not very important, but still useful: It would be nice if StdOut is added to the document on the fly, not waiting until the shelled program exits. That would allow programs which can remain open and insert text as needed, for example a "text soundboard" (a program which inserts standard texts at the push of a button), or when triggered by external events.

[Taitava]

Thanks. No need to post the same message twice, to two different places 🙂 .

And here's a link to a new idea discussion: #64. 🙂

[Troberg]

Yep, I know, it was a mistake. I intended to post here, but effed up. :)

[Taitava]

It's time for part 2: #68

[Taitava]

Here's a sneak peak of status bar output. I've run a command git status --short (shows changed files in Git version control system, in a short format).

Status bar shows permanently the last output line.

Hover the status bar item, and it shows the whole output. Not perhaps the most user friendly way to display output (the black box is narrow, won't get wider even if output text is wide). But I don't want to use too much time and energy on creating a completely new box for this thing. Status bar as an output channel is meant for short output, anyway.

[Troberg]

This is going to do so much for the extensibility of Obsidian. I expect a lot of stuff to appear which will have ShellCommand as a dependency.

[FelipeRearden]

Owww, this is happening !!!!

Status bar as an output channel is meant for short output, anyway.

100% agree. A place for short messages/notifications about something important to the user.

[Taitava]

Yes, the git status --short is a really handy feature for me in status bar. That way I can quickly check if I have done any modifications to the files or not, just by pressing Ctrl + S. Yes, I've taken that hotkey away from Save current file in my test vault to use it for Git status, because in Obsidian there's no need to manually save files IMO.

[FelipeRearden]

That's a great idea! It's the first user case for this new feature!

[Taitava]

About the option to apply linebreaks for the following:

• Insert into the current file at caret location.
• Insert into the top of the current file.
• Insert into the bottom of the current file.

Well, first of all, caret position output channel is already implemented, but it does not have the linebreak option. That's purely because I did not read @FelipeRearden's proposal accurately at the first time (sorry!), I only noticed the linebreak options for the top/bottom features.

Secondly, I'm planning to postpone this linebreak setting, due to UI reasons. I still need to think carefully how I will add output channel specific settings to the UI:

• Not all output channels will use the linebreak setting.
• Insert output to another note in background (without opening the destination note) does also need a setting for defining the target file name/relative path.

I'm thinking about creating a new modal for these settings, that would be opened from the current extra options modal:

The red marks show places for new icon buttons that would open the new modal containing the currently selected output channel's settings. The icon only appears if that channel uses any channel specific settings, so e.g. Ignore and Notification/Error balloon would not have the icon visible.

This new modal is still just an idea, which may change.

[FelipeRearden]

I only noticed the linebreak options for the top/bottom features.

No problem at all @Taitava !!!!! :)

I still need to think carefully how I will add output channel specific settings to the UI:

100% agree. We have a puzzle here, because each type of output will have different parameters (just like Insert output to another note in background needs the file path and Current file:top needs "just" a window for selection)

And we need to show what is the section made inside the modal (red dot)

I'm gonna think about this to see if I have an idea.

Let's keep talking :)

[Taitava]

I feel this puzzle can have as many moving pieces as imagination can throw us. Adding a line break to the output is just one thing. Here's a few more, but I'm just not sure if I'll want to implement them all:

• Only one linebreak? Why not multiple linebreaks? Simple: define a number of linebreaks you want, and then you can have more space between the outputted text and earlier content. Set this to 0 to disable adding linebreaks at all.
• Add linebreaks before or after the outputted content? Well, create two linebreak settings: how many linebreaks before the outputted content and how many linebreaks after it.
• What if I don't like linebreaks? I WANT SPACES! Raaahhhh! Should this actually be a separator that a user can customise to be whatever they want? Increases complexity.
• Or just shoot a fly with a bazooka: introduce output templates. Let a user define a template file that contains e.g.:

  Newlines, spaces, or text preceding the output.
  {{output}}
  Newlines, spaces, or text trailing the output.

  But this could be achieved via normal shell commands, too. And it opens up a totally new direction for development: what kind of variables should the template file support, and should it support e.g. Templater community plugin or Templates core plugin? I don't know, I'll have to see if there is any practical need for this kind of thing.

Btw, how about trimming the output? E.g. echo on Windows always puts a newline at the end of the output. I don't know if there's an option to disable the newline for echo. Perhaps on Mac/Linux there is? Maybe this could be just a simple checkbox in each command's settings? Like: Trim whitespaces out of the beginning and end of output? (Description: Whitespace: All space, tab, and newline characters, but not removed in the middle of output).

And we need to show what is the section made inside the modal (red dot)

I'm so quick to change my mind: Instead of creating a new modal, I'm now thinking about creating fields that appear/hide whether they are needed or not, in the current extra options modal. It would be easier for a user when the fields will be accessible in the same modal as the output channel selection. Much simpler than a new modal.

[FelipeRearden]

I feel this puzzle can have as many moving pieces as imagination can throw us. Adding a line break to the output is just one thing. Here's a few more, but I'm just not sure if I'll want to implement them all:

It's much easier to think about develop new feature when we already have a workflow to match (or a pain to cure).

• Your case with status bar and my case with tags

Btw, how about trimming the output? E.g. echo on Windows always puts a newline at the end of the output. I don't know if there's an option to disable the newline for echo. Perhaps on Mac/Linux there is?

The only thing that I learn after start talking with you is thatecho adds a line break and printf don't

Trim whitespaces out of the beginning and end of output? (Description: Whitespace: All space, tab, and newline characters, but not removed in the middle of output).

Great idea!

I'm so quick to change my mind: Instead of creating a new modal, I'm now thinking about creating fields that appear/hide whether they are needed or not, in the current extra options modal. It would be easier for a user when the fields will be accessible in the same modal as the output channel selection. Much simpler than a new modal.

100% agree.

[Taitava]

New output channels for both stdout and stderr:

* [x]  A statusbar text.

* [x]  The clipboard.

* [x]  Insert into the top of the current file.

* [x]  Insert into the bottom of the current file.

These are now released. (Just for the record here too).

[FelipeRearden]

Tests on MacOS here #68

[FelipeRearden]

'm so quick to change my mind: Instead of creating a new modal, I'm now thinking about creating fields that appear/hide whether they are needed or not, in the current extra options modal. It would be easier for a user when the fields will be accessible in the same modal as the output channel selection. Much simpler than a new modal.

@Taitava,

A few comments about our new channels for the future based on a new fields/modal:

1. Current File: TOP --> option to move the cursor to the TOP as well (maybe after the output of the SC);
2. Current File: BOTTOM --> option to move the cursor to the BOTTOM as well (maybe after the output of the SC);
3. Clipboard: --> option to disable the notification that appears with executing a SC with the clipboard output;

I'm still testing statusbar:)

Have a great day and thank you very much for these amazing new features !!!!

[Taitava]

• Current File: TOP --> option to move the cursor to the TOP as well (maybe after the output of the SC);

• Current File: BOTTOM --> option to move the cursor to the BOTTOM as well (maybe after the output of the SC);

I need to think about these.

(maybe after the output of the SC)

I think the UI can get a bit messy if there will be an option to select will the cursor go before or after the output that was inserted to top/bottom.

• Clipboard: --> option to disable the notification that appears with executing a SC with the clipboard output;

This can be done.

[FelipeRearden]

I need to think about these.

Ok !!!!!!!

I think the UI can get a bit messy if there will be an option to select will the cursor go before or after the output that was inserted to top/bottom.

yeah, this is the type of feature that we have to see working on the screen to really see if is good or not :)

This can be done.

Great !!!!!!!!

[Taitava]

Currently, when outputting to status bar, if the output text is empty, and the status bar contains text from an old shell command execution, the old text will not be removed from the status bar, because empty output text is considered to be kind of "no output at all, so do nothing" thing.

I think I should change this so that the status bar will be emptied if the newest output is empty. And of course emptying the status bar only means emptying the part of status bar that is controlled by SC, all other status bar content (e.g. word count) will stay intact.

[Taitava]

This change was done in #124 and released in 0.9.0.

[Taitava]

I added a new idea to the list:

Open a file in Obsidian. The output should be a path to a file, relative from the vault's root. The file needs to exist, and it must reside in the vault. If the conditions are not met, an error message will be displayed.