[Feature / Question] Access to exact raw args?

[mcblum]

Description

Hello again!

I'm working on implementing a small CLI to assist my family in configuring their Ubiquiti products so that they "just work". One of the quirks of this process is that sometimes you need to change the user's group when running a command when changing the config on the device. The process they suggest is to basically re-execute the command you just ran but with a different group. Doing a little investigation, it appears to me that Bashly has already processed some things about the command args even when initialize.sh runs - is that correct?

This is the code snippet that they recommend as the way to re-execute the command but with a different group:

if [ "$(id -g -n)" != 'vyattacfg' ] ; then
    exec sg vyattacfg -c "/bin/vbash $(readlink -f $0) $@"
fi

I can, of course, add some config to each command and re-construct the exact command that was run, but, before I do that, is there a way to access the raw args within the Bashly structure?

Thank you, the new features added recently are dope.
Matt

[DannyBen]

Hmm - I love how you stretch bashly to its limits :)

Well - I think I know what your issue is.

As you can see at the bottom of your generated script, initialize is the first thing being called:

if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
  initialize
  run "$@"
fi

And the initialize function only does minimal declaration before embedding the content of your initialize.sh:

# :command.initialize
initialize() {
  declare -g version="0.1.0"
  set -e

  # src/initialize.sh
  echo "$*"
}

However - as can be seen - the call to initialize does not pass "$@", so there is no way for you to "preprocess" the args beforehand, or store them, or use them.

Need to think a little about what would make sense here, as it can be a fragile situation.

[DannyBen]

One thing I can offer you right away:

If you look inside the run function in your generated script, you will see that this happens:

run() {
  ...
  normalize_input "$@"
  parse_requirements "${input[@]}"
  before_hook
  ...
}

This takes the raw input, normalizes it - which just some simple tasks of converting -abc to -a -b -c and so on. This function then creates the input array, which is the same as "$@", only normalized:

Immediately after that, it calls the before hook function. So, for your use case - I would probably check if this before hook is a suitable case for you to use the input array, or save it for future use.

In other words, try this:

# src/before.sh
echo "${input[@]}"

If this is going to be helpful, I can commit to keeping this input array as a documented intentional feature.

The same input array is of course available inside your command function, so you can probably create your own lib function that is called before every call to every command, and it will do that recall you need.

[pcrockett]

in hindsight, as long as input is an undocumented unintentional feature, it might be wise to change those kinds of variable names so they are harder to accidentally overwrite. ex: __BASHLY_INPUT or something like that. i can totally see myself defining an input variable in a before hook for whatever reason, then being surprised when things stop working.

[DannyBen]

The decision to avoid namespacing global variables and internal functions (think inspect_args()), along with the decision to use as few of them as possible, was very intentional. As a user, I am working on bash scripts, I do not want to be constantly made aware of the framework and its needs.

This is sort of a symbiotic relationship between the developer and bashly.

If the word "bashly" appears in a generated script (except the optional info header), I consider this a design flaw.

That said - perhaps it would be a good idea to document all the different variables or functions that may be considered "reserved words" in a bashly environment.

The thing is that over-documenting becomes pointless at a certain level. I mean, if someone happens to bump into this issue, it will be easier for them to look inside the script and identify the problem than to go to some documentation site and find out where the documentation author chose to place it, if at all.

[DannyBen]

@pcrockett another thing I just observed: The input array that is accessible in the before hook has already served its purpose, so even if you use this name anywhere, it should not be a problem.

This code snippet from run() shows why:

run() {
  ...
  normalize_input "$@"   # creates `input` array
  parse_requirements "${input[@]}"   # consumes `input` here and converts it to `args`
  before_hook
  ...
}

[pcrockett]

sort of a symbiotic relationship between the developer and bashly.

good point. especially in bash-land, it's kind of impossible to have truly "black box" abstractions. better to have more of a "power tool" philosophy. so i think i agree.

[mcblum]

I appreciate all of this discussion and also the thoughtfulness that goes in to planning Bashly. I'm still working on my Ruby, and I'm hoping to be able to contribute more actual code soon instead of just breaking things :)

Immediately after that, it calls the before hook function. So, for your use case - I would probably check if this before hook is a suitable case for you to use the input array, or save it for future use.

I'll give that a try today!

@DannyBen do you think it would be 1. Desirable for the framework and 2. A good first PR to add something like a raw_args var which is also accessible later on, should it be needed? I don't know why so many of the things I do seem to run me up against permissions / user issues, but the number of times I end up reading something like "the solution to this is to check if group xyz was used when calling the script and, if not, just re-call it but modify the group"...

Alternatively, if input isn't going anywhere, I may just be able to use that, too.

[DannyBen]

Well, you touched on several issues, and I will start from what I think might be the most important.

As a long time linux user who used many different distributions in a variety of settings (personal, corporate, startup) - I can tell you that in my experience, if you run into group related problems that are not easily and logically solved by adding the users to the group, then something is not set up correctly. There should usually be no need under normal circumstances to think about groups at runtime.

So, from this standpoint, I would try to eliminate the problem instead of solving it. Or in this case, look at the source instead of trying to create awkward subshell execs that manipulate groups at runtime.

Now, for the bashly-specific issue:

1. There should be no need for users to modify the incoming arguments. I am willing to be convinced otherwise, if there is a good real life use case. This is bashly's way of saying - "you deal with your code, I will deal with the user's input".
2. There should normally be no need for users to read the incoming raw arguments (the command line itself). However, I can accept the fact that sometimes it might be needed. This is already solved in the input array, for the most part.
3. In case we all reach a different conclusion - that perhaps we should allow users more control over it - then I would look in the initialize / normalize_input / before hook / run relationship, and evaluate what can be done to allow users to inject logic (with or without write access) to the raw input line.
4. Maybe it can be a good first PR, assuming we reach an understanding of what (if at all) we want to do.

[DannyBen]

I would like to put this here, in case it resonates.

There may be a simple way to allow anyone to interfere with the "$@" input - read and write it - perhaps without too much mess.
The one thing I had to keep in mind - and I hope I did - was the fact that run command can be called by users from other commands, so the handling of its "$@" parameters, and the processing of the input line must remain inside that run function.

So, the idea here is simple:

1. Pass "$@" to initialize - unlike what is happening now. This will make them accessible to users who just want to read them, and maybe store them for later use.
2. Furthermore, if users want to modify them (assuming we want to allow this use case), then they simply need to declare a global array, which I named in this example raw_input but we can use another name (just not input).

# :command.initialize
initialize() {
  declare -g version="0.1.0"
  set -e

  # src/initialize.sh
  # here we can read the raw input by examining "$@"
  # in addition, if we want to change them, we need to create a new array
  # called "raw_input"
  declare -g -a raw_input=("modified" "args")

  echo "in initialize"
}

# ...

if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
  initialize "$@"  # <- Passing "$@" now

  # Instead of calling just `run "$@"`, we see if the `raw_input` array 
  # was defined by the user.
  if declare -p raw_input &>/dev/null; then
    run "${raw_input[@]}"
  else
    run "$@"
  fi
fi

Running this with any input will show:

$ ./download asd
args:
- ${args[source]} = modified
- ${args[target]} = args

This is still incomplete, since it only lets people mess with the input during the initialize phrase.
The before hook's behavior is still the same - it has read access to the (now possibly modified-) input array.

And since the before hook is declared (by documentation) to have access to the parsed args array, it means that this cannot change. It cannot have write access to the input array without a breaking change, which is probably not a good idea anyways (i.e. having args here is better than having input).

Is it good? Is it enough? Is it confusing?.... I am not sure yet.

[mcblum]

@DannyBen first, I agree with everything you said. There shouldn't ever need to be a need to mess with the users at runtime. And yet... an example I had last week was writing a script to manage Vyatta router configs. By default, the admin user isn't part of nor does it run using the vyatta group, but for certain operations you have to run that as the group or it explodes. There may be a better way of dealing with it, but all of the other scripts I could find as well as the official docs were indicating to just re-call with the group changed.

As to your proposed solution, I think that's a great implementation. I would guess that the vast majority of people do not need to do any of this, but for those of us who either 1. Don't know how to use Linux or 2. Come across the rare edge case, this gives an option to save the args as they were passed in their entirely should we need to do something dumb.

[DannyBen]

Released in v1.3.1 - thanks for the issue.