Required options in conflict with typical use of --help

[wasserwerk]

A possibility to defining an option as required is great idea, but this is in conflict with typical use of --help or --version. When I start a program only with --help I expect only a usage output, and version number for --version. In this case the required options should be ignored. When a start the program for "normal use" the required options should be checked. At the moment I get an error message fatal: missing required options: "--level" when I start example.sh -h.

Can you solve this problem?

[libre-man]

Hi!

Thanks for reporting this issue. It was known that this issue existed, however a fix is unclear. I think there are two possible options. The first being adding an extra option to the parse function that enables or disables checking of required options. The second being an extra option when defining an argument that this arg bypasses the required checks.

I want to prevent possible bugs where checking these special arguments is forgotten after parsing. So maybe a condition could be helpful here.

What would your ideal solution look like?

[libre-man]

What also could be an option is an extra option in define-opts. This could be callback-when-given or something like that that would be called directly after the option is parsed and parsing also terminates directly after (by signaling a condition).

[wasserwerk]

An ideal solution is not trivial. The issue is deeper as the mentioned problem with options like --help or --version. A simple require parameter don't cover cases where you have dependencies between groups of options. An example: you have options --min and --max. When you define --max you must define --min. But you have also options --mean and --width, where you must define both or nothing of them, but... when you decide to use min/max you can not define mean/width.

But I think solving the --help problem is good first step. I'm not an Common Lisp expert, but I think your callback idea is a good aproach. I like this. I've tested how unix tools behave. When I run ls --help --version (cygwin on windows) I get only usage output and ls terminates. When I run ls --version --help I get only version number and ls terminates. A callback can fullfil this task.

[libre-man]

I think that it would be nice to add these constraints some way, however I think that doing this in the definition of options is a bit too much. Maybe adding some utility functions so they can be expressed in a nice way would be better.

I'll add something like the callback approach the next days!

[wasserwerk]

It is really difficult question. I have thought a lot in recent days and I tend to prefer the solution of complete definition of options. Let me show you an example.

(opts:define-opts
  (:name :help
   :description "show usage and exit"
   :short #\h
   :long "help"
   :exit t)
  (:name :version
   :description "show version and exit"
   :short #\v
   :long "version"
   :exit t)
  (:name :level
   :description "the program will run on LEVEL level"
   :short #\l
   :long "level"
   :required t
   :arg-parser #'parse-integer
   :meta-var "LEVEL")
  (:name :priority
   :description "Priority, only for levels < 5"
   :short #\p
   :long "priority"
   :required t
   :dependent '((lambda (:level) (< :level 5))))
  (:name :min
   :description "min value"
   :short #\n
   :long "min"
   :required t
   :dependent '(:max)
   :prohibited '(:start :width))
  (:name :max
   :description "max value"
   :short #\x
   :long "max"
   :required t
   :dependent '(:min)
   :prohibited '(:start :width))
  (:name :start
   :description "start value"
   :short #\s
   :long "start"
   :required t
   :dependent '(:width)
   :prohibited '(:min :max))
  (:name :width
   :description "width"
   :short #\w
   :long "width"
   :required t
   :dependent '(:start)
   :prohibited '(:min :max)))

You will find some new ideas:

1. :dependent defines dependencies to other options. It can be a list of this options (see min/max, start/width) or an lambda expression for more complicated cases like option prioroty which must be defined only when option level was defined with value less than 5.

2. :prohibited: specify options which can not be defined, when given option is used. This should also supported lambdas.

3. :exit t signalize thet when this options was found the application should execute some code and terminate immediately.

Note that min/max and start/width are all four labeled as required. But because there are defined prohibitions, the option parser should ignore prohibited options.

One idea for short names for option. I think it would be better to allow strings, not only single characters. It would be easer to name options when an application have many of them.

[tanders]

Have there been perhaps any developments concerning this matter, if only for simple help and version arguments?

I tried a simple hack as a way around for now as shown at the end of this message, where for opts:missing-required-option it simply checks whether help has been specified. The main change is the following.

;;; some surrounding code missing
	(opts:missing-required-option (con)
	  ;; hard-coded option checking in argv, as variable options is not bound yet
	  (unless (or (member "-h" argv :test #'equal)
		      (member "--help" argv :test #'equal))
	    (format t "fatal: ~a~%" con)
	    (opts:exit 1)))

However, while this avoids the error when now main is called with only the help option, the help message is not printed either.

$ ./myapp --help
free args:

Can you perhaps see what I am missing here? Thanks!

Full Function Definition

(defun main (&rest argv)
  (multiple-value-bind (options free-args)
      (handler-case
	  (handler-bind ((opts:unknown-option #'unknown-option))
	    (opts:get-opts argv))
	(opts:missing-arg (condition)
	  (format t "fatal: option ~s needs an argument!~%"
		  (opts:option condition)))
	(opts:arg-parser-failed (condition)
	  (format t "fatal: cannot parse ~s as argument of ~s~%"
		  (opts:raw-arg condition)
		  (opts:option condition)))
	(opts:missing-required-option (con)
	  ;; hard-coded option checking in argv, as variable options is not bound yet
	  (unless (or (member "-h" argv :test #'equal)
		      (member "--help" argv :test #'equal))
	    (format t "fatal: ~a~%" con)
	    (opts:exit 1))))
    ;; Here all options are checked independently, it's trivial to code any
    ;; logic to process them.
    (when-option (options :help)
		 (opts:describe
		  :prefix "example—program to demonstrate unix-opts library"
		  :suffix "so that's how it works…"
		  :usage-of "example.sh"
		  :args     "[FREE-ARGS]"))
    (when-option (options :verbose)
		 (format t "OK, running in verbose mode…~%"))
    (when-option (options :level)
		 (format t "I see you've supplied level option, you want ~a level!~%" it))
    (when-option (options :output)
		 (format t "I see you want to output the stuff to ~s!~%"
			 (getf options :output)))
    ;; always executed
    (format t "free args: ~{~a~^, ~}~%" free-args)
    )
  )

[iamFIREcracker]

Have there been perhaps any developments concerning this matter, if only for simple help and version arguments?

I tried a simple hack as a way around for now as shown at the end of this message, where for opts:missing-required-option it simply checks whether help has been specified. The main change is the following.

;;; some surrounding code missing
	(opts:missing-required-option (con)
	  ;; hard-coded option checking in argv, as variable options is not bound yet
	  (unless (or (member "-h" argv :test #'equal)
		      (member "--help" argv :test #'equal))
	    (format t "fatal: ~a~%" con)
	    (opts:exit 1)))

However, while this avoids the error when now main is called with only the help option, the help message is not printed either.

Have you tried invoking the OPTS:SKIP-OPTION restart from within the OPTS:MISSING-REQUIRED-OPTION handler?

(defun parse-opts (&optional (argv (opts:argv)))
  (multiple-value-bind (options)
      (handler-case
          (handler-bind ((opts:missing-required-option (lambda (condition)
                                                         (if (or (member "-h" argv :test #'equal)
                                                                 (member "--help" argv :test #'equal)
                                                                 (member "-v" argv :test #'equal)
                                                                 (member "--version" argv :test #'equal))
                                                           (invoke-restart 'opts:skip-option)
                                                           (progn
                                                             (format t "~a~%" condition)
                                                             (opts:exit 1))))))
              (opts:get-opts argv))
        (opts:unknown-option (condition)
          (format t "~a~%" condition)
          (opts:exit 1))
        (opts:missing-arg (condition)
          (format t "~a~%" condition)
          (opts:exit 1)))
    (if (getf options :help)
      ...

Important bits:

• We do want to invoke the OPTS:SKIP-OPTION restart when any of the special options is set -- this way the library will use NIL and move on
• HANDLER-BIND -- and not HANDLER-CASE -- needs to be used to handle OPTS:MISSING-REQUIRED-OPTION, especially if we want to call INVOKE-RESTART (with HANDLER-CASE the stack is already unwound, when the handler runs. Thus the restart established in the function is gone.)

[digikar99]

You will find some new ideas:

Is that an exhaustive list of use cases?

[rbugajewski]

Have there been perhaps any developments concerning this matter, if only for simple help and version arguments?

Is it still the case that to get usable behavior, you have to either stop using handlers and roll your own validation logic, or to create a PR for the ideas in the comment above?