Argparse: "Default" interacts incorrectly/non-intuitively with action='append' and action='extend'

[rhetzler]

Bug report

Bug description:

The documentation for default claims:

The default keyword argument of add_argument(), whose value defaults to None, specifies what value should be used if the command-line argument is not present
which (implicitly) sets the expectation that the default is not used if the command-line argument is present

in it's defense, the strange behaviour is documented in append but not extend

If the default value is non-empty, the default elements will be present in the parsed value for the option, with any values from the command line appended after those default values

However, this behaviour is non-intuitive. The fact that the values will be prepended to the end-user's input is not obvious, furthermore, the end-user is unable to specify a set of parameters that don't contain undesired default values.

    DEFAULT_SET=['a','b','c','d']
    parser = argparse.ArgumentParser()
    parser.add_argument('--items',nargs='+',action='extend',dest='items',choices=DEFAULT_SET,default=DEFAULT_SET)
    args = parser.parse_args()

    print(args.items)

$ python3 example.py --items a --items b c # expected: items will contain 'a', 'b', and 'c'
['a', 'b', 'c', 'd', 'a', 'b', 'c']

Now it's fair to say that there's a workaround:

    parser.add_argument('--items',nargs='+',action='extend',dest='items',choices=DEFAULT_SET,default=None)
    args = parser.parse_args()
    if args.items == None:
        args.items = DEFAULT_SET 

however, isn't this the pattern that default= aims to eliminate/alleviate in the first place?

If the purpose of DEFAULT is to supply a value in the absense of user input, the fact that it supplies a value even when the user supplies input is an undesired side effect and not really default behaviour.

If the code needs to always be processing a set of values regardless of user input, then that really should be clearly stated in the code/logic, rather than a side effect of the input parsing

    do_something(DEFAULT_SET + args.items) # no ambiguity

hope this makes sense. thank you!

CPython versions tested on:

3.8

Operating systems tested on:

macOS

[hpaulj]

With all action classes, the default is inserted into the args Namespace at the start of parsing.

At the end of parsing, if the args value is still the same as the default, and it is a string, it (may) be passed through the type function. I'd have to check if it's passed by the choices. Quick tests with extend suggest it is not.

Anyways, with extend, and the default is a list, each user input will be added to that list in the normal extend method.

In your example, both --items a and --items b c extend what was already in the Namespace. It does not know, in any way, that 'a' is the first item, and 'b c' the second.

So default isn't 'prepended to the user inputs. User inputs are added to the default, and any previous user inputs.

For default 'store' action, user inputs overwrite the default in a transparent manner. 'append' and 'extend' handle the defaults in the same way, just differing in what list method is used. If you have more questions, I suggest looking at the __call__ method of the respective Action subclasses.

[serhiy-storchaka]

The documentation for extend requires improvements (see also #84545). But the behavior is correct and pretty straightforward. Reclassified this issue as the documentation issue.