Control the amount of information logged with --verbose flag

[ifariasg]

Is your feature request related to a problem? Please describe.

When using the slurm executor and the --verbose flag the size of the log file generated grows very quick for large workflows.
In my workflow composed of ~4000+ rules the log file can grow to 1gb+ which makes it difficult to explore.

I might not be entirely correct here, but my understanding is that with the --verbose flag the slurm log is composed of (in somewhat of an order):

1. DAG build phase with all all rules and why they need to be rerun
2. Rule submission information (including the CLI command invoked by the executor to submit a job)
3. Queue state (sacct state of each job), which is printed on every status check.

Step 3) in particular is the one that makes the log file grow in size very quick in workflows with hundreds of rules being run simultaneously.

I use the --verbose flag to keep watch of the command that snakemake submits for each rule, specifically to keep track of how input functions/wildcard evaluation and resources are being mapped to slurm.

Describe the solution you'd like

A slurm-executor specific --slurm-verbose flag where you can decide what is printed to the log file. Options could be:

• "dag" (to print 1)
• "rule" (to print 2)
• "args" (to print the CLI command that accompanies each rule/group)
• "queue" (to print 3, the full job status queries)
• "default" (equivalent to not using --verbose)

Describe alternatives you've considered
Allow for an alternative to print/log the args used by snakemake for each job submission.

Additional context

[cmeesters]

Thank you for bringing this issue to my attention! I have been knocked out for a while and was unable to respond. Only now working backwards to solve issues, albeit at a lower pace than before.

I'm afraid, there is only one global logger instance. While I understand the issue and think that you have a point, changing this behaviour is not straightforward. Will put it on my to-do list. If you want to go ahead and make a PR, it would be most appreciated.