Layout how parameters are combined

[enryH]

Description of feature

Parameter for the pipeline have an hierachical order accoring to @jpfeuffer :

If an SDRF file is provided,

1. SDRF parameters - set or no - will of precedence over all other parameters
2. Command line parameters
3. JSON provided parameters

No matter what, my plan is to dump in pipeline_info the actual parameters that were used. For parameters that could be sample-specific this might be a challenge (as these cannot be easily aligned fully with the pipeline paramters).

My first aim is to lay out the process in detail in the documentation - and maybe at least ensure that parameters not explicitly set in the SDRF file are picked up from the command line or json file provided parameters.

[jpfeuffer]

I think CLI vs Json params is handled via nextflow and should be correctly reflected in the pipeline_info.
Only params in the sdrf might overwrite them.
Something to consider is that we initially planned and partially allow already that different files may be processed with different parameters if they have different values in those (e.g. the tolerance) columns in the SDRF.

[ypriverol]

I think that is possible but then proteomicsLFQ do not allow it. I think when proteomicsLFQ read the parameters for the merge it gets confused and raise an error, for the ID pipeline is totally possible.

[jpfeuffer]

This should work even in quant. IDMerger should use the broadest settings from all of them for the rest of the workflow. If you search one with 10ppm and another with 20ppm it should pick 20ppm. But yeah it is not recommended anyway.

[ypriverol]

I tested this long time back and didn't work, I will try it again and know why or if it works now.

[enryH]

CREATE_INPUT_CHANNEL module (process) list them. So if we could decide for each to take the min or max in case there are diverse values, we could make things align.

quantms/subworkflows/local/create_input_channel/main.nf

Lines 105 to 118 in ebc207a

	// for sdrf read from config file, without it, read from params
	if (is_sdrf.toString().toLowerCase().contains("false")) {
	meta.labelling_type = params.labelling_type
	meta.dissociationmethod = params.ms2_fragment_method
	meta.fixedmodifications = params.fixed_mods
	meta.variablemodifications = params.variable_mods
	meta.precursormasstolerance = params.precursor_mass_tolerance
	meta.precursormasstoleranceunit = params.precursor_mass_tolerance_unit
	meta.fragmentmasstolerance = params.fragment_mass_tolerance
	meta.fragmentmasstoleranceunit = params.fragment_mass_tolerance_unit
	meta.enzyme = params.enzyme
	meta.acquisition_method = params.acquisition_method
	}
	else {

[enryH]

One suggestion I got is to parse the parameters from the SDRF file using an input schema. assets/input_schema.json, see e.g. sample sheet schema specification. Soon there will be an online tool do it and test the processing.

The samplesheet (SDRF) is parsed part of the UTILS_NEXTFLOW_PIPELINE nf-core template workflow, which then can be the input to BIGBIO_QUANTMS.

To define which parameters are actually used in the end, the ones set in nextflow.config or in the SDRF file is still not entirely clear. I guess it will be rather a documentation issue, although we could think about combining the params map and the sdrf input maps. Dumping can be done using collectfile channel operator.

[ypriverol]

Basically, you are saying SDRF -> input_schema.json .. then all the way nf-core tools?

[enryH]

I think it is an alternative to the parsing of the SDRF file in CREATE_INPUT_CHANNEL where we create maps from each row of the SDRF. I can start from the nf-core template and only create a schema for the SDRF. So the SDRF will have a json schema to pick out the meta maps. I put a brief hint to remember

33f09dc

[jpfeuffer]

The problem is that they do not support ontology lookups. I told them multiple times already that this will be needed. Our validation is quite extensive. So even if you manage to parse the sdrf with that, you will still need to validate it with our parser.

[ypriverol]

I think we can have a validation step and then a conversion no?