Parquet schema for speclib.parquet?

[jpfeuffer]

Hi @vdemichev

I was checking the Readme.md (and some other discussions) but could not find any description of the parquet file that can be used to pass a pre-generated speclib to DIA-NN.

Would you be able to provide a parquet schema for the most important/required columns such that tools can easily write this format for usage in DIA-NN?

I tried to extract it from Skyline's source code but it looks very minimal there
https://github.com/ProteoWizard/pwiz/blob/7c29cd39f89c191eb2a95e6e6bb750c78f9a9987/pwiz_tools/BiblioSpec/src/DiaNNSpecLibReader.cpp#L653

[vdemichev]

Hi Julianus,

Yes, it's not specified, only parquet files generated by DIA-NN should be used.

Best,
Vadim

[vdemichev]

If you'd like to generate a library with some other tool, please save it in a .tsv format (e.g. same as FragPipe does or as DIA-NN 1.8.1 does). In general, quite a wide-range of common .tsv formats are supported by DIA-NN.

[jpfeuffer]

Ok. I could use EasyPQP output for now. This looks like the most clearly specified and seems to be used by both FragPipe and OpenSWATH. I can also write it as TSV for now (although it might be nice to support their new parquet output at some point since you already seem to include parquet libraries).

Their specification is:

        let fields = vec![
            Field::new("PrecursorMz", DataType::Float64, false),
            Field::new("ProductMz", DataType::Float64, false),
            Field::new("PrecursorCharge", DataType::Int32, false),
            Field::new("ProductCharge", DataType::Int32, false),
            Field::new("LibraryIntensity", DataType::Float64, false),
            Field::new("NormalizedRetentionTime", DataType::Float64, false),
            Field::new("PeptideSequence", DataType::Utf8, true),
            Field::new("ModifiedPeptideSequence", DataType::Utf8, true),
            Field::new("ProteinId", DataType::Utf8, true),
            Field::new("UniprotId", DataType::Utf8, true),
            Field::new("GeneName", DataType::Utf8, true),
            Field::new("FragmentType", DataType::Utf8, true),
            Field::new("FragmentSeriesNumber", DataType::Int32, false),
            Field::new("Annotation", DataType::Utf8, true),
            Field::new("PrecursorIonMobility", DataType::Float64, false),
            Field::new("TransitionGroupId", DataType::Int64, false),
            Field::new("TransitionId", DataType::Int64, false),
            Field::new("Decoy", DataType::Int32, false),
            Field::new("DetectingTransition", DataType::Int32, false),
            Field::new("IdentifyingTransition", DataType::Int32, false),
            Field::new("QuantifyingTransition", DataType::Int32, false),
        ];

Does that sound about right?

[vdemichev]

Whatever is FragPipe format is read by DIA-NN - FragPipe team adapted it. What's the use case? I mean why not just use DIA-NN itself for library creation?

[jpfeuffer]

Ah it's just for benchmarking different/own library prediction algorithms.