Add tests with split 0 input files to raw data unit tests

Author: wddgit

DataFormats/Common/test/TestTriggerResultsFormat.sh

@@ -10,7 +10,25 @@ file=testTriggerResults.root
 
 cmsRun ${LOCAL_TEST_DIR}/test_readTriggerResults_cfg.py "$file" || die "Failure using test_readTriggerResults_cfg.py $file" $?
 
-oldFiles="testTriggerResults_CMSSW_13_0_0.root  testTriggerResults_CMSSW_13_1_0_pre3.root"
+# The old files read below were generated as follows.
+#
+#     Check out the release indicated in the filename. Then cherry pick
+#     the commit that adds the original version of the file
+#     DataFormats/Common/test/TestWriteTriggerResults.cc.
+#     Except for BuildFile.xml, this only adds new test files.
+#     There may be minor conflicts or issues in test/BuildFile.xml
+#     that need to be resolved.
+#
+# Run cmsRun with DataFormats/Common/test/create_triggerresults_test_file_cfg.py
+# as the configuration and rename the file that creates.
+
+# Note that TriggerResults is declared in the classes_def.xml file with
+# a requirement that the product is always written with split level 0.
+# Most other raw data products are written with the default split level
+# for the output file. That is why all the test input files in this
+# shell script were written with split level 0.
+
+oldFiles="testTriggerResults_CMSSW_13_0_0_split_0.root  testTriggerResults_CMSSW_13_1_0_pre3_split_0.root"
 for file in $oldFiles; do
   inputfile=$(edmFileInPath DataFormats/Common/data/$file) || die "Failure edmFileInPath DataFormats/Common/data/$file" $?
   cmsRun ${LOCAL_TEST_DIR}/test_readTriggerResults_cfg.py "$inputfile" || die "Failed to read old file $file" $?

DataFormats/DetId/README.md

@@ -7,4 +7,4 @@ The type `std::vector<DetId>` is part of the RAW data, and any changes must be b
 * a file written by the same release
 * files written by (some) earlier releases
 
-If the persistent format of class `std::vector<DetId>` gets changed in the future, please adjust the `TestReadVectorDetId` and `TestWriteVectorDetId` modules accordingly. It is important that every member container has some content in this test. Please also add a new file to the [https://github.com/cms-data/DataFormats-DetId/](https://github.com/cms-data/DataFormats-DetId/) repository, and update the `TestVectorDetId` unit test to read the newly created file. The file name should contain the release or pre-release with which it was written.
+If the persistent format of class `std::vector<DetId>` gets changed in the future, please adjust the `TestReadVectorDetId` and `TestWriteVectorDetId` modules accordingly. It is important that every member container has some content in this test. Please also add new files to the [https://github.com/cms-data/DataFormats-DetId/](https://github.com/cms-data/DataFormats-DetId/) repository, and update the `TestVectorDetId` unit test to read the newly created files. There should be one file written with split level 0 and one with split level 99. The file name should contain the release or pre-release with which it was written and the split level.

DataFormats/DetId/test/TestVectorDetId.sh

@@ -10,7 +10,25 @@ file=testVectorDetId.root
 
 cmsRun ${LOCAL_TEST_DIR}/test_readVectorDetId_cfg.py "$file" || die "Failure using test_readVectorDetId_cfg.py $file" $?
 
-oldFiles="testVectorDetId_CMSSW_13_2_4.root"
+# The old files read below were generated as follows.
+#
+#     Check out the 13_2_4 release and cherry pick the commit that
+#     adds the original version of the file
+#     DataFormats/DetId/test/TestWriteVectorDetId.cc.
+#     Except for BuildFile.xml this only adds new test files.
+#     There may be minor conflicts or issues in test/BuildFile.xml
+#     that need to be resolved.
+#
+# Run cmsRun with DataFormats/DetId/test/create_VectorDetId_test_file_cfg.py
+# as the configuration and rename the file that creates.
+#
+# By default, split level 99 is used (maximum possible splitting).
+# If the suffix "_split_0" is near the end of the filename, the
+# following was added to the configuration of the output module:
+#     "splitLevel = cms.untracked.int32(0)"
+#
+
+oldFiles="testVectorDetId_CMSSW_13_2_4_split_99.root testVectorDetId_CMSSW_13_2_4_split_0.root"
 for file in $oldFiles; do
   inputfile=$(edmFileInPath DataFormats/DetId/data/$file) || die "Failure edmFileInPath DataFormats/DetId/data/$file" $?
   cmsRun ${LOCAL_TEST_DIR}/test_readVectorDetId_cfg.py "$inputfile" || die "Failed to read old file $file" $?

DataFormats/FEDRawData/test/TestFEDRawDataCollectionFormat.sh

@@ -10,7 +10,25 @@ file=testFEDRawDataCollection.root
 
 cmsRun ${LOCAL_TEST_DIR}/test_readFEDRawDataCollection_cfg.py "$file" || die "Failure using test_readFEDRawDataCollection_cfg.py $file" $?
 
-oldFiles="testFEDRawDataCollection_CMSSW_13_0_0.root testFEDRawDataCollection_CMSSW_13_1_0_pre3.root"
+# The old files read below were generated as follows.
+#
+#     Check out the release indicated in the filename. Then cherry pick
+#     the commit that adds the original version of the file
+#     DataFormats/FEDRawData/test/TestWriteFEDRawDataCollection.cc.
+#     Except for BuildFile.xml, this only adds new test files.
+#     There may be minor conflicts or issues in test/BuildFile.xml
+#     that need to be resolved.
+#
+# Run cmsRun with DataFormats/FEDRawData/test/create_FEDRawDataCollection_test_file_cfg.py
+# as the configuration and rename the file that creates.
+
+# Note that FEDRawDataCollection is declared in the classes_def.xml file with
+# a requirement that the product is always written with split level 0.
+# Most other raw data products are written with the default split level
+# for the output file. That is why all the test input files in this
+# shell script were written with split level 0.
+
+oldFiles="testFEDRawDataCollection_CMSSW_13_0_0_split_0.root testFEDRawDataCollection_CMSSW_13_1_0_pre3_split_0.root"
 for file in $oldFiles; do
   inputfile=$(edmFileInPath DataFormats/FEDRawData/data/$file) || die "Failure edmFileInPath DataFormats/FEDRawData/data/$file" $?
   cmsRun ${LOCAL_TEST_DIR}/test_readFEDRawDataCollection_cfg.py "$inputfile" || die "Failed to read old file $file" $?

DataFormats/HLTReco/test/TestTriggerEventFormat.sh

@@ -10,7 +10,25 @@ file=testTriggerEvent.root
 
 cmsRun ${LOCAL_TEST_DIR}/test_readTriggerEvent_cfg.py "$file" || die "Failure using test_readTriggerEvent_cfg.py $file" $?
 
-oldFiles="testTriggerEvent_CMSSW_13_0_0.root testTriggerEvent_CMSSW_13_1_0_pre3.root"
+# The old files read below were generated as follows.
+#
+#     Check out the release indicated in the filename. Then cherry pick
+#     the commit that adds the original version of the file
+#     DataFormats/HLTReco/test/TestWriteTriggerEvent.cc.
+#     Except for BuildFile.xml, this only adds new test files.
+#     There may be minor conflicts or issues in test/BuildFile.xml
+#     that need to be resolved.
+#
+# Run cmsRun with DataFormats/HLTReco/test/create_TriggerEvent_test_file_cfg.py
+# as the configuration and rename the file that creates.
+
+# Note that TriggerEvent is declared in the classes_def.xml file with
+# a requirement that the product is always written with split level 0.
+# Most other raw data products are written with the default split level
+# for the output file. That is why all the test input files in this
+# shell script were written with split level 0.
+
+oldFiles="testTriggerEvent_CMSSW_13_0_0_split_0.root testTriggerEvent_CMSSW_13_1_0_pre3_split_0.root"
 for file in $oldFiles; do
   inputfile=$(edmFileInPath DataFormats/HLTReco/data/$file) || die "Failure edmFileInPath DataFormats/HLTReco/data/$file" $?
   cmsRun ${LOCAL_TEST_DIR}/test_readTriggerEvent_cfg.py "$inputfile" || die "Failed to read old file $file" $?

DataFormats/L1Scouting/README.md

@@ -6,3 +6,6 @@ Any changes to the L1 scouting data formats must be backwards compatible.
 In order to ensure the L1 Scouting formats can be read by future CMSSW releases,
 there is a `TestWriteL1ScoutingDataFormats` unit test, which makes use of the `TestReadL1Scouting` analyzer and the `TestWriteL1Scouting` producer.
 The unit test checks that objects can be written and read properly. 
+If any of the data formats is changed, then 2 new input files
+for the test should be created, one for split level 0 and one for
+split level 99.

DataFormats/L1Scouting/test/TestL1ScoutingFormat.sh

@@ -10,14 +10,37 @@ file=testL1Scouting.root
 
 cmsRun ${LOCAL_TEST_DIR}/read_L1Scouting_cfg.py --inputFile "$file" || die "Failure using read_L1Scouting_cfg.py $file" $?
 
+# The old files read below were generated as follows.
+#
+# Check out the release in the filename and use it without modification to make
+# files with split level 99 (maximum possible splitting for each product)
+#
+# Then execute:
+#
+#   cmsRun DataFormats/L1Scouting/test/create_L1Scouting_test_file_cfg.py
+#
+# Rename the output file.
+#
+# The versions of the classes are encoded in the filenames in
+# alphabetical order. This order is also the order the classes
+# appear in classes_def.xml.
+#
+# For the split level 0 files, do the exact same thing except
+# add the following to the output module configuration.
+#     "splitLevel = cms.untracked.int32(0)"
+
 # test file for muon, jet, e/gamma and energy sums data formats
-oldFile="testL1Scouting_v3_v3_v3_v3_v3_13_3_0_pre5.root"
-inputfile=$(edmFileInPath DataFormats/L1Scouting/data/$oldFile) || die "Failure edmFileInPath DataFormats/L1Scouting/data/$oldFile" $?
-cmsRun ${LOCAL_TEST_DIR}/read_L1Scouting_cfg.py --inputFile "$inputfile" --bmtfStubVersion 0 || die "Failed to read old file $oldFile" $?
+oldFiles="testL1Scouting_v3_v3_v3_v3_v3_14_0_0_split_99.root testL1Scouting_v3_v3_v3_v3_v3_14_0_0_split_0.root"
+for file in $oldFiles; do
+  inputfile=$(edmFileInPath DataFormats/L1Scouting/data/$file) || die "Failure edmFileInPath DataFormats/L1Scouting/data/$file" $?
+  cmsRun ${LOCAL_TEST_DIR}/read_L1Scouting_cfg.py --inputFile "$inputfile" --bmtfStubVersion 0 || die "Failed to read old file $file" $?
+done
 
 # added BMTF input stubs data format
-oldFile="testL1Scouting_v3_v3_v3_v3_v3_v3_14_1_0_pre4.root"
-inputfile=$(edmFileInPath DataFormats/L1Scouting/data/$oldFile) || die "Failure edmFileInPath DataFormats/L1Scouting/data/$oldFile" $?
-cmsRun ${LOCAL_TEST_DIR}/read_L1Scouting_cfg.py --inputFile "$inputfile" --bmtfStubVersion 3 || die "Failed to read old file $oldFile" $?
+oldFiles="testL1Scouting_v3_v3_v3_v3_v3_v3_14_1_0_pre5_split_99.root testL1Scouting_v3_v3_v3_v3_v3_v3_14_1_0_pre5_split_0.root"
+for file in $oldFiles; do
+  inputfile=$(edmFileInPath DataFormats/L1Scouting/data/$file) || die "Failure edmFileInPath DataFormats/L1Scouting/data/$file" $?
+  cmsRun ${LOCAL_TEST_DIR}/read_L1Scouting_cfg.py --inputFile "$inputfile" --bmtfStubVersion 3 || die "Failed to read old file $file" $?
+done
 
 exit 0

DataFormats/L1ScoutingRawData/test/TestSDSRawDataCollectionFormat.sh

@@ -10,7 +10,22 @@ file=testSDSRawDataCollection.root
 
 cmsRun ${LOCAL_TEST_DIR}/read_SDSRawDataCollection_cfg.py "$file" || die "Failure using read_SDSRawDataCollection_cfg.py $file" $?
 
-oldFile="testSDSRawDataCollection_v3_CMSSW_13_3_0_pre5.root"
+# The old files read below were generated as follows.
+#
+# Check out the release in the filename and use it without modification.
+# Then execute:
+#
+#   cmsRun DataFormats/L1ScoutingRawData/test/create_SDSRawDataCollection_test_file_cfg.py
+#
+# Rename the output file. The versions of the class is encoded in the filename.
+#
+# Note that SDSRawDataCollection is declared in the classes_def.xml file with
+# a requirement that the product is always written with split level 0.
+# Most other raw data products are written with the default split level
+# for the output file. That is why all the test input files in this
+# shell script were written with split level 0.
+
+oldFile="testSDSRawDataCollection_v3_CMSSW_14_0_0_split_0.root"
 inputfile=$(edmFileInPath DataFormats/L1ScoutingRawData/data/$oldFile) || die "Failure edmFileInPath DataFormats/L1ScoutingRawData/data/$oldFile" $?
 cmsRun ${LOCAL_TEST_DIR}/read_SDSRawDataCollection_cfg.py "$inputfile" || die "Failed to read old file $oldFile" $?
 

DataFormats/L1TGlobal/README.md

@@ -7,4 +7,4 @@ The class `GlobalObjectMapRecord` is part of the RAW data, and any changes must
 * a file written by the same release
 * files written by (some) earlier releases
 
-If the persistent format of class `GlobalObjectMapRecord` gets changed in the future, please adjust the `TestReadGlobalObjectMapRecord` and `TestWriteGlobalObjectMapRecord` modules accordingly. It is important that every member container has some content in this test. Please also add a new file to the [https://github.com/cms-data/DataFormats-L1TGlobal/](https://github.com/cms-data/DataFormats-L1TGlobal/) repository, and update the `TestGlobalObjectMapRecord` unit test to read the newly created file. The file name should contain the release or pre-release with which it was written.
+If the persistent format of class `GlobalObjectMapRecord` gets changed in the future, please adjust the `TestReadGlobalObjectMapRecord` and `TestWriteGlobalObjectMapRecord` modules accordingly. It is important that every member container has some content in this test. Please also add new files to the [https://github.com/cms-data/DataFormats-L1TGlobal/](https://github.com/cms-data/DataFormats-L1TGlobal/) repository, and update the `TestGlobalObjectMapRecord` unit test to read the newly created files. There should be one file written with split level 0 and one with split level 99. The file name should contain the release or pre-release with which it was written and the split level.

DataFormats/L1TGlobal/test/TestGlobalObjectMapRecordFormat.sh

@@ -10,7 +10,22 @@ file=testGlobalObjectMapRecord.root
 
 cmsRun ${LOCAL_TEST_DIR}/test_readGlobalObjectMapRecord_cfg.py "$file" || die "Failure using test_readGlobalObjectMapRecord_cfg.py $file" $?
 
-oldFiles="testGlobalObjectMapRecord_CMSSW_13_0_0.root testGlobalObjectMapRecord_CMSSW_13_1_0_pre3.root"
+# The old files read below were generated as follows.
+#
+#     Check out the release in the filename and cherry pick the commit that
+#     adds the original version of the file:
+#     DataFormats/L1TGlobal/test/TestWriteGlobalObjectMapRecord.cc
+#
+# Run cmsRun with DataFormats/L1TGlobal/test/create_GlobalObjectMapRecord_test_file_cfg.py
+# as the configuration and rename the file that creates.
+#
+# By default, split level 99 is used (maximum possible splitting).
+# If the suffix "_split_0" is near the end of the filename, the
+# following was added to the configuration of the output module:
+#     "splitLevel = cms.untracked.int32(0)"
+
+oldFiles="testGlobalObjectMapRecord_CMSSW_13_0_0_split_99.root testGlobalObjectMapRecord_CMSSW_13_0_0_split_0.root"
+oldFiles+=" testGlobalObjectMapRecord_CMSSW_13_1_0_pre3_split_99.root testGlobalObjectMapRecord_CMSSW_13_1_0_pre3_split_0.root"
 for file in $oldFiles; do
   inputfile=$(edmFileInPath DataFormats/L1TGlobal/data/$file) || die "Failure edmFileInPath DataFormats/L1TGlobal/data/$file" $?
   cmsRun ${LOCAL_TEST_DIR}/test_readGlobalObjectMapRecord_cfg.py "$inputfile" || die "Failed to read old file $file" $?