ConfDB does not properly handle a PSet in a PSet in a VPSet

[Dr15Jones]

A module with the following parameter set

            prod = cms.EDProducer("D",
                                  a_ = cms.VPSet(
                                      cms.PSet(foo = cms.PSet(bar = cms.int32(1)))))

will not be properly parsed by ConfdbLoadParamsFromConfigs141.py. The label foo will be lost. This problem was found as part of the discussion in cms-sw/cmssw#47653.

The underlying issue is with the following lines

hlt-confdb/parser/ConfdbLoadParamsFromConfigs141.py

Lines 1391 to 1392 in b61ec72

	if(psetname.find("[") != -1 and psetname.find("]") != -1):
	parametername = ''

[mmusich]

I am confused by this issue.
If I run the test introduced at #108

$ python3 parser/TestConfdbLoadParamsFromConfigs.py

in the current HEAD of hlt-confdb (commit 08ffdba) I get:

SELECT name,id  FROM u_paramtypes
SELECT u_moduletypes.type, u_moduletypes.id FROM u_moduletypes
SELECT a.id, a.name, a.cvstag, a.id_pkg FROM u_moduletemplates a, u_modt2rele c  WHERE c.id_release=0 AND a.name = 'D' AND c.id_modtemplate=a.id
SELECT * from u_modtelements WHERE id_modtemplate = 1 ORDER BY id 
	Ignoring unknown parameter type _OptionalParameter
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','1','string','b_','0','1','0','value', :PLOB) PLOB= 
.SELECT name,id  FROM u_paramtypes
SELECT u_moduletypes.type, u_moduletypes.id FROM u_moduletypes
SELECT a.id, a.name, a.cvstag, a.id_pkg FROM u_moduletemplates a, u_modt2rele c  WHERE c.id_release=0 AND a.name = 'D' AND c.id_modtemplate=a.id
SELECT * from u_modtelements WHERE id_modtemplate = 1 ORDER BY id 
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','2','PSet','a_','0','1','0','', :PLOB) PLOB= 
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','2','PSet','foo','1','1','0','', :PLOB) PLOB= 
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','1','int32','bar','2','1','0','1', :PLOB) PLOB= 
.SELECT name,id  FROM u_paramtypes
SELECT u_moduletypes.type, u_moduletypes.id FROM u_moduletypes
SELECT a.id, a.name, a.cvstag, a.id_pkg FROM u_moduletemplates a, u_modt2rele c  WHERE c.id_release=0 AND a.name = 'D' AND c.id_modtemplate=a.id
SELECT * from u_modtelements WHERE id_modtemplate = 1 ORDER BY id 
	Ignoring unknown parameter type _RequiredParameter
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','1','string','b_','0','1','0','value', :PLOB) PLOB= 
.SELECT name,id  FROM u_paramtypes
SELECT u_moduletypes.type, u_moduletypes.id FROM u_moduletypes
SELECT a.id, a.name, a.cvstag, a.id_pkg FROM u_moduletemplates a, u_modt2rele c  WHERE c.id_release=0 AND a.name = 'D' AND c.id_modtemplate=a.id
SELECT * from u_modtelements WHERE id_modtemplate = 1 ORDER BY id 
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','3','VPSet','a_','0','1','0','', :PLOB) PLOB= 
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','2','PSet','','1','1','0','', :PLOB) PLOB= 
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','2','PSet','foo','2','1','0','', :PLOB) PLOB= 
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','1','int32','bar','3','1','0','1', :PLOB) PLOB= 
.SELECT name,id  FROM u_paramtypes
SELECT u_moduletypes.type, u_moduletypes.id FROM u_moduletypes
SELECT a.id, a.name, a.cvstag, a.id_pkg FROM u_moduletemplates a, u_modt2rele c  WHERE c.id_release=0 AND a.name = 'D' AND c.id_modtemplate=a.id
SELECT * from u_modtelements WHERE id_modtemplate = 1 ORDER BY id 
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','3','VPSet','a_','0','1','0','', :PLOB) PLOB= 
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','2','PSet','','1','1','0','', :PLOB) PLOB= 
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','1','int32','foo','2','1','0','1', :PLOB) PLOB= 
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','2','PSet','','1','1','1','', :PLOB) PLOB= 
INSERT INTO u_modtelements (id_modtemplate,moetype, paramtype,name, lvl, tracked, ord, value, valuelob)  VALUES ('1','1','int32','foo','2','1','0','2', :PLOB) PLOB= 
.
----------------------------------------------------------------------
Ran 5 tests in 0.004s

OK

I think this issue was already addressed at 2a841b7.

[mmusich]

Indeed after doing git reset --hard HEAD~3 I have a failure. So I think this is issue is solved and can be closed.

[Martin-Grunewald]

This is only a partial fix which will fail if someone puts a PSet with name/label VPSet...

[mmusich]

This is only a partial fix which will fail if someone puts a PSet with name/label VPSet...

so this case is not covered by new unit tests?

[Martin-Grunewald]

VPSet is also used elsewhere triggering the assert true.

[Dr15Jones]

@mmusich I only looked to see if a new pull request had been used to address the issue. I had not looked to see if a commit had gone directly into the working branch (as I'm not accustomed to working that way). Sorry for creating the issue (although it does seem a new issue and an extension to the unit test would be useful to remember the newly discussed case).

[Martin-Grunewald]

The residual failure still occurs with a VPSet parameter with name/label VPSet (as then the blanking out of the parameter name still occurs). The underlying problem is due to the mixed use of using VPSet[#] within ConfDb parsing to denote a vector element (PSet of a VPSet) which does not have a name/label, and a named nested PSet which do need a label/name.

But at least for all other cases it is fixed now, and now RecoTauProducer looks OK.