Make a comprehensive behaviour of hist_hooks task #81
Description
Currently when running on a single dataset (data or mc) the plotting task fails because it tries to subtract empty list of hists (in case of running on data) from the empty list (in case of running on mc). The error is not quite comprehensive.
=============================== entering sandbox ===============================
task : cf.PlotVariables1D_httcp_config_ana_0__afa2ba9bd8
sandbox: bash::$CF_BASE/sandboxes/venv_columnar_dev.sh
================================================================================
INFO: luigi-interface - Done scheduling tasks
INFO: luigi-interface - Running Worker with 1 processes
INFO: luigi-interface - [pid 1525591] Worker Worker(salt=210562245, workers=1, host=lxplus972.cern.ch, username=mwitt, pid=1525269) running cf.PlotVariables1D(effective_workflow=local, branch=0, analysis=httcp.config.analysis_httcp.analysis_httcp, version=ff_exp_and_p2_tau_eta2p3, config=run3_2022_preEE_mutau_limited, shift=nominal, local_shift=nominal, calibrators=main, selector=main, producers=main, categories=cat_mutau_sr, variables=mutau_lep1_pt,N_jets_pT_20_eta_4_7_Tight,mutau_mvis,mutau_delta_r,mutau_lep0_iso,mutau_lep0_eta,mutau_lep0_phi,mutau_l
ep0_ip_sig,mutau_lep0_pt,mutau_lep1_eta,mutau_lep1_phi,mutau_lep1_mass,mutau_lep1_decayModePNet,mutau_lep1_decayMode,mutau_mt,mutau_pt,puppi_met_pt,puppi_met_phi,phi_cp_incl,phi_cp_mu_pi,phi_cp_mu_rho, datasets=wj_incl_madgraph, processes=wj, weight_producer=main, hist_hooks=good_old_abcd, file_types=pdf,png, plot_suffix=NO_STR)
plotting N_jets_pT_20_eta_4_7_Tight in cat_mutau_sr
invoking hist hook 'good_old_abcd'
failed (took 0.43 seconds)
ERROR: luigi-interface - [pid 1525591] Worker Worker(salt=210562245, workers=1, host=lxplus972.cern.ch, username=mwitt, pid=1525269) failed cf.PlotVariables1D(effective_workflow=local, branch=0, analysis=httcp.config.analysis_httcp.analysis_httcp, version=ff_exp_and_p2_tau_eta2p3, config=run3_2022_preEE_mutau_limited, shift=nominal, local_shift=nominal, calibrators=main, selector=main, producers=main, categories=cat_mutau_sr, variables=mutau_lep1_pt,N_jets_pT_20_eta_4_7_Tight,mutau_mvis,mutau_delta_r,mutau_lep0_iso,mutau_lep0_eta,mutau_lep0_phi,mutau_
lep0_ip_sig,mutau_lep0_pt,mutau_lep1_eta,mutau_lep1_phi,mutau_lep1_mass,mutau_lep1_decayModePNet,mutau_lep1_decayMode,mutau_mt,mutau_pt,puppi_met_pt,puppi_met_phi,phi_cp_incl,phi_cp_mu_pi,phi_cp_mu_rho, datasets=wj_incl_madgraph, processes=wj, weight_producer=main, hist_hooks=good_old_abcd, file_types=pdf,png, plot_suffix=NO_STR)
Traceback (most recent call last):
File "/eos/user/m/mwitt/CPinHToTauTauData/venvs/venv_columnar_dev_3cbb5aff/lib/python3.9/site-packages/luigi/worker.py", line 210, in run
new_deps = self._run_get_new_deps()
File "/eos/user/m/mwitt/CPinHToTauTauData/venvs/venv_columnar_dev_3cbb5aff/lib/python3.9/site-packages/luigi/worker.py", line 138, in _run_get_new_deps
task_gen = self.task.run()
File "/afs/cern.ch/user/m/mwitt/public/CPinHToTauTau/modules/columnflow/modules/law/law/decorator.py", line 231, in wrapper
result = call(state)
File "/afs/cern.ch/user/m/mwitt/public/CPinHToTauTau/modules/columnflow/modules/law/law/decorator.py", line 388, in call
return fn(task, *args, **kwargs)
File "/afs/cern.ch/user/m/mwitt/public/CPinHToTauTau/modules/columnflow/modules/law/law/decorator.py", line 244, in wrapper
return decorator(fn, _opts, *args, **kwargs)
File "/afs/cern.ch/user/m/mwitt/public/CPinHToTauTau/modules/columnflow/modules/law/law/decorator.py", line 272, in log
return fn(task, *args, **kwargs)
File "/afs/cern.ch/user/m/mwitt/public/CPinHToTauTau/modules/columnflow/modules/law/law/decorator.py", line 231, in wrapper
result = call(state)
File "/afs/cern.ch/user/m/mwitt/public/CPinHToTauTau/modules/columnflow/columnflow/tasks/framework/decorators.py", line 46, in call
return fn(task, *args, **kwargs)
File "/afs/cern.ch/user/m/mwitt/public/CPinHToTauTau/modules/columnflow/columnflow/tasks/plotting.py", line 154, in run
hists = self.invoke_hist_hooks(hists,category_inst)
File "/afs/cern.ch/user/m/mwitt/public/CPinHToTauTau/modules/columnflow/columnflow/tasks/framework/mixins.py", line 2473, in invoke_hist_hooks
hists = func(self, hists, category_inst)
File "/afs/cern.ch/user/m/mwitt/public/CPinHToTauTau/data_driven/hist_hooks.py", line 98, in qcd_estimation
data_num, mc_num = get_hists_from_reg(config, hists,sr.aux['abcd_regs']['dr_num'])
File "/afs/cern.ch/user/m/mwitt/public/CPinHToTauTau/data_driven/hist_hooks.py", line 93, in get_hists_from_reg
data_hist = sum(data_hists[1:], data_hists[0].copy())
IndexError: list index out of range
INFO: luigi-interface - This progress looks :( because there were failed tasks
=============================== leaving sandbox ================================
task : cf.PlotVariables1D_httcp_config_ana_0__afa2ba9bd8
sandbox: bash::$CF_BASE/sandboxes/venv_columnar_dev.sh
================================================================================
We need to add either a check of the number of histograms, or throw an exception when running on a single dataset.