uzhcms / exovvntuplizerrunii

Ntuplizer for VV->lvj/jj

# EXOVVNtuplizerRunII

Ntuplizer for searches for heavy resonances decaying to dibosons

installation instructions

Setting up CMSSW (for september reprocessing):

cmsrel CMSSW_9_4_9
cd CMSSW_9_4_9/src
cmsenv
git cms-init

getting the ntuplizer codes

cd $CMSSW_BASE/src
export GITUSER=`git config user.github`
git clone https://github.com/${GITUSER}/EXOVVNtuplizerRunII 
cd EXOVVNtuplizerRunII
git remote add UZHCMS https://github.com/UZHCMS/EXOVVNtuplizerRunII
git fetch UZHCMS
git checkout -b DevelopmentBranch_9_4_0 UZHCMS/94X_ntuplizer
cd $CMSSW_BASE/src
scram b -j 8

update the cut-based electron ID and MVAID to V2. HEEP are included by default in 94X

(https://twiki.cern.ch/twiki/bin/view/CMS/CutBasedElectronIdentificationRun2)

cd $CMSSW_BASE/src
git cms-merge-topic guitargeek:EgammaID_9_4_X
cd $CMSSW_BASE/src
scram b -j 8

running for data and MC

Just set the proper flag in python/ntuplizerOptions_generic_cfi.py

cmsRun config_generic.py 

to recluster jets and MET, or to add the Higgs-tagger the following flags can be changed:

config["DOAK8RECLUSTERING"] = False
config["DOHBBTAG"] = False
config["DOAK8PRUNEDRECLUSTERING"] = False
config["DOMETRECLUSTERING"] = False

If you want to use Higgs tagger the first two flags must all be set to True.

Batch submission

Config file creation

Config file creation can be done via the createConfig.py script. It requires a text file with a list of input data sets, see e.g. samples/QCD_HT_RunIISpring15MiniAODv2.txt. To run:

python tools/createConfig.py samples/QCD_HT_RunIISpring15MiniAODv2.txt

When running over data, this requires the -d flag. The script will automatically determine if the data sets are available on the T3 storage element. Also, --help will provide more information (e.g. allows changing the default number of jobs per event). If you run the script from a different directory, you need to provide the location of the template file.

Job submission

Submit your jobs using the submitJobsOnT3batch.py script with the generated config files like this:

python submitJobsOnT3batch.py -C myconfig.cfg

Once the jobs are done, they can be checked for completeness like this:

python submitJobsOnT3batch.py -C myconfig.cfg --check

Resubmit jobs like this:

python submitJobsOnT3batch.py -C myconfig.cfg --resubmit 1,4,7

And eventually copied to the SE (path given in the config file):

python submitJobsOnT3batch.py -C myconfig.cfg --copy

Finally, note that, when you run on crab, you have to enable

config.JobType.sendExternalFolder = True

as described at https://twiki.cern.ch/twiki/bin/viewauth/CMS/MultivariateElectronIdentificationRun2#Recipes_and_implementation

exovvntuplizerrunii's Issues

use correct primary vertex

instead of using the first vertex (index 0), should use the first that passes quality cuts, see https://github.com/ikrav/EgammaWork/blob/ntupler_and_VID_demos/ElectronNtupler/plugins/ElectronNtuplerVIDDemo.cc#L271
This is in particular needed in the ElectronNtuplizer, but probably also in other places.

Isolation variables for the leptons redundancy

Dear All,
I'm editing the Ntuplizer code to see if it can be skimmed a bit. I found that for the lepton isolation variable there is a lot of redundancy. For instance the variables used in the isolation computation are saved using the values accessed directly from the particles and the ones in the userfloat of the particles (for example mu.photonIso mu.userfloat("photonIso") ) and they should be the same, to my knowledge.
Can we delete one of the 2?
Then the same thing applies for different kind of isolations. I think the most used are the lep_SemileptonicPFIso and lep_SemileptonicCorrPFIso from my side and maybe the mu_trackIso for the muons.
the SemileptonicPFIso means for electrons this means iso= RhoCorrectedIso03 and for the muons iso = (sumChargedHadronPt + std::max( 0. ,sumNeutralHadronEt + sumPhotonEt - 0.5 * sumPUPt)
I think we could remove these other ones if people don't use them :
mu_pfRhoCorrRelIso03
mu_pfRhoCorrRelIso03Boost
mu_pfRhoCorrRelIso04
mu_pfRhoCorrRelIso04Boost
mu_pfDeltaCorrRelIso
mu_pfDeltaCorrRelIsoBoost
mu_pfRelIso
mu_pfRelIsoBoost
Please let me know if I can delete them or you need some of this variables

Missing 2017 triggers

There are some triggers missing needed for tautau analyses:

HLT_IsoMu20_eta2p1_LooseChargedIsoPFTau27_eta2p1_CrossL1_v # HLT_IsoMu20_eta2p1 is here
HLT_Ele35_WPTight_Gsf_v
HLT_Ele24_eta2p1_WPTight_Gsf_LooseChargedIsoPFTau30_eta2p1_CrossL1_v
HLT_Mu12_TrkIsoVVL_Ele23_CaloIdL_TrackIdL_IsoVL_DZ_v

See https://twiki.cern.ch/twiki/bin/viewauth/CMS/SMTauTau2017#Final_selection_for_the_categori

EXOVVNtuplizerRunII/Ntuplizer/plugins/TriggersNtuplizer.cc

Lines 95 to 186 in 0a5ffbe

	trigName.find("HLT_IsoMu20_eta2p1") != std::string::npos ||
	trigName.find("HLT_IsoMu24_eta2p1") != std::string::npos ||
	trigName.find("HLT_IsoTkMu24_eta2p1") != std::string::npos ||
	trigName.find("HLT_IsoMu24_v") != std::string::npos ||
	trigName.find("HLT_IsoTkMu24_v") != std::string::npos ||
	trigName.find("HLT_IsoMu27_v") != std::string::npos ||
	trigName.find("HLT_IsoTkMu27_v") != std::string::npos ||
	trigName.find("HLT_IsoMu27_eta2p1") != std::string::npos ||
	trigName.find("HLT_IsoTkMu27_eta2p1") != std::string::npos ||
	trigName.find("HLT_Mu45_eta2p1") != std::string::npos ||
	//trigName.find("HLT_Mu50_eta2p1") != std::string::npos ||
	trigName.find("HLT_Mu50_v") != std::string::npos ||
	trigName.find("HLT_TkMu50_v") != std::string::npos ||
	trigName.find("HLT_Ele27_WPTight_Gsf") != std::string::npos ||
	trigName.find("HLT_Ele27_WPLoose_Gsf") != std::string::npos ||
	trigName.find("HLT_Ele27_WPLoose_Gsf_WHbbBoost") != std::string::npos ||
	trigName.find("HLT_Ele27_eta2p1_WPLoose") != std::string::npos ||
	trigName.find("HLT_Ele27_eta2p1_WP75_Gsf") != std::string::npos ||
	trigName.find("HLT_Ele23_CaloIdL_TrackIdL_IsoVL") != std::string::npos ||
	trigName.find("HLT_Ele32_eta2p1_WP75_Gsf") != std::string::npos ||
	trigName.find("HLT_Ele45_WPLoose_Gsf_v") != std::string::npos ||
	trigName.find("HLT_Ele105_CaloIdVT_GsfTrkIdT") != std::string::npos ||
	trigName.find("HLT_Ele115_CaloIdVT_GsfTrkIdT") != std::string::npos ||
	trigName.find("HLT_Ele145_CaloIdVT_GsfTrkIdT") != std::string::npos ||
	trigName.find("HLT_Ele50_CaloIdVT_GsfTrkIdT_PFJet165") != std::string::npos ||
	trigName.find("HLT_Ele22_eta2p1_WPLoose_Gsf_LooseIsoPFTau20") != std::string::npos ||
	trigName.find("HLT_Ele27_eta2p1_WPLoose_Gsf_DoubleMediumIsoPFTau35") != std::string::npos ||
	trigName.find("HLT_Ele27_eta2p1_WPLoose_Gsf_LooseIsoPFTau20") != std::string::npos ||
	trigName.find("HLT_IsoMu16_eta2p1_MET30_JetIDCleaned_LooseIsoPFTau50") != std::string::npos ||
	trigName.find("HLT_IsoMu16_eta2p1_MET30_LooseIsoPFTau50_Trk30_eta2p1_v1") != std::string::npos ||
	//H->tautau triggers
	trigName.find("HLT_LooseIsoPFTau50") != std::string::npos||
	trigName.find("HLT_IsoMu17_eta2p1_LooseIsoPFTau20_SingleL1_v") != std::string::npos||
	trigName.find("HLT_IsoMu17_eta2p1_LooseIsoPFTau20_v") != std::string::npos||
	trigName.find("HLT_IsoMu19_eta2p1_LooseIsoPFTau20_SingleL1_v") != std::string::npos||
	trigName.find("HLT_IsoMu19_eta2p1_LooseIsoPFTau20_v") != std::string::npos||
	trigName.find("HLT_IsoMu21_eta2p1_LooseIsoPFTau20_SingleL1_v") != std::string::npos||
	trigName.find("HLT_IsoMu24_eta2p1") != std::string::npos||
	trigName.find("HLT_IsoMu17_eta2p1") != std::string::npos||
	trigName.find("HLT_IsoMu18_v") != std::string::npos||
	trigName.find("HLT_IsoMu22_v") != std::string::npos||
	trigName.find("HLT_IsoMu27_v") != std::string::npos||
	trigName.find("HLT_IsoMu20_v") != std::string::npos||
	trigName.find("HLT_IsoMu22_eta2p1_v") != std::string::npos||
	trigName.find("HLT_IsoMu24_v") != std::string::npos||
	trigName.find("HLT_IsoTkMu18_v") != std::string::npos||
	trigName.find("HLT_IsoTkMu20_v") != std::string::npos||
	trigName.find("HLT_IsoTkMu22_v") != std::string::npos||
	trigName.find("HLT_IsoTkMu22_eta2p1_v") != std::string::npos||
	trigName.find("HLT_IsoTkMu24_v") != std::string::npos||
	trigName.find("HLT_IsoTkMu27_v") != std::string::npos||
	trigName.find("HLT_Ele22_eta2p1_WPLoose_Gsf_LooseIsoPFTau20_SingleL1_v") != std::string::npos||
	trigName.find("HLT_Ele24_eta2p1_WPLoose_Gsf_LooseIsoPFTau20_SingleL1_v") != std::string::npos||
	trigName.find("HLT_Ele24_eta2p1_WPLoose_Gsf_LooseIsoPFTau20_v") != std::string::npos||
	trigName.find("HLT_Ele27_eta2p1_WPLoose_Gsf_LooseIsoPFTau20_SingleL1_v") != std::string::npos||
	trigName.find("HLT_Ele32_eta2p1_WPLoose_Gsf_LooseIsoPFTau20_SingleL1_v") != std::string::npos||
	trigName.find("HLT_Ele22_eta2p1_WP75_Gsf_LooseIsoPFTau20") != std::string::npos||
	trigName.find("HLT_Ele25_eta2p1") != std::string::npos||
	trigName.find("HLT_Ele22_eta2p1_WP75_Gsf_v") != std::string::npos||
	trigName.find("HLT_Ele32_eta2p1_WP75_Gsf") != std::string::npos||
	trigName.find("HLT_Ele23_WPLoose_Gsf") != std::string::npos||
	trigName.find("HLT_Ele23_WPTight_Gsf") != std::string::npos||
	trigName.find("HLT_Ele24_eta2p1_WPLoose_Gsf_v") != std::string::npos||
	trigName.find("HLT_Ele25_WPTight_Gsf_v") != std::string::npos||
	trigName.find("HLT_Ele25_eta2p1_WPLoose_Gsf_v") != std::string::npos||
	trigName.find("HLT_Ele25_eta2p1_WPTight_Gsf_v") != std::string::npos||
	trigName.find("HLT_Ele27_WPLoose_Gsf") != std::string::npos||
	trigName.find("HLT_Ele27_WPTight_Gsf") != std::string::npos||
	trigName.find("HLT_Ele27_eta2p1_WPLoose_Gsf_v") != std::string::npos||
	trigName.find("HLT_Ele27_eta2p1_WPTight_Gsf_v") != std::string::npos||
	trigName.find("HLT_Ele32_eta2p1_WPTight_Gsf_v") != std::string::npos||
	trigName.find("HLT_Ele32_WPTight_Gsf_v") != std::string::npos||
	trigName.find("HLT_Ele45_WPLoose_Gsf_L1JetTauSeeded") != std::string::npos||
	trigName.find("HLT_DoubleMediumIsoPFTau40_Trk1_eta2p1_Reg") != std::string::npos||
	trigName.find("HLT_DoubleMediumIsoPFTau35_Trk1_eta2p1_Reg") != std::string::npos||
	// Double leptons
	trigName.find("HLT_Ele23_Ele12_CaloIdL_TrackIdL_IsoVL_DZ_v") != std::string::npos||
	trigName.find("HLT_DoubleEle24_22_eta2p1_WPLoose_Gsf_v") != std::string::npos||
	trigName.find("HLT_DoubleEle37_Ele27_CaloIdL_GsfTrkIdVL_v") != std::string::npos||
	trigName.find("HLT_Mu17_TrkIsoVVL_TkMu8_TrkIsoVVL_DZ_v") != std::string::npos||
	trigName.find("HLT_Mu17_TrkIsoVVL_Mu8_TrkIsoVVL_DZ_v") != std::string::npos||
	trigName.find("HLT_Mu17_TrkIsoVVL_TkMu8_TrkIsoVVL_v") != std::string::npos||
	trigName.find("HLT_Mu17_TrkIsoVVL_Mu8_TrkIsoVVL_v") != std::string::npos||
	trigName.find("HLT_Mu30_TkMu11_v") != std::string::npos||
	trigName.find("HLT_Mu8_TrkIsoVVL_Ele23_CaloIdL_TrackIdL_IsoVL_v") != std::string::npos||
	trigName.find("HLT_Mu23_TrkIsoVVL_Ele12_CaloIdL_TrackIdL_IsoVL_v") != std::string::npos||
	trigName.find("HLT_Mu8_TrkIsoVVL_Ele17_CaloIdL_TrackIdL_IsoVL_v") != std::string::npos||
	trigName.find("HLT_Mu17_TrkIsoVVL_Ele12_CaloIdL_TrackIdL_IsoVL_v") != std::string::npos||
	trigName.find("HLT_Mu23_TrkIsoVVL_Ele8_CaloIdL_TrackIdL_IsoVL_v") != std::string::npos||
	trigName.find("HLT_Mu8_TrkIsoVVL_Ele23_CaloIdL_TrackIdL_IsoVL_DZ_v") != std::string::npos||
	trigName.find("HLT_Mu23_TrkIsoVVL_Ele12_CaloIdL_TrackIdL_IsoVL_DZ_v") != std::string::npos||

Does anybody use "triggerObject_firedTrigger" and "triggerObject_filterIDs" ?

I'm doing "trimming" of the code. Does anybody use

EXOVVNtuplizerRunII/Ntuplizer/plugins/TriggersNtuplizer.cc

Lines 397 to 398 in 32f4eb7

	nBranches_->triggerObject_firedTrigger.push_back(vfiredTrigger);
	nBranches_->triggerObject_filterIDs.push_back(vfilterIDs);

If yes, let me know soon - If not, I would remove them.

Trigget typo.

PFHT900_V should be PFHT900_v
https://github.com/UZHCMS/EXOVVNtuplizerRunII/blob/master/Ntuplizer/plugins/TriggersNtuplizer.cc#L72

and add trigName.find("AK8DiPFJet280_200_TrimMass30_BTagCSV0p41") != std::string::npos ||

Delete stale branch

As we discussed this week,
I'm going to delete following stale branches.

Please let me know if you have objections.
Otherwise, I will delete them this week.

GenPatch
cgalloni-patch-1
cgalloni-patch-2
revert-124-80X_ntuplizer
clelange-triggerSimplification

You can see all stale branches here
https://github.com/UZHCMS/EXOVVNtuplizerRunII/branches/stale
(I would keep master & 80X related branches)

Test run

I skimmed some information from the trigger ntuplizers.
Together with modifications from Camilla and Alberto,
I think it is very helpful now to run the test job with MC,
and analyze which collection still occupies what amount.

@zucchett Since you did this before, can you do this again ?
(of course, once you get well from the sick !)

Thanks !

check option of job submission script doesn't work for files not stored at T3

default is to use dcap, but should be xrootd for files on remote sites

add all missing jet ID variables to the ntuple

that means all variables that used to calculate quality cuts for jets, can be copied from dijet ntupliser, see https://github.com/CMSDIJET/DijetRootTreeMaker/

add all missing muon ID variables to ntuple

i.e. all variables that are used to calculate whether a certain ID (such as tight) has been passed

Jets

Hi all! @dani is being so kind and going through the JetsNtuplizer.
I had a chat with @cgalloni and the three of us propose the following:
Only keep PUPPI AK8 (these are standard) BUT add CHS tauN + softdrop+pruned mass as userfloats. The names will be such that jetAK8 branches will refer to PUPPI. The specific CHS userfloats will have CHS in their names. tau2 will for instance be PUPPI tau 2. PUPPI softdrop mass will just be jetAK8_softdrop_mass
Only keep AK4 CHS (these are standard). This means no change essentially.

Camilla want to launch a production ASAP and will do so once dani and I make the above changes. Then we will continue cleaning out the code after (removing pruned subjets etc)

git cms-merge-topic ahinzmann:ecf_74

The merge-topic clones many packages that are not needed and it takes long time to compile them. Solution to try:

• git fetch CMSSW_7_4_X
• git cherry-pick ef1fc36223728cba675877035c57d1368f6b05f6

submitJobsOnT3batch.py --check option is useless when using JSON file

The number of events in the ntuple tree will be smaller than the one of the input files if a JSON is used. Script should at least check the output logs for exceptions if running over all files worked.

Potential bug in forward jet selection

Even if i don't use the framework anymore, Deborah spotted a potential issue related to the jet ID in the JetsNtuplizer class:

https://github.com/UZHCMS/EXOVVNtuplizerRunII/blob/10210_ntuplizer_BcMu/Ntuplizer/plugins/JetsNtuplizer.cc#L111

The problem lies that the abs(eta) is computed, and not fabs(eta) as it should, leading to some dangerous approximation in the output. It may also depends on the C++ compiler.

More details from Deborah here:

https://docs.google.com/document/d/1jMulr2AqE856pCZfGL9dI9apOZ00ZxCPtv3OJUk1-U4/edit

You may be interested in fixing it.

dump all trigger decisions dynamically

instead of defining just a few trigger decisions, loop over the available triggers in the dataset and dump them all as individual booleans

overwrite option for submission script

Currently the batch submission does not overwrite files in the user directory.
It would be good to get a warning at submission/resubmission if the output file already exists and ask to delete it if job is sucessful.

genParticle_N vs. actual genParticle vector size

EXOVVNtuplizerRunII/Ntuplizer/plugins/GenParticlesNtuplizer.cc

Line 30 in 47332c4

nBranches_->genParticle_N = genParticles_->size();

We are filtering out unwanted particles, so we should define genParticle_N as the number of particles that pass this filter:
https://github.com/UZHCMS/EXOVVNtuplizerRunII/blob/94X_ntuplizer/Ntuplizer/plugins/GenParticlesNtuplizer.cc#L49

We could e.g. do something like
nBranches_->genParticle_N = nBranches_->genParticle_pt.size
else count with an integer.

add muon trigger objects (for triggers used) in ntuple

need that for trigger matching in selection, objects:

• HLT_IsoMu24_eta2p1
• HLT_Mu45_eta2p1