rsmith54 / factorytools Goto Github PK

We're wasting a lot of space now that we're starting to write out some ints and bools and storing them as doubles. Not sure if there's a clean way to do this, but it might be good to try and think of a way to make this more space-efficient.

Can see an example here:
http://aipanda056.cern.ch/media/filebrowser/7f5157df-8a2c-48fc-b63f-24ed9cc0837e/tarball_PandaJob_2780162004_ANALY_SWT2_CPB/athena_stdout.txt

I don't think it can be something on our end, but maybe we're using some incompatible packages. Anybody else seeing these kinds of issues?

-L

BasicEventSelection::e... INFO    Event number = 1000
BasicEventSelection::e... INFO    Event number = 2000
BasicEventSelection::e... INFO    Event number = 3000
BasicEventSelection::e... INFO    Event number = 4000
BasicEventSelection::e... INFO    Event number = 5000
BasicEventSelection::e... INFO    Event number = 6000
BasicEventSelection::e... INFO    Event number = 7000
BasicEventSelection::e... INFO    Event number = 8000
BasicEventSelection::e... INFO    Event number = 9000
BasicEventSelection::e... INFO    Event number = 10000
BasicEventSelection::e... INFO    Event number = 11000
BasicEventSelection::e... INFO    Event number = 12000
BasicEventSelection::e... INFO    Event number = 13000
BasicEventSelection::e... INFO    Event number = 14000
ORToolBjet_MuJetORT       ERROR   /build2/atnight/localbuilds/nightlies/AnalysisSUSY-2.3.X/AnalysisSUSY/rel_nightly/AssociationUtils/Root/MuJetOverlapTool.cxx:137 (virtual StatusCode ORUtils::MuJetOverlapTool::findOverlaps(const MuonContainer&, const JetContainer&) const): Failed to call "vtx != nullptr"
ORToolBjet_MuJetORT       ERROR   /build2/atnight/localbuilds/nightlies/AnalysisSUSY-2.3.X/AnalysisSUSY/rel_nightly/AssociationUtils/Root/MuJetOverlapTool.cxx:118 (virtual StatusCode ORUtils::MuJetOverlapTool::findOverlaps(const IParticleContainer&, const IParticleContainer&) const): Failed to call "findOverlaps(static_cast<const xAOD::MuonContainer&>(cont1), static_cast<const xAOD::JetContainer&>(cont2))"
ORToolBjet                ERROR   /build2/atnight/localbuilds/nightlies/AnalysisSUSY-2.3.X/AnalysisSUSY/rel_nightly/AssociationUtils/Root/OverlapRemovalTool.cxx:139 (StatusCode ORUtils::OverlapRemovalTool::removeOverlap(const ToolHandle<ORUtils::IOverlapTool>&, const IParticleContainer*, const IParticleContainer*) const): Failed to call "tool->findOverlaps(*cont1, *cont2)"
ORToolBjet                ERROR   /build2/atnight/localbuilds/nightlies/AnalysisSUSY-2.3.X/AnalysisSUSY/rel_nightly/AssociationUtils/Root/OverlapRemovalTool.cxx:116 (virtual StatusCode ORUtils::OverlapRemovalTool::removeOverlaps(const ElectronContainer*, const MuonContainer*, const JetContainer*, const TauJetContainer*, const PhotonContainer*, const JetContainer*) const): Failed to call "removeOverlap(m_muJetORT, muons, jets)"
SUSYObjDef_xAOD           ERROR   /build2/atnight/localbuilds/nightlies/AnalysisSUSY-2.3.X/AnalysisSUSY/rel_nightly/SUSYTools/Root/SUSYObjDef_xAOD.cxx:2201 (virtual StatusCode ST::SUSYObjDef_xAOD::OverlapRemoval(const ElectronContainer*, const MuonContainer*, const JetContainer*, const PhotonContainer*, const TauJetContainer*)): Failed to call "m_orTool->removeOverlaps(electrons, muons, jets, taujet, gamma)"
virtual EL::StatusCode... ERROR   Failed to execute: "m_objTool->OverlapRemoval(electrons_nominal, muons_nominal, jets_nominal )"
EventLoop:/build2/atnight/localbuilds/nightlies/AnalysisSUSY-2.3.X/AnalysisSUSY/rel_nightly/EventLoop/Root/Worker.cxx:46:exception: Algorithm::execute returned StatusCode::FAILURE
Caught exception while executing algorithm:
EventLoop:/build2/atnight/localbuilds/nightlies/AnalysisSUSY-2.3.X/AnalysisSUSY/rel_nightly/EventLoop/Root/Worker.cxx:46:exception: Algorithm::execute returned StatusCode::FAILURE
xAOD::TFileAccessTracer   INFO    Sending file access statistics to http://rucio-lb-prod.cern.ch:18762/traces/
Finished executing root
Fri Mar  4 16:42:43 CST 2016

Just to remind myself we need to think about how to do systematics.

I think it might be easiest to set this up from the run script and set the names of the output trees based on that.

As of UCATLAS/xAODAnaHelpers@7e4d2b6

the metadata and cutflow histogram output streams are configurable. So with this, we can put those histograms into the same stream. This will be a crap-ton easier when downloading datasets from the grid as we'll only need to grab one container instead of three. Let's do that.

Specify release of xAODAnaHelpers in readme

Following the instructions again, this time with the rcSetup Base,2.3.45 version.

firstly, it tells me to change the root version to 6.04, which i do through the provided source
I then

rc find_packages
rc compile

and I get the following error at the end,

At global scope:
cc1plus: warning: unrecognized command line option "-Wno-tautological-undefined-compare"
make: *** [/afs/cern.ch/work/j/jaoliver/private/RJigsawTools/RootCoreBin/obj/x86_64-slc6-gcc49-opt/xAODAnaHelpers/obj/BasicEventSelection.o] Error 1
make: *** Waiting for unfinished jobs....
RootCore: Error failed to compile package xAODAnaHelpers

I had this error last week during the non-stable release, is it possible its been re-introduced?

Cheers,

jason

We should make sure any code which can fail uses the STRONG_CHECK to avoid missing errors.

When running on the grid any .ds file i provide gets a ".root" appended to the end. I've provided an the output below. I looked to where the error was but couldn't pin it down

INFO:root:creating new sample handler
INFO:root:creating new job
INFO:root:creating algorithms
BasicEventSelection::B... INFO Calling constructor
INFO:root:adding basicEventSelection to algs
BasicEventSelection::s... INFO Calling setupJob
INFO:root:adding mcEventVeto to algs
INFO:root:adding calibrateST to algs
INFO:root:adding preselectDileptonicWW to algs
INFO:root:adding selectDileptonicWW to algs
INFO:root:adding postselectDileptonicWW to algs
INFO:root:adding calculateRJigsawVariables to algs
INFO:root:adding calculateRegionVars to algs
INFO:root:adding writeOutputNtupleSR to algs
INFO:root:adding writeOutputNtupleCR1L to algs
INFO:root:adding writeOutputNtupleCR0L to algs
INFO:root:submit_dir already exists.
INFO:root:Overwriting previous submitDir
INFO:root:creating driver
INFO:root:grid driver
gridname: user.jaoliver.%in:name[2]%.%in:name[3]%.120516_lvlv
INFO:root:submit job
BasicEventSelection::B... INFO Calling constructor
TFile::TFile ERROR file /afs/cern.ch/work/j/jaoliver/private/RJigsawTools_110516/submit_dir/input/RJigsawTools/data/mc15_13TeV_SM_SUSY2_p2470_failed.ds.root can not be opened (No such file or directory)
TROOT::WriteTObject ERROR The current directory (PyROOT) is not associated with a file. The object (sample) has not been written.
TFile::TFile ERROR file /afs/cern.ch/work/j/jaoliver/private/RJigsawTools_110516/submit_dir/hist/RJigsawTools/data/mc15_13TeV_SM_SUSY2_p2470_failed.ds.root can not be opened (No such file or directory)
TROOT::WriteTObject ERROR The current directory (PyROOT) is not associated with a file. The object (sample) has not been written.
^Z

This can be split into separate issues later, but to make this work out as a general purpose RJigsaw toolset, we need to have other calculators for different decay trees. This essentially requires taking the code from the RestFrames examples and placing it in this framework.

SUSY2 Backgrounds

Diboson should only run over:

mc15_13TeV.361063.Sherpa_CT10_llll.merge.DAOD_SUSY2.e3836_s2608_s2183_r7267_r6282_p2470
mc15_13TeV.361064.Sherpa_CT10_lllvSFMinus.merge.DAOD_SUSY2.e3836_s2608_s2183_r7267_r6282_p2470
mc15_13TeV.361065.Sherpa_CT10_lllvOFMinus.merge.DAOD_SUSY2.e3836_s2608_s2183_r7267_r6282_p2470
mc15_13TeV.361066.Sherpa_CT10_lllvSFPlus.merge.DAOD_SUSY2.e3836_s2608_s2183_r7267_r6282_p2470
mc15_13TeV.361067.Sherpa_CT10_lllvOFPlus.merge.DAOD_SUSY2.e3836_s2608_s2183_r7267_r6282_p2470
mc15_13TeV.361077.Sherpa_CT10_ggllvv.merge.DAOD_SUSY2.e4641_s2726_r7326_r6282_p2482
mc15_13TeV.361073.Sherpa_CT10_ggllll.merge.DAOD_SUSY2.e3836_s2608_s2183_r6869_r6282_p2482
mc15_13TeV.361068.Sherpa_CT10_llvv.merge.DAOD_SUSY2.e3836_s2608_s2183_r7267_r6282_p2470
mc15_13TeV.361084.Sherpa_CT10_WqqZll.merge.DAOD_SUSY2.e3836_s2608_s2183_r7326_r6282_p2482
mc15_13TeV.361086.Sherpa_CT10_ZqqZll.merge.DAOD_SUSY2.e3926_s2608_s2183_r7326_r6282_p2482

should these non-integer values be this way?

Things that need to be done...

• Make dev-Dylan compatible with master.
• In the TLS region calculator, write out lepton signed PDGID as well.
• Also add to the 2L calculator a variable isSS and isSF for ease of use later
• Write out a variable that encodes whether or not the leptons are in the same hemisphere

We want to try and run a production of these samples this weekend.

-L

Testing #80 changes I had the old issue where mcEventWeight was being retrieved as an int still, even though we fixed this since I was running the _zl code where it wasn't fixed.

To avoid similar things in the future, it would be good if we could move the things that every analysis needs to a common place (EventNumber, RunNumber, mcEventWeight etc... ).

Some kind of bookkeeping histograms will need to be implemented for the output files since we're throwing away events in these algorithms.

Right now the calculate function is empty. We need to add the correct calculations from the example in RestFrames.

It'd be nice to have a 0L-esque merging step to write out a weight that includes the cross section weighting per major background. Can look at:

https://github.com/lawrenceleejr/ZeroLeptonRun2/blob/master/Root/FilterUpdateMerge.cxx

for inspiration.

I think this is what we actually want to do to figure out the right CBK object :

martin tripiana [email protected]
11:44 AM (1 hour ago)

to atlas-phys-sus.
Dear all,

 we wanted to bring your attention to an issue observed by Ximo & Othmane (&others) with the way we (used to) access the CBK metadata in our SUSY derivations.

Long story short, the usual prescription of taking the lowest-n cycle among the complete
cbks available doesn't not longer apply in the new derivations. Or so it seems from some
few samples we looked at.

As you see below, the old rcp can give odd results now, as we have the info for more
primitive steps in the prod chain and of all the steps in the relevant train.

We put a quick fix example in the SUSYToolsTester code in the trunk just before which gives the expected results for now.

https://svnweb.cern.ch/trac/atlasoff/browser/PhysicsAnalysis/SUSYPhys/SUSYTools/trunk/util/SUSYToolsTester.cxx#L130

Please take note of this and report back if you find a problem with the new approach.

Best regards,
M/C

Issue Example (before fix!):

NEW 20.7
mc15_13TeV.361070.Sherpa_CT10_llvvjj_ss_EW6.merge.DAOD_SUSY2.e3836_s2608_s2183_r7725_r7676_p2613_tid08198517_00/DAOD_SUSY2.08198517._000001.pool.root.1
AllExecutedEvents : desc = N/A : inputStream = StreamDAOD_SUSY2 : cycle = 6 : allEvents = 4348
AllExecutedEvents : desc = N/A : inputStream = StreamAOD : cycle = 5 : allEvents = 10000
SUSY3KernelAug : desc = N/A : inputStream = StreamAOD : cycle = 5 : allEvents = 10000
SUSY3KernelSkim : desc = N/A : inputStream = StreamAOD : cycle = 5 : allEvents = 5842
SUSY2KernelAug : desc = N/A : inputStream = StreamAOD : cycle = 5 : allEvents = 10000
SUSY2KernelSkim : desc = N/A : inputStream = StreamAOD : cycle = 5 : allEvents = 4348
SUSY5KernelSkim : desc = N/A : inputStream = StreamAOD : cycle = 5 : allEvents = 8341
SUSY5KernelAug : desc = N/A : inputStream = StreamAOD : cycle = 5 : allEvents = 8341
AllExecutedEvents : desc = N/A : inputStream = StreamAOD : cycle = 4 : allEvents = 10000
AllExecutedEvents : desc = N/A : inputStream = StreamAOD : cycle = 3 : allEvents = 8000
AllExecutedEvents : desc = N/A : inputStream = StreamESD : cycle = 2 : allEvents = 8000
AllExecutedEvents : desc = N/A : inputStream = StreamRDO : cycle = 1 : allEvents = 15912
AllExecutedEvents : desc = N/A : inputStream = StreamRDO : cycle = 0 : allEvents = 31736
SUSYToolsTester INFO CutBookkeepers Accepted 31736 SumWei 31942.399574 sumWei2 32593.949977 (WRONG!)

OLD 20.1
SUSYToolsTester INFO Opening file: /nfs/atlas-data07/tripiana/xAOD/CBKtest/mc15_13TeV.361070.Sherpa_CT10_llvvjj_ss_EW6.merge.DAOD_SUSY2.e3836_s2608_s2183_r7267_r6282_p2499_tid08215078_00/DAOD_SUSY2.08215078._000001.pool.root.1

SUSYToolsTester INFO Number of events in the file: 4669
AllExecutedEvents : desc = N/A : inputStream = StreamDAOD_SUSY2 : cycle = 2 : allEvents = 4669
AllExecutedEvents : desc = N/A : inputStream = StreamAOD : cycle = 1 : allEvents = 10000
SUSY2KernelAug : desc = N/A : inputStream = StreamAOD : cycle = 1 : allEvents = 10000
SUSY2KernelSkim : desc = N/A : inputStream = StreamAOD : cycle = 1 : allEvents = 4669
AllExecutedEvents : desc = N/A : inputStream = StreamAOD : cycle = 0 : allEvents = 10000
SUSYToolsTester INFO CutBookkeepers Accepted 10000 SumWei 10080.925944 sumWei2 10388.937223

And actually in general, it would be nice if we were smart enough to stop looking for things that don't exist in our derivation in subsequent in CalibrateST calls.

Having an issue compiling properly,

I follow the instructions and when I run the test example I get the following error:

TFile::TFile ERROR file /afs/cern.ch/work/j/jaoliver/private/RJigsawTools/RootCoreBin/data/ElectronEfficiencyCorrection/efficiencySF.offline.TightLLH_d0z0_v10.2015.13TeV.rel20p0.25ns.v05.root does not exist
TAsgElectronEfficiency... ERROR /build1/atnight/localbuilds/nightlies/AnalysisBase-2.3.X/AnalysisBase/rel_nightly/ElectronEfficiencyCorrection/Root/TElectronEfficiencyCorrectionTool.cxx:301 (virtual int Root::TElectronEfficiencyCorrectionTool::initialize()): TAsgElectronEfficiencyCorrectionTool_id_TightLLH (file: /build1/atnight/localbuilds/nightlies/AnalysisBase-2.3.X/AnalysisBase/rel_nightly/ElectronEfficiencyCorrection/Root/TElectronEfficiencyCorrectionTool.cxx, line: 301) ! No ROOT file found here: /afs/cern.ch/work/j/jaoliver/private/RJigsawTools/RootCoreBin/data/ElectronEfficiencyCorrection/efficiencySF.offline.TightLLH_d0z0_v10.2015.13TeV.rel20p0.25ns.v05.root
AsgElectronEfficiencyC... ERROR /build1/atnight/localbuilds/nightlies/AnalysisBase-2.3.X/AnalysisBase/rel_nightly/ElectronEfficiencyCorrection/Root/AsgElectronEfficiencyCorrectionTool.cxx:158 (virtual StatusCode AsgElectronEfficiencyCorrectionTool::initialize()): Could not initialize the TElectronEfficiencyCorrectionTool!
SUSYObjDef_xAOD ERROR /afs/cern.ch/work/j/jaoliver/private/RJigsawTools/SUSYTools/Root/SUSYToolsInit.cxx:515 (StatusCode ST::SUSYObjDef_xAOD::SUSYToolsInit()): Failed to call "elecEfficiencySFTool_id->initialize()"
SUSYObjDef_xAOD ERROR /afs/cern.ch/work/j/jaoliver/private/RJigsawTools/SUSYTools/Root/SUSYObjDef_xAOD.cxx:492 (virtual StatusCode ST::SUSYObjDef_xAOD::initialize()): Failed to call "this->SUSYToolsInit()"
virtual EL::StatusCode... ERROR Failed to execute: "m_objTool->initialize()"

Any idea how to fix this / the origin of this error?

Cheers,

Jason

These should select the events based on whatever criteria we need in each of these regions, and then to fit in the framework we should do something like add a decoration to the EventInfo object (ala Sklimmer code).

@mattleblanc 's issue is what happens when the SampleHandler object is empty. This is actually quite annoying and should have some higher verbosity (probably not by default since SampleHandler spits out a lot of stuff with sh.printContent() ).

I made a few quick changes to look at this, but probably this is just some annoying failure in fillSampleHandler (maybe SampleHandler has changed?) .

This goes along with whatever was going on with FAX yesterday, but that's not our fault .

Out of the box, the current master (07904e4) doesn't compile. First we need to add SUSYTools to the README since we've switched to AnalysisBase.

And then in the compilation of xAODAnaHelpers, I get the following error. So there's some incompatibility in the various versions in the README. @rsmith54, I know you have a working setup - can you update the readme to use the tags you're using?

-L

/afs/cern.ch/work/l/leejr/public/TwoLeptonFactory/xAODAnaHelpers/Root/BasicEventSelection.cxx: In member function ‘virtual EL::StatusCode BasicEventSelection::execute()’:
/afs/cern.ch/work/l/leejr/public/TwoLeptonFactory/xAODAnaHelpers/Root/BasicEventSelection.cxx:600:39: error: no matching function for call to ‘CP::PileupReweightingTool::apply(const EventInfo&)’ 
       m_pileuptool->apply( *eventInfo ); // NB: this call automatically decorates eventInfo with:
                                       ^
/afs/cern.ch/work/l/leejr/public/TwoLeptonFactory/xAODAnaHelpers/Root/BasicEventSelection.cxx:600:39: note: candidate is:
In file included from /afs/cern.ch/work/l/leejr/public/TwoLeptonFactory/xAODAnaHelpers/xAODAnaHelpers/BasicEventSelection.h:9:0,
                 from /afs/cern.ch/work/l/leejr/public/TwoLeptonFactory/xAODAnaHelpers/Root/BasicEventSelection.cxx:29:
/afs/cern.ch/work/l/leejr/public/TwoLeptonFactory/RootCoreBin/include/PileupReweighting/PileupReweightingTool.h:87:26: note: virtual StatusCode CP::PileupReweightingTool::apply(const EventInfo&, bool)
       virtual StatusCode apply ( const xAOD::EventInfo& eventInfo, bool mu_dependent );
                          ^
/afs/cern.ch/work/l/leejr/public/TwoLeptonFactory/RootCoreBin/include/PileupReweighting/PileupReweightingTool.h:87:26: note:   candidate expects 2 arguments, 1 provided
Compiling IParticleHistsAlgo.o
At global scope:
cc1plus: warning: unrecognized command line option "-Wno-tautological-undefined-compare"
make: *** [/afs/cern.ch/work/l/leejr/public/TwoLeptonFactory/RootCoreBin/obj/x86_64-slc6-gcc49-opt/xAODAnaHelpers/obj/BasicEventSelection.o] Error 1
make: *** Waiting for unfinished jobs....
RootCore: Error failed to compile package xAODAnaHelpers

Just letting you know that when you run

python RJigsawTools/util/run_lvlv.py --doOverwrite --inputDS RJigsawTools/data/mc15_13TeV_HIGG3D1_TEST.ds --gridTag 012516_lvlv

It no longer prompts you asking "did you mean to include --driver grid", rather it runs it on lxplus. Not sure if this is intentional, but i found the prompt to be helpful

python RJigsawTools/util/run_lvlv.py --doOverwrite --inputDS RJigsawTools/data/mc15_13TeV_HIGG3D1_TEST.ds --gridTag 012516_lvlv --driver grid

is what I run for the grid, as usual

I created a branch for this , but there are some small things we should update (README, example files, etc), and we should also consider if everything can be moved over to 20.7 so we can put the branch in the master.

Not exactly sure what I want here, but something smarter can probably be done to set the calculator (right now using an enum).

GeV are easier.

From output ntuples:

root [2] treesSR->Scan("mcEventWeight")
************************
*    Row   * mcEventWe *
************************
*        0 *         0 *
*        1 *         0 *
*        2 *         0 *
*        3 *         0 *
*        4 *         0 *
*        5 *         0 *
*        6 *         0 *
*        7 *         0 *
*        8 *         0 *
*        9 *         0 *
*       10 *         0 *

etc. Should figure this out.

Need to add another algorithm to calculate non RJR variables - e.g. meff, HT, whatever.

We should have some way that we check that clearEvent has been called for each calculator. This is mostly a reminder to myself, but I'm also not sure the best way to do this.

Would be pretty fast and easy to do. Just start n processes and wait for them to finish. Will be important once we start pulling a full production and can significantly speed up the workflow easily.

In things like the RegionVarsCalculator, we are copying the vectors over into the map after the push_backs.

This is definitely slower than allocating the vectors on the heap, and also slower than assigning the map with an empty vector and push_back'ing into a reference to the vector from the map, but I think it's better to keep it since it's a lot simpler as is. IF we have problems with speed at some point we can reconsider this.

We need a reasonable preselection in the SelectDileptonicWW events source file.

xAODAnaHelpers has deprecated 2.3.X support luckily.

I'll update the README and make sure everything works.

UCATLAS/xAODAnaHelpers#589

MET is calculated in CalibrateST, but we need to calculate it after our selections if we use a tighter hard object selection in our selection algs than SUSYTools. It might be nicest to just overwrite it so the calculation alg can use the same name.

so we have them.

Ugh typed up a whole thing and lost it..... Try again...

Unless I'm wrong (please let me be), there's no way to create an IParticleContainer containing multiple kinds of particles. e.g. If I want to hand an electron and a muon to the calculator, I need to create a new container object that contains all the interesting objects. This can, for example, be a container of xAOD::Particle objects as defined here:

http://acode-browser.usatlas.bnl.gov/lxr/source/atlas/Event/xAOD/xAODParticleEvent/xAODParticleEvent/versions/Particle_v1.h

This seems to be the simplest non-abstract IParticle type. The problem is, that since I can't cast between derived classes, I'm trying to create an xAOD::Particle from each object. But for example, if I create an xAOD::Particle and try to set the momentum (by any of the methods) I get a run-time crash:

python: /build2/atnight/localbuilds/nightlies/AnalysisSUSY-2.3.X/AnalysisSUSY/rel_nightly/RootCore/include/AthContainers/AuxElement.icc:433: SG::AuxElement::
Accessor<T>::reference_type SG::AuxElement::Accessor<T>::operator()(SG::AuxElement&) const [with T = float; SG::AuxElement::Accessor<T>::reference_type = float&]: Assertion `e.container() != 0' failed.
Aborted (core dumped)

I'm tempted to just store things as vectors of TLorentzVectors, and would therefore need to change the calculator class to take those instead of IParticleContainers. But if there's a cleaner way to do it, I'm happy to.

Thoughts?

Right now the _lvlv calculator directly follows the H->WW example from RestFrames. This might not be the exactly correct way to set things up.

It would be good to used ordered maps. In actually trying to play with ntuples, the randomness of the std::map has made it just a little annoying to just open a file and peek at something. Not urgent, but would be nice at some point.

Some events seem to have very high (>1e9 GeV) for example for H2PP, but the bulk seem fine. Gotta investigate these events...

For clarity

We need a calculator for the 2L strong decay tree.

Already adapted the lvlv example to take in a gridDS option in #35. Need to extend to allow for a file of DS names.

We should have some additional run scripts for other selections/calculator setups to demonstrate use of the different selection algs and calculators.

Current merging script creates trees with a new branch ("normweight") which look fine in the intermediate output files (one per DSID). But then it invokes hadd via the os module. The resulting file has mostly alright norm weights but some events obtain radical values (e.g. -1.64e-46 and -2.3e-262). Looking at the individual files, there is no such problem, and as soon as I hadd two together, the corruption appears.

I'm working on validating the output of RJigsawTools vs ZeroLeptonRun2.

Despite a difference in jet definitions which affects the MEff filtering, the unweighted distributions look good at high scales where this should matter less.

However, the norm weights differ from ZLR2. These should be the same. For the ttbar sample, ours is ~3e-5, and ZLR2's merging step gives ~2.2e-5. This is probably because we're pulling the sum of weights from the XAH cutflow histogram which just counts the events run over. We instead need to count the pre-derivation-skimming events. However turning on the CBC accessing with m_useMetaData in XAH causes a crash. Submitted an issue here:

UCATLAS/xAODAnaHelpers#565

-L

Lepton SFs will of course need to be implemented for ID and triggering. Can follow the example of zero lepton for starters:

https://github.com/lawrenceleejr/ZeroLeptonRun2/blob/master/Root/ZeroLeptonCRWT.cxx#L478

We should move the common setup code in the run scripts to another file.