Question: Must ENERG_LO/ENERG_HI binning be identical for different IRFs? about gamma-astro-data-formats

[bkhelifi]

Hello,
I think that they might not for two reasons
1/ the software functioning and accuracy should not depend on the bins scheme or strategy. It is very dangerous
2/ When making a spectrum, the used binning can change strongly between a weak source and a strong one. So, the integration of the migration matrix should be done very accurately over the user binning, based on a fine binning of the migration matrix. The area, in contrary, vary smoothly and interpolations are easy to make without strong computing errors.
So, I can easily conceived that at the end, after computing the software inaccuracies, the binning of the migration matrix could be much finer that the area one.

[cdeil]

Hi @TarekHC, just to confirm what @bkhelifi wrote.

Must the energy binning of (for instance) the effective area be identical to the migration matrix by definition?

Short answer: No.

Are they assumed/checked to be identical within ctools/gammapy?

Short answer: No. (not 100% sure for ctools)

---

So basically the "IRF grids" are independent of the "analysis grids", i.e. for analysis users can choose energy, offset, skymap, ... binnings they like, and the science tool software interpolates and / or integrates onto that analysis grid. Usually IRF grids are chosen to sample the IRF dependencies well when using linear interpolation, and for science analysis the grids can be finer or coarser, depending on the goal of the analysis.

There are some caveats to that: e.g. in HESS we sometimes do event reconstruction or train gamma-hadron in energy bands, which then leads to steps in the IRFs at the energy band edges.
In these cases one can get artefacts at the energy edges when blindly interpolating IRFs onto new grids. At the moment this could maybe still be used by doing tricks like having two very small bins at the edges of these energy bands, but in the (far) future we might want to add more complicated declarative schemes for such things to the IRF format, that e.g. say to the science tools: "you really should be using this energy grid for analysis with these IRFs". Hopefully we will be able to go with a simple scheme and not need such complexities though all the way.

---

@TarekHC - Close the issue?

If you had this question, probably others will too.
Do you think it's worth / do you have time to add a paragraph explaining this to the spec?

Maybe start a section and bullet list of such general explanations for IRFs here?
http://gamma-astro-data-formats.readthedocs.io/en/latest/irfs/index.html
Or start a new subpage with "IRF notes" at the end of the IRF section?
http://gamma-astro-data-formats.readthedocs.io/en/latest/general/index.html

[TarekHC]

Hi @cdeil ,

There are some caveats to that: e.g. in HESS we sometimes do event reconstruction or train gamma-hadron in energy bands, which then leads to steps in the IRFs at the energy band edges.

This worries me slightly. The cuts we apply to CTA MC data to maximize sensitivity are not smooth at all. That would mean that interpolating some IRFs vs E would lead to very weird features. Although probably in the future we will have smoother outputs.

@TarekHC - Close the issue?

Yes, the answer is clear.

Do you think it's worth / do you have time to add a paragraph explaining this to the spec?

Next week I'll read them again, and see if a note should be added.

[jknodlseder]

I would not suppose that any of the binnings need to be identical. The software should transparently deal with rebinning or interpolation as needed. That's at least what ctools do.

…

Le 7 oct. 2016 à 14:42, Tarek Hassan ***@***.***> a écrit : Hi all, One simple question: Must the energy binning of (for instance) the effective area be identical to the migration matrix by definition? Are they assumed/checked to be identical within ctools/gammapy? @cdeil <https://github.com/cdeil> @jknodlseder <https://github.com/jknodlseder> If the answer is yes, we should add some clarifications in the specs (I could do it). But I want to make sure how consistent they need to be all IRF axes ENERGY, THETA, etc... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#69>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC2oV3LRKzqRrecJFRzIvZk47Cs0sVPoks5qxj4-gaJpZM4KQ_Lc>.