Presentation of the initiative at the DPG spring meeting about gamma-astro-data-formats

[maxnoe]

If no one has any comments to the last version of the abstract, I'll hand it in in the afternoon.

[maxnoe]

I adapted this from the ADASS abstract but had to shorten to make room for the last paragraph and to make it fit into the submission limit:

Open initiative for a VHE Open Data Format

The operation of the next-generation gamma-ray telescopes as observatories and the wish of currently operating instruments to archive and publish their data in a more accessible format poses the issue of developing an open format for gamma-ray data.

A first attempt at defining a common specification for high-level gamma-ray astronomical data has been initiated by members of different Imaging Atmospheric Cherenkov Telescopes (IACT) experiments with the "Data formats for gamma-ray astronomy" initiative.
The current specification defines formats for high-level gamma-ray data like lists of candidate photons and instrument response function, based on FITS files.

Open-source software for gamma-ray analyses, including gammapy and ctools, have recently developed support for this format and, as a result, a series of publications relying on standardized datasets and software have been issued.

Currently, an effort to formalize the endeavor is underway, creating a Coordination Committee formed from representatives of the participating instruments to steer the future development of the data formats.
In this talk, current developments and future plans will be presented, including the already implemented extension to ground-based wide-field experiments like HAWC and possible extension to other messengers.

[maxnoe]

Is someone else at the spring meeting I could add as a second author? E.g. @lmohrmann ?

[TarekHC]

Hi Max.

If I remember correctly, in the past I presented the joint-crab paper in the DPG!

Some comments from my side. Implement only those you agree with!

Open initiative for a VHE Open Data Format

I would not repeat "open" in the title.

The operation of the next-generation gamma-ray telescopes as observatories and the wish of currently operating instruments to archive and publish their data in a more accessible format poses the issue of developing an open format for gamma-ray data.

I feel two important points are missing in this paragraph. 1) I would also refer to open-source science tools to analyze the data (if not, the format is a bit useless). 2) joint multi-instrument analysis is another of the main scientific drivers for current-generation instruments to push for this... So if you can find a way of adding both points to the sentence, it would be ideal.

A first attempt at defining a common specification for high-level gamma-ray astronomical data has been initiated by members of different Imaging Atmospheric Cherenkov Telescopes (IACT) experiments with the "Data formats for gamma-ray astronomy" initiative.

I would probably define it as a github repository used for storing documentation, and as a forum for discussion.

The current specification defines formats for high-level gamma-ray data like lists of candidate photons and instrument response function, based on FITS files.

based on FITS files -> serialized as FITS files? you probably know more than me about the wording of these technical things...

Open-source software for gamma-ray analyses, including gammapy and ctools, have recently developed support for this format and, as a result, a series of publications relying on standardized datasets and software have been issued.

I would probably move some of this to the first sentence (the fact that this software exist, not the fact that recent publications exist).

No more comments from my side... as you can see, only subtleties you can ignore!

[adonath]

Thanks @maxnoe, I agree it's a good idea to present the initiative at the DPG spring meeting.

Open initiative for a VHE Open Data Format

I agree with @TarekHC here, the repeated "open" sounds odd. What about "Community initiative..."?

including gammapy and ctools, ...

Minor: the recommended spelling is "Gammapy" as a proper name.

[maxnoe]

Thanks all, here is a new version (deadline the day after tomorrow).

Community Initiative for a VHE Open Data Format

The operation of the next-generation gamma-ray telescopes as observatories, the wish of currently operating instruments to archive and publish their data in an accessible format,
and enabling multi-instrument analyses are strong reasons for developing an open, software independent format for gamma-ray data.

A first attempt of a common specification has been developed by members of different Imaging Atmospheric Cherenkov Telescopes (IACT) within the "Data formats for gamma-ray astronomy" initiative.
The current version defines formats for high-level gamma-ray data, including event lists of candidate photons and instrument response functions, serialized as FITS files.

Open-source software for gamma-ray analyses, including gammapy and ctools, have recently developed support for this format and, as a result, a series of publications relying on standardized datasets and software have been issued.

Currently, an effort to formalize the endeavor is underway, creating a Coordination Committee formed from representatives of the participating instruments to steer the future development of the specification.

In this talk, current developments and future plans will be presented, including the already implemented extension to ground-based wide-field experiments and possible extension to other messengers.

[maxnoe]

I feel two important points are missing in this paragraph.

I added the second aspect, the first is in a paragraph further down and space is really limited. I had to shuffle around some parts, anticipating a second author.

I would probably define it as a github repository used for storing documentation, and as a forum for discussion.

Given the space limitation, I think this is too technical for the abstract. I think it's only important for the abstract that there is an open initative with that goal, not how it works in detail, right?

serialized as FITS files?

Done

What about "Community initiative..."?

Done

Minor: the recommended spelling is "Gammapy" as a proper name.

Done

[lmohrmann]

@maxnoe I had not planned to submit an abstract (but support of course a presentation about GADF there). I don't know whether I will be attending on any of the days. If it's still possible to add me as co-author, feel free to do so!

[maxnoe]

@lmohrmann Yes, I could do it, but maybe there's someone who will certainly attend the conference?

[maxnoe]

I submitted the abstract now, I can still make changes if someone has last minute comments.

[maxnoe]

The talk is next week and I prepared the slides here: (private for now): https://github.com/open-gamma-ray-astro/gadf-talk-dpg2022/blob/main/gadf_dpg2022.pdf

[TarekHC]

Hi @maxnoe

Nice slides! Some comments here:

• Proprietary analysis software -> Mostly proprietary analysis software (VERITAS's eventDisplay is open source, as Gernot will surely tell you)
• Mostly based on ROOT -> Maybe adding some info about ROOT is helpful for some external people? "Mostly based on CERN's ROOT library" or something like that... For the DPG probably doesn't matter...
• Current generation telescopes ... -> "Current-generation telescopes" or better "The current generation of telescopes", and I would extend it in this direction "need a better format and science tools to allow long-term archival and analysis"
• "on specifying a file format for event lists and instrument response" -> on specifying a file format for event lists and instrument response "components"? (or equivalent?) Leaving it singular reads very weird to me.
• sl 8: "C. Nigro et al 2019" should appear in the reference of the paper

Of course, just suggestions. Feel free to ignore (except the last one, that one I feel is important).

[maxnoe]

@TarekHC Thanks for the comments, I will implement them (actually I already implemented the first one!)

CERN's ROOT library" or something like that... For the DPG probably doesn't matter...

It's in the IT session of the particle physics DPG meeting, so I guess almost everyone will know and love ROOT

"need a better format and science tools to allow long-term archival and analysis"

This is what I will say, I tried to keep the bullet points concise as it's mostly Wall of Text already

[cboisson]

Just a quick comment on the last sentence "Interoperability with other standards, e. g. the Virtual Observatory" : concerning the VO that is not a format standard (but a schema) it is better to write "interoperability with Virtual Observatory services"