nnpdf / pineappl

PineAPPL is not an extension of APPLgrid

NNPDF: An open-source machine learning framework for global analyses of parton distributions

The NNPDF collaboration determines the structure of the proton using Machine Learning methods. This is the main repository of the fitting and analysis frameworks. In particular it contains all the necessary tools to reproduce the NNPDF4.0 PDF determinations.

Documentation

The documentation is available at https://docs.nnpdf.science/

Install

See the NNPDF installation guide for the conda package, and how to build from source.

Please note that the conda based workflow described in the documentation is the only supported one. While it may be possible to set up the code in different ways, we won't be able to provide any assistance.

We follow a rolling development model where the tip of the master branch is expected to be stable, tested and correct. For more information see our releases and compatibility policy.

Cite

This code is described in the following paper:

@article{NNPDF:2021uiq,
    author = "Ball, Richard D. and others",
    collaboration = "NNPDF",
    title = "{An open-source machine learning framework for global analyses of parton distributions}",
    eprint = "2109.02671",
    archivePrefix = "arXiv",
    primaryClass = "hep-ph",
    reportNumber = "Edinburgh 2021/13, Nikhef-2021-020, TIF-UNIMI-2021-12",
    doi = "10.1140/epjc/s10052-021-09747-9",
    journal = "Eur. Phys. J. C",
    volume = "81",
    number = "10",
    pages = "958",
    year = "2021"
}

If you use the code to produce new results in a scientific publication, please follow the Citation Policy, particularly in regards to the papers relevant for QCD NNLO and EW NLO calculations incorporated in the NNPDF dataset.

Contribute

We welcome bug reports or feature requests sent to the issue tracker. You may use the issue tracker for help and questions as well.

If you would like contribute to the code, please follow the Contribution Guidelines.

pineappl's Issues

Ensure backwards compatibility

Changing structs results in different file format (implicitly defined through serde and bincode). As a result, loading previously generated grids with the changed version of PineAPPL won't work. Since grid generation is a very costly operation, this is not desirable.

To avoid this and allow changes in structs we should adopt a strategy that is outlined here and first check if bincode supports this.

Add APPLgrid importer

We need a subcommand in pineappl which is able to convert APPLgrids into PineAPPL grids.

Implement fully dynamic scale in LagrangeSubgrid

Remove the following todo!(); by resizing the grid properly:

https://github.com/N3PDF/pineappl/blob/026421a372cef058656887ecca3765141cc4c746/pineappl/src/lagrange_subgrid.rs#L198-L200

Add subcommand that shows differences between two grids

Make lhapdf crate separate

The lhapdf crate is a dependency for lhapdf_cli, but it should be a separate crate and not a member of this workspace.

Add offline file-size optimisation

Possible optimisations:

• symmetrisation of the initial states if the hadronic initial states are the same
• better file format, which doesn't save zeros; if the optimisation is offline, we can try several sparse matrix ideas like DOK, LIL, COO, CSR, BSR, ...
• optimise luminosity function:
  • strip empty entries
  • strip numerical zeros, see also NNPDF/pinecards#107
  • #199
  • strip entries with factor = 0.0 (why are they there in the first place?)
  • #246
• static scale detection; remove interpolation of Q^2 in this case
• #119
• use bincode's varint encoding, part of #118
• #117
• #151
• #202

Add fastNLO importer

Similar to issue #17.

Add functionality to remap bins

Monte Carlo generators often do not support two- or three-dimensional distributions, which is why we need to map them to a one-dimensional distribution during their generation. It would be nice to have functionality in PineAPPL which allows to 'remap' the one-dimensional distribution to the original two- or three-dimensional one. To do so, we simply need a global vector which collects, for each bin, the new limits and possibly multiplicative factors.

Improve error messages

Whenever the methods of the crate pineappl return an error the error messages are not helpful.

Improve remap subcommand

The subcommand remap needs to understand more complicated specifications for bin limits.

Add new subcommand showing the size of each initial-state

The program pineappl needs a new subcommand showing the size of each initial-state for each bin.

Add switch to choose between different subgrids

Right now we have two subgrids, which are LagrangeSubgrid and NtupleSubgrid, and by default LagrangeSubgrid is chosen. We should let the user decide this.

Support initial-state hadrons different than protons

Right now PineAPPL doesn't save the type of hadrons in the initial state and assumes it's a proton.

Show size of each contribution/correction

pineappl needs a new subcommand that, for each bins, list the size for each order.

Add support for DIS grids

We need support for observables that require only a single PDF convolution: DIS.

Integrate with mg5_aMC

To use PineAPPL together with mg5_aMC we should add a HEPTool which install both Rust and PineAPPL.

Add subcommand for fully differential output

We need a subcommand that convolutes the grid with a PDF, but is differential in bin, lumi, and order.

Add new subcommand `subgrids`

This subcommand should show technical information about the various subgrids:

• subgrid type
• x1, x2, q2 grid values
• which part of grid is empty (in q2, in x1, x2)
• grids are zero

Upload the sphinx docs to readthedocs

Create an action which uploads the sphinx docs for the master branch in the readthedocs website.

Optimize SparseArray3

Optimize SparseArray3 for the case in which the second dimension is larger than the third; in this case the two axes are best swapped.

Add functionality to remove bins from grids

We need a method in Grid and a subcommand in the CLI to remove specific bins from a PineAPPL grid.

@felixhekhorn @alecandido

Improve diff subcommand

• Make sure that the differences are always visible; when the difference is smaller than a tenth of a permille, the difference is not visible. Instead of showing differences in per cent we should show relative differences fixed in commit bd869e5.
• The order of the perturbative contributions seems to be random; this has to be fixed fixed in commit 81908de.
• The differences shown as per cent were actually just relative differences fixed in commit 0b96587.

Upload wheels to pypi

Create an action which uploads python wheels for the master branch when a new release is created.

pineappl_capi needs a buildsystem

PineAPPL needs a build system that installs the header, the shared library, and the pkgconfig file telling other build systems where to find the files.

Get a Zenodo DOI

Get a digital object identifier from https://zenodo.org/ and add citation information.

Add switches in convolute for pineappl to select orders

The subcommand convolute in pineappl needs a switch to select/deselect orders that contribute to the convolution.

Integrate with apfelcomb

Write an implementation in apfelcomb to PineAPPL and find out if Grid::convolute is sufficient to do this or if any other methods are needed.

Add subcommand to calculate PDF uncertainties

We need separate command to calculate PDF uncertainties for (possibly multiple) PDF sets.

Improve convolution time

The subcommand convolute of pineappl needs roughly 8.5 seconds on cigno for the convolution of CMSDY2D11 with a PDF set. Roughly two thirds of the time is spent loading the grid (about 800 MB large), but the remainder is spent convoluting. The latter should be optimized by caching the luminosity function.

Add new subcommand to read/write/add metadata

The program pineappl needs new subcommands to read/write/add metadata:

• cuts
• description of the content
• link to the arxiv paper
• link to the experimental data on hepforge
• parameters
• run card
• generator information
• units: pb, fb, ..., MeV, GeV, ...
• centre-of-mass energy
• labels of the observables, latex-compatible labels, ...
• grid author/person in charge

Add an entry to the changelog for the new python module

Add constant subgrid

PineAPPL needs a new grid type which is able to store (x1,x2,Q^2)-independent corrections, for instance to convert NNLO C factors into subgrids. These subgrids can then merged with the NLO grids, 'upgrading' them to NNLO precision.

Add Fortran interface

Fix unnecessary GridMergeError

Remove the artificial restriction of https://github.com/N3PDF/pineappl/blob/2981e2d3b0df55d100378e2eced33a28a7577d9a/pineappl/src/grid.rs#L544-L548.

Update READMEs

All four READMEs should be updated.

PineAPPL needs a logo

Do not show scale-variation uncertainties when NLO log grids are not present

In this case the uncertainties are wrong.

Decrease grid file size

The grids that are currently produced are rather large (CMSDY2D11 is 800 MB large), but this can be optimized using several strategies:

• many of out grids have basically a static scale, so that we should be able to remove the interpolation in Q2,
• inside the non-zero Q2 grids many bins are probably empty, which can be optimized when writing to disk using a simple compression algorithm.

Add subcommand to calculate integrated cross section grid

This subcommand should accept a PineAPPL file, combine all bins of it into a single unit-sized grid and save the result into a separate file.

Add CI and more bling

We need continuous integration and badges for crates.io.

Support `merge` where the input and output files are the same

A useful subcommand is pineappl merge output.pineappl input.pineappl --scale 0.5 to scale the grid. This works as long as the input and output file are not the same, which is useful, however. This requires being able to compress the file inside Rust.

Add new subcommand q2

We need a subcommand that can show us, or least allows us to reverse engineer, the scale choice. To that end it is useful to know which Q^2 bins of the grid are filled (LagrangeSubgrid) or the distribution of the Q^2 values (NtupleSubgrid).

Add Arch User Repository badge

Hi, I just updated a PKGINFO to the Arch user repository: https://aur.archlinux.org/packages/pineappl/
mainly for my own convenience but I thought it would be nice to let you know:

This package installs also the python package.

I tried this package in the PR that @scarrazza added in vegasflow: N3PDF/vegasflow#56 and seems to work.

Improve the output of the PineAPPL CLI

When using pineappl it is not clear what the meaning of each column in the output is.

Change the way subgrids are imported and exported

Add more checks for merging and optimization

When merging and optimizing grids we must perform more thorough checks if the grids are compatible with each other, for instance whether they have the same x1, x2, and q2 grids.

Fix installation directory of pineappl_cli

cargo-c installs the header at ${includedir}/pineappl_capi/pineappl_capi.h, but we need it at ${includedir}/pineappl_capi.h

Support resummed results

Add subcommand to change the central scale choice

Since we have the log(muR) and log(muF) dependent grids we can not only calculate theory uncertainties, but also change the central scale choice.

Implement theory uncertainties

The subcommand convolute of pineappl should print theory uncertainties calculated from a 3-, 7- and maybe a 9-point scale variation.