peppedilillo / mescal Goto Github PK

calibrate.py could use a further preprocessing filter step to address retriggering events.
what we need is a function filter_retrigger(data, gate) which takes a pandas DataFrame (the data) and a gate value in us (nominally 20) and returns a pandas DataFrame containing all events in data but those having the immediately preceding events at times smaller than the event's time minus the gate value.

the function should be implemented in specutilities.py and called in calibrate.py.

    # this is calibrate.py
    with console.status("Preprocessing.."):
        filter_mult = (lambda df: df[(df['NMULT'] < 2) | ((df['NMULT'] == 2) & (df['EVTYPE'] == 'S'))])

        data = add_evtype_tag(data, couples={q: get_couples('fm1', q) for q in 'ABCD'})
        data = filter_mult(data)
        > HERE FILTER RETRIGGERS <
        ....

i believe implementation could mimic the logic of specutilities.add_evtype_tag or in any case should rely on numpy or pandas (no loops or list comprehensions). this because of code efficiency.

Within Xmode, the detectPeaks function (from Specutilities) is used to both apply a smoothing algorithm (if necessary, usually depending on noise levels) and to automatically find the calibration lines and return their position.

Both the smoothing and detection algorithms use several input values, such as "window" (a range within which the maximum value is searched for) or "prominence" (a measure of the prominence of the spectral feature the algorithm is looking for, by comparing it to its vicinity). Playing with these parameters may provide better results.

However, currently these parameters are set by direct input in both the Specutilities and Xmode codes. It would be better to choose which ones should be fixed and which ones should not, and thus the latter should be accesible within the first lines of code, where all parameters are set right from the beginning.

Implement X and S spectra obtained summing events over all SDDs/scintillator couples.

(Easy example of how to use the Issues tool)

The ranges-step variables define the binning with which the spectra are built and from which the Gain and Offset values are obtained.

In the latest calsw version, ranges-step is defined manually and separately in X and S mode calibrations. However, in S mode, if the ranges-step value differs from that used beforehand on X mode, a normalization factor should be introduced to maintain the validity of the Gain and Offset.

Throughout the code there are several imported modules that are not in use, mainly due to copypasting the whole import header into all branches of the code. It would be best to check which are currently being used (in all the different modalities of a given code) and which are not, and then erase the latter from the imports section.

The output writing format is already included within the script, with the functions write_to_***. It'd be useful to include an If which decides, based on a user-input flag (-csv, for example), in which format to write down the data.

It'd be useful to have the option to calibrate the X-ray range with a quadratic LS, instead of a linear LS. This would imply also a modification of the Smode calibration script, since instead of using Gain and Offset, it would need to use Gain, Offset + one extra parameter (since its a quadratic now).

Linearity plots (done with the linPlot function in Specutilities) could be updated to include residuals.
Currently, residual plots are done within the calibrate function in Specutilities, but they are not percentile residuals. Plus, these plots currently overlap information with those done within linPlot.

In Xmode, the current way to fit the calibration spectrum uses a pedestal guess within the hist function in Specutilities.
There are two pedestal guesses in this function: "pedestal", which is only reading a user-generated file with all pedestal values for all channels, or "pedestal2", which guesses the pedestal value for each channel by using the guess_pedestal function within Specutilities. This function checks for peaks and returns the average value of the first two it finds (if more than two exist).

The original idea was to implement directly guess_pedestal, but in several cases there are more than two peaks that we need to discard (pedestal + noise peaks, which can be more than one). Thus, when there are more than 2 peaks to discard, guess_pedestal does not work properly.

Maybe by iterating guess_pedestal N times until a calibration condition is met?

It'd be best to upgrade the S-mode code to do a separate calibration, such as the X-mode one, with more than only one feature.
As of now, cal_Smode uses the 661.657 spectral feature of Cs137, and nothing else. But eventually other sources (Co60?) could be used that have more than one feature.

It'd be nice to implement a better mode to decide which binning to use for the Xmode calibration: An automatic binning? Or maybe as an user-input?

Write down Specutilities Documentation in the Discussions section.

It'd be useful to have an optional Smode calibration in which the gamma-ray range is calibrated with its own linear function, using 2 or more lines (similar to Xmode).