ntia / scos-actions Goto Github PK

Sensor cal is loaded from file on container restart, so y-factor calibration results do not persist.

We rely heavily on self-calibrations running on fielded sensors. Currently, the sensor and sigan calibration files must have the correct key structure already to store the results of a calibration. This means that calibrating a new set of parameters requires updating the calibration files.

The required nested key structure should be created automatically based on the parameters used in the Y-factor calibration, so that field calibrations can be performed and saved for parameter combinations not already present in the calibration files.

Preselector web relay timeout errors occasionally occur when running the calibration action. As-is, this causes the action to fail and the sensor calibration not to be updated. For scheduled calibrations, this can mean that much larger gaps than are expected will exist between consecutive sensor calibrations.

The action can and should retry calibration in the event of an error, up to some limit of retries, which may or may not be made configurable in YAML.

Existing metadata structure for SEA edge data products is overly cumbersome. Restructuring and reducing it will allow for easier bulk post-processing of edge data products. Some changes should also be made to the data result.

Data/Processing

• Re-organize processing such that the IIR filter is applied before the FFT is computed. This is important since the calibration uses the filter, and the calibration result is applied to the IQ samples before the FFT even when the filter is not.
• If possible, implement more aggressive parallelization to improve the efficiency and speed of edge computations.
• Add mean/max channel power component (2x15 pts for 15 channels), in addition to existing components.
• Clean up logger for data product and calibration actions by stripping non-sigan parameters from those attempted to set on sigan

Metadata

• Restructure metadata to be more concise and simple, grouping most metadata by channel.
• Add sensor readouts to metadata: temperatures, humidity, preselector door sensor, etc.

• monitor_sigan action behavior depends on the is_available value from the radio interface. This causes unexpected behavior, such as the status being reported as "healthy" when there are actually errors which were not caught in the radio interface itself.
• test_required_components (implemented in MeasurementAction) also relies on is_available, which makes it susceptible to similar problems
• test_required_components is overridden in calibration and SEA data product actions, but these implementations still have the same issue.
• When the signal analyzer is unreachable, a power cycle should be attempted

It looks like SCOS Actions' use of NTIA/SigMF instead of the official gnuradio/SigMF is a relic from a time before NTIA/sigmf-ns-ntia existed. The official SigMF has a v1.0.0 release, which we should be using instead of an unmaintained and out-of-date fork.

This will also make the SigMF dependency installable from PyPi along with most other dependencies, which should speed up SCOS build times a little bit, since it's one fewer package that will need to be built from source.

Merge #48 into SEA branch and make corresponding updates to SEA data product action.

Causes failure to initialize scos_actions.discover

The metadata handling should be revisiting specifically to eliminate excessive number of arguments in constructors and investigate use of json schema to perform validation.

The code could use a PyTest for the individual actions, especially those that apply some form of processing (namely single frequency FFTs). We should generate a signal IQ, feed that into the processing algorithm and ensure the returned result is what we expect. This would prevent any accidental mistakes when adjusting the algorithms. Once the Generic Measurement Info branch is pulled in, this can be implemented (this issue is serving as a reminder).

Or, identify an alternative solution that avoids hard coding.

Issue

When running the Y-Factor calibration action, the preselector is switched first to the noise diode on path, then to the noise diode off path.

If the Y-Factor action is run, then a measurement action is run, the measurement action needs to set the antenna path. Otherwise, the switch is still set to the noise diode path, and the measurement will not be performed correctly.

Proposed Solution

I see two solutions to this:

1. Update the Y-Factor action to read the preselector's RF path at the start, then at the end of the action, reset the RF path to whatever it was before.
2. Update all measurement actions to include preselector configuration so that the correct path is used.

Option 1 has the benefit of not touching other actions, but could still leave the preselector in the wrong state for a given measurement action.

Option 2 seems better, but requires a preselector (using the ITS Preselector package) to run any action (which may not be an issue)

Temporary Workaround

This can be worked around in the meantime by manually setting the preselector state using its web interface before running a measurement action.