mbadv / multibeam_tools

Python-based tools for assessing Kongsberg multibeam echosounder performance

Multibeam Tools

• Code: GitHub repo
• Project page: Multibeam Advisory Committee

General Info

Multibeam Tools are a set of Python tools for evaluating multibeam system installation and performance during Sea Acceptance Trials or Quality Assessment Testing.

The tools are intended to simplify the multibeam evaluation process with a more user-friendly and flexible interface.

multibeam_tools's Issues

Swath accuracy: add percentiles to result plot

Add results plot option for percentiles within each beamwise bin (currently plotting mean +/- standard deviations).

Speed issue

Currently, the computation takes a bit too long.

There are a few ways to speed up the computation. We should discuss the options in person.

BIST plotter: add SOG adjustment to plot STW for RX Noise test

Speed over ground (SOG) is usually applied (parsed from file name or from SIS 5 BIST data) for RX Noise plots.

Add speed through water (STW) as a test parameter, based on SOG (as parsed) with user-defined adjustment for apparent currents along ship course. For example, if SOG = 10 kts (parsed from the BIST file or filename), and the ship is going into a 1 kt current, the STW is 11 kts.

When STW is selected for plotting, the user enters a fixed current value (e.g., 1.5 kts) and selects ship direction as 'with' current (STW = SOG - current) or 'against' current (STW = SOG + current). Default will be 0 kt current (or groupbox for STW adjustment will be disabled).

BIST plotter 0.2.1 fails while making Atlantis EM124 TX BIST history plot

Plotting more than one TX Channels BIST fails when making the history plot. Plotting one file at a time does not fail during the history plot step (with one file in the history).

Example BISTs included.

....
ztx_limits= [40.0, 150.0]
i= 8
i= 9
i= 10
i= 11
i= 12
i= 13
year matches
Traceback (most recent call last):
File "multibeam_tools\apps\bist_plotter.py", line 1383, in plot_bist
File "multibeam_tools\libs\read_bist.py", line 2041, in plot_tx_z_history
File "site-packages\matplotlib_init_.py", line 1810, in inner
File "site-packages\matplotlib\axes_axes.py", line 1611, in plot
File "site-packages\matplotlib\axes_base.py", line 393, in _grab_next_args
File "site-packages\matplotlib\axes_base.py", line 370, in _plot_args
File "site-packages\matplotlib\axes_base.py", line 231, in _xy_from_xy
ValueError: x and y must have same first dimension, but have shapes (864,) and (1728,)

20220924-230338-EM124_60-combined.txt
20221223-110048-EM124_60-combined.txt

Can we add the project code in Pydro?

Among other advantages, it would help to make the tools more discoverable!

Swath accuracy: add NOAA CO-OPS tide import

Add tide file import support for NOAA CO-OPS .csv exported for a given tide station

Example: https://tidesandcurrents.noaa.gov/waterlevels.html?id=8419870&units=metric&bdate=20210724&edate=20210730&timezone=GMT&datum=MSL&interval=6&action=data

swath accuracy plotter does not (yet) handle UTM zone changes

Adding flexibility to handle UTM zone changes... v0.0.1 code may be inoperable until then. Will push 0.0.2 when it's ready.

Swath coverage plotter: add GSF import

Add GSF import option to swath coverage and accuracy plotters

Note: UKHO gsfpy package runs only on Linux machines

Swath accuracy: add histogram

Add histogram plot(s) for filtered crossline results (matching soundings shown in other plots, after all filtering). Consider adding 'raw' and 'filtered' sounding histograms (toggled by the user?) to help illustrate the impacts of filtering on the crossline analysis.

Swath accuracy: color by mode / archive

Add options to plot multiple modes of crosslines and color the results accordingly. For example, identify unique modes and calculate results for each, then let user toggle visibility and color options. This could be expanded to mark 'archive' results against 'current' results, or similar comparisons across vessels or sonars.

The plotter already identifies each unique mode (combination of depth mode, swath mode, and pulse form) in the crossline file set and offers filtering by mode. However, the results are calculated each time a unique mode is plotted, rather than storing the results for each mode and simply changing visibility of these results on each plot. Plotting multiple modes is not currently supported.

For speed and ease of use, the plotter should be updated to calculate results across all modes, store results, and simply update plots as desired; recalculations should be done only after additional import, removal, or filtering of reference surface or crossline data.

Swath accuracy: uncertainty is shown as zeros (when not parsed)

Reference surface uncertainty is shown as zero when it is not parsed (e.g., from an average Dynamic Surface export to .xyz, rather than a CUBE surface export to .xyz).

Update plotter so that the uncertainty plot is not shown (uncheck 'Show uncertainty plot if parsed' option) and add note in activity log for user re: no uncertainty data available. Re-enable / re-check this option if uncertainty is parsed.

Swath accuracy: results plot in meters

Add meters option for plotting results (not just %WD) for comparison with earlier MAC plots

Swath coverage: Improve detection of missing sectors in dual-swath mode

Recent Kilo Moana EM122 data is missing the outermost starboard sector for the second swath (when dual swath is enabled) for some files (e.g., 2021 QAT file 0012, inbound from deep to shallow). This shows up as all RX beams associated with the outermost TX sector being flagged as 'rejected' in Qimera, which is also the flag used to detect outermost soundings for the swath coverage plotter. The first outbound file (0001, shallow to deep) does not show this behavior. This came to light after feedback from a PI on board and was missed during the 2021 QAT.

Add a swath coverage plotter feature that checks for persistent mismatches between the TX sector associated with the outermost RX beam on each side, within each swath (e.g., for only swath 1 and only swath 2, if dual-swath is enabled). Warn the user if it appears that one side is consistently unable to report outermost soundings from the expected outer TX sector number for that swath number.

Alternatively, as a first pass to help users detect this, add an option to color the soundings by TX sector or swath number; this would highlight any anomalies by making the reduced coverage trends stand out with one color that is different from the remaining 'normal' outermost coverage.

See attached images for examples of inbound coverage with missing outer stbd sector on swath 2.

swath accuracy plotter improvements

Outlining the next few steps for the swath accuracy plotter:

1. Continue separation of GUI and function libraries; wherever possible, develop swath_fun.py for consistent, shared use by swath accuracy and coverage plotters (and update swath coverage plotter as appropriate to use swath_fun.py)

2. KMALL support: finish parsing position datagrams so data can be handled in same work flow as current .all position handling (and later processing steps)

3. Add text box options with reference surface name, projection

4. Add 95% CI plots (meters, not %WD) like previous MAC reports with options to plot NOAA/IHO standards

5. Add automatic detection of UTM zone from ref surf file name. Eventually, add ref surf import in other formats.

6. Add options for ref surf filtering: sounding density, slope

7. Add options for crossline filtering: fixed interval (e.g., +/- %WD or +/- meters) from ref surf or (iteratively) from the mean within angle bins; backscatter

8. Add options to color points by mode combos (e.g., Deep/Dual/Mixed in red, Very Deep/Single/FM in blue, etc.) and legend option with mean and st. dev. lines, at a minimum

9. Add histogram plot for difference from ref surf

10. Add percentiles to binwise beam calculations

Swath coverage plotter: accommodate different depth references in .all and .kmall

EX2000 EM304 SAT files (SIS 5 .kmall) converted to .all are reading 6.8 m shallower than the native .kmall files in the swath coverage plotter; this difference is same as TX array Z offset (6.86 m down) from the mapping system origin (granite block). According to the datagram formats, the .kmall reference point is the mapping system origin (granite block) whereas the .all reference point is the TX array. The coverage plotter is simply using the XYZ88 datagram depths for .all files and MRZ depth datagrams without consideration of the different depth references.

Proposed solution:

As a starting point, reference all coverage data to sea surface, assuming this is used primarily for surface vessels. This ensures an apples-to-apples comparison of achieved coverage versus depth from sea surface, not depth from different references from different file types. This would require parsing the installation parameters datagrams to apply vertical adjustments of [origin to waterline] for .kmall files and [TX array to waterline] for .all files.

For the same reasons, recalculate swath angles using the Z from sea surface and acrosstrack distance. The RX beam angles (e.g., from RRA datagram) are currently used when available; however, the RX angles re RX array are not available for all data, and it is not clear whether Kongsberg uses this angle re RX array or a simple arctan(acrosstrack/depth) for nominal swath angles when applying runtime parameters (e.g., swath angle limits). Update all angle-based filtering in the swath coverage plotter to use simplified, consistent sea-surface-based nominal beam angles for surface vessels. Update figure titles/axes/filenames accordingly.

Once this is addressed, add an option for the user to plot achieved coverage versus depth from other references, namely the mapping system origin (granite block for EX) or TX array, rather than sea surface/waterline. This will be useful for evaluating achieved coverage in the echosounder reference frame for vessels with deep array installations relative to total depth (e.g., SailDrone with a deep keel/gondola operating in relatively shallow water) or underwater vehicles where achieved coverage versus depth from the vehicle is much more meaningful than depth from sea surface.

In any case, the nominal beam angles calculated from arctan(acrosstrack/depth) should be used in place of the parsed angles re RX array (or perhaps an option can be added to select the beam angle reference, if there is lingering uncertainty about how Kongsberg applies angle params; for instance, if an RX array is tilted a few deg like on the FH, are the user's runtime angle params referenced from / applied to the array or to local level plane?).

Add a license file

For instance, LGPL v3: https://github.com/hydroffice/hyo2_openbst/blob/master/LICENSE