pamguard / pamguard

Detection, Classification, and Localisation of marine mammal and other bioacoustic signals

This is the main code repository for the PAMGuard software.

This repository was created on 7 January 2022 from sourceforge SVN repository at https://sourceforge.net/p/pamguard/svn/HEAD/tree/ revision r6278.

If you are a PAMGuard developer, you should clone and branch this repository and share with any collaborators in your own workspace. When your work is ready, contact the PAMGuard team to have your changes merged back into this repo.

pamguard's Issues

Bug in SoundSpotClassifier, GenericDLClassifier, and KetosClassifier

private function and an overwritten function with the same name and parameters.

@OverRide
public void newResult(GenericPrediction modelResult, GroupedRawData groupedRawData) {
modelResult.setClassNameID(getClassNameIDs(genericModelParams));
modelResult.setBinaryClassification(isBinaryResult(modelResult, genericModelParams));
newResult(modelResult, groupedRawData);
}

The problem is that both functions are ambiguous, which means that in my case, newResult was calling itself endlessly. To fix this, the private function must be renamed.

Binary Database does not work with DLLocalisation

Maybe it wasn't intended to, but DLLocalisation is an option.

Building from source

Hello,

I wonder if you can help, it appears that the geomajas link in the POM file is down.
Do you know of an alternative @douggillespie ?

I am trying to build a version that has the command/udp function to work with; and it appears the latest release version doesn't, it was fixed a few months after according to the commits.

That or can we get release of a new version posted?

Thank you,

New "overwrite existing output data" option clears entire binary file folder

Recently, I noticed a new feature that brings up a dialog when the record button is pressed, giving you the option to either overwrite binary files (if any are already present in the folder) or to not continue with processing. While I think this is a great addition in theory, I just discovered that instead of simply overwriting individual binary files that happen to have the same name when a new file is processed (as was previously the case), it now apparently just deletes every binary file that was in the folder. I just lost ten days of Whistle and Moan Detector data from that, as I was trying to process different audio files from a different time frame while using the same folder.

I would really appreciate it if there was an option to do it the old way, and if it was made more clear that the new option deletes all your binary files.

Cheers.

Sound Acquisition time zone issues

My team has occasionally come across an issue where Sound Acquisition seems to automatically interpret SoundTrap audio file dates as being in local time even when UTC is selected in the options dialog, as shown in the first image where the date in the audio file name is 2022-12-01-00:00:00, but that gets converted eight hours forward as if the original was in Canada/Pacific. Apparently some of the SoundTrap data we have was accidentally processed in local time, which is probably the culprit, but I'm not sure where in the code it reads the WAV file's metadata for a time zone (I'm assuming I'm just missing something). However, if I set the time zone to Canada/Pacific, it's now at 4am instead of midnight (second image), which doesn't make any sense anyway.

Any ideas?
Holly

Jitter in time base display during wrapping

There is a jitter in the time base display spectrogram when using wrap mode. This makes the spectrogram difficult to use. The issue is due to the fact that JavaFX uses double values for pixel measurements and has been rectified in the next release.

Reference paper of bearing localization algorithm

Could you please provide some reference papers on algorithms used in the bearingLocaliser and Group3DLocaliser modules? I am very interested in them, thank you

Skip option in whistle data selector should blank options

The whistle skip option presumably means that whistles data selector options are not used. In that case the user interface should make this intuitive by disabling the options when skip is selected.

Merging multiple files when processing file folders

This was broken and was not correctly separating files and restarting if the merge contiguous files option was selected. This is fixed and will be available in the next release.

Click classificatio symbols not showing in BT display

Symbols are not being coloured properly in PAMGuard. There seems to be two issues here.

The first is the most critical where symbols are not showing in PAMGuard. An example .psfx that shows this is here

The second is more general and may be difficult to fix. PAMGuard is first opened in viewer mode the click classi
fication symbols do not show properly. For example, if porpoise clicks are red triangles then these are not visible when PAMGuard is first opened - instead the clicks appear as black dots. The symbols are fixed by deselecting and then selecting "Colour by Click Classification" in the right click menu but this is incredibly confusing for users.

Note that is appears this is an issue when opening an older database with a new version of PAMGuard and/or opening the first time after processing data in real time mode. Once fixed, upon reopening PAMGuard is fine.

Example porpoise data - when first opened the clicks should be red but are black. Note there are two databases. Porpoise_DB opens fine but Porpoise_DB_original, which was generated with an older version of PG opens the clicks with the wrong colour.

https://drive.google.com/drive/folders/1SCTFgFRcu29QmpvuK3syClh1p-XD7_QF?usp=sharing

ClickDetection version 1 localization not applied in Viewer Mode

Click localization never applied to superceded ClickDetector data unit (module version 1) containing TDOA data in viewer mode. Not checked on other superceded module versions.

TD/Fx Display Freezes with Clicks

Hi All,

This is in reference to a long time issue I've had with the TD/Fx display freezes when I have clicks in one of the panels and am logging clicks to a DGL event (as I can't do that in the click detector panel). The problem was a little better I thought when I used the amplitude - stem button, but still problematic. The freezing happens after 30 minutes or after 2 minutes and seems to do it less if I have less data loaded, although if I have more than an hour of data loaded then it freezes more often (for all other panels, I have a day of data loaded and no freezing). I've tried de-selecting some clicks, and always de-select the unclassified. It makes it pretty time consuming to assign clicks to an event and

I have a Win 10, 32 GB RAM (but PG only has access to 16 GB, and I've tried more, less, with no difference), I have had this problem on different computers (others working through data and my laptop) and with all datasets and all versions of PAMGuard since about 2019 when I started it. Jamie and I have tried trouble shooting a lot but he can't replicate the problem. I am overlaying audio if that helps (to confirm species, etc.) but not in the TD/Fx display and the problem occurs without overlay as well.

Here's a video displaying the problem (one of my tests for Mike a while back, same freezing happens though - look at min 6):
https://screencast-o-matic.com/watch/crit14ViGB1

Here's a set of test files that have had the issue occur.
https://drive.google.com/drive/folders/1kvDAGPphWr-dL_IQXwdyya0sbvWtXfwL?usp=sharing

Appreciate any help!

Bearing Calculator mirrors at 45° Angle.

The Bearing Calculation Results are mirrored on the 45° Axis.
a 270° Bearing Signal will show at the 180°
a 0° Bearing will show at 90° and so on.

Offline analysis and viewer memory leak

The new(ish) background datablocks attached to 'normal' datablocks were not getting their data cleared when the main datablock data were cleared, so were accumulating too much data and bring the system down through lack of memory.

Datamap not updating after SoundTrap click import

The datamap does not update after SoundTrap clicks have been imported. Need to add PamController.getInstance().updateDataMap() to the DWVConvertObserver at line 402 ImportBCLDialog....?

Non chronological order processing wav files.

Report by Brian Miller.

The behaviour of the Files and Folders Acquisition behaves very differently depending on weather I select just the folder name from the dialog, or whether I navigate into the folder and select all the files within within it. If I select all the files within the folder, then the behaviour seems to be consistent with prior versions of Pamguard, and the files are sorted by time/alphabetically and analysed in chronological order by the timestamp within the file name. If I select the just the folder, then the data appears to be analysed in random (non-chronological nor alphabetical) order. This random order happens regardless of whether I have checked the “Merge continuous files” checkbox or not. For my datasets, I’m not treating the files as continuous anyway – since for my instruments the filename timestamps use the more precise clock than the one in the A/D converter. But for someone who wants contiguous files merged this would likely be a problematic bug.

Add template names to matched click classifier

Template names have now been added to imported templates - the template names are the names of the files and should be available in xml settings

Hardware-Software-Channel configuration doesn't apply to bearing calculator

if you skip a hydrophone in the configuration, it is not skipped in the bearing calculator. For example, if I have 8 hydrophones and want to localize on 0,1,2,4,5,6,7 and skip 3 using the Hardware-Software-Channels the bearing calculator uses the positions of 0,1,2,3,4,5,6

ROCCA Memory leak

when unable to find raw data for a whistle, ROCCA would double the raw data keep time. Since it regularly can't find this data at the end of the file (e.g. when threading causes a whistle to arrive after the raw data are cleaned up) the keep time kept doubling until the memory usage of the raw datablock would kill PAMGuard. I've now hard limited this to 15 minutes, which is loads more than should ever be needed. A similar problem occurred in the Ishmael Localiser.

The Advanced Settings Panel for the RawDeepLearningClassifier crashes

I unfortunately cannot pinpoint the exact reason why it crashes, it doesn't log anything on the console, but instead crashed to windows with a Java(TM) Platform SE binary is unresponsive error.

Logger forms data not plotting

Logger forms data were not plotting correctly. This was due to the required data selection functions not being fully implemented. This is now fixed and it should be possible to select what to plot based on selections from drop down lists within any form.

DIFAR module crashing in recent releases (e.g. 2.02.09)

DIFAR module seems to be crashing with null pointer errors in recent versions of Pamguard. Crashes don't happen in 2.01.05, but seem to happen in more recent versions.

Recent versions of Pamguard crash or go painfully slow when processing lots of wav files

Hi,

I don't have much time to work on Pamguard code these days, but do have enough time to file a bug-report/feature regression.

I'm no longer able to effectively process large folders of wav files using the attached psf (spectrogram, Ishmael spectrogram correlation detectors, sqlite database). In 2.02.02 Pamguard just crashes hard and all windows dissapear, and the log (attached) provides no indication of what is going on. In 2.02.05 the log includes more detail, and pamguard manages to avoid a crash, but after about 1000 files starts running extremely slowly.

The last known good version of Pamguard where I could successfully complete this task was 2.02.00 (r5437).

Cheers,
Brian
PamguardLogs.zip
kerguelen2015_fullYear_Bm-Ant-ABZ_SpectroCorrelation_250Hz.zip

Migrating from SVN to Git

@douggillespie I've noticed that you've put this on GitHub - this is great! just to add that it's also possible to migrate from SVN to Git and keep all past history of revisions in one repository. Since your SVN repository is public, I don't see a reason why not to do this. Instructions are e.g. at https://git-scm.com/book/en/v2/Git-and-Other-Systems-Migrating-to-Git and we can try this before many things happening here.

"2-second glitch" in Whistle and Moan Detector

The Whistle and Moan Detector has this issue where false detections (as in contours are drawn when there isn't anything there, including in dead silence) are often produced within the first two seconds or so of sound files. This isn't a recent issue - it was happening as far back as 2020 when I started using PAMGuard. My team uses the WMD extensively, so a good chunk of our data is false detections produced by the 2-second glitch (as we like to call it). This also results in the data map in Viewer Mode having an absolute snowstorm of false markings on it, which makes it impossible to pick out minor encounters from outright false ones.

If this isn't an easy issue to outright fix, a good temporary solution could be to add an option in the WMD to not produce contours within the first two seconds of each file. That would be super helpful for us in the meantime.

Cheers.

Threading hydrophone locator bug

The threading hydrophone locator was not working correctly in the latest release V2.02.02. The problem was particularly apparent in Viewer mode but may have given false array locations in normal mode under some circumstances. This is fixed in the code and will be in the next release.

GlobalWavFolderArg flag doesn't override RecorderSettings outputFolder

using -wavfilefolder flag doesn't override output directory

Bug with whistle contour (ROCCA)

Hello,
It's impossible for me to use ROCCA classifier with some files. The whistle contour doesn't appear after selecting the beginning and the end of the whistle or with "pick points". With other files, it works very good and some it doesn't do the exact whistle and go on places where there is no whistle even with a thresold of 10.
I see that the red vertical line (filter) is not on all the lenght of the files that doesn't work, and just at the end or doesnt' appear at all.
How can I resolve the problem and pick the whistles contour from my .wav files ? Thank you.

UDP Control & Command Syntax

Hello @douggillespie

Two issues - I wonder if you can point me the right way. Firstly I appear to be unable to run a UDP command into Pamguard, its compiled OK (thanks for that), the port is open but it disconnects and closes after the first try and only a restart of the software opens it again.

For example:

Pamguard running --port 6666

Another cmd window ---> ncat -u -v localhost 6666 (I am using ncat for sending udp traffic from the nmap package)
command: ping
response: cmd ping not found (with a return/new line after the command in the response)
From then on there are no more responses to any commands & all connections are closed.
After this point the port on pamguard is closed (checked with nmap, and interestingly if you can the port that also crashes it) and it does not reopen till I restart the software. Any clues?

Secondly --- can I please ask for some more clarity on the syntax you expect for the 3 commands listed

1. tellmodule
2. getxmlsettings
3. setxmlsettings

This may be because my attempts at "getxmlsettings" thus far have only resulted in either "No settings" or "Unspecified modules for settings"

For example getxmlsettings "Sound Acquisition" just returns no settings - which seems wrong?

Part of this is (and i have tried to search but I cannot locate it) there doesn;t appear to be a clear list of available module names, some like Sound Acquisition have a different type of Data Acquisition, others like Array Manager are both the name and type...

e.g command summary produces:
"Sound Acquisition:ch0,-0.0,-10.4,-46.6,ch1,0.0,-4.6,-22.1,<\Data Acquisition>"

So I presumed the module name is - Sound Acquisition? or is it Data? or is actually just Acquisition (as the kind of root module?)

Finally how did you then envisage someone then parsing the new xml file with setxmlsettings? Something like:
setxmlsettings ? or
setxmlsettings <type name?> <systemType Class="java.lang.String" Value="Audio File"/> (with escape characters used?, just the varrying parts? or the entire file escaped?)

Anyway if you could put an example for each just to read back a default value that would be really helpful.

Thank you very much for your time,

Feature request: Connect deep learning to DIFAR module

The deep learning classifier is great, and I'd be really keen to use the detections and classifications from this module as input into the DIFAR localisation module.