tphakala / birdnet-go

Realtime BirdNET soundscape analyzer

BirdNET-Go

BirdNET-Go is an AI solution for continuous avian monitoring and identification

• 24/7 realtime bird song analysis of soundcard capture, analysis output to log file, SQLite or MySQL
• Utilizes BirdNET AI model trained with more than 6500 bird species
• Local processing, Internet connectivity not required
• Easy to use Web user interface for data visualisation
• BirdWeather.com API integration
• Realtime log file output can be used as overlay in OBS for bird feeder streams etc.
• Minimal runtime dependencies, BirdNET Tensorflow Lite model is embedded in compiled binary
• Provides endpoint for Prometheus data scraping
• Runs on Windows, Linux and macOS
• Low resource usage, works on Raspberry Pi 3 and equivalent 64-bit single board computers

Help Improve BirdNET-Go

I am seeking web developer with experience in Go HTML templating, Tailwind CSS, HTMX, Alpine.js, or similar frameworks to join in enhancing the BirdNET-Go application web UI. Your expertise can make a real difference in how users engage with this app and connect with birdlife. If you have the skills and a keen interest in contributing to a nature-focused project, I'd love to hear from you.

Web Dashboard

Installation

For detailed installation instructions, see the installation documentation.

Building

For instructions on how to build the project, see the building documentation.

Usage

BirdNET-Go CLI

Usage:
  birdnet [command]

Available Commands:
  authors     Print the list of authors
  completion  Generate the autocompletion script for the specified shell
  directory   Analyze all *.wav files in a directory
  file        Analyze an audio file
  help        Help about any command
  license     Print the license of Go-BirdNET
  realtime    Analyze audio in realtime mode

Flags:
      --debug               Enable debug output
  -h, --help                help for birdnet
      --locale string       Set the locale for labels. Accepts full name or 2-letter code. (default "finnish")
      --overlap float       Overlap value between 0.0 and 2.9
      --sensitivity float   Sigmoid sensitivity value between 0.0 and 1.5 (default 1)
      --threshold float     Confidency threshold for detections, value between 0.1 to 1.0 (default 0.8)

Use "birdnet [command] --help" for more information about a command.

There is more detailed usage documentation at wiki https://github.com/tphakala/birdnet-go/wiki

License

Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International

Authors

Tomi P. Hakala

Contributions by Hampus Carlsson, Jan Vrska, @twt--, @aster1sk, @hoover67

Please let me know if you are missing from contributors list!

BirdNET AI model by the K. Lisa Yang Center for Conservation Bioacoustics at the Cornell Lab of Ornithology in collaboration with Chemnitz University of Technology. Stefan Kahl, Connor Wood, Maximilian Eibl, Holger Klinck.

BirdNET label translations by Patrick Levin for BirdNET-Pi project by Patrick McGuire.

birdnet-go's Issues

Location Filter Threshold is fixed but should be set by User #enhancement

Edit: No Idea how I can add Labels...

Hey,

I dug up some stuff from BirdNET
It is really important that the user of BirdNET-Go can set the locationFilterThreshold for themselves!

See line 62
BirdNET-Analyzer; config.py

I think you defined it in Line 33
rangefilter.go

If the threshold is fixed (like it is right now) I can't detect certain species that are definitely present on my location.

On BirdNET-Pi (Nachtzuster Fork- but its basically the same) it looks like this:

The link from the photo
here

There is also a small species list tester- so i know where i set my threshold
For the extreme i set it to 0.95 - see Picture below

And for my location I have it set to 0.0018 so i can detect most owls in my region.
As you can see the list gets MUCH longer

This is not needed for the person that just wants to know their birds in the garden. But for people that are interested in ornithology and Enthusiasts (like me) it is Really important!

Otherwise a lot of birds don't get detected even tho they are present, calling and are in the birdnet library.

Docker image fails to upload soundscapes to BirdWeather

When posting soundscapes to BirdWeather when running the birdnet-go Docker image I get the following error -

2024/03/09 17:42:45 Request to upload soundscape failed: Post "https://app.birdweather.com/api/v1/stations/XXXXXXXXXX/soundscapes?timestamp=2024-03-09T17:42:42.000+0000": tls: failed to verify certificate: x509: certificate signed by unknown authority

I believe this can be resolved by adding the ca-certificates package to the image.

Location Filter threshold should be set by user

EDIT: I opened a new issue (it's the same) but with the right "tag"- I couldn't edit it on iPad GitHub 🙄

Still learning to use GitHub like its intended

Hey,

I dug up some stuff from BirdNET
It is really important that the user of BirdNET-Go can set the locationFilterThreshold for themselves!

See line 62
BirdNET-Analyzer; config.py

I think you defined it in Line 33
rangefilter.go

If the threshold is fixed (like it is right now) I can't detect certain species that are definitely present on my location.

On BirdNET-Pi (Nachtzuster Fork- but its basically the same) it looks like this:

The link from the photo
here

There is also a small species list tester- so i know where i set my threshold
For the extreme i set it to 0.95 - see Picture below

And for my location I have it set to 0.0018 so i can detect most owls in my region.
As you can see the list gets MUCH longer

This is not needed for the person that just wants to know their birds in the garden. But for people that are interested in ornithology and Enthusiasts (like me) it is Really important!

Otherwise a lot of birds don't get detected even tho they are present, calling and are in the birdnet library.

Documenting Web UI Usage

Hi! Fantastic project BTW!

I am setting up an instance here and I am unable to determine how to get the Web UI running. Do you have any documentation on setting this up?

Thank you.

#bug species_config.csv just works with language set to English

I set up the species config and it only works if I have birdnetgo set to English. (In config.yaml)

I tried the common names for German (inside the species config) but that doesn't change the output (NO Output)

This works:

Config.yaml 
Language = en 

Species_config.csv
BirdNames in English [ tawny owl,0.2

This does not work:

Config.yaml 
Language = de 

Species_config.csv
BirdNames in English 
(neither with German names [ waldkauz,0.2 ])

Running birdnet-go in docker on Raspberry Pi

I'm currently trying to run birdnet-go on a raspberry pi with docker.

The Pi is running a default, clean Raspbian and docker was installed with this
command curl -fsSL https://get.docker.com | sh.

Running this command:

docker run -ti \
-p 8080:8080 \
--env ALSA_CARD=0 \
--env TZ="Europe/Stockholm" \
--device /dev/snd \
ghcr.io/tphakala/birdnet-go:latest

Gives following error message, which suggests that there are no
linux/arm/v7 images for the raspberry pi architecture:

Unable to find image 'ghcr.io/tphakala/birdnet-go:latest' locally
latest: Pulling from tphakala/birdnet-go
docker: no matching manifest for linux/arm/v7 in the manifest list entries.
See 'docker run --help'.

Adding linux/arm/v7 to the docker workflow could maybe build the images for raspberry pi, but I haven't tested yet.

Is there an easier way to run birdnet-go on the pi? I haven't found documentation explicitly for the pi so I'm quite unsure.

Thanks in advance!

LocationFilterThreshold

Implemented with commit 954c681

EDIT: "I've just seen that maybe the issue is on my side, I'll get to this in the evening."

EDIT_2: It's working flawlessly! Great work and fast commit ❤️
Thank you

Ursprünglich gepostet von @MueJosh in #96 (comment)

Unfortunately it still doesn't work as it should..
I don't know where the issue could be.
My settings haven't changed, I updated the container a few days ago. (No idea how I can find out which version I'm running. It says :latest - but it's doing this all the time)

# BirdNET-Go configuration

debug: false # print debug messages, can help with problem solving
# Node specific settings
main:
    name: BirdNET-Go_03 # name of node, can be used to identify source of notes
    timeas24h: true # true for 24-hour time format, false for 12-hour time format
    log:
        enabled: true # true to enable log file
        path: birdnet.log # path to log file
        rotation: daily # daily, weekly or size
        maxsize: 1048576 # max size in bytes for size rotation
        rotationday: 0 # day of the week for weekly rotation, 0 = Sunday
# BirdNET model specific settings
birdnet:
    sensitivity: !!float 1 # sigmoid sensitivity, 0.1 to 1.5
    threshold: 0.3 # threshold for prediction confidence to report, 0.0 to 1.0
    overlap: 0.5 # overlap between chunks, 0.0 to 2.9
    threads: 4 # 0 to use all available CPU threads
    locale: en # language to use for labels
    latitude: 48.XXXX # latitude of recording location for prediction filtering
    longitude: 7.XXX0 # longitude of recording location for prediction filtering
    locationfilterthreshold: 0.0018 # rangefilter species occurrence threshold
# Realtime processing settings

Location edited out because privacy, the last digit on longitude is "0" maybe that's an issue? Idk

Add Greek birdnames

First of all: Great work & thanks for sharing your efforts!

How can I add modern greek species names to the localizations already present?

Thanks,

Uwe

Listening device mismatch in Docker

When I run arecord -l, I get:

**** List of CAPTURE Hardware Devices ****
card 1: Device [USB Audio Device], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #14 

However, when I start the Docker container, I see:

BirdNET-Go build date: 2024-05-01T17:38:41Z, using config file: /root/.config/birdnet-go/config.yaml
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
System details: linux debian 12.5 on  hardware
Starting analyzer in realtime mode. Threshold: 0.8, sensitivity: 1, interval: 15
[2024-05-03T08:05:27-04:00] [INFO] ⇨ http server started on [::]:8080
Capture Devices
    0: Discard all samples (playback) or generate zero samples (capture), 6e756c6c, 
    1: Default Audio Device, 64656661756c74, [default]
    2: Default Audio Device, 73797364656661756c74, 
    3: USB Audio Device, USB Audio, 3a312c30, 
Listening on device: Default Audio Device (device ID 64656661756c74)

I could just be misunderstanding what the logs are trying to say, or how devices on the host exist with the Docker environment, but I would expect it to be listening on the USB Audio device that matches that arecord output.

It is recording clips and when I play them back they sound find, but the detections are not correct; it's not really detecting anything. There are occasional detections, but they are all false positives, it's not picking up any of the normal birds that it would normally be detecting ever couple of minutes.

Stopped detecting

Several days ago, my SD card ran out of space so BirdNET mostly stopped working. I cleared out files to make space, but now when I start birdnet-go I get some messages that look like errors related to audio capture, which I don't remember showing up before.

Read config file: /home/birdnet/.config/birdnet-go/config.yaml
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Starting BirdNET-Go Analyzer in realtime mode
Threshold: 0.8, sensitivity: 1, interval: 15
[2024-04-29T07:32:15-04:00] [INFO] ⇨ http server started on [::]:8080
Capture Devices
    0: Discard all samples (playback) or generate zero samples (capture), 6e756c6c, [ok]
    1: Playback/recording through the PulseAudio sound server, 64656661756c74, [ok]
    2: Rate Converter Plugin Using Libav/FFmpeg Library, 6c617672617465, [miniaudio: Resource does not exist]
    3: Rate Converter Plugin Using Samplerate Library, 73616d706c6572617465, [miniaudio: Resource does not exist]
    4: Rate Converter Plugin Using Speex Resampler, 737065657872617465, [miniaudio: Resource does not exist]
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
    5: JACK Audio Connection Kit, 6a61636b, [miniaudio: Resource does not exist]
ALSA lib pcm_oss.c:397:(_snd_pcm_oss_open) Cannot open device /dev/dsp
    6: Open Sound System, 6f7373, [miniaudio: Resource does not exist]
    7: PulseAudio Sound Server, 70756c7365, [ok]
    8: Plugin using Speex DSP (resample, agc, denoise, echo, dereverb), 7370656578, [miniaudio: Resource does not exist]
    9: Plugin for channel upmix (4,6,8), 75706d6978, [miniaudio: Resource does not exist]
    10: Plugin for channel downmix (stereo) with a simple spacialization, 76646f776e6d6978, [miniaudio: Resource does not exist]
    11: USB Audio Device, USB Audio, 3a312c30, [ok]
Listening ...

It had been running perfectly since ~March 30 when I updated to that release. I've updated to the most recent birdnet-go release and updated all the system packages, but I keep getting the same messages. It's persisted through several restarts. The program is running, and the web UI works, it just never detects anything.

BirdWeather stops BirdNET-go

Only issue with Birdweather uploads is that uploaded wav files are silent, there is no way that problem covered by this issue would cause loss of any files such as birdnet-go config file.

Please file a new issue with as much details as you can provide.

Ursprünglich gepostet von @tphakala in #68 (comment)
Edit: the config.yaml doesnt get lost, i just couldn't access it when it "crashed" (birdnet-go)

If I fill the BirdWeather Id and enable BirdWeather upload, the system stops after some time.
The BirdWeather Id is fine, it works on birdnetpi (I haven't used it for a week or so)
On birdnetgo- BirdWeather doesn't show anything (yep, I waited for an hour)

Website config (set it up on website)

No output for a few minutes- restarting the console and logging back in and it works again for a few minutes.

Config.yaml

Some of the output of webui.log

[2024-04-08T18:58:45+02:00] [INFO] 172.17.0.1 GET /assets/htmx.min.js 200 <nil>
[2024-04-08T19:05:52+02:00] [INFO] ⇨ http server started on [::]:8080
[2024-04-08T19:07:29+02:00] [INFO] ⇨ http server started on [::]:8080
[2024-04-08T19:08:02+02:00] [INFO] 172.17.0.1 GET / 200 <nil>
[2024-04-08T19:08:03+02:00] [INFO] 192.168.178.23 GET / 200 <nil>
[2024-04-08T19:08:03+02:00] [INFO] 192.168.178.23 GET /assets/tailwind.css 200 <nil>
[2024-04-08T19:08:03+02:00] [INFO] 192.168.178.23 GET /assets/htmx.min.js 200 <nil>
[2024-04-08T19:08:03+02:00] [INFO] 192.168.178.23 GET /assets/custom.css 200 <nil>
[2024-04-08T19:08:03+02:00] [INFO] 192.168.178.23 GET /assets/alpinejs.min.js 200 <nil>
[2024-04-08T19:08:03+02:00] [INFO] 192.168.178.23 GET /top-birds?date= 200 <nil>
[2024-04-08T19:08:03+02:00] [INFO] 192.168.178.23 GET /last-detections?numDetections=5 200 <nil>
[2024-04-08T19:08:04+02:00] [INFO] 192.168.178.23 GET /spectrogram?clip=clips%2F2024%2F04%2Fpasser_domesticus_70p_20240408T185844Z.wav 500 code=500, message=Failed to generate spectrogram
[2024-04-08T19:08:04+02:00] [INFO] 192.168.178.23 GET /spectrogram?clip=clips%2F2024%2F04%2Fstreptopelia_decaocto_74p_20240408T184129Z.wav 200 <nil>
[2024-04-08T19:08:05+02:00] [INFO] 192.168.178.23 GET /spectrogram?clip=clips%2F2024%2F04%2Fstreptopelia_decaocto_79p_20240408T184034Z.wav 200 <nil>
[2024-04-08T19:08:06+02:00] [INFO] 192.168.178.23 GET /spectrogram?clip=clips%2F2024%2F04%2Fstreptopelia_decaocto_72p_20240408T184455Z.wav 200 <nil>
[2024-04-08T19:08:07+02:00] [INFO] 192.168.178.23 GET /spectrogram?clip=clips%2F2024%2F04%2Fstreptopelia_decaocto_74p_20240408T184053Z.wav 200 <nil>
[2024-04-08T19:10:24+02:00] [INFO] ⇨ http server started on [::]:8080
[2024-04-08T19:10:42+02:00] [INFO] 172.17.0.1 GET /settings 200 <nil>
[2024-04-08T19:10:42+02:00] [INFO] 172.17.0.1 GET /assets/tailwind.css 200 <nil>
[2024-04-08T19:10:43+02:00] [INFO] 172.17.0.1 GET /assets/custom.css 200 <nil>
[2024-04-08T19:10:43+02:00] [INFO] 192.168.178.23 GET /assets/htmx.min.js 200 <nil>
[2024-04-08T19:10:43+02:00] [INFO] 192.168.178.23 GET /assets/alpinejs.min.js 200 <nil>
[2024-04-08T19:10:54+02:00] [INFO] 172.17.0.1 POST /update-settings 302 <nil>
[2024-04-08T19:10:54+02:00] [INFO] 172.17.0.1 GET /settings?update=success 200 <nil>
[2024-04-08T19:10:54+02:00] [INFO] 172.17.0.1 GET /assets/tailwind.css 200 <nil>
[2024-04-08T19:10:54+02:00] [INFO] 192.168.178.23 GET /assets/htmx.min.js 200 <nil>
[2024-04-08T19:10:54+02:00] [INFO] 192.168.178.23 GET /assets/alpinejs.min.js 200 <nil>
[2024-04-08T19:10:54+02:00] [INFO] 172.17.0.1 GET /assets/custom.css 200 <nil>
[2024-04-08T19:10:55+02:00] [INFO] 172.17.0.1 GET / 200 <nil>
[2024-04-08T19:10:55+02:00] [INFO] 192.168.178.23 GET /assets/htmx.min.js 200 <nil>
[2024-04-08T19:10:55+02:00] [INFO] 192.168.178.23 GET /assets/alpinejs.min.js 200 <nil>
[2024-04-08T19:10:55+02:00] [INFO] 172.17.0.1 GET /assets/tailwind.css 200 <nil>
[2024-04-08T19:10:55+02:00] [INFO] 172.17.0.1 GET /assets/custom.css 200 <nil>
[2024-04-08T19:10:56+02:00] [INFO] 172.17.0.1 GET /top-birds?date= 200 <nil>
[2024-04-08T19:10:56+02:00] [INFO] 172.17.0.1 GET /last-detections?numDetections=5 200 <nil>
[2024-04-08T19:10:58+02:00] [INFO] 172.17.0.1 GET /spectrogram?clip=clips%2F2024%2F04%2Fstreptopelia_decaocto_72p_20240408T191038Z.wav 200 <nil>
[2024-04-08T19:11:00+02:00] [INFO] 172.17.0.1 GET /spectrogram?clip=clips%2F2024%2F04%2Fturdus_merula_91p_20240408T190805Z.wav 200 <nil>
[2024-04-08T19:11:00+02:00] [INFO] 192.168.178.23 GET /spectrogram?clip=clips%2F2024%2F04%2Fstreptopelia_decaocto_74p_20240408T184129Z.wav 200 <nil>
[2024-04-08T19:11:04+02:00] [INFO] 192.168.178.23 GET /spectrogram?clip=clips%2F2024%2F04%2Fpasser_domesticus_70p_20240408T185844Z.wav 500 code=500, message=Failed to generate spectrogram
[2024-04-08T19:11:16+02:00] [INFO] 192.168.178.23 GET /spectrogram?clip=clips%2F2024%2F04%2Fstreptopelia_decaocto_72p_20240408T184455Z.wav 200 <nil>
[2024-04-08T19:14:38+02:00] [INFO] 172.17.0.1 GET / 200 <nil>
[2024-04-08T19:14:38+02:00] [INFO] 172.17.0.1 GET /assets/tailwind.css 200 <nil>
[2024-04-08T19:14:38+02:00] [INFO] 172.17.0.1 GET /assets/alpinejs.min.js 200 <nil>
[2024-04-08T19:14:38+02:00] [INFO] 172.17.0.1 GET /assets/custom.css 200 <nil>
[2024-04-08T19:14:38+02:00] [INFO] 172.17.0.1 GET /assets/htmx.min.js 200 <nil>
[2024-04-08T19:14:39+02:00] [INFO] 172.17.0.1 GET /last-detections?numDetections=5 200 <nil>
[2024-04-08T19:14:39+02:00] [INFO] 172.17.0.1 GET /top-birds?date=2024-04-08 200 <nil>
[2024-04-08T19:14:44+02:00] [INFO] 172.17.0.1 GET /spectrogram?clip=clips%2F2024%2F04%2Fpasser_domesticus_80p_20240408T191350Z.wav 200 <nil>
[2024-04-08T19:14:45+02:00] [INFO] 172.17.0.1 GET /spectrogram?clip=clips%2F2024%2F04%2Fpasser_domesticus_74p_20240408T191132Z.wav 200 <nil>
[2024-04-08T19:14:45+02:00] [INFO] 172.17.0.1 GET /spectrogram?clip=clips%2F2024%2F04%2Fpasser_domesticus_73p_20240408T191252Z.wav 200 <nil>
[2024-04-08T19:19:18+02:00] [INFO] 192.168.178.23 GET / 200 <nil>

Provide pre-compiled libtensorflow_c libraries

For easier deployment I should provide pre-compiled libtensorflow_c files for at least Linux and Windows, for macOS it is available through homebrew

Feature request: mqtt publishing of detections

Could you please consider publishing bird detections to an mqtt broker?

Capture longer audio clips than 3 seconds

Currently only contents of detection buffer is saved to disk, this is only 3 seconds of audio. Method of capturing longer clips for additional context should be added.

Maybe another larger buffer for saving clips should be used?

Adding some stats to MQTT messages

Hey, I set this up the other day and it's been great so far. Feels way more modern than the others. Great job!

I enabled MQTT so I could do notifications from Home Assistant. However I could do with some more context in the messages.

What do you think about adding number of sightings in a given day to MQTT or some other stats?

I don't really want to be notified for every bird visit but maybe the first time the bird has been seen that day (or that week?).

I'd be interested in helping with this if this would be a welcome change.

Dashboard displays yesterday

When opening the web dashboard I get yesterday's data displayed and have to use the date picker to select today. Also when reloading the browser window I get yesterday (Firefox, on Windows).

I just tried Edge and Chrome, they both opened yesterday too. And they have never opened the website before so it's not a caching issue.

Timezone, time and date are set correctly on the pi that Birdnet-go is running on and the PC my web browser is running on.

Fix code scanning alert - Uncontrolled data used in path expression

Tracking issue for:

• https://github.com/tphakala/birdnet-go/security/code-scanning/3

Primative log support webui.log

Hey, githubs having some issues right now. I can fork, but their backend are down for me so I can't do any pull requests no actully see my fork. Github for vs is having a fit too. So i'll just comment here : 👍

I've added a little function to read webui.log

utils.go -->
Add import "bufio"
Plus the following function

// readWebLog reads the content of the web.log file from the root directory and returns it as a string.
// It returns an error if there is an issue opening or reading the file.
func readWebLog() (string, error) {
	// Open the web.log file
	file, err := os.Open("webui.log")
	if err != nil {
		return "", err
	}
	defer file.Close()

	var content string
	scanner := bufio.NewScanner(file)

	// Read the file line by line and append each line to the content string
	for scanner.Scan() {
		content += scanner.Text() + "\n"
	}

	// Check if there was an error during scanning
	if err := scanner.Err(); err != nil {
		return "", err
	}

	// Return the content of the file
	return content, nil
}

In routes.go

// getLogsHandler handles GET requests to the /logs endpoint.
// It reads the content of the webui.log file and renders the logs view with the content.
func (s *Server) getLogsHandler(c echo.Context) error {
	// Read the content of web.log
	logContent, err := readWebLog()
	if err != nil {
		// Return an HTTP error if there is an issue reading the file
		return echo.NewHTTPError(http.StatusInternalServerError, "Failed to read web.log: "+err.Error())
	}

	// Render the logs view and pass the logContent as data
	return c.Render(http.StatusOK, "logs", map[string]interface{}{
		"LogContent": logContent,
	})
}

In the func (s *Server) initRoutes() {
Add the route

	// Add a route to handle GET requests to the /logs endpoint
	s.Echo.GET("/logs", s.getLogsHandler)

in logs.html

{{define "logs"}}
<span class="col-span-12">
    {{if .LogContent}}
        <pre>{{.LogContent}}</pre>
    {{else}}
        It didn't work
    {{end}}
</span>
{{end}}

It's basic but it does show whats being appended to webui.log (used cat webui.log and i'm getting the same stuff)
Sorry to do it like this but i'll probs forget to do a pull request if i don't do it now. The logs.html needs some work. I did get help from gpt, been ages since i've programmed any webstuff. More used to doing sience stuff (load flow)

Recommended hardware documentation

There should be a recommended hardware documentation

• Low cost quick start setup
• Mobile or remote setup (maybe)
• Cost-effective high fidelity setup

Also recommended storage solutions (sd cards, USB drives, NAS, etc)

Add BirdNET metamodel support

BirdNET metamodel should be used to filter list of probable predictions based on geographical location and week

Some audio captures start at wrong time missing bird call completely

There is a bug in audio capture ring buffer implementation which causes some audio captures to be at wrong point in time, causing some bird calls to missing from capture all together. At the moment it seems that when this happens capture is from time before call was detected.

Issue when webui logging is false

Any time I set webserver logging to false I get some errors when loading the UI. The page still partially loads, but it's broken.

BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Starting BirdNET-Go Analyzer in realtime mode
Threshold: 0.8, sensitivity: 1, interval: 15
Logging disabled
⇨ http server started on [::]:8080
Listening ...
echo: http: panic serving [fe80::823:50f4:1d31:1884%wlan0]:55092: runtime error: invalid memory address or nil pointer dereference
goroutine 51 [running]:
net/http.(*conn).serve.func1()
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:1868 +0xb0
panic({0xc18b00?, 0x12367c0?})
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/runtime/panic.go:920 +0x26c
github.com/tphakala/birdnet-go/internal/logger.(*Logger).log(0x0, {0xcc9c4e, 0x3}, 0x0, {0xcd2ab7?, 0x20?}, {0x400019f738?, 0x40000ba0c1?, 0x400019f788?})
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/logger/logger.go:157 +0x8c
github.com/tphakala/birdnet-go/internal/logger.(*Logger).Info(...)
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/logger/logger.go:133
github.com/tphakala/birdnet-go/internal/httpcontroller.(*Server).setupCustomLogger.func1({_, _}, {{0xc16cf324375557d0, 0x97f1a7db, 0x54c0ce0}, 0x0, {0x0, 0x0}, {0x40000ba0c1, 0x1e}, ...})
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/httpcontroller/init.go:120 +0xf4
github.com/labstack/echo/v4/middleware.RequestLoggerConfig.ToMiddleware.func1.1({0xe17138, 0x4000124000})
	/home/thakala/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/request_logger.go:378 +0x6ac
github.com/labstack/echo/v4.(*Echo).ServeHTTP(0x40001ec480, {0xe0c9f0?, 0x4000108000}, 0x40002a0000)
	/home/thakala/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:669 +0x374
net/http.serverHandler.ServeHTTP({0x4000218210?}, {0xe0c9f0?, 0x4000108000?}, 0x6?)
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2938 +0xbc
net/http.(*conn).serve(0x40000f8240, {0xe0e208, 0x40002181b0})
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2009 +0x518
created by net/http.(*Server).Serve in goroutine 24
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:3086 +0x4cc
echo: http: panic serving [fe80::823:50f4:1d31:1884%wlan0]:55095: runtime error: invalid memory address or nil pointer dereference
goroutine 7 [running]:
net/http.(*conn).serve.func1()
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:1868 +0xb0
panic({0xc18b00?, 0x12367c0?})
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/runtime/panic.go:920 +0x26c
github.com/tphakala/birdnet-go/internal/logger.(*Logger).log(0x0, {0xcc9c4e, 0x3}, 0x0, {0xcd2ab7?, 0x20?}, {0x4000985738?, 0x4000307b61?, 0x4000985788?})
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/logger/logger.go:157 +0x8c
github.com/tphakala/birdnet-go/internal/logger.(*Logger).Info(...)
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/logger/logger.go:133
github.com/tphakala/birdnet-go/internal/httpcontroller.(*Server).setupCustomLogger.func1({_, _}, {{0xc16cf32443bb8ec8, 0x9ff2a8d2, 0x54c0ce0}, 0x0, {0x0, 0x0}, {0x4000307b61, 0x1e}, ...})
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/httpcontroller/init.go:120 +0xf4
github.com/labstack/echo/v4/middleware.RequestLoggerConfig.ToMiddleware.func1.1({0xe17138, 0x40000a6320})
	/home/thakala/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/request_logger.go:378 +0x6ac
github.com/labstack/echo/v4.(*Echo).ServeHTTP(0x40001ec480, {0xe0c9f0?, 0x4000272000}, 0x40001e6d00)
	/home/thakala/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:669 +0x374
net/http.serverHandler.ServeHTTP({0x40001f13e0?}, {0xe0c9f0?, 0x4000272000?}, 0x6?)
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2938 +0xbc
net/http.(*conn).serve(0x40002a2240, {0xe0e208, 0x40002181b0})
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2009 +0x518
created by net/http.(*Server).Serve in goroutine 24
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:3086 +0x4cc
echo: http: panic serving [fe80::823:50f4:1d31:1884%wlan0]:55094: runtime error: invalid memory address or nil pointer dereference
goroutine 6 [running]:
net/http.(*conn).serve.func1()
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:1868 +0xb0
panic({0xc18b00?, 0x12367c0?})
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/runtime/panic.go:920 +0x26c
github.com/tphakala/birdnet-go/internal/logger.(*Logger).log(0x0, {0xcc9c4e, 0x3}, 0x0, {0xcd2ab7?, 0x20?}, {0x4000341738?, 0x400071c0c1?, 0x4000341788?})
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/logger/logger.go:157 +0x8c
github.com/tphakala/birdnet-go/internal/logger.(*Logger).Info(...)
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/logger/logger.go:133
github.com/tphakala/birdnet-go/internal/httpcontroller.(*Server).setupCustomLogger.func1({_, _}, {{0xc16cf32443ba07ac, 0x9ff121c9, 0x54c0ce0}, 0x0, {0x0, 0x0}, {0x400071c0c1, 0x1e}, ...})
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/httpcontroller/init.go:120 +0xf4
github.com/labstack/echo/v4/middleware.RequestLoggerConfig.ToMiddleware.func1.1({0xe17138, 0x400012e0a0})
	/home/thakala/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/request_logger.go:378 +0x6ac
github.com/labstack/echo/v4.(*Echo).ServeHTTP(0x40001ec480, {0xe0c9f0?, 0x4000882000}, 0x40003ae600)
	/home/thakala/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:669 +0x374
net/http.serverHandler.ServeHTTP({0x400024c2d0?}, {0xe0c9f0?, 0x4000882000?}, 0x6?)
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2938 +0xbc
net/http.(*conn).serve(0x40002a21b0, {0xe0e208, 0x40002181b0})
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2009 +0x518
created by net/http.(*Server).Serve in goroutine 24
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:3086 +0x4cc
echo: http: panic serving [fe80::823:50f4:1d31:1884%wlan0]:55096: runtime error: invalid memory address or nil pointer dereference
goroutine 5 [running]:
net/http.(*conn).serve.func1()
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:1868 +0xb0
panic({0xc18b00?, 0x12367c0?})
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/runtime/panic.go:920 +0x26c
github.com/tphakala/birdnet-go/internal/logger.(*Logger).log(0x0, {0xcc9c4e, 0x3}, 0x0, {0xcd2ab7?, 0x20?}, {0x4000981738?, 0x4000307b31?, 0x4000981788?})
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/logger/logger.go:157 +0x8c
github.com/tphakala/birdnet-go/internal/logger.(*Logger).Info(...)
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/logger/logger.go:133
github.com/tphakala/birdnet-go/internal/httpcontroller.(*Server).setupCustomLogger.func1({_, _}, {{0xc16cf32443bf128a, 0x9ff62cf1, 0x54c0ce0}, 0x0, {0x0, 0x0}, {0x4000307b31, 0x1e}, ...})
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/httpcontroller/init.go:120 +0xf4
github.com/labstack/echo/v4/middleware.RequestLoggerConfig.ToMiddleware.func1.1({0xe17138, 0x40002128c0})
	/home/thakala/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/request_logger.go:378 +0x6ac
github.com/labstack/echo/v4.(*Echo).ServeHTTP(0x40001ec480, {0xe0c9f0?, 0x4000150000}, 0x40001e6c00)
	/home/thakala/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:669 +0x374
net/http.serverHandler.ServeHTTP({0x40001f12c0?}, {0xe0c9f0?, 0x4000150000?}, 0x6?)
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2938 +0xbc
net/http.(*conn).serve(0x40002a2120, {0xe0e208, 0x40002181b0})
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2009 +0x518
created by net/http.(*Server).Serve in goroutine 24
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:3086 +0x4cc
echo: http: panic serving [fe80::823:50f4:1d31:1884%wlan0]:55097: runtime error: invalid memory address or nil pointer dereference
goroutine 48 [running]:
net/http.(*conn).serve.func1()
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:1868 +0xb0
panic({0xc18b00?, 0x12367c0?})
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/runtime/panic.go:920 +0x26c
github.com/tphakala/birdnet-go/internal/logger.(*Logger).log(0x0, {0xcc9c4e, 0x3}, 0x0, {0xcd2ab7?, 0x20?}, {0x4000985738?, 0x40000ba0f1?, 0x4000985788?})
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/logger/logger.go:157 +0x8c
github.com/tphakala/birdnet-go/internal/logger.(*Logger).Info(...)
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/logger/logger.go:133
github.com/tphakala/birdnet-go/internal/httpcontroller.(*Server).setupCustomLogger.func1({_, _}, {{0xc16cf32449135e40, 0xa54a7838, 0x54c0ce0}, 0x0, {0x0, 0x0}, {0x40000ba0f1, 0x1e}, ...})
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/httpcontroller/init.go:120 +0xf4
github.com/labstack/echo/v4/middleware.RequestLoggerConfig.ToMiddleware.func1.1({0xe17138, 0x40002120a0})
	/home/thakala/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/request_logger.go:378 +0x6ac
github.com/labstack/echo/v4.(*Echo).ServeHTTP(0x40001ec480, {0xe0c9f0?, 0x40001081c0}, 0x40002a0100)
	/home/thakala/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:669 +0x374
net/http.serverHandler.ServeHTTP({0x40009d2030?}, {0xe0c9f0?, 0x40001081c0?}, 0x6?)
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2938 +0xbc
net/http.(*conn).serve(0x40002a2090, {0xe0e208, 0x40002181b0})
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2009 +0x518
created by net/http.(*Server).Serve in goroutine 24
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:3086 +0x4cc
echo: http: panic serving [fe80::823:50f4:1d31:1884%wlan0]:55098: runtime error: invalid memory address or nil pointer dereference
goroutine 55 [running]:
net/http.(*conn).serve.func1()
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:1868 +0xb0
panic({0xc18b00?, 0x12367c0?})
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/runtime/panic.go:920 +0x26c
github.com/tphakala/birdnet-go/internal/logger.(*Logger).log(0x0, {0xcc9c4e, 0x3}, 0x0, {0xcd2ab7?, 0x20?}, {0x4000985738?, 0x40000ba121?, 0x4000985788?})
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/logger/logger.go:157 +0x8c
github.com/tphakala/birdnet-go/internal/logger.(*Logger).Info(...)
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/logger/logger.go:133
github.com/tphakala/birdnet-go/internal/httpcontroller.(*Server).setupCustomLogger.func1({_, _}, {{0xc16cf3244e0df848, 0xaa451265, 0x54c0ce0}, 0x0, {0x0, 0x0}, {0x40000ba121, 0x1e}, ...})
	/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/httpcontroller/init.go:120 +0xf4
github.com/labstack/echo/v4/middleware.RequestLoggerConfig.ToMiddleware.func1.1({0xe17138, 0x4000212140})
	/home/thakala/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/request_logger.go:378 +0x6ac
github.com/labstack/echo/v4.(*Echo).ServeHTTP(0x40001ec480, {0xe0c9f0?, 0x40001082a0}, 0x40002a0200)
	/home/thakala/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:669 +0x374
net/http.serverHandler.ServeHTTP({0x40009d21e0?}, {0xe0c9f0?, 0x40001082a0?}, 0x6?)
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2938 +0xbc
net/http.(*conn).serve(0x40001f6240, {0xe0e208, 0x40002181b0})
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2009 +0x518
created by net/http.(*Server).Serve in goroutine 24
	/home/thakala/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:3086 +0x4cc

Fix all golangci-lint issues

I would like to see all linting issues resolved such that the linting gate can actually be useful. I could probably go through and fix most of them quite quickly. However, I wonder which strategy should be used to deal with "unused" code such as variables and in some cases empty branches.

Unusued variables
For the unused variables etc, should we just go ahead and remove them? Or should we comment them out? I would prefer removing them since we can always go back in git history and bring them back if desired, while commenting bloats the code...

Empty branches
From a very quick look, many of these appears to be commented out logging. For example this file in internal/analysis/realtime.go:

func closeDataStore(store datastore.Interface) {
	if err := store.Close(); err != nil {
		//logger.Error("main", "Failed to close database: %v", err)
	} else {
		//logger.Info("main", "Successfully closed database")
	}
}

Do we want to enable the logging again?

These are currently all linting issues:

internal/httpcontroller/init.go:63:4: ineffectual assignment to err (ineffassign)
                        err = s.Echo.StartAutoTLS(":" + settings.WebServer.Port)
                        ^
root@601846a86cb9:/workspaces/BirdNET-Go# golangci-lint run       
internal/myaudio/buffers.go:35:18: Error return value of `ringBuffer.Write` is not checked (errcheck)
        ringBuffer.Write(data)
                        ^
internal/myaudio/buffers.go:85:16: Error return value is not checked (errcheck)
                                ProcessData(bn, data, startTime)
                                           ^
internal/myaudio/capture.go:53:23: Error return value of `malgoCtx.Uninit` is not checked (errcheck)
        defer malgoCtx.Uninit()
                             ^
internal/myaudio/capture.go:135:19: Error return value of `device.Stop` is not checked (errcheck)
        defer device.Stop()
                         ^
cmd/root.go:29:12: Error return value is not checked (errcheck)
        setupFlags(rootCmd, settings)
                  ^
internal/birdnet/rangefilter.go:230:6: func `mergeSpeciesLists` is unused (unused)
func mergeSpeciesLists(list1, list2 []string) []string {
     ^
internal/analysis/processor/actions.go:37:2: field `pcmData` is unused (unused)
        pcmData      []byte
        ^
internal/analysis/processor/execute.go:18:5: var `speciesActionsMap` is unused (unused)
var speciesActionsMap map[string]SpeciesActionConfig
    ^
internal/analysis/processor/processor.go:38:2: field `pcmDataExt` is unused (unused)
        pcmDataExt []byte
        ^
internal/httpcontroller/init.go:93:2: S1000: should use for range instead of for { select {} } (gosimple)
        for {
        ^
internal/myaudio/process.go:63:2: S1021: should merge variable declaration with assignment on next line (gosimple)
        var elapsedTime time.Duration
        ^
internal/mqtt/mqtt.go:54:5: S1002: should omit comparison to bool constant, can be simplified to `!c.internalClient.IsConnected()` (gosimple)
        if c.internalClient.IsConnected() == false {
           ^
internal/observation/observation.go:97:11: printf: fmt.Sprintf format %.1f has arg note.BeginTime of wrong type time.Time (govet)
                line := fmt.Sprintf("%d\tSpectrogram 1\t1\t%s\t%.1f\t%.1f\t0\t15000\t%s\t%s\t%.4f\n",
                        ^
internal/observation/observation.go:157:11: printf: fmt.Sprintf format %f has arg note.BeginTime of wrong type time.Time (govet)
                line := fmt.Sprintf("%f,%f,%s,%s,%.4f\n",
                        ^
internal/httpcontroller/init.go:63:4: ineffectual assignment to err (ineffassign)
                        err = s.Echo.StartAutoTLS(":" + settings.WebServer.Port)
                        ^
internal/analysis/processor/processor.go:199:4: SA9003: empty branch (staticcheck)
                        if p.Settings.Debug {
                        ^
internal/analysis/realtime.go:138:9: SA9003: empty branch (staticcheck)
        } else {
               ^
internal/analysis/realtime.go:136:2: SA9003: empty branch (staticcheck)
        if err := store.Close(); err != nil {

Add directory command for analysing all files in a dir

directory command should be added for analysing all sound files within directory

examample

birdnet directory /data/sounds

Import observations from BirdNET-Pi

This may already exist, but I couldn't find a mention. It'd be great to be able to bring over an export of observations from a BirdNET-Pi database

Fix file analysis code

File analysis code is currently broken and file analysis is removed from command line options. Code should fixed and cli options restored.

Add youtube as source for realtime analysis

With this library youtube should be easily added as source for realtime analysis
https://github.com/kkdai/youtube

example

birdnet realtime --youtube http://yoube.com/?v=foo

Feature to override default config file

I'm using kubernetes for running birdnet-go and find it convenient to have all configuration files etc in a separate repository, following the gitops strategy with argocd. One issue I am having currently is the config file. I'm quite happy with the defaults of it and would only like to change certain values. However, right now it is all or nothing, meaning that I have to put it all inside a large configmap, then mount it into the container filesystem.

Would it be possible to add some sort of feature for those who are happy with the default configuration, to override only certain fields of it? For example, lets say I would only want to change webserver port, then maybe a config-override.yaml could contain these two fields:

webserver:
  port: 80

which then would keep the default values, only changing the port to 80 instead of 8080

Support mp3 and flac input file types

Currently only WAV input file type is supported, should add support for mp3 and flac files also

Permission error when running as systemd service

This could be something very obvious that I'm missing, I don't deal with systemd all that often.

My system has a user called birdnet. I have created the following systemd service

[Unit]
Description=BirdNET-Go realtime analysis
After=network.target
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=5
User=birdnet
ExecStart=/home/birdnet/birdnet-go/birdnet-go realtime

[Install]
WantedBy=multi-user.target

When I try to start that service, I get

Feb 27 13:30:43 birdnet systemd[1]: Started birdnet-go.service - BirdNET-Go realtime analysis.
Feb 27 13:30:43 birdnet birdnet-go[110182]: Read config file: /home/birdnet/.config/birdnet-go/config.yaml
Feb 27 13:30:43 birdnet birdnet-go[110182]: BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Feb 27 13:30:43 birdnet birdnet-go[110182]: Starting BirdNET-Go Analyzer in realtime mode
Feb 27 13:30:43 birdnet birdnet-go[110182]: Threshold: 0.8, sensitivity: 1, interval: 15
Feb 27 13:30:43 birdnet birdnet-go[110182]: 2024/02/27 13:30:43 rename webui.log webui.log.20240227133043: permission denied
Feb 27 13:30:43 birdnet systemd[1]: birdnet-go.service: Main process exited, code=exited, status=1/FAILURE
Feb 27 13:30:43 birdnet systemd[1]: birdnet-go.service: Failed with result 'exit-code'.

I have tried also moving the installation to /opt/birdnet-go and checked permissions on all relevant files and directories. It seems like no matter what I do, running the program as the birdnet user directly always works, and running it via systemd as the birdnet user always fails with this renaming error. Also notable is that it seems to attempt to rename the webui.log file, even if there is no existing webui.log file in the same directory as birdnet-go.

I am able to work around this by providing an absolute path for the webserver log path in config.yaml. I think I would generally assume that the paths specified in the config are always relative to the program binary, but maybe that's a bad assumption.

Once I get past the renaming error, I get a new error:

Feb 27 13:43:36 birdnet systemd[1]: Started birdnet-go.service - BirdNET-Go realtime analysis.
Feb 27 13:43:36 birdnet birdnet-go[110371]: Read config file: /home/birdnet/.config/birdnet-go/config.yaml
Feb 27 13:43:36 birdnet birdnet-go[110371]: BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
Feb 27 13:43:36 birdnet birdnet-go[110371]: Starting BirdNET-Go Analyzer in realtime mode
Feb 27 13:43:36 birdnet birdnet-go[110371]: Threshold: 0.8, sensitivity: 1, interval: 15
Feb 27 13:43:37 birdnet birdnet-go[110371]: [2024-02-27T13:43:37-05:00] [INFO] ⇨ http server started on [::]:8080
Feb 27 13:43:37 birdnet birdnet-go[110371]: ALSA lib pcm_dsnoop.c:566:(snd_pcm_dsnoop_open) unable to open slave
Feb 27 13:43:37 birdnet birdnet-go[110371]: ALSA lib pcm_dsnoop.c:566:(snd_pcm_dsnoop_open) unable to open slave
Feb 27 13:43:37 birdnet birdnet-go[110371]: ALSA lib pcm_dsnoop.c:566:(snd_pcm_dsnoop_open) unable to open slave
Feb 27 13:43:37 birdnet birdnet-go[110371]: 2024/02/27 13:43:37 Device init failed miniaudio: Failed to open backend device
Feb 27 13:43:37 birdnet systemd[1]: birdnet-go.service: Main process exited, code=exited, status=1/FAILURE
Feb 27 13:43:37 birdnet systemd[1]: birdnet-go.service: Failed with result 'exit-code'.

I don't have a good idea for how to deal with that one.

But the tl;dr is that something seems to be handled differently when running as a user versus running as a user via systemd, which leads to a number of problems.

Enhancing RTSP Audio Stream Analysis in BirdNET-Go

Background

Currently, BirdNET-Go supports only a single RTSP stream. There is a need to develop a solution that allows for the support of multiple streams.

Things to Consider

1. Architecture for Supporting Multiple Streams

• Single vs. Multiple Processes: Determine whether a single BirdNET-Go process should handle multiple streams or if this functionality should be distributed across multiple BirdNET-Go instances, each updating a single database.

2. Audio Processing Strategy

• Downmixing vs. Individual Analysis: If opting for a single process approach, should audio from multiple RTSP streams be downmixed into a single stream for processing, or should each stream be analyzed individually? This decision involves trade-offs with simplicity, performance, and detection accuracy.

3. Performance and Limitations

• Inference Performance: With the potential increase in audio streams, BirdNET's inference performance must be carefully considered. The number of streams that can be effectively supported is limited by the hardware used by the user.

4. User Interface Changes

• User Interface Changes: What changes are needed for the Web UI in the case of multiple audio stream sources?

5. Implementation of RTSP Stream Analysis

• Native Go vs. FFmpeg Approach: Assess whether RTSP stream analysis should be implemented natively in Go, moving away from the current reliance on FFmpeg. FFmpeg has proven reliable and supports a wide range of audio codecs, Go has a limited set of audio libraries available for audio decoding.

This issue aims to gather feedback, suggestions, and potential contributions from the community to address these considerations comprehensively.

Implement audio file purging

Old wav files should be purged once disk space is starting to get low, purging should be flexible so that users can select which species recordings should be deleted last.

Multiple nodes, single MySQL database

Thanks for your hard work on this project.

I have setup a single MySQL database with 2 BirdNet-Go nodes sending their detections to that database. I noticed that there's a source_node field in the database so thought the web interface for each node would show only their respective detections, however the web interface on both nodes shows all detections, along with broken spectrograms for detections from the other node as the wav/png file doesn't exist on that node.

Should the web interface show only that nodes detections, or is it meant to show detections from all nodes?

look for custom species threshold file in default config paths

custom species threshold file should be locatable with main config file

Add CONTRIBUTING.md documentation

I should write contributing instructions, here are at least some topics which should be covered

Topics to Include in CONTRIBUTING.md

1. Introduction

• Brief overview of the project.
• The importance of contributions and how they can help.
• Encourage contributions of all kinds (code, documentation, ideas, etc.).

2. Code of Conduct

• Link to the project's Code of Conduct.
• Emphasize the importance of respectful and inclusive interactions.

3. Getting Started

• Instructions on setting up a development environment, preferred OS etc.
• Steps to install the Go compiler and any other prerequisites.

4. How to Contribute

• Steps to fork the repository and clone it locally.
• How to set up the development branch and keep it in sync with the main project.
• Guidelines for creating a new feature or fixing a bug (branch naming, coding standards, etc.).

5. Submitting Changes

• Instructions on how to submit a pull request.
• The process project uses to review and merge pull requests.
• What information to include in a pull request (e.g., descriptions of changes, screenshots for UI changes, etc.).

6. Coding Guidelines

• Coding standards specific to Go and project's style (formatting, variable naming, etc.).
• Any linting or formatting tools contributors should use.

7. Testing

• How to write tests for the new features or bug fixes.
• Running existing tests to ensure contributions don't break anything.

8. Documentation

• Guidelines for updating or adding documentation.
• Importance of comments in code for complex logic.

9. Community and Communication

• How contributors can ask for help or discuss features (e.g., GitHub issues, Discord, mailing list, etc.).
• Encourage updates or discussions on ongoing work to avoid duplication of effort.

10. Acknowledgment of Contributors

• How the project recognizes contributors (e.g., listing in the README, a contributors file, etc.).

11. Where to Find More Help

• Additional resources or documentation.
• External links that might be helpful for understanding Go, Linux development environment setup, etc.

Fix WAV encoding of BirdWeather uploads

WAV files uploaded to BirdWeather are incorrectly encoded resulting non-working audio clip uploads

Refactor spectrogram generation to improve Web UI responsiveness

Current implementation for generating spectrograms causes delays in Web UI navigation, this could be improved.

Connecting to non-existing rtsp stream spawns lots of ffmpeg processes

Got around to trying out the rtsp streaming feature and it works really well... Once you have filled in the correct url!

The scenario can be illustrated by starting the server while specifying a rtsp server that does not exist:

realtime --rtsp rtsp://madeupserverthatdoesnotexist.com:8554/mystream --rtsptransport tcp

Once started, the server will try to reconnect over and over again. Each time spawning a new ffmpeg process.

Console output snippet

2024/03/18 20:25:15 Starting ffmpeg with command:  /usr/bin/ffmpeg -rtsp_transport tcp -i rtsp://madeupserverthatdoesnotexist.com:8554/mystream -loglevel error -vn -f s16le -ar 48000 -ac 1 pipe:1
Restarting audio capture
2024/03/18 20:25:16 Error reading from ffmpeg: EOF
2024/03/18 20:25:16 Starting ffmpeg with command:  /usr/bin/ffmpeg -rtsp_transport tcp -i rtsp://madeupserverthatdoesnotexist.com:8554/mystream -loglevel error -vn -f s16le -ar 48000 -ac 1 pipe:1
2024/03/18 20:25:16 Error reading from ffmpeg: EOF
Restarting audio capture
2024/03/18 20:25:16 Starting ffmpeg with command:  /usr/bin/ffmpeg -rtsp_transport tcp -i rtsp://madeupserverthatdoesnotexist.com:8554/mystream -loglevel error -vn -f s16le -ar 48000 -ac 1 pipe:1
2024/03/18 20:25:16 Error reading from ffmpeg: EOF
Restarting audio capture
2024/03/18 20:25:16 Starting ffmpeg with command:  /usr/bin/ffmpeg -rtsp_transport tcp -i rtsp://madeupserverthatdoesnotexist.com:8554/mystream -loglevel error -vn -f s16le -ar 48000 -ac 1 pipe:1
2024/03/18 20:25:17 Error reading from ffmpeg: EOF
Restarting audio capture
2024/03/18 20:25:17 Starting ffmpeg with command:  /usr/bin/ffmpeg -rtsp_transport tcp -i rtsp://madeupserverthatdoesnotexist.com:8554/mystream -loglevel error -vn -f s16le -ar 48000 -ac 1 pipe:1
2024/03/18 20:25:17 Error reading from ffmpeg: EOF
...
... Just keeps going...
...

Htop view

The processes will very quickly pile up and bring your computer down (my poor old laptop server :( ). I'm now setting a cpu limit for BirdNet-Go in my deployments to be on the safe side.

I have tried with your most recent change and the behavior is the same there too.

panic: runtime error: invalid memory address or nil pointer dereference

When running birdnet-go using "birdnet-go realtime" I get the following error. I've documented the steps of my installation here, in case I missed something: #82 (comment)

Any help would be really appreciated! Thanks!

PS: It is btw weird that it says 'locale not set' as I did set them to German (de)...

birdnet-go realtime
Read config file: /home/username/.config/birdnet-go/config.yaml
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
BirdNET locale not set, using English as default
Starting BirdNET-Go Analyzer in realtime mode
Threshold: 0.8, sensitivity: 1, interval: 15
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x9b5698]

goroutine 1 [running]:
github.com/tphakala/birdnet-go/internal/analysis.RealtimeAnalysis(0x40001ec240)
/home/thakala/go/src/github.com/tphakala/birdnet-go/internal/analysis/realtime.go:47 +0x1f8
github.com/tphakala/birdnet-go/cmd/realtime.Command.func1(0x40001e4e00?, {0xcca26a?, 0x4?, 0xcca1a2?})
/home/thakala/go/src/github.com/tphakala/birdnet-go/cmd/realtime/realtime.go:20 +0x20
github.com/spf13/cobra.(*Command).execute(0x400011cf00, {0x54f6ba0, 0x0, 0x0})
/home/thakala/go/pkg/mod/github.com/spf13/[email protected]/command.go:983 +0x840
github.com/spf13/cobra.(*Command).ExecuteC(0x400011cc00)
/home/thakala/go/pkg/mod/github.com/spf13/[email protected]/command.go:1115 +0x344
github.com/spf13/cobra.(*Command).Execute(0x40000466e8?)
/home/thakala/go/pkg/mod/github.com/spf13/[email protected]/command.go:1039 +0x1c
main.main()
/home/thakala/go/src/github.com/tphakala/birdnet-go/main.go:33 +0xd8

Features to add from bird weather

Hey,

Just some ideas, they might have already been added or put in the sin bin.
Having a quick look at Birdweather API, there's some pretty nifty stuff.

As your stated, you mainly want this app to be local. Maybe a toggle could be added to grab stuff from external sources?

Going back to the API, a nifty thing would be to add the probabilty of said bird to the detections. i.e I sometimes get a false positive of the green sandpiper, I don't think at 1250m in the snow this is all too probable. But there is a 5 percent chance that at my location, one could be seen etc...
One could add weather, or even other sightings.

If it's a nay, pin this so others don't ask.

Cheers,

Automatic config reload on config change

Config files should be automatically reloaded on change

Fix code scanning alert - Uncontrolled data used in path expression

Tracking issue for:

• https://github.com/tphakala/birdnet-go/security/code-scanning/2

Highlight rare bird identifications

Implement a user configurable list of rare birds and highlight detections for species matching this list

Switch flag parsing to spf13/cobra

Added features require more flags and using golang standard flag parser becomes cumbersome, I should switch to spf13/cobra instead.

Few notes for parsing model

File analysis mode
birdnet file input.wav --threshold=0.8 --sensitivity=1.25 --locale=fi

Realtime analysis mode
birdnet realtime --threshold=0.9 --logpath=/var/log/birdnet.log --savepath=./audioclips/ --locale=fi

Support exporting mp3 and flac clips

Currently only format for sound clip exports is WAV, support for mp3 and flac should be added

Add resampling for non 48000 Hz input files

Input files which do not match required 48000 Hz resolution should be resampled internally

add to dockerhub

Good morning. I am watching this project carefully because I have been a birdnet-pi user for several months. I have synology nas with docker manager and I would like to test birdNET-go. Is there any plan to make a deposit on docker hub? thank you in advance for your development

Loic

Add rtsp source support for realtime analysis

Realtime analysis should support RTSP streams as source for audio data

example

birdnet realtime --rtsp rtsp://user@pass@url

HomeAssistant Add-on

There is demand for HomeAssistant Add-on

Tasks to achieve this:

• Add HomeAssistant dependencies to Docker image build
• What else?

Help with birdweather token

Hi, it may be a dumb question but : where can I find birdweather access token ?