bluecherrydvr / bluecherry-apps

Bluecherry surveillance system (server application)

Building from source

Currently building from source requires running pre-configured scripts to generate some of the required files for a build. In the past we used chroots for build environments, however it is possible to build for your specific distributions using the following command:

In the root of bluecherry-apps run the following command.

scripts/build_pkg_native.sh

Example:

root@buildtest:~/git/bluecherry-apps# scripts/build_pkg_native.sh

Please do not use 'dh' specific commands, the scripts generate several files (version.h for example) and the Debian control files for you.

Dependencies (server):

• mysql (debian: libmysqlclient-dev)
• libbsd (debian: libbsd-dev)
• php (debian: php5-dev)
• udev (debian: libudev-dev)

Dependencies (libav):

• yasm
• alsa-lib (debian: libasound2-dev)

Bundled libraries:

• libconfig
• libav

---

bluecherry-apps's Issues

Remove RTSP path as a required field

Needed for MJPEG only support

Fix timezone issues

http://community.bluecherrydvr.com/responses/time-in-event-viewer-is-out-by-1-hour

Web: Cycle cameras on live view layout

web: impossible to rename a camera with an empty name

name field collapses, so there is no way to rename the camera if the name was removed once

http://improve.bluecherrydvr.com/issues/1216

Add support for audio during client live view

http://improve.bluecherrydvr.com/issues/560

Investigate using Pulse for analog audio recording

Analog audio recordings (from solo devices) are currently done with the ALSA API, which seems to conflict with PulseAudio's desire to have control of all devices (snd_pcm_open returns EBUSY). I suspect that the problems with the current code run deeper than that, and from a casual glance, Pulse may match our recording pattern better anyway.

Look at using Pulse rather than ALSA for audio capture; if that fails, we must find a way to get ALSA recording working regardless of Pulse.

http://improve.bluecherrydvr.com/issues/1070

Experiment with flag thresholds to reduce false motion events

Currently, any frame flagged as having motion by the driver will begin a motion event. The threshold required to trigger a flag is configurable, but this still results in a huge number of false positive motion events, often generated by very brief movement like noise or shadows. Most actual motion will be much more consistent, and last over a period longer than a single frame.

Implement a server-level threshold, where a motion event is started only after a number of frames have been flagged within a certain period. This number should be fairly low.

One suggested approach for the threshold is to trigger only when at least N% of frames in the past X seconds contain motion, where N is a low tunable value, and X is a short period of time (generally between .25 and 2 seconds?). A fraction of frames within period X should be used over a static count to avoid false negatives on low framerate devices.

http://improve.bluecherrydvr.com/issues/955

Delete untracked files from recording paths during cleanup

When cleanup is needed for a recording path, we should (after #1062) delete files that aren't tracked in the database. Aside from avoiding non-empty directories during configuration, we should consider only deleting files that look like they were created by this software (e.g. following the media folder/filename pattern). The goal here is to avoid losing track of media files - for example, during a clean reinstallation - and filling up recording paths with unnecessary and invisible files.

http://improve.bluecherrydvr.com/issues/1063

Support IP cameras that only provide MJPEG streams

IP camera recording threads freezing up

There have been several reports of recording threads for IP cameras (particularly the AXIS m3011?) freezing, which makes the camera silently stop streaming and recording until the server is manually restarted.

We need to detect frozen recording threads by tracking their activity from the main thread, and should warn loudly if any are unresponsive. I'm not sure what we can do to respond to unresponsive threads, though - we can't terminate the thread, and starting a new one for the same device seems like a bad idea. The best solution I can think of is to kill the process, so it will be restarted and can resume recording - but that is a bit of a nuclear option.

http://improve.bluecherrydvr.com/issues/1153

Transcoding CGI script

We're going to need a CGI script to transcode a source (or pass-thru if the source is MJPEG already) from bc-server to an HTTP MJPEG feed.

It is going to be a replacement for www/media/mjpeg.php.

If it's for me, it can be written in any language; PHP, Python, whatever, even C. It just has to do the job.

It doesn't need to be identical in usage to mjpeg.php, it could even require a full source URL to be passed via POST; or it could support only bc-server from localhost, whatever you feel is simpler.

Restarting bluecherry service rejects license key.

I have a customer with the following setup:

• No internet access at all.
• eth0 was working, power problem took out the ethernet port so a new ethernet card was installed which is now named eth1
• eth1 is setup for DHCP, even though there isn't a DHCP server active
• eth1:100 (virtual interface) is setup with an IP of 192.168.0.100 with a gateway of 192.168.0.1 (which doesn't exist), IP cameras are on 192.168.0.200-208.

I was able to connect internet to the machine and reset the license key on the server and deleted and reinstalled the same key on the v2 installation and the key was accepted. As soon as I restarted the bluecherry service, it popped up this error:

Mar 15 09:48:43 dvr bc-server31106: F(): Authorization failed for license WRT2-WYDJ-66UO-SYCJ. The machine ID may have changed, which requires manual reauthorization.

I then deleted the key, reset it on our side, then did a manual authorization, installed the machine ID and the license key was accepted. The bluecherry service worked and I then restarted the bluecherry service and the same error populated again.

http://improve.bluecherrydvr.com/issues/1205

Export configuration settings

Add afterId parameter to events/ request

Adding afterId parameter will allow us to only download new events on the client and add them to existing event view (very usefull in main window).
That will help users with lots of events:
http://community.bluecherrydvr.com/responses/problem-with-client-events-201

http://improve.bluecherrydvr.com/issues/1186

Web: Support low bandwidth mode

A option in live view should enable low bandwidth mode where only 1 or 2FPS is pulled from the server.

RTSP streaming stops working after several hours, returns error, recordings still work

After several hours the RTSP live streaming dies, recording to disk still works (which is odd), MJPEG live feed also works.

Mar 20 19:13:30 mrbdvr1 bc-server[1060]: E(3/Duke's): Error writing to live stream: Invalid argument
Mar 20 19:13:30 mrbdvr1 bc-server[1060]: E(2/Gym): Error writing to live stream: Invalid argument
Mar 20 19:13:30 mrbdvr1 bc-server[1060]: E(1/Kitchen): Error writing to live stream: Invalid argument
Mar 20 19:13:30 mrbdvr1 bc-server[1060]: E(3/Duke's): Error writing to live stream: Invalid argument
Mar 20 19:13:30 mrbdvr1 bc-server[1060]: E(4/Grand Hall 1): Error writing to live stream: Invalid argument
Mar 20 19:13:30 mrbdvr1 bc-server[1060]: E(3/Duke's): Error writing to live stream: Invalid argument
Mar 20 19:13:30 mrbdvr1 bc-server[1060]: E(2/Gym): Error writing to live stream: Invalid argument

http://improve.bluecherrydvr.com/issues/1207

Invalid handling of too many records in Events results

From @vogel

Fix for bug #1015 disallows sending more than 15000 event records from server to clients.
This can be overriden by adding limit=25000 parameter to URL request.

But I think that sending even more events should not be a problem as long as we dont save all events in memory before sending them.
This line:
$events = data::query($query);
executes this:
$db->fetchAll()
which loads all results into memory as an array.

Using http://php.net/manual/en/mysqli-result.fetch-row.php and sending records one by one will allow us to send as many results as we want.

Delete events after a video is deleted off the filesystem

Events remain in the 'event list' even if the video is long deleted.

http://community.bluecherrydvr.com/responses/why-are-old-recordings-events-still-shown-if-they-have-been-overwritten

Add ids parameter to events/ request

ids parameter should be added to events request in following format:

ids=1,2,3,4

This will allow client to download only given subset of events.
For example:
client already displays 10 000 events, 20 of them are 'in progress'
client asks for newsets events using "afterId" parameter
client adds these events to display list
client asks for 20 'in progress' events using ids paremter
client updates these events in display list
Then we will have nicely working events lists that updated only neccessary items and does not loose selection.

http://improve.bluecherrydvr.com/issues/1187

Ability to set duration time for video splitting

Currently the video files are cut and restarted every 15 minutes, customers want the ability to change that

Support for SMART / software raid status changes

Add support for a utility to run and detect if the drive(s) used for storage are SMART capable (most are) or software raid and check the status of the drive every X amount of minutes. If an error or problem is detected report that to the event log and mark it as critical. It might also be useful to send an email to the system administrator notifying them of the status.

Change permissions to Bluecherry defaults when storage path is added

Currently users have to manually change permissions when storage paths are added. We need a wrapper that checks the path, and runs a command (sudo without a password for the wrapper?) that changes permissions to bluecherry.bluecherry and 755 for the storage path.

@ruminsam Is this added in your disk management work already?

RTSP client streaming problems (IP Cam Viewer)

Client reports adding a generic RTSP url in IP Cam Viewer (http://www.bluecherrydvr.com/2013/09/accessing-your-cameras-from-third-party-applications-or-mobile-devices/) causes the feed to crash and the server feed locks up

Remove one second wait cycles for cameras during startup.

Currently there is a fixed one second pause that occurs between each camera when the server starts up. This means the cameras are staggered by one second, which is fine for a couple cameras, but if you have 40 cameras on a server the last one start recording for forty seconds.

Terminate / ban remote client connections

Currently we support banning web client connections but we do not support disconnecting RTP streams.

Add mysql database table checking / repair crashed tables

A mysql 'crashed' table can make for a very bad day, and the end user won't know bad things are happening unless they look in the logs or notice nothing is being recorded:

Jan 16 09:59:17 BigBrother bc-server[924]: (SQL ERROR): [INSERT INTO EventsCam (time,level_id,device_id,type_id,length,media_id) VALUES(1326733147,'warn',13,'motion',-1,10837154)] => Table './bluecherry/EventsCam' is marked as crashed and last (automatic?) repair failed

Jan 16 09:59:17 BigBrother bc-server[924]: (SQL ERROR): [INSERT INTO EventsCam (time,level_id,device_id,type_id,length,media_id) VALUES(1326733157,'alrm',13,'not found',-1,NULL)] => Table './bluecherry/EventsCam' is marked as crashed and last (automatic?) repair failed

ID column in AvailableDevices is not reset

Currently this column is not used in the web, since ids are changed each time bc-detect runs.

If IDs are not used in the server we need to remove the column or truncate the table instead of deleting old records.

http://improve.bluecherrydvr.com/issues/1018

Server: Recording during motion detection stops and ignores post record

As noted here: http://community.bluecherrydvr.com/responses/motion-detection-seems-inconsistent#comment-2665

http://improve.bluecherrydvr.com/issues/1206

Remove RTSP path as a required field in the database

http://improve.bluecherrydvr.com/issues/1199

Change wording on where to purchase licenses

Currently the license warning says this:

bc-server994: E: [licensing] The 30 day trial of Bluecherry has expired. You can buy a license from http://www.bluecherrydvr.com

It should say purchase from:

http://store.bluecherry.net/licenses

Crash when MJPEG RTSP feed is detected from ACTi cameras

Unknown if this crash is specifically related to ACTi's MJPEG feeds over RTP.

Jul 27 15:13:33 ladon bc-server[5990]: RTSP session for rtsp://Admin:[email protected]:7070/ has multiple video streams. Only the first stream will be recorded.
Jul 27 15:13:33 ladon bc-server[5990]: I(1/Lobby): RTP stream started: Video: mjpeg
Jul 27 15:13:33 ladon bc-server[5990]: I(1/Lobby): avlib error: colorspace not supported in jpeg
Jul 27 15:13:33 ladon kernel: [ 6697.021638] bc-server[18853]: segfault at 4 ip 013e057a sp b6dfef70 error 4 in libavcodec.so.53[f62000+6cc000]
Jul 27 15:13:33 ladon kernel: [ 6697.034710] init: bluecherry main process (5990) killed by SEGV signal
Jul 27 15:13:33 ladon kernel: [ 6697.034924] init: bluecherry main process ended, respawning
Jul 27 15:13:33 ladon bc-server[18956]: I: Started Bluecherry daemon
Jul 27 15:13:33 ladon bc-server[18956]: I: SQL database connection opened
Jul 27 15:13:33 ladon bc-server[18956]: Event 5 left in-progress at shutdown; setting duration
Jul 27 15:13:33 ladon bc-server[18956]: I: Not licensed; running in trial mode
Jul 27 15:13:33 ladon bc-server[18956]: I(1/Lobby): Camera configured
Jul 27 15:13:33 ladon bc-server[18956]: I(1/Lobby): Switching to new recording schedule 'continuous'
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib fatal: No JPEG data found in image
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib error: invalid id 30
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib fatal: No JPEG data found in image
Jul 27 15:13:34 bc-server[18956]: last message repeated 3 times
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib error: mjpeg: unsupported coding type (c6)
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib fatal: No JPEG data found in image
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib error: mjpeg: unsupported coding type (c5)
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib error: only 8 bits/component accepted
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib fatal: No JPEG data found in image
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib error: mjpeg: unsupported coding type (cd)
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib error: mjpeg: unsupported coding type (ca)
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib error: only 8 bits/component accepted
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib fatal: No JPEG data found in image
Jul 27 15:13:34 ladon bc-server[18956]: I(1/Lobby): avlib error: invalid id 231

http://improve.bluecherrydvr.com/issues/1131

Add better statuses for network camera failures

Currently we log any error as 'Not Found'. We need to support at least these two:

'Offline' (for connection timeouts)
'Authentication failed'

We also need verbose handling of errors. The actual error received from the network camera should be attached to the XML so the client can read and display it.

Add option to close camera in live view

One of the options should allow to close the feed.

http://improve.bluecherrydvr.com/issues/1089

Log motion during continuous recording

We will need the server to 'mark' motion events in the database during continuous recording so they uniquely show up during the playback part of the UI.

http://improve.bluecherrydvr.com/issues/344

Add field for deleted ip cameras

bluecherry.Devices.deleted

http://improve.bluecherrydvr.com/issues/1203

Support conditional events

I'd like notification support to be based on conditional statements, instead of 'email when motion happens'. I've attached some screen shots of how Axis cameras work for notification and 'events'.

Web: Show estimated recording time for available storage devices based on enabled devices.

@abelykh or @ruminsam

In the storage area we should show how many hours, or days of recording is available based on historical information.

Allow machine id updates in some cases

In some cases if the machine id is changed the confirmation should be done automatically.

http://improve.bluecherrydvr.com/issues/1173

Web: Layouts are not user specific

Mobile interface for iPhone / Android web apps

By using one of the javascript frameworks below we can create a touch-enabled version of the web interface for live viewing, events and eventually event playback:

http://jqtouch.com/
http://www.iwebkit.net
http://www.sencha.com/products/touch/ (Commercial)
http://www.sproutcore.com/

HTTP streaming of past events from a web browser

Using a flash player a customer should be able to stream videos from a web browser without losing much quality.

We really need this to be HTML5 based, however HTML5 doesn't support the Matroska container in it's video tag. We can't tie up the CPU either transcoding videos on the fly, so we need to look into some way to support this.

Support on-camera motion detection for Axis cameras

Axis supports on-camera motion detection with notification via meta data, see the docs repo for more information.

Cut recordings in 15 minute blocks hourly, instead of 15 minutes from the start time for continuous recordings

To help better keep files in sync, and to help cleanup the time-line, event durations should be cut off four times an hour, every 15 minutes (:00, :15, :30, :45).

This will have no effect on motion events that exceed 15 minutes (they will remain cut off after 15 minutes)

At the same time as this change, we should implement proper cutting just before a keyframe, so that we never lose any video frames between continuous recordings.

Add ONVIF support

ONVIF is a standard that allows cameras to be detected and configured in a universal way. Links are below:

http://www.onvif.org/
http://www.openipcam.com/forum/index.php/topic,155.0.html
http://en.wikipedia.org/wiki/Open_Network_Video_Interface_Forum

Add API version call to server

Client should be ablt to determine which version of API server supports (for example: if it supports afterId or ids parameters to events call).
It should be a simple integer updated each time a new API call/parameter is available.

http://improve.bluecherrydvr.com/issues/1189

Real-time server-to-client communication

We need to work out a system for real-time communications from the server and web interface to the client. This is necessary/useful for:

Real-time event notification (especially motion)
Notification of recording status
Notification of device changes (added/removed/reconfigured)
..?
The actual protocol of this can be nearly anything, even as simple as a never-ending XML stream over HTTP. However, it must be possible for the client to maintain a connection and receive immediate notification of the situations above and anything else we find useful.

Details of implementation can be discussed at a later date.

Invalid Video Format, assuming MP4V-ES

Tested with 2.0.3 and 2.0.4:

root@BC-DVR:~# tail -f /var/log/bluecherry.log
Apr 29 14:53:23 BC-DVR bc-server[8129]: I: SQL database connection opened
Apr 29 14:53:23 BC-DVR bc-server[8129]: Media /var/lib/bluecherry/recordings/2013/04/29/000001/14-53-14.mkv left in-progress so updating length to 8
Apr 29 14:53:23 BC-DVR bc-server[8129]: I: Found 0000:02:05.0[solo6110] id 0, 8 ports
Apr 29 14:53:24 BC-DVR bc-server[8129]: I: Licensed for 8 devices
Apr 29 14:53:24 BC-DVR bc-server[8129]: I(1/Port 1 on Card 1): Camera configured
Apr 29 14:53:24 BC-DVR bc-server[8129]: I(1/Port 1 on Card 1): Switching to new recording schedule 'continuous'
Apr 29 14:53:24 BC-DVR bc-server[8129]: W(1/Port 1 on Card 1): Invalid Video Format, assuming MP4V-ES
Apr 29 14:53:26 BC-DVR bc-server[8129]: W(1/Port 1 on Card 1): Invalid Video Format, assuming MP4V-ES
Apr 29 14:53:26 BC-DVR bc-server[8129]: E(1/Port 1 on Card 1): Cannot decode video frame: Operation not permitted
Apr 29 14:53:26 BC-DVR bc-server[8129]: E(1/Port 1 on Card 1): No video frame for snapshot

http://improve.bluecherrydvr.com/issues/1213

Test import from GitHub to Trello

Unable to delete cameras / foreign key issues

Aug 2 13:57:42 myhost apache2[14764]: (SQL ERROR): [DELETE FROM Devices WHERE id='5'] => Cannot delete or update a parent row: a foreign key constraint fails (bluecherry.EventsCam, CONSTRAINT EventsCam_ibfk_3 FOREIGN KEY (device_id) REFERENCES Devices (id))

http://improve.bluecherrydvr.com/issues/1136

Find an alternative to dbconfig-common

Between customers not following the dbconfig-common instructions and random issues with dbconfig-common and SQL upgrades we need to come up with a better solution for database upgrades and migrations.

http://improve.bluecherrydvr.com/issues/1135