revmischa / cloudcam

IP camera surveillance management system using AWS IoT with support for Axis cameras.

                                                                                     *
                                                       *                           **
                       **                             ***                          **
                       **                              *                           **
***  ****               **    ***                               ****               **
 **** **** *    ***      **    ***  *** **** ****    ***       * **** *    ****    **  ***      ****
  **   ****    * ***     **     ***  *** **** ***  *  ***     **  ****    * ***  * ** * ***    * ***  *
  **          *   ***    **      **   **  **** ****    **    ****        *   ****  ***   ***  *   ****
  **         **    ***   **      **   **   **   **     **      ***      **         **     ** **    **
  **         ********    **      **   **   **   **     **        ***    **         **     ** **    **
  **         *******     **      **   **   **   **     **          ***  **         **     ** **    **
  **         **          **      *    **   **   **     **     ****  **  **         **     ** **    **
  ***        ****    *    *******     **   **   **     **    * **** *   ***     *  **     ** **    **
   ***        *******      *****      ***  ***  ***    *** *    ****     *******   **     **  ***** **
               *****                   ***  ***  ***    ***               *****     **    **   ***   **
                                                                                          *
                                                                                         *
                                                                                        *
                                                                                       *


Updated: 6/12/2024, 12:06:48 PM UTC

📩 Latest Blog Posts

• Updates in the JS ecosystem around the recently-released Node 22 and V8
• Will We Still Need Programmers?
• Mastering JavaScript Tree-Shaking
• Create a Test User in Cognito
• War and Ukrainian Linguistic Changes

📺 Latest Talks

• Building A Real-World IoT Application From Start to Finish With AWS
• Serverless Application Architecture
• Intro to AWS Lambda
• Basics of UNIX Sockets
• Linux Container Basics: Docker, Kubernetes

cloudcam's Issues

gstreamer library

Does it make sense to replace an exec() call with an actual call to gst_pipeline_new()?
I noticed Axis ACAP has glib available.
I assume we'd just be using gstreamer on the client side for RTPVideoSink (maybe audio too some day)

Frame timestamping and synchronization

(from @exp)

Given two cameras observing the same scene, each with acceptable jitter, how closely should the two views be synced?
Each camera will have its own free-running but quite inaccurate clock periodically checked by NTP, but there's no guarantee their NTP servers are closely linked and significant jitter and delay is to be expected no matter what.
Each frame is captured with an offset which should be very close between cameras but will ultimately vary based on its local clock rate and other internal factors such as temperature or system load.
Each media stream encodes its understanding of the current time into the PCR (Program Clock Reference) and this is used to synchronise individual decoders, but it too is limited to the clock quality inside a cheap network appliance, ie not great.
None of these can be trivially synchronised with each other without a process of clock reconstruction, that is determining on the receiver side at which time the sender actually captured or transmitted the frame (PTS vs PCR).
Receivers must also undergo this process as multiple cloud instances are very unlikely to have strong clock synchronisation. There are protocols to get a much better idea of what time it actually is, ie the Precision Time Protocol. I know that AWS has or does support some form of PTP use.
This is just a lot of blogging but really the question is: Do you even care about showing the same scene from two angles and having them be well synced, or would a few frames either way make little difference? If it does matter, a careful process of clock reconstruction has to happen on input, and the datastore you use must be able to handle different frame to frame timings, and even the offsets that occur when playing back two slightly mis-timed sources. (Axis Cam1 could be 60.0002fps wheras Axis Cam2 could be 59.9994).

Serverless transforms

If it will be beneficial, we should convert the cloudformation templates to use the new Serverless transforms which should make them more concise.

https://github.com/awslabs/serverless-application-model

https://github.com/awslabs/serverless-application-model/tree/master/examples/2016-10-31

Meetecho promo

Meetecho guy said we should post the project on https://janus.conf.meetecho.com/docs/resources
And post a writeup some day on https://groups.google.com/forum/#!forum/meetecho-janus

Maybe see if anyone's interested

Low priority

SRTP Janus Support

Goal: allow clients to stream to Janus with SRTP-encrypted streams
Implementation: see notes @ https://github.com/revmischa/cloudcam/wiki/janus
Will likely need to write a custom Janus plugin [#5]

Janus hosting

• Find somewhere to run dockerized Janus with DTLS support (lightsail?).
• Create a service registry of Janus instances available for usage - integrating cloudwatch health checks and/or ELB would be cool. Maybe use PostgreSQL, DynamoDB or consul?
• Need secret storage for certs (https://aws.amazon.com/certificate-manager/ or Vault or KMS ?))

Building ACAP Application

I am having an issue compiling the ACAP application for an axis target.

make -C axis dist
make[1]: Entering directory '/home/ubuntu/development/cloudcam/axis'
create-package.sh armv6
make armv6-axis-linux-gnueabimake[2]: Entering directory '/home/ubuntu/development/cloudcam/axis'
rm -f  lib/libaws-iot.a
rm -f cloudcam *.o core
make -C  clean
make[5]: *** clean: No such file or directory.  Stop.
Makefile:54: recipe for target 'clean-mbedtls' failed
make[4]: *** [clean-mbedtls] Error 2
/home/ubuntu/axis/emb-app-sdk_2_0_3/tools/build/rules/targets.mak:84: recipe for target 'checkclean' failed
make[3]: *** [checkclean] Error 2
/home/ubuntu/axis/emb-app-sdk_2_0_3/tools/build/rules/targets.mak:23: recipe for target 'armv6-axis-linux-gnueabi' failed
make[2]: *** [armv6-axis-linux-gnueabi] Error 1
make[2]: Leaving directory '/home/ubuntu/development/cloudcam/axis'
Makefile:62: recipe for target 'dist' failed
make[1]: *** [dist] Error 2
make[1]: Leaving directory '/home/ubuntu/development/cloudcam/axis'
Makefile:89: recipe for target 'dist' failed
make: *** [dist] Error 2

I also noticed that the ./axis/Makefile references files that do not exist like ./axis/cloudcam_axis.c and ./axis/lib/libaws_iot.a. I get the feeling that this implementation is incomplete.

I also noticed that when running the host build, a configuration file is missing:

DEBUG:   cloudcam_init_iot_client L#32 current dir: .

DEBUG:   cloudcam_init_iot_client L#33 
AWS IoT SDK Version 2.1.1-

ERROR: cloudcam_init_iot_client L#42 error: payload parsing config file ./config.json: unable to open ./config.json: No such file or directory at ./config.json

WARN:  main L#44 cloudcam_init_iot_client failed, exiting

Please advise.

Integrate JSON parsing library

AWSIoT embedded C SDK uses a pretty terrible JSON library. I was working on adding Jansson, I don't think I finished it. Integrate Jansson (or another library if you prefer) for JSON parsing and document construction.

See thumbnail_requested_handler

Janus Cloudcam Plugin

I imagine we will need a custom Janus plugin for cloudcam to handle registration of mountpoints and relaying status to cloudcam.
This is just an assumption though - maybe the existing API and streaming plugin provide enough functionality for us? Except for lack of SRTP support

Continuous Integration

Tests + lint for client lib and lambda code would be nice.
I like Travis-CI but whatever works.

Let's add tests wherever possible. I love tests and TDD.

API for frontend

Sad to say but some day we will need a user interface. It will need some sort of API to talk to.

Lambdas are nice but making a coherent set of APIs out of them will be challenging without something like chalice or flask or step functions. I don't know what the best solution for our project will be. Do some research.
It'd be nice to take advantage of API Gateway's ability to spit out SDKs and Swagger.

Public stream view

Goal: a user should be able to create a "public view" page of a stream. It would be a URL that anyone can visit and see a live stream from a camera. Just the stream, nothing else.

Investigate recording options

Goal: come up with a plan of how we will implement video and event recording, archiving, and playback.

Some people may want to stream the cams to the cloud 24/7 (maybe at very low bitrate unless someone's viewing), but some will want to buffer locally and some won't have enough bandwidth.

I believe Axis cameras can do two encoded streams at once with different bitrates, so in some setups maybe a camera is always streaming to the recording server (Janus or ?) at a very low bitrate, whereas the higher-bitrate encoder could be used when someone is viewing live.

A way to estimate how much storage and bandwidth a recording will use will be very important; storage of continuous video recordings is considerable.

An option to record only when motion is detected will be key. I believe Axis cams support this built-in. Not sure if we want to handle that on the server side, most likely no.

Consider use cases of people with one camera vs many camera, upstream bandwidth, local vs cloud recording server, Janus support, Axis support, local buffering, space/quality long term storage tradeoffs, AWS Glacier for long-term archiving, "events" (see Axis docs), auto-start/stop based on motion, how playback will work.

Automate AWS environment creation

Goal: make it easy for a developer or user to create or upgrade their AWS environment to run a CloudCam service.
Long-term goal: make a package that can be sold and easily deployed from AWS Marketplace.

Some components will probably be: IAM roles, Lambdas, Step Functions, S3 bucket, IoT (Thing category, rules), API gateway, RDS (PostgreSQL Aurora).
In future maybe Greengrass, X-ray, Rekognition, Elastic Transcoder, Cognito, Glacier, WAF/Shield, CloudWatch metrics.

I suggest using CloudFormation.

Synchronisation

Given two cameras observing the same scene, each with acceptable jitter, how closely should the two views be synced?

Each camera will have its own free-running but quite inaccurate clock periodically checked by NTP, but there's no guarantee their NTP servers are closely linked and significant jitter and delay is to be expected no matter what.

Each frame is captured with an offset which should be very close between cameras but will ultimately vary based on its local clock rate and other internal factors such as temperature or system load.

Each media stream encodes its understanding of the current time into the PCR (Program Clock Reference) and this is used to synchronise individual decoders, but it too is limited to the clock quality inside a cheap network appliance, ie not great.

None of these can be trivially synchronised with each other without a process of clock reconstruction, that is determining on the receiver side at which time the sender actually captured or transmitted the frame (PTS vs PCR).

Receivers must also undergo this process as multiple cloud instances are very unlikely to have strong clock synchronisation. There are protocols to get a much better idea of what time it actually is, ie the Precision Time Protocol. I know that AWS has or does support some form of PTP use.

This is just a lot of blogging but really the question is: Do you even care about showing the same scene from two angles and having them be well synced, or would a few frames either way make little difference? If it does matter, a careful process of clock reconstruction has to happen on input, and the datastore you use must be able to handle different frame to frame timings, and even the offsets that occur when playing back two slightly mis-timed sources. (Axis Cam1 could be 60.0002fps wheras AxisCam2 could be 59.9994).

Deployment issues

I was able to build the source code.

However, after running ./deploy_stack as in:

cd cloudformation && ./deploy-stack.sh

I get some errors like:

ERROR: missing configuration variable: STACK_NAME
ERROR: missing configuration variable: S3_CODE_BUCKET
..
..

How am I supposed to set these variables?