mahyarmirrashed / fiducial-tracker

Track fiducials from multiple video streams and publish location information to listening peers.

Fiducial Tracker

An array of cameras sends video streams to the server module. There, the server, using zxingcpp, a general purpose barcode code scanning library, analyzes for QR code fiducials, and published their tracked locations to listening clients.

Communication between cameras and server is accomplished using REQ/REP sockets. This is so that the server can regulate the input camera feed to reduce staleness of communicated frames from video streams.

Conda Environment

To run the code, setup the necessary conda environment by running conda env create -f environment.yml.

Firebase Connection

To publish the tracked fiducial locations to your Firebase instance, set the FIREBASE_DATABASE_URL environment variable. If you are setting the environment with a .env file, create the .env in the same folder as this README. Then, download the firebase RTDB certificate JSON and pass it as the --firebase-certificate parameter for the client submodule.

fiducial-tracker's Issues

[Feat]: Add Ability to Send Location Information from Server to Receivers

Servers should be able to send location information to receivers. Multiple fiducial locations should be able to be communicated.

[Feat]: Add Ability to Stream Videos to Server

Client programs should have an added flag for taking videos as inputs. These videos should then be streamed from the clients over to the server. The server will "analyze" the streams and then pass some data off to the listeners.

[Feat]: Add FPS Regulating on Camera Submodule

Camera submodule should listen to recommended FPS from server submodule when it receives a VideoStreamResponseMessage.

[Feat]: Add Camera Ability to Prompt for User Calibration

Add a feature for the camera to receive or prompt the user for required calibration.

[Feat]: Use `pyzbar` for QR Code Detection and Localization

Since pyzbar would be used anyway to get the data from the QR codes, why not take advantage of the fact that it already provides a polygon around the QR code when it scans for it? The images that we are passing are small and it is optimized for finding QR codes. In this manner, its speed might be equal to or better than YOLO. Not to mention, it would not require having a GPU.

[Feat]: Add Interface for Darknet Library Methods

The driver program for the server module will require interfacing with the methods in the compiled darknet library.

[Fix]: Increase Incoming Frame Rate of Video Streams to Server

When streaming video from the client to the server, the frame rate is noticeably low. With a single client, we should expect at least to be able to process 30 frames per second.

[Feat]: Add Ability to Automatically Compile the `libdarknet.so` Library File

As a framework, darknet, can instead be referenced as a factory that generates libdarknet.so library files that can be interfaced with through Python. Instructions, including installer scripts, should be included so that users can obtain copies of the library file when running the program on their own computers.