VIAME is a computer vision application designed for do-it-yourself artificial intelligence including object detection, object tracking, image/video annotation, image/video search, image mosaicing, size measurement, rapid model generation, and tools for the evaluation of different algorithms. Originally targetting marine species analytics, it now contains many common algorithms and libraries, and is also useful as a generic computer vision toolkit. It contains a number of standalone tools for accomplishing the above, a pipeline framework which can connect C/C++, python, and matlab nodes together in a multi-threaded fashion, and, lastly, multiple algorithms resting on top of the pipeline infrastructure. Both a desktop and web version exist for deployments in different types of environments.
The User's Quick-Start Guide, Tutorial Videos, and Developer's Manual are more comprehensive, but select entries are also listed below broken down by individual functionality:
Build and Install Guide <> All Examples <> GUIs for Annotation and Visualization <> Object Detectors <> Object Trackers <> Detector Training API <> Video Search and Rapid Model Generation <> Scoring of Detectors <> Detection File Formats <> Calibration and Image Enhancement <> Image Registration and Mosaicing <> Stereo Measurement and Depth Maps <> KWIVER Pipelining Overview <> Core Class and Pipelining Info <> Web Interface <> How to Integrate Your Own Plugin <> Example Plugin Templates <> Embedding Detectors in C++ Code
For a full installation guide and description of the various flavors of VIAME, see the quick-start guide, above. The desktop version is provided as either a .msi, .zip or .tar file. Alternatively, docker files are available for both VIAME Desktop and Web (below). A sample instance of VIAME Web is also online, hosted at viame.kitware.com. For desktop installs, extract the binaries (or use the msi Windows installation wizard) and place them in a directory of your choosing, for example /opt/noaa/viame on Linux or C:\Program Files\VIAME on Windows. If using packages built with GPU support, make sure to have sufficient video drivers installed, version 451.82 or higher. The best way to install drivers depends on your operating system, see below. Lastly, run through some of the examples to validate the installation. The binaries are quite large, in terms of disk space, due to the inclusion of multiple default model files and programs, but if just building your desired features from source (e.g. for embedded apps) they are much smaller.
Installation Requirements:
Windows 7, 8, or 10 64-Bit, RHEL/CentOS 7/8 64-Bit, or Ubuntu 16.04/18.04/20.04 64-Bit
6 Gb of Disk Space for the Full Installation
Installation Recommendations:
NVIDIA Drivers (Version 451.82+
Windows
[1]
[2]
Ubuntu
[1]
[2]
CentOS
[1]
[2])
A CUDA-enabled GPU with 8 Gb or more VRAM
Windows Desktop Binaries:
VIAME v0.15.2 Windows 7*/8/10, GPU Enabled, Wizard (.msi), Default Models
VIAME v0.15.2 Windows 7*/8/10, GPU Enabled, Mirror1 (.zip)
VIAME v0.15.2 Windows 7*/8/10, GPU Enabled, Mirror2 (.zip)
VIAME v0.15.2 Windows 7*/8/10, CPU Only, Mirror1 (.zip)
VIAME v0.15.2 Windows 7*/8/10, CPU Only, Mirror2 (.zip)
Ubuntu Desktop Binaries:
VIAME v0.15.1 Ubuntu 18.04/20.04, GPU Enabled, Mirror1 (.tar.gz)
VIAME v0.15.1 Ubuntu 18.04/20.04, GPU Enabled, Mirror2 (.tar.gz)
VIAME v0.15.1 Ubuntu 16.04, GPU Enabled, Mirror1 (.tar.gz)
VIAME v0.15.1 Ubuntu 16.04, GPU Enabled, Mirror2 (.tar.gz)
CentOS or Other Linux Desktop Binaries:
VIAME v0.15.1 RHEL/CentOS 7/8, GPU Enabled, Mirror1 (.tar.gz)
VIAME v0.15.1 RHEL/CentOS 7/8, GPU Enabled, Mirror2 (.tar.gz)
VIAME v0.15.1 Generic Linux, GPU Enabled, Mirror1 (.tar.gz)
VIAME v0.15.1 Generic Linux, GPU Enabled, Mirror2 (.tar.gz)
*Windows 7 requires some updates and service packs installed, e.g. KB2533623.
Web Applications:
VIAME Online Web Annotator and Public Annotation Archive
VIAME Web Local Installation Instructions
VIAME Web Source Repository
DIVE Standalone Desktop Annotator:
DIVE Installers (Linux, Mac, Windows)
SEAL Desktop Custom Distribution:
SEAL v0.14.0 Windows 7/8/10, GPU Enabled (.zip)
SEAL v0.14.0 Windows 7/8/10, CPU Only (.zip)
SEAL v0.14.0 CentOS 7, GPU Enabled (.tar.gz)
SEAL v0.14.0 Generic Linux, GPU Enabled (.tar.gz)
Optional Patches:
Motion Detector Models, All OS
Alternative Generic Detector for IQR, All OS
Arctic Seals Models, Windows
Arctic Seals Models, Linux
HabCam Models (Scallop, Skate, Flatfish), All OS
Low Memory GPU (For 4+ Gb Cards), All OS
MOUSS Model Set 1 (Deep 7 Bottomfish), All OS
MOUSS Model Set 2 (Deep 7 Bottomfish), All OS
MOUSS Sample Project, All Linux
Penguin Head FF Models, All OS
Sea Lion Models, All OS
SEFSC 100-200 Class Fish Models, All OS
EM Tuna Detectors, All OS
Note: To install Add-Ons and Patches, copy them into an existing VIAME installation folder. To use project files extract them into your working directory of choice. Custom distributions contain a full installation, only with non-default features turned on, and should not be copied into existing installations because they are a full installation and bad things will happen.
Docker images are available on: https://hub.docker.com. For a default container with just core algorithms, runnable via command-line, see:
kitware/viame:gpu-algorithms-latest
This image is headless (ie, it contains no GUI) and contains a VIAME desktop (not web) installation in the folder /opt/noaa/viame. For links to the VIAME-Web docker containers see the above section in the installation documentation. Most add-on models are not included in the instance but can be downloaded via running the script download_viame_addons.sh in the bin folder.
These instructions are intended for developers or those interested in building the latest master branch. More in-depth build instructions can be found here, but the software can be built either as a super-build, which builds most of its dependencies alongside itself, or standalone. To build VIAME requires, at a minimum, Git, CMake, and a C++ compiler. Installing Python and CUDA is also recommended. If using CUDA, version 10.1 or 9.2 are preferred, with CUDNN 7.0 (not 8.0, yet). Other CUDA versions may or may not work. On both Windows and Linux it is also highly recommended to use Anaconda3 5.2.0 for python, which is the most tested distribution used by developers. If using other python distributions having numpy installed, at a minimum, is necessary and there some issues on certain versions with the plugin system used on different version to be fixed on future versions.
To build on the command line in Linux, use the following commands, only replacing [source-directory] and [build-directory] with locations of your choice. While these directories can be the same, it's good practice to have a 'src' checkout then a seperate 'build' directory alongside it:
git clone https://github.com/VIAME/VIAME.git [source-directory]
cd [source-directory] && git submodule update --init --recursive
Next, create a build directory and run the following cmake command (or alternatively
use the cmake GUI if you are not using the command line interface):
mkdir [build-directory] && cd [build-directory]
cmake -DCMAKE_BUILD_TYPE:STRING=Release [source-directory]
Once your cmake command has completed, you can configure any build flags you want
using 'ccmake' or the cmake GUI, and then build with the following command on Linux:
make -j8
Or alternatively by building it in Visual Studio or your compiler of choice on Windows. On Linux, '-j8' tells the build to run multi-threaded using 8 threads, this is useful for a faster build though if you get an error it can be difficult to see it, in which case running just 'make' might be more helpful. For Windows, currently VS2017 is the desired compiler, though select versions of 2015 and 2019 also work.
There are several optional arguments to viame which control which plugins get built, such as those listed below. If a plugin is enabled that depends on another dependency such as OpenCV) then the dependency flag will be forced to on. If uncertain what to turn on, it's best to just leave the default enable and disable flags which will build most (though not all) functionalities. These are core components we recommend leaving turned on:
| Flag | Description |
|---|---|
| VIAME_ENABLE_OPENCV | Builds OpenCV and basic OpenCV processes (video readers, simple GUIs) |
| VIAME_ENABLE_VXL | Builds VXL and basic VXL processes (video readers, image filters) |
| VIAME_ENABLE_PYTHON | Turns on support for using python processes (multiple algorithms) |
| VIAME_ENABLE_PYTORCH | Installs all pytorch processes (detectors, trackers, classifiers) |
And a number of flags which control which system utilities and optimizations are built, e.g.:
| Flag | Description |
|---|---|
| VIAME_ENABLE_CUDA | Enables CUDA (GPU) optimizations across all processes (PyTorch, etc...) |
| VIAME_ENABLE_CUDNN | Enables CUDNN (GPU) optimizations across all processes |
| VIAME_ENABLE_VIVIA | Builds VIVIA GUIs (tools for making annotations and viewing detections) |
| VIAME_ENABLE_KWANT | Builds KWANT detection and track evaluation (scoring) tools |
| VIAME_ENABLE_DOCS | Builds Doxygen class-level documentation for projects (puts in install tree) |
| VIAME_BUILD_DEPENDENCIES | Build VIAME as a super-build, building all dependencies (default behavior) |
| VIAME_INSTALL_EXAMPLES | Installs examples for the above modules into install/examples tree |
| VIAME_DOWNLOAD_MODELS | Downloads pre-trained models for use with the examples and interfaces |
And lastly, a number of flags which build algorithms or interfaces with more specialized functionality:
| Flag | Description |
|---|---|
| VIAME_ENABLE_TENSORFLOW | Builds TensorFlow object detector plugin |
| VIAME_ENABLE_DARKNET | Builds Darknet (YOLO) object detector plugin |
| VIAME_ENABLE_BURNOUT | Builds Burn-Out based pixel classifier plugin |
| VIAME_ENABLE_SMQTK | Builds SMQTK plugins to support image/video indexing and search |
| VIAME_ENABLE_SCALLOP_TK | Builds Scallop-TK based object detector plugin |
| VIAME_ENABLE_SEAL_TK | Builds Seal multi-modality GUI |
| VIAME_ENABLE_ITK | Builds ITK cross-modality image registration |
| VIAME_ENABLE_UW_CLASSIFIER | Builds UW fish classifier plugin |
| VIAME_ENABLE_MATLAB | Turns on support for and installs all matlab processes |
| VIAME_ENABLE_LANL | Builds an additional (Matlab) scallop detector |
VIAME ├── cmake # CMake configuration files for subpackages ├── docs # Documentation files and manual (pre-compilation) ├── configs # All system-runnable config files and models │ ├── pipelines # All processing pipeline configs │ │ └── models # All models, which only get downloaded based on flags │ ├── prj-linux # Default linux project files │ └── prj-windows # Default windows project files ├── examples # All runnable examples and example tutorials ├── packages # External projects used by the system │ ├── kwiver # Processing backend infastructure │ ├── fletch # Dependency builder for things which don't change often │ ├── kwant # Scoring and detector evaluation tools │ ├── vivia # Baseline desktop GUIs (v1.0) │ └── ... # Assorted other packages (typically for algorithms) ├── plugins # Integrated algorithms or wrappers around external projects │ └── ... # Assorted plugins (detectors, depth maps, filters, etc.) ├── tools # Standalone tools or scripts, often building on the above └── README.md # Project introduction page that you are reading └── RELEASE_NOTES.md # A list of the latest updates in the system per version
If you already have a checkout of VIAME and want to switch branches or update your code, it is important to re-run:
git submodule update --init --recursive
After switching branches to ensure that you have on the correct hashes of sub-packages within the build. Very rarely you may also need to run:
git submodule sync
Just in case the address of submodules has changed. You only need to run this command if you get a "cannot fetch hash #hashid" error.
VIAME is released under a BSD-3 license.
A non-exhaustive list of relevant papers used within the project alongside contributors can be found here.
VIAME was developed with funding from multiple sources, with special thanks to those listed here.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent