wpilibsuite / cscore Goto Github PK

CS core could use some love in the HTML world.
Two way data bindings with the live values of the camera could be achieved with a JS library like angular.
The server should probably just provide static HTML with dynamic JSON instead of dynamic JSON and dynamic HTML.
You could also throw in some styling to make the UI look a bit more friendly.

If this is something you are interested in providing I'd need some help in the C++ world but I could do the rest of the HTML JS stuff.

Hi,
Sorry if this is a noob question. Im trying to compile the usbcvstream program for my camera running on a nanopi. It looks like I'm missing LLVM headers (like raw_ostream) which wasn't listed in the required packages in the README. Should these have been installed with one of those or do I actually need them at all? In addition if I'm already using GCC why do I even need this headers, aren't they associated with Clang which is a different compiler altogether?

Thanks!

Are there tests for this? Should we have travis configured on this project?

In Java11, Win64:

Yes, I have telemetry on with CameraServerJNI.setTelemetryPeriod(.5); and I can verify that it is by successfully utilizing VideoSource.getActualFPS(). Is there a separate thing I am missing or have I stumbled across a bug?

Repeated calls to UsbCamera.setExposure() (e.g. in the teleopPeriodic() method) causes a "too many streams" error.

For tuning purposes. Probably just a subclass of cvsource.

Would make it easier for users to figure out what to put in the 'path' if they didn't want to use the automatic indexed camera access when using multiple cameras.

Would it be possible for you to generate a JavaDoc for cscore and add it to the main WPILibJ JavaDoc?

If adding it to the main JavaDoc is impossible, is there a way I could download it?

Thanks!

• Implement HttpCamera
• Implement HttpCamera properties
• Intermediate FPS support
• Total ordering of UsbCamera streamon/streamoff (to avoid USB bandwidth limits when switching sources with SetSource) (#69)
• Per-MJPEG server image scaling and FPS (when more than one sink connected) so dashboard settings don't change image processing
• UsbCamera: Scaled 0-100 brightness, etc settings (not just raw)
• UsbCamera: Exposure quirk for Microsoft LifeCam
• WPILib: Finish NT bidirectional settings support
• MjpegServer: Write error message (e.g. "camera disconnected") as text on a black image for dashboard display (configurable setting)
• Add NiSource and NiSink to interface to nivision
• Add RawSource and RawSink to interface to nivision or other libraries
• Hoist Java OpenCV to Java layer (using RawSource/RawSink) to avoid static libs issues
• Performance improvement: don't copy UsbCamera image to memory but rather keep the kernel mmap buffer open? Tradeoff on this due to user code delays (low priority)
• Cache compressed / resized / color-converted images in Frame
• MjpegServer: Improve HTML "root page" (e.g. allow settings changes)
• Logger C/Java API (copy from ntcore)
• Cross platform support (Windows)
• Cross platform support (Mac)
• MjpegServer: Support multiple streams multiplexed onto a single server port, selected by name GET parameter
• Better error message for too much USB bandwidth usage (STREAMON failed due to no space left on device)
• CameraServer: Fix NT settings
• CameraServer: Provide access to MjpegServer object
• CameraServer: Auto-increment camera number in startAutomaticCapture

Hi -
I am pretty impressed with the cscore implementation running on our Raspberry Pi.
We are successfully serving a webpage from our RPi at port 8081. I am also able to add OpenCV image operations and view the results on port 8082 (the usbcvstream example).

Part of the use of our camera streams has been for autonomous, which involves OpenCV thresholding of H, S and V around a particular color (mainly green) to focus on targets. I was interested in extending the existing webpage controls by adding min and max sliders for H, S and V. I have found some of the HTML page construction code in mjpegserverImpl.cpp. Is there any documentation you can point me to to add controls to the webpage?

Thanks!
Rob

Currently if the process doesn't have permission to open a USB camera, CSCore simply keeps trying to open the camera every 2 seconds without logging an error message. It would be useful to actually print an error message in this scenario (otherwise people will stare at things like dmesg until someone mentions to check permissions).

Currently only specific GET request combinations are allowed. In particular, HTTP request parameters must be in a certain order for the case of "/?action=stream" (it must be the first request parameter). With the URL parsing utilities we have in wpiutil, this can be made much more accepting.

Reference: http://stackoverflow.com/questions/4227328/faq-why-does-dynamic-cast-only-work-if-a-class-has-at-least-1-virtual-method

Reference: pybind/pybind11#645 (comment)

If this seems reasonable, I can probably make a PR for this tonight. Thoughts?

Things like:

• How many streams are currently connected?
• How much bandwidth are they using?
• Resolution/framerate?

MjpegServer has an internal parameter "m_compression" which sets the JPEG compression. This parameter cannot be set anywhere via a program. The only method is to make an HTTP request to set it, which is not really feasible.
This parameter should be exposed for setting.

None of the GET responses properly respond with CORS headers. This causes Javascript connections to not work correctly.

Would be nice to be able to curl a snapshot of the camera without needing to do a full stream connection.

Due to the sometimes long delay incurred during connecting a USB camera, the common desire to switch between cameras, and the problems of USB camera bandwidth limitations, more control over camera connection and disconnection is desirable, particularly for USB cameras. Currently the library auto-disconnects when no sinks are listening, necessitating hacky methods for keeping the camera connection open to avoid reconnect delays when there is a sink being switched between two sources. Also, if two cameras combined exceed the USB camera bandwidth, there is no way to synchronize one stopping and the other starting when setSource is used to switch.

It seems like the cleanest approach would be to make this auto-disconnection feature optional and probably default it to disabled (at least for USB cameras) and provide API access to explicitly connect/disconnect the camera.

This was demonstrated at the computer vision seminar at the FRC Championships St. Louis. An audience member connected to the robot's radio, went to the address read aloud by the presenters, and started messing with the settings to hamper the ability of the GRIP pipeline that was demonstrated to find the water bottle's label.

Since the values did not update live, the presenters came to the conclusion that the lighting changed, that auto-exposure was too strong, and that the camera was faulty.

The presenters could have been properly informed about the situation if the settings were live updating, and politely asked the audience to stop changing the camera settings.

It would be nice if the results from enumerateVideoModes was available on the webserver, as that's more convenient then parsing programmatically to determine which modes a camera supports.

source.createBoolProperty, createIntProperty, et al.

Hi all -
Background: building a RPi vision processor for use with our Roborio. Using Raspbian stretch.
Have successfully build and published wpiutil and ntcore, would like to also build cscore.
Currently have OpenCV 2.4.13 build and installed.

When I start the build, the following occurs:

pi@raspberrypi:~/cscore ./gradlew -PreleaseBuild -PskipAllTests build

FAILURE: Build failed with an exception.

• What went wrong:
  A problem occurred configuring root project 'cscore'.

Exception thrown while executing model rule: DependencyConfigRules#setupDependencyDownloads(ModelMap, DependencyConfigSpec, BinaryContainer, ProjectLayout, BuildConfigSpec)
Could not resolve all files for configuration ':nativeDeps'.
> Could not find opencv-cpp-linuxarm.zip (org.opencv:opencv-cpp:3.2.0).
Searched in the following locations:
http://first.wpi.edu/FRC/roborio/maven/development/org/opencv/opencv-cpp/3.2.0/opencv-cpp-3.2.0-linuxarm.zip
> Could not find wpiutil-cpp-linuxarm.zip (edu.wpi.first.wpiutil:wpiutil-cpp:3.424242.0.0-beta-2-20171111160242-4-g3e4e526).
Searched in the following locations:
file:/home/pi/releases/maven/development/edu/wpi/first/wpiutil/wpiutil-cpp/3.424242.0.0-beta-2-20171111160242-4-g3e4e526/wpiutil-cpp-3.424242.0.0-beta-2-20171111160242-4-g3e4e526-linuxarm.zip

Any pointers? Do I need OpenCV 3 for cscore? What other dependencies?

Thanks,
Rob

https://github.com/wpilibsuite/cameraserver/blob/master/include/cameraserver_c.h#L103

Allocates using malloc, so needs an associated free function

I am trying to use cscore to stream a kinect's RGB and depth data over a MJPEG connection, but when I call PutFrame, I get a segfault. Here's a gist of the C++ code I'm using: https://gist.github.com/DrWateryCat/c225e436d5761bbb21fd622c78e78972 and here's the stacktrace by valgrind:

==25030== Use of uninitialised value of size 8
==25030==    at 0x5AA09F5: cv::Mat::copyTo(cv::_OutputArray const&) const (in /usr/lib/x86_64-linux-gnu/libopencv_core.so.2.4.9)
==25030==    by 0x5EFAF60: cs::CvSourceImpl::PutFrame(cv::Mat&) (in /usr/local/lib/libcscore.so)
==25030==    by 0x5EFC460: cs::PutSourceFrame(int, cv::Mat&, int*) (in /usr/local/lib/libcscore.so)
==25030==    by 0x40685E: cs::CvSource::PutFrame(cv::Mat&) (cscore_oo.inl:343)
==25030==    by 0x404393: main (libfreenect_opencvtest.cpp:328)
==25030== 
==25030== Invalid read of size 8
==25030==    at 0x5AA09F5: cv::Mat::copyTo(cv::_OutputArray const&) const (in /usr/lib/x86_64-linux-gnu/libopencv_core.so.2.4.9)
==25030==    by 0x5EFAF60: cs::CvSourceImpl::PutFrame(cv::Mat&) (in /usr/local/lib/libcscore.so)
==25030==    by 0x5EFC460: cs::PutSourceFrame(int, cv::Mat&, int*) (in /usr/local/lib/libcscore.so)
==25030==    by 0x40685E: cs::CvSource::PutFrame(cv::Mat&) (cscore_oo.inl:343)
==25030==    by 0x404393: main (libfreenect_opencvtest.cpp:328)
==25030==  Address 0xfc2010040 is not stack'd, malloc'd or (recently) free'd
==25030== 
==25030== 
==25030== Process terminating with default action of signal 11 (SIGSEGV)
==25030==  Access not within mapped region at address 0xFC2010040
==25030==    at 0x5AA09F5: cv::Mat::copyTo(cv::_OutputArray const&) const (in /usr/lib/x86_64-linux-gnu/libopencv_core.so.2.4.9)
==25030==    by 0x5EFAF60: cs::CvSourceImpl::PutFrame(cv::Mat&) (in /usr/local/lib/libcscore.so)
==25030==    by 0x5EFC460: cs::PutSourceFrame(int, cv::Mat&, int*) (in /usr/local/lib/libcscore.so)
==25030==    by 0x40685E: cs::CvSource::PutFrame(cv::Mat&) (cscore_oo.inl:343)
==25030==    by 0x404393: main (libfreenect_opencvtest.cpp:328)
==25030==  If you believe this happened as a result of a stack
==25030==  overflow in your program's main thread (unlikely but
==25030==  possible), you can try to increase the size of the
==25030==  main thread stack using the --main-stacksize= flag.
==25030==  The main thread stack size used in this run was 8388608.

I get the following error when trying to set the UsbCamera to auto exposure mode. I did not test with the HttpCamera class, but it gave the same issue for several different USB cameras. The setExposure and setBrightness methods all work correctly, it is just the auto exposure. This method was being called in the robotInit method of my Robot file as seen in the trace below.

CS: ERROR: ioctl VIDIOC_S_EXT_CTRLS failed at UsbCameraProperty.cpp:66: Invalid argument (UsbUtil.cpp:122)
ERROR 1 ERROR Unhandled exception: VideoException [edu.wpi.cscore.VideoException: property write failed] at [edu.wpi.cscore.CameraServerJNI.setCameraExposureAuto(Native Method), edu.wpi.cscore.VideoCamera.setExposureAuto(VideoCamera.java:51), com.kylecorry.frc.vision.CameraSource.setExposureAuto(CameraSource.java:130), org.usfirst.frc.team5112.robot.Robot.robotInit(Robot.java:34), edu.wpi.first.wpilibj.IterativeRobot.startCompetition(IterativeRobot.java:64), edu.wpi.first.wpilibj.RobotBase.main(RobotBase.java:247)] edu.wpi.first.wpilibj.RobotBase.main(RobotBase.java:249)

Maybe a log message saying "device allocation is 40", or something... to help teams debug potential bandwidth issues they may run into.

Last year I implemented something that would write images to disk periodically, but only if a USB flash drive was inserted into the roborio. It was pretty useful, and seems like the sort of generic utility that a team would want to have.

https://github.com/robotpy/robotpy-cscore/blob/master/cscore/imagewriter.py

I actually hadn't thought of this until now, but maybe it could be implemented as a sink? Something to think about.

Hi -
We are using a Microsoft LifeCam Cinema camera that exhibits the same exposure setting 'quirk' documented in the code for the HD-3000. Right now the code adjusts how exposure is controlled by checking for the camera type in UsbCameraImpl.cpp:

m_hd3000 = GetDescription(desc).endswith("LifeCam HD-3000");

Request: can the code also check for LifeCam Cinema? I will be updating my local version of this build to ensure the update works.

Thanks,
Rob

Hello, I've successfully crossed compiled gcc 6.3.0 and its relative binutils, gdb and sysroot for the roborio (This version upgrade isn't necessary but I perceive it to be a greater issue). Everything works fine until I attempt to compile cscore (As being in conjunction with allwpilib). I get a undefined reference error (see below) to a function in HttpUtil.cpp. I was receiving the same error for a native build (Using gcc 6.2.0). I am curious to hear back from you guys. I don't know the code layout and am wondering if there is a version check define somewhere where the preprocessor removes those functions -- Or this could just an absurd single instanced error... 😐

:cscoreSourceZip
:build
:outputVersions
:arm:compileJava
Download http://first.wpi.edu/FRC/roborio/maven/development/org/opencv/opencv-ja
va/3.1.0/opencv-java-3.1.0.pom
Download http://first.wpi.edu/FRC/roborio/maven/development/org/opencv/opencv-ja
va/3.1.0/opencv-java-3.1.0.jar
Download https://repo1.maven.org/maven2/com/google/errorprone/error_prone_core/2
.0.9/error_prone_core-2.0.9.pom
Download https://repo1.maven.org/maven2/com/google/errorprone/error_prone_parent
/2.0.9/error_prone_parent-2.0.9.pom
Download https://repo1.maven.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent
-7.pom
Download https://repo1.maven.org/maven2/com/google/errorprone/error_prone_annota
tion/2.0.9/error_prone_annotation-2.0.9.pom
Download https://repo1.maven.org/maven2/com/github/stephenc/jcip/jcip-annotation
s/1.0-1/jcip-annotations-1.0-1.pom
Download https://repo1.maven.org/maven2/org/pcollections/pcollections/2.1.2/pcol
lections-2.1.2.pom
Download https://repo1.maven.org/maven2/com/google/guava/guava/19.0/guava-19.0.p
om
Download https://repo1.maven.org/maven2/com/google/guava/guava-parent/19.0/guava
-parent-19.0.pom
Download https://repo1.maven.org/maven2/com/google/code/findbugs/jFormatString/3
.0.0/jFormatString-3.0.0.pom
Download https://repo1.maven.org/maven2/com/google/code/findbugs/jsr305/3.0.0/js
r305-3.0.0.pom
Download https://repo1.maven.org/maven2/org/checkerframework/dataflow/1.8.10/dat
aflow-1.8.10.pom
Download https://repo1.maven.org/maven2/com/google/errorprone/javac/1.9.0-dev-r2
973-2/javac-1.9.0-dev-r2973-2.pom
Download https://repo1.maven.org/maven2/com/google/errorprone/error_prone_annota
tions/2.0.9/error_prone_annotations-2.0.9.pom
Download https://repo1.maven.org/maven2/org/checkerframework/javacutil/1.8.10/ja
vacutil-1.8.10.pom
Download https://repo1.maven.org/maven2/com/google/errorprone/error_prone_core/2
.0.9/error_prone_core-2.0.9.jar
Download https://repo1.maven.org/maven2/com/google/errorprone/error_prone_annota
tion/2.0.9/error_prone_annotation-2.0.9.jar
Download https://repo1.maven.org/maven2/com/github/stephenc/jcip/jcip-annotation
s/1.0-1/jcip-annotations-1.0-1.jar
Download https://repo1.maven.org/maven2/org/pcollections/pcollections/2.1.2/pcol
lections-2.1.2.jar
Download https://repo1.maven.org/maven2/com/google/guava/guava/19.0/guava-19.0.j
ar
Download https://repo1.maven.org/maven2/com/google/code/findbugs/jFormatString/3
.0.0/jFormatString-3.0.0.jar
Download https://repo1.maven.org/maven2/com/google/code/findbugs/jsr305/3.0.0/js
r305-3.0.0.jar
Download https://repo1.maven.org/maven2/org/checkerframework/dataflow/1.8.10/dat
aflow-1.8.10.jar
Download https://repo1.maven.org/maven2/com/google/errorprone/javac/1.9.0-dev-r2
973-2/javac-1.9.0-dev-r2973-2.jar
Download https://repo1.maven.org/maven2/com/google/errorprone/error_prone_annota
tions/2.0.9/error_prone_annotations-2.0.9.jar
Download https://repo1.maven.org/maven2/org/checkerframework/javacutil/1.8.10/ja
vacutil-1.8.10.jar
:arm:processResources UP-TO-DATE
:arm:classes
:arm:jniHeadersCscore
:downloadOpenCvHeaders
Download http://first.wpi.edu/FRC/roborio/maven/development/org/opencv/opencv-he
aders/3.1.0/opencv-headers-3.1.0.pom
Download http://first.wpi.edu/FRC/roborio/maven/development/org/opencv/opencv-he
aders/3.1.0/opencv-headers-3.1.0.jar
:arm:unzipOpenCvHeaders
:arm:downloadOpenCvJni_linux-arm
Download http://first.wpi.edu/FRC/roborio/maven/development/org/opencv/opencv-jn
i/3.1.0/opencv-jni-3.1.0.pom
Download http://first.wpi.edu/FRC/roborio/maven/development/org/opencv/opencv-jn
i/3.1.0/opencv-jni-3.1.0-linux-arm.jar
:arm:unzipOpenCvJni_linux-arm
:arm:downloadOpenCvNatives_linux-arm
Download http://first.wpi.edu/FRC/roborio/maven/development/org/opencv/opencv-na
tives/3.1.0/opencv-natives-3.1.0.pom
Download http://first.wpi.edu/FRC/roborio/maven/development/org/opencv/opencv-na
tives/3.1.0/opencv-natives-3.1.0-linux-arm.jar
:arm:unzipOpenCvNatives_linux-arm
:arm:downloadWpiUtil
Download http://first.wpi.edu/FRC/roborio/maven/development/edu/wpi/first/wpilib
/wpiutil/1.0.2-20170209040313-1-ge665632/wpiutil-1.0.2-20170209040313-1-ge665632
.pom
Download http://first.wpi.edu/FRC/roborio/maven/development/edu/wpi/first/wpilib
/wpiutil/1.0.2-20170209040313-1-ge665632/wpiutil-1.0.2-20170209040313-1-ge665632
-arm.zip
:arm:unzipWpiUtil
:arm:compileCscoreSharedLibraryCscoreCpp
:arm:linkCscoreSharedLibrary
:arm:cscoreSharedLibrary
:arm:assembleDependentsCscoreSharedLibrary
:arm:compileEnum_usbExecutableEnum_usbCpp
:arm:compileCscoreStaticLibraryCscoreCpp
:arm:createCscoreStaticLibrary
:arm:cscoreStaticLibrary
:arm:linkEnum_usbExecutable
C:\Users\USER\Desktop\cscore\cscore\arm\build\libs\cscore\static\libcscore.a(Ht
tpUtil.o): In function cs::HttpRequest::SetAuth(cs::HttpLocation const&):
C:\Users\USER\Desktop\cscore\cscore\src/HttpUtil.cpp:305: undefined reference t
o wpi::Base64Encode(llvm::StringRef, std::__cxx11::basic_string<char, std::char
_traits<char>, std::allocator<char> >*)
collect2.exe: error: ld returned 1 exit status

:arm:linkEnum_usbExecutable FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task :arm:linkEnum_usbExecutable.
> A build operation failed.
      Linker failed while linking enum_usb.

Regards, David Smerkous

Haven't dug into this quite yet, but http://localhost:1182/?action=command&draw=1 doesn't seem to get reflected in settings.json.

No equivalent free exists either. I'll PR this when I have time.

Encoding/decoding might run into patent licensing issues, but at a minimum we should support streaming directly from a camera that outputs h.264.

I'm seeing the following build error after executing this build command in macOS on both the HEAD and v1.0.2 tag release:

./gradlew build -PskipArm

/Users/rich/github/cscore/src/Image.h:35:9: error: no matching constructor for initialization of 'std::vector<uchar>' (aka 'vector<unsigned char>')
      : m_data{capacity, default_init_allocator<uchar>{}} {
        ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/vector:505:5: note: candidate constructor not viable: no known conversion from 'default_init_allocator<uchar>' (aka 'default_init_allocator<unsigned char>') to 'const value_type' (aka 'const unsigned char') for 2nd argument
    vector(size_type __n, const_reference __x); ...

I see that in the included file default_init_allocator.h some problem related to vectors looks to have been solved using the included StackOverflow discussion linked in the comments of the source code:

// From: http://stackoverflow.com/questions/21028299/is-this-behavior-of-vectorresizesize-type-n-under-c11-and-boost-container
// Credits: Casey and Howard Hinnant

Is the error that I'm seeing related to this solution not working?