vvvv / vl.opencv

A VL wrapper for OpenCVSharp

VL.OpenCV

Computer vision library with the following features:

• Image filters
• Feature detection and tracking
• Aruco 3D marker tracking
• Camera and projector calibration
• ...

Not GPU accalerated!

For use with vvvv, the visual live-programming environment for .NET: http://visualprogramming.net

Getting started

• Install as described here via commandline:

  nuget install VL.OpenCV

• Usage examples and more information are included in the pack and can be found via the Help Browser

Contributing

• Please see our Contribution Guide
• For custom development requests, please get in touch

Credits

Based on OpenCvSharp which in turn in based on OpenCV. Also using SharpDX MediaFoundation.

vl.opencv's Issues

VideoIn crashes when selected enum entry not available

In SupportedVideoFormats the entry is then -1 and I get a index out of bounds exception. Anything we can do about that?

How do I use the VIdeoWriter?

Hi. The video writer has been marked as done, but I've been trying to record video from my camera and it doesn't create file. How should I be doing this?

Renderer break under certain coniditons with VideoPlayer

Firstly I noticed this only seems to happen with square videos (i.e. 720x720 or 1080x1080 videos).

versions:
vvvv = 2021.3.3 also tried 2021.4.0-0458
vl.opencv = 2.1.0
opencvsharp = 4.4.5.3.20210725
opencvsharp.runtime.win = 4.4.5.3.20210725

Error:
If you create a Renderer node and a VideoPlayer node (with a square video file loaded) and then connect the two of them you get an error from the Renderer node.

2 different work arounds:

1. If you create a VideoPlayer node (with a square video file loaded) drag the output link and then double click and create a Renderer node directly attached to the link, it works.

2. When you get the Renderer error, switch the video file to something that isnt square and then back to the square video.

Tooltip Error

Float64 Images fail to get converted correctly before being previewed:

Unsupported depth of input image:
'VDepth::contains(depth)'
where
'depth' is 6 (CV_64F)

at OpenCV.Filter.ConvertColor.c__DisplayClass37779189.b__0(InputArray Arg 1, OutputArray Arg 2) in C:\Users\H\Documents\vvvv\beta-preview\Packages\VL.OpenCV.0.2.106-alpha\VL.OpenCV.vl:line 183723
at OpenCV.Filter.Filter__1M_1_.Update(Boolean Apply, Action`2 Operation, CvImage Input, CvImage& Output) in C:\Users\H\Documents\vvvv\beta-preview\Packages\VL.OpenCV.0.2.106-alpha\VL.OpenCV.vl:line 188290

Cheers

rename demo patches to Root

to have them autorun

review VideoWriter

make it work for non-realtime cases

Undistort is distorting even more

Thoroughly tried the camera calibration example and it seems like the undistort node is distorting the image even more instead of doing the opposite.

I managed to get a reprojection error of 0.38, so pretty good. But applying this result with the undistort node gives you even more distorted corners and you get black pixels on the edges, as if the distortion is inverted.

review Houghlines

Aruco Marker: no way to enable CornerRefinement?

By default the CornerRefinement is disabled. Which makes the detection a bit jittery and only outputs integer pixel values.
From the documentation It can be enabled by:
bool doCornerRefinement
but exactly this parameter doesn't show up in VL under Aruco-->DetectorParamters, all the other parameters are there.

add single/spread versions for all draw nodes

review namings

Bind Renderer to help patches

Renderer: add "Show Cursor" input

default=true

add cache region to yolo and objectdetector

mark VideoFile nodes as experimental

Xenko-to-Stride update required for the pack

Gamma 2020.2.225 and above are currently breaking VL.OpenCV 0.2.133-alpha. I think this may be related to this error, or is this seperate?:

1 Channel Float Tooltips Preview strange

32FC1 images produce quite strange previews. Maybe the images could be normalized to between 0-255 and output as a a 8UC1, but this may not be the ideal. Dark images will become bright, perhaps beyond preference.

ObjectDetector (CustomFile) always sees empty files

On b36, VL.OpenCV 0.2.10-alpha.

When setting the Classifier File pin of ObjectDetector (CustomFile), the node turns pink and complains the file is empty (tooltip says !empty()). The file is indeed not empty, vvvv can display its content with Reader (String).

Opening the ObjectDetector, I noticed the ptr property of the CascadeClassifier keeps changing at every frame, since the Condition of the If region is always true, is that intended behavior ?

EDIT : The tooltip of the Path IOBox going in ObjectDetector says the size property is null :/

GPU support

gpumat and some other benefits would be awesome, right?

i just discovered there's new instructions on how to build and customize opencv binaries on the repo here: https://github.com/shimat/opencvsharp#how-to-customize-opencv-binaries-yourself
and even a separate repo for this purpose: https://github.com/shimat/opencv_files

i know this was not working some while ago, but maybe now is the time to try again?

sort out transformation btw. opencv/opengl/dx

see also #3

• document the problem and solution

VideoIn not working with creative webcam

while the webcam works fine in vvvv beta and anything else

ToProjectionTransform shift in Y (Principalpoint offset) inverted

The problem shows up when a camera has a principal Point shifted in Y direction. (or the calibration simply thinks so)

When feeding the "Camera Matrix" from "Calibrate Camera" into "ToProjectionTransform" to get a valid projection matrix in v4 the shift in Y is going the other direction.

A quick fix is to multiply slice 9 of the v4 matrix by -1.

Fix Apply pin in (Scalar) filter nodes

CvImage should implement IImage

add Warp node

and make it work with our transforms

Add HSV OpticalFlow visualization

review ImageReader

.Sink error on install

I cant get the example up and running. There seems to be an issue with .Sink Renderer Stack
not found stack
dependency sysytem.reflection.vl

Im new to vvvv so sorry if im doing something obvious wrong but how can i fix this?

Can't use math mat expressions without ToMat function returning the result

I know I can forward the OpenCV Nuget to get this command but I think users wouldn't be expecting that to be necessary because you have forwarded the * / + and - expression nodes already

Could you also forward the ToMat node so the results of the expressions can be obtained?

(I suppose this setup could be done by wrapping the mats to the CVImage type, but on the other hand maybe they performfaster if the matexpr's are combined and pass ToMat just once.

refactor default mat situation

Optical flow and resizing crash

See https://discourse.vvvv.org/t/application-error-with-vl-opencv/17970

Camera calibration wrong calculation in BoardCorners

In the VL patch BoardCorners the "Board Size" gets decremented by one and multiplied by "Square Side Length" and then fed into a LinearSpread. The decrement is actually not necessary. Gives better reprojection errors.

fix performance of experimental Renderer

DrawText improvements

• fails with empty text input
• set default text input "vvvv"
• the default position should be visible (now out of image topleft)
• can the fontsize be in pixels?

Transform (Perspective) crashes vl

review VideoReader

Normalization DataType Enum

An Enum would be easier to use than an Int for this filter.

CUDA support enabled?

Just wanted to ask if the GPU support in the underlying opencvsharp library is enabled/working...?

I Can't build anything

Using version 0.2.133
I tried building the simple app attached but building process hangs forever as it tried to compile the VL.OpenCV package.

Please test this.
QuickVidTest1.zip

Harmonize/clean existing help patches

Renderer improvements

• add bounds input
• add label input
• add aspect ratio scale and make it default
• add info overlay

Consider adding cache to FromImage

Crop bug on one channel image

When cropping one channel image it becomes distorted (channels are messed up).
(I'm rendering it with Skia.)
See the patch attached.

p.s. The workaround is to ConvertColor (GRAY > BRGA) after the Crop, which is not right, but maybe gives a hint on what is wrong.

CropBug.zip

add data getter nodes

update to OpenCVSharp4

• make sure all demos work

Git housekeeping

• rename master to main
• create develop branch
• change github actions to build on tag
• tags on main build release nuget
• tags on dev build prerelease nuget
• auto parse version from tag value and replace in nuspec file

review SeamlessClone

offer reactive VideoIn version

make vvvv specific conversion nodes

• create demo patches
  see also #12

UploadImage/OpenCV crash when F8

• open \VL.OpenCV.0.2.46-alpha\vvvv\girlpower\DX9Demo.v4p
• open vl editor
• press F8 -> crash

when removing the UploadImage node before pressing F8, the crash doesn't happen.

seems ocv related,
"C:\Users\joreg\Desktop\vvvvs\vvvv_50alpha36.1_x64\girlpower\VL\DX\DynamicBuffersAndTextures.v4p"
works

VideoIn - missing device specs output