wieden-kennedy / cinder-kcb2 Goto Github PK

View Code? Open in Web Editor NEW

48.0 48.0 27.0 13.97 MB

Cinder wrapper for Kinect 2 Common Bridge

C 13.69% C++ 86.31%

cinder-kcb2's People

Contributors

Stargazers

Watchers

cinder-kcb2's Issues

Cinder 0.8.6

Hi Guys,

I am trying to compile the project using Cinder 0.8.6 but I am getting lots of errors of datatypes not included in Cinder and being used in KinectV2.h

For example the file "cinder/gl/VboMesh.h" doesn't exist in Cinder.

The types Channel16uRef, Surface8uRef ..etc don't exist in Cinder

Any help would be really appropriated

Thanks

Error when using TinderBox-Win: "Unable to copy from C:/cinder/blocks/Cinder-KCB2/lib/$(PlatformTarget)/KCBv2.dll ..."

Hey,

I can compile the BodyApp example but TinderBox seems to have trouble resolving the PLATFORMTARGET variable when creating a new project with KCB2 (copy) selected.

"Unable to copy from C:/cinder/blocks/Cinder-KCB2/lib/$(PlatformTarget)/KCBv2.dll ..."

PointCloud not working with latest cinder master.

MayaCam.h is removed from latest code and replaced with CameriaUi.h
Kinect.h is also out of sync with cinder master code.

I have done some changes to PointCloud and kinect.h.

The app is showing blank image. I would like to know what is wrong in my code.

Kinect.h - Replaced addVertices to addPositions

PointCloudApp.cpp is as follows 👎

/*
*

Copyright (c) 2015, Wieden+Kennedy
Stephen Schieberl
All rights reserved.
*
Redistribution and use in source and binary forms, with or
without modification, are permitted provided that the following
conditions are met:
*
Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in
the documentation and/or other materials provided with the
distribution.
*
Neither the name of the Ban the Rewind nor the names of its
contributors may be used to endorse or promote products
derived from this software without specific prior written
permission.
*
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*
*/

include "cinder/app/App.h"

include "cinder/CameraUi.h"

include "cinder/gl/GlslProg.h"

include "cinder/gl/Texture.h"

include "cinder/gl/VboMesh.h"

include "cinder/params/Params.h"

include "cinder/gl/Texture.h"

include "cinder/gl/Draw.h"

include "cinder/gl/Scoped.h"

include "Kinect2.h"

class PointCloudApp : public ci::app::App
{
public:
void draw() override;
void mouseDrag(ci::app::MouseEvent event) override;
void resize() override;
void setup() override;
void update() override;
private:
ci::Channel16uRef mChannelDepth;
Kinect2::DeviceRef mDevice;
ci::Surface8uRef mSurfaceColor;
ci::Surface32fRef mSurfaceDepthToCameraTable;
ci::Surface32fRef mSurfaceDepthToColorTable;
long long mTimeStamp;
long long mTimeStampPrev;

void                        loadGlsl();
ci::gl::GlslProgRef         mGlslProg;
ci::gl::TextureRef          mTextureColor;
ci::gl::TextureRef          mTextureDepth;
ci::gl::TextureRef          mTextureDepthToCameraTable;
ci::gl::TextureRef          mTextureDepthToColorTable;
ci::gl::VboMeshRef          mVboMesh;

ci::CameraUi                mMayaCam;
ci::CameraPersp camera;

float                       mFrameRate;
bool                        mFullScreen;
ci::params::InterfaceGlRef  mParams;

};

include "cinder/app/RendererGl.h"

using namespace ci;
using namespace ci::app;
using namespace std;

void PointCloudApp::draw()
{
gl::viewport(getWindowSize());
gl::clear();
gl::setMatrices(camera);
gl::enableAlphaBlending();
gl::enableDepthRead();
gl::enableDepthWrite();

if (mSurfaceColor) {
    if (mTextureColor) {
        mTextureColor->update(*mSurfaceColor);
    }
    else {
        mTextureColor = gl::Texture::create(*mSurfaceColor);
    }
    mTextureColor->bind(0);
}
if (mChannelDepth) {
    if (mTextureDepth) {
        gl::ScopedTextureBind scopeTextureBind(mTextureDepth->getTarget(), mTextureDepth->getId());
        glTexSubImage2D(mTextureDepth->getTarget(), 0, 0, 0,
            mTextureDepth->getWidth(), mTextureDepth->getHeight(),
            GL_RED_INTEGER, GL_UNSIGNED_SHORT, mChannelDepth->getData());
    }
    else {
        mTextureDepth = gl::Texture::create(
            mChannelDepth->getWidth(), mChannelDepth->getHeight(),
            gl::Texture::Format().dataType(GL_UNSIGNED_SHORT).internalFormat(GL_R16UI));
    }
    mTextureDepth->bind(1);
}
if (mSurfaceDepthToCameraTable && !mTextureDepthToCameraTable) {
    mTextureDepthToCameraTable = gl::Texture::create(*mSurfaceDepthToCameraTable);
    mTextureDepthToCameraTable->bind(2);
}
if (mSurfaceDepthToColorTable) {
    if (mTextureDepthToColorTable) {
        mTextureDepthToColorTable->update(*mSurfaceDepthToColorTable);
    }
    else {
        mTextureDepthToColorTable = gl::Texture::create(
            *mSurfaceDepthToColorTable,
            gl::Texture::Format().dataType(GL_FLOAT));
    }
    mTextureDepthToColorTable->bind(3);
}

gl::ScopedGlslProg scopeGlsl(mGlslProg);
gl::setDefaultShaderVars();
mGlslProg->uniform("uTextureColor", 0);
mGlslProg->uniform("uTextureDepth", 1);
mGlslProg->uniform("uTextureDepthToCameraTable", 2);
mGlslProg->uniform("uTextureDepthToColorTable", 3);

gl::draw(mVboMesh);

if (mTextureColor) {
    mTextureColor->unbind();
}
if (mTextureDepth) {
    mTextureDepth->unbind();
}
if (mTextureDepthToCameraTable) {
    mTextureDepthToCameraTable->unbind();
}
if (mTextureDepthToColorTable) {
    mTextureDepthToColorTable->unbind();
}

mParams->draw();

}

void PointCloudApp::loadGlsl()
{
try {
mGlslProg = gl::GlslProg::create(gl::GlslProg::Format()
.vertex(loadAsset("cloud.vert"))
.fragment(loadAsset("cloud.frag")));
}
catch (gl::GlslProgCompileExc ex) {
console() << "GLSL Error: " << ex.what() << endl;
quit();
}
catch (gl::GlslNullProgramExc ex) {
console() << "GLSL Error: " << ex.what() << endl;
quit();
}
catch (...) {
console() << "Unknown GLSL Error" << endl;
quit();
}
}

void PointCloudApp::mouseDrag(MouseEvent event)
{
bool middle = event.isMiddleDown() || (event.isMetaDown() && event.isLeftDown());
bool right = event.isRightDown() || (event.isControlDown() && event.isLeftDown());
mMayaCam.mouseDrag(event.getPos(), event.isLeftDown() && !middle && !right, middle, right);
}

void PointCloudApp::resize()
{

//CameraPersp cam = mMayaCam.getCamera();
mMayaCam = CameraUi(&camera, getWindow());
camera.setAspectRatio(getWindowAspectRatio());

gl::enableVerticalSync();

}

void PointCloudApp::setup()
{
gl::enable(GL_TEXTURE_2D);

mFrameRate = 0.0f;
mFullScreen = false;
mTimeStamp = 0L;
mTimeStampPrev = mTimeStamp;

loadGlsl();

mDevice = Kinect2::Device::create();
mDevice->start();
mDevice->connectColorEventHandler([&](const Kinect2::ColorFrame& frame)
{
    mSurfaceColor = frame.getSurface();
});
mDevice->connectDepthEventHandler([&](const Kinect2::DepthFrame& frame)
{
    mChannelDepth = frame.getChannel();
    mTimeStamp = frame.getTimeStamp();
});

//////////////////////////////////////////////////////////////////////////////////////////////

ivec2 sz = Kinect2::DepthFrame().getSize();
vector<vec2> vertices;
for (int32_t x = 0; x < sz.x; ++x) {
    for (int32_t y = 0; y < sz.y; ++y) {
        vertices.push_back(vec2(x, y) / vec2(sz));
    }
}

gl::VboRef vbo = gl::Vbo::create(GL_ARRAY_BUFFER, vertices.size() * sizeof(vec2), &vertices[0], GL_STATIC_DRAW);

geom::BufferLayout layout;
layout.append(geom::Attrib::POSITION, 2, sizeof(vec2), 0);
vector<pair<geom::BufferLayout, gl::VboRef>> vertexArrayBuffers = { make_pair(layout, vbo) };

mVboMesh = gl::VboMesh::create(vertices.size(), GL_POINTS, vertexArrayBuffers);

//////////////////////////////////////////////////////////////////////////////////////////////

mParams = params::InterfaceGl::create("Params", ivec2(200, 120));
mParams->addParam("Frame rate", &mFrameRate, "", true);
mParams->addParam("Full screen", &mFullScreen).key("f");
mParams->addButton("Load GLSL", [&]() { loadGlsl(); }, "key=g");
mParams->addButton("Quit", [&]() { quit(); }, "key=q");

resize();

}

void PointCloudApp::update()
{
mFrameRate = getAverageFps();

if (mFullScreen != isFullScreen()) {
    setFullScreen(mFullScreen);
    mFullScreen = isFullScreen();
}

if (!mSurfaceDepthToCameraTable) {
    mSurfaceDepthToCameraTable = mDevice->mapDepthToCameraTable();
}

if ((mTimeStamp != mTimeStampPrev) && mSurfaceColor && mChannelDepth) {
    mTimeStampPrev = mTimeStamp;

    mSurfaceDepthToColorTable = Surface32f::create(mChannelDepth->getWidth(), mChannelDepth->getHeight(), false, SurfaceChannelOrder::RGB);
    vector<ivec2> positions = mDevice->mapDepthToColor(mChannelDepth);

    vec2 sz(Kinect2::ColorFrame().getSize());

    Surface32f::Iter iter = mSurfaceDepthToColorTable->getIter();
    vector<ivec2>::iterator v = positions.begin();
    while (iter.line()) {
        while (iter.pixel()) {
            iter.r() = (float)v->x / sz.x;
            iter.g() = 1.0f - (float)v->y / sz.y;
            iter.b() = 0.0f;
            ++v;
        }
    }
}

}

CINDER_APP(PointCloudApp, RendererGl(RendererGl::Options().coreProfile()), [](App::Settings* settings)
{
settings->prepareWindow(Window::Format().size(1280, 960).title("Point Cloud App"));
settings->setFrameRate(60.0f);
})

Kinect2::Body::Hand getState() function not working properly

Hello!

In the BodyApp example the function getState() for the Hand object,is always returning 0 (Unknown State) and LowConfidenceTracking, no matter what. Maybe there's an error getting the info from the SDK?

Lag when using Kinect2

I get a periodic lag in my Kinect2 data when using this API. This doesn't occur with the main Kinect2 API (non KCB) and it is definitely in the kinect data coming in (my app continues to run at 60fps as the Kinect2 data halts and then starts up again).

Need to get FloorClipPlane

Hey!
I was wondering if you were planning on adding the ability to get the Floor Clip Plane in this cinder block.
Here is the API reference for IBodyFrame: https://msdn.microsoft.com/en-us/library/microsoft.kinect.kinect.ibodyframe.aspx
And here is the floor clip plane: https://msdn.microsoft.com/en-us/library/microsoft.kinect.kinect.ibodyframe.get_floorclipplane.aspx

I am happy to add it in myself, but I was wondering if this would be a different type of frame, that I would connect an event handler to etc, or if this should be built into the existing Body data.

Visual Studio 2013

Is this supported? I am getting errors both when trying to use the block (Unable to copy from .... blocks/Cinder-KCB2/lib/$(PlatformTarget)/$(Configuration) etc) and when building, the faceApp for example:

Error 1 error C2039: 'Channel8uRef' : is not a member of 'cinder'
\cinder-kcb2\src\kinect2.h 64 1 FaceApp
Error 14 error C2039: 'vec2' : is not a member of 'cinder' \cinder-kcb2\src\kinect2.h 72 1 FaceApp

I copied the repo into the blocks folder, then launched the tinderbox exe from the master branch. I'm not very familiar with Windows tools, as I usually work in Linux, so could be an error on my side.

[glNext] Compile error on channel & surface checks

The boolean operators on channels & surfaces have been removed: the implicit shared_ptr pattern has been replaced with the typedef'ed shared_ptr refs. As such, all the if-statements checking if a channel or a surface exist now fail on compile.

Just a heads up...

The library hogs my CPU to 100%, even when Kinect isn't connected

On my Quadcore laptop, the library hogs my CPU to 100%, even when Kinect isn't connected.

Expose DepthFrameToCameraSpaceTable

Hi,

I was wondering if it'd be useful to expose this "depth-to-camera-space" lookup table in your block. This lookup table can be used to map the depth frame pixels to 3d positions in camera space. Converted to a texture, it can be used to "inverse-project" the depth map in a vertex shader.

I currently am using the following function (the name & the assert call might not be ideal):

ci::Surface32f Device::getDepthFrameToCameraSpaceTable() const {

    PointF* tableEntries = nullptr;
    uint32_t tableCount;
    GetDepthFrameToCameraSpaceTable( mKinect, &tableCount, &tableEntries );

    Vec2i size = Frame::getDepthSize();
    assert( size.x * size.y == tableCount );
    ci::Surface32f lookupSurface( size.x, size.y, false );

    Surface32f::Iter iter = lookupSurface.getIter();
    size_t i = 0;
    while( iter.line() ) {
        while( iter.pixel() ) {
            iter.r() = tableEntries[i].X;
            iter.g() = tableEntries[i].Y;
            iter.b() = 1.0f;
            ++i;
        }
    }
    return lookupSurface;
}

Used in the vertex shader, it would look something like this:

vDepth = texture( uDepth, ciTexCoord0 ).r;
pos.xy = vDepth * texture( uDepthToCameraSpace, ciTexCoord0 ).rg; //or texelFetch
pos.z = vDepth;

Just a suggestion. Feel free to close this "issue".

PointCloudApp exe runs but point cloud is not drawn

The sample PointCloudApp works as expected when I compile and run from Visual Studio 2013. However, when I try to run the built .exe, the app runs but the point cloud never draws. Tried from both Cinder release and master.

Crash after destroying Device, stale update signal

Just wanted to report this, and this is a simple fix.

Device::Process::Process() attempting to reference a deleted function

This error occurs on Line 745 in Kinect2.cpp and halts compiling of the BasicApp (syntax error). Also an error where this is called:
function "Kinect2::Device::Process::operator=(const Kinect2::Device::Process &)" (declared implicitly) cannot be referenced -- it is a deleted function [project name]

Is there something I'm missing?
Cinder v 0.9.0
Visual Studio 2015
Windows 10

Occasional hang during shutdown [glNext]

I'm using the block in an app that has an override on shutdown(). Occasionally the app won't shut all the way down, and if I pause it in the debugger, it's usually somewhere in the update loop / idling a thread somewhere, e.g. on this line. I'm managing the Kinect in a class, and in that class' destructor I'm disconnecting all handlers that I previously connected, and calling Device::stop(). This is intermittent so I'm sorry I can't be more specific... just wanted to get it up here while I'm thinking about it.

Expose a way to identify the number of detected bodies

According to the MSDN docs:

"The BodyIndexFrame Class represents a frame that is computed based on the depth image. This image tells you which depth or infrared pixels belong to tracked people and which belong to the background. The pixel values in this frame are 8-bit unsigned integers, where 0-5 map directly to the BodyData index in the BodyFrame Class. Values greater than the value obtained from BodyFrameSource.BodyCount Property indicate that the pixel is part of the background, not associated with a tracked body. This frame is useful for green screening applications..."
https://msdn.microsoft.com/en-us/library/dn782033.aspx

There doesn't seem to be a way to access this BodyFrameSource.BodyCount from the KCB2 API. BodyFrame::getBodies().size() always returns 6, for example, regardless whether any bodies are visible. Is there another way?

Samples not building against latest Cinder master

Hello,

I'm having some trouble getting a few of the samples to build. These are the specific errors I'm getting for each. The BodyApp sample however builds fine.

I'll add that I'm just getting my feet wet in Cinder. Any suggestions would be much appreciated.

BasicApp

1>  BasicApp.cpp
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(71): error C2039: 'viewport' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(71): error C3861: 'viewport': identifier not found
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(72): error C2039: 'clear' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(72): error C3861: 'clear': identifier not found
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(73): error C2039: 'setMatricesWindow' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(73): error C3861: 'setMatricesWindow': identifier not found
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(74): error C2039: 'enableAlphaBlending' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(74): error C3861: 'enableAlphaBlending': identifier not found
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(78): error C2039: 'draw' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(78): error C2660: 'BasicApp::draw' : function does not take 3 arguments
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(82): error C2039: 'draw' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(82): error C2660: 'BasicApp::draw' : function does not take 3 arguments
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(86): error C2039: 'draw' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(86): error C2660: 'BasicApp::draw' : function does not take 3 arguments
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(90): error C2039: 'draw' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(90): error C2660: 'BasicApp::draw' : function does not take 3 arguments
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(98): error C2039: 'enable' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\basicapp\src\basicapp.cpp(98): error C3861: 'enable': identifier not found

FaceApp

1>  FaceApp.cpp
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(80): error C2039: 'draw' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(80): error C2660: 'FaceApp::draw' : function does not take 3 arguments
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(114): error C2039: 'draw' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(114): error C2660: 'FaceApp::draw' : function does not take 1 arguments
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(126): error C2039: 'drawStrokedRect' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(126): error C3861: 'drawStrokedRect': identifier not found
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(128): error C2039: 'drawSolidCircle' : is not a member of 'cinder::gl'
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\faceapp\src\faceapp.cpp(128): error C3861: 'drawSolidCircle': identifier not found

PointCloudApp

1>  PointCloudApp.cpp
1>c:\users\braitsch\code\cinder\blocks\cinder-kcb2\samples\pointcloud\src\pointcloudapp.cpp(39): fatal error C1083: Cannot open include file: 'cinder/MayaCamUI.h': No such file or directory
1>  Kinect2.cpp

Cheers,
Stephen

Unable to compile the samples

Hi there,
When I compiled the code to run the samples. it showed the following errors:

Error 1
error C2039: 'DeviceOptions' : is not a member of 'Kinect2'

Error 2
error C2228: left of '.enableColor' must have class/struct/union

Error 3
error C2660: 'Kinect2::Device::start' : function does not take 1 arguments

Error 4
error C3861: 'DeviceOptions': identifier not found

Error 5
error C2039: 'connectFrameEventHandler' : is not a member of 'Kinect2::Device'

The errors refers to the following lines of the code:

mDevice->start(Kinect2::DeviceOptions().enableColor()); //enable specific sources
mDevice->connectFrameEventHandler([&](const Kinect2::Frame& frame)
{
    mFrame = frame;
});

I am not sure if it is something related to the project configuration , or it is the "Kinect2" header which has an issue.

So I would like to know if you can help me to solve it ?

Thanks in advance.

Regards,

/ Abdullah

wieden-kennedy / cinder-kcb2 Goto Github PK

cinder-kcb2's People

Contributors

Stargazers

Watchers

Forkers

cinder-kcb2's Issues

include "cinder/app/App.h"

include "cinder/CameraUi.h"

include "cinder/gl/GlslProg.h"

include "cinder/gl/Texture.h"

include "cinder/gl/VboMesh.h"

include "cinder/params/Params.h"

include "cinder/gl/Texture.h"

include "cinder/gl/Draw.h"

include "cinder/gl/Scoped.h"

include "Kinect2.h"

include "cinder/app/RendererGl.h"

Recommend Projects

Recommend Topics

Recommend Org