ks07 / gpkinectclient Goto Github PK

Occlusion destroys the shape detection.

Need to crop off pixels from the top edge (when transposed).

Max loss I've seen is 22 (which happens more often than not), although if we can deal with a small bit of distortion in the top left of the image we can possibly go back to 15. Probably the neatest solution is to cut it off at 20, and blur away the small extremities (which we will be doing anyway).

Some smaller boxes can catch out the rectangle detector, especially when we are unlucky with sensor noise on corners or edges of the box. Given a spout of bad luck we may also detect rectangles in sensor noise. Some potential improvements I can see:

• Re-introduce the filtering/averaging across frames used by Chris in early Kinect work, to reduce the probability of holes due to noise.
• Average the content of a detected rectangle and check that it matches the layer/box we think we are looking at. We might be able to tweak the rotation to minimise the amount of whitespace captured by the bounding box.
• Use alternative detection to FindContours, or increase confidence with this second metric. Hough lines?
• Add dimensions or area to box definitions, and filter out detected rectangles that are not close enough to the expected numbers.
• Add a simple button-press user verification step to the game, if projection is powerful enough.
• Tweak parameters to the poly approximation algorithm, or contour detector.
• Don't discount polys with >4 edges if we can do our own approximation to a rectangle (i.e. by checking angles between 'corners' and along 'lines' are within some acceptable tolerances)

This ties in with issue #9 - some of these may not be suitable if we choose to double up boxes per layer, and some of them may indeed be necessary.

Need to make sure that detected geometry does not cover excluded zones (i.e. bases). There is currently some rudimentary protection against this but it's not ideal. We might want to physically block the bases in the real world - better for us and the players.

The new thresholding approach is limited in the number of boxes we can detect, at least without a more accurate kinect or a bigger play area. We could alleviate this by allowing boxes with the same height, with sufficiently differing geometries that we can split a solid shape into two squares accurately. Perhaps a square box and very-non-square box (also know as rectangular) would make this easy. Drastically different sizes would also help.

We want to be able to see markers for abilities.

We could add detection for shapes other than rectangles.

We need to create some boxes with differing heights. Ideally coated for projection, and ideally weighted at the bottom to force them into the right orientation.

Being able to switch to debug views without recompilation would be helpful.

We may want to recalibrate on the fly. Need to add support to update the calibration from a remote request.

Detection needs to be calibrated for our chosen Kinect/object setup.

Fallback frames need to be calibrated properly, and need to check that they actually work nicely. Layer definitions may also need to be set for these as we might not be able to ensure the rig keeps things constant enough (as we calibrate based on the mean it is possible for us to accidentally shift all values slightly).

We might want to store frames taken in the current session and use these as fallback images, rather than an old image.