KinectProjectorToolkit without user recognition

[carlosjmramos]

Hello Gene,

I'm trying to adapt this code to project on other surface than the user body (a piece of cloth instead). I'm struggling with the DepthMap, because I need the DepthMap to apply a threshold to know where the piece of cloth is hanging in the rope. But if I use that method in your code I'm mixing DepthMap with DepthMaptoRealWorld and when the image is projected the shape/image projected doesn't match with the real cloth.

Do you have any hint how I can do this to project in other surface than the body? Because your code uses the user recognition in kinect to trigger the interaction. I'm checking the code in the classes of your library, trying to see how can I make it working without the user. Any help or hint appreciated :)

Thanks,
Carlos

[genekogan]

the main method you need to retrieve the right point is convertKinectToProjector(PVector kinectPoint). So you could apply the threshold to the depth image (maybe after making a copy of it), then use openCv to pull out the contours, then use convertKinectToProjector on each point in the contour. But in principle, this library probably needs to be updated for Processing 3/libfreenect.

[carlosjmramos]

I think I'm doing that, but the image projected doesn't match with the real cloth whereas if I use my body the projection fits perfectly.

Do you think it's because I'm thresholding using the kinect.depthMap ?

1st step - // Threshold the depth image

int[] rawDepth = kinect.depthMap();
for (int i=0; i < rawDepth.length; i++) {
if (rawDepth[i] >= minDepth && rawDepth[i] <= maxDepth) {
depthImg.pixels[i] = color(255);
} else {
depthImg.pixels[i] = color(0);
}
// Draw the thresholded image
depthImg.updatePixels();

2nd step - use openCv to pull out the contours and apply convertKinectToProjector on each point

opencv.loadImage(depthImg);

projectedContours = new ArrayList();
ArrayList contours = opencv.findContours();
for (Contour contour : contours) {
if (contour.area() > 10000) {
ArrayList cvContour = contour.getPoints();
for (int i=1; i == contour.numPoints(); i++){
PVector cvContourREAL = kpc.convertKinectToProjector(cvContour.get(i));
cvContour.set(i, cvContourREAL);
}
ProjectedContour projectedContour = kpc.getProjectedContour(cvContour, 1.0);
projectedContours.add(projectedContour);
}
}

But the when I draw the contours with your method, I have the mismatch...

[genekogan]

the second method is much better than the first. but it doesn't look like you are ever thresholding the opencv image. run opencv.threshold before getting the contours

[jing1234]

Hi Carlos @carlosjmramos,
Did you solve your problem? I want to project image on a box. Can you share your implement code to me ?
Thanks,
Jing

[jing1234]

@carlosjmramos Hi Carlos,
I'm totally new about this implement. For my project i'm really need your help. Please!!!

Thanks,
jing