Binary Thresholding based on distance

[jashshah999]

Hi, I want to implement binary thresholding based on distance. If the depth value of a pixel is between x and y, it should make it white, else make it black. Right now I am using a very crappy method (pixel-wise) but I was wondering if this repo has something that can make it fast because it is extremely slow right now. This is the current code,

import ch.bildspur.realsense.*;
import ch.bildspur.realsense.type.*;

RealSenseCamera camera = new RealSenseCamera(this);

int count = 0;

void setup()
{
  size(640, 480);
  camera.enableDepthStream(640, 480);
  camera.enableColorizer(ColorScheme.Cold);
  camera.start();
}

void draw()
{
  background(0);

  // read frames
  camera.readFrames();
  
  PImage depth = camera.getDepthImage();

  if (count == 2) {
    applyBinaryThreshold(depth, 0.64f, 1.01f);
  }
  
  image(depth, 0, 0, 640, 480);
}

void applyBinaryThreshold(PImage img, float lower, float upper) {
  img.loadPixels();
  for (int i = 0; i < img.pixels.length; i++) {
    float depth = camera.getDistance(i % img.width, i / img.width);
    if (depth >= lower && depth <= upper) {
      img.pixels[i] = color(255); // Set to white
    } else {
      img.pixels[i] = color(0); // Set to black
    }
  }
  img.updatePixels();
}

void mouseClicked(){
  println(mouseX, mouseY);
  println(camera.getDistance(mouseX, mouseY));
}

void keyPressed() {
  if (key == 'f' || key == 'F') {
    count = (count + 1) % 3; // Cycle through 0, 1, and 2
    
    camera.clearFilters(); // Clear existing filters
    
    switch(count) {
      case 0:
        // Raw output (no filters)
        println("Raw output");
        break;
      case 1:
        // Only temporal filter
        camera.addTemporalFilter(0.58f, 58, PersistencyIndex.ValidIn1_Last8);
        println("With temporal filter");
        break;
      case 2:
        // Only temporal filter (custom binary thresholding is applied in draw function)
        camera.addTemporalFilter(0.58f, 58, PersistencyIndex.ValidIn1_Last8);
        println("With temporal filter and custom binary thresholding");
        break;
    }
    
    camera.start(); // Restart the camera after applying filters
  }
}

Please help if there is a pre-existing function or if there is a faster way to do this.

[cansik]

There is an example on how to render the depth buffer: https://github.com/cansik/realsense-processing/blob/master/examples/UseDepthBuffer/UseDepthBuffer.pde

I think the performance issues are due to the GetDistance() method which is slow for so many calls. It's better to work on the depth buffer directly.

Also it makes sense to scale down the depth buffer with the DecimationFilter.

And you could use an opencv method for thresholding, which is implemented fast enough, or even better, implement a processing shader.

[jashshah999]

Sorry but I do not understand how this will help. How do I modify my binary thresholding function to incorporate this? Could you provide a sample where there is a function that takes as an input a PI image and then does the binary thresholding using the example you just mentioned - thank you

[cansik]

No, I'm not going to write code for you. I have provided examples which should be enough to help. What you have to do is to replace your GetDistance method with reading the depth buffer before the for-loops and then look it up during the for-loops.

If you have a more specific question about one of the hints, feel free to ask.

[jashshah999]

Thanks - I was not asking for the code but I just wanted to know how to use it because if I simply copy paste the code you provided - I do not get any output - the window is crashing so not sure what the issue is.

That is why I had asked for an example program. This should work out of the box right (?)

Please let me know if there is something I am missing