pixel data from RPiVideoGrabber?

[owenplanchart]

Hi All,

pi3b (buster). Pi cam v2.1

Does the pixel data from RPiVideoGrabber.getPixels() get sent to the cpu or is it somehow gpu only. I am having no luck getting openCV to see the pixel data from this function but not entirely sure how this addon works under the hood.

Do I need to convert it into an fbo?

Any help appreciated

[jvcleave]

It uses an fbo internally to get it to the CPU.

[owenplanchart]

Thank you for your prompt reply,

In that case can you see any reason why this code wouldn't work with ofxAruco. This sketch works with the ps3eye but I cant get it to work with Pi camera module using the RPiVideoGrabber. The video feed works but I cannot see the marker:

void ofApp::setup(){
    // OSC sender info
    sender.setup(HOST, PORT);
    ////////////////////////////
    
    ofSetFrameRate(60);
    ofSetWindowTitle("Chandelier Tracker");
    ofSetVerticalSync(true);
    ofSetLogLevel(OF_LOG_VERBOSE);
    string boardName = "boardConfiguration.yml";
    float markerSize = 0.15;
    string markerFile = "marker.xml";
    string cameraIntrinsics = "intrinsics.yml";
 
    doDrawInfo	= true;
		
    consoleListener.setup(this);

    ofxOMXCameraSettings settings;
    videoGrabber.initGrabber(1280,720);
    settings.enablePixels = true;
    settings.enableTexture = true;
    filterCollection.setup();    
    
    aruco.setup("intrinsics.int", videoGrabber.getWidth(), videoGrabber.getHeight(), boardName, markerSize);

    aruco.getBoardImage(board.getPixels());
    board.update(); 
    showMarkers = true;
    showBoardImage = false;
    
    ofEnableAlphaBlending();
 }

//--------------------------------------------------------------
void ofApp::update(){
    // do not use update function on the Pi
 }

//--------------------------------------------------------------
void ofApp::draw(){
    ofSetColor(255); 
    videoGrabber.update();
    if(videoGrabber.isFrameNew()){
        aruco.detectBoards(videoGrabber.getPixels());
     }
    videoGrabber.draw(0, 0);

    if (showMarkers) {
        vector <aruco::Marker> markers = aruco.getMarkers();
        for (int i = 0; i<aruco.getNumMarkers(); i++) {
            aruco.begin(i);
            drawMarker(0.15, ofColor::white);
            aruco.end();
        }
    }

[jvcleave]

maybe change

 if(videoGrabber.isFrameNew()){
        aruco.detectBoards(videoGrabber.getPixels());
     }

to

        aruco.detectBoards(videoGrabber.getPixels());

[owenplanchart]

that didn't do it unfortunately.

[owenplanchart]

I noticed that the original ofxAruco code uses
in the .h file

ofVideoGrabber grabber
ofBaseVideoDraws * video

and then in the .cpp file
video = &grabber

I couldnt do that with the RPiVideoGrabber, could this have something to do with it?

[pranavb104]

@owenplanchart I tried doing something similar with the Aruco markers and pi camera. You need to convert the image type to RGB from RGBA. The ofxAruco library only takes in RGB (3 channel) data for processing. :D