centralCam Issue

[jgeller112]

@lkfink @ShreshthSaxena In a docker container, access to the usb camera is not allowed/throws an errors. What do you recommend in this case? Is it okay to just run SocialEyes natively outside the docker container?

[jgeller112]

I dont think there is a way to gain access to system webcam in docker container but I could be wrong.

[ShreshthSaxena]

Hi @jgeller112

It is possible to access the usb camera in the container if we pass it explicitly during container setup.

But you're right, it is totally possible to run the centralCam module outside the docker as well. It's really only the ADB and glasses API calls in glassesRecord module that rely on linux dependencies at the moment. All other modules should have cross-platform compatibility.

I'm assuming that your docker and centralCam will be running on the same computer, so in that case they're also sharing the same clock and won't require any further time sync.

[jgeller112]

Hi @ShreshthSaxena @lkfink!

We ran SocialEyes in the classroom today! Overall, it went pretty smoothly (Yay!). However, the centralCam video stopped recording after 40 minutes. It showed errors about the width and height not being an integer. I am not sure what that is about. Any ideas what happened and how we can fix it?

[ShreshthSaxena]

That is exciting to hear!

Was the centralCam connected as a peripheral (USB or integrated cam) or was it an IP/network camera? The heigth and width error reflects that the data received by SocialEyes wasn't an image frame so it might be a hardware or network related drop. There is a "frame_drop_threshold" parameter in CentralCam's config.json file, which sets a tolerance for dropped frames so one could increase it to be more tolerant for longer durations of data loss.

[jgeller112]

That is exciting to hear!

Was the centralCam connected as a peripheral (USB or integrated cam) or was it an IP/network camera? The heigth and width error reflects that the data received by SocialEyes wasn't an image frame so it might be a hardware or network related drop. There is a "frame_drop_threshold" parameter in CentralCam's config.json file, which sets a tolerance for dropped frames so one could increase it to be more tolerant for longer durations of data loss.

The webcam was attached as USB device. Any idea how high the frame drop threshold should be set?

[jgeller112]

Also, the video we recorded did not have a clear view of the slides/projector due to student heads. Is that an issue you encountered?

[lkfink]

Super cool you (nearly successfully) ran a study in the classroom today! I'm curious if you had any glasses issues at any point? We often need to restart a few of the devices remotely due to PL app crashes.

As for central cam: which camera are you using? And did you end up running in or outside Docker? (Just curious).

For head occlusions: SuperGlue (the model used for homography) is fairly robust to occlusions (which always occur for us with audience members, or in your case students, having people in the rows in front of them). But for the central cam, we tend to position above the audience, with a clear view of the stage, above everyone's heads. Could you do something similar in the classroom? It's not that the occlusions can't be handled but rather that the central camera frames will be the ones you would use for final analyses so it'll be nice to have a clear view for visualizations, etc.

[jgeller112]

Super cool you (nearly successfully) ran a study in the classroom today! I'm curious if you had any glasses issues at any point? We often need to restart a few of the devices remotely due to PL app crashes.

As for central cam: which camera are you using? And did you end up running in or outside Docker? (Just curious).

For head occlusions: SuperGlue (the model used for homography) is fairly robust to occlusions (which always occur for us with audience members, or in your case students, having people in the rows in front of them). But for the central cam, we tend to position above the audience, with a clear view of the stage, above everyone's heads. Could you do something similar in the classroom? It's not that the occlusions can't be handled but rather that the central camera frames will be the ones you would use for final analyses so it'll be nice to have a clear view for visualizations, etc.

We luckily did not have to restart any of the devices (ran for ~1 hour). To be honest, the TUI is a bit clunky--every time we run we have to restart ABD to get all the metrics to appear. For one participant, the recoding message disappeared but other metrics were populated (thankfully the data was captured).

Yeah. We are thinking about positioning the usb webcam above the audience.

[jgeller112]

We are running the TUI in docker and centralcam outside it.

It is not clear how to give docker access to usb webcam.

I think this should be updated in readme.

[jgeller112]

Super cool you (nearly successfully) ran a study in the classroom today! I'm curious if you had any glasses issues at any point? We often need to restart a few of the devices remotely due to PL app crashes.

As for central cam: which camera are you using? And did you end up running in or outside Docker? (Just curious).

For head occlusions: SuperGlue (the model used for homography) is fairly robust to occlusions (which always occur for us with audience members, or in your case students, having people in the rows in front of them). But for the central cam, we tend to position above the audience, with a clear view of the stage, above everyone's heads. Could you do something similar in the classroom? It's not that the occlusions can't be handled but rather that the central camera frames will be the ones you would use for final analyses so it'll be nice to have a clear view for visualizations, etc.

@lkfink when you have to restart the app/restart what do you do with the previous video file?

[jgeller112]

@ShreshthSaxena,

I played around with this some more.

Here is my json file. I increased frame_drop to 6000 and I still get this error

Failed to grab frame Error: expected a sequence of integers or a single integer, got '{'width': 1920, 'height': 1080}'

{
  "output_dir": "outputs/",
  "frame_drop_threshold": 6000,
  "show_preview": true,
  "camera_input": "standard",
  "standard": {
    "cam_src": 0,
    "write_to_raw": false,
    "fps_calc_window": 180, 
     "resolution": {
            "width": 1920,
            "height": 1080
        }
  },
    "arducam": {
        "arducam_config": "arducam_config/Mira220_MIPI_2Lane_1600x1400_12b_30fps.cfg",
        "verbose": 0,
        "scale_width": -1
    }
}

[jgeller112]

We are running another class session on Thursday and would love to get this working.

[jgeller112]

For some reason, I think this issue has to do with computer going to sleep? I plugged it into power and changed some of the sleep settings and now I can get longer recordings.

I also changed the drop threshold to 1000

[ShreshthSaxena]

Hi @jgeller112 sorry to just see this. Good to know that disabling sleep time seems to solve it!
It is indeed an error due to the hardware not sending frames anymore so it makes sense that the computer going to sleep causes it.