Webgazer Calibration Issues

[LClarkeW]

Hi there, figured this was more of a jspsych issue than a webgazer issue seeing as I'm working with the webgazer-validate plugin. When running through a generic camera calibration, I run in to trouble pretty quickly after the jsPsychWebgazerCalibrateCamera plugin instantiation. I can successfully complete around 30 second of the calibration before this error is thrown:

Uncaught TypeError: calculateSampleRate(...) is null
    validation_done index.ts:213
    next_validation_point index.ts:157
    watch_dot index.ts:149
    watch_dot index.ts:144 
....

Tracked down the error to the aforementioned plugin-webgazer-validate that's imported (the current version published through npm). The troublesome function as follows:

function calculateSampleRate(gazeData) {
      var mean_diff = [];
      if (gazeData.length == 0) {
        return 0;
      }
      for (var i = 0; i < gazeData.length; i++) {
        if (gazeData[i].length > 1) {
          var t_diff = [];
          for (var j = 1; j < gazeData[i].length; j++) {
            t_diff.push(gazeData[i][j].t - gazeData[i][j - 1].t);
          }
          mean_diff.push(
            t_diff.reduce(function (a, b) {
              return a + b;
            }, 0) / t_diff.length
          );
        }
      }
      if (mean_diff.length > 0) {
        return (
          1000 /
          (mean_diff.reduce(function (a, b) {
            return a + b;
          }, 0) /
            mean_diff.length)
        );
      } else {
        return null;
      }
    }
  }
}

Best guess here is that gazeData is never correctly populated, so calculateSampleRate always returns null. I don't know enough about how webgazer functions internally to diagnose the problem. I'm running the experiment in browser with all my dependencies imported through npm. I was curious to know if anyone had run into a similar issue and had some wisdom to share. I am of course happy to provide any more specific information that may provide a clearer picture. Copied below is the timeline I export for the calibration:

import jsPsych from 'jspsych';
import CALIBRATION_CONFIG from './calibration-config';
import jsPsychHtmlButtonResponse from '@jspsych/plugin-html-button-response'
import jsPsychWebgazerCalibrateCamera from '@jspsych/plugin-webgazer-validate'
import jsPsychWebgazerValidateCamera from '@jspsych/plugin-webgazer-validate';
import jsPsychWebgazerInitCamera from '@jspsych/plugin-webgazer-init-camera';
import jsPsychFullscreen from '@jspsych/plugin-fullscreen';

var precheck_0 = {
  type: jsPsychHtmlButtonResponse,
  stimulus: `
    <p>This experiment uses your camera for eye tracking.</p>
    <p>No video is saved from this experiment. The only data recorded are predictions about where you are looking.</p>
    <p>Before we begin, there are a few things to do that can improve the accuracy of the eye tracking.</p>
  `,
  choices: ['Let\'s go!']
}

var precheck_1 = {
  type: jsPsychHtmlButtonResponse,
  stimulus: `
    <p>If you wear glasses and can complete the task without them, please remove your glasses.</p>
    <p>If you need to keep your glasses on to read this text, you can leave them on.</p>
  `,
  choices: ['OK']
}

var precheck_2 = {
  type: jsPsychHtmlButtonResponse,
  stimulus: `
    <p>Please close additional browser tabs and apps running in the background.</p>
    <p>The eye tracker works best when there are fewer things running at the same time.</p>
  `,
  choices: ['OK']
}

var precheck_3 = {
  type: jsPsychHtmlButtonResponse,
  stimulus: `
    <p>Try to avoid bright lights behind you.</p>
    <p>The eye tracker works best if your face is well lit from the front.</p>
    <p>If you can, adjust the lighting in your room.</p>
  `,
  choices: ['OK']
}

var precheck_4 = {
  type: jsPsychHtmlButtonResponse,
  stimulus: `
    <p>Turn off anything that may produce an alert or notification during the task.</p>
  `,
  choices: ['OK']
}

var camera_instructions = {
  type: jsPsychHtmlButtonResponse,
  stimulus: `
    <p>In order to participate you must allow the experiment to use your camera.</p>
    <p>You will be prompted to do this on the next screen.</p>
    <p>If you do not wish to allow use of your camera, you cannot participate in this experiment.<p>
    <p>It may take up to 30 seconds for the camera to initialize after you give permission.</p>
  `,
  choices: ['Got it'],
}

var init_camera = {
  type: jsPsychWebgazerInitCamera
}

var fullscreen_enter = {
  type: jsPsychFullscreen,
  fullscreen_mode: true
}

var calibration_instructions_1 = {
  type: jsPsychHtmlButtonResponse,
  stimulus: `
    <p>Now you'll calibrate the eye tracking, so that the software can use the image of your eyes to predict where you are looking.</p>
    <p>You'll see a series of dots appear on the screen. Look at each dot and click on it.</p>
  `,
  choices: ['Got it'],
}

var calibration = {
  type: jsPsychWebgazerCalibrateCamera,
  calibration_points: CALIBRATION_CONFIG.CALIBRATION_POINTS,
  repetitions_per_point: 3,
  randomize_calibration_order: true,
}

var validation_instructions_1 = {
  type: jsPsychHtmlButtonResponse,
  stimulus: `
    <p>Now we'll measure the accuracy of the calibration.</p>
    <p>Keep your head still, and move your eyes to look at the dots.</p>
    <p style="font-weight: bold;">You do not need to click on the dots this time.</p>
  `,
  choices: ['Got it'],
  post_trial_gap: 1000
}

var validation = {
  type: jsPsychWebgazerValidateCamera,
  validation_points: CALIBRATION_CONFIG.VALIDATION_POINTS,
  validation_point_coordinates: 'percent',
  show_validation_data: false,
  roi_radius: CALIBRATION_CONFIG.ROI_TOLERANCE_PIXELS,
  time_to_saccade: 1000,
  validation_duration: 2000,
  data: {
    task: 'validate'
  }
}

var recalibrate_instructions = {
  type: jsPsychHtmlButtonResponse,
  stimulus: `
    <p>The accuracy of the calibration is a little lower than we'd like.</p>
    <p>Let's try calibrating one more time.</p>
    <p>Keep your head still, look at the dots, and click on them.<p>
  `,
  choices: ['OK'],
}

var recalibrate = {
  timeline: [recalibrate_instructions, calibration, validation_instructions_1, validation],
  conditional_function: function(){
    var validation_data = jsPsych.data.get().filter({task: 'validate'}).values()[0];
    return validation_data.percent_in_roi.some(function(x){
      return x < CALIBRATION_CONFIG.MINIMUM_PERCENT_IN_ROI;
    });
  },
  data: {
    phase: 'recalibration'
  }
}

var calibrate_done = {
  type: jsPsychHtmlButtonResponse,
  stimulus: `
    <p>Great, we're done with calibration!</p>
    <p>Remember to keep your head still as you continue to the experiment.</p>
  `,
  choices: ['OK']
}

var initial_calibration_timeline = {
  timeline: [
    precheck_0,
    precheck_1,
    precheck_2,
    precheck_3,
    precheck_4,
    camera_instructions,
    init_camera,
    fullscreen_enter,
    calibration_instructions_1,
    calibration,
    validation_instructions_1,
    validation,
    recalibrate,
    calibrate_done
  ],
  data: {
    phase: 'initial-calibration'
  }
}

export default initial_calibration_timeline;

[jodeleeuw]

Hey @LClarkeW,

Are you using the original webgazer or the fork?

[LClarkeW]

I'm am now! Apologies, didn't read the documentation close enough. Unfortunately, the same error crops up after using the proper jspych webgazer fork (https://cdn.jsdelivr.net/gh/jspsych/jspsych@[email protected]/examples/js/webgazer/webgazer.js). Besides a couple new errors thrown that I've never seen, I don't really have much to go on.