Data Loss When Integrating jsPsych Experiment with SONA

[ghost]

Hello,

I programmed an experiment using jsPsych, and I was using Firestore on Firebase to save my experiment data. When I was running my program locally, all of the data was successfully saved to Firebase. However, once I integrated SONA into my experiment (I added a SONA URL to my code that redirects the participants to a credit-granting page), I noticed that more than half of the data was not saved. I was wondering if anyone had any similar issues with data loss or had any potential solutions/pointers to resolve this problem?

In case this is useful, here is my original code:

/* Run the experiment. */
var sona_id = jsPsych.data.urlVariables()['sona_id'];

function writeData(data) {
    let doc_name = "Participant Data " + sona_id;
    var collection = firebase.firestore().collection(doc_name);
    let dataArray = data.values();
    for (let i = 0; i < dataArray.length; i++) {
        let document_num = 1;
        let newData = {
            id: sona_id,
            demographics: dataArray[i].demographics,
            Phase: dataArray[i].Phase,
            .
            .
            .
            responseTimeAssess: dataArray[i].responseTimeAssess
        };

        collection.add(newData); 
    }
}

jsPsych.init({
    timeline: timeline,
    on_finish: function(data) {
        console.log("Finished");
        window.location.assign("**sona_study_url**"+sona_id);
        let saveData;
        saveData = jsPsych.data.get().ignore(['correct', 'rt', 'stimulus', 'response', 'trial_type', 'trial_index',
            'internal_node_id', 'time_elapsed']).filterCustom(function(trial) {return trial.Phase != null});
        writeData(saveData);
    }
})

I also attempted to use an async function and await, but this resulted in the same amount of data loss on average:

/* Run the experiment. */
var sona_id = jsPsych.data.urlVariables()['sona_id'];

async function addCollection(collectionName, collectionData) {
    try {
        var writeOperation = await collectionName.add(collectionData);
        console.log(writeOperation);
    } catch {
        console.log("ERROR");
    }
}

function writeData(data) {
    let doc_name = "Participant Data " + sona_id;
    var collection = firebase.firestore().collection(doc_name);
    let dataArray = data.values();
    for (let i = 0; i < dataArray.length; i++) {
        let document_num = 1;
        let newData = {
            id: sona_id,
            demographics: dataArray[i].demographics,
            Phase: dataArray[i].Phase,
            .
            .
            .
            responseTimeAssess: dataArray[i].responseTimeAssess
        };

        addCollection(collection, newData);
    }
}

jsPsych.init({
    timeline: timeline,
    on_finish: function(data) {
        console.log("Finished");
        window.location.assign("**sona_study_url**"+sona_id);
        let saveData;
        saveData = jsPsych.data.get().ignore(['correct', 'rt', 'stimulus', 'response', 'trial_type', 'trial_index',
            'internal_node_id', 'time_elapsed']).filterCustom(function(trial) {return trial.Phase != null});
        writeData(saveData);
    }
})

Thank you very much for your time, and any help would be much appreciated!

[becky-gilbert]

Hi @rmtham, I think the problem is that this line redirects participants to a different page:

window.location.assign("**sona_study_url**"+sona_id)

And that line comes before the data saving bit of code. This is a problem because the data saving happens over the network and therefore is affected by the connection speed, which means that sometimes the participant is redirected before the data has saved. So instead what you want to happen is for the data to save first, and only when that's completed, redirect the participant to the SONA completion page.

To fix this, you can try moving the window.location.assign("**sona_study_url**"+sona_id) line so that it runs after data saving is complete, for instance by using JavaScript promises and the .then syntax, or by adding an "on success" callback function argument to your writeData function.

[ghost]

Hi @becky-gilbert ,

I just wanted to say a huge thank you for all your help and for pointing me in the right direction! It took me a while to figure out how to program this up, but I was able to resolve the data loss problem by using JavaScript promises and the .then syntax. In case this would be helpful to anyone in the future (and hopefully save them some time), here is how I modified my code:

// Write Data to Firebase
var sona_id = jsPsych.data.urlVariables()['sona_id'];

function writeData(data) {
    let doc_name = "Participant Data " + sona_id;
    // let doc_name = "Participant Data " + uniqueID();
    console.log(doc_name);
    var collection = firebase.firestore().collection(doc_name);
    let dataArray = data.values();
    let num_collection_added = 0;
    for (let i = 0; i < dataArray.length; i++) {
        let newData = {
            id: sona_id,
            demographics: dataArray[i].demographics,
            Phase: dataArray[i].Phase,
            .
            .
            .
            responseTimeAssess: dataArray[i].responseTimeAssess
        };

        num_collection_added++;

        const promise = collection.add(newData);

        if (num_collection_added === dataArray.length) {
            promise.then((value) =>
            {
                console.log(value);
                window.location.assign("**sona_url**" + sona_id);
            }).catch(error => {
                console.log(error);
            });
        }
    }
}

As a quick note: I included an if statement because I wanted the data to save only once all of the collections from the dataArray are added.

Thank you again for your help, I really appreciate it!

[becky-gilbert]

@rmtham wonderful! Great to hear that you resolved the problem, and thanks very much for sharing your code.