jsPsych.randomization.shuffleNoRepeats outputs something incorrect #2807

Oriane4972 · 2022-10-10T22:07:35Z

Oriane4972
Oct 10, 2022

Hello,

This is a follow-up to issue #2802. In that prior issue, the application was hanging because there was no permutation of trials that could satisfy all 4 constraints that I had placed on jsPsych.randomization.shuffleNoRepeats. Now, I'm facing a different issue: despite limiting my constraints to 2 (instead of 4), the function(a,b) { return (a.color === b.color) || (a.word === b.word) } outputs something incorrect. Specifically, despite asking that no two consecutive stimuli feature the same color or the same word, it sometimes happens.

I looked at the documentation for jsPsych.randomization.shuffleNoRepeats, but I seem to be following the example provided exactly, so I have no idea why my code does not prevent two identical words or colors from appearing in a row. I'm pasting the updated code below (the issue in on line 510).

Thank you for any help you may be able to provide!

<!DOCTYPE html>
<html>

<head>
    <title>Attempt</title>
    <style>
        .tab {
            display: inline-block;
            margin-left: 40px;
        }
        
        .center {
            margin: 0;
            position: fixed;
            top: 5%;
            left: 50%;
            -ms-transform: translate(-50%, -50%);
            transform: translate(-50%, -50%);
        }
    </style>
    <script src="https://unpkg.com/[email protected]"></script>
    <script src="https://unpkg.com/@jspsych/[email protected]"></script>
    <script src="https://unpkg.com/@jspsych/[email protected]"></script>
    <link href="https://unpkg.com/[email protected]/css/jspsych.css" rel="stylesheet" type="text/css" />
</head>

<body></body>
<script>
    var jsPsych = initJsPsych({
        on_finish: function() {
            //jsPsych.data.get().localSave('csv', 'mydata.csv');
            window.location = "https://yalesurvey.ca1.qualtrics.com/jfe/form/SV_5yWvFUxmyEt2mDs?subjectid=" + subject_id + ""
        },
        on_interaction_data_update: function(data) {
            console.log(JSON.stringify(data))
        }
    });

    var timeline = [];

    var numbers = ["one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten", "eleven", "twelve"];

    var colors = ["red", "green", "blue", "black"];

    var interaction_data = jsPsych.data.getInteractionData();
    // log data to console in json format
    console.log(interaction_data.json());

    var trial = {
        type: jsPsychBrowserCheck
    };

    // capture info from Prolific ID
    var subject_id = jsPsych.data.getURLVariable('subjectid');

    //jsPsych.data.addProperties({
    //    subject: subject_id
    //});

    var instructions = {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: `
    <p>In this agility test, you will see words presented in different colors.</p>
    <p>Your task is to indicate the <u>COLOR</u> that each word is printed in, 
        and ignore what the word actually says.</p>
    <p>Indicate the color of the word by pressing either of the following keys:</p>
    <p>- d for <b><FONT COLOR=red>red</FONT></b> words.</p>
    <p>- f for <b><FONT COLOR=green>green</FONT></b> words.</p>
    <p>- j for <b><FONT COLOR=blue>blue</FONT></b> words.</p>
    <p>- k for <b>black</b> words.</p>
    <p><u>Example:</u> if you see the word <b><FONT COLOR=green>RED</FONT></b> printed in the color green, press ' f ' for green words,
        regardless of the meaning of the word.</p>   
    <p>Press space to start the practice round.</p>
    <div style='width: 700px;'>
    </div>`,
        data: {
            task: 'instructions'
        },
        post_trial_gap: 1000
    };

    var instructions2 = {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: `
    <p>Get ready:</p>
    <p>Place your index and middle fingers over the ' d ', ' f ', ' j ', and ' k ' keys.</p>
    <div style='width: 700px;'>
    </div>`,
        choices: "NO_KEYS",
        data: {
            task: 'instructions'
        },
        trial_duration: 4000
    };

    var myArray_numbers = jsPsych.randomization.repeat(numbers, 1);

    //function choose_color() {
    //    return jsPsych.randomization.sampleWithoutReplacement(["red", "green", "blue", "black"], 1);
    //};
    //var myArray = ["red", "green", "blue", "black", "red", "green", "blue", "black", choose_color()[0], choose_color()[0]];

    var myArray_colors = jsPsych.randomization.repeat(colors, 3);

    //function shuffle_colors(tableau) {
    //    return jsPsych.randomization.shuffle(tableau);
    //};

    function shuffle_colors(tableau) {
        console.log("shuffle_colors", tableau);
        return jsPsych.randomization.shuffleNoRepeats(tableau, function(a, b) {
            return a === b
        });
    };

    var color_assignment = shuffle_colors(myArray_colors);
    console.log(color_assignment);

    function translation(color) {
        if (color === 'red') {
            return 'd';
        } else {
            if (color === 'green') {
                return 'f';
            } else {
                if (color === 'blue') {
                    return 'j';
                } else {
                    if (color === 'black') {
                        return 'k';
                    } else {
                        return "problem";
                    }
                }
            }
        }
    };

    function parameters(i) {

        var color_selected;
        var color_translated;

        var color_selected = color_assignment[i];
        var color_translated = translation(color_selected);

        return {
            stimulus: '<p style = "color: ' + color_selected + '; font-size: 60pt; "> ' + numbers[i] + ' <\/p>',
            word: myArray_numbers[i],
            color: color_selected,
            correct_response: color_translated,
            stim_type: "control"
        };
    };

    var practice_stimuli = [
        parameters(0),
        parameters(1),
        parameters(2),
        parameters(3),
        parameters(4),
        parameters(5),
        parameters(6),
        parameters(7),
        parameters(8),
        parameters(9),
        parameters(10),
        parameters(11)
    ];

    console.log(practice_stimuli);

    var fixation = {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: '<div style="font-size:60px;">+</div>',
        choices: "NO_KEYS",
        trial_duration: function() {
            return jsPsych.randomization.sampleWithoutReplacement([100, 150, 200, 250, 300, 350, 400, 450, 500], 1)[0];
        },
        data: {
            task: "fixation"
        },
        prompt: '<p class = "center" style = "color:grey;">d = red<span class="tab"></span><span class="tab"></span>f = green<span class="tab"></span><span class="tab"></span>j = blue<span class="tab"></span><span class="tab"></span>k = black</p>'
    };

    var practice_trial = {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: jsPsych.timelineVariable('stimulus'),
        choices: ["d", "f", "j", "k"],
        data: {
            task: 'practice',
            word: jsPsych.timelineVariable('word'),
            color: jsPsych.timelineVariable('color'),
            congruency: jsPsych.timelineVariable('stim_type'),
            correct_response: jsPsych.timelineVariable('correct_response')
        },
        prompt: '<p class = "center" style = "color:grey;">d = red<span class="tab"></span><span class="tab"></span>f = green<span class="tab"></span><span class="tab"></span>j = blue<span class="tab"></span><span class="tab"></span>k = black</p>',
        on_finish: function(data) {
            console.log(data);
            data.correct = jsPsych.pluginAPI.compareKeys(data.response, data.correct_response);
        }
    };

    var feedback_trial = {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: function() {
            if (jsPsych.data.get().last(1).values()[0].correct == 0) {
                return '<div style=" font-size: 60px;font-style: italic;">Incorrect</div>'
            } else {
                return '<div style=" font-size: 60px;font-style: italic;">Correct</div>'
            }
        },
        choices: "NO_KEYS",
        trial_duration: 1000,
        prompt: '<p class = "center" style = "color:grey;">d = red<span class="tab"></span><span class="tab"></span>f = green<span class="tab"></span><span class="tab"></span>j = blue<span class="tab"></span><span class="tab"></span>k = black</p>'
    };

    var practice_procedure = {
            timeline: [fixation, practice_trial, feedback_trial],
            timeline_variables: practice_stimuli,
            randomize_order: false,
            repetitions: 1
        }
        //timeline.push(practice_procedure);

    var repeat_practice_message = {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: `<p>Fewer than 80% of your responses were correct.</p>
            <p>For this reason, you must take another round of practice before taking the real test.</p>
            <p>Press space to re-start the practice round.</p>
            <div style='width: 700px;'>
            </div>`,
        data: {
            task: 'failed practice message'
        },
        post_trial_gap: 1000
    };

    function practice_accuracy() {
        const practice_trials = jsPsych.data.get().filter({
            task: 'practice'
        }).last(practice_stimuli.length);

        const practice_correct_trials = practice_trials.filter({
            correct: true
        });

        return practice_correct_trials.count() / practice_trials.count();
    };

    function practice_failed() {
        return practice_accuracy() < .8
    };

    var repeat_practice_conditional = {
        timeline: [repeat_practice_message],
        type: jsPsychHtmlKeyboardResponse,
        conditional_function: practice_failed,
    };

    var proceed_to_test = {
        timeline: [instructions, instructions2, practice_procedure, repeat_practice_conditional],
        loop_function: practice_failed,
    };
    timeline.push(proceed_to_test);

    var instructions3 = {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: `
    <p>Practice is over. We will now start the real test. </p>
    <p>Try to respond as <b>quickly</b> and <b>accurately</b> as you can, because you will be timed.</p>   
    <p>Press space when you are ready.</p> 
    <br></br>
    <p><u>Reminder</u>: Your task is to indicate the <u>COLOR</u> that each word is printed in, 
        and ignore what the word actually says.</p>
    <p>Indicate the color of the word by pressing either of the following keys:</p>
    <p>- d for <b><FONT COLOR=red>red</FONT></b> words.</p>
    <p>- f for <b><FONT COLOR=green>green</FONT></b> words.</p>
    <p>- j for <b><FONT COLOR=blue>blue</FONT></b> words.</p>
    <p>- k for <b>black</b> words.</p>
    <p><u>Example:</u> if you see the word <b><FONT COLOR=green>RED</FONT></b> printed in the color green, press ' f ' for green words,
        regardless of the meaning of the word.</p>
    <div style='width: 700px;'>
    </div>`,
        data: {
            task: 'instructions'
        },
        post_trial_gap: 2000
    };

    var test_stimuli = [{
        stimulus: "<p id = 'stroop_stim' style = 'color: red; font-size: 60pt;'>red<\/p>",
        word: "red",
        color: "red",
        correct_response: "d",
        stim_type: "congruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: red; font-size: 60pt;'>red<\/p>",
        word: "red",
        color: "red",
        correct_response: "d",
        stim_type: "congruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: red; font-size: 60pt;'>red<\/p>",
        word: "red",
        color: "red",
        correct_response: "d",
        stim_type: "congruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: green; font-size: 60pt;'>red<\/p>",
        word: "red",
        color: "green",
        correct_response: "f",
        stim_type: "incongruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: blue; font-size: 60pt;'>red<\/p>",
        word: "red",
        color: "blue",
        correct_response: "j",
        stim_type: "incongruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: black; font-size: 60pt;'>red<\/p>",
        word: "red",
        color: "black",
        correct_response: "k",
        stim_type: "incongruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: red; font-size: 60pt;'>green<\/p>",
        word: "green",
        color: "red",
        correct_response: "d",
        stim_type: "incongruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: green; font-size: 60pt;'>green<\/p>",
        word: "green",
        color: "green",
        correct_response: "f",
        stim_type: "congruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: green; font-size: 60pt;'>green<\/p>",
        word: "green",
        color: "green",
        correct_response: "f",
        stim_type: "congruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: green; font-size: 60pt;'>green<\/p>",
        word: "green",
        color: "green",
        correct_response: "f",
        stim_type: "congruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: blue; font-size: 60pt;'>green<\/p>",
        word: "green",
        color: "blue",
        correct_response: "j",
        stim_type: "incongruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: black; font-size: 60pt;'>green<\/p>",
        word: "green",
        color: "black",
        correct_response: "k",
        stim_type: "incongruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: red; font-size: 60pt;'>blue<\/p>",
        word: "blue",
        color: "red",
        correct_response: "d",
        stim_type: "incongruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: green; font-size: 60pt;'>blue<\/p>",
        word: "blue",
        color: "green",
        correct_response: "f",
        stim_type: "incongruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: blue; font-size: 60pt;'>blue<\/p>",
        word: "blue",
        color: "blue",
        correct_response: "j",
        stim_type: "congruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: blue; font-size: 60pt;'>blue<\/p>",
        word: "blue",
        color: "blue",
        correct_response: "j",
        stim_type: "congruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: blue; font-size: 60pt;'>blue<\/p>",
        word: "blue",
        color: "blue",
        correct_response: "j",
        stim_type: "congruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: black; font-size: 60pt;'>blue<\/p>",
        word: "blue",
        color: "black",
        correct_response: "k",
        stim_type: "incongruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: red; font-size: 60pt;'>black<\/p>",
        word: "black",
        color: "red",
        correct_response: "d",
        stim_type: "incongruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: green; font-size: 60pt;'>black<\/p>",
        word: "black",
        color: "green",
        correct_response: "f",
        stim_type: "incongruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: blue; font-size: 60pt;'>black<\/p>",
        word: "black",
        color: "blue",
        correct_response: "j",
        stim_type: "incongruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: black; font-size: 60pt;'>black<\/p>",
        word: "black",
        color: "black",
        correct_response: "k",
        stim_type: "congruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: black; font-size: 60pt;'>black<\/p>",
        word: "black",
        color: "black",
        correct_response: "k",
        stim_type: "congruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: black; font-size: 60pt;'>black<\/p>",
        word: "black",
        color: "black",
        correct_response: "k",
        stim_type: "congruent"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: red; font-size: 60pt;'>xxxx<\/p>",
        word: "xxxx",
        color: "red",
        correct_response: "d",
        stim_type: "control"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: red; font-size: 60pt;'>xxxx<\/p>",
        word: "xxxx",
        color: "red",
        correct_response: "d",
        stim_type: "control"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: red; font-size: 60pt;'>xxxx<\/p>",
        word: "xxxx",
        color: "red",
        correct_response: "d",
        stim_type: "control"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: green; font-size: 60pt;'>xxxx<\/p>",
        word: "xxxx",
        color: "green",
        correct_response: "f",
        stim_type: "control"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: green; font-size: 60pt;'>xxxx<\/p>",
        word: "xxxx",
        color: "green",
        correct_response: "f",
        stim_type: "control"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: green; font-size: 60pt;'>xxxx<\/p>",
        word: "xxxx",
        color: "green",
        correct_response: "f",
        stim_type: "control"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: blue; font-size: 60pt;'>xxxx<\/p>",
        word: "xxxx",
        color: "blue",
        correct_response: "j",
        stim_type: "control"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: blue; font-size: 60pt;'>xxxx<\/p>",
        word: "xxxx",
        color: "blue",
        correct_response: "j",
        stim_type: "control"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: blue; font-size: 60pt;'>xxxx<\/p>",
        word: "xxxx",
        color: "blue",
        correct_response: "j",
        stim_type: "control"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: black; font-size: 60pt;'>xxxx<\/p>",
        word: "xxxx",
        color: "black",
        correct_response: "k",
        stim_type: "control"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: black; font-size: 60pt;'>xxxx<\/p>",
        word: "xxxx",
        color: "black",
        correct_response: "k",
        stim_type: "control"
    }, {
        stimulus: "<p id = 'stroop_stim' style = 'color: black; font-size: 60pt;'>xxxx<\/p>",
        word: "xxxx",
        color: "black",
        correct_response: "k",
        stim_type: "control"
    }];

    var repeatedSet = jsPsych.randomization.repeat(test_stimuli, 4);
    var shuffled = jsPsych.randomization.shuffleNoRepeats(repeatedSet, function(a, b) {
        return (a.color === b.color) || (a.word === b.word)
    });

    console.log(shuffled);

    var test_trial = {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: jsPsych.timelineVariable('stimulus'),
        choices: ["d", "f", "j", "k"],
        data: {
            task: 'test',
            word: jsPsych.timelineVariable('word'),
            color: jsPsych.timelineVariable('color'),
            congruency: jsPsych.timelineVariable('stim_type'),
            correct_response: jsPsych.timelineVariable('correct_response')
        },
        prompt: '<p class = "center" style = "color:grey;">d = red<span class="tab"></span><span class="tab"></span>f = green<span class="tab"></span><span class="tab"></span>j = blue<span class="tab"></span><span class="tab"></span>k = black</p>',
        on_finish: function(data) {
            console.log(data);
            data.correct = jsPsych.pluginAPI.compareKeys(data.response, data.correct_response);
        }
    };

    var test_procedure = {
        timeline: [fixation, test_trial],
        timeline_variables: shuffled,
        randomize_order: false,
        repetitions: 1
    };

    var debrief_block = {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: function() {
            var trials = jsPsych.data.get().filter({
                task: 'test'
            });
            var correct_trials = trials.filter({
                correct: true
            });
            var accuracy = Math.round(correct_trials.count() / trials.count() * 100);
            var rt = Math.round(correct_trials.select('rt').mean());

            return `<p>You responded correctly on ${accuracy}% of the trials.</p>
      <p>Your average response time was ${rt} milliseconds.</p>
      <p>Your performance on this test would pass the bar for most industries.</p>
      <p>Press any key to complete the experiment. Thank you!</p>`;

        },
        data: {
            task: 'debrief'
        }
    };

    var actual_test = {
        timeline: [instructions3, test_procedure, debrief_block]
    };
    timeline.push(actual_test);

    jsPsych.run(timeline);
</script>

</html>

Answered by jodeleeuw

Oct 11, 2022

Hi again @Oriane4972,

There was indeed a bug in the shuffleNoRepeats function in the case where the equalityFunction used a logical OR to check for equality. I've patched this with #2811, and it will be fixed when we publish v7.3.1 shortly.

Thanks for bringing this to our attention!

View full answer

jodeleeuw · 2022-10-11T21:20:12Z

jodeleeuw
Oct 11, 2022
Maintainer

I just wrote a test case for this and can confirm something is buggy here. I think it has to do with the fact that the array items are objects and we hadn't tested on that before. I am working on a fix and will let you know what I figure out.

0 replies

jodeleeuw · 2022-10-11T22:15:39Z

jodeleeuw
Oct 11, 2022
Maintainer

Hi again @Oriane4972,

There was indeed a bug in the shuffleNoRepeats function in the case where the equalityFunction used a logical OR to check for equality. I've patched this with #2811, and it will be fixed when we publish v7.3.1 shortly.

Thanks for bringing this to our attention!

3 replies

jodeleeuw Oct 12, 2022
Maintainer

7.3.1 is now published, and this should now work if you use it. Let me know if there are still problems!

Oriane4972 Oct 12, 2022
Author

Thanks @jodeleeuw! Do you know roughly when we can expect v7.3.1 to be released?

In the meantime, where can I find instructions for building jspsych from source?

Oriane4972 Oct 12, 2022
Author

Whoops! Looks like we both hit "send" at the same time. This is fantastic news -- thank you!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

jsPsych.randomization.shuffleNoRepeats outputs something incorrect #2807

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments 3 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

jsPsych.randomization.shuffleNoRepeats outputs something incorrect #2807

Uh oh!

Uh oh!

Oriane4972 Oct 10, 2022

Replies: 2 comments · 3 replies

Uh oh!

jodeleeuw Oct 11, 2022 Maintainer

Uh oh!

Uh oh!

jodeleeuw Oct 11, 2022 Maintainer

Uh oh!

jodeleeuw Oct 12, 2022 Maintainer

Uh oh!

Oriane4972 Oct 12, 2022 Author

Uh oh!

Oriane4972 Oct 12, 2022 Author

Oriane4972
Oct 10, 2022

Replies: 2 comments 3 replies

jodeleeuw
Oct 11, 2022
Maintainer

jodeleeuw
Oct 11, 2022
Maintainer

jodeleeuw Oct 12, 2022
Maintainer

Oriane4972 Oct 12, 2022
Author

Oriane4972 Oct 12, 2022
Author