issues with image-audio-response plugin

[Divyanshi09]

Hi! I am using the image-audio-response plugin to design a language switching task: each image will be displayed along with a language cue telling the participant to name the image in one language or the other, each image will displayed for 3 seconds which is also the audio response duration. From the audio recordings thus generated, I will manually calculate the rt of their response.

The plugin is working fine (thanks for creating it!) but I have one issue. Even after setting 'wait_for_mic_approval' to false, it asks for mic permission before every trial. I don't know why it is still happening-- I would like for it to ask for mic approval before the trials begin, but not before every single trial.

Also, I don't quite understand how the audio recording is being saved in these trials-- to be honest I am not sure if I have managed to make jsPsych save audio recordings in my script:

        var switch_trial = {
        type: 'image-audio-response',
        stimulus: jsPsych.timelineVariable('stimulus'),
        buffer_length: 3000, 
        allow_playback: false,
        response_ends_trial: true,
        wait_for_mic_approval: false,
        post_trial_gap: 250,
        data: jsPsych.timelineVariable('data'),
        recording_indicator_type: 1
    };
    var practice_trial_procedure = {
        timeline: [fixation , switch_trial],
        timeline_variables: practice_trial_info,
        randomize_order: true
    };  
    timeline.push(practice_trial_procedure);

    jsPsych.init({
        timeline:timeline,
        on_data_update: function(data){
        },
        on_finish: function() {
        jsPsych.data.displayData();
    }
    });

I also do not understand the use of 'post processing'. Furthermore, I am going to upload this jsPsych script on Gorilla Experiment Builder and I was wondering if there is anything that I need to be aware of before transferring it to Gorilla-- will Gorilla save the audio files similar to how it saves jsPsych rt data?
Thank you for your time and forgive me for my lack of knowledge on this.

[jodeleeuw]

Hi @Divyanshi09,

The image-audio-response plugin isn't officially part of jsPsych yet, so you may have to direct questions at the author(s) of the plugin directly. Though sometimes the authors are also browsing this forum!

[becky-gilbert]

Hi @Divyanshi09, sorry for the delay getting back to you! The audio response plugins are still work in progress, but you can find some documentation on the image-audio-response plugin here.

Even after setting 'wait_for_mic_approval' to false, it asks for mic permission before every trial. I don't know why it is still happening-- I would like for it to ask for mic approval before the trials begin, but not before every single trial.

Are you currently running your task locally, by opening the file directly in the browser? Or on a server? If you're opening the file directly in the browser, then you're likely to be asked for mic permission on every trial. This should change once your task is running on a server. This issue is mentioned in the docs.

I also do not understand the use of 'post processing'.

The postprocessing parameter is explained in the docs here. By default, the audio recordings will be converted to a base 64 string so that they can be saved with the rest of the jsPsych data, then you can convert the base 64 strings back into audio files offline. This postprocessing parameter gives you the option of saving the audio recording to your server as a separate file - see here for an example of a postprocessing function that saves the data to a JATOS server.

will Gorilla save the audio files similar to how it saves jsPsych rt data?

I'm afraid I don't know whether Gorilla allows you to save audio files separately from the rest of the task data, but you could check with them. If not, you can use the default postprocessing function to save the audio as a base 64 string, just keep in mind that this can cause data storage and transfer problems if you're recording a lot of audio during your task (see the postprocessing section in the docs).