Visual AND auditory stimuli in the same trial

[Miko999]

Hello!

I'm quite new to javascript and jspsych and am attempting to implement a metronome response task (see: https://doi.org/10.3758/s13414-020-02131-x), mostly using the audio-keyboard-response plugin.
This is a continuous performance task with regularly spaced audio stimuli (i.e., metronome beats), where participants have to tap a key along with the stimuli. The stimuli must be equally spaced to be 'rhythmic' for the purposes of this task.
The original had visual stimuli appear when participants responded, depending on the timing of their response relative to the auditory stimuli. I wanted to do something similar to show that the participants' response has been recorded. I could add this in using "on_finish", but I'm not sure if I can tie it to the response within the same trial. Along with that, I wanted to have a fixation cross on screen at all times (though it could be covered by the visual stimuli from the response).

So basically:

• Can I get a visual stimulus to appear during an audio-keyboard-response trial when the participant responds, regardless of when they respond during the trial? (If so, how?)
• Can I have a background image in audio-keyboard-response trials, to include a fixation cross? (If so, how?)

Any advice would be much appreciated!
I have to do this all in Gorilla(.sc), since that's how my supervisor wants us to implement online tasks, and jspsych seems like the best option (the Gorilla 'task creator' GUI has very limited logic/branching options).

[jodeleeuw]

Hi @Miko999

Can I get a visual stimulus to appear during an audio-keyboard-response trial when the participant responds, regardless of when they respond during the trial? (If so, how?)

Can you tell us a little more about the task structure? Is one trial a long audio clip with many metronome beats? Does the visual stimulus appear for a short time after each keypress? I think the way to solve your problem will depend on how the trials need to be organized.

Can I have a background image in audio-keyboard-response trials, to include a fixation cross? (If so, how?)

The best way to do this will probably depend on how you answer the other questions, but the simplest option is to use the prompt parameter:

const trial = {
	type: jsPsychAudioKeyboardResponse,
	stimulus: 'audio.mp3',
	prompt: '<p style="font-size: 48px;">+</p>'
}

[Miko999]

Hello @jodeleeuw !
Thanks for your response and for making this a discussion (I'm still not use to github).

For my version, each 'trial' is 2000 ms (for now) with a 75 ms audio clip, making each trial one beat, where participants can respond by key press at any time but are trying to sync their response with the beat. Each response triggers a visual stimulus for a short period (I don't know the exact of time in the original study; short enough to be noticeable but not so long as to disrupt the next trial).
At every 20th trial/beat, they must respond with a different key press (hence metronome COUNTING task). This would trigger a slightly different visual stimulus when participants are correct, and they start back at 'trial 1' in a new 'block'.
They are also able to self-report with third key when they lose count.
The 'blocks' are actually 25 'trials' because we let them go a bit after the 20th beat, to see if they miscounted or do not notice that they miscounted.

I hope that makes sense.
It's not perfectly accurate to call them 'blocks' because the task length is dictated by number of trials, rather than number of blocks. I've been attempting to make two 'counters' for trial number within a block and total trial number across blocks, to implement conditional loops for this.

Whenever participants incorrectly indicate the 20th beat, report losing count, or don't do either before 25 trials are up, they answer a few questions about their thoughts before their last response. After this, the count resets and they start a new block.

The original version was written up in python and is available on their osf page if that helps: https://osf.io/7jxny

[jodeleeuw]

I think with the precise timing needs you'll probably get the best results with the WebAudio API and a custom plugin. You could connect together a bunch of individual trials using the audio-keyboard-response plugin, but I'm worried that there will be lags that are noticeable at 30 beats per minute. With the WebAudio API, you could schedule all of the playback very precisely.

Have you tried stringing together a series of trials with the audio-keyboard-response plugin to see if the beat is steady?

[Miko999]

I am using the audio-keyboard-response plugin and it seems to do fine strung together or with a node for repetitions.
My problem is that I don't know how to get a visual stimulus to appear after the participants respond.
I also can't seem to get the counters to work.

Here's what part of my code looks like for the trials:

  var beat = 1;
   var trial = 1;

   var ptrials = {
       type: "audio-keyboard-response",
       stimulus: beatsound,
       choices: ['downarrow','rightarrow','uparrow'],
       trial_duration: 2000,
       response_ends_trial: false,
       data: {
           block: 'practice',
           beat: beat,
           trial: trial
       },
      on_finish: function(data) {
          //beat++; 
          //trial++
// I've tried many iterations of increasing 'beat' and 'trial' counters and nothing seems to work.
          if(data.key_press == '39') {
              return "<p> right key pressed </p>";
// I've also tried this with 'rightarrow' instead of '39' and data.reponse
          }
      }
   };

var pprocedure = {
       timeline: [ptrials],
       repetitions: 3,
     // on_trial_finish: beat = beat + 1
// another spot where I tried many iterations of attempts to increase the counter
   };

   timeline.push(pprocedure);

I've look up other discussions and jspsych documentation but this is turning into a lot of trial and error. It likely doesn't help that I'm doing this in Gorilla.