change/remove stimuli on audio response onset

[ssantomartino]

Hi everyone,

I'm pretty new to jsPsych and still trying to figure it out... I am trying to create an experiment where the participant sees an image and they then have to name the image. So far, this is just the jspsych-image-audio-response plugin from @becky-gilbert

However, I need to modify the plugin a bit so that once the participant begins to name the image, it disappears. This is the part I am having trouble with... is there a way to determine the onset of the recording (aka when they start speaking) and then remove the stimuli upon this action?

Any help is appreciated!

Thanks,
S

[becky-gilbert]

Hi @ssantomartino, this is probably a little tricky to do, but would be very cool if you get it to work! You'll need to do some real-time processing of the audio input. Here are some posts on stack overflow and JavaScript libraries related to detecting the onset of speech or silence:
https://stackoverflow.com/a/62212935
https://stackoverflow.com/questions/24515978/html-audio-recording-until-silence
https://github.com/solyarisoftware/WeBAD
https://stackoverflow.com/questions/46543341/how-can-i-extract-the-preceding-audio-from-microphone-as-a-buffer-when-silence/46781986#46781986
https://stackoverflow.com/a/54130720
I think most of these approaches are similar in that they use the Web Audio API's script processor node or analyser node to get access to the real-time audio input data, and then use some logic to test whether the input is above/below some threshold for some given period of time. The basic example on the analyser node docs page looks useful, as it shows how to access the incoming audio data in order to draw changes in amplitude/frequency over time on the page.

Let us know if need any further guidance. And please let us know if you get it working, as I'm sure others would be interested!