Discussion: meetings/2025-06-24

[hbelski]

This week Lucas (@lrudelt) and Nicholas (@Dedalus9) will present their statistical analysis.

[jeromelecoq]

If time permits, we can go back to the discussion of variants for the next SLAP2 experiments as well as lessons learned for the Neuropixels and Two photon experiments.
https://docs.google.com/presentation/d/1WFPHhe7XP8DtXoCHj_VGkGm4ukBfVv69UaC4gMWOlio/edit?usp=sharing

[Dedalus9]

Hi all,
I won’t have statistical power analysis results to present today, but I will have them ready for the next meeting. I plan to share preliminary results as soon as possible, ideally by Wednesday or Thursday.

The analysis will focus on estimating the amount of experimental data required to reliably detect changes in ROI tuning—specifically, how much data is needed to determine how an ROI's bassline tuning impacts the orientation tuning profile across the 11 tested orientations.

[Dedalus9]

@KasparP You had mentioned that there were measurements of the magnitude of photobleaching. Where are these measurements?

[lrudelt]

Hi all,
I also had a first look into sample size dependence of interesting measures regarding the oddball analysis. It got delayed, because I first wanted to replicate the analysis by Hamm et al. PNAS 2021 more closely, for better comparability. In that study, they quantify differences in stimulus responses averaged over a population of responsive units (units that are more active for the given stimulus than some factor above the std of baseline). When I apply this analysis to the SLAP2 data I also find a more pronounced depression for 0 and 45 degrees (see below for DMD 1 from the 2025-05-08 data set)

To investigate how this analysis of population averages depends on sample size, I subsampled the available trials and computed the mean and confidence intervals for the subsamples. From this plot, one can see that already with 20 to 30 trials one can safely distinguish the conditions.

However, any analysis of single ROI properties will depend much more sensitively on the number of trials available per condition. For example, as a rule of thumb, the minimum of trials to compute std is about 30 trials.

[lrudelt]

To tackle how the discriminability of the responses of individual ROIs between oddball and control conditions behaves as a function of number of trials, I followed a suggestion by @Sarruedi. Sarah suggested to use the area under the ROC curve of a classifier that tries to classify trials as oddball or control, based on the average response in the stimulus window of that trial.

I used a simple classifier that classifies the trial as the second condition if the trial response is larger or smaller than a certain decision threshold, depending on the sign of the mean difference between the two conditions. The threshold is varied systematically to study discriminability across thresholds (see ROC curve below).

Below, I show the ROC curves for the different orientations that occur during the oddball block, as well as the three comparisons we have discussed so far (control 1 vs oddball, control 2 vs oddball, control 1 vs control 2). It is obtained by sweeping the decision threshold, affecting both the number of true and false positives.

Computing the area under the curve (AUC) gives a measure how well the two conditions can be discriminated. I repeated the above analysis a hundred of times for each trial number, subsampling the available trials and computing the ROC AUC. The following plot summarizes this analysis.

Interestingly, the ROC AUC does not decrease with number of trials, which I thought initially, but rather has a small positive bias for very small number of trials. So I think how to read this plots is rather when the ROC AUC gives consistent results, because the discriminability of two conditions should be a quantity that is ideally invariant to the concrete realization that we see. It looks that we reach this consistent AUC in most cases, but it is important to note that this is also because we reach the maximum number of available trials, thus we are always sampling close to the same thing.

[jeromelecoq]

To tackle how the discriminability of the responses of individual ROIs between oddball and control conditions behaves as a function of number of trials, I followed a suggestion by @Sarruedi. Sarah suggested to use the area under the ROC curve of a classifier that tries to classify trials as oddball or control, based on the average response in the stimulus window of that trial.

I used a simple classifier that classifies the trial as the second condition if the trial response is larger or smaller than a certain decision threshold, depending on the sign of the mean difference between the two conditions. The threshold is varied systematically to study discriminability across thresholds (see ROC curve below).

Below, I show the ROC curves for the different orientations that occur during the oddball block, as well as the three comparisons we have discussed so far (control 1 vs oddball, control 2 vs oddball, control 1 vs control 2). It is obtained by sweeping the decision threshold, affecting both the number of true and false positives.

Computing the area under the curve (AUC) gives a measure how well the two conditions can be discriminated. I repeated the above analysis a hundred of times for each trial number, subsampling the available trials and computing the ROC AUC. The following plot summarizes this analysis.

Interestingly, the ROC AUC does not decrease with number of trials, which I thought initially, but rather has a small positive bias for very small number of trials. So I think how to read this plots is rather when the ROC AUC gives consistent results, because the discriminability of two conditions should be a quantity that is ideally invariant to the concrete realization that we see. It looks that we reach this consistent AUC in most cases, but it is important to note that this is also because we reach the maximum number of available trials, thus we are always sampling close to the same thing.

Very helpful! Thanks Lucas

[jeromelecoq]

Here is what I get if we follow the stats we discussed :

[jeromelecoq]

Adding a block that would present all conditions 16 times just before the oddball block is also possible. That would cost us :
0.686 * 4 * 16 ( ie, 0deg, 45 deg, 90 deg, pure gray) = 43 seconds for each serie.

[lrudelt]

Thanks Jerome, this looks great!

Sorry for taking the discussion over time yesterday.

I would like to post one result here that might be important to consider when finalizing the future stimulus design next week. I split the oddball and control blocks into subblocks and quantified the mean response plus 95 % bootstrapping CIs to see how responses evolve with time. I did this for both, ROIs that are responsive during the first control block, and ROIs that are active during the second control block. The bar widths do not reflect time! I split the oddball block for 45 and 90 trials only into 4 subblocks because we have much fewer trials available as for the standard 0 degree direction.

Averaged response of ROIs active during the first control block

Averaged response of ROIs active during the second control block

Important points

• Synaptic adaptation/depletion: When considering ROIs that are active during the first control (synapses that are strong and could subject to synaptic depletion) it looks that there is synaptic adaptation/depletion that could evolve over long times (one to three minutes or so). We have found similarly long timescales of synaptic depletion in hippocampal synapses recently (https://www.nature.com/articles/s42003-025-07956-6). The transient decrease during first control could be mediated by habituation at the beginning of the experiment as suggested by @KasparP and @koweiss. One thing that will be hard to account for is a potential transient release from depression after the oddball block (for 0 and 45 degrees, especially). The fact that depression from the oddball might affect responses for potentially minutes makes responses at the second control block to appear smaller than the first control (this might be the reason for the uniform depression of ROIs across orientations that @Sarruedi saw in her plots ).
• Long-term changes due to plasticity: Especially for ROIs that are active during control 2, one can nicely see that there are non-transient, long-term changes in responses (for 0 and 45 degrees). Thus in the stimulus design above the second oddball block will be very likely affected by these plasticity changes. This makes it hard to compare responses across predictive contexts (oddball block 1 and 2), because responses might be different not only because of a different predictive context, but also because of plasticity induced by oddball block 1.

Remark to stimulus design: My concern yesterday was that in the suggested stimulus design, we might run into problems to disentangle response differences that are caused by different predictive context, and long-term plasticity. Both are very interesting to study, but maybe we could do so using two separate stimulus designs?

Proposal:

• For studying learning, change the above suggestion by replacing the first oddball block by a learning control (e.g., grey screen or just a very long orientation block) to show that it is the repeated presentation of a grating that drives the plasticity and not just representational drift (as discussed in the previous meeting).
• For studying contextual modulation (predictive processing), take out the control block 2 and instead have one long oddball block where the standard orientation changes every three minutes or so. This ensures that both predictive contexts are similarly affected by plasticity. This way, we could also increase the number of oddball trials a bit to improve statistical power.
• As @KasparP suggested, we might want to take out the omission oddball (at least for the learning stimulus) to give us more trials

[jeromelecoq]

This is a very insightful analysis !
Indeed clearly there is dynamic that is established as each block unfold
We are planning an exp next Monday. Based on this, I propose we extend the first orientation block as Karim and you suggested. Maybe twice as long ?

Then we can discuss your second proposal.

[lrudelt]

Great!
Twice as long could help already, although we might want to match the duration of the extra orientation block with the duration of the oddball block to be able to compare plasticity under the orientation (control) and the oddball condition (if this is the aim of today's experiment). This would mean, for example, to replace the entire oddball 1 block with the orientation stimulus.

[jeromelecoq]

I want to be careful there as we make the session longer and longer, we are getting into session duration that trigger animal stress for animal that are not imaged on a regular scheduled.

I posted a new figure with more details

[jeromelecoq]

I think 57min is reasonable and is a minimal increase from our previous runs.
I can work to exclude consecutive oddballs if we have consensus on this.
Alternatively we can add one block of 16 repeats of the standard to exclude any oddball at the very onset.

[koweiss]

I'm catching up on this detailed analysis (beyond what I had mentioned during our meeting last week regarding the need to consider adaptation effects for repeated stimuli), and it strikes me how the numbers @lrudelt is reporting regarding possible synaptic adaptation/depletion agree with our recent study https://iopscience.iop.org/article/10.1088/1741-2552/add37b/meta. In a nutshell, what we did in that study is to use repeated depolarizing soma targeted optogenetic stimuli to a given cell as a way to study its adaptation. The little trick we used is that we 'let the cell tell us' when to present that stimulus by optically clamping it to a desired Ca2+ level in a closed loop setting, and studied how this protocol influences both the transient effect (once the cell experiences that stimulus for the first time) as well as the steady state effect (once the cell experiences it over multiple repetitions). We also compared the cell's 'impulse response' before and after this optical clamp as a way to measure the cell's return to homeostasis after this "cruise control" is turned off, and whether representational drift took place or not (although this was not extensively characterized in this paper).
One main finding is that both transient and steady state effects were a function of the activity target level parameters (response magnitude and clamp duration), but full return to homeostasis typically lasted around ~1 hr.

@jeromelecoq I suggest also adding information about the temporal structure of stimulus presentation in this figure to inform the analysis if we want to take these adaptation effects into consideration.
https://docs.google.com/presentation/d/1WFPHhe7XP8DtXoCHj_VGkGm4ukBfVv69UaC4gMWOlio/edit?usp=sharing

[jeromelecoq]

I posted a new figure with more details

[jeromelecoq]

After updates :
https://docs.google.com/presentation/d/1WFPHhe7XP8DtXoCHj_VGkGm4ukBfVv69UaC4gMWOlio/edit?usp=sharing

[jeromelecoq]

Version 2 is here : https://github.com/AllenNeuralDynamics/openscope-community-predictive-processing/blob/main/code/stimulus-control/src/Standard_oddball_slap2_version2.bonsai