[Task]: Post-hoc priors

[mcfanda]

Hi,
it seems that in post-hoc, the prior probabilities of the comparisons are computed using pH0^(2/k), where k is the number of levels of the factor. However, in Westfall, Johnson and Utts, 1997, the suggested adjustment is pH0^(1/k). What is the idea behind using pH0^(2/k)?
thanks

[tomtomme]

@vandenman
any insights on this one?

[vandenman]

@mcfanda Apologies for the delay, @tomtomme thanks for assigning this to me!

Following Westfall (1997), we have

$p(\mu_i =\mu) = \tau$

for all $i$.

So the probability that $p(\mu_1 =\mu_2) = p(\mu_1 =\mu)\times p(\mu_2 =\mu)= \tau^2$ and
$p(\mu_1 =\mu_2 = \dots=\mu_K) =\tau^K$.

Now, we assign a value to the hypothesis that all levels are equal, i.e.,

$p(\mu_1 =\mu_2 = \dots=\mu_K) =\tau^K) = p$

from which it follows that

$\tau = p^\frac{1}{k}$

However, for a specific comparison, we are interested in, say $\mu_1 =\mu_2$, so
$p(\mu_1 =\mu_2) = \tau^2 = (p^\frac{1}{k})^2 = p^\frac{2}{k}$

For example, suppose $k = 5$ and we set $p = 1/2$. Then we obtain $p(\mu_1 =\mu_2) = 1 / 2^\frac{2}{5} \approx 1.3195079107728942$.

The prior odds are given by $\frac{1 - p(\mu_1 =\mu_2)}{p(\mu_1 =\mu_2)} \approx 0.31950791077289437$

This matches JASP:

In the paper by Westfall, they write

Simplifying $\left(\Pi_0^2\right)^\frac{1}{t}$ yields what we use in JASP and what I derived above.

Westfall, P. H., Johnson, W. O., & Utts, J. M. (1997). A Bayesian perspective on the Bonferroni adjustment. Biometrika, 84(2), 419-427.

but see also section 6/ page 90 of

van den Bergh, D., Van Doorn, J., Marsman, M., Draws, T., Van Kesteren, E. J., Derks, K., ... & Wagenmakers, E. J. (2020). A tutorial on conducting and interpreting a Bayesian ANOVA in JASP. L’Année psychologique, 120(1), 73-96.
https://shs.cairn.info/revue-l-annee-psychologique-2020-1-page-73?lang=en

[mcfanda]

Could you send a pdf. I cannot realizzati the formula Thanks

…

On Fri, Mar 28, 2025, 8:57 PM Don van den Bergh ***@***.***> wrote: @mcfanda <https://github.com/mcfanda> Apologies for the delay, @tomtomme <https://github.com/tomtomme> thanks for assigning this to me! Following Westfall (1997), we have $p(\mu_i =\mu) = \tau$ for all $i$. So the probability that $p(\mu_1 =\mu_2) = p(\mu_1 =\mu)\times p(\mu_2 =\mu)= \tau^2$ and $p(\mu_1 =\mu_2 = \dots=\mu_K) =\tau^K$. Now, we assign a value to the hypothesis that all levels are equal, i.e., $p(\mu_1 =\mu_2 = \dots=\mu_K) =\tau^K) = p$ from which it follows that $\tau = p^\frac{1}{k}$ However, for a specific comparison, we are interested in, say $\mu_1 =\mu_2$, so $p(\mu_1 =\mu_2) = \tau^2 = (p^\frac{1}{k})^2 = p^\frac{2}{k}$ For example, suppose $k = 5$ and we set $p = 1/2$. Then we obtain $p(\mu_1 =\mu_2) = 1 / 2^\frac{2}{5} \approx 1.3195079107728942$. The prior odds are given by $1 - p(\mu_1 =\mu_2) / p(\mu_1 =\mu_2) \approx 0.31950791077289437$ This matches JASP: image.png (view on web) <https://github.com/user-attachments/assets/5910f724-a515-4f94-9997-1e788e761be2> In the paper by Westfall, they write image.png (view on web) <https://github.com/user-attachments/assets/f9d82f60-cb71-4ff3-9289-8db6857995ec> Simplifying $\left(\Pi_0^2\right)^(1/t)$ yields what we use in JASP and what I derived above. Westfall, P. H., Johnson, W. O., & Utts, J. M. (1997). A Bayesian perspective on the Bonferroni adjustment. Biometrika, 84(2), 419-427. but see also section 6/ page 90 of van den Bergh, D., Van Doorn, J., Marsman, M., Draws, T., Van Kesteren, E. J., Derks, K., ... & Wagenmakers, E. J. (2020). A tutorial on conducting and interpreting a Bayesian ANOVA in JASP. L’Année psychologique, 120(1), 73-96. https://shs.cairn.info/revue-l-annee-psychologique-2020-1-page-73?lang=en — Reply to this email directly, view it on GitHub <#3284 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABML2WN5L5NHK2KN2UIIEQL2WWLQJAVCNFSM6AAAAABYQY6KNGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRSGMZDGMJQHE> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: vandenman]*vandenman* left a comment (jasp-stats/jasp-issues#3284) <#3284 (comment)> @mcfanda <https://github.com/mcfanda> Apologies for the delay, @tomtomme <https://github.com/tomtomme> thanks for assigning this to me! Following Westfall (1997), we have $p(\mu_i =\mu) = \tau$ for all $i$. So the probability that $p(\mu_1 =\mu_2) = p(\mu_1 =\mu)\times p(\mu_2 =\mu)= \tau^2$ and $p(\mu_1 =\mu_2 = \dots=\mu_K) =\tau^K$. Now, we assign a value to the hypothesis that all levels are equal, i.e., $p(\mu_1 =\mu_2 = \dots=\mu_K) =\tau^K) = p$ from which it follows that $\tau = p^\frac{1}{k}$ However, for a specific comparison, we are interested in, say $\mu_1 =\mu_2$, so $p(\mu_1 =\mu_2) = \tau^2 = (p^\frac{1}{k})^2 = p^\frac{2}{k}$ For example, suppose $k = 5$ and we set $p = 1/2$. Then we obtain $p(\mu_1 =\mu_2) = 1 / 2^\frac{2}{5} \approx 1.3195079107728942$. The prior odds are given by $1 - p(\mu_1 =\mu_2) / p(\mu_1 =\mu_2) \approx 0.31950791077289437$ This matches JASP: image.png (view on web) <https://github.com/user-attachments/assets/5910f724-a515-4f94-9997-1e788e761be2> In the paper by Westfall, they write image.png (view on web) <https://github.com/user-attachments/assets/f9d82f60-cb71-4ff3-9289-8db6857995ec> Simplifying $\left(\Pi_0^2\right)^(1/t)$ yields what we use in JASP and what I derived above. Westfall, P. H., Johnson, W. O., & Utts, J. M. (1997). A Bayesian perspective on the Bonferroni adjustment. Biometrika, 84(2), 419-427. but see also section 6/ page 90 of van den Bergh, D., Van Doorn, J., Marsman, M., Draws, T., Van Kesteren, E. J., Derks, K., ... & Wagenmakers, E. J. (2020). A tutorial on conducting and interpreting a Bayesian ANOVA in JASP. L’Année psychologique, 120(1), 73-96. https://shs.cairn.info/revue-l-annee-psychologique-2020-1-page-73?lang=en — Reply to this email directly, view it on GitHub <#3284 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABML2WN5L5NHK2KN2UIIEQL2WWLQJAVCNFSM6AAAAABYQY6KNGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONRSGMZDGMJQHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[JohnnyDoorn]

@mcfanda posthocpriors.pdf

Thanks @vandenman for taking the time for this detailed response!

[mcfanda]

Thank you very much for your clear and instructive explanation. It makes perfect sense now.