wrong ttest result

[SterArcher]

Hi!

I wanted to compare two groups using ttest function.

The violin plot shows significant overlap, but the ttest from your package returns significant result.

pg.ttest(df["Skupina"], df["masa"])

           T  dof alternative     p-val         CI95%  cohen-d    BF10  \

T-test 3.828714 56 two-sided 0.000327 [0.16, 0.51] 1.00547 79.764

       power  

T-test 0.964251

When using scipy i get non-significant result.

# Perform t-test
group1 = df[df["Skupina"] == 1]["masa"]
group2 = df[df["Skupina"] == 2]["masa"]
t_stat, p_value = ttest_ind(group1, group2)

print(f"T-statistic: {t_stat}, P-value: {p_value}")

T-statistic: -1.2849905790443539, P-value: 0.20971011250741314

Why would that be? The latter result was also confirmed by another person in R.

Thank you!

[raphaelvallat]

Hi,

I believe that the correct code to replicate the SciPy result is:

import pingouin as pg
group1 = df[df["Skupina"] == 1]["masa"]
group2 = df[df["Skupina"] == 2]["masa"]
pg.ttest(group1, group2)

Alternatively, you can use:

import pingouin as pg
pg.pairwise_tests(data=df, dv="masa", between="Skupina")

[SterArcher]

Thank you for taking the time and correcting my mistake.

Interestingly the ttest metric between pg and scipy is a bit different.

Above is scipy, below pg.

[remrama]

@SterArcher the stats should be identical but take special note of the correction argument in pg.ttest() and the corresponding equal_var argument in scipy.stats.ttest_ind(). The default for scipy is to assume equal variance between groups, but pingouin attempts to determine that on its own (by default).

import pingouin as pg
from scipy.stats import ttest_ind

df = pg.read_dataset("penguins").dropna(subset="bill_length_mm")

group1 = df[df["species"] == "Adelie"]["bill_length_mm"].to_numpy()
group2 = df[df["species"] == "Chinstrap"]["bill_length_mm"].to_numpy()

pg_result = pg.ttest(group1, group2, correction="auto")
t, p, dof = pg_result.loc["T-test", ["T", "p-val", "dof"]]
print("Pinguoin with correction ('auto' applies in this case)")
print(f"T-stat: {t:.2f}, DoF: {dof:.0f}, p-value: {p}")
print("-" * 20)

scipy_result = ttest_ind(group1, group2)
t, p, dof = scipy_result.statistic, scipy_result.pvalue, scipy_result.df
print("Scipy without correction")
print(f"T-stat: {t:.2f}, DoF: {dof:.0f}, p-value: {p}")
print("-" * 20)

scipy_result = ttest_ind(group1, group2, equal_var=False)
t, p, dof = scipy_result.statistic, scipy_result.pvalue, scipy_result.df
print("Scipy with correction")
print(f"T-stat: {t:.2f}, DoF: {dof:.0f}, p-value: {p}")
print("-" * 20)

pg_result = pg.ttest(group1, group2, correction=False)
t, p, dof = pg_result.loc["T-test", ["T", "p-val", "dof"]]
print("Pinguoin without correction")
print(f"T-stat: {t:.2f}, DoF: {dof:.0f}, p-value: {p}")

Pinguoin with correction ('auto' applies in this case)
T-stat: -21.86, DoF: 107, p-value: 2.900144477264413e-41
--------------------
Scipy without correction
T-stat: -23.80, DoF: 217, p-value: 2.011759018655462e-62
--------------------
Scipy with correction
T-stat: -21.86, DoF: 107, p-value: 2.900144477264413e-41
--------------------
Pinguoin without correction
T-stat: -23.80, DoF: 217, p-value: 2.011759018655462e-62

[SterArcher]

@remrama Thank you so much for taking the time and explaining it, appreciate it!