[Python] Confusing behaviour of scipy.stats.ttest_ind

I'm using (or trying to use scipy.stats.ttest_ind) like this:

from scipy.stats import ttest_ind, norm

def test_ttest_ind():
rng = np.random.default_rng(43)

rvs1 = stats.norm.rvs(loc=15, scale=10, size=500, random_state=rng)
rvs2 = stats.norm.rvs(loc=15, scale=10, size=500, random_state=rng)

print("Same same")
print(ttest_ind(rvs1, rvs2))
print(ttest_ind(rvs1, rvs2, equal_var=False))
print()

print("Same size diff variance")
rvs3 = stats.norm.rvs(loc=15, scale=20, size=500, random_state=rng)
print(ttest_ind(rvs1, rvs3))
print(ttest_ind(rvs1, rvs3, equal_var=False))
equal_var=False))
print()

print("Diff size diff variance")
rvs4 = stats.norm.rvs(loc=15, scale=20, size=100, random_state=rng)
print(ttest_ind(rvs1, rvs4))
print(ttest_ind(rvs1, rvs4, equal_var=False))
equal_var=False))
print()



But the results are confusing to me...

Same same
TtestResult(statistic=np.float64(0.7788368), pvalue=np.float64(0.43626), df=np.float64(998.0))
TtestResult(statistic=np.float64(0.7788368), pvalue=np.float64(0.43626), df=np.float64(997.167))

# OK all the above looks good.


Same size diff variance
TtestResult(statistic=np.float64(0.5116953), pvalue=np.float64(0.60897), df=np.float64(998.0))
TtestResult(statistic=np.float64(0.5116953), pvalue=np.float64(0.60901), df=np.float64(730.112))

# The variance *is* different... So how come pooling it or not has no effect?


Diff size diff variance
TtestResult(statistic=np.float64(-5.125124), pvalue=np.float64(4.0e-07), df=np.float64(598.0))
TtestResult(statistic=np.float64(-3.577155), pvalue=np.float64(0.00051), df=np.float64(111.951))

# Now the sample size differs, pooling the variance seems to have a huge effect! Why?



Sorry if this is a very stupid question, I'm getting stuck with 'more advanced stuff', so looking to understand the basics before I know I'm confused about more complicated stuffs.

Continue reading...