Welch's t-Test Statistic

Compute the t statistic and the degrees of freedom of Welch's two-sample t-test.

To decide whether a new model is really better than the old one, you compare two sets of scores, for example from repeated runs. The t statistic is the gap between the means divided by its standard error. Student's t-test assumes both groups have the same variance. Welch's version does not, which makes it the safer default:

v_a = var(a, ddof=1) / n_a
v_b = var(b, ddof=1) / n_b
t   = (mean(a) - mean(b)) / sqrt(v_a + v_b)
df  = (v_a + v_b)^2 / (v_a^2 / (n_a - 1) + v_b^2 / (n_b - 1))

The degrees of freedom formula is the Welch-Satterthwaite approximation. It is usually not a whole number.

Function signature

def welch_t_test(a, b):

Example

Input

welch_t_test(np.array([1.0, 2.0, 3.0, 4.0]), np.array([2.0, 4.0, 6.0, 8.0, 10.0]))

Output

(-2.25143632, 5.52078775)

Reasoning

The means are 2.5 and 6. The sample variances are 1.667 and 10, so the squared standard errors are 0.417 and 2. The gap -3.5 divided by sqrt(2.417) = 1.555 gives t = -2.251. The second group is much noisier, so df is close to its own n_b - 1 = 4 rather than the 7 of Student's test.

Constraints and notes

• Each sample has at least 2 values and a non-zero variance.
• Use the sample variance with ddof=1.
• Return two plain floats.