请注意，在这种情况下，f1_scores_Algo_A和f1_scores_Algo_B都是列表，可以被视为两个独立的分数样本：

from sklearn.metrics import f1_score
from scipy import stats

from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from mlxtend.data import iris_data
from sklearn.model_selection import train_test_split

algo_A = LogisticRegression(random_state=1, max_iter=1000)  # try your algos / models
algo_B = DecisionTreeClassifier(random_state=1, max_depth=3)

X, y = iris_data() # try your dataset

f1_scores_Algo_A, f1_scores_Algo_B = [], []

for i in range(100):
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

    y_pred = algo_A.fit(X_train, y_train).predict(X_test)
    f1_scores_Algo_A.append(f1_score(y_test, y_pred, average='micro'))

    y_pred = algo_B.fit(X_train, y_train).predict(X_test)
    f1_scores_Algo_B.append(f1_score(y_test, y_pred, average='micro'))

下一个图显示了使用模型的不同训练-检验拆分获得的F1分数的分布。

现在，我们可以做成对的t-test：

t_value,p_value=stats.ttest_ind(f1_scores_Algo_A, f1_scores_Algo_B)
print('Test statistic is %f'%float("{:.6f}".format(t_value)))
# Test statistic is 2.457321
print('p-value for two tailed test is %f'%p_value)
# p-value for two tailed test is 0.014858

因此，我们可以在5%的显著性水平下拒绝零假设（2个独立样本具有相同的平均得分）。