keras 验证准确度非常好,但预测结果不佳

vcirk6k6  于 2022-11-30  发布在  其他
关注(0)|答案(4)|浏览(224)

我正在建立一个keras模型来对猫和狗进行分类。我使用了迁移学习和瓶颈特性,并对vgg模型进行了微调。现在我得到了非常好的验证准确率,如97%,但当我开始预测时,我得到了非常糟糕的分类报告和混淆矩阵。可能是什么问题?
下面是微调的代码和我得到的结果

base_model = applications.VGG16(weights='imagenet', include_top=False, input_shape=(150,150,3))
print('Model loaded.')

# build a classifier model to put on top of the convolutional model
top_model = Sequential()
top_model.add(Flatten(input_shape=base_model.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(2, activation='sigmoid'))

# note that it is necessary to start with a fully-trained
# classifier, including the top classifier,
# in order to successfully do fine-tuning
top_model.load_weights(top_model_weights_path)

# add the model on top of the convolutional base
# model.add(top_model)
model = Model(inputs=base_model.input, outputs=top_model(base_model.output))

# set the first 25 layers (up to the last conv block)
# to non-trainable (weights will not be updated)
for layer in model.layers[:15]:
    layer.trainable = False

# compile the model with a SGD/momentum optimizer
# and a very slow learning rate.
model.compile(loss='binary_crossentropy',
              optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
              metrics=['accuracy'])

# prepare data augmentation configuration
train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

test_datagen = ImageDataGenerator(rescale=1. / 255)

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='categorical')

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='categorical')

model.summary()

# fine-tune the model
model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size,
    verbose=2)
scores=model.evaluate_generator(generator=validation_generator,
steps=nb_validation_samples // batch_size)
print("Accuracy = ", scores[1])

Y_pred = model.predict_generator(validation_generator, nb_validation_samples // batch_size)

y_pred = np.argmax(Y_pred, axis=1)

print('Confusion Matrix')

print(confusion_matrix(validation_generator.classes, y_pred))

print('Classification Report')

target_names = ['Cats', 'Dogs']

print(classification_report(validation_generator.classes, y_pred, target_names=target_names))
model.save("model_tuned.h5")

准确度= 0.974375
混淆矩阵[[186 214] [199 201]]
分类报告

precision    recall  f1-score   support

    Cats       0.48      0.47      0.47       400
    Dogs       0.48      0.50      0.49       400

微观平均值0.48 0.48 0.48 800宏观平均值0.48 0.48 0.48 800加权平均值0.48 0.48 0.48 800

xeufq47z

xeufq47z1#

我认为问题在于您应该在验证生成器中添加shuffle = False

validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='categorical',
shuffle=False)

问题是默认行为是打乱图像的顺序,因此

validation_generator.classes

与发生器不匹配

jobtbby3

jobtbby32#

你的模型有两个问题。首先,如果你有多个输出神经元,你需要使用softmax激活:

top_model.add(Dense(2, activation='softmax'))

然后你必须使用categorical_crossentropy损失,二进制交叉熵只适用于当你有一个S形激活的输出神经元时。

model.compile(loss='categorical_crossentropy',
              optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
              metrics=['accuracy'])
enxuqcxy

enxuqcxy3#

此问题通常有两个原因:
1.最常见的一种是当我们用不同形式的图像实现(预测)模型时(可能忘记归一化或混淆了高度和宽度)。
1.第二种情况是当一类的样本比其他类的样本多很多时。假设有1000个样本A和100个样本B。如果模型只估计A,那么它的正确率将达到90%。这在数学上被称为“局部最小值”,即使验证结果产生0.9的准确度,实现也将是可怕的。
简而言之,你在处理不平衡的数据吗?在这种情况下,有时很难避免局部最小值。这会是这里的问题吗?

6ovsh4lw

6ovsh4lw4#

不知何故,Keras模型的predict_generator()并没有像预期的那样工作。我宁愿一个接一个地循环所有的测试图像,并在每次迭代中获得每个图像的预测。我使用Plaid-ML Keras作为我的后端,为了获得预测,我使用了以下代码。

import os
from PIL import Image
import keras
import numpy

print("Prediction result:")
dir = "/path/to/test/images"
files = os.listdir(dir)
correct = 0
total = 0
#dictionary to label all traffic signs class.
classes = {
    0:'This is Cat',
    1:'This is Dog',
}
for file_name in files:
    total += 1
    image = Image.open(dir + "/" + file_name).convert('RGB')
    image = image.resize((100,100))
    image = numpy.expand_dims(image, axis=0)
    image = numpy.array(image)
    image = image/255
    pred = model.predict_classes([image])[0]
    sign = classes[pred]
    if ("cat" in file_name) and ("cat" in sign):
        print(correct,". ", file_name, sign)
        correct+=1
    elif ("dog" in file_name) and ("dog" in sign):
        print(correct,". ", file_name, sign)
        correct+=1
print("accuracy: ", (correct/total))

相关问题