cnn- keras fit_generator回调中的值错误

jei2mxaa  于 2023-01-13  发布在  其他
关注(0)|答案(2)|浏览(165)

model.fit_generator()中的错误。作为一个Python的初级程序员,我不确定这个错误指示什么。
我正在尝试用python迁移学习来用imagenet训练VGG19,我在回调中遇到了一个值错误。有人能建议我应该对这个代码做些什么修改吗?
我尝试在colab中执行以下代码,但遇到错误

!apt-get install -y -qq software-properties-common python-software-properties module-init-tools
!add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null
!apt-get update -qq 2>&1 > /dev/null
!apt-get -y install -qq google-drive-ocamlfuse fuse
from google.colab import auth
auth.authenticate_user()
from oauth2client.client import GoogleCredentials
creds = GoogleCredentials.get_application_default()
import getpass
!google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL
vcode = getpass.getpass()
!echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}
!mkdir -p drive
!google-drive-ocamlfuse drive
!pip install opencv-python
!pip install opencv-contrib-python
!apt update && apt install -y libsm6 libxext6
!pip install -q keras
from glob import glob

import cv2
import numpy as np

from sklearn.utils import shuffle
from sklearn.model_selection import train_test_split

from keras.utils import to_categorical
from keras.models import Model
from keras.layers import Dropout, Dense, Flatten
from keras.optimizers import SGD
from keras.losses import categorical_crossentropy
from keras.regularizers import l2
from keras.applications.vgg19 import VGG19
model = VGG19(include_top=False, weights='imagenet', pooling='avg')
for layer in model.layers:
    layer.trainable = False
x = model.output
predictions = Dense(7, activation='softmax')(x)
model_final = Model(input=model.input, output=predictions)
from keras.callbacks import ReduceLROnPlateau
lr_reducer = ReduceLROnPlateau(monitor='val_loss', factor=0.9, patience=4, verbose=1)
model_final.compile(loss=categorical_crossentropy,
                  optimizer=SGD(lr=0.001, momentum=0.9, nesterov=True),
                  metrics=['accuracy'])
model_final.fit(np.array(X_train), np.array(y_train),
              batch_size=32,
              epochs=10,
              verbose=1,
              validation_split=0.1,
              shuffle=True)
for layer in model_final.layers[7:]:
  layer.trainable = True
model_final.compile(loss=categorical_crossentropy,
                  optimizer=SGD(lr=0.001, momentum=0.9, nesterov=True),
                  metrics=['accuracy'])
from keras.preprocessing.image import ImageDataGenerator
train_generator = ImageDataGenerator(
    featurewise_center = True,
    featurewise_std_normalization = True,
    rotation_range=30,
    shear_range=0.2,
    zoom_range=0.2,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True)

train_generator.fit(np.array(X_train))

test_generator = ImageDataGenerator(
    featurewise_center = True,
    featurewise_std_normalization = True)

test_generator.fit(np.array(X_train))

model_final.fit_generator(train_generator.flow(np.array(X_train), np.array(y_train), batch_size=32),
                          validation_data=test_generator.flow(np.array(X_test), np.array(y_test)),
                          steps_per_epoch=len(X_train)/32, 
                          epochs=50)
ValueError                                Traceback (most recent call last)
<ipython-input-39-f9af6d0d8994> in <module>()
      2                           validation_data=test_generator.flow(np.array(X_test), np.array(y_test)),
      3                           steps_per_epoch=len(X_train)/32,
----> 4                           epochs=50)

2 frames
/usr/local/lib/python3.6/dist-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
     66     if (val_gen and not isinstance(validation_data, Sequence) and
     67             not validation_steps):
---> 68         raise ValueError('`validation_steps=None` is only valid for a'
     69                          ' generator based on the `keras.utils.Sequence`'
     70                          ' class. Please specify `validation_steps` or use'

ValueError: `validation_steps=None` is only valid for a generator based on the `keras.utils.Sequence` class. Please specify `validation_steps` or use the `keras.utils.Sequence` class.
3htmauhk

3htmauhk1#

您必须在model_final.fit_generator中指定validation_steps。这是因为生成器不知道将使用的数据总数,它只知道batch_size(默认为batch_size=32)。因此,您必须通过提供每个时期中的步骤数来手动告知生成器何时停止加载数据。step实际上表示批处理数。
如果要在每个时期使用所有测试数据进行验证:

model_final.fit_generator(train_generator.flow(np.array(X_train), np.array(y_train), batch_size=32),
                          validation_data=test_generator.flow(np.array(X_test), np.array(y_test), batch_size=32),
                          steps_per_epoch=len(X_train)/32, 
                          validation_steps=len(X_test)/32,
                          epochs=50)

该错误消息还提到,只有当您使用从Sequence继承的生成器时,validation_step=None才有效。在这种情况下,validation_step将自动设置为len(validation_data)。请参见此处。之所以可以这样做,是因为__len__(self)方法是在Sequence对象中定义的。请参见此处,但不是在您的ImageDataGenerator

sczxawaw

sczxawaw2#

steps_per_epoch=len(X_train)/32 , validation_steps=len(X_test)/32, epochs=50)--按steps_per_epoch=len(X_train)//32 , validation_steps=len(X_test)//32, epochs=50)进行更正
解释-当你想得到小数形式的除法结果,比如5//2 = 2.5,在python中我们使用单个'/'。但是当你想得到整数形式的结果,我们使用'//',例如-- 5//2 = 2.5,5//2=2。
steps_per_epoch不能为小数,因此应使用5//2。

相关问题