keras 我试图从谷歌COLAB导入数据集,现在我得到了“列表目录:路径应为字符串、字节、os.PathLike、整数或None,而不是BatchDataset”

qjp7pelc  于 2023-01-17  发布在  其他
关注(0)|答案(1)|浏览(154)

我试图从谷歌COLAB导入数据集,已经链接到谷歌驱动器太。
这就是我现在使用的代码。

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Activation, Dropout, Flatten, Dense, Conv2D, MaxPooling2D
from tensorflow.keras.losses import sparse_categorical_crossentropy
from tensorflow.keras.optimizers import Adam
import matplotlib.pyplot as plt
from keras.preprocessing.image import ImageDataGenerator
from PIL import Image
import tensorflow as tf

# dimensions of our images.
img_width, img_height = 150, 150

# Model configuration
batch_size = 50
img_width, img_height, img_num_channels = 32, 32, 3
loss_function = sparse_categorical_crossentropy
no_classes = 100
no_epochs = 100
optimizer = Adam()

train_ds = tf.keras.utils.image_dataset_from_directory(
  '/content/drive/MyDrive/Colab Notebooks/Training_Data',
  validation_split=0.2,
  subset="training",
  seed=123,
  image_size=(img_height, img_width),
  batch_size=batch_size)

val_ds = tf.keras.utils.image_dataset_from_directory(
  '/content/drive/MyDrive/Colab Notebooks/Training_Data',
  validation_split=0.2,
  subset="validation",
  seed=123,
  image_size=(img_height, img_width),
  batch_size=batch_size)

# Determine shape of the data
input_shape = (img_width, img_height, img_num_channels)

model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, (3, 3)))   
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='categorical_crossentropy',
              optimizer='Adam',
              metrics=['accuracy'])

# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)

train_generator = train_datagen.flow_from_directory(
    train_ds,
    target_size=(img_width, img_height),
    batch_size = batch_size,
    class_mode='categorical')

validation_generator = test_datagen.flow_from_directory(
    val_ds,
    target_size=(img_width, img_height),
    batch_size = batch_size,
    class_mode='categorical')

model.fit(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    val_ds=validation_generator,
    validation_steps=nb_validation_samples // batch_size)

现在我得到了这个错误。

TypeError                                 Traceback (most recent call last)
<ipython-input-35-1a98ad8aaf01> in <module>()
     82     target_size=(img_width, img_height),
     83     batch_size = batch_size,
---> 84     class_mode='categorical')
     85 
     86 validation_generator = test_datagen.flow_from_directory(

2 frames
/usr/local/lib/python3.7/dist-packages/keras_preprocessing/image/directory_iterator.py in __init__(self, directory, image_data_generator, target_size, color_mode, classes, class_mode, batch_size, shuffle, seed, data_format, save_to_dir, save_prefix, save_format, follow_links, subset, interpolation, dtype)
    113         if not classes:
    114             classes = []
--> 115             for subdir in sorted(os.listdir(directory)):
    116                 if os.path.isdir(os.path.join(directory, subdir)):
    117                     classes.append(subdir)

TypeError: listdir: path should be string, bytes, os.PathLike, integer or None, not BatchDataset

我不知道下一步该怎么办,我承认编程不是我的专长,但我需要它,因为它涉及到我的论文,我不知道现在该怎么办。有人能帮我解决这个问题吗?我觉得我快成功了。

eanckbw9

eanckbw91#

正如@Dr. Snoopy和@ Vishal Balaji正确提到的,您应该给予directory,而不是直接将目录路径放在image_dataset_from_directory API中。

train_dataset="/content/drive/MyDrive/MY WORK/cats_and_dogs_filtered/train"

img_width, img_height = 150, 150
batch_size = 32
#We cannot put the direct path of the directory here
train_ds = tf.keras.utils.image_dataset_from_directory(
  train_dataset,
  validation_split=0.2, 
  subset="training",
  seed=123,
  image_size=(img_height, img_width))

您正在使用两个不同的API(image_dataset_from_directoryflow_from_directory)从目录导入数据集,并尝试使用这两个API训练模型。您可以使用这两个API中的任意一个。
检查以下代码,通过使用flow_from_directory API导入和扩充数据集来训练模型:

# this is the augmentation configuration we will use for training
train_datagen = tf.keras.preprocessing.image.ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True,)

train_generator = train_datagen.flow_from_directory(
    train_dataset,
    target_size=(150,150),
    batch_size = 32,
    class_mode='sparse')  # Used 'Sparse' class_mode for Image dataset

# We should apply the same data preprocessing to the validation data while model training

validation_generator = train_datagen.flow_from_directory(
    validation_dataset,
    target_size=(150,150),
    batch_size = 32,
    class_mode='sparse')

请查看此gist作为参考,以了解更多详细信息。

相关问题