如何在keras中合并两个预训练模型?

dw1jzc5e  于 2023-06-23  发布在  其他
关注(0)|答案(2)|浏览(229)

我想合并两个预训练模型(DenseNet169和InceptionV3),也可以是任意两个。已按照以下链接中的步骤操作,但未成功。已尝试串联和串联,但仍出错。我可能在某个地方犯了一些错误。这是我的第一个stackoverflow问题,如果您能提供帮助,我将不胜感激。https://datascience.stackexchange.com/questions/39407/how-to-make-two-parallel-convolutional-neural-networks-in-keras第一种情况:我尝试了无池化

model1 = DenseNet169(weights='imagenet', include_top=False, input_shape=(300,300,3))
out1 = model1.output    

model2 = InceptionV3(weights='imagenet', include_top=False, input_shape=(300,300,3))
out2 = model2.output

from keras.layers import concatenate 
from keras.layers import Concatenate 

x = concatenate([out1, out2])  # merge the outputs of the two   models
out = Dense(10, activation='softmax')(x)  # final layer of the network

我得到了这个错误:
ValueError:Concatenate层需要输入匹配的形状,除了concat轴。得到输入形状:【(无,9,9,1664),(无,8,8,2048)】
第二种情况:尝试使用平均池,能够连接,但在训练过程中出错

model1 = DenseNet169(weights='imagenet', include_top=False, pooling='avg', input_shape=(300,300,3))
out1 = model1.output   

model2 = InceptionV3(weights='imagenet', include_top=False, pooling='avg', input_shape=(300,300,3))
out2 = model2.output

x = concatenate([out1, out2])  # merge the outputs of the two models
out = Dense(10, activation='softmax')(x)  # final layer of the network

model = Model(inputs=[model1.input, model2.input], outputs=[out])

model.compile(optimizer=Adam(), loss='categorical_crossentropy',metrics=['accuracy'])
history = model.fit_generator(generator=data_generator_train,
                          validation_data=data_generator_val,
                          epochs=20,
                          verbose=1
                         )

第二个案例中的错误:ValueError:检查模型输入时出错:传递给模型的Numpy数组列表不是模型所需的大小。本应看到2个数组,但得到的却是以下1个数组的列表:[array([[[[0.17074525,0.10469133,0.08226486],[0.19852941,0.13124999,0.11642157],[0.36528033,0.3213197,0.3085095 ],...,[0.19082414,0.17801011,0.15840226...

smdncfj3

smdncfj31#

**第二种情况:**由于您的模型需要两个输入,您的data_generator_traindata_generator_val应该返回/产生相应模型和输出的两个输入的列表。您可以通过更新__data_generation方法的返回值来实现这一点

def __data_generation(...):
    ...
    # consider X as input image and y as the label of your model
    return [X, X], keras.utils.to_categorical(y, num_classes=self.n_classes)

**第一种情况:**由于model 2 (8x8)输出的空间大小与model 1输出(9x9)不同(较小),因此可以在连接之前对model 2输出应用零填充。

out1 = model1.output
out2 = model2.output
out2 = ZeroPadding2D(((0,1), (0,1)))(out2)
x = concatenate([out1, out2])

对于第一种情况,你也需要像第二种情况一样修改你的数据生成器。

dba5bblo

dba5bblo2#

Second Case Structure为True,但考虑连接两个模型,每个模型都有自己的输入,如果两个模型的输入相似,只需通过如下重复输入来拟合模型:

model.fit([X_train,X_train], y_train)

我自己实现了你的问题,它工作得非常好。

model1 = DenseNet169(weights='imagenet', include_top=False)
model2 = InceptionV3(weights='imagenet', include_top=False)

model1_out = model1.output
model1_out=GlobalAveragePooling2D()(model1_out)

model2_out = model2.output
model2_out=GlobalAveragePooling2D()(model2_out)

x = concatenate([model1_out, model2_out])

x = Dense(10, activation='softmax')(x) 

model=Model(inputs=[model1.input,model2.input],outputs=x)

model.fit([X_train,X_train], y_train)

相关问题