tensorflow 当data_format = "NDHWC"时，ConvertFusedBatchNorm返回未初始化的值,

tgabmvqs 于 10个月前发布在其他

关注(0)|答案(5)|浏览(93)

问题类型

Bug

你是否在TF nightly中复现了这个bug?

是的

来源

source

Tensorflow版本

v1.12.1-88697-g620bee79ab3 2.12.0-dev20230201

自定义代码

无

OS平台和发行版

Ubuntu 22.04

移动设备

无响应*

Python版本

Python 3.10

Bazel版本

5.3.0

GCC/编译器版本

gcc-11

CUDA/cuDNN版本

CUDA-11.8/cudnn-8.7.0/TensorRT-8.5.3

GPU型号和内存

RTX3090

当前行为？

See code snippet:
https://github.com/tensorflow/tensorflow/blob/4aec415b3f06b19c380d1a0ca92cc2de0d74cc21/tensorflow/compiler/tf2tensorrt/convert/convert_nodes.cc#L4399-L4436
In the case of NDHWC layout (triggered by the code below) an uninitialized value is returned from ConvertFusedBatchNorm which causes an exception to be raised.
I would expect it to build correctly. Changing ConvertFusedBatchNorm to do the same thing for NDHWC as for NHWC gets rid of the crash, but I don't know if this is correct.

独立代码以重现问题

import tensorflow as tf
import numpy as np
from tensorflow.keras.layers import (
    BatchNormalization,
    Conv3D,
    Dense,
    Flatten,
    Input,
)
from tensorflow.keras.models import Model
from tensorflow.python.compiler.tensorrt import trt_convert as trt
inputs = Input(shape=(24, 24, 64, 1), name="x")
x = inputs
x = Conv3D(16, (3, 3, 3), activation="relu", padding="same")(x)
x = BatchNormalization()(x)
x = Flatten()(x)
x = Dense(128, activation="relu")(x)
x = Dense(128)(x)
m = Model(inputs=[inputs], outputs=[x])
m.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    metrics=[tf.keras.metrics.SparseCategoricalAccuracy()],
)
model_dir = "/tmp/model"
tf.keras.models.save_model(m, model_dir)
converter = trt.TrtGraphConverterV2(input_saved_model_dir=model_dir,
                        precision_mode=trt.TrtPrecisionMode.FP16)
trt_func = converter.convert()
def input_fn():
    a = np.random.rand(1024, 24, 24, 64, 1).astype(np.float32)
    yield [a]
converter.build(input_fn=input_fn)

5条答案

按热度按时间

fwzugrvs1#

你好，@froody
对于延迟表示歉意，我认为你正在尝试将数据格式为 NHWC 的输入Tensor转换为 NDHWC ,所以我不确定是否可以做到这一点，但我认为输入Tensor的数据格式应该等于输出Tensor的数据格式，因此 NHWC 转换为 NCHW ,NDHWC 转换为 NCDHW ,甚至在日志输出中也清楚地显示为 INVALID_ARGUMENT: Rank of perm for transpose does not match with that of the input.。
你可以参考这个 official documentation 的源代码，以下是一些参考资料 Ref-1 、 Ref-2 、 Ref-3 ,它们可能有助于解决你的问题。
每个字母的意义可能有助于理解：

N: number of images in the batch
H: height of the image
W: width of the image
C: number of channels of the image (ex: 3 for RGB, 1 for grayscale)
D: Depth