pytorch 当我们返回ReLU(x)时,inplace是否重要

xoshrz7s  于 2023-08-05  发布在  其他
关注(0)|答案(1)|浏览(120)

下面两个类有什么区别吗?我知道inplace是什么(如果inplace是True,则不需要执行x = function(x),而只需执行function(x)来修改x)。但是这里因为我们return self.conv(x),应该没关系吧?

class ConvBlock(nn.Module):

    def __init__(
        self,
        in_channels,
        out_channels,
        down=True,
        use_act=True,
        **kwargs
        ):
        super().__init__()
        self.conv = nn.Sequential((nn.Conv2d(in_channels, out_channels,
                                  padding_mode='reflect',
                                  **kwargs) if down else nn.ConvTranspose2d(in_channels,
                                  out_channels, **kwargs)),
                                  nn.InstanceNorm2d(out_channels),
                                  (nn.ReLU() if use_act else nn.Identity()))

    def forward(self, x):
        return self.conv(x)

class ConvBlockInplace(nn.Module):

    def __init__(
        self,
        in_channels,
        out_channels,
        down=True,
        use_act=True,
        **kwargs
        ):
        super().__init__()
        self.conv = nn.Sequential((nn.Conv2d(in_channels, out_channels,
                                  padding_mode='reflect',
                                  **kwargs) if down else nn.ConvTranspose2d(in_channels,
                                  out_channels, **kwargs)),
                                  nn.InstanceNorm2d(out_channels),
                                  (nn.ReLU(inplace=True) if use_act else nn.Identity()))

    def forward(self, x):
        return self.conv(x)

字符串

lfapxunr

lfapxunr1#

inplace运算完成了精确的计算量。但是,如果任务是内存受限的,则内存访问较少。那就有关系了
我使用the ptflops flops counter生成以下统计信息

ConvBlock(
  0.0 M, 100.000% Params, 0.015 GMac, 100.000% MACs, 
  (conv): Sequential(
    0.0 M, 100.000% Params, 0.015 GMac, 100.000% MACs, 
    (0): Conv2d(0.0 M, 100.000% Params, 0.014 GMac, 93.333% MACs, 3, 10, kernel_size=(3, 3), stride=(1, 1), padding_mode=reflect)
    (1): InstanceNorm2d(0.0 M, 0.000% Params, 0.0 GMac, 3.333% MACs, 10, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False)
    (2): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 3.333% MACs, )
  )
)
Computational complexity:       0.01 GMac
Number of parameters:           280     
Warning: module ConvBlockInplace is treated as a zero-op.
ConvBlockInplace(
  0.0 M, 100.000% Params, 0.015 GMac, 100.000% MACs, 
  (conv): Sequential(
    0.0 M, 100.000% Params, 0.015 GMac, 100.000% MACs, 
    (0): Conv2d(0.0 M, 100.000% Params, 0.014 GMac, 93.333% MACs, 3, 10, kernel_size=(3, 3), stride=(1, 1), padding_mode=reflect)
    (1): InstanceNorm2d(0.0 M, 0.000% Params, 0.0 GMac, 3.333% MACs, 10, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False)
    (2): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 3.333% MACs, inplace=True)
  )
)
Computational complexity:       0.01 GMac
Number of parameters:           280

字符串

相关问题