使用BatchNorm进行训练时,Pytorch预期每个通道有多个值

7hiiyaii  于 2022-12-13  发布在  其他
关注(0)|答案(1)|浏览(209)

我写了这样的代码:

  1. import numpy as np
  2. import torch
  3. from torch.utils.data import TensorDataset, dataloader
  4. inputDim = 10
  5. n = 1000
  6. X = np.random.rand(n,inputDim)
  7. y = np.random.rand(0,2,n)
  8. tensor_x = torch.Tensor(X)
  9. tensor_y = torch.Tensor(y)
  10. Xy = (tensor_x, tensor_y)
  11. XyLoader = dataloader.DataLoader(Xy, batch_size = 16, shuffle = True, drop_last = True)
  12. model = torch.nn.Sequential(
  13. torch.nn.Linear(inputDim, 200),
  14. torch.nn.ReLU(),
  15. torch.nn.BatchNorm1d(num_features=200),
  16. torch.nn.Linear(200,100),
  17. torch.nn.Tanh(),
  18. torch.nn.BatchNorm1d(num_features=100),
  19. torch.nn.Linear(100,1),
  20. torch.nn.Sigmoid()
  21. )
  22. optimizer = torch.optim.Adam(model.parameters(), lr= 0.001)
  23. loss_fn = torch.nn.BCELoss()
  24. nepochs = 1000
  25. for epochs in range(nepochs):
  26. for X,y in XyLoader:
  27. batch_size = X.shape[0]
  28. y_hat = model(X.view(batch_size,-1))
  29. loss = loss_fn(y_hat, y)
  30. optimizer.zero_grad()
  31. loss.backward()
  32. optimizer.step()
  33. with torch.no_grad():
  34. xt = torch.tensor(np.random.rand(1,inputDim))
  35. y2 = model(xt.float())
  36. print(y2.detach().numpy()[0][0])

我对torch.nn.BatchNorm1d做错了什么?如果我运行代码时没有两行代码,一切都“正常”,那会有什么问题?

kfgdxczn

kfgdxczn1#

在您的例子中,PyTorch只是抱怨nn.BatchNorm1d的输入的形状,这里的输入应该具有(B, C, L)的形状。C是嵌入维数,L是输入序列的长度/时间步。通常,PyTorch中的1d指的是一个序列,例如一个标记化的句子,其中的每个标记(总共L)表示为C维向量,并堆叠在dim 1上。要修复此错误,可以执行以下操作

  1. tensor_x = torch.as_tensor(X).unsqueeze(1) # use as_tensor to avoid unnecessary data copy
  2. tensor_y = torch.as_tensor(y)

完整修改代码:

  1. import numpy as np
  2. import torch
  3. from torch.utils.data import TensorDataset, DataLoader
  4. inputDim = 10
  5. n = 1000
  6. X = np.random.rand(n, inputDim)
  7. y = np.random.rand(0, 2, n)
  8. tensor_x = torch.as_tensor(X).unsqueeze(1)
  9. tensor_y = torch.as_tensor(y)
  10. Xy = (tensor_x, tensor_y)
  11. XyLoader = DataLoader(Xy, batch_size = 16, shuffle = True, drop_last = True)
  12. model = torch.nn.Sequential(
  13. torch.nn.Linear(inputDim, 200),
  14. torch.nn.ReLU(),
  15. torch.nn.BatchNorm1d(num_features=1),
  16. torch.nn.Linear(200, 100),
  17. torch.nn.Tanh(),
  18. torch.nn.BatchNorm1d(num_features=1),
  19. torch.nn.Linear(100,1),
  20. torch.nn.Sigmoid()
  21. )
  22. optimizer = torch.optim.Adam(model.parameters(), lr= 0.001)
  23. loss_fn = torch.nn.BCELoss()
  24. nepochs = 1000
  25. for epochs in range(nepochs):
  26. for X, y in XyLoader:
  27. batch_size = X.shape[0]
  28. y_hat = model(X.view(batch_size,-1))
  29. loss = loss_fn(y_hat, y)
  30. optimizer.zero_grad()
  31. loss.backward()
  32. optimizer.step()
  33. with torch.no_grad():
  34. xt = torch.as_tensor(np.random.rand(1, inputDim), dtype=torch.float32).unsqueeze(1)
  35. y2 = model(xt.float())
  36. print(y2.detach().numpy()[0][0])
展开查看全部

相关问题