这是我第一次在stack overflow中发帖,所以如果我犯了任何错误,请原谅我。
我有10000个数据,每个数据都有一个0和1的标签。我想使用lstm进行分类,因为这是时间序列数据。
input_dim = 1
hidden_dim = 32
num_layers = 2
output_dim = 1
# Here we define our model as a class
class LSTM(nn.Module):
def __init__(self, input_dim, hidden_dim, num_layers, output_dim):
super(LSTM, self).__init__()
self.hidden_dim = hidden_dim
self.num_layers = num_layers
self.lstm = nn.LSTM(input_dim, hidden_dim, num_layers, batch_first=True)
self.fc = nn.Linear(hidden_dim, output_dim)
def forward(self, x):
#Initialize hidden layer and cell state
h0 = torch.zeros(self.num_layers, x.size(0), self.hidden_dim).requires_grad_()
c0 = torch.zeros(self.num_layers, x.size(0), self.hidden_dim).requires_grad_()
# We need to detach as we are doing truncated backpropagation through time (BPTT)
# If we don't, we'll backprop all the way to the start even after going through another batch
out, (hn, cn) = self.lstm(x, (h0.detach(), c0.detach()))
# Index hidden state of last time step
# out.size() --> 100, 32, 100
# out[:, -1, :] --> 100, 100 --> just want last time step hidden states!
out = self.fc(out[:, -1, :])
# For binomial Classification
m = torch.sigmoid(out)
return m
model = LSTM(input_dim=input_dim, hidden_dim=hidden_dim, output_dim=output_dim, num_layers=num_layers)
loss = nn.BCELoss()
optimiser = torch.optim.Adam(model.parameters(), lr=0.00001, weight_decay=0.00006)
num_epochs = 100
# Number of steps to unroll
seq_dim =look_back-1
for t in range(num_epochs):
y_train_class = model(x_train)
output = loss(y_train_class, y_train)
# Zero out gradient, else they will accumulate between epochs
optimiser.zero_grad(set_to_none=True)
# Backward pass
output.backward()
# Update parameters
optimiser.step()
这是一个结果的示例
这段代码最初来自kaggle,我对它们进行了分类编辑。拜托,你能告诉我我做错了什么吗?
1条答案
按热度按时间h7appiyu1#
神经网络训练中的一个常见现象是,它们最初会收敛到问题的一个非常简单的解决方案,在这个解决方案中,它们输出一个恒定的预测,从而使训练数据的误差最小化。我猜在你的训练数据中
0
及1
学校离学校很近0.5423
. 根据您的模型是否具有足够的复杂性,当给定更多的学习步骤时,它可能会学习根据输入做出更具体的预测。虽然增加纪元的数量可能会有所帮助,但您可以使用当前设置做一些更好的事情。目前,每个历元只执行一个优化器步骤。通常,您希望每批执行一步,并以(小)批(例如32个输入)的方式循环数据。要做到这一点,最好使用一个dataloader,您可以在其中定义一个批大小,并在epoch循环中循环dataloader,类似于本例。