I think there is some calculation error in the source code of Informer’s paper. (https://github.com/zhouhaoyi/Informer2020)
When training the model, the source code for calculating the loss is shown below.
def _select_criterion(self):
criterion = nn.MSELoss()
return criterion
def train(self, setting):
(Omitted...)
criterion = self._select_criterion()
(Omitted...)
for i, (batch_x,batch_y,batch_x_mark,batch_y_mark) in enumerate(train_loader):
iter_count += 1
model_optim.zero_grad()
pred, true = self._process_one_batch(
train_data, batch_x, batch_y, batch_x_mark, batch_y_mark)
loss = criterion(pred, true)
train_loss.append(loss.item())
(Omitted...)
return self.model
def _process_one_batch(self, dataset_object, batch_x, batch_y, batch_x_mark, batch_y_mark):
(Omitted...)
# encoder - decoder
if self.args.use_amp:
with torch.cuda.amp.autocast():
if self.args.output_attention:
outputs = self.model(batch_x, batch_x_mark, dec_inp, batch_y_mark)[0]
else:
outputs = self.model(batch_x, batch_x_mark, dec_inp, batch_y_mark)
else:
if self.args.output_attention:
outputs = self.model(batch_x, batch_x_mark, dec_inp, batch_y_mark)[0]
else:
outputs = self.model(batch_x, batch_x_mark, dec_inp, batch_y_mark)
if self.args.inverse:
outputs = dataset_object.inverse_transform(outputs)
f_dim = -1 if self.args.features=='MS' else 0
batch_y = batch_y[:,-self.args.pred_len:,f_dim:].to(self.device)
return outputs, batch_y
In def train, the shape of the pred and the true must be the same,
but the true uses all or only the last vector, depending on the situation, and the pred uses all vectors.
If self.args.features==’MS’, is an error in the loss calculation?