A complete error is:
ValueError: The model did not return a loss from the inputs, only the following keys: logits. For reference, the inputs it received are input_ids,attention_mask.
However, dataset
contains label
and train_dataset
in the argument has label
in column_names
.
from transformers import Trainer, TrainingArguments
batch_size = 64
logging_steps = len(emotions_encoded["train"])
training_args = TrainingArguments(output_dir = "model_out",
num_train_epochs=2,
learning_rate = 2e-5,
per_device_train_batch_size=batch_size,
per_device_eval_batch_size=batch_size,
weight_decay=0.01,
evaluation_strategy="epoch",
disable_tqdm=False,
logging_steps=logging_steps,
push_to_hub=False,
log_level="error")
from transformers import Trainer
trainer = Trainer(model=model, args=training_args,
compute_metrics=compute_metrics,
train_dataset=emotions_encoded["train"], # column_names shows <class 'list'>: ['text', 'label', 'input_ids', 'attention_mask']
eval_dataset=emotions_encoded["validation"],
tokenizer=tokenizer)
trainer.train()
This is what I found:
train_dataset
in the following trainer.py
function has column_names
value <class 'list'>: ['label', 'input_ids', 'attention_mask']
def get_train_dataloader(self) -> DataLoader:
return DataLoader(
train_dataset,
batch_size=self._train_batch_size,
sampler=train_sampler,
collate_fn=data_collator,
drop_last=self.args.dataloader_drop_last,
num_workers=self.args.dataloader_num_workers,
pin_memory=self.args.dataloader_pin_memory,
worker_init_fn=seed_worker,
)
but datasets arrow_dataset.py
function:
def _getitem(self, key: Union[int, slice, str, ListLike[int]], **kwargs) -> Union[Dict, List]:
format_columns = kwargs["format_columns"] if "format_columns" in kwargs else self._format_columns
because kwargs
is None
, thereafter self._format_columns
is used and its value is <class 'list'>: ['input_ids', 'attention_mask']
. Here model
is lost.
And that finally caused dataloder.py class _BaseDataLoaderIter
function:
def __next__(self) -> Any:
with torch.autograd.profiler.record_function(self._profile_name):
if self._sampler_iter is None:
# TODO(https://github.com/pytorch/pytorch/issues/76750)
self._reset() # type: ignore[call-arg]
data = self._next_data()
data
contains only ‘input_ids’and ‘attention_mask’.
I am a beginner with the boot NLP with Transformers