I’m trying to train a transformers model. Everything runs smoothly (loading data, initializing Trainer class, etc.) up until the trainer.train() statement. Then I get this error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Input In [64], in <cell line: 1>()
----> 1 trainer.train()
File /libraries/Custom/lib/python3.9/site-packages/transformers/trainer.py:1561, in Trainer.train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
1559 hf_hub_utils.enable_progress_bars()
1560 else:
-> 1561 return inner_training_loop(
1562 args=args,
1563 resume_from_checkpoint=resume_from_checkpoint,
1564 trial=trial,
1565 ignore_keys_for_eval=ignore_keys_for_eval,
1566 )
File /libraries/Custom/lib/python3.9/site-packages/transformers/trainer.py:1590, in Trainer._inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
1588 logger.debug(f"Currently training with a batch size of: {self._train_batch_size}")
1589 # Data loader and number of training steps
-> 1590 train_dataloader = self.get_train_dataloader()
1592 # Setting up training control variables:
1593 # number of training epochs: num_train_epochs
1594 # number of training steps per epoch: num_update_steps_per_epoch
1595 # total number of training steps to execute: max_steps
1596 total_train_batch_size = self._train_batch_size * args.gradient_accumulation_steps * args.world_size
File /libraries/Custom/lib/python3.9/site-packages/transformers/trainer.py:830, in Trainer.get_train_dataloader(self)
827 dataloader_params["worker_init_fn"] = seed_worker
828 dataloader_params["prefetch_factor"] = self.args.dataloader_prefetch_factor
--> 830 return self.accelerator.prepare(DataLoader(train_dataset, **dataloader_params))
File /libraries/Custom/lib/python3.9/site-packages/torch/utils/data/dataloader.py:241, in DataLoader.__init__(self, dataset, batch_size, shuffle, sampler, batch_sampler, num_workers, collate_fn, pin_memory, drop_last, timeout, worker_init_fn, multiprocessing_context, generator, prefetch_factor, persistent_workers, pin_memory_device)
238 raise ValueError('timeout option should be non-negative')
240 if num_workers == 0 and prefetch_factor != 2:
--> 241 raise ValueError('prefetch_factor option could only be specified in multiprocessing.'
242 'let num_workers > 0 to enable multiprocessing.')
243 assert prefetch_factor > 0
245 if persistent_workers and num_workers == 0:
ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing.
I apologize if it seems like I haven’t done enough research before asking, but I can’t even begin to understand how to debug this. I would be grateful to get any help on this. If any additional info is needed, I will try editing the question accordingly asap