I am trying to fine tune a Bert Pretrained model and I am using Transformers trainer and I use the TrainingArguments to tune some hyperparameters
training_args = TrainingArguments(
output_dir='/content',
do_train=True,
do_eval=True,
num_train_epochs=100,
per_device_train_batch_size=32,
per_device_eval_batch_size=16,
warmup_steps=100,
weight_decay=0.05,
logging_strategy='steps',
#logging_dir='./multi-class-logs',
logging_steps=50,
evaluation_strategy="steps",
eval_steps=50,
save_strategy="steps",
load_best_model_at_end=True
)
but I get this error
/usr/local/lib/python3.10/dist-packages/transformers/training_args.py:1474: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
warnings.warn(
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-32-b28a2072fb72> in <cell line: 1>()
----> 1 training_args = TrainingArguments(
2 output_dir='/content',
3 do_train=True,
4 do_eval=True,
5 num_train_epochs=100,
4 frames
/usr/local/lib/python3.10/dist-packages/transformers/training_args.py in _setup_devices(self)
2053 if not is_sagemaker_mp_enabled():
2054 if not is_accelerate_available():
-> 2055 raise ImportError(
2056 f"Using the `Trainer` with `PyTorch` requires `accelerate>={ACCELERATE_MIN_VERSION}`: "
2057 "Please run `pip install transformers[torch]` or `pip install accelerate -U`"
ImportError: Using the `Trainer` with `PyTorch` requires `accelerate>=0.21.0`: Please run `pip install transformers[torch]` or `pip install accelerate -U`
---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.
To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------
I am running this on colab jupyter notebbok with T4 GPU accelerator
the veriosn of pytorch accelerate is 0.32.1
I run also the command pip install transformers[torch]
I try the soultion mentionned here ImportError: Using the Trainer with PyTorch requires accelerate = 0.20.1 but it doesn’t work
2