Unable to solve dtype issue using UnslothAI fine tuning for Llama 3.1 8B model
I am new to fine tuning LLMs and I have been trying to run the notebooks provided by UnSlothAI. For this question, I am running the code for fine-tuning LLaMa 3.1 8B model as posted here
Finetuning LLama3 on hardware specification data
i want to train llama3-8B model on intel Xeon series CPUs specifications(cores, Gflops, cache, frequency, etc.)
GPU space required to fine-tune llama3 70B
I am wondering how much GPU space would be required to be able to fine-tune Llama3-Instruct-70B. This can be using 4bit quantisation and QLoRA to minimise GPU space used.
Why am I facing an OOM error when finetuning on a 3090Ti GPU but others on smaller GPUs don’t face this issue?
I’m following the code, exactly, as per the tutorial mentioned here. I’ve seen others on Reddit running 12GB machines mention that they follow this tutorial and they’re able to run the code in the blog as it is, except that they change the batch size to 3 to avoid an OOM error.