Relative Content

Tag Archive for llama

How do you finetune Llama 3 for a regression task?

I am trying to finetune Llama-3-8B for a regression task using PEFT where it would output a float to “score” the input. I have tried to adjust the model for it by changing the number of output logits to 1 and changing the loss function to mean-squared-error. I have formatted my data by providing the instruction along with the input statement, tokenized it, and kept the output float value as it is in “labels”. I am getting an error:

LLama3 Response Incompleteness

I am getting incomplete answer from meta-llama/Meta-Llama-3-8B-Instruct what can i do to improve the response completeness