I am new to LLM programming in Python and I am trying to fine-tune the instructlab/merlinite-7b-lab model on my Mac M1. My goal is to teach this model to a new music composer Xenobi Amilen I have invented.

The text of this composer is here.

Using the new Ilab CLI from RedHat I created this training set for the model. It is a JSONL file with 100 questions/answers about the invented composer.

I wrote this Python script to train the model. I tested all the parts related to the tokenizer, datasets and it seems to work. However, the final train got this error:

RuntimeError: Placeholder storage has not been allocated on MPS device!
  0%|          | 0/75 [00:00<?, ?it/s]                                                                                                                                        

I found a lot of articles about this error on Google and also StackOverflow like this, for example. The problem seems that in addition to the model I have to send to mps also the input parameters, but it’s not clear to me how to change my code to do that.

I tried several fixes but had no luck. Can anyone can help?