How to convert a AutoModelForCausalLM object to a dspy model object?
import dspy
Configure language model
llm = dspy.HFModel(model=’model’)
This method takes a string as input for the model if i have a quantized model object of the class AutoModelForCausalLM How i can convert the model object to dspy object?
as direct assignment gives error on inference
llm = model #previously created as AutoModelForCausalLM class object
llm(“Testing testing, is anyone out there?”)
File /opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py:623, in LlamaModel.forward(self, input_ids, attention_mask, position_ids, past_key_values, inputs_embeds, use_cache, output_attentions, output_hidden_states, return_dict)
621 raise ValueError(“You cannot specify both decoder_input_ids and decoder_inputs_embeds at the same time”)
622 elif input_ids is not None:
–> 623 batch_size, seq_length = input_ids.shape
624 elif inputs_embeds is not None:
625 batch_size, seq_length, _ = inputs_embeds.shape
AttributeError: ‘str’ object has no attribute ‘shape’