I have trained peft model and saved it in huggingface. No i want to merge it with base model.
i have used following code.

from peft import PeftModel, PeftConfig,AutoPeftModelForCausalLM
from transformers import AutoModelForCausalLM,pipeline,AutoTokenizer,BitsAndBytesConfig

config = PeftConfig.from_pretrained("sanduntg/ORPO_peft_llama3_8B")
base_model_name = "meta-llama/Meta-Llama-3-8B"

tokenizer = AutoTokenizer.from_pretrained(base_model_name)
print("Tokenizer vocab size:", len(tokenizer))

base_model = AutoModelForCausalLM.from_pretrained(base_model_name,device_map="auto")
base_model.resize_token_embeddings(len(tokenizer))

base_model = PeftModel.from_pretrained(model=base_model, model_id="sanduntg/ORPO_peft_llama3_8B")

base_model = base_model.merge_and_unload()
base_model.save_pretrained("merged_adapters")

But in this base_model = PeftModel.from_pretrained(model=base_model, model_id="sanduntg/ORPO_peft_llama3_8B") step gives following error

RuntimeError: Error(s) in loading state_dict for PeftModelForCausalLM:
    size mismatch for base_model.model.model.embed_tokens.weight: copying a param with shape torch.Size([128258, 4096]) from checkpoint, the shape in current model is torch.Size([128256, 4096]).
    size mismatch for base_model.model.lm_head.weight: copying a param with shape torch.Size([128258, 4096]) from checkpoint, the shape in current model is torch.Size([128256, 4096]).

I have tried above steps with resize to token embeddings. But it not worked. When it training i used the following configuration

peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=['up_proj', 'down_proj', 'gate_proj', 'k_proj', 'q_proj', 'v_proj', 'o_proj']
)