So, I’m a complete beginner on how to use any LLMs, given below is the code which I was using on my jupyter notebook to run a prompt to convert binary to decimal. However I was getting completely incorrect responses on Llama 3.
!pip install -r requirements.txt
import json
import torch
from transformers import (AutoTokenizer,
AutoModelForCausalLM,
BitsAndBytesConfig,
pipeline)
config_data = json.load(open("config.json"))
HF_TOKEN = config_data["HF_TOKEN"]
model_name = "meta-llama/Meta-Llama-3-8B"
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)
tokenizer = AutoTokenizer.from_pretrained(model_name, use_auth_token=HF_TOKEN)
tokenizer.pad_token = tokenizer.eos_token
model = AutoModelForCausalLM.from_pretrained(
model_name,
quantization_config=bnb_config,
use_auth_token=HF_TOKEN
)
text_generator = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
)
def convert_to_decimal(html_input):
prompt = (f"I'm giving you a input as a binary number"
f"I need you to convert the binary number to its decimal equivalent. "
f"Do not give any explanation or add any further text. "
f"Only provide the transformed output once. "
f"Don't add any other text except the output. "
f"For input '101', the output is '5'.nn"
f"{html_input}")
response = text_generator(prompt, max_new_tokens=20, temperature=0.2, return_full_text=False)
generated_text = response[0]['generated_text'].strip()
return generated_text
html_input = "1011"
output = convert_to_decimal(html_input)
print(output)
This is my code which I was trying to use, I kept getting the output as
1101 1110 1111….. till it reached the maximum number of tokens. Any help is appreciated about where am I going wrong.
New contributor
L lawliet is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.