I am trying to use ChatHugging face by locally downloaded model files available on huggingface for the models, however in the output I see the message which user has inputted also gets returned alongwith the role. How can i just get the message returned by the model
I tried with other tools like Ollama with ChatOllama and this gives the expected output, but I dont want to use these 3rd part tools as I only have access to model files available on huggingface and published by verified publishers
Below is the code i am using
from typing import Annotated
from typing_extensions import TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from langchain_core.messages import HumanMessage, SystemMessage, AIMessage
from langchain_huggingface import ChatHuggingFace,HuggingFacePipeline
model_id = "../downloaded_models/TinyLlama-1.1B-Chat-v1.0"
hf = HuggingFacePipeline.from_model_id(
model_id=model_id,
task="text-generation")
llm = ChatHuggingFace(llm=hf)
from langchain_core.prompts import PromptTemplate, ChatPromptTemplate
# template = """<|system|>
# you are a helpful assistant</s>
# <|user|>
# {input}</s>
# <|assistant|>"""
# prompt = PromptTemplate.from_template(template)
prompt = ChatPromptTemplate.from_messages([SystemMessage(content="you are a helpful assistant"),
HumanMessage(content="what is the capital of india?"),
AIMessage(content="The capital of india is new delhi")])
class State(TypedDict):
messages: Annotated[list, add_messages]
graph_builder = StateGraph(State)
def chatbot(state: State):
return {"messages": [llm.invoke(state["messages"])]}
graph_builder.add_node("chatbot", chatbot)
graph_builder.add_edge(START, "chatbot")
graph_builder.add_edge("chatbot", END)
graph = graph_builder.compile()
while True:
user_input = input("User: ")
if user_input.lower() in ["quit", "exit", "q"]:
print("Goodbye!")
break
for event in graph.stream({"messages": ("user", user_input)}):
for value in event.values():
print("Assistant:", value["messages"][-1].content)
output:
User: what is a model?
Assistant: <|user|>
what is a model?</s>
<|assistant|>
A model is a representation of a real-world system or phenomenon that is used to understand and predict its behavior. It is a mathematical or statistical representation of a system that can be used to simulate or predict its behavior. Models can be used in various fields, including engineering, physics, economics, and biology.
In engineering, models are used to design and test mechanical, electrical, and chemical systems. They are used to predict the behavior of these systems under different conditions, such as temperature, pressure, and flow rates. Models can also be used to simulate the behavior of complex systems, such as aerospace vehicles or power grids, and to optimize their performance.
In physics...