I was able to write a code where i upload an image and the salesforce LLM can convert that into text. Instead of just text, i want it to write a story(upto 30 words). I’m using huggingface repos.

from dotenv import find_dotenv, load_dotenv
from transformers import pipeline
from langchain import PromptTemplate, LLMChain, OpenAI

load_dotenv(find_dotenv())

# img2txt

def img2txt(url):
    image_to_text = pipeline("image-to-text", model="Salesforce/blip-image-captioning-large")

    text = image_to_text(url)
    
    print(text)
    return text

img2txt('image3.jpeg')

# llm
def generate_story(scenario):
    template = """
    You are a story teller;
    You can generate a short story based on a simple narrative, the story should not be more than 20 words;

    CONTEXT: {scenario}
    STORY:
    """

    prompt = PromptTemplate(template=template, input_variables=["scenario"])

    story_llm = LLMChain(llm=OpenAI(
        model_name="gpt-3.5-turbo", temprature=1), prompt=prompt, verbose=True)
    
    story = story_llm.predict(scenario=scenario)

    print(story)
    return story

I’m having tough time on the second part and cant figure out the second part. This is all new to me. Used Meta.ai and copilot, but nothing helps. The image 2 text part works but cnt get it to write a paragraph.