I am trying to create a LangChain ReAct agent that manages GitHub issues. I created a function like this:

    def create_issue(self, repo_name: str, title: str, body: str, assignee: str, label: str) -> List[Issue]:
        repo = self.__get_repository(repo_name)
        return repo.create_issue(title=title, body=body, assignee=assignee, labels=[label])

Using the PyGithub library. Then I created a tool like this:

@tool
def create_github_issue(args: str) -> str:
    """
    my description here
    """
    argument validation code
    call the above create_issue function

Then I have a prompt like this in my agent:

    PROMPT_TEMPLATE="""You are helpful, respectful ...
You have access to the following tools to achieve this goal:
{tools}

To answer the question you can use the following process:

Question: ...
Thought: ...
Action: ...
Action Input: ...
Observation: ...
... (the flow Thought/Action/Action Input/Observation could repeat max N time until you find the Final Answer)
Thought: ...
Final Answer: ...

Begin!

Question: {input}
Thought:{agent_scratchpad}"""

The problem is that the input argument arg of the create_github_issue is indeterministic. Sometimes I get:
repo name, title, body, assignee, label

Others I get:
repo name=xxxxxx, title=yyyyy, body=uuuuu, assignee=xxxxx, label=ttttt

And other variants. It’s tough to understand how to parse these parameters.

I read all the possible articles on the Internet, watched many YouTube videos, etc. The only thing I found to have more deterministic parameters is to act on the prompt (the one related to the tool description and the one of the agent). However, it does not always work as expected.

Then in all my infinite experiments, I found that sometimes LLMs (I use LLama 3.1 with Ollama provider) do very bad reasoning or run extra actions not necessary.

Regarding the problem of parameters management, I tried also to use Pydantic but it’s not clear to me the real benefits of using it, I noticed that sometimes I got parse errors because it receives strange formatting. Moreover, I have little control over the exceptions it raises.

I would like to understand, in my specific case, what should be the best approach to use to have the most benefits and deterministic result from this technology in my specific scenario.