Currently when the user sends input to the LLM it has to be in the format of:

<|im_start|>user
User input here.<|im_end|>

This is very cumbersome and makes prompt engineering very tedious.
Right now i am using llama.cpp to run a llama2 model locally in the terminal – is there a way to automatically insert user input between the two tokens (im_start and im_end)?

I might have approached this problem in the wrong way, but the following is what I have tried:

My current (so far unsuccessful) method is to use a bash script to launch the LLM in another terminal and redirecting the output of the current terminal to the LLM-terminal. Later I was going to make a script that the first “launcher-terminal” would, which would format all user input and send it to the LLM-terminal.

Sofar I haven’t been able to get the communication between the terminals working. The script would be trivial for me to make. It is moreso the bash-scripting that I’m struggling with.

I suspect my bash implementation is highly flawed, I’m not very experienced with it.

RUN_PATH="/llama.cpp/main"
MODEL_PATH="/llama.cpp/models/dolphin-2.9-llama3-8b-q4_K_M.gguf"
INIT_PATH="prompt.txt"

FLAGS="-n 512 --keep -1 --ctx-size 2048 --temp 0.0"

LAUNCH_CMD=".${RUN_PATH} -m .${MODEL_PATH} ${FLAGS} --color  --interactive-first --file .${INIT_PATH}"

# gnome-terminal -- "${LAUNCH_CMD}"
# Here im launching an impostor python program which just prints the input it receives.
gnome-terminal -- python3 impostor.py

LAUNCHER_PID="${$}"
LLM_PID="$PPID"

# It seems I don't get the PIDs i expect to get.
echo "Launcher PID: ${LAUNCHER_PID}"
echo "LLM PID:  ${LLM_PID}"

# Set STD-output of the launcher to LLM
ps -p $LLM_PID << ps -p $LAUNCHER_PID
/bin/bash

When typing echo b in the launcher terminal, it does not show up in the LLM-terminal