How to Immediately Cancel an Asyncio Task That uses the Ollama Python Library to generate an answer?
I’m using ollama to generate answers from large language models (LLMs) with ollama python api. I want to cancel the response generation by clicking the stop button. The problem is that the task cancellation works only if the response generation has already started printing. If the task is still processing and getting ready to print, the cancellation does not work, and the response gets printed regardless.