I am using asyncio to asynchronously launch a process, and then read data from the stdout, my code looks like the following:
process = await asyncio.subprocess.create_subprocess_exe(subprocess_args, stderr=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE)
I am then asynchronously reading from the stdout of this process, and for every line read, I am calling a user-provided callback function, my code looks like:
async for line in process.stdout:
user_provided_callback(line)
However, I noticed that is user_provided_callback raises an error, the process is not properly disposed of, so I see warnings such as “Resource warning: unclosed transport”, because I am running this code through pytest unit tests, warnings are treated as errors so this is causing a failure.
I thought I would have to manually shut down the process in the case of failure, so I changed my code to:
async for line in process.stdout:
try:
user_provided_callback(line)
except Exception as err:
process.terminate()
raise err
This explicitly terminates the process in case the callback fails, but I am still seeing this warning. I then modified the code above to:
async for line in process.stdout:
try:
user_provided_callback(line)
except Exception as err:
process.terminate()
**process._transport.close()**
raise err
(part in bold is the new line), after explicitly closing the transport, now the warning goes away, however this accesses a private variable so it does not seem like the right fix. What is the proper way to handle an exception in this scenario? I am looking for a cross platform solution as I want to support windows and linux.
Raj Kumar is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.