I have the following go program, called start_python_process.go
:
package main
import (
"fmt"
"log"
"os/exec"
)
func main() {
cmd := exec.Command("python", "python_process.py")
output, err := cmd.StdoutPipe()
if err != nil {
log.Fatal(err)
}
input, err := cmd.StdinPipe()
if err != nil {
log.Fatal(err)
}
err = cmd.Start()
if err != nil {
log.Fatal(err)
}
bytes_written, err := input.Write([]byte("Welcome, Python.nSeriously.nWelcome to the world of lightweight concurrency with easy to understand, well-defined semantics. And congratulations on shipping optional GIL in Python 3.13!n"))
if err != nil {
log.Fatal(err)
}
/* err = input.Close()
if err != nil {
log.Fatal(err)
}*/
fmt.Printf("bytes written %vn", bytes_written)
buffer := make([]byte, 7)
bytes_read, err := output.Read(buffer)
if err != nil {
log.Fatal(err)
}
fmt.Println(string(buffer))
fmt.Printf("bytes read %vn", bytes_read)
cmd.Wait()
}
And the following Python program, called python_process.py
:
import sys
with open("debug_file", "w") as f:
for line in sys.stdin:
line = line.rstrip()
print(line)
print(line, file=f)
I’m expecting the following behaviour when running go run start_python_process.go
:
I’m expecting that the Go program will write 186 bytes to the stdin of the Python program, and after that write finishes, that the Go program will block until the Python program echoes the bytes, and as soon as the first seven bytes are ready (“Welcome”), it will write them on the screen and then block indefinitely (as the Stdin pipe is never closed).
However, I don’t see this expected behaviour. What I see is that the Go program blocks on the output.Read
indefinitely. When I open a separate terminal to look at the debug_file
I see an empty file (the file is created, but there is no input written to the file by Python).
When I send the interrupt signal to the go process, the debug_file
gets fully written with the expected message:
Welcome, Python.
Seriously.
Welcome to the world of lightweight concurrency with easy to understand, well-defined semantics. And congratulations on shipping optional GIL in Python 3.13!
My interpretation is that the Write function is actually buffered, and these 186 bytes do not hit the python code until I send an interrupt signal to the Go code. This does not feature anywhere in the documentation, cmd.StdinPipe()
is supposed to implement io.Writer
, which quite clearly is not supposed to be buffered, as opposed to bufio.Writer
, which does feature a .Flush()
method to flush the buffer. No such method exists for the io.Writer
.
Is this essentially a bug in Go or am I missing something?
If this is of any help, I’m on MacOS and that’s the output of go version
:
go version go1.23.0 darwin/amd64