I am trying to understand the difference between CPU Bound vs IO Bound process. ChatGPT suggested that multi-threading/parallel processing can help a CPU bound process; However, I think that having multithreading can help with IO Bound process.
Let’s say the application is talking with some API, and has to make various API calls. If we have multi-threading, we can make those API calls in parallel, and boost the performance.
1
If you are truly I/O bound, such as your network link being saturated, then no matter how many processes you run you will not be able to transmit requests and receive responses any faster.
If you are I/O bound, adding processes does not give you more I/O capability.
5
I/O performance has two aspects that you must distinguish: lag and throughput. One measures how long you have to wait for the first byte of a response, the other how long you have to wait additionally for the last byte.
As pjc50 said, if your network link is already saturated (no more packets can go through) it doesn’t matter how smartly you organize your program – even a quantum computer couldn’t help you.
But if your bottleneck is mainly lag (e.g. a web API that takes five seconds to respond at all), then sending requests in parallel will absolutely speed things up, and multi-threading is a convenient way of doing that.
2
In general, multi-threading access to a single resource (CPU, network, disk, etc.) will not improve performance. Multi-threading will improve performance when the theads access different resources.
Multiple cores improve performance by increasing the capability of the CPU resource.
Is there anything wrong with my understanding?
As long as the API is non-blocking, a single threaded system can also make those API calls in parallel. By the assumption, we are IO bound, so there is plenty of CPU time to send each request and service each response, as they arrive.
4