I’m using Datastax’s python driver to submit concurrent requests like so:
concurrent.execute_concurrent(…concurrency=50)
The total number of requests submitted is around 200, and some result sets have thousands of rows.
I occasionally run into the following error:
exception while fetching results (‘Unable to complete the operation against any hosts’, {<Host: 127.0.0.1:9043 us-east>: ConnectionShutdown(‘CRC mismatch on header fc3b22. Received 85″, computed 67cd61.’), <Host: 10.0.0.235:9042 us-east>: ConnectionException(‘Host has been marked down or removed’)})
How can I avoid this error? I suspect it’s occurring due to the large amount of data being transferred over the network but can’t pinpoint the exact root cause. What is a safe level of concurrency?
driver version:’3.29.1′
cassandra version: cqlsh 6.1.0 | Cassandra 4.1.4 | CQL spec 3.4.6 | Native protocol v5]
- I tried playing around with different levels of concurrency – I need to set it at least 50 otherwise the queries take way too long to finish executing.
- nodetool status shows that all the nodes are up
- optimized my queries so that exact partition key is specified in each query – none of them use the “IN” operator on partitions.
- used prepared statements
Fathima Khazana is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.