When I stress test on my Scylla inside a docker I can’t get more than 2000 successful write per second and write requests more than that will lead to time out even though I have dedicated a lot of resources but Scylla doesn’t use much of it.

the time out error:
Operation timed out for keyspace1.messagedb - received only 0 responses from 1 CL=LOCAL_SERIAL.

here is part of my docker compose:

  some-scylla:
    image: scylladb/scylla:latest
    container_name: some-scylla
    restart: always
    ports:
      - "9333:10000"
    command: [
      "--smp", "8",
      "--memory", "10G",
      "--experimental", "0",
      "--io-setup", "0",
      "--disable-version-check"
    ]
    volumes:
      - scylla3:/var/lib/scylla
      - ./scylla.d/memory.conf:/etc/scylla.d/memory.conf
      - ./scylla.d/io.conf:/etc/scylla.d/io.conf
      - ./scylla.yaml:/etc/scylla/scylla.yaml
      - ./scylla.d/io_properties.yaml:/etc/scylla.d/io_properties.yaml
      - ./scylla-server:/etc/default/scylla-server

here is my memory.conf:
MEM_CONF="--memory=10G"

here is my cpuset.conf:
CPUSET="--smp 16"

when I use the same client to send request to cassandra I get 40k success write per second.

I already turned off the developer mode and configured the io.conf and io_properties.yaml using its built-in scylla_io_setup and fed it to scylla and tried tuning it in scylla.yaml

here is my scylla.yaml:

num_tokens: 1024
commitlog_sync: periodic
commitlog_sync_period_in_ms: 200
commitlog_segment_size_in_mb: 1024
schema_commitlog_segment_size_in_mb: 4096
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          - seeds: "127.0.0.1"
listen_address: localhost
native_transport_port: 9042
native_shard_aware_transport_port: 19042
read_request_timeout_in_ms: 2000
write_request_timeout_in_ms: 2000
cas_contention_timeout_in_ms: 1000
endpoint_snitch: SimpleSnitch
rpc_address: localhost
rpc_port: 9160
api_port: 10000
api_address: 127.0.0.1
batch_size_warn_threshold_in_kb: 1024
batch_size_fail_threshold_in_kb: 8192
partitioner: org.apache.cassandra.dht.Murmur3Partitioner
commitlog_total_space_in_mb: -1
developer_mode: false
murmur3_partitioner_ignore_msb_bits: 12
force_schema_commit_log: true
task_ttl_in_seconds: 10
consistent_cluster_management: true
strict_is_not_null_in_views: true
api_ui_dir: /opt/scylladb/swagger-ui/dist/
api_doc_dir: /opt/scylladb/api/api-doc/