We have a ClickHouse server (version 24.4.1.2088) running inside a Docker container alongside a Django web server on an 8GB machine.
The max_server_memory_usage_to_ram_ratio is set to 0.4.
We notice that the memory usage of ClickHouse grows over time, and then we get a lot of Exception: Memory limit (total) exceeded after reaching 3GB memory usage. The memory usage is reset to baseline after the server is restarted.
The thing is that we don’t have any SELECT queries running yet; we just do async inserts for now. There are around 4 inserts per second.
We have two tables with 45 million and 12 million rows.
It was written here that:
“If your system has less than 16 GB of RAM, you may experience various memory exceptions because default settings do not match this amount of memory. The recommended amount of RAM is 32 GB or more. You can use ClickHouse in a system with a small amount of RAM, even with 2 GB of RAM, but it requires additional tuning and can ingest at a low rate.”
Given our current setup and data size, what would be a good configuration for a low memory machine? If an upgrade is needed, how much memory would be required to handle our current data size?