I have a huge single-column table with engine=Log:
SELECT * FROM addresses_tmp LIMIT 5
┌─address──────────────────────────────────┐
1. │ 18a0a8bdcbd1fec1785224cfc486ccf02dc3ef5d │
2. │ 3ca0a8d9744b229f81fae2f59892b546c20a744e │
3. │ 4456058ebd1ae161348b5aae51d86aef423513a6 │
4. │ a3230a93a31f924a2713af72733d522873434025 │
5. │ 4960323c0fbd63ae068ea313c67bb2a3bc133baf │
└──────────────────────────────────────────┘
I try to insert it into a ReplacingMergeTree table:
create table addresses engine=ReplacingMergeTree() primary key address as select row_number() over() as id, * from (select * from addresses_tmp)
but it fails with memory error:
Code: 241. DB::Exception: Received from localhost:9000. DB::Exception:
Memory limit (total) exceeded: would use 27.89 GiB (attempt to
allocate chunk of 5248943 bytes), maximum: 27.86 GiB.
OvercommitTracker decision: Query was selected to stop by
OvercommitTracker.:
How else can I perform conversion to MergeTree or ReplacingMergeTree and deduplicate the table?