In PostgreSQL database I have four big tables:
table
(25 milions of rows) (20 Gib)table_link
(60 milions of rows) (17 Gib)table_rank
(25 milions of rows) (8 Gib)table_calc
(25 milions of rows) (65 Gib)
I have a problem with syncing this data to elasticsearch index table_index
which takes around 6 hours. Mostly because queries takes quite a long time. Right now I join all these tables together and select fields that I want to have in my index.
Is there any way to transfer this data to one index without joins? Like transfer data to separated indexes and then merge them on id if possible? Or are there any other ways to do this?
I already tried to first sync data from table
and then bulk update documents (adding fields) with data from table_link
, table_rank
, table_calc
. But that did not work because of overhead.