In a distributed bare-metal Apache Drill, complex concurrent queries have two issue:
-
hooking the cluster resources, especially CPU, this can be somehow controlled by Linux “cgroup”.
-
the Drill seems to be serving concurrent queries as first-come-first-served, this means – even if the second query is very simple and it should not take time, it will have to wait for the first-coming complex heavy query to be finished first, which is not acceptable at all in a production environment.
my question is: is there a workaround to resolve the second problem, if not, what are the alternatives from technology stack that might help in this case?
we tried changing some Apache Drill configuration parameters related to concurrent queries and queue management.