We are using jdbc sinks (apache flink) with which we are hitting the database maximal session count, especially when we increase parallelism.
Our tests showed that if we increase our default parallelism by setting parallelism.default property more than 1 we have more inactive sessions on the database. When we increase it furthermore, we will come to a point where we can see some exception because jdbc connections cannot established anymore.
By executing our tests with different parallelism.default properties we have encountered following behavior:
application.parallelism.default = 1 active session / inactive session -> 64/35
application.parallelism.default = 2 active session / inactive session -> 66/70
application.parallelism.default = 3 active session / inactive session -> 63/105
application.parallelism.default = 4 active session / inactive session -> 64/140
application.parallelism.default = 5 active session / inactive session -> 64/175
application.parallelism.default = 6 active session / inactive session -> 64/210
application.parallelism.default = 7 active session / inactive session -> 64/245
application.parallelism.default = 8 active session / inactive session -> 64/280
Is there any chance to limit the session used by the jdbc sinks?
We want to know if there is a proper why to keep the inactive sessions under control used by our jdbc sinks.
user26565994 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.