I am trying to connect Azure Databricks to an on-premises SQL Server database. My on-prem environment requires connecting to a Cisco VPN, after which I authenticate using a username and password. Once connected to the VPN, I use SQL Server credentials to access the database.

Here are the details of my setup:

Azure Databricks: Hosted in Azure.
On-premises Database: SQL Server, accessible only after connecting to the Cisco VPN.
VPN Details: Requires a username and password to establish a connection.
Database Access: SQL Server credentials (username and password) are used post-VPN connection.
My questions are:

What is the recommended approach to connect Databricks to this on-prem database considering the VPN requirement?
Is it possible to establish the VPN connection from Databricks itself, or do I need to set up a different solution, such as Azure VPN Gateway or another intermediary?
Are there any specific libraries or configurations in Databricks that support secure connections to SQL Server in such scenarios?
I have considered using JDBC for the SQL Server connection, but I am unsure how to handle the VPN step. If there are additional steps or configurations required, please provide detailed instructions or references.

Thank you for your help!