We are currently trying to access our built-in serverless SQL pool from our notebook. Our goal is to be able to drop views from the notebook, which is something we need to put into our pipeline as per our business requirements. We have attempted several methods but have not yet succeeded.

We initially tried using this Stack Overflow post for guidance, but encountered an error on the line regarding the token.
It’s important to note that we can only get an Azure Synapse Analytics (Artifact) linked service to work instead of an Azure SQL Database linked service. When we create a linked service for the Azure SQL Database, we receive the following message: “Check the linked service configuration is correct and make sure the SQL Database firewall allows the integration runtime to access.”
To address this, we attempted to change the SQL Database firewall settings. Although we found some methods online, they all seem to apply to dedicated pools rather than serverless. We also consulted with a Microsoft expert who believed it was possible to achieve our goal, but he had not done it before.

We have also tried the code below to connect the SQL Pool from our notebook.

%%spark

spark.conf.set("spark.sql.catalog.spark_catalog.connectionString",
               "Server=tcp:<Servername>.database.windows.net,1433;Database=<SQL database name>;Authentication=ActiveDirectoryMSI")

However when we try to query the view we get the error [TABLE_OR_VIEW_NOT_FOUND].

Honestly, we are not too sure on what the issue is or how to resolve, so any advice is greatly appreciated