I have a use case to run a SQL bigquery in required format using python recipie in dataiku.
In the local, I am able to set the path of service account credentials of a .json file and pass that file as credential to my query. However, I am unable to figure out option to use that .json file in Dataiku.

I have tried two options:

Option1:

import dataiku
from dataiku import pandasutils as pdu
import pandas as pd
import google.cloud.bigquery as bigquery

from google.cloud import bigquery
import os
import pandas as pd

cred_file = "adc.json"
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]=cred_file
os.environ["GOOGLE_CLOUD_PROJECT"] = "PROJECT_ID"
def query_bq(QUERY):
    client = bigquery.Client(project="PROJECT_ID")
    query_job = client.query(QUERY)  # API request
    rows = query_job.result()  # Waits for query to finish
    df = rows.to_dataframe()
    return df


# Initialize BigQuery client
client = bigquery.Client()

# Step 1: Fetch SQL snippets from BigQuery
fetch_sql_query = """
SELECT DISTINCT Col1, Col2, col_3, Col_4 FROM  `BQ_Table_name`
"""

query_job = client.query(fetch_sql_query)
snippets = query_job.result()

Error:

DefaultCredentialsError: File adc.json was not found.

Option2:

#created a managed folder in Dataiku and uploaded the adc.json file into it.
folder_path = dataiku.Folder("folder_name").get_path()

credentials = service_account.Credentials.from_service_account_file(
"folder_path/adc.json"
)
client = bigquery.Client(credentials=credentials, project='PROJECT_ID')

Error:

MalformedError: Service account info was not in the expected format, missing fields token_uri, client_email.

FYI: I am able to run the option1 in my local without any errors, This issue is while trying to execute the code in Dataiku Python notebook.