I have this code for writing a parquet file to SQL database table but it is not working
import pandas as pd
import pyodbc
try:
cnxn = pyodbc.connect(
r'Driver={SQL Server};'
r'Server=localhost,1433;' # Assuming the default port 1433
r'Database=fatigue_life_test_db;'
r'Trusted_Connection=yes;'
)
cursor = cnxn.cursor()
print("Connection established.")
except pyodbc.Error as e:
print("Error in connection:", e)
finally:
path = r'pathtoparquerttest.parquet'
df = pd.read_parquet(path)
df.to_sql('Test_data_table', con=cnxn, if_exists='append')
if 'cnxn' in locals() and cnxn:
cnxn.close()
The code gives the following error: pandas.errors.DatabaseError: Execution failed on sql ' SELECT name FROM sqlite_master WHERE type IN ('table', 'view') AND name=?; ': ('42S02', "[42S02] [Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name 'sqlite_master'. (208) (SQLExecDirectW); [42S02] [Microsoft][ODBC SQL Server Driver][SQL Server]Statement(s) could not be prepared. (8180)")
I tried doing this code according to pandas documentation but it still doesn’t work. The parquet file in question is several gigabytes so could the size be an issue?
user25884369 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1