So yesterday i was having issues trying to define all column via data frame’s to_sql method. For the most part this works just fine.

One file i was trying to process, however is giving me an odd error. See below.

When I run this :

 data.to_sql
    (
        name=f'tbl{table_name}'
        , schema='stage'
        , con=odbc_ntt.con
        , if_exists='replace'
        , index=False, dtype=sqlalchemy.types.NVARCHAR(length=2000)
    )

I get error:

 UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 2: character maps to <undefined>  

When I run this, it runs just fine.

data.to_sql
    (
        name=f'tbl{table_name}'
        , schema='stage'
        , con=odbc_ntt.con
        , if_exists='replace'
        , index=False, dtype=sqlalchemy.types.NVARCHAR
    )    

stack trace:

python -m trace --trace 
Traceback (most recent call last):
  File "C:ProgramDataAnaconda3librunpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:ProgramDataAnaconda3librunpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:ProgramDataAnaconda3libtrace.py", line 755, in <module>
    main()
  File "C:ProgramDataAnaconda3libtrace.py", line 735, in main
    code = compile(fp.read(), opts.progname, 'exec')
  File "C:ProgramDataAnaconda3libencodingscp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 2: character maps to <undefined> 

Now, I did check the file for ssing characters or otherwise bad actors. These did not exist in the source file. I tried enforicng Utf-8 encoding, nothing.

Im kind of at a loss here because the obvious things arent sticking. Any assistance appreciated <3