In Windows, piping Unicode characters to a text file causes a UnicodeEncodeError to be raised. How to avoid this error?
(env6) C:UsersMichael>python --version
Python 3.11.9
(env6) C:UsersMichael>python -c "print('📂')"
📂
(env6) C:UsersMichael>python -c "print('📂')" > log.txt
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:anaconda3envsenv6Libencodingscp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character 'U0001f4c2' in position 0: character maps to <undefined>
The UnicodeEncodeError occurs because the default encoding in the Windows command prompt is cp1252, which cannot encode certain Unicode characters. cp1252 is a legacy single-byte character encoding that is used by default (as the “ANSI code page”) in Microsoft Windows. The correct modern encoding to use is utf-8.
To avoid the error, you can explicitly set the encoding to utf-8 when redirecting the output to a file. You can set the PYTHONIOENCODING environment variable to utf-8 temporarily for the command:
set PYTHONIOENCODING=utf-8 && python -c "print('📂')" > log.txt
This sets the environment variable PYTHONIOENCODING to utf-8 for the duration of the command.