I’ve bug checked this thoroughly, and the full script is hundreds of lines long. I know that the bug is introduced when outputting to csv via the df.to_csv method.
The method is randomly adding lineterminators which aren’t called for in any way.
I spotted others complain of a similar issue on Windows, but I’m running mine on a Linux server. It actually works fine in windows. The proposed method also doesn’t work. https://github.com/pandas-dev/pandas/issues/20353
Has anybody come across this before? Do you know of any solutions or alternatives for returning the data to csv?