I have a csv dataset which I’m trying to read into a DataFrame using pd.read_csv() but the csv has a number of bad rows which can’t be type cast to the types I’m expecting. Is there a way to skip all these rows within the read_csv() function?
The issue is that the csv is 28,000 lines and the only way I know of to fix this is to read it line-by-line, adding each line into the DataFrame if thet don’t throw an error which would be more expensive than I’d like.