i am changing the spark dataframe to a pandas dataframe in jupyter notebook,and there are many warn messages like this “WARN RowBasedKeyValueBatch: Calling spill() on RowBasedKeyValueBatch. Will not spill but return 0.”. Does that means something wrong? if not,what can i do to make this not display? my code is simple just like the following:
result_df_pandas = result_df.toPandas();
Does that means something wrong? if not,what can i do to make this not display?
New contributor
tiredtian is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.