I have a file Large_File.pkl
that I would like load as a cudf.DataFrame
object. However, it seems like there is no direct pandas.read_pickle()
alternatives in cudf.
The work-around I have found so far is to load the pickle object first as a pandas.DataFrame
object then convert it to a cudf.DataFrame
:
import pandas as pd
import cudf
# Load the pickle file into a pandas DataFrame
pd_dataframe = pd.read_pickle('Large_File.pkl')
# Convert the pandas DataFrame to a cuDF DataFrame
cudf_dataframe = cudf.DataFrame.from_pandas(pd_dataframe)
I am wondering whether there are more memory-efficient solutions to this problem?