When whorking with pandas, it is necessary to use apply or map function. In my case, I have a very long (14 hours) traitement of my data to do and I want to save the DataFrame if any error raise in the middle.

To be mode concrete, consider the following code

import pandas as pd
from math import log

data = pd.DataFrame()
data['x'] = [1,2,-1,4,5]

try:
    data['y'] = data['x'].apply(log)
except Exception as e:
    data.to_pickle('data.pkl')
    raise(e)

When it is executed, it raise a ValueException when trying to compute log(-1). I would like to save my already computed data and the re-raised the Exception.
Unfortunatly this is not working. When you try

data_bis = pd.read_pickle('data.pkl')
print(data_bis)

You get only the x column. As far as I understand, first pandas create a new Series with the computed values and then append it to the dataframe.

Do you have any idea how to save already computed data before re-raising the exception ?