Summary
Procedure:
I have three functions. Function A, B, and C. Function A uses apply() to apply function B and C to a global Pandas DataFrame.
Problem:
Inspecting the results shows that only Function B was applied to the global dataframe
Other notes:
If I apply Function C from the python interpreter, then it works.
The long version
The three main functions in this problem are:
load_paypal(): Loads data into a gobal Pandas DataFrame and applies the other two functions on a couple columns.
read_cash(): reads in the value, strips dollar signs, commas etc and returns a number
read_date(): reads a string and returns a datetime.
The problem I’m having is that when I use apply() to apply read_cash, it appears to work but read_date doesn’t. Additionally, when I use the read_date function with apply from the python interpreter, with the exact same code, I get the expected results, ie it works.
Functions
load_paypal
<code>def load_paypal():
global paypal_data
paypal_data = pd.DataFrame( pd.read_csv(open("Download.csv") ) )
paypal_data = paypal_data.fillna(0)
cash_names = ('Gross', 'Fee', 'Net', 'Shipping and Handling Amount', 'Sales Tax', 'Balance')
for names in cash_names:
paypal_data[names].apply( ryan_tools.read_cash )
paypal_data = paypal_data.rename(columns = { paypal_data.columns[0] : 'Date'})
paypal_data['Date'].apply( ryan_tools.read_date )
print( paypal_data['Date'] ) # The 'Date' datatype is still a string here
print( paypal_data['Net'] ) # The 'Net' datatype is proven to be converted
# to a number over here( It definitely starts out as a string )
return
</code>
<code>def load_paypal():
global paypal_data
paypal_data = pd.DataFrame( pd.read_csv(open("Download.csv") ) )
paypal_data = paypal_data.fillna(0)
cash_names = ('Gross', 'Fee', 'Net', 'Shipping and Handling Amount', 'Sales Tax', 'Balance')
for names in cash_names:
paypal_data[names].apply( ryan_tools.read_cash )
paypal_data = paypal_data.rename(columns = { paypal_data.columns[0] : 'Date'})
paypal_data['Date'].apply( ryan_tools.read_date )
print( paypal_data['Date'] ) # The 'Date' datatype is still a string here
print( paypal_data['Net'] ) # The 'Net' datatype is proven to be converted
# to a number over here( It definitely starts out as a string )
return
</code>
def load_paypal():
global paypal_data
paypal_data = pd.DataFrame( pd.read_csv(open("Download.csv") ) )
paypal_data = paypal_data.fillna(0)
cash_names = ('Gross', 'Fee', 'Net', 'Shipping and Handling Amount', 'Sales Tax', 'Balance')
for names in cash_names:
paypal_data[names].apply( ryan_tools.read_cash )
paypal_data = paypal_data.rename(columns = { paypal_data.columns[0] : 'Date'})
paypal_data['Date'].apply( ryan_tools.read_date )
print( paypal_data['Date'] ) # The 'Date' datatype is still a string here
print( paypal_data['Net'] ) # The 'Net' datatype is proven to be converted
# to a number over here( It definitely starts out as a string )
return
ryan_tools.read_date
<code>def read_date(text):
for fmt in ( '%m/%d/%y' , '%M/%D/%y' , '%m/%d/%Y', '%Y/%m/%d', '%Y/%M/%D', 'Report Date :%m/%d/%Y', '%Y%M%D' , '%Y%m%d' ):
try:
return datetime.datetime.strptime(text, fmt)
except ValueError:
pass
raise ValueError('No Valid Date found')
</code>
<code>def read_date(text):
for fmt in ( '%m/%d/%y' , '%M/%D/%y' , '%m/%d/%Y', '%Y/%m/%d', '%Y/%M/%D', 'Report Date :%m/%d/%Y', '%Y%M%D' , '%Y%m%d' ):
try:
return datetime.datetime.strptime(text, fmt)
except ValueError:
pass
raise ValueError('No Valid Date found')
</code>
def read_date(text):
for fmt in ( '%m/%d/%y' , '%M/%D/%y' , '%m/%d/%Y', '%Y/%m/%d', '%Y/%M/%D', 'Report Date :%m/%d/%Y', '%Y%M%D' , '%Y%m%d' ):
try:
return datetime.datetime.strptime(text, fmt)
except ValueError:
pass
raise ValueError('No Valid Date found')
ryan_tools.read_cash
<code>def read_cash(text):
text = str(text)
if text == '':
return 0
temp = text.replace(' ', '')
temp = text.replace(',', '')
temp = temp.replace('$', '')
if ('(' in temp or ')' in temp):
temp = temp.replace('(', '')
temp = temp.replace(')', '')
ans = float(temp) * -1.0
return ans
ans = round(float(temp),2)
return ans
</code>
<code>def read_cash(text):
text = str(text)
if text == '':
return 0
temp = text.replace(' ', '')
temp = text.replace(',', '')
temp = temp.replace('$', '')
if ('(' in temp or ')' in temp):
temp = temp.replace('(', '')
temp = temp.replace(')', '')
ans = float(temp) * -1.0
return ans
ans = round(float(temp),2)
return ans
</code>
def read_cash(text):
text = str(text)
if text == '':
return 0
temp = text.replace(' ', '')
temp = text.replace(',', '')
temp = temp.replace('$', '')
if ('(' in temp or ')' in temp):
temp = temp.replace('(', '')
temp = temp.replace(')', '')
ans = float(temp) * -1.0
return ans
ans = round(float(temp),2)
return ans
Notes: ryan_tools is just my general file of commonly used useful functions
.apply() is not an in-place operation(i.e., it returns a new object rather than modifying the original):
<code>In [3]: df = pd.DataFrame(np.arange(10).reshape(2,5))
In [4]: df
Out[4]:
0 1 2 3 4
0 0 1 2 3 4
1 5 6 7 8 9
In [5]: df[4].apply(lambda x: x+100)
Out[5]:
0 104
1 109
Name: 4, dtype: int64
In [6]: df
Out[6]:
0 1 2 3 4
0 0 1 2 3 4
1 5 6 7 8 9
</code>
<code>In [3]: df = pd.DataFrame(np.arange(10).reshape(2,5))
In [4]: df
Out[4]:
0 1 2 3 4
0 0 1 2 3 4
1 5 6 7 8 9
In [5]: df[4].apply(lambda x: x+100)
Out[5]:
0 104
1 109
Name: 4, dtype: int64
In [6]: df
Out[6]:
0 1 2 3 4
0 0 1 2 3 4
1 5 6 7 8 9
</code>
In [3]: df = pd.DataFrame(np.arange(10).reshape(2,5))
In [4]: df
Out[4]:
0 1 2 3 4
0 0 1 2 3 4
1 5 6 7 8 9
In [5]: df[4].apply(lambda x: x+100)
Out[5]:
0 104
1 109
Name: 4, dtype: int64
In [6]: df
Out[6]:
0 1 2 3 4
0 0 1 2 3 4
1 5 6 7 8 9
What you probably want is to reassign the column to the new one created by your .apply():
<code>paypal_data['Date'] = paypal_data['Date'].apply(ryan_tools.read_date)
</code>
<code>paypal_data['Date'] = paypal_data['Date'].apply(ryan_tools.read_date)
</code>
paypal_data['Date'] = paypal_data['Date'].apply(ryan_tools.read_date)
0