TypeError: 'function' object is not iterable
I keep getting this error when I feel that the filter function would achieve my desired result if only it would work !..
What I’d like to know is… What am I missing when I decide to use ‘filter’, (which to me looks neater than using ‘loc’ to achieve the same outcome), and almost always end up with this error.
Here’s a simple example.
import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.rand(4,4), columns=['a', 'b', 'c', 'd'])
df.filter(lambda x:x['b'] > 0.5, axis=1)
Traceback (most recent call last):
Cell In[190], line 1
df.filter(lambda x:x['b'] > 0.5, axis=1)
File ~/.local/lib/python3.8/site-packages/pandas/core/generic.py:5454 in filter
return self.reindex(**{name: [r for r in items if r in labels]})
TypeError: 'function' object is not iterable
In [1]: df.apply(lambda x:x['b'] > 0.5, axis=1)
Out[1]:
0 True
1 True
2 False
3 False
dtype: bool
In [2]: df.loc[df['b']> 0.5]
Out[2]:
a b c d
0 0.578342 0.82178 0.414652 0.630859
1 0.181232 0.91806 0.772260 0.502921
Even the most basic lambda function creates an error, eg.
In [5]: tru = lambda x: True
In [6]: df['b'].filter(tru)
Traceback (most recent call last):
Cell In[196], line 1
df['b'].filter(tru)
File ~/.local/lib/python3.8/site-packages/pandas/core/generic.py:5454 in filter
return self.reindex(**{name: [r for r in items if r in labels]})
TypeError: 'function' object is not iterable
Once again
In [7]: df['b'].apply(tru)
Out[7]:
0 True
1 True
2 True
3 True
Name: b, dtype: bool
In my mind, filter requires a function to return a bool, which the simplistic example clearly does, yet ‘filter’ fails and again ‘apply’ works.
I don’t see why a function is not iterable for filter but still works with apply.
What is there ‘under the hood’ that makes the difference in outcome?
I’d be very grateful if some can explain and give me the ‘eureka’ moment of understanding so that I’ll not fall in this trap again.
As explained above, I tried df.filter(function)
which failed, yet
df.loc[df.apply(function)]
yields the correct result.
John Tweed is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.