I have a data frame with a MultiIndex, and one of the index columns is a DateTime:
df
symbol | timestamp | trade_count |
foo |
---|---|---|---|
ABC | 2023-01-19 00:00:00-05:00 | 113381.0 | 0.0 |
2023-01-20 00:00:00-05:00 | 79924.0 | 0.0 | |
… | … | … | … |
XYZ | 2024-08-08 00:00:00-04:00 | 92877.0 | 0.0 |
2024-08-09 00:00:00-04:00 | 156009.0 | 0.0 |
I would like to say: “for all rows where symbol==ABC and trade_count>100,000, set foo to 123.4”. So in other words, in the data shown above, the top row would have foo get set to 123.4, and it would be left as 0.0 in the other three rows.
I can get a set of Booleans representing whether I want to change the value of foo with:
idx = pd.IndexSlice
df.loc[idx['ABC', :], 'trade_count']>100000
symbol | timestamp | trade_count |
---|---|---|
ABC | 2023-01-19 00:00:00-05:00 | True |
2023-01-20 00:00:00-05:00 | False |
I would have thought that to actually make the change, it would be something like one of these:
df.loc[idx['ABC', :], 'foo'].loc[df.loc[idx['ABC', :], 'trade_count']>100000] = 123.4
df.loc[idx['ABC', :], 'foo'].loc[df['trade_count']>100000] = 123.4
df.loc[idx['ABC', :], 'foo'][df['trade_count']>100000] = 123.4
I will admit that I don’t fully understand the difference between the above three statements. Something about changing a copy vs changing the original. All three of those statements run without errors, but when I print the DataFrame, none of the values of foo
are actually changed.
My next thought was maybe it’s something about the DateTime index column, e.g. maybe there isn’t equality between two DateTimes that represent the same value? So I tried this:
bools = (df.loc[idx['ABC', :], 'trade_count']>100000).values # Gives a Series of Trues and Falses
df.loc[idx['ABC', :], 'asdf'].loc[bools] = 123.4
…but again when I print out df
, the value of foo
is still 0.0 in every row.
Can anyone please help me figure out what I’m doing wrong? Thank you!
Jon Keller is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.