I have a function which gives me the following output (pd df):

column dtypes are:
timestamp Int64
duration Int64
trial_type object
blink Int64
message object
dtype: object

Now I wrote a unit test to test the output for the first row:

@pytest.mark.parametrize(     
"folder, expected",     
[("emg", [9199380, pd.NA, np.nan, pd.NA, "RECORD_START"])]
)
def test_physioevents_value(folder, expected, eyelink_test_data_dir):
    input_dir = eyelink_test_data_dir / folder
    asc_file = asc_test_files(input_dir=input_dir, suffix="*_events")[0]
    events = _load_asc_file(asc_file)
    events_after_start = _df_events_after_start(events)
    physioevents_reordered = _df_physioevents(events_after_start)
    physioevents_eye1 = _physioevents_eye1(physioevents_reordered)
    assert physioevents_eye1.iloc[0].tolist() == expected

And I get the following error when running the test:

E AssertionError: assert [9199380, <NA>…CORD_START’] == [9199380, <NA>…CORD_START’]
E
E (pytest_assertion plugin: representation of details failed: missing.pyx:392: TypeError: boolean value of NA is ambiguous.
E Probably an object has a faulty repr.)

tests/test_edf2bids.py:670: AssertionError

So I guess I cannot use pd.NA to check if the value in that field is <NA>. However, I also cannot check it using “<NA>”, i.e. encoding it as a string.

Does anyone have an idea how I can check if <NA>s in the dataframe exist?

I tried changing the dtypes so that every column has the dtype ‘object’. However, that’s not really what I want because I want to have a numeric dtype wherever I have numbers. Also, if I change the dtype to object, the output of the test suddenly tells me that the “blink” column would hold folats, which is not true because it’s always either a 0 or a 1 (and also, I don’t want floats…).

To be precise: timestamp and duration hold numerics plus nans, trial_type holds strings plus nans, blink holds numerics (0 and 1) plus nans, and message hold strings plus nans.