Since asking this, I started looking into using polars to handle my data also. From the docs for map_rows
and in general, there are many warnings about using python functions rather than their own expression system as it’s much slower. Is it possible to use polars expressions to find contiguous sequences in dataframe rows?
An extra question would be if the @njit decorator on my existing UDF (from the previous question) has any effect on the running of the polars map_rows
. If it’s relevant I’m currently using the polars Array type instead of List (but I can change if needed).