Relative Content

Tag Archive for pythonpandasperformancetime

How to Find Overlapping Epoch Ranges in a Dataframe

I have a very large dataframe that includes a categorical column with multiple categories and an epoch column associating each category with a number of epochs. I am trying to determine all of the epoch ranges where ALL of the categories overlap. The data is not always measured in a consistent manner, so the epochs do not necessarily match 1 to 1. My thought was to make a table with columns for epoch and each category and have a row for each epoch within the timeframe and merge my data to the nearest epoch to find where all categories are overlapping, but this is obviously a horribly inefficient way to do it. Below is a sample dataframe.

How to Find Overlapping Epoch Ranges in a Dataframe

I have a very large dataframe that includes a categorical column with multiple categories and an epoch column associating each category with a number of epochs. I am trying to determine all of the epoch ranges where ALL of the categories overlap. The data is not always measured in a consistent manner, so the epochs do not necessarily match 1 to 1. My thought was to make a table with columns for epoch and each category and have a row for each epoch within the timeframe and merge my data to the nearest epoch to find where all categories are overlapping, but this is obviously a horribly inefficient way to do it. Below is a sample dataframe.

How to Find Overlapping Epoch Ranges in a Dataframe

I have a very large dataframe that includes a categorical column with multiple categories and an epoch column associating each category with a number of epochs. I am trying to determine all of the epoch ranges where ALL of the categories overlap. The data is not always measured in a consistent manner, so the epochs do not necessarily match 1 to 1. My thought was to make a table with columns for epoch and each category and have a row for each epoch within the timeframe and merge my data to the nearest epoch to find where all categories are overlapping, but this is obviously a horribly inefficient way to do it. Below is a sample dataframe.