I have a kind of huge dataset (about 100GB) based on blockchain data. I want to merge two tables based on the transactionHash, which would be impossible (O(n^2)) except because these two tables are both ordered by blockNumber, so this can be done in O(|A|+|B|)=O(n).

I’m working with a large Dask DataFrame (data) and I need to filter rows based on a specific ID (id12) in the column named ‘ID’. Additionally, I want to select only two columns (‘col1’ and ‘col2’) from the filtered DataFrame. Here’s the approach I’m currently using:

With Dask I try to create a column that has type list with integers. For example:

So, I’ve been wrestling with this script that’s supposed to work with a database. The database has these columns: open, high, low, close volume, and unix_timestamp. I’m trying to add some preprocessed indicators to the script, but it’s throwing this traceback error at me.

So, I’ve been wrestling with this script that’s supposed to work with a database. The database has these columns: open, high, low, close volume, and unix_timestamp. I’m trying to add some preprocessed indicators to the script, but it’s throwing this traceback error at me.

Traceback (most recent call last):
File “C:UsersPycharmProjectspythonProject2processing 2.py”, line 369, in process_data
df = calculate_cycles(df, open_arr, high_arr, low_arr, close_arr)
File “C:UsersPycharmProjectspythonProject2processing 2.py”, line 55, in calculate_cycles
cdl_inside_df = ta.cdl_inside(open_arr, high_arr, low_arr, close_arr)
File “C:UsersAppDataLocalProgramsPythonPython310libsite-packagespandas_tacandlescdl_inside.py”, line 16, in cdl_inside
inside = (high.diff() < 0) & (low.diff() > 0)
AttributeError: ‘NoneType’ object has no attribute ‘diff’
Error processing data: ‘NoneType’ object has no attribute ‘diff’
Loading data from D:New folderdataNew folder – CopyNew folderoutputNew folderdata_1.parquet