I’m migrating some python C extension to numpy 2. The extension basically gets a list of 2D numpy arrays and generates a new 2D array by combining them (average, median, etc,). The difficulty is that the input and output arrays are byteswapped. I cannot byteswap the input arrays to machine order (they are too many to fit in memory). So
- I read an element of each input array,
- bitswap them to machine order,
- then cast them to a list of doubles,
- perform my operation on the list to obtain a double
- cast the double to the dtype of the output array
- bitswap again
- and write the result in the output array
To achieve this (using numpy 1.x C-API) I was using something like:
PyArray_Descr* descr_in = PyArray_DESCR((PyArrayObject*)input_frame_1);
PyArray_CopySwapFunc* swap_in = descr_in->f->copyswap;
PyArray_VectorUnaryFunc* cast_in = PyArray_GetCastFunc(descr_in, NPY_DOUBLE);
And something slightly different but similar for the output. I use the function swap_in
to read a value from the input array, bitswap it and write it into a buffer and then cast_in
to cast the contents of the buffer into a double.
In numpy 2, the copyswap
function is still accesible with a different syntax:
PyArray_CopySwapFunc* swap_in = PyDataType_GetArrFuncs(descr_in)->copyswap;
But the cast
function is not. Although the member is still in the struct, most of its values are NULL. So this doesn’t work:
PyArray_VectorUnaryFunc* cast_in = PyDataType_GetArrFuncs(descr_in)->cast[NPY_DOUBLE];
The documentation says
PyArray_GetCastFunc is removed. Note that custom legacy user dtypes can still provide a castfunc as their implementation, but any access to them is now removed. The reason for this is that NumPy never used these internally for many years. If you use simple numeric types, please just use C casts directly. In case you require an alternative, please let us know so we can create new API such as PyArray_CastBuffer() which could use old or new cast functions depending on the NumPy version.
So the function has been removed, but there isn’t a clear path to subtitute it with something else. What is the correct way of read and write values from/to bitswapped arrays?