I have this numpy array ‘dat’ which is a 3d array of size 2048 MB and of shape (512,1024,512) when I do asizeof.asizeof(dat) from pympler, I get the error: ValueError: invalid option: reset(base=-2147483504)
but in a ipython session if I have defined any numpy array prior to this, then the asizeof.asizeof(dat) works without the error and I get 4096 MB instead of the actual 2048 MB. Now I understand pympler might include the buffer size, metadata etc but 2 GB for such things is too much! Kindly help me here!
- I tried creating a random array using
dat = np.random.rand(512,1024,512)
and appliedasizeof.asizeof(dat)
then I get the result without any error. - I also tried reading in only a 2d slice of the array ‘dat’ then I get the resulting memory value without any error using
asizeof.asizeof(data)
- I also tried this, read in the ‘dat’ array and set ‘dat_temp = dat*1’ then surprisingly
asizeof.asizeof(dat_temp)
works butasizeof.asizeof(dat)
gives the ‘ValueError’