Background

I want to convert the header information of a FITS data into a dictionary and save it as a binary file using Python. And of course, the binary data has to be loaded later.

Problem

The dictionary of FITS header data saved as a pickle file cannot be loaded, and I want to know how to fix this.

Here is the minimal reproducible example and the error message.

import astropy.io.fits as fits
import numpy as np
import os
import pickle

cwd = os.getcwd()
fits_path = os.path.join(cwd, 'hmi.M_45s.20200101_020130_TAI.2.magnetogram.fits')
path = os.path.join(cwd, 'demo.pkl')
hdul = fits.open(fits_path)
hdul.verify('silentfix')
print(hdul.info())
# hdul[0] is often the primary Header Data Unit (HDU) and hdul[1] is the first extension HDU
hdr = dict(hdul[1].header)
hdr['COMMENT'] = str(hdr['COMMENT'])
img  = np.transpose(hdul[1].data)
hdul.close()

with open(path, 'wb') as f:
    pickle.dump(hdr, f)
with open(path, 'rb') as f:
    hdr_loaded = pickle.load(f)

Filename: cwdhmi.M_45s.20200101_020130_TAI.2.magnetogram.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU       6   ()
  1  COMPRESSED_IMAGE    1 CompImageHDU    114   (4096, 4096)   int32
None
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[49], line 21
     19     pickle.dump(hdr, f)
     20 with open(path, 'rb') as f:
---> 21     hdr_loaded = pickle.load(f)

TypeError: __new__() missing 1 required positional argument: 'table_header'

With my environment (python 3.9.18), dumping and loading of img work fine, while they don’t for hdr.

You can download the FITS data provided by the Helioseismic and Magnetic Imager (HMI) onboard the Solar Dynamic Observatory (SDO) from here, by entring

hmi.M_45s[2020.01.01_02:01:30_TAI]

to the “RecordSet” and providing your email address in the “Notify”. Then click “check parameters” and “submit”. A link to download the data will be shown in a while.

What I Searched

I guessed that the issue is related to FITS data. Also I found a similar problem Pickling an astropy.io.fits.CompImageHDU fails, but I am not sure how to fix my code.