I appreciate this might be impossible in the most general, case but is there a way to convert a pyarrow table into record batches of a fixed maximum byte size? I know I can request a maximum row size, so this should be possible if I can find the maximum row size for a given schema. I didn’t see anything in the docs of either pa.Schema
or pa.Field
about their sizes?