pyarrow: not-nullable field in schema does not prevent null values in corresponding table column
I am fairly new to PyArrow and Arrow, so I may ask something stupid. I have read the documentation of the field()
method, and as far as I understand I can say that a filed does not allow NULL values by specifying nullable=False
. So, I have tried this example:
Convert hexadecimal StringArray to FixedSizeBinaryArray
I have an array of ethereum addresses in hexadecimal format (20 bytes), encoded as an hexadecimal string. How can I transform that into a FixedSizeBinaryArray?
Size of pyarrow Table in bytes
I have a basic pyarrow.Table. What’s the best way to get it’s size in bytes?
How do I combine columns from multiple files into a single arrow table
I have several thousand text files with methylation data in them. These are tab separated files, where I am only interested in two columns, the name of the methylation probe and a value in the column called “Beta”.