I have a pyarrow Schema defined and a list of native Python dictionaries. I can use
pyarrow.Table.from_pylist(list_of_python_objects, schema=SCHEMA)
and that will create a table matching the schema. However, I’d like to be able to validate the Python objects against the pyarrow schema, similarly to how pyarrow.json.ParseOptions.unexpected_field_behavior = "error"
will raise an exception if the object has fields in it that aren’t in the provided schema.
Right now, the implementation of from_pylist
iterates over the schema by field name and has no logic to detect and raise an exception if extra keys are in the input objects.