how to create a custom Spark DataWriter that works on ColumnarBatch instead of InternalRow
I want to use the Spark SQL DataSourceV2 API and create a custom DataWriter that is able to get the data in the internal ColumnarBatch representation such that I can leverage the columnar representation for efficient serialization of the data before I write out to my data storage.
how to create a custom Spark DataWriter that works on ColumnarBatch instead of InternalRow
I want to use the Spark SQL DataSourceV2 API and create a custom DataWriter that is able to get the data in the internal ColumnarBatch representation such that I can leverage the columnar representation for efficient serialization of the data before I write out to my data storage.
how to create a custom Spark DataWriter that works on ColumnarBatch instead of InternalRow
I want to use the Spark SQL DataSourceV2 API and create a custom DataWriter that is able to get the data in the internal ColumnarBatch representation such that I can leverage the columnar representation for efficient serialization of the data before I write out to my data storage.
how to create a custom Spark DataWriter that works on ColumnarBatch instead of InternalRow
I want to use the Spark SQL DataSourceV2 API and create a custom DataWriter that is able to get the data in the internal ColumnarBatch representation such that I can leverage the columnar representation for efficient serialization of the data before I write out to my data storage.
how to create a custom Spark DataWriter that works on ColumnarBatch instead of InternalRow
I want to use the Spark SQL DataSourceV2 API and create a custom DataWriter that is able to get the data in the internal ColumnarBatch representation such that I can leverage the columnar representation for efficient serialization of the data before I write out to my data storage.