Is it possible to read ragged tensors with tfio’s from_parquet
? Without specifying a RaggedTensorSpec
it returns them as one (joined) tensor, truncated after n rows. Specifying a (possibly incorrect?) RaggedTensorSpec
gives an InvalidArgumentError
.
import pandas as pd
import tensorflow_io as tfio
df = pd.DataFrame({'x': [0], [0, 1], [0, 1, 2]})
df.to_parquet('/tmp/df.pq')
## wrong layout without spec
ds = tfio.IODataset.from_parquet('/tmp/df.pq')
for x in ds.take(3):
print(x)
# OrderedDict([(b'x.list.element', <tf.Tensor: shape=(), dtype=int64, numpy=0>)])
# OrderedDict([(b'x.list.element', <tf.Tensor: shape=(), dtype=int64, numpy=0>)])
# OrderedDict([(b'x.list.element', <tf.Tensor: shape=(), dtype=int64, numpy=1>)])
## error with spec
ds = tfio.IODataset.from_parquet(
'/tmp/df.pq',
columns={'x': tf.RaggedTensorSpec(shape=[3, None], ragged_rank=1)})
# InvalidArgumentError: {{[...]}} slice index 0 of dimension 0 out of bounds