What I have done in AWS:
I have a datalake in AWS. I use AWS Glue Crawlers to get the schema of data (.parquet format) in different partitions (e.g. year=2000/month=1/day=1/file1.parquet, year=2000/month=1/day=1/file2.parquet) and populate a table in a database in AWS Glue Catalog.
I use AWS Athena to query the table in the database using pyathena and use this data locally for some processing.