I want to fetch Metadata of a s3 bucket (with prefix) from a glue job. Like I want to get filename, time of creation, file size etc.

Inside my glue job (pyspark) I’m converting csv files to parquet and storing them in some s3 location, and I want to get this parquet files details.

I tried using paginator from boto3 but it wasn’t working. Is there any simpler approach?