I have data in ADLS which is partitioned as shown in image [Partitioned Data structure]
Partitioned Data structure

• The plan from the below query does not have the filter stage on the partition.

EXPLAIN ANALYZE select *
from read_parquet(‘abfss://.dfs.core.windows.net//SkyCAST/FlightReportLevel1_1/**.parquet’,hive_partitioning = true, union_by_name=true)
where ReportId=7031

Analyze plan

How do I know for sure if the query is not doing a full scan of all the folders ?

Appreciate your help in advance.

Thanks

EXPLAIN ANALYZE select *
from read_parquet(‘abfss://.dfs.core.windows.net//SkyCAST/FlightReportLevel1_1/**.parquet’,hive_partitioning = true, union_by_name=true)
where ReportId=7031

The filter stage is not appearing in the plan produced.