we have pipelines that are continuously writing to iceberg. In the snapshot table we can see summary for each snapshot id. The summary shows number of added/deleted /total records for each snapshot id . Is there anyway to read only newly added records in the latest snapshot.
i tried using method. But looks like it only support append mode
spark.read
.format("iceberg")
.option("start-snapshot-id", "10963874102873")
.option("end-snapshot-id", "63874143573109")
.load("path/to/table")