The following merge clause supports updates, and additional rows. However, if a row is deleted it will still show the row that was deleted. Can someone help modify the Merge clause such that rows that have been deleted won’t appear.
try:
(deltadf.alias("t")
.merge(
partdf.alias("s"),
"s.primary_key_hash = t.primary_key_hash")
.whenMatchedUpdateAll("s.change_key_hash <> t.change_key_hash")
.whenNotMatchedInsertAll().
execute()
)