I am using delta live tables in medallion architecture.I have columns id, timestamp, operation, deleted_flag. I have operations “Insert” and “delete”. When the operation is “insert” the deleted_flag will be null and for “delete” it will be yes. This is correctly populating in bronze table. but for silver table i am taking bronze table as source and performing apply_changes. But after that for series of insert and delete operations for same id , if last operation is “insert” then the deleted flag should be null. But i am getting as yes. Can someone please help me. I don’t want to delete old records in silver table
I want the deleted_flag to be null for last insert if there are a series of inserts and deleted for same id. and if last operation is delete then it should be “yes”
Tanmayi Annareddy US is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1