I am comparing the title in movies.csv, which matches the title element in the json files, and then appending the filename to list.txt in the sink. Assume “Big Hero” and “Moana” are match in file1.json and file5.json. However, ADF returns both entire file1.json and file5.json to the sink.

I only want to see the filename “file1.json” and “file5.json” in list.txt.

title.csv:

• The Lion King
• Big Hero
• Moana

the final result in list.txt (sink):

• file1.json

• file5.json

  ADF dataflow setup:

  source option>column to store file name>list

  derivedColumn1>Columns>Title and expression lower(trim(record.metadata.Title))

  lookup1>primary stream> derivedColumn1

  lookup1>lookup stream> sourceCSV

  filter1>filter on> isNull(sourceCSV@title) == false()

  sink>column data> list