I am trying to build an application where I need to read file which will be placed by another application occasionally ,once or twice a day. I have to create a spark ETL to listen to this folder. once the file received, read the file , do some transformations and load the data to DB.
How do I implement the listening part of this project . Should we use Spark streaming for this?