Fairly new to Synapse/ADF and currently hitting an issue with what should be an obvious way of doing things…

Scenario: (I’ve intentionally separated it into multiple pipelines to build it up slowly)

Pipeline 1: Execute a web process, set variable to the output.job_id of said process
enter image description here

Pipeline 2: Grab output.job_id variable as a param from pipeline 1 and run another web activity to get output.status.

enter image description here

So far all this works fine – now the tricky part.

Status will be either Queued, In Progress, Complete or Error. – I put this in a switch

If Queued (almost always will be for the first 30-60 seconds), wait 15 seconds, add to variable_timer and check status again.
Keep doing this loop until status changes OR variable_timer hits 5400 seconds.

This also seems to work as expected in an Until iteration and is in Pipeline 3.
enter image description here

In the current toolset, once the Until loop is essentially complete, it goes back to pipeline 2, gets the status, and most likely this time, it will be In Progress so it should then go down a similar route to Queued.

Once In Progress is complete, it goes back to Pipeline 2 and most likely it will be Complete in which case end or error.

• The problem
  It seems I can’t go from the Queued pipeline 3 BACK to Pipeline 2. No errors but hitting Debug does nothing.

I then thought I could simplify it and bring the check status with the loop so:
Web activity>Switch>Until Pipeline for Queued>once out of loop, go back to web activity
and this time would most likely be Web activity>Switch>Until Pipeline for In Progress>, and so on.

This is how we currently do it in another toolset….but no, it seems going from loop success/completion back to web activity puts it into a cycle which it won’t accept.

How do I achieve this?

Any video/site explaining Switch, Until or IF seems to be very static – ie, look for condition, once satisfied, end.

In my case, if its queued, keep checking status until queued is no more and then check status again which will now be either in progress, error or complete).
If In progress, keep checking status until in progress is no more and then check status again which will now be either error or complete.
If error, add fail step and end job.
If complete, add 1 second wait and end job.

Apologies appreciate this is very wordy !

Thanks in advance