I want to use the foreach
–do
construct to iterate over a series of experiments.
From the docs, I understand the standard approach is as follows:
dvc.yaml
stages:
prepare_data:
foreach: ${experiments}
do:
cmd: python stages/prepare_data.py "${item}"
deps:
- data/data_source.csv
- stages/prepare_data.py
params:
- prepare_data
outs:
- input_data/${item}.csv
fit_model:
foreach: ${experiments}
do:
... etc
But is it possible to instead loop over each experiment and execute the stages in sequence:
foreach: ${experiments}
do:
stages:
prepare_data:
cmd: python stages/prepare_data.py "${item}"
deps:
- data/data_source.csv
- stages/prepare_data.py
params:
- prepare_data
outs:
- input_data/${item}.csv
fit_model:
... etc
This doesn’t work but I just want to make sure I’m not missing something because it would be nice in my case.
'./dvc.yaml' validation failed: 2 errors.
extra keys not allowed, in foreach
extra keys not allowed, in do, line 3, column 3
2 do:
3 stages: