I’m trying to tune a stacked learner using the mlr3 package. My original code can be found in this post, where I used the auto_tuner function. In the new version, I removed the prediction threshold tuning po("threshold"), but most importantly, I used the ti function to be able to consider multiple performance metrics:

inner_resampling <- mlr3::rsmp ("cv", folds = 2)

performance_measures <- c(mlr3::msr("classif.sensitivity"), mlr3::msr("classif.specificity"), 
                           mlr3::msr("classif.acc"), mlr3::msr("classif.auc"))

terminator <- mlr3tuning::trm("evals", n_evals = 100)

tuner <- mlr3tuning:::tnr("mbo")

## Create tuning instance
  instance <- mlr3tuning::ti(task = task, ## comme dans mon precedent post
                                           learner = lrn_graph_1, ## comme dans mon precedent post
                                           resampling = inner_resampling,
                                           measures = performance_measures,
                                           terminator = terminator)

## Run the tuning process  
  tuner$optimize(create_tuning_instance)

When I run the tuning process, I get this warning:

INFO  [00:32:16.375] [mlr3] Applying learner 'classif.kknn' on task 'data' (iter 1/3)
INFO  [00:32:16.504] [mlr3] Applying learner 'classif.kknn' on task 'data' (iter 2/3)
INFO  [00:32:16.632] [mlr3] Applying learner 'classif.kknn' on task 'data' (iter 3/3)

WARNING: reaching max number of iterations

WARNING: reaching max number of iterations

This leads me to several questions:

• I have the impression (but I could be wrong) that the warning is caused by the terminator. Also, I was wondering if a possible solution to avoid this warning would be to specify a sample size in the initial design of the “mbo” (for example, n = 1000 in a Latin hypercube sampling or Sobol sequence) and set terminator <- mlr3tuning::trm("none"). If this is a possible solution, how can we see the default parameter values of “tnr(“mbo”)”? I tested tuner$param_set but everything is NULL.

• Is it correct to tune a stacked learner with a level 0 and level 1 (where level 1 calculates a weighted average of the predictions of all learners from level 0) using the ti() function? I saw this very useful example https://mlr-org.com/gallery/pipelines/2020-04-27-tuning-stacking/ but it uses the auto-tuner function. So, I have a doubt.

• If this is possible, I would like to speed up the execution of the tuning process by using parallel computing. I am also new to this and I was wondering if it was possible to parallelize tuning with a stacked learner? I haven’t found an example. If yes, is it enough to add a command like this in my script future::plan("multisession", workers = 4)?

Any help would be greatly appreciated.