The question is complete in the title. For the context I am training an alignment model using a CTC loss. In this case the CTC loss requires the as target value the sequence of labels (without label duration). Additionally I’d like to use a metric that evaluates the probability distribution of the model using the sequence of labels including repetitions (label durations). So the model has one output but tow different target values.

I’ve seen this question and answer but in the present case the two target tensors have different dimensions. It appears quite unfortunate to estblish a complex aggrgation and splitting methdo, here noting espeically that the duration extened label sequence should not be used by the loss funtion used for training.

For the moment I imagine overwriting train_step and test_step, but would prefer if there would be a simpler approach.