I’m trying to use a weighted loss function to handle class imbalance in my data. My problem is a multi-class and multi-output problem. For example (my data has five output/target columns (output_1, output_2, output_3) and I have three classes (class_0, class_1, and class_2) in each target column. I am currently using pytorch’s cross entropy loss function https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html and I see that it has a weight parameter but my understanding is that this the same weight would be applied uniformly to each output/target, but I want to apply separate weights for each class in each output/target.
For concreteness, I could have data that looks like this
| A | B | C | D | E | OUTPUT_1 | OUTPUT_2 | OUTPUT_3 |
|---|---|---|---|---|---|---|---|
| 5.65 | 3.56 | 0.94 | 9.23 | 6.43 | 0 | 2 | 1 |
| 7.43 | 3.95 | 1.24 | 7.22 | 2.66 | 0 | 0 | 0 |
| 9.31 | 2.42 | 2.91 | 2.64 | 6.28 | 2 | 0 | 2 |
| 8.19 | 5.12 | 1.32 | 3.12 | 8.41 | 0 | 2 | 0 |
| 9.35 | 1.92 | 3.12 | 4.13 | 3.14 | 0 | 1 | 1 |
| 8.43 | 9.72 | 7.23 | 8.29 | 9.18 | 1 | 0 | 2 |
| 4.32 | 2.12 | 3.84 | 9.42 | 8.19 | 0 | 1 | 0 |
| 3.92 | 3.91 | 2.90 | 8.19 | 8.41 | 2 | 0 | 2 |
| 7.89 | 1.92 | 4.12 | 8.19 | 7.28 | 0 | 1 | 2 |
| 5.21 | 2.42 | 3.10 | 0.31 | 1.31 | 2 | 0 | 0 |
whereby,
the proportion in output 1 is : 0 = 0.6, 1 = 0.1, 2 = 0.3
the proportion in output 2 is : 0 = 0.4, 1 = 0.3, 2 = 0.3
the proportion in output 3 is : 0 = 0.4, 1 = 0.2, 2 = 0.4
I want to apply the class weight based on the distribution of classes in each output column such that it renormalizes (or rebalances? not sure what the terminology to use here is) class 1 to 0.15 and class 0 and class 2 to 0.425 each (so for output_1 the weights would be [0.425/0.6, 0.15/0.1, 0.425/0.3], for output 2 it’ll be [0.425/0.4, 0.15/0.3, 0.425/0.3] etc). Rather, what I understand the weight parameter in pytorch’s crossentropy loss function is currently doing, is it’ll apply a single class weight to each output column. I’m wondering if i’m missing something and there’s a way to do this using pytorch’s crossentropyloss function?