I have a very simple dataset of employee age and years of experience as features and income as label. The ask is to predict the income level using various regressors and I am using 4: Decision Trees (DT), Random Forest (RF), K-Nearest Neighbor (KNN) and Linear Regression (LR). The following table gives the income level as predicted by each of the 4 regressors.
My questions are:
- Why is DT predicting such tight income levels, (almost equal to the actual income of the data set) while other regressors seem to be working fine? Is it because the data is too small for the DT to train / the number of features too low (only 2 features) or is it something else?
- How come DT gives absurd results but RF does not?
- The R^2 and MSE for DT is less than and more than that for LR, respectively, but still, DT seems to be producing more tighter range. How?
Not including any code here as this is more of a contextual question. Can provide code if needed.