When I use XGBoost for regression prediction, changing the order of features greatly affects the accuracy of the model, and the importance of features only shows that the first feature is very high (for example, if I place a in the first one, a will be high, and if I place b in the first one, b will be high), while the rest will be very low. Moreover, changing the order of feature indicators (such as changing x1 to x2, x2 to x1) can have a significant impact on the accuracy of the model. I would like to ask why and how to solve this problem.
I standardized the data and randomly removed some features, but the problem still persists
user26020431 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.