I’m trying to evaluate regression models created using lightgbm, xgboost, and randomforest, with aic. My approach is to order features in terms of feature importance, fit, predict, and calculate aic, then drop the least important feature, and repeat. The goal is to find the least number of features required to achieve the lowest aic, and avoid overfitting. I have a general function below for calculating aic. I’m wondering if given my end goal it makes sense to use this general function to calculate aic for these three different models, or do I need a specific function for each model? If a different aic function is required for each model can anyone please suggest the functions?

code:

from sklearn.metrics import mean_squared_error
import numpy as np
import pandas as pd

def aic(y_true, y_pred,X_train):

    # calculate mean squared error
    mse = mean_squared_error(y_true,y_pred)
    
    # number of features
    n_features = X_train.shape[1]
    
    n=len(y_true)
    
    aic= 2 * n_features - 2 * n * np.log(mse)
    
    return aic