I am trying to optimise the premium charged to customers for profit, whilst targeting an overall conversion/retention rate. However, the optimisation process is currently failing, providing this output when I run ‘result’:

       fun: -298.2633664479155
     jac: array([-0.82517242, -0.60799408, -0.44937897, -0.82517242, -0.72691727,
       -0.84170532, -0.74860764, -0.18024063, -0.74860764, -0.96907043])
 message: 'Singular matrix C in LSQ subproblem'
    nfev: 11
     nit: 1
    njev: 1
  status: 6
 success: False
       x: array([338.39, 409.57, 453.83, 337.21, 377.76, 330.7 , 366.62, 559.69,
       366.95, 156.04])

The following is the script I used.

The jacobian, highlights that for increases in premium retention drops off, which is expected, and in varies by policy which is expected.

Data

cost_prices, initial_premiums

(array([293.2 , 371.08, 479.59, 328.2 , 295.23, 273.17, 303.83, 368.09,
        229.72, 208.64]),
 array([338.39, 409.57, 453.83, 337.21, 377.76, 330.7 , 366.62, 559.69,
        366.95, 156.04]))

Objective Function

def objective(premiums, cost_prices):
    propensities = propensity_to_renew(premiums)
    profit = np.sum((premiums - cost_prices) * propensities)
    return -profit  # Negate since we are minimizing in scipy

Propensity Model

This uses the logit function, where the linear predictors range from 2.5 to -3, for a change in premium range of 0.7 to 2 (not rate increases are capped at 1.3 due to bound below).

Here is the propensity output for the initial premiums used:

    [0.8251735557295322,
 0.607996891575633,
 0.4493802625513401,
 0.8251735557295322,
 0.7269196749118002,
 0.841705770676268,
 0.7486076220729461,
 0.18023960377963455,
 0.7486076220729461,
 0.969072689633726]

For this model I think expected profit versus premium looks very reasonable (see below). So given the constraints I would have expected to find an optimum (quadratic graph below). Note the peak below (close to 0.9) is very close to my input of 1 (this is basically a premium change value).

Target Overall retention Constraint

def retention_constraint(premiums):
    propensities = propensity_to_renew(premiums)
    target_retention = 0.6  # Example target retention rate
    return np.mean(propensities) - target_retention

Define constraints for the optimizer

constraints = [{'type': 'eq', 'fun': retention_constraint}]

Bounds

bounds = [(0.7*premium,1.3*premium) for premium in initial_premiums]

Perform Optimisation

result = opt.minimize(objective, initial_premiums, args=(cost_prices), bounds=bounds, constraints=constraints)

optimal_premiums = result.x
optimal_profit = -result.fun

print("Optimal Premiums:", optimal_premiums)
print("Optimal Profit:", optimal_profit)

Output (Note Optimised Premium is the same as the input initial Premium)

Optimal Premiums: [338.39 409.57 453.83 337.21 377.76 330.7  366.62 559.69 366.95 156.04]
Optimal Profit: 298.2633664479155

Can someone please give advice? Happy to provide additional detail if needed.

Thanks