I am trying to determine how I can generate and then loop through a large number of combinations of lists of lists to see which combination is “best” following a certain crieteria.

I have 6 list of lists where each subliist is some variable length but the same length across that specific list. An example of what these lists look like:

A = [[1,2,3],[4,5,6],[7,8,9]]
B = [[11,12],[13,14],[15,16],[7,18]]
C = [[10],[20],[30],[40],[50],[60],[70]]

and so on.

I can use itertools.product() and itertools.chain() to create a master list

D = [1,2,3,11,12,10],[1,2,3,11,12,20],...[7,8,9,7,18,70]]

I then would loop through D and use the values from each sublist as indices to call values from some master array and perform a calculation with the goal of determining which sublist in D gives the highest value.

Edit: didnt think was relevant but the calculation is I take every combination from the sublist in D that I am looking at and use those 2 values as the indices for the master 2×2 array which pulls a number that it then averages across all combinations for a given sublist.

The issue I am facing is some of my initial lists have lengths on the order of 10k and creating all possible combinations with these ends up being very computationally demanding.

Is there a better way to either approach the problem or to loop through this number of combinations?