I’ve made two different functions to calculate all possible combination of elements from n arrays (n not fixed, this is very important to notice) where each array contains a different number of elements (Cartesian product, so in each combo there’s one element from each array). Since these arrays are containing strings, I’ve performed a memory optimization utilizing possible indexes to compute combinations (since saving an integer is much less costly then a string in memory, or at least in most of the cases) and then referenced the combination in this way: combo=[0,1,5,6] => actualCombo=[arrays[0][0],arrays[1][1],arrays[2][5],arrays[3][6]]

This works pretty well until we reach 7/8 degrees of freedom (translated 7 or 8 arrays to be combined). In these cases in fact my PC runs out of memory even if I start node with 12GB of available memory (using the parameter --max-old-space-size)

Is there any solution to this? I’ve tried something like Bull to implement queues and to limit the number of jobs but no improvement. I’ll leave you below here the two functions used with a little explanation for both and the conclusions.

First function:

function cartesian(a, b, ...c) {
    return b ? cartesian(a.flatMap(d => b.map(e => [].concat(d, e))), ...c) : a
}

This works in principle with any type of arrays (of strings, numbers, objects) but as I said I’ve passed to this function an array of possible indexs like the following: let array1=['John','Doe','foo'] => passedVersionOfArray1=[0,1,2]

Second function (actually a group of functions):

function multiplication(lengths) {
    let prod = 1
    lengths.forEach(elem => prod *= elem);
    return prod
}

function firstCombo(length) {
    let combo = []
    for (let i = 0; i < length; i++) combo.push(0)
    return combo
}

function cartesian(previous, lengths) {
    let next = [...previous]
    for (let i = 1; i <= lengths.length; i++) {
        if (previous.at(-i) != lengths.at(-i) - 1) {
            next[next.length - i] = previous.at(-i) + 1
            break
        }
        else next[next.length - i] = 0
    }
    return next
}

const lengths = matrix.flatMap(elem => elem.length)
let previous = firstCombo(matrix.length)
const combos = [previous]
for (let i = 0; i < multiplication(lengths) - 1; i++) {
    const next = cartesian(previous, lengths)
    combos.push(next)
    previous = next
}

These functions as you might notice works only with combos of indexes and it works by incrementing by one the combination of integers in the following way [0,0,0,0] => [0,0,0,1]

Conclusions: at first I thought that the second approach might have been less memory consumptive since the first one is a recursive function. But it didn’t work. Is there any method to work in batches? Is there a method to partially calculate the combinations, work with them, and then resume the calculation so that GC can free memory space before the second batch of combos?