I’ve setup a more basic version of a test after running into a problem trying to use worker threads to run queries in parallel with different data sources. This works the first time, but after I have a main thread task, it stops working the next time it tries to create workers again. It will start the workers but then the program exits trying to add them into the workerQueue array.

Basic code structure is something like this

while(true) {
   task = task[i]
   if(task?.wait === false) { 
     const worker = new Worker('worker.js',{workerData: task}) workerQueue.push(worker);
   } else { 
     const results = await Promise.allSettled(setup listeners, resolve or reject based on exit/messages/etc.)
     workerQueue.forEach(... handle terminating and closing listeners)      
     workerQueue.length = 0; /*
      (or workerQueue = [] .. tried a few things to see if it made a difference, including adding statuses to avoid clearing the queue altogether, but none of these corrected the behavior)*/
     const errors = results.filter...
     if(errors.length > 0) throw new Error('blah')
     if(!task) break; // break loop, no more tasks 
     await runTask(task); // run a main thread task      
   }
   i++ //continue the loop either creating new worker threads if wait = false, or pausing if we hit a task that doesn't have a wait to run a task on main thread

If this seems crappy I’m still learning, but this seemed to work in principal just fine, until I started processing task arrays that switched from needing workers to not needing, then to needing again. Once I hit that this breaks…

Sometimes, I get a stack trace from node (only when I upgraded to current release did I get this). I’m not sure if it’s related, but… my guess is it probably is.

It provides some hex type data that I can’t copy and paste unfortunately, but the meaning of it isn’t clear and nothing comes up on a google search for it. The initial line just says it’s a fatal error in line 0 and the check failed node->IsInUse()