I have a design question about custom memory allocator / thread pool, or any structure that needs to be accessed globally.

Take the memory allocator as an example. Currently my design is more or less the following pseudo code:

namespace MyDumbMemAlloc {

struct SomeFancyHeapAllocatedMemoryPool {
  SomeFancyHeapAllocatedMemoryPool (arguments...) { ... }
  ~SomeFancyHeapAllocatedMemoryPool() { ... }
  template <typename T> T* dispatch (size_t size) { ... }
  template <typename T> void recall (T* p) { ... }
};
SomeFancyHeapAllocatedMemoryPool* SFHAMP = nullptr;

struct Activate {
  Activate (arguments...) { 
    SFHAMP = new SomeFancyHeapAllocatedMemoryPool(arguments...); 
  }
  ~Activate() { delete SFHAMP; SFHAMP = nullptr; }
};

struct Alloc {
  template <typename T> T* allocate (size_t size) { 
    return SFHAMP->dispatch<T>(size); 
  }
  template <typename T> void deallocate (T* p, size_t size) { 
    SFHAMP->recall<T>(p, size); 
  }
}
}

void f1(int size) {
  std::vector <int, MyDumbMemAlloc::Alloc> v(size);
  // Do many things with v.
}

void f2(int size) {
  std::vector <int, MyDumbMemAlloc::Alloc> v(size);
  // Do many things with v.
}

void f3(int size) {
  std::vector <int, MyDumbMemAlloc::Alloc> v(size);
  // Do many things with v.
}

int main() {
  auto ac = MyDumbMemAlloc::Activate(arguments...); // I don't want 
  // the existence of this line.
  
  f1(10000);
  f2(20000);
  f3(30000);
  // Object `ac` is auto destructed before going out of scope, and so is `SFHADS`.
} 

The above works perfectly fine, but I don’t recall any established allocator library, e.g. the Boost Pool allocator https://www.boost.org/doc/libs/1_47_0/libs/pool/doc/index.html, needs one to do some “activation”.

An obvious solution is to initialize the memory pool inside allocate() when the function is called the first time, and to destruct the pool inside deallocate() if there is no outstanding buffers:

namespace MyDumbMemAlloc {

struct SomeFancyHeapAllocatedMemoryPool {
  SomeFancyHeapAllocatedMemoryPool (arguments...) { ... }
  ~SomeFancyHeapAllocatedMemoryPool() { ... }
  template <typename T> T* dispatch (size_t size) { ... }
  template <typename T> void recall (T* p) { ... }
};
SomeFancyHeapAllocatedMemoryPool* SFHAMP = nullptr;


struct Alloc {
  template <typename T> T* allocate (size_t size) { 
    if ( SFHAMP == nullptr ) 
      SFHAMP = new SomeFancyHeapAllocatedMemoryPool(arguments...);
    return SFHAMP->dispatch<T>(size); 
  } 
  template <typename T> void deallocate (T* p, size_t size) { 
    SFHAMP->recall<T>(p, size); 
    if ( SFHAMP->allBuffersHaveBeenRecalled() ) {
      delete SFHAMP; 
      SFHAMP = nullptr;
    }
  } 
}
}

void f1(int size) {
  std::vector <int, MyDumbMemAlloc::Alloc> v(size);
  // Do many things with v.
} 

void f2(int size) {
  std::vector <int, MyDumbMemAlloc::Alloc> v(size);
  // Do many things with v.
} 

void f3(int size) {
  std::vector <int, MyDumbMemAlloc::Alloc> v(size);
  // Do many things with v.
} 

int main() {
  f1(10000); // Problem: memory pool is constructed and then destructed.
  f2(20000); // Problem: memory pool is constructed again and then destructed.
  f3(30000); // Problem: memory pool is constructed again and then destructed.
}  

My guess is that those established allocator libraries take the second approach to avoid some “activation” mechanism, but how do they skip the frequent pool construction / destruction cycles when the program uses only one buffer (as shown in the example above) ? The pool construction and destruction may be expensive. Is it even avoidable at all?

The same question applies to thread pool. Consider the following code:

int main() {
  auto po = std::execution::par_unseq;
  
  // Parallel STL uses TBB as the backend. I think the TBB thread pool
  //   is created during the first time for_each() being called.
  std::for_each(po, begin, end, unaryFun);  
  // Worker threads are put to sleep.
  
  // ... Other things are done by the main thread.
  
  std::for_each(po, begin, end, unaryFun); // Threads wake up and run again.
  
  // ... Other things are done by the main thread.
  
} // Right before going out of scope, how does the thread pool know to destruct
//     itself ? 

TBB once had a memory leak problem C++: memory leak in tbb. Whenever I compiled my program with sanitizers, I had to set export ASAN_OPTIONS=alloc_dealloc_mismatch=0 to avoid crashing. I always thought the leaky problem is exactly due to the thread pool not being deleted going out of scope. There seemed to be no remedy without some “activation” mechanism.

However, the newer version oneTBB no longer has this problem. How did they solve it? I don’t believe the answer is as dumb as that the thread pool is constructed and destructed inside every for_each() call. Was any compiler magic involved ?

Many thanks!