A thread pool is a group of pre-instantiated, idle threads which stand ready to be given work. These are preferred over instantiating new threads for each task when there is a large number of short tasks to be done rather than a small number of long ones. This prevents having to incur the overhead of creating a thread a large number of times.

Implementation will vary by environment, but in simplified terms, you need the following:

• A way to create threads and hold them in an idle state. This can be accomplished by having each thread wait at a barrier until the pool hands it work. (This could be done with mutexes as well.)
• A container to store the created threads, such as a queue or any other structure that has a way to add a thread to the pool and pull one out.
• A standard interface or abstract class for the threads to use in doing work. This might be an abstract class called Task with an execute() method that does the work and then returns.

When the thread pool is created, it will either instantiate a certain number of threads to make available or create new ones as needed depending on the needs of the implementation.

When the pool is handed a Task, it takes a thread from the container (or waits for one to become available if the container is empty), hands it a Task, and meets the barrier. This causes the idle thread to resume execution, invoking the execute() method of the Task it was given. Once execution is complete, the thread hands itself back to the pool to be put into the container for re-use and then meets its barrier, putting itself to sleep until the cycle repeats.

Thread pool is a collection of managed threads usually organized in a queue, which execute the tasks in the task queue.

Creating a new thread object every time you need something to be executed asynchronously is expensive. In a thread pool you would just add the tasks you wish to be executed asynchronously to the task queue and the thread pool takes care of assigning an available thread, if any, for the corresponding task. As soon as the task is completed, the the now available thread requests another task (assuming there is any left).

Thread pool helps you avoid creating or destroying more threads, than would really be necessary.

I would start by creating a class with a queue of threads and a queue of tasks. Then implement a method which adds a task to the task queue and move on from there. Obviously, you should also make it possible to set the maximum allowed threads in a thread pool.

Real Life Example;

1. Facility: Operating system
2. Sections: Applications
3. People: Threads

You have a facility there 12 people are working. There are 3 sections of this facility. Kitchen, restrooms and security. If you do not use thread pool technique, that’s how it works: All 12 people will be standing in a meeting room, if new customers come by facility and ask for tasks, then you will separate people in groups and send them to do their work, and come back to meeting room. But, before they go to their duty, there is a preparation phase. They need to wear correct uniform, equip certain devices and walk to that section, finish work and come back. So, once every time they finish their job (thread ends), they need to walk back to meeting room, undress uniform, take out equipment and wait for next job. These refer to creating thread context, it’s memory allocation and tracking information by OS. It is too much time consuming for OS to re-organize new thread needs.

If you are using thread pooling, then, in the early morning, you will assign 6 people to kitchen, 2 people to restroom and 4 people to security. So, they will only do their preparation once in a day. Even if there is no customers at the kitchen, those 4 people will be there, idling, for any upcoming tasks. They do not need to go back to meeting room until kitchen closes (app ends). These 4 people are in the Kitchen app pool, and ready to serve quickly. But, you cannot promise they are working all day along, since kitchen may become idle time to time. Same logic applies for restrooms and security as well.

In first scenario, you do not waste any thread for any task, BUT it will take good amount of time to prepare every single thread for each task. In second one, you prepare threads in advance, so you cannot guarantee you will be using all threads for all tasks but, OS mostly makes great optimization on it, so you can safely rely on it.

In multithreaded application, thread pool is a “pool of available threads” that can be used by your application. Usually, e.g. .NET, it is all managed so you just assign tasks and once a thread is free, it is going to perform it.
So to implement a threadpool, I would expect to create a concept where tasks are automatically taken by free threads without explicit thread creation for each task.