I have different python classes and all of them having a method called push. What I am trying to is
- Create multiple class objects one from each class
- Call push method
- All these methods should execute in parallel
- Log messages coming from push method as and when its complete
- Wait for all the methods to complete and exit scrip
I have tried multiple ways using
def push_wrapper(obj):
obj.push()
class_objects=[class1_obj1,class2_obj2,class3_obj3]
with Pool(processes=max_thread_count) as pool:
pool.map(push_wrapper, class_objects)
pool.close()
pool.join()
With this getting getting this error raise TypeError(f"cannot pickle {self.__class__.__name__!r} object")
There are some other approaches like using pool.apply_async
but they are not waiting for all the methods to complete exiting immediately . When i add job.wait()
along with pool.apply_async
its waiting for all threads to complete but I want to print result of the thread as and when its complete
Let me know what is the best approach to address this use case