Relative Content

Tag Archive for pythonparallel-processingmultiprocessing

Using multiprocessing inside a class

I have a Python class that I use for a series of computations in which I am initialising a number of different pandas.DataFrames stored inside nested dictionaries. The bottleneck in initialising the class is as follows:
I have a regular grid of points spanning the Earth and a set of polygons (representing tectonic plates) that also cover the Earth. Of each point in the grid, I need to find in which plate they are located and assign the ID of that plate to a list.
Because this is relatively slow (i.e. takes 10s of seconds per iteration instead of <1s), I would like to parallelise this section (and only this section!) of the class. I am new to parallelisation, however, and I ran into some issues.

Using multiprocessing inside a class

I have a Python class that I use for a series of computations in which I am initialising a number of different pandas.DataFrames stored inside nested dictionaries. The bottleneck in initialising the class is as follows:
I have a regular grid of points spanning the Earth and a set of polygons (representing tectonic plates) that also cover the Earth. Of each point in the grid, I need to find in which plate they are located and assign the ID of that plate to a list.
Because this is relatively slow (i.e. takes 10s of seconds per iteration instead of <1s), I would like to parallelise this section (and only this section!) of the class. I am new to parallelisation, however, and I ran into some issues.