Quick Python: Concurrent Futures
Another way to perform concurrency in python is to use the
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor def add(x, y): return x + y with ThreadPoolExecutor(max_workers=4) as executor: future = executor.submit(add, 1, 2) result = future.result(timeout=30) # unit: seconds
max_workers=None then it will default to the number of processors on the machine multiplied by 5.
timeout=None then there is no time limit applied.
You can also apply a function to a list or iterables
def double(x): return 2 * x with ThreadPoolExecutor() as executor: future = executor.map(function_handle, [1, 2, 3]) result = future.result()
Instead of threads, it is also possible to spawn processes to side-step the global interpreter lock. The documentation warns that only picklable objects can be executed and returned though.
def add(x, y): return x + y with ProcessPoolExecutor() as executor: future = executor.submit(add, 1, 2) result = future.result()