Brandon Rozek

Photo of Brandon Rozek

PhD Student @ RPI studying Automated Reasoning in AI and Linux Enthusiast.

Quick Python: Concurrent Futures

Published on

Updated on

Warning: This post has not been modified for over 2 years. For technical posts, make sure that it is still relevant.

Another way to perform concurrency in python is to use the concurrent.futures module.

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
def add(x, y):
    return x + y
with ThreadPoolExecutor(max_workers=4) as executor:
    future = executor.submit(add, 1, 2)
    result = future.result(timeout=30) # unit: seconds

If max_workers=None then it will default to the number of processors on the machine multiplied by 5.

If timeout=None then there is no time limit applied.

You can also apply a function to a list or iterables

def double(x):
    return 2 * x
with ThreadPoolExecutor() as executor:
    future = executor.map(function_handle, [1, 2, 3])
    result = future.result()

Instead of threads, it is also possible to spawn processes to side-step the global interpreter lock. The documentation warns that only picklable objects can be executed and returned though.

def add(x, y):
    return x + y
with ProcessPoolExecutor() as executor:
    future = executor.submit(add, 1, 2)
    result = future.result()
Reply via Email Buy me a Coffee
Was this useful? Feel free to share: Hacker News Reddit Twitter

Published a response to this? :