Concurrency in Python - concepts, frameworks and best practices

Loading

Upvotes: DownVotes:
Age: a year     Page Views: 447
Votes / View: 7    Wilson Score: 0.44




Have you run in situations where concurrent execution could speed upyour Python code? Are you using a GUI toolkit?This talk gives you the background to use concurrency in your codewithout shooting yourself in the foot - which is quite easy if you don'tunderstand how concurrent execution differs from linear execution!The presentation starts with explaining some concepts like concurrency,parallelism, resources, atomic operations, race conditions anddeadlocks.Then we discuss the commonly-used approaches to concurrency:multithreading with the ``threading`` module, multiprocessing with the``multiprocessing`` module, and event loops (which include the``asyncio`` framework). Each of these approaches has its typical usecases, which are explained.You can implement concurrency on a number of abstraction levels. Thelowest level consists of primitives like locks, events, semaphores andso on. A higher abstraction level is using queues, typically with workerthreads or processes. Even higher abstraction levels are active objects(hiding primitives or queues behind an API; this includes 'actors' ifyou heard of them), the thread and process pools in``concurrent.futures`` and the ``asyncio`` framework. Finally, you can'outsource' concurrency by leaving it to a message broker, which is adistinct process that receives and distributes messages.The talk closes with some tips and best practices, mainly: