# Parallel Computing¶

Before you dive into this, let me just tell you the punchline of this entire page right up front: parallelism is the last tool you want to turn to for speed. It is not a silver bullet, it will generally take you significant time to implement, the speed improvements from parallelism are generally much smaller than what you get from other performance improvement methods (see Understanding Performance and Performance Solutions), and the headaches of parallelizing code are many.

## Why is Paralleism important?¶

Moore’s law kinda is dead! see: single thread performance

Further reading / resources:

https://www.youtube.com/watch?v=zX4ZNfvw1cw

## Limits of Parallelism¶

Most task are at least partially serial, and that limits speed up. If your program is 90% parallelizable, all the cores in the world will only get you a 10x speed-up!

### Amdahl’s Law¶

$\frac{1}{(1-P)+\frac{P}{N}}$

## Types of Parallelism¶

### Multi-threading¶

DANGER! HERE BE DRAGONS! RACE CONDITIONS!

## How do I paralleize?¶

joblib

https://sebastianraschka.com/Articles/2014_multiprocessing.html

https://wiki.python.org/moin/ParallelProcessing

dask

[ ]: