Vectorization in Python¶
Vectorizing code is a technique that will typically enable you to create faster and more readable code. Vectorization is the process of performing computation on a set of values at once instead of explicitly looping through individual elements one at a time. The difference can be readily seen in a simple example. Lets say we want to add together each element of two arrays, \(a = [a_0, a_1, a_2]\) and \(b = [b_0, b_1, b_2]\) such that we create a new array \(c = [a_0 + b_0, a_1 + b_1, a_2 + b_2]\). Let’s show how this is done in a nonvectorized way (using loops) and through vectorization.
import numpy as np # create our data a = np.array([1, 2, 3]) b = np.array([2, 3, 4])
Let’s first complete this using a nonvectorized approach that uses a for loop…
# nonvectorized approach c =  # Start with an empty array that we will populate in the loop for i in range(len(a)): c.append(a[i] + b[i]) print('Nonvectorized approach ->', c)
Nonvectorized approach -> [3, 5, 7]
Now, let’s try a vectorized approach…
# vectorized approach c = a + b print('Vectorized approach ->', c)
Vectorized approach -> [3 5 7]
One thing is immediately clear - the notation of the vectorized approach is far clearer than the nonvectorized approach. It says exactly what is happening - an element-wise addition of vectors. This approach is even clearer with 2D matrices since 2D matrices would require nested loops. In data science we are often working with vectors or matrices of data and need to perform element-wise operations on them as we discussed before, so vectorized notation is often preferable for clarity of the code.
Performance benefits of vectorization¶
Perhaps the largest benefit is not the code clarity, but the performance improvement that vectorization provides: increase in computational efficiency/speed. Let’s explore this aspect of vectorization through a simple example. Lets say we have a large array of numbers and we want to double each of them.
We can start by using a nonvectorized approach which loops through each element in the array, doubling it along the way. Let’s create a function that does just that:
def double_nonvectorized(array): doubled = array.copy() for i in range(len(array)): doubled[i] = array[i] * 2 return doubled
Next, let’s create the equivalent function, but vectorize it:
def double_vectorized(array): return array * 2
Again, note how comparatively simple this version is. Now let’s verify they produce the same, correct output:
array = np.array([1,2,3,4]) print('Nonvectorized = ',double_nonvectorized(array)) print('Vectorized = ',double_vectorized(array))
Nonvectorized = [2 4 6 8] Vectorized = [2 4 6 8]
Now lets evaluate the speed difference between the two. To do that, we need to time our code. There are many strategies for doing so, but let’s use Python’s built in timing package
time. There’s often some variability in how long it takes to run depending on other processes that are going on, so lets run the code 5 times each and take the average. We’ll get to practice many of the tools we’ve been learning in the course so far.
time package has a function
time that returns the current time in seconds; so taking the difference between
time.time() between two points in time provides the number of seconds that have elapsed.
import time def timer(function,argument,num_runs): total_time = 0 # Rerun the code num_runs times for i in range(num_runs): t0 = time.time() # Capture the initial time function(argument) # Run the function we're timing t1 = time.time() # Capture the final time total_time += t1-t0 return total_time / num_runs # Return the average across the runs
All that’s left to be done is to create a very large array that we want to double and time how long it takes
big_array = np.arange(1000000) num_runs = 5 time_nonvectorized = timer(double_nonvectorized,big_array,num_runs) time_vectorized = timer(double_vectorized,big_array,num_runs) print('Nonvectorized code took ',time_nonvectorized, 's') print('Vectorized code took ',time_vectorized, 's') print('Vectorized code was ', time_nonvectorized/time_vectorized ,' times faster')
Nonvectorized code took 0.26927862167358396 s Vectorized code took 0.0003796100616455078 s Vectorized code was 709.3558598166059 times faster
That’s more than a 700x speedup! For the exact same code!!!! Note: if you rerun this code, your output will likely vary depending on the computer you’re using and the activity on that computer, but you will consistently see a speedup.
OK, so why does this happen? The answer is twofold.
First, in the vectorized function, Python (or rather, the
numpy code written in C that gets called) is designed to understand that it’s about to do something to every entry of an array, so it remembers where the array is located, and so only has to look up where to find the array once.
In addition, arrays are typed, meaning that Python also knows that every entry of the array it’s modifying is an integer. As a result, it doesn’t have to check the type of every entry in the array when the operation is vectorized, it checks once and knows that it’s working with an array of integers.
Caution: optimize when needed¶
While starting with efficient coding practices is the best approach, there are times when the solutions we can think of for a problem are effective, but inefficient. If you don’t think of the most optimized approach to a problem at the start, that’s okay. If you follow the practices of creating modular code and never duplicating code, you can always refactor your code later and improve its efficiency through optimization. Don’t let the perfect be the enemy of the good. Get a working solution created for your problem or analysis and if its sufficiently modular, you can optimize the performance as you identify opportunities to do so without delaying your minimum viable product.
Vectorization syntax parallels much of the math of data science¶
Often in data science we use linear algebra to perform matrix operations. Linear regression, principle components analysis, correlation analyses - all involve matrix operations. Many of these matrix operations can be directly expressed through vectorized operations in much the same way that the math would be expressed. For example we often need to multiply one matrix by another - a common operation for a data scientist. We will explore in the exercises just how much more concise and readable vectorized code can be in such circumstances, and how much of an increase in speed it results in.
Vectorization can drastically increase the speed of execution versus looping over arrays
Vectorization keeps code simpler and more readable so it’s easier to understand and build on later
Much of the math of data science is similar to vectorized implementations, making it easier to translate into vectorized code
While performance may be important for your particular problem, prioritize module implementations that can be optimized later over a delayed deliverable