Python Code Optimization Tips For Developers

Optimizing Slow Code First 

When you run your Python program, the source code .py is compiled using CPython into bytecode (.pyc file) saved in a folder (_pycache_) and at the end interpreted by Python virtual M2M code (machine to machine). Since Python uses an interpreter to execute the bytecode at runtime which makes the process a bit slower.    

To make your execution faster on CPython, get that include two different interpreters and the feature of its predecessor PyPy3.6-beta and Python 2.7. Compared to CPython it is 7.6X faster which is critical for coding. 

Programs in CPython takes a lot of space, but PyPy takes up less space in programs. Have issues in Python coding?

Spotting Performing Issues in Python?

Tools such as PyCallGraph, cProfile, and gProf2dot can be useful in faster and more efficient profiling of Python codes:

cProfile is to identify how long the various parts of Python codes are executed.

PyCallGraph creates visual representations of graphs to represent the relationship between Python codes.

From pycallgraph import PyCallGraph

From pycallgraph.output import GraphvizOutput

class Juice:

    def drink(self):
        pass


class Person:

    def __init__(self):
        self.no_juices()

    def no_juices(self):
        self.juices = []

    def add_juice(self, juice):
        self.juices.append(juice)

    def drink_juices(self):
        [juice.drink() for juice in self.juices]
        self.no_juices()


def main():
    graphviz = GraphvizOutput()
    graphviz.output_file = 'basic.png'

    with PyCallGraph(output=graphviz):
        person = Person()
        for a in xrange(10):
            person.add_juice(Banana())
        person.drink_juices()


if __name__ == '__main__':
    main()

Concatenation of Strings

Python strings are generally immutable and subsequently perform quite slow. There are several ways to optimize Python strings in different situations. For example, the + (plus) operator is the ideal option for the few Python string objects.

Iterator such as a List has multiple Strings, the best way to concatenate them is by utilizing the

.join() method

Let’s see an example of how the .join() and the += operator works:

import timeit

# create a list of 1000 words
list_of_words = ["foo "] * 1000

def using_join(list_of_words):  
    return "".join(list_of_words)

def using_concat_operator(list_of_words):  
    final_string = ""
    for i in list_of_words:
        final_string += i
    return final_string

print("Using join() takes {} s".format(timeit.timeit('using_join(list_of_words)', 'from __main__ import using_join, list_of_words')))

print("Using += takes {} s".format(timeit.timeit('using_concat_operator(list_of_words)', 'from __main__ import using_concat_operator, list_of_words'))) 

Output

$ python join-vs-concat.py 
Using join() takes 14.0949640274 s  
Using += takes 79.5631570816 s  
$ 
$ python join-vs-concat.py 
Using join() takes 13.3542580605 s  
Using += takes 76.3233859539 s  

List Comprehension vs Loops in Python Coding

For example, if you want to get the list of the cube root of all the odd numbers in a defined range using the

for loop

new_list = []  
for n in range(1, 9):  
    if n % 3 == 0:
        new_list.append(n**3)

new_list = [ n**3 for n in range(1,9) if n%3 == 0]  

So, it’s clear that list comprehension is more precise, shorter, and notably faster in execution time than

for loops

Optimizing Memory Footprint in Python

Dealing with loops in Python coding, you will need to create a complete list of integers for help in executing the for-loops. Here, in this case, functions such as

range

xrange

xrange

xrange

If you need to generate a large number of integers, then

xrange

xrange

Let’s see the difference between these two functions in terms of memory consumption -

xrange

range

$ python
Python 2.7.10 (default, Oct 23 2015, 19:19:21)  
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.  
>>> import sys
>>> 
>>> r = range(1000000)
>>> x = xrange(1000000)
>>> 
>>> print(sys.getsizeof(r))
8000072  
>>> 
>>> print(sys.getsizeof(x))
40  
>>> 
>>> print(type(r))
<type 'list'>  
>>> print(type(x))
<type 'xrange'>

Hence, it’s clear that

xrange

xrange

Summary

Hope these optimization tips enables robust and reliable execution of Python codes at multiple levels of granularity from profiling to data structures to string concatenation and memory optimization with

xrange

function