Prevent Mistakes In Python

This article is a collection of tips to Python programmers of all experience levels to help prevent mistakes in their code.

Catch All Exceptions

Sometimes it is necessary to catch any possible error raised in your program. However, using a traditional catch-all is not a good idea.

The following code snippet is the traditional way of catching all errors in Python. However, this can cause unexpected problems. This method will cause certain exceptions that should be raised independent from a catch-all for example: GeneratorExit, KeyboardInterrupt and SystemExit.

FACT: These exceptions actually all inherit BaseException

1
2
3
4
5

try:
    # Some Code
    pass
except:
    # Catch ALL Errors

A better solution is to specify the Exception class to catch all non-system exceptions:

1
2
3
4
5

try:
    # Some Code
    pass
except Exception:
    # Catch Non-System Errors

This will allow an exception such as KeyboardInterrupt to still be raised, which is important as this exception in particular allows the user to halt the program execution. Handling this in a catch-all could lead to an unexpected program hang, since the program would never be able to quit.

Objects Passed As References

In Python every value is stored as an object and when passing objects around, they are always passed by reference. If not handled correctly you may encounter errors in your code. Take a look at the following example:

1
2
3
4
5
6
7
8

def do_something(v):
    v.append(5)
    return v

my_list = [1, 2, 3, 4]
other_list = do_something(my_list)
print(my_list)
print(other_list)

Before running the code we may expect the second list to be different to the first. However, that is not really what happens. Instead the original list is altered.

1
2

[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5]

To prevent this, instead the copy() method can be called on the list. This will create a copy of the original list ensuring the original will not be altered by the called function.

FACT: This will actually create a shallow copy

Now when running the improved code we will get the result we expect.

1
2
3
4

my_list = [1, 2, 3, 4]
other_list = do_something(my_list.copy())
print(my_list)
print(other_list)

1
2

[1, 2, 3, 4]
[1, 2, 3, 4, 5]

More advanced copying may be required in other situations; in that case Python has a copy module that can be imported.

Using Objects As Defaults

Defining defaults for parameters in functions is a common task. However, careful consideration must be taken, preventing unexpected results. Since objects are passed as references.

Take a look at the following example:

1
2
3
4
5
6

def do_something(v=[]):
    v.append(5)
    return v

print(do_something())
print(do_something())

Before running we may expect two lists containing [5] to be printed. However, that is not what happens. Instead we get:

1
2

[5]
[5, 5]

Why is this? It’s because when the function first runs a reference of the list gets created and used as the default value. So the next time it’s ran; instead of a new list getting created the existing reference gets used.

To prevent this from happening we can instead use a default of None and then create the list in the function body instead, thus preventing the re-use like before. See the below code:

1
2
3
4
5
6
7
8

def do_something(v=None):
    if v is None:
        v = []
    v.append(5)
    return v

print(do_something())
print(do_something())

1
2

[5]
[5]

Closing IO

When dealing with system IO, such as opening a file. It is important to close the IO handle when you are finished, this frees memory and removes any possible locks that were active on the file.

The below code uses a common way of opening a file and closing a file:

1
2
3

fo = open("my_file.txt", "r")
second_line = fo.read().split("\n")[1]
fo.close()

This code has the potential to crash whilst the file is open, causing any locks to remain in place. There are two possible approaches to fix this.

The first way we could fix this; is by using a “try finally” block. As shown in the code below:

1
2
3
4
5

try:
    fo = open("my_file.txt", "r")
    second_line = fo.read().split("\n")[1]
finally:
    fo.close()

This “finally” statement will always run last; even if the program encounters an error during the file being open.

Another approach is by utilising the context manager, which is also the most ideal method to use.

1
2

with open("my_file.txt", "r") as fo
    second_line = fo.read().split("\n")[1]

As you can see we do not even need to specify the close(), since the context manager will do it for us; all with the amazing with block. This also has the added benefit of also still closing the IO handle if an error occurred whilst it is open.

Since the context manager will always ensure the IO handle is closed when it is no longer needed, it is the most ideal way of handling safe IO cleanup in Python programs.

Multi Inheritance

As Python is a Object Orientated Programming language, it features inheritance. However, unlike similar languages it also supports multi-inheritance. This means one class can inherit from not just one; but many classes. Whilst this may have it’s advantages, depending on how it’s used it can also introduce it’s own issues.

Take the following example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

class Base1:
    def do_something(self):
        print("One")

class Base2:
    def do_something(self):
        print("Two")

class Derived(Base1, Base2):
    pass

a = Derived()
a.do_something()

When calling the function do_something() it is not clear which function will get called. If you thought the result would be Two you would not be correct, it’s actually One! That’s because multi-inheritance is applied from right to left.

Now imagine there are many base classes we are inheriting from, some may even come from external libraries not under our control. How do we ever know when our derived class is calling the correct inherited method? We don’t, or at least it’s not clear.

Whilst multi-inheritance is a valid language feature; I would suggest to stay clear of it, since it can cause unexpected results without adding much advantage to your code.

References

• https://docs.python.org/3/library/exceptions.html
• https://docs.python.org/3/reference/datamodel.html
• https://docs.python.org/3/reference/compound_stmts.html#with
• https://docs.python.org/3/reference/datamodel.html#context-managers
• https://docs.python.org/3/tutorial/classes.html