Month: January 2017

Summary

In my opinion, the best way in Python to safely coerce things to integers requires use of an (almost) “naked” except, which is a construct I rarely want to use. Read on to see how I arrived at this conclusion, or you can jump ahead to what I think is the best solution.

The Problem

Suppose you had to write a Python function to convert to integer string values representing temperatures, like this list —

['22', '24', '24', '24', '23', '27']

The strings come from a file that a human has typed in, so even though most of the values are good, a few will have errors ('25C') that int() will reject.

Let’s Explore Some Solutions

You might write a function like this —

def force_to_int(value):
    """Given a value, returns the value as an int if possible.
    Otherwise returns None.
    """
    try:
        return int(value)
    except ValueError:
        return None

Here’s that function in action at the Python prompt —

>>> print(force_to_int('42'))
42
>>> print(force_to_int('oops'))
None

That works! However, it’s not as robust as it could be.

Suppose this function gets input that’s even more unexpected, like None —

>>> print(force_to_int(None))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 6, in force_to_int
TypeError: int() argument must be a string or a number, not 'NoneType'

Hmmm, let’s write a better version that catches TypeError in addition to ValueError —

def force_to_int(value):
    """Given a value, returns the value as an int if possible.
    Otherwise returns None.
    """
    try:
        return int(value)
    except (ValueError, TypeError):
        return None

Let’s give that a try at the Python prompt —

>>> print(force_to_int(None))
None

Aha! Now we’re getting somewhere. Let’s try some other types —

>>> import datetime
>>> print(force_to_int(datetime.datetime.now()))
None
>>> print(force_to_int({}))
None
>>> print(force_to_int(complex(3,3)))
None
>>> print(force_to_int(ValueError))
None

OK, looks good! Time to pop open a cold one and…

Wait, I can still feed input to this function that will break it. Watch this —

>>> class Unintable():
 ...    def __int__(self):
 ...        raise ArithmeticError
 ...
 >>>
 >>> trouble = Unintable()
 >>> print(force_to_int(trouble))
 Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "<stdin>", line 6, in force_to_int
   File "<stdin>", line 3, in __int__
 ArithmeticError

Dang!

While the class Unintable is contrived, it reminds us that classes control their own conversion to int, and can raise any error they please, even a custom error. A scenario that’s more realistic than the Unintable class might be a class that wraps an industrial sensor. Calling int() on an instance normally returns a value representing pressure or temperature. However, it might reasonably raise a SensorNotReadyError.

And Finally, the Naked Except

Since any exception is possible when calling int(), our code has to accomodate that. That requires the ugly “naked” except. A “naked” except is an except statement that doesn’t specify which exceptions it catches, so it catches all of them, even SyntaxError. They give bugs a place to hide, and I don’t like them. Here, I think it’s the only choice —

def force_to_int(value):
    """Given a value, returns the value as an int if possible.
    Otherwise returns None.
    """
    try:
        return int(value)
    except:
        return None

At the Python prompt —

>>> print(int_or_else(trouble))
 None

Now the bones of the function are complete.

Complete, Except For One Exception

Graham Dumpleton‘s comment below pointed out that there’s a difference between what I call a ‘naked’ except —

except:

And this —

except Exception:

The former traps even SystemExit which you don’t want to trap without good reason. From the Python documentation for SystemExit —

It inherits from BaseException instead of Exception so that it is not accidentally caught by code that catches Exception. This allows the exception to properly propagate up and cause the interpreter to exit.

The difference between these two is only a side note here, but I wanted to point it out because (a) it was educational for me and (b) it explains why I’ve updated this post to hedge on what I was originally calling a ‘naked’ except.

The Final Version

We can make this a bit nicer by allowing the caller to control the non-int return value, giving the “naked” except a fig leaf, and changing the function name —

def int_or_else(value, else_value=None):
    """Given a value, returns the value as an int if possible. 
    If not, returns else_value which defaults to None.
    """
    try:
        return int(value)
    # I don't like catch-all excepts, but since objects can raise arbitrary
    # exceptions when executing __int__(), then any exception is
    # possible here, even if only TypeError and ValueError are 
    # really likely.
    except Exception:
        return else_value

At the Python prompt —

>>> print(int_or_else(trouble))
None
>>> print(int_or_else(trouble, 'spaghetti'))
spaghetti

So there you have it. I’m happy with this function. It feels bulletproof. It contains an (almost) naked except, but that only covers one simple line of code that’s unlikely to hide anything nasty.

You might also want to read a post I made about the exception handling choices in this post.

I release this code into the public domain, and I’ll even throw in the valuable Unintable class for free!

The image in this post is public domain and comes to us courtesy of Wikimedia Commons.