How Best to Coerce Python Objects to Integers?

Summary

In my opinion, the best way in Python to safely coerce things to integers requires use of an (almost) “naked” except, which is a construct I rarely want to use. Read on to see how I arrived at this conclusion, or you can jump ahead to what I think is the best solution.

The Problem

Suppose you had to write a Python function to convert to integer string values representing temperatures, like this list —

['22', '24', '24', '24', '23', '27']

The strings come from a file that a human has typed in, so even though most of the values are good, a few will have errors ('25C') that int() will reject.

Let’s Explore Some Solutions

You might write a function like this —

def force_to_int(value):
    """Given a value, returns the value as an int if possible.
    Otherwise returns None.
    """
    try:
        return int(value)
    except ValueError:
        return None

Here’s that function in action at the Python prompt —

>>> print(force_to_int('42'))
42
>>> print(force_to_int('oops'))
None

That works! However, it’s not as robust as it could be.

Suppose this function gets input that’s even more unexpected, like None

>>> print(force_to_int(None))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 6, in force_to_int
TypeError: int() argument must be a string or a number, not 'NoneType'

Hmmm, let’s write a better version that catches TypeError in addition to ValueError

def force_to_int(value):
    """Given a value, returns the value as an int if possible.
    Otherwise returns None.
    """
    try:
        return int(value)
    except (ValueError, TypeError):
        return None

Let’s give that a try at the Python prompt —

>>> print(force_to_int(None))
None

Aha! Now we’re getting somewhere. Let’s try some other types —

>>> import datetime
>>> print(force_to_int(datetime.datetime.now()))
None
>>> print(force_to_int({}))
None
>>> print(force_to_int(complex(3,3)))
None
>>> print(force_to_int(ValueError))
None

OK, looks good! Time to pop open a cold one and…

Wait, I can still feed input to this function that will break it. Watch this —

>>> class Unintable():
 ...    def __int__(self):
 ...        raise ArithmeticError
 ...
 >>>
 >>> trouble = Unintable()
 >>> print(force_to_int(trouble))
 Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "<stdin>", line 6, in force_to_int
   File "<stdin>", line 3, in __int__
 ArithmeticError

Dang!

While the class Unintable is contrived, it reminds us that classes control their own conversion to int, and can raise any error they please, even a custom error. A scenario that’s more realistic than the Unintable class might be a class that wraps an industrial sensor. Calling int() on an instance normally returns a value representing pressure or temperature. However, it might reasonably raise a SensorNotReadyError.

And Finally, the Naked Except

Since any exception is possible when calling int(), our code has to accomodate that. That requires the ugly “naked” except. A “naked” except is an except statement that doesn’t specify which exceptions it catches, so it catches all of them, even SyntaxError. They give bugs a place to hide, and I don’t like them. Here, I think it’s the only choice —

def force_to_int(value):
    """Given a value, returns the value as an int if possible.
    Otherwise returns None.
    """
    try:
        return int(value)
    except:
        return None

At the Python prompt —

>>> print(int_or_else(trouble))
 None

Now the bones of the function are complete.

Complete, Except For One Exception

Graham Dumpleton‘s comment below pointed out that there’s a difference between what I call a ‘naked’ except —

except:

And this —

except Exception:

The former traps even SystemExit which you don’t want to trap without good reason. From the Python documentation for SystemExit —

It inherits from BaseException instead of Exception so that it is not accidentally caught by code that catches Exception. This allows the exception to properly propagate up and cause the interpreter to exit.

The difference between these two is only a side note here, but I wanted to point it out because (a) it was educational for me and (b) it explains why I’ve updated this post to hedge on what I was originally calling a ‘naked’ except.

The Final Version

We can make this a bit nicer by allowing the caller to control the non-int return value, giving the “naked” except a fig leaf, and changing the function name —

def int_or_else(value, else_value=None):
    """Given a value, returns the value as an int if possible. 
    If not, returns else_value which defaults to None.
    """
    try:
        return int(value)
    # I don't like catch-all excepts, but since objects can raise arbitrary
    # exceptions when executing __int__(), then any exception is
    # possible here, even if only TypeError and ValueError are 
    # really likely.
    except Exception:
        return else_value

At the Python prompt —

>>> print(int_or_else(trouble))
None
>>> print(int_or_else(trouble, 'spaghetti'))
spaghetti

So there you have it. I’m happy with this function. It feels bulletproof. It contains an (almost) naked except, but that only covers one simple line of code that’s unlikely to hide anything nasty.

You might also want to read a post I made about the exception handling choices in this post.

I release this code into the public domain, and I’ll even throw in the valuable Unintable class for free!

The image in this post is public domain and comes to us courtesy of Wikimedia Commons.

10 thoughts on “How Best to Coerce Python Objects to Integers?”

    1. Hi Paulo,
      I agree that logging mitigates the problems of a naked except. In this function, it feels like overkill, although one could argue that if some code calls int_or_else() on something really inappropriate like a `datetime` instance, that code is probably pretty confused and you might want to see that in your log file.

  1. As a general rule it is better to use ‘except Exception’ and not just ‘except’. This is because by using a bare ‘except’, you will catch and suppress special exceptions such as ‘SystemExit’ generated by calling ‘sys.exit()’ when code tries to shutdown an application.

      1. You’ve changed the code but the text still talks about using a naked except which is liable to cause a bit of confusion, Bruce 🙂

  2. The novice believes that the first priority is to stop the program from crashing. The expert understands that crashing (especially with an exception, but even a segfault) is actually very helpful and useful.

    The correct way to coerce a string to an integer in Python is:

    int(s)

    Don’t treat exceptions as things you should suppress at all costs. Treat them as useful, meaningful responses from the request to convert something to an integer.

    Instead, what you may want to look into is an atoi-style integer conversion, where you accept a series of digits and then ignore anything after them. That means that “25 C” comes out as 25, rather than None. It’s also a well-defined response that can be easily explained with a simple regex.

    Don’t mask NameError in your code – not even in a single line. Don’t mask problems from trying to intify a timestamp. Don’t swallow every exception and just return None, which is like ripping out your car’s dashboard and replacing it with a single response “Car is not moving, try again”. You wouldn’t accept it in a car; don’t accept it in software either. This kind of thing is what makes a typical PHP codebase a nightmare to debug, because PHP encourages you to just carry on regardless of errors. You don’t need to do it in Python.

    1. Hi Chris,
      There are times when I want exceptions to propagate, and times when I don’t. In this case I’m assuming the caller is in the latter category and explicitly doesn’t want to see any exceptions. Otherwise, there’s no point to the int_or_else() function.

      1. The exact point Chris wanted to make is that you _cannot_ make that assumption for your caller. If the caller wants to explicitly silence errors, let them write except themselves.

        And, by the way, why do you think exceptions are somehow special? How about this:

        class Mu:
        def __int__(self):
        while True:
        pass

        (Un)fortunately, there is no substitute for _thinking_ about your code.

  3. Chris’ comment is very much my ideas on this as well. Since your post already describes that classes can override their own int-coersion with custom code you’ve already presented the problem really. It is errors in this code you’re right away completely swallowing.

Leave a Reply

Your email address will not be published.