Category: coding

My Code Got Cyndi Lauper-ed, and I’m Glad

Let’s start with a quick musical quiz. Who wrote this 1983 pop hit? I’ll give you a hint — it’s not the person who sings it.

You can see the songwriting credits here (assuming that link survives time), but for most of you it won’t be a useful clue.

Those of you who grew up in Philadelphia in the 1980s might recognize the name Robert Hazard, leader of Robert Hazard and the Heroes, author of Escalator of Life, Change Reaction, and Out of the Blue, and, of course, Girls Just Wanna Have Fun. Robert’s popularity never grew much out of the Philly/NJ area, but Cyndi Lauper’s version of Girls Just Wanna Have Fun sold a zillion copies worldwide and touched a lot of people who had never heard of Robert Hazard, and never will.

I have something in common with him — someone has given my work far more exposure than I ever expected it to get. (Another thing we have in common is growing up in the same small Philadelphia suburb. But without Cyndi Lauper’s involvement, that’s just trivia.)

I was surprised to learn from http://pypi-ranking.info that posix_ipc, one of my open source packages, is currently in the top .5% (½ of 1%) of the most downloaded on PyPI. Now, posix_ipc might be good at what it does, but it fills a tiny niche that’s nowhere near big enough to justify all of those 1.7-million-and-counting downloads. Why is it a top 1% download? Because it has become part of something much bigger — the massively popular OpenStack.

OpenStack didn’t have to rewrite portions of posix_ipc (like Cyndi did with Girls Just Wanna Have Fun, with Robert’s permission). They haven’t yet made a video of it that includes a nod to the Marx Brothers (like Cyndi did, with or without Robert’s permission). And as far as I know, OpenStack has yet to be nominated for a Grammy. But they have shown me the value of putting something out into the world, because you never know where it will end up.

So thanks, OpenStack! And thanks to Robert Hazard for music I enjoyed growing up (and still do). R.I.P, Robert.

 

My Function Failed Inspection

Python’s inspect module allows one to examine the signature of functions, like so:

$ python3
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 23 2015, 02:52:03)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import inspect
>>> def f(foo=42):
... pass
...
>>> print(inspect.signature(f))
(foo=42)

I wanted to use function signature inspection during unit testing of my sysv_ipc and posix_ipc modules to ensure that my code matched its documentation.

Unfortunately, inspect doesn’t work with functions written in C, as you can see in the example below that uses math.sqrt().

$ python3
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 23 2015, 02:52:03)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import inspect
>>> import math
>>> inspect.signature(math.sqrt) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/inspect.py", line 2055, in signature return _signature_internal(obj) File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/inspect.py", line 1957, in _signature_internal skip_bound_arg=skip_bound_arg) File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/inspect.py", line 1890, in _signature_from_builtin raise ValueError("no signature found for builtin {!r}".format(func)) ValueError: no signature found for builtin <built-in function sqrt> 
>>>

Since posix_ipc and sysv_ipc are written in C, I have to test their functions and methods with keyword arguments by calling each one with each keyword argument specified.

Maybe there is no current working directory

A few years ago I wrote a Django app for a client. One part of the app called os.getcwd(), and another part (that I thought of as completely separate) used a temporary directory to build PDFs.

Occasionally the call to os.getcwd() would raise an error. I was confused. How can there be no current directory? It took me a while to figure it out, but in hindsight it’s kind of obvious (as these things often are).

My PDF-building code created a temporary directory, set that directory to be the current working directory, and then removed the directory once the PDF was built. After that, there was no current working directory. It’s easy to demonstrate —

$ python3
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 23 2015, 02:52:03)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import tempfile
>>> temp_dir = tempfile.TemporaryDirectory()
>>> os.chdir(temp_dir.name)
>>> os.getcwd()
'/private/var/folders/9f/4zptd_j10dx343w0r7tvkm0r0000gn/T/tmpfcwpe1xs'
>>> temp_dir.cleanup()
>>> os.getcwd()
Traceback (most recent call last):
File "", line 1, in
FileNotFoundError: [Errno 2] No such file or directory
>>>

I have a feeling I’m not the only developer who was foolish enough to assume that os.getcwd() would never fail. At least now I know better, and you do too!

 

 

I’m not that type of variable

This is a story about the details of a C type leaking into Python.

As part of testing my Python wrapper for SysV IPC, I wrote tests for the time-related attributes of the IPC objects that change when something happens to the object. For instance, when someone sends a message to a queue, the queue’s last_send_time attribute (msg_stime in the C code) is updated to the current time.

I have a hard time imagining many programmers care about these attributes. Knowing the last time someone changed the uid of a message queue, for instance, just doesn’t have many use cases. But they’re part of the SysV IPC API and so I want my package to expose them.

I wrote tests to ensure that each one changed when it was supposed to. The tests failed consistently although the code worked when I tested it “by hand” in an interactive shell. Here’s the relevant portion of a failing test:

 def test_property_last_change_time(self):
     """exercise MessageQueue.last_change_time"""
     original_last_change_time = self.mq.last_change_time
     # This might seem like a no-op, but setting the UID to
     # any value triggers a call to msgctl(...IPC_STAT...)
     # which should set last_change_time.
     self.mq.uid = self.mq.uid
     # Ensure the time actually changed.
     self.assertNotEqual(self.mq.last_change_time,
                         original_last_change_time)

The problem is obvious, right? No, it wasn’t obvious to me, either.

The problem is that in C, a message queue’s last change time (msg_ctime) is of variable type time_t which is typedef-ed as an integral type (int or long) on most (all?) systems. Because the test above executed in less than 1 second, the assertion always failed. Setting self.mq.uid correctly caused an update to the last change time (msg_ctime), it was just being updated to the same value that had been saved in the first line of the test.

My solution was to add a little sleeping, like so –

 def test_property_last_change_time(self):
     """exercise MessageQueue.last_change_time"""
     original_last_change_time = self.mq.last_change_time
     time.sleep(1.1)
     # This might seem like a no-op, but setting the UID to
     # any value triggers a call to msgctl(...IPC_STAT...)
     # which should set last_change_time.
     self.mq.uid = self.mq.uid
     # Ensure the time actually changed.
     self.assertNotEqual(self.mq.last_change_time,
                         original_last_change_time)

That ensured that the value stored in original_last_change_time at the start of the test would differ from self.mq.last_change_time by at least 1 at the end of the test.

Lessons learned: don’t forget about C types, especially when you’re wrapping a C API.