How To Set Up CentOS to Build Linux Wheels for Python

PEP 513 gives guidelines on how to build broadly compatible Linux platform wheels for Python. The PEP names CentOS 5.11 as the reference OS on which a Python wheel must run if it is to earn the right to the manylinux1 name.

The surest way to get one’s binary package to run on CentOS 5.11 is to build it there. This post explains how I set up a CentOS 5.11 VirtualBox guest to build manylinux1 Python wheels.

PEP 513 offers a prebuilt Docker container of CentOS 5.11. If you’re on Linux and/or you’re familiar with Docker, that’s probably a better route than building a VM.

Note that I’m not at all a Linux expert. If I’ve done something foolish or incorrect, I’d like to hear about it in the comments. Please be nice. =)

Why CentOS 5.11?

CentOS has a few things going for it that make it a good choice —

  • It’s free
  • As a derivative of Red Hat Enterprise Linux, it’s a conservative distro, so the libraries on it are likely older than the libraries on contemporaneous distros.
  • At the time PEP 513 was written, CentOS 5.11 was already over a year old. That increases the odds that other distros will have the libraries that it has.

It’s age is also a disadvantage, because at this stage the only updates CentOS 5 will receive are critical security updates and the CentOS team “recommends that you start moving workloads from CentOS 5 to CentOS Linux 6 or CentOS Linux 7”.

CentOS 5.11 Setup

Download the CentOS 5.11 ISO and install under VirtualBox. During the CentOS installation, I opted to disable SELinux. Since I only use this installation for builds and not as a server or daily desktop, I don’t feel the need for high security.

Once CentOS is installed, let the updater download and install its patches.

Next, you’ll want to install VirtualBox guest additions to make the guest OS easier to use. In order to do that, you first have to add yourself to the sudoers file.

Add Yourself to sudoers

Open a terminal and enter the following commands —

  1. su -
  2. vim /etc/sudoers
  3. At the end of the file, add this line:
    your_username ALL=(ALL) ALL
  4. Save the file with :wq!
  5. Type exit to exit the su - shell.

Now you should be able to run commands with sudo.

Build the VirtualBox Guest Additions

  1. Install GCC:
    sudo yum install gcc gcc-c++
  2. Insert the Guest Additions CD.
  3. Start a terminal and cd /media/VBOXADDITIONS_xxxx. Note that the exact name of the VBOXADDITIONS directory changes with each each version of VirtualBox.
  4. sudo ./VBoxLinuxAdditions.run
  5. Eject the Guest Additions CD and reboot.

Add Packages

Install the things you’ll need to build Python, and to use pip —

sudo yum install xz zlib zlib-devel openssl-devel pcre-devel sqlite-devel

Build Python

CentOS 5.11 comes with Python 2.4. You will undoubtedly want a newer Python, so download and untar the source code for the Python you want to use and then build it. I built Python 2.7.11 with these steps —

 sudo ./configure --enable-unicode=ucs4
 sudo make altinstall

It’s important to use UCS4 (as opposed to the default UCS2) during the configure step to increase your odds of being compatible with the Pythons built for other Linux distros.

make altinstall tells Python to install itself in such a way that it doesn’t interfere with the default (system) Python.

Once my Python was built, I added a symlink to make it the default Python in my shell —

 sudo ln -s /usr/local/bin/python2.7 /usr/local/bin/python

At this point, if you start a new terminal and type python, you should
get Python 2.7.11.

Download and Install pip

wget --no-check-certificate https://bootstrap.pypa.io/get-pip.py
sudo python get-pip.py

This also installs setuptools and wheel, both of which we need.

Fix pip

At this point, pip will malfunction as described in issue 1918. The fix is not difficult.

Use sudo vim to edit these 3 scripts —

  • /usr/local/bin/pip
  • /usr/local/bin/pip2
  • /usr/local/bin/pip2.7

In each, change the first line from this —

#!/usr/bin/python

to this —

#!/usr/local/bin/python

Enjoy, and Build Some Wheels!

You’re done! Go build some great wheels!

 

My Code Got Cyndi Lauper-ed, and I’m Glad

Let’s start with a quick musical quiz. Who wrote this 1983 pop hit? I’ll give you a hint — it’s not the person who sings it.

You can see the songwriting credits here (assuming that link survives time), but for most of you it won’t be a useful clue.

Those of you who grew up in Philadelphia in the 1980s might recognize the name Robert Hazard, leader of Robert Hazard and the Heroes, author of Escalator of Life, Change Reaction, and Out of the Blue, and, of course, Girls Just Wanna Have Fun. Robert’s popularity never grew much out of the Philly/NJ area, but Cyndi Lauper’s version of Girls Just Wanna Have Fun sold a zillion copies worldwide and touched a lot of people who had never heard of Robert Hazard, and never will.

I have something in common with him — someone has given my work far more exposure than I ever expected it to get. (Another thing we have in common is growing up in the same small Philadelphia suburb. But without Cyndi Lauper’s involvement, that’s just trivia.)

I was surprised to learn from http://pypi-ranking.info that posix_ipc, one of my open source packages, is currently in the top .5% (½ of 1%) of the most downloaded on PyPI. Now, posix_ipc might be good at what it does, but it fills a tiny niche that’s nowhere near big enough to justify all of those 1.7-million-and-counting downloads. Why is it a top 1% download? Because it has become part of something much bigger — the massively popular OpenStack.

OpenStack didn’t have to rewrite portions of posix_ipc (like Cyndi did with Girls Just Wanna Have Fun, with Robert’s permission). They haven’t yet made a video of it that includes a nod to the Marx Brothers (like Cyndi did, with or without Robert’s permission). And as far as I know, OpenStack has yet to be nominated for a Grammy. But they have shown me the value of putting something out into the world, because you never know where it will end up.

So thanks, OpenStack! And thanks to Robert Hazard for music I enjoyed growing up (and still do). R.I.P, Robert.