Using distutils

distutils is the Python standard for distributing modules. A package built upon distutils can be easily built and installed. It is similar in spirit to Perl’s ExtUtils::MakeMaker.

Building and installing a distutils-based package

To package a python module using distutils, create a script called setup.py in the root of your module. This script will be invoked to build, test, and install the package:

$ python setup.py build
$ python setup.py test
$ python setup.py install

build, test, and install are called commands in distutils lingo.

It is important to invoke setup.py with the Python you want to install the module for; distutils decides where to put the .py files based on the executable.

A Basic setup.py

A basic setup.py looks like:

from distutils.core import setup

setup(
    name = 'MyPythonModule',
    version = '1.00',
    description = 'This is my Python module.',

    author = 'Otto M. Ation',
    author_email =  'otto@boston.com',
)

distutils implements its commands with classes; build and install are standard commands. The test command is not standard, however, so we’ll have to add it.

Adding a new command: automated tests

By default, distutils does not have a test command. I consider this to be a bug. Adding commands to your setup script entails creating a distutils.core.Command subclass that implements the functionality of your new command, and then telling setup about it.

To create a test command, we first need to create a class to handle it:

from distutils.core import Command
from unittest import TextTestRunner, TestLoader
from glob import glob
from os.path import splitext, basename, join as pjoin, walk
import os

class TestCommand(Command):
    user_options = [ ]

    def initialize_options(self):
        self._dir = os.getcwd()

    def finalize_options(self):
        pass

    def run(self):
        '''
        Finds all the tests modules in tests/, and runs them.
        '''
        testfiles = [ ]
        for t in glob(pjoin(self._dir, 'tests', '*.py')):
            if not t.endswith('__init__.py'):
                testfiles.append('.'.join(
                    ['tests', splitext(basename(t))[0]])
                )

        tests = TestLoader().loadTestsFromNames(testfiles)
        t = TextTestRunner(verbosity = 1)
        t.run(tests)

The user_options member and finalize_options, initialize_options, and run methods are required by Command’s interface (Command is an abstract class, and your tests will not run without implementing those methods).

The interesting bits are in run. This method does several things: Finds all the test modules in the tests subdirectory, and uses the TestLoader class to load the unittest.TestCase-based test classes from them. Finally, a TextTestRunner instance is created to run actually invoke the tests (using its run method).

Another useful command is clean. Because Python compiles to byte code, running your test command is going to leave lots of .pyc files in your build directory, so the clean command will, well, clean those files up. An implementation might look like this:

class CleanCommand(Command):
    user_options = [ ]

    def initialize_options(self):
        self._clean_me = [ ]
        for root, dirs, files in os.walk('.'):
            for f in files:
                if f.endswith('.pyc'):
                    self._clean_me.append(pjoin(root, f))

    def finalize_options(self):
        pass

    def run(self):
        for clean_me in self._clean_me:
            try:
                os.unlink(clean_me)
            except:
                pass

Again, the empty finalize_options method and user_options member are ignored; the interesting things are the initialize_options method, will accumulates a list of the pyc files, and run which iterates over them and deletes them.

Once the classes have been created, add them to the call to setup using the cmdclass option:

setup(
    # As before

    cmdclass = { 'test': TestCommand, 'clean': CleanCommand }
)

Creating a distribution

The name and version keys are mainly used for creating a source distribution, which is done with the sdist command:

$ python setup.py sdist

The resulting tarball is named $name-$version.tar.gz, and is suitable for installation and distribution.