4.1 Modifying Python's Search Path

4 Custom Installation

Sometimes, the alternate installation schemes described in section 3 just don't do what you want. You might want to tweak just one or two directories while keeping everything under the same base directory, or you might want to completely redefine the installation scheme. In either case, you're creating a custom installation scheme.

You probably noticed the column of ``override options'' in the tables describing the alternate installation schemes above. Those options are how you define a custom installation scheme. These override options can be relative, absolute, or explicitly defined in terms of one of the installation base directories. (There are two installation base directories, and they are normally the same--they only differ when you use the Unix ``prefix scheme'' and supply different --prefix and --exec-prefix options.)

For example, say you're installing a module distribution to your home directory under Unix--but you want scripts to go in ~/scripts rather than ~/bin. As you might expect, you can override this directory with the --install-scripts option; in this case, it makes most sense to supply a relative path, which will be interpreted relative to the installation base directory (your home directory, in this case):

python setup.py install --home=~ --install-scripts=scripts

Another Unix example: suppose your Python installation was built and installed with a prefix of /usr/local/python, so under a standard installation scripts will wind up in /usr/local/python/bin. If you want them in /usr/local/bin instead, you would supply this absolute directory for the --install-scripts option:

python setup.py install --install-scripts=/usr/local/bin

(This performs an installation using the ``prefix scheme,'' where the prefix is whatever your Python interpreter was installed with-- /usr/local/python in this case.)

If you maintain Python on Windows, you might want third-party modules to live in a subdirectory of prefix, rather than right in prefix itself. This is almost as easy as customizing the script installation directory--you just have to remember that there are two types of modules to worry about, pure modules and non-pure modules (i.e., modules from a non-pure distribution). For example:

python setup.py install --install-purelib=Site --install-platlib=Site

The specified installation directories are relative to prefix. Of course, you also have to ensure that these directories are in Python's module search path, such as by putting a .pth file in prefix. See section 4.1 to find out how to modify Python's search path.

If you want to define an entire installation scheme, you just have to supply all of the installation directory options. The recommended way to do this is to supply relative paths; for example, if you want to maintain all Python module-related files under python in your home directory, and you want a separate directory for each platform that you use your home directory from, you might define the following installation scheme:

python setup.py install --home=~ \
                        --install-purelib=python/lib \
                        --install-platlib=python/lib.$PLAT \
                        --install-scripts=python/scripts
                        --install-data=python/data

or, equivalently,

python setup.py install --home=~/python \
                        --install-purelib=lib \
                        --install-platlib='lib.$PLAT' \
                        --install-scripts=scripts
                        --install-data=data

$PLAT is not (necessarily) an environment variable--it will be expanded by the Distutils as it parses your command line options, just as it does when parsing your configuration file(s).

Obviously, specifying the entire installation scheme every time you install a new module distribution would be very tedious. Thus, you can put these options into your Distutils config file (see section 5):

[install]
install-base=$HOME
install-purelib=python/lib
install-platlib=python/lib.$PLAT
install-scripts=python/scripts
install-data=python/data

or, equivalently,

[install]
install-base=$HOME/python
install-purelib=lib
install-platlib=lib.$PLAT
install-scripts=scripts
install-data=data

Note that these two are not equivalent if you supply a different installation base directory when you run the setup script. For example,

python setup.py --install-base=/tmp

would install pure modules to /tmp/python/lib in the first case, and to /tmp/lib in the second case. (For the second case, you probably want to supply an installation base of /tmp/python.)

You probably noticed the use of $HOME and $PLAT in the sample configuration file input. These are Distutils configuration variables, which bear a strong resemblance to environment variables. In fact, you can use environment variables in config files on platforms that have such a notion but the Distutils additionally define a few extra variables that may not be in your environment, such as $PLAT. (And of course, on systems that don't have environment variables, such as Mac OS 9, the configuration variables supplied by the Distutils are the only ones you can use.) See section 5 for details.

4.1 Modifying Python's Search Path

When the Python interpreter executes an import statement, it searches for both Python code and extension modules along a search path. A default value for the path is configured into the Python binary when the interpreter is built. You can determine the path by importing the sys module and printing the value of sys.path.

$ python
Python 2.2 (#11, Oct  3 2002, 13:31:27)
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-112)] on linux2
Type ``help'', ``copyright'', ``credits'' or ``license'' for more information.
>>> import sys
>>> sys.path
['', '/usr/local/lib/python2.3', '/usr/local/lib/python2.3/plat-linux2', 
 '/usr/local/lib/python2.3/lib-tk', '/usr/local/lib/python2.3/lib-dynload', 
 '/usr/local/lib/python2.3/site-packages']
>>>

The null string in sys.path represents the current working directory.

The expected convention for locally installed packages is to put them in the .../site-packages/ directory, but you may want to install Python modules into some arbitrary directory. For example, your site may have a convention of keeping all software related to the web server under /www. Add-on Python modules might then belong in /www/python, and in order to import them, this directory must be added to sys.path. There are several different ways to add the directory.

The most convenient way is to add a path configuration file to a directory that's already on Python's path, usually to the .../site-packages/ directory. Path configuration files have an extension of .pth, and each line must contain a single path that will be appended to sys.path. (Because the new paths are appended to sys.path, modules in the added directories will not override standard modules. This means you can't use this mechanism for installing fixed versions of standard modules.)

Paths can be absolute or relative, in which case they're relative to the directory containing the .pth file. Any directories added to the search path will be scanned in turn for .pth files. See site module" >the documentation for the site module for more information.

A slightly less convenient way is to edit the site.py file in Python's standard library, and modify sys.path. site.py is automatically imported when the Python interpreter is executed, unless the -S switch is supplied to suppress this behaviour. So you could simply edit site.py and add two lines to it:

import sys
sys.path.append('/www/python/')

However, if you reinstall the same major version of Python (perhaps when upgrading from 2.2 to 2.2.2, for example) site.py will be overwritten by the stock version. You'd have to remember that it was modified and save a copy before doing the installation.

There are two environment variables that can modify sys.path. PYTHONHOME sets an alternate value for the prefix of the Python installation. For example, if PYTHONHOME is set to "/www/python", the search path will be set to ['', '/www/python/lib/python2.2/', '/www/python/lib/python2.3/plat-linux2', ...].

The PYTHONPATH variable can be set to a list of paths that will be added to the beginning of sys.path. For example, if PYTHONPATH is set to "/www/python:/opt/py", the search path will begin with ['/www/python', '/opt/py']. (Note that directories must exist in order to be added to sys.path; the site module removes paths that don't exist.)

Finally, sys.path is just a regular Python list, so any Python application can modify it by adding or removing entries.

See About this document... for information on suggesting changes.