Introduction

    The Modules/Setup mechanism has some flaws:

    * People have to remember to uncomment bits of Modules/Setup in
      order to get all the possible modules.

    * Moving Setup to a new version of Python is tedious; new modules
      have been added, so you can't just copy the older version, but
      have to reconcile the two versions.

    * Users have to figure out where the needed libraries, such as
      zlib, are installed.

Proposal

    Use the Distutils to build the modules that come with Python.

    The changes can be broken up into several pieces:

    1. The Distutils needs some Python modules to be able to build
       modules.  Currently I believe the minimal list is posix, _sre,
       and string.

       These modules will have to be built before the Distutils can be
       used, so they'll simply be hardwired into Modules/Makefile and
       be automatically built.

    2. A top-level setup.py script will be written that checks the
       libraries installed on the system and compiles as many modules
       as possible.

    3. Modules/Setup will be kept and settings in it will override
       setup.py's usual behavior, so you can disable a module known
       to be buggy, or specify particular compilation or linker flags.
       However, in the common case where setup.py works correctly,
       everything in Setup will remain commented out.  The other
       Setup.* become unnecessary, since nothing will be generating
       Setup automatically.

    The patch was checked in for Python 2.1, and has been subsequently 
    modified.

Implementation

    Patch #102588 on SourceForge contains the proposed patch.
    Currently the patch tries to be conservative and to change as few
    files as possible, in order to simplify backing out the patch.
    For example, no attempt is made to rip out the existing build
    mechanisms.  Such simplifications can wait for later in the beta
    cycle, when we're certain the patch will be left in, or they can
    wait for Python 2.2.
    
    The patch makes the following changes:

    * Makes some required changes to distutils/sysconfig (these will
      be checked in separately)

    * In the top-level Makefile.in, the "sharedmods" target simply 
      runs "./python setup.py build", and "sharedinstall" runs
      "./python setup.py install".  The "clobber" target also deletes
      the build/ subdirectory where Distutils puts its output.

    * Modules/Setup.config.in only contains entries for the gc and thread
      modules; the readline, curses, and db modules are removed because 
      it's now setup.py's job to handle them.

    * Modules/Setup.dist now contains entries for only 3 modules --
      _sre, posix, and strop.

    * The configure script builds setup.cfg from setup.cfg.in.  This
      is needed for two reasons: to make building in subdirectories
      work, and to get the configured installation prefix.

    * Adds setup.py to the top directory of the source tree.  setup.py
      is the largest piece of the puzzle, though not the most
      complicated.  setup.py contains a subclass of the BuildExt
      class, and extends it with a detect_modules() method that does
      the work of figuring out when modules can be compiled, and adding 
      them to the 'exts' list.

Unresolved Issues

    Do we need to make it possible to disable the 3 hard-wired modules
    without manually hacking the Makefiles?  [Answer: No.]

    The Distutils always compile modules as shared libraries.  How do
    we support compiling them statically into the resulting Python
    binary?

    [Answer: building a Python binary with the Distutils should be
    feasible, though no one has implemented it yet.  This should be
    done someday, but isn't a pressing priority as messing around with
    the top-level Makefile.pre.in is good enough.]

Copyright

    This document has been placed in the public domain.