Abstract
Making a Python release is an arduous process that takes a
minimum of half a day's work even for an experienced releaser.
Until recently, most -- if not all -- of that burden was borne by
Guido himself. But several recent releases have been performed by
other folks, so this PEP attempts to collect, in one place, all
the steps needed to make a Python release. It is organized as a
recipe and you can actually print this out and check items off as
you complete them.
How to Make A Release
Here are the steps taken to make a Python release. Some steps are
more fuzzy than others because there's little that can be
automated (e.g. writing the NEWS entries). Where a step is
usually performed by An Expert, the name of that expert is given.
Otherwise, assume the step is done by the Release Manager (RM),
the designated person performing the release. Almost every place
the RM is mentioned below, this step can also be done by the BDFL
of course!
XXX: We should include a dependency graph to illustrate the steps
that can be taken in parallel, or those that depend on other
steps.
We use the following conventions in the examples below. Where a
release number is given, it is of the form X.YaZ, e.g. 2.1a3 for
Python 2.1 alpha 3, where "a" == alpha, "b" == beta, "rc" ==
release candidate.
Final releases are named "releaseXY". The branch tag is
"releaseXY-maint" because this will point to the long lived
maintenance branch. The fork tag on the trunk is
"releaseXY-fork". If a micro release number is used, then we'll
say X.Y.MaZ.
Note: This document has been updated to reflect the more
streamlined procedures used to release Python 2.3 (including the
alphas and betas).
___ Impose a check-in freeze. Send a message to
python-dev@python.org telling people not to make any check-ins
on the tree until further notice.
At this point, nobody except the RM should make any commits to
the branch (or his duly assigned agents, i.e. Guido the BDFL,
Fred Drake for documentation, or Tim Peters for Windows). If
the RM screwed up and some desperate last minute change to the
branch is necessary, it can mean extra work for Fred and Tim.
So try to avoid this!
___ Log into irc.freenode.net and join the #python-dev channel.
You probably need to coordinate with other people around the
world. This IRC channel is where we've arranged to meet.
___ The most important thing to do is to update the Misc/NEWS file.
Tim will need this in order to do the Windows release and he
likes to stay up late. This step can be pretty tedious, so it's
best to get to it immediately after making the branch, or even
before you've made the branch.
Add high level items new to this release. E.g. if we're
releasing 2.2a3, there must be a section at the top of the file
explaining "What's new in Python 2.2a3". It will be followed by
a section entitled "What's new in Python 2.2a2".
Note that you /hope/ that as developers add new features to the
trunk, they've updated the NEWS file accordingly. You can't be
positive, so double check. If you're a Unix weenie, it helps to
verify with Tim Peters about changes on Windows, and Jack Jansen
about changes on the Mac.
This command should help you:
% cvs log | python Tools/scripts/logmerge.py > /tmp/news.txt
IOW, you're printing out all the cvs log entries from the
previous release until now. You can then troll through the
news.txt file looking for interesting things to add to NEWS.
___ For major releases (e.g. 2.3 final), move any historical "what's
new" entries from Misc/NEWS to Misc/HISTORY.
___ Check with the IDLE maintainer to be sure that
Lib/idlelib/NEWS.txt has been similarly updated.
___ For final (non-alpha/beta/candidate) releases, make sure the
release date is fully spelled out in Doc/commontex/boilerplate.tex.
___ Tag and/or branch the tree for release X.YaZ
If you're releasing an alpha/beta/release candidate, you will
just tag the tree. If you are releasing a final release, you
will both tag the trunk and create the long-lived maintenance
branch.
All Python development happens on the trunk. While it's
sometimes challenging to keep people from checking things in
while you're making a release, it's still preferred to creating
a short-lived release branch.
Practically speaking, we tag and branch just before making the
release. Tagging too early causes too much merging work.
___ Do a CVS update with the -A, -d, and -P flags, e.g.
% cvs -q update -d -P -A
To tag the tree, do the following:
___ cvs tag rXYaZ
To create a maintenance branch the following steps are taken:
___ CVS tag the trunk with the symbolic name "releaseXY-fork", e.g.
% cvs tag releaseXY-fork
___ Make the branch with the symbolic name "releaseXY-maint", e.g.
% cvs tag -b releaseXY-maint
___ Check out a clean version of the branch into a new directory.
You'll be doing a lot of work in this directory and you want
to keep it straight from your trunk working directory. E.g.
% export CVSROOT=cvs.sf.net:/cvsroot/python
% cvs -q co -d python-22a3 -r release23-maint python/dist/src
___ cd into the branch directory.
___ Change Include/patchlevel.h in two places, to
reflect the new version number you've just created. You'll want
to change the PY_VERSION macro, and one or several of the
version subpart macros just above PY_VERSION, as appropriate.
___ IDLE maintains its own versioning and NEWS file (Lib/idlelib/NEWS.txt).
There should be a number of entries reflecting new development, under a
temporary header. Update that header to reflect IDLE's new version and
release date. Then update Lib/idlelib/idlever.py to show a matching
version.
___ Change the "%define version" line of Misc/RPM/python-2.3.spec to
the same string as PY_VERSION was changed to above. E.g.
%define version 2.3.1
The following line, "%define libvers", should reflect the
major/minor number as one would usually see in the
"/usr/lib/python<libvers>" directory name. E.g.
%define libvers 2.3
You also probably want to reset the %define release line
to '1pydotorg' if it's not already that.
If the new release uses a major/minor version which is
different than is in the name of the current
"Misc/RPM/python-*.spec" file, rename the file:
% mv python-2.3.spec python-2.4.spec
% cvs remove python-2.3.spec
% cvs add python-2.4.spec
% cvs commit
___ If this is a release candidate, mail Sean <jafo@tummy.com>
noting the impending release, so that RPMs can be built and
tested.
___ Update the README file, which has a big banner at the top
proclaiming its identity.
___ If the major (first) or minor (middle) digit of the version
number changes, also update the LICENSE file.
___ There's a copy of the license in
Doc/commontex/license.tex; Fred usually takes care of that.
___ Check the years on the copyright notice. If the last release
was some time last year, add the current year to the copyright
notice in several places:
___ README
___ LICENSE
___ Python/getcopyright.c
___ Doc/commontex/copyright.tex
___ PC/python_nt.rc sets up the DLL version resource for Windows
(displayed when you right-click on the DLL and select
Properties).
___ PCbuld/python20.wse sets up the Windows installer version
resource (displayed when you right-click on the installer .exe
and select Properties).
___ The license.ht file for the distribution on the website
contains what purports to be an HTML-ized copy of the LICENSE
file from the distribution.
___ For a final release, edit the first paragraph of
Doc/whatsnew/whatsnewXX.tex to include the actual release date;
e.g. "Python 2.3 was released on August 1, 2003."
There's no need to edit this for alpha or beta releases. Note
that Andrew often takes care of this.
___ At this point, Fred will create the formatted versions of the
documentation and push the appropriate files out to their FTP
locations on www.python.org. The HTML format is used to build
the HTML Help format for the Windows installer, but the RM
doesn't need this to build the source distribution. The HTML
Help format will typically be generated by whoever builds the
Windows installer.
Once Fred is done, there can be no further checkins on the
branch in the Doc/ directory -- not even by the RM.
Building the documentation is done using the Makefile in the
Doc/ directory. Once all the external tools are installed (see
the "Documenting Python" manual for information on the required
tools), use these commands to build the formatted documentation
packages::
$ make clobber
...
$ make PAPER=a4 paperdist
...
$ make distfiles
...
The packages can be installed on the FTP server using commands
like these:
$ VERSION=`tools/getversioninfo`
$ TARGET=/ftp/ftp.python.org/pub/python/doc/$VERSION
$ ssh creosote.python.org mkdir $TARGET
$ scp *-$VERSION.* creosote.python.org:$TARGET
___ For final releases, publish the documentation on python.org.
This must be done by someone with write access to the python.org
CVS repository.
Start by creating a new directory and filling it with the
standard boilerplate. $VERSION is the same as for uploading the
documentation, above; $OLDVERSION is the most recently published
version on the site.
$ cd .../pydotorg/doc/
$ mkdir $VERSION
$ cvs add $VERSION
$ cd $OLDVERSION
$ cp .cvsignore Makefile index.ht download.ht ../$VERSION
$ cd ../$VERSION
$ cvs add .cvsignore Makefile *.ht
Now make the following edits:
- in Makefile, change the value of ROOT_OFFSET to doc/$VERSION
- in index.ht, change:
- the version number to $VERSION in two places: the Title:
header, and the <h3> at the top of the page
- the release date, in the <h3> at the top of the page
- if the minor release number changed (for example, from 2.3
to 2.4), the title and link to the "What's New" document
(search for "whatsnew")
- in download.ht, change:
- the version number to $VERSION in two places: the Title:
header, and the <h3> at the top of the page
- the release date, in the <h3> at the top of the page
- if the minor release number changed (for example, from 2.3
to 2.4), the title and link to the "What's New" document
(search for "whatsnew")
- replace the large table of downloads with the content of the
pkglist.html file generated by the documentation build
process
Now, the web content has all been prepared, but there's still
some dancing to do to make it all work right. To be safe, we
can commit the new files to CVS, but we're *not* ready to
install them on the site yet:
$ cvs commit -m \
"Add website content for Python $VERSION documentation."
Log into creosote.python.org using SSH and unpack a copy of the
documentation into place:
# on creosote:
$ cd /ftp/www.python.org/doc
$ tar xjf \
/ftp/ftp.python.org/pub/python/doc/$VERSION/html-$VERSION.tar.bz2
$ mv Python-Docs-$VERSION $VERSION
$ find $VERSION -type d | xargs chmod g+s
Now head back to your pydotorg checkout on your workstation, and
push website content into place:
$ cd .../pydotorg/doc/$VERSION
$ make install
Point your browser at this URL and check it out:
http://www.python.org/doc/$VERSION/
There are three more changes that need to happen in the
top-level doc/ directory of the website content. The first of
these can happen any time after what's already happened in this
process, and the last two should happen as soon as the release
announcement has been made. Those are described in a separate
step of this checklist.
At this time, edit the versions.ht file in doc/ to make add the
new release at the top. The previous latest release should drop
down to the top of the long list of released versions, and the
new release should replace the previous most recent release.
There should be a blank line between the link to the development
documentation and the most recent release, and another blank
line between the most recent release and the long list of older
releases. (Is should be fairly easy to figure this out while
looking at the file.)
___ Thomas grabs the HTML to build the Windows helpfile.
The HTML files are unpacked into a new src/html directory, and
runs this command to create the project files for MS HTML
Workshop:
% python ..\Doc\tools\prechm.py -v 2.3 python23
HTML Workshop is then fired up on the created python23.hhp file,
finally resulting in an python23.chm file.
___ Tim Peters grabs the HTML Help format and uses this to build the
Windows installer.
___ Tim performs his Windows magic, generating an installer
executable. He uploads this file to SourceForge, and then sends
the RM a notice which includes the location and MD5 checksum of
the Windows executable.
Note that Tim used to upload the installer to www.python.org,
but has had problems with ssh for a while now.
Note that Tim's creation of the Windows executable may generate
a few more commits on the branch. Tim will be responsible for
merging Windows-specific changes from trunk to branch, and from
branch to trunk.
___ Sean Reifschneider grabs the HTML and uses this to build the
Linux RPMs. Sean performs his Red Hat magic, generating a set
of RPMs. He uploads these files to python.org. He then sends
the RM a notice which includes the location and MD5 checksum of
the RPMs.
___ Download the Windows executable from SourceForge to
creosote.python.org. Tell Tim so he can remove the file from
SourceForge.
___ Time to build the source tarball. If you created a branch, be
sure to cd to your working directory for the branch. E.g.
% cd .../python-22a3
___ Do a "cvs update" in this directory. Do NOT include the -A flag
if you're working on a branch, but do include it if you're
working on the trunk.
You should not see any "M" files, but you may see several "P" or
"U" files. I.e. you better not have any uncommitted changes in
your working directory, but you may pick up some of Fred's or
Tim's last minute changes.
___ If you've seen updates to existing files, update the cvs tag:
% cvs tag -F r22a3
If you created a maintenance branch and you've changed any files
since you branched, tag the tree -- in the branch -- now with
something like
% cvs tag r23
This is the tag you will use below.
___ Change to a neutral directory, i.e. one in which you can do a
fresh, virgin, cvs export of the branch. You will be creating a
new directory at this location, to be named "Python-X.YaZ". Do
a CVS export of the tagged branch.
% cd ~
% export CVSROOT=cvs.sf.net:/cvsroot/python
% cvs export -rr23c2 -d Python-2.3c2 python/dist/src
___ Generate the tarball. Note that we're not using the `z' option
on the tar command because 1) that's only supported by GNU tar
as far as we know, and 2) we're going to max out the compression
level, which isn't a supported option.
% tar cf - Python-2.3c2 | gzip -9 > Python-2.3c2.tgz
___ Calculate the MD5 checksum of the tgz file you just created
% md5sum Python-2.3c2.tgz
Note that if you don't have the md5sum program, there is a
Python replacement in the Tools/scripts/md5sum.py file.
___ Now you want to perform the very important step of checking the
tarball you just created, to make sure a completely clean,
virgin build passes the regression test. Here are the best
steps to take:
% cd /tmp
% tar zxvf ~/Python-2.3c2.tgz
% cd Python-2.3c2
% ls
(Do things look reasonable?)
% ./configure
(Loads of configure output)
% make test
(Do all the expected tests pass?)
If you're feeling lucky and have some time to kill, run the full
test suite:
% make TESTOPTS='-u all' test
If the tests pass, then you can feel good that the tarball is
fine. If some of the tests fail, or anything else about the
freshly unpacked directory looks weird, you better stop now and
figure out what the problem is.
___ Upload the tgz file to creosote.python.org using scp.
___ Tim has been having trouble uploading to creosote, so he will
usually put the file on SF, giving you the file name and the md5
checksum. It's best to do a wget from creosote to SF, but
calculating the URL can be not-fun. You can usually get the URL
from the file download page, if you start the download and then
immediately cancel it.
___ While you're waiting, you can start twiddling the web pages to
include the announcement.
___ If necessary, and if you have the right permissions (the
python.org sysadmins must set this up for you), check out the
web site CVS tree by doing:
% cvs -d :ext:<you>@creosote.python.org:/usr/local/cvsroot co pydotorg
___ In the python.org web site CVS tree, cd to the X.Y
subdirectory, and copy index.ht to new-index.ht. Be sure to
do a "cvs update" first!
% cd .../pydotorg
% cvs -q up -P -d
% cd 2.2
% cp index.ht new-index.ht
___ Edit the file for content: usually you can globally replace
X.Ya(Z-1) with X.YaZ. However, you'll need to think about the
"What's New?" section.
___ Copy the Misc/NEWS file to NEWS.txt in the X.Y directory for
python.org; this contains the "full scoop" of changes to
Python since the previous release for this version of Python.
___ Also, update the MD5 checksums.
___ Preview the web page by doing a "make" -- NOT a "make install".
View the page via a file: url.
___ Similarly, edit the ../index.ht file, i.e. the python.org home
page. In the Big Blue Announcement Block, move the paragraph
for the new version up to the top and boldify the phrase
"Python X.YaZ is out". Edit for content, and preview as
above. Do NOT do a "make install" yet!
___ Also on the ../index.ht file (still the python.org home page),
update the link information so that the release status is
correct. Update the links in the left-hand navigation
sidebar. Still do NOT do a "make install"!
___ Now we're waiting for the scp to creosote to finish. Da de da,
da de dum, hmm, hmm, dum de dum.
___ Now you need to go to creosote.python.org and move all the files
in place over there. Our policy is that every Python version
gets its own directory, but each directory may contain several
releases. We keep all old releases, moving them into a "prev"
subdirectory when we have a new release.
So, there's a directory called "2.2" which contains
Python-2.2a2.exe and Python-2.2a2.tgz, along with a "prev"
subdirectory containing Python-2.2a1.exe and Python-2.2a1.tgz.
So...
___ On creosote, cd to ~ftp/pub/python/X.Y creating it if
necessary.
___ Move the previous release files to a directory called "prev"
creating the directory if necessary (make sure the directory
has g+ws bits on). If this is the first alpha release of a
new Python version, skip this step.
___ Move the .tgz file and the .exe file to this directory. Make
sure they are world readable. They should also be group
writable, and group-owned by webmaster.
___ md5sum the files and make sure they got uploaded intact.
___ Update the X.Y/bugs.ht file if necessary. It is best to get
BDFL input for this step.
___ Now preview the new-index.ht file once more. IMPORTANT: follow
every link on the page to make sure it goes where you expect it
to go, and that what you expect to be there is there.
___ If everything looks good, move new-index.ht to index.ht and do a
"make install" in this directory. Go up to the parent directory
(i.e. the root of the web page hierarchy) and do a "make
install" there too. You're release is now live!
___ Now it's time to write the announcement for the mailing lists.
This is the fuzzy bit because not much can be automated. You
can use one of Guido's earlier announcements as a template, but
please edit it for content!
Once the announcement is ready, send it to the following
addresses:
python-list@python.org
python-announce@python.org
python-dev@python.org
___ Send a SourceForge News Item about the release. From the
project's "menu bar", select the "News" link; once in News,
select the "Submit" link. Type a suitable subject (e.g. "Python
2.2c1 released" :-) in the Subject box, add some text to the
Details box (at the very least including the release URL at
www.python.org and the fact that you're happy with the release)
and click the SUBMIT button.
Feel free to remove any old news items.
___ Make the last two changes to the documentation area on
python.org. (Remember those from the documentation items above?
It's time now.)
In your pydotorg checkout, edit the file doc/index.ht to update
to the right version number and release date. Commit the
changes to CVS and push the file to the website:
$ cd .../pydotorg/doc/
$ make install
This will cause the doc/index.html file to claim to point to the
new docs, but they won't. Run (don't walk!) to
creosote.python.org, and update a symlink in the doc/ tree:
# on creosote:
$ cd /ftp/www.python.org/doc/
$ rm current && ln -s $VERSION current
Good. Stop running.
Now it's time to do some cleaning up. These steps are very important!
___ If you made a non-maintenance branch, be sure to merge it into
the trunk! Now that we've released this branch, we don't need
it any more. We've already tagged it so we can always reproduce
it. Note that merging branches is a bit of a black art, but
here's what's worked for us.
NOTE: If this was an X.Y major release, we will be using this as
the maintenance branch for a long time to come.
___ Check out a completely clean, virgin working directory of the
trunk, by doing this in the directory that is the parent of
your branch working directory python-XYaZ:
% cvs -d <cvsroot> co -d python-clean python/dist/src
___ Run a diff against your branch by doing this in the common
parent directory containing both python-clean and python-XYaZ:
% diff -r python-clean python-22a2 | grep ^diff | grep -v CVS \
> /tmp/diffcmd.sh
___ Edit diffcmd.sh to get rid of files that you know don't have
important changes. You're looking for files that have updates
in the branch that haven't made it to the trunk.
Generally you can ignore any changes to the Doc or Mac
subdirectories, or any changes to Windows related files. The
sub-RMs for those parts will take care of any necessary merges
from the branch to the trunk.
If you've been diligent about merging changes from the trunk
into the branch, there shouldn't be many of these files.
___ Edit /tmp/diffcmd.sh, changing all the -r's into -u's. Run
the /tmp/diffcmd.sh command like so:
% sh /tmp/diffcmd.sh > /tmp/pydiff.txt
___ Attempt to patch your python-clean working directory. Do this
first, noting that --dry-run does not actually apply any
patches, it just makes sure that the patch command runs
successfully to completion:
% patch -p1 --dry-run < /tmp/pydiff.txt
___ If this goes well, run it again, taking out the --dry-run
option. If this fails, or if it prompts you for a file to
patch, try using -p0 instead of -p1. Otherwise, your diff
command was messed up, so try again.
___ cd to python-clean and do a "cvs commit". Use as your log
message something like "Merging the rXYaZ-maint tag back into
the trunk".
___ Edit the file Include/patchlevel.h so that the PY_VERSION
string says something like "X.YaZ+". Note the trailing `+'
indicating that the trunk is going to be moving forward with
development. E.g. the line should look like:
#define PY_VERSION "2.2a2+"
Make sure that the other PY_ version macros contain the
correct values. Commit this change.
___ For the extra paranoid, do a completely clean test of the
release. This includes downloading the tarball from
www.python.org.
___ Make sure the md5 checksums match. Then unpack the tarball,
and do a clean make test.
% make distclean
% ./configure
% make test
To ensure that the regression test suite passes. If not, you
screwed up somewhere!
Step 5 ...
Verify! This can be interleaved with Step 4. Pretend you're a
user: download the files from python.org, and make Python from it.
This step is too easy to overlook, and on several occasions we've
had useless release files. Once a general server problem caused
mysterious corruption of all files; once the source tarball got
built incorrectly; more than once the file upload process on SF
truncated files; and so on.
What Next?
Rejoice. Drink. Be Merry. Write a PEP like this one. Or be
like unto Guido and take A Vacation.
You've just made a Python release!
Actually, there is one more step. You should turn over ownership
of the branch to Jack Jansen. All this means is that now he will
be responsible for making commits to the branch. He's going to
use this to build the MacOS versions. He may send you information
about the Mac release that should be merged into the informational
pages on www.python.org. When he's done, he'll tag the branch
something like "rX.YaZ-mac". He'll also be responsible for
merging any Mac-related changes back into the trunk.
Final Release Notes
The Final release of any major release, e.g. Python 2.2 final, has
special requirements, specifically because it will be one of the
longest lived releases (i.e. betas don't last more than a couple
of weeks, but final releases can last for years!).
For this reason we want to have a higher coordination between the
three major releases: Windows, Mac, and source. The Windows and
source releases benefit from the close proximity of the respective
release-bots. But the Mac-bot, Jack Jansen, is 6 hours away. So
we add this extra step to the release process for a final
release:
___ Hold up the final release until Jack approves, or until we
lose patience <wink>.
Windows Notes
Windows has a GUI installer, various flavors of Windows have
"special limitations", and the Windows installer also packs
precompiled "foreign" binaries (Tcl/Tk, expat, etc). So Windows
testing is tiresome but very necessary.
Concurrent with uploading the installer, Tim installs Python from
it twice: once into the default directory suggested by the
installer, and later into a directory with embedded spaces in its
name. For each installation, he runs the full regression suite
from a DOS box, and both with and without -0.
He also tries *every* shortcut created under Start -> Menu -> the
Python group. When trying IDLE this way, you need to verify that
Help -> Python Documentation works. When trying pydoc this way
(the "Module Docs" Start menu entry), make sure the "Start
Browser" button works, and make sure you can search for a random
module (Tim uses "random" <wink>) and then that the "go to
selected" button works.
It's amazing how much can go wrong here -- and even more amazing
how often last-second checkins break one of these things. If
you're "the Windows geek", keep in mind that you're likely the
only person routinely testing on Windows, and that Windows is
simply a mess.
Repeat all of the above on at least one flavor of Win9x, and one
of NT/2000. On NT/2000, try both an Admin and a plain User (not
Power User) account.
WRT Step 5 above (verify the release media), since by the time
release files are ready to download Tim has generally run many
Windows tests on the installer he uploaded, he usually doesn't do
anything for Step 5 except a full byte-comparison ("fc /b" if
using a Windows shell) of the downloaded file against the file he
uploaded.
Copyright
This document has been placed in the public domain.
![[Python]](../pics/PyBanner001.gif){kind=small}