1   Frequently Asked Questions for Developers

Date:$Date: 2006-01-03 03:01:22 +0100 (Tue, 03 Jan 2006) $
Version:$Revision: 8753 $
Web site:http://www.python.org/dev/devfaq.html

Contents

1.1   General Information

1.1.1   What is SourceForge?

SourceForge is a free hosting service for open source projects. The main website is found at http://sourceforge.net.

1.1.3   Who is who? What is a nowonder?

To get the real name, click on "View members" in the "Developer Info" box on the SourceForge project page. This presents a list of developers, giving their names and SourceForge IDs. A separate list of committers lists people with commit privileges to the Subversion repository.

1.1.4   How can I become a developer?

There's only one way to become a developer, and that's through the School of Hard Knocks.

1.2   Subversion (svn)

1.2.1   Where can I learn about Subversion (svn)?

Subversion has its official web site at http://subversion.tigris.org/ (it is also known as svn thanks to that being the name of the executable of Subversion itself). A book on Subversion published by O'Reilly Media, Version Control with Subversion, is available for free online.

With Subversion installed, you can run the help tool that comes with Subversion to get help:

svn help

This will give you the needed information to use the tool. The man page for svn is rather scant and thus not worth reading.

1.2.2   What do I need to use Subversion?
1.2.2.1   UNIX

First, you need to download Subversion. Most UNIX-based operating systems have binary packages available for Subversion. Also, most package systems also have Subversion available.

If you have checkin rights, you need OpenSSH. This is needed to verify your identity when performing commits.

1.2.2.2   Windows

You have several options on Windows. One is to download Subversion itself which will give you a command-line version. Another option is to download TortoiseSVN which integrates with Windows Explorer.

If you have checkin rights, you will also need an SSH client. Download PuTTY and friends (PuTTYgen, Pageant, and Plink) for this. All other questions in this FAQ will assume you are using these tools.

Once you have both Subversion and PuTTY installed you must tell Subversion where to find an SSH client. Do this by editing %APPDATA%\Subversion\config (on Win9x, this might be c:\windows\Application Data\Subversionconfig) to have the following section:

[tunnels]
ssh="c:/path/to/putty/plink.exe" -T

Obviously change the path to be the proper one for your system. The -T option prevents a pseudo-terminal from being created.

You can use Pageant to prevent from having to type in your password for your SSH 2 key constantly (for how to do this, see XXX). If you prefer not to have another program running, you need to create a profile in PuTTY.

Go to Session:Saved Sessions and create a new profile named svn.python.org. In Session:Hose Name, enter svn.python.org. In SSH/Auth:Private key file select your private key. In Connection:Auto-login username enter pythondev.

With this set up, paths are slightly different than most other settings in that the username is not required. Do take notice of this when choosing to check out a project!

1.2.3   How do I get a checkout of the repository (read-only and read-write)?

Regardless of whether you are checking out a read-only or read-write version of the repository, the basic command is the same:

svn checkout <URL> [PATH]

<URL> is the specified location of the project within the repository that you would like to check out (those paths are discussed later). The optional [PATH] argument specifies the local directory to put the checkout into. If left out then the tail part of <URL> is used for the directory name.

For a read-only checkout, the format of <URL> is:

http://svn.python.org/projects/<path>

with <path> representing the path to the project. A list of projects can be viewed at http://svn.python.org/view/ . Do note that any subdirectory may also be checked out individually.

For a read-write checkout (with a caveat for Windows users using PuTTY without Pageant), the format for <URL> is:

svn+ssh://pythondev@python.org/<path>

There are three critical differences between a read-only URL and a read-write URL. One is the protocol being specified as svn+ssh and not http. Next, notice the addition of the username pythondev (note that for Windows users using PuTTY without Pageant should leave off pythondev@ if PuTTY was set up following the instructions in this FAQ). Lastly, note that project was removed from the path entirely for a read-write checkout.

The Subversion repository has many projects under it. Chances are, though, most people will be interested in the trunk (the current version under development), the latest release of Python, and the PEPs (Python Enhancement Proposals). For a read-only checkout, those three projects can be checked out using the following <URL> paths , respectively, of:

For read-write checkouts (which require that you have commit privileges), the URLs for the same projects are:

  • svn+ssh://pythondev@svn.python.org/python/trunk
  • svn+ssh://pythondev@svn.python.org/python/branches/release24-maint
  • svn+ssh://pythondev@svn.python.org/peps/trunk

Obviously the note above checking out released versions of Python applies to read-write checkouts just as it did for read-only ones.

1.2.4   How do I update my working copy to be in sync with the repository?

Run:

svn update

from the directory you wish to update (and all subdirectories).

1.2.6   Where can I find a downloadable snapshot of the source code?

Visit http://svn.python.org/snapshots/ to download a tarball containing a daily snapshot of the repository.

1.2.8   How do I get commit privileges on the svn repository if I had the same privileges on the CVS repository?

If you have established commit privileges as listed at http://sourceforge.net/project/memberlist.php?group_id=5470 but do not have them yet for the Subversion repository, then there are a few steps you must take:

XXX add links to Q for each point

  1. Generate an SSH 2 public key
  2. Email key and name (first and last required, middle optional)
  3. Wait for verification email that the key has been installed
  4. Verify access
1.2.9   How do I verify that my commit privileges are working?
1.2.9.1   UNIX

If you are listed as a committer at http://www.python.org/dev/committers , then you should be able to execute:

ssh pythondev@svn.python.org

and have the following print out to your terminal:

( success ( 1 2 ( ANONYMOUS EXTERNAL ) ( edit-pipeline ) ) )

If something else is printed, then there is a problem with your SSH 2 public key and you should contact pydotorg at python dot org .

1.2.9.2   Windows

If you are using Pageant (discussed in XXX), you can verify that your SSH 2 key is set up properly by running:

c:\path\to\putty\plink.exe pythondev@svn.python.org

Using the proper path to your PuTTY installation, you should get a response from the server that says:

( success ( 1 2 ( ANONYMOUS EXTERNAL ) ( edit-pipeline ) ) )

If there is a failure, run plink with -v to analyse the problem.

If you are using a profile in PuTTY, the best way to test is to try to log in through Open.

1.2.10   What configuration settings should I use?

Make sure the following setttings are in your Subversion config file (~/.subversion/config under UNIX):

[miscellany]
enable-auto-props = yes

[auto-props]
* = svn:eol-style=native
*.c = svn:keywords=Id
*.h = svn:keywords=Id
*.py = svn:keywords=Id
*.txt = svn:keywords=Author Date Id Revision

The [auto-props] line specifies the beginning of the section in the config file. The svn:eol-style setting tells Subversion to check out files using the native line endings on your OS. It will also automatically convert line endings upon committal so that they are consistent across all platforms. The svn:keywords settings are to automatically substitute $keyword$ arguments in files that match the pattern. *.txt has more options so as to cover all needed keywords for PEPs.

The [miscellany] section and its one option make Subversion apply the various rules in the [auto-props] section automatically to all added or imported files into the respository.

1.2.11   How do I add a file or directory to the repository?

Simply specify the path to the file or directory to add and run:

svn add PATH

Subversion will skip any directories it already knows about. But if you want new files that exist in any directories specified in PATH, specify --force and Subversion will check all directories for new files.

You will then need to run svn commit (as discussed in XXX) to commit the file to the repository.

1.2.12   How do I commit a change to a file?

To have any changes to a file (which include adding a new file or deleting an existing one), you use the command:

svn commit [PATH]

[PATH] is optional and will commit all changes in your local copy to the repository. DO NOT USE THIS!!! You should specify the specific files to be committed unless you are absolutely positive that everything that will be committed is meant to go in this commit.

To abort a commit that you are in the middle of, perform a commit with no message (i.e., close the text editor without adding any text for the message). Subversion will ask if you want to abort the commit or not at that point.

If you do not like the default text editor Subversion is having you use for entering commmit message, you may specify if in your config file with the editor-cmd option in the [helpers] section.

1.2.13   How do I delete a file or directory in the repository?

Specify the path to be removed with:

svn delete PATH

Any modified files or that are not checked in will not be deleted in the working copy on your machine.

1.2.14   What files are modified locally in my working copy?

Running:

svn status [PATH]

will list any differences between your working copy and the repository. Some key indicators that can appear in the first column of output are:

  • A

    Scheduled to be added

  • D

    Scheduled to be deleted

  • M

    Modified locally

  • ?

    Not under version control

1.2.16   How do I revert a file I have modified back to the version in the respository?

Running:

svn revert PATH

will change PATH to match the version in the repository, throwing away any changes you made locally. If you run:

svn revert -R

from the root of your local repository it will recursively restore everything to match up with the main server.

1.2.17   How do I find out who edited or what revision changed a line last?

You want:

svn blame PATH

This will output to stdout every line of the file along with what revision number last touched that line and who committed that revision. Since it is printed to stdout, you probably want to pipe the output to a pager:

svn blame PATH | less
1.2.18   How can I see a list of log messages for a file or specific revision?

To see the log messages for a specific file, run:

svn log PATH

That will list all messages that pertain to the file specified in PATH.

If you want to view the log message for a specific revision, run:

svn log --verbose -r REV

With REV substituted with the revision number. The --verbose flag should be used to get a listing of all files modified in that revision.

1.2.19   How can I edit the log message of a committed revision?

Use:

svn propedit -r <revision> --revprop svn:log

Replace <revision> with the revision number of the commit whose log message you wish to change.

1.2.20   How do I get a diff between the repository and my working copy for a file?

The diff between your working copy and what is in the repository can be had with:

svn diff PATH

This will work off the current revision in the repository. To diff your working copy with a specific revision, do:

svn diff -r REV PATH

Finally, to generate a diff between two specific revisions, use:

svn diff -r REV1:REV2 PATH

Notice the : between REV1 and REV2.

1.2.21   How do I undo the changes made in a recent committal?

Assuming your bad revision is NEW and OLD is the equivalent of NEW - 1, then run:

svn merge -r NEW:OLD PATH

This will revert all files back to their state in revision OLD. The reason that OLD is just NEW - 1 is you do not want files to be accidentally reverted to a state older than your changes, just to the point prior.

1.3   SSH

1.3.1   How do I generate an SSH 2 public key?
1.3.1.1   UNIX

Run:

ssh-keygen -t dsa

This will generate a two files; your public key and your private key. Your public key is the file ending in .pub.

1.3.1.2   Windows

Use PuTTYgen to generate your public key. Choose the "SSH2 DSA" radio button, have it create an OpenSSH formatted key, choose a password, and save the private key to a file. Copy the section with the public key (using Alt-P) to a file; that file now has your public key.

1.3.2   Is there a way to prevent from having to enter my password for my SSH 2 public key constantly?
1.3.2.1   UNIX

Use ssh-agent and ssh-add to register your private key with SSH for your current session. The simplest solution, though, is to use KeyChain, which is a shell script that will handle ssh-agent and ssh-add for you once per login instead of per session.

1.3.2.2   Windows

Running Pageant will prevent you from having to type your password constantly. If you add a shortcut to Pageant to your Autostart group and edit the shortcut so that the command line includes an argument to your private key then Pageant will load the key every time you log in.

1.4   Patches

1.4.1   How to make a patch?

If you are using CVS (anonymous or developer) you can use CVS to make the patches for you. Just edit your local copy and enter the following command:

svn diff | tee ~/name_of_the_patch.diff

Else you can use the diff util which comes with most operating systems (a Windows version is available as part of the cygwin tools).

1.4.2   How do I apply a patch?

For the general case, to apply a patch go to the directory that the patch was created from (usually /dist/src/) and run:

patch -p0 < name_of_the_patch.diff

The -p option specifies the number of directory separators ("/" in the case of UNIX) to remove from the paths of the files in the patch. -p0 leaves the paths alone.

1.4.3   How do I undo an applied patch?

Undoing a patch differs from applying one by only a command-line option:

patch -R -p0 < name_of_the_patch.diff

Another option is to have 'patch' create backups of all files by using the -b command-line option. See the man page for 'patch' on the details of use.

1.4.4   How to submit a patch?

Please consult the patch submission guidelines at http://www.python.org/patches/ .

1.4.5   How to test a patch?

Start by testing on your own system. Python comes with an extensive regression test suite by running the Lib/test/regrtest.py script. For more thorough documentation, read the documentation for the test package at http://www.python.org/doc/lib/module-test.html.

If you suspect the patch may impact other operating systems, test as many as you have easy access to. You can get help on alternate platforms by contacting the people listed on http://www.python.org/moin/PythonTesters, who have volunteered to support a particular operating system.

1.4.6   How to change the status of a patch?

To change the status of a patch or assign it to somebody else you have to be a) a SourceForge-recognized Python developer and b) a patch administrator. Unfortunately the SourceForge default for developers is not to be patch administrators. Contact one of the project administrators if the following does not work for you.

Click on the patch itself. In the screen that comes up, there is a drop-box for "Assigned To:" and a drop-box for "Status:" where you can select a new responsible developer or a new status respectively. After selecting the appropriate victim and status, hit the "Submit Changes" button at the bottom of the page.

Note: If you are sure that you have the right permissions and a drop-box does not appear, check that you are actually logged in to SourceForge!

For more information about the use of the "Status:" and "Assigned To:" fields consult the Patch Manager Guidelines; A recent copy can be found in the Appendix of this FAQ.

1.5   Bugs

1.5.1   Where can I submit/view bugs for Python?

The Python project uses SourceForge's bug tracking facility. Go to http://sourceforge.net/bugs/?group_id=5470 for all bug management needs. You will need to create a SourceForge account for yourself before submitting the first bug report; anonymous reports have been disabled since it was too difficult to get in contact with submitters.

1.5.2   How do I use the sourceforge bug manager?

By default, you will see the list of all Open bugs. You can change which bugs you're viewing by selecting the assigned_to/status/area/type select boxes.

To submit a bug, use the "Submit a Bug" link, near the top of the page.

1.6   Appendix A

1.6.1   Patch Manager Guidelines

Intended use of SourceForge patch Resolution, Status & "Assigned To" fields Revision 4 16-Mar-2001

In general, the Resolution and Status fields should be close to self-explanatory, and the "Assigned to:" field should be the person responsible for taking the next step in the patch process. Both fields are expected to change value over the life of a patch; the normal workflow is detailed below.

When you've got the time and the ability, feel free to move any patch that catches your eye along, whether or not it's been assigned to you. And if you're assigned to a patch but aren't going to take reasonably quick action (for whatever reason), please assign it to someone else ASAP: at those times you can't actively help, actively get out of the way.

If you're an expert in some area and know that a patch in that area is both needed and non-controversial, just commit your changes directly -- no need then to get the patch mechanism involved in it.

You should add a comment to every patch assigned to you at least once a week, if only to say that you realize it's still on your plate. This rule is meant to force your attention periodically: patches get harder & harder to deal with the longer they sit.

1.6.1.1   Status Open, Resolution None

The initial state of all patches. The patch is under consideration, but has not been reviewed yet, or s under review but not yet Accepted or Rejected.

The Resolution will normally change to Accepted or Rejected next. The person submitting the patch should (if they can) assign it to the person they most want to review it.

Else the patch will be assigned via [xxx a list of expertise areas should be developed] [xxx but since this hasn't happened and volunteers are too few, andom assignment is better than nothing: if you're a Python developer, xpect to get assigned out of the blue!]

Discussion of major patches is carried out on the Python-Dev mailing list. For simple patches, the SourceForge comment mechanism should be sufficient. [xxx an email gateway would be great, ditto Ping's Roundup] For the reviewer: If you're certain the patch should be applied, change the Resolution to Accepted and assign it back to the submitter (if possible) for checkin. If you're certain the patch should never be accepted, change the Resolution to Rejected, Status to Closed, and assign it to None.

If you have specific complaints that would cause you to change your mind, explain them clearly in a comment, leave the status Open, and reassign back to the submitter. If you're uncertain, leave the status Open, explain your uncertainies in a comment, and reassign the patch to someone you believe can address your remaining questions; or leave the status Open and bring it up on Python-Dev.

1.6.1.2   Status Open, Resolution Accepted

The powers that be accepted the patch, but it hasn't been applied yet. [xxx flesh out -- Guido Bottleneck avoidable here?]

The Status will normally change to Closed next.

The person changing the Resolution to Accepted should, at the same time, assign the patch to whoever they believe is most likely to be able & willing to apply it (the submitter if possible).

1.6.1.3   Status Closed, Resolution Accepted

The patch has been accepted and applied.

The previous Resolution was Accepted, or possibly None if the submitter was Guido (or moral equivalent in some particular area of expertise).

1.6.1.4   Status Closed, Resolution Rejected

The patch has been reviewed and rejected.

There are generally no transitions out of this state: the patch is dead.

The person setting this state should also assign the patch to None.

1.6.1.5   Status Open, Resolution Out of date

revious Resolution was Accepted or Postponed, but the patch no longer works.

Please enter a comment when changing the Resolution to "Out of date", to record the nature of the problem and the previous state.

Also assign it back to the submitter, as they need to upload a new version.

1.6.1.6   Status Open, Resolution Postponed

The previous Resolution was None or Accepted, but for some reason (e.g., pending release) the patch should not be reviewed or applied until further notice.

The Resolution will normally change to None or Accepted next.

Please enter a comment when changing the Resolution to Postponed, to record the reason, the previous Resolution, and the conditions under which the patch should revert to Resolution None or Accepted. Also assign the patch to whoever is most likely able and willing to decide when the state should change again.

1.6.1.7   Status Deleted

Bit bucket.

Use only if it's OK for the patch and its SourceForge history to disappear. As of 09-July-2000, SF does not actually throw away Deleted patches, but that may change.