Python Programming Environment - Session Overview

Ken Manheimer, ken.manheimer@nist.gov, 19-May-1995

Session Purpose

As a Python programmer, i need tools that will help me capitalize on python's versatility, and on the growing, increasingly diverse contingent of software libraries. As Python grows, i wish to help promote the development of such tools. That's what i see as the focus of this session.

(Two personal traits particularly shape my own programming environment concerns:

  • I'm sorta lazy - like most people, i want to work effectively, and avoid unnecessary work.
  • My memory for names, locations, and such, is pretty feeble, particularly when compared with the reliability of the computer's "memory". So i would like, as much as possible, to depend on the computer to keep track of the location of things...

    So my interests tend towards tools and conventions that help organize and track new and existing software components and tools, to help me find and reuse what's already there.)

    Some Current Programming Environment Issues

    The following list is not meant to be a comprehensive survey of the important issues in software management tools. Rather, it includes some items that i see as being of current relevance. Suggestions of other items to discuss are welcome.

    (The list specifically includes a few issues that i promised to pursue during the last workshop. Other items are motivated partly by my own continuing software mgmt tools concerns and interests, and partly by suggestions and questions that i have seen go by on the list.)

    Object identification - docstrings, attribute identification

    Proposals that we discussed at the last workshop have made their way, with refinements, into the language. Also, a new implementation of the 'dir()' builtin has been added to the language as an alternative, within the module 'newdir.py'. Ultimately, these are the sorts of things that are necessary in the development of a comprehensive object examination system, as part of an environment browser.

    Docstrings

    Docstrings are brief descriptive strings that can be associated with many of the complex python objects, by assigning to those objects '__doc__' attribute. In addition, the definition syntax for callables (functions, methods, and classes) and modules include optional, formal slots for docstring declarations. In particular, when the first executable statement of a callable or a module is a string literal, that string becomes the value of the callable's __doc__ property.

    We have only started to formulate conventions about the format of docstrings. As with GNU emacs docstrings, we suggest that the first line of the string be a self-contained, very terse description of the purpose of the object, and subsequent lines describe the public properties and behaviors of the object.

    For an extensive example of the use of docstrings, see the newimp.py module which is in the distribution library for Python 1.2.

    Where to go from here with docstrings? Several items suggest themselves:

    Of course, there are more. Suggestions?

    Import module packages/nesting

    Also as promised at the last workshop, i have designed, proposed, and contributed a working prototype which implements module-nesting "packages". The prototype is included in the Python 1.2 distribution, in the module 'newimp.py'.

    Description of the new import enhancements

    Why packages? Packages enable module nesting and sibling module imports. 'Til now, the python module namespace was flat, which means every module had to have a unique name, in order to not conflict with names of other modules on the load path. Furthermore, suites of modules could not be structurally affiliated with one another.

    With packages, a suite of, eg, email-oriented modules can include a module named 'mailbox', without conflicting with the, eg, 'mailbox' module of a shared-memory suite - 'email.mailbox' vs 'shmem.mailbox'. Packages also enable modules within a suite to load other modules within their package without having the package name hard-coded. Similarly, package suites of modules can be loaded as a unit, by loading the package that contains them.

    Usage: once installed (newimp.install(); newimp.revert() to revert to the prior __import__ routine), 'import ...' and 'from ... import ...' can be used to:

    Difference between new import mechanism and Java's surprisingly similar import mechanism

    As Guido mentions in his Works In Progress page, Sun's new web language, Java, provides a very similar module-nesting capability. However, there is a subtle difference that turns out to be important.

    As Guido succinctly puts it: 

       Java got this wrong -- their "import A.B" creates a local name B, so
   it is hard to use two unrelated packages that happen to define a
   module with the same name, like "P.main" and "Q.main"
    ... because both P.main and Q.main would try to populate the current namespace with the name 'main', and hence conflict.

    In the new python mechanism, the 'import P.main' would establish module P in the current namespace. You would then reach the 'P's component, 'main', as 'P.main', without colliding with the 'Q's 'main' component.

    Furthermore, Python's 'from X import Y' form allows you to deliberately import a component of a module directly into the current namespace, when you specifically wish. I'm not sure whether or not Java has an equivalent of the 'from' form, which might explain why they made the choice they did.

    Some further prospects for import enhancements.

    I would like to have the import mechanism make it easy to hook in alternate search and load mechanisms, to, eg, enable easy hookup with the web and other media. This would essentially entail making the relationship of the load mechanisms to the module name, or file-type, keyed through a table lookup. Developers of new import types could then hook in their special load mechanisms by adding new entries to the appropriate places in the tables. For instance: Restructuring the code to accomplish this sort of thing would take some attention, however, and i would like to find a volunteer with time and motivation to take it on.

    Runtime environment - debuggers, profilers, env browsers

    In thinking about the prospects for advanced environment browsers for python, i've reached an interesting branch-point. I see two primary and divergent options for embedding the user interface for such tools:
    1. With all the attention on it, we could plan to use a python GUI mechanism, as it emerges. As, eg, Guido embedded the wdb window debugger interface in stdwin.
    2. Alternately, we could work immediately on embedding such interfaces in emacs.
    I may be biased, because i am both an intensive emacs user and emacs hacker, but i see some very compelling advantages to using emacs as the runtime-tools interface substrate. The primary disadvantage is the same one that always comes with emacs involvement - it's complicated for the user.

    It also is sort-of one-way - it would be a good (i think) substrate for embedding our programming-environment tools, but would not be so good as a basis for a python GUI toolkit. (Or, at least, not without some mondo hacking, to make an emacs "API" available from within python!) (Did i hear someone gasp?-) My inclination is to recommend both avenues - emacs immediately, and elegant-and-easy-to-use GUI interfaces as they coelesce. I don't think the different substrates will interfere with eachother, each having somewhat divergent audiences - user-friendly point-and-click with the GUI's, vs programmer-friendly (and user fiendish:-) customize-and-automate-out-the-wazoo of emacs.

    With the subject broached, i figure i might as well take the opportunity to mention some of the emacs interfaces which i see as useful, exciting prospects:

    I have to mention one other, non-emacs-affiliated option, re code and document indexing: the Glimpse indexing and query system. It looks very promising and useful in it's own right, and is easy to use. And it's even more intriguing when you consider the potentials inherent in it's connection with the Harvest network information discovery and access system.

    Documentation formats and efforts

    1. Do we need, and can we muster a documentation project, a la Linux?

    And,

    2. What format should we primarily support?

    I haven't thought about this very much, but have increasingly hit up against the second issue. I figure the issue is quite ripe, and just yearning to be resolved. Some offhand thoughts:

    Programming style practices

    Here's the beginnings of a grab-bag of emerging coding style and conventions, from a discussion with Guido.

    Perhaps we can start accumulating a list. I'd be willing to include it in my Python bestiary, if i ever get time to update it...