16 Other Language Changes

Here are all of the changes that Python 2.3 makes to the core Python language.

The yield statement is now always a keyword, as described in section 2 of this document.

A new built-in function enumerate() was added, as described in section 7 of this document.

Two new constants, True and False were added along with the built-in bool type, as described in section 9 of this document.

The int() type constructor will now return a long integer instead of raising an OverflowError when a string or floating-point number is too large to fit into an integer. This can lead to the paradoxical result that isinstance(int(expression), int) is false, but that seems unlikely to cause problems in practice.

Built-in types now support the extended slicing syntax, as described in section 15 of this document.

A new built-in function, sum(iterable, start=0), adds up the numeric items in the iterable object and returns their sum. sum() only accepts numbers, meaning that you can't use it to concatenate a bunch of strings. (Contributed by Alex Martelli.)

list.insert(pos, value) used to insert value at the front of the list when pos was negative. The behaviour has now been changed to be consistent with slice indexing, so when pos is -1 the value will be inserted before the last element, and so forth.

list.index(value), which searches for value within the list and returns its index, now takes optional start and stop arguments to limit the search to only part of the list.

Dictionaries have a new method, pop(key[, default]), that returns the value corresponding to key and removes that key/value pair from the dictionary. If the requested key isn't present in the dictionary, default is returned if it's specified and KeyError raised if it isn't.

>>> d = {1:2}
>>> d
{1: 2}
>>> d.pop(4)
Traceback (most recent call last):
  File "stdin", line 1, in ?
KeyError: 4
>>> d.pop(1)
2
>>> d.pop(1)
Traceback (most recent call last):
  File "stdin", line 1, in ?
KeyError: 'pop(): dictionary is empty'
>>> d
{}
>>>

There's also a new class method, dict.fromkeys(iterable, value), that creates a dictionary with keys taken from the supplied iterator iterable and all values set to value, defaulting to None.

(Patches contributed by Raymond Hettinger.)

Also, the dict() constructor now accepts keyword arguments to simplify creating small dictionaries:

>>> dict(red=1, blue=2, green=3, black=4)
{'blue': 2, 'black': 4, 'green': 3, 'red': 1}

(Contributed by Just van Rossum.)

The assert statement no longer checks the __debug__ flag, so you can no longer disable assertions by assigning to __debug__. Running Python with the -O switch will still generate code that doesn't execute any assertions.

Most type objects are now callable, so you can use them to create new objects such as functions, classes, and modules. (This means that the new module can be deprecated in a future Python version, because you can now use the type objects available in the types module.) For example, you can create a new module object with the following code:

>>> import types
>>> m = types.ModuleType('abc','docstring')
>>> m
<module 'abc' (built-in)>
>>> m.__doc__
'docstring'

A new warning, PendingDeprecationWarning was added to indicate features which are in the process of being deprecated. The warning will not be printed by default. To check for use of features that will be deprecated in the future, supply -Walways::PendingDeprecationWarning:: on the command line or use warnings.filterwarnings().

The process of deprecating string-based exceptions, as in raise "Error occurred", has begun. Raising a string will now trigger PendingDeprecationWarning.

Using None as a variable name will now result in a SyntaxWarning warning. In a future version of Python, None may finally become a keyword.

The xreadlines() method of file objects, introduced in Python 2.1, is no longer necessary because files now behave as their own iterator. xreadlines() was originally introduced as a faster way to loop over all the lines in a file, but now you can simply write for line in file_obj. File objects also have a new read-only encoding attribute that gives the encoding used by the file; Unicode strings written to the file will be automatically converted to bytes using the given encoding.

The method resolution order used by new-style classes has changed, though you'll only notice the difference if you have a really complicated inheritance hierarchy. Classic classes are unaffected by this change. Python 2.2 originally used a topological sort of a class's ancestors, but 2.3 now uses the C3 algorithm as described in the paper ``A Monotonic Superclass Linearization for Dylan''. To understand the motivation for this change, read Michele Simionato's article ``Python 2.3 Method Resolution Order'', or read the thread on python-dev starting with the message at http://mail.python.org/pipermail/python-dev/2002-October/029035.html. Samuele Pedroni first pointed out the problem and also implemented the fix by coding the C3 algorithm.

Python runs multithreaded programs by switching between threads after executing N bytecodes. The default value for N has been increased from 10 to 100 bytecodes, speeding up single-threaded applications by reducing the switching overhead. Some multithreaded applications may suffer slower response time, but that's easily fixed by setting the limit back to a lower number using sys.setcheckinterval(N). The limit can be retrieved with the new sys.getcheckinterval() function.

One minor but far-reaching change is that the names of extension types defined by the modules included with Python now contain the module and a "." in front of the type name. For example, in Python 2.2, if you created a socket and printed its __class__, you'd get this output:

>>> s = socket.socket()
>>> s.__class__
<type 'socket'>

In 2.3, you get this:

>>> s.__class__
<type '_socket.socket'>

One of the noted incompatibilities between old- and new-style classes has been removed: you can now assign to the __name__ and __bases__ attributes of new-style classes. There are some restrictions on what can be assigned to __bases__ along the lines of those relating to assigning to an instance's __class__ attribute.

16.1 String Changes

The in operator now works differently for strings. Previously, when evaluating X in Y where X and Y are strings, X could only be a single character. That's now changed; X can be a string of any length, and X in Y will return True if X is a substring of Y. If X is the empty string, the result is always True.

>>> 'ab' in 'abcd'
True
>>> 'ad' in 'abcd'
False
>>> '' in 'abcd'
True

Note that this doesn't tell you where the substring starts; if you need that information, use the find() string method.

The strip(), lstrip(), and rstrip() string methods now have an optional argument for specifying the characters to strip. The default is still to remove all whitespace characters:

>>> '   abc '.strip()
'abc'
>>> '><><abc<><><>'.strip('<>')
'abc'
>>> '><><abc<><><>\n'.strip('<>')
'abc<><><>\n'
>>> u'\u4000\u4001abc\u4000'.strip(u'\u4000')
u'\u4001abc'
>>>

(Suggested by Simon Brunning and implemented by Walter Dörwald.)

The startswith() and endswith() string methods now accept negative numbers for the start and end parameters.

Another new string method is zfill(), originally a function in the string module. zfill() pads a numeric string with zeros on the left until it's the specified width. Note that the % operator is still more flexible and powerful than zfill().

>>> '45'.zfill(4)
'0045'
>>> '12345'.zfill(4)
'12345'
>>> 'goofy'.zfill(6)
'0goofy'

(Contributed by Walter Dörwald.)

A new type object, basestring, has been added. Both 8-bit strings and Unicode strings inherit from this type, so isinstance(obj, basestring) will return True for either kind of string. It's a completely abstract type, so you can't create basestring instances.

Interned strings are no longer immortal and will now be garbage-collected in the usual way when the only reference to them is from the internal dictionary of interned strings. (Implemented by Oren Tirosh.)

16.2 Optimizations

The creation of new-style class instances has been made much faster; they're now faster than classic classes!

The sort() method of list objects has been extensively rewritten by Tim Peters, and the implementation is significantly faster.

Multiplication of large long integers is now much faster thanks to an implementation of Karatsuba multiplication, an algorithm that scales better than the O(n*n) required for the grade-school multiplication algorithm. (Original patch by Christopher A. Craig, and significantly reworked by Tim Peters.)

The SET_LINENO opcode is now gone. This may provide a small speed increase, depending on your compiler's idiosyncrasies. See section 20 for a longer explanation. (Removed by Michael Hudson.)

xrange() objects now have their own iterator, making for i in xrange(n) slightly faster than for i in range(n). (Patch by Raymond Hettinger.)

A number of small rearrangements have been made in various hotspots to improve performance, such as inlining a function or removing some code. (Implemented mostly by GvR, but lots of people have contributed single changes.)

The net result of the 2.3 optimizations is that Python 2.3 runs the pystone benchmark around 25% faster than Python 2.2.

See About this document... for information on suggesting changes.