5.11.1 shlex Objects

A shlex instance has the following methods:

get_token ()

Return a token. If tokens have been stacked using push_token(), pop a token off the stack. Otherwise, read one from the input stream. If reading encounters an immediate end-of-file, an empty string is returned.

push_token (str)

Push the argument onto the token stack.

Instances of shlex subclasses have some public instance variables which either control lexical analysis or can be used for debugging:

commenters

The string of characters that are recognized as comment beginners. All characters from the comment beginner to end of line are ignored. Includes just "#" by default.

wordchars

The string of characters that will accumulate into multi-character tokens. By default, includes all ASCII alphanumerics and underscore.

whitespace

Characters that will be considered whitespace and skipped. Whitespace bounds tokens. By default, includes space, tab, linefeed and carriage-return.

quotes

Characters that will be considered string quotes. The token accumulates until the same quote is encountered again (thus, different quote types protect each other as in the shall.) By default, includes ASCII single and double quotes.

Note that any character not declared to be a word character, whitespace, or a quote will be returned as a single-character token.

Quote and comment characters are not recognized within words. Thus, the bare words "ain't" and "ain#t" would be returned as single tokens by the default parser.

lineno

Source line number (count of newlines seen so far plus one).

token

The token buffer. It may be useful to examine this when catching exceptions.