2.1 The Basics

The Python runtime sees all Python objects as variables of type PyObject*. A PyObject is not a very magnificent object - it just contains the refcount and a pointer to the object's ``type object''. This is where the action is; the type object determines which (C) functions get called when, for instance, an attribute gets looked up on an object or it is multiplied by another object. I call these C functions ``type methods'' to distinguish them from things like [].append (which I will call ``object methods'' when I get around to them).

So, if you want to define a new object type, you need to create a new type object.

This sort of thing can only be explained by example, so here's a minimal, but complete, module that defines a new type:

#include <Python.h>

staticforward PyTypeObject noddy_NoddyType;

typedef struct {
    PyObject_HEAD
} noddy_NoddyObject;

static PyObject*
noddy_new_noddy(PyObject* self, PyObject* args)
{
    noddy_NoddyObject* noddy;

    if (!PyArg_ParseTuple(args,":new_noddy")) 
        return NULL;

    noddy = PyObject_New(noddy_NoddyObject, &noddy_NoddyType);

    return (PyObject*)noddy;
}

static void
noddy_noddy_dealloc(PyObject* self)
{
    PyObject_Del(self);
}

static PyTypeObject noddy_NoddyType = {
    PyObject_HEAD_INIT(NULL)
    0,
    "Noddy",
    sizeof(noddy_NoddyObject),
    0,
    noddy_noddy_dealloc, /*tp_dealloc*/
    0,          /*tp_print*/
    0,          /*tp_getattr*/
    0,          /*tp_setattr*/
    0,          /*tp_compare*/
    0,          /*tp_repr*/
    0,          /*tp_as_number*/
    0,          /*tp_as_sequence*/
    0,          /*tp_as_mapping*/
    0,          /*tp_hash */
};

static PyMethodDef noddy_methods[] = {
    { "new_noddy", noddy_new_noddy, METH_VARARGS },
    {NULL, NULL}
};

DL_EXPORT(void)
initnoddy(void) 
{
    noddy_NoddyType.ob_type = &PyType_Type;

    Py_InitModule("noddy", noddy_methods);
}

Now that's quite a bit to take in at once, but hopefully bits will seem familiar from the last chapter.

The first bit that will be new is:

staticforward PyTypeObject noddy_NoddyType;

This names the type object that will be defining further down in the file. It can't be defined here because its definition has to refer to functions that have no yet been defined, but we need to be able to refer to it, hence the declaration.

The staticforward is required to placate various brain dead compilers.

typedef struct {
    PyObject_HEAD
} noddy_NoddyObject;

This is what a Noddy object will contain. In this case nothing more than every Python object contains - a refcount and a pointer to a type object. These are the fields the PyObject_HEAD macro brings in. The reason for the macro is to standardize the layout and to enable special debugging fields to be brought in debug builds.

For contrast

typedef struct {
    PyObject_HEAD
    long ob_ival;
} PyIntObject;

is the corresponding definition for standard Python integers.

Next up is:

static PyObject*
noddy_new_noddy(PyObject* self, PyObject* args)
{
    noddy_NoddyObject* noddy;

    if (!PyArg_ParseTuple(args,":new_noddy")) 
        return NULL;

    noddy = PyObject_New(noddy_NoddyObject, &noddy_NoddyType);

    return (PyObject*)noddy;
}

This is in fact just a regular module function, as described in the last chapter. The reason it gets special mention is that this is where we create our Noddy object. Defining PyTypeObject structures is all very well, but if there's no way to actually create one of the wretched things it is not going to do anyone much good.

Almost always, you create objects with a call of the form:

PyObject_New(<type>, &<type object>);

This allocates the memory and then initializes the object (i.e. sets the reference count to one, makes the ob_type pointer point at the right place and maybe some other stuff, depending on build options). You can do these steps separately if you have some reason to -- but at this level we don't bother.

We cast the return value to a PyObject* because that's what the Python runtime expects. This is safe because of guarantees about the layout of structures in the C standard, and is a fairly common C programming trick. One could declare noddy_new_noddy to return a noddy_NoddyObject* and then put a cast in the definition of noddy_methods further down the file -- it doesn't make much difference.

Now a Noddy object doesn't do very much and so doesn't need to implement many type methods. One you can't avoid is handling deallocation, so we find

static void
noddy_noddy_dealloc(PyObject* self)
{
    PyObject_Del(self);
}

This is so short as to be self explanatory. This function will be called when the reference count on a Noddy object reaches 0 (or it is found as part of an unreachable cycle by the cyclic garbage collector). PyObject_Del() is what you call when you want an object to go away. If a Noddy object held references to other Python objects, one would decref them here.

Moving on, we come to the crunch -- the type object.

static PyTypeObject noddy_NoddyType = {
    PyObject_HEAD_INIT(NULL)
    0,
    "Noddy",
    sizeof(noddy_NoddyObject),
    0,
    noddy_noddy_dealloc, /*tp_dealloc*/
    0,                   /*tp_print*/
    0,                   /*tp_getattr*/
    0,                   /*tp_setattr*/
    0,                   /*tp_compare*/
    0,                   /*tp_repr*/
    0,                   /*tp_as_number*/
    0,                   /*tp_as_sequence*/
    0,                   /*tp_as_mapping*/
    0,                   /*tp_hash */
};

Now if you go and look up the definition of PyTypeObject in object.h you'll see that it has many, many more fields that the definition above. The remaining fields will be filled with zeros by the C compiler, and it's common practice to not specify them explicitly unless you need them.

This is so important that I'm going to pick the top of it apart still further:

    PyObject_HEAD_INIT(NULL)

This line is a bit of a wart; what we'd like to write is:

    PyObject_HEAD_INIT(&PyType_Type)

as the type of a type object is ``type'', but this isn't strictly conforming C and some compilers complain. So instead we fill in the ob_type field of noddy_NoddyType at the earliest oppourtunity -- in initnoddy().

0,

XXX why does the type info struct start PyObject_*VAR*_HEAD??

    "Noddy",

The name of our type. This will appear in the default textual representation of our objects and in some error messages, for example:

>>> "" + noddy.new_noddy()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: cannot add type "Noddy" to string

    sizeof(noddy_NoddyObject),

This is so that Python knows how much memory to allocate when you call PyObject_New.

0,

This has to do with variable length objects like lists and strings. Ignore for now...

Now we get into the type methods, the things that make your objects different from the others. Of course, the Noddy object doesn't implement many of these, but as mentioned above you have to implement the deallocation function.

    noddy_noddy_dealloc, /*tp_dealloc*/

From here, all the type methods are nil so I won't go over them yet - that's for the next section!

Everything else in the file should be familiar, except for this line in initnoddy:

    noddy_NoddyType.ob_type = &PyType_Type;

This was alluded to above -- the noddy_NoddyType object should have type ``type'', but &PyType_Type is not constant and so can't be used in its initializer. To work around this, we patch it up in the module initialization.

That's it! All that remains is to build it; put the above code in a file called noddymodule.c and

from distutils.core import setup, Extension
setup(name = "noddy", version = "1.0",
    ext_modules = [Extension("noddy", ["noddymodule.c"])])

in a file called setup.py; then typing

$ python setup.py build%$

at a shell should produce a file noddy.so in a subdirectory; move to that directory and fire up Python -- you should be able to import noddy and play around with Noddy objects.

That wasn't so hard, was it?

See About this document... for information on suggesting changes.