1.10.1 Reference Counting in Python

There are two macros, Py_INCREF(x) and Py_DECREF(x), which handle the incrementing and decrementing of the reference count. Py_DECREF() also frees the object when the count reaches zero. For flexibility, it doesn't call free() directly -- rather, it makes a call through a function pointer in the object's type object. For this purpose (and others), every object also contains a pointer to its type object.

The big question now remains: when to use Py_INCREF(x) and Py_DECREF(x)? Let's first introduce some terms. Nobody ``owns'' an object; however, you can own a reference to an object. An object's reference count is now defined as the number of owned references to it. The owner of a reference is responsible for calling Py_DECREF() when the reference is no longer needed. Ownership of a reference can be transferred. There are three ways to dispose of an owned reference: pass it on, store it, or call Py_DECREF(). Forgetting to dispose of an owned reference creates a memory leak.

It is also possible to borrow^1.2 a reference to an object. The borrower of a reference should not call Py_DECREF(). The borrower must not hold on to the object longer than the owner from which it was borrowed. Using a borrowed reference after the owner has disposed of it risks using freed memory and should be avoided completely.^1.3

The advantage of borrowing over owning a reference is that you don't need to take care of disposing of the reference on all possible paths through the code -- in other words, with a borrowed reference you don't run the risk of leaking when a premature exit is taken. The disadvantage of borrowing over leaking is that there are some subtle situations where in seemingly correct code a borrowed reference can be used after the owner from which it was borrowed has in fact disposed of it.

A borrowed reference can be changed into an owned reference by calling Py_INCREF(). This does not affect the status of the owner from which the reference was borrowed -- it creates a new owned reference, and gives full owner responsibilities (i.e., the new owner must dispose of the reference properly, as well as the previous owner).

Footnotes

...borrow^1.2

The metaphor of ``borrowing'' a reference is not completely correct: the owner still has a copy of the reference.

... completely.^1.3

Checking that the reference count is at least 1 does not work -- the reference count itself could be in freed memory and may thus be reused for another object!