Author:	Jim Fulton, Zope Corporation, jim@zope.com

Abstract

Developers often need to extend the processing of existing functions or methods. This paper discusses how an event system can be used to provide this extensibility. Zope has a layered event system, built on an extremely simple policy-free base. Additional layers, built on this base layer, progressively add additional policy. Zope provides an adapter-based subscription framework that provides a simple rule-based system. This paper presents this framework, with examples showing both how events are used and how the event system is used to extend itself.

Object extensibility

There are a number of ways we can extend objects. We can use inheritance or adaptation to add or override methods or attributes. Sometimes, rather than adding or replacing a method, we need to add extra processing as a method is executing. We want to add processing at various points in a method's execution.

Sometimes, there is a need to extend what happens inside individual methods. For example, when an object is added to a system, we might record meta data or index the object's data or meta data. The actions we want to take are likely to be application dependent. It shouldn't be the responsibility of the code that adds the object to take these extra actions.

There are a number of ways to add processing as a method executes:

Callbacks

The caller of a method can pass functions to be called at various times in the processing. This doesn't work very well if many callbacks are needed, and it's inflexible, because only one function can be passed for each callback.
Hooks

The object with the method can provide an API for setting hooks to be called when certain things happen. These are basically callbacks that are set independently. This is more flexible than passing callbacks to the application method. It's straightforward to set as many and only as many hooks as needed, but as with callbacks, only one function can be provided per hook.
Events

Events are similar to hooks, except that multiple callback functions, called "subscribers" can be provided for each event. In addition, subscribers are registered by event type, and event types are arranged in a multiple-inheritance graph. Subscribers registered for general types are called for many specific types of events.

The Zope event system, which can be easily used outside of Zope provides an extensible event system that is composed of layers, starting with a very simple policy-free layer on which additional adapter-based layers are provided.

Events and subscribers

Events are objects that define plug points for additional processing. Application code identifies situations where other code can execute by publishing events:

>>> import zope.event
>>> event = "Something happened"
>>> zope.event.notify(event)

Events can be any kind of object. We used a string in the example above, although events are usually instances that provide data useful to subscribers.

The base event system

The Zope event system is built up in layers. The bottom layer is an extremely simple policy-free system for registering and notifying subscribers:

subscribers = []

def notify(event):
    for subscriber in subscribers:
        subscriber(event)

This is implemented in the zope.event module.

Applications subscribe by adding items to zope.event.subscribers. When an event is published, all subscribers are called. The base event system provides no policy for dispatching events. Every event is dispatched to every subscriber. Applications implement event-dispatching policies by providing base subscribers that dispatch to other subscribers based on properties of the event. Zope provides a type-based dispatching layer based on adaptation.

Brief introduction to adapters

Adapters are used to convert from one kind of object to another. A simple example is a class that converts Euclidean coordinates to polar coordinates:

>>> class Euclidean:
...     def __init__(self, x, y):
...         self.x, self.y = float(x), float(y)

>>> import math
>>> class Polar:
...     def __init__(self, euclidean):
...         self._euclidean = euclidean
...
...     @property
...     def r(self):
...         return math.hypot(self._euclidean.x,
...                           self._euclidean.y)
...
...     @property
...     def a(self):
...         return math.atan2(self._euclidean.y, 
...                           self._euclidean.x)

>>> e = Euclidean(1, 2)
>>> p = Polar(e)
>>> "%.2f" % p.r, "%.2f" % p.a
('2.24', '1.11')

The Polar class adapts Euclidean coordinates to polar coordinates.

Note that the adapter wraps the adapted object, however, this is an implementation detail. For example, the Polar class could have computed the polar coordinates in the constructor and discarded the original object.

In Zope, adapters are normally looked up based on interfaces. Suppose we define interfaces IEuclidean and IPolar:

>>> from zope import interface

>>> class IEuclidean(interface.Interface):
...     x = interface.Attribute("horizontal")
...     y = interface.Attribute("vertical")

>>> class IPolar(interface.Interface):
...     r = interface.Attribute("radius")
...     a = interface.Attribute("angle")

We can decorate the classes to indicate that they implement the corresponding interfaces:

>>> class Euclidean:
...     interface.implements(IEuclidean)
...
...     def __init__(self, x, y):
...         self.x, self.y = float(x), float(y)

In this example, we've declared that instances of Euclidean provide IEuclidean.

>>> from zope import component
>>> class Polar:
...     component.adapts(IEuclidean)
...     interface.implements(IPolar)
...
...     def __init__(self, euclidean):
...         self._euclidean = euclidean
...
...     @property
...     def r(self):
...         return math.hypot(self._euclidean.x,
...                           self._euclidean.y)
...
...     @property
...     def a(self):
...         return math.atan2(self._euclidean.y, 
...                           self._euclidean.x)

Here we've declared that instances of Polar provide IPolar. We've also declared that the Polar class adapts IEuclidean.

We can register the adapter:

>>> component.provideAdapter(Polar)

which allows us to adapt euclidean coordinates to IPolar by simply calling IPolar:

>>> e = Euclidean(2, 1)
>>> p = IPolar(e)
>>> "%.2f" % p.r, "%.2f" % p.a
('2.24', '0.46')

An advantage of interface-based adaptation is that we can have looser coupling. In the example above, we didn't rely on the Polar class to do the adaptation. If some other class was registered to provide IPolar, we'd automatically use it instead.

Adapters are a little bit like methods, in that they can be inherited. Suppose we define 3-dimensional Euclidean coordinates:

>>> class IEuclidean3D(IEuclidean):
...     z = interface.Attribute("Z :)")

>>> class Euclidean3D:
...     interface.implements(IEuclidean3D)
...
...     def __init__(self, x, y, z):
...         self.x, self.y, self.z = float(x), float(y), float(z)

We can adapt our 3d objects using our original adapter, because IEuclidean3D extends IEuclidean:

>>> e = Euclidean3D(1, 3, 2)
>>> p = IPolar(e)
>>> "%.2f" % p.r, "%.2f" % p.a
('3.16', '1.25')

We can provide a more specific adapter for 3-dimensional Euclidean coordinates. In this case, we'll provide one that provides cylindrical coordinates:

>>> class ICylindrical(IPolar):
...     z = interface.Attribute("Z :)")

>>> class Cylindrical:
...     component.adapts(IEuclidean3D)
...     interface.implements(ICylindrical)
...
...     def __init__(self, p):
...         x, y = p.x, p.y
...         self.r = math.hypot(x, y)
...         self.a = math.atan2(y, x)
...         self.z = p.z

>>> component.provideAdapter(Cylindrical)

Note that, in this example, we performed our computation in the adapter constructor. The adapter doesn't wrap the adapted object.

Now, we can use this adapter to get cylindrical coordinates:

>>> p = ICylindrical(e)
>>> p.__class__.__name__
'Cylindrical'

Note that. because this adapter adapts a more specific interface, it is preferred even when adapting to IPolar:

>>> p = IPolar(e)
>>> "%.2f" % p.r, "%.2f" % p.a, p.z
('3.16', '1.25', 2.0)

>>> p.__class__.__name__
'Cylindrical'

Normally, when looking up an adapter, we get a single adapter that is the most specific for the adapted object. This is similar to attribute inheritance in classes.

When adapting a single object to a single interface, as in the examples above, if an object already provides the interface, then the object itself is returned. (The __conform__ method, as defined by PEP 246 is also supported.)

Note on component registration

In the examples given here, we call provideAdapter and similar functions for registering components. We show these calls with the component implementations. In practice, we separate component registration from component implementation. We find that we often want to make component configuration decisions independent of component implementation. This is important for an application server like Zope, which has to support flexible application configuration.

It happens that separating configuration from implementation is valuable for simpler programs when testing is considered. When testing components, it's useful to test in isolation. Each test needs to create a configuration of components to suit its needs. The component architecture includes a function to clear all registrations. This is intended to be used by tests so that they can create controlled configurations.

Subscription adapters

Unlike regular adapters, subscription adapters are used when we want multiple adapters that adapt an object to a particular interface.

Consider validation. We have documents and we want to assess whether they meet some sort of standards.

>>> class IDocument(zope.interface.Interface):
...     summary = interface.Attribute("Document summary")
...     body = interface.Attribute("Document text")

>>> class Document:
...     interface.implements(IDocument)
...
...     def __init__(self, summary, body):
...         self.summary, self.body = summary, body

We define a validation interface:

>>> class IValidate(interface.Interface):
...     def validate(ob):
...         """Determine whether the object is valid.
...         
...         Return a string describing a validation problem.
...         An empty string is returned to indicate that the
...         object is valid.
...         """

Now, we may want to specify various validation rules for documents. For example, we might require that the summary be a single line:

>>> class SingleLineSummary:
...     component.adapts(IDocument)
...     interface.implements(IValidate)
...
...     def __init__(self, doc):
...         self.doc = doc
...
...     def validate(self):
...         if '\n' in self.doc.summary:
...             return 'Summary should only have one line'
...         else:
...             return ''

Or we might require the body to be at least 1000 characters in length:

>>> class AdequateLength:
...     component.adapts(IDocument)
...     interface.implements(IValidate)
...
...     def __init__(self, doc):
...         self.doc = doc
...
...     def validate(self):
...         if len(self.doc.body) < 1000:
...             return 'too short'
...         else:
...             return ''

We can register these as subscription adapters:

>>> component.provideSubscriptionAdapter(SingleLineSummary)
>>> component.provideSubscriptionAdapter(AdequateLength)

We can then use the subscribers to validate objects:

>>> doc = Document("A\nDocument", "blah")
>>> [adapter.validate()
...  for adapter in component.subscribers([doc], IValidate)]
['Summary should only have one line', 'too short']

To find the subscription adapters we use the subscribers function. The first argument is a list of objects to adapt. Why a list? Zope supports multi-adapters. An adapter can be defined that adapts multiple objects. Adapters that adapt a single object are an important special case, so a simple interface-call API is provided to handle this special case. The subscribers function uses the more general calling convention, taking a list of objects.

We use subscription adapters to provide a more powerful event-dispatch mechanism. Subscription adapters adapt events and optionally other objects. We can dispatch to event subscribers based on event and other object types.

Event subscribers are different from other subscription adapters in that the caller of event subscribers doesn't expect to interact with them in any direct way. For example, an event publisher doesn't expect to get any return value. Because subscribers don't need to provide an API to their callers, it is more natural to define them with functions, rather than classes. For example, in a document-management system, we might want to record creation times for documents:

>>> import datetime

>>> def documentCreated(event):
...     event.doc.created = datetime.datetime.utcnow()

In this example, we have a function that takes an event and performs some processing. It doesn't actually return anything. This is a special case of a subscription adapter that adapts an event to nothing. All of the work is done when the adapter "factory" is called. We call subscribers that don't actually create anything "handlers". There are special APIs for registering and calling them.

To register the subscriber above, we define a document-created event:

>>> class IDocumentCreated(interface.Interface):
...     doc = interface.Attribute("The document that was created")

>>> class DocumentCreated:
...     interface.implements(IDocumentCreated)
...
...     def __init__(self, doc):
...         self.doc = doc

We'll also change our handler definition to:

>>> @component.adapter(IDocumentCreated)
... def documentCreated(event):
...     event.doc.created = datetime.datetime.utcnow()

This marks the handler as an adapter of IDocumentCreated events.

Now we'll register the handler:

>>> component.provideHandler(documentCreated)

Now, if we can create an event and use the handle function to call handlers registered for the event:

>>> component.handle(DocumentCreated(doc))
>>> doc.created.__class__.__name__
'datetime'

Extending the event system

We used the 'component.handle` function to call our object-created event handler. This API is specific to Zope's component architecture. We'd like to be able to use the more general zope.event.notify call. To do that, however, we'll need to extend the basic event system to use component.handle. This is very simple, we simply register component.handle:

>>> zope.event.subscribers.append(component.handle)

Now, we can use the more general zope.event.notify call:

>>> doc2 = Document("A document", "blah " * 1000)

>>> zope.event.notify(DocumentCreated(doc2))

>>> doc2.created.__class__.__name__
'datetime'

Here, we extended the basic policy-free event system to add dispatching based on event type. We can build additional layers by registering additional basic subscribers, or by registering additional type-based event handlers. In the following section, we'll look at some examples from Zope that provide more powerful event dispatching.

Examples using subscribers to add policy

In the document-management example, we defined a document-specific event, DocumentCreated. In a system with many types, we don't want to define events for each type. It would be better to define more general event types, such as:

>>> class IObjectEvent(interface.Interface):
...     "Something happened to an object"
...
...     object = interface.Attribute("The document that was created")

>>> class IObjectModified(IObjectEvent):
...     "An object was modified"

>>> class ObjectModified:
...     interface.implements(IObjectModified)
...
...     def __init__(self, object):
...         self.object = object

Now suppose we want to set the modification time whenever documents are modified. We can't simply handle object-modified events, because we only want to set the modification time of documents. In this case, we want to dispatch to our handler based on both the event type and the object type. We can extend the event system to provide this dispatching by registering a handler for object events:

>>> @component.adapter(IObjectEvent)
... def objectDispatch(event):
...     component.handle(event.object, event)

>>> component.provideHandler(objectDispatch)

Here we pass two arguments to handle, the object and the event. This causes a handler to be looked up based on the object type and the event type. Now we can define our handler for document-modification events.

>>> @component.adapter(IDocument, IObjectModified)
... def documentCreated(doc, event):
...     doc.modified = datetime.datetime.utcnow()

>>> component.provideHandler(documentCreated)

With this in place, we can handle object modification events for documents:

>>> zope.event.notify(ObjectModified(doc))
>>> doc.modified.__class__.__name__
'datetime'

But not for other objects:

>>> zope.event.notify(ObjectModified(e))
>>> e.modified
Traceback (most recent call last):
...
AttributeError: Euclidean3D instance has no attribute 'modified'

We can take this further. Suppose we wish to allow subscribers for specific instances. We might define an interface such as:

>>> class IObservable(interface.Interface):
...
...    def handle(event):
...        "Notify object event handlers"

Objects that support object-based event subscribers would be adaptable to IObservable. (Zope provides an implementation of a similar, but more complicated interface that also supports subscriber registration.)

Now, if we want to dispatch to observable objects, we can extend the event system with the following subscriber:

>>> @component.adapter(interface.Interface, IObjectEvent)
... def instanceDispatch(object, event):
...     observable = IObservable(object, None)
...     if observable is not None:
...         observable.handle(event)

>>> component.provideHandler(instanceDispatch)

Let's extend our document class to provide minimal observability:

>>> class ObservableDocument(Document):
...     interface.implements(IObservable)
...
...     def handle(self, event):
...         print 'observed', event.__class__.__name__

Now, if we publish an event for an instance:

>>> doc = ObservableDocument('An observable', 'eek'*1000)
>>> zope.event.notify(ObjectModified(doc))
observed ObjectModified

We see that the handler was called.

Of course, since this was a document, its modification time was also set:

>>> doc.modified.__class__.__name__
'datetime'

Related technologies

Aspect-Oriented Programming

Aspect-oriented programming (AOP) provides a way to extend software to deal with "cross-cutting concerns". Software components are written to perform specific functions and deal with specific requirements. This is the primary or intrinsic functionality of the software. Within a system, we may want to extend the software to satisfy secondary requirements imposed by the system. Some examples of secondary concerns include:

Logging some activities
Recording meta data, such as modification times
Enforcing security

We don't want to have to modify the original software to satisfy the concerns.

In AOP, locations in the original software are identified where extra processing should occur. These are called "point cuts". The point cuts are recorded separately from the original software. Separate code, called "advice" is defined and associated with the point cuts. The point cuts and advice, together, are called the "aspects". A weaving program reads:

The point cut definitions,
The advice configuration, and
The original software.

The weaver then generates new software that calls the advice code when the point cuts are reached in the execution. This weaving is sometimes done at run time, modifying byte codes on the fly.

Our event system is somewhat similar to AOP. Events correspond to point cuts and subscribers to advice. Event registration plays the role of the configuration of advice for point cuts. The main difference is that events must be explicitly published from original code. It's possible that weaving techniques might be used to insert event-publication code at points in the code.

Rule-based systems

Rule-based systems use databases of rules with "if" and "then" parts. Rules "fire" or execute when the conditions in their "if" parts are satisfied. These rules are held in a database, rather than procedural program text.

The event system described here is a very simple rule-based system. Handler registrations are the rules. The event types used in the registrations provide the "if" parts of the rules and the handlers provide the "then" parts. Subscribers can generate events, causing additional rules to fire. Because the rules are expressed through registrations, rather than in static code, the rules are much easier to change, and compose.

Summary

Event systems provide a way to extend the processing within methods. The Zope event system is extremely simple, but supports extension to provide gradually more sophisticated event-dispatch policies. We've seen how to use event subscribers to manage object meta data and to extend the event system. Powerful type-based dispatching is made possible using the adaptation facilities of the component architecture.

It is worth considering adding the basic event system described here to the Python standard library. It would allow library developers to publish events without depending on a specific event-dispatch implementation. Specific applications could add custom event systems on top of the basic event system to meet their custom event dispatching needs.

A Layered Event System to Provide Method Extensibility