Building Chandler Parcels

Overview

Chandler manages data we're all accustomed to using every day -- email, contacts, tasks, notes -- and unifies and organizes it all in a unique way. Chandler works for common data types out of the box, and it's easy to extend Chandler to work with custom data.

In Chandler a user's data is made up of Items and collections of Items, which we call Item Collections. An Item could be a calendar event, email message, task, note, photo, etc. An Item Collection is a heterogenous group of these Items. The center table in the screenshot above demonstrates a collection of arbitrary Items. This reflects one of our design goals for Chandler -- instead of trapping data in individual application "silos" (email in an mail reader, calendar events in a calendar application) Chandler provides a flexible, unified domain where all the data can live.

Out of the box, Chandler will support types commonly found in a Personal Information Manager: email, contacts, calendar events, tasks, and notes. Chandler's data handling is designed to make it easy for users to plug in their own data types and behaviors -- as simple as adding attributes to an existing Item type or as complex as defining a new Item type, with its own schema, data handling code, and UI code.

Our example extension for Chandler, ZaoBao, extends Chandler to deal with RSS feed data. Eventually we want to enable end users to extend and customize Chandler with minimal to no programming.

To get a feel for Chandler's design and see how customization works, we'll walk though the major steps that make up the ZaoBao example. For a more detailed walkthrough of the same extension, see the tutorial at http://wiki.osafoundation.org/bin/view/Chandler/ZaoBaoTutorial

Basics of the Chandler UI

First, lets cover some of the basic elements of Chandler's user interface:

At first glance, our three pane view looks pretty familiar. The Sidebar contains a set of Item Collections, and one of them is selected. The Summary View displays the Items from the Item Collection, and one Item is selected. The Detail View displays the selected Item.

ZaoBao is going to extend Chandler to add RSS Items to our mix of Items. To integrate RSS data, we need to...

1. Extend Chandler's schema by defining two new kinds of Items: one for an RSS channel (or feed), and one for the individual RSS news Items.
2. Create a background task that will periodically retreive RSS Items for the RSS channels in our repository.
3. Connect the RSS channels and Items to the user interface.
   • Add a menu handler to add collections of RSS Items to the Sidebar.
   • Extend schema for RSS Items to display properly in the Summary View.
   • Define a UI element for RSS Items to display properly in the *Detail View.

1. Adding RSS Item and RSS Channel: Extending Chandler's Schema

Before we start extending Chandler's schema, we'll introduce you briefly to Chandler's Repository. Chandler's Repository is a single database that stores all information about all data used in Chandler. An Item is the simplest kind of object that can be stored in the Repository. Once Items are created in the Repository, they will persist across all invocations of Chandler until they are explicitly deleted. The Repository API tries to make working with Items look like working with regular Python objects.

An Item is really just a bunch of Attribute values. For example, a calendar event Item would have attributes such as startTime, endTime, etc. Every Item has a Kind that determines what Attributes that Item can have. For example, calendar event Items would have a Kind of CalendarEvent, which would list the Attribute definitions for startTime, endTime, etc.

For ZaoBao, let's define a Kind for an RSS channel:

  <Kind itsName="RSSChannel">
    <displayName>RSS Channel</displayName>

    <!-- define new attributes -->
    <Attribute itsName="url">
      <displayName>URL</displayName>
      <cardinality>single</cardinality>
      <type itemref="String"/>
    </Attribute>
  
    <Attribute itsName="items">
      <displayName>Items</displayName>
      <cardinality>list</cardinality>
      <type itemref="doc:RSSItem"/>
      <inverseAttribute itemref="doc:RSSItem/channel"/>
    </Attribute>
  </Kind>

The XML above gives the name of the Kind (itsName="RSSChannel"), and lists the Attribute definitions for this Kind (Attribute itsName="url").

Now let's define a Kind for RSS Items, representing a single news Item in an RSS channel.

  <Kind itsName="RSSItem">
    <superKinds itemref="content:ContentItem"/>
    <displayName>RSS Item</displayName>

    <!-- define new attributes -->
    <Attribute itsName="channel">
      <displayName>Channel</displayName>
      <cardinality>single</cardinality>
      <type itemref="doc:RSSChannel"/>
    </Attribute>

    <Attribute itsName="author">
      <displayName>Author</displayName>
      <cardinality>single</cardinality>
      <type itemref="String"/>
    </Attribute>

    <Attribute itsName="date">
      <displayName>Date</displayName>
      <cardinality>single</cardinality>
      <type itemref="DateTime"/>
    </Attribute>

    <Attribute itsName="content">
      <cardinality>single</cardinality>
      <type itemref="Lob"/>
    </Attribute>

  </Kind>

As you can see, RSSItem contains four attributes: channel, author, date, and content. The attribute channel refers back to the original RSSChannel, and content is a Lob - or Large Object. We'll be storing potentially large chunks of HTML in this attribute so we'll use Lob rather than String.

Behind the scenes: Repository

Everything in the Repository is an Item, including Kinds themselves.

You may have noticed that our RSS Item has a SuperKind called ContentModel. This reflects an inheritance hierarchy: the Items that we think of as the user's data all have Kinds that descend from ContentModel.

Adding behavior with Python

This data definition is useful, but we probably want to define some behavior for RSSChannel=s and =RSSItem=s. We can define methods and implement them using Python. All we need to do is add a =<classes> tag to our Kind definition. For example, we might want to be able to call a method, Update() on an RSSChannel Item which updates the RSS info from the server.

  <Kind itsName="RSSChannel">

    <classes key="python">osaf.examples.zaobao.RSSData.RSSChannel</classes>

    ... rest of RSSChannel definition ...
  </Kind>

We can then define the corresponding class in the file osaf/examples/zaobao/RSSData.py:

class RSSChannel(Item):
    def Update(self):
        ... retrieve the RSS feed here ...

2. Periodically getting new RSS Items: WakeupCaller

Most RSS Aggregators will periodically poll the source of the feed in order to download new Items. We'd like ZaoBao to do the same thing. Chandler provides an Item called a WakeupCaller which is used to implement these kinds of background or periodic tasks.

<wakeupCall:WakeupCall itsName="ZaoBao">
    <wakeupCallClass>osaf.examples.zaobao.ZaoBaoWakeupCall.WakeupCall</wakeupCallClass>
    <callOnStartup>True</callOnStartup>
    <enabled>True</enabled>
    <repeat>True</repeat>
    <delay>00:00:30:00</delay>
</wakeupCall:WakeupCall>

The above XML generates an Item of the WakeupCall Kind in the Repository, set with the given Attributes. The WakeupCaller system will automatically find all Items in the repository of Kind WakeupCall and run them as often as they are specified. The WakeupCall defined above will be activated every 30 minutes. The behavior of the WakeupCall is defined by the Python class specified in the wakeupCallClass tag. The receiveWakeupCall method of this class will be called at the interval specified by the WakeupCall Item (above).

class WakeupCall(WakeupCaller.WakeupCall):

    def receiveWakeupCall(self, wakeupCallItem):

        # We need the view for most repository operations
        view = wakeupCallItem.itsView

        # We need the Kind object for RSSChannels
        channelKind = RSSData.RSSChannel.getKind(view)

        # Find all objects with the corresponding kind
        for channel in KindQuery().run([channelKind]):
                channel.Update()

        ...

        # We want to commit the changes to the repository
        view.commit()

The ZaoBao WakeupCall retreives new RSS data every 30 minutes.

Behind the scenes: Threading

wxWidgets, like all GUI frameworks, needs to run in a single thread. We're currently using the Twisted Reactor to schedule work that needs to happen outside that thread. The Reactor runs in a separate thread, and is used to schedule the WakeupCallers found in the Repository.

A thread in the Repository has its own Repository View, which is an independent connection to the Repository. Each WakeupCaller has its own Repository View, as does the main wxWidgets UI thread. Different tasks and threads communicate with each other through the repository. After ZaoBao's WakeupCaller creates new Items in its Repository View, it calls commit() on the Repository View, which pushes the Items to the Repository. The UI thread calls refresh() on its Repository View during its OnIdle cycle, picking up the new changes.

3. Connecting the data to the user interface

Our next step is to get RSS Items to show up in Chandler's interface. We'll add a menu handler to add collections of RSS Items to the Sidebar, modify the schema so the Summary View can display RSS Items, and extend the Detail View to display RSS Item attributes.

Creating a Menu Handler

Let's start by adding a new menu item for creating a Collection of RSS Items from an RSS feed. A menu is an Item of Kind MenuItem, so we'll define just like all the other Items we've seen so far.

  <MenuItem itsName="NewZaoBaoChannel">
    <blockName>NewZaoBaoChannelItem</blockName>
    <title>New ZaoBao Channel</title>
    <event itemref="doc:NewZaoBaoChannelEvent"/>
    <parentBlock itemref="main:NewItemMenu"/>
  </MenuItem>

The MenuItem is attached to the NewItemMenu, which is the menu you see when you select File->New. When you select the menu item, it will fire the NewZaoBaoChannelEvent, which we'll define next. As with menu items, events are also Items.

  <BlockEvent itsName="NewZaoBaoChannelEvent">
    <blockName>NewZaoBaoChannel</blockName>
    <dispatchEnum>SendToBlockByReference</dispatchEnum>
    <destinationBlockReference itemref="doc:ZaoBaoController"/>
    <commitAfterDispatch>True</commitAfterDispatch>
  </BlockEvent>

When a NewZaoBaoChannelEvent is fired, it will be sent to the onNewZaoBaoChannelEvent method of ZaoBao's controller, ZaoBaoController.

class ZaoBaoController(Block.Block):

    def onNewZaoBaoChannelEvent(self, event):
        """
        create the channel, and add it to the Sidebar
        """
        url = application.dialogs.Util.promptUser(wx.GetApp().mainFrame, "New Channel", "Enter a URL for the RSS Channel", "http://")
        if url and url != "":
            # create the zaobao channel
            channel = osaf.examples.zaobao.RSSData.NewChannelFromURL(view=self.itsView, url=url, update=True)
            
            # now post the new collection to the sidebar
            mainView = Globals.views[0]
            mainView.postEventByName ('AddToSidebarWithoutCopying', {'items': [channel.items]})

The controller calls NewChannelFromURL to create an RSSChannel object and populate it with data from the RSS feed.

Next, the list of RSSItems is added to the Sidebar. channel.items is an ItemCollection that contains the RSSItem objects. This collection is managed by the RSSChannel : Items will be added each time the RSS feed is updated. We add this ItemCollection to the Sidebar by posting an event to the view. This ItemCollection will appear as a single entry in the Sidebar.

def NewChannelFromURL(view, url, update = True):
    """
    Create the channel in the repository
    """

    # Since RSSChannel subclasses Item, the constructor for RSSChannel
    # will ensure that the Item is created in the repository
    channel = RSSChannel(view)
    channel.url = url

    if update:
        data = feedparser.parse(url)
        channel.Update(data)

    return channel    

NewChannelFromURL takes care of calling the feedparser and filling in the RSS data.

Behind the scenes: CPIA (Chandler Presentation Interaction Architecture)

The high-level UI elements in Chandler (like the Sidebar, the Detail View, the Summary View, Menus, the Toolbar and Status Bar) are also Items, and they are stored in the repository just like any other Kind of Item. UI element kinds are derived from the 'Block' Kind, and so we call them "Blocks". Blocks represent the "View/Controller" in our variant of a Model-View-Controller architecture. Collections of ContentModel Items (RSS Items, calendar events, mail messages, etc.) make up the "Model". Each Block has a "contents" Attribute, connecting it to its Model. In the case of the Summary View, its "contents" is the ItemCollection selected in the Side Bar. In the case of the Detail View, its "contents" is the currently selected Item in the Summary View. The actual GUI implementation of the Block is handled by a wxWidgets peer.

Displaying Items in Summary View

When the user clicks on a collection in the Sidebar, the Summary View automatically displays the Items in that collection. In RSSChannel collections the Items are RSSItems.

By default, the Summary View displays the who, about, and date attributes in its columns.

As we have defined it, the RSSItem does not define all of these attributes. This is normal as these are very generic attributes. Typically a Kind will define a series of "redirect" attributes with these names. These "redirect" attributes are like virtual attributes that actually refer to values in other attributes. Typically, about refers to the title of an Item, who refers to an author or creator of an Item, and date refers to some relevant date stored in the Item such as the due date or the start date.

In the case of RSSItem, about should redirect to the title of the article, and who should redirect to the author.

We will define these "redirect" attributes in the original definition for the Kind:

<Kind itsName="RSSItem">
    ... definition for RSSItem ...

    <Attribute itsName="about">
        <redirectTo value="displayName"/>
    </Attribute>

    <Attribute itsName="who">
        <redirectTo value="author"/>
    </Attribute>

</Kind>

This is all the code that is required to display Items in this list. Chandler will take care of all the rest.

Behind the scenes: ItemCollections

Under the hood, ItemCollections are actually implemented as queries against the Repository. We specify a "rule" for the query, as well as a list of Items that are included in the collection (in addition to the rule), or excluded from the collection (an exception to the rule). In our simple example, the RSSChannel ItemCollections are actually a degenerate case: just the list of inclusions.

Displaying an Item's Detail View

When the user clicks on an Item in the Summary View, the Detail View will display details about that particular Item. Similar to the Summary View, the Detail View knows about certain attributes and can display them automatically. Unlike the Summary View, each Kind can display a different set of attributes in the Detail View, in its own unique way.

A Kind can define the user interface for the Detail View by providing a "Trunk Subtree" which defines a list of user interface elements to display:

<detail:DetailTrunkSubtree>
  <!-- this DetailTrunkSubtree is for RSSItems -->
  <key itemref="zbSchema:RSSItem"/>

  <!-- define UI Elements -->
  <rootBlocks itemref="detail:MarkupBar"/>
  <rootBlocks itemref="doc:LinkArea"/>
  <rootBlocks itemref="doc:AuthorArea"/>
  <rootBlocks itemref="doc:CategoryArea"/>
</detail:DetailTrunkSubtree>

Each rootBlocks attribute refers to a widget or fragment of user interface which will be displayed in the Detail View. Chandler will ensure that these user interface fragments will display and edit the correct data in a consistent way.

Loading the Items into Chandler: Parcels

We have now defined schema Kind Items, defined a WakeupCaller Item for a background task, defined a Menu Block and associated Event to add new behavior, and defined a DetailTrunkSubtree Item to customize the UI. We've also associated Python code with several of these Items. We add these enhancements to Chandler by defining a Parcel. A Parcel defines Items to be loaded into the repository. Once those Items are in the repository, they are discovered by queries against the repository, or they are discovered because they are linked to other well known Items in the repository.

Chandler uses parcels to define much of the UI framework, and all of the PIM functionality.

Parcel XML is stored in files, all named parcel.xml. Parcel XML and Python modules are placed in one of a few well known directories. Chandler recursively scans these directories at startup, and loads each parcel.xml directly into the repository. While Chandler is running it finds the various data definitions in the repository and activates them when appropriate. This in turn causes the corresponding Python code to be loaded and run. A Parcel is the name for this combination of parcel.xml files and the associated Python .py files. Note that each directory containing Python modules is also a Python package.

This is one of the primary patterns that drives Chandler: the repository controls all data in Chandler, and Chandler uses it to discover new application schemas and data. Behavior is determined by code that is attached to this data.

Behind the scenes: parcel.xml is merely a bootstrapping mechanism to get data loaded into the Repository. The Repository is the primary data store that Chandler uses to keep all user data, schemas, and more. If data that is declared in your parcel.xml is changed within the Repository, those changes will not be serialized back into parcel.xml.

Where Chandler is today

In this paper, you have seen how to:

• Define new data types
• Populate the repository with new data
• Display that data in the Sidebar, Summary View, and Detail View

The next step is to spend some time developing your own data types and application behavior. The steps described here will get you some basic functionality out of your data but Chandler's true potential comes to light when you begin to explore some of its more advanced capabilities. When Items have been properly defined, many of these capabilities "just work" for your new Kind.

Sharing

Chandler has a built-in infrastructure for sharing individual Items or entire ItemCollections through WebDAV. The most immediate and obvious use for this is the capability to share Calendar data. Users can share their data with others and allow them make changes, allowing collaboration using open standards. By creating your parcel's data as Chandler Items, your data will also be able to take advantage of Chandler's sharing capabilities.

Stamping and dynamic types

Future Directions

The website for the Chandler Projects is at http://wiki.osafoundation.org.

You can join our mailing lists via this page: http://wiki.osafoundation.org/bin/view/Chandler/OsafMailingLists.

Chandler developers hang out in IRC at irc://irc.osafoundation.org:6667#chandler.

This work is licensed under a Creative Commons License.