Introduction
This PEP describes an extension to instance attribute lookup and
modification machinery, which allows pure-Python implementations
of many interesting programming models. This PEP tracks the
status and ownership of this feature. It contains a description
of the feature and outlines changes necessary to support the
feature. This PEP summarizes discussions held in mailing list
forums, and provides URLs for further information, where
appropriate. The CVS revision history of this file contains the
definitive historical record.
Background
The semantics for Python instances allow the programmer to
customize some aspects of attribute lookup and attribute
modification, through the special methods __getattr__() and
__setattr__() [1].
However, because of certain restrictions imposed by these methods,
there are useful programming techniques that can not be written in
Python alone, e.g. strict Java Bean-like[2] interfaces and Zope
style acquisitions[3]. In the latter case, Zope solves this by
including a C extension called ExtensionClass[5] which modifies
the standard class semantics, and uses a metaclass hook in
Python's class model called alternatively the "Don Beaudry Hook"
or "Don Beaudry Hack"[6].
While Zope's approach works, it has several disadvantages. First,
it requires a C extension. Second it employs a very arcane, but
truck-sized loophole in the Python machinery. Third, it can be
difficult for other programmers to use and understand (the
metaclass has well-known brain exploding properties). And fourth,
because ExtensionClass instances aren't "real" Python instances,
some aspects of the Python runtime system don't work with
ExtensionClass instances.
Proposals for fixing this problem have often been lumped under the
rubric of fixing the "class/type dichotomy"; that is, eliminating
the difference between built-in types and classes[7]. While a
laudable goal itself, repairing this rift is not necessary in
order to achieve the types of programming constructs described
above. This proposal provides an 80% solution with a minimum of
modification to Python's class and instance objects. It does
nothing to address the type/class dichotomy.
Proposal
This proposal adds a new special method called __findattr__() with
the following semantics:
* If defined in a class, it will be called on all instance
attribute resolutions instead of __getattr__() and
__setattr__().
* __findattr__() is never called recursively. That is, when a
specific instance's __findattr__() is on the call stack, further
attribute accesses for that instance will use the standard
__getattr__() and __setattr__() methods.
* __findattr__() is called for both attribute access (`getting')
and attribute modification (`setting'). It is not called for
attribute deletion.
* When called for getting, it is passed a single argument (not
counting `self'): the name of the attribute being accessed.
* When called for setting, it is called with third argument, which
is the value to set the attribute to.
* __findattr__() methods have the same caching semantics as
__getattr__() and __setattr__(); i.e. if they are present in the
class at class definition time, they are used, but if they are
subsequently added to a class later they are not.
Key Differences with the Existing Protocol
__findattr__()'s semantics are different from the existing
protocol in key ways:
First, __getattr__() is never called if the attribute is found in
the instance's __dict__. This is done for efficiency reasons, and
because otherwise, __setattr__() would have no way to get to the
instance's attributes.
Second, __setattr__() cannot use "normal" syntax for setting
instance attributes, e.g. "self.name = foo" because that would
cause recursive calls to __setattr__().
__findattr__() is always called regardless of whether the
attribute is in __dict__ or not, and a flag in the instance object
prevents recursive calls to __findattr__(). This gives the class
a chance to perform some action for every attribute access. And
because it is called for both gets and sets, it is easy to write
similar policy for all attribute access. Further, efficiency is
not a problem because it is only paid when the extended mechanism
is used.
Related Work
PEP 213 [9] describes a different approach to hooking into
attribute access and modification. The semantics proposed in PEP
213 can be implemented using the __findattr__() hook described
here, with one caveat. The current reference implementation of
__findattr__() does not support hooking on attribute deletion.
This could be added if it's found desirable. See example below.
Examples
One programming style that this proposal allows is a Java
Bean-like interface to objects, where unadorned attribute access
and modification is transparently mapped to a functional
interface. E.g.
class Bean:
def __init__(self, x):
self.__myfoo = x
def __findattr__(self, name, *args):
if name.startswith('_'):
# Private names
if args: setattr(self, name, args[0])
else: return getattr(self, name)
else:
# Public names
if args: name = '_set_' + name
else: name = '_get_' + name
return getattr(self, name)(*args)
def _set_foo(self, x):
self.__myfoo = x
def _get_foo(self):
return self.__myfoo
b = Bean(3)
print b.foo
b.foo = 9
print b.foo
A second, more elaborate example is the implementation of both
implicit and explicit acquisition in pure Python:
import types
class MethodWrapper:
def __init__(self, container, method):
self.__container = container
self.__method = method
def __call__(self, *args, **kws):
return self.__method.im_func(self.__container, *args, **kws)
class WrapperImplicit:
def __init__(self, contained, container):
self.__contained = contained
self.__container = container
def __repr__(self):
return '<Wrapper: [%s | %s]>' % (self.__container,
self.__contained)
def __findattr__(self, name, *args):
# Some things are our own
if name.startswith('_WrapperImplicit__'):
if args: return setattr(self, name, *args)
else: return getattr(self, name)
# setattr stores the name on the contained object directly
if args:
return setattr(self.__contained, name, args[0])
# Other special names
if name == 'aq_parent':
return self.__container
elif name == 'aq_self':
return self.__contained
elif name == 'aq_base':
base = self.__contained
try:
while 1:
base = base.aq_self
except AttributeError:
return base
# no acquisition for _ names
if name.startswith('_'):
return getattr(self.__contained, name)
# Everything else gets wrapped
missing = []
which = self.__contained
obj = getattr(which, name, missing)
if obj is missing:
which = self.__container
obj = getattr(which, name, missing)
if obj is missing:
raise AttributeError, name
of = getattr(obj, '__of__', missing)
if of is not missing:
return of(self)
elif type(obj) == types.MethodType:
return MethodWrapper(self, obj)
return obj
class WrapperExplicit:
def __init__(self, contained, container):
self.__contained = contained
self.__container = container
def __repr__(self):
return '<Wrapper: [%s | %s]>' % (self.__container,
self.__contained)
def __findattr__(self, name, *args):
# Some things are our own
if name.startswith('_WrapperExplicit__'):
if args: return setattr(self, name, *args)
else: return getattr(self, name)
# setattr stores the name on the contained object directly
if args:
return setattr(self.__contained, name, args[0])
# Other special names
if name == 'aq_parent':
return self.__container
elif name == 'aq_self':
return self.__contained
elif name == 'aq_base':
base = self.__contained
try:
while 1:
base = base.aq_self
except AttributeError:
return base
elif name == 'aq_acquire':
return self.aq_acquire
# explicit acquisition only
obj = getattr(self.__contained, name)
if type(obj) == types.MethodType:
return MethodWrapper(self, obj)
return obj
def aq_acquire(self, name):
# Everything else gets wrapped
missing = []
which = self.__contained
obj = getattr(which, name, missing)
if obj is missing:
which = self.__container
obj = getattr(which, name, missing)
if obj is missing:
raise AttributeError, name
of = getattr(obj, '__of__', missing)
if of is not missing:
return of(self)
elif type(obj) == types.MethodType:
return MethodWrapper(self, obj)
return obj
class Implicit:
def __of__(self, container):
return WrapperImplicit(self, container)
def __findattr__(self, name, *args):
# ignore setattrs
if args:
return setattr(self, name, args[0])
obj = getattr(self, name)
missing = []
of = getattr(obj, '__of__', missing)
if of is not missing:
return of(self)
return obj
class Explicit(Implicit):
def __of__(self, container):
return WrapperExplicit(self, container)
# tests
class C(Implicit):
color = 'red'
class A(Implicit):
def report(self):
return self.color
# simple implicit acquisition
c = C()
a = A()
c.a = a
assert c.a.report() == 'red'
d = C()
d.color = 'green'
d.a = a
assert d.a.report() == 'green'
try:
a.report()
except AttributeError:
pass
else:
assert 0, 'AttributeError expected'
# special names
assert c.a.aq_parent is c
assert c.a.aq_self is a
c.a.d = d
assert c.a.d.aq_base is d
assert c.a is not a
# no acquisiton on _ names
class E(Implicit):
_color = 'purple'
class F(Implicit):
def report(self):
return self._color
e = E()
f = F()
e.f = f
try:
e.f.report()
except AttributeError:
pass
else:
assert 0, 'AttributeError expected'
# explicit
class G(Explicit):
color = 'pink'
class H(Explicit):
def report(self):
return self.aq_acquire('color')
def barf(self):
return self.color
g = G()
h = H()
g.h = h
assert g.h.report() == 'pink'
i = G()
i.color = 'cyan'
i.h = h
assert i.h.report() == 'cyan'
try:
g.i.barf()
except AttributeError:
pass
else:
assert 0, 'AttributeError expected'
C++-like access control can also be accomplished, although less
cleanly because of the difficulty of figuring out what method is
being called from the runtime call stack:
import sys
import types
PUBLIC = 0
PROTECTED = 1
PRIVATE = 2
try:
getframe = sys._getframe
except ImportError:
def getframe(n):
try: raise Exception
except Exception:
frame = sys.exc_info()[2].tb_frame
while n > 0:
frame = frame.f_back
if frame is None:
raise ValueError, 'call stack is not deep enough'
return frame
class AccessViolation(Exception):
pass
class Access:
def __findattr__(self, name, *args):
methcache = self.__dict__.setdefault('__cache__', {})
missing = []
obj = getattr(self, name, missing)
# if obj is missing we better be doing a setattr for
# the first time
if obj is not missing and type(obj) == types.MethodType:
# Digusting hack because there's no way to
# dynamically figure out what the method being
# called is from the stack frame.
methcache[obj.im_func.func_code] = obj.im_class
#
# What's the access permissions for this name?
access, klass = getattr(self, '__access__', {}).get(
name, (PUBLIC, 0))
if access is not PUBLIC:
# Now try to see which method is calling us
frame = getframe(0).f_back
if frame is None:
raise AccessViolation
# Get the class of the method that's accessing
# this attribute, by using the code object cache
if frame.f_code.co_name == '__init__':
# There aren't entries in the cache for ctors,
# because the calling mechanism doesn't go
# through __findattr__(). Are there other
# methods that might have the same behavior?
# Since we can't know who's __init__ we're in,
# for now we'll assume that only protected and
# public attrs can be accessed.
if access is PRIVATE:
raise AccessViolation
else:
methclass = self.__cache__.get(frame.f_code)
if not methclass:
raise AccessViolation
if access is PRIVATE and methclass is not klass:
raise AccessViolation
if access is PROTECTED and not issubclass(methclass,
klass):
raise AccessViolation
# If we got here, it must be okay to access the attribute
if args:
return setattr(self, name, *args)
return obj
# tests
class A(Access):
def __init__(self, foo=0, name='A'):
self._foo = foo
# can't set private names in __init__
self.__initprivate(name)
def __initprivate(self, name):
self._name = name
def getfoo(self):
return self._foo
def setfoo(self, newfoo):
self._foo = newfoo
def getname(self):
return self._name
A.__access__ = {'_foo' : (PROTECTED, A),
'_name' : (PRIVATE, A),
'__dict__' : (PRIVATE, A),
'__access__': (PRIVATE, A),
}
class B(A):
def setfoo(self, newfoo):
self._foo = newfoo + 3
def setname(self, name):
self._name = name
b = B(1)
b.getfoo()
a = A(1)
assert a.getfoo() == 1
a.setfoo(2)
assert a.getfoo() == 2
try:
a._foo
except AccessViolation:
pass
else:
assert 0, 'AccessViolation expected'
try:
a._foo = 3
except AccessViolation:
pass
else:
assert 0, 'AccessViolation expected'
try:
a.__dict__['_foo']
except AccessViolation:
pass
else:
assert 0, 'AccessViolation expected'
b = B()
assert b.getfoo() == 0
b.setfoo(2)
assert b.getfoo() == 5
try:
b.setname('B')
except AccessViolation:
pass
else:
assert 0, 'AccessViolation expected'
assert b.getname() == 'A'
Here's an implementation of the attribute hook described in PEP
213 (except that hooking on attribute deletion isn't supported by
the current reference implementation).
class Pep213:
def __findattr__(self, name, *args):
hookname = '__attr_%s__' % name
if args:
op = 'set'
else:
op = 'get'
# XXX: op = 'del' currently not supported
missing = []
meth = getattr(self, hookname, missing)
if meth is missing:
if op == 'set':
return setattr(self, name, *args)
else:
return getattr(self, name)
else:
return meth(op, *args)
def computation(i):
print 'doing computation:', i
return i + 3
def rev_computation(i):
print 'doing rev_computation:', i
return i - 3
class X(Pep213):
def __init__(self, foo=0):
self.__foo = foo
def __attr_foo__(self, op, val=None):
if op == 'get':
return computation(self.__foo)
elif op == 'set':
self.__foo = rev_computation(val)
# XXX: 'del' not yet supported
x = X()
fooval = x.foo
print fooval
x.foo = fooval + 5
print x.foo
# del x.foo
Reference Implementation
The reference implementation, as a patch to the Python core, can be
found at this URL:
http://sourceforge.net/patch/?func=detailpatch&patch_id=102613&group_id=5470
References
[1] http://www.python.org/doc/current/ref/attribute-access.html
[2] http://www.javasoft.com/products/javabeans/
[3] http://www.digicool.com/releases/ExtensionClass/Acquisition.html
[5] http://www.digicool.com/releases/ExtensionClass
[6] http://www.python.org/doc/essays/metaclasses/
[7] http://www.foretec.com/python/workshops/1998-11/dd-ascher-sum.html
[8] http://www.python.org/doc/howto/rexec/rexec.html
[9] PEP 213, Attribute Access Handlers, Prescod
http://www.python.org/peps/pep-0213.html
Rejection
There are serious problems with the recursion-protection feature.
As described here it's not thread-safe, and a thread-safe solution
has other problems. In general, it's not clear how helpful the
recursion-protection feature is; it makes it hard to write code
that needs to be callable inside __findattr__ as well as outside
it. But without the recursion-protection, it's hard to implement
__findattr__ at all (since __findattr__ would invoke itself
recursively for every attribute it tries to access). There seems
to be no good solution here.
It's also dubious how useful it is to support __findattr__ both
for getting and for setting attributes -- __setattr__ gets called
in all cases alrady.
The examples can all be implemented using __getattr__ if care is
taken not to store instance variables under their own names.
Copyright
This document has been placed in the Public Domain.
![[Python]](../pics/PyBanner002.gif){kind=small}