Python metaclasses in depth
Metaclasses have a reputation for being among the blackest of Python’s black magic. As it turns out, they’re actually pretty simple, but they require you to understand some concepts. There are two ways to learn metaclasses in Python:
- Skip the details and get straight to a useful implementation of a metaclass.
- Understand the core concepts behind metaclasses.
Number 1 is useful, but will only get you so far. If you really want to understand metaclasses, you need to take approach number 2.
With that in mind, I’m going to start with the basics of how classes are constructed in Python. Let’s consider the following class:
class SomeClass(object):
x = 1
y = 2If you’re ready to learn about metaclasses, the above statement shouldn’t require much thought to understand. But let’s stop to think about it a bit anyway. In Python, everything is an object. Therefore, classes are also objects. But if SomeClass is an object, it must be an instance of some class. Let’s find out what that class is:
>>> type(SomeClass) <type 'type'>
So apparently, it’s an instance of type. We just saw the most common usage of type: to query an object’s type. But have you ever read the help on that function?
>>> help(type) Help on class type in module __builtin__: class type(object) | type(object) -> the object's type | type(name, bases, dict) -> a new type ...
So it turns out that type is not only a builtin function, it’s also a class! How do we instantiate an instance of type? You’ve already seen the most obvious way: using a class statement. But did you know that you can create a class using type?
SomeClass = type('SomeClass', (object,), {'x' : 1, 'y' : 2})For all intents and purposes, the above statement is equivalent to the prior definition of SomeClass. But this way is ugly and isn’t used very commonly. That said, we’ve demonstrated something: type isn’t just any class. It’s the class of classes. It’s a metaclass.
Let’s go a step further though. How does the compiler generate the dictionary that’s the third argument to type? As it turns out, classes have something in common with functions: they have local namespaces. You might have seen the locals function used like this:
def some_func():
x = 1
y = 2
print locals()If you execute this, the output is this:
>>> some_func()
{'y': 2, 'x': 1}If classes have their own namespaces, then they must also be able to use the locals function as well:
>>> class SomeClass(object):
... x = 1
... y = 2
... print locals()
...
{'y': 2, 'x': 1, '__module__': '__main__'}The only difference here is that the namespace is generated at import time and passed into the type function.
So this is all interesting, but we haven’t really seen anything terribly useful yet. Let’s go further. We’ve seen that type is the class of classes. But if type is a class, then we must be able to subclass it. There are a number of reasons you might want to do this. For instance, you may sometimes want to attach a property to a class rather than to an instance. Let’s do just that:
class SomeMetaClass(type):
@property
def z(self):
print 'In class property z'
return self.x + self.y
>>> SomeClass = SomeMetaClass('SomeClass', (object,), {'x' : 1, 'y' : 2})
>>> SomeClass.z
In class property z
3But we don’t want to use this same ugly notation for creating SomeClass. Python provides syntactic sugar for this. We can instead define SomeClass like this:
class SomeClass(object):
__metaclass__ = SomeMetaClass
x = 1
y = 2A more common use of metaclasses is to create a class constructor. Let’s attach z to the class directly rather than defining a property:
>>> class SomeMetaClass(type): ... def __init__(self, name, bases, dict): ... self.z = self.x + self.y ... >>> class SomeClass(object): ... __metaclass__ = SomeMetaClass ... x = 1 ... y = 2 ... >>> SomeClass.z 3
As we can see, we’ve defined a constructor for SomeClass. Now let’s
go a bit further. What if we want to change the base class of
SomeClass? That can be done, but we have to use a __new__ method.
I’m going to presume that you know a bit about __new__ methods. If
you don’t, you might want to read up on
them.
>>> class SomeMetaClass(type):
... def __new__(cls, name, bases, cls_dict):
... new_cls = type.__new__(cls, name, (dict,), cls_dict)
... return new_cls
...
>>> class SomeClass(object):
... __metaclass__ = SomeMetaClass
... x = 1
... y = 2
...
>>> x = SomeClass()
>>> x['foo'] = 'bar'
>>> x
{'foo': 'bar'}That should hopefully give you an idea of what metaclasses are and how to use them. If you’re even more lost than you were before, don’t worry. This is just one of those things that requires an “aha!” moment. You might also want to check out Michael Foord’s great Meta-classes Made Easy for a different perspective.
