==================
Decorators Are Fun
==================

:Author: Matt Wilson <matt@tplus1.com>

* `decoratortalk.py module`_
* `Slides`_
* `HTML version`_
* `original reST file`_
* `My blog`_

.. _`decoratortalk.py module`: http://scratch.tplus1.com/decoratortalk/decoratortalk.py
.. _`Slides`: http://scratch.tplus1.com/decoratortalk/slides.html
.. _`HTML version`: http://scratch.tplus1.com/decoratortalk/index.html
.. _`original reST file`: http://scratch.tplus1.com/decoratortalk/talk.rst
.. _`My blog`: http://blog.tplus1.com

What are decorators?
====================

.. image:: christopher-lowell.jpg

Decorators are functions that wrap around other functions.  Sometimes a
decorator monkeys with the stuff going into or out of the decorated function.

Decorators depend on two fun facts...

Functions can be parameters
===========================

map applies a function to every element of a sequence.

>>> map(len,  ['a', 'bb', 'ccc'])
[1, 2, 3]
>>> from string import upper
>> map(upper, ['a', 'bb', 'ccc'])
['A', 'BB', 'CCC']

Functions can make functions
============================

>>> def add_n_to_x(n):
...     def add_to_x(x):
...         return n + x
...     return add_to_x
...
>>> add3_to_x = add_n_to_x(3)

What is add3_to_x?

.. class:: incremental

>>> type(add3_to_x)
<type 'function'>


Functions making functions
==========================

What will add3_to_x(8) return?

.. class:: incremental

>>> add3_to_x(8)
11

What about add_n_to_x(7)(3)?

.. class:: incremental

>>> add_n_to_x(7)(3)
10

Partial function application
============================

Ruby won't get this until 1.9, so feel smug.

>>> def f(a, b):
...     "Returns a minus b"
...     return a - b
...
>>> from functools import partial
>>> subtract_one = partial(f, b=1)

People argue over whether this is currying or not.

Usage
=====

functools.partial returns a new object based on the original function, except
one or more parameters are set to fixed values.

>>> subtract_one(5)
4

>>> subtract_one_from_three = \
... partial(subtract_one, a=3)

>>> subtract_one_from_three()
2

Stop for discussion
===================

.. image:: dalek.jpg

Simple decorator examples
=========================

"What you do at Initech is, you take the specifications from the customers
and you bring them down to the software engineers."

.. image:: smykowsky.jpg

smykowsky
=========

::

    >>> def smykowsky(f):
    ...
    ...     def h(*args, **kwargs):
    ...         return f(*args, **kwargs)
    ...
    ...     return h

* Every time smykowsky is called, it makes a new function h and then returns it.
* h doesn't get called when smykowsky gets called.

Worse than useless
==================

smykowsky mangles the function signature of the function he decorates.

f requires exactly two parameters:

>>> f(3, 1)
2
>>> f(1, 2, 3) # kaboom!
Traceback (most recent call last):
    ...
TypeError: f() takes exactly 2 arguments (3 given)

Worse than useless
==================

We can inspect the function f to find out the parameters it requires:

>>> import inspect
>>> inspect.getargspec(f)
(['a', 'b'], None, None, None)

Worse than useless
==================

Now see what happens when smykowski gets involved:

>>> h = smykowsky(f)
>>> inspect.getargspec(h)
([], 'args', 'kwargs', None)

The @ symbol
============

>>> @smykowsky
... def g():
...     return True

Is a shortcut for:

>>> def g():
...     return True
...
>>> g = smykowsky(g)

The downside of @
=================

The inner function is trapped forever! 

.. image:: doctor-orpheus.jpg

Less ridiculous
===============

::

    >>> def as_string(f):
    ...     "convert results of function f to string."
    ...     
    ...     def g(*args, **kwargs):
    ...         return str(f(*args, **kwargs))
    ...     return g

What will as_string(f)(3, 1) + "abc" return?

.. class:: incremental

>>> as_string(f)(3, 1) + "abc"
'2abc'

Use a decorator for logging
===========================

Panicked developers sometimes do this:

>>> from decoratortalk import log
>>> def gross(a, b):
...     log.debug("a is %s and b is %s." % (a, b))
...     y = f(a, b)
...     log.debug("y is %s." % y)
...     return y

That logging can be moved into a decorator instead...

Use a decorator for logging
===========================

>>> from decoratortalk import debuglogger
>>> f(3, 1)
2
>>> logging_f = debuglogger(f)
>>> logging_f(3, 1)
args is (3, 1) and kwargs is {}.
results from calling f is 2.
2

Why use decorators?
===================

* Reuse code
* Is a gateway drug to the larger world of functional programming
* Separate code into layers, in other words, aspects
* You tell me

Build a decorator factory
=========================

.. image:: babushka.jpg

add1 
====

I'll revise this in the next slide::

    >>> def add1(f):
    ...     "Return a new function returns 1 + f(...)"
    ...     
    ...     def g(*args, **kwargs):
    ...         return 1 + f(*args, **kwargs)
    ...
    ...     return g


add1 revisited
==============

Instead of writing add1 from scratch, it would be much nicer to be able
to do something like:

>>> from decoratortalk import add_n
>>> add1 = add_n(1)

We could reuse add_n to make other decorators too:

>>> subtract5 = add_n(-5)

Introducing add_n
=================
::

    >>> def add_n(n):
    ...     "Return a decorator that adds n."
    ...     def dec(f):
    ...         def h(*args, **kwargs):
    ...             return n + f(*args, **kwargs)
    ...         return h
    ...     return dec

The decorator module
====================

Two problems with decorators so far:

* They mangle the function signature.
* Writing decorator factories requires too much thinking.

.. image:: michele-simionato.jpg

Smykowsky part two
==================

You have to write your decorator with a parameter f.

::

    >>> from decorator import decorator
    >>> def smykowsky2(f, *args, **kwargs):
    ...     return f(*args, **kwargs)

Trying it out
=============

>>> dec = decorator(smykowsky2)
>>> f2 = dec(f)
>>> from inspect import getargspec as g
>>> g(f) == g(f2)
True

We get the docstring too!
=========================

>>> print f.__doc__
Returns a minus b
>>> f3 = smykowsky(f)
>>> print f3.__doc__
None
>>> print f2.__doc__
Returns a minus b

Less silly example
==================

Functions often coerce parameters into certain types, sort of like
this::

    >>> def j(x):
    ...     if isinstance(x, (list, tuple)):
    ...         xlist = x
    ...     else:
    ...         xlist = [x]
    ...
    ...     # Now do the real stuff

Use a decorator
===============

::

    >>> def as_list(f, x):
    ...     "Assure the arg x is a list."
    ...     if isinstance(x, (list, tuple)):
    ...         xlist = x
    ...     else:
    ...         xlist = [x]
    ...     return f(xlist)

Test drive
==========

>>> from decorator import decorator
>>> @decorator(as_list)
... def j2(x):
...     print type(x), x
...

>>> j2(99)
<type 'list'> [99]

>>> j2([1, 2])
<type 'list'> [1, 2]

Better decorator factories
==========================

The decorator module version also offers a better approach for decorator
factories::

    >>> class AddX(object):
    ...    def __init__(self, x):
    ...        self.x = x
    ...    def call(self, f, *args, **kwargs):
    ...        return self.x + f(*args, **kwargs)
    ...
    >>> AddX = decorator(AddX)

Better decorator factories
==========================

>>> @AddX(13)
... def f(a, b):
...    return a + b

What will f(1, 1) return?

.. class:: incremental

>>> f(1, 1)
15

Some real-world examples
========================

Developers working on Python 3k

.. image:: concrete.jpg

classmethod
===========

Both classes and instances can call methods decorated as classmethods.

>>> class C(object):
...     @classmethod
...     def spam(cls):
...         return "delicious!"
...

classmethod
===========

Now classes and instances can call these methods.

>>> C.spam()
'delicious!'
>>> c1 = C()
>>> c1.spam()
'delicious!'

Alternate constructors
======================

>>> d = {}
>>> d.fromkeys('abc', 99)
{'a': 99, 'c': 99, 'b': 99}
>>> dict.fromkeys('def', 98)
{'e': 98, 'd': 98, 'f': 98}

An instance made other new instances.

staticmethod
============

If you decorate with @staticmethod, your method won't get an extra
parameter like self or cls.  Some reasons to use staticmethod:

* Make Java programmers feel less homesick
* Put associated functions inside the relevant class, so we don’t
  pollute the module namespace
* Makes it obvious to the reader that a method won't alter the object.

staticmethods for DI
====================

staticmethods useful when you want to add functions to a class:

::

    >>> class C(object):
    ...     g1 = f
    ...     g2 = staticmethod(f)
    >>> i = C()

What will happen if I call i.g1(3, 1)?

staticmethods for DI
====================
::

    >>> i.g1(3, 1)
    ------------------------------------------------------------
    Traceback (most recent call last):
      File "<ipython console>", line 1, in <module>
    TypeError: f() takes exactly 2 arguments (3 given)

However, g2 will work fine::

    >>> i.g2(9, 8)
    1

Properties
==========

They look like attributes, but are really methods.

>>> class Rectangle(object):
...     def calculate_area(self):
...         return self.height * self.width
...
...     area = property(calculate_area)

Properties
==========

>>> r = Rectangle()
>>> r.height, r.width = 3, 4
>>> r.calculate_area()
12
>>> r.area
12

>>> getattr(r, 'area')
12

property can be a decorator
===========================

>>> class Square(object):
...     @property
...     def area(self):
...         return self.side ** 2
...
>>> sq = Square()
>>> sq.side = 3
>>> sq.area
9

properties in 2.6
=================

Specify setters and deleters as properties::

    class C(object):
        @property
        def x(self):
            return self._x

        @x.setter
        def x(self, value):
            self._x = value


properties in 2.6
=================

Subclasses can tweak getters::

    class D(C):
        @C.x.getter
        def x(self):
            return self._x * 2

        @x.setter
        def x(self, value):
            self._x = value / 2



Decorators can just watch
=========================

A decorator doesn't HAVE to tweak inputs or return values.

You can use a decorator to tell another system about the presence of a
function.

>>> functions = list()
>>> def register_function(f):
...     functions.append(f)
...     return f
...

Generic functions
=================

Dispatching on parameters rather than types

.. image:: cat.jpg

Pet groomer example
===================

* Groom dogs and cats differently.
* Offer a standard and a premium package.

Four possible procedures total {A, B, C, D}

========  === ===
...       Dog Cat
Standard  A   C
Premium   B   D
========  === ===


First version of groom
======================

::

    >>> def groom1(pet, package):
    ...     "Groom according to pet and package types"
    ...     if isinstance(pet, Dog):
    ...         if isinstance(package, Standard):
    ...             A()
    ...         else:
    ...             B()

First version of groom
======================

::

    ...     elif isinstance(pet, Cat):
    ...         if isinstance(package, Standard):
    ...             C()
    ...         else:
    ...             D()


Same idea, without the elsifs
=============================

>>> def groom2(pet, package):
...     dispatcher = {(Dog, Standard):A,
...                   (Dog, Premium):C,
...                   (Cat, Standard):B,
...                   (Cat, Premium):D}
...     key = (type(pet), type(package))
...     f = dispatcher[key]
...     f()

Why is this bad?
=================

We have one function that is going to keep growing longer and longer and
longer as we add more branches.

.. image:: dog.jpg

Using classes helps
===================

We can make groom a method rather than a function.  Dog.groom and Cat.groom
still need to test the type of package.

>>> class Cat(Pet):
...
...     def groom(self, package):
...         if isinstance(package, Standard):
...             C()
...         else:
...             D()

Single Dispatching
==================

When I call mittens.groom(x), Python runs Cat.groom(mittens, x).

mittens is effectively a parameter that python dispatches on.

Since mittens's type is the only parameter python looks at in order to find
the right method to run, this is called "single dispatch".

Ancestral Wisdom
================

In 1987, common lisp introduced generic methods.  Generic methods can dispatch
based on the types and values of any of the parameter. 

.. image:: lisplogo.png

Common Lisp version of groom
============================

::

    (defgeneric groom (pet grooming-package))

    (defmethod groom ((pet dog) 
                      (grooming-package standard-groom)) 
        "cheapskate dog owner" 
        (A))

But you came here for python instead...

PEAK-Rules
==========

>>> from peak.rules import abstract, when, after
>>> @abstract
... def groom3(pet, package):
...     "Like defgeneric"
... 
>>> @when(groom3, (Cat, Premium))
... def snootycat(x1, x2):
...     D()

In use
======

>>> groom3(Cat(), Premium())
wash
dry
hairball medicine
A ridiculous haircut

Why is this better than subclassing?
====================================

Now the vet wants to know if your dog is young or old.  Young dogs get
different treatments than old dogs.

Will you subclass by this dimension too?

The @after rule
===============

Imagine that the premium package for dogs is really just the standard
package plus something else::  

    >>> def B():
    ...     A()
    ...     print 'toothbrushing'

PEAK Rules can first run one rule, then run another, so this is not
necessary.

The @after rule
===============

First make a rule that applies to dogs getting the standard or premium
package:

>>> @when(groom3, (Dog, Package))
... def everydog(x1, x2):
...     A()
... 

Remember Package is the superclass of Standard and Premium.  So this rule
will match any grooming.

The @after rule
===============

Now make a rule that runs AFTER the first rule.

>>> def B2():
...     "Like B, but doesn't call A first"
...     print 'toothbrushing'

>>> @after(groom3, (Dog, Premium))
... def fancydog(x1, x2):
...     B2()

In use
======

>>> groom3(Dog(), Standard())
wash
flea dip
dry

>>> groom3(Dog(), Premium())
wash
flea dip
dry
toothbrushing

More fancier
============

You can do more than test on types!  Anything goes!

>>> @abstract
... def f(i):
...     "abstract!"
...     

>>> @when(f, "i % 2")
... def f(i):
...     print "%d must be an odd number." % i
... 

Disambiguation
==============

PEAK-Rules isn't magic.  If you give it a parameter that matches two
"when" rules, it will fuss.

If you give it a parameter that matches NO rules, it will fuss.

Apocrypha
=========

Stuff that I can talk about if time permits.

doctests are great
==================

I can test all the code samples in the text file because of the doctest
module.

Doctests in a function's docstring guarantee that the function works and
provide examples of intended use.

You can store really long doctests in a separate file.

Multiple Decorators can be tricky
=================================

>>> @as_string
... @add1
... def g(a, b):
...     return a + b
...
>>> g(3, 4)
'8'

Now switch the order
====================

>>> @add1
... @as_string
... def g(a, b):
...     return a + b
...
>>> g(3, 4)
Traceback (most recent call last):
    ...
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Class decorators
================

Apply that debuglogger to every method in a class::

    >>> from inspect import getmembers, ismethod
    >>> from decoratortalk import debuglogger
    >>> def debug_everything(C):
    ...     "danger -- this alters the C param."
    ...     for name, value in getmembers(C, ismethod):
    ...         setattr(C, name, debuglogger(value))
    ...     return C
    ...

Class decorators
================

In 2.5, you  have to apply it like this::

    >>> debug_everything(C) #2.5

In 2.6, you can do this::

    >>> @debug_everything
    ... class C(object):
    ...     def f1(self, a):
    ...         return a + 1
    ... 

Class decorators
================

Here's how it works::

    >>> c = C()
    >>> c.f1(99)
    args is (<__main__.C object at 0x8366fec>, 99) and kwargs is {}.
    results from calling f1 is 100.
    100


Class decorators vs metaclasses
===============================

* both allow clever postprocessing.
* metaclasses do their magic on subclasses.
* metaclasses may block the creation of a class or substitute something
  else in place of the original class.
* both can make your code gorgeous or horrifying.

Links
=====

* `PEP 318 <http://peak.telecommunity.com/DevCenter/RulesReadme>`_ introduced decorators.

* `Michele Simionato's decorator package
  <http://www.phyast.pitt.edu/~micheles/python/documentation.html>`_

* Guido wrote a blog entry called `Five-minute multimethods in Python
  <http://www.artima.com/weblogs/viewpost.jsp?thread=101605>`_.

* David Mertz wrote a `multimethods package
  <http://www.ibm.com/developerworks/linux/library/l-pydisp.html>`_

* `Practical Common Lisp <http://gigamonkeys.com/book>`_

* `PEAK Rules project <http://peak.telecommunity.com/DevCenter/RulesReadme>`_

* `What's new in 2.6 <http://docs.python.org/dev/whatsnew/2.6.html>`_