The Power of Python Decorators

At their core, Python’s decorators allow you to extend and modify the behavior of a callable (functions, methods, and classes) without permanently modifying the callable itself.

Any sufficiently generic functionality you can tack on to an existing class or function’s behavior makes a great use case for decoration. This includes the following:

• logging
• enforcing access control and authentication
• instrumentation and timing functions
• rate-limiting
• caching and more

Sure, decorators are relatively complicated to wrap your head around for the first time, but they’re a highly useful feature that you’ll often encounter in third-party frameworks and the Python standard library. Explaining decorators is also a make or break moment for any good Python tutorial. I’ll do my best here to introduce you to them step by step.

Before you dive in however, now would be an excellent moment to refresh your memory on the properties of first-class functions in Python. The most important first-class functions takeaways for understanding decorators are:

Python Decorator Basics

Now, what are decorators really? They decorate or wrap another function and let you execute code before and after the wrapped function runs.

Decorators allow you to define reusable building blocks that can change or extend the behavior of other functions. And, they let you do that without permanently modifying the wrapped function itself. The function’s behavior changes only when it’s decorated.

What might the implementation of a simple decorator look like? In basic terms, a decorator is a callable that takes a callable as input and returns another callable.

The following function has that property and could be considered the simplest decorator you could possibly write:

def null_decorator(func):
    return func

As you can see, null_decorator is a callable (it’s a function), it takes another callable as its input, and it returns the same input callable without modifying it.

Let’s use it to decorate (or wrap) another function:

def greet():
    return 'Hello!'

greet = null_decorator(greet)

>>> greet()
'Hello!'

In this example, I’ve defined a greet function and then immediately decorated it by running it through the null_decorator function. I know this doesn’t look very useful yet. I mean, we specifically designed the null decorator to be useless, right? But in a moment this example will clarify how Python’s special-case decorator syntax works.

Instead of explicitly calling null_decorator on greet and then reassigning the greet variable, you can use Python’s @ syntax for decorating a function more conveniently:

@null_decorator
def greet():
    return 'Hello!'

>>> greet()
'Hello!'

Putting an @null_decorator line in front of the function definition is the same as defining the function first and then running through the decorator. Using the @ syntax is just syntactic sugar and a shortcut for this commonly used pattern.

Note that using the @ syntax decorates the function immediately at definition time. This makes it difficult to access the undecorated original without brittle hacks. Therefore you might choose to decorate some functions manually in order to retain the ability to call the undecorated function as well.

Decorators Can Modify Behavior

Now that you’re a little more familiar with the decorator syntax, let’s write another decorator that actually does something and modifies the behavior of the decorated function.

Here’s a slightly more complex decorator which converts the result of the decorated function to uppercase letters:

def uppercase(func):
    def wrapper():
        original_result = func()
        modified_result = original_result.upper()
        return modified_result
    return wrapper

Instead of simply returning the input function like the null decorator did, this uppercase decorator defines a new function on the fly (a closure) and uses it to wrap the input function in order to modify its behavior at call time.

The wrapper closure has access to the undecorated input function and it is free to execute additional code before and after calling the input function. (Technically, it doesn’t even need to call the input function at all.)

Time to see the uppercase decorator in action. What happens if you decorate the original greet function with it?

@uppercase
def greet():
    return 'Hello!'

>>> greet()
'HELLO!'

I hope this was the result you expected. Let's take a closer look at what just happened here. Unlike null_decorator, our uppercase decorator returns a different function object when it decorates a function. Exit CTRL + D from python shell and define once again our python functions:

def greet():
    return 'Hello!'

def null_decorator(func):
    return func

def uppercase(func):
    def wrapper():
        original_result = func()
        modified_result = original_result.upper()
        return modified_result
    return wrapper

>>> greet
<function greet at 0x7f01ee5b3510>

>>> null_decorator(greet)
<function greet at 0x7f01ee5b3510>

>>> uppercase(greet)
<function uppercase.<locals>.wrapper at 0x7f01ee5b36a8>

And as you saw earlier, it needs to do that in order to modify the behavior of the decorated function when it finally gets called. The uppercase decorator is a function itself. And the only way to influence the "future behavior" of an input function it decorates is to replace (or wrap) the input function with a closure.

That's why uppercase defines and returns another function (the closure) that can then be called at a later time, run the original input function, and modify its result. Decorators modify the behavior of a callable through a wrapper closure so you don’t have to permanently modify the original. The original callable isn’t permanently modified—its behavior changes only when decorated.

This lets you tack on reusable building blocks, like logging and other instrumentation, to existing functions and classes. It makes decorators such a powerful feature in Python that it’s frequently used in the standard library and in third-party packages.

Applying Multiple Decorators to a Function

Perhaps not surprisingly, you can apply more than one decorator to a function. This accumulates their effects and it’s what makes decorators so helpful as reusable building blocks.

Here's an example. The following two decorators wrap the output string of the decorated function in HTML tags. By looking at how the tags are nested, you can see which order Python uses to apply multiple decorators:

def strong(func):
    def wrapper():
        return '<strong>' + func() + '</strong>'
    return wrapper


def emphasis(func):
    def wrapper():
        return '<em>' + func() + '</em>'
    return wrapper

Now let's take these two decorators and apply them to our greet function at the same time. You can use the regular @ syntax for that and just "stack" multiple decorators on top of a single function:

@strong
@emphasis
def greet():
    return 'Hello!'

What output do you expect to see if you run the decorated function? Will the @emphasis decorator add its <em> tag first, or does @strong have precedence? Here's what happens when you call the decorated function:

>>> greet()
'<strong><em>Hello!</em></strong>'

This clearly shows in what order the decorators were applied: from bottom to top. First, the input function was wrapped by the @emphasis decorator, and then the resulting (decorated) function got wrapped again by the @strong decorator. If you break down the above example and avoid the @ syntax to apply the decorators, the chain of decorator function calls looks like this:

decorated_greet = strong(emphasis(greet))

Decorating Functions That Accept Arguments

All examples so far only decorated a simple nullary greet function that didn't take any arguments whatsoever. Up until now, the decorators you saw here didn't have to deal with forwarding arguments to the input function. If you try to apply one of these decorators to a function that takes arguments, it will not work correctly. How do you decorate a function that takes arbitrary arguments?

This is where Python's *args and **kwargs feature3 for dealing with variable numbers of arguments comes in handy. The following proxy decorator takes advantage of that:

def proxy(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

There are two notable things going on with this decorator:

• It uses the * and ** operators in the wrapper closure definition to collect all positional and keyword arguments and stores them in variables (args and kwargs).
• The wrapper closure then forwards the collected arguments to the original input function using the * and ** "argument unpacking" operators.

How to Write "Debuggable" Decorators

When you use a decorator, really what you're doing is replacing one function with another. One downside of this process is that it "hides" some of the metadata attached to the original (undecorated) function. For example, the original function name, its docstring, and parameter list are hidden by the wrapper closure:

def greet():
    """Return a friendly greeting."""
    return 'Hello!'

decorated_greet = uppercase(greet)

If you try to access any of that function metadata, you'll see the wrapper closure's metadata instead:

>>> greet.__name__
'greet'
>>> greet.__doc__
'Return a friendly greeting.'

>>> decorated_greet.__name__
'wrapper'
>>> decorated_greet.__doc__
None

This makes debugging and working with the Python interpreter awkward and challenging. Thankfully there's a quick fix for this: the functools.wraps decorator included in Python’s standard library. You can use functools.wraps in your own decorators to copy over the lost metadata from the undecorated function to the decorator closure. Here's an example:

import functools

def uppercase(func):
    @functools.wraps(func)
    def wrapper():
        return func().upper()
    return wrapper

Applying functools.wraps to the wrapper closure returned by the decorator carries over the docstring and other metadata of the input function:

import functools

def uppercase(func):
    @functools.wraps(func)
    def wrapper():
        return func().upper()
    return wrapper

@uppercase
def greet():
    """Return a friendly greeting."""
    return 'Hello!'

>>> greet.__name__
'greet'
>>> greet.__doc__
'Return a friendly greeting.'

Stateful Decorators

Sometimes, it's useful to have a decorator that can keep track of state. As a simple example, we will create a decorator that counts the number of times a function is called.

import functools

def count_calls(func):
    @functools.wraps(func)
    def wrapper_count_calls(*args, **kwargs):
        wrapper_count_calls.num_calls += 1
        print(f"Call {wrapper_count_calls.num_calls} of {func.__name__!r}")
        return func(*args, **kwargs)
    wrapper_count_calls.num_calls = 0
    return wrapper_count_calls

@count_calls
def say_whee():
    print("Whee!")

The state — the number of calls to the function—is stored in the function attribute .num_calls on the wrapper function. Here is the effect of using it:

>>> say_whee()
Call 1 of 'say_whee'
Whee!

>>> say_whee()
Call 2 of 'say_whee'
Whee!

>>> say_whee.num_calls
2

Classes as Decorators

The typical way to maintain state is by using classes. In this section, you'll see how to rewrite the @count_calls example from the previous section using a class as a decorator.

Recall that the decorator syntax @my_decorator is just an easier way of saying func = my_decorator(func). Therefore, if my_decorator is a class, it needs to take func as an argument in its __init__() method. Furthermore, the class instance needs to be callable so that it can stand in for the decorated function.

Therefore, a typical implementation of a decorator class needs to implement __init__() and __call__():

import functools

class CountCalls:
    def __init__(self, func):
        functools.update_wrapper(self, func)
        self.func = func
        self.num_calls = 0

    def __call__(self, *args, **kwargs):
        self.num_calls += 1
        print(f"Call {self.num_calls} of {self.func.__name__!r}")
        return self.func(*args, **kwargs)

@CountCalls
def say_whee():
    print("Whee!")

The __init__() method must store a reference to the function and can do any other necessary initialization. The __call__() method will be called instead of the decorated function. It does essentially the same thing as the wrapper() function in our earlier examples.

This @CountCalls decorator works the same as the one in the previous section:

>>> say_whee()
Call 1 of 'say_whee'
Whee!

>>> say_whee()
Call 2 of 'say_whee'
Whee!

>>> say_whee.num_calls
2

Key Takeaways

• Decorators define reusable building blocks you can apply to a callable to modify its behavior without permanently modifying the callable itself.
• The @ syntax is just a shorthand for calling the decorator on an input function. Multiple decorators on a single function are applied bottom to top (decorator stacking).
• As a debugging best practice, use the functools.wraps helper in your own decorators to carry over metadata from the undecorated callable to the decorated one.