<jupyter_start><jupyter_text>Copyright 2016 Google Inc. All Rights Reserved.Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License at    http://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License. Table of Contents<jupyter_code>%%javascript
// From https://github.com/kmahelona/ipython_notebook_goodies
$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')<jupyter_output><empty_output><jupyter_text>Basics There's lots of guides out there on decorators ([this](http://thecodeship.com/patterns/guide-to-python-function-decorators/) one is good), but I was never really sure when I would need to use decorators. Hopefully this will help motivate them a little more. Here I hope to show you:* When decorators might come in handy* How to write one* How to generalize using `*args` and `**kwargs` sorcery. You should read this if:* You've heard of decorators and want to know more about them, and/or* You want to know what `*args` and `**kwargs` mean.If you're here just for `*args` and `**kwargs`, start reading [here](args).  MotivationLet's say you're defining methods on numbers:<jupyter_code>def add(n1, n2):
    return n1 + n2

def multiply(n1, n2):
    return n1 * n2

def exponentiate(n1, n2):
    """Raise n1 to the power of n2"""
    import math
    return math.pow(n1, n2)<jupyter_output><empty_output><jupyter_text>Well, we only want these functions to work if both inputs are numbers. So we could do:<jupyter_code>def is_number(n):
    """Return True iff n is a number."""
    # A number can always be converted to a float
    try:
        float(n)
        return True
    except ValueError:
        return False
    
def add(n1, n2):
    if not (is_number(n1) and is_number(n2)):
        print("Arguments must be numbers!")
        return
    return n1 + n2

def multiply(n1, n2):
    if not (is_number(n1) and is_number(n2)):
        print("Arguments must be numbers!")
        return
    return n1 * n2

def exponentiate(n1, n2):
    """Raise n1 to the power of n2"""
    if not (is_number(n1) and is_number(n2)):
        print("Arguments must be numbers!")
        return
    import math
    return math.pow(n1, n2)<jupyter_output><empty_output><jupyter_text>But this is yucky: we had to copy and paste code. This should always make you sad! For example, what if you wanted to change the message slightly? Or to return an error instead? You'd have to change it everywhere it appears... We want the copy & pasted code to live in just one place, so any changes just go there (DRY code: Don't Repeat Yourself). So let's **refactor**.<jupyter_code>def validate_two_arguments(n1, n2):
    """
    Returns True if n1 and n2 are both numbers.
    """
    if not (is_number(n1) and is_number(n2)):
        return False
    return True

def add(n1, n2):
    if validate_two_arguments(n1, n2):
        return n1 + n2

def multiply(n1, n2):
    if validate_two_arguments(n1, n2):
        return n1 * n2

def exponentiate(n1, n2):
    """Raise n1 to the power of n2"""
    if validate_two_arguments(n1, n2):
        import math
        return math.pow(n1, n2)<jupyter_output><empty_output><jupyter_text>This is definitely better. But there's still some repeated logic. Like, what if we want to return an error if we don't get numbers, or print something before running the code? We'd still have to make the changes in multiple places. The code isn't DRY.   Basic decoratorsWe can refactor further with the **decorator pattern**.We want to write something that looks like        @decorator    def add(n1, n2):        return n1 + n2so that all the logic about validating `n1` and `n2` lives in one place, and the functions just do what we want them to do.         Since the @ syntax just means `add = decorator(add)`, we know the decorator needs to take a function as an argument, and it needs to return a function. (This should be confusing at first. Functions returning functions are scary, but think about it until that doesn't seem outlandish to you.)This returned function should act the same way as `add`, so it should take two arguments. And within this returned function, we want to first check that the arguments are numbers. If they are, we want to call the original function that we decorated (in this case, `add`). If not, we don't want to do anything. Here's what that looks like (there's a lot here, so use the comments to understand what's happening):<jupyter_code># The decorator: takes a function.
def validate_arguments(func):
    # The decorator will be returning wrapped_func, a function that has the 
    # same signature as add, multiply, etc.
    def wrapped_func(n1, n2):
        # If we don't have two numbers, we don't want to run the function. 
        # Best practice ("be explicit") is to raise an error here 
        # instead of just returning None.
        if not validate_two_arguments(n1, n2):
            raise Exception("Arguments must be numbers!")
        # We've passed our checks, so we can call the function with the passed in arguments.
        # If you like, think of this as
        #   result = func(n1, n2)
        #   return result
        # to distinguish it from the outer return where we're returning a function.
        return func(n1, n2)
    # This is where we return the function that has the same signature.
    return wrapped_func
@validate_arguments
def add(n1, n2):
    return n1 + n2
# Don't forget, the @ syntax just means
# add = validate_decorator(add)

print(add(1, 3))
try:
    add(2, 'hi')
except Exception as e:
    print("Caught Exception: {}".format(e))<jupyter_output>4
Caught Exception: Arguments must be numbers!<jupyter_text>This pattern is nice because we've even refactored out all the validation logic (even the "if blah then blah" part) into the decorator.   Generalizing with \*args and \**kwargsWhat if we want to validate a function that has a different number of arguments?<jupyter_code>@validate_arguments  # Won't work!
def add3(n1, n2, n3):
    return n1 + n2 + n3

add3(1, 2, 3)<jupyter_output><empty_output><jupyter_text>We can't decorate this because the wrapped function expects 2 arguments. Here's where we use the `*` symbol. I'll write out the code so you can see how it looks, and we'll look at what `*args` is doing below.<jupyter_code># The decorator: takes a function.
def validate_arguments(func):
    # Note the *args! Think of this as representing "as many arguments as you want". 
    # So this function will take an arbitrary number of arguments.
    def wrapped_func(*args):
        # We just want to apply the check to each argument.
        for arg in args:
            if not is_number(arg):
                raise Exception("Arguments must be numbers!")
        # We also want to make sure there's at least two arguments.
        if len(args) < 2:
            raise Exception("Must specify at least 2 arguments!")
        
        # We've passed our checks, so we can call the function with the 
        # passed-in arguments.
        # Right now, args is a tuple of all the different arguments passed in 
        # (more explanation below), so we want to expand them back out when 
        # calling the function.
        return func(*args)
    return wrapped_func
@validate_arguments  # This works
def add3(n1, n2, n3):
    return n1 + n2 + n3

add3(1, 2, 3)
@validate_arguments  # And so does this
def addn(*args):
    """Add an arbitrary number of numbers together"""
    cumu = 0
    for arg in args:
        cumu += arg
    return cumu
print(addn(1, 2, 3, 4, 5))
# range(n) gives a list, so we expand the list into positional arguments...
print(addn(*range(10)))<jupyter_output>15
45<jupyter_text>`*args`What is this `*` nonsense?You've probably seen `*args` and `**kwargs` in documentation before. Here's what they mean:* When calling a function, `*` **expands an iterable** into **positional arguments**.     * Terminology note: in a call like `bing(1, 'hi', name='fig')`, `1` is the first positional argument, `'hi'` is the second positional argument, and there's a keyword argument `'name'` with the value `'fig'`.* When defining a signature, `*args` represents an **arbitrary number of positional arguments**.<jupyter_code>def foo(*args):
    print("foo args: {}".format(args))
    print("foo args type: {}".format(type(args)))
    
# So foo can take an arbitrary number of arguments
print("First call:")
foo(1, 2, 'a', 3, True)

# Which can be written using the * syntax to expand an iterable
print("\nSecond call:")
l = [1, 2, 'a', 3, True]
foo(*l)<jupyter_output>First call:
foo args: (1, 2, 'a', 3, True)
foo args type: <type 'tuple'>

Second call:
foo args: (1, 2, 'a', 3, True)
foo args type: <type 'tuple'><jupyter_text>Back to the decorator(If you're just here for \*args and \*\*kwargs, skip down to [here](kwargs))So let's look at the decorator code again, minus the comments:    def validate_decorator(func):        def wrapped_func(*args):            for arg in args:                if not is_number(arg):                    print("arguments must be numbers!")                    return            return func(*args)        return wrapped_func        * `def wrapped_func(*args)` says that `wrapped_func` can take an arbitrary number of arguments.* Within `wrapped_func`, we interact with `args` as a tuple containing all the (positional) arguments passed in. * If all the arguments are numbers, we call `func`, the function we decorated, by **expanding** the `args` tuple back out into positional arguments: `func(*args)`.* Finally the decorator needs to return a function (remember that the `@` syntax is just sugar for `add = decorator(add)`.Congrats, you now understand decorators! You can do tons of other stuff with them, but hopefully now you're equipped to read the other guides online. --- As for `**kwargs`: * When calling a function, `**` **expands a dict** into **keyword arguments**.* When defining a signature, `**kwargs` represents an **arbitrary number of keyword arguments**.<jupyter_code>def bar(**kwargs):
    print("bar kwargs: {}".format(kwargs))

# bar takes an arbitrary number of keyword arguments
print("First call:")
bar(location='US-PAO', ldap='awan', age=None)

# Which can also be written using the ** syntax to expand a dict
print("\nSecond call:")
d = {'location': 'US-PAO', 'ldap': 'awan', 'age': None}
bar(**d)<jupyter_output>First call:
bar kwargs: {'age': None, 'location': 'US-PAO', 'ldap': 'awan'}

Second call:
bar kwargs: {'age': None, 'location': 'US-PAO', 'ldap': 'awan'}<jupyter_text>And in case your head doesn't hurt yet, we can do both together:<jupyter_code>def baz(*args, **kwargs):
    print("baz args: {}. kwargs: {}".format(args, kwargs))
    
# Calling baz with a mixture of positional and keyword arguments
print("First call:")
baz(1, 3, 'hi', name='Joe', age=37, occupation='Engineer')

# Which is the same as
print("\nSecond call:")
l = [1, 3, 'hi']
d = {'name': 'Joe', 'age': 37, 'occupation': 'Engineer'}
baz(*l, **d)<jupyter_output>First call:
baz args: (1, 3, 'hi'). kwargs: {'age': 37, 'name': 'Joe', 'occupation': 'Engineer'}

Second call:
baz args: (1, 3, 'hi'). kwargs: {'age': 37, 'name': 'Joe', 'occupation': 'Engineer'}<jupyter_text>--- Advanced decoratorsThis section will introduce some of the many other useful ways you can use decorators. We'll talk about* Passing arguments into decorators* `functools.wraps`* Returning a different function* Decorators and objects.Use the [table of contents](toc) at the top to make it easier to look around. Decorators with argumentsA common thing to want to do is to do some kind of configuration in a decorator. For example, let's say we want to define a `divide_n` method, and to make it easy to use we want to hide the existence of integer division. Let's define a decorator that converts arguments into floats.<jupyter_code>def convert_arguments(func):
    """
    Convert func arguments to floats.
    """
    # Introducing the leading underscore: (weakly) marks a private 
    # method/property that should not be accessed outside the defining
    # scope. Look up PEP 8 for more. 
    def _wrapped_func(*args):
        new_args = [float(arg) for arg in args]
        return func(*new_args)
    return _wrapped_func

@convert_arguments
@validate_arguments
def divide_n(*args):
    cumu = args[0]
    for arg in args[1:]:
        cumu = cumu / arg
    return cumu
# The user doesn't need to think about integer division!
divide_n(103, 2, 8)<jupyter_output><empty_output><jupyter_text>But now let's say we want to define a `divide_n_as_integers` function. We could write a new decorator, or we could alter our decorator so that we can specify what we want to convert the arguments to. Let's try the latter. (For you smart alecks out there: yes you could use the `//` operator, but you'd still have to replicate the logic in `divide_n`. Nice try.)<jupyter_code>def convert_arguments_to(to_type=float):
    """
    Convert arguments to the given to_type by casting them.
    """
    def _wrapper(func):
        def _wrapped_func(*args):
            new_args = [to_type(arg) for arg in args]
            return func(*new_args)
        return _wrapped_func
    return _wrapper


@validate_arguments
def divide_n(*args):
    cumu = args[0]
    for arg in args[1:]:
        cumu = cumu / arg
    return cumu


@convert_arguments_to(to_type=int)
def divide_n_as_integers(*args):
    return divide_n(*args)


@convert_arguments_to(to_type=float)
def divide_n_as_float(*args):
    return divide_n(*args)


print(divide_n_as_float(7, 3))
print(divide_n_as_integers(7, 3))<jupyter_output>2.33333333333
2<jupyter_text>Did you notice the tricky thing about creating a decorator that takes arguments? **We had to create a function to "return a decorator".** The outermost function, `convert_arguments_to`, returns a function that takes a function, which is what we've been calling a "decorator". To think about why this is necessary, let's start from the form that we wanted to write, and unpack from there. We wanted to be able to do:    @decorator(decorator_arg)    def myfunc(*func_args):        pass        Unpacking the syntactic sugar gives us    def myfunc(*func_args):        pass    myfunc = decorator(decorator_arg)(myfunc)      Written this way, it should immediately be clear that `decorator(decorator_arg)` **returns a function that takes a function**. So that's how you write a decorator that takes an argument: it actually has to be a function that takes your decorator arguments, and returns a function that takes a function.   functools.wrapsIf you've played around with the examples above, you might've seen that the name of the wrapped function changes after you apply a decorator... And perhaps more importantly, the docstring of the wrapped function changes too (this is important for when generating documentation, e.g. with Sphinx).<jupyter_code>@validate_arguments
def foo(*args):
    """foo frobs bar"""
    pass


print(foo.__name__)
print(foo.__doc__)<jupyter_output>wrapped_func
None<jupyter_text>[`functools.wraps`](https://docs.python.org/2/library/functools.htmlfunctools.wraps) solves this problem. Use it as follows:<jupyter_code>from functools import wraps

def better_validate_arguments(func):
    @wraps(func)
    def wrapped_func(*args):
        for arg in args:
            if not is_number(arg):
                raise Exception("Arguments must be numbers!")
        if len(args) < 2:
            raise Exception("Must specify at least 2 arguments!")
        return func(*args)
    return wrapped_func


@better_validate_arguments
def bar(*args):
    """bar frobs foo"""
    pass


print(bar.__name__)
print(bar.__doc__)<jupyter_output>bar
bar frobs foo<jupyter_text>Think of the `@wraps` decorator making it so that `wrapped_func` knows what function it originally wrapped.   Returning a different functionDecorators don't even have to return the function that's passed in. You can have some fun with this...<jupyter_code>def jedi_mind_trick(func):
    def _jedi_func():
        return "Not the droid you're looking for"
    return _jedi_func

    
@jedi_mind_trick
def get_droid():
    return "Found the droid!"


get_droid()<jupyter_output><empty_output>