Instant Search

Journey to the Decorators world.

Hi Guys , So i was reading about decorators and its very interesting topic in functional programming so i thought to write about them.
Before understanding Decorators we have to understand some pre-things. So i am dividing this section into three parts :

First Section :

About Functions properties :

 

 

  • what are first class ?
  • What is variable lifetime ?
  • Assign function to variables.
  • How use function’s return value ?
  • function can be passed as parameters to other functions.
  • Function can return other functions
  • Inner functions have access to the enclosing scope
  • Function can capture local state

 

Second Section :

About Nested functions :

  • What are nested function?
  • Can we access nested function directly ?
  • Return a function and return data
  • Why nested function return ? Is it necessary to return something from nested function  ?

Third Section :

Everything About Decorators :

  • What are decorators?
  • rules of decorators
  • what is syntactic sugar
  • How parameter and argument works in decorator ?
  • how parameter in main and nested function works ? and how parameter in below decorator function work?
  • accumulate of decorators or order of execution of decorators
  • Decorator that takes argument
  • Variable length parameter issue and solution.
  • difference between  @hello and @hello()
  • how chain of decorator works ?
  • funtools
  • multiple decorators to single function
  • Classes instead of Functions
  • Some examples
  • decorating a decorator
  • Create singleton class with a decorator

 

Ok so its going to be little long but if you want to dive deep into the decorator world from basic to advance then you should continue .. (With cup of coffee) :

 

k let’s start :

 

About Functions properties :

 

  • what are first class ?

A Python function is not only a higher-order function, but also a first-class function, which satisfies following four criteria:

  1. can be created at runtime
  2. can be assigned to a variable
  3. can be passed as a argument to a function
  4. can be returned as the result of a function

Ok so if you have not read my Previous post on objects  post then i request you to read that then continue :

In python everything is object means , Functions are first class and we can pass functions to other functions and we can treat functions same like any other value in python. Means they are not “so” special they works same as they are any other value like any variable.

Python’s functions are first-class objects. You can assign them to variables, store them in data structures, pass them as arguments to other functions, and even return them as values from other functions.

Functions are objects :
All data in a Python program is represented by objects or relations between objects. Things like strings, lists, modules, and functions are all objects. There’s nothing particularly special about functions in Python.

so let’s define a function first :

Ok so we have a function now as i said functions are first class then we can assign function to any variable :

Now we assigned firs to hello function now what is amazing part here that now if we call firs with parameter then it will work same like hello because hello and firs are pointing same object.

So now let’s take another example :

We all have use print function so let’s do a trick

Now run this program and you will get output :

 

So we can assign many names to the same function it will refer to same object , they are not containing any values.

Assign function to variables.

We are going to delete main function in first program.

 

Now we deleted hello function still firs pointing the same object , even hello is deleted and now we can access our function via firs variable. or we can call our function via this.

Here is also one thing to remember that while assigning variable or name to function we always assign like  normal name  “func”  we doesn’t use “func()” when we are not returning anything from function .

Ok so what the point is that when we refer any function to variable like

Then now foo is storing what func() will return but if we are not returning anything then we can assign foo to func like this :

Let’s try same example there :

We have not returned anything from “hello” function and we assigning a name to function with “()” and then calling that variable as function so now it will give error :

Because when we do

it means we must have to return something from func so that foo can store that return value. foo always store that value which its front function(whom its referring)  return.

And most important that if we want to use that variable as function caller then we must return the function from main function :

 

As you can see now i returned 1 from function hello but i am trying to use firs(20) as a caller of hello so now we have to return hello instead of 1 :

You can ask why we must return that function itself from its function then the answer is :

when we assign any value to function like :

so foo always store what func return and if we return the func itself then foo will store that function because its returned by its front function so now foo will work as function caller.

 

What is variable lifetime ?

when we call the function that time variable initialize and then when function return something and exit then those variables destroy. if we try to access those variable after function exist then we can’t access.

How to use function return value :

if we want to access function value after func exist then we can’t access them but if function returned something then we can use that returned value in anywhere in program. so question is how we can use that returned value , Answer is by referring a variable to function.

let’s define a function :

So now i want to use that x value in second function like :

When we try to run this program we get this error :

Because as i said when function exist then its variable destroyed and we can’t access its variable after the function execution.

But we want to use x value in second program then how can i ?

The solution is first we have to save x value for later use and this work done by “return” so we have to return x in first program.
then as i told who will save this return value ?? We can save this return value by referring function to one variable . like

so foo always store what func return (when we use func()  , not func) so we can do this something like this:

 

We first return the value of x and then we save its value by referring the hello function by ref variable then we can now use this x value anywhere in the program we just have to use ref variable.

Function can be passed as parameters to other functions.

 

ok as we know function are first class object so we can pass them as parameter to other function :

As you can see we defined a function and then we defined a second function now we pass a parameter in second function and then we call that parameter as function. Because we pass first function as argument in second so second function parameter is now first function. If you want we can see fun function real name by .__name__ keyword

Second example :

We passed many functions in another built-in function called “List”.

 

Function can return other function:

we returned nested function in first function.
This is closure which i have covered in depth in my previous post.

 

As you can see inner function can access the inclosing scope.

One important thing to remember is not to confuse “return a function” with “return a data value”.

 

 

Function can capture local state :

 

As you can see “by” function didn’t have any parameter still it accessed the x value from hello function.

Notice how they no longer have a text parameter? But somehow they can still access the textparameter defined in the parent function. In fact, they seem to capture and “remember” the value of that argument.

Functions that do this are called lexical closures (or just closures, for short). A closure remembers the values from its enclosing lexical scope even when the program flow is no longer in that scope.

In practical terms this means not only can functions return behaviors but they can also pre-configure those behaviors.

 

one thing you can ask that what are double parenthesis there ??

 

 

Even an inexperienced programmer should be able to make sense of most of this. Reading from left to right, it looks like we want to print the output of a function, and we’re passing an integer – 3 – to that function. However, the second pair of parentheses doesn’t quite make sense.

This notation is different from what we would do if we wanted to pass two arguments to a function; in that case, we’d put them all inside a single pair of parentheses and separate them via commas:

 

So what does the first example using two pairs of parentheses accomplish?

The use of a double parentheses is actually an indicator of one of Python’s coolest features – and that is that functions are themselves, objects! What does this mean?

Let’s work our way up to the snippet above by first defining a very simple function – something that takes an integer as an argument, adds one to it, and returns the result. Pretty simple:

 

However, let’s try to do something a little more interesting. What if we embed this “addone” function within a wrapper function, that throws a second integer into the mix?

You can see that we’re calling the wrapper function with the double parentheses. This is possible because our wrapper function actually returns object representing the addone function, not it’s result.

This is because (on line 4), we are returning “addone”, not “addone()”
In sequence, “funcwrapper(3)” is evaluated first, and it returns the “addone” function itself. Because the first parameter (y) is set to 3, the function that is returned will end up evaluating x + 3 + 1.

Because we have a second pair of parenthesis (2), this ends up being the parameter to the embedded “addone” function, which means that x is set to 2.

 

So we can simple access inner function by this trick hello()() if outer function returned the inner function.

 

So out first section is done , Take a cup of coffee now and come back we will discuss second section.

 

 

Second Section:

Nested Function:

  • What are nested function?

so you have already used nested functions and i have written many times about what are nested functions and how they use.

for reminding again let’s take little tour of nested function :

When we define a function inside a function its called nested function.

Here “bye” is nested function.

How access nested function ?

We can access nested function by some methods , I am going to tell some of them methods :

using return

Using closure

Using func()()

We can’t access nested function directly.

 

Using return method :

we can access nested function by returning nested function from outer function.
#there is difference between returning function and returning data value i have explained above don’t confuse.

Here you can see i have accessed the nested function by returning nested function in outer function. This is one method.

Using closure :

I have written a brief document on closure , here i am  going to give a short example :

Here yu is closure and its accessing the nested function.You can read more about closure in deep in my previous post on closure.

Using func()() method:

We can access nested function by this method too.

#keep in mind that no matter which method we use we must have to return from nested function either function (like func)  or either function data (like func())  . In all method return of nested function is MUST.

Why nested function return ? Is it necessary to return something from nested function  ?

for this question i assume you know about closure , if not then i request please read my previous post on closure and return function .

So why return is necessary for two reasons :

for closure , for nested function access

for decorator for using function value in decorator wrapper function

 

for closure , for nested function access:

Return values are useful for future use and that’s why we return something from functions.

As you know when we write foo=func() then its calle closure and foo always store what func() return so if we want to access inner function via closure then what should we store in closure ? a function call of inner and how we can store ? by returning inner function from outer function . So that’s the trick . When we return inner function from outer function then foo=func()  , foo store that value and when we call foo() then inner function called.

for decorator for using function value in decorator wrapper function :

Why we use return in below of decorator function ?

This is advance example , probably you should see this after decorator section :
Suppose we have a decorator :

 

we would not need this value in decorator because we will need decorator values in below functions  we would need hello values in hal and other functions which will be define below decorator or when we will put decorator before any function definition. not below functions values in decorator
but i am giving this example just to show use of “return” ,

 

So as you can see we have defined two functions and one decorator now if i want to use rt value from hal function to bye function how i can ?

let’s try :

first i tried to print the value of rt there in bye function then it gave me None and i can’t use that value with anything :

#note that hal and bye functions are same because of decorators that’s why we will be able to use hal values in bye function.
So how can i use there ?

well , simply return the value from hal function so that return will save this value for future use and then you can simply use this value by x(*args ) which is same hal(rt)

#note : x is hal function ad *args is rt there. both are pointing same function and parameter.

So now :

So now we can easily access rt value there now even i can perform operations with this value :

Ok now our nested function section finish here. Now we are going to understand main topic on which this post is about :

Decorators : Decorators consider as difficult stuff in python but i have gave and explained enough information and  Prerequisite which are necessary to understand decorators :

Ok i will tell about each and every thing about decorator (except not more deep in class decorator ):

 

 

What are decorators :

what are decorators really? They “decorate” or “wrap” another function and let you execute code before and after the wrapped function runs.

Decorators allow you to define reusable building blocks that can change or extend the behavior of other functions. And they let you do that without permanently modifying the wrapped function itself. The function’s behavior changes only when it’s decorated.

Now what does the implementation of a simple decorator look like? In basic terms, a decorator is a callable that takes a callable as input and returns another callable.

 

a function can return another function.

 

If you can return a function, you can pass one as a parameter:

 

So as you have seen we can do something before any function and after any function via decorator :

Now what is pythonic decorator or syntactic sugar :

so

The decoration in Python is usually not performed in the way we did it in our previous example, even though the notation foo = our_decorator(foo) is catchy and easy to grasp. This is the reason, why we used it! You can also see a design problem in our previous approach. “foo” existed in the same program in two versions, before decoration and after decoration.

We will do a proper decoration now. The decoration occurrs in the line before the function header. The “@” is followed by the decorator function name.

We will rewrite now our initial example. Instead of writing the statement

 

we can write

So we can do above code in pythonic decorator way by very easily like :

or

The previous example, using the decorator syntax:

 

 

Rules of decorator :

So how create a decorator :

first create a function , pass an parameter into this function. (remember this parameter will be work as a function call for that function which will be use with decorator or which will be defined below decorator.)

Now second step create a nested function which will be treat as wrapper for decorator. This wrapper function is actually that function which will work with decorator.

third step is to do the stuff which you want before calling the decorating function and do stuff after calling the decorating function.  func(x) will work like caller for that function.

 

Now return the nested function in first function. Its necessery.

Now our decorator is complete it will print before and after the function where we want.

Let’s test it

Now here is the tricky part , You have to call the first function with testingfunc  as  parameter to this function.

 

Here you can see when we did this then now testingfunc == func because both are parameter and argument of same function.

Now when we called that closure then it will print before and after this testing function.

output is :

That’s it , That’s how decorator works.

How parameter and argument works in decorator :

This is very important part of decorator :
so let’s take example of any decorator :

So now i will describe in brief how parameter and argument works in decorator :

first line :

hello(func)
So when you write any decorator function and then you have to pass a parameter in first function which is just before the wrapper (bye) function. Why we need parameter there because when we pass any parameter there in first function , that is not just parameter that will become function call when we will pass function as parameter.

As you can see in example we pass “hi” as parameter in function hello so when we pass
hello(hi) and we had hello(func) so now func==hi

As we know and i have pointed out that function “hi” and function” bye()”  are same and func()  is caller of function “hi”

So as you can see in the image func is actually the function “Hi” when we pass “hi” as paramter in hello.

So we must have to pass parameter in main outer function:

second thing what about nested function parameter so we have to pass nested function parameter if there is any parameter in function below of decorator :

 

SIDENOTE : default you can give def hello(*args, **kwargs) as default so this will be good for variable length arguments now no matter how many paramter decorator below function have we don’t have to take tension about it. Let’s see it by live example :

So when we write a decorator and then we don’t pass any parameter in inner function but there are parameter in function below decorator then what happens :

It shows error so now we simply give 1 argument to to hi() now what happens :

As we know and i have pointed out that function “hi” and function” bye()”  are same and func()  is caller of function “hi”  so now as its pointing out the error now i have one parameter to bye for correct the error :

#bye and hi are same function but func() is the caller of hi() function.

 

Now again new error because now we have given parameter to bye and hi but we have not passed any argument in func() which is caller of hi() function. so now we have to pass argument because we have given parameter.

Now it run smoothly no error :

now let’s take this example :

so now its very serious condition that one function taking one parameter and second is two parameter but both are using same decorator now what’s the solution :

  • Variable length parameter issue and solution.

So what we have to do is simply pass *args and **kwargs parameters to wrapper of decorator function then no matter how many function function pass how many parameter it will adjust automatically.

 

Now we passed *args and **kwargs to wrapper of decorator function (bye) now it can take variable length of arguments.

This is best solution , always writing decorators you can pass *args and **kwargs to wrapper so it will smoothly pass all functions arguments no matter how many anyone is taking.

Advance use of decorators :

 

Decorators which takes arguments:

For understanding this thing you first have to understand how three function decorator works :

or another second method of accessing nested function :

 

As you can see we have used three functions so for accessing them we have to first make a closure which store first function return (which is inner function call) then we have to make closure of this closure which will store inner function return so this is (wrapper function) now we can do this things very easily via pythonic way of decorator :

 

 

We have to pass argument because for the sake of extrafun(hi) parameter because @extrafun is directly potinting to that function.

Before rushing to the solution, let’s write a little reminder:

 

 

It’s exactly the same. “my_decorator” is called. So when you @my_decorator, you are telling Python to call the function ‘labelled by the variable “my_decorator“‘.

This is important! The label you give can point directly to the decorator—or not.

Let’s get evil. ☺

 

 

 

No surprise here.

Let’s do EXACTLY the same thing, but skip all the pesky intermediate variables:

 

 

 

Let’s make it even shorter:

 

 

 

Hey, did you see that? We used a function call with the “@” syntax! 🙂

 

 

Accumulate of decorators or order of execution of decorators :

Chain of decorators :

so if we use more than one decorators in functions then is order matter ?

Yes , Order matter :

 

Now if we change the order of decorators then :

 

So hence order matter.

Decorating a decorator :

 

Now let’s go something more deep into decorator and let’s decorate a decorator :

or second example is from stackoverflow :

 

 

 

 

It can be used as follows:

 

 

So now what are funtools ?

funtool are also decorator :

why we use them ?

When you use a decorator, you’re replacing one function with another. In other words, if you have a decorator

 

 

then when you say

 

 

it’s exactly the same as saying

 

 

and your function f is replaced with the function with_logging. Unfortunately, this means that if you then say

 

 

it will print with_logging because that’s the name of your new function. In fact, if you look at the docstring for f, it will be blank because with_logging has no docstring, and so the docstring you wrote won’t be there anymore. Also, if you look at the pydoc result for that function, it won’t be listed as taking one argument x; instead it’ll be listed as taking *args and **kwargs because that’s what with_logging takes.

If using a decorator always meant losing this information about a function, it would be a serious problem. That’s why we have functools.wraps. This takes a function used in a decorator and adds the functionality of copying over the function name, docstring, arguments list, etc. And since wraps is itself a decorator, the following code does the correct thing:

 

 

So now let’s see one more example:

suppose we have this program :

 

wait ! wait ! have you seen both funtions are printing that they are function “he” means we lost the original gh function. Now what we have to do is use the functools.wrap which will copy all original data while using decorators :

 

Some best use and examples of decorators :

calculating how much time its take to execute any function

second program , print $ before final bill

 

How many times a function called?

 

Output:

Chain of Decorators

Output:

 

 

Using a Class as a Decorator
We will rewrite the following decorator as a class:

 

The following decorator implemented as a class does the same “job”:
class decorator2(object):

 

Both versions return the same output:

 

 

 

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *