Instant Search

Closure in Python.

Ok , Hi guys so i was reading about Closure in python and during this topic i have not learn only about closure even i learn some other important topics too so let’s divide this section of closure in these parts :

  • What is function ?
  • What is Nested Function?
  • What are free variables and Bound variables ?
  • What are lexical scope and dynamic scope ?
  • What is non-local keyword ?
  • Why non-local keyword is needed ?
  • What is closure and what are rules for creating closures ?
  • why nested function are not closure , what is the difference between nested function and closure ?
  • How free variable is related to closure?
  • What is scoping bug ?
  • what __closure__ does?

Hmm, So we will get to know all of these awesome things in a single post and most important that these things are necessary in function depth knowledge.

What is function ?

So in python a function is created by “def” keyword and its takes parameter and return the result.
You can read my this post to know more deep about function and return values.

simple example of python function is :

So after creating a function we must have to call the function otherwise it will not activate.  according to strict rules of function programming we must return in any function , when we don’t return anything in function and call  the function without returning anything then it return default value which is “None”

What is Nested Function?

So we talked about function now take another step and let’s know what is nested function. So nested is function is when we define a function inside another function. The outer function is called “Enclosed function” and the inner function is called “Nested function. Nested function can access values from enclosed scope but can’t modify them , if you want to modify those values too then we have to declare them in nested function too or use “Non-local” keyword.

 

So here hello(x) is Enclosed function and hello2 is nested function. hello2 can access hello variables but it can’t modify them.

 

What are free variable and bound variable :

 

Ok so we know about python scope and namespace , If you don’t know about them i suggested please read my this post to know in deep about namespace and scope rules in python.
When you reference a variable in an expression, the Python interpreter will traverse the scope to resolve the reference in this order:

The  function’s scope:

Any enclosing scopes (like other containing functions)
The scope of the module that contains the code (also called the global scope)
The built-in scope (that contains functions like len and str)
There is a great difference between referring to a name, and assigning to a name.
“return a function” with “return a data value”.

 

So Now a free variable is which is use in local scope but have not defined there , It defined somewhere else.

In short, free variables are variables that are used locally, but defined in an enclosing scope.

A free variable in a code block is a variable that is used there but not defined there.

They often overlap.

Some examples:

 

In the first example, x is a global variable and, in foo, x is a free variable. So this is a case where they overlap.

In the second example, there are two variables named “x”: The global variable, and the variable local to foo. There are no free variables.

(Side note: To reference the global variable x from within foo, use the statement global x.)

In the third example, x is local to bar and y is local to foo. In foo, x is a free variable. There are no global variables.

 

according to the python official documentation :

If a name is bound in a block, it is a local variable of that block, unless declared as nonlocal. If a name is bound at the module level, it is a global variable. (The variables of the module code block are local and global.) If a variable is used in a code block but not defined there, it is a free variable.

So how we can identify which variables are free ?

Simple , we can use locals() in current lexical scope (current function block) to know about free variables.

so in this program :

 

In this example you can see :

 

So d is free variable there.

What is lexical scope and dynamic scope :

Ok so lexical scoping (sometimes known as static scoping ) is a convention used with many programming languages that sets the scope (range of functionality) of a variable so that it may only be called (referenced) from within the block of code in which it is defined. The scope is determined when the code is compiled. A variable declared in this fashion is sometimes called a private variable.

Means if we defined a variable in a function then that block of code called lexical scope for that variable.

 

So lexical scope define that a variable can be called in a current block of function then its it lexical scope.

The opposite approach is known as dynamic scoping . Dynamic scoping creates variables that can be called from outside the block of code in which they are defined. A variable declared in this fashion is sometimes called a public variable.

 

What is non-local keyword ?

it works in exactly the same way as the global statement, except that it is used to refer to variables that are neither global nor local to the function.

It takes the one “closest” to the point of reference in the source code. This is called “Lexical Scoping” and is standard for >40 years now.

Python’s class members are really in a dictionary called __dict__ and will never be reached by lexical scoping.

 

Global functionality within iterated functions rather than the body of the code itself. A Global statement between functions if you will.

Let’s understand this by two examples one is without non-local Variable and second one with non-local variable.

 

Without Non-local variable :

So here we got the output f1 111 and f2 12 , Because second print is taking the value from the enclosed function which is x=11 Now let’s see same example with non-local keyword:

As you can see now we are getting both result same , it means now x=111 is working as global for enclose function too. So nonlocal keyword is use for making a variable global but only in enclose scope.

#glitch 1 :

Take same example in this way :

We used nonlocal keyword still why we are getting f2=12 ?

 

Its because of  python’s flow of execution rule :

Python program always start to run from the first line of the code and move on , so if it find any function then it escape that function because there is no call of this function and then it continue execute whatever is after function , if there is call after function then now it will execute funtion.

So in first example when python started running code then it started from first line and found the def function and escaped it but just after few lines of code it found a call of function which is hello2() function calling so now it execute the function hello2() before going to execute rest of code of first function so when it execute hello2() then x value is now become global because this function returned.

while in second example when it started running from first line then it found function hello and it escape the function then it move on and it doesn’t find any call of any fucntion instead it found print() so it just print the x old value. that’s why we got f2=12

 

Why non-local keyword is needed ?

Non-local keyword is needed because when we have nested function in a function then we can “ONLY” can access the enclosed function’s variables but we can’t modify them , if we want to modify them in nested function then we need to declare them again via nonlocal or simple variable = value

So now let’s take two example :

 

This program will give this error :

Why its giving error ??
Because python scope rule works like this :

 

Any enclosing scopes (like other containing functions)
The scope of the module that contains the code (also called the global scope)
The built-in scope (that contains functions like len and str)
There is a great difference between referring to a name, and assigning to a name.
“return a function” with “return a data value”.

 

Encountering this problem is sometimes called the scoping bug because it can be so surprising to newbies. But this is the intended result. This behavior prevents local variables in a function from polluting the containing module.

 

So what’s actually happening in this program :

Ok so when we do d=d+1 we are actually assigning “d” in hello1() function which makes “d” to local variable of function hello1(). Now if you remember when there is any local variable then python doesn’t try to search that variable in enclosed , global or built-in scope it uses local because the searching variable rule is :

local variable –> enclosed —> global —> built-in

So we are using d in enclosed and trying to modify them so its making d local to hello1() function so when python try to find that variable in local of hello1() it doesn’t found and that’s why its giving error.

Now let’s see same program with nonlocal keyword:

So as we can see we just assign d in hello2() local with non-local keyword and now it takes value from enclose function because it become global now for enclosed function.

now let’s come on the main topic which is what is closure and what are the rules for creating closure :

 

What is difference between return a function and return a value ?

 

So let’s take two example :

 

now its returning data value as return value now let’s see how its return function :

Now its returning a function as return value.

What is closure :

A Closure is a function object that remembers values in enclosing scopes even if they are not present in memory.

A closure uses variables from enclosing scopes. That’s why it’s called a “closure,” it “closes over” or “encloses” those variables.

If a nested function doesn’t use anything from an enclosing scope, then it’s not a closure. There’s no reason for it to be a nested function, instead of just another function defined elsewhere.

As a rule a nested function *is* called a closure.
the fact that a function uses a variable from an enclosing scope is what earns it the label ‘closure’, if it doesn’t use any then it is just a nested function
“nested function” means something with syntax, “closure” means something at runtime
Since that’s usually *why* it’s defined as a nested function.
all closures are nested functions, but not all nested functions are closures.

“closures” are functions which are capable to remember the environment they were created. This functionality, allows you to use variables or methods within the closure wich, in other way,you wouldn’t be able to use either because they don’t exist anymore or they are out of reach due to scope.

 

ok enough definitions now its time to take some action :

 

 

So in above example closur is closure function and it store what hello returned and hello is returning function hell2() which means closur is just a reference to the hell2() function and it remember the values of func1 even when it returned and exit from memory.

 

 

 

 

A CLOSURE is a function object that remembers values in enclosing scopes regardless of whether those scopes are still present in memory. If you have ever written a function that returned another function, you probably may have used closures even without knowing about them.

A quick look at closures

For example, consider the following function generate_power_func which returns another function.

The inner function nth_power is called a closure because, as you will see shortly, it will have access to n which is defined in generate_power_func (the enclosing scope) even after program flow is leaves it. If you want to get too technical, you can say that the function nth_power is closed over the variable n. Let’s call generate_power_func and assign the result to another variable to examine this further.

As expected, when generate_power_func(4) was executed, it created an nth_power function object (at 0x00C46630), and returned it, which we just assigned to raised_to_4 (you can see that id(raised_to_4) == 0x00C46630 == id(nth_power)). Now let’s also delete the name of the original function generate_power_func from the global namespace.

Now it’s time for the closure magic …

Wait a minute! How did this work? We defined n = 4 outside of the local scope of nth_power. How does raised_to_4 (the nth_power function object) know that the value of n is 4? It makes sense that generate_power_func would know about n (and its value, 4) when the program flow is within generate_power_func. But the program flow is currently not within generate_power_func. For that matter generate_power_func does not even exist in the namespace anymore.

The nth_power function object returned by generate_power_func is a closure because it knows about the details of the variable n from the enclosing scope.

 

What are the rules for creating closures ?

four rule of closure:
f = foo(12) now f will store that value which foo will return
We must have a nested function (function inside a function).
The nested function must refer to a value defined in the enclosing function.
The enclosing function must return the nested function.
If a function does not use free variables it doesn’t form a closure.

 

so when we call function using any reference like foo=f(12) and if there are nested function in program and main function is returning nested function then this foo will store then value which main function returning and if its returning nested function it means this foo will be reference to direct call of the nested function.

We must have a nested function.
The nested function must refer to a value defined in the enclosing function. means if nested function is not referring the value define in enclosing function then its not form closer.

A closure occurs when a function has access to a local variable from an enclosing scope that has finished its execution.

When make_printer is called, a new frame is put on the stack with the compiled code for the printer function as a constant and the value of msg as a local. It then creates and returns the function. Because the function printer references the msg variable, it is kept alive after the make_printer function has returned.

So, if your nested functions don’t

  1. access variables that are local to enclosing scopes,
  2. do so when they are executed outside of that scope,

then they are not closures.

 

Closures are functions that inherit variables from their enclosing environment. When you pass a function callback as an argument to another function that will do I/O, this callback function will be invoked later, and this function will — almost magically — remember the context in which it was declared, along with all the variables available in that context.

  • If a function does not use free variables it doesn’t form a closure.
  • If there is another inner level which uses free variables — all previous levels save the lexical environment ( example at the end )
  • function attributes func_closure in python < 3.X or __closure__ in python > 3.X save the free variables.
  • Every function in python has this closure attributes, but it doesn’t save any content if there is no free variables.

example: of closure attributes but no content inside as there is no free variable.

NB: FREE VARIABLE IS MUST TO CREATE A CLOSURE.

I will explain using the same snippet as above:

And all Python functions have a closure attribute so let’s examine the enclosing variables associated with a closure function.

 

 

  • How free variable is related to closure?

If there is no free variable in nested function then it means the nested function is not using the variable from enclosed function and if its not using the data from enclosed function then it means its not a closure. So free variable is must condition for forming closure.

 

What __closure__ does?

we can know if the  function is closure of not by using __closure__ method.

 

The closure attribute returns a tuple of cell objects which contain details of the variables defined in the enclosing scope. the first element of “d” which could be None or a tuple of cells that contain bindings for the function’s free variables and it is read-only.

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *