Techno Blender
Digitally Yours.

Side Effects Of Python Functions. When does a function change a variable… | by Pan Cretan | Aug, 2022

0 79


When does a function change a variable defined outside its own scope?

Have you wondered how a Python function can change variables defined outside its scope? It may confuse developers coming from other languages, but predicting side effects is straightforward once you understand few fundamental principles.

Photo by Nick Fewings on Unsplash

Function side effects can be difficult to debug and most of the times are not intended. Instead, returning objects from functions is better done using the return statement with a single object or a tuple. In Python, side effects can occur in two ways:

  • Changing the value of a mutable object
  • Changing the binding of a variable defined outside the local namespace

We will go through these aspects using a set of examples. Before doing so we will revisit how Python passes arguments to functions.

If you have used other programming languages you may have heard the terms pass-by-value and pass-by-reference. Pass-by-value means that a copy of the argument is passed to the function, whilst pass-by-reference means that the function receives a reference to the argument. You may be surprised to hear that Python uses neither of the above. I was completely baffled when I started working with Python because I thought that there are only two options and Python functions clearly fit neither.

Before going to the Python mechanism for passing function arguments lets quickly revisit name binding in Python. Consider the following code:

The first statement does two things. First, it creates a string object and then it binds this object to the variable a. The second statement binds another variable, b , to the object bound to a. The third statement creates another string object and rebinds variable b to it, whilst a remains bound to the string 'hello'. Given that there is still one variable bound to the string 'hello' it is not garbage collected. This seems trivial, but pay attention to the words used. We did not use expressions such as “allocate memory to a string variable aand store the string 'hello' in it”. Instead we talk about creating objects and binding variables to them.

What about a mutable object, such as a list? The above applies equally to mutable objects. However, there is a caveat. Look at the following code:

that prints

a -> 2199273128320 [10, 2, 3]
b -> 2199273128320 [10, 2, 3]

Changing one element of the list bound to achanges btoo. In fact, this expression is misleading. There is only one list and both a and bare bound to it as can be seen from memory location returned by the id() function.

Lets look at a function next:

This prints

outside function x -> 140731263354656, 5
inside function x_f -> 140731263354656, 5
{'x_f': 5}
inside function x_f -> 140731263354816, 10
{'x_f': 10}
outside function x -> 140731263354656, 5

What is going on here? Before invoking the function we created an integer object and bound variable x to it. The argument is then passed to the function. We can see that the function parameter x_f is initialised to the same memory location pointed to by the variable x outside the function. This is reflected in the local namespace of the function returned by the locals() function. Inside the function we create a new integer object with the value 10 that is rebound to the local variable x_f. This does not affect the object bound to x outside the function. We can confirm this by the memory locations returned by the id() function and the final print statement that shows that variable x continues to be bound to the integer 5.

What does this mean? We clearly did not pass the argument by reference because the function did not change the value bound to x. We also did not pass the argument by value because initially x_f was pointing to the same memory location as x. Passing arguments to Python functions follow some kind of a hybrid pattern. We can say that we passed a reference, but the reference was passed by value, that is also known as pass-by-assignment. It is a bit mind boggling but predictable once variable binding in Python is understood.

Based on the previous section one may think that Python functions cannot change the passed arguments at all. This is partially true. Python functions cannot totally change the argument in the sense that the argument cannot be made to point to a different location in memory as a whole. If the parameter inside the function is bound to a new object, then there would be no relation to the argument outside the function. However, the argument can be partially or fully changed and this is one of the ways that functions can have side effects in Python. Lets see an example:

that prints

outside function x -> 2199273130560, [1, 2, 3]
inside function x_f -> 2199273130560, [1, 2, 3]
{'x_f': [1, 2, 3]}
inside function x_f -> 2199273130560, [1, 10, 3]
{'x_f': [1, 10, 3]}
outside function x -> 2199273130560, [1, 10, 3]

The function had a side effects as it changed the second element of the passed argument array. This behaviour is completely predictable if we keep in mind the fundamental principles introduced above. The function operates on the same array defined outside the function as we can see by the output of the id() function that always prints the same memory address.

Can we change the whole content of the array? The following example shows that we can:

that prints

outside function x -> 2199272856256, [1, 2, 3]
inside function x_f -> 2199272856256, [1, 2, 3]
{'x_f': [1, 2, 3]}
inside function x_f -> 2199272856256, [10, 20, 30, 40]
{'x_f': [10, 20, 30, 40]}
outside function x -> 2199272856256, [10, 20, 30, 40]

We did not only change the value of all elements of the array, but we even changed its length. But we did not change its address. This is something Python functions cannot do when objects are passed as argument. But this can be done via the second mechanism in which functions can introduce side effects as we will see in the next section. But before this, I wanted to mention that there are ways to create a local copy of the array inside the function so that side effects can be avoided as shown below:

that prints

outside function x -> 2199273194880, [1, 2, 3]
inside function x_f -> 2199273194880, [1, 2, 3]
{'x_f': [1, 2, 3]}
inside function x_f -> 2199272856256, [1, 10, 3]
{'x_f': [1, 10, 3]}
outside function x -> 2199273194880, [1, 2, 3]

We can see that creating a copy of the array inside the function prevented the side effect. The same could be achieved by simply passing a copy of x to the function, i.e. calling the function with f(x[:]).

But things can still go wrong in the case of nested arrays:

that prints

outside function x -> 2199272857344, [1, [-1, -2, -3], 3]
inside function x_f -> 2199272857344, [1, [-1, -2, -3], 3]
inside function x_f -> 2199273194240, ['we guard against this side effect', [-1, 'we do not guard against this side effect', -3], 3]
outside function x -> 2199272857344, [1, [-1, 'we do not guard against this side effect', -3], 3]

The behaviour above can be understood using the same fundamental principles. Copying the array inside the function means that the immutable elements of the array are rebound if changed. All elements are immutable, apart from the second element of the array that keeps being bound to the same array as before. We can verify this with the id() function:

that prints

outside function x
2199273306608, 1000
2199273249088, [-1000, -2000, -3000]
.. 2199273307216, -1000
.. 2199273307280, -2000
.. 2199273306448, -3000
2199273306384, 3000
inside function x_f
2199273306608, 1000
2199273249088, [-1000, -2000, -3000]
.. 2199273307216, -1000
.. 2199273307280, -2000
.. 2199273306448, -3000
2199273306384, 3000
inside function x_f
2199272920720, we guard against this side effect
2199273249088, [-1000, 'we do not guard against this side effect', -3000]
.. 2199273307216, -1000
.. 2199273257072, we do not guard against this side effect
.. 2199273306448, -3000
2199273306384, 3000
outside function x
2199273306608, 1000
2199273249088, [-1000, 'we do not guard against this side effect', -3000]
.. 2199273307216, -1000
.. 2199273257072, we do not guard against this side effect
.. 2199273306448, -3000
2199273306384, 3000

Using the deepcopy() function of the copy module is in fact the safest way to avoid side effects.

Python searches in four namespaces when a variable is used: local, enclosing, global and built-in. The namespaces are searched in this order until the name is found and if not found Python raises a NameError exception. We will not cover all pesky details of namespaces (see this tutorial if you are curious) but instead focus on the global and local namespace. Let’s see an example:

prints

{'x': 'local'}
global

The reason is that inside the function we create a new string object and bind a local variable named x to it as also seen by the output of the locals()function. The global variable x is kept bound to the string 'global' as we would have expected. Contrast the above with the following:

that prints

x = 'global'
def f():
global x
x = 'local'
print(locals())

f()
print(x)

We can see that the variable x outside the function is now bound to the string object created within the function. This is a side effect and a prime example of things to avoid. The same can be achieved by directly accessing the globals dictionary. In fact it is not even necessary to bind the variable x before the function is called:

that prints

{}
local

This is in fact terrible practice. Readers of the code that have looked inside the function will be perplexed. Similarly to global side-effects can also be caused when using nonlocal in nested functions.


When does a function change a variable defined outside its own scope?

Have you wondered how a Python function can change variables defined outside its scope? It may confuse developers coming from other languages, but predicting side effects is straightforward once you understand few fundamental principles.

Photo by Nick Fewings on Unsplash

Function side effects can be difficult to debug and most of the times are not intended. Instead, returning objects from functions is better done using the return statement with a single object or a tuple. In Python, side effects can occur in two ways:

  • Changing the value of a mutable object
  • Changing the binding of a variable defined outside the local namespace

We will go through these aspects using a set of examples. Before doing so we will revisit how Python passes arguments to functions.

If you have used other programming languages you may have heard the terms pass-by-value and pass-by-reference. Pass-by-value means that a copy of the argument is passed to the function, whilst pass-by-reference means that the function receives a reference to the argument. You may be surprised to hear that Python uses neither of the above. I was completely baffled when I started working with Python because I thought that there are only two options and Python functions clearly fit neither.

Before going to the Python mechanism for passing function arguments lets quickly revisit name binding in Python. Consider the following code:

The first statement does two things. First, it creates a string object and then it binds this object to the variable a. The second statement binds another variable, b , to the object bound to a. The third statement creates another string object and rebinds variable b to it, whilst a remains bound to the string 'hello'. Given that there is still one variable bound to the string 'hello' it is not garbage collected. This seems trivial, but pay attention to the words used. We did not use expressions such as “allocate memory to a string variable aand store the string 'hello' in it”. Instead we talk about creating objects and binding variables to them.

What about a mutable object, such as a list? The above applies equally to mutable objects. However, there is a caveat. Look at the following code:

that prints

a -> 2199273128320 [10, 2, 3]
b -> 2199273128320 [10, 2, 3]

Changing one element of the list bound to achanges btoo. In fact, this expression is misleading. There is only one list and both a and bare bound to it as can be seen from memory location returned by the id() function.

Lets look at a function next:

This prints

outside function x -> 140731263354656, 5
inside function x_f -> 140731263354656, 5
{'x_f': 5}
inside function x_f -> 140731263354816, 10
{'x_f': 10}
outside function x -> 140731263354656, 5

What is going on here? Before invoking the function we created an integer object and bound variable x to it. The argument is then passed to the function. We can see that the function parameter x_f is initialised to the same memory location pointed to by the variable x outside the function. This is reflected in the local namespace of the function returned by the locals() function. Inside the function we create a new integer object with the value 10 that is rebound to the local variable x_f. This does not affect the object bound to x outside the function. We can confirm this by the memory locations returned by the id() function and the final print statement that shows that variable x continues to be bound to the integer 5.

What does this mean? We clearly did not pass the argument by reference because the function did not change the value bound to x. We also did not pass the argument by value because initially x_f was pointing to the same memory location as x. Passing arguments to Python functions follow some kind of a hybrid pattern. We can say that we passed a reference, but the reference was passed by value, that is also known as pass-by-assignment. It is a bit mind boggling but predictable once variable binding in Python is understood.

Based on the previous section one may think that Python functions cannot change the passed arguments at all. This is partially true. Python functions cannot totally change the argument in the sense that the argument cannot be made to point to a different location in memory as a whole. If the parameter inside the function is bound to a new object, then there would be no relation to the argument outside the function. However, the argument can be partially or fully changed and this is one of the ways that functions can have side effects in Python. Lets see an example:

that prints

outside function x -> 2199273130560, [1, 2, 3]
inside function x_f -> 2199273130560, [1, 2, 3]
{'x_f': [1, 2, 3]}
inside function x_f -> 2199273130560, [1, 10, 3]
{'x_f': [1, 10, 3]}
outside function x -> 2199273130560, [1, 10, 3]

The function had a side effects as it changed the second element of the passed argument array. This behaviour is completely predictable if we keep in mind the fundamental principles introduced above. The function operates on the same array defined outside the function as we can see by the output of the id() function that always prints the same memory address.

Can we change the whole content of the array? The following example shows that we can:

that prints

outside function x -> 2199272856256, [1, 2, 3]
inside function x_f -> 2199272856256, [1, 2, 3]
{'x_f': [1, 2, 3]}
inside function x_f -> 2199272856256, [10, 20, 30, 40]
{'x_f': [10, 20, 30, 40]}
outside function x -> 2199272856256, [10, 20, 30, 40]

We did not only change the value of all elements of the array, but we even changed its length. But we did not change its address. This is something Python functions cannot do when objects are passed as argument. But this can be done via the second mechanism in which functions can introduce side effects as we will see in the next section. But before this, I wanted to mention that there are ways to create a local copy of the array inside the function so that side effects can be avoided as shown below:

that prints

outside function x -> 2199273194880, [1, 2, 3]
inside function x_f -> 2199273194880, [1, 2, 3]
{'x_f': [1, 2, 3]}
inside function x_f -> 2199272856256, [1, 10, 3]
{'x_f': [1, 10, 3]}
outside function x -> 2199273194880, [1, 2, 3]

We can see that creating a copy of the array inside the function prevented the side effect. The same could be achieved by simply passing a copy of x to the function, i.e. calling the function with f(x[:]).

But things can still go wrong in the case of nested arrays:

that prints

outside function x -> 2199272857344, [1, [-1, -2, -3], 3]
inside function x_f -> 2199272857344, [1, [-1, -2, -3], 3]
inside function x_f -> 2199273194240, ['we guard against this side effect', [-1, 'we do not guard against this side effect', -3], 3]
outside function x -> 2199272857344, [1, [-1, 'we do not guard against this side effect', -3], 3]

The behaviour above can be understood using the same fundamental principles. Copying the array inside the function means that the immutable elements of the array are rebound if changed. All elements are immutable, apart from the second element of the array that keeps being bound to the same array as before. We can verify this with the id() function:

that prints

outside function x
2199273306608, 1000
2199273249088, [-1000, -2000, -3000]
.. 2199273307216, -1000
.. 2199273307280, -2000
.. 2199273306448, -3000
2199273306384, 3000
inside function x_f
2199273306608, 1000
2199273249088, [-1000, -2000, -3000]
.. 2199273307216, -1000
.. 2199273307280, -2000
.. 2199273306448, -3000
2199273306384, 3000
inside function x_f
2199272920720, we guard against this side effect
2199273249088, [-1000, 'we do not guard against this side effect', -3000]
.. 2199273307216, -1000
.. 2199273257072, we do not guard against this side effect
.. 2199273306448, -3000
2199273306384, 3000
outside function x
2199273306608, 1000
2199273249088, [-1000, 'we do not guard against this side effect', -3000]
.. 2199273307216, -1000
.. 2199273257072, we do not guard against this side effect
.. 2199273306448, -3000
2199273306384, 3000

Using the deepcopy() function of the copy module is in fact the safest way to avoid side effects.

Python searches in four namespaces when a variable is used: local, enclosing, global and built-in. The namespaces are searched in this order until the name is found and if not found Python raises a NameError exception. We will not cover all pesky details of namespaces (see this tutorial if you are curious) but instead focus on the global and local namespace. Let’s see an example:

prints

{'x': 'local'}
global

The reason is that inside the function we create a new string object and bind a local variable named x to it as also seen by the output of the locals()function. The global variable x is kept bound to the string 'global' as we would have expected. Contrast the above with the following:

that prints

x = 'global'
def f():
global x
x = 'local'
print(locals())

f()
print(x)

We can see that the variable x outside the function is now bound to the string object created within the function. This is a side effect and a prime example of things to avoid. The same can be achieved by directly accessing the globals dictionary. In fact it is not even necessary to bind the variable x before the function is called:

that prints

{}
local

This is in fact terrible practice. Readers of the code that have looked inside the function will be perplexed. Similarly to global side-effects can also be caused when using nonlocal in nested functions.

FOLLOW US ON GOOGLE NEWS

Read original article here

Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.

Leave a comment