Big Data Data Science python Tutorials

Python for Spark Tutorial – Methods and Functions in Python


In the last tutorial, we’ve learned about the different control structures in Python. Now that we know control structures and basics of Python programming, let’s have a look into how to encapsulate code into functions. Therefore, we will have a look at methods and functions in Python.

Methods

Due to the overall differences with C-like languages, also the method definition is slightly different. However, it is very similar to the control structures layout. Let’s have a look

def functionname(args):
FUNCTION BLOCK
return value

A function in Python is always defined with “def”. After that, a function name is provided. Values passed to the function are then in parentheses. Due to the dynamic aspects of Python, it doesn’t know any dedicated type definitions. This means that values are passed by their name to the function. After the “:”, the function block starts. Everything within the function block needs to be indented. Python can also return values by adding the “return” statement. The following function adds one to a number:

def myiter(n):
    return n + 1

val = myiter(33)

val

Output:

34

As you would expect, it is also possible to add different levels by indenting control structures and alike. The following function creates an array from 0-4 (in the range of 5) and calls a function that iterates over each item in the array.

def printall(vals):
    for val in vals:
        print(val)

values = range(5)

printall(values)

Output:

0
1
2
3
4

Lambda Expressions

A very cool feature of most modern programming languages is the availability of Lambda expressions. With this, it is possible to significantly reduce the code complexity by writing easy functions in one-liners. For instance, the first sample of the iterator could also be written as a lambda expression. Basically, a lambda expression is a function defined in-line to be called on each item in a list or an array. It is very useful for data manipulations. Basically, a Lambda expression is introduced with the following statement:

lambda variable: STATEMENT

In this case, variable is one or more variable(s) to work with in the following statement. For instance, if it is the previous sample, it would mean a number. Thus we would only use one variable. If it would be a dictionary, it can also be just one (and the key/values are available as methods) or you would provide both, for instance as x, y. A sample lambda expression matching with the previous one is this:

v = map(lambda x: x + 1, values)

printall(v)

Output:

1
2
3
4
5

In the above statement, we used the “map” function out of python that is capable of calling a lambda function on the specified iterable (array in our case). We re-used the existing array specified in the statement above. Each item of the array is now changed in its value by one. As you can see, it is very easy to work with lambda expressions in Python and they are very useful to keep your code simple and clean.

In the next tutorial, we will have a look at how to encapsulate methods and functions into classes and packages. We will also have a look at inheritance.

I lead a team of Senior Experts in Data & Data Science as Head of Data & Analytics and AI at A1 Telekom Austria Group. I also teach this topic at various universities and frequently speak at various Conferences. In 2010 I wrote a book about Cloud Computing, which is often used at German & Austrian Universities. In my home country (Austria) I am part of several organisations on Big Data & Data Science.

0 comments on “Python for Spark Tutorial – Methods and Functions in Python

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: