More Special Features in Python

Last Updated on June 21, 2023

Python is an superior programming language! It is no doubt one of many hottest languages for creating AI and machine learning functions. With a very easy-to-learn syntax, Python has some explicit choices that distinguish it from completely different languages. In this tutorial, we’ll discuss some distinctive attributes of the Python programming language.

After ending this tutorial, you may know:

• Constructs for itemizing and dictionary comprehension
• How to utilize the zip and enumerate options
• What are function contexts and inside designers
• What is the purpose of generators in Python

Kick-start your enterprise with my new e ebook Python for Machine Learning, along with step-by-step tutorials and the Python provide code recordsdata for all examples.

Let’s get started.

Python Special Features
Photo by M Mani, some rights reserved.

Tutorial Overview

This tutorial is break up into 4 elements; they’re:

1. List and dictionary comprehension
2. Zip and enumerate options
3. Function contexts and inside designers
4. Generators in Python with an occasion of Keras generator

Import Section

The libraries used on this tutorial are imported inside the code beneath.

from tensorflow import keras<br />from tensorflow.keras.preprocessing.image import ImageDataGenerator<br />import numpy as np<br />import matplotlib.pyplot as plt<br />import math

1 2 3 4 5

from tensorflow import keras from tensorflow.keras.preprocessing.image import ImageDataGenerator import numpy as np import matplotlib.pyplot as plt import math

List Comprehension

List comprehension provides a short, straightforward syntax for creating new lists from present ones. For occasion, suppose we require a model new itemizing, the place each new merchandise is the earlier merchandise multiplied by 3. One methodology is to utilize a for loop as confirmed beneath:

original_list = [1, 2, 3, 4]<br />times3_list = []</p><p>for i in original_list:<br /> times3_list.append(i*3)<br />print(times3_list)

1 2 3 4 5 6

original_list = [1, 2, 3, 4] times3_list = []   for i in original_list:         times3_list.append(i*3) print(times3_list)

Output

[3, 6, 9, 12]

1	[3, 6, 9, 12]

The shorter methodology using itemizing comprehension requires solely a single line of code:

time3_list_awesome_method = [i*3 for i in original_list]<br />print(time3_list_awesome_method)

1 2	time3_list_awesome_method = [i*3 for i in original_list] print(time3_list_awesome_method)

Output

[3, 6, 9, 12]

1	[3, 6, 9, 12]

You might even create a model new itemizing primarily based totally on a specific criterion. For occasion, if we wish solely even numbers added to the model new itemizing:

even_list_awesome_method = [i for i in original_list if i%2==0]<br />print(even_list_awesome_method)

1 2	even_list_awesome_method = [i for i in original_list if i%2==0] print(even_list_awesome_method)

Output

[2, 4]

1

[2, 4]

It may also be potential to have an else associated to the above. For occasion, we’ll go away all even numbers intact and change the odd numbers with zero:

new_list_awesome_method = [i if i%2==0 else 0 for i in original_list]<br />print(new_list_awesome_method)

1 2	new_list_awesome_method = [i if i%2==0 else 0 for i in original_list] print(new_list_awesome_method)

Output

[0, 2, 0, 4]

1	[0, 2, 0, 4]

Want to Get Started With Python for Machine Learning?

Take my free 7-day email correspondence crash course now (with sample code).

Click to sign-up and likewise get a free PDF Ebook mannequin of the course.

Dictionary Comprehension

Dictionary comprehension is rather like itemizing comprehension, apart from now now we’ve (key, value) pairs. Here is an occasion; we’ll modify each value of the dictionary by concatenating the string ‘amount ‘ to each value:

original_dict = {1: ‘one’, 2: ‘two’, 3: ‘three’, 4: ‘4’}<br />new_dict = {key:’amount ‘ + value for (key, value) in original_dict.devices()}<br />print(new_dict)

1 2 3

original_dict = {1: ‘one’, 2: ‘two’, 3: ‘three’, 4: ‘4’} new_dict = {key:‘amount ‘ + value for (key, value) in original_dict.devices()} print(new_dict)

Output

{1: ‘main’, 2: ‘amount two’, 3: ‘amount three’, 4: ‘amount 4’}

1	{1: ‘main’, 2: ‘amount two’, 3: ‘amount three’, 4: ‘amount 4’}

Again, conditionals are moreover potential. We can choose in order so as to add (key, value) pairs primarily based totally on a criterion inside the new dictionary.

#Only add keys which can be bigger than 2<br />new_dict_high_keys = {key:’amount ‘ + value for (key, value) in original_dict.devices() if key>2}<br />print(new_dict_high_keys)</p><p># Only change values with key>2<br />new_dict_2 = {key:(‘amount ‘ + value if key>2 else value) for (key, value) in original_dict.devices() }<br />print(new_dict_2)

1 2 3 4 5 6 7

#Only add keys which can be bigger than 2 new_dict_high_keys = {key:‘amount ‘ + value for (key, value) in original_dict.devices() if key>2} print(new_dict_high_keys)   # Only change values with key>2 new_dict_2 = {key:(‘amount ‘ + value if key>2 else value) for (key, value) in original_dict.devices() } print(new_dict_2)

Output

{3: ‘amount three’, 4: ‘amount 4’}<br />{1: ‘one’, 2: ‘two’, 3: ‘amount three’, 4: ‘amount 4’}

1 2	{3: ‘amount three’, 4: ‘amount 4’} {1: ‘one’, 2: ‘two’, 3: ‘amount three’, 4: ‘amount 4’}

Enumerators and Zip in Python

In Python, an iterable is printed as any data development that will return all its devices, individually. This method, it’s best to make the most of a for loop to extra course of all devices one after the opposite. Python has two further constructs that make for loops less complicated to utilize, i.e., enumerate() and zip().

Enumerate

In typical programming languages, you desire a loop variable to iterate by way of fully completely different values of a container. In Python, that’s simplified by offering you with entry to a loop variable along with one value of the iterable object. The enumerate(x) function returns two iterables. One iterable varies from 0 to len(x)-1. The completely different is an iterable with a value equal to devices of x. An occasion is confirmed beneath:

establish = [‘Triangle’, ‘Square’, ‘Hexagon’, ‘Pentagon’]</p><p># enumerate returns two iterables<br />for i, n in enumerate(establish):<br /> print(i, ‘establish: ‘, n)

1 2 3 4 5

establish = [‘Triangle’, ‘Square’, ‘Hexagon’, ‘Pentagon’]   # enumerate returns two iterables for i, n in enumerate(establish):     print(i, ‘establish: ‘, n)

Output

0 establish: Triangle<br />1 establish: Square<br />2 establish: Hexagon<br />3 establish: Pentagon

1 2 3 4

0 establish:  Triangle 1 establish:  Square 2 establish:  Hexagon 3 establish:  Pentagon

By default, enumerate begins at 0, nonetheless we’ll start at one other amount if we specify it. This is useful in some situations, as an illustration:

data = [1,4,1,5,9,2,6,5,3,5,8,9,7,9,3]<br />for n, digit in enumerate(data[5:], 6):<br /> print(“The %d-th digit is %d” % (n, digit))

1 2 3

data = [1,4,1,5,9,2,6,5,3,5,8,9,7,9,3] for n, digit in enumerate(data[5:], 6):     print(“The %d-th digit is %d” % (n, digit))

The 6-th digit is 2<br />The 7-th digit is 6<br />The 8-th digit is 5<br />The 9-th digit is 3<br />The 10-th digit is 5<br />The 11-th digit is 8<br />The 12-th digit is 9<br />The 13-th digit is 7<br />The 14-th digit is 9<br />The 15-th digit is 3

1 2 3 4 5 6 7 8 9 10

The 6-th digit is 2 The 7-th digit is 6 The 8-th digit is 5 The 9-th digit is 3 The 10-th digit is 5 The 11-th digit is 8 The 12-th digit is 9 The 13-th digit is 7 The 14-th digit is 9 The 15-th digit is 3

Zip

Zip lets you create an iterable object of tuples. Zip takes as an argument a variety of containers $(m_1, m_2, ldots, m_n)$ and creates the i-th tuple by pairing one merchandise from each container. The i-th tuple is then $(m_{1i}, m_{2i}, ldots, m_{ni})$. If the handed objects have fully completely different lengths, then the entire number of tuples formed has a dimension equal to the minimal dimension of handed objects.

Below are examples of using every zip() and enumerate().

sides = [3, 4, 6, 5]<br />colors = [‘red’, ‘green’, ‘yellow’, ‘blue’]<br />shapes = zip(establish, sides, colors)</p><p># Tuples are created from one merchandise from each itemizing<br />print(set(shapes))</p><p># Easy to utilize enumerate and zip collectively for iterating by way of a variety of lists in a single go<br />for i, (n, s, c) in enumerate(zip(establish, sides, colors)):<br /> print(i, ‘Shape- ‘, n, ‘; Sides ‘, s)

1 2 3 4 5 6 7 8 9 10

sides = [3, 4, 6, 5] colors = [‘red’, ‘green’, ‘yellow’, ‘blue’] shapes = zip(establish, sides, colors)   # Tuples are created from one merchandise from each itemizing print(set(shapes))   # Easy to utilize enumerate and zip collectively for iterating by way of a variety of lists in a single go for i, (n, s, c) in enumerate(zip(establish, sides, colors)):     print(i, ‘Shape- ‘, n, ‘; Sides ‘, s)

Output

{(‘Triangle’, 3, ‘purple’), (‘Square’, 4, ‘inexperienced’), (‘Hexagon’, 6, ‘yellow’), (‘Pentagon’, 5, ‘blue’)}<br />0 Shape- Triangle ; Sides 3<br />1 Shape- Square ; Sides 4<br />2 Shape- Hexagon ; Sides 6<br />3 Shape- Pentagon ; Sides 5

1 2 3 4 5

{(‘Triangle’, 3, ‘purple’), (‘Square’, 4, ‘inexperienced’), (‘Hexagon’, 6, ‘yellow’), (‘Pentagon’, 5, ‘blue’)} 0 Shape-  Triangle ; Sides  3 1 Shape-  Square ; Sides  4 2 Shape-  Hexagon ; Sides  6 3 Shape-  Pentagon ; Sides  5

Function Context

Python permits nested options, the place you’ll define an inside function inside an outer function. There are some superior choices related to nested options in Python.

• The outer function can return a take care of to the within function.
• The inside function retains all its environment and variables native to it and in its enclosing function even when the outer function ends its execution.

An occasion is given beneath, with an proof inside the suggestions.

def circle(r):<br /> house = 0<br /> def area_obj():<br /> nonlocal house<br /> house = math.pi * r * r<br /> print(“area_obj”)<br /> return area_obj </p><p>def circle(r):<br /> area_val = math.pi * r * r<br /> def house():<br /> print(area_val)<br /> return house </p><p># returns area_obj(). The value of r handed is retained<br />circle_1 = circle(1)<br />circle_2 = circle(2)</p><p># Calling area_obj() with radius = 1<br />circle_1()<br /># Calling area_obj() with radius = 2<br />circle_2()

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

def circle(r):     house = 0     def area_obj():         nonlocal house         house = math.pi * r * r         print(“area_obj”)     return area_obj       def circle(r):     area_val = math.pi * r * r     def house():         print(area_val)     return house       # returns area_obj(). The value of r handed is retained circle_1 = circle(1) circle_2 = circle(2)   # Calling area_obj() with radius = 1 circle_1() # Calling area_obj() with radius = 2 circle_2()

Output

3.141592653589793<br />12.566370614359172

1 2	3.141592653589793 12.566370614359172

Decorators in Python

Decorators are a powerful operate of Python. You can use decorators to customize the working of a class or a function. Think of them as a function utilized to a special function. Use the function establish with the @ picture to stipulate the decorator function on the adorned function. The decorator takes a function as an argument, giving a complete lot of flexibility.

Consider the following function square_decorator() that takes a function as an argument and likewise returns a function.

• The inside nested function square_it()takes an argument arg.
• The square_it()function applies the function to arg and squares the result.
• We can cross a function equivalent to sin to square_decorator(), which in flip would return $sin^2(x)$.
• You may even write your private customized function and use the square_decorator() function on it using the actual @ picture as confirmed beneath. The function plus_one(x) returns x+1. This function is adorned by the square_decorator(), and subsequently we get $(x+1)^2$.

def square_decorator(function):<br /> def square_it(arg):<br /> x = function(arg)<br /> return x*x<br /> return square_it</p><p>size_sq = square_decorator(len)<br />print(size_sq([1,2,3]))</p><p>sin_sq = square_decorator(math.sin)<br />print(sin_sq(math.pi/4))</p><p>@square_decorator<br />def plus_one(a):<br /> return a+1</p><p>a = plus_one(3)<br />print(a)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

def square_decorator(function):     def square_it(arg):         x = function(arg)         return x*x     return square_it   size_sq = square_decorator(len) print(size_sq([1,2,3]))   sin_sq = square_decorator(math.sin) print(sin_sq(math.pi/4))   @square_decorator def plus_one(a):     return a+1   a = plus_one(3) print(a)

Output

9<br />0.4999999999999999<br />16

1 2 3

9 0.4999999999999999 16

Generators in Python

Generators in Python allow you to generate sequences. Instead of writing a return assertion, a generator returns a variety of values via a variety of yield statements. The first identify to the function returns the first value from yield. The second identify returns the second value from yield and so forth.

The generator function could also be invoked via subsequent().Every time subsequent() is called, the next yield value is returned. An occasion of manufacturing the Fibonacci sequence as a lot as a given amount x is confirmed beneath.

def get_fibonacci(x):<br /> x0 = 0<br /> x1 = 1<br /> for i in fluctuate(x):<br /> yield x0<br /> temp = x0 + x1<br /> x0 = x1<br /> x1 = temp</p><p>f = get_fibonacci(6)<br />for i in fluctuate(6):<br /> print(subsequent(f))

1 2 3 4 5 6 7 8 9 10 11 12 13

def get_fibonacci(x):     x0 = 0     x1 = 1     for i in fluctuate(x):         yield x0         temp = x0 + x1         x0 = x1         x1 = temp     f = get_fibonacci(6) for i in fluctuate(6):     print(subsequent(f))

Output

0<br />1<br />1<br />2<br />3<br />5

1 2 3 4 5 6

0 1 1 2 3 5

Example of Data Generator in Keras

One use of a generator is the data generator in Keras. It is useful on account of we do not want to maintain all data in memory nonetheless want to create it on the fly when the teaching loop desires it. Remember, in Keras, a neural neighborhood model is expert in batches, so a generator is to emit batches of knowledge. The function beneath is from our earlier put up, “Using CNN for financial time sequence prediction“:

def datagen(data, seq_len, batch_size, targetcol, kind):<br /> “As a generator to offer samples for Keras model”<br /> batch = []<br /> whereas True:<br /> # Pick one dataframe from the pool<br /> key = random.choice(itemizing(data.keys()))<br /> df = data[key]<br /> input_cols = [c for c in df.columns if c != targetcol]<br /> index = df.index[df.index < TRAIN_TEST_CUTOFF]<br /> reduce up = int(len(index) * TRAIN_VALID_RATIO)<br /> if kind == ‘put together’:<br /> index = index[:split] # fluctuate for the teaching set<br /> elif kind == ‘professional’:<br /> index = index[split:] # fluctuate for the validation set<br /> # Pick one place, then clip a sequence dimension<br /> whereas True:<br /> t = random.choice(index) # select one time step<br /> n = (df.index == t).argmax() # uncover its place inside the dataframe<br /> if n-seq_len+1 < 0:<br /> proceed # can’t get adequate data for one sequence dimension<br /> physique = df.iloc[n-seq_len+1:n+1]<br /> batch.append([frame[input_cols].values, df.loc[t, targetcol]])<br /> break<br /> # if we get adequate for a batch, dispatch<br /> if len(batch) == batch_size:<br /> X, y = zip(*batch)<br /> X, y = np.expand_dims(np.array(X), 3), np.array(y)<br /> yield X, y<br /> batch = []

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

def datagen(data, seq_len, batch_size, targetcol, kind):     “As a generator to offer samples for Keras model”     batch = []     whereas True:         # Pick one dataframe from the pool         key = random.choice(itemizing(data.keys()))         df = data[key]         input_cols = [c for c in df.columns if c != targetcol]         index = df.index[df.index < TRAIN_TEST_CUTOFF]         reduce up = int(len(index) * TRAIN_VALID_RATIO)         if kind == ‘put together’:             index = index[:split]   # fluctuate for the teaching set         elif kind == ‘professional’:             index = index[split:]   # fluctuate for the validation set         # Pick one place, then clip a sequence dimension         whereas True:             t = random.choice(index)      # select one time step             n = (df.index == t).argmax()  # uncover its place inside the dataframe             if n–seq_len+1 < 0:                 proceed # can’t get adequate data for one sequence dimension             physique = df.iloc[n–seq_len+1:n+1]             batch.append([frame[input_cols].values, df.loc[t, targetcol]])             break         # if we get adequate for a batch, dispatch         if len(batch) == batch_size:             X, y = zip(*batch)             X, y = np.expand_dims(np.array(X), 3), np.array(y)             yield X, y             batch = []

The function above is to pick a random row of a pandas dataframe as a kick off point and clip the next a variety of rows as a one-time interval sample. This course of is repeated a variety of situations to assemble many time intervals into one batch. When we purchase adequate interval samples, on the second to the ultimate line inside the above function, the batch is dispatched using the yield command. You might have already seen that generator options do not have a return assertion. In this occasion, the function will run endlessly. This is useful and important on account of it permits our Keras teaching course of to run as many epochs as we wish.

If we do not use a generator, we would need to convert the dataframe into all potential time intervals and maintain them in memory for the teaching loop. This will probably be a complete lot of repeating data (on account of the time intervals are overlapping) and take up a complete lot of memory.

Because it is useful, Keras has some generator function predefined inside the library. Below is an occasion of the ImageDataGenerator(). We have loaded the cifar10 dataset of 32×32 footage in x_train. The data is said to the generator via the motion() methodology. The subsequent() function returns the next batch of knowledge. In the occasion beneath, there are 4 calls to subsequent(). In each case, 8 footage are returned as a result of the batch dimension is 8.

Below is the entire code that moreover reveals all footage after every identify to subsequent().

(x_train, y_train), _ = keras.datasets.cifar10.load_data()<br />datagen = ImageDataGenerator()<br />data_iterator = datagen.motion(x_train, y_train, batch_size=8)</p><p>fig,ax = plt.subplots(nrows=4, ncols=8,figsize=(18,6),subplot_kw=dict(xticks=[], yticks=[]))</p><p>for i in fluctuate(4):<br /> # The subsequent() function will load 8 footage from CIFAR<br /> X, Y = data_iterator.subsequent()<br /> for j, img in enumerate(X):<br /> ax[i, j].imshow(img.astype(‘int’))

1 2 3 4 5 6 7 8 9 10 11

(x_train, y_train), _ = keras.datasets.cifar10.load_data() datagen = ImageDataGenerator() data_iterator = datagen.motion(x_train, y_train, batch_size=8)   fig,ax = plt.subplots(nrows=4, ncols=8,figsize=(18,6),subplot_kw=dict(xticks=[], yticks=[]))   for i in fluctuate(4):     # The subsequent() function will load 8 footage from CIFAR     X, Y = data_iterator.subsequent()     for j, img in enumerate(X):         ax[i, j].imshow(img.astype(‘int’))

Further Reading

This half provides further property on the topic in case you might be in search of to go deeper.

Python Documentation

• Python Documentation at python.org

Books

• Think Python: How to Think Like a Computer Scientist by Allen B. Downey
• Programming in Python 3: A Complete Introduction to the Python Language by Mark Summerfield
• Python Programming: An Introduction to Computer Science by John Zelle

API Reference

• Keras ImageDataGenerator

Summary

In this tutorial, you discovered some explicit choices of Python.

Specifically, you realized:

• The aim of itemizing and dictionary comprehension
• How to utilize zip and enumerate
• Nested options, function contexts, and inside designers
• Generators in Python and the ImageDataGenerator in Python

Do you’ve got any questions regarding the Python choices talked about on this put up? Ask your questions inside the suggestions beneath, and I’ll do my biggest to answer.

 

Get a Handle on Python for Machine Learning!

Be More Confident to Code in Python

…from learning the wise Python ideas

Discover how in my new Ebook:
Python for Machine Learning

It provides self-study tutorials with a lot of of working code to equip you with experience along with:
debugging, profiling, duck typing, decorators, deployment,
and quite extra…

Showing You the Python Toolbox at a High Level for
Your Projects

See What’s Inside

More On This Topic

• 

  How to Perform Feature Selection for Regression Data

• 

  How to Develop a Deep Learning Photo Caption…

• 

  How to Perform Feature Selection With Numerical Input Data

• 

  Recursive Feature Elimination (RFE) for Feature…

• 

  How to Develop a Feature Selection Subspace Ensemble…

• 

  How to Use Polynomial Feature Transforms for Machine…