Python Generators

Instead of returning a single value, a function can generate an entire sequence of results if it uses the yield statement.
A generator is almost exactly the same as a plain function and it can be used in exactly the same way as an iterator.

There is one definite sign of identifying a generator and that is — yield. Any function that uses yield is a generator.

Lets create a countdown timer with the help of a generator.

Output:

We can use both next() and __next__() method to call the next value in the sequence. The next() method makes the generator function run until it reaches the next yield value. So this is what happens:
1) We define c=countdown(5).
2) We define next(c) or c.__next__()which calls the generator function.
2) countdown prints the print statement, and enters the while loop. On encountering the yield sign, it returns the value of num, thereby stopping further execution of the countdown function.
4) Once the value is returned, we move forward by calling the generator function again with next(c) .
5) countdown returns to the function where it last stopped the execution. It decreases the value of num by 1 and continues the while loop by entering it again.
6) Everything that follows from here is repeated(steps 2–5) until the condition in while loop turns False .
7) Calling the generator function now, will return a StopIteration error, because there is nowhere to go now.

Now calling next(c) every time is tedious! So instead of that we use my favorite python loop for!

for — I think is really powerful, for you can use it to iterate in a list of items, lines of a file, results of a generator function or any other object that supports iteration. So when you write for i in s , s can be any of the items that I mentioned above!

Before moving further, I’d like you to try out this command in your terminal: tail -f file_name | grep search_term (you will have to type ctrl+c to stop the execution of the commands).
tail is used to print the last 10(default value) lines of a file and is often used to monitor log files, while grep is used as a search tool, it prints lines matching the given search_term(pattern). (You can run man command_name in your terminal to find out more about these commands.). When we use the -f flag, it continuously monitors the given file, so as soon as you append a new line to the given file, it will print the appended sentence.

To make things interesting, lets write a python program to duplicate the behavior of these UNIX commands.

Output:

grep here is a generator that looks for a specific substring in a sequence of lines, while tail generator reads lines from a given file and returns them to grep generator.

(tail command, as I said before, prints last 10 sentences of a file, but I was unable to duplicate that behavior, instead tail generator here, will scan the whole file.)

References: [1] Python Essential Reference 4th Edition (2009), [2]

Update: For a better understanding, check out this article by Vincent Driessen.