Contents
I. Advanced Python Features
Many features in Python can be implemented in multiple ways. In Python, more code is not better; less is better. More complex code is not better; simpler is better. Based on this philosophy, Python provides many very useful advanced features.
1.1 List Comprehensions
(1) Simple list comprehensions
For example, suppose we want to generate the squares of the first 10 numbers, [1x1, 2x2, 3x3, ..., 10x10]. The conventional approach requires a loop.
L = []
for x in range(1, 11):
L.append(x * x)This approach is too cumbersome. A list comprehension can replace the loop above with a single line:
[x * x for x in range(1,11)]
# [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]When writing a list comprehension, place the element to generate, x*x, first, followed by the for loop. This creates the list.
(2) Nested loops and loops with multiple variables
List comprehensions support multiple loops. For example:
[m + n for m in 'ABC' for n in 'XYZ']
# ['AX', 'AY', 'AZ', 'BX', 'BY', 'BZ', 'CX', 'CY', 'CZ']Loops with three or more levels are rarely used.
Because a for loop supports two or even more variables, a list comprehension supports the same functionality:
d = {'x': 'A', 'y': 'B', 'z': 'C' }
[k + '=' + v for k, v in d.items()]
# ['y=B', 'x=A', 'z=C'](3) Using if in a list comprehension
A for loop with if:
[x for x in range(1, 11) if x % 2 == 0]
# [2, 4, 6, 8, 10]The if after the for is a filtering condition.
(4) Using if…else in a list comprehension
A for loop with if…else:
[x if x % 2 == 0 else -x for x in range(1, 11)]
# [-1, 2, -3, 4, -5, 6, -7, 8, -9, 10]The part before the for is an expression. It must calculate a result from the x obtained by the for loop, and that result becomes an element in the list.
In the example above, the expression before the for, x if x % 2 == 0 else -x, can calculate a definite result from x. If only if is used without else, consider the expression x if x % 2 == 0: it cannot calculate a result from x, so the program reports an error.
In summary, in a list comprehension, before for, if ... else is an expression, while after for, if is a filtering condition and cannot include else.
1.2 Generators
Because of memory limitations, a list has finite capacity. If we need to create a list containing 100万 elements but only need to access the first few elements each time, it not only occupies a large amount of storage space, but also wastes the space taken by the vast majority of the other elements.
If the elements in a list can be calculated according to an algorithm, we can continuously calculate them during iteration. This avoids creating the complete list all at once and saves a great deal of space. This mechanism of calculating while iterating is called a generator.
(1) Creating a generator
The simplest way to create a generator is to change [] in a list comprehension to (), which creates a generator:
g = (x * x for x in range(10))
# <generator object <genexpr> at 0x1022ef630>You can access the elements of a generator with a for loop:
g = (x * x for x in range(10))
for n in g:
print(n)(2) Creating a generator with yield
If a function definition contains the yield keyword, that function is no longer an ordinary function, but a generator function.
def odd():
print('step 1')
yield 1
print('step 2')
yield(3)
print('step 3')
yield(5)You can obtain the generator’s next return value with the next() function. When the last element has been calculated and there are no more elements, a StopIteration error is raised.
The execution flow of a generator function differs from that of an ordinary function. An ordinary function executes sequentially and returns when it encounters a return statement or reaches the last line of the function. A function that has become a generator executes each time next() is called and returns when it encounters a yield statement. The next time it executes, it continues from the yield statement where it last returned.
When calling this generator function, first create a generator object, then use the next() function repeatedly to obtain the next return value:
>>> o = odd()
>>> next(o)
step 1
1
>>> next(o)
step 2
3
>>> next(o)
step 3
5
>>> next(o)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIterationAfter changing a function into a generator function, we almost never use next() to obtain the next return value. Instead, we iterate directly with a for loop.
for n in odd():
print(n)II. Functional Programming
2.1 Higher-Order Functions
(1) Variables can refer to functions
A function itself can be assigned to a variable; in other words, a variable can refer to a function.
>>> f = abs
>>> f(-10)
10This shows that the variable f now refers to the abs function itself. Calling the abs() function directly is exactly the same as calling the variable f().
(2) Function names are also variables
A function name is actually a variable that refers to a function! For the abs() function, the function name abs can be regarded as a variable that refers to a function capable of calculating an absolute value.
>>> abs = 10
>>> abs(-10)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callableAfter abs is made to refer to 10, the function can no longer be called through abs(-10) because the variable abs no longer refers to the absolute-value function; it refers to the integer 10 instead! (Of course, real code would never be written this way.)
(3) Passing a function
A function can receive another function as a parameter. Such a function is called a higher-order function.
def add(x, y, f):
return f(x) + f(y)When we call add(-5, 6, abs), the parameters x, y, and f receive -5, 6, and abs, respectively, thereby calculating the sum of the absolute values.
(4) map/reduce
The map() function accepts two parameters: a function and an Iterable. map applies the supplied function to each element in the sequence in turn and returns the results as a new Iterator.
For example, suppose we have a function f(x)=x2 and want to apply it to the list [1, 2, 3, 4, 5, 6, 7, 8, 9]. We can use map() as follows:
>>> def f(x):
... return x * x
...
>>> r = map(f, [1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> list(r)
[1, 4, 9, 16, 25, 36, 49, 64, 81]The reduce() function accepts two parameters: a function and an Iterable.
reduce applies a function to a sequence [x1, x2, x3, ...]. This function must accept two parameters. reduce then continues accumulating by applying the function to the result and the next element in the sequence, with the following effect:
reduce(f, [x1, x2, x3, x4]) = f(f(f(x1, x2), x3), x4)For example, convert the sequence [1, 3, 5, 7, 9] into the integer 13579:
>>> from functools import reduce
>>> def fn(x, y):
... return x * 10 + y
...
>>> reduce(fn, [1, 3, 5, 7, 9])
13579(5) filter
Python’s built-in filter() function is used to filter a sequence.
filter() accepts a function and a sequence, applies the supplied function to each element in turn, and then decides whether to keep or discard the element based on whether the return value is True or False.
For example, remove the even numbers from a list and retain only the odd numbers:
def is_odd(n):
return n % 2 == 1
list(filter(is_odd, [1, 2, 4, 5, 6, 9, 10, 15]))
# 结果: [1, 5, 9, 15](6) sorted
Sorting is another algorithm frequently used in programs. Whether bubble sort or quicksort is used, the core of sorting is comparing the values of two elements.
Python’s built-in sorted() function can sort a list:
>>> sorted([36, 5, -12, 9, -21])
[-21, -12, 5, 9, 36]The sorted() function is also a higher-order function. It can also accept a key function to implement custom sorting:
>>> sorted([36, 5, -12, 9, -21], key=abs)
[5, 9, -12, -21, 36]By default, strings are sorted by comparing their ASCII values:
>>> sorted(['bob', 'about', 'Zoo', 'Credit'])
['Credit', 'Zoo', 'about', 'bob']To ignore case and sort alphabetically, simply use a key function that maps the strings for case-insensitive sorting:
>>> sorted(['bob', 'about', 'Zoo', 'Credit'], key=str.lower)
['about', 'bob', 'Credit', 'Zoo']To sort in reverse order, there is no need to change the key function; pass the third parameter, reverse=True:
>>> sorted(['bob', 'about', 'Zoo', 'Credit'], key=str.lower, reverse=True)
['Zoo', 'Credit', 'bob', 'about']2.2 Returning Functions
(1) A function as a return value
In addition to accepting functions as parameters, a higher-order function can also return a function as its result.
For example, what if we do not need to calculate the sum immediately, but want to calculate it later in the code as needed? Instead of returning the result of the sum, we can return the function that calculates it:
def lazy_sum(*args):
def sum():
ax = 0
for n in args:
ax = ax + n
return ax
return sumWhen we call lazy_sum(), it does not return the result of the sum; it returns the summing function:
>>> f = lazy_sum(1, 3, 5, 7, 9)
>>> f
<function lazy_sum.<locals>.sum at 0x101c6ed90>The sum is not actually calculated until the function f is called:
>>> f()
25(2) Closures
In the example above, the inner function sum can refer to the parameters and local variables of the outer function lazy_sum.
A closure is a function that can read variables inside other functions.
One point to note is that the returned function does not execute immediately; it executes only when f() is called.
When returning a closure, remember this: the returned function should not refer to any loop variable or any variable that will subsequently change.
def count():
fs = []
for i in range(1, 4):
def f():
return i*i
fs.append(f)
return fs
f1, f2, f3 = count()
>>> f1()
9
>>> f2()
9
>>> f3()
9Using a closure means that the inner function refers to a local variable of the outer function. If we only read the value of the outer variable, the returned closure function works normally when called. However, if we assign a value to the outer variable, the Python interpreter treats x as a local variable of the function fn(), and it reports an error.
def inc():
x = 0
def fn():
nonlocal x
x = x + 1
return x
return fn
f = inc()
print(f()) # 1
print(f()) # 2The reason is that x, as a local variable, has not been initialized, so x+1 cannot be calculated directly. However, we actually want to refer to what is inside the inc() function, namely x, so inside the fn() function we must add a nonlocal x declaration. After this declaration is added, the interpreter treats fn()’s x as a local variable of the outer function. It has already been initialized, so x+1 can be calculated correctly.
2.3 Anonymous Functions
The lambda keyword denotes an anonymous function, and the x before the colon denotes the function parameter.
An anonymous function has one limitation: it can contain only one expression. There is no need to write return; the return value is the result of that expression.
>>> list(map(lambda x: x * x, [1, 2, 3, 4, 5, 6, 7, 8, 9]))
[1, 4, 9, 16, 25, 36, 49, 64, 81]An anonymous function is also a function object. It can be assigned to a variable and then called through that variable:
>>> f = lambda x: x * x
>>> f(5)
25An anonymous function can also be returned as a return value:
def build(x, y):
return lambda: x * x + y * y2.4 Decorators
Suppose we want to enhance a function—for example, by automatically printing a log before and after the function call—but do not want to modify the function’s definition. This approach of dynamically adding functionality while the code is running is called a “decorator.”
In essence, a decorator is a higher-order function that returns a function.
def log(func):
def wrapper(*args, **kw):
print('call %s():' % func.__name__)
return func(*args, **kw)
return wrapperConsider the log function above. Because it is a decorator, it accepts a function as a parameter and returns a function. (The __name__ attribute provides the function’s name.)
To use the log function, use Python’s @ syntax to place the decorator at the function definition:
@log
def now():
print('2022-6-28')Placing @log at the definition of the now() function is equivalent to executing the following statement:
now = log(now)Calling the now() function not only runs the now() function itself, but also prints a line of log output before the now() function runs:
>>> now()
call now():
2022-6-28Functions have attributes such as __name__, but after a function is decorated with a decorator, its __name__ changes from the original 'now' to 'wrapper'. Therefore, the original function’s __name__ and other attributes must be copied to the wrapper() function; otherwise, some code that relies on the function signature will fail.
A complete decorator can be written as follows:
import functools
def log(func):
@functools.wraps(func)
def wrapper(*args, **kw):
print('call %s():' % func.__name__)
return func(*args, **kw)
return wrapperA decorator with parameters:
import functools
def log(text):
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kw):
print('%s %s():' % (text, func.__name__))
return func(*args, **kw)
return wrapper
return decorator2.5 Partial Functions
Python’s functools module provides partial function functionality. It fixes some parameters of a function (that is, sets default values) and returns a new function, making the new function simpler to call.
For example, the int() function provides an additional base parameter whose default value is 10. Passing the base parameter enables base-N conversion:
>>> int('12345', base=8)
5349
>>> int('12345', 16)
74565functools.partial helps us create a partial function. We do not need to define int2() ourselves; instead, we can create a new function int2 directly with the following code:
>>> import functools
>>> int2 = functools.partial(int, base=2)
>>> int2('1000000')
64
>>> int2('1010101')
85
Comments