Python Review Notes 4—File and Directory Operations

I. Reading and Writing Files

1.1 Reading Files

(1) Opening a File

Use Python’s built-in open() function, passing in the file name and mode specifier, to open a file object:

>>> import os
>>> f = open('/Users/michael/test.txt', 'r')

The mode specifier 'r' means read, so we have successfully opened a file.

Python provides the with statement to avoid errors:

with open('/path/to/file', 'r') as f:
    print(f.read())

The code above is equivalent to:

try:
    f = open('/path/to/file', 'r')
    print(f.read())
finally:
    if f:
        f.close()

(2) Reading File Contents

Method	Purpose
read()	Reads the entire file at once
read(size)	Reads at most size bytes each time
readline()	Reads one line at a time
readlines()	Reads all content at once and returns it as a list of lines

(3) Closing a File

Call the close() method to close the file. A file must be closed after use because the file object occupies operating system resources.

>>> f.close()

(4) Reading Files in Other Encodings

To read binary files, such as images and videos, simply open the file in 'rb' mode:

>>> f = open('/Users/michael/test.jpg', 'rb')
>>> f.read()
b'\xff\xd8\xff\xe1\x00\x18Exif\x00\x00...' # 十六进制表示的字节

To read a text file that is not encoded in UTF-8, give the open() function an encoding parameter. For example, to read a GBK-encoded file:

>> f = open('/Users/michael/gbk.txt', 'r', encoding='gbk')
>>> f.read()
'测试'

1.2 Writing Files

Call open() and pass the mode specifier 'w' or 'wb' to write a text file or binary file:

>>> f = open('/Users/michael/test.txt', 'w')
>>> f.write('Hello, world!')
>>> f.close()

You can call write() repeatedly to write to a file, but you must call f.close() to close it. When we write a file, the operating system often does not write the data to disk immediately. Instead, it caches the data in memory and writes it gradually when idle. Only when the close() method is called does the operating system guarantee that all unwritten data is written to disk.

Therefore, we usually still use the with statement when writing files:

with open('/Users/michael/test.txt', 'w') as f:
    f.write('Hello, world!')

II. File and Directory Operations

Python’s built-in os module can directly call interface functions provided by the operating system.

2.1 Directory Operations

(1) Viewing a Directory

# 查看当前目录的绝对路径:
>>> os.path.abspath('.')
'/Users/michael'

(2) Creating a Directory

# 在某个目录下创建一个新目录，首先把新目录的完整路径表示出来:
>>> os.path.join('/Users/michael', 'testdir')
'/Users/michael/testdir'
# 然后创建一个目录:
>>> os.mkdir('/Users/michael/testdir')

(3) Deleting a Directory

# 删掉一个目录:
>>> os.rmdir('/Users/michael/testdir')

(4) Joining and Splitting Paths

When joining two paths, do not concatenate strings directly. Use the os.path.join() function instead.

>>> os.path.join('/Users/michael', 'testdir')
'/Users/michael/testdir'

When splitting a path, likewise do not split the string directly. Use the os.path.split() function instead.

>>> os.path.split('/Users/michael/testdir/file.txt')
('/Users/michael/testdir', 'file.txt')

(5) Getting a File Extension

os.path.splitext() lets you get the file extension directly:

>>> os.path.splitext('/path/to/file.txt')
('/path/to/file', '.txt')

2.2 File Operations

(1) Renaming a File

# 对文件重命名:
>>> os.rename('test.txt', 'test.py')

(2) Deleting a File

# 删掉文件:
>>> os.remove('test.py')

(3) Copying and Moving Files

The os module does not provide functions for copying and moving files, but the shutil module provides the copyfile() function and can be regarded as a supplement to the os module.

shutil.copy(src_file, target_path) shutil.move(src_file, target_path)

import shutil
shutil.copy('/Users/michael/testdir1/file.txt', '/Users/michael/testdir2')
shutil.move('/Users/michael/testdir1/file.txt', '/Users/michael/testdir2')

III. JSON

To pass objects between different programming languages, the objects must be serialized into a standard format. The most common approach is to serialize them as JSON.

Because JSON is represented as a string, it can be read by all languages and can also be conveniently stored on disk or transmitted over a network. JSON is not only a standard format but is also faster than XML, and it can be read directly on Web pages, making it very convenient.

3.1 Python dict -> JSON

Python’s built-in json module provides very comprehensive conversion from Python objects to JSON format. First, let’s see how to turn a Python object into JSON:

>>> import json
>>> d = dict(name='Bob', age=20, score=88)
>>> json.dumps(d)
'{"age": 20, "score": 88, "name": "Bob"}'

The dumps() method returns a str whose content is standard JSON. The dump() method can write JSON directly to a file-like Object.

3.2 JSON -> Python dict

To deserialize JSON into a Python object, use loads() or the corresponding load() method. The former deserializes a JSON string, while the latter reads and deserializes a string from a file-like Object:

>>> json_str = '{"age": 20, "score": 88, "name": "Bob"}'
>>> json.loads(json_str)
{'age': 20, 'score': 88, 'name': 'Bob'}

3.3 Python Class -> JSON

Because we normally use a class to represent an object, serializing an object directly will certainly raise an error. This is because a Student object is not an object that can be serialized as JSON.

import json

class Student(object):
    def __init__(self, name, age, score):
        self.name = name
        self.age = age
        self.score = score

s = Student('Bob', 20, 88)
print(json.dumps(s))

# 输出
Traceback (most recent call last):
  ...
TypeError: <__main__.Student object at 0x10603cc50> is not JSON serializable

However, the dumps() method also provides many optional parameters. Among them, the optional default parameter supports providing a conversion function. By passing an object instance and a conversion function to dumps(), serialization can be implemented.

The concise version is as follows:

print(json.dumps(s, default=lambda obj: obj.__dict__))

This works because an instance of a class usually has a __dict__ attribute. It is a dict used to store instance variables.

3.4 JSON -> Python Class

If we want to deserialize JSON into a Student object instance, the loads() method first converts it into a dict object. Then, the object_hook function we pass in is responsible for converting the dict into a Student instance:

def dict2student(d):
    return Student(d['name'], d['age'], d['score'])

>>> json_str = '{"age": 20, "score": 88, "name": "Bob"}'
>>> print(json.loads(json_str, object_hook=dict2student))
<__main__.Student object at 0x10cd3c190>