Corresponding course https://www.bilibili.com/video/BV18p4y167Md
Operating System Correspondence Course Operating System Practical Lectures 45
Table of Contents CSAPP An in-depth understanding of computer systems comes from the open class of Microsoft boss
yangminz
https://www.bilibili.com/video/BV17K4y1N7Q2 (under update)
Table of Contents C++ Object-Oriented Programming Series from Master Hou Jie’s CPP Object-Oriented Programming Series (Completed)
An introduction to the basic usage and options of commands.
Man is the abbreviation of manual, which displays the specific information of the instruction.
When man date is executed, DATE(1) appears. The number in it represents the type of instruction. Commonly used numbers and their types are as follows:
code name | type |
---|---|
1 | Instructions or executable files that users can operate in the shell environment |
5 | Configuration file |
8 | Administrative commands available to system administrators |
info is similar to man, but info divides the document into pages, and each page can be jumped.
/usr/share/doc stores a complete set of documentation for the software.
Before shutting down, you need to use the who command to check if there are other users online.
In order to speed up the reading and writing of disk files, the file data located in the memory will not be synchronized to the disk immediately, so a sync synchronization operation is required before shutting down.
# # shutdown [-krhc] 时间 [信息] |
-k : 不会关机,只是发送警告信息,通知所有在线的用户
-r : 将系统的服务停掉后就重新启动
-h : 将系统的服务停掉后就立即关机
-c : 取消已经在进行的 shutdown 指令内容
The path to the executable file is declared in PATH, separated by :.
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/dmtsai/.local/bin:/home/dmtsai/bin
sudo allows ordinary users to use root executable commands, but only users added in the /etc/sudoers configuration file can use this command.
RPM and DPKG are the two most common types of software package management tools:
A Linux distribution is an integrated version of the Linux kernel and various application software. Package management tools based on Commercial distributions Community distributions RPM Red Hat Fedora/CentOS DPKG Ubuntu Debian
Order | effect |
---|---|
:w | write to disk |
:w! | Forces writing to disk when the file is read-only. Whether it can be written depends on the user's permissions on the file. |
:q | leave |
:q! | Forced to leave without saving |
:wq | Write to disk and leave |
:wq! | Force write to disk and then exit |
The GNU Project, translated as the Genuine Project, aims to create a completely free operating system called GNU, and its content software is completely released under the GPL. The full name of GPL is GNU General Public License, which includes the following contents: freedom to run this program for any purpose; freedom to reproduce; freedom to improve this program and publicly release improvements. ¶Choose an open source license (opens new window) How to choose an open source license? (opens new window)
IDE (ATA) stands for Advanced Technology Attachment. The maximum interface speed is 133MB/s. Because the parallel port cable has poor anti-interference performance and the cable takes up a lot of space, which is not conducive to the internal heat dissipation of the computer, it has been gradually replaced by SATA.
SATA's full name is Serial ATA, which is an ATA interface that uses a serial port. It has strong anti-interference, has much lower requirements for the length of the data cable than ATA, and supports hot-swappable and other functions. The interface speed of SATA-II is 300MiB/s, while the new SATA-III standard can reach a transmission speed of 600MiB/s. SATA data cables are also much thinner than ATA, which is beneficial to air circulation in the chassis and makes it easier to organize the cables.
The full name of SCSI is Small Computer System Interface. It has experienced many generations of development, from the early SCSI-II to the current Ultra320 SCSI and Fiber-Channel, with various interface types. SCSI hard drives are widely used in workstation-level personal computers and servers. Therefore, they use more advanced technologies, such as a high disc speed of 15,000 rpm, and have lower CPU usage during transmission. However, the unit price is also higher than ATA and SATA hard drives of the same capacity. More expensive.
Every piece of hardware in Linux is treated as a file, including disks. Disks are named according to the disk interface type. Common disk file names are as follows:
The determination of the serial number after the file name is related to the order in which the system detects the disks, and has nothing to do with the slot position in which the disks are inserted.
There are two main formats of disk partition tables, one is the more restrictive MBR partition table, and the other is the newer and less restrictive GPT partition table.
In the MBR, the first sector is the most important. It contains the master boot record (MBR) and the partition table. The master boot record occupies 446 bytes and the partition table occupies 64 bytes.
The partition table is only 64 bytes and can only store up to 4 partitions, which are the primary partition (Primary) and the extended partition (Extended). There is only one extended partition, which uses other sectors to record additional partition tables. Therefore, more partitions can be divided through extended partitions. These partitions are called logical partitions.
Linux also treats partitions as files. The naming method of partition files is: disk file name + number, such as /dev/sda1. Note that the numbering of logical partitions starts from 5.
Different disks have different sector sizes, for example 512 bytes and 4k for the latest disks. In order to be compatible with all disks, GPT uses logical block addresses (Logical Block Address, LBA) on defined sectors. The default size of LBA is 512 bytes.
The first block of GPT records the main boot record (MBR), followed by 33 blocks to record partition information, and the last 33 blocks are used to back up partition information. The first of these 33 blocks is the GPT header record. This part records the location and size of the partition table itself and the location of the backup partition. It also contains the check code (CRC32) of the partition table. The operating system can use this Check code to determine whether the GPT is correct. If there is an error, you can use the backup partition to restore it.
GPT does not have the concept of extended partitions. They are all primary partitions. Each LAB can be divided into 4 partitions, so a total of 4 * 32 = 128 partitions can be divided.
MBR does not support hard drives above 2.2 TB, while GPT supports up to 233 TB = 8 ZB.
BIOS (Basic Input/Output System) is a firmware (software embedded in hardware). The BIOS program is stored in a read-only memory whose contents will not be lost after a power outage.
BIOS is the first program executed by the computer when it is turned on. This program knows the disk that can be booted and reads the main boot record (MBR) of the first sector of the disk. The main boot record (MBR) performs the boot management. program, this boot management program will load the core files of the operating system.
The boot manager in the Main Boot Record (MBR) provides the following functions:
menu, load core files, and transfer to other boot management programs. The transfer function can be used to implement multi-boot. You only need to install the boot management program of another operating system on the boot sector of another partition. When starting the boot management program, you can choose to start the current operating system or start the current operating system through the menu. Transfer it to other boot management programs to start another operating system.
In the figure below, the boot management program in the main boot record (MBR) of the first sector provides two menus: M1, M2. M1 points to the Windows operating system, and M2 points to the boot sectors of other partitions, which contains Another boot management program provides a menu pointing to Linux.
To install multi-boot, it is best to install Windows first and then Linux. Because the main boot record (MBR) will be overwritten when installing Windows, Linux can choose to install the boot management program in the main boot record (MBR) or the boot sector of other partitions, and can set the menu of the boot management program.
BIOS cannot read the GPT partition table, but UEFI can.
Formatting a partition is to create a file system on the partition. A partition can usually only be formatted with one file system, but technologies such as disk arrays can format a partition with multiple file systems.
The most important components are as follows:
In addition, it also includes:
It means that the block where the content of a file is located is too scattered.
The block sizes supported in the Ext2 file system are 1K, 2K and 4K. Different sizes limit the maximum size of a single file and the file system.
size | 1KB | 2KB | 4KB |
---|---|---|---|
largest single file | 16 GB | 256GB | 2TB |
Maximum file system 2TB 8TB 16TB One block can only be used by one file, and the unused part is directly wasted. Therefore, if you need to store a large number of small files, it is best to choose a smaller block.
The inode specifically contains the following information:
inode has the following characteristics:
The inode records the block number where the file content is located, but each block is very small. A large file requires hundreds of thousands of blocks. An inode has a limited size and cannot directly reference so many block numbers. Therefore, indirect, double indirect, and triple indirect references were introduced. Indirect reference means that the reference block recorded by the inode records reference information.
When a directory is created, an inode and at least one block are allocated. The content recorded by block is the inode number and file name of all files in the directory. It can be seen that the inode of the file itself does not record the file name, but the file name is recorded in the directory. Therefore, operations such as adding files, deleting files, and changing file names are related to the w permission of the directory.
If there is a sudden power outage, an error will occur in the file system. For example, the block bitmap was only modified before the power outage, but the data was not actually written into the block. The ext3/ext4 file system introduces a log function, which can be used to repair the file system.
Mounting uses a directory as the entry point to the file system, that is, after entering the directory, the file system data can be read.
In order to maintain consistency in the directory structure of different Linux distributions, the Filesystem Hierarchy Standard (FHS) specifies the directory structure of Linux.
The three most basic directories are as follows:
There are three types of users: file owners, groups and others. Different users have different file permissions.
When using ls to view a file, the information of a file will be displayed, such as drwxr-xr-x. 3 root root 17 May 6 00:14 .config
. The explanation of this information is as follows:
Common file types and their meanings are:
In the 9-digit file permission field, every 3 digits are in a group, and there are 3 groups in total. Each group represents the file permissions for the file owner, the group to which it belongs, and other people. The three digits in a set of permissions are r, w, and x permissions, indicating readable, writable, and executable.
There are three types of file times:
List the information of a file or directory. The information of the directory is the files contained in it.
# # ls [-aAdfFhilnrRSt] file|dir
-a : 列出全部的文件
-d : 仅列出目录本身
-l : 以长数据串行列出,包含文件的属性与权限等等数据
cd [相对路径或绝对路径]
# # mkdir [-mp] 目录名称
-m : 配置目录权限
-p : 递归创建目录
rmdir [-p] 目录名称
-p : 递归删除目录
# # touch [-acdmt] filename
-a : 更新 atime
-c : 更新 ctime,若该文件不存在则不建立新文件
-m : 更新 mtime
-d : 后面可以接更新日期而不使用当前日期,也可以使用 --date= "日期或时间"
-t : 后面可以接更新时间而不使用当前时间,格式为[YYYYMMDDhhmm]
Copy the file. If there are more than two source files, the destination file must be a directory.
cp [-adfilprsu] source destination
-a : 相当于 -dr --preserve=all 的意思,至于 dr 请参考下列说明
-d : 若来源文件为链接文件,则复制链接文件属性而非文件本身
-i : 若目标文件已经存在时,在覆盖前会先询问
-p : 连同文件的属性一起复制过去
-r : 递归持续复制
-u : destination 比 source 旧才更新 destination,或 destination 不存在的情况下才复制
--preserve=all : 除了 -p 的权限相关参数外,还加入 SELinux 的属性, links, xattr 等也复制了
# # rm [-fir] 文件或目录
-r : 递归删除
Move files.
# # mv [-fiu] source destination
# # mv [options] source1 source2 source3 .... directory
-f : force 强制的意思,如果目标文件已经存在,不会询问而直接覆盖
A set of permissions can be represented by numbers. At this time, the three bits of a set of permissions are regarded as the bits of a binary number. The weight of each bit from left to right is 4, 2, and 1, that is, the digital weight corresponding to each permission. The values are r: 4, w: 2, x: 1.
# # chmod [-R] xyz dirname/filename
Example: Change the permissions of the .bashrc file to -rwxr-xr--.
# # chmod 754 .bashrc
You can also use symbols to set permissions.
# # chmod [ugoa] [+-=] [rwx] dirname/filename
- u: 拥有者
- g: 所属群组
- o: 其他人
- a: 所有人
- +: 添加权限
- -: 移除权限
- =: 设定权限
Example: Add write permissions to the .bashrc file for all users.
# # chmod a+w .bashrc
**File names are not stored in the contents of a file, but in the directory in which the file is located. **Therefore, having w permission on the file cannot modify the file name. Directories store file lists, and the permissions of a directory are the permissions on its file list.
**Therefore, the r permission of the directory means that the file list can be read; the w permission means that the file list can be modified, specifically, adding, deleting files, and modifying the file name; the x permission can make the directory a working directory, and the x permission can make the directory a working directory. It is the basis of r and w permissions. If a directory cannot be made a working directory, there will be no way to read the file list or modify the file list. **
# # ln [-sf] source_filename dist_filename
-s : 默认是 hard link,加 -s 为 symbolic link
-f : 如果目标文件存在时,先删除目标文件
Create an entry in the directory to record the file name and inode number. This inode is the inode of the source file. Delete any entry and the file will still exist, as long as the number of references is not zero. It has the following restrictions: it cannot span file systems and cannot link directories.
# # ln /etc/crontab .
# # ll -i /etc/crontab crontab
34474855 -rw-r--r--. 2 root root 451 Jun 10 2014 crontab
34474855 -rw-r--r--. 2 root root 451 Jun 10 2014 /etc/crontab
The symbolic link file saves the absolute path of the source file, and will be located at the source file when reading. It can be understood as a shortcut to Windows. When the source file is deleted, the linked file cannot be opened. Links can be made to directories.
# # ll -i /etc/crontab /root/crontab2
34474855 -rw-r--r--. 2 root root 451 Jun 10 2014 /etc/crontab
53745909 lrwxrwxrwx. 1 root root 12 Jun 23 22:31 /root/crontab2 - > /etc/crontab
# # cat [-AbEnTv] filename
-n : 打印出行号,连同空白行也会有行号,-b 不会
# # head [-n number] filename
-n : 后面接数字,代表显示几行的意思
Command search.
# # which [-a] command
-a : 将所有指令列出,而不是只列第一个
File search. It's faster because it only searches a few specific directories. ## whereis [-bmsu] dirname/filename
File search. You can search using keywords or regular expressions. locate uses the /var/lib/mlocate/ database to search. It is stored in memory and updated once a day, so you cannot use locate to search for newly created files. You can use updatedb to update the database immediately.
# # locate [-ir] keyword
-r: 正则表达式
File search. You can search using the file's properties and permissions.
# # find [basedir] [option]
example: find . -name " shadow* "
(1) Time-related options
-mtime n : 列出在 n 天前的那一天修改过内容的文件
-mtime +n : 列出在 n 天之前 (不含 n 天本身) 修改过内容的文件
-mtime -n : 列出在 n 天之内 (含 n 天本身) 修改过内容的文件
-newer file : 列出比 file 更新的文件
(2) Options related to file owners and groups
-uid n
-gid n
-user name
-group name
-nouser : 搜索拥有者不存在 /etc/passwd 的文件
-nogroup: 搜索所属群组不存在于 /etc/group 的文件
(3) Options related to file permissions and names
-name filename
-size [+-]SIZE: 搜寻比 SIZE 还要大 (+) 或小 (-) 的文件。这个 SIZE 的规格有: c: 代表 byte,k: 代表 1024bytes。所以,要找比 50KB 还要大的文件,就是 -size +50k
-type TYPE
-perm mode : 搜索权限等于 mode 的文件
-perm -mode : 搜索权限包含 mode 的文件
-perm /mode : 搜索权限包含任一 mode 的文件
There are many compressed file names under Linux, the common ones are as follows:
extension | Compressor |
---|---|
*.Z | compress |
*.zip | zip |
*.gz | gzip |
*.bz2 | bzip2 |
*.xz | xz |
*.tar | The data packaged by the tar program is not compressed. |
*.tar.gz | Files packaged by the tar program and compressed by gzip |
*.tar.bz2 | Files packaged by the tar program and compressed by bzip2 |
*.tar.xz | Files packaged by the tar program and compressed by xz |
gzip is the most widely used compression command in Linux and can decompress files compressed by compress, zip and gzip.
After gzip compression, the source file no longer exists. There are 9 different compression levels available. You can use zcat, zmore, and zless to read the contents of compressed files.
$ gzip [-cdtv#] filename
-c : 将压缩的数据输出到屏幕上
-d : 解压缩
-t : 检验压缩文件是否出错
-v : 显示压缩比等信息
-# # : ## 为数字的意思,代表压缩等级,数字越大压缩比越高,默认为 6
Provides higher compression ratio than gzip. View commands: bzcat, bzmore, bzless, bzgrep.
$ bzip2 [-cdkzv#] filename
-k : 保留源文件
Provides better compression ratio than bzip2. It can be seen that the compression ratios of gzip, bzip2, and xz are constantly optimized. However, it should be noted that the higher the compression ratio, the longer the compression time will be.
View commands: xzcat, xzmore, xzless, xzgrep.
$ xz [-dtlkc#] filename
The compression command can only compress one file, while the pack can pack multiple files into one large file. Tar can not only be used for packaging, but you can also use gip, bzip2, and xz to compress the packaged files.
$ tar [-z | -j | -J] [cv] [-f 新建的 tar 文件] filename... ==打包压缩
$ tar [-z | -j | -J] [tv] [-f 已有的 tar 文件] ==查看
$ tar [-z | -j | -J] [xv] [-f 已有的 tar 文件] [-C 目录] ==解压缩
-z : 使用 zip;
-j : 使用 bzip2;
-J : 使用 xz;
-c : 新建打包文件;
-t : 查看打包文件里面有哪些文件;
-x : 解打包或解压缩的功能;
-v : 在压缩/解压缩的过程中,显示正在处理的文件名;
-f : filename: 要处理的文件;
-C 目录 : 在特定目录解压缩。
Usage | Order |
---|---|
Pack and compress | tar -jcv -f filename.tar.bz2 The name of the file or directory to be compressed |
Check | tar -jtv -f filename.tar.bz2 |
Unzip | tar -jxv -f filename.tar.bz2 -C Directory to decompress |
You can request the kernel to provide services through Shell, and Bash is one type of Shell.
Command history: record used commands Command and file completion: Shortcut key: tab Naming alias: For example, lm is the alias of ls -al shell scripts Wildcard: For example, ls -l /usr/bin/X* lists /usr/bin All files below starting with X
$
before the variable, or you can use the form ${}
;$ x=abc
$ echo $x
$ echo ${x}
If there are spaces in the variable content, double quotes or single quotes must be used.
x="lang is $LANG"
, then the value of x is lang is zh_TW.UTF-8;x='lang is $LANG'
, then the value of x is lang is $LANG
. 指令
or $(指令)
to assign the execution results of instructions to variables. For example, version=$(uname -r), the value of version is 4.15.0-22-generic. You can use the export command to convert custom variables into environment variables. Environment variables can be used in subprograms. The so-called subprograms are sub-Bash generated by the current Bash. Bash variables can be declared as arrays and integer numbers. Note that numeric types do not have floating point numbers. If not declared, the default is string type. Variables are declared using the declare command: $ declare [-aixr] variable
-a : 定义为数组类型
-i : 定义为整数类型
-x : 定义为环境变量
-r : 定义为 readonly 类型
Use [ ] to index an array:
$ array[1]=a
$ array[2]=b
$ echo ${array[1]}
$PATH
variable and execute it.Redirection refers to using files instead of standard input, standard output, and standard error output.
1 | code | operator |
---|---|---|
Standard input (stdin) | 0 | < or << |
Standard output (stdout) | 1 | > or >> |
Standard error output (stderr) | 2 | 2> or 2>> |
Among them, one arrow indicates redirection in an overwriting manner, while two arrows indicates redirection in an appending manner.
Unnecessary standard output and standard error output can be redirected to /dev/null, which is equivalent to throwing it into the trash. If you need to redirect standard output and standard error output to a file at the same time, you need to convert one output to another output. For example, 2>&1 means converting standard error output to standard output.
$ find /home -name .bashrc > list 2>&1
$ nohup latte-dock --replace > /dev/null 2>&1 & # 没有任何输出的后台任务
Pipelining is to use the standard output of one command as the standard input of another command. Pipelining can be used when the data needs to be processed in multiple steps before we can get what we want.
Use | between commands to separate individual pipeline commands.
$ ls -al /etc | less
cut splits the data and extracts the desired part. The segmentation process proceeds line by line.
$ cut
-d : 分隔符
-f : 经过 -d 分隔后,使用 -f n 取出第 n 个区间
-c : 以字符为单位取出区间
$ last
root pts/1 192.168.201.101 Sat Feb 7 12:35 still logged in
root pts/1 192.168.201.101 Fri Feb 6 12:13 - 18:46 (06:33)
root pts/1 192.168.201.254 Thu Feb 5 22:37 - 23:53 (01:16)
$ last | cut -d ' ' -f 1
$ export
declare -x HISTCONTROL= " ignoredups "
declare -x HISTSIZE= " 1000 "
declare -x HOME= " /home/dmtsai "
declare -x HOSTNAME= " study.centos.vbird "
.....(其他省略).....
$ export | cut -c 12-
used for sorting.
$ sort [-fbMnrtuk] [file or stdin]
-f : 忽略大小写
-b : 忽略最前面的空格
-M : 以月份的名字来排序,例如 JAN,DEC
-n : 使用数字
-r : 反向排序
-u : 相当于 unique,重复的内容只出现一次
-t : 分隔符,默认为 tab
-k : 指定排序的区间
$ cat /etc/passwd | sort -t ' : ' -k 3
root:x:0:0:root:/root:/bin/bash
dmtsai:x:1000:1000:dmtsai:/home/dmtsai:/bin/bash
alex:x:1001:1002::/home/alex:/bin/bash
arod:x:1002:1003::/home/arod:/bin/bash
You can take only one duplicate data.
$ uniq [-ic]
-i : 忽略大小写
-c : 进行计数
$ last | cut -d ' ' -f 1 | sort | uniq -c
1
6 (unknown
47 dmtsai
4 reboot
7 root
1 wtmp
Output redirection will redirect the output content to a file, and tee can not only complete this function, but also preserve the output on the screen. That is, using the tee command, an output is sent to both the file and the screen.
$ tee [-a] file
Used to delete characters in a line or replace characters.
$ tr [-ds] SET1 ...
-d : 删除行中 SET1 这个字符串
$ last | tr ' [a-z] ' ' [A-Z] '
Convert tab characters to space characters.
$ col [-xb]
-x : 将 tab 键转换成对等的空格键
Convert tabs to a certain number of spaces, the default is 8.
$ expand [-t] file
-t : tab 转为空格的数量
Merge rows with the same data together.
$ join [-ti12] file1 file2
-t : 分隔符,默认为空格
-i : 忽略大小写的差异
-1 : 第一个文件所用的比较字段
-2 : 第二个文件所用的比较字段
Paste the two lines directly together.
$ paste [-d] file1 file2
-d : 分隔符,默认为 tab
split divides a file into multiple files.
$ split [-bl] file PREFIX
-b : 以大小来进行分区,可加单位,例如 b, k, m 等
-l : 以行数来进行分区。
- PREFIX : 分区文件的前导名称
g/re/p (globally search a regular expression and print), use regular expressions to globally search and print.
$ grep [-acinv] [--color = auto] 搜寻字符串 filename
-c : 统计个数
-i : 忽略大小写
-n : 输出行号
-v : 反向选择,也就是显示出没有 搜寻字符串 内容的那一行
--color=auto : 找到的关键字加颜色显示
$ grep -n ' the ' regular_express.txt
8:I can ' t finish the test.
12:the symbol ' * ' is represented as start.
15:You are the best is mean you are the no. 1.
16:The world Happy is the same with "glad".
18:google is the best tools for search keyword
Because { and } have special meaning in the shell, they must be escaped using escape characters.
$ grep -n ' go{2,5}g ' regular_express.txt
For formatted output. It does not belong to the pipeline command. You need to use the $() form when transmitting data to printf.
$ printf ' %10s %5i %5i %5i %8.2f n ' $( cat printf.txt )
DmTsai 80 60 92 77.33
VBird 75 55 80 70.00
Ken 60 90 70 73.33
It was created by Alfred Aho, Peter Weinberger, and Brian Kernighan. The name awk is the initials of the three founders.
awk processes one line at a time. The smallest unit of processing is a field. The naming method of each field is