Shell string operations

$ i=5;j=9423483247234;$ echo $i | grep -q "^[0-9]$"$ echo $?0$ echo $j | grep -q "^[0-9]+$" $ echo $?0

Example: Character combination (lowercase letters, uppercase letters, combination of both)

 $ c="A"; d="fwefewjuew"; e="fewfEFWefwefe"$ echo $c | grep -q "^[AZ]$"$ echo $d | grep -q "^[az]+$" $ echo $e | grep -q "^[a-zA-Z]+$"

Example: combination of letters and numbers

 $ ic="432fwfwefeFWEwefwef"$ echo $ic | grep -q "^[0-9a-zA-Z]+$"

Example: Space or Tab key, etc.

 $ echo " " | grep " "$ echo -e "t" | grep "[[:space:]]" #[[:space:]] will match both spaces and TAB keys $ echo -e " t" | grep "[[:space:]]"$ echo -e "t" | grep "" #To press the TAB key on the keyboard, not the character

Example: Match email addresses

 $ echo "[email protected]" | grep "[0-9a-zA-Z.]*@[0-9a-zA-Z.]"[email protected]

Example: Match URL address (take http link as an example)

 $ echo "http://news.lzu.edu.cn/article.jsp?newsid=10135" | grep "^http://[0-9a-zA-Z./=?]+$"http ://news.lzu.edu.cn/article.jsp?newsid=10135

illustrate:

 /dev/null

and

 /dev/zero

The equipment is very interesting. It is like a black hole. Anything that falls into it will disappear. The latter is also an energy box, and you can always get 0 from there until you exit.

 [[:space:]]

yes

 grep

Marker used to match spaces or TAB key characters. Please check the help for other markers:

 man grep

All of the above are used

 grep

to perform pattern matching, in fact

 sed

,

 awk

Both can be used for pattern matching. For knowledge about regular expressions used in matching, please refer to the relevant information below.

If you want to determine whether a string is empty, you can determine whether its length is zero. You can use

 test

commanding

 -z

options to implement, see the specific usage

 test

Order,

 man test

Example: Determine whether a character is a printable character

 $ echo "tn" | grep "[[:print:]]"tn$ echo $?0$ echo -e "tn" | grep "[[:print:]]"$ echo $?1

length of string

In addition to the character types that make up a string, what other properties does a string have? The number of characters that make up the string.

Next we will calculate the length of a string, that is, the number of all characters, and briefly introduce several methods for finding the number of specified characters in a string.

Example: Calculate the length of a string

That is to count the number of all characters, there are various calculation methods, choose the best one and use it:

 $ var="get the length of me"$ echo ${var} # This is equivalent to $varget the length of me$ echo ${#var}20$ expr length "$var"20$ echo $var | awk '{ printf("%dn", length($0));}'20$ echo -n $var | wc -c20

Example: Count the number of certain specified characters or character combinations

 $ echo $var | tr -cd g | wc -c2$ echo -n $var | sed -e 's/[^g]//g' | wc -c2$ echo -n $var | sed -e 's /[^gt]//g' | wc -c5

Example: Count the number of words

For more related information, see the word statistics related examples in "Numerical Computation".

 $ echo $var | wc -w5$ echo "$var" | tr " " "n" | grep get | uniq -c1$ echo "$var" | tr " " "n" | grep get | wc -l1

illustrate:

 ${}

Operators are a "big name" in Bash and are capable of quite a lot of tasks. For details, see "Thirteen Questions about Shell" written by someone on the Internet.

 $(())

and

 $()

besides

 ${}

What's the difference? "Bar.

display of string

Next, we discuss how to control the display of characters in the terminal.

Example: Control character display position, color, background, etc. on the screen

 $ echo -e "33[31;40m" #Set the foreground color to black and the background color to red $ echo -e "33[11;29H Hello, World!" #In line 11, column 29 of the screen Start printing the string Hello, World!

Example: Dynamically display the current system time at a certain location on the screen

 $ while :; do echo -e "33[11;29H "$(date "+%Y-%m-%d %H:%M:%S"); done

Example: Filter out certain control strings

use

 col

The command filters certain control characters when processing such as

 script

,

 screen

This is useful when waiting for the output of the screenshot command.

 $ screen -L$ cat /bin/cat$ exit$ cat screenlog.0 | col -b # After filtering some control characters, you can keep readable operation logs

String storage

In our opinion, a string is just a series of characters, but for the convenience of operation, we can often make the string show a certain structure. Here, we don't care about the actual storage structure of the string in memory, only the logical structure it presents. For example, a string like this:

 get the length of me"

, we can present it from different aspects.

Render a character by its position in a string

In this way we can find a certain substring by specifying the position. This can usually be done using pointers in C language. In shell programming, there are many tools available, such as

 expr

,

 awk

Both provide similar methods to implement substring query actions. Both almost support pattern matching

 match

and exact match

 index

. This will be described in detail later in string operations.

Get each part of a string based on a certain delimiter

The most common ones here are line separators, spaces or

 TAB

separator. The former is used as a line number. We seem to have become accustomed to it, because our editor handles the line separator "inexplicably" in this way (under UNIX, it is

 \n

, there are some differences under other systems, such as under Windows

 rn

). And spaces or

 TAB

Keys are often used to separate fields in a database, and this seems to be a common thing.

Because of this, a large number of excellent line editing tools have been produced, such as

 grep

,

 awk

,

 sed

wait. In terms of string splitting "inline" (let's put it this way, it means processing a single line, that is, the string no longer contains the line separator),

 cut

and

 awk

Provides very superior "in-line" (processing of a single line) processing capabilities.

It is more convenient to process various parts separated by separators

The delimiter is also used, but in order to more conveniently operate the various parts of the string after splitting, we abstract the data structure of "array", which makes it more convenient for us to obtain a specified part through the subscript.

 bash

provides such a data structure, and the excellent

 awk

It is also provided, and we will briefly introduce their usage here.

Example: Split a string into an array of strings

The array data structure provided by Bash uses numbers as subscripts, which is the same as the subscripts starting from 0 in C language.

 $ var="get the length of me"$ var_arr=($var) #Store the string var into the string array var_arr. By default, spaces are used as separators. $ echo ${var_arr[0]} ${var_arr[1 ]} ${var_arr[2]} ${var_arr[3]} ${var_arr[4]}get the length of me$ echo ${var_arr[@]} #For the entire string, * can be used instead of @, the same below get the length of me$ echo ${#var_arr[@]} #Similar to finding the length of a string, the `#` operator can also be used to find the number of array elements 5

You can also directly assign a value to an array element

 $ var_arr[5]="new_element"$ echo ${var_arr[5]}6$ echo ${var_arr[5]}new_element

Bash actually also provides an "array"-like functionality, which is

 for i in

, which can easily obtain various parts of a string, for example:

 $ for i in $var; do echo -n $i"_"; doneget_the_length_of_me_

 awk

array in , pay attention to compare it with

 Bash

Similarities and differences of the arrays in

 split

Split a line by spaces and store it in an array

 var_arr

, and returns the array length. Note: The subscript of the first element here is not 0, but 1

 $ echo $var | awk '{printf("%d %sn", split($0, var_arr, " "), var_arr[1]);}'5 get

In fact, the above operations are very similar

 awk

Own row processing function:

 awk

By default, a line is divided into multiple fields according to spaces, and can be passed

 $1

,

 $2

,

 $3``...

to get,

 $0

Represents the entire line.

Here's

 NF

It is the total number of fields in the row, similar to the length of the array above. It also provides a function to access a string through a similar "subscript".

 $ echo $var | awk '{printf("%d | %s %s %s %s %s | %sn", NF, $1, $2, $3, $4, $5, $0);}'5 | get the length of me | get the length of me

 awk

The "array" function of

 for

Quote, note that this and

 Bash

inside

 for

Not quite the same,

 i

Not the element itself, but the subscript:

 $ echo $var | awk '{split($0, var_arr, " "); for(i in var_arr) printf("%s ",var_arr[i]);}'of me get the length4 5 1 2 3

In addition, it can be seen from the above results that after

 for

After processing, the entire result is not arranged in the principle character order, but it is also meaningful if it is just iterating out all elements.

 awk

There is also more "powerful" processing power. Its subscript can be not a number, but a string, thus becoming an "associative" array. This "association" is very convenient in some aspects. For example, to replace a certain system call name in a file with an address based on the function address mapping table in another file, you can do this:

 $ cat symbolsys_exitsys_readsys_close$ ls /boot/System.map*$ awk '{if(FILENAME ~ "System.map") map[$3]=$1; else {printf("%sn", map[$1])} }'  /boot/System.map-2.6.20-16-generic symbolc0129a80c0177310c0175d80

In addition, awk also supports using the delete function to delete an array element. If necessary in some situations, don't forget that awk also supports two-dimensional arrays.

String general operations

String operations include substring retrieval, substring query, substring insertion, substring deletion, substring replacement, substring comparison, substring sorting, substring hexadecimal conversion, substring encoding conversion, etc.

Get substring

The main methods for obtaining substrings are:

Go directly to the specified position to find the substring

Character matching to find substring

Example: Get a substring based on position

For example, where to start and how many characters to take?

 $ var="get the length of me"$ echo ${var:0:3}get$ echo ${var:(-2)} # In the opposite direction me$ echo `expr substr "$var" 5 3` # Remember to quote $var, otherwise expr will parse errors due to spaces the$ echo $var | awk '{printf("%sn", substr($0, 9, 6))}'length

 awk

Bundle

 $var

Separate into multiple variables according to spaces, in order

 $1

,

 $2

,

 $3

,

 $4

,

 $5

 $ echo $var | awk '{printf("%sn", $1);}'get$ echo $var | awk '{printf("%sn", $5);}'me

Almost omitted

 cut

A small tool that is similar to awk.

 -d

Specify the delimiter, as in

 awk

use

 -F

The specified delimiter is the same;

 -f

Specify "domain", like awk's

 $number

.

 $ echo $var | cut -d" " -f 5

Example: Match characters to find substrings

Use Bash's built-in support to find strings:

 $ echo ${var%% *} #Start from the right and delete all characters to the right of the leftmost space get$ echo ${var% *} #Start from the right and delete all characters to the right of the first space get the length of$ echo ${var##* } #Start from the left and delete all characters to the left of the rightmost space me$ echo ${var#* } #Start from the left and delete all characters to the left of the first space the length of me

Delete all

 Space + letter combination

string:

 $ echo $var | sed 's/ [az]*//g'get$ echo $var | sed 's/[az]* //g'me

 sed

There is a function to print (p) by address (line), remember to use it first

 tr

Replace spaces with line numbers:

 $ echo $var | tr " " "n" | sed -n 1pget$ echo $var | tr " " "n" | sed -n 5pme

 tr

It can also be used to get substrings, it can be similar to

 #

and

 %

To "remove" some strings to achieve substrings:

 $ echo $var | tr -d " "getthelengthofme$ echo $var | tr -cd "[az]" #Remove all spaces and only retain the letter string. Pay attention to the usage of -c and -d getthelengthofme

illustrate:

 %

and

 #

The directions for deleting characters are different, the former is on the right and the latter is on the left.

 %%

and

 %

,

 ##

and

 #

The direction is that the former is the maximum match and the latter is the minimum match. (For a good memory method, see the keyboard memory method of people on the Internet:

 #

,

 $

,

 %

are the three keys on the keyboard from left to right)

 tr

of

 -c

The options are

 complement

the abbreviation of

 invert

,and

 -d

The option is delete,

 tr -cd "[az]"

This will keep all the letters

For string interception, there are actually some commands. If

 head

,

 tail

You can achieve interesting functions by intercepting the specified number of lines or bytes in front and behind a string. For example:

 $ echo "abcdefghijk" | head -c 4abcd$ echo -n "abcdefghijk" | tail -c 4hijk

Query substring

Substring queries include:

Returns the substring itself that matches a pattern

Returns the position of the substring in the target string

Preparation: Before performing the following operations, please prepare a file test.txt, which contains the content "consists of" for the following example.

Example: Query the position of a substring in the target string

 expr index

It seems that it can only return the position where the first character of a certain character or multiple characters appears.

 $ var="get the length of me"$ expr index "$var" t3

awk can find strings, and match can also match regular expressions.

 $ echo $var | awk '{printf("%dn", match($0,"the"));}'5

Example: Query a substring and return the rows containing the substring

 awk

,

 sed

These functions can be realized, but

 grep

best at

 $ grep "consists of" test.txt # Query the lines that the file contains "consists of" and print these lines $ grep "consists[[:space:]]of" -n -H test.txt # Print the file name and the location of the substring The line number of the line and the content of the line $ grep "consists[[:space:]]of" -n -o test.txt # Only print the line number and the content of the matched substring itself $ awk '/consists of/{ printf("%s:%d:%sn",FILENAME, FNR, $0)}' test.txt #See it? The result is the same as grep $ sed -n -e '/consists of/=;/consists of/p' test.txt #The line number can also be printed

illustrate:

 awk

,

 grep

,

 sed

Both can find specified strings through pattern matching, but they each have their own strengths. We will continue to use and compare them in subsequent chapters to discover their respective advantages.

Let’s treat the file content as a large string here. File operations will be introduced specifically in the following chapters, so there will be a more in-depth analysis and introduction to the operation of storing strings in the file content.

substring replacement

Substring replacement is to replace a specified substring with another string, which includes the operations of "insert substring" and "delete substring". For example, if you want to insert a string before a substring, you can replace the original substring with "substring + new string". If you want to delete a substring, replace the substring with an empty string. However, some tools provide some special usage for inserting and deleting substrings, so I will introduce them specifically. In addition, if you want to replace a certain substring, you usually first find the substring (query the substring) and then replace it. In fact, many tools reflect this in their use and design.

Example: Replace spaces in variable var with underscores

use

 {}

Operator, remember? Tutorials from people on the internet

 $ var="get the length of me"$ echo ${var/ /_} #Replace the first space with an underscore get_the length of me$ echo ${var// /_} #Replace all spaces with an underscore get_the_length_of_me

use

 awk

,

 awk

Provides a minimal replacement function for transformations

 sub

and global replacement function

 gsub

,similar

 /

and

 //

 $ echo $var | awk '{sub(" ", "_", $0); printf("%sn", $0);}'get_the length of me$ echo $var | awk '{gsub(" " , "_", $0); printf("%sn", $0);}'get_the_length_of_me

use

 sed

, substring replacement can be

 sed

Specialties:

 $ echo $var | sed -e 's/ /_/' #s <= substitudeget_the length of me$ echo $var | sed -e 's/ /_/g' #See if you can, just two short commands to achieve it The minimum match and maximum match g <= globalget_the_length_of_me

Have forgotten

 tr

An order? You can replace a single character with:

 $ echo $var | tr " " "_"get_the_length_of_me$ echo $var | tr '[az]' '[AZ]' #This is interesting, replace all lowercase letters with uppercase letters GET THE LENGTH OF ME

illustrate:

 sed

There are also very interesting tag usages, which I will introduce below.

One of the more interesting string replacements is the inversion of the entire file line. This can be done by

 tac

Command implementation, it will reverse all the lines in the file. In a sense, sorting is actually a string replacement.

insert substring

Insert a substring at the specified position. This position may be the position of a certain substring, or it may be a certain length from the beginning of a file. Through the above exercise, we found that the two are actually similar.

Formula: insert substring = replace "old substring" with "old substring+new substring" or "new substring+old substring"

Example: Insert an underscore before or after a space in the var string

use{}:

 $ var="get the length of me"$ echo ${var/ /_ } #Insert a string before the specified string get_ the length of me$ echo ${var// /_ }get_ the_ length_ of_ me$ echo ${var/ / _} #Insert a string after the specified string get _the length of me$ echo ${var// / _}get _the _length _of _me

Do you still need to demonstrate for others? Here we mainly introduce how to use sed to insert characters, because its label function is very interesting:

 (

and

 )

Store unmatched strings as a label, in matching order:

 1

,

 2

...

 $ echo $var | sed -e 's/( )/_1/'get_ the length of me$ echo $var | sed -e 's/( )/_1/g'get_ the_ length_ of_ me$ echo $var | sed -e 's/( )/1_/'get _the length of me$ echo $var | sed -e 's/( )/1_/g'get _the _length _of _me

have a look

 sed

The order of the tags is not

 1

,

 2

..., see?

 2

and

 1

After changing the position,

 the

and

 get

The position of has been changed:

 $ echo $var | sed -e 's/([az]*) ([az]*) /2 1 /g'the get of length me

 sed

There are also special insertion instructions,

 a

and

 i

, respectively indicating inserting the specified character after the matching line and before the line.

 $ echo $var | sed '/get/a test'get the length of metest$ echo $var | sed '/get/i test'testget the length of me

Delete substring

Deleting a substring: It should be very simple. Replacing the substring with "empty" (nothing) means deletion. Let’s briefly review substitutions.

Example: Remove all spaces in the var string.

Encouragement: I don’t know what word this substitution will turn into. Who knows it? But Chinese is connected together, so have you ever thought about how difficult Chinese is? It turns out that you are also a language genius, and English is not terrible. You have the talent to learn it, as long as you have the intention.

reuse

 {}

 $ echo ${var// /}getthelengthofme

reuse

 awk

 $ echo $var | awk '{gsub(" ","",$0); printf("%sn", $0);}'

reuse

 sed

 $ echo $var | sed 's/ //g'getthelengthofme

There are simpler

 tr

Order,

 tr

You can also delete the spaces, see

 $ echo $var | tr -d ""getthelengthofme

What if you want to delete all strings after the first space? still remember

 {}

of

 #

and

 %

Usage? If you don’t remember, go back to the beginning of this section and start reviewing. (In fact, deleting substrings and taking substrings are two complementary operations. Deleting some unwanted substrings will also obtain other desired substrings at the same time - this world is a "binary" world, very interesting)

Substring comparison

This one is simple: remember

 test

How to use the command?

 man test

. It can be used to determine whether two strings are equal. In addition, there are "whether strings are equal" and "whether strings match another string."

 "

Is there a relationship between the two questions? Two strings are equal if they match exactly. Therefore, the string matching method used above can also be used here.

Substring sorting

I almost forgot this important content. Substring sorting is often used. Common ones include alphabetical order, numerical order, etc. positive or negative order.

 sort

The command can be used to do this work. Like other line processing commands, it operates on a line basis. In addition, it is similar to

 cut

and

 awk

, you can specify the delimiter and specify the column to be sorted.

 $ var="get the length of me"$ echo $var | tr ' ' 'n' | sort #Forward sorting getlengthmeofthe$ echo $var | tr ' ' 'n' | sort -r #Reverse sorting theofmelengthget $ cat > data.txt1 2 3 4 5 6 7 8 9 10 11 12 13 14 1541 45 44 44 26 44 42 20 20 38 37 25 45 45 4516 17 18 19 20 21 22 23 24 25 26 27 28 29 3044 20 30 39 35 38 38 28 25 30 36 20 24 32 3331 32 33 34 35 36 37 38 39 40 41 42 43 44 4541 33 51 39 20 20 44 37 38 39 42 40 37 50 5046 47 48 49 50 51 52 53 54 55 5642 43 41 42 45 42 19 39 75 17 17$ cat data.txt | sort -k 2 -n1 2 3 4 5 6 7 8 9 10 11 12 13 14 1516 17 18 19 20 21 22 23 24 25 26 27 28 29 3044 20 30 39 35 38 38 28 25 30 36 20 24 32 3331 32 33 34 35 36 37 38 39 40 41 42 43 44 4541 33 51 39 20 20 44 37 38 39 42 40 37 50 5042 43 41 42 45 42 19 39 75 17 1741 45 44 44 26 44 42 20 20 38 37 25 45 45 4546 47 48 49 50 51 52 53 54 55 56

Substring base conversion

If alphanumeric characters are used for counting, then there is a problem with base conversion. In the section "Numerical Calculation", it has been introduced

 bc

Command, review it here.

 $ echo "ibase=10;obase=16;10" | bcA

illustrate:

 ibase

Specify the input base,

 obase

Indicate the output base, so by adjusting

 ibase

and

 obase

, you can turn it however you want!

substring encoding conversion

What is character encoding? This needs no introduction. Have you ever seen those web pages displayed in a mess? Most of the time it is caused by the inconsistency between the "encoding" used by the browser and the actual "encoding" used by the web page. Character encoding usually refers to converting a sequence of "printable" characters into a binary representation, while character decoding performs the opposite process. If the two processes do not match, so-called "garbled characters" will appear.

In order to solve the "garbled code" problem? Encoding conversion is required. Under Linux, we can use

 iconv

This tool is used to perform related operations. This situation is often encountered when moving files between different operating systems and exchanging files between different editors. Currently, the commonly used Chinese character encoding under Windows is

 gb2312

, while under Linux, it is mostly used

 utf8

.

 $ nihao_utf8=$(echo "Hello")$ nihao_gb2312=$(echo $nihao_utf8 | iconv -f utf8 -t gb2312)

Advanced string operations

In fact, when programming in Bash, you spend most of your time processing strings, so it is very important to master this section.

regular expression

Example: Processing URL addresses

URL address (URL (Uniform Resoure Locator: Uniform Resource Locator) is the address of a WWW page) is almost a playmate in our daily life. We have reached the point where we cannot leave it. There are many operations on it, including determining the URL address. Validity, intercept various parts of the address (server type, server address, port, path, etc.) and perform further operations on each part.

Let's deal with this URL address in detail: ftp://anonymous:[email protected]/software/scim-1.4.7.tar.gz

 $ url="ftp://anonymous:[email protected]/software/scim-1.4.7.tar.gz"

Match the URL address and determine the validity of the URL address

 $ echo $url | grep "ftp://[az]*:[az]*@[az./-]*"

Interception server type

 $ echo ${url%%:*}ftp$ echo $url | cut -d":" -f 1ftp

Intercept domain name

 $ tmp=${url##*@} ; echo ${tmp%%/*}mirror.lzu.edu.cn

intercept path

 $ tmp=${url##*@} ; echo ${tmp%/*}mirror.lzu.edu.cn/software

Intercept file name

 $ basename $urlscim-1.4.7.tar.gz$ echo ${url##*/}scim-1.4.7.tar.gz

Intercept file type (extension)

 $ echo $url | sed -e 's/.*[0-9].(.*)/1/g'tar.gz

Example: Match a specific range of lines in a file

First prepare a test file README

 Chapter 7 -- Exercises7.1 please execute the program: mainwithoutreturn, and print the return value of it with the command "echo $?", and then compare the return of the printffunction, they are the same.7.2 it will depend on the execution mode, interactive or redirection to a file,if interactive, the "output" action will accur after the n char with the linebuffer mode, else, it will be really "printed" after all of the strings havebeen stayed in the buffer.7.3 there is no another effective method in most OS. because argc and argv arenot global variables like environ.

Then start experimenting,

Before printing out the answer, specify the line range: line 7 to line 9, and just found the answer to question 2

 $ sed -n 7,9p README7.2 it will depend on the execution mode, interactive or redirection to a file,if interactive, the "output" action will accur after the n char with the linebuffer mode, else, it will be really "printed" after all of the strings have

In fact, because the content format of this file is very unique, there is an easier way

 $ awk '/7.2/,/^$/ {printf("%sn", $0);}' README7.2 it will depend on the execution mode, interactive or redirection to a file,if interactive, the "output " action will accur after the n char with the linebuffer mode, else, it will be really "printed" after all of the strings have been stayed in the buffer.

With the above knowledge, you can perform these tasks very easily: modify the file name of a file, such as adjusting its encoding, downloading all the files in a web page

 pdf

Documentation etc. Just do these as exercises yourself.

Handle formatted text

When I usually do work, most of the time I deal with some "formatted" text, such as something like

 /etc/passwd

Such text with fixed rows and columns is also similar to

 tree

The kind of text with a tree structure that the command outputs, and of course other text with a specific structure.

Regarding the processing of tree-structured text, you can refer to another blog post I wrote earlier: Source code analysis: static analysis C program function call diagram

In fact, as long as you grasp some characteristics of the characteristic structure and according to the specific application, it will not be difficult to deal with it.

The following introduces the operation of specific text, so as to

 /etc/passwd

file as an example. For help and usage of this file, please go to

 man 5 passwd

Check. Let's perform some meaningful operations on this file and related files.

Example: Select specific columns

Select the username and group ID columns in the /etc/passwd file

 $ cat /etc/passwd | cut -d":" -f1,4

Select the group name and group ID columns in the /etc/group file

 $ cat /etc/group | cut -d":" -f1,3

Example: File association operations

What if you want to find out which groups all users belong to?

 $ join -o 1.1,2.1 -t":" -1 4 -2 3 /etc/passwd /etc/grouproot:rootbin:bindaemon:daemonadm:admlp:lppop:popnobody:nogroupfalcon:users

illustrate:

 join

The command is used to connect two files, which is somewhat similar to the connection of two tables in a database.

 -t

Specify the delimiter,

 -1 4 -2 3

Specify the group according to column 4 of the first file and column 3 of the second file.

 ID

to connect,

 -o``1.1,2.1

It means that only the first column of the first file and the first column of the second file are output, so that we can get the result we want. However, unfortunately, this result is not accurate. If you perform the following operations, you will You will find:

 $ cat /etc/passwd | sort -t":" -n -k 4 > /tmp/passwd$ cat /etc/group | sort -t":" -n -k 3 > /tmp/group$ join -o 1.1,2.1 -t":" -1 4 -2 3 /tmp/passwd /tmp/grouphalt:rootoperator:rootroot:rootshutdown:rootsync:rootbin:bindaemon:daemonadm:admlp:lppop:popnobody:nogroupfalcon:usersgames:users

You can see that this result is correct, so use it in the future

 join

Be sure to pay attention to this issue, otherwise taking a more conservative approach seems to be better at ensuring correctness. For more discussion about file connections, please refer to the subsequent information.

The above involves processing the specified column in a formatted row, including interception (such as

 SQL

of

 select

usage), connection (e.g.

 SQL

of

 join

usage), sorting (e.g.

 SQL

of

 order by

Usage), you can split a formatted line by specifying a separator. In addition, there are many ways to "intercept", not only

 cut

,

 awk

, even through

 IFS

Specify the delimiter

 read

This can also be done with commands, for example:

 $ IFS=":"; cat /etc/group | while read C1 C2 C3 C4; do echo $C1 $C3; done

Therefore, by becoming familiar with these usages, our work will become very flexible and interesting.

At this point, you need to do a simple exercise, how to convert user names and user IDs corresponding to columns into rows, that is, data similar to the following:

 $ cat /etc/passwd | cut -d":" -f1,3 --output-delimiter=" "root 0bin 1daemon 2

Convert to:

 $ cat aroot bin daemon0 1 2

And convert it back, what can I do? Remember things like

 tr

,

 paste

,

 split

and other commands can be used.

Reference method:

Positive conversion: first intercept the user name column and save it to the file

 user

, and then intercept the user

 ID

Deposit

 ID

, and then use the two files

 paste -s

The commands are connected together, thus completing the forward conversion.

Inverse conversion: first use the result obtained by forward conversion to

 split -1

Split into two files, and then use the two split files

 tr

separator

 t

Replace with

 n

, only use

 paste

The command connects the two files together, thus completing the reverse conversion.

References

"Advanced Bash Script Programming Guide" operates strings and specifies the type of variables

"Shell Thirteen Questions"

 $(())

and

 $()

besides

 ${}

What's the difference?

Regular Expressions - User guide

Regular Expression Tutorial

Grep Tutorial

Sed Tutorial

awkTutorial

sedTutorial

An awk Primer

The origin of some strange UNIX command names

Hone your skills in building regular expression patterns

Basic 11: File classification, merging and splitting (sort, uniq, join, cut, paste, split)

Simplify data extraction using Linux text tools

SED Single Line Script Quick Reference (Unix Stream Editor)

postscript

This section was supposed to be completed last week, but I have been too busy these days. I have only written a "first draft" now, and will add specific examples when I have time. The examples in this section are probably the most interesting, so you should study a few interesting examples carefully.

It seems to be around 1 o'clock when I finished writing the above part, just now

 check

After checking some typos and grammar, I added another section, namely "Storage Structure of Strings", and it's almost done now.

 half past 2

Well, good night, friends.

On the 26th, add two sections "Substring Base Conversion" and "Substring Encoding Conversion" and a processing

 URL

Example of address.