File Globbing and Regular Expressions

Introduction

File globing and regular expressions are powerful tools needed by every system administrator.

Learning Objectives

When you have successfully completed this assignment you will be able to:

• Describe the use of common wild cards.

• Write a simple regular expression.

• Use sed to modify text in a file.

• Use tr to replace non-printing characters.

Preparation

• To make the results stand out alias grep to grep --color=auto.
• Create a temporary directory to work in mkdir ~/temp.
• Create a file to work with echo "Mark Twain" "Huckleberry Finn" "Tom Sawyer" > twain.txt.

File Globing (Wild Cards)

Wild cards can be used with common commands to match file names. While similar to regular expressions there are not the same. Regular expressions do not match file or directory names but strings.

Example 1

Match all files beginning with exam followed by any number of characters.

$ ls example*
exam example examples example12

Example 2

Match all files beginning with example followed by a single character.

$ ls example?
examples

Regular Expressions (regex)

Regular expressions are strings of special characters that describe a pattern for searching. The following table show some of the characters special to regular expressions.

Literal Strings

The simplest regular expression is a literal string. The following matches each line with Mark Twain but only if on the same line.

$ grep "Mark Twain" twain.txt

or if the quotes are left out both Mark and Twain separately.

$ grep Mark Twain twain.txt

Note the grep is case sensitive. Filtering on mark twain returns nothing unless -i option is included.

The -v option reverses the search. This will return all the lines that do not have Mark Twain.

$ grep -v "Mark Twain" twain.txt

Special (Metacharacters)

The Period

The period returns a single character. Here n preceded by one character.

$ grep ".n" twain.txt

Or two ns followed by a single character.

$ grep ".nn" twain.txt

The Asterisk

An asterisk denotes zero or more characters.

$ grep "n*" twain.txt

Search for String at Start of Line

Filter lines that start with Mark *Twain

$ grep ^"Mark Twain" twain.txt

Search for a String at the End of Line

Filter lines that end with Mark *Twain. This will fail if there is a space before the newline character.

$ grep ^"Mark Twain" twain.txt

$ cat -E twain.txt
Space at end  Mark Twain $

Brace Expansion

Here brace expansion is used to create files and bracket expansion is used to display them.

$ for i in {0,1,2,3,4,5,6,7,8,9}; do touch file$i; done
$ ls file[0-9]
file0 file1 file2 file3 file4 file5 file6 file7 file8 file9

Sequence

{1..10}

{a..bb}

echo {a..z}

$ touch file-{1..5}.txt

Comma Separated List

{ aa,bb,cc,dd }

$ echo {aaa,bbb,ccc,ddd}

Character Classes

Usage: grep [[:punct:]]

sed

sed is a stream editor for filtering and transforming text and supports regular expressions. Try these examples. By default sed outputs to STDOUT and does not alter the original file.

sed s/apple/orange/ regex_sample

sed s/^apple/orange/ regex_sample

tr

The tr command can replace non-printing characters.

This example will replace the new line character with a space putting all words on one line.

tr '

' ' ' < regex_sample

This example will replace all lower case letters with upper case letters.

tr '[:lower:]' '[:upper:]' < regex_sample

Debug the Assignment

Resources

• man regex
• man glob
• IBM DevelopersWorks
• Wikibooks: Regular Expressions
• Wikibooks: grep
• Wikibooks: sed
• Wikibooks: awk
• Awk by example +Introducing filters and regular expressions

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.