| Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting | ||
|---|---|---|
| Prev | Chapter 12. External Filters, Programs and Commands | Next |
Commands for more advanced users
-exec COMMAND \;
Carries out COMMAND on each file that find matches. The command sequence terminates with \; (the ";" is escaped to make certain the shell passes it to find literally).
bash$ find ~/ -name '*.txt' /home/bozo/.kde/share/apps/karm/karmdata.txt /home/bozo/misc/irmeyc.txt /home/bozo/test-scripts/1.txt |
If COMMAND contains {}, then find substitutes the full path name of the selected file for "{}".
1 find ~/ -name 'core*' -exec rm {} \;
2 # Removes all core dump files from user's home directory. |
1 find /home/bozo/projects -mtime 1
2 # Lists all files in /home/bozo/projects directory tree
3 #+ that were modified within the last day.
4 #
5 # mtime = last modification time of the target file
6 # ctime = last status change time (via 'chmod' or otherwise)
7 # atime = last access time
8
9 DIR=/home/bozo/junk_files
10 find "$DIR" -type f -atime +5 -exec rm {} \;
11 # ^^
12 # Curly brackets are placeholder for the path name output by "find."
13 #
14 # Deletes all files in "/home/bozo/junk_files"
15 #+ that have not been accessed in at least 5 days.
16 #
17 # "-type filetype", where
18 # f = regular file
19 # d = directory, etc.
20 # (The 'find' manpage has a complete listing.) |
1 find /etc -exec grep '[0-9][0-9]*[.][0-9][0-9]*[.][0-9][0-9]*[.][0-9][0-9]*' {} \;
2
3 # Finds all IP addresses (xxx.xxx.xxx.xxx) in /etc directory files.
4 # There a few extraneous hits. How can they be filtered out?
5
6 # Perhaps by:
7
8 find /etc -type f -exec cat '{}' \; | tr -c '.[:digit:]' '\n' \
9 | grep '^[^.][^.]*\.[^.][^.]*\.[^.][^.]*\.[^.][^.]*$'
10 #
11 # [:digit:] is one of the character classes
12 #+ introduced with the POSIX 1003.2 standard.
13
14 # Thanks, Stephané Chazelas. |
![]() | The -exec option to find should not be confused with the exec shell builtin. |
Example 12-3. Badname, eliminate file names in current directory containing bad characters and whitespace.
1 #!/bin/bash
2 # badname.sh
3 # Delete filenames in current directory containing bad characters.
4
5 for filename in *
6 do
7 badname=`echo "$filename" | sed -n /[\+\{\;\"\\\=\?~\(\)\<\>\&\*\|\$]/p`
8 # badname=`echo "$filename" | sed -n '/[+{;"\=?~()<>&*|$]/p'` also works.
9 # Deletes files containing these nasties: + { ; " \ = ? ~ ( ) < > & * | $
10 #
11 rm $badname 2>/dev/null
12 # ^^^^^^^^^^^ Error messages deep-sixed.
13 done
14
15 # Now, take care of files containing all manner of whitespace.
16 find . -name "* *" -exec rm -f {} \;
17 # The path name of the file that "find" finds replaces the "{}".
18 # The '\' ensures that the ';' is interpreted literally, as end of command.
19
20 exit 0
21
22 #---------------------------------------------------------------------
23 # Commands below this line will not execute because of "exit" command.
24
25 # An alternative to the above script:
26 find . -name '*[+{;"\\=?~()<>&*|$ ]*' -exec rm -f '{}' \;
27 # (Thanks, S.C.) |
Example 12-4. Deleting a file by its inode number
1 #!/bin/bash
2 # idelete.sh: Deleting a file by its inode number.
3
4 # This is useful when a filename starts with an illegal character,
5 #+ such as ? or -.
6
7 ARGCOUNT=1 # Filename arg must be passed to script.
8 E_WRONGARGS=70
9 E_FILE_NOT_EXIST=71
10 E_CHANGED_MIND=72
11
12 if [ $# -ne "$ARGCOUNT" ]
13 then
14 echo "Usage: `basename $0` filename"
15 exit $E_WRONGARGS
16 fi
17
18 if [ ! -e "$1" ]
19 then
20 echo "File \""$1"\" does not exist."
21 exit $E_FILE_NOT_EXIST
22 fi
23
24 inum=`ls -i | grep "$1" | awk '{print $1}'`
25 # inum = inode (index node) number of file
26 # ----------------------------------------------------------------------
27 # Every file has an inode, a record that hold its physical address info.
28 # ----------------------------------------------------------------------
29
30 echo; echo -n "Are you absolutely sure you want to delete \"$1\" (y/n)? "
31 # The '-v' option to 'rm' also asks this.
32 read answer
33 case "$answer" in
34 [nN]) echo "Changed your mind, huh?"
35 exit $E_CHANGED_MIND
36 ;;
37 *) echo "Deleting file \"$1\".";;
38 esac
39
40 find . -inum $inum -exec rm {} \;
41 # ^^
42 # Curly brackets are placeholder
43 #+ for text output by "find."
44 echo "File "\"$1"\" deleted!"
45
46 exit 0 |
See Example 12-27, Example 3-4, and Example 10-9 for scripts using find. Its manpage provides more detail on this complex and powerful command.
A filter for feeding arguments to a command, and also a tool for assembling the commands themselves. It breaks a data stream into small enough chunks for filters and commands to process. Consider it as a powerful replacement for backquotes. In situations where command substitution fails with a too many arguments error, substituting xargs often works. Normally, xargs reads from stdin or from a pipe, but it can also be given the output of a file.
The default command for xargs is echo. This means that input piped to xargs may have linefeeds and other whitespace characters stripped out.
bash$ ls -l total 0 -rw-rw-r-- 1 bozo bozo 0 Jan 29 23:58 file1 -rw-rw-r-- 1 bozo bozo 0 Jan 29 23:58 file2 bash$ ls -l | xargs total 0 -rw-rw-r-- 1 bozo bozo 0 Jan 29 23:58 file1 -rw-rw-r-- 1 bozo bozo 0 Jan 29 23:58 file2 |
ls | xargs -p -l gzip gzips every file in current directory, one at a time, prompting before each operation.
![]() | An interesting xargs option is -n NN, which limits to NN the number of arguments passed. ls | xargs -n 8 echo lists the files in the current directory in 8 columns. |
![]() | Another useful option is -0, in combination with find -print0 or grep -lZ. This allows handling arguments containing whitespace or quotes. find / -type f -print0 | xargs -0 grep -liwZ GUI | xargs -0 rm -f grep -rliwZ GUI / | xargs -0 rm -f Either of the above will remove any file containing "GUI". (Thanks, S.C.) |
Example 12-5. Logfile: Using xargs to monitor system log
1 #!/bin/bash 2 3 # Generates a log file in current directory 4 # from the tail end of /var/log/messages. 5 6 # Note: /var/log/messages must be world readable 7 # if this script invoked by an ordinary user. 8 # #root chmod 644 /var/log/messages 9 10 LINES=5 11 12 ( date; uname -a ) >>logfile 13 # Time and machine name 14 echo --------------------------------------------------------------------- >>logfile 15 tail -$LINES /var/log/messages | xargs | fmt -s >>logfile 16 echo >>logfile 17 echo >>logfile 18 19 exit 0 20 21 # Note: 22 # ---- 23 # As Frank Wang points out, 24 #+ unmatched quotes (either single or double quotes) in the source file 25 #+ may give xargs indigestion. 26 # 27 # He suggests the following substitution for line 15: 28 # tail -$LINES /var/log/messages | tr -d "\"'" | xargs | fmt -s >>logfile 29 30 31 32 # Exercise: 33 # -------- 34 # Modify this script to track changes in /var/log/messages at intervals 35 #+ of 20 minutes. 36 # Hint: Use the "watch" command. |
As in find, a curly bracket pair serves as a placeholder for replacement text.
Example 12-6. Copying files in current directory to another
1 #!/bin/bash
2 # copydir.sh
3
4 # Copy (verbose) all files in current directory ($PWD)
5 #+ to directory specified on command line.
6
7 E_NOARGS=65
8
9 if [ -z "$1" ] # Exit if no argument given.
10 then
11 echo "Usage: `basename $0` directory-to-copy-to"
12 exit $E_NOARGS
13 fi
14
15 ls . | xargs -i -t cp ./{} $1
16 # ^^ ^^ ^^
17 # -t is "verbose" (output command line to stderr) option.
18 # -i is "replace strings" option.
19 # {} is a placeholder for output text.
20 # This is similar to the use of a curly bracket pair in "find."
21 #
22 # List the files in current directory (ls .),
23 #+ pass the output of "ls" as arguments to "xargs" (-i -t options),
24 #+ then copy (cp) these arguments ({}) to new directory ($1).
25 #
26 # The net result is the exact equivalent of
27 #+ cp * $1
28 #+ unless any of the filenames has embedded "whitespace" characters.
29
30 exit 0 |
Example 12-7. Killing processes by name
1 #!/bin/bash
2 # kill-byname.sh: Killing processes by name.
3 # Compare this script with kill-process.sh.
4
5 # For instance,
6 #+ try "./kill-byname.sh xterm" --
7 #+ and watch all the xterms on your desktop disappear.
8
9 # Warning:
10 # -------
11 # This is a fairly dangerous script.
12 # Running it carelessly (especially as root)
13 #+ can cause data loss and other undesirable effects.
14
15 E_BADARGS=66
16
17 if test -z "$1" # No command line arg supplied?
18 then
19 echo "Usage: `basename $0` Process(es)_to_kill"
20 exit $E_BADARGS
21 fi
22
23
24 PROCESS_NAME="$1"
25 ps ax | grep "$PROCESS_NAME" | awk '{print $1}' | xargs -i kill {} 2&>/dev/null
26 # ^^ ^^
27
28 # -----------------------------------------------------------
29 # Notes:
30 # -i is the "replace strings" option to xargs.
31 # The curly brackets are the placeholder for the replacement.
32 # 2&>/dev/null suppresses unwanted error messages.
33 # -----------------------------------------------------------
34
35 exit $? |
Example 12-8. Word frequency analysis using xargs
1 #!/bin/bash 2 # wf2.sh: Crude word frequency analysis on a text file. 3 4 # Uses 'xargs' to decompose lines of text into single words. 5 # Compare this example to the "wf.sh" script later on. 6 7 8 # Check for input file on command line. 9 ARGS=1 10 E_BADARGS=65 11 E_NOFILE=66 12 13 if [ $# -ne "$ARGS" ] 14 # Correct number of arguments passed to script? 15 then 16 echo "Usage: `basename $0` filename" 17 exit $E_BADARGS 18 fi 19 20 if [ ! -f "$1" ] # Check if file exists. 21 then 22 echo "File \"$1\" does not exist." 23 exit $E_NOFILE 24 fi 25 26 27 28 ######################################################## 29 cat "$1" | xargs -n1 | \ 30 # List the file, one word per line. 31 tr A-Z a-z | \ 32 # Shift characters to lowercase. 33 sed -e 's/\.//g' -e 's/\,//g' -e 's/ /\ 34 /g' | \ 35 # Filter out periods and commas, and 36 #+ change space between words to linefeed, 37 sort | uniq -c | sort -nr 38 # Finally prefix occurrence count and sort numerically. 39 ######################################################## 40 41 # This does the same job as the "wf.sh" example, 42 #+ but a bit more ponderously, and it runs more slowly (why?). 43 44 exit 0 |
All-purpose expression evaluator: Concatenates and evaluates the arguments according to the operation given (arguments must be separated by spaces). Operations may be arithmetic, comparison, string, or logical.
returns 8
returns 2
returns 15
The multiplication operator must be escaped when used in an arithmetic expression with expr.
Increment a variable, with the same effect as let y=y+1 and y=$(($y+1)). This is an example of arithmetic expansion.
Extract substring of $length characters, starting at $position.
Example 12-9. Using expr
1 #!/bin/bash 2 3 # Demonstrating some of the uses of 'expr' 4 # ======================================= 5 6 echo 7 8 # Arithmetic Operators 9 # ---------- --------- 10 11 echo "Arithmetic Operators" 12 echo 13 a=`expr 5 + 3` 14 echo "5 + 3 = $a" 15 16 a=`expr $a + 1` 17 echo 18 echo "a + 1 = $a" 19 echo "(incrementing a variable)" 20 21 a=`expr 5 % 3` 22 # modulo 23 echo 24 echo "5 mod 3 = $a" 25 26 echo 27 echo 28 29 # Logical Operators 30 # ------- --------- 31 32 # Returns 1 if true, 0 if false, 33 #+ opposite of normal Bash convention. 34 35 echo "Logical Operators" 36 echo 37 38 x=24 39 y=25 40 b=`expr $x = $y` # Test equality. 41 echo "b = $b" # 0 ( $x -ne $y ) 42 echo 43 44 a=3 45 b=`expr $a \> 10` 46 echo 'b=`expr $a \> 10`, therefore...' 47 echo "If a > 10, b = 0 (false)" 48 echo "b = $b" # 0 ( 3 ! -gt 10 ) 49 echo 50 51 b=`expr $a \< 10` 52 echo "If a < 10, b = 1 (true)" 53 echo "b = $b" # 1 ( 3 -lt 10 ) 54 echo 55 # Note escaping of operators. 56 57 b=`expr $a \<= 3` 58 echo "If a <= 3, b = 1 (true)" 59 echo "b = $b" # 1 ( 3 -le 3 ) 60 # There is also a "\>=" operator (greater than or equal to). 61 62 63 echo 64 echo 65 66 67 68 # String Operators 69 # ------ --------- 70 71 echo "String Operators" 72 echo 73 74 a=1234zipper43231 75 echo "The string being operated upon is \"$a\"." 76 77 # length: length of string 78 b=`expr length $a` 79 echo "Length of \"$a\" is $b." 80 81 # index: position of first character in substring 82 # that matches a character in string 83 b=`expr index $a 23` 84 echo "Numerical position of first \"2\" in \"$a\" is \"$b\"." 85 86 # substr: extract substring, starting position & length specified 87 b=`expr substr $a 2 6` 88 echo "Substring of \"$a\", starting at position 2,\ 89 and 6 chars long is \"$b\"." 90 91 92 # The default behavior of the 'match' operations is to 93 #+ search for the specified match at the ***beginning*** of the string. 94 # 95 # uses Regular Expressions 96 b=`expr match "$a" '[0-9]*'` # Numerical count. 97 echo Number of digits at the beginning of \"$a\" is $b. 98 b=`expr match "$a" '\([0-9]*\)'` # Note that escaped parentheses 99 # == == + trigger substring match. 100 echo "The digits at the beginning of \"$a\" are \"$b\"." 101 102 echo 103 104 exit 0 |
![]() | The : operator can substitute for match. For example, b=`expr $a : [0-9]*` is the exact equivalent of b=`expr match $a [0-9]*` in the above listing.
|
This example illustrates how expr uses the escaped parentheses -- \( ... \) -- grouping operator in tandem with regular expression parsing to match a substring.
Perl, sed, and awk have far superior string parsing facilities. A short sed or awk "subroutine" within a script (see Section 34.2) is an attractive alternative to expr.
See Section 9.2 for more on using expr in string operations.