Combining Find And Shell Commands With -Exec
Locating Files for Batch Operations
The find command in Linux provides a powerful way to locate files and directories based on flexible criteria. With over 50 different tests, find can match files by name, size, modification time, permissions, ownership, and many other attributes.
To use find effectively, understanding how to build complex criteria with logical operators is key. Tests can be combined to zero in on specific files for various administrative and automation tasks.
The find Command for Locating Files Based on Criteria
At its simplest, find takes a starting directory and outputs any matching files and folders under that path:
# Find all files under /var/log find /var/log
By adding tests, the results can be narrowed down based on specific attributes:
# Find files over 1MB in size find /home -size +1M
Dozens of useful tests like -name, -mtime, -type, and -user exist to matching based on file names, timestamps, types, owners, and more. Tests can be combined to select files based on multiple specific criteria.
Matching Files with Tests Like -name, -size, -mtime
The -name test matches based on file name:
# Find files named access.log find /var/log -name access.log
-size checks against the size of files:
# Find files over 100MB in size find /backup -size +100M
While -mtime matches based on the last modification time in days:
# Find files modified over 180 days ago find /var/log -mtime +180
Some other useful tests and examples include:
-
-type f
# Find regular files
-
-perm 755
# Match based on permissions
-
-user john
# Match files owned by john
-
-group admins
# Match files owned by admins group
Logical Operators to Build Complex Criteria
By combining tests with logical operators, you can craft precise criteria to pinpoint files based on multiple attributes. The main logical operators are:
-
-and
: Match only if both conditions are met
-
-or
: Match if either condition is met
-
-not
: Negates or reverses the result of a test
Some examples of logical operators:
# Files over 1GB owned by john find /home -size +1G -and -user john # Files over 2 days old OR named tmp find /var/tmp -mtime +2 -or -name tmp # Files that don't start with temp find /var/tmp -not -name "temp*"
Grouping long sets of tests is also possible by escaping parentheses:
# Long criteria grouped
find /var/log \( -name "*.log" -and -mtime +180 \) \
-or \( -size +500M -and -exec gzip {} \; \)
Performing Actions on Matching Files
Locating files with find is just half of the usefulness. The real power comes from being able to perform operations and actions on the matches.
This is accomplished with the -exec option, which runs any shell command on the results by substituting {} for the file name.
The -exec option to perform commands on results
The basic syntax for -exec is:
find starting_path tests -exec command_on_matches {} \;
For example, to compress *.log files over 10 days old under /var/log:
find /var/log -name *.log -mtime +10 -exec gzip {} \;
The {} will be replaced by the name of each matching file to compress it with gzip.
Passing {} as the matching file name
The {} syntax lets you easily operate on the files find outputs without manual intervention:
# Delete old temp files
find /tmp -name "tmp*" -mtime +10 -exec rm {} \;
# Change permissions on media files
find /home -type f -exec chmod 644 {} \;
It works the same for any number of matches – the command runs individually for every resulting file name.
Terminating commands properly with \;
The semicolon \; is important for terminating the -exec properly for each run:
# Good, will run rm properly per file
find . -type f -name "*" -exec rm {} \;
# Bad, tries to delete * literally
find . -type f -name "*" -exec rm {}
Without the \; files will not substitute right and operations can fail or be destructive.
Use Cases and Examples
Combining find with -exec opens many possibilities for practical use cases. Any repetitive task based on file attributes is a candidate for automation with find.
Archiving Old Log Files
Log rotation is essential for limiting disk consumption. Finding and compressing old logs is easy:
# Archive logs over 180 days
find /var/log -type f -mtime +180 -exec gzip {} \;
# Delete archived logs over 1 year old
find /var/log -name "*.gz" -mtime +365 -exec rm {} \;
Finding and Deleting Temporary Files
Clearing out old temporary files helps reclaim wasted disk space:
# Remove files not modified in over 90 days
find /tmp -type f -mtime +90 -exec rm {} \;
Changing Permissions on Directories
Updating permissions recursively is simple with find:
# Add group write to trees
find /home -type d -exec chmod g+w {} \;
# Remove world write
find /home -type d -exec chmod o-w {} \;
Other Common Automation Tasks
Many other administrative tasks are also good candidates for find automation:
- Searching logs for failures or errors
- Updating access times on inactive files
- Moving old files to other partitions
- Removing outdated backups
Anything needing recursive scaling across files can benefit from find with -exec.
Improving Efficiency and Readability
When chaining together many find commands with -exec, there are some best practices to make things cleaner and more efficient.
Grouping Long Lists of Criteria
Finding files often requires long lists of criteria. Grouping them with escaping makes things readable:
find /var/log \( -name "*.log" -and -mtime +180 \) \
-or \( -size +500M -and -exec gzip {} \; \)
Splitting Long -exec Lines
Similarly, splitting -exec across lines keeps things clean:
find /home -type f -mtime +180
-exec gzip {} \;
-exec mv {} /archive \;
Proper Quoting and Escapes
When executing commands, proper escaping and quoting ensures operate as intended:
find . -type f -exec sh -c 'rm "{}" ;' \;
This quotes the {} to allow for filenames with spaces.
Putting It in a Script
For reusable batch operations, put commands in a script:
#!/bin/bash
# Find files
find "/home" -name "*.tmp" -mtime +30
# Take action
-exec rm {} \;
This way you can repeatedly automate tasks consistently.
