How to find files or file contents in the shell
Some of the most useful commands when working in bash are find and grep. These commands or a combinations of them let you find files or file contents in just about any way imaginable. There are other tools that can also do this, but as I have worked with some frequency on small embedded systems these can be the best ones available. Even on systems where you have alternatives, knowning these can still be effective and useful since they are available just about everywhere.
find:
General format -
Find takes glob patterns by default if you do not specify otherwise.
find /location modifiers+options "pattern" actions Specific example -
This would find any file larger than 10M with a name matching the pattern passed as the search.
find / -type f -size 10M -name "search" -printf Looking for files or directories
You can search case sensitive (name) or insensitive (iname). The name options take glob patterns. For regex, keep reading. type f specifies to search file names, type d directories.
find / -type f -name "search" # search files
find / -type d -iname "search" # search directoriesUsing regex with find
The -regex/-iregex flags allow you to specify a regex to search for.
The -regextype flag allows you to choose regular expression format. I often use the posix-egrep so that my find and grep regex syntax is the same and I don’t have to try and figure out what the exact syntax is.
# You can choose the type of regex
-regextype posix-egrep
# and then pass the regular expression to this flag
-regex # case sensitive regex
-iregex # case insensitive regex.Same search with name and regex -
sudo find / -type f -name fstab
sudo find / -type f -regex ".*/fstab" Specifying the regextype -
find . -regextype posix-egrep -maxdepth 1 -regex '.*[12][890][0-9][0-9].*\.txt' Other Useful options for find
name/iname - name of file with glob pattern.
type - can specify file, directory, link, pipe, socket
maxdepth/-mindepth 1 - Allows depth of recursiveness to be specified
not - invert the match
Actions -
These are things that can be done with the results of the search.
printf/0/ - printing options for the output
delete - get rid of what you searched for. I think of this as turning find into an advanced rm command because it can find subsets of files/directories with mush greater ease.
Exclude directories with find -
Not posix compliant, but easy to understand. Otherwise you have to use the prune syntax.
find ./ -type f -not -path "*node_modules*" -not -path "*coverage*" -iname "*.js" This could also be done with the prune method, which requires each new command prefaced with -o, -prune after each find option to exclude, and -print at the end to actually print the result.
find . -type f -regex "*node_modules*" -prune -o -regex "*coverage*" -prune -o -iname "*.js" -printShow results to the specified depth with full path:
The -maxdepth option prevents searching below that level. Find is recursive by default so I use this anytime I don’t want it searching every possible location.
find ./ -maxdepth 3 -type d Getting result paths without ./ -
If you want a path from the current directory without the ./ at the start, use * as the location:
find * -type f -iname "*.txt" Delete files with find
Its like rm but with the precision of find.
find /home/jadesrochers/Dropbox/ -type f -iname "*(sed)*" -delete
find . -type f -regextype posix-extended -regex "^[^-]+\.edf" -delete Deleting by piping to xargs/rm -
You can also pass the found files along to rm, which can be beneficial if you use xargs and there are lots of them.
find . -type f -regex ".*\.temp.*" -print0 | xargs -0 rm Copy files with find, xargs, and cp
find ~/ -type f -iname "plunderboo" -print0 | xargs -0 -I % cp % ~/backups Pass find results to grep using xargs:
find sources.list.d/ -type f -print0 | xargs -0 grep -i regex Pruning
Allows excluding patterns, though as I pointed out before you can also use -not if compatiblity is not the goal.
find . -type d -regex ".*\.svn.*" -prune -o -type f -iname "log" -printThe find arguments (-type d -regex “..svn.”) that precede the prune determine what gets ignored. The -type f -iname afterward is what will get printed.
Pruning and printing -
Prune by default still prints the pruned content. To print only the remaining content, use another statement with -print
afterward.
find . -type f -iname "*svn*" -prune -o -type f -iname "*xml" -print Prune paths -
The leading wildcard means the directory is avoided at any level.
find . -path "*node_modules" -prune -o -type f -iname "*js" -print Find and cp loop example
In this case I needed to do an operation with some logic, so I threw a loop in there because I need to be able to manipulate the name to do the copy operation.
for j in $(find . -type f -iname "*edf" | sort); do cp "$j" "sg${SG}.edf";
SG=$((SG+1)); done Change permissions en mass with find/exec
Find lends itself to mass operations on files.
sudo find /home/jadesrochers/bin/python/personalwebsite -name *.py -type f -exec chmod 764 {} + grep -
grep is for finding patterns within files using regular expressions and printing out what was found/where, depending on the options used. You can extract the pattern, surrounding content, or just get the file names.
What it does well
Find patterns in files and print that information out in a variety of formats.
What it is not as good at
Specifying which files/directories to search. You can use —include/—exclude to do this, but they are limited to glob patterns so a quick find piped to grep is often faster.
General format -
Grep takes regexes by default, unlike find which does globs by default.
Grep options pattern file/path
Very common options -
-E - specify extended regular expressions. I almost always leave this on because the extended regex is easier to use.
-i - Case insensitive search.
Recursive searching -
-r - search all files recursively, but do not follow symlinks.
-R - search recursively, including following links.
Printing options -
-l - only print out the file name
-A,B,C - These options specify a number of lines to be printed
before, after, or around the line with the text matched
-n - show line numbers in the results.
Specifying what to search -
While the file selection methods are limited, they also work well enough for many cases.
Search file or files -
You can use one or more globs to specify files.
grep -i "isnil" *js
grep -i "test" *js *json Or you can use dir paths with globs
grep -i "test" datatodb/*js datatodb/*json Or just specify a single file
grep -i -n "pipeAsync" filehelpers.js Including or excluding only certain types of files -
Includes and excludes are glob pattern only, and only work with Recusive (-r or -R) searches.
grep -R --include "*.cpp" "regex" ./
grep -r --include "*log" "regex" ./ Directory exclusion -
Also glob only. The {} braces are used for expansion.
sudo grep -iRE -C 2 -n --exclude-dir={node*,coverage} "regex" ./ These work, but if youy are more familiar with find already for locating specific files/directories then just use that and send the output to grep with exec or xargs.
Use grep in combination with find and exec/xargs -
When the basic grep options are not enough for selecting which files to search, just use a find command along with xargs or exec to search the files passed by find.
How to do it with find/exec
find . -type f -regex .*\.log -exec grep -E -o "regex" {} \;
# Exec, but allow grep to also get the file names.
find . -type f -exec grep -l 'ansible' {} + Using find/grep with xargs -
The main advantage is that xargs handles a lot of ouput better than exec since it passes many results to a single instance of the grep process. You can also access the file names without any special syntax.
find . -type f -print0 | xargs -0 grep -no "ansible"
# A detailed find with xargs to search those files with grep
find ./src -maxdepth 3 -type f -regextype "posix-extended" -iregex ".*js" -print0 | xargs -0 grep -E -i -C 2 "async"
# another example
find . -type f -iname "*md" -print0 | xargs -0 grep -in 'awk'find and grep can locate just about anything you need to
As a team find and grep can find just about anything in any set of files you to find them in. I do find the possible range of syntax impossible to remember, but you probably will only use a few options regularly, and if other ones are needed check the —help to look for what you might need. Many tutorials go through tons of options, but over time I found that it was not the range of options but the ability to use a few options well that was most useful.
Use each for its role
I often use find and grep together even for simple operations so that I only have to remember find syntax for finding the files and grep for content searching.