Lesson 3 of 7 · 8 min · Bash basics

Reading files, input and command-line options

Covers: Ubuntu 24.04 LTS, bash 5.2. Updated 13 Nov 2026. Part of the DevOps Engineer roadmap.

What you will learn

  • Reading a file or command output line by line without breaking on spaces
  • Splitting delimited fields with IFS and read
  • Here-documents and here-strings for multi-line text
  • Parsing options with getopts, and printing a proper usage message
  • Temporary files and directories

In simple words

Useful scripts take input: a file to read, an answer from a person, or settings typed after the script's name. This lesson shows how to read a file one line at a time without mistakes, and how to accept options such as -v, the way real commands do.

Words to know

  • Argument: a value typed after the script's name, such as the file to work on.
  • Option: a switch such as -v that changes how the script behaves.
  • Here-document: a way to put several lines of text inside a script and feed them to a command.
  • Temporary file: a file a script creates for its own short use and deletes when it finishes.

Looping over lines

The correct pattern to read a file line by line:

while IFS= read -r line; do
  echo "line: $line"
done < file.txt
  • read takes one line from stdin into the variable.
  • IFS= stops read from trimming leading and trailing spaces.
  • -r stops it from treating backslashes as escape characters.
  • < file.txt feeds the file to the whole loop.

To loop over a command's output, use process substitution so the loop runs in the current shell (a plain pipe would run it in a subshell and lose variable changes):

while IFS= read -r line; do echo "line: $line"; done < <(printf 'one\ntwo words\n')
line: one
line: two words

Splitting fields

Set IFS (the input field separator) for the read command only:

printf 'name,qty\nweb,2\ndb,1\n' | while IFS=, read -r n q; do echo "$n has $q"; done
name has qty
web has 2
db has 1

This is fine for simple CSV. Real CSV with quotes needs a proper tool (Python's csv, jq for JSON). Never parse JSON, YAML or HTML with grep and sed: use jq, yq or a script language.

Looping over files

for f in *.log; do echo "file: [$f] lines: $(wc -l < "$f")"; done
file: [a b.log] lines: 1
file: [c.log] lines: 1

The brackets in the output show that the name with a space stayed in one piece, because every use of $f is quoted. For files in subdirectories use find ... -print0 | while IFS= read -r -d '' f; do ...; done, or find ... -exec.

Here-documents and here-strings

A here-document gives a command several lines of text. Quote the delimiter ('EOF') to stop expansion inside:

cat <<EOF > config.txt
host=$(hostname)
EOF

cat <<'EOF' > script.sh
echo "$HOME is not expanded here"
EOF

Without quotes the variables and commands inside expand, as for host= above. <<-EOF strips leading tabs so you can indent. A here-string passes one string as input: grep -c a <<< "banana".

Here-documents are the standard way to write config files or send multi-line input to commands (ssh host bash -s <<'EOF', mysql <<'EOF').

Asking for input

read -r -p "Delete everything? [y/N] " answer
[[ $answer == [yY] ]] || { echo "aborted"; exit 1; }
read -r -s -p "Password: " pw; echo        # -s hides what is typed

In automation nobody is typing. A good script accepts values as arguments or environment variables and only asks if running in a terminal ([[ -t 0 ]] tests whether stdin is a terminal).

Options with getopts

Real scripts accept options like -v or -n NAME. getopts parses them:

usage() { echo "usage: $0 [-v] [-n NAME] file" >&2; exit 2; }

verbose=0 name=""
while getopts "vn:h" opt; do
  case $opt in
    v) verbose=1 ;;
    n) name=$OPTARG ;;
    h) usage ;;
    *) usage ;;
  esac
done
shift $((OPTIND - 1))          # remove the parsed options; "$@" is now the remaining arguments
[[ $# -eq 1 ]] || usage
file=$1
  • In "vn:h", a letter followed by : takes a value (-n NAME), available in $OPTARG.
  • shift $((OPTIND-1)) leaves the non-option arguments.
  • getopts handles short options only. For --long options, use a case loop over "$@" or a Python script.
  • Exit code 2 for wrong usage and 1 for runtime failures is a common convention. Print usage and errors to stderr.

With no option, the variables keep their defaults (verbose=0 name=none in our test).

Temporary files

Never use fixed names like /tmp/data.txt: two runs collide, and another user could plant a symlink there. Use mktemp and clean up with a trap:

tmp=$(mktemp -d)                     # a new private directory
trap 'rm -rf "$tmp"' EXIT            # removed when the script ends, however it ends
echo "working in $tmp"

mktemp creates the path with unpredictable name and mode 600/700. The trap command is explained in the next lesson.

Handling output and errors of commands

output=$(some_command 2>&1) || { echo "failed: $output" >&2; exit 1; }   # capture both, handle failure
if ! command -v jq >/dev/null; then echo "jq is required" >&2; exit 1; fi  # check a tool exists
mapfile -t lines < <(grep ERROR app.log)                                    # read lines into an array

command -v name is the portable way to check that a program is installed. mapfile (also readarray) reads lines into an array.

Common mistakes

  • for line in $(cat file) splits on every space. Use while IFS= read -r.
  • Piping into while and expecting variables to persist: the loop ran in a subshell. Use < <(...) or a redirect.
  • Forgetting -r and losing backslashes.
  • Fixed temp file names.
  • Interactive prompts in scripts that run in CI or cron.
  • Parsing structured data (JSON) with regular expressions.

Try it yourself

  1. Read /etc/passwd and print the username and shell of every account whose shell ends in bash.
  2. Write a script with options -s SERVER (required) and -q (quiet) and a helpful usage message; test missing and unknown options.
  3. Generate an nginx config snippet with a here-document using a variable for the port.
  4. Make a script that creates a temp directory, writes to it and always removes it, even when you press Ctrl+C halfway.

Check yourself

Why does for line in $(cat file) fail for lines with spaces?

$(cat file) is unquoted, so the shell splits the output at every space, tab and newline, giving words, not lines. while IFS= read -r line; do ...; done < file keeps whole lines.

What does IFS=, read -r a b do?

It splits one line from stdin at commas into a and b (the rest into b), setting IFS only for that command.

What is the difference between <<EOF and <<'EOF'?

With an unquoted delimiter, variables and command substitutions inside the here-document are expanded. With a quoted delimiter the text is passed exactly as written.

Why should a script use mktemp?

It creates a uniquely named, private file or directory, avoiding collisions between runs and predictable-name attacks.

Interview questions

How do you parse command-line options in Bash?

getopts for short options, with a case statement and shift $((OPTIND-1)) afterwards; a manual while/case over "$@" for long options.

How do you read a file line by line safely?

while IFS= read -r line; do ...; done < file.

How can a script check whether a required tool is installed?

command -v tool >/dev/null 2>&1 || { echo "tool is required" >&2; exit 1; }.

Summary

  • Read lines with while IFS= read -r; use < <(cmd) to avoid a subshell; split fields with IFS=,.
  • Here-documents write multi-line text; quote the delimiter to prevent expansion.
  • getopts parses short options; print usage to stderr and exit 2 on misuse.
  • mktemp plus trap ... EXIT gives safe, self-cleaning temporary files.

Going deeper

See how much of this you can explain

A free mock interview asks you new questions on Shell scripting with Bash for DevOps Engineer and scores every answer, with what to fix.