Reading Files

Scripts often need to read data from files. Learn the best techniques for reading files line by line, in bulk, or field by field.

Reading Entire File

#!/bin/bash

# Method 1: cat command
CONTENT=$(cat file.txt)
echo "$CONTENT"

# Method 2: Redirect
CONTENT=$(< file.txt)
echo "$CONTENT"

# Method 3: Read into variable
CONTENT=$(<file.txt)

Reading Line by Line

The most common pattern:

#!/bin/bash
while IFS= read -r line; do
    echo "Line: $line"
done < file.txt

Key components:

• IFS= prevents trimming leading/trailing whitespace
• -r prevents backslash interpretation
• < file.txt redirects file to the loop

Exercise: Read File Lines

Read and process file contents:

Processing Specific Fields

#!/bin/bash
# CSV file: name,age,city
while IFS=',' read -r name age city; do
    echo "Name: $name, Age: $age, City: $city"
done < data.csv

# Colon-separated (like /etc/passwd)
while IFS=':' read -r user _ uid gid _ home shell; do
    echo "User: $user, UID: $uid, Home: $home"
done < /etc/passwd

Reading into Array

#!/bin/bash

# Read all lines into array
readarray -t LINES < file.txt

# Or using mapfile (same as readarray)
mapfile -t LINES < file.txt

echo "Total lines: ${#LINES[@]}"
echo "First line: ${LINES[0]}"
echo "Last line: ${LINES[-1]}"

Reading Specific Lines

#!/bin/bash

# First N lines
head -n 5 file.txt

# Last N lines
tail -n 5 file.txt

# Specific line number
sed -n '5p' file.txt

# Range of lines
sed -n '5,10p' file.txt

# In script with arrays
readarray -t LINES < file.txt
echo "${LINES[4]}"    # Fifth line (0-indexed)

Handling Empty Lines

#!/bin/bash
while IFS= read -r line || [[ -n "$line" ]]; do
    # Skip empty lines
    [[ -z "$line" ]] && continue

    echo "Processing: $line"
done < file.txt

Note: || [[ -n "$line" ]] handles files without final newline.

Exercise: Count Lines

Count lines in a file:

Reading from Command Output

#!/bin/bash

# Process command output line by line
while IFS= read -r file; do
    echo "Found: $file"
done < <(find . -name "*.txt")

# Or with pipe (creates subshell - variables don't persist)
find . -name "*.txt" | while IFS= read -r file; do
    echo "Found: $file"
done

Safe File Reading

#!/bin/bash

FILE="$1"

# Check file exists
if [[ ! -f "$FILE" ]]; then
    echo "Error: File not found: $FILE" >&2
    exit 1
fi

# Check file is readable
if [[ ! -r "$FILE" ]]; then
    echo "Error: File not readable: $FILE" >&2
    exit 1
fi

# Now safe to read
while IFS= read -r line; do
    echo "$line"
done < "$FILE"

Reading Binary Data

For binary files, use different approaches:

#!/bin/bash
# Read bytes as hex
xxd file.bin | head

# Read with dd
dd if=file.bin bs=1 count=10 2>/dev/null | xxd

# Check file type
file some_file

Reading with Timeout

#!/bin/bash
# Read with 5-second timeout
if read -t 5 -r line < file.txt; then
    echo "Got: $line"
else
    echo "Timeout or empty"
fi

Key Takeaways

• Use while IFS= read -r line for line-by-line reading
• IFS= preserves whitespace, -r preserves backslashes
• Use readarray -t LINES to read into an array
• Check file existence and readability before reading
• Use < <(command) instead of pipes to preserve variables
• || [[ -n "$line" ]] handles files without trailing newline
• Set custom IFS to parse delimited data