String Substitution

String substitution transforms strings using various bash techniques. Learn to modify text efficiently using parameter expansion, tr, and sed.

Parameter Expansion Substitution

Replace patterns within variables:

#!/bin/bash
TEXT="The quick brown fox"

# Replace first match
echo "${TEXT/quick/slow}"      # The slow brown fox

# Replace all matches
TEXT2="foo bar foo bar"
echo "${TEXT2//foo/baz}"       # baz bar baz bar

# Replace at beginning
echo "${TEXT/#The/A}"          # A quick brown fox

# Replace at end
echo "${TEXT/%fox/dog}"        # The quick brown dog

Delete by Substitution

Replace with empty string:

#!/bin/bash
TEXT="Hello World"

# Remove first occurrence
echo "${TEXT/o/}"         # Hell World

# Remove all occurrences
echo "${TEXT//o/}"        # Hell Wrld

# Remove spaces
SPACED="  lots  of  spaces  "
echo "${SPACED// /}"      # lotsofspaces

Exercise: Remove Characters

Remove unwanted characters:

The tr Command

Translate or delete characters:

#!/bin/bash

# Translate characters
echo "hello" | tr 'a-z' 'A-Z'     # HELLO
echo "HELLO" | tr 'A-Z' 'a-z'     # hello

# Delete characters
echo "hello123" | tr -d '0-9'     # hello
echo "hello world" | tr -d ' '    # helloworld

# Squeeze repeated characters
echo "hellooo" | tr -s 'o'        # hello

# Replace characters
echo "hello" | tr 'elo' '310'     # h3110

tr Character Classes

# Using predefined classes
echo "Hello123" | tr -d '[:digit:]'   # Hello
echo "Hello123" | tr -d '[:alpha:]'   # 123

# Complement (delete everything except)
echo "Hello123" | tr -cd '[:digit:]'  # 123
echo "Hello123!" | tr -cd '[:alnum:]' # Hello123

Class	Characters
[:alpha:]	Letters
[:digit:]	Digits
[:alnum:]	Letters and digits
[:space:]	Whitespace
[:upper:]	Uppercase
[:lower:]	Lowercase
[:punct:]	Punctuation

Exercise: Uppercase Conversion

Convert to uppercase:

Basic sed Substitution

Stream editor for more complex substitutions:

#!/bin/bash

# Basic substitution: s/old/new/
echo "hello world" | sed 's/world/universe/'   # hello universe

# Global substitution: s/old/new/g
echo "foo foo foo" | sed 's/foo/bar/g'         # bar bar bar

# Case insensitive: s/old/new/i
echo "Hello HELLO" | sed 's/hello/hi/gi'       # hi hi

# Delete lines matching pattern
echo -e "one\ntwo\nthree" | sed '/two/d'       # one\nthree

sed Delimiters

Use any delimiter (helpful with paths):

# Default: /
sed 's/old/new/'

# Alternative delimiters
sed 's|/usr/bin|/opt/bin|'
sed 's#/path/to#/new/path#'
sed 's@pattern@replacement@'

sed with Capture Groups

Capture and reuse matched text:

#!/bin/bash

# Capture groups with \( \) and reference with \1
echo "John Smith" | sed 's/\(.*\) \(.*\)/\2, \1/'
# Output: Smith, John

# Rearrange date format
echo "2024-01-15" | sed 's/\([0-9]*\)-\([0-9]*\)-\([0-9]*\)/\3\/\2\/\1/'
# Output: 15/01/2024

# Add prefix to captured text
echo "error: something went wrong" | sed 's/error: \(.*\)/ERROR: [\1]/'
# Output: ERROR: [something went wrong]

Multiple Substitutions

Chain multiple operations:

#!/bin/bash

# Multiple -e options
echo "hello world" | sed -e 's/hello/hi/' -e 's/world/there/'
# Output: hi there

# Semicolon separator
echo "foo bar" | sed 's/foo/baz/; s/bar/qux/'
# Output: baz qux

Exercise: Format Phone Number

Use sed to format a string:

awk for Field Manipulation

Extract and transform fields:

#!/bin/bash

# Print specific fields
echo "John,25,Engineer" | awk -F',' '{print $1}'        # John
echo "John,25,Engineer" | awk -F',' '{print $1,$3}'     # John Engineer

# Rearrange fields
echo "first last" | awk '{print $2, $1}'                # last first

# Add text between fields
echo "a b c" | awk '{print $1 "-" $2 "-" $3}'           # a-b-c

# Transform field values
echo "hello" | awk '{print toupper($0)}'                 # HELLO

Combining Tools

Chain substitution tools:

#!/bin/bash

# Clean and normalize input
INPUT="  Hello,   WORLD!  123  "
CLEAN=$(echo "$INPUT" | \
    tr -s ' ' | \           # Squeeze spaces
    tr 'A-Z' 'a-z' | \      # Lowercase
    tr -d '[:punct:]' | \   # Remove punctuation
    sed 's/^ *//;s/ *$//')  # Trim

echo "$CLEAN"    # hello world 123

Quick Reference

# Parameter expansion
${var/pattern/string}     # Replace first
${var//pattern/string}    # Replace all
${var/#pattern/string}    # Replace at start
${var/%pattern/string}    # Replace at end

# tr command
tr 'abc' 'xyz'            # Translate chars
tr -d 'abc'               # Delete chars
tr -s 'a'                 # Squeeze repeated

# sed command
sed 's/old/new/'          # Replace first
sed 's/old/new/g'         # Replace all
sed 's/old/new/i'         # Case insensitive
sed -e 'cmd1' -e 'cmd2'   # Multiple commands

Key Takeaways

• ${var/old/new} for simple in-variable substitution
• tr is fast for character-by-character translation
• sed 's/old/new/' for pattern-based substitution
• Use // in parameter expansion or /g in sed for global replacement
• Capture groups in sed: \(...\) referenced as \1, \2
• Chain commands with pipes for complex transformations
• Choose the right tool: parameter expansion for simple cases, sed for complex patterns