How to use the command 'sed' (with examples)

sed, short for “stream editor”, is a powerful command-line utility that allows users to perform basic text changes to a file or input stream in a programmatic way. It is most often used for substitution, deletion, and insertion of text and is a staple tool for quickly editing structured text. Its capability to script transformations makes it a core component in many Unix-based shell scripts.

Use case 1: Replace all apple (basic regex) occurrences with mango (basic regex) in all input lines and print the result to stdout

Code:

command | sed 's/apple/mango/g'

Motivation:

In scenarios where you have large datasets or logs with repetitive occurrences of a specific word that need to be changed uniformly, sed provides a fast and efficient way to execute these changes instantly, avoiding the need for time-consuming manual edits or cumbersome find-and-replace operations in text editors.

Explanation:

• command: Represents the command that outputs text, which acts as an input for sed.
• sed: Invokes the sed command-line tool.
• 's/apple/mango/g': This is a sed substitution command. Here, s signifies a substitution, apple is the search pattern, mango is the replacement string, and g is a global flag indicating that all occurrences in each line should be replaced.

Example output:

Assuming the input is:

apple is a fruit. I have an apple.

The output will be:

mango is a fruit. I have a mango.

Use case 2: Execute a specific script [f]ile and print the result to stdout

Code:

command | sed -f path/to/script.sed

Motivation:

When dealing with complex text processing where multiple search and replace operations are needed, writing each command separately in the terminal can be cumbersome. Using a script file allows users to predefine a sequence of sed operations, simplifying execution and improving readability and reusability.

Explanation:

• command: Represents the command that outputs text which will be processed by sed.
• sed: Invokes the sed command.
• -f path/to/script.sed: The -f flag specifies a file containing sed commands. This script file is pre-written with multiple sed instructions to be applied to the input.

Example output:

Assuming script.sed contains:

s/apple/mango/g
s/orange/peach/g

And given input:

apple and orange are fruits.

The output will be:

mango and peach are fruits.

Use case 3: Delay opening each file until a command containing the related w function or flag is applied to a line of input

Code:

command | sed -fa path/to/script.sed

Motivation:

This usage is beneficial in situations where a script needs to modify a file but should only open the target file when certain conditions are met. This minimizes resource usage and can prevent unnecessary file I/O operations, optimizing script performance.

Explanation:

• command: Serves as the source of the input for sed.
• sed: Initiates the sed command.
• -fa path/to/script.sed: The -f flag again points to a file containing sed instructions to execute. The -a option tells sed to delay the opening of any output files specified with w commands until input lines matching the provided pattern occur.

Example output:

Given script.sed has:

/start/ w output.txt

And input:

here is some text
start processing from here
additional text

Output to stdout:

here is some text
start processing from here
additional text

output.txt would start filling from “start processing from here”.

Use case 4: Replace all apple (extended regex) occurrences with APPLE (extended regex) in all input lines and print the result to stdout

Code:

command | sed -E 's/(apple)/\U\1/g'

Motivation:

There are times when regular expressions come in handy for more advanced pattern matching. Extending regex capabilities allows users to create complex matches or group portions of matches, opening a broad spectrum of sophisticated text transformations.

Explanation:

• command: Provides input data for sed.
• sed: Calls the sed utility.
• -E: Enables extended regular expressions, supporting parentheses and other features.
• 's/(apple)/\U\1/g': The substitution pattern utilizes capturing groups, where (apple) is captured, and \U\1 converts the match (first captured group) to uppercase, replacing each occurrence.

Example output:

For input:

apple pie and apple juice.

The output becomes:

APPLE pie and APPLE juice.

Use case 5: Print just the first line to stdout

Code:

command | sed -n '1p'

Motivation:

In tasks that require isolating the first record or line of a dataset, such as retrieving header details from a CSV, obtaining quick insights, or limiting output for performance, extracting just the first line proves efficient and reduces unnecessary data processing.

Explanation:

• command: Supplies input data for the command.
• sed: Initiates the text editing tool.
• -n: Suppresses the default output behavior of sed.
• '1p': Directs sed to print the first line. Suppression of usual output due to -n ensures only the specified line gets printed.

Example output:

Input is:

First line for display
Second line ignored

Output will be:

First line for display

Use case 6: Replace all apple (basic regex) occurrences with mango (basic regex) in a specific file and overwrite the original file in place

Code:

sed -i 's/apple/mango/g' path/to/file

Motivation:

For those needing to permanently update a file without generating backups or side-effects from original content, sed offers in-place editing. This is invaluable for scripts that require batch updates across numerous files or configurations by overwriting them directly.

Explanation:

• sed: Invokes the sed command.
• -i: Stands for in-place editing, updating the file directly.
• 's/apple/mango/g': Specifies the substitution operation, where all occurrences of apple replace mango across lines.
• path/to/file: Identifies the target file for sed to perform operations on in-place.

Example output:

Given a file content of:

I love apple.
Apple pie is tasty.

After execution, the file content changes to:

I love mango.
Apple pie is tasty.

Conclusion:

Understanding and mastering the sed command empowers users with a versatile tool for string processing and text manipulation. Whether replacing text, executing complex scripts, or performing regex-enabled transformations, sed enhances data manipulation capabilities in batch processes, making it an indispensable ally for programmers and system administrators alike.