Power Up Your Command Line Skills with These sed Examples

Have you been looking for another way to perform editing operations on texts? Look no further! This tutorial has got you covered with stream editor sed examples.

In a nutshell, sed allows you to perform a wide range of edits on files and text streams in a non-interactive way. You make the editing decisions as you call the commands. And in this tutorial, you will learn how to use sed commands and effectively streamline your workflow.

Read on and power up your command line skills!

Prerequisites

Ensure you have the following to follow along with the hands-on demonstration in this tutorial:

• A command line interface (e.g., Terminal on macOS or Linux) – This tutorial uses Ubuntu Linux 20.04, but the commands should work on most popular operating systems.

Related:How to Install Ubuntu 20.04 [Step-by-Step]

• A text editor (e.g., nano, vi, or Sublime Text) – This tutorial uses Nano.

Related:Essential Tips for Installing & Using Sublime Text on Ubuntu

Installing sed on Ubuntu

sed is generally included as a standard utility in most Unix-like operating systems, including Linux and macOS. But regardless, double-checking if you already have sed on your system would be okay and is a basic procedure before performing text transformations.

1. Open your terminal, and run the below sed command to check which sed version is installed on your system.

sed --version

If sed is installed, you will see the sed version number and some information about the build, as shown below.

Otherwise, you will see the error message below if sed is not installed.

This error message indicates that the sed command was not found in your system’s PATH environment variable.

2. Next, run the [apt-get update below command to update and install sed on your system.

This command fetches the latest package information from the configured package sources, including the package’s dependencies, files, and scripts. Running this command is necessary to ensure you install the latest version of sed and any other packages you may need.

Related:Learning Ubuntu Apt Get Through Examples

sudo apt-get update -y && sudo apt-get install sed -y

3. Lastly, run the following command to verify the sed command is installed on your system.

This command prints the full path to the sed executable, which in this example is /usr/bin/sed.

Once you have installed sed and verified it is in your PATH, you can use the sed command to perform various text transformations and manipulations.

which sed

Replacing Strings in a Text File

sed is a popular stream editor available on most Unix-like operating systems, and one of sed’s most handy features is replacing strings in a text file.

Below are some scenarios, but are not limited to, where replacing strings is helpful:

• Updating a list of URLs to reflect a new domain name.
• Changing product IDs in a database or CSV file.
• Formatting data for import into another system and many more.

The basic syntax for using sed is as follows where:

sed command options input_file | output_file

Suppose you have a text file called urls.txt that contains a list of URLs and want to update all of the URLs to use a new domain name.

1. Run the echo commands below to create the urls.txt file in the current directory. These commands do not provide output but write specified texts to the file.

echo "http://old-domain.com/page1" > urls.txt && echo "http://old-domain.com/page2" >> urls.txt && echo "http://old-domain.com/page3" >> urls.txt

2. Next, run the cat command to view the contents of the urls.txt file. cat urls.txt

cat urls.txt

The output below confirms the three old URLs written in the urls.txt file.

3. Now, run the command below to find and substitute (s) all occurrences of the string old-domain.com with new-domain.com in the urls.txt file.

sed 's/old-domain.com/new-domain.com/' urls.txt

You will see that the URLs in your input file have been updated with the new domain, as shown.

Replacing the Nth Occurrences of a String

In addition to the basic search-and-replace functionality, sed has additional built-in commands that allow you to perform more complex text transformations. One such case is replacing the nth occurrence of the pattern string in a single line of text.

For example, you wish to update the second field in a CSV file with a new value or update the third occurrence of a string in a database dump.

The basic syntax for this command is as follows where:

• pattern – is the string to replace.
• replacement – is the string to use as a replacement.
• n – is the nth occurrence of the string to replace (e.g., 1=first occurrence, 2=second occurrence…).

sed 's/pattern/replacement/n' input_file

To see how to replace the nth occurrence of a string, follow these steps:

Run the following echo command to create a line of text with three occurrences of the pattern quick. The sed command then replaces (s) the second occurrence (2) of the pattern quick with the string fast.

echo "The quick and quick and quick brown fox jumps over the lazy dog" | sed 's/quick/fast/2'

As you can see below, only the second occurrence of the pattern quick has been replaced with the string fast, while all other occurrences remain unchanged.

The nth command is case-sensitive by default. As a result, the command only looks for the exact match of the pattern. To ignore the letter case of the pattern, jump to the following step.

💡 Note that the nth command counts occurrences of the pattern from left to right. Moreover, the nth command only works on a single line of text. To perform a substitution on multiple lines, use a different method demonstrated in the following section.

Next, run the below command to replace the quick pattern’s second (2) occurrence with fast while ignoring (i) the letter case.

The i option comes in handy when working with text with inconsistent capitalization.

echo "The Quick and QuiCk and Quick brown fox jumps over the lazy dog" | sed 's/quick/fast/2i'

In the output below, the second occurrence of the word quick (QuiCk) is successfully replaced with the string fast — regardless of the letter case.

Replacing All Occurrences of a Matching Pattern/String

Instead of replacing the nth occurrence of a string, perhaps you plan to replace all its occurrences. Doing so dramatically helps with your plan to mass-update every occurrence of a Copyright string with the current year, for example.

1. Create a file called input.txt with the following contents in your preferred text editor.

This is a quick test.
The quick brown fox jumps over the lazy dog.
Quick, you need to finish this task before it gets too late.

2. Next, run the following command to replace (s) all occurrences of the word quick with fast from the input.txt file. Once replaced, the changes are written to the output.txt file.

sed 's/quick/fast/g' input.txt > output.txt

3. Now, run the command below to view the output.txt file’s contents and verify if the replacement has been made.

cat output.txt

You will see all occurrences of the word quick have been replaced with fast, as shown below. Similar to the nth command, the g command is case-sensitive and replaces only the exact matches.

The Quick string will not get replaced, as shown below, but you can use the i option to perform case-insensitive substitutions.

Replacing a Matching Pattern String on a Specific Line

Perhaps you already know which line of the string you want to replace lies. Why replace all occurrences if you can replace strings on specific lines? This feature is useful when making changes to a particular line in a file or stream but leaving the other lines untouched.

Below is the syntax for replacing a string from a specific line, where n represents the line number.

sed 'n s/pattern/replacement/' input_file output_file

1. Create a file called myfile.txt with the following content.

test1
test2
test3

2. Next, run the below sed command to perform the following:

• Search for the word test2 from the second line (2) of the myfile.txt file.Replace (s) the word test2 with test2_changed.Write the changes (>) to a new file called myfile_changed.txt.

sed '2 s/test2/test2_changed/' myfile.txt > myfile_changed.txt

3. Lastly, run the below cat command to view the contents of the myfile_changed.txt file.

cat myfile_changed.txt

As you can see below, the second line of the myfile.txt file has been replaced with test2_changed, while all other lines remain unchanged.

Outputting Changed Lines of Strings Exclusively

Instead of printing the entire content of a file after making changes, you may want to print the changed lines exclusively. How? By using the p flag together with the -n option.

Below is the syntax in printing changed lines of strings exclusively where:

• The -n option tells sed to suppress the default behavior of printing every line.
• The p flag tells sed to print the output of the command.

sed -n 's/pattern/replacement/p' input_file output_file

Open your myfile.txt file and replace the content with the following. Notice that the second line is incorrect since the word first should be second instead.

This is the first line of my file.
This is the first line of my file.
This is the third line of my file.

Now, run the below sed command to replace the string first with second in the second line (2) of the myfile.txt but only output the modified lines (p).

sed -n '2 s/first/second/p' myfile.txt

You will see the output of the modified second line only on your terminal, as shown below.

Conclusion

In this tutorial, you have used different sed usage examples to perform search-and-replace operations. Moreover, you touched on more advanced techniques, such as replacing specific occurrences and outputting only the modified lines.

sed is a powerful command line tool that lets you perform various text manipulation tasks. Whether making global replacements across an entire file or targeted replacements on specific lines, sed will do the trick.

Now, why not explore more about regular expressions? Regular expressions are essential to many text manipulation tools, including sed. These expressions are a time-saver and lessen exerted efforts. When? While you perform complex searches and replace operations on large amounts of text.