How to use the command xargs (with examples)

The xargs command is used to execute a command with piped arguments coming from another command, a file, or any other input source. It treats the input as a single block of text and splits it into separate pieces on spaces, tabs, newlines, and end-of-file. This allows for flexible and efficient execution of commands with multiple arguments.

Use case 1: Run a command using the input data as arguments

Code:

arguments_source | xargs command

Motivation: Sometimes we have a set of data that we want to pass as arguments to a command. Instead of manually specifying each argument, we can use xargs to handle this for us. This can be particularly useful when dealing with large sets of data or when the input is constantly changing.

Explanation:

• arguments_source: The source of the arguments. It can be a command that produces the arguments, a file containing the arguments, or any other method of providing the arguments.
• xargs: The command itself.
• command: The command to be executed, using the input data as arguments.

Example output: Let’s say we have a file called names.txt containing the following names:

John
Jane
Michael
Emily

We can use xargs to pass each name in the file as an argument to the echo command:

cat names.txt | xargs echo

Output:

John Jane Michael Emily

The echo command is executed with the arguments provided by xargs, resulting in the names being printed on a single line.

Use case 2: Run multiple chained commands on the input data

Code:

arguments_source | xargs sh -c "command1 && command2 | command3"

Motivation: In some cases, we may need to run multiple commands on the input data, with each command depending on the output of the previous one. Using xargs in combination with sh allows us to chain these commands together and process the input data in a more efficient and streamlined manner.

Explanation:

• arguments_source: The source of the arguments.
• xargs: The command itself.
• sh -c: Invokes the shell to run the chained commands.
• "command1 && command2 | command3": The chained commands to be executed.

Example output: Let’s say we have a file called numbers.txt containing a list of numbers:

1
2
3
4

We can use xargs and sh to calculate the square of each number and then sum the squares:

cat numbers.txt | xargs sh -c "echo $1*$1 | awk '{sum += $1} END {print sum}'"

Output:

30

The sh -c command executes the chained commands. In this example, command1 (echo) calculates the square of each number, command2 (awk) sums the squares, and command3 (print) outputs the final sum.

Use case 3: Delete all files with a .backup extension

Code:

find . -name '*.backup' -print0 | xargs -0 rm -v

Motivation: When we want to delete multiple files that match a certain pattern, such as files with a specific extension, it can be cumbersome to manually specify each file. xargs can simplify this task by taking the matched files from the find command and passing them as arguments to the rm command.

Explanation:

• find . -name '*.backup' -print0: Searches for files with the .backup extension in the current directory and prints their names, separated by null characters.
• xargs: The command itself.
• -0: Specifies that the arguments are separated by null characters, allowing for correct handling of file names with spaces or special characters.
• rm -v: The command to delete the files, with the -v option for verbose output.

Example output: Suppose we have the following files in the current directory:

file1.backup
file2.backup
file3.backup

Running the find command followed by xargs with rm will delete all the files with the .backup extension:

find . -name '*.backup' -print0 | xargs -0 rm -v

Output:

removed './file1.backup'
removed './file2.backup'
removed './file3.backup'

The rm command is executed for each file name provided by xargs, resulting in the deletion of the files.

Use case 4: Execute the command once for each input line with placeholder substitution

Code:

arguments_source | xargs -I _ command _ optional_extra_arguments

Motivation: Sometimes we want to execute a command multiple times, with each execution using a different input value. xargs allows us to achieve this by substituting a placeholder (marked as _) in the command with the current input line.

Explanation:

• arguments_source: The source of the arguments.
• xargs: The command itself.
• -I _: Specifies the placeholder _ to be replaced with the input line when executing the command.
• command: The command to be executed, with the placeholder _ representing the input line.
• optional_extra_arguments: Any additional arguments that need to be passed to the command.

Example output: Let’s say we have a file called cities.txt containing a list of cities:

London
Paris
New York

We can use xargs to execute the echo command for each city, with the city name replacing the placeholder _ in the command:

cat cities.txt | xargs -I _ echo "Welcome to _!"

Output:

Welcome to London!
Welcome to Paris!
Welcome to New York!

The echo command is executed for each city name provided by xargs, resulting in a personalized welcome message for each city.

Use case 5: Parallel runs of up to max-procs processes at a time

Code:

arguments_source | xargs -P max-procs command

Motivation: When dealing with computationally intensive tasks or a large number of input arguments, it can be beneficial to run multiple instances of a command in parallel. xargs with the -P option allows us to control the number of processes running simultaneously, maximizing the utilization of available system resources and potentially reducing the overall execution time.

Explanation:

• arguments_source: The source of the arguments.
• xargs: The command itself.
• -P max-procs: Specifies the maximum number of processes to run in parallel.
• command: The command to be executed.

Example output: Suppose we have a file called urls.txt containing a list of URLs:

https://example1.com
https://example2.com
https://example3.com

We can use xargs to download the content of each URL in parallel by executing the curl command for each URL:

cat urls.txt | xargs -P 3 curl -O

Output:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   131  100   131    0     0    781      0 --:--:-- --:--:-- --:--:--   781
100   508  100   508    0     0   1022      0 --:--:-- --:--:-- --:--:--  1022
100 42337  100 42337    0     0  62166      0 --:--:-- --:--:-- --:--:-- 62166

The curl command is executed in parallel for each URL provided by xargs, resulting in the downloading of the content from each URL. The -P 3 option limits the maximum number of concurrent curl processes to 3.