awk (with examples)

1: Print the fifth column in a space-separated file:

awk '{print $5}' path/to/file

Motivation: This command is useful when you need to extract specific columns from a file. In this case, we want to print the fifth column (field) from a space-separated file.

Explanation:

• awk invokes the awk programming language.
• '{print $5}' is the awk program which tells awk to print the fifth field from each line.
• path/to/file is the path to the file on which we want to apply the awk program.

Example output: If the contents of the file are:

apple banana carrot dog
elephant fish grape horse

The command will output:

carrot
grape

2: Print the second column of the lines containing “foo” in a space-separated file:

awk '/foo/ {print $2}' path/to/file

Motivation: This command is useful when you want to extract a specific column, but only from the lines that match a certain pattern. In this case, we want to print the second column of the lines that contain the string “foo”.

Explanation:

• /foo/ is a pattern matching expression that searches for the string “foo” in the input file.
• print $2 is the action that tells awk to print the second field if the pattern is matched.
• path/to/file is the path to the file on which we want to apply the awk program.

Example output: If the contents of the file are:

hello world
foo bar
apple foo carrot

The command will output:

bar
carrot

3: Print the last column of each line in a file, using a comma as a field separator:

awk -F ',' '{print $NF}' path/to/file

Motivation: This command is useful when you want to extract the last column from each line in a file, but the columns are separated by a different character than the default space. In this case, the columns are separated by commas.

Explanation:

• awk -F ',' sets the field separator to a comma.
• print $NF tells awk to print the last field from each line.
• path/to/file is the path to the file on which we want to apply the awk program.

Example output: If the contents of the file are:

apple,banana,carrot
elephant,fish,grape,horse

The command will output:

carrot
horse

4: Sum the values in the first column of a file and print the total:

awk '{s+=$1} END {print s}' path/to/file

Motivation: This command is useful when you want to calculate the sum of the values in a specific column of a file. In this case, we want to sum the values in the first column.

Explanation:

• awk invokes the awk programming language.
• '{s+=$1}' is the awk program which tells awk to add the value of the first field to the variable s for each line.
• END {print s} is the special pattern END which is triggered after processing all lines. It tells awk to print the value of s.
• path/to/file is the path to the file on which we want to apply the awk program.

Example output: If the contents of the file are:

10
20
30

The command will output:

60

5: Print every third line starting from the first line:

awk 'NR%3==1' path/to/file

Motivation: This command is useful when you want to extract every nth line from a file. In this case, we want to print every third line, starting from the first line.

Explanation:

• awk invokes the awk programming language.
• NR%3==1 is the condition that selects every third line. NR represents the current line number and % is the modulo operator.
• path/to/file is the path to the file on which we want to apply the awk program.

Example output: If the contents of the file are:

line1
line2
line3
line4
line5
line6

The command will output:

line1
line4
line6

6: Print different values based on conditions:

awk '{if ($1 == "foo") print "Exact match foo"; else if ($1 ~ "bar") print "Partial match bar"; else print "Baz"}' path/to/file

Motivation: This command is useful when you want to perform conditional operations based on the values in a specific column. In this case, we want to print different values depending on the value in the first column.

Explanation:

• awk invokes the awk programming language.
• '{if ($1 == "foo") print "Exact match foo"; else if ($1 ~ "bar") print "Partial match bar"; else print "Baz"}' is the awk program that performs different actions based on the value in the first field.
  • if ($1 == "foo") checks if the first field equals “foo”.
  • print "Exact match foo" is the action to be performed if the condition is true.
  • else if ($1 ~ "bar") checks if the first field contains the string “bar”.
  • print "Partial match bar" is the action to be performed if the condition is true.
  • else print "Baz" is the action to be performed if none of the conditions are true.
• path/to/file is the path to the file on which we want to apply the awk program.

Example output: If the contents of the file are:

foo apple
bar1
baz
bar2

The command will output:

Exact match foo
Partial match bar
Baz
Partial match bar

7: Print all lines where the 10th column value equals the specified value:

awk '($10 == value)' path/to/file

Motivation: This command is useful when you want to filter out lines from a file based on the value in a specific column. In this case, we want to print all lines where the value in the 10th column is equal to a specified value.

Explanation:

• awk invokes the awk programming language.
• ($10 == value) is the condition that selects lines where the value in the 10th field is equal to the specified value.
• path/to/file is the path to the file on which we want to apply the awk program.

Example output: If the contents of the file are:

1 2 3 4 5 6 7 8 9 10
1 2 3 4 5 6 7 8 9 20
1 2 3 4 5 6 7 8 9 10

The command awk '($10 == 10)' path/to/file will output:

1 2 3 4 5 6 7 8 9 10
1 2 3 4 5 6 7 8 9 10

8: Print all the lines where the 10th column value is between a min and a max:

awk '($10 >= min_value && $10 <= max_value)' path/to/file

Motivation: This command is useful when you want to filter out lines from a file based on the value in a specific column within a certain range. In this case, we want to print all lines where the value in the 10th column is between a minimum and a maximum value.

Explanation:

• awk invokes the awk programming language.
• ($10 >= min_value && $10 <= max_value) is the condition that selects lines where the value in the 10th field is greater than or equal to the minimum value and less than or equal to the maximum value.
• path/to/file is the path to the file on which we want to apply the awk program.

Example output: If the contents of the file are:

1 2 3 4 5 6 7 8 9 10
1 2 3 4 5 6 7 8 9 20
1 2 3 4 5 6 7 8 9 30

The command awk '($10 >= 10 && $10 <= 20)' path/to/file will output:

1 2 3 4 5 6 7 8 9 10
1 2 3 4 5 6 7 8 9 20

Conclusion

The awk command is a versatile programming language for working on files. It allows you to extract specific columns, filter lines based on conditions, perform calculations, and much more. The examples provided demonstrate different use cases of the awk command, showcasing its flexibility and power when it comes to file processing and manipulation.