The tsv-filter command is a tool that allows you to filter lines of a Tab-Separated Values (TSV) file by running tests against individual fields. It can be useful for quickly extracting specific data from a large TSV file based on certain criteria.

Use case 1: Print the lines where a specific column is numerically equal to a given number

Code:

tsv-filter -H --eq field_name:number path/to/tsv_file

Motivation: You may have a TSV file containing a large amount of data, and you only want to extract the lines where a specific column has a certain numerical value. Using the --eq option allows you to do this efficiently.

Explanation:

• tsv-filter: The command itself.
• -H: Indicates that the first line of the TSV file is a header row.
• --eq: Specifies the test to be performed, in this case, checking if the value of the specified field is equal to the given number.
• field_name:number: The name of the column to be checked followed by the desired number.
• path/to/tsv_file: The path to the TSV file to be filtered.

Example Output:

field1    field2    field3
1         apple     red
3         banana    yellow

In this example, the TSV file has three columns: field1, field2, and field3. By running the command with the appropriate parameters, we are left with only the lines where field1 is numerically equal to 3.

Use case 2: Print the lines where a specific column has a comparison with a given number

Code:

tsv-filter --eq|ne|lt|le|gt|ge column_number:number path/to/tsv_file

Motivation: In some cases, you may want to filter lines based on various numerical comparisons with a specific column. This use case allows you to do this easily with the --eq, --ne, --lt, --le, --gt, and --ge options.

Explanation:

• tsv-filter: The command itself.
• --eq|ne|lt|le|gt|ge: Specifies the comparison test to be performed. You can choose from equality (--eq), non-equality (--ne), less than (--lt), less than or equal to (--le), greater than (--gt), or greater than or equal to (--ge).
• column_number:number: Specifies the column number to be checked followed by the desired number.
• path/to/tsv_file: The path to the TSV file to be filtered.

Example Output:

field1    field2    field3
1         apple     red
2         banana    yellow

In this example, we want to print the lines where the values in field1 are less than or equal to 2. The output shows the two lines that satisfy this condition.

Use case 3: Print the lines where a specific column has a comparison with a given string

Code:

tsv-filter --str-eq|ne|in-fld|not-in-fld column_number:string path/to/tsv_file

Motivation: Similar to the previous use case, you may want to filter lines based on various string comparisons with a specific column. This use case allows you to do this using the --str-eq, --str-ne, --in-fld, and --not-in-fld options.

Explanation:

• tsv-filter: The command itself.
• --str-eq|ne|in-fld|not-in-fld: Specifies the string comparison test to be performed. You can choose from equality (--str-eq), non-equality (--str-ne), part of (--in-fld), or not part of (--not-in-fld).
• column_number:string: Specifies the column number to be checked followed by the desired string.
• path/to/tsv_file: The path to the TSV file to be filtered.

Example Output:

field1    field2    field3
1         apple     red
2         apple     yellow

In this example, we want to print the lines where field2 is equal to “apple”. The output shows the two lines that meet this condition.

Use case 4: Filter for non-empty fields

Code:

tsv-filter --not-empty column_number path/to/tsv_file

Motivation: When dealing with large TSV files, you may want to filter out lines that have empty fields in a specific column. This use case allows you to easily extract only the lines where the specified column is not empty.

Explanation:

• tsv-filter: The command itself.
• --not-empty: Specifies that the column should not be empty.
• column_number: Specifies the column number to be checked.
• path/to/tsv_file: The path to the TSV file to be filtered.

Example Output:

field1    field2    field3
1         apple     red
2         banana    yellow

In this example, we want to filter for non-empty values in the second column of the TSV file. The output shows the lines that have non-empty values in the specified column.

Use case 5: Print the lines where a specific column is empty

Code:

tsv-filter --invert --not-empty column_number path/to/tsv_file

Motivation: On the contrary to the previous use case, you may want to filter out lines where a specific column is empty and keep only the lines where the column is empty. This use case allows you to achieve that by using the --invert option.

Explanation:

• tsv-filter: The command itself.
• --invert: Specifies to invert the result of the test, i.e., keep lines that do not match the specified condition.
• --not-empty: Specifies that the column should not be empty.
• column_number: Specifies the column number to be checked.
• path/to/tsv_file: The path to the TSV file to be filtered.

Example Output:

field1    field2    field3
3                   green

In this example, we want to print the lines where the third column is empty. The output shows the line where the condition is met.

Use case 6: Print the lines that satisfy two conditions

Code:

tsv-filter --eq column_number1:number --str-eq column_number2:string path/to/tsv_file

Motivation: In some cases, you may want to filter lines based on multiple conditions simultaneously. This use case allows you to achieve that by specifying two conditions using the --eq and --str-eq options.

Explanation:

• tsv-filter: The command itself.
• --eq: Specifies the test to be performed, checking if the value of the specified column is equal to the given number.
• --str-eq: Specifies the test to be performed, checking if the value of the specified column is equal to the given string.
• column_number1:number: Specifies the column number and the desired number for the first condition.
• column_number2:string: Specifies the column number and the desired string for the second condition.
• path/to/tsv_file: The path to the TSV file to be filtered.

Example Output:

field1    field2    field3
2         apple     yellow

In this example, we want to print the lines where field1 is equal to 2 and field2 is equal to “apple”. The output shows the line that satisfies both conditions.

Use case 7: Print the lines that match at least one condition

Code:

tsv-filter --or --eq column_number1:number --str-eq column_number2:string path/to/tsv_file

Motivation: Sometimes, you may want to filter lines that match at least one of multiple conditions. This use case allows you to achieve that by using the --or option in conjunction with another condition.

Explanation:

• tsv-filter: The command itself.
• --or: Specifies that at least one of the conditions should be satisfied.
• --eq: Specifies the test to be performed, checking if the value of the specified column is equal to the given number.
• --str-eq: Specifies the test to be performed, checking if the value of the specified column is equal to the given string.
• column_number1:number: Specifies the column number and the desired number for the first condition.
• column_number2:string: Specifies the column number and the desired string for the second condition.
• path/to/tsv_file: The path to the TSV file to be filtered.

Example Output:

field1    field2    field3
1         apple     red
2         apple     yellow
4         banana    green

In this example, we want to print the lines where field1 is equal to 2 or field2 is equal to “apple”. The output shows the lines that satisfy at least one of these conditions.

Use case 8: Count matching lines, interpreting first line as a Header

Code:

tsv-filter --count -H --eq field_name:number path/to/tsv_file

Motivation: Besides filtering lines based on specific criteria, you may also want to count the number of lines that match a certain condition. This use case allows you to count the matching lines, assuming the first line of the TSV file is a header.

Explanation:

• tsv-filter: The command itself.
• --count: Specifies to count the number of matching lines.
• -H: Indicates that the first line of the TSV file is a header row.
• --eq: Specifies the test to be performed, checking if the value of the specified field is equal to the given number.
• field_name:number: The name of the column to be checked followed by the desired number.
• path/to/tsv_file: The path to the TSV file to be filtered.

Example Output:

2

In this example, we want to count the number of lines where the value in the specified column is equal to 1. The output shows that there are 2 lines that match this condition.

Conclusion:

The tsv-filter command is a flexible tool for filtering lines of TSV files based on various conditions. Whether you need to extract specific values, count matching lines, or filter for non-empty fields, this command provides a range of options to accomplish your data extraction needs. Experiment with the different use cases presented in this article to effectively filter and extract data from TSV files.