Adding Timestamps with the 'ts' Command (with examples)

The ’ts’ command is a versatile utility from the moreutils package that is used to prepend timestamps to each line of input from standard input (stdin). This tool is particularly useful for logging and tracking events, especially in real-time data processing, as it helps to understand the timing and order of events. Its flexibility allows you to format timestamps in various ways and even convert existing timestamps to relative formats for easier interpretation.

Use case 1: Add a timestamp to the beginning of each line

Code:

command | ts

Motivation:

Adding a timestamp to each line of output from a command allows one to log when specific output was generated. This is particularly beneficial in debugging and monitoring applications, where understanding the sequence of events or actions can help diagnose issues or performance bottlenecks. By providing a timestamp, you’re able to easily correlate log entries with times of interest, such as when an error occurred or when a new user connected to a service.

Explanation:

• command: This represents any command whose output you want to timestamp. For example, it could be a script that outputs log data or a data stream.
• |: This is a pipe, which is a standard way in Unix-like systems to redirect the output of one command to another.
• ts: Executes the ’ts’ command, which adds a timestamp to the beginning of each input line.

Example output:

If you have a command that outputs the following:

Started processing data
Data chunk 1 complete
Data chunk 2 complete
Process finished

Using ts, the output will look like:

Oct 13 14:30:45 Started processing data
Oct 13 14:30:47 Data chunk 1 complete
Oct 13 14:31:00 Data chunk 2 complete
Oct 13 14:31:15 Process finished

Use case 2: Add timestamps with microsecond precision

Code:

command | ts "%b %d %H:%M:%.S"

Motivation:

High-resolution timestamps including microseconds are crucial in scenarios where events occur in quick succession and must be precisely ordered. For financial transactions, network packet analysis, or high-frequency trading systems, having microsecond precision can be the difference between an actionable insight and misleading information. This format allows developers and analysts to detect and resolve minute performance issues and optimizations that can significantly enhance processing efficiency.

Explanation:

• command: Again, this is the source of the log or output which needs timestamping.
• |: A pipe for connecting the command output to the ’ts’ command.
• ts "%b %d %H:%M:%.S":
  • "%b %d %H:%M:%.S": This is a format string where:
    • %b denotes the abbreviated month name.
    • %d represents the day of the month.
    • %H:%M specifies the hour and minute.
    • %.S provides the seconds, including microseconds.

Example output:

Oct 13 14:30:45.123456 Started processing data
Oct 13 14:30:47.654321 Data chunk 1 complete
Oct 13 14:31:00.789012 Data chunk 2 complete
Oct 13 14:31:15.345678 Process finished

Use case 3: Add incremental timestamps with microsecond precision, starting from zero

Code:

command | ts -i "%H:%M:%.S"

Motivation:

Incremental timestamps are useful for performance testing and benchmarking software systems. By starting from zero, developers can see precisely how many seconds or microseconds have elapsed between each output line of the command. This is ideal for optimizing processes or understanding the lifecycle of a script, function, or application behavior over time without necessarily focusing on real-world clock time.

Explanation:

• command: The process or script whose output you are analyzing.
• |: The pipe directs the command output to ’ts'.
• ts -i "%H:%M:%.S":
  • -i: This flag indicates that ts should use incremental timestamps starting from zero.
  • "%H:%M:%.S": This is the format for the elapsed time in hours, minutes, and second fractions with microsecond precision.

Example output:

00:00:00.000001 Started processing data
00:00:02.530000 Data chunk 1 complete
00:00:13.120000 Data chunk 2 complete
00:00:27.539999 Process finished

Use case 4: Convert existing timestamps in a text file (e.g., a log file) into relative format

Code:

cat path/to/file | ts -r

Motivation:

Converting absolute timestamps in logs to relative timestamps can simplify understanding the time difference between events. When reviewing logs, it’s often more valuable to see the time elapsed between actions rather than static timestamps, especially for long-running processes. It can highlight delays, bottlenecks, or unusual variances that require attention.

Explanation:

• cat path/to/file: Outputs the contents of the file located at “path/to/file”.
• |: Pipes the file content to the ’ts’ utility.
• ts -r:
  • -r: Flag to convert existing timestamps to a relative time format, based on the difference from the previous read timestamp.

Example output:

If the file consists of absolute timestamps:

2023-10-13 14:30:00 Started
2023-10-13 14:30:10 Data chunk 1 complete
2023-10-13 14:31:20 Data chunk 2 complete
2023-10-13 14:32:45 Process finished

Using ts -r, it might convert to:

00:00:00 Started
00:00:10 Data chunk 1 complete
00:01:10 Data chunk 2 complete
00:02:25 Process finished

Conclusion:

The ’ts’ command in the moreutils package provides an efficient way to manage timestamps in logging and data stream outputs. Through its various formatting and processing options, it adds clarity and depth to system events and processes, aiding developers and engineers in accurate diagnostics and system performance evaluation. Whether you require real-time insight into event occurrences or need detailed incremental timings for performance tuning, ’ts’ serves as a powerful yet simple tool to enhance your data handling capabilities.