How to use the command "trace-cmd" (with examples)

The “trace-cmd” command is a utility used to interact with the Ftrace Linux kernel internal tracer. It is essential for tracing system events and capturing detailed information for debugging and performance analysis. However, it can only be run as the root user.

Use case 1: Display the status of tracing system

Code:

trace-cmd stat

Motivation:

• Checking the status of the tracing system can provide insights into the current state of tracing activities, such as active tracing sessions, running plugins, and buffer usage.

Explanation:

• “trace-cmd”: The command itself.
• “stat”: The subcommand to display the tracing system’s status.

Example output:

Tracers: ['nop', 'function', 'wakeup', 'wakeup_rt', 'wakeup_dl', 'function_graph', 'mmiotrace', 'blk', 'hwlat', 'osnoise', 'timerlat', 'futex', 'ksyms'] 

Current Tracing Buffer Size: 8 MB (8388608 bytes) total

Use case 2: List available tracers

Code:

trace-cmd list -t

Motivation:

• Before starting tracing, it is crucial to verify the available tracers on the system. This step allows you to choose the appropriate tracer for your specific needs.

Explanation:

• “trace-cmd”: The command itself.
• “list”: The subcommand to list available options.
• “-t”: The option to specify that we want to list available tracers.

Example output:

Plugin Name                     Available            
---------------------------- ---------------------
nop                            yes
function                       yes
wakeup                         yes
wakeup_rt                      yes
wakeup_dl                      yes
function_graph                 yes
mmiotrace                      yes
blk                            yes
hwlat                          yes
osnoise                        yes
timerlat                       yes
futex                          yes
ksyms                          yes

Use case 3: Start tracing with a specific plugin

Code:

trace-cmd start -p timerlat

Motivation:

• Starting tracing with a specific plugin allows you to capture events associated with that particular aspect of the system. For instance, using the “timerlat” plugin helps analyze the latency caused by timers in the Linux kernel.

Explanation:

• “trace-cmd”: The command itself.
• “start”: The subcommand to initiate tracing.
• “-p timerlat”: The option and argument to specify the particular plugin for tracing, in this case, the “timerlat” plugin.

Use case 4: View the trace output

Code:

trace-cmd show

Motivation:

• Viewing the trace output helps to analyze the recorded events and their timestamps. It provides crucial information for troubleshooting performance issues or understanding the system behavior.

Explanation:

• “trace-cmd”: The command itself.
• “show”: The subcommand to display the trace output.

Example output:

...
14) bash-7254  [000]  3594.928235: sched_migrate_task:      0x1fd/630   flags: all_cpus(1), system-wide(0), high_priority(0), idle(0), task_new(0) 
15) bash-7254  [000]  3594.928250: sched_migrate_task:      0x1a3/419   flags: all_cpus(1), system-wide(0), high_priority(0), idle(0), task_new(0) 
16) bash-7254  [000]  3594.928260: sched_migrate_task:      0x1a4/420   flags: all_cpus(1), system-wide(0), high_priority(0), idle(0), task_new(0) 
...

Use case 5: Stop the tracing but retain the buffers

Code:

trace-cmd stop

Motivation:

• Stopping tracing while retaining the buffers is helpful when you need to pause the tracing temporarily without losing the captured data. It allows you to resume the tracing session later and continue to collect events.

Explanation:

• “trace-cmd”: The command itself.
• “stop”: The subcommand to stop the tracing.

Use case 6: Clear the trace buffers

Code:

trace-cmd clear

Motivation:

• Clearing the trace buffers is necessary to remove all the captured data from previous tracing sessions. This step ensures a clean slate for the subsequent tracing activities.

Explanation:

• “trace-cmd”: The command itself.
• “clear”: The subcommand to clear the trace buffers.

Use case 7: Clear the trace buffers and stop tracing

Code:

trace-cmd reset

Motivation:

• Clearing the trace buffers and stopping the tracing session is useful when you want to completely reset the tracing environment and start fresh. This step ensures all the previous data is discarded and a new tracing session can be initiated.

Explanation:

• “trace-cmd”: The command itself.
• “reset”: The subcommand to clear the trace buffers and stop tracing.

Conclusion:

The “trace-cmd” command is a powerful utility for interacting with the Ftrace Linux kernel internal tracer. Each use case mentioned above demonstrates different functionalities of the command, such as displaying the tracing system’s status, starting and stopping tracing, clearing the trace buffers, and viewing the trace output. Understanding these use cases will enable users to effectively utilize “trace-cmd” for debugging and performance analysis purposes.