Utilizing the 'sar' Command for System Performance Monitoring (with examples)

The ‘sar’ command, short for System Activity Report, is a powerful tool in Unix and Linux systems used to collect, report, or save system activity information. It provides insights into various system performance metrics, such as CPU usage, memory statistics, disk status, network activity, and more. By offering detailed reports, it helps system administrators diagnose potential system bottlenecks or pinpoint areas for optimization. With the ability to monitor and record data over time, ‘sar’ is a staple in performance monitoring and analysis.

Use case 1: Reporting I/O and Transfer Rate to Physical Devices

Code:

sar -b 1

Motivation:

Monitoring disk I/O activity and transfer rates is vital in ensuring your storage devices are performing optimally. High I/O operation times can indicate a performance bottleneck in disk operations, which can severely affect application performance. Using this command, administrators can observe real-time data to diagnose issues or validate expected behavior after a system update or change.

Explanation:

• -b: This flag invokes reports on I/O and transfer rate statistics. It shows metrics like the number of transactions, blocks per second, and total blocks used.
• 1: This numeric argument specifies that the command should output I/O statistics every second.

Example Output:

Linux 5.4.0-73-generic (server01)     10/02/2023     _x86_64_    (8 CPU)

08:00:01 AM       tps      rtps      wtps   bread/s   bwrtn/s
08:00:02 AM    100.04     50.03     50.01    2048.0    4096.0
08:00:03 AM    102.06     52.05     50.01    2100.0    4200.0

Use case 2: Reporting Network Device Statistics

Code:

sar -n DEV 2 10

Motivation:

Network performance is a critical aspect of server operations. Slow network speeds or excessive packet loss can dramatically affect the user experience and service delivery. This command allows monitoring of network device statistics, such as packets transferred and received, which can help in diagnosing network-related issues.

Explanation:

• -n DEV: This option instructs ‘sar’ to report on network device statistics like packets transmitted and received, errors, and drops.
• 2: The frequency in seconds at which data is collected.
• 10: The total number of outputs generated, effectively running this collection for 20 seconds.

Example Output:

08:00:01 AM       IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s
08:00:03 AM        eth0    1000.00    900.00   1200.00    800.00      0.00      0.00     50.00
08:00:05 AM        eth0    1050.00    940.00   1300.00    820.00      0.00      0.00     45.00

Use case 3: Reporting CPU Utilization

Code:

sar -u ALL 2

Motivation:

CPU utilization is often the first metric checked in system performance analysis. Understanding the distribution of system load across CPU cores and user/kernel space can help in detecting imbalances or identifying overly demanding processes. Monitoring CPU performance in real time is essential for effective load balancing and capacity planning.

Explanation:

• -u ALL: Command option to display CPU utilization reports for all cores, summarizing both user and system loads.
• 2: Every 2 seconds, data collection and reporting occur, enabling real-time insights into CPU usage.

Example Output:

08:00:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
08:00:03 AM     all      20.00      0.00     15.00      5.00      0.00     60.00
08:00:05 AM     all      22.00      0.00     13.00      4.00      0.00     61.00

Use case 4: Reporting Memory Utilization

Code:

sar -r ALL 1 20

Motivation:

Effective memory management is critical for optimal system performance and avoiding resource exhaustion that may lead to crashes. By monitoring memory usage, administrators can track application consumption patterns and detect memory leaks or inefficiencies.

Explanation:

• -r ALL: Option to report on all available memory utilization metrics, such as free and used memory, cache, and buffers.
• 1: The interval, meaning memory statistics are reported every 1 second.
• 20: Specifies the total number of iterations the report is generated, providing a detailed view over 20 seconds.

Example Output:

08:00:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
08:00:02 AM    262144   1048576     80.00    32768    512000   1572864     70.00
08:00:03 AM    258000   1052720     80.35    34000    510000   1577000     70.12

Use case 5: Reporting Run Queue Length and Load Averages

Code:

sar -q 1

Motivation:

The run queue length and load average are invaluable metrics for evaluating overall system load. These indicators help determine if a system is overtasked and whether CPU requests are being efficiently serviced. This data can assist in diagnosing systemic issues or in tuning the system for better performance.

Explanation:

• -q: Specifies to report on the system’s run queue and load average statistics, which gives insights into system responsiveness and potential bottlenecks.
• 1: Sets the frequency at which data is displayed to every second.

Example Output:

08:00:01 AM  runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15
08:00:02 AM        2       143      0.02      0.10      0.28
08:00:03 AM        3       145      0.03      0.11      0.29

Use case 6: Reporting Paging Statistics

Code:

sar -B 5

Motivation:

Paging is a process of managing memory and tracking page requests and faults. Excessive paging indicates memory pressure, which can slow down the system. By monitoring page-ins, page-outs, and faults, system administrators can identify if there is a need for more RAM or if application behavior needs adjusting.

Explanation:

• -B: Command option to report paging activity, detailing information about page-in/page-out operations and the swap system’s performance.
• 5: Indicates that reports should be generated every 5 seconds, allowing for close monitoring of memory paging activity.

Example Output:

08:00:01 AM pgpgin/s pgpgout/s   fault/s  majflt/s  pgfree/s pgscank/s pgscand/s pgsteal/s    %vmeff
08:00:06 AM      0.00      0.00     10.00      0.00    100.00      5.00     10.00      2.00      40.00
08:00:11 AM      1.00      0.50     12.00      0.00    120.00      6.00     12.00      3.00      50.00

Conclusion:

The ‘sar’ command is a versatile and powerful utility for system performance monitoring. By understanding and using its various options, system administrators can gain insights into critical performance metrics such as CPU usage, memory activity, network throughput, and more. These capabilities make it easier to diagnose issues, perform optimizations, and ensure the stability and efficiency of Linux systems. Whether dealing with routine maintenance or diagnosing unexpected performance issues, the ‘sar’ command’s detailed reports provide comprehensive data for making informed decisions.