How to use the command 'sacct' (with examples)

The ‘sacct’ command is used to display accounting data from the Slurm service. It provides information about the jobs that have been submitted and their associated attributes such as job id, job name, partition, account, number of allocated CPUs, job state, and job exit codes.

Use case 1: Display job id, job name, partition, account, number of allocated CPUs, job state, and job exit codes for recent jobs

Code:

sacct

Motivation: This use case is useful when you want to get an overview of recent jobs and their important attributes. It can help you track the progress of jobs, identify any issues, and gather statistics for analysis.

Explanation: The command ‘sacct’ without any arguments will display accounting data for recent jobs. It provides a tabular view of job id, job name, partition, account, number of allocated CPUs, job state, and job exit codes.

Example output:

JobID    JobName    Partition    Account    AllocCPUS    State    ExitCode
-------  ---------  -----------  ---------  -----------  -------  --------
1        job1       compute      user1      4            COMPLETED  0:0
2        job2       compute      user2      8            FAILED     1:0
3        job3       gpu          user3      2            PENDING

Use case 2: Display job id, job state, job exit code for recent jobs

Code:

sacct --brief

Motivation: This use case is helpful when you only need a concise overview of the recent jobs and their state and exit codes. It provides a more condensed output compared to the default ‘sacct’ command.

Explanation: The ‘–brief’ option is used to display a shorter form of output. It only includes the job id, job state, and job exit code for recent jobs.

Example output:

JobID    State    ExitCode
-------  -------  --------
1        COMPLETED 0:0
2        FAILED    1:0
3        PENDING

Use case 3: Display the allocations of a job

Code:

sacct --jobs job_id --allocations

Motivation: This use case is useful when you want to view detailed information about the resource allocations of a specific job. It provides insights into how the job’s resources were utilized.

Explanation: The ‘–jobs’ option allows you to specify a job id for which you want to view allocations. The ‘–allocations’ option is used to display the allocations of the specified job.

Example output:

JobID  AllocCPUS  AllocNodes  ReqCPUS  ReqNodes  NCPUS
------ ---------- ----------- -------- --------- -----
1      4          1           4        1         4

Use case 4: Display elapsed time, job name, number of requested CPUs, and memory requested of a job

Code:

sacct --jobs job_id --format=Elapsed,JobName,ReqCPUS,ReqMem

Motivation: This use case is beneficial when you need specific information about a job, such as elapsed time, job name, number of requested CPUs, and memory requested. It allows you to quickly gather important details about a specific job.

Explanation: The ‘–format’ option specifies the attributes you want to display. In this use case, we are interested in ‘Elapsed’ (elapsed time), ‘JobName’ (job name), ‘ReqCPUS’ (number of requested CPUs), and ‘ReqMem’ (memory requested) attributes.

Example output:

Elapsed     JobName    ReqCPUS  ReqMem
----------  ---------  -------- -------
00:28:57    job1       4        16G

Use case 5: Display recent jobs that occurred from one week ago up to the present day

Code:

sacct --starttime=$(date -d "1 week ago" +'%F')

Motivation: This use case allows you to filter and display job information for a specific time range. For example, you can find all the jobs that have occurred in the past week. It can help you analyze job trends, monitor job activities, and identify any issues.

Explanation: The ‘–starttime’ option is used to specify the start time for filtering jobs. In this use case, we are using the ‘date’ command to calculate the start time as one week ago, and then passing it to the ‘–starttime’ option.

Example output:

JobID    JobName    Partition    Account    AllocCPUS    State    ExitCode
-------  ---------  -----------  ---------  -----------  -------  --------
1        job1       compute      user1      4            COMPLETED  0:0
2        job2       compute      user2      8            FAILED     1:0
3        job3       gpu          user3      2            PENDING

Use case 6: Output a larger number of characters for an attribute

Code:

sacct --format=JobID,JobName%100

Motivation: This use case is useful when you want to increase the number of characters displayed for a specific attribute. It can help accommodate longer job names that would otherwise be truncated in the default output.

Explanation: The ‘–format’ option is used to specify the format of the output. In this use case, the ‘JobName%100’ format specifies that the ‘JobName’ attribute should be displayed with a maximum width of 100 characters.

Example output:

JobID    JobName
-------  ----------------------------------------------------------------------------------------------------
1        a_very_long_job_name_that_would_be_truncated_in_default_output_exceeding_100_characters_goes_here
2        shorter_name
3        another_long_job_name_that_would_be_truncated

Conclusion:

The ‘sacct’ command is a powerful tool for displaying accounting data from the Slurm service. It offers various options to filter and format the output, allowing users to gather specific information about jobs that have been submitted. By utilizing the different use cases discussed in this article, users can effectively monitor job status, track resource utilization, and analyze job trends.