How to List Amazon S3 Buckets and Objects using AWS CLI (with examples)

The AWS Command Line Interface (CLI) provides a unified tool to manage your AWS services. The aws s3 ls command is a versatile tool within the AWS CLI that allows you to list your Amazon Simple Storage Service (S3) buckets, folders within those buckets, and files (objects) stored in those folders. By using different arguments with the aws s3 ls command, you can retrieve varying levels of detail about the contents within your S3 storage service.

Use case 1: List All Buckets

Code:

aws s3 ls

Motivation:

This command is essential for AWS users who manage multiple S3 buckets. It provides a quick overview of all the buckets available in your AWS account, making it easy to verify and organize your cloud storage resources.

Explanation:

• aws: The AWS CLI command to interact with AWS services.
• s3: The sub-command used specifically for actions related to Amazon S3.
• ls: A shorthand for ’list,’ this command lists the resources.

Example Output:

2019-10-25 12:45:34 bucket-one
2020-02-14 17:21:30 project-documents
2020-06-11 09:54:12 photos-bucket

Use case 2: List Files and Folders in the Root of a Bucket

Code:

aws s3 ls s3://bucket_name

Motivation:

When you need to quickly assess the contents of an S3 bucket, listing the root files and folders helps provide an immediate insight into the structure and organization of resources within that specific bucket. This can be particularly useful for confirming the existence of primary directories or main files.

Explanation:

• s3://bucket_name: Specifies the S3 bucket. The s3:// prefix is an indication that you are referring to a resource within S3.

Example Output:

2021-03-27 15:00:19        365 file1.txt
2021-04-14 11:05:30        689 folder1/
2021-04-21 09:30:12      23109 folder2/

Use case 3: List Files and Folders Directly Inside a Directory

Code:

aws s3 ls bucket_name/path/to/directory/

Motivation:

For users managing complex file structures within a bucket, this command is useful to drill down into specific directories and understand their contents. This is particularly beneficial for developers or administrators who need to organize files or track updates in certain folders.

Explanation:

• bucket_name/path/to/directory/: The path specifies the exact location within the bucket, allowing you to list only the desired subdirectory.

Example Output:

2021-07-22 18:20:45        128 file2.txt
2021-07-28 08:40:55        512 file3.txt
2021-08-02 09:51:35      20480 another_folder/

Use case 4: List All Files in a Bucket

Code:

aws s3 ls --recursive bucket_name

Motivation:

When performing audits or backups, it is often necessary to have a complete view of all objects within a bucket. This command provides a comprehensive listing of every file, regardless of how deeply nested they are within the directory structure.

Explanation:

• --recursive: A flag that enables listing of all files under the specified bucket, including those nested in directories.

Example Output:

2021-09-12 11:25:02        134 file4.txt
2021-09-12 11:35:43        760 folder3/file5.txt
2021-09-12 11:40:21       1024 folder3/subfolder1/file6.txt

Use case 5: List All Files in a Path with a Given Prefix

Code:

aws s3 ls --recursive bucket_name/path/to/directory/prefix

Motivation:

This is particularly useful for filtering out large datasets or logs where naming conventions adhere to a pattern. It helps in focusing on specific resources within a bucket by only listing items that start with the specified prefix.

Explanation:

• path/to/directory/prefix: The combination provides a path and a prefix to filter the results, effectively narrowing down the list of objects to those that match the initial string pattern.

Example Output:

2021-10-10 17:45:10        543 prefix-file7.txt
2021-10-11 14:01:13       1923 prefix-folder4/prefix-file8.txt

Use case 6: Display Help

Code:

aws s3 ls help

Motivation:

Accessing help directly within the CLI is a convenient way to understand command usage, especially for beginners or when using aws s3 ls complex arguments. It ensures users can quickly refer to syntax and functionality queries without needing external resources.

Explanation:

• help: When appended to any AWS CLI command, this displays detailed information about the command’s syntax, options, and usage scenarios directly in the terminal.

Example Output:

NAME
    ls - List S3 objects and common prefixes under a prefix or all ...

SYNOPSIS
    aws s3 ls [<S3Uri>] [--recursive] [--page-size <value>] ...
...

Conclusion

The aws s3 ls command is a powerful tool in the AWS CLI arsenal that provides users with the ability to efficiently navigate and manage their S3 resources. By understanding and utilizing the different use cases, AWS users can enhance their productivity and effectively organize their cloud storage assets. Whether it’s getting a list of buckets, examining directory contents, or filtering through large data sets, aws s3 ls offers various ways to customize the listing experience to meet specific needs.