AWS S3 Command Examples (with examples)

Introduction:

AWS S3 (Simple Storage Service) provides storage through web services interfaces. In this article, we will explore eight different use cases of the aws s3 command, which is the AWS CLI for S3. These examples will demonstrate various functionalities, such as listing files in a bucket, syncing files between local and bucket, removing files from a bucket, and more. By understanding these use cases, you will be able to efficiently manage S3 buckets using the AWS CLI.

Prerequisites:

Before proceeding, make sure you have the AWS CLI installed and configured on your system. You can find installation instructions and configuration details in the AWS CLI User Guide.

Use Case 1: Show files in a bucket

To view the files present in an S3 bucket, you can use the following command:

aws s3 ls bucket_name

Motivation: This command is useful when you want to get a glimpse of the files stored in a specific bucket without downloading or syncing them.

Arguments:

• bucket_name: The name of the bucket you want to list the files from.

Example Output:

2022-01-01 15:30:00   1000 file1.txt
2022-01-01 15:31:00   2000 file2.jpg
2022-01-01 15:32:00   3000 file3.png

The output displays the last modified timestamp, file size, and file name for each file in the bucket.

Use Case 2: Sync files and directories from local to bucket

To synchronize files and directories from your local system to an S3 bucket, use the sync command as follows:

aws s3 sync path/to/files s3://bucket_name

Motivation: This use case is helpful when you want to upload or backup local files to your S3 bucket, ensuring that both the source and destination remain synchronized.

Arguments:

• path/to/files: The path to the local files or directory that you want to sync.
• bucket_name: The name of the S3 bucket where you want to sync the files.

Example Output:

upload: path/to/files/file1.txt to s3://bucket_name/file1.txt
upload: path/to/files/file2.jpg to s3://bucket_name/file2.jpg
upload: path/to/files/file3.png to s3://bucket_name/file3.png

The output shows the uploaded files and their corresponding paths in the bucket.

Use Case 3: Sync files and directories from bucket to local

To sync files and directories from an S3 bucket to your local system, use the sync command with reversed arguments:

aws s3 sync s3://bucket_name path/to/target

Motivation: This use case is useful when you want to retrieve files or create local backups from an S3 bucket to your local system while maintaining synchronization between the source and destination.

Arguments:

• bucket_name: The name of the S3 bucket from which you want to sync the files.
• path/to/target: The location on your local system where you want to sync the files.

Example Output:

download: s3://bucket_name/file1.txt to path/to/target/file1.txt
download: s3://bucket_name/file2.jpg to path/to/target/file2.jpg
download: s3://bucket_name/file3.png to path/to/target/file3.png

The output shows the downloaded files and their corresponding paths on the local system.

Use Case 4: Sync files and directories with exclusions

While syncing files and directories, you can exclude specific files or directories using the --exclude flag. Here’s a command example:

aws s3 sync path/to/files s3://bucket_name --exclude path/to/file --exclude path/to/directory/*

Motivation: This use case is helpful when you want to sync files and directories but exclude certain items that don’t need to be synchronized.

Arguments:

• path/to/files: The path to the local files or directory that you want to sync.
• bucket_name: The name of the S3 bucket where you want to sync the files.
• path/to/file: The specific file that you want to exclude from syncing.
• path/to/directory: The directory that you want to exclude along with its contents.

Example Output:

...

The output will display the synced files and directories after excluding the specified file and directory.

Use Case 5: Remove file from bucket

To remove a file from an S3 bucket, you can use the rm command as follows:

aws s3 rm s3://bucket_name/path/to/file

Motivation: This command is useful when you want to delete a specific file from an S3 bucket.

Arguments:

• bucket_name: The name of the S3 bucket from which you want to remove the file.
• path/to/file: The path to the file that you want to remove.

Example Output:

delete: s3://bucket_name/path/to/file

The output confirms that the specified file has been successfully deleted from the bucket.

Use Case 6: Preview changes only

If you want to preview the changes without actually executing the command, you can use the --dryrun option. This option works with any aws s3 command. Here’s an example:

aws s3 any_command --dryrun

Motivation: This option allows you to preview the potential changes that would occur if the command were executed. It assists in verifying your intentions before making actual modifications to your S3 bucket.

Arguments:

• any_command: Any valid aws s3 command that you want to preview.

Example Output:

...

The output will display the expected changes or modifications that would occur without actually making any modifications to the bucket.

Conclusion:

In this article, we explored eight different use cases of the aws s3 command. These examples provide a comprehensive understanding of various S3 management tasks, such as listing files, syncing between local and bucket, removing files, and more. By leveraging the AWS CLI and these examples, you can efficiently manage your S3 buckets, ensuring effective file synchronization and maintenance.