How to use the command 'dd' (with examples)

The dd command is a versatile and powerful tool found in Unix and Unix-like operating systems. It is primarily used for converting and copying files or data streams from one location to another. Despite its somewhat cryptic syntax, dd is a favorite among many system administrators and developers due to its capability to perform low-level operations on file and file systems, such as cloning disks or backing up data. Below, we will explore various use cases for the dd command through practical examples.

Use case 1: Make a bootable USB drive from an isohybrid file

Code:

dd if=path/to/file.iso of=/dev/usb_drive status=progress

Motivation:

Creating bootable USB drives is a common requirement, especially when installing new operating systems or running live environments. With dd, you can convert an ISO image into a bootable USB drive easily. The use of status=progress gives real-time updates, making it user-friendly amidst the otherwise opaque process.

Explanation:

• if=path/to/file.iso: Specifies the input file, which is the ISO image you wish to write to the USB.
• of=/dev/usb_drive: Specifies the output file, which in this case is the target USB device. It is crucial to ensure this points to the correct device to avoid data loss.
• status=progress: Provides periodic updates on the transfer progress, showing the number of bytes copied.

Example Output:

104857600 bytes (105 MB, 100 MiB) copied, 2 s, 52.2 MB/s

Use case 2: Clone a drive to another drive with 4 MiB block size

Code:

dd bs=4M conv=fsync if=/dev/source_drive of=/dev/dest_drive

Motivation:

Cloning drives is often necessary for backup purposes, system migration, or duplication. Using dd, a perfect bit-for-bit copy of a storage device can be made. The bs=4M setting optimizes the copying speed, and conv=fsync ensures data integrity by flushing writes to disk before completion.

Explanation:

• bs=4M: Sets the block size to 4 mebibytes, which can improve performance by reducing overhead from multiple read/write operations.
• conv=fsync: Forces dd to physically write data from memory to disk before completing. This ensures that all writes are safely committed.
• if=/dev/source_drive: Indicates the source drive for cloning.
• of=/dev/dest_drive: Designates the target drive to which data will be cloned.

Example Output:

10+0 records in  
10+0 records out  
41943040 bytes (42 MB, 40 MiB) copied, 4 s, 10.6 MB/s

Use case 3: Generate a file with a specific number of random bytes

Code:

dd bs=100 count=1 if=/dev/urandom of=path/to/random_file

Motivation:

Generating random bytes is useful for creating keys, tokens, or random data files for testing purposes. By leveraging the randomness from urandom, dd can generate a file with cryptographically secure random data.

Explanation:

• bs=100: Sets the block size to 100 bytes.
• count=1: Indicates that only one block of the specified size should be written, resulting in a 100-byte file.
• if=/dev/urandom: Uses the kernel’s pseudorandom number generator as an input source.
• of=path/to/random_file: Defines where the output file containing the random data will be stored.

Example Output:

1+0 records in  
1+0 records out  
100 bytes copied, 0.00023 s, 434 kB/s

Use case 4: Benchmark the write performance of a disk

Code:

dd bs=1M count=1024 if=/dev/zero of=path/to/file_1GB

Motivation:

Understanding a disk’s write performance is crucial for optimization and evaluation. By writing a large block of data from /dev/zero, you can easily measure throughput and evaluate performance in a controlled environment.

Explanation:

• bs=1M: Sets the block size to 1 mebibyte for large, efficient writes.
• count=1024: Combines to create a 1 GiB file by writing 1024 blocks of the defined size.
• if=/dev/zero: Uses a special file that supplies null bytes, creating a consistent data stream.
• of=path/to/file_1GB: Indicates where the 1 GiB test file will be generated.

Example Output:

1024+0 records in  
1024+0 records out  
1073741824 bytes (1.1 GB) copied, 8.00145 s, 134 MB/s

Use case 5: Create a system backup as an IMG file

Code:

dd if=/dev/drive_device of=path/to/file.img status=progress

Motivation:

Having a complete backup of your system in an IMG file format is a safeguard against data loss, corruption, or hardware failure. This file can easily be restored, ensuring data continuity and reducing downtime.

Explanation:

• if=/dev/drive_device: References the complete drive to be backed up.
• of=path/to/file.img: Points to the IMG file where the backup will be stored.
• status=progress: Provides real-time feedback on the progress of the backup.

Example Output:

256000000 bytes (256 MB, 244 MiB) copied, 5.012 s, 51.1 MB/s

Use case 6: Check the progress of an ongoing dd operation

Code:

kill -USR1 $(pgrep -x dd)

Motivation:

The dd command does not provide feedback by default. Using this method allows users to manually check the progress of a dd operation in a different terminal.

Explanation:

• kill -USR1: Sends a user-defined signal to the specified process.
• $(pgrep -x dd): Finds the process ID of the dd command to which the signal should be sent.

Example Output:

123456+0 records in  
123456+0 records out  
12345600 bytes (12.3 MB) copied, 1.234 s, 10.0 MB/s

Conclusion:

The dd command is a comprehensive utility that can perform a wide range of tasks from creating bootable drives to cloning disks and benchmarking storage performance. Each example provided underscores its power and versatility, making dd an indispensable tool for tasks requiring precise data manipulation.