How to use the command dd (with examples)

The dd command is an essential Unix utility primarily used for converting and copying files. Its simplicity yet potency makes it popular for tasks ranging from creating bootable drives to backing up entire systems. With its many options and flexibility, dd can perform a variety of disk-related operations efficiently.

Use case 1: Make a bootable USB drive from an isohybrid file (such as archlinux-xxx.iso) and show the progress

Code:

dd if=path/to/file.iso of=/dev/usb_drive status=progress

Motivation:

Creating a bootable USB drive is a fundamental task for system administrators and tech-savvy users alike. Whether for OS installation, recovery purposes, or even for customized user environments, bootable USB drives provide a portable, easy-to-use solution. Using dd for this operation ensures that the image is transferred bit by bit, maintaining bootability, and the ‘status=progress’ flag provides real-time feedback, crucial for such lengthy operations.

Explanation:

• if=path/to/file.iso: Specifies the input file, in this case, the ISO image to be written.
• of=/dev/usb_drive: Specifies the output file, referring to the target USB drive device path.
• status=progress: Provides real-time updates on the operation’s progress.

Example Output:

1020232+0 records in
1020232+0 records out
523248352 bytes (523 MB) copied, 92.1236 s, 5.7 MB/s

Use case 2: Clone a drive to another drive with 4 MB block, ignore error and show the progress

Code:

dd bs=4m conv=noerror if=/dev/source_drive of=/dev/dest_drive status=progress

Motivation:

Drive cloning is a critical aspect of data management and disaster recovery. Cloning drives can serve purposes like upgrading hardware, performing data migrations, or creating backups. Employing dd with no error handling ensures that the cloning process continues even if encounters bad sectors, thereby preserving whatever data can be retained. This use case is beneficial for transferring vast amounts of data reliably.

Explanation:

• bs=4m: Sets the block size to 4 megabytes, optimizing the speed of data transfer.
• conv=noerror: Instructs dd to continue operation despite errors, a useful flag when dealing with potentially failing drives.
• if=/dev/source_drive: Denotes the source drive to be cloned.
• of=/dev/dest_drive: Denotes the destination drive where data will be copied.
• status=progress: Provides continuous updates.

Example Output:

0+1 records in
0+1 records out
9469432 bytes (9.5 MB) copied, 1.456 s, 6.5 MB/s

Use case 3: Generate a file with a specific number of random bytes using the kernel random driver

Code:

dd bs=100 count=1 if=/dev/urandom of=path/to/random_file

Motivation:

Random byte file generation is useful in various scenarios like cryptographic operations, testing, and simulation tasks. This use case shows how to reliably generate a binary file filled with random data sourced from the kernel’s random number generator. This can be crucial for cryptography or creating mock datasets.

Explanation:

• bs=100: Sets the block size to 100 bytes.
• count=1: Designates that dd should output one block, ensuring a file with 100 random bytes.
• if=/dev/urandom: Uses /dev/urandom to source random data from the system.
• of=path/to/random_file: Specifies the destination file for the random bytes.

Example Output:

1+0 records in
1+0 records out
100 bytes copied, 0.000237234 s, 421 kB/s

Use case 4: Benchmark the write performance of a disk

Code:

dd bs=1024 count=1000000 if=/dev/zero of=path/to/1GB_file

Motivation:

Disk benchmarking is essential for evaluating system performance, especially for storage devices. Knowing how fast data can be written to disk under typical conditions can guide decisions in optimization and hardware upgrades. Using dd to write zeros to a file as a proxy for disk speed testing demonstrates its utility in diagnostics and performance testing.

Explanation:

• bs=1024: Sets the block size to 1024 bytes (1 KB).
• count=1000000: Repeats the operation one million times, leading to an approximately 1 GB file.
• if=/dev/zero: Uses /dev/zero, which provides an infinite source of zero bytes for the test.
• of=path/to/1GB_file: Designates the output path of the zero-filled file, effectively measuring disk write speed.

Example Output:

1000000+0 records in
1000000+0 records out
1024000000 bytes (1.0 GB) copied, 12.3456 s, 82.9 MB/s

Use case 5: Create a system backup, save it into an IMG file, and show the progress

Code:

dd if=/dev/drive_device of=path/to/file.img status=progress

Motivation:

Backups are a central component of IT infrastructure, helping in data recovery and continuity, especially when hardware fails. Creating disk images is an efficient way to preserve the state of a system. The ability to restore such an image allows for quick disaster recovery, thus minimizing downtime. Additionally, the progress status helps track backup processes, ensuring reliability and confidence that the operation is proceeding correctly.

Explanation:

• if=/dev/drive_device: Specifies the input source, usually a disk or partition to back up.
• of=path/to/file.img: Indicates the output file path where the disk image will be stored.
• status=progress: Provides updates regarding the backup operation’s progress, offering transparency.

Example Output:

15003248+0 records in
15003248+0 records out
769695232 bytes (7.7 GB) copied, 350.4560 s, 22.1 MB/s

Use case 6: Check the progress of an ongoing dd operation (run this command from another shell)

Code:

kill -USR1 $(pgrep ^dd)

Motivation:

Long-running dd operations often lack visibility regarding their progress, possibly leaving users uncertain about their completion time. While the ‘status=progress’ flag aids visibility, for older versions of dd or operations started without it, sending a signal to reveal current status is indispensable. This command provides reassurance that the operation is ongoing and gives a snapshot of its status without interruption.

Explanation:

• kill -USR1: Sends a USR1 signal to the dd process, instructing it to output current progress.
• $(pgrep ^dd): Uses pgrep to find the process ID of dd, ensuring the signal is sent to the correct process.

Example Output:

status update:
1048576+0 records in
1048576+0 records out
1073741824 bytes (1.1 GB) copied, 43.218 s, 24.8 MB/s

Conclusion:

The dd command, while simple in its essence, provides versatile functionality essential across a variety of system administration tasks, from data transfer to system recovery. With its wide array of options, it empowers users to handle data in a low-level, efficient manner, tailored to their specific needs and contexts.