Dd utility in Unix/Linux

dd (dataset definition) is a UNIX utility used for copying, converting files, and reading data. The name is inherited from the operator DD (Dataset Definition) from the JCL language.

As in UNIX, many objects (hard/floppy drives, COM/LPT ports, memory of computer, memory with code/data-each process) is presented in the form of special files, the application range of the utility dd is much broader than it seems at first glance. Regularly there is a need not just to copy a file or multiple files (what is the utility cp) and copy the first n bytes of the file, skip m bytes from the beginning, to read the file with the defective media to broadcast the contents of the file in ASCII, to reverse the order of the bytes in the file (Little-Endian vs. Big-Endian), just copy a very large file or all together. For this purpose, and serves as a dd.

Among other things, this utility allows you to copy the regions from the files of “raw” devices, for example, to backup the boot sector of the hard disk, or to read fixed chunks of data from special files like /dev/zero or /dev/random.

The name of the utility dd is sometimes jokingly decode as “disk destroyer”, “data destroyer”, “delete data” or “kill disk”, because the utility allows you to perform low-level operations on hard drives at the slightest mistake (such as reversing the if and of parameters) it is possible to lose part of the data on the disk (or even all). There are more “respectful” nickname “disk duplicator” because in practice, its main application are copies, images, and backups of partitions.

The syntax of the utility:

$ dd if= < filepath of=kudanovich drugiye

where:

  • filepath — the file Path to the image from where the data will be taken.
  • kudanovich — Where to copy the data (flash drive, disk, file)
  • drogear — Various usage parameters.

Possible parameters:

  • bs — This option specifies the block size (block size) to read and write data at a time. For example, you can specify 512, 1m, 8m. I recommend not to use big numbers, or preferably is a 4-8MB;
  • cbs — This parameter is used to specify how many bytes you need to record at once;
  • count — This option specifies how to copy blocks and the block size is set to bs option;
  • conv Option is to convert the file in accordance with the comma-separated list of symbols. Each character can be one of the following and represents a type conversion ascii, ebcdic, ibm, block, unblock, lcase, nocreat, excl, notrunc, ucase, swab, noerror, sync, fdatasync, fsync. A description of the data types will be lower.
  • ibs — This option specifies how many need to count bytes at a time (the Default value is 512 bytes);
  • obs — This option specifies how many need to write bytes at a time (the Default value is 512 bytes);
  • seek — This option specifies how many to skip the number of bytes (obs-sized blocks) at the beginning of the output;
  • skip — This option specifies how many to skip the number of bytes (ibs-sized blocks) at the beginning of the input;
  • status — Indicates how detailed you need to conclude;
  • iflag, oflag Allows you to specify additional flags work for the device input and output, basic of them are: nocache, nofollow.

Parameters conv:

  • ascii Conversion EBCDIC to ASCII.
  • ebcdic — the Conversion from ASCII to EBCDIC.
  • ibm Convert ASCII to alternate EBCDIC.
  • blockpad — Completion of new line records with spaces to cbs-size.
  • unblock Replace trailing spaces in cbs-size records with the new string.
  • lcase Change upper case to lower case.
  • nocreat Not sozdavayte output file.
  • excl — Error if the output file already exists.
  • notrunc — do Not truncate the output file.
  • ucase is to Replace the lower case on the top.
  • swab — Swap every pair of input bytes.
  • noerror — Continue after read errors.
  • sync — Adds each input block values of NUL to ibs-size; when used with block or unblock, use the block with spaces, not NUL.
  • fdatasync Physically writes the data to the output file until the end of the process.
  • fsync is Similar, but also are written to the metadata.

The parameters for the iflag, oflag:

  • append append mode (makes sense only for output; conv=notrunc offer).
  • direct — Use direct input/output (I/O) for the data.
  • directory Issues a fail if you use the directory.
  • dsync — Use synchronization I/O for data.
  • sync likewise, but also for metadata.
  • fullblock — Accumulate full blocks of input (iflag only).
  • nonblock — Uses non-blocking input/output (I/O).
  • noatime — does Not update the access time (often used).
  • noctty — do Not assign controlling terminal from file.
  • nofollow — does Not pass via the symbolic links.

For count, seek, skip, bs, cbs, ibs, obs, you can use the following dimension:

  • c=4
  • w=8
  • b=512
  • kB=1000
  • K=1024
  • MB=1000*1000
  • M=1024*1024
  • xM=M
  • GB=1000*1000*1000
  • G=1024*1024*1024

Examples of dd utility in Unix/Linux

Let’s start with the most simple — to help:

$ dd --help

Note: in Unix (for example on MacOS) may not work, this has the following solution:

$ man dd

Learn version is available here:

$ dd --version

Note: it Works on Linux. On Unix, probably will not work.

-=== EXAMPLE 1 ===-

To record the image on a flash drive or disk, you can use:

$ sudo dd if=2019-04-08-raspbian-stretch-lite.img of=/dev/disk3 bs=1m

-=== EXAMPLE 2 ===-

If you want to copy the disk to another, for example, you can do it as follows:

# dd if=/dev/disk2 of=/dev/disk3 bs=4096 conv=noerror,sync

-=== EXAMPLE 3 ===-

Also, it is sometimes useful to compress data and write them down somewhere (Tested only with gzip, bzip2). For example, you want to make a backup. Use bzip2 compression:

# dd if =/dev/disk2 | bzip2 disk2.img.gz

Or, use gzip:

$ sudo dd if=/dev/disk3 | gzip -c > /Users/captain/raspberrypi.img.gz
Password:
250347520+0 records in
250347520+0 records out
128177930240 bytes transferred in 6767.601942 secs (18939933 bytes/sec)

Do not forget to combine options (can be used in this case: bs=8m”, ” status=progress, conv=fsync)

PS: for R you can use:

# gzip -dc /Users/captain/raspberrypi.img.gz | dd of=/dev/disk2

You can remotely create a backup, for example:

# ssh [email protected]_ssh_host "dd if=/dev/disk2 | gzip -1 -" | dd of=your_backup_here.gz

As another solution, you can do backup at the specified time and copy it over the network. For example:

# dd if=/dev/disk2 | ssh [email protected]_ssh_host "dd of=/Users/captain/raspberrypi.img.gz".

PS: I Should add SSH keys ssh will work this scheme

-=== EXAMPLE 4 ===-

To format the drive low-level way using dd, example looks like this:

# dd if=/dev/zero of=/dev/disk2

PS: you Can reset only a certain number of bytes, for example:

# dd if=/dev/zero of=/home/captain/the_file_here bs=512 count=1 conv=notrunc

You can check the same dd, but converting the data in hex:

# dd if=/dev/disk2 | hexdump -C

Must be 0. Or, you can check the other method:

# dd if=/dev/disk2 | hexdump -C | grep [^00] 

It is possible to test, take and to copy 10 GB of zeros and redirect them to /dev/null (nowhere):

# dd if=/dev/zero of=/dev/null bs=100M count=100

Destroy the superblock, you can:

# dd if=/dev/zero count=1 bs=1024 seek=1 of=/dev/disk2 

or another solution:

# dd if=/dev/zero count=1 bs=4096 seek=0 of=/dev/disk2 

To clear the first 113 MB partition:

# dd if=/dev/zero of=/dev/disk2
 bs=113m count=1

-=== EXAMPLE 5 ===-

MBR located in the first 512 bytes of the hard disk, and consists of the partition table, bootloader and a few extra bytes. Sometimes, it is necessary bekapit, restore etc Backup you can do this:

# dd if=/dev/disk2 of=just_mbr_in.img bs=512 count=1

To recover, use:

# dd if=just_mbr_in.img of=/dev/disk2

If you want to make a backup boot data MBR excluding the partition table, example here:

# dd if=/dev/disk2 of=/tmp/disk2_mbr2.img bs=446 count=1

Back:

# dd if=/home/test/just_mbr_in.img of=/dev/disk2 bs=446 count=1

To copy the MBR to a floppy disk:

# dd if=/dev/disk2 of=/dev/fd0 bs=512 count=1

View MBR and using the dd utility:

# dd if=/dev/disk2 count=1 | hexdump -C

-=== EXAMPLE 6 ===-

Using dd to generate a file, and then use them as containers for other file systems, even in encrypted form. Technology is as follows:
Using dd creates a file full of zeros (with random numbers score is not rational: a long and pointless):

# dd if=/dev/zero of=your_image_is.crypted bs=1M count=1000

To fill the disk random values:

# dd if=/dev/random of=/dev/disk2

Overwrite filedisk n-times, so (in my case, will be overwritten 20 times):

# for i in {1..20};do dd if=/dev/random of=/dev/disk2; done

-=== EXAMPLE 7 ===-

Create ISO from CD/DVD ROM and the local storage (file on your disk):

# dd if=/dev/cdrom of=/your_image_from_cdrom.iso

To create a partition (useful for backup):

# dd if=/dev/disk2 of=/your_disk2_1.img
 bs=4M count=4430
# dd if=/dev/disk2 of=/your_disk2_2.img
 bs=4M count=8860
[...]

After such copying, you can expand it all back as follows:

# dd if=/your_disk2_1.img of=/dev/disk2
bs=8M
# dd if=/your_disk2_2.img of=/dev/disk2
 seek=4430 bs=8M
# dd if=/your_disk2_3.img of=/dev/disk2
 seek=8860 bs=8M

To create an ISO of Florica is possible here so:

# dd if=/dev/fd0 of=~/floppy_here.img

-=== EXAMPLE 8 ===-

To convert the data format of the file( for example EBCDIC to ASCII):

# dd if=textfile.ebcdic of=textfile.ascii conv=ascii

To convert the format of the data file( e.g. ASCII to EBCDIC):

# dd if=textfile.ascii of=textfile.ebcdic conv=ebcdic

-=== EXAMPLE 9 ===-

For example you have a file with the text:

# cat file_with_lowercase.txt
test dd convert

If you need to convert the contents of a file to uppercase:

# dd if=~/file_with_lowercase.txt of=~/file_with_uppercase.txt conv=ucase

Another example:

# ls-l | dd conv=ucase

If you need to convert file contents to lower case:

# dd if=~/file_with_uppercase.txt of=~/file_with_lowercase.txt conv=lcase

-=== EXAMPLE 10 ===-

To view your virtual memory, use the command:

# dd if=/proc/kcore | hexdump -C | less

-=== EXAMPLE 11 ===-

To check what filesystem is installed, run:

# dd if=/proc/filesystems | hexdump -C | less

-=== EXAMPLE: 12 ===-

Check all loaded modules using dd:

# dd if=/proc/kallsyms | hexdump -C | less

-=== EXAMPLE 13 ===-

To watch the interrupt table, command is:

# dd if=/proc/interrupts | hexdump -C | less

-=== EXAMPLE 14 ===-

To check how many seconds the system worked:

# dd if=/proc/uptime | hexdump -C | less

-=== EXAMPLE 15 ===-

To check the partitions and also, their sizes (in KB), use:

# dd if=/proc/partitions | hexdump -C | less

-=== EXAMPLE 16 ===-

Output of statistics memory:

# dd if=/proc/meminfo | hexdump -C | less

-=== EXAMPLE 17 ===-

A good way to verify the presence of the detectors on the device with the dd command:

# dd if=/dev/disk2 of=/dev/null bs=1m 

-=== EXAMPLE 17 ===-

It is also possible with this utility to check the file for viruses (but needed for this, you need to install ClamAV):

# dd if=/home/captain/your_file_for_checking.gz | clamscan -

Link to ClamAV:

To install ClamAV on Debian/Ubuntu/ Linux Mint

Install ClamAV on a RedHat/Centos/Fedora

-=== EXAMPLE 18 ===-

Test the speed of read/write hard drives:

# dd if=/home/your_big_file_here of=/dev/null
# dd if=/dev/zero of=/home/your_big_file_here
bs=1024 count=1000000

-=== EXAMPLE 19 ===-

Output the file to stdout like:

# dd if=/home/your_file_here

-=== EXAMPLE 20 ===-

Found another example with which you can search for a string throughout the tab (Even if imposed security), you can use a LiveCD and boot from it and execute:

# dd if=/dev/disk2 bs=16065 | hexdump -C
 | grep 'search text string here'

-=== EXAMPLE 21 ===-

To read the BIOS:

# dd if=/dev/mem bs=1k skip=768 count=256 
2>/dev/null | strings -n 8

-=== EXAMPLE 22 ===-

Found a funny example:

# echo-n "hello man" | dd cbs=1 
> conv=unblock 2> /dev/null
h
e
l
l
o

m
a
n

-=== EXAMPLE 23 ===-

To remove/ignore the first 111 bytes of the standard input:

# dd ibs= 111 skip=1

-=== 24 ===-

Make a quick backup over the network using Netcat:

# dd if=/dev/disk2 | nc-l 10001 
nc $system_to_backup_IP 10001 | dd
 of=sysbackupsda.img 

-=== EXAMPLE 25 ===-

Create a temporary paging space:

# dd if=/dev/zero of=tmpswap bs=1k
count=1000000
# chmod 600 tmpswap
# mkswap tmpswap
# swapon tmpswap

-=== EXAMPLE 26 ===-

Determine the speed of sequential I / o your drive. Reading 1 GB file:

# dd if=/dev/disk2 of=/dev/null bs=1024k 
count=1024

-=== EXAMPLE 27 ===-

You can add a progress bar, for example:

# dd bs=4M if=2019-04-08-raspbian-stretch-lite.img | pv | of=/dev/disk3 conv=fsync

Or:

# sudo dd if=2019-04-08-raspbian-stretch-lite.img of=/dev/disk3 bs=1m status=progress conv=fsync

That’s it for me, “the dd Utility in Unix/Linux” is completed.

Source: linux-notes.org

(Visited 90 times, 1 visits today)