21

There is probably simple trick to do this, but I can't figure from man page.

How do I cut last 1MB from file with undetermined size for example, by using dd?

zetah
  • 2,057

3 Answers3

40

Well, assuming you have stat and bash, you can get the file size with:

stat -c %s your_file

If you want to extract the last $amount bytes for that file with dd, you could:

dd if=your_file of=extracted_part \
   bs=1 count=$amount \
   skip=$(( $(stat -c %s your_file) - $amount ))

But the saner approach would be to use tail:

tail -c $(( 1024*1024 )) your_file > target_file
Mat
  • 52,586
  • No -c option for head? – ADTC Dec 31 '14 at 03:54
  • I think he meant +$(( 1024*1024 )) : -c, --bytes=K output the last K bytes; alternatively, use -c +K to output bytes starting with the Kth of each file – Vanuan Jul 27 '18 at 14:21
  • While tail might look saner, using dd (with bs=1k) saved my day when I had to cut out last few megabytes of 49 GB log file -- it was done instantly, and tail was never able to finish because it had to read an discard too much data. – alx Dec 13 '21 at 06:39
5
dd --help
Usage: dd [OPERAND]...
  or:  dd OPTION
Copy a file, converting and formatting according to the operands.

  bs=BYTES        read and write BYTES bytes at a time (also see ibs=,obs=)
  cbs=BYTES       convert BYTES bytes at a time
  conv=CONVS      convert the file as per the comma separated symbol list
  count=BLOCKS    copy only BLOCKS input blocks
  ibs=BYTES       read BYTES bytes at a time (default: 512)
  if=FILE         read from FILE instead of stdin
  iflag=FLAGS     read as per the comma separated symbol list
  obs=BYTES       write BYTES bytes at a time (default: 512)
  of=FILE         write to FILE instead of stdout
  oflag=FLAGS     write as per the comma separated symbol list
  seek=BLOCKS     skip BLOCKS obs-sized blocks at start of output
  skip=BLOCKS     skip BLOCKS ibs-sized blocks at start of input
  status=noxfer   suppress transfer statistics

BLOCKS and BYTES may be followed by the following multiplicative suffixes:
c =1, w =2, b =512, kB =1000, K =1024, MB =1000*1000, M =1024*1024, xM =M
GB =1000*1000*1000, G =1024*1024*1024, and so on for T, P, E, Z, Y.

If the file size is exactly 10MB, 1024*10 10240K This will leave the last 1024K. You should specify the block size you are working with using the ibs and obs options. 

1M = 1024K 
1024*9 = 9216  

dd if=/10/MB/file of=/9/MB/file count=9216K ibs=1K obs=1K
dd if=/10/MB/file of=/9/MB/file count=9M ibs=1M obs=1M

You can also skip the first 1MB of a file, using the skip option to read to the end of the file skipping the first 1MB. 

dd if=/10/MB/file of=/9/MB/file skip=1M ibs=1M obs=1M

Using the seek option you can write a to a specific place in your output file. Say you want to keep the first 1MB and over write the last 8MB.

dd if=/10/MB/file of=/9/MB/file skip=1M seek=1M count=8M ibs=1M obs=1M

You probably need to get some details about your file size to make sure you get the right amount of data in and out. 

ls -s --block-size 1K ./my/10MB/file

man ls

       --block-size=SIZE
              use SIZE-byte blocks.  See SIZE format below

       -s, --size
              print the allocated size of each file, in blocks

       SIZE  may  be (or may be an integer optionally followed by) one of fol‐
       lowing: KB 1000, K 1024, MB 1000*1000, M 1024*1024, and so on for G, T,
       P, E, Z, Y.
Mat
  • 52,586
nelaaro
  • 1,333
0

To truncate a file without copying it using dd, you want to set the input file to /dev/null and let dd truncate the output (target) file at the seek point. For example:

dd if=/dev/null of=somefile.txt bs=1M seek=10

will truncate somefile.txt at 10MB.

However, because your file is of variable length and you want to remove a fixed amount from the end, look at the truncate command. This works much more simply as truncate <file> --size=X where X can be an absolute or relative amount. If the new size is larger, it will also expand a file (using a hole). See:

$ dd if=/dev/urandom of=somefile.txt bs=1M count=13
13631488 bytes (14 MB, 13 MiB) copied, 0.0661225 s, 206 MB/s
$ ls -lh somefile.txt 
-rw-rw-r-- 1 13M Sep 28 20:18 somefile.txt
$ truncate -s -3M somefile.txt  # remove the trailing 3MB
$ ls -lh somefile.txt 
-rw-rw-r-- 1 10M Sep 28 20:18 somefile.txt

The same using dd, but without a relative size:

$ dd if=/dev/urandom of=somefile.txt bs=1M count=13
13631488 bytes (14 MB, 13 MiB) copied, 0.0669367 s, 204 MB/s
$ ls -lh somefile.txt 
-rw-rw-r-- 1 13M Sep 28 20:21 somefile.txt
$ dd if=/dev/null of=somefile.txt bs=1M seek=10
0 bytes copied, 3.6978e-05 s, 0.0 kB/s
$ ls -lh somefile.txt 
-rw-rw-r-- 1 10M Sep 28 20:22 somefile.txt
rand'Chris
  • 141