3

I found a few examples that claimed this but I could not manage to get dd to convert a file with a given block size to double that block size.

dd if=disk256bytesectors.img of=disk512bytesectors.img cbs=256 ibs=512 obs=512 conv=sync

My disk img is 10 megabytes in size I was expecting a new image of 20 megabytes, but no.

So what I need to happen is each 256 byte block get converted to a 512 byte block with the second half of each 512 byte block being null, zero, space or anything.

The dd manual state conv=sync

pads any input block shorter than ibs to that size with null bytes before conversion and output.

But the input block will never be shorter than itself?!?! So what does that mean?

Anyway how can I do this?

Walter ZAMBOTTI
  • 145
  • 4

4 Answers4

4

You might be able to use socat which uses a Unix domain socket when you specify exec: for a command. This can be made of type datagram (type=2) to ensure the input to dd is only 256 bytes.

For example, for 2 bytes per datagram (-b2) padded to 4 (bs=4):

$ echo abcdefx | socat -u -b2 - exec:'dd bs=4 conv=sync',type=2 | od -c   
0000000   a   b  \0  \0   c   d  \0  \0   e   f  \0  \0   x  \n  \0  \0

Note, the above use of a pipe for stdin to socat is just for this test. Normally, you should provide the input file directly on stdin so that socat has no trouble reading whole "blocks" of the wanted -b size (i.e. <file socat -u ...).

meuh
  • 51,383
  • Seems to work. I tested with -b256 and bs=512, the output was identical to what bbe had given me. The manual says -b sends at most such-and-such number of bytes per step. When dd requests 512 bytes, will it always get exactly 256? If for any reason it gets less then the output will go out of sync. Does type=2 guarantee this exactness maybe? – Kamil Maciorowski May 06 '20 at 20:35
  • type=2 is a sort of "packet" mode, where each write made towards dd keeps its structure, so a dd read larger than the packet will only get one packet, thus preserving the "structure". If you provide your file as stdin to socat (as in <myfile socat -u ...), then socat will always be able to read the 256 bytes it is asked to. If you use a pipe, as in my example, and the command you are using to fill the pipe is only sending a few bytes at a time, socat might not get the full 256 bytes, and so will send a short packet to dd. So just ensure you provide your file as stdin directly. – meuh May 06 '20 at 21:14
  • I knew socat is a powerful tool but I didn't know this trick. Thanks. +1. – Kamil Maciorowski May 06 '20 at 21:21
2

Instead of using dd I would use a binary editor,

bbe is available in Ubuntu's repository universe, and can be installed with

sudo apt install bbe

I think it is available in most Linux distros. See man bbe for details about the command line in the script below.

#!/bin/bash

if ! test -f 256dots
then
 for i in {1..256};do echo -n '.';done > 256dots
fi

filler=$(cat 256dots)

bbe -b :256 -e "A $filler" d256.img > d512.img
  • First a file is created to be used to fill a shell variable with dots (you can modify it if you wish).
  • Next blocks of 256 bytes are read from the input file and the filler is appended.
  • Then all of it is written to the output file, 512 bytes for each input block of 256 bytes.

This shellscript can certainly be improved, but it works for me with the following file names

  • input file: d256.img

  • output file: d512.img

sudodus
  • 6,421
2

Main problem

So what I need to happen is each 256 byte block get converted to a 512 byte block with the second half of each 512 byte block being null, zero, space or anything.

I don't think dd can do this.

bbe is "a sed-like editor for binary files". In Debian it's in the bbe package.

Run bbe, specifying block length of 256. In the below code the $(…) part is responsible for generating a script that looks like this: A \0\0\0\0…. There will be 256 appearances of \0. The script means "after each block append 256 NULL bytes".

<disk256bytesectors.img bbe -b :256 -e "$(
   printf 'A '; yes '\0' | head -n 256 | tr -d '\n'
)" >disk512bytesectors.img

This way each 256-byte block from the input will grow to 512 bytes in the output.

Side question about dd

conv=sync […] But the input block will never be shorter than itself?!?! So what does that mean?

It means if dd reads less than ibs in a single read operation (example) then with conv=sync it will append NULL bytes.

Kamil Maciorowski
  • 21,864
2

You can use dd to pad input by also using dd to provide the input to pad. For example, assuming sizes of 2 and 4 rather than 256 and 512:

dd bs=2 count=1 |
dd bs=4 count=1 conv=sync

The first dd will only read 2 bytes, once, and the 2nd dd will read 2 bytes, then pad it to 4 bytes, once.

You need only do this in a loop, somewhat inefficiently, until there is no more data (when dd reports: 0 bytes copied). For example, for the input string abcdefx:

#!/bin/bash
echo abcdefx | ( exec 3>&1
while   log=$(dd bs=2 count=1 status=none |
              dd bs=4 count=1 conv=sync 2>&1 1>&3)
        ! [[ "$log" =~ [^0-9]'0 bytes copied' ]]
do :
done ) | od -c

The result is

0000000   a   b  \0  \0   c   d  \0  \0   e   f  \0  \0   x  \n  \0  \0
meuh
  • 51,383