How can I put a bit mask on /dev/zero so that I can get bytes other than zero?

Question

How can I put a bit mask on /dev/zero so that I can have a source not only for 0x00 but also for any byte between 0x01 and 0xFF?

Answer 1

You cannot easily do that.

You might consider writing your own kernel module providing such a device. I don't recommend that.

You could write a tiny C program writing an infinite stream of same bytes on some pipe (or on stdout) or FIFO.

You could use tr(1) to read from /dev/zero and translate every 0 byte to somethng else.

You could use perhaps yes(1), at least if you can afford having newlines (or else pipe it into tr -d '\n'...)

Answer 2

The following bash code is set to work with the byte being representred in binary. However you can easily change it to handle ocatal, decimal or hex by simply changing the radix r value of 2 to 8, 10 or 16 respectively and setting b= accordingly.

r=2; b=01111110
printf -vo '\\%o' "$(($r#$b))"; </dev/zero tr '\0' "$o"

EDIT - It does handle the full range of byte values: hex 00-FF (when I wrote 00-7F below, I was considering only single-byte UTF-8 characters).

If, for example, you only want 4 bytes (characters in the UTF-8 'ASCII'-only hex 00-7F range), you can pipe it into head: ... | head -c4

Output (4 chars):

~~~~

To see the output in 8-bit format, pipe it into xxd (or any other 1's and 0's byte dump*):
eg. b=10000000 and piping to: ... | head -c4 | xxd -b

0000000: 10000000 10000000 10000000 10000000                    ....

Answer 3

Well, if you literally want to achieve this, you can use a LD_PRELOAD hook. The basic idea is to rewrite a function from the C library and use it instead of the normal one.

Here is a simple example where we override the read() function to XOR the output buffer with 0x42.

#define _GNU_SOURCE
#include <string.h>
#include <errno.h>
#include <sys/types.h>
#include <dlfcn.h> 
#include <unistd.h>

static int dev_zero_fd = -1;

int open64(const char *pathname, int flags)
{
    static int (*true_open64)(const char*, int) = NULL;
    if (true_open64 == NULL) {
        if ((true_open64 = dlsym(RTLD_NEXT, "open64")) == NULL) {
            perror("dlsym");
            return -1;
        }        
    }
    int ret = true_open64(pathname, flags);
    if (strcmp(pathname, "/dev/zero") == 0) {
        dev_zero_fd = ret;
    }
    return ret;
}


ssize_t read(int fd, void *buf, size_t count)
{
    static ssize_t (*true_read)(int, void*, size_t) = NULL;
    if (true_read == NULL) {
        if ((true_read = dlsym(RTLD_NEXT, "read")) == NULL) {
            perror("dlsym");
            return -1;
        }        
    }    

    if (fd == dev_zero_fd) {
        int i;
        ssize_t ret = true_read(fd, buf, count);    
        for (i = 0; i < ret; i++) {
            *((char*)buf + i) ^= 0x42;
        }
        return ret;
    }

    return true_read(fd, buf, count);    
}

A naive implementation would XOR 0x42 on every file we read, which would have undesirable consequences. In order to solve this problem, I also hooked the open() function, making it fetch the file descriptor associated with /dev/zero. Then, we only perform the XOR in on our read() function if fd == dev_zero_fd.

Usage:

$ gcc hook.c -ldl -shared -o hook.so
$ LD_PRELOAD=$(pwd)/hook.so bash #this spawns a hooked shell
$ cat /dev/zero
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

Answer 4

In terms of speed, the fastest I found was:

$ PERLIO=:unix perl -e '$s="\1" x 65536; for(;;){print $s}' | pv -a > /dev/null
[4.02GiB/s]

For comparison:

$ tr '\0' '\1' < /dev/zero | pv -a > /dev/null
[ 765MiB/s]
$ busybox tr '\0' '\1' < /dev/zero | pv -a > /dev/null
[ 399MiB/s]

$ yes $'\1' | tr -d '\n' | pv -a > /dev/null
[26.7MiB/s]

$ dash -c 'while : ; do echo -n "\1"; done' | pv -a > /dev/null
[ 225KiB/s]

$ bash -c 'while : ; do echo -ne "\1"; done' | pv -a > /dev/null
[ 180KiB/s]

$ < /dev/zero pv -a > /dev/null
[5.56GiB/s]
$ cat /dev/zero | pv -a > /dev/null
[2.82GiB/s]

Answer 5

It's kind of pointless to try and bitmask/xor zero bytes, isn't it? Taking a byte and xoring it with zero is a no-op.

Just create a loop that gives you the bytes you want and put it behind a pipe or named pipe. It'll behave pretty much the same as a character device (won't waste CPU cycles when idle):

mkfifo pipe
while : ; do echo -n "a"; done > pipe &

And if you want to super-optimize it, you can use the C code below:

#include <stdio.h>
#include <string.h>

int main(int argc, char **argv) { 
  char c = argc == 1+1 ? argv[1][0] : 'y';

  char buff[BUFSIZ];
  memset(buff, c, BUFSIZ);

  for(;;){ 
    write(1, buff, sizeof(buff)); 
  }
}

compile & run

$ CFLAGS=-O3 make loop
./loop "$the_byte_you_want" > pipe

Performance test:

./loop 1 | pv -a >/dev/null 

2.1GB/s on my machine (even slightly faster than cat /dev/zero | pv -a >/dev/null)

Answer 6

Read zeros, translate each zero to your pattern!

We read zero bytes out of /dev/zero, and use tr to apply a bit mask to each of the bytes by translating each zero byte:

$ </dev/zero tr '\000' '\176' | head -c 10
~~~~~~~~~~$

Octal 176 is the ascii code of ~, so we get 10 ~. (The $ at the end of the output indicates in my shell that there was no line end - it could look different for you)

So, let's create 0xFF bytes: Hex 0xFF is octal 0377. The leading zero is left out for the tr command line; At the end, hexdump is used to make the output readable.

$ </dev/zero tr '\000' '\377' | head -c 10 | hexdump
0000000 ffff ffff ffff ffff ffff               
000000a

You need to use the octal codes of the characters here, instead of the hexadecimal. So it's the range from \000 to octal \377 (same as 0xFF).
Use ascii -x and ascii -o to get a table of the characters with hexadecimal or octal index numbers.
(For a table with decimal and hexadecimal, just ascii).

Quite fast

It runs fairly fast, compared to just using the zeros: cat /dev/zero is only four times as fast, while it can make perfect use of IO buffering, which tr can not.

$ </dev/zero tr '\000' '\176' | pv -a >/dev/null
[ 913MB/s]
$ </dev/zero cat | pv -a >/dev/null

[4.37GB/s]

Answer 7

Depends what you want to do with the data and how flexible you want to use it.

Worst case if you need speed, you could do the same as the /dev/zero, and just compile the /dev/one, /dev/two, .. /dev/fourtytwo .. and so on devices.

In most cases it should be better to create the data directly where it is needed, so inside a program/script as a constant. With more information people could better help you.

Answer 8

Infinte printf loop

Reeplace \u00 with the byte you want.

while true ; do printf "\u00" ; done | yourapp

C++ code:

#include<cstdio>

int main(){
 char out=Byte;
 while(true)
 fwrite(&out,sizeof(out),1,stdout);
}

Compile: reeplace Byte with the value you want.

g++ -O3 -o bin file.cpp -D Byte=0x01

Use

./bin | yourapp