5

Sometimes I have problems with opening a file using a graphical text editor -- I'm using geany. The file can be read by vim without a problem. I checked the file, and there wasn't anything wrong with it, except some lines. This is for example .bash_history file:

 776 reboot
 777 ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^     @^@^@^@^@^@^@geany /etc/fstab
....
....
 823 reboot 
 824 ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@exit

I don't know what ^@ is, but after removing each line that has it, the file can be read again in geany. Maybe the reboot action has to do something with it? But I have other reboot entries in the file and the ^@ characters appear only in two or three places.

This is only an example file, I saw the characters in some other files, one thing seems to be the same -- it concerns only big files, those that have many lines.

Does anyone know what ^@ means, where it came from and why vim has no problems with reading the file whereas geany can't read it at all?

Mikhail Morfikov
  • 10,549
  • 7
    They are basically null characters. This might give you more info: http://stackoverflow.com/questions/2398393/identifying-and-removing-null-characters-in-unix – Ketan Jan 08 '14 at 04:27
  • I checked the link, and the situation described there and on superuser concerns utf-16, but the file I have is coded in utf-8 -- /root/.bash_history: UTF-8 Unicode text, with very long lines. The other difference is that they have ^@ between each normal character, but in my case I have the whole line of these ^@, and it only occurs 2-3 times in some file. – Mikhail Morfikov Jan 08 '14 at 04:59
  • A related question is https://unix.stackexchange.com/questions/477537/ . – JdeBP Mar 09 '19 at 09:25

1 Answers1

4

When ever you have stray characters in a file you can enlist the assistance of the tools od or hexdump.

Examples

First we'll show what octal dump (od) shows when we tell it to dump the contents of file a.txt in a hexidecimal format (-x).

od

$ od -x a.txt 
0000000 3737 2036 6572 6f62 746f 370a 3737 0020
0000020 0000 0000 0000 0000 0000 0000 0000 0000
*
0000140 0000 0000 0000 0000 0000 0000 0000 6567
0000160 6e61 2079 652f 6374 662f 7473 6261 380a
0000200 3332 7220 6265 6f6f 2074 380a 3432 0020
0000220 0000 0000 0000 0000 0000 0000 0000 0000
*
0000320 7865 7469 000a
0000325

We can use hexdump to do something similar, showing the data in hexidecimal format, however it will also show the value as an ASCII character if possible.

hexdump

$ hexdump -C a.txt 
00000000  37 37 36 20 72 65 62 6f  6f 74 0a 37 37 37 20 00  |776 reboot.777 .|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 67 65  |..............ge|
00000070  61 6e 79 20 2f 65 74 63  2f 66 73 74 61 62 0a 38  |any /etc/fstab.8|
00000080  32 33 20 72 65 62 6f 6f  74 20 0a 38 32 34 20 00  |23 reboot .824 .|
00000090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000000d0  65 78 69 74 0a                                    |exit.|
000000d5

Looking at the above output you'll notice several sequences of 00 00 00. These are those ^@ characters you were asking about originally.

Incidentally the character 00 is the null character.

slm
  • 369,824
  • 3
    But what are they doing in the file? How did they get there? – Mikhail Morfikov Jan 08 '14 at 08:40
  • 1
    They look like garbage characters that have gotten included either when you where hitting backspace on the command line or a copy/paste of some previously run command where extra garbage characters were accidentally included. Or control characters from your prompt were included, but were converted to null, since they're non-printable characters. – slm Jan 08 '14 at 12:04
  • I tried to reproduce it based on this what you're saying but nothing seems to work. Maybe is it a terminal fault? I'm using a different one now. – Mikhail Morfikov Jan 08 '14 at 13:44
  • 4
    Other common causes of NULs in a text file are: 1. a process truncated the file while another process was writing it, so the second process's writes suddenly went past the current EOF, so the gap became sparsely allocated all-zero space. 2. The file was modified shortly before a system crash and the filesystem recovered the file's size without recovering all the data. –  Jan 08 '14 at 14:06
  • @WumpusQ.Wumbley - thanks for giving more possible reasons! – slm Jan 08 '14 at 14:23
  • @MikhailMorfikov - hard to say why, I'd need to have a look at the file and need to know more about your system and your setup. – slm Jan 08 '14 at 14:23
  • @slm -- What do you want to know? – Mikhail Morfikov Jan 08 '14 at 15:03
  • @MikhailMorfikov - it would be too hard to do it via SE. – slm Jan 08 '14 at 15:10