22

I just read the answers to Removing a newline character at the end of a file and everyone said to delete the last character. My question is, isn't the eof character the last one?

Enlico
  • 1,555
sworwitz
  • 273

3 Answers3

23

ASCII control characters have definitions from the 1960s (actually preceding what you might consider a network). Not all of those control characters are used in the way that they were defined for telecommunications equipment back then.

On Unix-like systems, there is no need for an EOF character; none is used. The system can tell applications how many bytes are in a file:

  • On some other systems (seen in VMS, DOS, Windows), a control-Z may act as an end-of-file marker because in older versions the system could not tell some applications how many bytes are in the file.

    In the case of VMS, the limitation was due to the way the C runtime worked. Assembly-language applications could (and did) get the correct file size.

  • Unix systems in the shell conventionally use control-D to tell an application that an end of input (file) has been reached, but the control-D is not stored in the file.

In C, EOF is purposely made -1 to indicate that it is not a valid character. Standard I/O returns EOF when an end-of-file condition is detected — not a special character.

By the way, files need not end with a newline (ASCII line-feed) character. Text editors can cope with files which are all printable text but lack a trailing newline.

Community
  • 1
Thomas Dickey
  • 76,765
  • 11
    POSIX defines a text file as a file containing a sequence of lines and in turn each line as a sequence of non-newline characters followed by one newline. Thus a file ending with anything but 0x0A is not a conforming text file. – Damian Yerrick Oct 08 '16 at 22:07
  • 2
    I'm aware of that, which is why I pointed out that text editors work. (Binary files have no such constraint). – Thomas Dickey Oct 08 '16 at 22:14
  • It's really worth noting that files intended to be handled as text which don't have a trailing newline are still arguably bad form (even if typical text editors have been coded to compensate for such files), at least if you actually want it to be broadly user-friendly/compatible, because the lack of a trailing newline can add additional difficulties in various circumstances (concatenating/printing multiple text files, parsing with typical command-line tools, minimal editors like busybox's vi, etc). – mtraceur Oct 08 '16 at 22:58
  • (1) Before VMS, RT-11 RSX-11 TOPS-10 had filesystems only precise to a block and needed an EOF character. So did CP/M, which apparently copied it from DEC and was in turn copied by early MS-DOS and then passed down to Windows. (2) In Unix it's the tty driver not the shell, as described in more detail by JohanM, although people usually run shells on tty devices. – dave_thompson_085 Oct 09 '16 at 12:58
  • Sure - DEC was back there (and note that I mentioned older versions). Whether it was the origin of the CP/M feature would be an interesting topic to explore (not here); I mentioned those cases to give some background to the alternatives. – Thomas Dickey Oct 09 '16 at 14:18
17

A file does not end with an End of File character, as the previous answers correctly state. But I think the answers and comments contain some inaccuracies worth pointing out:

  • The ASCII character set does not contain an exact EOF character. There are several "end" control characters: End of Text (3), End of Transmission (4), End of Transmission Block (23), End of Medium (25). File Separator (28) maybe comes closest to an EOF character. Code 26 is "Substitute", not EOF.

  • Ctrl-D is only associated with terminal input. For example the command cat filea fileb filec > outfile does not involve Ctrl-D. By the way, you can change the terminal EOF character to something else than Ctrl-D using the stty command.

  • Strictly speaking, Ctrl-D (or whatever you have changed to) is not an EOF key code. What it does is make the read system call return with what input is available, just like pressing return makes the read system call return a line of characters to the caller. By convention a return value of zero from the read system call (i.e. zero characters read) signals an end of file condition. However, the input file is not closed automatically, and, if the input comes from the terminal, it is not put in an "end of file" state. You can write a program that continues reading from the terminal even after an "end of file" and the read call can return non-zero for the next input line.

  • The analogy between the eof and eol characters can be seen if Ctrl-D is pressed when some input has already been written on the line. For example, if you write "abc" and the press Ctrl-D the read call returns, this time with a return value of 3 and with "abc" stored in the buffer passed as argument. Because read does not return 0, this is not interpreted as an EOF condition by the convention above. Similarly, pressing return to makes the read call return with the whole input line (including newline). You can try this out with the cat command: write some characters on the line and press Ctrl-D. You'll see the characters echoed back to you and cat waiting for more input.

  • All the above only applies when the terminal is in the "cooked" mode, as opposed to "raw" mode, in which line input processing is minimized. In raw mode a Ctrl-D character really is delivered to the input buffer.

chicks
  • 1,112
Johan Myréen
  • 13,168
7

EOF is not a character. It is a state which indicates no more characters to read from a file stream. When you enter EOF command from the terminal, you are signalling the OS to close the input stream, not putting in a special character.

Munir
  • 3,332
  • 1
    Yes but in ASCII table EOF is 26 so I thought the last byte was the binary representation of 26. So how could a program which reads an input know where it ends? – sworwitz Oct 08 '16 at 17:10
  • ASCII was meant for passing information over a network. In that case, you need an EOF character. (ASCII had a lot of control codes as well. Not everything was printable.) In case of file streams, the size of the file is already known through the file system so the OS can tell when there is no more data to read. – Munir Oct 08 '16 at 17:14
  • @sworwitz: With regards to C, input reading functions that return a character per call return an int (usually a 32 bit number but must be minimum 16 bits) not a char. The function signals and EOF by returning -1 (0xffffffff) which is not a valid 8 bit value so it won't be confused by any ASCII character, not even 0xff. Functions that return a string also returns the length of the read data. This length may be used to signal no data or end of data (again, length can be -1). Finally, there is also a function that you can call that will tell you if a stream has reached the end – slebetman Oct 08 '16 at 19:09
  • Ok thank you! So when in bash I press Ctrl+d I give in input the ASCII character, right? – sworwitz Oct 08 '16 at 19:17
  • @sworwitz Not exactly. Before bash gets its hands on the input, it is massaged by the TTY driver. This driver intercepts Ctrl-D and sends an EOF to bash (Where EOF is not a character, but a special file status) – Stig Hemmer Oct 10 '16 at 07:47