Converting DOS text files to Unix

Question

I have a bunch of PHP scripts that are in DOS format (vi displayed [noeol] [dos] in the status line) and as expected they weren't usable.

I ran dos2unix over them to fix the line endins:

dos2unix index.php

but when I open the files in vi I still see [noeol] on the status line.

Upon checking them in vi's hex editor mode (:%!xxd) I don't see anything that jumps out at me.

In the original files the line ends looked like: 0x0D 0x0A

If after running through dos2unix they look like: 0x0A

I also checked to ensure that the last line of the file has a proper end of line and that's there too (0x0A).

What else could be causing this, should I worry?

I'm running CentOS 5.6 x64.

Does this answer your question? How to add a newline to the end of a file? — enharmonic, Apr 21 '23 at 20:43
@enharmonic - No, and why are you asking this on a nearly 12 year old question where I already accepted an answer and my problem is long since solved? — Kev, Apr 21 '23 at 21:11
I flagged as a duplicate, and StackExchange (un)helpfully auto-creates a comment with that text, as if I asked it. — enharmonic, Apr 21 '23 at 22:13
@enharmonic ah, that's an annoying behaviour and had forgotten about this. My apologies if I came across a bit short. — Kev, Apr 21 '23 at 23:12

enzotib · Accepted Answer · 2011-08-25T15:25:48.180

5

As @Klox said, after running dos2unix, the remaining problem vi shows is the missing newline on the last line.

If you open the file in vi and save it, it will add the newline for you.

Or in a batch you can do the whole thing with ed:

#!/bin/bash
for f in <list-of-files-goes-here>; do
    ed -s -- "$f" <<<$',s/\r//g\nw' >/dev/null
done

edited Aug 25 '11 at 15:25

answered Aug 25 '11 at 14:02

enzotib

51,661

Thanks, I knocked up a variation in that. Is dos2unix broken? – Kev Aug 25 '11 at 15:29
@Kev: i don't know, i consider it an un-useful tool, given many other methods exist to do the same, without installing nothing new. – enzotib Aug 25 '11 at 15:50
1

I eventually did: vi +":w ++ff=unix" +":q" ${f} in a loop. – Kev Aug 25 '11 at 15:52

score 2 · Answer 2 · answered Aug 25 '11 at 13:49

2

It is just warning you that you don't have an end of line (\n) on the last line.

answered Aug 25 '11 at 13:49

Klox

401

That was the first thing I checked and it's there. – Kev Aug 25 '11 at 13:56
@Kev: when you save, vi insert the last line newline. Have you checked after saving? – enzotib Aug 25 '11 at 14:04
Yes that works as does :set ff=unix, but I have over 200 files and wanted to batch convert them. – Kev Aug 25 '11 at 14:10
Use a shell for loop to run echo >> myfile on each file. – Klox Aug 25 '11 at 15:14

score 1 · Answer 3 · answered Aug 25 '11 at 13:50

1

DOS machines enter a carriage return at the end of lines (eol), it should look like this in vi ^M.

Try running this:

:%s/^M//g

answered Aug 25 '11 at 13:50

nopcorn

9,559

score 1 · Answer 4 · answered Aug 25 '11 at 15:24

1

You may also use ed to batch convert your php files since ed will add the missing final newlines for you as well.

Using ed there is no need for tmp files either.

# using Bash
IFS=$'\n'
for file in *.php; do
   ed -s "$file" <<< $'H\n,g/\r*$/s///\nwq'
   #printf '%s\n' H $',g/\r*$/s///' wq | ed -s "$file"
done

answered Aug 25 '11 at 15:24

tim

11
1

I would point out that I came to almost the same solution without seeing your answer :) – enzotib Aug 25 '11 at 15:51

score 0 · Answer 5 · answered Mar 11 '15 at 15:25

0

try this

find . -type f | xargs -I {} dos2unix {}

answered Mar 11 '15 at 15:25

wdog

101

score 0 · Answer 6 · answered Aug 25 '11 at 15:26

If you use vim (instead of an old skool vi), you can do this:

:set ff=dos
:e!

and you'll be able to edit the PHP files without all the ugly "^M" suffixes. I believe it will save as a DOS "text file" format, too. This can be useful if you share the files with (ugh!) Windows somehow.

A little editorializing: the two-byte end-of-line token chosed for MS-DOS has got to have been one of the biggest problems that MS-DOS foisted off on the world, after segments, drive-letters and "\" as directory seperator. That two-byte token is still the reason for a difference between "text" and "binary". Ickk.

Converting DOS text files to Unix

6 Answers6