Bash/Korn shell script edited on Windows throws error '...^M: not found'

Question

I wrote a Bash script in Windows using Notepad++.

cxStopAllServicesOnSERVER.sh

#!/usr/bin/bash
cd "/some/path"
echo "Hi. Logs can be found at "`pwd`"/cxStartStopLogger.log"
echo "["`date`"]*** STOPPING ALL SERVICES ON SERVER ***" >> "cxStartStopLogger.log"
exit

Now after uploading it and setting the required file permissions, I tried executing it as follows:

bash-3.00$ cat cxStopAllServicesOnSERVER.sh #Let's have a look at the code.
#!/usr/bin/bash
cd "/some/path/"
echo "Hi. Logs can be found at "`pwd`"/cxStartStopLogger.log"
echo "["`date`"]*** STOPPING ALL SERVICES ON SERVER ***" >> "cxStartStopLogger.log"

bash-3.00$ # Code and hashbang 'looks' correct, 
bash-3.00$ # if there is any issue with the format (EOL characters and such) 
bash-3.00$ # we cannot see it using cat.
bash-3.00$ 
bash-3.00$ sh cxStopAllServicesOnSERVER.sh
cxStopAllServicesOnSERVER.sh[2]: /some/path^M:  not found
Hi. Logs can be found at /some/path/cxStartStopLogger.log
bash-3.00$ # Note that ^M appears at the end of the line.

bash-3.00$ bash -x cxStopAllServicesOnSERVER.sh
+ cd $'/some/path\r'
: No such file or directory1.sh: line 2: cd: /some/path
++ pwd
' echo 'Hi. Logs can be found at /some/path/cxStartStopLogger.log
Hi. Logs can be found at /some/path/cxStartStopLogger.log
++ date
+ echo '[Sun' Nov 18 00:28:17 EST '2012]*** STOPPING ALL SERVICES ON SERVER ***'
bash-3.00$ # Note that '\r' return character appears at the end of the line.

Problem: When I change the code to korn shell I face the same issue. Appears that an incorrect character is getting added at the end of the line.

NOTE: I found the solution and have posted the same as an answer. Feel free to update or improve it to help other beginners that may face the same issue. Thanks!

Answer 1

After snooping around various forums and some Stack-overflow post I found the issue.

Here is an explanation to better understand and 'see' the root cause of the errors.

First off if you are writing code on Windows, better arm yourself with Notepad++. It's basically the Swiss Army knife of text-editors in my opinion.

Now to have a look at the EOL (end-of-line) characters and other control symbols do the following:

1.Open the file in Notepad++.

Select View > Show Symbol > Show All Characters from the Menu bar

You should now see the symbols as follows:

Aha! Windows/MS-DOS uses CR+LF to indicate end-of-lines. Now UNIX uses LF character to indicate line termination (EOL character).

2.1.Let's convert the Windows EOL into UNIX EOLs:

Select EDIT > EOL Conversion > UNIX Format

You should see something like this:

This explains why we saw the '\r' character getting appended at the end of the line. This is because UNIX recognized the '\n' or LF (linefeed) as the line termination/EOL character but ignore the '\r' or ^M (carriage return) character.

We can also test for the EOL issue by

bash-3.00$ head myScript.sh | cat -vet | head -1
#usr/bin/bash^M$
bash-3.00$ #We see that ^M, that is \r is found at the end of each line. 

2.2.If you would like convert the file from Windows to UNIX EOL format on UNIX then there are two ways of fixing this:

$ dos2unix myScript.sh

or

Use the solution provided by @Chris Down as below:

$ sed -i 's/\r$//' myScript.sh

References:

1. For more information on line termination and what-is-the-difference-between-r-and-n refer these links.

Answer 2

Your script contains CRLF line endings, whereas Unix uses LF line endings. The following will remove them, if you have GNU sed:

sed -i 's/\r$//' cxStopAllServicesOnSERVER.sh

If you don't have GNU sed, write to a temporary file, and then move it over the top of the old file.

As an aside, if you invoke a script using sh, you will override the shebang. That's probably not what you want.