304

At the beginning of a bash shell script is the following line: 

IFS=$'\n'

What is the meaning behind this collection of symbols?

PythonJin
  • 103
Abdul Al Hazred
  • 26,630

4 Answers4

335

IFS stands for "internal field separator". It is used by the shell to determine how to do word splitting, i. e. how to recognize word boundaries.

Try this in a shell like bash (other shells may handle this differently, for example zsh):

mystring="foo:bar baz rab"
for word in $mystring; do
  echo "Word: $word"
done

The default value for IFS consists of whitespace characters (to be precise: space, tab and newline). Each character can be a word boundary. So, with the default value of IFS, the loop above will print:

Word: foo:bar
Word: baz
Word: rab

In other words, the shell thinks that whitespace is a word boundary.

Now, try setting IFS=: before executing the loop. This time, the result is:

Word: foo
Word: bar baz rab

Now, the shell splits mystring into words as well -- but now, it only treats a colon as the word boundary.

The first character of IFS is special: It is used to delimit words in the output when using the special $* variable (example taken from the Advanced Bash Scripting Guide, where you can also find more information on special variables like that one):

$ bash -c 'set w x y z; IFS=":-;"; echo "$*"'
w:x:y:z

Compare to:

$ bash -c 'set w x y z; IFS="-:;"; echo "$*"'
w-x-y-z

Note that in both examples, the shell will still treat all of the characters :, - and ; as word boundaries. The only thing that changes is the behaviour of $*.

Another important thing to know is how so-called "IFS whitespace" is treated. Basically, as soon as IFS includes whitespace characters, leading and trailing whitespace is stripped from the string to be split before processing it and a sequence of consecutive whitespace characters delimits fields as well. However, this only applies to those whitespace characters which are actually present in IFS.

For example, let's look at the string "a:b:: c d " (trailing space and two space characters between c and d).

  1. With IFS=: it would be split into four fields: "a", "b", "" (empty string) and " c d " (again, two spaces between c and d). Note the leading and trailing whitespace in the last field.
  2. With IFS=' :', it would be split into five fields: "a", "b", "" (empty string), "c" and "d". No leading and trailing whitespace anywhere.

Note how multiple, consecutive whitespace characters delimit two fields in the second example, while multiple, consecutive colons don't (since they are not whitespace characters).

As for IFS=$'\n', that is a ksh93 syntax also supported by bash, zsh, mksh and FreeBSD sh (with variations between all shells). Quoting the bash manpage:

Words of the form $'string' are treated specially. The word expands to "string", with backslash-escaped characters replaced as specified by the ANSI C standard.

\n is the escape sequence for a newline, so IFS ends up being set to a single newline character.

Vlastimil Burián
  • 28,462
Tblue
  • 3,989
  • 1
  • 11
  • 7
  • 6
    This is good, but, in my opinion, you'd do a lot better to read and understand the POSIX spec rather than the bash scripting guide, or whatever. In the main, the information available at links like that is lacking in important ways. Anyway, thus misses two crucial points concerning shell splitting - globbing and IFS whitespace. – mikeserv Feb 14 '15 at 16:43
  • 1
    @mikeserv Thanks, I added some information on IFS whitespace. Didn't know about that. :) – Tblue Feb 14 '15 at 17:26
  • 7
    Not as relevant, but if you're curious, you might want to look at how unset IFS makes a shell behave very differently than IFS=. Also the first byte in IFS is special to "${named_array[*]}" as well - but neither matters when the expansion is unquoted.. – mikeserv Feb 14 '15 at 17:30
  • A few more points: 1- word splitting, governed by $IFS is one of the two main things performed upon expanding an unquoted variable in list context (it's the split part of the split+glob operator). The other one is globbing. When using work splitting, you generally need to issue set -f to disable the glob part. – Stéphane Chazelas Jul 20 '16 at 14:12
  • 2- $IFS and the split+glob operator is used on unquoted parameter expansion ($var, $#...) but also for unquoted command substitution (\...`` and $(...)) and unquoted arithmetic expansion ($((...)) or $[...] in some shells). – Stéphane Chazelas Jul 20 '16 at 14:14
  • 5
    3- $IFS is also used by the read builtin command – Stéphane Chazelas Jul 20 '16 at 14:15
  • 4- While $IFS was the internal field separator initially in the Bourne shell, in POSIX shells, it's rather the internal field delimiter as in a:b: with IFS=: is split into a and b only (not a, b and the empty string). – Stéphane Chazelas Jul 20 '16 at 14:16
  • What's the purpose of the oddball IFS=$'\n' versus the more familiar IFS="\n" or IFS="\x0A"? The shell auto-escapes double-quoted strings by default. – Beejor Dec 09 '17 at 02:27
  • 1
    @Beejor: The $'' syntax expands the \n, so it breaks on an actual newline instead of \ and n. – SilverWolf Apr 19 '18 at 23:24
  • +1 for the double quotes question, I also don't understand how $'\n' is different from "\n", if both expand the \n to a newline`? which one should I use for better portability? – szx Nov 25 '20 at 15:02
  • 6
    @szx "\n" (or'\n') is not the same as $'\n'. The first string consists of the characters backslash and n. $'\n' results in a single newline character. For best portability, you cannot use $'\n' since it's only supported by certain shells. One hacky way to set IFS to a newline character would be IFS=$(printf '\n.'); IFS=${IFS%.} (command expansion removes trailing newlines, so we add and then remove another trailing character to work around that). – Tblue Nov 29 '20 at 18:35
38

Inside dollared single quotes, some characters are evaluated specially. For example, \n is translated to new line.

So, this particular line assigns newline to the variable IFS. IFS, in turn, is a special variable in bash: Internal Field Separator. As man bash says, it

is used for word splitting after expansion and to split lines into words with the read builtin command. The default value is <space><tab><newline>.

choroba
  • 47,233
  • 14
    +1 for mentioning dollared single quotes which is different from simple single quotes. – Snowcrash Feb 13 '18 at 13:42
  • 5
    @Snowcrash +1 for saying +1 for mentioning of dollared single quotes* which is different from simple single quotes*. Sorry, couldn't help it :) But actually it's a really good thing to point out because it is important! – Pryftan Feb 21 '19 at 17:09
  • 2
    @Pryftan +1 for +1 for +1... you know... it's really important. – 0xc0de Jun 13 '19 at 10:44
  • @0xc0de Definitely agree! Thanks for that! :) – Pryftan Jul 16 '19 at 21:26
28

For short, IFS=$'\n' assign newline \n to variable IFS.

$'string' construct is a quoting mechanism which use to decode ANSI C like escape sequences. This syntax comes from ksh93, and was portable to modern shell like bash, zsh, pdksh, busybox sh.

This syntax is not defined by POSIX, but was accepted for SUS issue 7.

cuonglm
  • 153,898
2

I prefered to explain $IFS via example:
Suppsoe you want to cp or mv or another file proccessing, IFS is empty by defualt, When your files have meta char or space such as :
Linux Administration.pdf or Free Software Fundation.ogg , Sure you'll have probelm, Because : Linux consider a seperate param and Administartion consider a seperate param.So bash has built-in variable , Then you can initialize to IFS==$(echo -en "\n\b") , Then bash discard any meta char and space between file name, For example:

#!/bin/bash
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
mymusicdir=~/test/dd
find $mymusicdir -name "*" -execdir rename 's/ /_/g' "{}" +
IFS=$SAVEIFS
PersianGulf
  • 10,850