37

All shell builtins share the same manual page:

BUILTIN(1)                BSD General Commands Manual               BUILTIN(1)

NAME
     builtin, !

etc.

Then there is a little text describing what shell builtins are, and then a list that looks something like this:

  Command       External    csh(1)    sh(1)
       !             No          No        Yes
       %             No          Yes       No

But if we do man grep we get sections such as

  • Bugs
  • History
  • See also
  • Standards
  • Description

etc.

Don't shell builtins have their own history, description and arguments like -A or -r? Why isn't that provided in the manual pages and how would I learn to use them correctly and efficiently?

DisplayName
  • 11,688

3 Answers3

28

Because builtins are part of the shell. Any bugs or history they have are bugs and history of the shell itself. They are not independent commands and don't exist outside the shell they are built into.

The equivalent, for bash at least, is the help command. For example:

$ help while
while: while COMMANDS; do COMMANDS; done
    Execute commands as long as a test succeeds.

    Expand and execute COMMANDS as long as the final command in the
    `while' COMMANDS has an exit status of zero.

    Exit Status:
    Returns the status of the last command executed.

All bash builtins have help pages. Even help itself:

$ help help
help: help [-dms] [pattern ...]
    Display information about builtin commands.

    Displays brief summaries of builtin commands.  If PATTERN is
    specified, gives detailed help on all commands matching PATTERN,
    otherwise the list of help topics is printed.

    Options:
      -d    output short description for each topic
      -m    display usage in pseudo-manpage format
      -s    output only a short usage synopsis for each topic matching
        PATTERN

    Arguments:
      PATTERN   Pattern specifiying a help topic

    Exit Status:
    Returns success unless PATTERN is not found or an invalid option is given.

Inspired by @mikeserv's sed script, here's a little function that will print the relevant section of a man page using Perl. Add this line to your shell's initialization file (~/.bashrc for bash):

manperl(){ man "$1" | perl -00ne "print if /^\s*$2\b/"; }

Then, you run it by giving it a man page and the name of a section:

$ manperl bash while
       while list-1; do list-2; done
       until list-1; do list-2; done
              The while command continuously executes the list list-2 as long as the last command in the list list-1 returns an exit
              status of zero.  The until command is identical to the while command, except that the test is negated; list-2 is  exe‐
              cuted  as  long  as the last command in list-1 returns a non-zero exit status.  The exit status of the while and until
              commands is the exit status of the last command executed in list-2, or zero if none was executed.

$ manperl grep SYNOPSIS
SYNOPSIS
       grep [OPTIONS] PATTERN [FILE...]
       grep [OPTIONS] [-e PATTERN | -f FILE] [FILE...]

$ manperl rsync "-r"
       -r, --recursive
              This tells rsync to copy directories recursively.  See also --dirs (-d).
terdon
  • 242,166
  • Oh, so does shell builtins also have some thing like man pages in man bash? – DisplayName Nov 09 '14 at 18:43
  • 2
    @DisplayName they are bash. They are part of it and yes, they are explained in the SHELL BUILTIN COMMANDS section of the bash man page. Their "man pages" are help builtin_name. – terdon Nov 09 '14 at 18:45
  • 3
    What isn't clear is why they weren't given man pages. Man pages are just files on the MANPATH. They don't have to correspond to separate binaries. There's no reason in principle why bash couldn't have shipped with man pages for its builtins - rather than having an internal help system. – Francis Davey Nov 09 '14 at 22:49
  • 4
    @FrancisDavey: But most builtins exist (with different extensions) in various shells. Manpages are not shell-specific; they are system-wide. – rici Nov 10 '14 at 00:22
  • 2
    @FrancisDavey As rici said, the commands aren't system wide. It would be a bit misleading to have a manpage for a command that's not present in every shell, but even worse, it would be very confusing to have a man page for a command that is present in multiple shells, but which behaves differently (e.g., accepts different arguments, has different syntax, etc.). – Joshua Taylor Nov 10 '14 at 02:56
  • It's nice to inspire. I still think attributions are silly... – mikeserv Nov 10 '14 at 17:10
  • @JoshuaTaylor - there are a ton of man pages for common utilities of all kinds. They're the POSIX ones - and I thought, up to yesterday, that everybody had them. Anyway they are detailed and useful, but their contents detail only POSIX portable options, syntax - only that which should work everywhere. They are basically this. – mikeserv Nov 10 '14 at 18:08
  • @mikeserv Yes, it makes sense to have man pages for utilities whose behavior is specified by some particular standards. The issue is in things where there are intentionally multiple versions on a system and they have different syntax/behavior/etc. As an interesting example, look at CSH PROGRAMMING CONSIDERED HARMFUL which provides a humorous exposition of some of the differences between csh and some other shells. It would be hard to have a good man page for if syntax if it's different for the different shells installed on a system. – Joshua Taylor Nov 10 '14 at 18:14
  • 2
    @mikeserv However, I would welcome man pages for shell builtins along the lines of what, e.g., git offers, where man git commit brings up a man page for git-commit. Something like man bash if would be wonderful. – Joshua Taylor Nov 10 '14 at 18:16
  • @rici Agreed about why there can't be a single man page for a builtin, e.g., man if would be ambiguous. Man pages for individual commands the way that, e.g., git does things, where man git commit pulls up the page for git-commit, would be great. E.g., man bash if. – Joshua Taylor Nov 10 '14 at 18:17
  • @JoshuaTaylor - yeah I just read about that git-commit thing yesterday in man man. GNU man does that - it reads in man arguments in pairs - like section,subsection. Pretty cool, but it needs to be supported by the format - groff can do a lot of stuff I don't understand with its formatted source. I've seen that csh thing before - I like it. It's actually the closest I've ever come to csh, I guess. – mikeserv Nov 10 '14 at 18:18
6

While it's true that some shell builtins may have a scant showing in a complete manual - especially for those bash-specific builtins that you're only likely to use on a GNU system (the GNU folks, as a rule, don't believe in man and prefer their own info pages) - the vast majority of POSIX utilities - shell builtins or otherwise - are very well represented in the POSIX Programmer's Guide.

Here's an excerpt from the bottom of my man sh (which is probably 20 pages long or so...)

enter image description here

All of those are there, and others not mentioned such as set, read, break... well, I don't need to name them all. But note the (1P) at the bottom right - it denotes the POSIX category 1 manual series - those are the man pages I'm talking about.

It may be that you just need to install a package? This looks promising for a Debian system. While help is useful, if you can find it, you should definitely get that POSIX Programmer's Guide series. It can be extremely helpful. And it's constituent pages are very detailed.

That aside, shell builtins are almost always listed in a specific section of the specific shell's manual. zsh, for example, has an entire separate man page for that - (I think it totals at 8 or 9 or so individual zsh pages - including zshall which is huge.)

You can just grep man of course:

man bash 2>/dev/null | 
grep '^[[:blank:]]*read [^`]*[-[]' -A14

   read [-ers] [-a aname] [-d  delim]  [-i  text]  [-n
   nchars]  [-N  nchars]  [-p prompt] [-t timeout] [-u
   fd] [name ...]
          One line is read from the standard input, or
          from  the  file descriptor fd supplied as an
          argument to the -u  option,  and  the  first
          word is assigned to the first name, the sec‐
          ond word to the second name, and so on, with
          leftover words and their intervening separa‐
          tors assigned to the last  name.   If  there
          are  fewer  words read from the input stream
          than names, the remaining names are assigned
          empty  values.   The  characters  in IFS are
          used to split the line into words using  the
          same  rules  the  shell  uses  for expansion

...which is pretty close to what I used to do when searching a shell man page. But help is pretty good in bash in most cases.

I've actually been working on a sed script to handle this kind of stuff recently. It's how I grabbed the section in the picture above. It's still longer than I like, but it's improving - and can be pretty handy. In its current iteration it will pretty reliably extract a context-sensitive section of text as matched to a section or subsection heading based on [a] pattern[s] given it on the command line. It colors its output and prints to stdout.

It works by evaluating indent levels. Non-blank input lines are generally ignored, but when it encounters a blank line it starts to pay attention. It gathers lines from there until it has verified that the current sequence definitely indents further in than did its first line before another blank line occurs or else it drops the thread and waits for the next blank. If the test is successful it attempts to match the lead line against its command-line args.

This means that a match pattern will match:

heading
    match ...
    ...
    ...
        text...

..and..

match
   text

..but not..

heading
    match
    match

    notmatch

..or..

         text

         match
         match
         text

         more text

If a match can be had it starts printing. It will strip the matched line's leading blanks from all lines it prints - so no matter the indent level it found that line on it prints it as if it were at the top. It will continue to print until it encounters another line at an equal or lesser-than indent level than its matched line - so entire sections are grabbed with just a heading match, including any/all subsections, paragraphs they might contain.

So basically if you ask it to match a pattern it will only do so against a subject heading of some kind and will color and print all of the text it finds within the section headed by its match. Nothing is saved as it does this except your first line's indent - and so it can be very fast and handle \newline separated input of virtually any size.

It took me awhile to figure out how to recurse into subheadings like the following:

Section Heading
    Subsection Heading

But I sorted it out eventually.

I did have to rework the whole thing for simplicity's sake, though. While before I had several small loops doing mostly the same things in slightly different ways to fit their context, by varying their means of recursion I managed to de-duplicate the majority of the code. Now there are two loops - one prints and one checks indent. Both depend on the same test - the print loop starts when the test passes and the indent loop takes over when it fails or begins on a blank line.

The whole process is very fast because most of the time it just /./deletes any non-blank line and moves on to the next - even results from zshall populate the screen instantly. This has not changed.

Anyway, it is very useful so far, though. For example, the read thing above can be done like:

mansed bash read

...and it gets the whole block. It can take any patterns or whatever, or multiple arguments, though the first is always the man page in which it should search. Here's a picture of some of its output after I did:

mansed bash read printf

enter image description here

...both blocks are returned whole. I often use it like:

mansed ksh '[Cc]ommand.*'

...for which it is quite useful. Also, getting SYNOPS[ES] makes it really handy:

enter image description here

Here it is if you want to give it a whirl - I won't blame you if you don't though.

mansed() {
MAN_KEEP_FORMATTING=1 man "$1" 2>/dev/null | ( shift
b='[:blank:]' s='[:space:]' bs=$(printf \\b) esc=$(printf '\033\[') n='\
' match=$(printf "\([${b}]*%s[${b}].*\)*" "$@")
sed -n "1p
    /\n/!{  /./{    \$p;d
        };x;    /.*\n/!g;s///;x
    :indent
        /.*\n\n/{s///;x
        };n;\$p;
        /^\([^${s}].*\)*$/{s/./ &/;h;   b indent
        };x;    s/.*\n[^-[]*\n.*//; /./!x;t
        s/[${s}]*$//;   s/\n[${b}]\{2,\}/${n} /;G;h
    };
    #test
    /^\([${b}]*\)\([^${b}].*\n\)\1\([${b}]\)/!b indent
        s//\1\2.\3/
    :print
    /^[${s}]*\n\./{ s///;s/\n\./${n}/
        /${bs}/{s/\n/ & /g;
            s/\(\(.\)${bs}\2\)\{1,\}/${esc}38;5;35m&${esc}0m/g
            s/\(_${bs}[^_]\)\{1,\}/${esc}38;5;75m&${esc}0m/g
            s/.${bs}//g;s/ \n /${n}/g
            s/\(\(${esc}\)0m\2[^m]*m[_ ]\{,2\}\)\{2\}/_/g
        };p;g;N;/\n$/!D
        s//./;  t print
    };
    #match
        s/\n.*/ /;  s/.${bs}//g
        s/^\(${match}\).*/${n}\1/
        /../{   s/^\([${s}]*\)\(.*\)/\1${n}/
        x;  s//${n}\1${n}. \2/; P
    };D
");}

Briefly, the workflow is:

  • any line which is not blank and which does not contain a \newline character is deleted from output.
    • \newline characters never occur in input pattern space. They can only be had as the result of an edit.
  • :print and :indent are both mutually dependent closed loops and are the only way to obtain a \newline.
    • :print's loop cycle begins if the leading characters on a line are a series of blanks followed by a \newline character.
    • :indent's cycle begins on blank lines - or on :print cycle lines which fail #test - but :indent removes all leading blank + \newline sequences from its output.
    • once :print begins it will continue to pull in input lines, strip leading whitespace up to the amount found on the first line in its cycle, translate overstrike and understrike backspace escapes into color terminal escapes, and print the results until #test fails.
    • before :indent begins it first checks hold space for any possible indent continuation (such as a Subsection), and then continues to pull in input as long as #test fails and any line following the first continues to match [-. When a line after the first does not match that pattern it is deleted - and subsequently so are all following lines until the next blank line.
  • #match and #test bridge the two closed loops.
    • #test passes when the leading series of blanks is shorter than the series followed by the last \newline in a line sequence.
    • #match prepends the leading \newlines needed to begin a :print cycle to any of :indent's output sequences which lead with a match to any command-line arg. Those sequences that don't are rendered empty - and the resulting blank line is passed back to :indent.
mikeserv
  • 58,310
  • 2
    Your sed-fu is strong. Of course, you can do the same thing with manperl(){ man $1 | perl -00ne "print if /^\s*$2\b/"; } and then manperl sh SYNOPSIS or manperl sh read :) – terdon Nov 10 '14 at 14:37
  • @terdon - no, you can't. This doesn't eat input. I could do the same thing as that like sed 'H;$!d;g;s/\(\(\n *\)match\([^\n]*\)\2 \)\{1,\}\)*.\{,1\}/\1/g' ...probably that works... but that requires swallowing the file and parsing it all at once. This works in a stream - it can handle input of any size provided the lines are not astronomically long. It prints as it works - and it parses all of man's \backslash escapes to boot. But man is just a single application for it - I've applied much of it to other problems as well... – mikeserv Nov 10 '14 at 15:00
  • 1
    I'm just yanking your chain since I can do what you describe with a tiny one liner. Note, however, that it does not swallow the file whole, it works in a stream. It just defines "lines" using \n\n instead of \n but still can handle any size input and prints as it works. See "paragraph mode" here: http://perldoc.perl.org/perlrun.html – terdon Nov 10 '14 at 15:20
  • @terdon Maybe that would have been a better way to go here. In sed it can be done like: '/./{H;$!d' -e '};x;now work the paragraph...'. I often do that, too. But I originally wrote the first part for watching a log live for unlimited amounts of time, and even that behavior was iffy - the buffer can explode under certain conditions. That was only half this size - man made it harder. I looked at man -H after getting the man synop above though, and I'm thinking it might be easier to work with the machine-genned HTML that groff can print on GNU systems. I'm kinda elbow-deep already thoug – mikeserv Nov 10 '14 at 15:29
  • @terdon - I did second-guess myself and try a paragraph-centered approach, but it is easier as is. This gets sections. Like mansed cmd DESCRIPTION gets DESCRIPTION section - and all included ones. A matched search is printed whole and as if its indent level was top one. It even skips false positives by ignoring paragraphs that match but do not then indent further. It matches its args through the color backspace escapes and doesn't handle those until it is definitely ready to print a line. All of that is very hard for me to do with much more data than a single line at a time. – mikeserv Nov 12 '14 at 21:25
3

Each shell has its own set of builtins. While there are commonalities, they each have their own peculiarities that need to be documented.

On systems like Linux and FreeBSD (and OSX, which inherits from FreeBSD) where each shell is provided as a separate package, there is no man page for builtins; instead, each builtin is documented in the shell's man page. So read the bash man page for the documentation of bash's kill builtin, read the dash man page for the documentation of dash's kill builtin, etc. There's also a man page for the kill standalone utility.

See Can I get individual man pages for the bash builtin commands? for a man function that shows bash's internal documentation instead of the man page if the argument is the name of a builtin.

There are unix variants that provide man pages for shell builtins — in fact, most commercial variants do. That's feasible because the system comes with either a single shell, or a set of known shells. The man page discusses the differences between shells. For example, the fg(1) man page on Solaris 10 has sections for sh, ksh and csh. The fg(1) man page on AIX 7.1 references “Korn shell” and “POSIX shell” but discusses them together (they happen to support exactly the same features for fg). The fg(1) man page on Tru64 5.0 discusses the ksh builtin and refers csh users to the csh(1) man page. SCO apparently comes with a single shell. You can install other shells as add-on packages on these operating systems; if you use a custom shell, you must remember that the man pages for builtins will not be relevant when using a non-default shell.