How to grep a specific line _and_ the first line of a file?

Question

Assuming a simple grep such as:

$ psa aux | grep someApp
1000     11634 51.2  0.1  32824  9112 pts/1    SN+  13:24   7:49 someApp

This provides much information, but as the first line of the ps command is missing there is no context for the info. I would prefer that the first line of ps be shown as well:

$ psa aux | someMagic someApp
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
1000     11634 51.2  0.1  32824  9112 pts/1    SN+  13:24   7:49 someApp

Of course, I could add a regex to grep specifically for ps:

$ ps aux | grep -E "COMMAND|someApp"

However, I would prefer a more general solution as there are other cases in which I would like to have the first line as well.

Seems like this would be a good use case for a "stdmeta" file descriptor.

Answer 1

Good way

Normally you can't do this with grep but you can use other tools. AWK was already mentioned but you can also use sed, like this:

sed -e '1p' -e '/youpattern/!d'

How it works:

1. Sed utility works on each line individually, running specified commands on each of them. You can have multiple commands, specifying several -e options. We can prepend each command with a range parameter that specifies if this command should be applied to specific line or not.

2. "1p" is a first command. It uses p command which normally prints all the lines. But we prepend it with a numerical value that specifies the range it should be applied to. Here, we use 1 which means first line. If you want to print more lines, you can use x,yp where x is first line to print, y is last line to print. For example to print first 3 lines, you would use 1,3p

3. Next command is d which normally deletes all the lines from buffer. Before this command we put yourpattern between two / characters. This is the other way (first was to specify which lines as we did with p command) of addressing lines that the command should be running at. This means the command will only work for the lines that match yourpattern. Except, we use ! character before d command which inverts its logic. So now it will remove all the lines that do not match specified pattern.

4. At the end, sed will print all the lines that are left in buffer. But we removed lines that do not match from the buffer so only matching lines will be printed.

To sum up: we print 1st line, then we delete all the lines that do not match our pattern from input. Rest of the lines are printed (so only lines that do match the pattern).

First line problem

As mentioned in comments, there is a problem with this approach. If specified pattern matches also first line, it will be printed twice (once by p command and once because of a match). We can avoid this in two ways:

1. Adding 1d command after 1p. As I already mentioned, d command deletes lines from buffer and we specify it's range by number 1, which means it will only delete 1st line. So the command would be sed -e '1p' -e '1d' -e '/youpattern/!d'

2. Using 1b command, instead of 1p. It's a trick. b command allows us to jump to other command specified by a label (this way some commands can be omitted). But if this label is not specified (as in our example) it just jumps to the end of commands, ignoring rest of the commands for our line. So in our case, last d command won't remove this line from buffer.

Full example:

ps aux | sed -e '1b' -e '/syslog/!d'

Using semicolon

Some sed implementations can save you some typing by using semicolon to separate commands instead of using multiple -e options. So if you don't care about being portable the command would be ps aux | sed '1b;/syslog/!d'. It works at least in GNU sed and busybox implementations.

Crazy way

Here's, however, rather crazy way to do this with grep. It's definitely not optimal, I'm posting this just for learning purposes, but you may use it for example, if you don't have any other tool in your system:

ps aux | grep -n '.*' | grep -e '\(^1:\)\|syslog'

How it works

1. First, we use -n option to add line numbers before each line. We want to numerate all the lines we we are matching .* - anything, even empty line. As suggested in comments, we can also match '^', result is the same.

2. Then we are using extended regular expressions so we can use \| special character which works as OR. So we match if the line starts with 1: (first line) or contains our pattern (in this case its syslog).

Line numbers problem

Now the problem is, we are getting this ugly line numbers in our output. If this is a problem, we can remove them with cut, like this:

ps aux | grep -n '.*' | grep -e '\(^1:\)\|syslog' | cut -d ':' -f2-

-d option specifies delimiter, -f specifies fields (or columns) we want to print. So we want to cut each lines on every : character and print only 2nd and all subsequent columns. This effectively removes first column with it's delimiter and this is exactly what we need.

Answer 2

How do you feel about using awk instead of grep?

chopper:~> ps aux | awk 'NR == 1 || /syslogd/'
USER              PID  %CPU %MEM      VSZ    RSS   TT  STAT STARTED      TIME COMMAND
root               19   0.0  0.0  2518684   1160   ??  Ss   26Aug12   1:00.22 /usr/sbin/syslogd
mrb               574   0.0  0.0  2432852    696 s006  R+    8:04am   0:00.00 awk NR == 1 || /syslogd/

• NR == 1: Number of record == 1; ie. the first line
• ||: or:
• /syslogd/: Pattern to search for

It might also be worth looking at pgrep, although this is more for scripts rather than user-facing output. It does avoid the grep command itself from appearing in the output, though.

chopper:~> pgrep -l syslogd
19 syslogd

Answer 3

ps aux | { IFS= read -r line; printf '%s\n' "$line";grep someApp;}

With some head implementations such as the builtin head of ksh93 (enabled with builtin head, though beware not all builds of ksh93 include it):

ps aux | { head -n1;grep someApp;}

With most head implementations however, that doesn't work when the input is not seekable (such as the pipe it is here) as they read their input by block.

With:

{ head -1;grep ok;} <<END
this is a test
this line should be ok
not this one
END

with most head implementations, you only get:

this is a test
this line should be ok

With shells that implement here documents with temporary files instead of pipes.

The line command, where available (it used to be a standard command, but was removed from the standard on the ground that the functionality was available via IFS= read -r) would work for that as it's guaranteed not to read more than one line of input.

With zsh, you can also use IFS= read -re (-e for echo, not to be confused with bash's -e for edit). It's also the only shell whose read won't choke on NUL bytes.

Answer 4

The HP/UX and procps implementations of ps support internal filter with the -C option.

Suppose you're looking for bash processes:

ps -fC bash

Will list all processes whose name is bash.

Answer 5

I tend to send the header to stderr:

ps | (IFS= read -r HEADER; echo "$HEADER" >&2; cat) | grep ps

This is usually sufficient for human reading purposes. e.g.:

  PID TTY          TIME CMD
 4738 pts/0    00:00:00 ps

The bracketed part could go into its own script for general use.

There's an added convenience in that the output can be further piped (to sort etc.) and the header will remain on top.

Answer 6

You could also use tee and head:

ps aux | tee >(head -n1) | grep syslog

Note however that as long as tee is unable to ignore SIGPIPE signals (see e.g. the discussion here) this approach needs a workaround to be reliable. The workaround is to ignore SIGPIPE signals, this can for example be done like this in bash like shells:

trap '' PIPE    # ignore SIGPIPE
ps aux | tee >(head -n1) 2> /dev/null | grep syslog
trap - PIPE     # restore SIGPIPE handling

Also note that the output order is not guaranteed.

Answer 7

Perhaps two ps commands would be easiest.

$ ps aux | head -1 && ps aux | grep someApp
USER             PID  %CPU %MEM      VSZ    RSS   TT  STAT STARTED      TIME COMMAND
100         3304   0.0  0.2  2466308   6476   ??  Ss    2Sep12   0:01.75 /usr/bin/someApp

Answer 8

You could use pidstat with:

pidstat -C someApp
or
pidstat -p <PID>

Example:

# pidstat -C java
Linux 3.0.26-0.7-default (hostname)    09/12/12        _x86_64_

13:41:21          PID    %usr %system  %guest    %CPU   CPU  Command
13:41:21         3671    0.07    0.02    0.00    0.09     1  java

Further Info: http://linux.die.net/man/1/pidstat

Answer 9

Put the following in your .bashrc file or copy/paste in shell first, for testing.

function psls { 
ps aux|head -1 && ps aux|grep "$1"|grep -v grep;
}

Usage: psls [grep pattern]

$ psls someApp
USER             PID  %CPU %MEM      VSZ    RSS   TT  STAT STARTED      TIME COMMAND
root              21   0.0  0.0  2467312   1116   ??  Ss   Tue07PM   0:00.17 /sbin/someApp

Make sure to source your .bashrc (or .bash_profile if you put it there instead):

source ~/.bashrc

The function will even auto-complete at the shell command line. As you stated in another answer, you can pipe the first line to a file to save one call to ps.

Answer 10

sort but keep header line at the top

# print the header (the first line of input)
# and then run the specified command on the body (the rest of the input)
# use it in a pipeline, e.g. ps | body grep somepattern
body() {
    if IFS= read -r header; then
      printf '%s\n' "$header"
    else # no first line or unterminated first line
      printf %s "$header"
      # return # you may want to return in that case
    fi
    "$@"
}

And use it like this

$ ps aux | body grep someApp
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
1000     11634 51.2  0.1  32824  9112 pts/1    SN+  13:24   7:49 someApp

Answer 11

Thanks mostly to Janis Papanagnou in comp.unix.shell, I use the following function:

grep1() {
    local header
    IFS= read -r header && printf "%s\n" "$header"; grep "$@"
}

This has a number of advantages:

• Should work with bash, dash, ksh, and zsh
• It’s a drop-in replacement for grep, so you can continue to use whichever flags you want: -i for case-insensitive matching, -E for extended regexes, etc.
• Always yields the same exit code as grep, in case you want to programmatically determine whether any lines actually matched
• Prints nothing if the input was empty

Usage example:

$ ps -rcA | grep1 databases
  PID TTY           TIME CMD
$ ps -rcA | grep1 -i databases
  PID TTY           TIME CMD
62891 ??         0:00.33 com.apple.WebKit.Databases

Answer 12

Another way with gnu ed:

ed -s '!ps aux' <<< $'2,$v/PATTERN/d\n,p\nq\n'

or, if the shell supports process substitution:

printf '%s\n' '2,$v/PATTERN/d' ,p q | ed -s <(ps aux)

that is:

2,$v/PATTERN/d  - remove all lines not matching pattern (ignore the header)
,p              - print the remaining lines
q               - quit

More portable, without gnu '!' or shell substitution - using only ed built-in r to read the output of ps aux into the buffer and then delete non-matching lines in the 2,$ range and print the result:

printf '%s\n' 'r !ps aux' '2,$v/PATTERN/d' ,p q | ed -s

---

And since the sed commands in the accepted answer output also the line matching themselves, with a sed that supports -f- and a shell that supports process substitution I would run:

printf '%s\n' '2,${' '/PATTERN/!d' '}' | sed -f - <(ps aux)

which pretty much does the same thing as the previous ed commands.

Answer 13

The Perl way:

ps aux | perl -ne 'print if /pattern/ || $.==1'

Way easier to read than sed, faster, no risk to pick undesired lines.

Answer 14

If that's only for grepping processes with full headers, I'd expand @mrb's suggestion:

$ ps -f -p $(pgrep bash)
UID        PID  PPID  C STIME TTY      STAT   TIME CMD
nasha     2810  2771  0  2014 pts/6    Ss+    0:00 bash
...

pgrep bash | xargs ps -fp will get the same result but without a subshell. If other formatting is required:

$ pgrep bash | xargs ps fo uid,pid,stime,cmd -p
  UID   PID STIME CMD
    0  3599  2014 -bash
 1000  3286  2014 /bin/bash
 ...

Answer 15

This solution allows to use the real grep or any other filtering command.

ps aux | { awk 'NR<=1 {print >"/dev/fd/3"; next}; {print}' | grep someApp ; } 3>&1

It should not have these problems which are present in some other solutions:

• doubling of the first line
• early exit before processing all the input
• skipping part of the file after the first line
• changing the order of the lines
• using unreliable hacks instead of proper process synchronization (to keep the line order)

It could be a base for small utility script or function. Note that it is possible that the redirection using /dev/fd/3 is not available on some systems.

How does it work?

The awk process reads the standard input and splits it to two outputs based on a given condition - here line number: NR<=1.

• The first line is sent to the file descriptor 3 and in awk it is not processed further (accomplished by the next command).
• The other lines are sent to the grep process through the pipe. The grepped result continues to the standard output.

The parent shell opened the file descriptor 3 for the child processes as a copy of the standard output (3>&1). This way both outputs go to the standard output in the parent shell.