56

If have a long text file and I want to display all the lines in which a given pattern occurs, I do:

grep -n form innsmouth.txt | cut -d : -f1

Now, I have a sequence of numbers (one number per line)

I would like to make a 2D graphical representation with the occurrence on the x-axis and the line number on the y-axis. How can I achieve this?

enter image description here

terdon
  • 242,166
  • 1
    Could you explain what you mean by occurrence? Do you mean how many times a particular number is found in the file? Or do you just want the actual value of the number on the x-axis and the line number that number was found on on the y-axis? – terdon Mar 15 '15 at 20:05
  • I mean by occurence simply in which order a pattern was found. e.g.: first time in line 400, second time in line 410 , third time in line 412 ... – Abdul Al Hazred Mar 15 '15 at 20:38
  • Or you can redirect the stdout data through pipe to a custom python script. This will allow you immense amount of customization and flexibility in parsing, pre-processing and visualizing the data. Here is a tutorial on this I wrote to do exactly as you intend. link – quartzfun Dec 18 '17 at 14:32

6 Answers6

72

You could use gnuplot for this:

 primes 1 100 |gnuplot -p -e 'plot "/dev/stdin"'

produces something like

enter image description here

You can configure the appearance of the graph to your heart's delight, output in various image formats, etc.

  • 3
    I have downloaded gnuplot and tried to test it by entering: seq 100 | gnuplot -p -e 'plot "/dev/stdin"' . strangely no graph appeared, but the exit code (echo $?) was 0 , so no error appeared either. – Abdul Al Hazred Mar 15 '15 at 22:18
  • @AbdulAlHazred did you install gnuplot or gnuplot-x11? if the former, afaik it only provides file output (i.e. generating pdf, png etc. files) rather than interactive plots direct to the screen. – steeldriver Mar 15 '15 at 22:55
  • @AbdulAlHazred: What happens if you just do seq 100 >seq.dat, then run gnuplot interactively and at the prompt type plot "seq.dat" ? – Nate Eldredge Mar 15 '15 at 23:02
  • @steeldriver I have an error Failed to initialize wxWidgets. with gnuplot-x11... Do I need to have one or the other? or can both gnuplot and gnuplot-x11be installed? – 3kstc Mar 17 '15 at 03:27
  • @nate thanks. all i had in my head was "graphviz" which does not draw what i call graphs – northern-bradley Aug 12 '17 at 09:55
  • @AbdulAlHazred it sounds like a terminal problem, see e.g. https://askubuntu.com/q/710783/383994 – Josip Rodin Mar 28 '18 at 13:15
  • 1
    Very nice; append notitle to plot without the title. – Victoria Stuart Aug 09 '18 at 23:00
15

I would do this in R. You'll have to install it but it shouold be available in your distributions repositories. For Debian-based systems, run

sudo apt-get install r-base

That should also bring in r-base-core but if it doesn't, run sudo apt-get install r-base-core as well. Once you have R installed, you could write a simple R script for this:

#!/usr/bin/env Rscript
args <- commandArgs(TRUE)
## Read the input data
a<-read.table(args[1])
## Set the output file name/type
pdf(file="output.pdf")
## Plot your data
plot(a$V2,a$V1,ylab="line number",xlab="value")
## Close the graphics device (write to the output file)
dev.off()

The script above will create a file called output.pdf. I tested as follows:

## Create a file with 100 random numbers and add line numbers (cat -n)
for i in {1..100}; do echo $RANDOM; done | cat -n > file 
## Run the R script
./foo.R file

On the random data I used, that produces:

enter image description here

I am not entirely sure what you want to plot but that should at least point you in the right direction.

terdon
  • 242,166
  • My Rscript v3.4.4 generates plots.pdf by default, regardless if using ggplot or plot. – Vorac May 27 '18 at 10:06
  • @Vorac did you mean to comment on another answer? What does ggplot have to do with it? And why is the default output filename relevant? – terdon May 27 '18 at 10:10
  • On my debian system this subset of your script is enough #!/usr/bin/env Rscript; args <- commandArgs(TRUE); a<-read.table(args[1]); plot(a$V2,a$V1,ylab="line number",xlab="value"); to generate an Rplots.pdf in the same directory. – Vorac May 27 '18 at 10:25
  • 2
    @Vorac yes, of course. But I want to choose the output file name. And, more importantly show how it can be done so it can be scripted. Otherwise, each time you run an RScript, it will use the same name and overwrite the output of a precious run. – terdon May 27 '18 at 10:30
15

If it might be that a very simple terminal printout would suffice, and that you could be satisfied by inverted axes, consider the following:

seq 1000   |
grep -n 11 |
while IFS=: read -r n match
do  printf "%0$((n/10))s\n" "$match"
done

The above charts an inverted trend on a 10% scale for every occurrence of the pattern 11 in the output of seq 1000.

Like this:

11
        110
        111
        112
        113
        114
        115
        116
        117
        118
        119
                  211
                            311
                                      411
                                                511
                                                          611
                                                                    711
                                                                              811
                                                                                        911

With dots and occurrence count it could be:

seq 1000    |
grep -n 11  | {
i=0
while IFS=: read -r n match
do    printf "%02d%0$((n/10))s\n" "$((i+=1))" .
done; }

...which prints...

01 .
02           .
03           .
04           .
05           .
06           .
07           .
08           .
09           .
10           .
11           .
12                     .
13                               .
14                                         .
15                                                   .
16                                                             .
17                                                                       .
18                                                                                 .
19                                                                                           .

You could get the axes like your example with a lot more work and tput - you'd need to do the \033[A escape (or its equivalent as is compatible with your terminal emulator) to move the cursor up a line for each occurrence.

If awk's printf supports space-padding like the POSIX-shell printf does, then you can use it to do the same - and likely far more efficiently as well. I, however, do not know how to use awk.

mikeserv
  • 58,310
  • Wow this creative, underutilized and probably sufficient for more cases than I realize. – Sridhar Sarnobat Aug 03 '22 at 04:39
  • Interesting. Can this be modified to read data from a csv file? (x,y) – FractalSpace Jan 01 '23 at 01:59
  • @FractalSpace do you mean a comma separated array of graphical plot points? yes, i think one might work up a program to do that with sort and sed but you might as well use gnuplot. – mikeserv Sep 24 '23 at 02:21
  • Thank you. Here's my TCP ping plot one-liner: TIMEFORMAT='%3R'; while true; do t=$( { time nc -z 8.8.8.8 53 &> /dev/null; } 2>&1 ); s=$(echo $t | tr -d '.' | sed 's/^0*//'); printf "%0$((s))s\n" "$t"; sleep 0.1; done – Vasiliy Sharapov Oct 10 '23 at 04:49
10

Check the package plotext which allows to plot data directly on terminal. It is very intuitive, as its syntax is very similar to matplotlib.

Here is a basic example:

import plotext as plt
y = plt.sin() # sinusoidal signal 
plt.scatter(y)
plt.title("Scatter Plot")
plt.show()

Scatter Plot

You can also plot bar plots:

Bar Plot

and even images: Image

An example of plotting a continuous data flow is shown here:

Data Stream

It can be installed with

pip install plotext

or with:

pip install "plotext[image]" 

to use plotext with images.

Glorfindel
  • 815
  • 2
  • 10
  • 19
Kriel
  • 201
1

Enhancing Nate's answer to have PDF output and to plot lines (requires the rsvg-convert):

| gnuplot -p -e 'set term svg; set output "|rsvg-convert -f pdf -o out.pdf /dev/stdin"; plot "/dev/stdin" with lines'
1

Possible way, it supports all kind of charts, not just line

plot="{chart:[line,{x: a},{y: b}],data:{a:[1,2,3],b:[2,4,1]}}"
curl -s http://pl0t.com/view.html | sed "s/{data}/$plot/" > play.html
open play.html

enter image description here

P.S. I'm the author of this tool.