How to merge duplicated history of commands in interactive shell?

Question

When you try to find a previous used command in an interactive shell session by ↑ (up arrow), you may get something like

$ls         # 1st time push `up arrow`
$ls         # 2nd time push `up arrow`
$ls         # 3rd time push `up arrow`
$ls         # 4th time push `up arrow`
$ls         # 5th time push `up arrow`
$ls         # 6th time push `up arrow`
$make       # 7th time push `up arrow`
$make       # 8th time push `up arrow`
$make       # 9th time push `up arrow`
$ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"   # Bingo!

I would like it better if it were like this:

$ls         # 1st time push `up arrow`
$make       # 2th time push `up arrow`
$ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"   # Bingo!

because the duplicated history is usually of no use.

How can I get Bash to do this?

You know that you can reuse the most recent ruby command by typing !r (Enter), right? — G-Man Says 'Reinstate Monica', Jul 27 '22 at 01:08

score 6 · Accepted Answer · edited Jul 27 '22 at 01:06

You can accomplish this by setting ignoredups in the HISTCONTROL variable:

HISTCONTROL="ignoredups"

Optionally export it,

export HISTCONTROL="ignoredups"

to make it an environment variable. From the bash(1) man page:

HISTCONTROL
A colon-separated list of values controlling how commands are saved on the history list. If the list of values includes ignorespace, lines which begin with a space character are not saved in the history list. A value of ignoredups causes lines matching the previous history entry to not be saved. A value of ignoreboth is shorthand for ignorespace and ignoredups. A value of erasedups causes all previous lines matching the current line to be removed from the history list before that line is saved. Any value not in the above list is ignored. If HISTCONTROL is unset, or does not include a valid value, all lines read by the shell parser are saved on the history list, subject to the value of HISTIGNORE. The second and subsequent lines of a multi-line compound command are not tested, and are added to the history regardless of the value of HISTCONTROL.

This does exactly what the question asks for. If the user enters the commands ruby …, make, make, make, ls, ls, ls, ls, ls and ls (as separate, consecutive lines), then the history list will be ruby …, make, ls. Pressing ↑ (up arrow) three times will return to the ruby command.

While this is a step in the right direction it won't do quite what the OP wants. It will NOT give them just three entries, one for make, one for ls and one for ruby ... it will just bunch up subsequent repeats. — tink, Feb 03 '15 at 00:50

score 1 · Answer 2 · edited Apr 13 '17 at 12:36

To add to @jordanm reply, I think you should use HISTCONTROL in fact but with the "erasedups" value.

"A value of erasedups causes all previous lines matching the current line to be removed from the history list before that line is saved."

export HISTCONTROL="ignoreboth:erasedups"

Add it to your ~/.bashrc to have it execute every time you log in.

Actually, this has been answered before

I wanted to add a comment but I can't because of my reputation.

VVK · Answer 3 · 2020-12-20T10:16:54.173

0

For macos, add the following lines to your ~/.zshrc file to avoid command duplicates in terminal history

setopt HIST_EXPIRE_DUPS_FIRST
setopt HIST_IGNORE_DUPS
setopt HIST_IGNORE_ALL_DUPS
setopt HIST_IGNORE_SPACE
setopt HIST_FIND_NO_DUPS
setopt HIST_SAVE_NO_DUPS

edited Dec 20 '20 at 10:16

answered Dec 20 '20 at 09:03

VVK

357
2
5

score 0 · Answer 4 · answered Jul 25 '22 at 16:41

0

I really couldn't find anything that works!

I create a bash file which I run on boot, but you can schedule it with cron too. It erase the duplicated from .bash_history.

arrVar=()
while read -r line; do
  s=${line# } # remove leading spaces
  s=${line%% } # remove trailing spaces
  if [[ ! " ${arrVar[*]} " =~ " ${s} " ]] && [[ "$s" != history* ]] ;
  then
    arrVar+=("$s")
  fi
done < ~/.bash_history
printf "%s\n" "${arrVar[@]}" > ~/.bash_history

answered Jul 25 '22 at 16:41

Yoram

1

Compared to some answers we get here, this is not bad. But (1) The OP describes a situation where, apparently, they run ls six times in a row, with no intervening commands (no, I don’t know why they would do that — maybe they’re monitoring the progress of some asynchronous activity, like a file transfer). And then they press “(up arrow)” twice, and they expect (want) the second press to take them back to the seventh previous command (make is shown as an example in the question), skipping the duplicate ls entries. Running your script at boot would not be useful. … (Cont’d) – G-Man Says 'Reinstate Monica' Jul 25 '22 at 19:36
(Cont’d) … You would have to run it very frequently; maybe even continuously. (2) Running a short script like that at boot is fairly inexpensive. But, as I just said, to make it do what the question asks for, you would have to run it very frequently. That would affect performance. (2b) Worse, if you (or any other user) are logged in and using bash while the script is running, there is likely to be a collision over the history file, possibly resulting in corruption. (3) ${line# } will remove only one leading space, and ${line%% } will remove only one trailing space. … (Cont’d) – G-Man Says 'Reinstate Monica' Jul 25 '22 at 19:36
(Cont’d) … (4) If you want to remove leading and trailing space(s), you should set s to line with the leading space(s) removed, and then remove the trailing space(s) *from s.* By twice giving s a value derived from line, you throw away the result of the first operation. (5) The ironic thing is that none of that is necessary, since read (with default value of IFS) will already strip leading and trailing spaces. … (Cont’d) – G-Man Says 'Reinstate Monica' Jul 25 '22 at 19:36
(Cont’d) … (6) Going back to point #1, this is not what the question asks for. If the user runs ls, and then 42 other commands, and then (an hour later) they run ls again, and then your script runs, it will keep only the oldest ls entry, so the user will have to press “(up arrow)” 43 times to re-run ls. (7) This script will probably mangle the history file if it contains multi-line commands. (8) It would be better if you provided a better explanation of what your script is doing. – G-Man Says 'Reinstate Monica' Jul 25 '22 at 19:37
uniq can do this : .... TMP=mktemp ; uniq < ~/.bash_history > $TMP && cat $TMP > ~/.bash_history && rm $TMP .... But you could just have HISTCONTROL in your ~/.bashrc "HISTCONTROL=ignoredups" – jmullee Aug 07 '22 at 21:04

How to merge duplicated history of commands in interactive shell?

4 Answers4