2

The Stage

sam@x230:~$ grep -E '^VERSION=|^NAME=' /etc/os-release 
NAME="Ubuntu"
VERSION="20.04.5 LTS (Focal Fossa)"
sam@x230:~$ echo ${BASH_VERSINFO[@]}
5 0 17 1 release x86_64-pc-linux-gnu
sam@x230:~$ tail -n $HISTSIZE ~/.bash_history  |grep '^ssh-add -l$' |wc -l
10
sam@x230:~$ grep -i hist .bashrc |egrep -v '^alias|^\s*#'
HISTCONTROL=ignoreboth
shopt -s histappend
HISTSIZE=1000
HISTFILESIZE=20000
sam@x230:~$  sed 's/\(ignoreboth\)$/\1:erasedups/' ~/.bashrc
sam@x230:~$ grep HISTCONTROL ~/.bashrc
HISTCONTROL=ignoreboth:erasedups
sam@x230:~$ export HISTCONTROL=ignoreboth:erasedups
sam@x230:~$ set |grep HIST
HISTCONTROL=ignoreboth:erasedups
HISTFILE=/home/sam/.bash_history
HISTFILESIZE=20000
HISTSIZE=1000

Scene 1:

sam@x230:~$ echo $HISTCONTROL
ignoreboth:erasedups
sam@x230:~$ ssh-add -l
256 SHA256:5FGaVpklzgSA1r/X8lb4SFaZ9hN0OfQXy+HOWpidcs4 default.sam@x230 (ED25519)
sam@x230:~$ tail -n $HISTSIZE ~/.bash_history  |grep '^ssh-add -l$' |wc -l
10
sam@x230:~$ exit

Tragic ending, in a new terminal:

sam@x230:~$ echo $HISTCONTROL
ignoreboth:erasedups
sam@x230:~$  tail -n $HISTSIZE ~/.bash_history  |grep '^ssh-add -l$' |wc -l
11

The experts

According to the man bash:

HISTCONTROL
              A colon-separated list of values controlling  how  commands  are
              saved  on  the history list.  If the list of values includes ig‐
              norespace, lines which begin with  a  space  character  are  not
              saved  in  the history list.  A value of ignoredups causes lines
              matching the previous history entry to not be saved.  A value of
              ignoreboth is shorthand for ignorespace and ignoredups.  A value
              of erasedups causes all previous lines matching the current line
              to  be  removed from the history list before that line is saved.
              Any value not in the above list is ignored.  If  HISTCONTROL  is
              unset,  or does not include a valid value, all lines read by the
              shell parser are saved on the history list, subject to the value
              of  HISTIGNORE.  The second and subsequent lines of a multi-line
              compound command are not tested, and are added  to  the  history
              regardless of the value of HISTCONTROL.

The critics

I expected that the last $HISTSIZE lines of my bash history will no longer contain duplicates (over multiple cycles of starting and exit of bash).

Where am I going wrong in my actions or my assumptions?

Note: I also tried with setting HISTCONTROL=erasedups, and the result is similar, i.e. duplicates remain (didn't test consecutive duplicates).

Samveen
  • 165
  • 1
    That's interesting. What happens if you set HISTCONTROL only to erasedups? And which version of bash do you use, on which OS/distro? – aviro Nov 27 '22 at 07:47
  • @aviro Ubuntu Focal (20.04), with bash 5.0 (info added to the question). There's no change with only erasedups. Much thanks pointing out the missing info :) – Samveen Nov 27 '22 at 11:35

1 Answers1

1

From the maintainer Chet Chet Ramey on gnu.bash.bug:

The erasedups feature works correctly: it removes matching commands from the history list, which is kept in memory. When the shell exits, by default it appends the current history list to the history file. If you want to force the history file to be completely rewritten, you can use history -w at shell exit to rewrite it. There is, unfortunately, currently no easy way to force the rewrite-at-exit behavior.

Furthermore, entries are only erased when you actually use them again.

Thus, you should find that if you:

  1. Start a new shell
  2. Execute ssh-add -l
  3. Execute history -w

Then you should have only one entry of ssh-add -l in your ~/.bash_history file.

How to fix this? How to clear all duplicates on a regular basis? That is another question, but the following might help:

mforbes
  • 111