How to check duplicate commands in $PATH?

Question

For example, youtube-dl has different ways to install it onto machine. I downloaded/installed it multiple times in not-same ways inadvertently, so that resulted that I have several youtube-dl executables in my $PATH directories, /usr/local/bin/youtube-dl, /home/username/.local/bin/youtube-dl, and /usr/local/bin/youtube-dl. So even I installed upgrade version "2021.01.03", youtube-dl --version shows me "2020.07.28", because youtube-dl in other directory in $PATH somehow overrided it.

So here, I want to check all of installed same-name files in $PATH so I can check their versions at once and can see which one is the latest, current one and which one should be deleted. So, Is there a way or CLI tool to do that? Thanks.

whereis youtube-dl or which youtube-dl might get you started — Panki, Jan 04 '21 at 15:18
This has been asked before, please see this post, or this post for several solutions. — number9, Jan 04 '21 at 18:23

score 2 · Accepted Answer · answered Jan 04 '21 at 15:57

2

Interactively, I would use:

ls -l $(type -ap youtube-dl)

to find the locations and timestamps of all the youtube-dl programs in my $PATH.

Of course, this doesn't work for executables that have spaces in their names, but youtube-dl isn't one of them.

answered Jan 04 '21 at 15:57

Jeff Schaller

67,283
35
116
255

This solution does not work recursively, or through the $PATH. – number9 Jan 04 '21 at 18:22
1

The question didn't ask for a recursive solution, and $PATH doesn't work that way. Why do you say it doesn't work for the $PATH variable? – Jeff Schaller Jan 04 '21 at 18:44

score 2 · Answer 2 · answered Jan 04 '21 at 19:12

2

In order to visit the PATH directories in order and list the location of files that would be executed you can use the which command:

$ which -a command_name

answered Jan 04 '21 at 19:12

Eduardo Trápani

12,834

I have to draw your attention to https://unix.stackexchange.com/a/85250/117549 – Jeff Schaller Jan 04 '21 at 21:02

Kusalananda · Answer 3 · 2021-01-07T12:22:42.903

The following script will output a list of duplicated executables found in the directories listed in the $PATH variable.

It does this by creating a tab-delimited two-column list of executables, where the first column is the directory pathname and the second is the filename of an executable.

This list is read by a simple awk program that counts how many times each executable is found and that collects all directory pathnames for each of them in a tab-delimited string.

At the end, the awk code outputs each executable whose exact name was found more than once, together with the tab-delimited list of directory pathnames where it was found. The directories will be listed in the order they were searched, i.e. an executable would be picked from the first directory listed if used without an explicit path on the command line (given the current value of $PATH).

#!/bin/sh
turn off filename globbing, and
split strings in unquoted variables on colons
set -f; IFS=:
set the positional parameters to the list of
directories in the PATH variable
set -- $PATH
turn on filename globbing again
(leave IFS as is, it won't affect anything else here)
set +f
loop over those directories and do the things
for dir do
        for pathname in "$dir"/; do
                [ -x "$pathname" ] && printf '%s\t%s\n' "$dir" "${pathname##/}"
        done
done |
awk -F '\t' '
{
        dir[$2] = ( dir[$2] == "" ? $1 : dir[$2] "\t" $1 )
        count[$2]++
}
END {
        for (util in dir)
                if (count[util] > 1)
                        printf "%s\t%s\n", util, dir[util]
}'

Example run on my OpenBSD system, passing the output through column -t to format it nicely, and sorting it:

$ sh script | column -t | sort
banner       /usr/bin                /usr/games
bash         /home/myself/local/bin  /usr/local/bin
bashbug      /home/myself/local/bin  /usr/local/bin
chgrp        /bin                    /usr/sbin
chown        /usr/sbin               /sbin
clang        /usr/bin                /usr/local/bin
clang++      /usr/bin                /usr/local/bin
clang-cpp    /usr/bin                /usr/local/bin
ld.lld       /usr/bin                /usr/local/bin
llvm-config  /usr/bin                /usr/local/bin
python3      /home/myself/local/bin  /usr/local/bin
python3.8    /home/myself/local/bin  /usr/local/bin
sysctl       /usr/sbin               /sbin
zsh          /home/myself/local/bin  /usr/local/bin

The output tells me that if I start bash on the command line, it will be picked up from /home/myself/local/bin, and that I have another bash executable in /usr/local/bin.

score 0 · Answer 4 · answered Jan 07 '21 at 12:26

I want to check all of installed same-name files in $PATH

Here's another suggestion, using find across the $PATH to get the file names, and awk to match up duplicates. You can roll it up into a single line if you want, but I've spread it across several here for readability,

( IFS=:; find $PATH -maxdepth 1 -type f ) |    # See below for alternative without maxdepth
    awk -F/ '
        {
            p[$NF] = p[$NF] "\n" $0;           # Save the pathname ($NF is last component)
            h[$NF] ++                          # Count this instance of the filename
        }
        END {
            for (f in h) {
                if (h[f] >1) { print p[f] }    # Print list iff we have multiple hits
            }
        }
    '

If your find doesn't have the -maxdepth action, you can replace that first line with this expansion

( IFS=:; for dir in $PATH; do find dir ! -path dir -prune -type f; done ) |

Unreadable but rolled up one-line version

(IFS=:; find $PATH -maxdepth 1 -type f)|awk -F/ '{p[$NF]=p[$NF]"\n"$0;h[$NF]++}END{for(f in h){if(h[f]>1){print p[f]}}}'

How to check duplicate commands in $PATH?

4 Answers4

turn off filename globbing, and

split strings in unquoted variables on colons

set the positional parameters to the list of

directories in the PATH variable

turn on filename globbing again

(leave IFS as is, it won't affect anything else here)

loop over those directories and do the things