78

I source the bashrc's of a few of my friends. So I end up having duplicate entries in my $PATH variable. I am not sure if that is causing commands to take a really long time to start. How does $PATH internally work in bash? Does having more PATHS slow my start up time?

davemackey
  • 105
balki
  • 4,407

6 Answers6

71

Having more entries in $PATH doesn't directly slow your startup, but it does slow each time you first run a particular command in a shell session (not every time you run the command, because bash maintains a cache). The slowdown is rarely perceptible unless you have a particularly slow filesystem (e.g. NFS, Samba or other network filesystem, or on Cygwin).

Duplicate entries are also a little annoying when you review your $PATH visually, you have to wade through more cruft.

It's easy enough to avoid adding duplicate entries.

case ":$PATH:" in
  *":$new_entry:"*) :;; # already there
  *) PATH="$new_entry:$PATH";; # or PATH="$PATH:$new_entry"
esac

Side note: sourcing someone else's shell script means executing code that he's written. In other words, you're giving your friends access to your account whenever they want.

Side note: .bashrc is not the right place to set $PATH or any other environment variable. Environment variables should be set in ~/.profile. See Which setup files should be used for setting up environment variables with bash?, Difference between .bashrc and .bash_profile.

Community
  • 1
Gilles 'SO- stop being evil'
  • 829,060
  • 9
    +1: can't emphasize that "giving your friends access to your account" enough emphasis. Even if there is no attempt at doing you harm, their script could be just what they need and still eat your lunch when you source it. – msw Jun 13 '11 at 11:41
  • One possible issue with this solution is if $new_entry is already the first entry in PATH, the ":$new_entry:" won't match. I fixed this in my profile by excluding the initial ':' colon. – Jeffrey Bauer May 19 '19 at 14:12
  • 1
    @JeffBauer I don't see the problem. I use case :$PATH: and not case $PATH so that it matches even if the entry is first or last. – Gilles 'SO- stop being evil' May 19 '19 at 20:05
  • Is the colon before the double-semicolon on the second line (:;;) a typo? – Big McLargeHuge Aug 19 '22 at 15:30
  • 1
    @BigMcLargeHuge : is a no-op command. I use it before ;; because some shells require a command before ;; (or used to, it looks like modern shells available on Linux don't). Even when not required, explicitly indicating that there's nothing to do is arguably good style, so the reader knows there isn't something missing. – Gilles 'SO- stop being evil' Aug 19 '22 at 15:43
  • Gotcha. So in a function I guess return would work as well. – Big McLargeHuge Aug 19 '22 at 16:03
45

I've seen people clean up duplicates from their PATH variable using awk and something like this:

PATH=$(printf "%s" "$PATH" | awk -v RS=':' '!a[$1]++ { if (NR > 1) printf RS; printf $1 }')

You could try adding that to your own bashrc and make sure you source the other files somewhere before running that.

An alternative would be to use the pathmerge utility.

As for your speed problem, this will not affect the startup time of the shell in any significant way but it may save some time doing tab completion for commands, especially when the command is not found in the path and it does repeated searches through the same folders looking for it.

A note on security: You should really heed Gilles' warnings about security here. By sourcing a file owned by another user you are giving a free pass to those users to execute their own code as your user every time you start a shell. If you don't trust those users with your password, you shouldn't be sourcing their shell files.

Stephen Kitt
  • 434,908
Caleb
  • 70,105
  • 8
    I like the awk one-liner, but it prints a trailing ORS ':'. So I modified it to read PATH=$(echo "$PATH" | awk -v RS=':' -v ORS=":" '!a[$1]++{if (NR > 1) printf ORS; printf $a[$1]}') – gkb0986 Sep 18 '13 at 06:01
  • The trailing : is not only a cosmetic issue. It is the same as adding . to your path, which is potentially dangerous. – wisbucky Apr 24 '17 at 23:31
  • I've edited the answer to include the fix from gkb0986. – Tim Lesher Jan 19 '18 at 13:24
  • @TimLesher The reason I'd never edited than into by answer is that it doesn't work for me .... and the original without it does work (including not leaving a trailing separator. I don't know what the difference is. – Caleb Jan 19 '18 at 14:07
  • @Tim printf $a[$1] can’t be right, that prints the field in the position matching the number of times the non-dupe has been seen, i.e. the first field. – Stephen Kitt Jan 19 '18 at 16:10
  • 1
    @gkb0986 This solution still fails if the path contains an escaped space, such as PATH=/bin:/foo\ bar:/usr/bin. I found a variant that avoids this at https://unix.stackexchange.com/a/124517/106102 – Alcamtar Feb 14 '19 at 20:49
  • @maharvey67 true, this solution does not cope with whitespace in the path. Nor with things like '%s'. – Steven Shaw Oct 24 '19 at 06:33
  • I can't get this fix to persist. It does clean duplicates from $PATH, but if I open a new Ubuntu WSL2 command prompt window, my $PATH is back to having duplicates. How can I make this permanent? – Kyle Vassella Dec 24 '20 at 15:39
  • @KyleVassella The best way is to figure out where you are adding duplicates in the first place. To do that you need a list of all the files that are sources when your shell starts (profiles, rc files, etc.). Somewhere in one of them you're adding the same things to the path twice. Even if you don't find the source of the problem you need to know that list anyway and know which one gets sources last, then you could add this hack to clean it up as the last step. You have to get it after whatever the problem cause is though or you'll just get it added again after it was cleaned. – Caleb Dec 29 '20 at 17:32
23

Based on @Gilles answer you may wrap it in a function to minimize typing:

function addToPATH {
  case ":$PATH:" in
    *":$1:"*) :;; # already there
    *) PATH="$1:$PATH";; # or PATH="$PATH:$1"
  esac
}

addToPATH /Applications/AIRSDK_Compiler/bin
addToPATH ~/.local/lib/npm/bin
hwde
  • 331
  • 2
  • 2
2

Only the first match in $PATH is executed, so any subsequent entries are not processed after that. That's why you should sometimes revise the order of the entries in your $PATH to make your environment behave as expected.

To answer your question: this shouldn't be the cause of slow startup.

Rajish
  • 797
  • 1
    But it takes longer when I type a command that does not exist. It will search the same folder twice for the command. – balki Jun 13 '11 at 08:50
  • @balki You mean completing a command with TAB? In that case you should check whether you complete definition doesn't look like complete -c which -a. You should delete the -a parameter. You can check that by issuing the command: complete | grep which. – Rajish Jun 13 '11 at 09:34
  • It could still be an issue if it searches the same directory that it's not in multiple times before finding it. – Random832 Jun 13 '11 at 13:44
0

PATH manipulation (manipulating any colon-separated list) is easy. I use Stephen Collyer's bash_path_funcs, described in Linux Journal way back in 2000:

https://www.linuxjournal.com/article/3645 https://www.linuxjournal.com/article/3768 https://www.linuxjournal.com/article/3935

The addpath function adds an entry to a path only if it is not there in the first place. delpath -n deletes all non-existent directories from a path.

You can get the pathfunc.tgz file from https://web.archive.org/web/20061210054813/http://www.netspinner.co.uk:80/Downloads/pathfunc.tgz

waltinator
  • 4,865
-1

To prevent duplicate entries in my PATH, I had to put the following in BOTH ~/.bash_profile and ~/.bashrc:

PATH=$(echo $(sed 's/:/\n/g' <<< $PATH | sort | uniq) | sed -e 's/\s/':'/g')

The main drawback is that it sorts PATH entries, but I think I can live with that.

John Reynolds
  • 1