How to organize installed programs in $HOME for shell to search for them when running them?

Question

I have written or downloaded some programs ( bash scripts, python scripts, java programs, C programs) and placed them under /home/t/program_files/. For each program, I create a directory under the path, and place all the files for the program in that directory. I even group the programs by their similarity in topics, by further creating directories such as OS, network, and document, and moving the directory of each program into one of such directories. I feel that way helps me manage the programs in a logical way, and I can find the file for a program more quickly.

If I want to run the programs, I have to specify the search paths for all the programs to PATH. It can be tedious to manually find out the path to each script or executable file and add it to $PATH. One quick way is to recursively add to PATH all the descendant directories of /home/t/program_files/, but the problem with this way is that it will add directories which don't directly contain executable or script files but other files such as Java and C source files, and there will be too many paths in PATH for shell to search for a command in a reasonably long time.

What are some good ways to solve the above problem?

• Gilles' reply suggested to use Stow. After a little reading about Stow, I understand that it creates symlinks and the symlinks are still organized in the same ways as their target files, which if I am correct doesn't help with the problem of specifying the search paths of the programs. Am I missing something about Stow which can help with my problem?

• Another way which I am wondering about is to go recursively under /home/t/program_files/, and check each file whether it is runnable, and if yes, create a symlink to it under /home/t/bin/ and maybe also add its parent directory's pathname to PATH.

• Additionally, I believe it is common that one has the needs to create and download third part programs and would like to install them in their home directory. What are some recommended ways to organize such programs? Do they organize the programs not by placing all the files of each program in its own directory?

Thanks.

Answer 1

I don't have experience with stow, nor do I know it, but I can suggest some logic as an answer to your question, also using links.

As you wrote, you keep your programs in /home/t/program_files/. So say you have there a program called GHR_Communicator, in a folder communicators, which in turn has an executable ghrc at the path:

/home/t/program_files/communicators/GHR_Communicator/ghrc

Now say that you designate a common folder for your private executables as /home/t/bin. This will be your only folder that you need to include in the PATH variable. In some distributions PATH contains it by default, so check your PATH for that and add it if needed.

The only thing you need to do to be able to call ghrc by this simple name is to create a link in ~/bin pointing at the executable.

$ ln -sr ~/program_files/communicators/GHR_Communicator/ghrc ~/bin

Now just call ghrc:

$ ghrc

And you have it executed.

Answer 2

I agree with Gilles that Stow is probably the right tools for the job. The basic model of Stow is that you install software into specific directories, one per piece of software; in the Stow manual you’ll find Perl and Emacs used as examples, installed respectively in /usr/local/stow/perl and /usr/local/stow/emacs. Inside those directories, you’ll have the usual structure — bin, share etc. Stow “installs” software by symlinking directories and/or files into a target, typically /usr/local. Installing perl will ensure that /usr/local/stow/perl/bin/perl is available as /usr/local/bin/perl, etc. Thus you only need to have /usr/local/bin on your PATH...

Stow can be told to use multiple Stow directories, which should solve your problem with it. As long as each individual piece of software has a directory structure matching the typical directory structure under /usr/local, it can be managed by Stow.

Stow also works well for programs stored under your home directory, with a variety of structures. That way you can install programs into separate directories, but still make them available from a single shared directory (e.g. ~/.local/bin).

The general approach to Stow, when building programs from source, is to run ./configure (or equivalent) with the final target prefix (/usr/local), and install the software with a destination directory inside a Stow directory (for example, /usr/local/stow/perl in the basic case covered by the manual; but /usr/local/Programming/perl would also work, using /usr/local/Programming as a Stow directory). Then you use Stow to make the software available by adding symlinks as appropriate.