2

I have almost 7,000 files in a single directory which makes loading that particular directory a tedious and time-consuming one. Is there any way to list and perform operations on files in that directory faster rather than distributing the files into multiple sub-folders(which makes it even more difficult)?

I'm using ext4. But also received the same from efs filesystems.

ctrl-alt-delor
  • 27,993
Snazzy Sanoj
  • 129
  • 1
    It's better to you make filter, you have some way, find, globbing and so on , But at first you explain exactly your task. – PersianGulf Mar 07 '15 at 20:22
  • 2
    Could you clarify what you mean by "loading"? Simply ls, or displaying the contents in a file manager (which one?), or something else? On my system listing 7,000 files with ls on an ext4 filesystem is pretty much instantaneous, as is displaying the directory in Thunar. – Stephen Kitt Mar 07 '15 at 20:56
  • @SnazzySanoj please answer comments, by amending your question. – ctrl-alt-delor Mar 07 '15 at 22:30
  • 3
    A lot of systems alias ls to something that makes it lstat every file. --color and -F do this. If you don't need ls to decorate the output, remove these aliases. Listing a 7000 file directory shouldn't be slow. – Mark Plotnick Mar 07 '15 at 23:48

1 Answers1

1

A large directory can be problematic for a number of reasons other than simply listing of files. For one thing, the time it takes to open a file in that directory will increase because the directory has to be read until the file is found. On many filesystems including ext*, directory entries not organized nor optimized for retrieval efficiency.

Answering your specific question, I think that you'll find using ls takes awhile due to the sorting involved. A solution is to run ls unsorted (if there is such an option in your distribution of ls). Specifically, I might ls unsorted to a file and the sort it. I then have the file to refer to without having to do another ls for a bit.

Another similar method is to use the find command and listout the directory contents into a file (which will be unsorted) and then go from there.

Thus my suggestion would be based on accessibility, effeciency and easily finding files, to use multiple sub-directories.

mdpc
  • 6,834
  • 1
    "A directory is not organized to speed up retrievals of specific entries": that completely depends on the filesystem in use, and many commonly-used filesystem types do optimize directory indices for just this ability. – Celada Mar 08 '15 at 05:09
  • OK, I qualified my response. Could you provide the commonly used filesystems that do what you state for the record? – mdpc Mar 08 '15 at 05:42
  • 2
    XFS automatically, ext4 (probably ext3 as well?) with the "dir_index" option enabled (which is part of the default options) (see mkfs.ext4 manpage), and I didn't bother to look up references but probably most of the other modern filesystem types as well: Btrfs, ZFS, JFS, all support efficient indexing for large directories. – Celada Mar 08 '15 at 07:19