Projectile with caching doesn't like new files

Question

I use projectile for my project browsing. I have seen that it's really slow, so I turned on caching. Now it's fast (not blazingly fast, sadly).

If I create a new file outside emacs itself, projectile doesn't find it. I can run cache invalidation but that takes a few minutes. How to fix this?

You can open the file and manually add it to the cache with `M-x` `projectile-cache-current-file`. If you don't want to then Emacs _must_ traverse the whole tree looking for files, so no optimizations can be made. — PythonNut, Feb 09 '15 at 02:53
I suggest opening a feature request for supporting the in-built `filenotify.el` library, which I believe would allow projectile to track directory changes and consequently be informed when a new file (or directory for tracking) was created (or deleted) so that it could automatically add (or remove) the files to/from the cache. Directory notifications are not recursive, so this feature would entail independently tracking every directory in the project. I don't know whether that would incur any noticeable penalties for a large project tree. — phils, Apr 12 '15 at 13:54

score 5 · Answer 1 · answered Feb 09 '15 at 10:34

Use the prefix argument, ie C-u C-c p f to quickly invalidate and find files in project. The suggestion in the comment about using projectile-cache-current-file is technically correct, but there is a hook on find-file that automatically calls this on every new file that is opened. That will be the fastest solution without invalidating the existing cache.

However, it sounds like the real issue is it takes too long to rebuild the cache by traversing the tree. For this it would help if there was a little more information about your environment. By default projectile-indexing-method is set to 'native on windows, which prefers an elisp implementation of find as it is more portable. Otherwise it uses an 'alien method, ie git ls, find, etc. If you are on linux the alien method is enabled by default though.

Otherwise you might want to chase down the rabbit hole of projectile-get-repo-files and projectile-get-ext-command. They specify where projectile looks for the list of files. It's possible you could override this with dir-locals or the like to force a faster lookup method.

score 3 · Answer 2 · answered Feb 10 '15 at 22:28

This does not speed up the file caching but it saves the manual step of having to invalidate the stale cache (which does not contain the new file).

Projectile automatically invalidates the cache if the /your/projectile/project/root/.projectile file is newer than the project cache file.

This check is done whenever you try to find any file in the project using projectile; projectile-find-file or C-c p f if one of the commands that does this check first.

(defun projectile-maybe-invalidate-cache (force)
  "Invalidate if FORCE or project's dirconfig newer than cache."
  (when (or force (file-newer-than-file-p (projectile-dirconfig-file)
                                           projectile-cache-file))
    (projectile-invalidate-cache nil)))

- From projectile.el source - lines 575-579

So the solution is to touch the .projectile file when you update the project. For example, you can alias the git commit, git pull, etc to do

touch /your/projectile/project/root/.projectile`

after doing whatever you were doing with the git commands.

So if someone in your team adds a new file to the project and (assuming you are using git to version control) you use your special aliased git pull, the cache will be invalidated automatically when you do projectile-find-file the next time.

Projectile with caching doesn't like new files

2 Answers2