This isn't the default behavior of Emacs, so it's likely that you're using a package that modifies it. I'm going to guess that you're using ido-mode. I occasionally get annoyed by this same aspect of ido-mode, but I don't think there's really a fix for it. If the search is taking long enough, you can hit C-g to cancel it, but if it's already finished the search that will cancel the ido-find-file command entirely. Another work around is to hit C-f again before it searches, this will drop you back to the default Emacs find-file.
Another thing I've started doing is using projectile-mode. This has a command to open another file in the same project (projectile-find-file, bound to C-c p f by default) which enumerates all of the files in the project before asking you to select one. You still get ido-mode's matching behavior, but since it has all the files in the project already it'll do less searching of unrelated projects. Of course, that requires that your files be in a "project" as Projectile defines it. On the other hand it treats any version-control repository as a project, which is very convenient.
And of course you have the option of disabling ido-mode entirely; just comment it out in your init file.