I'd like to continue using fuzzy matching with Ivy except in swiper.
The variable that determines which "regexp builder", as Ivy refers to these functions, is used for which collection function is ivy-re-builders-alist:
ivy-re-builders-alist is a variable defined in ‘ivy.el’.
Its value is ((t . ivy--regex-plus))
Documentation:
An alist of regex building functions for each collection function.
Each key is (in order of priority):
1. The actual collection function, e.g. ‘read-file-name-internal’.
2. The symbol passed by :caller into ‘ivy-read’.
3. ‘this-command’.
4. t.
Each value is a function that should take a string and return a
valid regex or a regex sequence (see below).
Possible choices: ‘ivy--regex’, ‘regexp-quote’,
‘ivy--regex-plus’, ‘ivy--regex-fuzzy’.
If a function returns a list, it should format like this:
’(("matching-regexp" . t) ("non-matching-regexp") ...).
The matches will be filtered in a sequence, you can mix the
regexps that should match and that should not match as you
like.
So, in order to change the default regexp builder from ivy--regex-plus to ivy--regex-fuzzy, but keep the former for swiper, you could
(setq ivy-re-builders-alist
'((swiper . ivy--regex-plus)
(t . ivy--regex-fuzzy)))
or, more programmatically,
(with-eval-after-load 'ivy
(push (cons #'swiper (cdr (assq t ivy-re-builders-alist)))
ivy-re-builders-alist)
(push (cons t #'ivy--regex-fuzzy) ivy-re-builders-alist))
This is described in more detail in (ivy) Completion Styles.
I don't really like [fuzzy matching] (at least not as a default, all the time)
Ivy allows you to rotate the regexp builder on the fly via its hydra interface. The fairly hidden last sentence of (ivy) ivy--regex-fuzzy alludes to this, and a more complete description can be found under (ivy) Hydra in the minibuffer, but it looks like the manual is a bit outdated given it's been a little while since the last release.
The upshot is that, since 2017-07-04, Ivy allows you to cycle through regexp builders during completion via C-om (ivy-rotate-preferred-builders). Edit: as pointed out by Asme Just in a comment, the default key binding was changed to C-oM on 2019-02-06.