I had to do quite some diving and hunting through documentation and forums, so I figured I might as well do a Q&A here for others:
How do I enable GnuPG passphrase prompting in the minibuffer? The Emacs 25.1 changelog says:
** pinentry.el allows GnuPG passphrase to be prompted through the minibuffer instead of a graphical dialog, depending on whether the gpg command is called from Emacs (i.e., INSIDE_EMACS environment variable is set). This feature requires newer versions of GnuPG (2.1.5 or later) and Pinentry (0.9.5 or later). To use this feature, add "allow-emacs-pinentry" to "~/.gnupg/gpg-agent.conf" and reload the configuration with "gpgconf --reload gpg-agent".
Doing just that, I just get an error message now and not even the graphical passphrase prompt.
This is GNU Emacs 25.2.2 on Lubuntu (Ubuntu Xenial) with GnuPG 2.1.11.