How can I retrieve an HTTPS URL on Mac OS X without warnings about an untrusted authority?

Question

The following code should in principle return the contents of https://badssl.com

(with-current-buffer
    (url-retrieve-synchronously "https://badssl.com")
  (buffer-string))

But with my GNU Emacs 25.0.50.1 (x86_64-apple-darwin13.4.0, NS appkit-1265.21 Version 10.9.5 (Build 13F1112)) of 2015-08-24 on OS X 10.10.5, run with -Q, it instead shows me this message:

Certificate information
Issued by:          COMODO RSA Domain Validation Secure Server CA
Issued to:          Domain Control Validated
Hostname:           *.badssl.com
Public key:         RSA, signature: RSA-SHA256
Protocol:           TLS1.2, key: ECDHE-RSA, cipher: AES-128-GCM, mac: AEAD
Security level:     Medium
Valid:              From 2015-04-09 to 2016-07-07


The TLS connection to badssl.com:443 is insecure for the following
reasons:

the certificate was signed by an unknown and therefore untrusted authority
certificate could not be verified

and asks whether to accept the certificate - No/Session/Always. It then works if I accept the certificate.

I do not experience the same problem with, for example, wget. Wget correctly downloads (without asking the question) https://badssl.com and rejects https://self-signed.badssl.com.

Moreover, there appear to be two separate customization groups - gnutls and tls. I have previously compiled emacs with support for gnutls (gnutls-available-p returns t). Gnutls was installed with homebrew.

If I set gnutls-verify-error to t, I instead get the error

gnutls-negotiate: Certificate validation failed badssl.com, verification code 42

so presumably gnutls was not being used in the first example. On OS X none of the files mentioned in gnutls-trustfiles (in /usr/ssl and /etc/ssl) exist.

So how can I configure emacs to use the default trusted root certificates on OS X?

Edit: If I install libressl from homebrew, and add /opt/brew/etc/libressl/cert.pem to gnutls-trustfiles, the issue disappears when using gnutls-verify-error set to t.

Edit 2: In my testing it didn't seem like the value of tls-certtool-program was actually used, but it may be incorrect for OS X anyway: the default certtool on OS X is not the GnuTLS certtool required by tls.el, and with homebrew's gnutls I think it should be gnutls-certtool instead.

Answer 1

Add /private/etc/ssl/cert.pem to gnutls-trustfiles:

1. M-x customize-variable
2. Customize variable: gnutls-trustfiles
3. Select the last [INS]
4. Bundle filename: /private/etc/ssl/cert.pem
5. Select [ Apply and Save ]

Answer 2

To solve this issue it seems sufficient to:

• Set gnutls-verify-error to t, and
• Add an appropriate trusted root certificate file to gnutls-trustfiles - the default setting for that list does not include any existing files on OS X and it doesn't seem to default to using the system keychain.

This can be done, for example, by installing libressl from homebrew and adding the file $(brew --prefix)/etc/libressl/cert.pem - homebrew says this file is created from the system keychain. Setting the variable gnutls-log-level to a positive value shows that it does actually use that file.

Note: With these settings I get an extra unrelated message from gnutls saying

gnutls.c: [audit] Note that the security level of the Diffie-Hellman key exchange has been lowered to 256 bits and this may allow decryption of the session data

This can be fixed by setting gnutls-min-prime-bits to a value higher than the emacs default (256) and higher than what gnutls considers insecure (768) such as 3072, see http://gnutls.org/manual/html_node/Selecting-cryptographic-key-sizes.html as well as http://www.keylength.com/en/3/ - this sets the minimum group size for Diffie-Hellman key exchange.

Answer 3

Adding /private/etc/ssl/cert.pem seems to do the job.