How to Renew Let's Encrypt Certificates Behind a Firewall

When using Let's Encrypt(opens in new tab) for SSL certificates, it's a good idea to have automatic certificate renewal enabled, so you can avoid receiving emails like this:

Let's Encrypt certificate expiration notice

Your certificate (or certificates) will expire in 7 days. We recommend renewing certificates automatically when they have a third of their total lifetime left.

However, you might receive such emails even though you already have successfully enabled automatic renewal. This can happen if a firewall protects your server by blocking inbound traffic, like in my case with my DigitalOcean server.

The Problem

When setting up a firewall for a web server, you usually restrict only inbound traffic. This allows just a few specific IPs to initiate connections with your server (e.g. your office network). On the other hand, you leave outbound traffic unrestricted, so your server can freely initiate connections with any IP on the internet. This is necessary if your application relies on remote APIs, for example.

To perform a certificate renewal, your server needs to pass a challenge. The most common one is the HTTP-01 challenge(opens in new tab):

Let's Encrypt gives a token to your ACME client(opens in new tab), and your ACME client puts a file on your web server. That file contains the token, plus a thumbprint of your account key. Once your ACME client tells Let's Encrypt that the file is ready, Let's Encrypt tries retrieving it. If our validation checks get the right responses from your web server, the validation is considered successful.

This means that in order to pass the challenge, Let's Encrypt must be able to access your server over HTTP on port 80. In other words, you not only need outbound traffic allowed — you need inbound traffic allowed as well… and your firewall blocks it.

The Solution

Your first thought might be to simply whitelist the IPs that Let's Encrypt uses. But they have stated that they don't publish their IPs publicly(opens in new tab):

We don't publish a list of IP addresses we use to validate, and these IP addresses may change at any time.

The most likely reason for this is security, as explained here(opens in new tab):

[…] validation from many different parts of the Internet will help make it harder for attackers to manipulate Internet routing (or DNS) in order to get certificates that they shouldn't be entitled to.

It appears that the only way to pass the HTTP-01 challenge is to:

1. Allow all inbound HTTP traffic on port 80
2. Run sudo certbot renew
3. Disable HTTP traffic again

This completely undermines the idea of automatic renewal… unless you automate it as well. In my case, I couldn't do that because I'm using a GUI firewall. But if you use ufw(opens in new tab), for example, then check Neil Brown's guide to automatically change firewall rules(opens in new tab).

Another solution is to use the DNS-01 challenge(opens in new tab) instead, but it requires that your DNS provider has an API, can be slower due to DNS propagation, and is generally harder to set up.

Conclusion

To perform automatic certificate renewal, your server needs to temporarily have port 80 opened, so it can pass the HTTP challenge by Let's Encrypt. Alternatively, a more involved approach with a DNS challenge can be used instead.