use LE SSL certificate with multiple domains associated to a website

pecus

sean Thanks Sean, I'll summarize but we can go private with a support ticket.

The CDN works (we use StackPath), but the setup on OpalStack is tricky.
The edge SSL certificate is for a domain that is not managed via OpalStack DNS. But we do need to add the domain to our website configuration so that host headers are correctly forwarded. The CMS uses the host header to serve content for the specific site.

When we have a opalstack.com domain (website.user.opalstacked.com) and an external domain (website.com), we cannot use the automatic generation of SSL certs via Let'sEncrypt.

sean

pecus I think as long as our setup knows the StackPath CDN IPs we'll be able to provision the certificate transparently as we do for sites that use Cloudflare or KeyCDN, but I'll have to get our LE dev to test and confirm. I'll open a ticket internally and will follow up here when more info is available.

sean

pecus we've added the StackPath CDN IPs to our setup so please try enabling LE for your site and let me know how it goes 🙂

pecus

sean unfortunately I still get

Unable to generate LetsEncrypt certificate. All site domains must either point to the site IP address (46.165.217.151) or use Opalstack DNS servers.

I verified that the IP address that my domain resolves to is not in the list of IPs that StackPath provides as Edge locations. This might explain why LE on OS does not work (but it does not explain why the IP is not in the block list)

sean

pecus Ok, please email the details to support and we'll take a closer look.

pecus

sean StackPath has provided the 2 anycast IP addresses they use for public facing sites:

151.139.128.11 and 151.139.128.10

These IPs don't call OpalStack in a CDN configuration but resolve the domains mapped to a site configuration.
Hope it helps (I opened a ticket to OS support referencing this issue)

sean

pecus OK, we've added those IPs as well. Please give it another go.

pecus

sean Certificate generation failed. I am not familiar with the challenge/response protocol of LE, but it might be blocked by the firewall ahead of the CDN.

Lets Encrypt failed to generate a certificate for your site: lightbox_prod. The site is served by a CDN which failed to serve the HTTP challenge.

I am working with StackPath support to verify if custom edge rules are needed in order to let the challenge through.

pecus

sean if manual certbot is available (with cron support to allow certificate renewal), I might be able to solve this by generating a certificate only for the site.user.opalstacked.com domain, skipping the user facing domain

sean

pecus for this to work you're going to need to let any URL matching /.well-known/acme-challenge/* pass through the CDN so it can be fetched directly from the server. See if StackPath will let you do that.

pecus

sean SOLVED

StackPath WAF has a Known Bot rule that by default filters Let's Encrypt ACME challenge client. Enabling it allows the challenge to go through to origin (and additional edge rule that bypasses the cache for /.well-known/acme-challenge/* helps forwarding the traffic as a passive proxy)

sean

pecus perfect, glad that worked out 🙂

pecus

I'm reopening (so to speak) this issue because recently we had a failure in renewing the certificate. It seems that one of the challenges is performed by a script that does not advertise itself as an acme client in the user agent string, instead using a generic user agent that gets blocked by our WAF. It is reported as python-requests/2.28.1.
We whitelisted any traffic to /.well-known/acme-challenge/* but are wondering if there was a change.

sean

pecus that's probably a pre-flight check from our end, ie we test the challenge URL before we hand everything off to certbot. It's worked like this for as long as I can remember.

pecus

makes sense, but you should probably customize the user agent to mimic the RFC for the ACME protocol. anything with acme inside would probably do (and, as a side effect probably be picked up as an allowed client by bot-screening waf rules)

sean

pecus thank you for the suggestion, I'll pass it along to the team.