Let's Encrypt News

We've Issued Our First IP Address Certificate

Since Let’s Encrypt started issuing certificates in 2015, people have repeatedly requested the ability to get certificates for IP addresses, an option that only a few certificate authorities have offered. Until now, they’ve had to look elsewhere, because we haven’t provided that feature.

Today, we’ve issued our first certificate for an IP address, as we announced we would in January. As with other new certificate features on our engineering roadmap, we’ll now start gradually rolling out this option to more and more of our subscribers.

Some Background on IP Address Certs

IP addresses are the underlying numerical addresses used on the Internet. Every device on the Internet has one (though, in modern practice, it might be shared with other devices, like when an entire home network shares a single public IP address). The Internet infrastructure uses them to route communications to their proper destination. IP addresses come in two forms, IPv4 and IPv6, and generally look like 54.215.62.21 (IPv4) or 2600:1f1c:446:4900::65 (IPv6).

Most Internet users rarely see or refer to IP addresses directly. Instead, they almost always use domain names like letsencrypt.org to refer to Internet services. The domain name system (DNS) is a part of the Internet infrastructure that’s responsible for allowing software to find the IP addresses associated with a particular domain name. For instance, your web browser can use DNS to find out that the service https://letsencrypt.org/ (Let’s Encrypt’s own website) is provided from the IP addresses 54.215.62.21 and 2600:1f1c:446:4900::65, among several others. This probably happened behind the scenes before you started reading this article! Your web browser needed to know our IP address in order to actually connect to our site and fetch this article.

Because we overwhelmingly tend to think and talk about Internet services in terms of domain names, those are the identifiers that are normally listed in certificates like those that Let’s Encrypt provides to our subscribers. Since you know us as “letsencrypt.org” and not as, say, “54.215.62.21,” it makes the most sense for our domain name to be on our certificate. After all, that’s what you’ll want your web browser to check against. This also gives Internet services more flexibility to be hosted in multiple locations, or to change where they’re hosted, without necessarily needing separate certificates for each server.

In principle, there’s no reason that a certificate couldn’t be issued for an IP address rather than a domain name, and in fact the technical and policy standards for certificates have always allowed this, with a handful of certificate authorities offering this service on a small scale. In Let’s Encrypt’s case, we’ve preferred to wait until some other pieces, like short-lived certs, were in place before we made this option available for our subscribers.

Why IP Address Certs Are Less Common

First and foremost, it’s because Internet users usually know services by domain names, not by IP addresses, and because IP addresses can easily change “behind the scenes” with no prior notice. For instance, a popular site could switch from one cloud hosting company to a different one, and update its DNS records to point at the new host. Most users wouldn’t ever notice the change at all, even though the site’s underlying IP addresses would be completely different.

Second, because IP addresses can change so easily, the sense of “ownership” one might have for them—or that a certificate authority might be able to attest to—tends to be weaker than for a domain name. If you’re hosting something in your house on a residential broadband connection, your Internet service provider most likely doesn’t guarantee that your IP address will stay the same over time. (That is, most home Internet users have a “dynamic IP address” from their ISPs, rather than a “static IP address.”) In that case, you have to contend with the possibility that that address may change often, possibly without warning, and that your old address may be assigned to somebody else.

Third, most Internet service operators don’t expect that end users will ever intentionally connect to their sites directly by IP address. In some cases, when an IP address is shared by different websites or different devices, connecting by IP address alone wouldn’t even work properly. In that case, there’s not much benefit to obtaining a certificate for the IP address!

How Let’s Encrypt Subscribers May Use IP Address Certs

Most current subscribers should be fine with their existing domain name certs and won’t need IP address certs. Subscribers who have a use for an IP address cert are typically already aware of that. A few use cases that we’re aware of include:

• A default page for hosting providers, in case someone pastes a server’s IP address into a browser instead of an individual site name (right now, this normally produces an error in the browser).

• A way to access your website if you don’t have a domain name at all (at some cost in reliability and convenience compared to getting a domain name).

• Securing DNS over HTTPS (DoH) or other infrastructure services. Having a certificate makes it much easier for DoH servers to prove their identities to clients. That could make it more feasible for DoH users or clients to enforce a requirement for a valid publicly-trusted certificate when connecting to DoH servers.

• Securing remote access to some home devices (like network-attached storage servers and Internet-of-things devices) even without a domain name.

• Securing ephemeral connections within cloud hosting infrastructure, like connections between one back-end cloud server and another, or ephemeral connections to administer new or short-lived back-end servers via HTTPS—as long as those servers have at least one public IP address available.

How To Get an IP Address Cert

IP address certificates are available right now in Staging. They should be generally available in Prod later in 2025, at the same time that short-lived certificates become generally available. Prior to general availability we may allow list issuance for a limited number of partners who can provide us with feedback.

Many Let’s Encrypt client applications should already be able to request certificates for IP addresses, although there can be minor technical changes required to support this in some client software.

As a matter of policy, Let’s Encrypt certificates that cover IP addresses must be short-lived certs, valid for only about six days. As such, your ACME client must support the draft ACME Profiles specification, and you must configure it to request the shortlived profile. And, probably not surprisingly, you can’t use the DNS challenge method to prove your control over an IP address; only the http-01 and tls-alpn-01 methods can be used.

If your client software requests an IP address cert with details that aren’t compatible with these policies, the order will be rejected by the ACME server. In this case, your client application may need to be updated or reconfigured. Feel free to ask for help on the Let’s Encrypt community forum if you encounter any problems, either as a client application developer or as an end user.

Tue, 01 Jul 2025 00:00:00 +0000

---

Expiration Notification Service Has Ended

Since Let’s Encrypt started issuing certificates in 2015, people have repeatedly requested the ability to get certificates for IP addresses, an option that only a few certificate authorities have offered. Until now, they’ve had to look elsewhere, because we haven’t provided that feature.

Today, we’ve issued our first certificate for an IP address, as we announced we would in January. As with other new certificate features on our engineering roadmap, we’ll now start gradually rolling out this option to more and more of our subscribers.

Some Background on IP Address Certs

IP addresses are the underlying numerical addresses used on the Internet. Every device on the Internet has one (though, in modern practice, it might be shared with other devices, like when an entire home network shares a single public IP address). The Internet infrastructure uses them to route communications to their proper destination. IP addresses come in two forms, IPv4 and IPv6, and generally look like 54.215.62.21 (IPv4) or 2600:1f1c:446:4900::65 (IPv6).

Most Internet users rarely see or refer to IP addresses directly. Instead, they almost always use domain names like letsencrypt.org to refer to Internet services. The domain name system (DNS) is a part of the Internet infrastructure that’s responsible for allowing software to find the IP addresses associated with a particular domain name. For instance, your web browser can use DNS to find out that the service https://letsencrypt.org/ (Let’s Encrypt’s own website) is provided from the IP addresses 54.215.62.21 and 2600:1f1c:446:4900::65, among several others. This probably happened behind the scenes before you started reading this article! Your web browser needed to know our IP address in order to actually connect to our site and fetch this article.

Because we overwhelmingly tend to think and talk about Internet services in terms of domain names, those are the identifiers that are normally listed in certificates like those that Let’s Encrypt provides to our subscribers. Since you know us as “letsencrypt.org” and not as, say, “54.215.62.21,” it makes the most sense for our domain name to be on our certificate. After all, that’s what you’ll want your web browser to check against. This also gives Internet services more flexibility to be hosted in multiple locations, or to change where they’re hosted, without necessarily needing separate certificates for each server.

In principle, there’s no reason that a certificate couldn’t be issued for an IP address rather than a domain name, and in fact the technical and policy standards for certificates have always allowed this, with a handful of certificate authorities offering this service on a small scale. In Let’s Encrypt’s case, we’ve preferred to wait until some other pieces, like short-lived certs, were in place before we made this option available for our subscribers.

Why IP Address Certs Are Less Common

First and foremost, it’s because Internet users usually know services by domain names, not by IP addresses, and because IP addresses can easily change “behind the scenes” with no prior notice. For instance, a popular site could switch from one cloud hosting company to a different one, and update its DNS records to point at the new host. Most users wouldn’t ever notice the change at all, even though the site’s underlying IP addresses would be completely different.

Second, because IP addresses can change so easily, the sense of “ownership” one might have for them—or that a certificate authority might be able to attest to—tends to be weaker than for a domain name. If you’re hosting something in your house on a residential broadband connection, your Internet service provider most likely doesn’t guarantee that your IP address will stay the same over time. (That is, most home Internet users have a “dynamic IP address” from their ISPs, rather than a “static IP address.”) In that case, you have to contend with the possibility that that address may change often, possibly without warning, and that your old address may be assigned to somebody else.

Third, most Internet service operators don’t expect that end users will ever intentionally connect to their sites directly by IP address. In some cases, when an IP address is shared by different websites or different devices, connecting by IP address alone wouldn’t even work properly. In that case, there’s not much benefit to obtaining a certificate for the IP address!

How Let’s Encrypt Subscribers May Use IP Address Certs

Most current subscribers should be fine with their existing domain name certs and won’t need IP address certs. Subscribers who have a use for an IP address cert are typically already aware of that. A few use cases that we’re aware of include:

• A default page for hosting providers, in case someone pastes a server’s IP address into a browser instead of an individual site name (right now, this normally produces an error in the browser).

• A way to access your website if you don’t have a domain name at all (at some cost in reliability and convenience compared to getting a domain name).

• Securing DNS over HTTPS (DoH) or other infrastructure services. Having a certificate makes it much easier for DoH servers to prove their identities to clients. That could make it more feasible for DoH users or clients to enforce a requirement for a valid publicly-trusted certificate when connecting to DoH servers.

• Securing remote access to some home devices (like network-attached storage servers and Internet-of-things devices) even without a domain name.

• Securing ephemeral connections within cloud hosting infrastructure, like connections between one back-end cloud server and another, or ephemeral connections to administer new or short-lived back-end servers via HTTPS—as long as those servers have at least one public IP address available.

How To Get an IP Address Cert

IP address certificates are available right now in Staging. They should be generally available in Prod later in 2025, at the same time that short-lived certificates become generally available. Prior to general availability we may allow list issuance for a limited number of partners who can provide us with feedback.

Many Let’s Encrypt client applications should already be able to request certificates for IP addresses, although there can be minor technical changes required to support this in some client software.

As a matter of policy, Let’s Encrypt certificates that cover IP addresses must be short-lived certs, valid for only about six days. As such, your ACME client must support the draft ACME Profiles specification, and you must configure it to request the shortlived profile. And, probably not surprisingly, you can’t use the DNS challenge method to prove your control over an IP address; only the http-01 and tls-alpn-01 methods can be used.

If your client software requests an IP address cert with details that aren’t compatible with these policies, the order will be rejected by the ACME server. In this case, your client application may need to be updated or reconfigured. Feel free to ask for help on the Let’s Encrypt community forum if you encounter any problems, either as a client application developer or as an end user.

Since its inception, Let’s Encrypt has been sending expiration notification emails to subscribers that have provided an email address to us via the ACME API. This service ended on June 4, 2025. The decision to end the service is the result of the following factors:

1. Over the past 10 years more and more of our subscribers have been able to put reliable automation into place for certificate renewal.
2. Providing expiration notification emails means that we have to retain millions of email addresses connected to issuance records. As an organization that values privacy, removing this requirement is important to us.
3. Providing expiration notifications costs Let’s Encrypt tens of thousands of dollars per year, money that we believe can be better spent on other aspects of our infrastructure.
4. Providing expiration notifications adds complexity to our infrastructure, which takes time and attention to manage and increases the likelihood of mistakes being made. Over the long term, particularly as we add support for new service components, we need to manage overall complexity by phasing out system components that can no longer be justified.

For those who would like to continue receiving expiration notifications, we recommend using a third party service such as Red Sift Certificates Lite (formerly Hardenize). Red Sift’s monitoring service providing expiration emails is free of charge for up to 250 certificates. More monitoring options can be found here.

We have deleted the email addresses provided to Let’s Encrypt via the ACME API that were stored in our CA database in association with issuance data. This doesn’t affect addresses signed up to mailing lists and other systems. They are managed in a separate ISRG system unassociated with issuance data.

Going forward, if an email address is provided to Let’s Encrypt via the ACME API, Let’s Encrypt will not store the address but will instead forward it to the general ISRG mailing list system unassociated with any account data. If the email address has not been seen before, that system may send an onboarding email with information about how to subscribe to various sources of updates.

If you’d like to stay informed about technical updates and other news about Let’s Encrypt and our parent nonprofit, ISRG, based on the preferences you choose, you can sign up for our email lists below:

Thu, 26 Jun 2025 00:00:00 +0000

---

Reflections on a Year of Sunlight

Since Let’s Encrypt started issuing certificates in 2015, people have repeatedly requested the ability to get certificates for IP addresses, an option that only a few certificate authorities have offered. Until now, they’ve had to look elsewhere, because we haven’t provided that feature.

Today, we’ve issued our first certificate for an IP address, as we announced we would in January. As with other new certificate features on our engineering roadmap, we’ll now start gradually rolling out this option to more and more of our subscribers.

Some Background on IP Address Certs

IP addresses are the underlying numerical addresses used on the Internet. Every device on the Internet has one (though, in modern practice, it might be shared with other devices, like when an entire home network shares a single public IP address). The Internet infrastructure uses them to route communications to their proper destination. IP addresses come in two forms, IPv4 and IPv6, and generally look like 54.215.62.21 (IPv4) or 2600:1f1c:446:4900::65 (IPv6).

Most Internet users rarely see or refer to IP addresses directly. Instead, they almost always use domain names like letsencrypt.org to refer to Internet services. The domain name system (DNS) is a part of the Internet infrastructure that’s responsible for allowing software to find the IP addresses associated with a particular domain name. For instance, your web browser can use DNS to find out that the service https://letsencrypt.org/ (Let’s Encrypt’s own website) is provided from the IP addresses 54.215.62.21 and 2600:1f1c:446:4900::65, among several others. This probably happened behind the scenes before you started reading this article! Your web browser needed to know our IP address in order to actually connect to our site and fetch this article.

Because we overwhelmingly tend to think and talk about Internet services in terms of domain names, those are the identifiers that are normally listed in certificates like those that Let’s Encrypt provides to our subscribers. Since you know us as “letsencrypt.org” and not as, say, “54.215.62.21,” it makes the most sense for our domain name to be on our certificate. After all, that’s what you’ll want your web browser to check against. This also gives Internet services more flexibility to be hosted in multiple locations, or to change where they’re hosted, without necessarily needing separate certificates for each server.

In principle, there’s no reason that a certificate couldn’t be issued for an IP address rather than a domain name, and in fact the technical and policy standards for certificates have always allowed this, with a handful of certificate authorities offering this service on a small scale. In Let’s Encrypt’s case, we’ve preferred to wait until some other pieces, like short-lived certs, were in place before we made this option available for our subscribers.

Why IP Address Certs Are Less Common

First and foremost, it’s because Internet users usually know services by domain names, not by IP addresses, and because IP addresses can easily change “behind the scenes” with no prior notice. For instance, a popular site could switch from one cloud hosting company to a different one, and update its DNS records to point at the new host. Most users wouldn’t ever notice the change at all, even though the site’s underlying IP addresses would be completely different.

Second, because IP addresses can change so easily, the sense of “ownership” one might have for them—or that a certificate authority might be able to attest to—tends to be weaker than for a domain name. If you’re hosting something in your house on a residential broadband connection, your Internet service provider most likely doesn’t guarantee that your IP address will stay the same over time. (That is, most home Internet users have a “dynamic IP address” from their ISPs, rather than a “static IP address.”) In that case, you have to contend with the possibility that that address may change often, possibly without warning, and that your old address may be assigned to somebody else.

Third, most Internet service operators don’t expect that end users will ever intentionally connect to their sites directly by IP address. In some cases, when an IP address is shared by different websites or different devices, connecting by IP address alone wouldn’t even work properly. In that case, there’s not much benefit to obtaining a certificate for the IP address!

How Let’s Encrypt Subscribers May Use IP Address Certs

Most current subscribers should be fine with their existing domain name certs and won’t need IP address certs. Subscribers who have a use for an IP address cert are typically already aware of that. A few use cases that we’re aware of include:

• A default page for hosting providers, in case someone pastes a server’s IP address into a browser instead of an individual site name (right now, this normally produces an error in the browser).

• A way to access your website if you don’t have a domain name at all (at some cost in reliability and convenience compared to getting a domain name).

• Securing DNS over HTTPS (DoH) or other infrastructure services. Having a certificate makes it much easier for DoH servers to prove their identities to clients. That could make it more feasible for DoH users or clients to enforce a requirement for a valid publicly-trusted certificate when connecting to DoH servers.

• Securing remote access to some home devices (like network-attached storage servers and Internet-of-things devices) even without a domain name.

• Securing ephemeral connections within cloud hosting infrastructure, like connections between one back-end cloud server and another, or ephemeral connections to administer new or short-lived back-end servers via HTTPS—as long as those servers have at least one public IP address available.

How To Get an IP Address Cert

IP address certificates are available right now in Staging. They should be generally available in Prod later in 2025, at the same time that short-lived certificates become generally available. Prior to general availability we may allow list issuance for a limited number of partners who can provide us with feedback.

Many Let’s Encrypt client applications should already be able to request certificates for IP addresses, although there can be minor technical changes required to support this in some client software.

As a matter of policy, Let’s Encrypt certificates that cover IP addresses must be short-lived certs, valid for only about six days. As such, your ACME client must support the draft ACME Profiles specification, and you must configure it to request the shortlived profile. And, probably not surprisingly, you can’t use the DNS challenge method to prove your control over an IP address; only the http-01 and tls-alpn-01 methods can be used.

If your client software requests an IP address cert with details that aren’t compatible with these policies, the order will be rejected by the ACME server. In this case, your client application may need to be updated or reconfigured. Feel free to ask for help on the Let’s Encrypt community forum if you encounter any problems, either as a client application developer or as an end user.

Since its inception, Let’s Encrypt has been sending expiration notification emails to subscribers that have provided an email address to us via the ACME API. This service ended on June 4, 2025. The decision to end the service is the result of the following factors:

1. Over the past 10 years more and more of our subscribers have been able to put reliable automation into place for certificate renewal.
2. Providing expiration notification emails means that we have to retain millions of email addresses connected to issuance records. As an organization that values privacy, removing this requirement is important to us.
3. Providing expiration notifications costs Let’s Encrypt tens of thousands of dollars per year, money that we believe can be better spent on other aspects of our infrastructure.
4. Providing expiration notifications adds complexity to our infrastructure, which takes time and attention to manage and increases the likelihood of mistakes being made. Over the long term, particularly as we add support for new service components, we need to manage overall complexity by phasing out system components that can no longer be justified.

For those who would like to continue receiving expiration notifications, we recommend using a third party service such as Red Sift Certificates Lite (formerly Hardenize). Red Sift’s monitoring service providing expiration emails is free of charge for up to 250 certificates. More monitoring options can be found here.

We have deleted the email addresses provided to Let’s Encrypt via the ACME API that were stored in our CA database in association with issuance data. This doesn’t affect addresses signed up to mailing lists and other systems. They are managed in a separate ISRG system unassociated with issuance data.

Going forward, if an email address is provided to Let’s Encrypt via the ACME API, Let’s Encrypt will not store the address but will instead forward it to the general ISRG mailing list system unassociated with any account data. If the email address has not been seen before, that system may send an onboarding email with information about how to subscribe to various sources of updates.

If you’d like to stay informed about technical updates and other news about Let’s Encrypt and our parent nonprofit, ISRG, based on the preferences you choose, you can sign up for our email lists below:

The Certificate Transparency ecosystem has been improving transparency for the web PKI since 2013. It helps make clear exactly what certificates each certificate authority has issued and makes sure errors or compromises of certificate authorities are detectable.

Let’s Encrypt participates in CT both as a certificate issuer and as a log operator. For the past year, we’ve also been running an experiment to help validate a next-generation design for Certificate Transparency logs. That experiment is now nearing a successful conclusion. We’ve demonstrated that the new architecture (called the “Static CT API”) works well, providing greater efficiency and making it easier to run huge and reliable CT log services with comparatively modest resources. The Static CT API also makes it easier to download and share data from CT logs.

The Sunlight log implementation, alongside other Static CT API log implementations, is now on a path to production use. Browsers are now officially accepting Static CT API logs into their log programs as a means to help guarantee that the contents of CA-issued certificates are all publicly disclosed and publicly accessible (see Safari’s and Chrome’s recent announcements), although the browsers also require the continued use of a traditional RFC 6962 log alongside the new type.

All of this is good news for everyone who runs, submits certificates to, or monitors a CT log: as the new architecture gets adopted, we can expect to see more organizations running more logs, at lower cost, and with greater overall capacity to keep up with the large volume of publicly-trusted certificates.

Certificate Transparency

Certificate Transparency (CT) was introduced in 2013 in response to concerns about how Internet users could detect misbehavior and compromise of certificate authorities. Prior to CT, it was possible for a CA to issue an inaccurate or malicious certificate that could be used to attack a relatively small number of users, and that might never come to wider attention. A team led by Google responded to this by creating a transparency log mechanism, where certificate authorities (like Let’s Encrypt) must disclose all of the certificates that we issue by submitting them to public log services. Web browsers now generally reject certificates unless the certificates include cryptographic proof (“Signed Certificate Timestamps”, or SCTs) demonstrating that they were submitted to and accepted by such logs.

The CT logs themselves use a cryptographic append-only ledger to prove that they haven’t deleted or modified their records. There are currently over a dozen CT log services, most of them also run by certificate authorities, including Let’s Encrypt’s own Oak log.

The Static CT API

The original 2013 CT log design has been used with relatively few technical changes since it was first introduced, but several other transparency logging systems have been created in other areas, such as sumdb for Golang, which helps ensure that the contents of Golang package updates are publicly recorded. While they were originally inspired by CT, more-recently invented transparency logs have improved on its design.

The current major evolution of CT was led by Filippo Valsorda, a cryptographer with an interest in transparency log mechanisms, with help from others in the CT ecosystem. Portions of the new design are directly based on sumdb. In addition to designing the new architecture, Valsorda also wrote the implementation that we’ve been using, called Sunlight, with support from Let’s Encrypt. We’re excited to see that there are now at least three other compatible implementations: Google’s trillian-tessera, Cloudflare’s Azul, and an independent project called Itko.

The biggest change for the Static CT API is that logs are now represented, and downloaded by verifiers, as simple collections of flat files (called “tiles,” so some implementers have also been referring to these as “tiled logs” or “tlogs”). Anyone who wants to download log data can do so just by downloading these files. This is great for log operators because these simple file downloads can be distributed in various ways, including caching by a CDN, which was less practical and efficient for the classic CT API.

The new design is also simpler and more efficient from the log operator’s perspective, making it cheaper to run logs. As we said last year, this may enable us and other operators to increase reliability and availability by running several separate logs, likely with lower overall resource requirements than a single traditional log.

Our Sunlight experiment

For the past year, we’ve run three Sunlight logs, called Twig, Willow, and Sycamore. We’ve been logging all of our own issued certificates, which represent a majority of the total volume of all publicly-trusted certificates, into our Sunlight logs. Sunlight logged these certificates quickly and correctly on relatively modest server hardware. Notably, each log’s write side was handled comfortably by just a single server. We also achieved high availability for these log services throughout the course of this experiment. (Because our Sunlight logs are not yet trusted by web browsers, we didn’t include the SCT proofs that they returned to us in the actual certificates we gave out to our subscribers; those proofs wouldn’t have been of use to our subscribers yet and would just have taken up space.)

A potential failure mode of traditional CT logs is that they could be unacceptably slow in incorporating newly-submitted certificates (known as missing the maximum merge delay), which can result in a log becoming distrusted. This isn’t a possibility for our new Sunlight-based logs: they always completely incorporate newly-submitted certificates before returning an SCT to the submitter, so the effective merge delay is zero! Of course, any log can suffer outages for a variety of reasons, but this feature of Sunlight makes it less likely that any outages will be fatal to a log’s continued operation.

We’ve demonstrated that Sunlight and the Static CT API work in practice, and this demonstration has helped to confirm the browser developers’ hope that Static CT API logs can become an officially-supported part of CT. As a result, the major browsers that enforce CT have now permitted Static CT API logs to apply for inclusion in browsers as publicly-trusted logs, and we’re preparing to apply for this status for our Willow and Sycamore logs with the Chrome and Safari CT log programs.

Let’s Encrypt will run at least these two logs, and possibly others over time, for the foreseeable future. Once they’re trusted by browsers, we’ll encourage other CAs to submit to them as well, and we’ll begin including SCTs from these logs in our own certificates (alongside SCTs from traditional CT logs).

How to participate

The new Static CT API and the rollout of tile-based logs will bring various changes and opportunities for community members.

New Certificate Transparency log operators

Companies and non-profit organizations could help support the web PKI by running a CT log and applying for it to be publicly trusted. Implementations like Sunlight will have substantially lower resource requirements than first-generation CT logs, particularly when cached behind a CDN. The biggest resource demands for a log operator will be storage and upstream bandwidth. A publicly-trusted log is also expected to maintain relatively high availability, because CAs need logs to be available in order to continue issuing certificates.

We don’t have statistics to share about the exact resource requirements for such a log yet, but after we have practical experience running a fully publicly-trusted Sunlight log, we should be able to make this more concrete. As noted above, the compute side of the log can be handled by a single server. Sunlight author Filippo Valsorda has recently started running a Sunlight log—also on just a single server—and offered more detailed cost breakdowns for that log’s setup, with an estimated total cost around $10,000 per year. The costs for our production Static CT API logs may be higher than those for Filippo’s log, but still far less than the costs for our traditional RFC 6962 logs.

As with trust decisions about CAs, browser developers are the authorities about which CT logs become publicly trusted. Although any person or organization can run a log, browser developers will generally prefer to trust logs whose continued availability they’re confident of—typically those run by stable organizations with experience running some form of public Internet services. Unlike becoming a certificate authority, running a log does not require a formal audit, as the validation of the log’s availability and correctness can be performed purely by observation.

Certificate authorities

Once the Willow and Sycamore logs are trusted by browsers, our fellow certificate authorities can choose to start logging certificates to them as part of their issuance processes. (Initially, you should still include at least one SCT from a traditional CT log in each certificate.) The details, including the log API endpoints and keys, are available at our CT log page. You can start submitting to these logs right away if you prefer; just bear in mind that the SCTs they return aren’t useful to subscribers yet, and won’t be useful until browsers are updated to trust the new logs.

CT data users

You can monitor CT in order to watch for certificate issuances for your own domain names, or as part of monitoring or security products or services, or for Internet security research purposes. Many of our colleagues have been doing this for some time as a part of various tools they maintain. The Static CT API should make this easier, because you’ll be able to download and share log tiles as sets of ordinary files.

If you already run such monitoring tools, please note that you’ll need to update your data pipeline in order to access Static CT API logs; since the read API is not backwards-compatible, CT API clients will need to be modified to support the new API. Without updated tools, your view of the CT system will become partial!

Also note that getting a complete view of all of CT will still require downloading data from traditional logs, which will probably continue to be true for several years.

Software developers

As logs based on the new API enter production use, it will be important to have tools to interact with and search these logs. We can all benefit from more software that understands how to do this. Since file downloads are such a familiar piece of software functionality, it will probably be easier for developers to develop against the new API compared to the original one.

We’ve also continued to see greater integration of transparency logging tools into other kinds of services, such as software updates. There’s a growing transparency log ecosystem that’s always in need of more tools and integrations. As we mentioned above, transparency logs are increasingly learning from one another, and there are also mechanisms for more direct integration between different kinds of transparency logs (known as “witnessing”). Software developers can help improve different aspects of Internet security by contributing to this active and growing area.

Conclusion

The Certificate Transparency community and larger transparency logging community have experienced a virtuous cycle of innovation, sharing ideas and implementation code between different systems and demonstrating the feasibility of new mechanisms and functionality. With the advent of tile-based logging in CT, the state of the art has moved forward in a way that helps log operators run our logs much more efficiently without compromising security.

We’re proud to have participated in this experiment and the engineering conversation around the evolution of logging architectures. Now that we’ve shown how well the new API really works at scale, we look forward to having publicly-trusted Sunlight logs later this year!

Wed, 11 Jun 2025 00:00:00 +0000

---

How We Reduced the Impact of Zombie Clients

Since Let’s Encrypt started issuing certificates in 2015, people have repeatedly requested the ability to get certificates for IP addresses, an option that only a few certificate authorities have offered. Until now, they’ve had to look elsewhere, because we haven’t provided that feature.

Today, we’ve issued our first certificate for an IP address, as we announced we would in January. As with other new certificate features on our engineering roadmap, we’ll now start gradually rolling out this option to more and more of our subscribers.

Some Background on IP Address Certs

IP addresses are the underlying numerical addresses used on the Internet. Every device on the Internet has one (though, in modern practice, it might be shared with other devices, like when an entire home network shares a single public IP address). The Internet infrastructure uses them to route communications to their proper destination. IP addresses come in two forms, IPv4 and IPv6, and generally look like 54.215.62.21 (IPv4) or 2600:1f1c:446:4900::65 (IPv6).

Most Internet users rarely see or refer to IP addresses directly. Instead, they almost always use domain names like letsencrypt.org to refer to Internet services. The domain name system (DNS) is a part of the Internet infrastructure that’s responsible for allowing software to find the IP addresses associated with a particular domain name. For instance, your web browser can use DNS to find out that the service https://letsencrypt.org/ (Let’s Encrypt’s own website) is provided from the IP addresses 54.215.62.21 and 2600:1f1c:446:4900::65, among several others. This probably happened behind the scenes before you started reading this article! Your web browser needed to know our IP address in order to actually connect to our site and fetch this article.

Because we overwhelmingly tend to think and talk about Internet services in terms of domain names, those are the identifiers that are normally listed in certificates like those that Let’s Encrypt provides to our subscribers. Since you know us as “letsencrypt.org” and not as, say, “54.215.62.21,” it makes the most sense for our domain name to be on our certificate. After all, that’s what you’ll want your web browser to check against. This also gives Internet services more flexibility to be hosted in multiple locations, or to change where they’re hosted, without necessarily needing separate certificates for each server.

In principle, there’s no reason that a certificate couldn’t be issued for an IP address rather than a domain name, and in fact the technical and policy standards for certificates have always allowed this, with a handful of certificate authorities offering this service on a small scale. In Let’s Encrypt’s case, we’ve preferred to wait until some other pieces, like short-lived certs, were in place before we made this option available for our subscribers.

Why IP Address Certs Are Less Common

First and foremost, it’s because Internet users usually know services by domain names, not by IP addresses, and because IP addresses can easily change “behind the scenes” with no prior notice. For instance, a popular site could switch from one cloud hosting company to a different one, and update its DNS records to point at the new host. Most users wouldn’t ever notice the change at all, even though the site’s underlying IP addresses would be completely different.

Second, because IP addresses can change so easily, the sense of “ownership” one might have for them—or that a certificate authority might be able to attest to—tends to be weaker than for a domain name. If you’re hosting something in your house on a residential broadband connection, your Internet service provider most likely doesn’t guarantee that your IP address will stay the same over time. (That is, most home Internet users have a “dynamic IP address” from their ISPs, rather than a “static IP address.”) In that case, you have to contend with the possibility that that address may change often, possibly without warning, and that your old address may be assigned to somebody else.

Third, most Internet service operators don’t expect that end users will ever intentionally connect to their sites directly by IP address. In some cases, when an IP address is shared by different websites or different devices, connecting by IP address alone wouldn’t even work properly. In that case, there’s not much benefit to obtaining a certificate for the IP address!

How Let’s Encrypt Subscribers May Use IP Address Certs

Most current subscribers should be fine with their existing domain name certs and won’t need IP address certs. Subscribers who have a use for an IP address cert are typically already aware of that. A few use cases that we’re aware of include:

• A default page for hosting providers, in case someone pastes a server’s IP address into a browser instead of an individual site name (right now, this normally produces an error in the browser).

• A way to access your website if you don’t have a domain name at all (at some cost in reliability and convenience compared to getting a domain name).

• Securing DNS over HTTPS (DoH) or other infrastructure services. Having a certificate makes it much easier for DoH servers to prove their identities to clients. That could make it more feasible for DoH users or clients to enforce a requirement for a valid publicly-trusted certificate when connecting to DoH servers.

• Securing remote access to some home devices (like network-attached storage servers and Internet-of-things devices) even without a domain name.

• Securing ephemeral connections within cloud hosting infrastructure, like connections between one back-end cloud server and another, or ephemeral connections to administer new or short-lived back-end servers via HTTPS—as long as those servers have at least one public IP address available.

How To Get an IP Address Cert

IP address certificates are available right now in Staging. They should be generally available in Prod later in 2025, at the same time that short-lived certificates become generally available. Prior to general availability we may allow list issuance for a limited number of partners who can provide us with feedback.

Many Let’s Encrypt client applications should already be able to request certificates for IP addresses, although there can be minor technical changes required to support this in some client software.

As a matter of policy, Let’s Encrypt certificates that cover IP addresses must be short-lived certs, valid for only about six days. As such, your ACME client must support the draft ACME Profiles specification, and you must configure it to request the shortlived profile. And, probably not surprisingly, you can’t use the DNS challenge method to prove your control over an IP address; only the http-01 and tls-alpn-01 methods can be used.

If your client software requests an IP address cert with details that aren’t compatible with these policies, the order will be rejected by the ACME server. In this case, your client application may need to be updated or reconfigured. Feel free to ask for help on the Let’s Encrypt community forum if you encounter any problems, either as a client application developer or as an end user.

Since its inception, Let’s Encrypt has been sending expiration notification emails to subscribers that have provided an email address to us via the ACME API. This service ended on June 4, 2025. The decision to end the service is the result of the following factors:

1. Over the past 10 years more and more of our subscribers have been able to put reliable automation into place for certificate renewal.
2. Providing expiration notification emails means that we have to retain millions of email addresses connected to issuance records. As an organization that values privacy, removing this requirement is important to us.
3. Providing expiration notifications costs Let’s Encrypt tens of thousands of dollars per year, money that we believe can be better spent on other aspects of our infrastructure.
4. Providing expiration notifications adds complexity to our infrastructure, which takes time and attention to manage and increases the likelihood of mistakes being made. Over the long term, particularly as we add support for new service components, we need to manage overall complexity by phasing out system components that can no longer be justified.

For those who would like to continue receiving expiration notifications, we recommend using a third party service such as Red Sift Certificates Lite (formerly Hardenize). Red Sift’s monitoring service providing expiration emails is free of charge for up to 250 certificates. More monitoring options can be found here.

We have deleted the email addresses provided to Let’s Encrypt via the ACME API that were stored in our CA database in association with issuance data. This doesn’t affect addresses signed up to mailing lists and other systems. They are managed in a separate ISRG system unassociated with issuance data.

Going forward, if an email address is provided to Let’s Encrypt via the ACME API, Let’s Encrypt will not store the address but will instead forward it to the general ISRG mailing list system unassociated with any account data. If the email address has not been seen before, that system may send an onboarding email with information about how to subscribe to various sources of updates.

If you’d like to stay informed about technical updates and other news about Let’s Encrypt and our parent nonprofit, ISRG, based on the preferences you choose, you can sign up for our email lists below:

The Certificate Transparency ecosystem has been improving transparency for the web PKI since 2013. It helps make clear exactly what certificates each certificate authority has issued and makes sure errors or compromises of certificate authorities are detectable.

Let’s Encrypt participates in CT both as a certificate issuer and as a log operator. For the past year, we’ve also been running an experiment to help validate a next-generation design for Certificate Transparency logs. That experiment is now nearing a successful conclusion. We’ve demonstrated that the new architecture (called the “Static CT API”) works well, providing greater efficiency and making it easier to run huge and reliable CT log services with comparatively modest resources. The Static CT API also makes it easier to download and share data from CT logs.

The Sunlight log implementation, alongside other Static CT API log implementations, is now on a path to production use. Browsers are now officially accepting Static CT API logs into their log programs as a means to help guarantee that the contents of CA-issued certificates are all publicly disclosed and publicly accessible (see Safari’s and Chrome’s recent announcements), although the browsers also require the continued use of a traditional RFC 6962 log alongside the new type.

All of this is good news for everyone who runs, submits certificates to, or monitors a CT log: as the new architecture gets adopted, we can expect to see more organizations running more logs, at lower cost, and with greater overall capacity to keep up with the large volume of publicly-trusted certificates.

Certificate Transparency

Certificate Transparency (CT) was introduced in 2013 in response to concerns about how Internet users could detect misbehavior and compromise of certificate authorities. Prior to CT, it was possible for a CA to issue an inaccurate or malicious certificate that could be used to attack a relatively small number of users, and that might never come to wider attention. A team led by Google responded to this by creating a transparency log mechanism, where certificate authorities (like Let’s Encrypt) must disclose all of the certificates that we issue by submitting them to public log services. Web browsers now generally reject certificates unless the certificates include cryptographic proof (“Signed Certificate Timestamps”, or SCTs) demonstrating that they were submitted to and accepted by such logs.

The CT logs themselves use a cryptographic append-only ledger to prove that they haven’t deleted or modified their records. There are currently over a dozen CT log services, most of them also run by certificate authorities, including Let’s Encrypt’s own Oak log.

The Static CT API

The original 2013 CT log design has been used with relatively few technical changes since it was first introduced, but several other transparency logging systems have been created in other areas, such as sumdb for Golang, which helps ensure that the contents of Golang package updates are publicly recorded. While they were originally inspired by CT, more-recently invented transparency logs have improved on its design.

The current major evolution of CT was led by Filippo Valsorda, a cryptographer with an interest in transparency log mechanisms, with help from others in the CT ecosystem. Portions of the new design are directly based on sumdb. In addition to designing the new architecture, Valsorda also wrote the implementation that we’ve been using, called Sunlight, with support from Let’s Encrypt. We’re excited to see that there are now at least three other compatible implementations: Google’s trillian-tessera, Cloudflare’s Azul, and an independent project called Itko.

The biggest change for the Static CT API is that logs are now represented, and downloaded by verifiers, as simple collections of flat files (called “tiles,” so some implementers have also been referring to these as “tiled logs” or “tlogs”). Anyone who wants to download log data can do so just by downloading these files. This is great for log operators because these simple file downloads can be distributed in various ways, including caching by a CDN, which was less practical and efficient for the classic CT API.

The new design is also simpler and more efficient from the log operator’s perspective, making it cheaper to run logs. As we said last year, this may enable us and other operators to increase reliability and availability by running several separate logs, likely with lower overall resource requirements than a single traditional log.

Our Sunlight experiment

For the past year, we’ve run three Sunlight logs, called Twig, Willow, and Sycamore. We’ve been logging all of our own issued certificates, which represent a majority of the total volume of all publicly-trusted certificates, into our Sunlight logs. Sunlight logged these certificates quickly and correctly on relatively modest server hardware. Notably, each log’s write side was handled comfortably by just a single server. We also achieved high availability for these log services throughout the course of this experiment. (Because our Sunlight logs are not yet trusted by web browsers, we didn’t include the SCT proofs that they returned to us in the actual certificates we gave out to our subscribers; those proofs wouldn’t have been of use to our subscribers yet and would just have taken up space.)

A potential failure mode of traditional CT logs is that they could be unacceptably slow in incorporating newly-submitted certificates (known as missing the maximum merge delay), which can result in a log becoming distrusted. This isn’t a possibility for our new Sunlight-based logs: they always completely incorporate newly-submitted certificates before returning an SCT to the submitter, so the effective merge delay is zero! Of course, any log can suffer outages for a variety of reasons, but this feature of Sunlight makes it less likely that any outages will be fatal to a log’s continued operation.

We’ve demonstrated that Sunlight and the Static CT API work in practice, and this demonstration has helped to confirm the browser developers’ hope that Static CT API logs can become an officially-supported part of CT. As a result, the major browsers that enforce CT have now permitted Static CT API logs to apply for inclusion in browsers as publicly-trusted logs, and we’re preparing to apply for this status for our Willow and Sycamore logs with the Chrome and Safari CT log programs.

Let’s Encrypt will run at least these two logs, and possibly others over time, for the foreseeable future. Once they’re trusted by browsers, we’ll encourage other CAs to submit to them as well, and we’ll begin including SCTs from these logs in our own certificates (alongside SCTs from traditional CT logs).

How to participate

The new Static CT API and the rollout of tile-based logs will bring various changes and opportunities for community members.

New Certificate Transparency log operators

Companies and non-profit organizations could help support the web PKI by running a CT log and applying for it to be publicly trusted. Implementations like Sunlight will have substantially lower resource requirements than first-generation CT logs, particularly when cached behind a CDN. The biggest resource demands for a log operator will be storage and upstream bandwidth. A publicly-trusted log is also expected to maintain relatively high availability, because CAs need logs to be available in order to continue issuing certificates.

We don’t have statistics to share about the exact resource requirements for such a log yet, but after we have practical experience running a fully publicly-trusted Sunlight log, we should be able to make this more concrete. As noted above, the compute side of the log can be handled by a single server. Sunlight author Filippo Valsorda has recently started running a Sunlight log—also on just a single server—and offered more detailed cost breakdowns for that log’s setup, with an estimated total cost around $10,000 per year. The costs for our production Static CT API logs may be higher than those for Filippo’s log, but still far less than the costs for our traditional RFC 6962 logs.

As with trust decisions about CAs, browser developers are the authorities about which CT logs become publicly trusted. Although any person or organization can run a log, browser developers will generally prefer to trust logs whose continued availability they’re confident of—typically those run by stable organizations with experience running some form of public Internet services. Unlike becoming a certificate authority, running a log does not require a formal audit, as the validation of the log’s availability and correctness can be performed purely by observation.

Certificate authorities

Once the Willow and Sycamore logs are trusted by browsers, our fellow certificate authorities can choose to start logging certificates to them as part of their issuance processes. (Initially, you should still include at least one SCT from a traditional CT log in each certificate.) The details, including the log API endpoints and keys, are available at our CT log page. You can start submitting to these logs right away if you prefer; just bear in mind that the SCTs they return aren’t useful to subscribers yet, and won’t be useful until browsers are updated to trust the new logs.

CT data users

You can monitor CT in order to watch for certificate issuances for your own domain names, or as part of monitoring or security products or services, or for Internet security research purposes. Many of our colleagues have been doing this for some time as a part of various tools they maintain. The Static CT API should make this easier, because you’ll be able to download and share log tiles as sets of ordinary files.

If you already run such monitoring tools, please note that you’ll need to update your data pipeline in order to access Static CT API logs; since the read API is not backwards-compatible, CT API clients will need to be modified to support the new API. Without updated tools, your view of the CT system will become partial!

Also note that getting a complete view of all of CT will still require downloading data from traditional logs, which will probably continue to be true for several years.

Software developers

As logs based on the new API enter production use, it will be important to have tools to interact with and search these logs. We can all benefit from more software that understands how to do this. Since file downloads are such a familiar piece of software functionality, it will probably be easier for developers to develop against the new API compared to the original one.

We’ve also continued to see greater integration of transparency logging tools into other kinds of services, such as software updates. There’s a growing transparency log ecosystem that’s always in need of more tools and integrations. As we mentioned above, transparency logs are increasingly learning from one another, and there are also mechanisms for more direct integration between different kinds of transparency logs (known as “witnessing”). Software developers can help improve different aspects of Internet security by contributing to this active and growing area.

Conclusion

The Certificate Transparency community and larger transparency logging community have experienced a virtuous cycle of innovation, sharing ideas and implementation code between different systems and demonstrating the feasibility of new mechanisms and functionality. With the advent of tile-based logging in CT, the state of the art has moved forward in a way that helps log operators run our logs much more efficiently without compromising security.

We’re proud to have participated in this experiment and the engineering conversation around the evolution of logging architectures. Now that we’ve shown how well the new API really works at scale, we look forward to having publicly-trusted Sunlight logs later this year!

Every night, right around midnight (mainly UTC), a horde of zombies wakes up and clamors for … digital certificates!

The zombies in question are abandoned or misconfigured Internet servers and ACME clients that have been set to request certificates from Let’s Encrypt. As our certificates last for at most 90 days, these zombie clients’ software knows that their certificates are out-of-date and need to be replaced. What they don’t realize is that their quest for new certificates is doomed! These devices are cursed to seek certificates again and again, never receiving them.

But they do use up a lot of certificate authority resources in the process.

The Zombie Client Problem

Unlike a human being, software doesn’t give up in frustration, or try to modify its approach, when it repeatedly fails at the same task. Our emphasis on automation means that the vast majority of Let’s Encrypt certificate renewals are performed by automated software. This is great when those renewals succeed, but it also means that forgotten clients and devices can continue requesting renewals unsuccessfully for months, or even years.

How might that happen? Most often, it happens when a device no longer has a domain name pointed to it. The device itself doesn’t know that this has changed, so it treats renewal failures as transient even though they are actually permanent. For instance:

• An organization may have allowed a domain name registration to lapse because it is no longer needed, but its servers are still configured to request certs for it.
• Or, a home user stopped using a particular dynamic-DNS domain with a network-attached storage device, but is still using that device at home. The device doesn’t realize that the user no longer expects to use the name, so it keeps requesting certs for it.
• Or, a web hosting or CDN customer migrated to a different service provider, but never informed the old service provider. The old service provider’s servers keep requesting certs unsuccessfully. If the customer was in a free service tier, there might not be invoices or charges reminding the customer to cancel the service.
• Or any number of other, subtler changes in a subscriber’s infrastructure, such as changing a firewall rule or some webserver configuration.

At the scale of Let’s Encrypt, which now covers hundreds of millions of names, scenarios like these have become common, and their impact has become substantial. In 2024, we noticed that about half of all certificate requests to the Let’s Encrypt ACME API came from about a million accounts that never successfully complete any validations. Many of these had completed validations and issued certificates sometime in the past, but nowadays every single one of their validation attempts fails, and they show no signs that this will change anytime soon.

Unfortunately, trying to validate those futile requests still uses resources. Our CA software has to generate challenges, reach out and attempt to validate them over the Internet, detect and report failures, and record all of the associated information in our databases and audit logs. And over time, we’ve seen more and more recurring failures: accounts that always fail their issuance requests have been growing at around 18% per year.

In January, we mentioned that we had been addressing the zombie client problem through our rate limit system. This post provides more detail on that progress. 

Our Rate Limit Philosophy

If you’ve used Let’s Encrypt as a subscriber, you may have run into one of our rate limits at some point, maybe during your initial setup process. We have eight different kinds of rate limits in place now; as our January post describes, they’ve become more algorithmically sophisticated and grown to address a wider range of problems. A key principle for Let’s Encrypt is that our rate limiting is not a punishment. We don’t think of rate limits as a way of retaliating against a client for misbehavior. Rate limits are simply a tool to maximize the efficient use of our limited resources and prevent people and programs from using up those resources for no constructive purpose.

We’ve consistently tried to design our rate limit mechanisms in line with that philosophy. So if a misconfiguration or misunderstanding has caused excessive requests in the past, we’re still happy to welcome the user in question back and start issuing them certificates again—once the problem has been addressed. We want the rate limits to put a brake on wasteful use of our systems, but not to frustrate users who are actively trying to make Let’s Encrypt work for them.

In addition, we’ve always implemented our rate limits to err on the side of permissiveness. For example, if the Redis instances where rate limits are tracked have an outage or lose data, the system is designed to permit more issuance rather than less issuance as a result.

We wanted to create additional limits that would target zombie clients, but in a correspondingly non-punitive way that would avoid any disruption to valid issuance, and welcome subscribers back quickly if they happened to notice and fix a long-time problem with their setups.

Our Zombie-Related Rate Limits and Their Impact

In planning a new zombie-specific response, we decided on a “pausing” approach, which can temporarily limit an account’s ability to proceed with certificate requests. The core idea is that, if a particular account consistently fails to complete validation for a particular hostname, we’ll pause that account-hostname pair. The pause means that any new order requests from that account for that hostname will be rejected immediately, before we get to the resource-intensive validation phase.

This approach is more finely targeted than pausing an entire account. Pausing account-hostname pairs means that your ability to issue certs for a specific name could be paused due to repeated failures, but you can still get all of your other certs like normal. So a large hosting provider doesn’t have to fear that its certificate issuance on behalf of one customer will be affected by renewal failures related to a problem with a different customer’s domain name. The account-specificity of the pause, in turn, means that validation failures from one subscriber or device won’t prevent a different subscriber or device from attempting to validate the same name, as long as the devices in question don’t share a single Let’s Encrypt account.

In September 2024, we began applying our zombie rate limits manually by pausing about 21,000 of the most recurrently-failing account-hostname pairs, those which were consistently repeating the same failed requests many times per day, every day. After implementing that first round of pauses, we immediately saw a significant impact on our failed request rates. As we announced at that time, we also began using a formula to automatically pause other zombie client account-hostname pairs from December 2024 onward. The associated new rate limit is called “Consecutive Authorization Failures per Hostname Per Account” (and is independent of the existing “Authorization Failures per Hostname Per Account” limit, which resets every hour).

This formula relates to the frequency of successive failed issuance requests for the same domain name by the same Let’s Encrypt account. It applies only to failures that happen again and again, with no successful issuances at all in between: a single successful validation immediately resets the rate limit all the way to zero. Like all of our rate limits, this is not a punitive measure but is simply intended to reduce the waste of resources. So, we decided to set the thresholds rather high in the expectation that we would catch only the most disruptive zombie clients, and ultimately only those clients that were extremely unlikely to succeed in the future based on their substantial history of failed requests. We don’t hurry to block requesters as zombies: according to our current formula, client software following the default established by EFF’s Certbot (two renewal attempts per day) would be paused as a zombie only after about ten years of constant failures. More aggressive failed issuance attempts will get a client paused sooner, but clients will generally have to fail hundreds or thousands of attempts in a row before they are paused.

Most subscribers using mainstream client applications with default configurations will never encounter this rate limit, even if they forget to deactivate renewal attempts for domains that are no longer pointed at their servers. As described below, our current limit is already providing noticeable benefits with minimal disruption, and we’re likely to tighten it a bit in the near future, so it will trigger after somewhat fewer consecutive failures.

Self-Service Unpausing

A key feature in our zombie issuance pausing mechanism is self-service unpausing. Whenever an account-hostname pair is paused, any new certificate requests for that hostname submitted by that account are immediately rejected. But this means that the “one successful validation immediately resets the rate limit counter” feature can no longer come into effect: once they’re paused, they can’t even attempt validation anymore.

So every rejection comes with an error message explaining what has happened and a custom link that can be used to immediately unpause that account-hostname pair and remove any other pauses on the same account at the same time. The point of this is that subscribers who notice at some point that issuance is failing and want to intervene to get it working again have a straightforward option to let Let’s Encrypt know that they’re aware of the recurring failures and are still planning to use a particular account. As soon as subscribers notify us via the self-service link, they’ll be able to issue certificates again.

Currently, the user interface for an affected subscriber looks like this:

This link would be provided via an ACME error message in response to any request that was blocked due to a pause account-hostname pair.

As it’s turned out, the unpause option shown above has only been used by about 3% of affected accounts! This goes to show that most of the zombies we’ve paused were, in fact, well and truly forgotten about.

However, the unpause feature is there for whenever it’s needed, and there may be cases when it will become more important. A very large integration could trigger the zombie-related rate limits if a newly-introduced software bug causes what looks like a very high volume of zombie requests in a very short time. In that case, once that bug has been noticed and fixed, an integrator may need to unpause its issuance on behalf of lots of customers at once. Our unpause feature permits unpausing 50,000 domain names on a single account at a time, so even the largest integrators can get themselves unpaused expeditiously in this situation.

Conclusion

We’ve been very happy with the results of our zombie mitigation measures, and, as far as we can tell, there’s been almost no impact for subscribers! Our statistics indicate that we’ve managed to reduce the load on our infrastructure while causing no detectable harm or inconvenience to subscribers’ valid issuance requests.

Since implementing the manual pauses in September and the automated pauses in December, we’ve seen:

• Over 100,000 account-hostname pairs have been paused for excessive failures.
• We received zero (!) associated complaints or support requests.
• About 3,200 people manually unpaused issuance.
• Failed certificate orders fell by about 30% so far, and should continue to fall over time as we fine-tune the rate limit formula and catch more zombie clients.

The new rate limit and the self-service unpause system are also ready to deal with circumstances that might produce more zombie clients in the future. For instance, we’ve announced that we’re going to be discontinuing renewal reminder emails soon. If some subscribers overlook failed renewals in the future, we might see more paused clients that result from unintentional renewal failures. We think taking advantage of the existing self-service unpause feature will be straightforward in that case. But it’s much better to notice problems and get them fixed up front, so please remember to set up your own monitoring to avoid unnoticed renewal failures in the future.

If you’re a subscriber who’s had occasion to use the self-service unpause feature, we’d love your feedback on the Community Forum about your experience using the feature and the circumstances that surrounded your account’s getting paused.

Also, if you’re a Let’s Encrypt client developer, please remember to make renewal requests at a random time (not precisely at midnight) so that the load on our infrastructure is smoothed out. You can also reduce the impact of zombie renewals by repeating failed requests somewhat less frequently over time (a “back-off” strategy), especially if the failure reason makes it look like a domain name may no longer be in use at all.

Wed, 04 Jun 2025 00:00:00 +0000

---

Sustaining a More Secure Internet: The Power of Recurring Donations

Since Let’s Encrypt started issuing certificates in 2015, people have repeatedly requested the ability to get certificates for IP addresses, an option that only a few certificate authorities have offered. Until now, they’ve had to look elsewhere, because we haven’t provided that feature.

Today, we’ve issued our first certificate for an IP address, as we announced we would in January. As with other new certificate features on our engineering roadmap, we’ll now start gradually rolling out this option to more and more of our subscribers.

Some Background on IP Address Certs

IP addresses are the underlying numerical addresses used on the Internet. Every device on the Internet has one (though, in modern practice, it might be shared with other devices, like when an entire home network shares a single public IP address). The Internet infrastructure uses them to route communications to their proper destination. IP addresses come in two forms, IPv4 and IPv6, and generally look like 54.215.62.21 (IPv4) or 2600:1f1c:446:4900::65 (IPv6).

Most Internet users rarely see or refer to IP addresses directly. Instead, they almost always use domain names like letsencrypt.org to refer to Internet services. The domain name system (DNS) is a part of the Internet infrastructure that’s responsible for allowing software to find the IP addresses associated with a particular domain name. For instance, your web browser can use DNS to find out that the service https://letsencrypt.org/ (Let’s Encrypt’s own website) is provided from the IP addresses 54.215.62.21 and 2600:1f1c:446:4900::65, among several others. This probably happened behind the scenes before you started reading this article! Your web browser needed to know our IP address in order to actually connect to our site and fetch this article.

Because we overwhelmingly tend to think and talk about Internet services in terms of domain names, those are the identifiers that are normally listed in certificates like those that Let’s Encrypt provides to our subscribers. Since you know us as “letsencrypt.org” and not as, say, “54.215.62.21,” it makes the most sense for our domain name to be on our certificate. After all, that’s what you’ll want your web browser to check against. This also gives Internet services more flexibility to be hosted in multiple locations, or to change where they’re hosted, without necessarily needing separate certificates for each server.

In principle, there’s no reason that a certificate couldn’t be issued for an IP address rather than a domain name, and in fact the technical and policy standards for certificates have always allowed this, with a handful of certificate authorities offering this service on a small scale. In Let’s Encrypt’s case, we’ve preferred to wait until some other pieces, like short-lived certs, were in place before we made this option available for our subscribers.

Why IP Address Certs Are Less Common

First and foremost, it’s because Internet users usually know services by domain names, not by IP addresses, and because IP addresses can easily change “behind the scenes” with no prior notice. For instance, a popular site could switch from one cloud hosting company to a different one, and update its DNS records to point at the new host. Most users wouldn’t ever notice the change at all, even though the site’s underlying IP addresses would be completely different.

Second, because IP addresses can change so easily, the sense of “ownership” one might have for them—or that a certificate authority might be able to attest to—tends to be weaker than for a domain name. If you’re hosting something in your house on a residential broadband connection, your Internet service provider most likely doesn’t guarantee that your IP address will stay the same over time. (That is, most home Internet users have a “dynamic IP address” from their ISPs, rather than a “static IP address.”) In that case, you have to contend with the possibility that that address may change often, possibly without warning, and that your old address may be assigned to somebody else.

Third, most Internet service operators don’t expect that end users will ever intentionally connect to their sites directly by IP address. In some cases, when an IP address is shared by different websites or different devices, connecting by IP address alone wouldn’t even work properly. In that case, there’s not much benefit to obtaining a certificate for the IP address!

How Let’s Encrypt Subscribers May Use IP Address Certs

Most current subscribers should be fine with their existing domain name certs and won’t need IP address certs. Subscribers who have a use for an IP address cert are typically already aware of that. A few use cases that we’re aware of include:

• A default page for hosting providers, in case someone pastes a server’s IP address into a browser instead of an individual site name (right now, this normally produces an error in the browser).

• A way to access your website if you don’t have a domain name at all (at some cost in reliability and convenience compared to getting a domain name).

• Securing DNS over HTTPS (DoH) or other infrastructure services. Having a certificate makes it much easier for DoH servers to prove their identities to clients. That could make it more feasible for DoH users or clients to enforce a requirement for a valid publicly-trusted certificate when connecting to DoH servers.

• Securing remote access to some home devices (like network-attached storage servers and Internet-of-things devices) even without a domain name.

• Securing ephemeral connections within cloud hosting infrastructure, like connections between one back-end cloud server and another, or ephemeral connections to administer new or short-lived back-end servers via HTTPS—as long as those servers have at least one public IP address available.

How To Get an IP Address Cert

IP address certificates are available right now in Staging. They should be generally available in Prod later in 2025, at the same time that short-lived certificates become generally available. Prior to general availability we may allow list issuance for a limited number of partners who can provide us with feedback.

Many Let’s Encrypt client applications should already be able to request certificates for IP addresses, although there can be minor technical changes required to support this in some client software.

As a matter of policy, Let’s Encrypt certificates that cover IP addresses must be short-lived certs, valid for only about six days. As such, your ACME client must support the draft ACME Profiles specification, and you must configure it to request the shortlived profile. And, probably not surprisingly, you can’t use the DNS challenge method to prove your control over an IP address; only the http-01 and tls-alpn-01 methods can be used.

If your client software requests an IP address cert with details that aren’t compatible with these policies, the order will be rejected by the ACME server. In this case, your client application may need to be updated or reconfigured. Feel free to ask for help on the Let’s Encrypt community forum if you encounter any problems, either as a client application developer or as an end user.

Since its inception, Let’s Encrypt has been sending expiration notification emails to subscribers that have provided an email address to us via the ACME API. This service ended on June 4, 2025. The decision to end the service is the result of the following factors:

1. Over the past 10 years more and more of our subscribers have been able to put reliable automation into place for certificate renewal.
2. Providing expiration notification emails means that we have to retain millions of email addresses connected to issuance records. As an organization that values privacy, removing this requirement is important to us.
3. Providing expiration notifications costs Let’s Encrypt tens of thousands of dollars per year, money that we believe can be better spent on other aspects of our infrastructure.
4. Providing expiration notifications adds complexity to our infrastructure, which takes time and attention to manage and increases the likelihood of mistakes being made. Over the long term, particularly as we add support for new service components, we need to manage overall complexity by phasing out system components that can no longer be justified.

For those who would like to continue receiving expiration notifications, we recommend using a third party service such as Red Sift Certificates Lite (formerly Hardenize). Red Sift’s monitoring service providing expiration emails is free of charge for up to 250 certificates. More monitoring options can be found here.

We have deleted the email addresses provided to Let’s Encrypt via the ACME API that were stored in our CA database in association with issuance data. This doesn’t affect addresses signed up to mailing lists and other systems. They are managed in a separate ISRG system unassociated with issuance data.

Going forward, if an email address is provided to Let’s Encrypt via the ACME API, Let’s Encrypt will not store the address but will instead forward it to the general ISRG mailing list system unassociated with any account data. If the email address has not been seen before, that system may send an onboarding email with information about how to subscribe to various sources of updates.

If you’d like to stay informed about technical updates and other news about Let’s Encrypt and our parent nonprofit, ISRG, based on the preferences you choose, you can sign up for our email lists below:

The Certificate Transparency ecosystem has been improving transparency for the web PKI since 2013. It helps make clear exactly what certificates each certificate authority has issued and makes sure errors or compromises of certificate authorities are detectable.

Let’s Encrypt participates in CT both as a certificate issuer and as a log operator. For the past year, we’ve also been running an experiment to help validate a next-generation design for Certificate Transparency logs. That experiment is now nearing a successful conclusion. We’ve demonstrated that the new architecture (called the “Static CT API”) works well, providing greater efficiency and making it easier to run huge and reliable CT log services with comparatively modest resources. The Static CT API also makes it easier to download and share data from CT logs.

The Sunlight log implementation, alongside other Static CT API log implementations, is now on a path to production use. Browsers are now officially accepting Static CT API logs into their log programs as a means to help guarantee that the contents of CA-issued certificates are all publicly disclosed and publicly accessible (see Safari’s and Chrome’s recent announcements), although the browsers also require the continued use of a traditional RFC 6962 log alongside the new type.

All of this is good news for everyone who runs, submits certificates to, or monitors a CT log: as the new architecture gets adopted, we can expect to see more organizations running more logs, at lower cost, and with greater overall capacity to keep up with the large volume of publicly-trusted certificates.

Certificate Transparency

Certificate Transparency (CT) was introduced in 2013 in response to concerns about how Internet users could detect misbehavior and compromise of certificate authorities. Prior to CT, it was possible for a CA to issue an inaccurate or malicious certificate that could be used to attack a relatively small number of users, and that might never come to wider attention. A team led by Google responded to this by creating a transparency log mechanism, where certificate authorities (like Let’s Encrypt) must disclose all of the certificates that we issue by submitting them to public log services. Web browsers now generally reject certificates unless the certificates include cryptographic proof (“Signed Certificate Timestamps”, or SCTs) demonstrating that they were submitted to and accepted by such logs.

The CT logs themselves use a cryptographic append-only ledger to prove that they haven’t deleted or modified their records. There are currently over a dozen CT log services, most of them also run by certificate authorities, including Let’s Encrypt’s own Oak log.

The Static CT API

The original 2013 CT log design has been used with relatively few technical changes since it was first introduced, but several other transparency logging systems have been created in other areas, such as sumdb for Golang, which helps ensure that the contents of Golang package updates are publicly recorded. While they were originally inspired by CT, more-recently invented transparency logs have improved on its design.

The current major evolution of CT was led by Filippo Valsorda, a cryptographer with an interest in transparency log mechanisms, with help from others in the CT ecosystem. Portions of the new design are directly based on sumdb. In addition to designing the new architecture, Valsorda also wrote the implementation that we’ve been using, called Sunlight, with support from Let’s Encrypt. We’re excited to see that there are now at least three other compatible implementations: Google’s trillian-tessera, Cloudflare’s Azul, and an independent project called Itko.

The biggest change for the Static CT API is that logs are now represented, and downloaded by verifiers, as simple collections of flat files (called “tiles,” so some implementers have also been referring to these as “tiled logs” or “tlogs”). Anyone who wants to download log data can do so just by downloading these files. This is great for log operators because these simple file downloads can be distributed in various ways, including caching by a CDN, which was less practical and efficient for the classic CT API.

The new design is also simpler and more efficient from the log operator’s perspective, making it cheaper to run logs. As we said last year, this may enable us and other operators to increase reliability and availability by running several separate logs, likely with lower overall resource requirements than a single traditional log.

Our Sunlight experiment

For the past year, we’ve run three Sunlight logs, called Twig, Willow, and Sycamore. We’ve been logging all of our own issued certificates, which represent a majority of the total volume of all publicly-trusted certificates, into our Sunlight logs. Sunlight logged these certificates quickly and correctly on relatively modest server hardware. Notably, each log’s write side was handled comfortably by just a single server. We also achieved high availability for these log services throughout the course of this experiment. (Because our Sunlight logs are not yet trusted by web browsers, we didn’t include the SCT proofs that they returned to us in the actual certificates we gave out to our subscribers; those proofs wouldn’t have been of use to our subscribers yet and would just have taken up space.)

A potential failure mode of traditional CT logs is that they could be unacceptably slow in incorporating newly-submitted certificates (known as missing the maximum merge delay), which can result in a log becoming distrusted. This isn’t a possibility for our new Sunlight-based logs: they always completely incorporate newly-submitted certificates before returning an SCT to the submitter, so the effective merge delay is zero! Of course, any log can suffer outages for a variety of reasons, but this feature of Sunlight makes it less likely that any outages will be fatal to a log’s continued operation.

We’ve demonstrated that Sunlight and the Static CT API work in practice, and this demonstration has helped to confirm the browser developers’ hope that Static CT API logs can become an officially-supported part of CT. As a result, the major browsers that enforce CT have now permitted Static CT API logs to apply for inclusion in browsers as publicly-trusted logs, and we’re preparing to apply for this status for our Willow and Sycamore logs with the Chrome and Safari CT log programs.

Let’s Encrypt will run at least these two logs, and possibly others over time, for the foreseeable future. Once they’re trusted by browsers, we’ll encourage other CAs to submit to them as well, and we’ll begin including SCTs from these logs in our own certificates (alongside SCTs from traditional CT logs).

How to participate

The new Static CT API and the rollout of tile-based logs will bring various changes and opportunities for community members.

New Certificate Transparency log operators

Companies and non-profit organizations could help support the web PKI by running a CT log and applying for it to be publicly trusted. Implementations like Sunlight will have substantially lower resource requirements than first-generation CT logs, particularly when cached behind a CDN. The biggest resource demands for a log operator will be storage and upstream bandwidth. A publicly-trusted log is also expected to maintain relatively high availability, because CAs need logs to be available in order to continue issuing certificates.

We don’t have statistics to share about the exact resource requirements for such a log yet, but after we have practical experience running a fully publicly-trusted Sunlight log, we should be able to make this more concrete. As noted above, the compute side of the log can be handled by a single server. Sunlight author Filippo Valsorda has recently started running a Sunlight log—also on just a single server—and offered more detailed cost breakdowns for that log’s setup, with an estimated total cost around $10,000 per year. The costs for our production Static CT API logs may be higher than those for Filippo’s log, but still far less than the costs for our traditional RFC 6962 logs.

As with trust decisions about CAs, browser developers are the authorities about which CT logs become publicly trusted. Although any person or organization can run a log, browser developers will generally prefer to trust logs whose continued availability they’re confident of—typically those run by stable organizations with experience running some form of public Internet services. Unlike becoming a certificate authority, running a log does not require a formal audit, as the validation of the log’s availability and correctness can be performed purely by observation.

Certificate authorities

Once the Willow and Sycamore logs are trusted by browsers, our fellow certificate authorities can choose to start logging certificates to them as part of their issuance processes. (Initially, you should still include at least one SCT from a traditional CT log in each certificate.) The details, including the log API endpoints and keys, are available at our CT log page. You can start submitting to these logs right away if you prefer; just bear in mind that the SCTs they return aren’t useful to subscribers yet, and won’t be useful until browsers are updated to trust the new logs.

CT data users

You can monitor CT in order to watch for certificate issuances for your own domain names, or as part of monitoring or security products or services, or for Internet security research purposes. Many of our colleagues have been doing this for some time as a part of various tools they maintain. The Static CT API should make this easier, because you’ll be able to download and share log tiles as sets of ordinary files.

If you already run such monitoring tools, please note that you’ll need to update your data pipeline in order to access Static CT API logs; since the read API is not backwards-compatible, CT API clients will need to be modified to support the new API. Without updated tools, your view of the CT system will become partial!

Also note that getting a complete view of all of CT will still require downloading data from traditional logs, which will probably continue to be true for several years.

Software developers

As logs based on the new API enter production use, it will be important to have tools to interact with and search these logs. We can all benefit from more software that understands how to do this. Since file downloads are such a familiar piece of software functionality, it will probably be easier for developers to develop against the new API compared to the original one.

We’ve also continued to see greater integration of transparency logging tools into other kinds of services, such as software updates. There’s a growing transparency log ecosystem that’s always in need of more tools and integrations. As we mentioned above, transparency logs are increasingly learning from one another, and there are also mechanisms for more direct integration between different kinds of transparency logs (known as “witnessing”). Software developers can help improve different aspects of Internet security by contributing to this active and growing area.

Conclusion

The Certificate Transparency community and larger transparency logging community have experienced a virtuous cycle of innovation, sharing ideas and implementation code between different systems and demonstrating the feasibility of new mechanisms and functionality. With the advent of tile-based logging in CT, the state of the art has moved forward in a way that helps log operators run our logs much more efficiently without compromising security.

We’re proud to have participated in this experiment and the engineering conversation around the evolution of logging architectures. Now that we’ve shown how well the new API really works at scale, we look forward to having publicly-trusted Sunlight logs later this year!

Every night, right around midnight (mainly UTC), a horde of zombies wakes up and clamors for … digital certificates!

The zombies in question are abandoned or misconfigured Internet servers and ACME clients that have been set to request certificates from Let’s Encrypt. As our certificates last for at most 90 days, these zombie clients’ software knows that their certificates are out-of-date and need to be replaced. What they don’t realize is that their quest for new certificates is doomed! These devices are cursed to seek certificates again and again, never receiving them.

But they do use up a lot of certificate authority resources in the process.

The Zombie Client Problem

Unlike a human being, software doesn’t give up in frustration, or try to modify its approach, when it repeatedly fails at the same task. Our emphasis on automation means that the vast majority of Let’s Encrypt certificate renewals are performed by automated software. This is great when those renewals succeed, but it also means that forgotten clients and devices can continue requesting renewals unsuccessfully for months, or even years.

How might that happen? Most often, it happens when a device no longer has a domain name pointed to it. The device itself doesn’t know that this has changed, so it treats renewal failures as transient even though they are actually permanent. For instance:

• An organization may have allowed a domain name registration to lapse because it is no longer needed, but its servers are still configured to request certs for it.
• Or, a home user stopped using a particular dynamic-DNS domain with a network-attached storage device, but is still using that device at home. The device doesn’t realize that the user no longer expects to use the name, so it keeps requesting certs for it.
• Or, a web hosting or CDN customer migrated to a different service provider, but never informed the old service provider. The old service provider’s servers keep requesting certs unsuccessfully. If the customer was in a free service tier, there might not be invoices or charges reminding the customer to cancel the service.
• Or any number of other, subtler changes in a subscriber’s infrastructure, such as changing a firewall rule or some webserver configuration.

At the scale of Let’s Encrypt, which now covers hundreds of millions of names, scenarios like these have become common, and their impact has become substantial. In 2024, we noticed that about half of all certificate requests to the Let’s Encrypt ACME API came from about a million accounts that never successfully complete any validations. Many of these had completed validations and issued certificates sometime in the past, but nowadays every single one of their validation attempts fails, and they show no signs that this will change anytime soon.

Unfortunately, trying to validate those futile requests still uses resources. Our CA software has to generate challenges, reach out and attempt to validate them over the Internet, detect and report failures, and record all of the associated information in our databases and audit logs. And over time, we’ve seen more and more recurring failures: accounts that always fail their issuance requests have been growing at around 18% per year.

In January, we mentioned that we had been addressing the zombie client problem through our rate limit system. This post provides more detail on that progress. 

Our Rate Limit Philosophy

If you’ve used Let’s Encrypt as a subscriber, you may have run into one of our rate limits at some point, maybe during your initial setup process. We have eight different kinds of rate limits in place now; as our January post describes, they’ve become more algorithmically sophisticated and grown to address a wider range of problems. A key principle for Let’s Encrypt is that our rate limiting is not a punishment. We don’t think of rate limits as a way of retaliating against a client for misbehavior. Rate limits are simply a tool to maximize the efficient use of our limited resources and prevent people and programs from using up those resources for no constructive purpose.

We’ve consistently tried to design our rate limit mechanisms in line with that philosophy. So if a misconfiguration or misunderstanding has caused excessive requests in the past, we’re still happy to welcome the user in question back and start issuing them certificates again—once the problem has been addressed. We want the rate limits to put a brake on wasteful use of our systems, but not to frustrate users who are actively trying to make Let’s Encrypt work for them.

In addition, we’ve always implemented our rate limits to err on the side of permissiveness. For example, if the Redis instances where rate limits are tracked have an outage or lose data, the system is designed to permit more issuance rather than less issuance as a result.

We wanted to create additional limits that would target zombie clients, but in a correspondingly non-punitive way that would avoid any disruption to valid issuance, and welcome subscribers back quickly if they happened to notice and fix a long-time problem with their setups.

Our Zombie-Related Rate Limits and Their Impact

In planning a new zombie-specific response, we decided on a “pausing” approach, which can temporarily limit an account’s ability to proceed with certificate requests. The core idea is that, if a particular account consistently fails to complete validation for a particular hostname, we’ll pause that account-hostname pair. The pause means that any new order requests from that account for that hostname will be rejected immediately, before we get to the resource-intensive validation phase.

This approach is more finely targeted than pausing an entire account. Pausing account-hostname pairs means that your ability to issue certs for a specific name could be paused due to repeated failures, but you can still get all of your other certs like normal. So a large hosting provider doesn’t have to fear that its certificate issuance on behalf of one customer will be affected by renewal failures related to a problem with a different customer’s domain name. The account-specificity of the pause, in turn, means that validation failures from one subscriber or device won’t prevent a different subscriber or device from attempting to validate the same name, as long as the devices in question don’t share a single Let’s Encrypt account.

In September 2024, we began applying our zombie rate limits manually by pausing about 21,000 of the most recurrently-failing account-hostname pairs, those which were consistently repeating the same failed requests many times per day, every day. After implementing that first round of pauses, we immediately saw a significant impact on our failed request rates. As we announced at that time, we also began using a formula to automatically pause other zombie client account-hostname pairs from December 2024 onward. The associated new rate limit is called “Consecutive Authorization Failures per Hostname Per Account” (and is independent of the existing “Authorization Failures per Hostname Per Account” limit, which resets every hour).

This formula relates to the frequency of successive failed issuance requests for the same domain name by the same Let’s Encrypt account. It applies only to failures that happen again and again, with no successful issuances at all in between: a single successful validation immediately resets the rate limit all the way to zero. Like all of our rate limits, this is not a punitive measure but is simply intended to reduce the waste of resources. So, we decided to set the thresholds rather high in the expectation that we would catch only the most disruptive zombie clients, and ultimately only those clients that were extremely unlikely to succeed in the future based on their substantial history of failed requests. We don’t hurry to block requesters as zombies: according to our current formula, client software following the default established by EFF’s Certbot (two renewal attempts per day) would be paused as a zombie only after about ten years of constant failures. More aggressive failed issuance attempts will get a client paused sooner, but clients will generally have to fail hundreds or thousands of attempts in a row before they are paused.

Most subscribers using mainstream client applications with default configurations will never encounter this rate limit, even if they forget to deactivate renewal attempts for domains that are no longer pointed at their servers. As described below, our current limit is already providing noticeable benefits with minimal disruption, and we’re likely to tighten it a bit in the near future, so it will trigger after somewhat fewer consecutive failures.

Self-Service Unpausing

A key feature in our zombie issuance pausing mechanism is self-service unpausing. Whenever an account-hostname pair is paused, any new certificate requests for that hostname submitted by that account are immediately rejected. But this means that the “one successful validation immediately resets the rate limit counter” feature can no longer come into effect: once they’re paused, they can’t even attempt validation anymore.

So every rejection comes with an error message explaining what has happened and a custom link that can be used to immediately unpause that account-hostname pair and remove any other pauses on the same account at the same time. The point of this is that subscribers who notice at some point that issuance is failing and want to intervene to get it working again have a straightforward option to let Let’s Encrypt know that they’re aware of the recurring failures and are still planning to use a particular account. As soon as subscribers notify us via the self-service link, they’ll be able to issue certificates again.

Currently, the user interface for an affected subscriber looks like this:

This link would be provided via an ACME error message in response to any request that was blocked due to a pause account-hostname pair.

As it’s turned out, the unpause option shown above has only been used by about 3% of affected accounts! This goes to show that most of the zombies we’ve paused were, in fact, well and truly forgotten about.

However, the unpause feature is there for whenever it’s needed, and there may be cases when it will become more important. A very large integration could trigger the zombie-related rate limits if a newly-introduced software bug causes what looks like a very high volume of zombie requests in a very short time. In that case, once that bug has been noticed and fixed, an integrator may need to unpause its issuance on behalf of lots of customers at once. Our unpause feature permits unpausing 50,000 domain names on a single account at a time, so even the largest integrators can get themselves unpaused expeditiously in this situation.

Conclusion

We’ve been very happy with the results of our zombie mitigation measures, and, as far as we can tell, there’s been almost no impact for subscribers! Our statistics indicate that we’ve managed to reduce the load on our infrastructure while causing no detectable harm or inconvenience to subscribers’ valid issuance requests.

Since implementing the manual pauses in September and the automated pauses in December, we’ve seen:

• Over 100,000 account-hostname pairs have been paused for excessive failures.
• We received zero (!) associated complaints or support requests.
• About 3,200 people manually unpaused issuance.
• Failed certificate orders fell by about 30% so far, and should continue to fall over time as we fine-tune the rate limit formula and catch more zombie clients.

The new rate limit and the self-service unpause system are also ready to deal with circumstances that might produce more zombie clients in the future. For instance, we’ve announced that we’re going to be discontinuing renewal reminder emails soon. If some subscribers overlook failed renewals in the future, we might see more paused clients that result from unintentional renewal failures. We think taking advantage of the existing self-service unpause feature will be straightforward in that case. But it’s much better to notice problems and get them fixed up front, so please remember to set up your own monitoring to avoid unnoticed renewal failures in the future.

If you’re a subscriber who’s had occasion to use the self-service unpause feature, we’d love your feedback on the Community Forum about your experience using the feature and the circumstances that surrounded your account’s getting paused.

Also, if you’re a Let’s Encrypt client developer, please remember to make renewal requests at a random time (not precisely at midnight) so that the load on our infrastructure is smoothed out. You can also reduce the impact of zombie renewals by repeating failed requests somewhat less frequently over time (a “back-off” strategy), especially if the failure reason makes it look like a domain name may no longer be in use at all.

At Let’s Encrypt we know that building a secure Internet isn’t just a technical challenge—it’s a long-term commitment. Over the past decade we’ve made enormous strides: from issuing billions of TLS certificates to continually innovating to keep the web safer and more accessible. But none of this would be possible without recurring donations from individuals and organizations around the world.

Recurring donations are more than just financial support; they allow us to plan, innovate, and keep improving with confidence, knowing that month after month, year after year, our supporters are there. This consistent backing empowers us to maintain a secure, privacy-respecting Internet for all.

Our tenth anniversary tagline, Encryption for Everybody, highlights this vision. It’s both a technical goal and a fundamental belief that secure communication should be available to everyone, everywhere.

When we asked our recurring donors why they give, their responses affirmed how essential this commitment is. One longtime supporter shared:

For some, it’s about paying it forward, helping future users benefit as they once did:

The next decade of Let’s Encrypt will likely be about maintaining that commitment to encryption for everybody. It’s about ensuring that our work remains reliable, accessible, and—most importantly—supported by people who believe in what we do. To everyone who’s been part of this journey, thank you. We couldn’t do it without you.

During Let’s Encrypt’s 10th Anniversary Year, we’re celebrating our community and reflecting on our journey. We’d be thrilled to hear from you. Connect with us on LinkedIn, our community forum, or email us at outreach@letsencrypt.org. Let’s keep building a secure Internet together!

Let’s Encrypt is a project of the nonprofit Internet Security Research Group, a 501(c)(3) nonprofit. To support our work, visit letsencrypt.org/donate.

Wed, 21 May 2025 00:00:00 +0000

---

Ending TLS Client Authentication Certificate Support in 2026

Since Let’s Encrypt started issuing certificates in 2015, people have repeatedly requested the ability to get certificates for IP addresses, an option that only a few certificate authorities have offered. Until now, they’ve had to look elsewhere, because we haven’t provided that feature.

Today, we’ve issued our first certificate for an IP address, as we announced we would in January. As with other new certificate features on our engineering roadmap, we’ll now start gradually rolling out this option to more and more of our subscribers.

Some Background on IP Address Certs

IP addresses are the underlying numerical addresses used on the Internet. Every device on the Internet has one (though, in modern practice, it might be shared with other devices, like when an entire home network shares a single public IP address). The Internet infrastructure uses them to route communications to their proper destination. IP addresses come in two forms, IPv4 and IPv6, and generally look like 54.215.62.21 (IPv4) or 2600:1f1c:446:4900::65 (IPv6).

Most Internet users rarely see or refer to IP addresses directly. Instead, they almost always use domain names like letsencrypt.org to refer to Internet services. The domain name system (DNS) is a part of the Internet infrastructure that’s responsible for allowing software to find the IP addresses associated with a particular domain name. For instance, your web browser can use DNS to find out that the service https://letsencrypt.org/ (Let’s Encrypt’s own website) is provided from the IP addresses 54.215.62.21 and 2600:1f1c:446:4900::65, among several others. This probably happened behind the scenes before you started reading this article! Your web browser needed to know our IP address in order to actually connect to our site and fetch this article.

Because we overwhelmingly tend to think and talk about Internet services in terms of domain names, those are the identifiers that are normally listed in certificates like those that Let’s Encrypt provides to our subscribers. Since you know us as “letsencrypt.org” and not as, say, “54.215.62.21,” it makes the most sense for our domain name to be on our certificate. After all, that’s what you’ll want your web browser to check against. This also gives Internet services more flexibility to be hosted in multiple locations, or to change where they’re hosted, without necessarily needing separate certificates for each server.

In principle, there’s no reason that a certificate couldn’t be issued for an IP address rather than a domain name, and in fact the technical and policy standards for certificates have always allowed this, with a handful of certificate authorities offering this service on a small scale. In Let’s Encrypt’s case, we’ve preferred to wait until some other pieces, like short-lived certs, were in place before we made this option available for our subscribers.

Why IP Address Certs Are Less Common

First and foremost, it’s because Internet users usually know services by domain names, not by IP addresses, and because IP addresses can easily change “behind the scenes” with no prior notice. For instance, a popular site could switch from one cloud hosting company to a different one, and update its DNS records to point at the new host. Most users wouldn’t ever notice the change at all, even though the site’s underlying IP addresses would be completely different.

Second, because IP addresses can change so easily, the sense of “ownership” one might have for them—or that a certificate authority might be able to attest to—tends to be weaker than for a domain name. If you’re hosting something in your house on a residential broadband connection, your Internet service provider most likely doesn’t guarantee that your IP address will stay the same over time. (That is, most home Internet users have a “dynamic IP address” from their ISPs, rather than a “static IP address.”) In that case, you have to contend with the possibility that that address may change often, possibly without warning, and that your old address may be assigned to somebody else.

Third, most Internet service operators don’t expect that end users will ever intentionally connect to their sites directly by IP address. In some cases, when an IP address is shared by different websites or different devices, connecting by IP address alone wouldn’t even work properly. In that case, there’s not much benefit to obtaining a certificate for the IP address!

How Let’s Encrypt Subscribers May Use IP Address Certs

Most current subscribers should be fine with their existing domain name certs and won’t need IP address certs. Subscribers who have a use for an IP address cert are typically already aware of that. A few use cases that we’re aware of include:

• A default page for hosting providers, in case someone pastes a server’s IP address into a browser instead of an individual site name (right now, this normally produces an error in the browser).

• A way to access your website if you don’t have a domain name at all (at some cost in reliability and convenience compared to getting a domain name).

• Securing DNS over HTTPS (DoH) or other infrastructure services. Having a certificate makes it much easier for DoH servers to prove their identities to clients. That could make it more feasible for DoH users or clients to enforce a requirement for a valid publicly-trusted certificate when connecting to DoH servers.

• Securing remote access to some home devices (like network-attached storage servers and Internet-of-things devices) even without a domain name.

• Securing ephemeral connections within cloud hosting infrastructure, like connections between one back-end cloud server and another, or ephemeral connections to administer new or short-lived back-end servers via HTTPS—as long as those servers have at least one public IP address available.

How To Get an IP Address Cert

IP address certificates are available right now in Staging. They should be generally available in Prod later in 2025, at the same time that short-lived certificates become generally available. Prior to general availability we may allow list issuance for a limited number of partners who can provide us with feedback.

Many Let’s Encrypt client applications should already be able to request certificates for IP addresses, although there can be minor technical changes required to support this in some client software.

As a matter of policy, Let’s Encrypt certificates that cover IP addresses must be short-lived certs, valid for only about six days. As such, your ACME client must support the draft ACME Profiles specification, and you must configure it to request the shortlived profile. And, probably not surprisingly, you can’t use the DNS challenge method to prove your control over an IP address; only the http-01 and tls-alpn-01 methods can be used.

If your client software requests an IP address cert with details that aren’t compatible with these policies, the order will be rejected by the ACME server. In this case, your client application may need to be updated or reconfigured. Feel free to ask for help on the Let’s Encrypt community forum if you encounter any problems, either as a client application developer or as an end user.

Since its inception, Let’s Encrypt has been sending expiration notification emails to subscribers that have provided an email address to us via the ACME API. This service ended on June 4, 2025. The decision to end the service is the result of the following factors:

1. Over the past 10 years more and more of our subscribers have been able to put reliable automation into place for certificate renewal.
2. Providing expiration notification emails means that we have to retain millions of email addresses connected to issuance records. As an organization that values privacy, removing this requirement is important to us.
3. Providing expiration notifications costs Let’s Encrypt tens of thousands of dollars per year, money that we believe can be better spent on other aspects of our infrastructure.
4. Providing expiration notifications adds complexity to our infrastructure, which takes time and attention to manage and increases the likelihood of mistakes being made. Over the long term, particularly as we add support for new service components, we need to manage overall complexity by phasing out system components that can no longer be justified.

For those who would like to continue receiving expiration notifications, we recommend using a third party service such as Red Sift Certificates Lite (formerly Hardenize). Red Sift’s monitoring service providing expiration emails is free of charge for up to 250 certificates. More monitoring options can be found here.

We have deleted the email addresses provided to Let’s Encrypt via the ACME API that were stored in our CA database in association with issuance data. This doesn’t affect addresses signed up to mailing lists and other systems. They are managed in a separate ISRG system unassociated with issuance data.

Going forward, if an email address is provided to Let’s Encrypt via the ACME API, Let’s Encrypt will not store the address but will instead forward it to the general ISRG mailing list system unassociated with any account data. If the email address has not been seen before, that system may send an onboarding email with information about how to subscribe to various sources of updates.

If you’d like to stay informed about technical updates and other news about Let’s Encrypt and our parent nonprofit, ISRG, based on the preferences you choose, you can sign up for our email lists below:

The Certificate Transparency ecosystem has been improving transparency for the web PKI since 2013. It helps make clear exactly what certificates each certificate authority has issued and makes sure errors or compromises of certificate authorities are detectable.

Let’s Encrypt participates in CT both as a certificate issuer and as a log operator. For the past year, we’ve also been running an experiment to help validate a next-generation design for Certificate Transparency logs. That experiment is now nearing a successful conclusion. We’ve demonstrated that the new architecture (called the “Static CT API”) works well, providing greater efficiency and making it easier to run huge and reliable CT log services with comparatively modest resources. The Static CT API also makes it easier to download and share data from CT logs.

The Sunlight log implementation, alongside other Static CT API log implementations, is now on a path to production use. Browsers are now officially accepting Static CT API logs into their log programs as a means to help guarantee that the contents of CA-issued certificates are all publicly disclosed and publicly accessible (see Safari’s and Chrome’s recent announcements), although the browsers also require the continued use of a traditional RFC 6962 log alongside the new type.

All of this is good news for everyone who runs, submits certificates to, or monitors a CT log: as the new architecture gets adopted, we can expect to see more organizations running more logs, at lower cost, and with greater overall capacity to keep up with the large volume of publicly-trusted certificates.

Certificate Transparency

Certificate Transparency (CT) was introduced in 2013 in response to concerns about how Internet users could detect misbehavior and compromise of certificate authorities. Prior to CT, it was possible for a CA to issue an inaccurate or malicious certificate that could be used to attack a relatively small number of users, and that might never come to wider attention. A team led by Google responded to this by creating a transparency log mechanism, where certificate authorities (like Let’s Encrypt) must disclose all of the certificates that we issue by submitting them to public log services. Web browsers now generally reject certificates unless the certificates include cryptographic proof (“Signed Certificate Timestamps”, or SCTs) demonstrating that they were submitted to and accepted by such logs.

The CT logs themselves use a cryptographic append-only ledger to prove that they haven’t deleted or modified their records. There are currently over a dozen CT log services, most of them also run by certificate authorities, including Let’s Encrypt’s own Oak log.

The Static CT API

The original 2013 CT log design has been used with relatively few technical changes since it was first introduced, but several other transparency logging systems have been created in other areas, such as sumdb for Golang, which helps ensure that the contents of Golang package updates are publicly recorded. While they were originally inspired by CT, more-recently invented transparency logs have improved on its design.

The current major evolution of CT was led by Filippo Valsorda, a cryptographer with an interest in transparency log mechanisms, with help from others in the CT ecosystem. Portions of the new design are directly based on sumdb. In addition to designing the new architecture, Valsorda also wrote the implementation that we’ve been using, called Sunlight, with support from Let’s Encrypt. We’re excited to see that there are now at least three other compatible implementations: Google’s trillian-tessera, Cloudflare’s Azul, and an independent project called Itko.

The biggest change for the Static CT API is that logs are now represented, and downloaded by verifiers, as simple collections of flat files (called “tiles,” so some implementers have also been referring to these as “tiled logs” or “tlogs”). Anyone who wants to download log data can do so just by downloading these files. This is great for log operators because these simple file downloads can be distributed in various ways, including caching by a CDN, which was less practical and efficient for the classic CT API.

The new design is also simpler and more efficient from the log operator’s perspective, making it cheaper to run logs. As we said last year, this may enable us and other operators to increase reliability and availability by running several separate logs, likely with lower overall resource requirements than a single traditional log.

Our Sunlight experiment

For the past year, we’ve run three Sunlight logs, called Twig, Willow, and Sycamore. We’ve been logging all of our own issued certificates, which represent a majority of the total volume of all publicly-trusted certificates, into our Sunlight logs. Sunlight logged these certificates quickly and correctly on relatively modest server hardware. Notably, each log’s write side was handled comfortably by just a single server. We also achieved high availability for these log services throughout the course of this experiment. (Because our Sunlight logs are not yet trusted by web browsers, we didn’t include the SCT proofs that they returned to us in the actual certificates we gave out to our subscribers; those proofs wouldn’t have been of use to our subscribers yet and would just have taken up space.)

A potential failure mode of traditional CT logs is that they could be unacceptably slow in incorporating newly-submitted certificates (known as missing the maximum merge delay), which can result in a log becoming distrusted. This isn’t a possibility for our new Sunlight-based logs: they always completely incorporate newly-submitted certificates before returning an SCT to the submitter, so the effective merge delay is zero! Of course, any log can suffer outages for a variety of reasons, but this feature of Sunlight makes it less likely that any outages will be fatal to a log’s continued operation.

We’ve demonstrated that Sunlight and the Static CT API work in practice, and this demonstration has helped to confirm the browser developers’ hope that Static CT API logs can become an officially-supported part of CT. As a result, the major browsers that enforce CT have now permitted Static CT API logs to apply for inclusion in browsers as publicly-trusted logs, and we’re preparing to apply for this status for our Willow and Sycamore logs with the Chrome and Safari CT log programs.

Let’s Encrypt will run at least these two logs, and possibly others over time, for the foreseeable future. Once they’re trusted by browsers, we’ll encourage other CAs to submit to them as well, and we’ll begin including SCTs from these logs in our own certificates (alongside SCTs from traditional CT logs).

How to participate

The new Static CT API and the rollout of tile-based logs will bring various changes and opportunities for community members.

New Certificate Transparency log operators

Companies and non-profit organizations could help support the web PKI by running a CT log and applying for it to be publicly trusted. Implementations like Sunlight will have substantially lower resource requirements than first-generation CT logs, particularly when cached behind a CDN. The biggest resource demands for a log operator will be storage and upstream bandwidth. A publicly-trusted log is also expected to maintain relatively high availability, because CAs need logs to be available in order to continue issuing certificates.

We don’t have statistics to share about the exact resource requirements for such a log yet, but after we have practical experience running a fully publicly-trusted Sunlight log, we should be able to make this more concrete. As noted above, the compute side of the log can be handled by a single server. Sunlight author Filippo Valsorda has recently started running a Sunlight log—also on just a single server—and offered more detailed cost breakdowns for that log’s setup, with an estimated total cost around $10,000 per year. The costs for our production Static CT API logs may be higher than those for Filippo’s log, but still far less than the costs for our traditional RFC 6962 logs.

As with trust decisions about CAs, browser developers are the authorities about which CT logs become publicly trusted. Although any person or organization can run a log, browser developers will generally prefer to trust logs whose continued availability they’re confident of—typically those run by stable organizations with experience running some form of public Internet services. Unlike becoming a certificate authority, running a log does not require a formal audit, as the validation of the log’s availability and correctness can be performed purely by observation.

Certificate authorities

Once the Willow and Sycamore logs are trusted by browsers, our fellow certificate authorities can choose to start logging certificates to them as part of their issuance processes. (Initially, you should still include at least one SCT from a traditional CT log in each certificate.) The details, including the log API endpoints and keys, are available at our CT log page. You can start submitting to these logs right away if you prefer; just bear in mind that the SCTs they return aren’t useful to subscribers yet, and won’t be useful until browsers are updated to trust the new logs.

CT data users

You can monitor CT in order to watch for certificate issuances for your own domain names, or as part of monitoring or security products or services, or for Internet security research purposes. Many of our colleagues have been doing this for some time as a part of various tools they maintain. The Static CT API should make this easier, because you’ll be able to download and share log tiles as sets of ordinary files.

If you already run such monitoring tools, please note that you’ll need to update your data pipeline in order to access Static CT API logs; since the read API is not backwards-compatible, CT API clients will need to be modified to support the new API. Without updated tools, your view of the CT system will become partial!

Also note that getting a complete view of all of CT will still require downloading data from traditional logs, which will probably continue to be true for several years.

Software developers

As logs based on the new API enter production use, it will be important to have tools to interact with and search these logs. We can all benefit from more software that understands how to do this. Since file downloads are such a familiar piece of software functionality, it will probably be easier for developers to develop against the new API compared to the original one.

We’ve also continued to see greater integration of transparency logging tools into other kinds of services, such as software updates. There’s a growing transparency log ecosystem that’s always in need of more tools and integrations. As we mentioned above, transparency logs are increasingly learning from one another, and there are also mechanisms for more direct integration between different kinds of transparency logs (known as “witnessing”). Software developers can help improve different aspects of Internet security by contributing to this active and growing area.

Conclusion

The Certificate Transparency community and larger transparency logging community have experienced a virtuous cycle of innovation, sharing ideas and implementation code between different systems and demonstrating the feasibility of new mechanisms and functionality. With the advent of tile-based logging in CT, the state of the art has moved forward in a way that helps log operators run our logs much more efficiently without compromising security.

We’re proud to have participated in this experiment and the engineering conversation around the evolution of logging architectures. Now that we’ve shown how well the new API really works at scale, we look forward to having publicly-trusted Sunlight logs later this year!

Every night, right around midnight (mainly UTC), a horde of zombies wakes up and clamors for … digital certificates!

The zombies in question are abandoned or misconfigured Internet servers and ACME clients that have been set to request certificates from Let’s Encrypt. As our certificates last for at most 90 days, these zombie clients’ software knows that their certificates are out-of-date and need to be replaced. What they don’t realize is that their quest for new certificates is doomed! These devices are cursed to seek certificates again and again, never receiving them.

But they do use up a lot of certificate authority resources in the process.

The Zombie Client Problem

Unlike a human being, software doesn’t give up in frustration, or try to modify its approach, when it repeatedly fails at the same task. Our emphasis on automation means that the vast majority of Let’s Encrypt certificate renewals are performed by automated software. This is great when those renewals succeed, but it also means that forgotten clients and devices can continue requesting renewals unsuccessfully for months, or even years.

How might that happen? Most often, it happens when a device no longer has a domain name pointed to it. The device itself doesn’t know that this has changed, so it treats renewal failures as transient even though they are actually permanent. For instance:

• An organization may have allowed a domain name registration to lapse because it is no longer needed, but its servers are still configured to request certs for it.
• Or, a home user stopped using a particular dynamic-DNS domain with a network-attached storage device, but is still using that device at home. The device doesn’t realize that the user no longer expects to use the name, so it keeps requesting certs for it.
• Or, a web hosting or CDN customer migrated to a different service provider, but never informed the old service provider. The old service provider’s servers keep requesting certs unsuccessfully. If the customer was in a free service tier, there might not be invoices or charges reminding the customer to cancel the service.
• Or any number of other, subtler changes in a subscriber’s infrastructure, such as changing a firewall rule or some webserver configuration.

At the scale of Let’s Encrypt, which now covers hundreds of millions of names, scenarios like these have become common, and their impact has become substantial. In 2024, we noticed that about half of all certificate requests to the Let’s Encrypt ACME API came from about a million accounts that never successfully complete any validations. Many of these had completed validations and issued certificates sometime in the past, but nowadays every single one of their validation attempts fails, and they show no signs that this will change anytime soon.

Unfortunately, trying to validate those futile requests still uses resources. Our CA software has to generate challenges, reach out and attempt to validate them over the Internet, detect and report failures, and record all of the associated information in our databases and audit logs. And over time, we’ve seen more and more recurring failures: accounts that always fail their issuance requests have been growing at around 18% per year.

In January, we mentioned that we had been addressing the zombie client problem through our rate limit system. This post provides more detail on that progress. 

Our Rate Limit Philosophy

If you’ve used Let’s Encrypt as a subscriber, you may have run into one of our rate limits at some point, maybe during your initial setup process. We have eight different kinds of rate limits in place now; as our January post describes, they’ve become more algorithmically sophisticated and grown to address a wider range of problems. A key principle for Let’s Encrypt is that our rate limiting is not a punishment. We don’t think of rate limits as a way of retaliating against a client for misbehavior. Rate limits are simply a tool to maximize the efficient use of our limited resources and prevent people and programs from using up those resources for no constructive purpose.

We’ve consistently tried to design our rate limit mechanisms in line with that philosophy. So if a misconfiguration or misunderstanding has caused excessive requests in the past, we’re still happy to welcome the user in question back and start issuing them certificates again—once the problem has been addressed. We want the rate limits to put a brake on wasteful use of our systems, but not to frustrate users who are actively trying to make Let’s Encrypt work for them.

In addition, we’ve always implemented our rate limits to err on the side of permissiveness. For example, if the Redis instances where rate limits are tracked have an outage or lose data, the system is designed to permit more issuance rather than less issuance as a result.

We wanted to create additional limits that would target zombie clients, but in a correspondingly non-punitive way that would avoid any disruption to valid issuance, and welcome subscribers back quickly if they happened to notice and fix a long-time problem with their setups.

Our Zombie-Related Rate Limits and Their Impact

In planning a new zombie-specific response, we decided on a “pausing” approach, which can temporarily limit an account’s ability to proceed with certificate requests. The core idea is that, if a particular account consistently fails to complete validation for a particular hostname, we’ll pause that account-hostname pair. The pause means that any new order requests from that account for that hostname will be rejected immediately, before we get to the resource-intensive validation phase.

This approach is more finely targeted than pausing an entire account. Pausing account-hostname pairs means that your ability to issue certs for a specific name could be paused due to repeated failures, but you can still get all of your other certs like normal. So a large hosting provider doesn’t have to fear that its certificate issuance on behalf of one customer will be affected by renewal failures related to a problem with a different customer’s domain name. The account-specificity of the pause, in turn, means that validation failures from one subscriber or device won’t prevent a different subscriber or device from attempting to validate the same name, as long as the devices in question don’t share a single Let’s Encrypt account.

In September 2024, we began applying our zombie rate limits manually by pausing about 21,000 of the most recurrently-failing account-hostname pairs, those which were consistently repeating the same failed requests many times per day, every day. After implementing that first round of pauses, we immediately saw a significant impact on our failed request rates. As we announced at that time, we also began using a formula to automatically pause other zombie client account-hostname pairs from December 2024 onward. The associated new rate limit is called “Consecutive Authorization Failures per Hostname Per Account” (and is independent of the existing “Authorization Failures per Hostname Per Account” limit, which resets every hour).

This formula relates to the frequency of successive failed issuance requests for the same domain name by the same Let’s Encrypt account. It applies only to failures that happen again and again, with no successful issuances at all in between: a single successful validation immediately resets the rate limit all the way to zero. Like all of our rate limits, this is not a punitive measure but is simply intended to reduce the waste of resources. So, we decided to set the thresholds rather high in the expectation that we would catch only the most disruptive zombie clients, and ultimately only those clients that were extremely unlikely to succeed in the future based on their substantial history of failed requests. We don’t hurry to block requesters as zombies: according to our current formula, client software following the default established by EFF’s Certbot (two renewal attempts per day) would be paused as a zombie only after about ten years of constant failures. More aggressive failed issuance attempts will get a client paused sooner, but clients will generally have to fail hundreds or thousands of attempts in a row before they are paused.

Most subscribers using mainstream client applications with default configurations will never encounter this rate limit, even if they forget to deactivate renewal attempts for domains that are no longer pointed at their servers. As described below, our current limit is already providing noticeable benefits with minimal disruption, and we’re likely to tighten it a bit in the near future, so it will trigger after somewhat fewer consecutive failures.

Self-Service Unpausing

A key feature in our zombie issuance pausing mechanism is self-service unpausing. Whenever an account-hostname pair is paused, any new certificate requests for that hostname submitted by that account are immediately rejected. But this means that the “one successful validation immediately resets the rate limit counter” feature can no longer come into effect: once they’re paused, they can’t even attempt validation anymore.

So every rejection comes with an error message explaining what has happened and a custom link that can be used to immediately unpause that account-hostname pair and remove any other pauses on the same account at the same time. The point of this is that subscribers who notice at some point that issuance is failing and want to intervene to get it working again have a straightforward option to let Let’s Encrypt know that they’re aware of the recurring failures and are still planning to use a particular account. As soon as subscribers notify us via the self-service link, they’ll be able to issue certificates again.

Currently, the user interface for an affected subscriber looks like this:

This link would be provided via an ACME error message in response to any request that was blocked due to a pause account-hostname pair.

As it’s turned out, the unpause option shown above has only been used by about 3% of affected accounts! This goes to show that most of the zombies we’ve paused were, in fact, well and truly forgotten about.

However, the unpause feature is there for whenever it’s needed, and there may be cases when it will become more important. A very large integration could trigger the zombie-related rate limits if a newly-introduced software bug causes what looks like a very high volume of zombie requests in a very short time. In that case, once that bug has been noticed and fixed, an integrator may need to unpause its issuance on behalf of lots of customers at once. Our unpause feature permits unpausing 50,000 domain names on a single account at a time, so even the largest integrators can get themselves unpaused expeditiously in this situation.

Conclusion

We’ve been very happy with the results of our zombie mitigation measures, and, as far as we can tell, there’s been almost no impact for subscribers! Our statistics indicate that we’ve managed to reduce the load on our infrastructure while causing no detectable harm or inconvenience to subscribers’ valid issuance requests.

Since implementing the manual pauses in September and the automated pauses in December, we’ve seen:

• Over 100,000 account-hostname pairs have been paused for excessive failures.
• We received zero (!) associated complaints or support requests.
• About 3,200 people manually unpaused issuance.
• Failed certificate orders fell by about 30% so far, and should continue to fall over time as we fine-tune the rate limit formula and catch more zombie clients.

The new rate limit and the self-service unpause system are also ready to deal with circumstances that might produce more zombie clients in the future. For instance, we’ve announced that we’re going to be discontinuing renewal reminder emails soon. If some subscribers overlook failed renewals in the future, we might see more paused clients that result from unintentional renewal failures. We think taking advantage of the existing self-service unpause feature will be straightforward in that case. But it’s much better to notice problems and get them fixed up front, so please remember to set up your own monitoring to avoid unnoticed renewal failures in the future.

If you’re a subscriber who’s had occasion to use the self-service unpause feature, we’d love your feedback on the Community Forum about your experience using the feature and the circumstances that surrounded your account’s getting paused.

Also, if you’re a Let’s Encrypt client developer, please remember to make renewal requests at a random time (not precisely at midnight) so that the load on our infrastructure is smoothed out. You can also reduce the impact of zombie renewals by repeating failed requests somewhat less frequently over time (a “back-off” strategy), especially if the failure reason makes it look like a domain name may no longer be in use at all.

At Let’s Encrypt we know that building a secure Internet isn’t just a technical challenge—it’s a long-term commitment. Over the past decade we’ve made enormous strides: from issuing billions of TLS certificates to continually innovating to keep the web safer and more accessible. But none of this would be possible without recurring donations from individuals and organizations around the world.

Recurring donations are more than just financial support; they allow us to plan, innovate, and keep improving with confidence, knowing that month after month, year after year, our supporters are there. This consistent backing empowers us to maintain a secure, privacy-respecting Internet for all.

Our tenth anniversary tagline, Encryption for Everybody, highlights this vision. It’s both a technical goal and a fundamental belief that secure communication should be available to everyone, everywhere.

When we asked our recurring donors why they give, their responses affirmed how essential this commitment is. One longtime supporter shared:

For some, it’s about paying it forward, helping future users benefit as they once did:

The next decade of Let’s Encrypt will likely be about maintaining that commitment to encryption for everybody. It’s about ensuring that our work remains reliable, accessible, and—most importantly—supported by people who believe in what we do. To everyone who’s been part of this journey, thank you. We couldn’t do it without you.

During Let’s Encrypt’s 10th Anniversary Year, we’re celebrating our community and reflecting on our journey. We’d be thrilled to hear from you. Connect with us on LinkedIn, our community forum, or email us at outreach@letsencrypt.org. Let’s keep building a secure Internet together!

Let’s Encrypt is a project of the nonprofit Internet Security Research Group, a 501(c)(3) nonprofit. To support our work, visit letsencrypt.org/donate.

Let’s Encrypt will no longer include the “TLS Client Authentication” Extended Key Usage (EKU) in our certificates beginning in 2026. Most users who use Let’s Encrypt to secure websites won’t be affected and won’t need to take any action. However, if you use Let’s Encrypt certificates as client certificates to authenticate to a server, this change may impact you.

To minimize disruption, Let’s Encrypt will roll this change out in multiple stages, using ACME Profiles:

• Today: Let’s Encrypt already excludes the Client Authentication EKU on our tlsserver ACME profile. You can verify compatibility by issuing certificates with this profile now.
• October 1, 2025: Let’s Encrypt will launch a new tlsclient ACME profile which will retain the TLS Client Authentication EKU. Users who need additional time to migrate can opt-in to this profile.
• February 11, 2026: the default classic ACME profile will no longer contain the Client Authentication EKU.
• May 13, 2026: the tlsclient ACME profile will no longer be available and no further certificates with the Client Authentication EKU will be issued.

Once this is completed, Let’s Encrypt will switch to issuing with new intermediate Certificate Authorities which also do not contain the TLS Client Authentication EKU.

For some background information, all certificates include a list of intended uses, known as Extended Key Usages (EKU). Let’s Encrypt certificates have included two EKUs: TLS Server Authentication and TLS Client Authentication.

• TLS Server Authentication is used to authenticate connections to TLS Servers, like websites.
• TLS Client Authentication is used by clients to authenticate themselves to a server. This feature is not typically used on the web, and is not required on the certificates used on a website.

After this change is complete, only TLS Server Authentication will be available from Let’s Encrypt.

This change is prompted by changes to Google Chrome’s root program requirements, which impose a June 2026 deadline to split TLS Client and Server Authentication into separate PKIs. Many uses of client authentication are better served by a private certificate authority, and so Let’s Encrypt is discontinuing support for TLS Client Authentication ahead of this deadline.

Wed, 14 May 2025 00:00:00 +0000

---

How Pebble Supports ACME Client Developers

Since Let’s Encrypt started issuing certificates in 2015, people have repeatedly requested the ability to get certificates for IP addresses, an option that only a few certificate authorities have offered. Until now, they’ve had to look elsewhere, because we haven’t provided that feature.

Today, we’ve issued our first certificate for an IP address, as we announced we would in January. As with other new certificate features on our engineering roadmap, we’ll now start gradually rolling out this option to more and more of our subscribers.

Some Background on IP Address Certs

IP addresses are the underlying numerical addresses used on the Internet. Every device on the Internet has one (though, in modern practice, it might be shared with other devices, like when an entire home network shares a single public IP address). The Internet infrastructure uses them to route communications to their proper destination. IP addresses come in two forms, IPv4 and IPv6, and generally look like 54.215.62.21 (IPv4) or 2600:1f1c:446:4900::65 (IPv6).

Most Internet users rarely see or refer to IP addresses directly. Instead, they almost always use domain names like letsencrypt.org to refer to Internet services. The domain name system (DNS) is a part of the Internet infrastructure that’s responsible for allowing software to find the IP addresses associated with a particular domain name. For instance, your web browser can use DNS to find out that the service https://letsencrypt.org/ (Let’s Encrypt’s own website) is provided from the IP addresses 54.215.62.21 and 2600:1f1c:446:4900::65, among several others. This probably happened behind the scenes before you started reading this article! Your web browser needed to know our IP address in order to actually connect to our site and fetch this article.

Because we overwhelmingly tend to think and talk about Internet services in terms of domain names, those are the identifiers that are normally listed in certificates like those that Let’s Encrypt provides to our subscribers. Since you know us as “letsencrypt.org” and not as, say, “54.215.62.21,” it makes the most sense for our domain name to be on our certificate. After all, that’s what you’ll want your web browser to check against. This also gives Internet services more flexibility to be hosted in multiple locations, or to change where they’re hosted, without necessarily needing separate certificates for each server.

In principle, there’s no reason that a certificate couldn’t be issued for an IP address rather than a domain name, and in fact the technical and policy standards for certificates have always allowed this, with a handful of certificate authorities offering this service on a small scale. In Let’s Encrypt’s case, we’ve preferred to wait until some other pieces, like short-lived certs, were in place before we made this option available for our subscribers.

Why IP Address Certs Are Less Common

First and foremost, it’s because Internet users usually know services by domain names, not by IP addresses, and because IP addresses can easily change “behind the scenes” with no prior notice. For instance, a popular site could switch from one cloud hosting company to a different one, and update its DNS records to point at the new host. Most users wouldn’t ever notice the change at all, even though the site’s underlying IP addresses would be completely different.

Second, because IP addresses can change so easily, the sense of “ownership” one might have for them—or that a certificate authority might be able to attest to—tends to be weaker than for a domain name. If you’re hosting something in your house on a residential broadband connection, your Internet service provider most likely doesn’t guarantee that your IP address will stay the same over time. (That is, most home Internet users have a “dynamic IP address” from their ISPs, rather than a “static IP address.”) In that case, you have to contend with the possibility that that address may change often, possibly without warning, and that your old address may be assigned to somebody else.

Third, most Internet service operators don’t expect that end users will ever intentionally connect to their sites directly by IP address. In some cases, when an IP address is shared by different websites or different devices, connecting by IP address alone wouldn’t even work properly. In that case, there’s not much benefit to obtaining a certificate for the IP address!

How Let’s Encrypt Subscribers May Use IP Address Certs

Most current subscribers should be fine with their existing domain name certs and won’t need IP address certs. Subscribers who have a use for an IP address cert are typically already aware of that. A few use cases that we’re aware of include:

• A default page for hosting providers, in case someone pastes a server’s IP address into a browser instead of an individual site name (right now, this normally produces an error in the browser).

• A way to access your website if you don’t have a domain name at all (at some cost in reliability and convenience compared to getting a domain name).

• Securing DNS over HTTPS (DoH) or other infrastructure services. Having a certificate makes it much easier for DoH servers to prove their identities to clients. That could make it more feasible for DoH users or clients to enforce a requirement for a valid publicly-trusted certificate when connecting to DoH servers.

• Securing remote access to some home devices (like network-attached storage servers and Internet-of-things devices) even without a domain name.

• Securing ephemeral connections within cloud hosting infrastructure, like connections between one back-end cloud server and another, or ephemeral connections to administer new or short-lived back-end servers via HTTPS—as long as those servers have at least one public IP address available.

How To Get an IP Address Cert

IP address certificates are available right now in Staging. They should be generally available in Prod later in 2025, at the same time that short-lived certificates become generally available. Prior to general availability we may allow list issuance for a limited number of partners who can provide us with feedback.

Many Let’s Encrypt client applications should already be able to request certificates for IP addresses, although there can be minor technical changes required to support this in some client software.

As a matter of policy, Let’s Encrypt certificates that cover IP addresses must be short-lived certs, valid for only about six days. As such, your ACME client must support the draft ACME Profiles specification, and you must configure it to request the shortlived profile. And, probably not surprisingly, you can’t use the DNS challenge method to prove your control over an IP address; only the http-01 and tls-alpn-01 methods can be used.

If your client software requests an IP address cert with details that aren’t compatible with these policies, the order will be rejected by the ACME server. In this case, your client application may need to be updated or reconfigured. Feel free to ask for help on the Let’s Encrypt community forum if you encounter any problems, either as a client application developer or as an end user.

Since its inception, Let’s Encrypt has been sending expiration notification emails to subscribers that have provided an email address to us via the ACME API. This service ended on June 4, 2025. The decision to end the service is the result of the following factors:

1. Over the past 10 years more and more of our subscribers have been able to put reliable automation into place for certificate renewal.
2. Providing expiration notification emails means that we have to retain millions of email addresses connected to issuance records. As an organization that values privacy, removing this requirement is important to us.
3. Providing expiration notifications costs Let’s Encrypt tens of thousands of dollars per year, money that we believe can be better spent on other aspects of our infrastructure.
4. Providing expiration notifications adds complexity to our infrastructure, which takes time and attention to manage and increases the likelihood of mistakes being made. Over the long term, particularly as we add support for new service components, we need to manage overall complexity by phasing out system components that can no longer be justified.

For those who would like to continue receiving expiration notifications, we recommend using a third party service such as Red Sift Certificates Lite (formerly Hardenize). Red Sift’s monitoring service providing expiration emails is free of charge for up to 250 certificates. More monitoring options can be found here.

We have deleted the email addresses provided to Let’s Encrypt via the ACME API that were stored in our CA database in association with issuance data. This doesn’t affect addresses signed up to mailing lists and other systems. They are managed in a separate ISRG system unassociated with issuance data.

Going forward, if an email address is provided to Let’s Encrypt via the ACME API, Let’s Encrypt will not store the address but will instead forward it to the general ISRG mailing list system unassociated with any account data. If the email address has not been seen before, that system may send an onboarding email with information about how to subscribe to various sources of updates.

If you’d like to stay informed about technical updates and other news about Let’s Encrypt and our parent nonprofit, ISRG, based on the preferences you choose, you can sign up for our email lists below:

The Certificate Transparency ecosystem has been improving transparency for the web PKI since 2013. It helps make clear exactly what certificates each certificate authority has issued and makes sure errors or compromises of certificate authorities are detectable.

Let’s Encrypt participates in CT both as a certificate issuer and as a log operator. For the past year, we’ve also been running an experiment to help validate a next-generation design for Certificate Transparency logs. That experiment is now nearing a successful conclusion. We’ve demonstrated that the new architecture (called the “Static CT API”) works well, providing greater efficiency and making it easier to run huge and reliable CT log services with comparatively modest resources. The Static CT API also makes it easier to download and share data from CT logs.

The Sunlight log implementation, alongside other Static CT API log implementations, is now on a path to production use. Browsers are now officially accepting Static CT API logs into their log programs as a means to help guarantee that the contents of CA-issued certificates are all publicly disclosed and publicly accessible (see Safari’s and Chrome’s recent announcements), although the browsers also require the continued use of a traditional RFC 6962 log alongside the new type.

All of this is good news for everyone who runs, submits certificates to, or monitors a CT log: as the new architecture gets adopted, we can expect to see more organizations running more logs, at lower cost, and with greater overall capacity to keep up with the large volume of publicly-trusted certificates.

Certificate Transparency

Certificate Transparency (CT) was introduced in 2013 in response to concerns about how Internet users could detect misbehavior and compromise of certificate authorities. Prior to CT, it was possible for a CA to issue an inaccurate or malicious certificate that could be used to attack a relatively small number of users, and that might never come to wider attention. A team led by Google responded to this by creating a transparency log mechanism, where certificate authorities (like Let’s Encrypt) must disclose all of the certificates that we issue by submitting them to public log services. Web browsers now generally reject certificates unless the certificates include cryptographic proof (“Signed Certificate Timestamps”, or SCTs) demonstrating that they were submitted to and accepted by such logs.

The CT logs themselves use a cryptographic append-only ledger to prove that they haven’t deleted or modified their records. There are currently over a dozen CT log services, most of them also run by certificate authorities, including Let’s Encrypt’s own Oak log.

The Static CT API

The original 2013 CT log design has been used with relatively few technical changes since it was first introduced, but several other transparency logging systems have been created in other areas, such as sumdb for Golang, which helps ensure that the contents of Golang package updates are publicly recorded. While they were originally inspired by CT, more-recently invented transparency logs have improved on its design.

The current major evolution of CT was led by Filippo Valsorda, a cryptographer with an interest in transparency log mechanisms, with help from others in the CT ecosystem. Portions of the new design are directly based on sumdb. In addition to designing the new architecture, Valsorda also wrote the implementation that we’ve been using, called Sunlight, with support from Let’s Encrypt. We’re excited to see that there are now at least three other compatible implementations: Google’s trillian-tessera, Cloudflare’s Azul, and an independent project called Itko.

The biggest change for the Static CT API is that logs are now represented, and downloaded by verifiers, as simple collections of flat files (called “tiles,” so some implementers have also been referring to these as “tiled logs” or “tlogs”). Anyone who wants to download log data can do so just by downloading these files. This is great for log operators because these simple file downloads can be distributed in various ways, including caching by a CDN, which was less practical and efficient for the classic CT API.

The new design is also simpler and more efficient from the log operator’s perspective, making it cheaper to run logs. As we said last year, this may enable us and other operators to increase reliability and availability by running several separate logs, likely with lower overall resource requirements than a single traditional log.

Our Sunlight experiment

For the past year, we’ve run three Sunlight logs, called Twig, Willow, and Sycamore. We’ve been logging all of our own issued certificates, which represent a majority of the total volume of all publicly-trusted certificates, into our Sunlight logs. Sunlight logged these certificates quickly and correctly on relatively modest server hardware. Notably, each log’s write side was handled comfortably by just a single server. We also achieved high availability for these log services throughout the course of this experiment. (Because our Sunlight logs are not yet trusted by web browsers, we didn’t include the SCT proofs that they returned to us in the actual certificates we gave out to our subscribers; those proofs wouldn’t have been of use to our subscribers yet and would just have taken up space.)

A potential failure mode of traditional CT logs is that they could be unacceptably slow in incorporating newly-submitted certificates (known as missing the maximum merge delay), which can result in a log becoming distrusted. This isn’t a possibility for our new Sunlight-based logs: they always completely incorporate newly-submitted certificates before returning an SCT to the submitter, so the effective merge delay is zero! Of course, any log can suffer outages for a variety of reasons, but this feature of Sunlight makes it less likely that any outages will be fatal to a log’s continued operation.

We’ve demonstrated that Sunlight and the Static CT API work in practice, and this demonstration has helped to confirm the browser developers’ hope that Static CT API logs can become an officially-supported part of CT. As a result, the major browsers that enforce CT have now permitted Static CT API logs to apply for inclusion in browsers as publicly-trusted logs, and we’re preparing to apply for this status for our Willow and Sycamore logs with the Chrome and Safari CT log programs.

Let’s Encrypt will run at least these two logs, and possibly others over time, for the foreseeable future. Once they’re trusted by browsers, we’ll encourage other CAs to submit to them as well, and we’ll begin including SCTs from these logs in our own certificates (alongside SCTs from traditional CT logs).

How to participate

The new Static CT API and the rollout of tile-based logs will bring various changes and opportunities for community members.

New Certificate Transparency log operators

Companies and non-profit organizations could help support the web PKI by running a CT log and applying for it to be publicly trusted. Implementations like Sunlight will have substantially lower resource requirements than first-generation CT logs, particularly when cached behind a CDN. The biggest resource demands for a log operator will be storage and upstream bandwidth. A publicly-trusted log is also expected to maintain relatively high availability, because CAs need logs to be available in order to continue issuing certificates.

We don’t have statistics to share about the exact resource requirements for such a log yet, but after we have practical experience running a fully publicly-trusted Sunlight log, we should be able to make this more concrete. As noted above, the compute side of the log can be handled by a single server. Sunlight author Filippo Valsorda has recently started running a Sunlight log—also on just a single server—and offered more detailed cost breakdowns for that log’s setup, with an estimated total cost around $10,000 per year. The costs for our production Static CT API logs may be higher than those for Filippo’s log, but still far less than the costs for our traditional RFC 6962 logs.

As with trust decisions about CAs, browser developers are the authorities about which CT logs become publicly trusted. Although any person or organization can run a log, browser developers will generally prefer to trust logs whose continued availability they’re confident of—typically those run by stable organizations with experience running some form of public Internet services. Unlike becoming a certificate authority, running a log does not require a formal audit, as the validation of the log’s availability and correctness can be performed purely by observation.

Certificate authorities

Once the Willow and Sycamore logs are trusted by browsers, our fellow certificate authorities can choose to start logging certificates to them as part of their issuance processes. (Initially, you should still include at least one SCT from a traditional CT log in each certificate.) The details, including the log API endpoints and keys, are available at our CT log page. You can start submitting to these logs right away if you prefer; just bear in mind that the SCTs they return aren’t useful to subscribers yet, and won’t be useful until browsers are updated to trust the new logs.

CT data users

You can monitor CT in order to watch for certificate issuances for your own domain names, or as part of monitoring or security products or services, or for Internet security research purposes. Many of our colleagues have been doing this for some time as a part of various tools they maintain. The Static CT API should make this easier, because you’ll be able to download and share log tiles as sets of ordinary files.

If you already run such monitoring tools, please note that you’ll need to update your data pipeline in order to access Static CT API logs; since the read API is not backwards-compatible, CT API clients will need to be modified to support the new API. Without updated tools, your view of the CT system will become partial!

Also note that getting a complete view of all of CT will still require downloading data from traditional logs, which will probably continue to be true for several years.

Software developers

As logs based on the new API enter production use, it will be important to have tools to interact with and search these logs. We can all benefit from more software that understands how to do this. Since file downloads are such a familiar piece of software functionality, it will probably be easier for developers to develop against the new API compared to the original one.

We’ve also continued to see greater integration of transparency logging tools into other kinds of services, such as software updates. There’s a growing transparency log ecosystem that’s always in need of more tools and integrations. As we mentioned above, transparency logs are increasingly learning from one another, and there are also mechanisms for more direct integration between different kinds of transparency logs (known as “witnessing”). Software developers can help improve different aspects of Internet security by contributing to this active and growing area.

Conclusion

The Certificate Transparency community and larger transparency logging community have experienced a virtuous cycle of innovation, sharing ideas and implementation code between different systems and demonstrating the feasibility of new mechanisms and functionality. With the advent of tile-based logging in CT, the state of the art has moved forward in a way that helps log operators run our logs much more efficiently without compromising security.

We’re proud to have participated in this experiment and the engineering conversation around the evolution of logging architectures. Now that we’ve shown how well the new API really works at scale, we look forward to having publicly-trusted Sunlight logs later this year!

Every night, right around midnight (mainly UTC), a horde of zombies wakes up and clamors for … digital certificates!

The zombies in question are abandoned or misconfigured Internet servers and ACME clients that have been set to request certificates from Let’s Encrypt. As our certificates last for at most 90 days, these zombie clients’ software knows that their certificates are out-of-date and need to be replaced. What they don’t realize is that their quest for new certificates is doomed! These devices are cursed to seek certificates again and again, never receiving them.

But they do use up a lot of certificate authority resources in the process.

The Zombie Client Problem

Unlike a human being, software doesn’t give up in frustration, or try to modify its approach, when it repeatedly fails at the same task. Our emphasis on automation means that the vast majority of Let’s Encrypt certificate renewals are performed by automated software. This is great when those renewals succeed, but it also means that forgotten clients and devices can continue requesting renewals unsuccessfully for months, or even years.

How might that happen? Most often, it happens when a device no longer has a domain name pointed to it. The device itself doesn’t know that this has changed, so it treats renewal failures as transient even though they are actually permanent. For instance:

• An organization may have allowed a domain name registration to lapse because it is no longer needed, but its servers are still configured to request certs for it.
• Or, a home user stopped using a particular dynamic-DNS domain with a network-attached storage device, but is still using that device at home. The device doesn’t realize that the user no longer expects to use the name, so it keeps requesting certs for it.
• Or, a web hosting or CDN customer migrated to a different service provider, but never informed the old service provider. The old service provider’s servers keep requesting certs unsuccessfully. If the customer was in a free service tier, there might not be invoices or charges reminding the customer to cancel the service.
• Or any number of other, subtler changes in a subscriber’s infrastructure, such as changing a firewall rule or some webserver configuration.

At the scale of Let’s Encrypt, which now covers hundreds of millions of names, scenarios like these have become common, and their impact has become substantial. In 2024, we noticed that about half of all certificate requests to the Let’s Encrypt ACME API came from about a million accounts that never successfully complete any validations. Many of these had completed validations and issued certificates sometime in the past, but nowadays every single one of their validation attempts fails, and they show no signs that this will change anytime soon.

Unfortunately, trying to validate those futile requests still uses resources. Our CA software has to generate challenges, reach out and attempt to validate them over the Internet, detect and report failures, and record all of the associated information in our databases and audit logs. And over time, we’ve seen more and more recurring failures: accounts that always fail their issuance requests have been growing at around 18% per year.

In January, we mentioned that we had been addressing the zombie client problem through our rate limit system. This post provides more detail on that progress. 

Our Rate Limit Philosophy

If you’ve used Let’s Encrypt as a subscriber, you may have run into one of our rate limits at some point, maybe during your initial setup process. We have eight different kinds of rate limits in place now; as our January post describes, they’ve become more algorithmically sophisticated and grown to address a wider range of problems. A key principle for Let’s Encrypt is that our rate limiting is not a punishment. We don’t think of rate limits as a way of retaliating against a client for misbehavior. Rate limits are simply a tool to maximize the efficient use of our limited resources and prevent people and programs from using up those resources for no constructive purpose.

We’ve consistently tried to design our rate limit mechanisms in line with that philosophy. So if a misconfiguration or misunderstanding has caused excessive requests in the past, we’re still happy to welcome the user in question back and start issuing them certificates again—once the problem has been addressed. We want the rate limits to put a brake on wasteful use of our systems, but not to frustrate users who are actively trying to make Let’s Encrypt work for them.

In addition, we’ve always implemented our rate limits to err on the side of permissiveness. For example, if the Redis instances where rate limits are tracked have an outage or lose data, the system is designed to permit more issuance rather than less issuance as a result.

We wanted to create additional limits that would target zombie clients, but in a correspondingly non-punitive way that would avoid any disruption to valid issuance, and welcome subscribers back quickly if they happened to notice and fix a long-time problem with their setups.

Our Zombie-Related Rate Limits and Their Impact

In planning a new zombie-specific response, we decided on a “pausing” approach, which can temporarily limit an account’s ability to proceed with certificate requests. The core idea is that, if a particular account consistently fails to complete validation for a particular hostname, we’ll pause that account-hostname pair. The pause means that any new order requests from that account for that hostname will be rejected immediately, before we get to the resource-intensive validation phase.

This approach is more finely targeted than pausing an entire account. Pausing account-hostname pairs means that your ability to issue certs for a specific name could be paused due to repeated failures, but you can still get all of your other certs like normal. So a large hosting provider doesn’t have to fear that its certificate issuance on behalf of one customer will be affected by renewal failures related to a problem with a different customer’s domain name. The account-specificity of the pause, in turn, means that validation failures from one subscriber or device won’t prevent a different subscriber or device from attempting to validate the same name, as long as the devices in question don’t share a single Let’s Encrypt account.

In September 2024, we began applying our zombie rate limits manually by pausing about 21,000 of the most recurrently-failing account-hostname pairs, those which were consistently repeating the same failed requests many times per day, every day. After implementing that first round of pauses, we immediately saw a significant impact on our failed request rates. As we announced at that time, we also began using a formula to automatically pause other zombie client account-hostname pairs from December 2024 onward. The associated new rate limit is called “Consecutive Authorization Failures per Hostname Per Account” (and is independent of the existing “Authorization Failures per Hostname Per Account” limit, which resets every hour).

This formula relates to the frequency of successive failed issuance requests for the same domain name by the same Let’s Encrypt account. It applies only to failures that happen again and again, with no successful issuances at all in between: a single successful validation immediately resets the rate limit all the way to zero. Like all of our rate limits, this is not a punitive measure but is simply intended to reduce the waste of resources. So, we decided to set the thresholds rather high in the expectation that we would catch only the most disruptive zombie clients, and ultimately only those clients that were extremely unlikely to succeed in the future based on their substantial history of failed requests. We don’t hurry to block requesters as zombies: according to our current formula, client software following the default established by EFF’s Certbot (two renewal attempts per day) would be paused as a zombie only after about ten years of constant failures. More aggressive failed issuance attempts will get a client paused sooner, but clients will generally have to fail hundreds or thousands of attempts in a row before they are paused.

Most subscribers using mainstream client applications with default configurations will never encounter this rate limit, even if they forget to deactivate renewal attempts for domains that are no longer pointed at their servers. As described below, our current limit is already providing noticeable benefits with minimal disruption, and we’re likely to tighten it a bit in the near future, so it will trigger after somewhat fewer consecutive failures.

Self-Service Unpausing

A key feature in our zombie issuance pausing mechanism is self-service unpausing. Whenever an account-hostname pair is paused, any new certificate requests for that hostname submitted by that account are immediately rejected. But this means that the “one successful validation immediately resets the rate limit counter” feature can no longer come into effect: once they’re paused, they can’t even attempt validation anymore.

So every rejection comes with an error message explaining what has happened and a custom link that can be used to immediately unpause that account-hostname pair and remove any other pauses on the same account at the same time. The point of this is that subscribers who notice at some point that issuance is failing and want to intervene to get it working again have a straightforward option to let Let’s Encrypt know that they’re aware of the recurring failures and are still planning to use a particular account. As soon as subscribers notify us via the self-service link, they’ll be able to issue certificates again.

Currently, the user interface for an affected subscriber looks like this:

This link would be provided via an ACME error message in response to any request that was blocked due to a pause account-hostname pair.

As it’s turned out, the unpause option shown above has only been used by about 3% of affected accounts! This goes to show that most of the zombies we’ve paused were, in fact, well and truly forgotten about.

However, the unpause feature is there for whenever it’s needed, and there may be cases when it will become more important. A very large integration could trigger the zombie-related rate limits if a newly-introduced software bug causes what looks like a very high volume of zombie requests in a very short time. In that case, once that bug has been noticed and fixed, an integrator may need to unpause its issuance on behalf of lots of customers at once. Our unpause feature permits unpausing 50,000 domain names on a single account at a time, so even the largest integrators can get themselves unpaused expeditiously in this situation.

Conclusion

We’ve been very happy with the results of our zombie mitigation measures, and, as far as we can tell, there’s been almost no impact for subscribers! Our statistics indicate that we’ve managed to reduce the load on our infrastructure while causing no detectable harm or inconvenience to subscribers’ valid issuance requests.

Since implementing the manual pauses in September and the automated pauses in December, we’ve seen:

• Over 100,000 account-hostname pairs have been paused for excessive failures.
• We received zero (!) associated complaints or support requests.
• About 3,200 people manually unpaused issuance.
• Failed certificate orders fell by about 30% so far, and should continue to fall over time as we fine-tune the rate limit formula and catch more zombie clients.

The new rate limit and the self-service unpause system are also ready to deal with circumstances that might produce more zombie clients in the future. For instance, we’ve announced that we’re going to be discontinuing renewal reminder emails soon. If some subscribers overlook failed renewals in the future, we might see more paused clients that result from unintentional renewal failures. We think taking advantage of the existing self-service unpause feature will be straightforward in that case. But it’s much better to notice problems and get them fixed up front, so please remember to set up your own monitoring to avoid unnoticed renewal failures in the future.

If you’re a subscriber who’s had occasion to use the self-service unpause feature, we’d love your feedback on the Community Forum about your experience using the feature and the circumstances that surrounded your account’s getting paused.

Also, if you’re a Let’s Encrypt client developer, please remember to make renewal requests at a random time (not precisely at midnight) so that the load on our infrastructure is smoothed out. You can also reduce the impact of zombie renewals by repeating failed requests somewhat less frequently over time (a “back-off” strategy), especially if the failure reason makes it look like a domain name may no longer be in use at all.

At Let’s Encrypt we know that building a secure Internet isn’t just a technical challenge—it’s a long-term commitment. Over the past decade we’ve made enormous strides: from issuing billions of TLS certificates to continually innovating to keep the web safer and more accessible. But none of this would be possible without recurring donations from individuals and organizations around the world.

Recurring donations are more than just financial support; they allow us to plan, innovate, and keep improving with confidence, knowing that month after month, year after year, our supporters are there. This consistent backing empowers us to maintain a secure, privacy-respecting Internet for all.

Our tenth anniversary tagline, Encryption for Everybody, highlights this vision. It’s both a technical goal and a fundamental belief that secure communication should be available to everyone, everywhere.

When we asked our recurring donors why they give, their responses affirmed how essential this commitment is. One longtime supporter shared:

For some, it’s about paying it forward, helping future users benefit as they once did:

The next decade of Let’s Encrypt will likely be about maintaining that commitment to encryption for everybody. It’s about ensuring that our work remains reliable, accessible, and—most importantly—supported by people who believe in what we do. To everyone who’s been part of this journey, thank you. We couldn’t do it without you.

During Let’s Encrypt’s 10th Anniversary Year, we’re celebrating our community and reflecting on our journey. We’d be thrilled to hear from you. Connect with us on LinkedIn, our community forum, or email us at outreach@letsencrypt.org. Let’s keep building a secure Internet together!

Let’s Encrypt is a project of the nonprofit Internet Security Research Group, a 501(c)(3) nonprofit. To support our work, visit letsencrypt.org/donate.

Let’s Encrypt will no longer include the “TLS Client Authentication” Extended Key Usage (EKU) in our certificates beginning in 2026. Most users who use Let’s Encrypt to secure websites won’t be affected and won’t need to take any action. However, if you use Let’s Encrypt certificates as client certificates to authenticate to a server, this change may impact you.

To minimize disruption, Let’s Encrypt will roll this change out in multiple stages, using ACME Profiles:

• Today: Let’s Encrypt already excludes the Client Authentication EKU on our tlsserver ACME profile. You can verify compatibility by issuing certificates with this profile now.
• October 1, 2025: Let’s Encrypt will launch a new tlsclient ACME profile which will retain the TLS Client Authentication EKU. Users who need additional time to migrate can opt-in to this profile.
• February 11, 2026: the default classic ACME profile will no longer contain the Client Authentication EKU.
• May 13, 2026: the tlsclient ACME profile will no longer be available and no further certificates with the Client Authentication EKU will be issued.

Once this is completed, Let’s Encrypt will switch to issuing with new intermediate Certificate Authorities which also do not contain the TLS Client Authentication EKU.

For some background information, all certificates include a list of intended uses, known as Extended Key Usages (EKU). Let’s Encrypt certificates have included two EKUs: TLS Server Authentication and TLS Client Authentication.

• TLS Server Authentication is used to authenticate connections to TLS Servers, like websites.
• TLS Client Authentication is used by clients to authenticate themselves to a server. This feature is not typically used on the web, and is not required on the certificates used on a website.

After this change is complete, only TLS Server Authentication will be available from Let’s Encrypt.

This change is prompted by changes to Google Chrome’s root program requirements, which impose a June 2026 deadline to split TLS Client and Server Authentication into separate PKIs. Many uses of client authentication are better served by a private certificate authority, and so Let’s Encrypt is discontinuing support for TLS Client Authentication ahead of this deadline.

How Pebble Supports ACME Client Developers

Together with the IETF community, we created the ACME standard to support completely automated certificate issuance. This open standard is now supported by dozens of clients. On the server side, did you know that we have not one but two open-source ACME server implementations?

The big implementation, which we use ourselves in production, is called Boulder. Boulder handles all of the facets and details needed for a production certificate authority, including policy compliance, database interfaces, challenge verifications, and logging. You can adapt and use Boulder yourself if you need to run a real certificate authority, including an internal, non-publicly-trusted ACME certificate authority within an organization.

The small implementation is called Pebble. It’s meant entirely for testing, not for use as a real certificate authority, and we and ACME client developers use it for various automated and manual testing purposes. For example, Certbot has used Pebble in its development process for years in order to perform a series of basic but realistic checks of the ability to request and obtain certificates from an ACME server.

Pebble is Easy to Use for ACME Client Testing

For any developer or team creating an ACME client application, Pebble solves a range of problems along the lines of “how do I check whether I’ve implemented ACME correctly, so that I could actually get certificates from a CA, without necessarily using a real domain name, and without running into CA rate limits during my routine testing?” Pebble is quick and easy to set up if you need to test an ACME client’s functionality.

It runs in RAM without dependencies or persistence; you won’t need to set up a database or a configuration for it. You can get Pebble running with a single golang command in just a few seconds, and immediately start making local ACME requests. That’s suitable for inclusion in a client’s integration test suite, making much more realistic integration tests possible without needing to worry about real domains, CA rate limits, or network outages.

We see Pebble getting used in the official test suites for ACME clients including getssl, Lego, Certbot, simp_le, and others. In many cases, every change committed to the ACME client’s code base is automatically tested against Pebble.

Pebble is Intentionally Different From Boulder

Pebble is also deliberately different from Boulder in some places in order to provide clients with an opportunity to interoperate with slightly different ACME implementations. The Pebble code explains that

For instance, the Let’s Encrypt service currently offers its newAccount resource at the path /acme/new-acct, whereas Pebble uses a different name /sign-me-up, so clients will be reminded to check the directory rather than assuming a specific path. Other substantive differences include:

• Pebble rejects 5% of all requests as having a invalid nonce, even if the nonce was otherwise valid, so clients can test how they respond this error condition
• Pebble only reuses valid authorizations 50% of the time, so clients can check their ability to perform validations when they might not have expected to
• Pebble truncates timestamps to a different degree of precision than Boulder
• Unlike Boulder, Pebble respects the notBefore and notAfter fields of new-order requests

The ability of ACME clients to work with both versions is a good test of their conformance to the ACME specification, rather than making assumptions about the current behavior of the Let’s Encrypt service in particular. This helps ensure that clients will work properly with other ACME CAs, and also with future versions of Let’s Encrypt’s own API.

Pebble is Useful to Both Let’s Encrypt and Client Developers as ACME Evolves

We often test out new ACME features by implementing them, at least in a simplified form, in Pebble before Boulder. This lets us and client developers experiment with support for those features even before they get rolled out in our staging service. We can do this quickly because a Pebble feature implementation doesn’t have to work with a full-scale CA backend.

We continue to encourage ACME client developers to use a copy of Pebble to test their clients’ functionality and ACME interoperability. It’s convenient and it’s likely to increase the correctness and robustness of their client applications.

Try Out Pebble Yourself

Want to try Pebble with your ACME client right now? On a Unix-like system, you can run

git clone https://github.com/letsencrypt/pebble/
cd pebble
go run ./cmd/pebble

Wait a few seconds; now you have a working ACME CA directory available at https://localhost:14000/dir! Your local ACME Server can immediately receive requests and issue certificates, though not publicly-trusted ones, of course. (If you prefer, we also offer other options for installing Pebble, like a Docker image.)

We welcome code contributions to Pebble. For example, ACME client developers may want to add simple versions of an ACME feature that’s not currently tested in Pebble in order to make their test suites more comprehensive. Also, if you notice a possibly unintended divergence between Pebble and Boulder or Pebble and the ACME specification, we’d love for you to let us know.

Wed, 30 Apr 2025 00:00:00 +0000

---

Ten Years of Let's Encrypt: Announcing support from Jeff Atwood

Since Let’s Encrypt started issuing certificates in 2015, people have repeatedly requested the ability to get certificates for IP addresses, an option that only a few certificate authorities have offered. Until now, they’ve had to look elsewhere, because we haven’t provided that feature.

Today, we’ve issued our first certificate for an IP address, as we announced we would in January. As with other new certificate features on our engineering roadmap, we’ll now start gradually rolling out this option to more and more of our subscribers.

Some Background on IP Address Certs

IP addresses are the underlying numerical addresses used on the Internet. Every device on the Internet has one (though, in modern practice, it might be shared with other devices, like when an entire home network shares a single public IP address). The Internet infrastructure uses them to route communications to their proper destination. IP addresses come in two forms, IPv4 and IPv6, and generally look like 54.215.62.21 (IPv4) or 2600:1f1c:446:4900::65 (IPv6).

Most Internet users rarely see or refer to IP addresses directly. Instead, they almost always use domain names like letsencrypt.org to refer to Internet services. The domain name system (DNS) is a part of the Internet infrastructure that’s responsible for allowing software to find the IP addresses associated with a particular domain name. For instance, your web browser can use DNS to find out that the service https://letsencrypt.org/ (Let’s Encrypt’s own website) is provided from the IP addresses 54.215.62.21 and 2600:1f1c:446:4900::65, among several others. This probably happened behind the scenes before you started reading this article! Your web browser needed to know our IP address in order to actually connect to our site and fetch this article.

Because we overwhelmingly tend to think and talk about Internet services in terms of domain names, those are the identifiers that are normally listed in certificates like those that Let’s Encrypt provides to our subscribers. Since you know us as “letsencrypt.org” and not as, say, “54.215.62.21,” it makes the most sense for our domain name to be on our certificate. After all, that’s what you’ll want your web browser to check against. This also gives Internet services more flexibility to be hosted in multiple locations, or to change where they’re hosted, without necessarily needing separate certificates for each server.

In principle, there’s no reason that a certificate couldn’t be issued for an IP address rather than a domain name, and in fact the technical and policy standards for certificates have always allowed this, with a handful of certificate authorities offering this service on a small scale. In Let’s Encrypt’s case, we’ve preferred to wait until some other pieces, like short-lived certs, were in place before we made this option available for our subscribers.

Why IP Address Certs Are Less Common

First and foremost, it’s because Internet users usually know services by domain names, not by IP addresses, and because IP addresses can easily change “behind the scenes” with no prior notice. For instance, a popular site could switch from one cloud hosting company to a different one, and update its DNS records to point at the new host. Most users wouldn’t ever notice the change at all, even though the site’s underlying IP addresses would be completely different.

Second, because IP addresses can change so easily, the sense of “ownership” one might have for them—or that a certificate authority might be able to attest to—tends to be weaker than for a domain name. If you’re hosting something in your house on a residential broadband connection, your Internet service provider most likely doesn’t guarantee that your IP address will stay the same over time. (That is, most home Internet users have a “dynamic IP address” from their ISPs, rather than a “static IP address.”) In that case, you have to contend with the possibility that that address may change often, possibly without warning, and that your old address may be assigned to somebody else.

Third, most Internet service operators don’t expect that end users will ever intentionally connect to their sites directly by IP address. In some cases, when an IP address is shared by different websites or different devices, connecting by IP address alone wouldn’t even work properly. In that case, there’s not much benefit to obtaining a certificate for the IP address!

How Let’s Encrypt Subscribers May Use IP Address Certs

Most current subscribers should be fine with their existing domain name certs and won’t need IP address certs. Subscribers who have a use for an IP address cert are typically already aware of that. A few use cases that we’re aware of include:

• A default page for hosting providers, in case someone pastes a server’s IP address into a browser instead of an individual site name (right now, this normally produces an error in the browser).

• A way to access your website if you don’t have a domain name at all (at some cost in reliability and convenience compared to getting a domain name).

• Securing DNS over HTTPS (DoH) or other infrastructure services. Having a certificate makes it much easier for DoH servers to prove their identities to clients. That could make it more feasible for DoH users or clients to enforce a requirement for a valid publicly-trusted certificate when connecting to DoH servers.

• Securing remote access to some home devices (like network-attached storage servers and Internet-of-things devices) even without a domain name.

• Securing ephemeral connections within cloud hosting infrastructure, like connections between one back-end cloud server and another, or ephemeral connections to administer new or short-lived back-end servers via HTTPS—as long as those servers have at least one public IP address available.

How To Get an IP Address Cert

IP address certificates are available right now in Staging. They should be generally available in Prod later in 2025, at the same time that short-lived certificates become generally available. Prior to general availability we may allow list issuance for a limited number of partners who can provide us with feedback.

Many Let’s Encrypt client applications should already be able to request certificates for IP addresses, although there can be minor technical changes required to support this in some client software.

As a matter of policy, Let’s Encrypt certificates that cover IP addresses must be short-lived certs, valid for only about six days. As such, your ACME client must support the draft ACME Profiles specification, and you must configure it to request the shortlived profile. And, probably not surprisingly, you can’t use the DNS challenge method to prove your control over an IP address; only the http-01 and tls-alpn-01 methods can be used.

If your client software requests an IP address cert with details that aren’t compatible with these policies, the order will be rejected by the ACME server. In this case, your client application may need to be updated or reconfigured. Feel free to ask for help on the Let’s Encrypt community forum if you encounter any problems, either as a client application developer or as an end user.

Since its inception, Let’s Encrypt has been sending expiration notification emails to subscribers that have provided an email address to us via the ACME API. This service ended on June 4, 2025. The decision to end the service is the result of the following factors:

1. Over the past 10 years more and more of our subscribers have been able to put reliable automation into place for certificate renewal.
2. Providing expiration notification emails means that we have to retain millions of email addresses connected to issuance records. As an organization that values privacy, removing this requirement is important to us.
3. Providing expiration notifications costs Let’s Encrypt tens of thousands of dollars per year, money that we believe can be better spent on other aspects of our infrastructure.
4. Providing expiration notifications adds complexity to our infrastructure, which takes time and attention to manage and increases the likelihood of mistakes being made. Over the long term, particularly as we add support for new service components, we need to manage overall complexity by phasing out system components that can no longer be justified.

For those who would like to continue receiving expiration notifications, we recommend using a third party service such as Red Sift Certificates Lite (formerly Hardenize). Red Sift’s monitoring service providing expiration emails is free of charge for up to 250 certificates. More monitoring options can be found here.

We have deleted the email addresses provided to Let’s Encrypt via the ACME API that were stored in our CA database in association with issuance data. This doesn’t affect addresses signed up to mailing lists and other systems. They are managed in a separate ISRG system unassociated with issuance data.

Going forward, if an email address is provided to Let’s Encrypt via the ACME API, Let’s Encrypt will not store the address but will instead forward it to the general ISRG mailing list system unassociated with any account data. If the email address has not been seen before, that system may send an onboarding email with information about how to subscribe to various sources of updates.

If you’d like to stay informed about technical updates and other news about Let’s Encrypt and our parent nonprofit, ISRG, based on the preferences you choose, you can sign up for our email lists below:

The Certificate Transparency ecosystem has been improving transparency for the web PKI since 2013. It helps make clear exactly what certificates each certificate authority has issued and makes sure errors or compromises of certificate authorities are detectable.

Let’s Encrypt participates in CT both as a certificate issuer and as a log operator. For the past year, we’ve also been running an experiment to help validate a next-generation design for Certificate Transparency logs. That experiment is now nearing a successful conclusion. We’ve demonstrated that the new architecture (called the “Static CT API”) works well, providing greater efficiency and making it easier to run huge and reliable CT log services with comparatively modest resources. The Static CT API also makes it easier to download and share data from CT logs.

The Sunlight log implementation, alongside other Static CT API log implementations, is now on a path to production use. Browsers are now officially accepting Static CT API logs into their log programs as a means to help guarantee that the contents of CA-issued certificates are all publicly disclosed and publicly accessible (see Safari’s and Chrome’s recent announcements), although the browsers also require the continued use of a traditional RFC 6962 log alongside the new type.

All of this is good news for everyone who runs, submits certificates to, or monitors a CT log: as the new architecture gets adopted, we can expect to see more organizations running more logs, at lower cost, and with greater overall capacity to keep up with the large volume of publicly-trusted certificates.

Certificate Transparency

Certificate Transparency (CT) was introduced in 2013 in response to concerns about how Internet users could detect misbehavior and compromise of certificate authorities. Prior to CT, it was possible for a CA to issue an inaccurate or malicious certificate that could be used to attack a relatively small number of users, and that might never come to wider attention. A team led by Google responded to this by creating a transparency log mechanism, where certificate authorities (like Let’s Encrypt) must disclose all of the certificates that we issue by submitting them to public log services. Web browsers now generally reject certificates unless the certificates include cryptographic proof (“Signed Certificate Timestamps”, or SCTs) demonstrating that they were submitted to and accepted by such logs.

The CT logs themselves use a cryptographic append-only ledger to prove that they haven’t deleted or modified their records. There are currently over a dozen CT log services, most of them also run by certificate authorities, including Let’s Encrypt’s own Oak log.

The Static CT API

The original 2013 CT log design has been used with relatively few technical changes since it was first introduced, but several other transparency logging systems have been created in other areas, such as sumdb for Golang, which helps ensure that the contents of Golang package updates are publicly recorded. While they were originally inspired by CT, more-recently invented transparency logs have improved on its design.

The current major evolution of CT was led by Filippo Valsorda, a cryptographer with an interest in transparency log mechanisms, with help from others in the CT ecosystem. Portions of the new design are directly based on sumdb. In addition to designing the new architecture, Valsorda also wrote the implementation that we’ve been using, called Sunlight, with support from Let’s Encrypt. We’re excited to see that there are now at least three other compatible implementations: Google’s trillian-tessera, Cloudflare’s Azul, and an independent project called Itko.

The biggest change for the Static CT API is that logs are now represented, and downloaded by verifiers, as simple collections of flat files (called “tiles,” so some implementers have also been referring to these as “tiled logs” or “tlogs”). Anyone who wants to download log data can do so just by downloading these files. This is great for log operators because these simple file downloads can be distributed in various ways, including caching by a CDN, which was less practical and efficient for the classic CT API.

The new design is also simpler and more efficient from the log operator’s perspective, making it cheaper to run logs. As we said last year, this may enable us and other operators to increase reliability and availability by running several separate logs, likely with lower overall resource requirements than a single traditional log.

Our Sunlight experiment

For the past year, we’ve run three Sunlight logs, called Twig, Willow, and Sycamore. We’ve been logging all of our own issued certificates, which represent a majority of the total volume of all publicly-trusted certificates, into our Sunlight logs. Sunlight logged these certificates quickly and correctly on relatively modest server hardware. Notably, each log’s write side was handled comfortably by just a single server. We also achieved high availability for these log services throughout the course of this experiment. (Because our Sunlight logs are not yet trusted by web browsers, we didn’t include the SCT proofs that they returned to us in the actual certificates we gave out to our subscribers; those proofs wouldn’t have been of use to our subscribers yet and would just have taken up space.)

A potential failure mode of traditional CT logs is that they could be unacceptably slow in incorporating newly-submitted certificates (known as missing the maximum merge delay), which can result in a log becoming distrusted. This isn’t a possibility for our new Sunlight-based logs: they always completely incorporate newly-submitted certificates before returning an SCT to the submitter, so the effective merge delay is zero! Of course, any log can suffer outages for a variety of reasons, but this feature of Sunlight makes it less likely that any outages will be fatal to a log’s continued operation.

We’ve demonstrated that Sunlight and the Static CT API work in practice, and this demonstration has helped to confirm the browser developers’ hope that Static CT API logs can become an officially-supported part of CT. As a result, the major browsers that enforce CT have now permitted Static CT API logs to apply for inclusion in browsers as publicly-trusted logs, and we’re preparing to apply for this status for our Willow and Sycamore logs with the Chrome and Safari CT log programs.

Let’s Encrypt will run at least these two logs, and possibly others over time, for the foreseeable future. Once they’re trusted by browsers, we’ll encourage other CAs to submit to them as well, and we’ll begin including SCTs from these logs in our own certificates (alongside SCTs from traditional CT logs).

How to participate

The new Static CT API and the rollout of tile-based logs will bring various changes and opportunities for community members.

New Certificate Transparency log operators

Companies and non-profit organizations could help support the web PKI by running a CT log and applying for it to be publicly trusted. Implementations like Sunlight will have substantially lower resource requirements than first-generation CT logs, particularly when cached behind a CDN. The biggest resource demands for a log operator will be storage and upstream bandwidth. A publicly-trusted log is also expected to maintain relatively high availability, because CAs need logs to be available in order to continue issuing certificates.

We don’t have statistics to share about the exact resource requirements for such a log yet, but after we have practical experience running a fully publicly-trusted Sunlight log, we should be able to make this more concrete. As noted above, the compute side of the log can be handled by a single server. Sunlight author Filippo Valsorda has recently started running a Sunlight log—also on just a single server—and offered more detailed cost breakdowns for that log’s setup, with an estimated total cost around $10,000 per year. The costs for our production Static CT API logs may be higher than those for Filippo’s log, but still far less than the costs for our traditional RFC 6962 logs.

As with trust decisions about CAs, browser developers are the authorities about which CT logs become publicly trusted. Although any person or organization can run a log, browser developers will generally prefer to trust logs whose continued availability they’re confident of—typically those run by stable organizations with experience running some form of public Internet services. Unlike becoming a certificate authority, running a log does not require a formal audit, as the validation of the log’s availability and correctness can be performed purely by observation.

Certificate authorities

Once the Willow and Sycamore logs are trusted by browsers, our fellow certificate authorities can choose to start logging certificates to them as part of their issuance processes. (Initially, you should still include at least one SCT from a traditional CT log in each certificate.) The details, including the log API endpoints and keys, are available at our CT log page. You can start submitting to these logs right away if you prefer; just bear in mind that the SCTs they return aren’t useful to subscribers yet, and won’t be useful until browsers are updated to trust the new logs.

CT data users

You can monitor CT in order to watch for certificate issuances for your own domain names, or as part of monitoring or security products or services, or for Internet security research purposes. Many of our colleagues have been doing this for some time as a part of various tools they maintain. The Static CT API should make this easier, because you’ll be able to download and share log tiles as sets of ordinary files.

If you already run such monitoring tools, please note that you’ll need to update your data pipeline in order to access Static CT API logs; since the read API is not backwards-compatible, CT API clients will need to be modified to support the new API. Without updated tools, your view of the CT system will become partial!

Also note that getting a complete view of all of CT will still require downloading data from traditional logs, which will probably continue to be true for several years.

Software developers

As logs based on the new API enter production use, it will be important to have tools to interact with and search these logs. We can all benefit from more software that understands how to do this. Since file downloads are such a familiar piece of software functionality, it will probably be easier for developers to develop against the new API compared to the original one.

We’ve also continued to see greater integration of transparency logging tools into other kinds of services, such as software updates. There’s a growing transparency log ecosystem that’s always in need of more tools and integrations. As we mentioned above, transparency logs are increasingly learning from one another, and there are also mechanisms for more direct integration between different kinds of transparency logs (known as “witnessing”). Software developers can help improve different aspects of Internet security by contributing to this active and growing area.

Conclusion

The Certificate Transparency community and larger transparency logging community have experienced a virtuous cycle of innovation, sharing ideas and implementation code between different systems and demonstrating the feasibility of new mechanisms and functionality. With the advent of tile-based logging in CT, the state of the art has moved forward in a way that helps log operators run our logs much more efficiently without compromising security.

We’re proud to have participated in this experiment and the engineering conversation around the evolution of logging architectures. Now that we’ve shown how well the new API really works at scale, we look forward to having publicly-trusted Sunlight logs later this year!

Every night, right around midnight (mainly UTC), a horde of zombies wakes up and clamors for … digital certificates!

The zombies in question are abandoned or misconfigured Internet servers and ACME clients that have been set to request certificates from Let’s Encrypt. As our certificates last for at most 90 days, these zombie clients’ software knows that their certificates are out-of-date and need to be replaced. What they don’t realize is that their quest for new certificates is doomed! These devices are cursed to seek certificates again and again, never receiving them.

But they do use up a lot of certificate authority resources in the process.

The Zombie Client Problem

Unlike a human being, software doesn’t give up in frustration, or try to modify its approach, when it repeatedly fails at the same task. Our emphasis on automation means that the vast majority of Let’s Encrypt certificate renewals are performed by automated software. This is great when those renewals succeed, but it also means that forgotten clients and devices can continue requesting renewals unsuccessfully for months, or even years.

How might that happen? Most often, it happens when a device no longer has a domain name pointed to it. The device itself doesn’t know that this has changed, so it treats renewal failures as transient even though they are actually permanent. For instance:

• An organization may have allowed a domain name registration to lapse because it is no longer needed, but its servers are still configured to request certs for it.
• Or, a home user stopped using a particular dynamic-DNS domain with a network-attached storage device, but is still using that device at home. The device doesn’t realize that the user no longer expects to use the name, so it keeps requesting certs for it.
• Or, a web hosting or CDN customer migrated to a different service provider, but never informed the old service provider. The old service provider’s servers keep requesting certs unsuccessfully. If the customer was in a free service tier, there might not be invoices or charges reminding the customer to cancel the service.
• Or any number of other, subtler changes in a subscriber’s infrastructure, such as changing a firewall rule or some webserver configuration.

At the scale of Let’s Encrypt, which now covers hundreds of millions of names, scenarios like these have become common, and their impact has become substantial. In 2024, we noticed that about half of all certificate requests to the Let’s Encrypt ACME API came from about a million accounts that never successfully complete any validations. Many of these had completed validations and issued certificates sometime in the past, but nowadays every single one of their validation attempts fails, and they show no signs that this will change anytime soon.

Unfortunately, trying to validate those futile requests still uses resources. Our CA software has to generate challenges, reach out and attempt to validate them over the Internet, detect and report failures, and record all of the associated information in our databases and audit logs. And over time, we’ve seen more and more recurring failures: accounts that always fail their issuance requests have been growing at around 18% per year.

In January, we mentioned that we had been addressing the zombie client problem through our rate limit system. This post provides more detail on that progress. 

Our Rate Limit Philosophy

If you’ve used Let’s Encrypt as a subscriber, you may have run into one of our rate limits at some point, maybe during your initial setup process. We have eight different kinds of rate limits in place now; as our January post describes, they’ve become more algorithmically sophisticated and grown to address a wider range of problems. A key principle for Let’s Encrypt is that our rate limiting is not a punishment. We don’t think of rate limits as a way of retaliating against a client for misbehavior. Rate limits are simply a tool to maximize the efficient use of our limited resources and prevent people and programs from using up those resources for no constructive purpose.

We’ve consistently tried to design our rate limit mechanisms in line with that philosophy. So if a misconfiguration or misunderstanding has caused excessive requests in the past, we’re still happy to welcome the user in question back and start issuing them certificates again—once the problem has been addressed. We want the rate limits to put a brake on wasteful use of our systems, but not to frustrate users who are actively trying to make Let’s Encrypt work for them.

In addition, we’ve always implemented our rate limits to err on the side of permissiveness. For example, if the Redis instances where rate limits are tracked have an outage or lose data, the system is designed to permit more issuance rather than less issuance as a result.

We wanted to create additional limits that would target zombie clients, but in a correspondingly non-punitive way that would avoid any disruption to valid issuance, and welcome subscribers back quickly if they happened to notice and fix a long-time problem with their setups.

Our Zombie-Related Rate Limits and Their Impact

In planning a new zombie-specific response, we decided on a “pausing” approach, which can temporarily limit an account’s ability to proceed with certificate requests. The core idea is that, if a particular account consistently fails to complete validation for a particular hostname, we’ll pause that account-hostname pair. The pause means that any new order requests from that account for that hostname will be rejected immediately, before we get to the resource-intensive validation phase.

This approach is more finely targeted than pausing an entire account. Pausing account-hostname pairs means that your ability to issue certs for a specific name could be paused due to repeated failures, but you can still get all of your other certs like normal. So a large hosting provider doesn’t have to fear that its certificate issuance on behalf of one customer will be affected by renewal failures related to a problem with a different customer’s domain name. The account-specificity of the pause, in turn, means that validation failures from one subscriber or device won’t prevent a different subscriber or device from attempting to validate the same name, as long as the devices in question don’t share a single Let’s Encrypt account.

In September 2024, we began applying our zombie rate limits manually by pausing about 21,000 of the most recurrently-failing account-hostname pairs, those which were consistently repeating the same failed requests many times per day, every day. After implementing that first round of pauses, we immediately saw a significant impact on our failed request rates. As we announced at that time, we also began using a formula to automatically pause other zombie client account-hostname pairs from December 2024 onward. The associated new rate limit is called “Consecutive Authorization Failures per Hostname Per Account” (and is independent of the existing “Authorization Failures per Hostname Per Account” limit, which resets every hour).

This formula relates to the frequency of successive failed issuance requests for the same domain name by the same Let’s Encrypt account. It applies only to failures that happen again and again, with no successful issuances at all in between: a single successful validation immediately resets the rate limit all the way to zero. Like all of our rate limits, this is not a punitive measure but is simply intended to reduce the waste of resources. So, we decided to set the thresholds rather high in the expectation that we would catch only the most disruptive zombie clients, and ultimately only those clients that were extremely unlikely to succeed in the future based on their substantial history of failed requests. We don’t hurry to block requesters as zombies: according to our current formula, client software following the default established by EFF’s Certbot (two renewal attempts per day) would be paused as a zombie only after about ten years of constant failures. More aggressive failed issuance attempts will get a client paused sooner, but clients will generally have to fail hundreds or thousands of attempts in a row before they are paused.

Most subscribers using mainstream client applications with default configurations will never encounter this rate limit, even if they forget to deactivate renewal attempts for domains that are no longer pointed at their servers. As described below, our current limit is already providing noticeable benefits with minimal disruption, and we’re likely to tighten it a bit in the near future, so it will trigger after somewhat fewer consecutive failures.

Self-Service Unpausing

A key feature in our zombie issuance pausing mechanism is self-service unpausing. Whenever an account-hostname pair is paused, any new certificate requests for that hostname submitted by that account are immediately rejected. But this means that the “one successful validation immediately resets the rate limit counter” feature can no longer come into effect: once they’re paused, they can’t even attempt validation anymore.

So every rejection comes with an error message explaining what has happened and a custom link that can be used to immediately unpause that account-hostname pair and remove any other pauses on the same account at the same time. The point of this is that subscribers who notice at some point that issuance is failing and want to intervene to get it working again have a straightforward option to let Let’s Encrypt know that they’re aware of the recurring failures and are still planning to use a particular account. As soon as subscribers notify us via the self-service link, they’ll be able to issue certificates again.

Currently, the user interface for an affected subscriber looks like this:

This link would be provided via an ACME error message in response to any request that was blocked due to a pause account-hostname pair.

As it’s turned out, the unpause option shown above has only been used by about 3% of affected accounts! This goes to show that most of the zombies we’ve paused were, in fact, well and truly forgotten about.

However, the unpause feature is there for whenever it’s needed, and there may be cases when it will become more important. A very large integration could trigger the zombie-related rate limits if a newly-introduced software bug causes what looks like a very high volume of zombie requests in a very short time. In that case, once that bug has been noticed and fixed, an integrator may need to unpause its issuance on behalf of lots of customers at once. Our unpause feature permits unpausing 50,000 domain names on a single account at a time, so even the largest integrators can get themselves unpaused expeditiously in this situation.

Conclusion

We’ve been very happy with the results of our zombie mitigation measures, and, as far as we can tell, there’s been almost no impact for subscribers! Our statistics indicate that we’ve managed to reduce the load on our infrastructure while causing no detectable harm or inconvenience to subscribers’ valid issuance requests.

Since implementing the manual pauses in September and the automated pauses in December, we’ve seen:

• Over 100,000 account-hostname pairs have been paused for excessive failures.
• We received zero (!) associated complaints or support requests.
• About 3,200 people manually unpaused issuance.
• Failed certificate orders fell by about 30% so far, and should continue to fall over time as we fine-tune the rate limit formula and catch more zombie clients.

The new rate limit and the self-service unpause system are also ready to deal with circumstances that might produce more zombie clients in the future. For instance, we’ve announced that we’re going to be discontinuing renewal reminder emails soon. If some subscribers overlook failed renewals in the future, we might see more paused clients that result from unintentional renewal failures. We think taking advantage of the existing self-service unpause feature will be straightforward in that case. But it’s much better to notice problems and get them fixed up front, so please remember to set up your own monitoring to avoid unnoticed renewal failures in the future.

If you’re a subscriber who’s had occasion to use the self-service unpause feature, we’d love your feedback on the Community Forum about your experience using the feature and the circumstances that surrounded your account’s getting paused.

Also, if you’re a Let’s Encrypt client developer, please remember to make renewal requests at a random time (not precisely at midnight) so that the load on our infrastructure is smoothed out. You can also reduce the impact of zombie renewals by repeating failed requests somewhat less frequently over time (a “back-off” strategy), especially if the failure reason makes it look like a domain name may no longer be in use at all.

At Let’s Encrypt we know that building a secure Internet isn’t just a technical challenge—it’s a long-term commitment. Over the past decade we’ve made enormous strides: from issuing billions of TLS certificates to continually innovating to keep the web safer and more accessible. But none of this would be possible without recurring donations from individuals and organizations around the world.

Recurring donations are more than just financial support; they allow us to plan, innovate, and keep improving with confidence, knowing that month after month, year after year, our supporters are there. This consistent backing empowers us to maintain a secure, privacy-respecting Internet for all.

Our tenth anniversary tagline, Encryption for Everybody, highlights this vision. It’s both a technical goal and a fundamental belief that secure communication should be available to everyone, everywhere.

When we asked our recurring donors why they give, their responses affirmed how essential this commitment is. One longtime supporter shared:

For some, it’s about paying it forward, helping future users benefit as they once did:

The next decade of Let’s Encrypt will likely be about maintaining that commitment to encryption for everybody. It’s about ensuring that our work remains reliable, accessible, and—most importantly—supported by people who believe in what we do. To everyone who’s been part of this journey, thank you. We couldn’t do it without you.

During Let’s Encrypt’s 10th Anniversary Year, we’re celebrating our community and reflecting on our journey. We’d be thrilled to hear from you. Connect with us on LinkedIn, our community forum, or email us at outreach@letsencrypt.org. Let’s keep building a secure Internet together!

Let’s Encrypt is a project of the nonprofit Internet Security Research Group, a 501(c)(3) nonprofit. To support our work, visit letsencrypt.org/donate.

Let’s Encrypt will no longer include the “TLS Client Authentication” Extended Key Usage (EKU) in our certificates beginning in 2026. Most users who use Let’s Encrypt to secure websites won’t be affected and won’t need to take any action. However, if you use Let’s Encrypt certificates as client certificates to authenticate to a server, this change may impact you.

To minimize disruption, Let’s Encrypt will roll this change out in multiple stages, using ACME Profiles:

• Today: Let’s Encrypt already excludes the Client Authentication EKU on our tlsserver ACME profile. You can verify compatibility by issuing certificates with this profile now.
• October 1, 2025: Let’s Encrypt will launch a new tlsclient ACME profile which will retain the TLS Client Authentication EKU. Users who need additional time to migrate can opt-in to this profile.
• February 11, 2026: the default classic ACME profile will no longer contain the Client Authentication EKU.
• May 13, 2026: the tlsclient ACME profile will no longer be available and no further certificates with the Client Authentication EKU will be issued.

Once this is completed, Let’s Encrypt will switch to issuing with new intermediate Certificate Authorities which also do not contain the TLS Client Authentication EKU.

For some background information, all certificates include a list of intended uses, known as Extended Key Usages (EKU). Let’s Encrypt certificates have included two EKUs: TLS Server Authentication and TLS Client Authentication.

• TLS Server Authentication is used to authenticate connections to TLS Servers, like websites.
• TLS Client Authentication is used by clients to authenticate themselves to a server. This feature is not typically used on the web, and is not required on the certificates used on a website.

After this change is complete, only TLS Server Authentication will be available from Let’s Encrypt.

This change is prompted by changes to Google Chrome’s root program requirements, which impose a June 2026 deadline to split TLS Client and Server Authentication into separate PKIs. Many uses of client authentication are better served by a private certificate authority, and so Let’s Encrypt is discontinuing support for TLS Client Authentication ahead of this deadline.

How Pebble Supports ACME Client Developers

Together with the IETF community, we created the ACME standard to support completely automated certificate issuance. This open standard is now supported by dozens of clients. On the server side, did you know that we have not one but two open-source ACME server implementations?

The big implementation, which we use ourselves in production, is called Boulder. Boulder handles all of the facets and details needed for a production certificate authority, including policy compliance, database interfaces, challenge verifications, and logging. You can adapt and use Boulder yourself if you need to run a real certificate authority, including an internal, non-publicly-trusted ACME certificate authority within an organization.

The small implementation is called Pebble. It’s meant entirely for testing, not for use as a real certificate authority, and we and ACME client developers use it for various automated and manual testing purposes. For example, Certbot has used Pebble in its development process for years in order to perform a series of basic but realistic checks of the ability to request and obtain certificates from an ACME server.

Pebble is Easy to Use for ACME Client Testing

For any developer or team creating an ACME client application, Pebble solves a range of problems along the lines of “how do I check whether I’ve implemented ACME correctly, so that I could actually get certificates from a CA, without necessarily using a real domain name, and without running into CA rate limits during my routine testing?” Pebble is quick and easy to set up if you need to test an ACME client’s functionality.

It runs in RAM without dependencies or persistence; you won’t need to set up a database or a configuration for it. You can get Pebble running with a single golang command in just a few seconds, and immediately start making local ACME requests. That’s suitable for inclusion in a client’s integration test suite, making much more realistic integration tests possible without needing to worry about real domains, CA rate limits, or network outages.

We see Pebble getting used in the official test suites for ACME clients including getssl, Lego, Certbot, simp_le, and others. In many cases, every change committed to the ACME client’s code base is automatically tested against Pebble.

Pebble is Intentionally Different From Boulder

Pebble is also deliberately different from Boulder in some places in order to provide clients with an opportunity to interoperate with slightly different ACME implementations. The Pebble code explains that

For instance, the Let’s Encrypt service currently offers its newAccount resource at the path /acme/new-acct, whereas Pebble uses a different name /sign-me-up, so clients will be reminded to check the directory rather than assuming a specific path. Other substantive differences include:

• Pebble rejects 5% of all requests as having a invalid nonce, even if the nonce was otherwise valid, so clients can test how they respond this error condition
• Pebble only reuses valid authorizations 50% of the time, so clients can check their ability to perform validations when they might not have expected to
• Pebble truncates timestamps to a different degree of precision than Boulder
• Unlike Boulder, Pebble respects the notBefore and notAfter fields of new-order requests

The ability of ACME clients to work with both versions is a good test of their conformance to the ACME specification, rather than making assumptions about the current behavior of the Let’s Encrypt service in particular. This helps ensure that clients will work properly with other ACME CAs, and also with future versions of Let’s Encrypt’s own API.

Pebble is Useful to Both Let’s Encrypt and Client Developers as ACME Evolves

We often test out new ACME features by implementing them, at least in a simplified form, in Pebble before Boulder. This lets us and client developers experiment with support for those features even before they get rolled out in our staging service. We can do this quickly because a Pebble feature implementation doesn’t have to work with a full-scale CA backend.

We continue to encourage ACME client developers to use a copy of Pebble to test their clients’ functionality and ACME interoperability. It’s convenient and it’s likely to increase the correctness and robustness of their client applications.

Try Out Pebble Yourself

Want to try Pebble with your ACME client right now? On a Unix-like system, you can run

git clone https://github.com/letsencrypt/pebble/
cd pebble
go run ./cmd/pebble

Wait a few seconds; now you have a working ACME CA directory available at https://localhost:14000/dir! Your local ACME Server can immediately receive requests and issue certificates, though not publicly-trusted ones, of course. (If you prefer, we also offer other options for installing Pebble, like a Docker image.)

We welcome code contributions to Pebble. For example, ACME client developers may want to add simple versions of an ACME feature that’s not currently tested in Pebble in order to make their test suites more comprehensive. Also, if you notice a possibly unintended divergence between Pebble and Boulder or Pebble and the ACME specification, we’d love for you to let us know.

As we touched on in our first blog post highlighting ten years of Let’s Encrypt: Just as remarkable to us as the technical innovations behind proliferating TLS at scale is, so too is the sustained generosity we have benefited from throughout our first decade.

With that sense of gratitude top of mind, we are proud to announce a contribution of $1,000,000 from Jeff Atwood. Jeff has been a longtime supporter of our work, beginning many years ago with Discourse providing our community forum pro bono; something Discourse still provides to this day. As best we can tell, our forum has helped hundreds of thousands of people get up and running with Let’s Encrypt—an impact that has helped billions of people use an Internet that’s more secure and privacy-respecting thanks to widely adopted TLS.

When we first spoke with Jeff about the road ahead for Let’s Encrypt back in 2023, we knew a few things wouldn’t change no matter how the Internet changes over the next decade:

1. Free TLS is the only way to ensure it is and remains accessible to as many people as possible.
2. Let’s Encrypt is here to provide a reliable, trusted, and sound service no matter the scale.
3. Generosity from our global community of supporters will be how we sustain our work.

We’re proud that Jeff not only agrees, but has chosen to support us in such a meaningful way. In discussing how Jeff might want us to best celebrate his generosity and recognize his commitment to our work, he shared:

Indeed, this contribution is significant because of its scale, but more importantly because of its signal: a signal that supporting the not-so-glamorous but oh-so-nerdy work of encryption at scale matters to the lives of billions of people every day; a signal that supporting free privacy and security afforded by TLS for all of the Internet’s five billion users just makes sense.

Ten years ago we set out to build a better Internet through easy to use TLS. If you or your organization have supported us throughout the years, thank you for joining Jeff in believing in the work of Let’s Encrypt. For a deeper dive into the impact of Let’s Encrypt and ISRG’s other projects, take a look at our most recent annual report.

Let’s Encrypt is a project of the nonprofit Internet Security Research Group, a 501(c)(3) nonprofit committed to protecting Internet users by lowering monetary, technological, and informational barriers to a more secure and privacy-respecting Internet. For more, visit abetterinternet.org. Press inquiries can be sent to press@abetterinternet.org

Tue, 18 Mar 2025 00:00:00 +0000

---

We Issued Our First Six Day Cert

Since Let’s Encrypt started issuing certificates in 2015, people have repeatedly requested the ability to get certificates for IP addresses, an option that only a few certificate authorities have offered. Until now, they’ve had to look elsewhere, because we haven’t provided that feature.

Today, we’ve issued our first certificate for an IP address, as we announced we would in January. As with other new certificate features on our engineering roadmap, we’ll now start gradually rolling out this option to more and more of our subscribers.

Some Background on IP Address Certs

IP addresses are the underlying numerical addresses used on the Internet. Every device on the Internet has one (though, in modern practice, it might be shared with other devices, like when an entire home network shares a single public IP address). The Internet infrastructure uses them to route communications to their proper destination. IP addresses come in two forms, IPv4 and IPv6, and generally look like 54.215.62.21 (IPv4) or 2600:1f1c:446:4900::65 (IPv6).

Most Internet users rarely see or refer to IP addresses directly. Instead, they almost always use domain names like letsencrypt.org to refer to Internet services. The domain name system (DNS) is a part of the Internet infrastructure that’s responsible for allowing software to find the IP addresses associated with a particular domain name. For instance, your web browser can use DNS to find out that the service https://letsencrypt.org/ (Let’s Encrypt’s own website) is provided from the IP addresses 54.215.62.21 and 2600:1f1c:446:4900::65, among several others. This probably happened behind the scenes before you started reading this article! Your web browser needed to know our IP address in order to actually connect to our site and fetch this article.

Because we overwhelmingly tend to think and talk about Internet services in terms of domain names, those are the identifiers that are normally listed in certificates like those that Let’s Encrypt provides to our subscribers. Since you know us as “letsencrypt.org” and not as, say, “54.215.62.21,” it makes the most sense for our domain name to be on our certificate. After all, that’s what you’ll want your web browser to check against. This also gives Internet services more flexibility to be hosted in multiple locations, or to change where they’re hosted, without necessarily needing separate certificates for each server.

In principle, there’s no reason that a certificate couldn’t be issued for an IP address rather than a domain name, and in fact the technical and policy standards for certificates have always allowed this, with a handful of certificate authorities offering this service on a small scale. In Let’s Encrypt’s case, we’ve preferred to wait until some other pieces, like short-lived certs, were in place before we made this option available for our subscribers.

Why IP Address Certs Are Less Common

First and foremost, it’s because Internet users usually know services by domain names, not by IP addresses, and because IP addresses can easily change “behind the scenes” with no prior notice. For instance, a popular site could switch from one cloud hosting company to a different one, and update its DNS records to point at the new host. Most users wouldn’t ever notice the change at all, even though the site’s underlying IP addresses would be completely different.

Second, because IP addresses can change so easily, the sense of “ownership” one might have for them—or that a certificate authority might be able to attest to—tends to be weaker than for a domain name. If you’re hosting something in your house on a residential broadband connection, your Internet service provider most likely doesn’t guarantee that your IP address will stay the same over time. (That is, most home Internet users have a “dynamic IP address” from their ISPs, rather than a “static IP address.”) In that case, you have to contend with the possibility that that address may change often, possibly without warning, and that your old address may be assigned to somebody else.

Third, most Internet service operators don’t expect that end users will ever intentionally connect to their sites directly by IP address. In some cases, when an IP address is shared by different websites or different devices, connecting by IP address alone wouldn’t even work properly. In that case, there’s not much benefit to obtaining a certificate for the IP address!

How Let’s Encrypt Subscribers May Use IP Address Certs

Most current subscribers should be fine with their existing domain name certs and won’t need IP address certs. Subscribers who have a use for an IP address cert are typically already aware of that. A few use cases that we’re aware of include:

• A default page for hosting providers, in case someone pastes a server’s IP address into a browser instead of an individual site name (right now, this normally produces an error in the browser).

• A way to access your website if you don’t have a domain name at all (at some cost in reliability and convenience compared to getting a domain name).

• Securing DNS over HTTPS (DoH) or other infrastructure services. Having a certificate makes it much easier for DoH servers to prove their identities to clients. That could make it more feasible for DoH users or clients to enforce a requirement for a valid publicly-trusted certificate when connecting to DoH servers.

• Securing remote access to some home devices (like network-attached storage servers and Internet-of-things devices) even without a domain name.

• Securing ephemeral connections within cloud hosting infrastructure, like connections between one back-end cloud server and another, or ephemeral connections to administer new or short-lived back-end servers via HTTPS—as long as those servers have at least one public IP address available.

How To Get an IP Address Cert

IP address certificates are available right now in Staging. They should be generally available in Prod later in 2025, at the same time that short-lived certificates become generally available. Prior to general availability we may allow list issuance for a limited number of partners who can provide us with feedback.

Many Let’s Encrypt client applications should already be able to request certificates for IP addresses, although there can be minor technical changes required to support this in some client software.

As a matter of policy, Let’s Encrypt certificates that cover IP addresses must be short-lived certs, valid for only about six days. As such, your ACME client must support the draft ACME Profiles specification, and you must configure it to request the shortlived profile. And, probably not surprisingly, you can’t use the DNS challenge method to prove your control over an IP address; only the http-01 and tls-alpn-01 methods can be used.

If your client software requests an IP address cert with details that aren’t compatible with these policies, the order will be rejected by the ACME server. In this case, your client application may need to be updated or reconfigured. Feel free to ask for help on the Let’s Encrypt community forum if you encounter any problems, either as a client application developer or as an end user.

Since its inception, Let’s Encrypt has been sending expiration notification emails to subscribers that have provided an email address to us via the ACME API. This service ended on June 4, 2025. The decision to end the service is the result of the following factors:

1. Over the past 10 years more and more of our subscribers have been able to put reliable automation into place for certificate renewal.
2. Providing expiration notification emails means that we have to retain millions of email addresses connected to issuance records. As an organization that values privacy, removing this requirement is important to us.
3. Providing expiration notifications costs Let’s Encrypt tens of thousands of dollars per year, money that we believe can be better spent on other aspects of our infrastructure.
4. Providing expiration notifications adds complexity to our infrastructure, which takes time and attention to manage and increases the likelihood of mistakes being made. Over the long term, particularly as we add support for new service components, we need to manage overall complexity by phasing out system components that can no longer be justified.

For those who would like to continue receiving expiration notifications, we recommend using a third party service such as Red Sift Certificates Lite (formerly Hardenize). Red Sift’s monitoring service providing expiration emails is free of charge for up to 250 certificates. More monitoring options can be found here.

We have deleted the email addresses provided to Let’s Encrypt via the ACME API that were stored in our CA database in association with issuance data. This doesn’t affect addresses signed up to mailing lists and other systems. They are managed in a separate ISRG system unassociated with issuance data.

Going forward, if an email address is provided to Let’s Encrypt via the ACME API, Let’s Encrypt will not store the address but will instead forward it to the general ISRG mailing list system unassociated with any account data. If the email address has not been seen before, that system may send an onboarding email with information about how to subscribe to various sources of updates.

If you’d like to stay informed about technical updates and other news about Let’s Encrypt and our parent nonprofit, ISRG, based on the preferences you choose, you can sign up for our email lists below:

The Certificate Transparency ecosystem has been improving transparency for the web PKI since 2013. It helps make clear exactly what certificates each certificate authority has issued and makes sure errors or compromises of certificate authorities are detectable.

Let’s Encrypt participates in CT both as a certificate issuer and as a log operator. For the past year, we’ve also been running an experiment to help validate a next-generation design for Certificate Transparency logs. That experiment is now nearing a successful conclusion. We’ve demonstrated that the new architecture (called the “Static CT API”) works well, providing greater efficiency and making it easier to run huge and reliable CT log services with comparatively modest resources. The Static CT API also makes it easier to download and share data from CT logs.

The Sunlight log implementation, alongside other Static CT API log implementations, is now on a path to production use. Browsers are now officially accepting Static CT API logs into their log programs as a means to help guarantee that the contents of CA-issued certificates are all publicly disclosed and publicly accessible (see Safari’s and Chrome’s recent announcements), although the browsers also require the continued use of a traditional RFC 6962 log alongside the new type.

All of this is good news for everyone who runs, submits certificates to, or monitors a CT log: as the new architecture gets adopted, we can expect to see more organizations running more logs, at lower cost, and with greater overall capacity to keep up with the large volume of publicly-trusted certificates.

Certificate Transparency

Certificate Transparency (CT) was introduced in 2013 in response to concerns about how Internet users could detect misbehavior and compromise of certificate authorities. Prior to CT, it was possible for a CA to issue an inaccurate or malicious certificate that could be used to attack a relatively small number of users, and that might never come to wider attention. A team led by Google responded to this by creating a transparency log mechanism, where certificate authorities (like Let’s Encrypt) must disclose all of the certificates that we issue by submitting them to public log services. Web browsers now generally reject certificates unless the certificates include cryptographic proof (“Signed Certificate Timestamps”, or SCTs) demonstrating that they were submitted to and accepted by such logs.

The CT logs themselves use a cryptographic append-only ledger to prove that they haven’t deleted or modified their records. There are currently over a dozen CT log services, most of them also run by certificate authorities, including Let’s Encrypt’s own Oak log.

The Static CT API

The original 2013 CT log design has been used with relatively few technical changes since it was first introduced, but several other transparency logging systems have been created in other areas, such as sumdb for Golang, which helps ensure that the contents of Golang package updates are publicly recorded. While they were originally inspired by CT, more-recently invented transparency logs have improved on its design.

The current major evolution of CT was led by Filippo Valsorda, a cryptographer with an interest in transparency log mechanisms, with help from others in the CT ecosystem. Portions of the new design are directly based on sumdb. In addition to designing the new architecture, Valsorda also wrote the implementation that we’ve been using, called Sunlight, with support from Let’s Encrypt. We’re excited to see that there are now at least three other compatible implementations: Google’s trillian-tessera, Cloudflare’s Azul, and an independent project called Itko.

The biggest change for the Static CT API is that logs are now represented, and downloaded by verifiers, as simple collections of flat files (called “tiles,” so some implementers have also been referring to these as “tiled logs” or “tlogs”). Anyone who wants to download log data can do so just by downloading these files. This is great for log operators because these simple file downloads can be distributed in various ways, including caching by a CDN, which was less practical and efficient for the classic CT API.

The new design is also simpler and more efficient from the log operator’s perspective, making it cheaper to run logs. As we said last year, this may enable us and other operators to increase reliability and availability by running several separate logs, likely with lower overall resource requirements than a single traditional log.

Our Sunlight experiment

For the past year, we’ve run three Sunlight logs, called Twig, Willow, and Sycamore. We’ve been logging all of our own issued certificates, which represent a majority of the total volume of all publicly-trusted certificates, into our Sunlight logs. Sunlight logged these certificates quickly and correctly on relatively modest server hardware. Notably, each log’s write side was handled comfortably by just a single server. We also achieved high availability for these log services throughout the course of this experiment. (Because our Sunlight logs are not yet trusted by web browsers, we didn’t include the SCT proofs that they returned to us in the actual certificates we gave out to our subscribers; those proofs wouldn’t have been of use to our subscribers yet and would just have taken up space.)

A potential failure mode of traditional CT logs is that they could be unacceptably slow in incorporating newly-submitted certificates (known as missing the maximum merge delay), which can result in a log becoming distrusted. This isn’t a possibility for our new Sunlight-based logs: they always completely incorporate newly-submitted certificates before returning an SCT to the submitter, so the effective merge delay is zero! Of course, any log can suffer outages for a variety of reasons, but this feature of Sunlight makes it less likely that any outages will be fatal to a log’s continued operation.

We’ve demonstrated that Sunlight and the Static CT API work in practice, and this demonstration has helped to confirm the browser developers’ hope that Static CT API logs can become an officially-supported part of CT. As a result, the major browsers that enforce CT have now permitted Static CT API logs to apply for inclusion in browsers as publicly-trusted logs, and we’re preparing to apply for this status for our Willow and Sycamore logs with the Chrome and Safari CT log programs.

Let’s Encrypt will run at least these two logs, and possibly others over time, for the foreseeable future. Once they’re trusted by browsers, we’ll encourage other CAs to submit to them as well, and we’ll begin including SCTs from these logs in our own certificates (alongside SCTs from traditional CT logs).

How to participate

The new Static CT API and the rollout of tile-based logs will bring various changes and opportunities for community members.

New Certificate Transparency log operators

Companies and non-profit organizations could help support the web PKI by running a CT log and applying for it to be publicly trusted. Implementations like Sunlight will have substantially lower resource requirements than first-generation CT logs, particularly when cached behind a CDN. The biggest resource demands for a log operator will be storage and upstream bandwidth. A publicly-trusted log is also expected to maintain relatively high availability, because CAs need logs to be available in order to continue issuing certificates.

We don’t have statistics to share about the exact resource requirements for such a log yet, but after we have practical experience running a fully publicly-trusted Sunlight log, we should be able to make this more concrete. As noted above, the compute side of the log can be handled by a single server. Sunlight author Filippo Valsorda has recently started running a Sunlight log—also on just a single server—and offered more detailed cost breakdowns for that log’s setup, with an estimated total cost around $10,000 per year. The costs for our production Static CT API logs may be higher than those for Filippo’s log, but still far less than the costs for our traditional RFC 6962 logs.

As with trust decisions about CAs, browser developers are the authorities about which CT logs become publicly trusted. Although any person or organization can run a log, browser developers will generally prefer to trust logs whose continued availability they’re confident of—typically those run by stable organizations with experience running some form of public Internet services. Unlike becoming a certificate authority, running a log does not require a formal audit, as the validation of the log’s availability and correctness can be performed purely by observation.

Certificate authorities

Once the Willow and Sycamore logs are trusted by browsers, our fellow certificate authorities can choose to start logging certificates to them as part of their issuance processes. (Initially, you should still include at least one SCT from a traditional CT log in each certificate.) The details, including the log API endpoints and keys, are available at our CT log page. You can start submitting to these logs right away if you prefer; just bear in mind that the SCTs they return aren’t useful to subscribers yet, and won’t be useful until browsers are updated to trust the new logs.

CT data users

You can monitor CT in order to watch for certificate issuances for your own domain names, or as part of monitoring or security products or services, or for Internet security research purposes. Many of our colleagues have been doing this for some time as a part of various tools they maintain. The Static CT API should make this easier, because you’ll be able to download and share log tiles as sets of ordinary files.

If you already run such monitoring tools, please note that you’ll need to update your data pipeline in order to access Static CT API logs; since the read API is not backwards-compatible, CT API clients will need to be modified to support the new API. Without updated tools, your view of the CT system will become partial!

Also note that getting a complete view of all of CT will still require downloading data from traditional logs, which will probably continue to be true for several years.

Software developers

As logs based on the new API enter production use, it will be important to have tools to interact with and search these logs. We can all benefit from more software that understands how to do this. Since file downloads are such a familiar piece of software functionality, it will probably be easier for developers to develop against the new API compared to the original one.

We’ve also continued to see greater integration of transparency logging tools into other kinds of services, such as software updates. There’s a growing transparency log ecosystem that’s always in need of more tools and integrations. As we mentioned above, transparency logs are increasingly learning from one another, and there are also mechanisms for more direct integration between different kinds of transparency logs (known as “witnessing”). Software developers can help improve different aspects of Internet security by contributing to this active and growing area.

Conclusion

The Certificate Transparency community and larger transparency logging community have experienced a virtuous cycle of innovation, sharing ideas and implementation code between different systems and demonstrating the feasibility of new mechanisms and functionality. With the advent of tile-based logging in CT, the state of the art has moved forward in a way that helps log operators run our logs much more efficiently without compromising security.

We’re proud to have participated in this experiment and the engineering conversation around the evolution of logging architectures. Now that we’ve shown how well the new API really works at scale, we look forward to having publicly-trusted Sunlight logs later this year!

Every night, right around midnight (mainly UTC), a horde of zombies wakes up and clamors for … digital certificates!

The zombies in question are abandoned or misconfigured Internet servers and ACME clients that have been set to request certificates from Let’s Encrypt. As our certificates last for at most 90 days, these zombie clients’ software knows that their certificates are out-of-date and need to be replaced. What they don’t realize is that their quest for new certificates is doomed! These devices are cursed to seek certificates again and again, never receiving them.

But they do use up a lot of certificate authority resources in the process.

The Zombie Client Problem

Unlike a human being, software doesn’t give up in frustration, or try to modify its approach, when it repeatedly fails at the same task. Our emphasis on automation means that the vast majority of Let’s Encrypt certificate renewals are performed by automated software. This is great when those renewals succeed, but it also means that forgotten clients and devices can continue requesting renewals unsuccessfully for months, or even years.

How might that happen? Most often, it happens when a device no longer has a domain name pointed to it. The device itself doesn’t know that this has changed, so it treats renewal failures as transient even though they are actually permanent. For instance:

• An organization may have allowed a domain name registration to lapse because it is no longer needed, but its servers are still configured to request certs for it.
• Or, a home user stopped using a particular dynamic-DNS domain with a network-attached storage device, but is still using that device at home. The device doesn’t realize that the user no longer expects to use the name, so it keeps requesting certs for it.
• Or, a web hosting or CDN customer migrated to a different service provider, but never informed the old service provider. The old service provider’s servers keep requesting certs unsuccessfully. If the customer was in a free service tier, there might not be invoices or charges reminding the customer to cancel the service.
• Or any number of other, subtler changes in a subscriber’s infrastructure, such as changing a firewall rule or some webserver configuration.

At the scale of Let’s Encrypt, which now covers hundreds of millions of names, scenarios like these have become common, and their impact has become substantial. In 2024, we noticed that about half of all certificate requests to the Let’s Encrypt ACME API came from about a million accounts that never successfully complete any validations. Many of these had completed validations and issued certificates sometime in the past, but nowadays every single one of their validation attempts fails, and they show no signs that this will change anytime soon.

Unfortunately, trying to validate those futile requests still uses resources. Our CA software has to generate challenges, reach out and attempt to validate them over the Internet, detect and report failures, and record all of the associated information in our databases and audit logs. And over time, we’ve seen more and more recurring failures: accounts that always fail their issuance requests have been growing at around 18% per year.

In January, we mentioned that we had been addressing the zombie client problem through our rate limit system. This post provides more detail on that progress. 

Our Rate Limit Philosophy

If you’ve used Let’s Encrypt as a subscriber, you may have run into one of our rate limits at some point, maybe during your initial setup process. We have eight different kinds of rate limits in place now; as our January post describes, they’ve become more algorithmically sophisticated and grown to address a wider range of problems. A key principle for Let’s Encrypt is that our rate limiting is not a punishment. We don’t think of rate limits as a way of retaliating against a client for misbehavior. Rate limits are simply a tool to maximize the efficient use of our limited resources and prevent people and programs from using up those resources for no constructive purpose.

We’ve consistently tried to design our rate limit mechanisms in line with that philosophy. So if a misconfiguration or misunderstanding has caused excessive requests in the past, we’re still happy to welcome the user in question back and start issuing them certificates again—once the problem has been addressed. We want the rate limits to put a brake on wasteful use of our systems, but not to frustrate users who are actively trying to make Let’s Encrypt work for them.

In addition, we’ve always implemented our rate limits to err on the side of permissiveness. For example, if the Redis instances where rate limits are tracked have an outage or lose data, the system is designed to permit more issuance rather than less issuance as a result.

We wanted to create additional limits that would target zombie clients, but in a correspondingly non-punitive way that would avoid any disruption to valid issuance, and welcome subscribers back quickly if they happened to notice and fix a long-time problem with their setups.

Our Zombie-Related Rate Limits and Their Impact

In planning a new zombie-specific response, we decided on a “pausing” approach, which can temporarily limit an account’s ability to proceed with certificate requests. The core idea is that, if a particular account consistently fails to complete validation for a particular hostname, we’ll pause that account-hostname pair. The pause means that any new order requests from that account for that hostname will be rejected immediately, before we get to the resource-intensive validation phase.

This approach is more finely targeted than pausing an entire account. Pausing account-hostname pairs means that your ability to issue certs for a specific name could be paused due to repeated failures, but you can still get all of your other certs like normal. So a large hosting provider doesn’t have to fear that its certificate issuance on behalf of one customer will be affected by renewal failures related to a problem with a different customer’s domain name. The account-specificity of the pause, in turn, means that validation failures from one subscriber or device won’t prevent a different subscriber or device from attempting to validate the same name, as long as the devices in question don’t share a single Let’s Encrypt account.

In September 2024, we began applying our zombie rate limits manually by pausing about 21,000 of the most recurrently-failing account-hostname pairs, those which were consistently repeating the same failed requests many times per day, every day. After implementing that first round of pauses, we immediately saw a significant impact on our failed request rates. As we announced at that time, we also began using a formula to automatically pause other zombie client account-hostname pairs from December 2024 onward. The associated new rate limit is called “Consecutive Authorization Failures per Hostname Per Account” (and is independent of the existing “Authorization Failures per Hostname Per Account” limit, which resets every hour).

This formula relates to the frequency of successive failed issuance requests for the same domain name by the same Let’s Encrypt account. It applies only to failures that happen again and again, with no successful issuances at all in between: a single successful validation immediately resets the rate limit all the way to zero. Like all of our rate limits, this is not a punitive measure but is simply intended to reduce the waste of resources. So, we decided to set the thresholds rather high in the expectation that we would catch only the most disruptive zombie clients, and ultimately only those clients that were extremely unlikely to succeed in the future based on their substantial history of failed requests. We don’t hurry to block requesters as zombies: according to our current formula, client software following the default established by EFF’s Certbot (two renewal attempts per day) would be paused as a zombie only after about ten years of constant failures. More aggressive failed issuance attempts will get a client paused sooner, but clients will generally have to fail hundreds or thousands of attempts in a row before they are paused.

Most subscribers using mainstream client applications with default configurations will never encounter this rate limit, even if they forget to deactivate renewal attempts for domains that are no longer pointed at their servers. As described below, our current limit is already providing noticeable benefits with minimal disruption, and we’re likely to tighten it a bit in the near future, so it will trigger after somewhat fewer consecutive failures.

Self-Service Unpausing

A key feature in our zombie issuance pausing mechanism is self-service unpausing. Whenever an account-hostname pair is paused, any new certificate requests for that hostname submitted by that account are immediately rejected. But this means that the “one successful validation immediately resets the rate limit counter” feature can no longer come into effect: once they’re paused, they can’t even attempt validation anymore.

So every rejection comes with an error message explaining what has happened and a custom link that can be used to immediately unpause that account-hostname pair and remove any other pauses on the same account at the same time. The point of this is that subscribers who notice at some point that issuance is failing and want to intervene to get it working again have a straightforward option to let Let’s Encrypt know that they’re aware of the recurring failures and are still planning to use a particular account. As soon as subscribers notify us via the self-service link, they’ll be able to issue certificates again.

Currently, the user interface for an affected subscriber looks like this:

This link would be provided via an ACME error message in response to any request that was blocked due to a pause account-hostname pair.

As it’s turned out, the unpause option shown above has only been used by about 3% of affected accounts! This goes to show that most of the zombies we’ve paused were, in fact, well and truly forgotten about.

However, the unpause feature is there for whenever it’s needed, and there may be cases when it will become more important. A very large integration could trigger the zombie-related rate limits if a newly-introduced software bug causes what looks like a very high volume of zombie requests in a very short time. In that case, once that bug has been noticed and fixed, an integrator may need to unpause its issuance on behalf of lots of customers at once. Our unpause feature permits unpausing 50,000 domain names on a single account at a time, so even the largest integrators can get themselves unpaused expeditiously in this situation.

Conclusion

We’ve been very happy with the results of our zombie mitigation measures, and, as far as we can tell, there’s been almost no impact for subscribers! Our statistics indicate that we’ve managed to reduce the load on our infrastructure while causing no detectable harm or inconvenience to subscribers’ valid issuance requests.

Since implementing the manual pauses in September and the automated pauses in December, we’ve seen:

• Over 100,000 account-hostname pairs have been paused for excessive failures.
• We received zero (!) associated complaints or support requests.
• About 3,200 people manually unpaused issuance.
• Failed certificate orders fell by about 30% so far, and should continue to fall over time as we fine-tune the rate limit formula and catch more zombie clients.

The new rate limit and the self-service unpause system are also ready to deal with circumstances that might produce more zombie clients in the future. For instance, we’ve announced that we’re going to be discontinuing renewal reminder emails soon. If some subscribers overlook failed renewals in the future, we might see more paused clients that result from unintentional renewal failures. We think taking advantage of the existing self-service unpause feature will be straightforward in that case. But it’s much better to notice problems and get them fixed up front, so please remember to set up your own monitoring to avoid unnoticed renewal failures in the future.

If you’re a subscriber who’s had occasion to use the self-service unpause feature, we’d love your feedback on the Community Forum about your experience using the feature and the circumstances that surrounded your account’s getting paused.

Also, if you’re a Let’s Encrypt client developer, please remember to make renewal requests at a random time (not precisely at midnight) so that the load on our infrastructure is smoothed out. You can also reduce the impact of zombie renewals by repeating failed requests somewhat less frequently over time (a “back-off” strategy), especially if the failure reason makes it look like a domain name may no longer be in use at all.

At Let’s Encrypt we know that building a secure Internet isn’t just a technical challenge—it’s a long-term commitment. Over the past decade we’ve made enormous strides: from issuing billions of TLS certificates to continually innovating to keep the web safer and more accessible. But none of this would be possible without recurring donations from individuals and organizations around the world.

Recurring donations are more than just financial support; they allow us to plan, innovate, and keep improving with confidence, knowing that month after month, year after year, our supporters are there. This consistent backing empowers us to maintain a secure, privacy-respecting Internet for all.

Our tenth anniversary tagline, Encryption for Everybody, highlights this vision. It’s both a technical goal and a fundamental belief that secure communication should be available to everyone, everywhere.

When we asked our recurring donors why they give, their responses affirmed how essential this commitment is. One longtime supporter shared:

For some, it’s about paying it forward, helping future users benefit as they once did:

The next decade of Let’s Encrypt will likely be about maintaining that commitment to encryption for everybody. It’s about ensuring that our work remains reliable, accessible, and—most importantly—supported by people who believe in what we do. To everyone who’s been part of this journey, thank you. We couldn’t do it without you.

During Let’s Encrypt’s 10th Anniversary Year, we’re celebrating our community and reflecting on our journey. We’d be thrilled to hear from you. Connect with us on LinkedIn, our community forum, or email us at outreach@letsencrypt.org. Let’s keep building a secure Internet together!

Let’s Encrypt is a project of the nonprofit Internet Security Research Group, a 501(c)(3) nonprofit. To support our work, visit letsencrypt.org/donate.

Let’s Encrypt will no longer include the “TLS Client Authentication” Extended Key Usage (EKU) in our certificates beginning in 2026. Most users who use Let’s Encrypt to secure websites won’t be affected and won’t need to take any action. However, if you use Let’s Encrypt certificates as client certificates to authenticate to a server, this change may impact you.

To minimize disruption, Let’s Encrypt will roll this change out in multiple stages, using ACME Profiles:

• Today: Let’s Encrypt already excludes the Client Authentication EKU on our tlsserver ACME profile. You can verify compatibility by issuing certificates with this profile now.
• October 1, 2025: Let’s Encrypt will launch a new tlsclient ACME profile which will retain the TLS Client Authentication EKU. Users who need additional time to migrate can opt-in to this profile.
• February 11, 2026: the default classic ACME profile will no longer contain the Client Authentication EKU.
• May 13, 2026: the tlsclient ACME profile will no longer be available and no further certificates with the Client Authentication EKU will be issued.

Once this is completed, Let’s Encrypt will switch to issuing with new intermediate Certificate Authorities which also do not contain the TLS Client Authentication EKU.

For some background information, all certificates include a list of intended uses, known as Extended Key Usages (EKU). Let’s Encrypt certificates have included two EKUs: TLS Server Authentication and TLS Client Authentication.

• TLS Server Authentication is used to authenticate connections to TLS Servers, like websites.
• TLS Client Authentication is used by clients to authenticate themselves to a server. This feature is not typically used on the web, and is not required on the certificates used on a website.

After this change is complete, only TLS Server Authentication will be available from Let’s Encrypt.

This change is prompted by changes to Google Chrome’s root program requirements, which impose a June 2026 deadline to split TLS Client and Server Authentication into separate PKIs. Many uses of client authentication are better served by a private certificate authority, and so Let’s Encrypt is discontinuing support for TLS Client Authentication ahead of this deadline.

How Pebble Supports ACME Client Developers

Together with the IETF community, we created the ACME standard to support completely automated certificate issuance. This open standard is now supported by dozens of clients. On the server side, did you know that we have not one but two open-source ACME server implementations?

The big implementation, which we use ourselves in production, is called Boulder. Boulder handles all of the facets and details needed for a production certificate authority, including policy compliance, database interfaces, challenge verifications, and logging. You can adapt and use Boulder yourself if you need to run a real certificate authority, including an internal, non-publicly-trusted ACME certificate authority within an organization.

The small implementation is called Pebble. It’s meant entirely for testing, not for use as a real certificate authority, and we and ACME client developers use it for various automated and manual testing purposes. For example, Certbot has used Pebble in its development process for years in order to perform a series of basic but realistic checks of the ability to request and obtain certificates from an ACME server.

Pebble is Easy to Use for ACME Client Testing

For any developer or team creating an ACME client application, Pebble solves a range of problems along the lines of “how do I check whether I’ve implemented ACME correctly, so that I could actually get certificates from a CA, without necessarily using a real domain name, and without running into CA rate limits during my routine testing?” Pebble is quick and easy to set up if you need to test an ACME client’s functionality.

It runs in RAM without dependencies or persistence; you won’t need to set up a database or a configuration for it. You can get Pebble running with a single golang command in just a few seconds, and immediately start making local ACME requests. That’s suitable for inclusion in a client’s integration test suite, making much more realistic integration tests possible without needing to worry about real domains, CA rate limits, or network outages.

We see Pebble getting used in the official test suites for ACME clients including getssl, Lego, Certbot, simp_le, and others. In many cases, every change committed to the ACME client’s code base is automatically tested against Pebble.

Pebble is Intentionally Different From Boulder

Pebble is also deliberately different from Boulder in some places in order to provide clients with an opportunity to interoperate with slightly different ACME implementations. The Pebble code explains that

For instance, the Let’s Encrypt service currently offers its newAccount resource at the path /acme/new-acct, whereas Pebble uses a different name /sign-me-up, so clients will be reminded to check the directory rather than assuming a specific path. Other substantive differences include:

• Pebble rejects 5% of all requests as having a invalid nonce, even if the nonce was otherwise valid, so clients can test how they respond this error condition
• Pebble only reuses valid authorizations 50% of the time, so clients can check their ability to perform validations when they might not have expected to
• Pebble truncates timestamps to a different degree of precision than Boulder
• Unlike Boulder, Pebble respects the notBefore and notAfter fields of new-order requests

The ability of ACME clients to work with both versions is a good test of their conformance to the ACME specification, rather than making assumptions about the current behavior of the Let’s Encrypt service in particular. This helps ensure that clients will work properly with other ACME CAs, and also with future versions of Let’s Encrypt’s own API.

Pebble is Useful to Both Let’s Encrypt and Client Developers as ACME Evolves

We often test out new ACME features by implementing them, at least in a simplified form, in Pebble before Boulder. This lets us and client developers experiment with support for those features even before they get rolled out in our staging service. We can do this quickly because a Pebble feature implementation doesn’t have to work with a full-scale CA backend.

We continue to encourage ACME client developers to use a copy of Pebble to test their clients’ functionality and ACME interoperability. It’s convenient and it’s likely to increase the correctness and robustness of their client applications.

Try Out Pebble Yourself

Want to try Pebble with your ACME client right now? On a Unix-like system, you can run

git clone https://github.com/letsencrypt/pebble/
cd pebble
go run ./cmd/pebble

Wait a few seconds; now you have a working ACME CA directory available at https://localhost:14000/dir! Your local ACME Server can immediately receive requests and issue certificates, though not publicly-trusted ones, of course. (If you prefer, we also offer other options for installing Pebble, like a Docker image.)

We welcome code contributions to Pebble. For example, ACME client developers may want to add simple versions of an ACME feature that’s not currently tested in Pebble in order to make their test suites more comprehensive. Also, if you notice a possibly unintended divergence between Pebble and Boulder or Pebble and the ACME specification, we’d love for you to let us know.

As we touched on in our first blog post highlighting ten years of Let’s Encrypt: Just as remarkable to us as the technical innovations behind proliferating TLS at scale is, so too is the sustained generosity we have benefited from throughout our first decade.

With that sense of gratitude top of mind, we are proud to announce a contribution of $1,000,000 from Jeff Atwood. Jeff has been a longtime supporter of our work, beginning many years ago with Discourse providing our community forum pro bono; something Discourse still provides to this day. As best we can tell, our forum has helped hundreds of thousands of people get up and running with Let’s Encrypt—an impact that has helped billions of people use an Internet that’s more secure and privacy-respecting thanks to widely adopted TLS.

When we first spoke with Jeff about the road ahead for Let’s Encrypt back in 2023, we knew a few things wouldn’t change no matter how the Internet changes over the next decade:

1. Free TLS is the only way to ensure it is and remains accessible to as many people as possible.
2. Let’s Encrypt is here to provide a reliable, trusted, and sound service no matter the scale.
3. Generosity from our global community of supporters will be how we sustain our work.

We’re proud that Jeff not only agrees, but has chosen to support us in such a meaningful way. In discussing how Jeff might want us to best celebrate his generosity and recognize his commitment to our work, he shared:

Indeed, this contribution is significant because of its scale, but more importantly because of its signal: a signal that supporting the not-so-glamorous but oh-so-nerdy work of encryption at scale matters to the lives of billions of people every day; a signal that supporting free privacy and security afforded by TLS for all of the Internet’s five billion users just makes sense.

Ten years ago we set out to build a better Internet through easy to use TLS. If you or your organization have supported us throughout the years, thank you for joining Jeff in believing in the work of Let’s Encrypt. For a deeper dive into the impact of Let’s Encrypt and ISRG’s other projects, take a look at our most recent annual report.

Let’s Encrypt is a project of the nonprofit Internet Security Research Group, a 501(c)(3) nonprofit committed to protecting Internet users by lowering monetary, technological, and informational barriers to a more secure and privacy-respecting Internet. For more, visit abetterinternet.org. Press inquiries can be sent to press@abetterinternet.org

Earlier this year we announced our intention to introduce short-lived certificates with lifetimes of six days as an option for our subscribers. Yesterday we issued our first short-lived certificate. You can see the certificate at the bottom of our post, or here thanks to Certificate Transparency logs. We issued it to ourselves and then immediately revoked it so we can observe the certificate’s whole lifecycle. This is the first step towards making short-lived certificates available to all subscribers.

The next step is for us to make short-lived certificates available to a small set of our subscribers so we can make sure our systems scale as expected prior to general availability. We expect this next phase to begin during Q2 of this year.

We expect short-lived certificates to be generally available by the end of this year.

How To Get Six-Day Certificates

Once short-lived certificates are an option for you, you’ll need to use an ACME client that supports ACME certificate profiles and select the short-lived certificate profile (“shortlived”). The lego client recently added this functionality.

In the meantime, the best way to prepare to take advantage of short-lived certificates is to make sure your ACME client is reliably renewing certificates in an automated fashion. If that’s working well then there should be no costs to switching to short-lived certificates.

You’ll also want to be sure your ACME client is running frequently - both for the sake of renewing short-lived certificates and so as to take advantage of ACME Renewal Information (ARI). ARI allows Let’s Encrypt to notify your client if it should renew early for some reason. ARI checks should happen at least once per day, and short-lived certificates should be renewed every two to three days, so we recommend having your client run at least once per day.

Shorter Certificate Lifetimes Are Good for Security

When the private key associated with a certificate is compromised, the recommendation has always been to have the certificate revoked so that people will know not to use it. Unfortunately, certificate revocation doesn’t work very well. This means that certificates with compromised keys (or other issues) may continue to be used until they expire. The longer the lifetime of the certificate, the longer the potential for use of a problematic certificate.

The primary advantage of short-lived certificates is that they greatly reduce the potential compromise window because they expire relatively quickly. This reduces the need for certificate revocation, which has historically been unreliable. Our six-day certificates will not include OCSP or CRL URLs. Additionally, short-lived certificates practically require automation, and we believe that automating certificate issuance is important for security.

Questions

If you have questions or comments about our plans, feel free to let us know on our community forums.

We’d like to thank Open Technology Fund for supporting this work.

Our First 6-Day Certificate

PEM format:

-----BEGIN CERTIFICATE-----
MIIDSzCCAtGgAwIBAgISA7CwFcGk4mQWEXMacRtxHeDvMAoGCCqGSM49BAMDMDIx
CzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQDEwJF
NjAeFw0yNTAyMTkxNzMwMDFaFw0yNTAyMjYwOTMwMDBaMAAwWTATBgcqhkjOPQIB
BggqhkjOPQMBBwNCAAQoSItt2V1aocI5dxrKR8iLfmm0KiVvOhiwKByzu2kLeC7C
0BdfAgtwdICdkuEhAXokhXLq6DNZZgmh5T4flVwZo4IB9zCCAfMwDgYDVR0PAQH/
BAQDAgeAMBMGA1UdJQQMMAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0j
BBgwFoAUkydGmAOpUWiOmNbEQkjbI79YlNIwVQYIKwYBBQUHAQEESTBHMCEGCCsG
AQUFBzABhhVodHRwOi8vZTYuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6
Ly9lNi5pLmxlbmNyLm9yZy8wKAYDVR0RAQH/BB4wHIIaaGVsbG93b3JsZC5sZXRz
ZW5jcnlwdC5vcmcwEwYDVR0gBAwwCjAIBgZngQwBAgEwggEFBgorBgEEAdZ5AgQC
BIH2BIHzAPEAdgDM+w9qhXEJZf6Vm1PO6bJ8IumFXA2XjbapflTA/kwNsAAAAZUf
d/zOAAAEAwBHMEUCIFNd51TfSNiJrO+294t49C5ANc4oC7gTUzf7xnlNlhKsAiEA
wi5hfiC9SsKLxlTQ0sctUxhLmdYh40r6ECWQS/yWw2AAdwDgkrP8DB3I52g2H95h
uZZNClJ4GYpy1nLEsE2lbW9UBAAAAZUfd/0TAAAEAwBIMEYCIQCs2NuZIUIloOaH
1t9eXDKb8bjoWESBPsK4i2BxMvEIswIhAOMNaQNyr1YkzrcNUz15qGV0oVLg5BJN
+ikWxXOdcRHFMAoGCCqGSM49BAMDA2gAMGUCMDANqy7G09AIwzXcd7SNl7uFwhC+
xlfduvp1PeEDHc/FA9K3mRYkGXuKtzNdOh7wcAIxALjEMDmBQiwXbB447oGkaZAe
0rqxA3EtNV5wj0obeObluj/NgUsVEG9OqiBIoggFRw==
-----END CERTIFICATE-----

openssl x509 -text output:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            03:b0:b0:15:c1:a4:e2:64:16:11:73:1a:71:1b:71:1d:e0:ef
        Signature Algorithm: ecdsa-with-SHA384
        Issuer: C=US, O=Let's Encrypt, CN=E6
        Validity
            Not Before: Feb 19 17:30:01 2025 GMT
            Not After : Feb 26 09:30:00 2025 GMT
        Subject:
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:28:48:8b:6d:d9:5d:5a:a1:c2:39:77:1a:ca:47:
                    c8:8b:7e:69:b4:2a:25:6f:3a:18:b0:28:1c:b3:bb:
                    69:0b:78:2e:c2:d0:17:5f:02:0b:70:74:80:9d:92:
                    e1:21:01:7a:24:85:72:ea:e8:33:59:66:09:a1:e5:
                    3e:1f:95:5c:19
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature
            X509v3 Extended Key Usage:
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Authority Key Identifier:
                93:27:46:98:03:A9:51:68:8E:98:D6:C4:42:48:DB:23:BF:58:94:D2
            Authority Information Access:
                OCSP - URI:http://e6.o.lencr.org
                CA Issuers - URI:http://e6.i.lencr.org/
            X509v3 Subject Alternative Name: critical
                DNS:helloworld.letsencrypt.org
            X509v3 Certificate Policies:
                Policy: 2.23.140.1.2.1
            CT Precertificate SCTs:
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : CC:FB:0F:6A:85:71:09:65:FE:95:9B:53:CE:E9:B2:7C:
                                22:E9:85:5C:0D:97:8D:B6:A9:7E:54:C0:FE:4C:0D:B0
                    Timestamp : Feb 19 18:28:32.078 2025 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:45:02:20:53:5D:E7:54:DF:48:D8:89:AC:EF:B6:F7:
                                8B:78:F4:2E:40:35:CE:28:0B:B8:13:53:37:FB:C6:79:
                                4D:96:12:AC:02:21:00:C2:2E:61:7E:20:BD:4A:C2:8B:
                                C6:54:D0:D2:C7:2D:53:18:4B:99:D6:21:E3:4A:FA:10:
                                25:90:4B:FC:96:C3:60
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : E0:92:B3:FC:0C:1D:C8:E7:68:36:1F:DE:61:B9:96:4D:
                                0A:52:78:19:8A:72:D6:72:C4:B0:4D:A5:6D:6F:54:04
                    Timestamp : Feb 19 18:28:32.147 2025 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:46:02:21:00:AC:D8:DB:99:21:42:25:A0:E6:87:D6:
                                DF:5E:5C:32:9B:F1:B8:E8:58:44:81:3E:C2:B8:8B:60:
                                71:32:F1:08:B3:02:21:00:E3:0D:69:03:72:AF:56:24:
                                CE:B7:0D:53:3D:79:A8:65:74:A1:52:E0:E4:12:4D:FA:
                                29:16:C5:73:9D:71:11:C5
    Signature Algorithm: ecdsa-with-SHA384
    Signature Value:
        30:65:02:30:30:0d:ab:2e:c6:d3:d0:08:c3:35:dc:77:b4:8d:
        97:bb:85:c2:10:be:c6:57:dd:ba:fa:75:3d:e1:03:1d:cf:c5:
        03:d2:b7:99:16:24:19:7b:8a:b7:33:5d:3a:1e:f0:70:02:31:
        00:b8:c4:30:39:81:42:2c:17:6c:1e:38:ee:81:a4:69:90:1e:
        d2:ba:b1:03:71:2d:35:5e:70:8f:4a:1b:78:e6:e5:ba:3f:cd:
        81:4b:15:10:6f:4e:aa:20:48:a2:08:05:47

Thu, 20 Feb 2025 00:00:00 +0000

---

Encryption for Everybody

Since Let’s Encrypt started issuing certificates in 2015, people have repeatedly requested the ability to get certificates for IP addresses, an option that only a few certificate authorities have offered. Until now, they’ve had to look elsewhere, because we haven’t provided that feature.

Today, we’ve issued our first certificate for an IP address, as we announced we would in January. As with other new certificate features on our engineering roadmap, we’ll now start gradually rolling out this option to more and more of our subscribers.

Some Background on IP Address Certs

IP addresses are the underlying numerical addresses used on the Internet. Every device on the Internet has one (though, in modern practice, it might be shared with other devices, like when an entire home network shares a single public IP address). The Internet infrastructure uses them to route communications to their proper destination. IP addresses come in two forms, IPv4 and IPv6, and generally look like 54.215.62.21 (IPv4) or 2600:1f1c:446:4900::65 (IPv6).

Most Internet users rarely see or refer to IP addresses directly. Instead, they almost always use domain names like letsencrypt.org to refer to Internet services. The domain name system (DNS) is a part of the Internet infrastructure that’s responsible for allowing software to find the IP addresses associated with a particular domain name. For instance, your web browser can use DNS to find out that the service https://letsencrypt.org/ (Let’s Encrypt’s own website) is provided from the IP addresses 54.215.62.21 and 2600:1f1c:446:4900::65, among several others. This probably happened behind the scenes before you started reading this article! Your web browser needed to know our IP address in order to actually connect to our site and fetch this article.

Because we overwhelmingly tend to think and talk about Internet services in terms of domain names, those are the identifiers that are normally listed in certificates like those that Let’s Encrypt provides to our subscribers. Since you know us as “letsencrypt.org” and not as, say, “54.215.62.21,” it makes the most sense for our domain name to be on our certificate. After all, that’s what you’ll want your web browser to check against. This also gives Internet services more flexibility to be hosted in multiple locations, or to change where they’re hosted, without necessarily needing separate certificates for each server.

In principle, there’s no reason that a certificate couldn’t be issued for an IP address rather than a domain name, and in fact the technical and policy standards for certificates have always allowed this, with a handful of certificate authorities offering this service on a small scale. In Let’s Encrypt’s case, we’ve preferred to wait until some other pieces, like short-lived certs, were in place before we made this option available for our subscribers.

Why IP Address Certs Are Less Common

First and foremost, it’s because Internet users usually know services by domain names, not by IP addresses, and because IP addresses can easily change “behind the scenes” with no prior notice. For instance, a popular site could switch from one cloud hosting company to a different one, and update its DNS records to point at the new host. Most users wouldn’t ever notice the change at all, even though the site’s underlying IP addresses would be completely different.

Second, because IP addresses can change so easily, the sense of “ownership” one might have for them—or that a certificate authority might be able to attest to—tends to be weaker than for a domain name. If you’re hosting something in your house on a residential broadband connection, your Internet service provider most likely doesn’t guarantee that your IP address will stay the same over time. (That is, most home Internet users have a “dynamic IP address” from their ISPs, rather than a “static IP address.”) In that case, you have to contend with the possibility that that address may change often, possibly without warning, and that your old address may be assigned to somebody else.

Third, most Internet service operators don’t expect that end users will ever intentionally connect to their sites directly by IP address. In some cases, when an IP address is shared by different websites or different devices, connecting by IP address alone wouldn’t even work properly. In that case, there’s not much benefit to obtaining a certificate for the IP address!

How Let’s Encrypt Subscribers May Use IP Address Certs

Most current subscribers should be fine with their existing domain name certs and won’t need IP address certs. Subscribers who have a use for an IP address cert are typically already aware of that. A few use cases that we’re aware of include:

• A default page for hosting providers, in case someone pastes a server’s IP address into a browser instead of an individual site name (right now, this normally produces an error in the browser).

• A way to access your website if you don’t have a domain name at all (at some cost in reliability and convenience compared to getting a domain name).

• Securing DNS over HTTPS (DoH) or other infrastructure services. Having a certificate makes it much easier for DoH servers to prove their identities to clients. That could make it more feasible for DoH users or clients to enforce a requirement for a valid publicly-trusted certificate when connecting to DoH servers.

• Securing remote access to some home devices (like network-attached storage servers and Internet-of-things devices) even without a domain name.

• Securing ephemeral connections within cloud hosting infrastructure, like connections between one back-end cloud server and another, or ephemeral connections to administer new or short-lived back-end servers via HTTPS—as long as those servers have at least one public IP address available.

How To Get an IP Address Cert

IP address certificates are available right now in Staging. They should be generally available in Prod later in 2025, at the same time that short-lived certificates become generally available. Prior to general availability we may allow list issuance for a limited number of partners who can provide us with feedback.

Many Let’s Encrypt client applications should already be able to request certificates for IP addresses, although there can be minor technical changes required to support this in some client software.

As a matter of policy, Let’s Encrypt certificates that cover IP addresses must be short-lived certs, valid for only about six days. As such, your ACME client must support the draft ACME Profiles specification, and you must configure it to request the shortlived profile. And, probably not surprisingly, you can’t use the DNS challenge method to prove your control over an IP address; only the http-01 and tls-alpn-01 methods can be used.

If your client software requests an IP address cert with details that aren’t compatible with these policies, the order will be rejected by the ACME server. In this case, your client application may need to be updated or reconfigured. Feel free to ask for help on the Let’s Encrypt community forum if you encounter any problems, either as a client application developer or as an end user.

Since its inception, Let’s Encrypt has been sending expiration notification emails to subscribers that have provided an email address to us via the ACME API. This service ended on June 4, 2025. The decision to end the service is the result of the following factors:

1. Over the past 10 years more and more of our subscribers have been able to put reliable automation into place for certificate renewal.
2. Providing expiration notification emails means that we have to retain millions of email addresses connected to issuance records. As an organization that values privacy, removing this requirement is important to us.
3. Providing expiration notifications costs Let’s Encrypt tens of thousands of dollars per year, money that we believe can be better spent on other aspects of our infrastructure.
4. Providing expiration notifications adds complexity to our infrastructure, which takes time and attention to manage and increases the likelihood of mistakes being made. Over the long term, particularly as we add support for new service components, we need to manage overall complexity by phasing out system components that can no longer be justified.

For those who would like to continue receiving expiration notifications, we recommend using a third party service such as Red Sift Certificates Lite (formerly Hardenize). Red Sift’s monitoring service providing expiration emails is free of charge for up to 250 certificates. More monitoring options can be found here.

We have deleted the email addresses provided to Let’s Encrypt via the ACME API that were stored in our CA database in association with issuance data. This doesn’t affect addresses signed up to mailing lists and other systems. They are managed in a separate ISRG system unassociated with issuance data.

Going forward, if an email address is provided to Let’s Encrypt via the ACME API, Let’s Encrypt will not store the address but will instead forward it to the general ISRG mailing list system unassociated with any account data. If the email address has not been seen before, that system may send an onboarding email with information about how to subscribe to various sources of updates.

If you’d like to stay informed about technical updates and other news about Let’s Encrypt and our parent nonprofit, ISRG, based on the preferences you choose, you can sign up for our email lists below:

The Certificate Transparency ecosystem has been improving transparency for the web PKI since 2013. It helps make clear exactly what certificates each certificate authority has issued and makes sure errors or compromises of certificate authorities are detectable.

Let’s Encrypt participates in CT both as a certificate issuer and as a log operator. For the past year, we’ve also been running an experiment to help validate a next-generation design for Certificate Transparency logs. That experiment is now nearing a successful conclusion. We’ve demonstrated that the new architecture (called the “Static CT API”) works well, providing greater efficiency and making it easier to run huge and reliable CT log services with comparatively modest resources. The Static CT API also makes it easier to download and share data from CT logs.

The Sunlight log implementation, alongside other Static CT API log implementations, is now on a path to production use. Browsers are now officially accepting Static CT API logs into their log programs as a means to help guarantee that the contents of CA-issued certificates are all publicly disclosed and publicly accessible (see Safari’s and Chrome’s recent announcements), although the browsers also require the continued use of a traditional RFC 6962 log alongside the new type.

All of this is good news for everyone who runs, submits certificates to, or monitors a CT log: as the new architecture gets adopted, we can expect to see more organizations running more logs, at lower cost, and with greater overall capacity to keep up with the large volume of publicly-trusted certificates.

Certificate Transparency

Certificate Transparency (CT) was introduced in 2013 in response to concerns about how Internet users could detect misbehavior and compromise of certificate authorities. Prior to CT, it was possible for a CA to issue an inaccurate or malicious certificate that could be used to attack a relatively small number of users, and that might never come to wider attention. A team led by Google responded to this by creating a transparency log mechanism, where certificate authorities (like Let’s Encrypt) must disclose all of the certificates that we issue by submitting them to public log services. Web browsers now generally reject certificates unless the certificates include cryptographic proof (“Signed Certificate Timestamps”, or SCTs) demonstrating that they were submitted to and accepted by such logs.

The CT logs themselves use a cryptographic append-only ledger to prove that they haven’t deleted or modified their records. There are currently over a dozen CT log services, most of them also run by certificate authorities, including Let’s Encrypt’s own Oak log.

The Static CT API

The original 2013 CT log design has been used with relatively few technical changes since it was first introduced, but several other transparency logging systems have been created in other areas, such as sumdb for Golang, which helps ensure that the contents of Golang package updates are publicly recorded. While they were originally inspired by CT, more-recently invented transparency logs have improved on its design.

The current major evolution of CT was led by Filippo Valsorda, a cryptographer with an interest in transparency log mechanisms, with help from others in the CT ecosystem. Portions of the new design are directly based on sumdb. In addition to designing the new architecture, Valsorda also wrote the implementation that we’ve been using, called Sunlight, with support from Let’s Encrypt. We’re excited to see that there are now at least three other compatible implementations: Google’s trillian-tessera, Cloudflare’s Azul, and an independent project called Itko.

The biggest change for the Static CT API is that logs are now represented, and downloaded by verifiers, as simple collections of flat files (called “tiles,” so some implementers have also been referring to these as “tiled logs” or “tlogs”). Anyone who wants to download log data can do so just by downloading these files. This is great for log operators because these simple file downloads can be distributed in various ways, including caching by a CDN, which was less practical and efficient for the classic CT API.

The new design is also simpler and more efficient from the log operator’s perspective, making it cheaper to run logs. As we said last year, this may enable us and other operators to increase reliability and availability by running several separate logs, likely with lower overall resource requirements than a single traditional log.

Our Sunlight experiment

For the past year, we’ve run three Sunlight logs, called Twig, Willow, and Sycamore. We’ve been logging all of our own issued certificates, which represent a majority of the total volume of all publicly-trusted certificates, into our Sunlight logs. Sunlight logged these certificates quickly and correctly on relatively modest server hardware. Notably, each log’s write side was handled comfortably by just a single server. We also achieved high availability for these log services throughout the course of this experiment. (Because our Sunlight logs are not yet trusted by web browsers, we didn’t include the SCT proofs that they returned to us in the actual certificates we gave out to our subscribers; those proofs wouldn’t have been of use to our subscribers yet and would just have taken up space.)

A potential failure mode of traditional CT logs is that they could be unacceptably slow in incorporating newly-submitted certificates (known as missing the maximum merge delay), which can result in a log becoming distrusted. This isn’t a possibility for our new Sunlight-based logs: they always completely incorporate newly-submitted certificates before returning an SCT to the submitter, so the effective merge delay is zero! Of course, any log can suffer outages for a variety of reasons, but this feature of Sunlight makes it less likely that any outages will be fatal to a log’s continued operation.

We’ve demonstrated that Sunlight and the Static CT API work in practice, and this demonstration has helped to confirm the browser developers’ hope that Static CT API logs can become an officially-supported part of CT. As a result, the major browsers that enforce CT have now permitted Static CT API logs to apply for inclusion in browsers as publicly-trusted logs, and we’re preparing to apply for this status for our Willow and Sycamore logs with the Chrome and Safari CT log programs.

Let’s Encrypt will run at least these two logs, and possibly others over time, for the foreseeable future. Once they’re trusted by browsers, we’ll encourage other CAs to submit to them as well, and we’ll begin including SCTs from these logs in our own certificates (alongside SCTs from traditional CT logs).

How to participate

The new Static CT API and the rollout of tile-based logs will bring various changes and opportunities for community members.

New Certificate Transparency log operators

Companies and non-profit organizations could help support the web PKI by running a CT log and applying for it to be publicly trusted. Implementations like Sunlight will have substantially lower resource requirements than first-generation CT logs, particularly when cached behind a CDN. The biggest resource demands for a log operator will be storage and upstream bandwidth. A publicly-trusted log is also expected to maintain relatively high availability, because CAs need logs to be available in order to continue issuing certificates.

We don’t have statistics to share about the exact resource requirements for such a log yet, but after we have practical experience running a fully publicly-trusted Sunlight log, we should be able to make this more concrete. As noted above, the compute side of the log can be handled by a single server. Sunlight author Filippo Valsorda has recently started running a Sunlight log—also on just a single server—and offered more detailed cost breakdowns for that log’s setup, with an estimated total cost around $10,000 per year. The costs for our production Static CT API logs may be higher than those for Filippo’s log, but still far less than the costs for our traditional RFC 6962 logs.

As with trust decisions about CAs, browser developers are the authorities about which CT logs become publicly trusted. Although any person or organization can run a log, browser developers will generally prefer to trust logs whose continued availability they’re confident of—typically those run by stable organizations with experience running some form of public Internet services. Unlike becoming a certificate authority, running a log does not require a formal audit, as the validation of the log’s availability and correctness can be performed purely by observation.

Certificate authorities

Once the Willow and Sycamore logs are trusted by browsers, our fellow certificate authorities can choose to start logging certificates to them as part of their issuance processes. (Initially, you should still include at least one SCT from a traditional CT log in each certificate.) The details, including the log API endpoints and keys, are available at our CT log page. You can start submitting to these logs right away if you prefer; just bear in mind that the SCTs they return aren’t useful to subscribers yet, and won’t be useful until browsers are updated to trust the new logs.

CT data users

You can monitor CT in order to watch for certificate issuances for your own domain names, or as part of monitoring or security products or services, or for Internet security research purposes. Many of our colleagues have been doing this for some time as a part of various tools they maintain. The Static CT API should make this easier, because you’ll be able to download and share log tiles as sets of ordinary files.

If you already run such monitoring tools, please note that you’ll need to update your data pipeline in order to access Static CT API logs; since the read API is not backwards-compatible, CT API clients will need to be modified to support the new API. Without updated tools, your view of the CT system will become partial!

Also note that getting a complete view of all of CT will still require downloading data from traditional logs, which will probably continue to be true for several years.

Software developers

As logs based on the new API enter production use, it will be important to have tools to interact with and search these logs. We can all benefit from more software that understands how to do this. Since file downloads are such a familiar piece of software functionality, it will probably be easier for developers to develop against the new API compared to the original one.

We’ve also continued to see greater integration of transparency logging tools into other kinds of services, such as software updates. There’s a growing transparency log ecosystem that’s always in need of more tools and integrations. As we mentioned above, transparency logs are increasingly learning from one another, and there are also mechanisms for more direct integration between different kinds of transparency logs (known as “witnessing”). Software developers can help improve different aspects of Internet security by contributing to this active and growing area.

Conclusion

The Certificate Transparency community and larger transparency logging community have experienced a virtuous cycle of innovation, sharing ideas and implementation code between different systems and demonstrating the feasibility of new mechanisms and functionality. With the advent of tile-based logging in CT, the state of the art has moved forward in a way that helps log operators run our logs much more efficiently without compromising security.

We’re proud to have participated in this experiment and the engineering conversation around the evolution of logging architectures. Now that we’ve shown how well the new API really works at scale, we look forward to having publicly-trusted Sunlight logs later this year!

Every night, right around midnight (mainly UTC), a horde of zombies wakes up and clamors for … digital certificates!

The zombies in question are abandoned or misconfigured Internet servers and ACME clients that have been set to request certificates from Let’s Encrypt. As our certificates last for at most 90 days, these zombie clients’ software knows that their certificates are out-of-date and need to be replaced. What they don’t realize is that their quest for new certificates is doomed! These devices are cursed to seek certificates again and again, never receiving them.

But they do use up a lot of certificate authority resources in the process.

The Zombie Client Problem

Unlike a human being, software doesn’t give up in frustration, or try to modify its approach, when it repeatedly fails at the same task. Our emphasis on automation means that the vast majority of Let’s Encrypt certificate renewals are performed by automated software. This is great when those renewals succeed, but it also means that forgotten clients and devices can continue requesting renewals unsuccessfully for months, or even years.

How might that happen? Most often, it happens when a device no longer has a domain name pointed to it. The device itself doesn’t know that this has changed, so it treats renewal failures as transient even though they are actually permanent. For instance:

• An organization may have allowed a domain name registration to lapse because it is no longer needed, but its servers are still configured to request certs for it.
• Or, a home user stopped using a particular dynamic-DNS domain with a network-attached storage device, but is still using that device at home. The device doesn’t realize that the user no longer expects to use the name, so it keeps requesting certs for it.
• Or, a web hosting or CDN customer migrated to a different service provider, but never informed the old service provider. The old service provider’s servers keep requesting certs unsuccessfully. If the customer was in a free service tier, there might not be invoices or charges reminding the customer to cancel the service.
• Or any number of other, subtler changes in a subscriber’s infrastructure, such as changing a firewall rule or some webserver configuration.

At the scale of Let’s Encrypt, which now covers hundreds of millions of names, scenarios like these have become common, and their impact has become substantial. In 2024, we noticed that about half of all certificate requests to the Let’s Encrypt ACME API came from about a million accounts that never successfully complete any validations. Many of these had completed validations and issued certificates sometime in the past, but nowadays every single one of their validation attempts fails, and they show no signs that this will change anytime soon.

Unfortunately, trying to validate those futile requests still uses resources. Our CA software has to generate challenges, reach out and attempt to validate them over the Internet, detect and report failures, and record all of the associated information in our databases and audit logs. And over time, we’ve seen more and more recurring failures: accounts that always fail their issuance requests have been growing at around 18% per year.

In January, we mentioned that we had been addressing the zombie client problem through our rate limit system. This post provides more detail on that progress. 

Our Rate Limit Philosophy

If you’ve used Let’s Encrypt as a subscriber, you may have run into one of our rate limits at some point, maybe during your initial setup process. We have eight different kinds of rate limits in place now; as our January post describes, they’ve become more algorithmically sophisticated and grown to address a wider range of problems. A key principle for Let’s Encrypt is that our rate limiting is not a punishment. We don’t think of rate limits as a way of retaliating against a client for misbehavior. Rate limits are simply a tool to maximize the efficient use of our limited resources and prevent people and programs from using up those resources for no constructive purpose.

We’ve consistently tried to design our rate limit mechanisms in line with that philosophy. So if a misconfiguration or misunderstanding has caused excessive requests in the past, we’re still happy to welcome the user in question back and start issuing them certificates again—once the problem has been addressed. We want the rate limits to put a brake on wasteful use of our systems, but not to frustrate users who are actively trying to make Let’s Encrypt work for them.

In addition, we’ve always implemented our rate limits to err on the side of permissiveness. For example, if the Redis instances where rate limits are tracked have an outage or lose data, the system is designed to permit more issuance rather than less issuance as a result.

We wanted to create additional limits that would target zombie clients, but in a correspondingly non-punitive way that would avoid any disruption to valid issuance, and welcome subscribers back quickly if they happened to notice and fix a long-time problem with their setups.

Our Zombie-Related Rate Limits and Their Impact

In planning a new zombie-specific response, we decided on a “pausing” approach, which can temporarily limit an account’s ability to proceed with certificate requests. The core idea is that, if a particular account consistently fails to complete validation for a particular hostname, we’ll pause that account-hostname pair. The pause means that any new order requests from that account for that hostname will be rejected immediately, before we get to the resource-intensive validation phase.

This approach is more finely targeted than pausing an entire account. Pausing account-hostname pairs means that your ability to issue certs for a specific name could be paused due to repeated failures, but you can still get all of your other certs like normal. So a large hosting provider doesn’t have to fear that its certificate issuance on behalf of one customer will be affected by renewal failures related to a problem with a different customer’s domain name. The account-specificity of the pause, in turn, means that validation failures from one subscriber or device won’t prevent a different subscriber or device from attempting to validate the same name, as long as the devices in question don’t share a single Let’s Encrypt account.

In September 2024, we began applying our zombie rate limits manually by pausing about 21,000 of the most recurrently-failing account-hostname pairs, those which were consistently repeating the same failed requests many times per day, every day. After implementing that first round of pauses, we immediately saw a significant impact on our failed request rates. As we announced at that time, we also began using a formula to automatically pause other zombie client account-hostname pairs from December 2024 onward. The associated new rate limit is called “Consecutive Authorization Failures per Hostname Per Account” (and is independent of the existing “Authorization Failures per Hostname Per Account” limit, which resets every hour).

This formula relates to the frequency of successive failed issuance requests for the same domain name by the same Let’s Encrypt account. It applies only to failures that happen again and again, with no successful issuances at all in between: a single successful validation immediately resets the rate limit all the way to zero. Like all of our rate limits, this is not a punitive measure but is simply intended to reduce the waste of resources. So, we decided to set the thresholds rather high in the expectation that we would catch only the most disruptive zombie clients, and ultimately only those clients that were extremely unlikely to succeed in the future based on their substantial history of failed requests. We don’t hurry to block requesters as zombies: according to our current formula, client software following the default established by EFF’s Certbot (two renewal attempts per day) would be paused as a zombie only after about ten years of constant failures. More aggressive failed issuance attempts will get a client paused sooner, but clients will generally have to fail hundreds or thousands of attempts in a row before they are paused.

Most subscribers using mainstream client applications with default configurations will never encounter this rate limit, even if they forget to deactivate renewal attempts for domains that are no longer pointed at their servers. As described below, our current limit is already providing noticeable benefits with minimal disruption, and we’re likely to tighten it a bit in the near future, so it will trigger after somewhat fewer consecutive failures.

Self-Service Unpausing

A key feature in our zombie issuance pausing mechanism is self-service unpausing. Whenever an account-hostname pair is paused, any new certificate requests for that hostname submitted by that account are immediately rejected. But this means that the “one successful validation immediately resets the rate limit counter” feature can no longer come into effect: once they’re paused, they can’t even attempt validation anymore.

So every rejection comes with an error message explaining what has happened and a custom link that can be used to immediately unpause that account-hostname pair and remove any other pauses on the same account at the same time. The point of this is that subscribers who notice at some point that issuance is failing and want to intervene to get it working again have a straightforward option to let Let’s Encrypt know that they’re aware of the recurring failures and are still planning to use a particular account. As soon as subscribers notify us via the self-service link, they’ll be able to issue certificates again.

Currently, the user interface for an affected subscriber looks like this:

This link would be provided via an ACME error message in response to any request that was blocked due to a pause account-hostname pair.

As it’s turned out, the unpause option shown above has only been used by about 3% of affected accounts! This goes to show that most of the zombies we’ve paused were, in fact, well and truly forgotten about.

However, the unpause feature is there for whenever it’s needed, and there may be cases when it will become more important. A very large integration could trigger the zombie-related rate limits if a newly-introduced software bug causes what looks like a very high volume of zombie requests in a very short time. In that case, once that bug has been noticed and fixed, an integrator may need to unpause its issuance on behalf of lots of customers at once. Our unpause feature permits unpausing 50,000 domain names on a single account at a time, so even the largest integrators can get themselves unpaused expeditiously in this situation.

Conclusion

We’ve been very happy with the results of our zombie mitigation measures, and, as far as we can tell, there’s been almost no impact for subscribers! Our statistics indicate that we’ve managed to reduce the load on our infrastructure while causing no detectable harm or inconvenience to subscribers’ valid issuance requests.

Since implementing the manual pauses in September and the automated pauses in December, we’ve seen:

• Over 100,000 account-hostname pairs have been paused for excessive failures.
• We received zero (!) associated complaints or support requests.
• About 3,200 people manually unpaused issuance.
• Failed certificate orders fell by about 30% so far, and should continue to fall over time as we fine-tune the rate limit formula and catch more zombie clients.

The new rate limit and the self-service unpause system are also ready to deal with circumstances that might produce more zombie clients in the future. For instance, we’ve announced that we’re going to be discontinuing renewal reminder emails soon. If some subscribers overlook failed renewals in the future, we might see more paused clients that result from unintentional renewal failures. We think taking advantage of the existing self-service unpause feature will be straightforward in that case. But it’s much better to notice problems and get them fixed up front, so please remember to set up your own monitoring to avoid unnoticed renewal failures in the future.

If you’re a subscriber who’s had occasion to use the self-service unpause feature, we’d love your feedback on the Community Forum about your experience using the feature and the circumstances that surrounded your account’s getting paused.

Also, if you’re a Let’s Encrypt client developer, please remember to make renewal requests at a random time (not precisely at midnight) so that the load on our infrastructure is smoothed out. You can also reduce the impact of zombie renewals by repeating failed requests somewhat less frequently over time (a “back-off” strategy), especially if the failure reason makes it look like a domain name may no longer be in use at all.

At Let’s Encrypt we know that building a secure Internet isn’t just a technical challenge—it’s a long-term commitment. Over the past decade we’ve made enormous strides: from issuing billions of TLS certificates to continually innovating to keep the web safer and more accessible. But none of this would be possible without recurring donations from individuals and organizations around the world.

Recurring donations are more than just financial support; they allow us to plan, innovate, and keep improving with confidence, knowing that month after month, year after year, our supporters are there. This consistent backing empowers us to maintain a secure, privacy-respecting Internet for all.

Our tenth anniversary tagline, Encryption for Everybody, highlights this vision. It’s both a technical goal and a fundamental belief that secure communication should be available to everyone, everywhere.

When we asked our recurring donors why they give, their responses affirmed how essential this commitment is. One longtime supporter shared:

For some, it’s about paying it forward, helping future users benefit as they once did:

The next decade of Let’s Encrypt will likely be about maintaining that commitment to encryption for everybody. It’s about ensuring that our work remains reliable, accessible, and—most importantly—supported by people who believe in what we do. To everyone who’s been part of this journey, thank you. We couldn’t do it without you.

During Let’s Encrypt’s 10th Anniversary Year, we’re celebrating our community and reflecting on our journey. We’d be thrilled to hear from you. Connect with us on LinkedIn, our community forum, or email us at outreach@letsencrypt.org. Let’s keep building a secure Internet together!

Let’s Encrypt is a project of the nonprofit Internet Security Research Group, a 501(c)(3) nonprofit. To support our work, visit letsencrypt.org/donate.

Let’s Encrypt will no longer include the “TLS Client Authentication” Extended Key Usage (EKU) in our certificates beginning in 2026. Most users who use Let’s Encrypt to secure websites won’t be affected and won’t need to take any action. However, if you use Let’s Encrypt certificates as client certificates to authenticate to a server, this change may impact you.

To minimize disruption, Let’s Encrypt will roll this change out in multiple stages, using ACME Profiles:

• Today: Let’s Encrypt already excludes the Client Authentication EKU on our tlsserver ACME profile. You can verify compatibility by issuing certificates with this profile now.
• October 1, 2025: Let’s Encrypt will launch a new tlsclient ACME profile which will retain the TLS Client Authentication EKU. Users who need additional time to migrate can opt-in to this profile.
• February 11, 2026: the default classic ACME profile will no longer contain the Client Authentication EKU.
• May 13, 2026: the tlsclient ACME profile will no longer be available and no further certificates with the Client Authentication EKU will be issued.

Once this is completed, Let’s Encrypt will switch to issuing with new intermediate Certificate Authorities which also do not contain the TLS Client Authentication EKU.

For some background information, all certificates include a list of intended uses, known as Extended Key Usages (EKU). Let’s Encrypt certificates have included two EKUs: TLS Server Authentication and TLS Client Authentication.

• TLS Server Authentication is used to authenticate connections to TLS Servers, like websites.
• TLS Client Authentication is used by clients to authenticate themselves to a server. This feature is not typically used on the web, and is not required on the certificates used on a website.

After this change is complete, only TLS Server Authentication will be available from Let’s Encrypt.

This change is prompted by changes to Google Chrome’s root program requirements, which impose a June 2026 deadline to split TLS Client and Server Authentication into separate PKIs. Many uses of client authentication are better served by a private certificate authority, and so Let’s Encrypt is discontinuing support for TLS Client Authentication ahead of this deadline.

How Pebble Supports ACME Client Developers

Together with the IETF community, we created the ACME standard to support completely automated certificate issuance. This open standard is now supported by dozens of clients. On the server side, did you know that we have not one but two open-source ACME server implementations?

The big implementation, which we use ourselves in production, is called Boulder. Boulder handles all of the facets and details needed for a production certificate authority, including policy compliance, database interfaces, challenge verifications, and logging. You can adapt and use Boulder yourself if you need to run a real certificate authority, including an internal, non-publicly-trusted ACME certificate authority within an organization.

The small implementation is called Pebble. It’s meant entirely for testing, not for use as a real certificate authority, and we and ACME client developers use it for various automated and manual testing purposes. For example, Certbot has used Pebble in its development process for years in order to perform a series of basic but realistic checks of the ability to request and obtain certificates from an ACME server.

Pebble is Easy to Use for ACME Client Testing

For any developer or team creating an ACME client application, Pebble solves a range of problems along the lines of “how do I check whether I’ve implemented ACME correctly, so that I could actually get certificates from a CA, without necessarily using a real domain name, and without running into CA rate limits during my routine testing?” Pebble is quick and easy to set up if you need to test an ACME client’s functionality.

It runs in RAM without dependencies or persistence; you won’t need to set up a database or a configuration for it. You can get Pebble running with a single golang command in just a few seconds, and immediately start making local ACME requests. That’s suitable for inclusion in a client’s integration test suite, making much more realistic integration tests possible without needing to worry about real domains, CA rate limits, or network outages.

We see Pebble getting used in the official test suites for ACME clients including getssl, Lego, Certbot, simp_le, and others. In many cases, every change committed to the ACME client’s code base is automatically tested against Pebble.

Pebble is Intentionally Different From Boulder

Pebble is also deliberately different from Boulder in some places in order to provide clients with an opportunity to interoperate with slightly different ACME implementations. The Pebble code explains that

For instance, the Let’s Encrypt service currently offers its newAccount resource at the path /acme/new-acct, whereas Pebble uses a different name /sign-me-up, so clients will be reminded to check the directory rather than assuming a specific path. Other substantive differences include:

• Pebble rejects 5% of all requests as having a invalid nonce, even if the nonce was otherwise valid, so clients can test how they respond this error condition
• Pebble only reuses valid authorizations 50% of the time, so clients can check their ability to perform validations when they might not have expected to
• Pebble truncates timestamps to a different degree of precision than Boulder
• Unlike Boulder, Pebble respects the notBefore and notAfter fields of new-order requests

The ability of ACME clients to work with both versions is a good test of their conformance to the ACME specification, rather than making assumptions about the current behavior of the Let’s Encrypt service in particular. This helps ensure that clients will work properly with other ACME CAs, and also with future versions of Let’s Encrypt’s own API.

Pebble is Useful to Both Let’s Encrypt and Client Developers as ACME Evolves

We often test out new ACME features by implementing them, at least in a simplified form, in Pebble before Boulder. This lets us and client developers experiment with support for those features even before they get rolled out in our staging service. We can do this quickly because a Pebble feature implementation doesn’t have to work with a full-scale CA backend.

We continue to encourage ACME client developers to use a copy of Pebble to test their clients’ functionality and ACME interoperability. It’s convenient and it’s likely to increase the correctness and robustness of their client applications.

Try Out Pebble Yourself

Want to try Pebble with your ACME client right now? On a Unix-like system, you can run

git clone https://github.com/letsencrypt/pebble/
cd pebble
go run ./cmd/pebble

Wait a few seconds; now you have a working ACME CA directory available at https://localhost:14000/dir! Your local ACME Server can immediately receive requests and issue certificates, though not publicly-trusted ones, of course. (If you prefer, we also offer other options for installing Pebble, like a Docker image.)

We welcome code contributions to Pebble. For example, ACME client developers may want to add simple versions of an ACME feature that’s not currently tested in Pebble in order to make their test suites more comprehensive. Also, if you notice a possibly unintended divergence between Pebble and Boulder or Pebble and the ACME specification, we’d love for you to let us know.

As we touched on in our first blog post highlighting ten years of Let’s Encrypt: Just as remarkable to us as the technical innovations behind proliferating TLS at scale is, so too is the sustained generosity we have benefited from throughout our first decade.

With that sense of gratitude top of mind, we are proud to announce a contribution of $1,000,000 from Jeff Atwood. Jeff has been a longtime supporter of our work, beginning many years ago with Discourse providing our community forum pro bono; something Discourse still provides to this day. As best we can tell, our forum has helped hundreds of thousands of people get up and running with Let’s Encrypt—an impact that has helped billions of people use an Internet that’s more secure and privacy-respecting thanks to widely adopted TLS.

When we first spoke with Jeff about the road ahead for Let’s Encrypt back in 2023, we knew a few things wouldn’t change no matter how the Internet changes over the next decade:

1. Free TLS is the only way to ensure it is and remains accessible to as many people as possible.
2. Let’s Encrypt is here to provide a reliable, trusted, and sound service no matter the scale.
3. Generosity from our global community of supporters will be how we sustain our work.

We’re proud that Jeff not only agrees, but has chosen to support us in such a meaningful way. In discussing how Jeff might want us to best celebrate his generosity and recognize his commitment to our work, he shared:

Indeed, this contribution is significant because of its scale, but more importantly because of its signal: a signal that supporting the not-so-glamorous but oh-so-nerdy work of encryption at scale matters to the lives of billions of people every day; a signal that supporting free privacy and security afforded by TLS for all of the Internet’s five billion users just makes sense.

Ten years ago we set out to build a better Internet through easy to use TLS. If you or your organization have supported us throughout the years, thank you for joining Jeff in believing in the work of Let’s Encrypt. For a deeper dive into the impact of Let’s Encrypt and ISRG’s other projects, take a look at our most recent annual report.

Let’s Encrypt is a project of the nonprofit Internet Security Research Group, a 501(c)(3) nonprofit committed to protecting Internet users by lowering monetary, technological, and informational barriers to a more secure and privacy-respecting Internet. For more, visit abetterinternet.org. Press inquiries can be sent to press@abetterinternet.org

Earlier this year we announced our intention to introduce short-lived certificates with lifetimes of six days as an option for our subscribers. Yesterday we issued our first short-lived certificate. You can see the certificate at the bottom of our post, or here thanks to Certificate Transparency logs. We issued it to ourselves and then immediately revoked it so we can observe the certificate’s whole lifecycle. This is the first step towards making short-lived certificates available to all subscribers.

The next step is for us to make short-lived certificates available to a small set of our subscribers so we can make sure our systems scale as expected prior to general availability. We expect this next phase to begin during Q2 of this year.

We expect short-lived certificates to be generally available by the end of this year.

How To Get Six-Day Certificates

Once short-lived certificates are an option for you, you’ll need to use an ACME client that supports ACME certificate profiles and select the short-lived certificate profile (“shortlived”). The lego client recently added this functionality.

In the meantime, the best way to prepare to take advantage of short-lived certificates is to make sure your ACME client is reliably renewing certificates in an automated fashion. If that’s working well then there should be no costs to switching to short-lived certificates.

You’ll also want to be sure your ACME client is running frequently - both for the sake of renewing short-lived certificates and so as to take advantage of ACME Renewal Information (ARI). ARI allows Let’s Encrypt to notify your client if it should renew early for some reason. ARI checks should happen at least once per day, and short-lived certificates should be renewed every two to three days, so we recommend having your client run at least once per day.

Shorter Certificate Lifetimes Are Good for Security

When the private key associated with a certificate is compromised, the recommendation has always been to have the certificate revoked so that people will know not to use it. Unfortunately, certificate revocation doesn’t work very well. This means that certificates with compromised keys (or other issues) may continue to be used until they expire. The longer the lifetime of the certificate, the longer the potential for use of a problematic certificate.

The primary advantage of short-lived certificates is that they greatly reduce the potential compromise window because they expire relatively quickly. This reduces the need for certificate revocation, which has historically been unreliable. Our six-day certificates will not include OCSP or CRL URLs. Additionally, short-lived certificates practically require automation, and we believe that automating certificate issuance is important for security.

Questions

If you have questions or comments about our plans, feel free to let us know on our community forums.

We’d like to thank Open Technology Fund for supporting this work.

Our First 6-Day Certificate

PEM format:

-----BEGIN CERTIFICATE-----
MIIDSzCCAtGgAwIBAgISA7CwFcGk4mQWEXMacRtxHeDvMAoGCCqGSM49BAMDMDIx
CzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQDEwJF
NjAeFw0yNTAyMTkxNzMwMDFaFw0yNTAyMjYwOTMwMDBaMAAwWTATBgcqhkjOPQIB
BggqhkjOPQMBBwNCAAQoSItt2V1aocI5dxrKR8iLfmm0KiVvOhiwKByzu2kLeC7C
0BdfAgtwdICdkuEhAXokhXLq6DNZZgmh5T4flVwZo4IB9zCCAfMwDgYDVR0PAQH/
BAQDAgeAMBMGA1UdJQQMMAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0j
BBgwFoAUkydGmAOpUWiOmNbEQkjbI79YlNIwVQYIKwYBBQUHAQEESTBHMCEGCCsG
AQUFBzABhhVodHRwOi8vZTYuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6
Ly9lNi5pLmxlbmNyLm9yZy8wKAYDVR0RAQH/BB4wHIIaaGVsbG93b3JsZC5sZXRz
ZW5jcnlwdC5vcmcwEwYDVR0gBAwwCjAIBgZngQwBAgEwggEFBgorBgEEAdZ5AgQC
BIH2BIHzAPEAdgDM+w9qhXEJZf6Vm1PO6bJ8IumFXA2XjbapflTA/kwNsAAAAZUf
d/zOAAAEAwBHMEUCIFNd51TfSNiJrO+294t49C5ANc4oC7gTUzf7xnlNlhKsAiEA
wi5hfiC9SsKLxlTQ0sctUxhLmdYh40r6ECWQS/yWw2AAdwDgkrP8DB3I52g2H95h
uZZNClJ4GYpy1nLEsE2lbW9UBAAAAZUfd/0TAAAEAwBIMEYCIQCs2NuZIUIloOaH
1t9eXDKb8bjoWESBPsK4i2BxMvEIswIhAOMNaQNyr1YkzrcNUz15qGV0oVLg5BJN
+ikWxXOdcRHFMAoGCCqGSM49BAMDA2gAMGUCMDANqy7G09AIwzXcd7SNl7uFwhC+
xlfduvp1PeEDHc/FA9K3mRYkGXuKtzNdOh7wcAIxALjEMDmBQiwXbB447oGkaZAe
0rqxA3EtNV5wj0obeObluj/NgUsVEG9OqiBIoggFRw==
-----END CERTIFICATE-----

openssl x509 -text output:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            03:b0:b0:15:c1:a4:e2:64:16:11:73:1a:71:1b:71:1d:e0:ef
        Signature Algorithm: ecdsa-with-SHA384
        Issuer: C=US, O=Let's Encrypt, CN=E6
        Validity
            Not Before: Feb 19 17:30:01 2025 GMT
            Not After : Feb 26 09:30:00 2025 GMT
        Subject:
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:28:48:8b:6d:d9:5d:5a:a1:c2:39:77:1a:ca:47:
                    c8:8b:7e:69:b4:2a:25:6f:3a:18:b0:28:1c:b3:bb:
                    69:0b:78:2e:c2:d0:17:5f:02:0b:70:74:80:9d:92:
                    e1:21:01:7a:24:85:72:ea:e8:33:59:66:09:a1:e5:
                    3e:1f:95:5c:19
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature
            X509v3 Extended Key Usage:
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Authority Key Identifier:
                93:27:46:98:03:A9:51:68:8E:98:D6:C4:42:48:DB:23:BF:58:94:D2
            Authority Information Access:
                OCSP - URI:http://e6.o.lencr.org
                CA Issuers - URI:http://e6.i.lencr.org/
            X509v3 Subject Alternative Name: critical
                DNS:helloworld.letsencrypt.org
            X509v3 Certificate Policies:
                Policy: 2.23.140.1.2.1
            CT Precertificate SCTs:
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : CC:FB:0F:6A:85:71:09:65:FE:95:9B:53:CE:E9:B2:7C:
                                22:E9:85:5C:0D:97:8D:B6:A9:7E:54:C0:FE:4C:0D:B0
                    Timestamp : Feb 19 18:28:32.078 2025 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:45:02:20:53:5D:E7:54:DF:48:D8:89:AC:EF:B6:F7:
                                8B:78:F4:2E:40:35:CE:28:0B:B8:13:53:37:FB:C6:79:
                                4D:96:12:AC:02:21:00:C2:2E:61:7E:20:BD:4A:C2:8B:
                                C6:54:D0:D2:C7:2D:53:18:4B:99:D6:21:E3:4A:FA:10:
                                25:90:4B:FC:96:C3:60
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : E0:92:B3:FC:0C:1D:C8:E7:68:36:1F:DE:61:B9:96:4D:
                                0A:52:78:19:8A:72:D6:72:C4:B0:4D:A5:6D:6F:54:04
                    Timestamp : Feb 19 18:28:32.147 2025 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:46:02:21:00:AC:D8:DB:99:21:42:25:A0:E6:87:D6:
                                DF:5E:5C:32:9B:F1:B8:E8:58:44:81:3E:C2:B8:8B:60:
                                71:32:F1:08:B3:02:21:00:E3:0D:69:03:72:AF:56:24:
                                CE:B7:0D:53:3D:79:A8:65:74:A1:52:E0:E4:12:4D:FA:
                                29:16:C5:73:9D:71:11:C5
    Signature Algorithm: ecdsa-with-SHA384
    Signature Value:
        30:65:02:30:30:0d:ab:2e:c6:d3:d0:08:c3:35:dc:77:b4:8d:
        97:bb:85:c2:10:be:c6:57:dd:ba:fa:75:3d:e1:03:1d:cf:c5:
        03:d2:b7:99:16:24:19:7b:8a:b7:33:5d:3a:1e:f0:70:02:31:
        00:b8:c4:30:39:81:42:2c:17:6c:1e:38:ee:81:a4:69:90:1e:
        d2:ba:b1:03:71:2d:35:5e:70:8f:4a:1b:78:e6:e5:ba:3f:cd:
        81:4b:15:10:6f:4e:aa:20:48:a2:08:05:47

2025 marks ten years of Let’s Encrypt. Already this year we’ve taken steps to continue to deliver on our values of user privacy, efficiency, and innovation, all with the intent of continuing to deliver free TLS certificates to as many people as possible; to deliver encryption for everybody.

And while we’re excited about the technical progress we’ll make this year, we’re also going to celebrate this tenth anniversary by highlighting the people around the world who make our impact possible. It’s no small village.

From a community forum that has provided free technical support, to our roster of sponsors who provide vital funding, to the thousands of individual supporters who contribute financially to Let’s Encrypt each year, free TLS at Internet scale works because people have supported it year in, year out, for ten years.

Each month we’ll highlight a different set of people behind our “everybody.” Who do you want to see us highlight? What use cases of Let’s Encrypt have you seen that amazed you? What about our work do you hope we’ll continue or improve as we go forward? Let us know on LinkedIn, or drop a note to outreach@letsencrypt.org.

Encryption for Everybody is our unofficial tagline for this tenth anniversary year. What we love about it is that, yes, it captures our commitment to ensuring anyone around the world can easily get a cert for free. But more importantly, it captures the reality that technical innovation won’t work without people believing in it and supporting it. We’re grateful that, for ten years (and counting!), our community of supporters has made an impact on the lives of billions of Internet users—an impact that’s made theWeb more secure and privacy respecting for everybody, everywhere.

Internet Security Research Group (ISRG) is the parent organization of Let’s Encrypt, Prossimo, and Divvi Up. ISRG is a 501(c)(3) nonprofit. If you’d like to support our work, please consider getting involved, donating, or encouraging your company to become a sponsor.

Fri, 14 Feb 2025 00:00:00 +0000

---

Social Media

• GitHub
• Google+
• LinkedIn
• Twitter
• Instagram