We run one of the largest Transport Layer Security (TLS) deployments in the world, and we are continually looking at ways to move TLS security forward. Last year, we shared how we led the deployment of TLS 1.3 on the internet as well as in our data centers. Certificates are an important element for TLS security, but managing them securely can present technical challenges. To address these challenges, we’ve been working with our counterparts at the Internet Engineering Task Force (IETF) from Cloudflare and Mozilla to develop an extension to the TLS protocol, called delegated credentials. Delegated credentials is an extremely effective method for securing certificates and is in the process of being adopted as a standard by IETF.
Why do we need delegated credentials?
In modern server deployments, a certificate could be deployed to thousands of servers, which could be distributed all over the world. It’s important to ensure the security of the certificate to prevent potential attackers from gaining control of a certificate, which would enable them to perform a man-in-the-middle (MITM) attack on traffic to the server.
Facebook has built several systems to protect our certificates with a defense-in-depth approach. We offload signing operations to a separate certificate service so that private keys do not have to be stored in the memory of Proxygen, our load balancer. We securely distribute private keys to the servers. As a precaution, in case these physical security measures fail, we’ve added a final protection by shortening the lifetime of certificates. Certificates are valid only for a certain amount of time, after which they expire and browsers will reject them. This way, we’ve limited how long a potential attacker could have access to a certificate before browsers reject it.
If a certificate becomes compromised before it expires, the only option currently available to server operators is to revoke the certificate. However, certificate revocation does not work well for modern browsers. Revocation checking mechanisms such as Online Certificate Status Protocol (OCSP) may require browsers to speak to a certificate authority and can be blocked. They can also add latency to the connection establishment. As a result, modern browsers do not rely on them and instead use alternative mechanisms, such as the CRLsets used by Chrome. Because of this, we try to reduce the validity of our certificates so that we are less reliant on revocation. The shorter the certificate lifetime, the less likely a certificate will need to be revoked before it expires. We have shortened the validity lifetime of our certificates from the current industry standard of one year to just a few months. This boosts our security by reducing the period during which a potential attacker could use a compromised certificate. It also reduces our reliance on revocation and is a robust end-to-end protection method. But we started wondering: Could we reduce the lifetime of certificates even further? Maybe to just a few days or even a few hours? Would any problems occur if we did this?
The trade-off between lifetime and reliability
When establishing a TLS connection (for example, to facebook.com) the browser must verify that a server’s certificate is valid. The browser verifies that the certificate has been signed by a trusted third party, called a certificate authority (CA), and that it’s not expired. If a website’s certificate is not valid, the browser will display warnings to users. These warnings are normally risky to bypass, and users are unlikely to continue to the site, thereby the website would appear like it was down. To stay up, servers need to obtain a new certificate whenever their existing certificate expires.
The CA is run by another organization. If we shorten the expiration time of a certificate, a server would need to fetch certificates from a CA much more frequently. At scale, this means failures could happen all the time. If we reduce the certificate expiration time to just a few hours, it will reduce the time a potential attacker could use the certificate — but a network being down between CAs and servers, or the CA service being down for a few hours, will also result in a server not being able to obtain a valid certificate. WIthout a valid certificate, the website will go down. To mitigate this risk, services like ours generally opt for a longer expiration time, so there is time to recover from any failures.
Delegated credentials allow a server to better balance this trade-off between security and reliability. A server generates a new delegated credential structure containing a public key and an expiration time, which can be as little as a few hours. It can then use its signed certificate (also known as a leaf certificate) obtained from a CA to sign the delegated credential. Since this delegated credential is generated and signed by a server, it does not need to contact the CA every time it needs to create a new delegated credential. At its heart, the delegated credential is a very simple data structure:
DelegatedCredential {
PublicKey // A public key to use for the TLS handshake
ExpirationTime // Time for which the credential is valid (few hours maybe)
Signature // The signature of the delegated credential by the leaf certificate
}Clients would tell the server that they support delegated credentials and verify that the delegated credential is correctly signed by the server’s leaf certificate. The public key of the delegated credential is then used as the key for the TLS connection.
Delegated credentials provide greater flexibility to server operators. Operators can issue each of their servers a separate delegated credential with a short validity time, instead of the real certificate private key, to add defense in depth. Since the delegated credential has its own public key, a server can also experiment with new public key algorithms for TLS (including Ed25519 public keys) even before CAs support it.
Try it out
We believe delegated credentials provide an effective way to add defense-in-depth by reducing certificate lifetimes without sacrificing reliability. We have already implemented this solution in Fizz, our implementation of TLS 1.3, and plan to deploy it more widely in the coming weeks. We’ve also set up a test endpoint for others to try delegated credentials. To access the endpoint, you can download the latest release of Mozilla Firefox and modify about:config to set security.tls.enable_delegated_credentials = true. Cloudflare also has setup support for delegated credentials.
We’d like to thank DigiCert for building out the infrastructure to issue the certificates needed for delegated credentials. This will soon become an internet standard, and we hope others in the industry will soon implement changes so that everyone can begin using delegated credentials. We hope other servers will also use it to make certificate deployment more secure.
This work would not have been possible without the contributions of Puneet Mehra and those who contributed to the IETF efforts.
