Linköpings universitetSE–581 83 Linköping

+46 13 28 10 00 , www.liu.se

Linköping University | Department of Computer and Information ScienceBachelor thesis, 16 ECTS | Information Technology

2017 | LIU-IDA/LITH-EX-G–17/072–SE

How Certificate TransparencyImpact the Performance

Carl Nykvist, Linus Sjöström

Supervisor : Niklas CarlssonExaminer : Nahid Shahmehri

Upphovsrätt

Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under 25 årfrån publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår.Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstakakopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och förundervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva dettatillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. Föratt garantera äktheten, säkerheten och tillgängligheten finns lösningar av teknisk och admin-istrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman iden omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sättsamt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sam-manhang som är kränkande för upphovsmannenslitterära eller konstnärliga anseende elleregenart. För ytterligare information om Linköping University Electronic Press se förlagetshemsida http://www.ep.liu.se/.

Copyright

The publishers will keep this document online on the Internet – or its possible replacement– for a period of 25 years starting from the date of publication barring exceptional circum-stances. The online availability of the document implies permanent permission for anyone toread, to download, or to print out single copies for his/hers own use and to use it unchangedfor non-commercial research and educational purpose. Subsequent transfers of copyrightcannot revoke this permission. All other uses of the document are conditional upon the con-sent of the copyright owner. The publisher has taken technical and administrative measuresto assure authenticity, security and accessibility. According to intellectual property law theauthor has the right to be mentioned when his/her work is accessed as described above andto be protected against infringement. For additional information about the Linköping Uni-versity Electronic Press and its procedures for publication and for assurance of documentintegrity, please refer to its www home page: http://www.ep.liu.se/.

c© Carl Nykvist, Linus Sjöström

Students in the 5 year Information Technology program complete a semester-long soft-ware development project during their sixth semester (third year). The project is completedin mid-sized groups, and the students implement a mobile application intended to be usedin a multi-actor setting, currently a search and rescue scenario. In parallel they study severaltopics relevant to the technical and ethical considerations in the project. The project culmi-nates by demonstrating a working product and a written report documenting the results ofthe practical development process including requirements elicitation. During the final stageof the semester, students create small groups and specialise in one topic, resulting in a bache-lor thesis. The current report represents the results obtained during this specialisation work.Hence, the thesis should be viewed as part of a larger body of work required to pass thesemester, including the conditions and requirements for a bachelor thesis.

Abstract

Security on the Internet is essential to ensure the privacy of an individual. Today, Trans-port Layer Security (TLS) and certificates are used to ensure this. But certificates are notenough in order to maintain confidentiality and therefore a new concept, Certificate Trans-parency (CT), has been introduced. CT improves security by allowing the analysis of sus-picious certificates. Validation by CT uses public logs that can return Signed CertificateTimestamp (SCT), which is a promise returned by the log indicating that the certificate willbe added to the log. A server may then deliver the SCT to a client in three different ways:X.509v3 extension, Online Certificate Status Protocol (OSCP) stapling and TLS extension.For further analysis, we have created a tool to collect data during TLS handshakes and datatransfer, including byte information, the certificates themselves, SCT delivery method andespecially timing information. From our dataset we see that most websites do not use CTand the ones that use CT almost only use X.509 extension to send their SCTs.

Contents

Abstract iv

Contents v

List of Figures vi

1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Research questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.4 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.5 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Theory 32.1 X.509 certificates and Certificate Authorities . . . . . . . . . . . . . . . . . . . . . 32.2 TLS/SSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.3 Certificate Transparency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.4 DV, OV and EV certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.5 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3 Method 103.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.2 Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.3 Validate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.4 Data collection setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 Results 154.1 Certificate Transparency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.2 Validation types in certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

5 Discussion 205.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.3 The work in a wider context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6 Conclusion 23

Bibliography 24

v

List of Figures

2.1 View of an X.509v3 certificate [SSL] . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.2 Handshake in TLS 1.2 [cloudfare] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3 Handshake in TLS 1.3 [cloudfare] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.4 How SCTs are delivered using X.509v3 Extension . . . . . . . . . . . . . . . . . . . 62.5 How SCTs are delivered using TLS Extension . . . . . . . . . . . . . . . . . . . . . . 62.6 How SCTs are delivered using OCSP Stapling . . . . . . . . . . . . . . . . . . . . . . 7

4.1 The distribution of how SCTs are delivered . . . . . . . . . . . . . . . . . . . . . . . 164.2 The distribution of the websites and how much used number of bytes on the SCTs

using different methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.3 The distribution of the websites and how long it takes to fetch the SCTs using

different methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.4 The distribution of SCTs using different public logs . . . . . . . . . . . . . . . . . . 174.5 The distribution of how many days the SCTs have been in the logs . . . . . . . . . 174.6 The different categories of websites that used OCSP . . . . . . . . . . . . . . . . . . 184.7 The timing distribution for CT enabled websites versus non-CT enabled websites . 184.8 The distribution of the three different validation types of a certificate . . . . . . . . 19

vi

1 Introduction

Nowadays, most of the people use the Internet daily. As the Internet grows fast, questionsregarding security are growing and security becomes more and more essential. As a client,you expect to get the information from the expected website and not someone else, and cer-tificates are one component that makes this possible. Certificates are messages that includeinformation about the key, information about the identity and a digital signature. All of theinformation included in a certificate is used to prove the ownership of a public key. Certifi-cates are used to connect an identity to a key because the client need to know that it is talkingto the correct identity or the right website.

In order to manage the creation, administration, distribution and revocation of these cer-tificates, Public Key Infrastructure (PKI) is used. The standard certificate used in PKI is X.509.Even though most of the X.509 certificates are trustworthy, there may be fake X.509 certifi-cates, and therefore, there is a problem of which certificates you should rely on and how toknow if they are valid.

A question regarding keys may be which public keys we can rely on during validation ofcertificates. To solve this, Certificate Authority (CA) has keys and certificates already storedin the software, for example in Google Chrome. A CA is an entity, for example a company,that undergoes annual checks because they need to be trusted. Nevertheless, mistakes can bemade and it is even possible for a CA to be compromised and go rogue. CAs may seem tobe trustworthy for a browser but they have a chance of being rogue. A mechanism that triesto avoid these things is Certificate Transparency (CT). CT protects visitors of a website frombeing duped by certificates that are not correct and not trustworthy.

1.1 Motivation

Secure communication is difficult to handle and is in constant development with new tech-niques and every day people find new security bugs in systems. This development makesus interested and motivated in studying how you can evaluate performance, meaning thetime it takes to handle the handshake, when Signed Certificate Timestamp (SCT) has beendelivered in different ways. CT is a new technique and we will definitely see this more in thefuture. Therefore it is important to understand how it works and how it is used to be sure thecommunication is secure and safe.

1

1.2. Aim

1.2 Aim

This report will focus on three ways for SCT to be delivered in certificates and possible im-pacts these ways may have on delay for certification download. The report will cover byteand timing information about the handshake, what method is most common when deliveringSCTs e.g. To get an understanding how this can impact the performance, we have to under-stand what certificates are used and how they work. We also need to know how to validatecertificates, how CT works, and why it is used.

1.3 Research questions

For this project we want some questions answered that will guide us through the project. Thequestions we have chosen to investigate are the following:

• How does the number of bytes of the SCTs that we download differ when using thethree methods to deliver the SCTs?

• Which SCT delivery method is used most and which one has the least delivery timecompared to each other?

• How much does CT affect the performance?

1.4 Delimitations

We have limited our testing to only a fixed number of websites for our crawler due to alimitation of time and computer capacity. The program that we created only crawl the topmost 1,000,000 visited websites according to Alexa1. This report does not cover what HTTPis and how it works in general. Encryption and how encryption works are not presented dueto that this report focus on the part of Secure Sockets Layer (SSL) that validates certificatesand not how the data is sent after the validation.

1.5 Thesis outline

In the next chapter we look at the theory that is needed for the reader to understand theresearch questions and the report. In Chapter 3 we explain our methodology, used to answerthe research questions and obtain our results. After that, in Chapter 4, we present our results.In Chapter 5 we discuss the results and our method, analysing if it worked or what waswrong in the methodology. In Chapter 6, which is the last chapter, we discuss the results inmore detail and present our conclusions based on the results.

1Alexa is a website that keeps track of the top websites, http://www.alexa.com

2

2 Theory

2.1 X.509 certificates and Certificate Authorities

For associating an online identity (representing e.g. a person) with a key, digital certificateshave for some time been the most commonly used approach on the Internet. The type stan-dard today is known as X.509 and this type has the following fields:

• Version: The version for the format of the certificate

• Serial Number: Unique identifier for each issuer

• Signature value: The signature of the issuer on the certificate

• Signature algorithm: An algorithm and its parameters used in creating a signature

• Issuer: The certificate authority who issued the certificate

• Validity: How long the certificate is valid for

• Subject: The entity for which the certificate is applied to

• Subject public key: The public key of the entity

• Extensions: Extra information, for example other names of the entity

When a certificate is downloaded to a computer it needs to be validated. To do so, thecomputer needs to check for whom the certificate applies to and who the publisher is. Further,the certificate also provides an algorithm used to sign it. The signature will be decrypted byusing the publisher public key. In this way, a hash value for the certificate is obtained and cannow be compared by using the same algorithm that was given from the certificate to check ifthe result is the same as the publisher’s. If it is a match then the certificate is validated. It isalso possible and often preferred to check if the certificate is valid within its expiration date,because certificates can expire.

Additional to X.509 certificate’s, there are different versions of this standard public keycertificates. The version that we will focus on is the X.509v3, and a view of what fields areincluded in that type of certificate are shown in Figure 2.1 [1].

3

2.2. TLS/SSL

There is still a problem though. How can the user really trust that this certificate’s publickey is from the website it says it is from. It could be a man-in-the-middle attack. To make thecertificate more trustworthy, a third-party CA can also sign the certificate. This means thatthere is a separate verifier for the certificate’s authenticity apart from the sender. X.509 hasthe ability to be signed by a CA which is more trustworthy than not having anyone signingit.

Figure 2.1: View of an X.509v3 certificate [2]

2.2 TLS/SSL

Transport Layer Security (TLS), and SSL, are protocols that provide privacy and data integrityover a computer network. Today’s websites and many applications, for example VoIP1, useTLS today to make sure they exchange data and information in a secure way. The TLS proto-col has two properties: the connection is private and the connection is reliable. To make surethe connection is private, TLS uses symmetric cryptography and one encryption algorithmused for that is Advanced Encryption Standard (AES). To verify messages integrity, it uses akeyed Message Authentication Code (MAC) in combination with a hashing algorithm, e.g.SHA-1 [3].

As mentioned, many web browsers use TLS, which is designed to communicate securelyover the network. There are different versions of TLS and the most recent one is TLS 1.2, but1.3 was released this semester and is in draft mode still. Figure 2.2 shows the handshake inTLS 1.2. The client first sends a message with their supported cipher suites. Cipher suitesinclude key exchange algorithms, encryption algorithms, message authentication code anda pseudorandom function. The server responds with the chosen cipher suite that shouldbe used in the key agreement algorithm, a certificate signed by a CA including a signature.Client receives that message and generates its own KeyShare and sends a finished messagesaying it is done with the handshake. The last part of the handshake is that the server receivesthe finished message and sends a finished message also to the client, and now the server andclient can send data.

We can see that the handshake in TLS 1.2 takes two round trip times in order to be ableto send data. This needs to happen every time a new connection is required. Thus, latencybecomes a significant concern. In TLS 1.3, there is a difference here in how long the connectionset-up time is and the handshake differ as well for TLS 1.3, which is shown in Figure 2.3. Inthis case, the client predicts a key agreement algorithm that the server will decide to use.Therefore, the client sends a KeyShare in the first message. The server receives this messageand sends his own KeyShare to run the same algorithm. Then it immediately sends a finishedmessage to the client making the client aware that it can start sending data.

1Voice-over-IP, a group of technologies to deliver voice communications over the Internet.

4

2.3. Certificate Transparency

Figure 2.2: Handshake in TLS 1.2 [4]

Figure 2.3: Handshake in TLS 1.3 [4]

2.3 Certificate Transparency

Certificate Transparency (CT) is a framework created to mitigate weaknesses in the SSL cer-tificate system, and a way for clients to assess CAs trustworthiness. Specifically, CT helps todiscover malicious certificates as well as those delivered in error. CT adds three componentsto the SSL certificate system: CT logs, monitors and auditors. The logs are public, meaninganyone can open them to verify correctness, examine them wholesale or scrutinise metadata,and of course to check if new SSL certificates have been logged. Everyone can get the meta-information in the logs and the information inside them. The CT logs are also append onlywhich means that certificates can only be added to the log and not be deleted or modified.The append only facility is implemented using a Merkle Hash Tree, a binary tree structure.This aids in the detection of misbehaviour with the logs, e.g. when an attempt is made to

5

2.3. Certificate Transparency

insert old certificates. The aim of these logs is the ability to control that the correct certificateare shown in the log[5].

When someone submits a certificate to a log, the log responds with a SCT. SCT is a promisefrom the log that it will add the certificate within some fixed time that is called MaximumMerge Delay (MMD). At present, almost every log sets the MMD to 24 hours. Inclusion proof,also called Merkle audit proof, is essentially where the log demonstrates the path through theMerkle tree to get from the individual entry to the root. The benefit of inclusion proof is thatyou can verify that one of the items within the tree is indeed the certificate.

During a TLS handshake, the server must provide the SCT with the certificate if it sup-ports CT. To do this, there are three different ways in which a SCT may be delivered with acertificate that CT supports. The first one is X.509 v3 extension and in this method, CAs sub-mit a precertificate to a log in which the log responds with a SCT. When the CA has the SCT,it can attach it as an X.509v3 extension to the precertificate and then deliver the certificate tothe server operator. This is also shown in Figure 2.4. Using X.509 extension to send the SCTdoes not require any modification on the server to be able to send the SCT [5].

Figure 2.4: How SCTs are deliveredusing X.509v3 Extension

Figure 2.5: How SCTs are deliveredusing TLS Extension

The second method for delivering SCTs is TLS extension during handshake, which isshown in Figure 2.5. In this case, it is the server operator that submits the certificate to thelog. After the server has received the SCT from the log, it uses the TLS extension with typeSCT to deliver the SCT to the client.

The third method used for delivering a SCT is Online Certificate Status Protocol (OCSP)stapling. The difference between OCSP and X.509 v3 Extension is that the server here sendsan OCSP query to the CA and then the server gets a response with a SCT from the CA. Theserver can then include the SCT in the OCSP extension during the TLS handshake process.Figure 2.6 shows how OCSP stapling works. With OCSP, the CA can give the certificate tothe server and at the same time request the SCT asynchronously from the log. Using OCSPstapling or TLS extensions requires the server to make some modification to support thedelivery of SCTs [5].

6

2.4. DV, OV and EV certificates

Figure 2.6: How SCTs are delivered using OCSP Stapling

It is important to mention here that SCT delivery can combine methods, for examplethrough X.509 extension and OCSP Stapling.

Two other components in CT were mentioned earlier: certificate monitors and certificateauditors. Monitors control logs to see if they behave correctly, making sure that there areno unauthorized certificates for example. Auditors take some information from a log andchecks with other information to see if the information is consistent, to verify the integrity ofthe public logs [6].

2.4 DV, OV and EV certificates

Domain Validated (DV) certificates are only validated against the domain name. There is noassurance that any specific entity is connected to the certificate, only a DNS domain name [7].

To provide more assurance, Organization Validated (OV) certificates require further val-idation. For an OV certificate, the CA verifies that the correct applicant is attempting to getthe certificate [8].

To provide the client with even greater confidence, Extended Validation (EV) certificatesare used. EV certificates are indicated by a green bar and organization name in the addressbar of web browsers such as Google Chrome. At the time of writing, Google Chrome requiresinclusion of SCTs only for EV certificates. From October 2017, Google will require that allcertificate types include a SCT. In contrast to DV certificates, EV certificates employ extendedvalidation, with the CA verifying physical addresses among other things, before it issues thecertificate. Thus, EV gives more assurance to the user that the company issuing a certificatereally is the owner of the company. Once validated by an appropriate EV authority, a web-browser’s address bar will indicate that the certificate is EV-trusted [9].

2.5 Related work

We are not the first to analyse certificates on the Internet. Holz et al. [2] concluded in a reportthat there have been a lot of certificates that are not completely valid. They also noticed thatthere had been a decreased use of MD5 hashing, which has been almost completely mapped.But using larger hashing algorithms would also affect the performance according to theirresults.

7

2.5. Related work

Chuat et al. [10] presented the first gossip protocols2 for CT that aim at detecting loginconsistencies. They concluded that it is possible to implement a gossip protocol that couldimprove the security for certificates at the expense of the performance or privacy. Anotherinteresting aspect they concluded was that most of the computational overhead informationcomes from the signature verification in the CT logs.

Durumeric et al. [11] completed 110 scans of the public IPv4 address space when HTTPSwas used during the TLS handshakes. The scans were divided into three parts. First, theydiscovered hosts using port 443, which is for HTTPS. In the second part, TLS handshake wasdone to collect the certificate chains. The last part was to parse the certificates and validatethem and they used OpenSSL to do this. Out of 8.1 million certificates, they retrieved that48% were self-signed certificates3, 33% were certificates signed by an unknown issuer, andremaining 13% were certificates signed by a known issuer but they are untrusted.

Amann et al. [12] collected more than 17 billion SSL sessions and certificate informationfrom different university networks with several thousand active users. They used an open-source script called Bro to collect information about the SSL connection, certificates, theircorresponding timestamps, TLS extensions etc. They concluded that it is difficult to iden-tify malicious certificates because the typical behavior they show also exist in good contextswhich makes it difficult to distinguish the characteristics.

Gustafsson et al. [13] presented a large-scale characterization of the CT landscape. Inthis paper, they used a combination of active and passive measurements, but look closerat certification transparency. They concluded that there were significant differences in theselection of root stores and how certificates are added to logs. For example, Google mostlyadd certificates to logs by crawling. These certificates are compared both in practice withthose seen in the public CT logs.

Ouvrier et al. [14] characterize the TLS/SSL landscape using passive measurements anddiscuss deficiencies in the different trust relationships that affect the security of online users.From their dataset, they for instance found out that wildcard and multi-domain certificatesare popular and that some clients still offer broken ciphers, for example RC4. 232 millionHTTPS sessions were observed in this report and 25% had weak security properties.

Gustafsson et al. [15] use active measurements from different geographic locations tolook closer at differences in the TLS/SSL usage and routes taken. Active measurements inthis case are a combination of trace routes and they use OpenSSL to record certificates usedin the handshake process. They observed that there are many alternative pathways throughnon-European Autonomous Systems (ASes) whose presence raises questions regarding theintegrity of the Internet paths because ASes may be conflicted by multiple laws and otheragreements.

Beurdouche et al. [16] implemented a TLS state machine, which is a set of functions thatgenerate and parse each message and can perform cryptographic operations. This machinecaptures the cipher suites in the messages sent between client and server from the beginningto the end of the handshake. By developing this tool, they found serious flaws in most TLSimplementations.

Asghari et al. [17] analyse for instance the market of SSL certificates. By using the SSLObservatory dataset, they found out that DV and OV certificate types are around half of thetotal number of certificates. In June 2010, the percentage that have an SSL certificate from thetop 500,000 websites on Alexa were 5.0%, while the percentage of which EV certificates wereused was 8.5% from the top 500,000 websites.

Lee et al. [18] developed a tool used to characterize the cryptographic strength of pub-lic servers running SSL/TLS. They evaluated 19,000 servers and most servers supported allSSL/TLS versions. Regarding cryptography, most of the servers analysed support weak cryp-

2Protocol where peers choose randomly other peers to communicate to over the network.3A certificate signed by the same entity whose identity it certifies.

8

2.5. Related work

tography and a reason for that is because they accept the largest number of connections eventough there may be security vulnerabilities.

Akhawe et al. [7] performed a large-scale measurement study of the most common TLSwarnings by using a set of passive network monitors located at different places. They got that98.46% of the filtered connections validate correctly which means that the false warning ratewas 1.54%. In addition to that, they also found that 73.50% of incorrect connections involveunknown CA issuers or self-signed certificates.

Zhang et al. [19] determined vulnerable certificates that needed to be reissued, as well ashow many vulnerable certificates that are not revoked by the domains. The dataset comesfrom Alexa top 1 million domains. They found out that most of the vulnerable certificateshave not been reissued and 60% of those domains did not revoke their vulnerable certificates.

9

3 Method

3.1 Implementation

In order to collect data for the analysis, we created a program in Java. This program workslike a crawler, meaning it collects various pieces of information when making requests to awebsite. We used our program to analyse the most popular websites on the Internet, rankedby Alexa.

The class TLSClient is responsible for making the SSL handshake to a specified website.The is done via Bouncy Castle1 which allows us to get the certificate, number of bytes sentduring handshake, and timing statistics. It also allows us to find out in which way the SCThas been delivered. TLSClient uses the SSLSocket in the Bouncy Castle library to get thecertificates and to get information regarding the handshake. However, Bouncy Castle limitedus, because it had very poor documentation and did not support retrieval of information suchas the server OS. We try to fetch the SCTs using three different methods: First, OCSP stapling,then TLS extension, and finally X.509 extension. We keep track of the different methods thatthe servers allow. We do not continue communicating with the website after making thehandshake. Because our analysis focuses on the performance impact of certificates, it it notdesirable to spend time downloading elements of the website such as the HTML page andembedded images that may negatively impact our results.

X509Downloader is the class that is the center of it all. It initiates TLSClient making theSSL handshake, making sure to decode the SCTs in order to collect the information about themost used logs. Other than keeping track of SCTs, it also checks what kind of validation thecertificate has; for example, whether or not it is an EV certificate. It does this by checking thecertificate policies extension: if the policy identifier id matches a known OID string, we returnthe validation type that has the specific OID. This class is also responsible for the creating allthe threads and keeping track of which websites have already been tested. Therefore, a map-ping is done which points a website to a boolean: if the boolean is true then the website hasbeen requested. The X509Downloader runs on 600 threads to make it faster. Many websitesmay time out, resulting in long wait times. By having many threads, the program can stillprogress while another thread is waiting on a website. If a website times out, the programwill try again up to two times before reporting back that the website is unavailable.

1A collection of Java APIs used in cryptography, https://www.bouncycastle.org.

10

3.1. Implementation

The program has seven types of failure cases, which we have defined in the list below.Some of them occurred after we got some exceptions in the program and therefore we madethis list to present them in an easy way. Our program does not take into consideration that awebsite may use a non-standard port for HTTPS connections.

• Fail 1: The website did not have a certificate or the program could not find it.

• Fail 2: This is an unknown error regarding the fetching of the certificate. The exceptionmessage will be printed.

• Fail 3: The website could not be connected to or timed out.

• Fail 4: The website did not exist or could not find a website with the given address; thatis, an unknown host.

• Fail 5: A TLS error occurred during the fetching of the certificate.

• Fail 6: This is an unknown error while downloading the certificate.

• Fail 7: This is an unknown error. At this point, the error could come from anywhereexcept downloading the certificates.

When the program is done downloading all certificates and SCTs it will sort them in theorder that the requests were made. Then, it writes them to a semi-structured file using aseparator.

The program will collect as much information as possible. Then, scripts will be used toextract the information that is relevant to the results. Each entry in the file consists of thefollowing, in this order:

• Serial number: Unique identifier for each issuer

• Version: The version for the format of the certificate

• Type: The type of the certificate, e.g. X.509

• Signature algorithm name: The name for the certificate signature algorithm.

• Signature algorithm OID: The signature algorithm OID string for the certificate

• Issuer name: The CA who issued the certificate

• Key usage: The purpose of the key contained in the certificate

• Signature length: The length of the signature algorithm

• Public key length: The length of the public key used

• Not before: The "not before" date from the validity period of the certificate

• Not after: The "not after" date from the validity period of the certificate

• Subject name: The name of the entity for which the certificate is applicable

• Subject unique ID: The subject unique ID value for the certificate

• Handshake time: The time between the first byte sent from us, and the last byte sentfrom the server prior to the actual HTTP request

• OCSP stapling time: The time between the first byte sent from us until the whole SCTbundle, through OCSP stapling, is downloaded.

11

3.1. Implementation

• TLS extension time: The time between the first byte sent from us and until the wholeSCT bundle, through TLS extension, is downloaded.

• X.509v3 extension time: The time between the first byte sent from us until the wholecertificate, X.509v3, is downloaded and then decoded to get the SCT through theX.509v3 extension.

• OCSP stapling used: A boolean that is true if the SCT has been delivered with OCSPstapling

• TLS extension used: A boolean that is true if the SCT has been delivered with TLSextension

• X.509v3 extension used: A boolean that is true if the SCT has been delivered within thecertificate

• Bytes received: How many bytes were received during the handshake

• Bytes sent: How many bytes were sent from the client

• OCSP stapling length: The length of the OCSP bundle, if it has been used

• TLS extension length: The length of the TLS bundle extension, if it has been used

• X.509v3 extension length: The length of X.509v3 bundle extension length, if it has beenused

• Public key algorithm: The algorithm used for encrypting the public key

• Validation time: The time it takes to validate the fetched certificate using Java’s ownmethod. It is probably always valid if we have fetched it

• Site: The website that is being visited by our program

• Position: The position the website had on Alexa.com

• IP Address: The IP address of the website

• Todays date: Todays date in the format yyyy-MM-dd HH:mm:ss, according to the com-puter’s time zone

• Normal handshake time: The handshake time it takes without trying to get the SCTs

• Validation type: The validation type the website uses. For example, EV, OV or DV

• SCT decode time: The time it takes to decode the SCT from bytes to readable informa-tion. This is done using our own written function in Java

• SCT bundle size: The size in bytes of the SCT bundle

• Number of SCTs: The number of SCTs contained in the SCT bundle

• The SCT bundle: Prints all the SCTs

– Log: The name of the log the SCT is in

– Log id: The id of the log the SCT is in

– Timestamp: The timestamp represented as a date with the format yyyy-MM-ddHH:mm:ss in UTC time

– Version: The version of the protocol to which the SCT conforms

– Size: The size in bytes of the SCT

12

3.2. Script

– Hash algorithm: The hash algorithm used, represented as a number

– Signature algorithm: The algorithm used for the signature, represented as a num-ber

– Signature size: The number of bytes the signature contains

– Signature: The actual signature, represented as Base64 encoded to String

3.2 Script

Scripts have been created in Java to parse the output from generated data sets to show onlythe relevant information we want to analyse.

• Script 1: This script collects the delivery time of the SCTs and sorts it in ascending order

• Script 2: This script is similar to the first one but instead gets the length of the SCTs inbytes

• Script 3: This script counts how many times each of the three methods has been usedto fetch a SCT. It also counts the number of websites that did not respond, did not useCT, did not use HTTPS or if any error occurred

• Script 4: This script is the log counter, which counts the number of logs that is used forall decoded SCTs

• Script 5: This script takes the timestamp from all the SCTs and counts how many daysago it was from today

• Script 6: This script prints all the websites where OCSP stapling was used

• Script 7: This script collects all the handshake timing from the ones that uses CT anddo not uses CT, and then print these to two separated files

• Script 8: This script counts the validation types used. It counts how many entries usedEV, OV, DV or Other

By doing this we can create graphs and visualize how the differences and distribution lookslike.

3.3 Validate

To make sure our program is showing the same results as it does when entering a website,we validate it with Wireshark2. Wireshark lets us look at the requests sent from and to us.By looking at the requests, we will be able to see the timing and other information regardingthe certificates. We can then compare the information with what our program has given forthe same website. If the data matches or is within a margin of error, our program shouldbehave according how it looks in the reality. If the program does not match at all, we willneed to look into why there is a difference and try to explain it. To make sure our results areaccurate, we will perform this process on more than one website, and distinguish betweentop websites and websites that are less popular.

A potential side-effect of multi-threading is unfair usage of the CPU. It takes a couple ofseconds before all of the threads are in use. Therefore, the first thread that runs will get thewhole CPU for the beginning of its request and may have a lower response time. To remedythis, we add dummy websites before and after the top websites. They will run as any otherwebsite in the program but will not be printed into the data file. The dummy websites are

2Free and open source packet analyzer, https://www.wireshark.org.

13

3.4. Data collection setup

all the same and chosen randomly from websites that use HTTPS. For the purposes of thisstudy, we used Fotbollskanalen3. The number of dummy websites is the same as the numberof threads we use, both in the beginning and at the end of the top websites. This is to ensurethat when we begin downloading the certificate from a website that will appear in the results,we are utilizing as many threads as we can. Because of this the first website in the top websitelist will not get any advantages regarding process power.

3.4 Data collection setup

Before starting the download of the certificates, the program will generate a readme file. Thisfile will contain information about the computer and its components before the downloadstarts. It is important to see what condition the computer is in when the program started be-cause it may have an affect on the results. The readme file contains the following information:

• Available processors: The number of cores available on the computer

• Free memory: The number of bytes free to use on the computer

• Maximum memory: The maximum number of bytes the computer has

• Total memory available to JVM: The number of bytes available for the JVM to use

• OS: The operating system the computer is using

• CPU: The name of the CPU the computer has and also the maximum clock frequency

3https://www.fotbollskanalen.se.

14

4 Results

In this section, the results and information that we got from our program will be presentedusing graphs.

4.1 Certificate Transparency

In Figure 4.1, the distribution of how SCTs are delivered is shown. The pie chart is dividedup into the following categories:

• TLS - If the SCT has been delivered by TLS extension

• X.509 - If the SCT has been delivered by X.509v3 extension

• OCSP - If the SCT has been delivered by OCSP stapling

• X.509 and TLS - If the same SCT has been delivered by TLS extension and X.509v3extension

• Did not use CT - The website did use HTTPS but did not use certificate transparency

• No HTTPS - The website do not use HTTPS

• No Response - Our program could not connect to the website and a timeout occurred

• TLS error - This can be caused by many reasons but for example that the certificate’svalidity could not be verified or the connection was lost

• Unknown error - Any other error has occurred and no certificate has been downloaded

Most of the websites did not use CT to deliver any SCT at all, 45.57% and 23.52% of allwebsites lead a TLS error. From the websites that use CT to deliver the SCT, X.509v3 is usedmostly, which is 10.49% and after that TLS extension is the most used method for deliveringthe SCT with 0.82%. There were 17 websites that used OCSP stapling which correspondedto 0.002%. It is possible that the same SCT can be delivered by for example OCSP and TLSand other combinations of the three methods. The only combination that was used from ourdataset was TLS and X.509v3 extension and there were 8 websites that used both of these

15

4.1. Certificate Transparency

which gave us 0.001%. Since there were no website that used any other combination of thethree methods, we did not choose to include these in Figure 4.1.

Figure 4.1: The distribution of how SCTs are delivered

Figure 4.2 is a Cumulative Distribution Function (CDF) graph that represents the distri-bution of how many bytes the SCTs contained depending on which of the three methods thathave been used. The Y-axis shows the distribution of the websites, and is sorted by numberof bytes. The X-axis shows how many bytes the different methods and websites uses. TheTLS extension is almost the same for all of the websites except some few websites where theyuse a lot of more bytes in the SCTs. The X.509 extension has more steps and the SCTs containmore bytes in general than TLS extension. Most of the values in the X.509 extension are be-tween 240 to 360 bytes. Since there are only 17 websites that used OCSP stapling, there arenot many points on that graph which makes it difficult to comment anything about it.

Figure 4.3 shows the CDF of the timing for the three different methods that may be usedto deliver SCTs. The Y-axis is a distribution of the websites starting with the quickest andending with the slowest. The X-axis shows the number of milliseconds it took to fetch theSCTs. TLS extension is the graph to the leftmost and is the fastest. X.509 extension has a verylarge difference between the lowest and the highest website and it is slower than the TLSextension. There were only 17 websites that used OCSP stapling which did not give us thatmany results.

Figure 4.2: The distribution of thewebsites and how much used num-ber of bytes on the SCTs using differ-ent methods

Figure 4.3: The distribution of thewebsites and how long it takes tofetch the SCTs using different meth-ods

16

4.1. Certificate Transparency

Since we decode the SCTs, we can collect information in which log each SCT has beenstored. This resulted in the bar graph shown in Figure 4.4. This bar diagram showthat the most common logs used nowadays are ct.ws.symantec.com with 26.397% andct.googleapis.com/pilot with 25.453%. There are also three less used than the top two logs,ct.googleapis.com/aviator with 12.133%, ct.googleapis.com/rockeeter with 17.361% and fi-nally ct1.digicert-ct.com/log with 10.160%. The log ct.googleapis.com/aviator is frozen. Ingeneral, a frozen log means that it is full and no more certificates can be added to it, but thecertificates in it right now can still be used. The aviator log though was frozen because itexceeded its MMD. Some websites still use logs that have ceased operation. These are thelogs that have ceased operation, ct.izenpe.com with 0.098%, ct.wosign.com with 0.006% andlog.certly.io with 0.078%. 0.007% of the websites used an unknown log. Known logs are de-clared by Certificate Transparency website. From the same website it is possible to see at whatdate the logs started being public and available. For example, Pilot was opened 2013-03-25and Rocketeer 2014-09-01 which is early compared to some logs that do not store many SCTs,e.g. Izenpe[5]

Figure 4.4: The distribution of SCTs using different public logs

It is also interesting to see how long some SCTs have been in the logs. Figure 4.5 showsa CDF that represents the distribution of how many days the SCTs have been stored in allpublic logs that were showed in the bar graph before. The Y-axis is a distribution of thewebsites sorted by number of days in logs. The X-axis shows the number days the SCTs hasbeen stored in the logs. As showed in the Figure 4.5, about a third of the SCTs have been inthe log for less than 100 days.

Figure 4.5: The distribution of how many days the SCTs have been in the logs

17

4.1. Certificate Transparency

Since we only got that 17 websites used OCSP, it is interesting to find out what typesof websites these are. For example if they are food market stores or software developmentcompanies etc. Figure 4.6 shows a pie graph over these categories. As we can see, mostof the websites that used OCSP are about security development, for example cryptographiccommunity that develops software packages.

Figure 4.6: The different categories of websites that used OCSP

Figure 4.7 shows a CDF of the handshake timings for websites that uses CT and for web-sites that do not use CT. The Y-axis is a distribution of the websites sorted by the handshaketime sorted in ascending order. The X-axis shows the number of milliseconds it took to makethe handshake. As shown, both plotted lines are close to each other but CT enabled websitesis a little bit faster in general because it has most of its website faster than then the ones thatdo not use CT.

Figure 4.7: The timing distribution for CT enabled websites versus non-CT enabled websites

18

4.2. Validation types in certificates

4.2 Validation types in certificates

Figure 4.8 shows the distribution of which validation types that is used on the top websitesthat uses HTTPS. The result shows that 4% of the websites uses EV to validate. 14% uses OVto validate. 57%, which is most websites, uses DV to validate. 25% of the websites uses anOID value that was unknown to the program, which we named Other.

Figure 4.8: The distribution of the three different validation types of a certificate

19

5 Discussion

In this section, a review of the method used to derive the results is presented as well a dis-cussion about the results.

5.1 Results

In Figure 4.1, we showed the different ways SCT can be delivered and which delivery wasmost common, by percent. As shown, most certificates did not use CT at all. A reason forthis may be that Google does not require all certificates to include SCTs nowadays. At themoment, Google only requires SCTs for EV certificates, which is explained in Section 2.3.Therefore, this category will probably change a lot in some months. If we were to run ourprogram again in December of this year, this category would not have been the largest onein the pie graph. Another reason why CT is not commonly used is because Chrome at themoment is the only browser that has implemented CT. However, this is expected to changesoon as Mozilla has plans to implement CT and the other browsers tends to follow Chromeand Mozilla. Wider adaption and broader policies (beyond EV certificates) are expected toimprove adaption.

From our results, we got that the time it took to get the SCT from X.509v3 extension wasgreater than the time for TLS and this is shown in the CDF in Figure 4.3. The smaller timedelta between different websites makes TLS extension a fast way to deliver the SCT com-pared to X.509 extension. A possible reason for this is that SCTs in TLS can be issued asyn-chronously. This becomes very clear when looking at Figure 2.4 and Figure 2.5. These figuresshows that the X.509v3 extension delivery method makes the server wait on the CA before itcan send the certificate and SCT. Meanwhile, using TLS extension makes it able to send thecertificate while getting the SCT. If the CA wants to embed an SCT with the X.509v3 extension,they have to wait for the log to issue the SCT. Another reason why TLS have faster deliverytime is because the TLS extension appears in the ServerHello message, which is the very firstpacket the server sends back, whereas the OCSP stapling and X.509 extension extensions areoften in a later packet. This means there is a greater chance for the latency between the clientand server to vary for OCSP stapling and X.509 extension when downloading SCTs. Also Fig-ure 4.3 shows that for the TLS extension method, a lot of websites have very similar deliverytimes. This could be because a lot of the websites that uses TLS extension are more up to dateand perhaps larger companies (e.g., Google and Netflix) have more than one server. If this is

20

5.2. Method

the case, they probably have their servers near or in Sweden and therefore we get a similarlow delivery time.

The performance impact for end users was designed to be minimal because inclusionproof checks can be done asynchronously; they do not have to block the user’s HTTPS con-nection. The reason why there are both an inclusion proof and SCT is that it allows CAs toissue without having to be delayed by the logs. An alternative mechanism here would be:CA submits the certificate, waits for inclusion proofs(which could take up to 24 hours), andthen the inclusion proofs can be embedded directly inside the certificate. CAs do not preferthe idea of having an up to 24 hours delay, and this is why inclusion proofs checks and SCTsare separate which means there is no delay in issuance at the moment. But you do have towait up to 24 hours after issuance to verify that the SCT has appeared in the log.

Figure 4.4 showed that Pilot, Symantec and Rocketeer logs were used most and a reasonfor this may be that these logs were submitted early compared to other logs.

As we saw in Figure 4.5, most of the SCTs have been in the log for more than 100 days.This SCTs could be the SCTs for the website that is less popular and less up to date. Facebookand Google, which are very popular websites, would probably have their certificates up todate and their SCTs in the log less than 100 days.

From the result at Figure 4.7, we saw that using CT was in general faster. This is unex-pected because using CT requires more data to be sent. However, the websites that use CTare probably more popular websites with more CPU on their servers and with better geo-graphical positions relative to Sweden. Therefore, the speed could be faster even though theyare using CT.

From the result presented in Figure 4.8, we saw that most websites had certificates of typeDV. This was something that we expected because DV certificates are often cheap or free. Webelieve that large companies and organizations use OV or EV certificates because they givemore trust to the user. Therefore, it may be that in the beginning of our trace, there weremany OV since we visited the popular websites and large organizations. The ones that usesEV are probably organizations that handle money or something equally important, and theywould want to give the visitors maximum trust. We can also see that 25% was unknown anda reason for this is we used a specific list of known OIDs and we may have missed somethere. For example, Mozilla has a CA certificate list where they have EV Policy OIDs columnand these OIDs has not been included in our scripts.

5.2 Method

If we would have written the program in C, using OpenSSL, we believe that the programmay have been faster. We believe that the decoding of X.509 certificates takes a significantamount of time, and that running C code would make this process faster than using Java.

We also started out with a single threaded program. This turned out to be very slowbecause when we tried to get the certificates, we timed out frequently and were unable tolower the wait time before we time out. Therefore, we made the program multi-threadedto download from multiple websites simultaneously. Because of this, if one thread werewaiting on an answer from a website, we would have another thread that was not waiting.This allowed the program to run freely, and not be stuck in a continuous state of waiting.

Since we had limited time to do this project, we did not have time to run our program onone million websites more than once. If we had this possibility, we could get a better meanvalue of the timings and confirm the results. There might be disturbances in the networkduring the tests and the network condition may vary from day to day, which could affect thedata. If we would have done it more than once, the reliability of the results increase becausehaving disturbances for the same website on two different moments is rare. This is especiallyunusual on the top websites. If someone were to run our experiment in the future, this wouldwould make it more likely that they would see similar results.

21

5.3. The work in a wider context

Also, our geographical position affects our tests. In Sweden we may have different serverscloser to us relative to, for example, the United Kingdom. Thus, the delivery times will varydepending on where in the world the program would run.

5.3 The work in a wider context

In our work, we limited ourselves to only focus on the delivery of SCTs when it comes towhich method they use and the time it takes to download them. The program could befurther improved and modified to collect information about the servers involved during thehandshakes with the top million websites. This information could be the location of the serverand what OS the servers use. If this were to be done, it may be interesting to see how muchthe locations of the servers affect the steps in the CDF-figures, and if there is a correlationbetween CT usage and CA choice.

22

6 Conclusion

By looking at the results of this thesis, we can draw a number of conclusions. One of the con-clusion is that most of the websites uses X.509 extension to deliver their SCTs. TLS extensionis the second most used delivery method, and it is not even close to the same amount used asX.509 extension. This distribution is probably because using X.509 extension does not requireany server modification. Meanwhile sending SCTs through TLS extension or OCSP staplingrequires server modification. In the future we believe that TLS will be more progressivelyused. This is because in our result from Figure 4.3 that shows us that using TLS extension re-quires less time to get the SCT. We also concluded that TLS uses the least number of bytes ingeneral. We saw in Figure 4.2 that very few websites sends their SCT, using X.509 extension,with very low number of bytes. But the rest of the websites that uses X.509 sends with eitherthe same as TLS or with even higher. TLS on the other hand uses almost always the samenumber of bytes except some very few website that uses a very large number of bytes. But aswe stated earlier, in general TLS extension uses less number of bytes to send their SCTs.

23

Bibliography

[1] R. Housley, W. Polk, W. Ford, and D. Solo. RFC3280: Internet X.509 Public Key Infrastruc-ture Certificate and Certificate Revocation List (CRL) Profile. IETF. 2002.

[2] R. Holz, L. Braun, N. Kammenhuber, and Carle G. The SSL landscape: A thorough anal-ysis of the X.509 PKI using active and passive measurements. In Proc. ACM SIGCOMMconference on Internet measurement (IMC). 2011.

[3] T. Dierks and E. Rescorla. The Transport Layer Security (TLS) Protocol Version 1.2. RFC5246. 2008.

[4] F. Valsorda. An overview of TLS 1.3 and QA. URL: https://blog.cloudflare.com/tls-1-3-overview-and-q-and-a (visited on 04/24/2017).

[5] Google certificate transparency team. Certificate Transparency. URL: https://www.certificate-transparency.org (visited on 04/15/2017).

[6] B. Laurie, A. Langley, and E. Käsper. RFC6962: Certificate Transparency. IETF. 2013.

[7] D. Akhawe, B. Amann, M. Vallentin, and R. Sommer. Here’s My Cert, So Trust Me,Maybe? Understanding TLS Errors on the Web. In Proc. International Conference on WorldWide Web. 2013.

[8] A. Delignat-Lavaud, M. Abadi, A. Birrell, I. Mironov, T. Wobber, and Y. Xie. Web PKI:Closing the Gap between Guidelines and Practices. In Proc NDSS. Annual Network andDistributed System Security Symposium. 2014.

[9] CA/Browser Forum. EV SSL Certificate Guidelines version 1.6.2. 2017.

[10] L. Chuat, P. Szalachowski, A. Perrig, B. Laurie, and E. Messeri. Efficient gossip protocolsfor verifying the consistency of certificate logs. In Proc. IEEE CNS. 2015.

[11] Z. Durumeric, J. Kasten, M. Bailey, and J. Halderman. Analysis of the HTTPS CertificateEcosystem. In Proc IMC. Conference on Internet Measurement Conference. 2013.

[12] B. Amann, R. Sommer, M. Vallentin, and S. Hall. No attack necessary: The surprising dy-namics of SSL trust relationships. In Proc. Annual Computer Security Applications Con-ference. 2013.

[13] J. Gustafsson, G. Overier, M. Arlitt, and N. Carlsson. A First Look at the CT Landscape:Certificate Transparency Logs in Practice. Proc. Passive and Active Measurement Confer-ence (PAM). 2017.

24

Bibliography

[14] G. Ouvrier, M. Laterman, M. Arlitt, and N. Carlsson. Characterizing the HTTPS TrustLandscape: A Passive View from the Edge. IEEE Communications Magazine. 2017.

[15] J. Gustafsson, R. Hiran, V. Krishnamoorthi, and N. Carlsson. The Hidden Mailman andHis Mailbag: Routing Path Analysis from a European Perspective. In Proc. IEEE Interna-tional Conference on Communications (IEEE ICC). 2017.

[16] B. Beurdouche, K. Bhargavan, A. Delignat-Lavaud, C. Fournet, M. Kohlweiss, A.Pironti, P. Strub, and J. Zinzindohoue. A messy state of the union: Taming the compositestate machines of TLS. In Proc. IEEE S&P. 2015.

[17] H. Asghari, M. van Eeten, A. Arnbak, and N. van Eijk. Security economics in the HTTPSvalue chain. In Proc. Workshop on the Economics of Information Security (WEIS). 2013.

[18] H. Lee, T. Malkin, and E. Nahum. Cryptographic strength of SSL/TLS servers: Current andrecent practices. In Proc. ACM SIGCOMM Conference on Internet Measurement (IMC).2007.

[19] L. Zhang, D. Choffnes, D. Levin, T. Dumitra, A. Mislove, A. Schulman, and C. Wilson.Analysis of SSL Certificate Reissues and Revocations in the Wake of Heartbleed. In Proc IMC.Conference on Internet Measurement Conference. 2014.

25