End-to-end Verifiable E-voting Trial for Polling Station Voting

Feng Hao∗, Shen Wang∗, Samiran Bag∗, Rob Procter∗, Siamak Shahandashti†

Maryam Mehrnezhad‡, Ehsan Toreini‡, Roberto Metere‡§, Lana Liu‡∗University of Warwick, UK†University of York, UK‡Newcastle University, UK§The Alan Turing Institute, UK

Abstract—On 2 May 2019, during United Kingdomlocal elections, an e-voting trial was conducted inGateshead, using a touch-screen end-to-end verifiablee-voting system. This was the first test of its kind inthe United Kingdom, and it presented a case study toenvisage the future of e-voting.

Index Terms—E-voting, End-to-end verifiability, DirectRecording Electronic

1. Introduction

An electronic voting (e-voting) system useselectronic technologies to record, store and pro-cess ballots in a digital form. In general, thereare two types of e-voting systems. One type isdesigned for local voting in a polling station,where a touch-screen machine, called DirectRecording Electronic (DRE), is typically usedto record votes. The other type is designed forremote voting, where voters can cast their votesfrom anywhere via the Internet.

Today, e-voting has already been deployedin a number of countries. It is used in manystates in America in various forms, e.g., based onoptical scan or DRE. In India, a fully electronicvoting system called Electronic Voting Machinehas been used in all national elections since2004. Brazil started its DRE-based elections in2002. In 2007, Estonia allowed Internet votingfor national elections for the first time. In 2019during the Estonian parliamentary election, 44%of ballots were cast using Internet voting.

The e-voting systems as currently used inreal-world elections in the above countries gen-erally work like a trusted “black-box” that is

critically dependent on the integrity of the in-ternal software implementation. However, votershave no means to verify the internal software.For example, as demonstrated by Springall etal. [1], if the server software in the Estonian In-ternet voting system had been compromised, theintegrity of the whole election would have beenlost without voters even knowing it. Publishingthe source code can help promote trust, but itcannot resolve the fundamental problem as onecannot guarantee that the same software is usedunmodified on the election day.

To address the trust problem on e-votingsoftware, Rivest and Wack first proposed thenotion of software independence: “a voting sys-tem is software-independent if an undetectedchange or error in its software cannot cause anundetectable change or error in an election out-come” [2]. The software-independence principleessentially requires that a voting system shouldguarantee security without depending on detailsof the internal software implementation, sincevoters have no means to access and verify thesoftware.

There are various approaches to build asoftware-independent voting system [2], amongwhich the most promising one involves applyingcryptography to make the voting system end-to-end (E2E) verifiable. Being E2E verifiableencompasses the following aspects:

• Cast-as-intended: a voter can verify thata ballot is cast correctly for the intendedcandidate.

• Recorded-as-cast: a voter can verify that

a cast ballot is recorded correctly in thesystem.

• Tallied-as-recorded: a public observercan verify that all the recorded ballotsare tallied correctly.

A system that satisfies the above requirements issaid to be E2E verifiable. Besides verifiability,an E2E voting system must also preserve voterprivacy, ensuring that the ability to verify thattheir vote cannot be misused to reveal how theyhave voted to a third party, say a coercer. Anoverview of the E2E verifiable voting systemsin a real-world setting can be found in [3].

The potential of an E2E voting system forreal-world elections has been demonstrated ina number of studies. In 2009, Helios, an E2EInternet voting system, was used to elect theuniversity president of the Universite Catholiquede Louvain (UCL) in Belgium [4]. Scantegrity,a scanner-based E2E voting system [5], wasadopted in the municipal elections of TakomaPark, USA in 2009 and 2011. In 2014, Preta Voter (PaV), based on a hybrid method us-ing a touch-screen machine and a scanner, wasadopted in the 2014 Victoria State election inAustralia [6].

Although progress has been made in E2Everifiable voting, large-scale deployments of thistechnology are still limited due to two mainreasons. First, most of the E2E voting systemsrequire a group of tallying authorities (TAs)who are supposedly trustworthy individuals withcomputing and cryptographic expertise to per-form the complex decryption and tallying op-erations. Finding and managing such TAs hasproved to be difficult [4]. Second, the E2E sys-tems tested in polling stations, such as Scant-egrity and PaV primarily use paper at the vot-ing stage. Although they improve the systemsecurity by introducing E2E verifiability, thecomplex handling of paper ballots (e.g., using aspecial pen in Scantegrity and tearing the ballotinto halves in PaV) is not any easier than thetraditional paper ballots.

Recent research in this field has shown thatit is possible to construct fully electronic E2Everifiable voting systems without involving anyTA, using a new paradigm called “self-enforcing

e-voting” (SEEV) [7]. The removal of TAs cansignificantly simplify election management andmake the system much more practical than be-fore. The first SEEV system, called DRE-i dueto Hao et al. [7], adopts a pre-computationstrategy to encrypt ballots before the electionin a structured way such that multiplying theciphertexts after the election will cancel outrandom factors and hence allow everyone toverify the integrity of the tallying result withoutTAs. A prototype of DRE-i has been used formobile phone-based classroom voting [8]. Thesecond SEEV system, called DRE-ip due toShahandashti and Hao [9], adopts an alternativereal-time computation strategy to encrypt ballotsduring voting, while keeping an aggregated formof the random factors in memory. When theelection has finished, the system publishes thefinal aggregation of the random factors alongwith other audit data to allow the public toverify the tallying integrity without involvingany TAs. By removing the need to store pre-computed ballots, DRE-ip provides a strongerguarantee of vote privacy than DRE-i and isparticularly suited for polling station voting. Atouch-screen based implementation of DRE-ipfor polling station voting was trialled in thecampus of Newcastle University in May 2017with positive feedback from voters.

Based on the initial success of the cam-pus trial, the research team reached out to theGateshead council in Newcastle, UK, with aproposal to trial the system with real votersin a realistic polling station environment. Thisproposal was supported by the electoral officialsin the Gateshead council, and was subsequentlyapproved by the council. It was agreed that an e-voting trial would be held on 2 May, 2019 at theGateshead Civic Center polling station as part ofthe UK local elections.

This trial differs from all previous e-votingpilots in the UK in that the trialled system is E2Everifiable rather than a “black-box”. Outside theUK, this trial also represents the first time thata fully-electronic E2E verifiable voting systemwas tested in a polling station by real voters.It was hoped that the results of this trial wouldpresent a useful case study for researchers aswell as election law and policy makers.

The rest of the paper is organized as follows.Section 2 explains the DRE-ip system that wasused in the Gateshead trial. Section 3 givesdetails of the trial. Sections 4 and 5 discuss thevoters’ feedback and results. Finally, Section 6concludes the paper with suggestions for futurework.

2. DRE-ip Voting Protocol

High-level view. At a high level, a self-enforcinge-voting (SEEV) system can be explained us-ing the analogy of a picture. Imagine an elec-tion as a picture that is formed of millions ofpixels. Every voter holds a key to one pixel.The voter’s privacy is protected because eachindividual pixel does not reveal the value oftheir vote. However, when all pixels are piecedtogether, they collectively show a picture thatis the election tally. Everyone will be able tocompute/verify the tally without involving anyTAs. However, only the tally, not an individualvote or any partial tally, can be learnt. If anattacker attempts to modify pixels or the tally,the tampering will be publicly noticeable sincethe mathematical relations between the pixelswill fail to be verified. DRE-ip [9] is an in-stantiation of a SEEV system based on real-time computation (as opposed to DRE-i [7] thatis based on pre-computation). The protocol issummarized below.

Setup. Let p and q be two large primes where qdivides p− 1. The protocol operates in the sub-group of Z?

p of prime order q. In this subgroup,g1 and g2 are two random generators whose dis-crete logarithm relationship is unknown. In theimplementation, this is realized by first choosinga non-identity element as g1 and then computingg2 based on using a one-way hash functionwith inclusion of election specific information inthe input, such as the date, title and questions.For simplicity, the DRE-ip protocol is describedhere for a single candidate (Yes/No) election. Itcan be easily extended for supporting multiplecandidates as shown in [9].

Voting. After authentication, a voter casts a voteon a DRE machine in two steps. First, they are

presented with a “Yes” or “No” option for thedisplayed candidate on the DRE screen. Oncethe voter makes a choice, the DRE prints the firstpart of the receipt, containing i, Ri = gri2 , Zi =gri1 g

vi1 where i is a unique ballot index number,

ri is a random number chosen uniformly from[1, q−1], and vi is either 1 or 0 (corresponding to“Yes” or “No” respectively). The ciphertext dataalso comes with a zero knowledge proof (ZKP)to prove that Ri and Zi are well-formed [9].

In the second step, the voter has the optionto either confirm or cancel the selection. In caseof “confirm”, the DRE updates the aggregatedvalues t and s in memory as in Equation 1,deletes individual values ri and vi, and marksthe ballot as confirmed on the receipt.

t =∑

vi and s =∑

ri. (1)

In case of “cancel”, the DRE reveals ri and vion the receipt, marks it a cancelled ballot andprompts the voter to choose again. The voter cancheck if the printed vi matches their previousselection and can dispute it if it does not. Thevoter can cancel as many ballots as they wish butcan only cast one confirmed ballot. Since votingis anonymous, the machine cannot guess if, afterhaving printed the first part of the receipt, thevoter is going to “confirm” or “cancel”.

After voting, the voter leaves the votingbooth with one receipt for the confirmed ballotand zero or more receipts for the cancelledballots. All data on the receipts are digitallysigned and are also available on a public electionwebsite. To ensure the vote is recorded, the voterjust needs to check if the same receipt has beenpublished on the election website.

Tallying. Once the election has finished, theDRE publishes the final values t and s on theelection website, in addition to all the receipts.Anyone will be able to verify the tallying in-tegrity by checking the published audit data, inparticular, whether the two equalities in Equa-tion 2 hold for the confirmed ballots:

∏Ri = gs2 and

∏Zi = gs1g

t1. (2)

3. E-voting Trial

Ethics. The trial was ethically approved by theElectoral Services of the Gateshead council andthe Research Ethics Committee of the Univer-sity of Warwick. Participation in this trial wasentirely voluntary. To avoid any perception oflikely bribery, no financial compensation wasallowed to pay for the voter’s time, not even freecoffee or tea. However, sweets were permitted.So, two packs of sweets (about £2.5 for 200pieces) were purchased and made available toall voters regardless whether they took part inthe trial or not.

Since the security of DRE-ip has been peerreviewed in a published paper, the main aimof the trial was to evaluate the usability of thesystem and its public acceptance in compari-son to traditional paper ballots. The initial planwas to conduct the trial as an exit poll, so thetallying results could have been compared withthe official election results. However, during theethics review, a concern was raised that sincean e-voting device was never used in any exitpoll before, some voters might confuse the trialwith the real election. To address this concern,it was decided to use dummy candidate namesfor the trial. The voting question and the dummycandidate names were provided by the Councilbased on a sample paper ballot used in an elec-tion education program. During the briefing, vot-ers were explicitly informed that the candidatenames used in the trial were dummy ones.

Implementation. The DRE-ip system used inthe trial was implemented using an elliptic curve(NIST P-256) rather than a finite field settingfor better efficiency. This does not change theprotocol specification. The system consisted ofa server and multiple clients. Each DRE clientcomprised a touch-screen Tablet (Google PixelC 10.2 inch) connected to a thermal printer(Epson TPM-P80). Two clients were installedin the trial venue, supporting voting in parallel.The clients were connected to a remote serverwhere all the cryptographic operations were per-formed. The network connection was providedvia a wireless dongle (Huawei 4G). Although theGateshead Civic Centre provided free WiFi to all

visitors on the election day, the 4G dongle wasused for the assurance of more reliable Internetconnectivity. All the electronic devices were off-the-shelf equipment and could work in battery-only mode. Portable power banks were includedin the setup in case the electrical power becameunavailable in the venue. Therefore, other thanrequiring a physical space, the trial setup hadminimum dependence on the IT infrastructurein the polling station. Figure 1 shows the setupon the election day. Since it was a trial, the DREclients were placed in an open space. In a realelection, each should be put in a separate votingbooth.

Figure 1: Trial setup at the polling station.

Election day. The trial was chosen to be heldat Gateshead Civic Center, which was the bus-iest polling station in Gateshead. On 2 May,2019, the Gateshead Civic Center polling stationopened at 6:30 am for voting. Voters walked intothe polling station (inside a hall as indicated

in Figure 1) to vote as normal using paperballots. Upon exiting the polling station, theywere invited to take part in a voluntary trialusing e-voting. Ninety-four voters (out of a totalof about 200 voters who attended that pollingstation) participated in the trial.

After the voter consented to participate, theywere first asked if they would like to watcha short 1-minute video demonstration on howto use the system. About one third of the par-ticipants chose to watch it, while the majoritydecided to vote straightaway.

To cast a vote, the voter first picked upa folded slip of paper with a random 9-digitpasscode from a glass jar. This passcode wouldallow the voter to log in to the DRE to cast a votewhile remaining anonymous. With the passcode,the voter chose one of the provided DRE clientsand started their voting session. Figure 2 shows aseries of screenshots to illustrate the voting pro-cess. First, the voter logged in to the DRE usingthe 9-digit passcode. The screen then displayeda list of candidates. The voter touched the screento select a candidate. Meanwhile the thermalprinter printed the first part of the receipt. Basedon [8], only a truncated hash (50 characters inCrockford’s base-32 encoding) was printed onthe receipt, while the complete crypto data in-cluding the digital signature was published at theelection website. Next, the voter needed to either”confirm” or ”cancel” the selection. If “cancel”was chosen, the DRE client would return tothe initial screen of the candidate selection andprint the second part of the receipt for the justcancelled ballot (Figure 3a). If “confirm” waschosen, the voting session would terminate andthe DRE client would print the rest of the receiptfor the just confirmed ballot (Figure 3b). Afterthe trial, voters were provided with an (optional)questionnaire to provide anonymous feedback.Results of the feedback will be presented in thenext section.

The polling station closed at 10:00 pm tomark the official end of the election. Secondslater, the tallying results for the e-voting trialwere published at the election website alongwith full audit data (which can be downloaded asan XML file). The audit data was subsequentlychecked by the research team and was found

to be verified successfully. The same verifica-tion could be performed by anyone using theprovided open-source software or any indepen-dently developed software. The system recorded93 confirmed ballots and 11 cancelled ballots,with a total of 94 participating voters. The appar-ent absence of one confirmed ballot was becauseone voter logged in to the tablet but chose to exitand eventually not to vote (this voter came to thepolling station to cast a protest vote and wantedto do the same on the e-voting system).

4. Participant Study Design

Questionnaire design. The main part of thequestionnaire was designed to assess the usabil-ity of the voting process based on a common setof System Usability Scale (SUS) statements firstdeveloped by John Brooke [10]. Respondentsindicate their agreement or disagreement witheach statement using a five-point Likert scale,where 1 = “strongly disagree”, 2 = “disagree”,3 = “neutral”, 4 = “agree” and 5 = “stronglyagree”. Based on pilot testing prior to the trial, itwas found that the first statement was potentiallyconfusing. The original statement was “I thinkthat I would like to use this system frequently”,and it was changed to ‘‘I think that I would liketo use this system in future elections” to better fitthe context of the trial. The rest of the statementswere left unchanged.

The usability assessment was focused onthe voting process instead of the verificationprocess. This was for two main reasons. First,since the trial was conducted with real votersin a busy polling station, the time available foreach participant to vote and to complete a surveywas limited. Second, while voting is mandatory,verification is an optional operation. In practice,dedicated auditors may be employed to verify“cast as intended” by casting cancelled ballotsat any time during the election day; voters maygive receipts to a helper in the polling station toverify “recorded as cast” by checking if the samereceipts are published at the election website;anyone with access to the election website isable to verify “tallied as recorded” by usingthe open-source software to check the publishedreceipts and the tally. Hence, none of these veri-

(a) Login. (b) List of candidates.

(c) Select candidate. (d) Cancel selection.

(e) Select candidate again. (f) Confirm selection.

Figure 2: DRE screenshots during voting.

(a) Cancelled ballot. (b) Confirmed ballot.

Figure 3: Example of receipts in the proof-of-concept implementation.

fication operations is mandatory for an ordinaryvoter. The assessment of the usability for theverification process will be done in future work.

In addition to the SUS questions, the ques-tionnaire also collected demographic informa-tion about the participant and their background,including gender, age, education, experienceof using computer/touch-screen devices, andwhether or not they had watched the video demobefore voting.

The last part of the questionnaire asked theparticipant “based on your experience of usingpaper ballots and e-voting, which system doyou prefer”? Participants were asked to indicatetheir preference on a 5-point scale, namely, (1)strongly prefer paper, (2) prefer paper, (3) neu-tral, (4) prefer e-voting and (5) strongly prefere-voting. Participants could optionally write freetext to explain their choice.

5. Results

Demographics. Based on 93 returned question-naires, the gender distributions among the par-ticipants were 39.8% “female”, 53.7% “male”,1.1% “other” and 3.2% “prefer not to say“.The age distributions were 1.1% “below 20”,8.8% “20-29”, 27.5% “30-39”, 24.2% “40-49”,23.1% “50-59”, 13.2% “above 60” and 2.2%“prefer not to say”. While 11.3% of the partic-ipants attended “secondary school”, others varyamong “college” (28.1%), “undergraduate de-gree” (22.5%), “postgraduate degree” (34.8%)

and “Prefer not to say” (3.4%). Experience ofusing computer or touch-screen devices rangedfrom “never” (3.4%), “occasionally” (6.7%), to“sometimes” (4.5%), “often” (30.3%) and “ex-tensively” (55.1%). Only 34.8% of the partici-pants indicated they had watched the video demoprior to voting.

Completion of voting. All participants wereable to complete voting without error. In general,a 9-digit passcode is all that was needed fora voter to carry out voting by themselves byfollowing the on-screen instructions. Only onevoter encountered difficulty in touch-screen vot-ing and asked the research team for help. Thisvoter pressed the tablet screen really hard like apush button, but the touch screen did not respondunder hard pressing. The issue was resolvedby advising the voter to touch the screen moregently. It turned out that this particular voter hadno prior experience of using any touch-screendevice.

SUS scores. Using the SUS computationmethod [10], the mean SUS score was 87.9(the standard deviation or STD was 13.8). Bythe commonly used criteria [11], this score isconsidered “excellent” in usability. It is higherthan the reported SUS score of 76 for Helios, 60for PaV, 58 for Scangrity, and is comparable to89 for STAR-vote [12]. It is worth noting thatthe user study in [12] was conducted in a labenvironment with 30 recruited volunteers (paid$25 each), while the Gateshead trial involved94 real voters in a real polling station with nopayment for each participant.

Based on the analysis using the Spearmancorrelation method, the SUS score is found tobe uncorrelated with the age, gender or ed-ucation background. However, it is positivelycorrelated with the voter’s experience of usingcomputer/touch-screen devices (Spearman cor-relation coefficient ρ = 0.28, and two-tailedp = 0.008). It is inversely correlated with thewatching of the video demo prior to voting(ρ = −0.35, p = 0.001): those who choseto watch the video scored lower in SUS. Thisis counter-intuitive, but may be due to a self-selection effect: since watching the video was

a voluntary choice, those who opted to watchit tended to be those who felt less comfortablewith touch-screen e-voting. This is corroboratedby the negative correlation between watching thevideo and the voter preference: those who choseto watch the video were more in favor of paperthan e-voting (ρ = −0.235, p = 0.027).

Voting time. The voting time was recorded fromthe moment that the voter started entering thepasscode to the finish of the voting session.It ranged from the minimum of 10 seconds tothe maximum of 116 seconds with an averagevalue of 33 seconds (STD = 17 seconds). Thiscompares favorably with previous studies, whichreport mean voting time 450 seconds for Helios,550 seconds for PaV, 620 seconds for Scant-egrity and 272 seconds for STAR-Vote [12]. Thesubstantially shorter voting time for DRE-ip isdue to two factors: 1) the touch-screen interfacewas entirely electronic without involving anymanual handling of paper ballots as in othersystems; and 2) the ”confirm/cancel” choice wassmoothly integrated into the voting process as anatural voter-initiated auditing step. Indeed, afterthe voter entered the passcode, they typicallytook only 2 touches on the screen to cast avote. More touches were needed only when thevoter opted to cancel the vote and re-start fromthe initial screen. Overall, the response time forinteracting with a touch screen is much quickerthan filling in a physical paper ballot by hand.

Voter preference. Between the traditional paperballots and the trialled DRE-ip system, there wasa clear preference among voters for the latter,as summarized in Figure 4. The choice of pref-erence was positively correlated with the SUSscore (ρ = 0.59, p = 0.000), which suggeststhat usability is one key factor in deciding thevoter preference.

Among those who preferred or strongly pre-ferred e-voting (55 in total), 48 of them providedwritten comments. Three main reasons can besummarized from the provided comments. Thedominant reason seems to be the ease of use:30 voters (out of 48, or 63%) commented thatthey preferred e-voting as they found it “easier”,“more convenient” and “simpler” than paper

100

0

40

30

20

1011

9

16

23

32

Stronglypreferpaper

Stronglyprefere-voting

Prefere-voting

Preferpaper Neutral

(a) Counts of voter preferences.100

75

50

25

0Stronglypreferpaper

Stronglyprefere-voting

Prefere-voting

Preferpaper Neutral

67.50

83.06 88.28 88.0496.21

(b) Correlation with SUS scores.

Figure 4: Summary of voter preferences.

ballots. The next reason appears security: 21voters (44%) mentioned that the ability to verifythe vote made them feel “safer” and “moresecure”, as one voter commented: “I can doublecheck my vote, it seems more secure/protectedthan hand counting paper ballots”. Another votercommented: “Given a receipt at the end gives as-surance that vote is counted”. The third reason isthe speed of voting: 20 voters (42%) commentedthat they preferred e-voting as they found it“quicker”, and “faster”. This is corroborated bythe mean voting time of 33 seconds reportedearlier. Besides these three, other reasons men-tioned in the comments included the use of e-voting being more “cost-effective” (4 voters) andmore “environment-friendly” (1 voter).

Among those preferring or strongly prefer-ring paper ballots (20), all of them providedwritten comments to explain their choice. Basedon the comments, two main reasons can beidentified. The first is down to the habituation:10 voters (out of 20, or 50%) commented thatthey were a “traditionalist”, “accustomed [topaper]” and “like the ritual of casting the pa-per vote”. One voter commented: “[paper vot-ing] has worked for hundreds of years. Why

change it now just because we can?” The secondreason concerns the security: 8 voters (40%)mentioned security as a reason they preferredpaper ballots. One voter commented: “I thinka computerized system could be easily hackedwhich could affect the outcome of the ballot”.Another commented: “I would not be confidentin the security of a system of this nature. It couldbe open to hacking or other manipulation”. Itis worth noting that “security” was one mainreason for both liking and disliking the trialede-voting system. Other reasons mentioned inthe comments included the paper ballot being“simpler” (1 voter), “quicker” (1 voter) and thatthe use of e-voting might disenfranchise peoplewho “have disabilities and/or dyslexia” or “donot use computers” (2 voters).

Among those choosing “neutral” (16), 13provided further comments. The comments men-tioned a range of reasons, such as simplicity,speed, security and tradition as covered above.One factor not covered before is that some voterschose “neutral” as they did not like coming tothe polling station to vote, as one commented:“Not much time difference to actual voting [be-tween paper and e-voting] - if I were able to doonline and at home would be more beneficial.”

Limitations and future work. The Gatesheadtrial was the first study to assess the feasibilityof touch-screen based E2E verifiable e-votingfor polling station voting. Although the userfeedback shows a clear preference on the tri-alled e-voting system over paper ballots, severallimitations of this study should be noted. First,the trial was confined to one polling stationin a north-east region of the UK. Whether theresult can be generalized to the whole votingpopulation remains to be investigated. Second,the number of participants in the trial (94) wasrelatively small. Third, as the participation of thetrial was entirely voluntary, there may be a self-selection bias on the survey result. Fourth, in thetrial, only the usability of voting is evaluated, notthe usability of verification. These limitationswill need to be addressed in further studies, e.g.,by conducting more trials in distributed regions.Finally, a trial to compare E2E verifiable Internetvoting and postal voting under a remote voting

setting has not been carried out before and willbe worthwhile to conduct as a future work.

6. Conclusion

This paper summarises the results of theGateshead e-voting trial, which was the firsttime that a fully electronic voting system withE2E verifiability was tested for polling stationvoting. Feedback from the participants in thistrial indicates a clear preference of verifiable e-voting over the traditional paper ballots. Thisis because many voters considered the former“safer”, “more secure”, “quicker” and “easierto use”. This shows the promising potential ofdeploying E2E verifiable e-voting in future elec-tions. However, this trial also shows that 20 outof the 91 participants (22%) still preferred orstrongly preferred paper voting. This indicatesthat deployment of e-voting in any real-worldelection should be undertaken with caution andwith a plan to support voters, especially thosewho may be unfamiliar with e-voting or maydislike it.

The Gateshead trial was conducted for adummy election within the existing legal frame-work of the UK election law, which only allowspaper ballots for statutory voting. The results ofthis trial hopefully present a useful case studyfor voting policy makers for the possible up-dating of the law, which was written at a timewhen paper ballots were the only possible meansof voting, but has not been updated to accountfor many developments of digital technologiesin the modern era.Acknowledgements. The research on SEEV was fundedby the ERC Starting Grant, No. 306994. The systemprototype was developed under the support of the InnovateUK, Cyber Security Academic Start-up Programme. Thetrial was supported in part by the Royal Society grant,ICA\R1\180226. We thank Joanne Little and membersin the Electoral Services team in the Gateshead councilfor supporting this trial. We thank Alice Scott of theUniversity of Warwick for the news report and the team atGlobal Initiative in Oxford for contributing to the trial.

References

[1] Drew Springall, Travis Finkenauer, Zakir Du-rumeric, Jason Kitcat, Harri Hursti, MargaretMacAlpine, and J Alex Halderman. Security anal-ysis of the Estonian internet voting system. InProceedings of the 2014 ACM SIGSAC Conference

on Computer and Communications Security, pages703–715, 2014.

[2] Ronald Rivest and Madars Virza. Software indepen-dence revisited. Real-World Electronic Voting: De-sign, Analysis and Deployment, CRC Press., 2016.

[3] Feng Hao and Peter YA Ryan. Real-World ElectronicVoting: Design, Analysis and Deployment. CRCPress, 2016.

[4] Ben Adida, Olivier De Marneffe, Olivier Pereira,Jean-Jacques Quisquater, et al. Electing a universitypresident using open-audit voting: Analysis of real-world use of Helios. EVT/WOTE, 9(10), 2009.

[5] Richard Carback, David Chaum, Jeremy Clark, JohnConway, Aleksander Essex, Paul S Herrnson, TravisMayberry, Stefan Popoveniuc, Ronald L Rivest,Emily Shen, et al. Scantegrity II municipal electionat Takoma Park: The first E2E binding governmentalelection with ballot privacy. USENIX Security, 2010.

[6] Craig Burton, Chris Culnane, and Steve Schneider.Verifiable Electronic Voting in Practice: the use ofvVote in the Victorian State Election. IEEE Securityand Privacy, 2016.

[7] Feng Hao, Matthew N Kreeger, Brian Randell, Dy-lan Clarke, Siamak F Shahandashti, and Peter Hyun-Jeen Lee. Every vote counts: Ensuring integrity inlarge-scale electronic voting. In 2014 Electronic Vot-ing Technology Workshop/Workshop on TrustworthyElections (EVT/WOTE 14), 2014.

[8] Feng Hao, Dylan Clarke, Brian Randell, and Sia-mak F Shahandashti. Verifiable classroom votingin practice. IEEE Security & Privacy, 16(1):72–81,2018.

[9] Siamak F Shahandashti and Feng Hao. Dre-ip: a ver-ifiable e-voting scheme without tallying authorities.In European Symposium on Research in ComputerSecurity, pages 223–240. Springer, 2016.

[10] John Brooke. SUS-A quick and dirty usabilityscale. Usability evaluation in industry, 189(194):4–7, 1996.

[11] Aaron Bangor, Philip T Kortum, and James T Miller.An empirical evaluation of the system usabilityscale. Intl. Journal of Human–Computer Interaction,24(6):574–594, 2008.

[12] Claudia Ziegler Acemyan, Philip Kortum, Michael DByrne, and Dan S Wallach. Summative usabilityassessments of STAR-Vote: A cryptographically se-cure e2e voting system that has been empiricallyproven to be easy to use. Human factors, 2018.

Feng Hao is a Professor of Security Engineering in theUniversity of Warwick, UK.Contact him at feng.hao@warwick.ac.uk.

Shen Wang is a PhD student in Computer Science in theUniversity of Warwick, UK.Contact him at Shin.Wang@warwick.ac.uk.

Samiran Bag is a Senior Research Fellow in the Univer-sity of Warwick, UK.Contact him at Samiran.Bag@warwick.ac.uk.

Rob Procter is Professor of Social Informatics in theUniversity of Warwick, UK.Contact him at Rob.Procter@warwick.ac.uk.

Siamak F. Shahandashti is a Lecturer (assistant profes-sor) in Cyber Security in the University of York, UK.Contact him at siamak.shahandashti@york.ac.uk.

Maryam Mehrnezhad is a Research Fellow at NewcastleUniversity, UK.Contact her at maryam.mehrnezhad@ncl.ac.uk.

Ehsan Toreini is a Research Associate at NewcastleUniversity, UK.Contact him at ehsan.toreini@ncl.ac.uk..

Roberto Metere is a Research Associate at NewcastleUniversity and The Alan Turing Institute, UK.Contact him at roberto.metere@ncl.ac.uk.

Lana Y.J. Liu is a Senior Lecturer in Accounting andFinance at Newcastle University Business School, New-castle University.Contact her at lana.liu@newcastle.ac.uk.