umr samovar -...
TRANSCRIPT
VETO’08 (19-20/3/2008)CIRM, Marseille Luminy, Université de la Méditerranée
E-voting verfication problems across the world
J. P. Gibson, E. Lallet, J-L. RaffyLe département LOgiciels-Réseaux (LOR)
UMR SAMOVAR(Services répartis, Architectures, MOdélisation, Validation, Administration des Réseaux)
LOR-SAMOVAR2 VETO 08
E-voting: worldwide
Nedap in Ireland USA: iVotronic in South Carolina
EU: vote counting
in Scotland
LOR-SAMOVAR3 VETO 08
E-voting: worldwide
Nedap in Ireland
They said: “The software remains under continuous development and is not of sufficient quality to enable its use to be confidently recommended.
Even if it can be demonstrated to work in most situations, the processes and documentation that underpin the design and development of this software are insufficient to enable its reliability to be assured with the necessary levels of confidence by analysis or inspection of the source code.
Functional testing has revealed programming errors and suggests the possible existence of others, thus further reducing confidence in the software.”
The commission was unable to recommend use of
Election Management Software:
LOR-SAMOVAR4 VETO 08
E-voting: worldwide
USA:
iVotronic in South Carolina
Attempts to vote for one candidate
on the iVotronic were repeatedly
changed to an opposing candidate
on the voter verification screen.
They said: “such vote-flipping is due to calibration errors — touches on the screen are simply registering incorrectly There is a well-defined, simple, 15-step process that poll workers can follow in order to re-calibrate the screen.”
LOR-SAMOVAR5 VETO 08
E-voting: worldwide
EU: vote counting
in Scotland
Officials said:
“The system counted the votes but was unable to consolidate the data
constrained in the machine before printing out the result”
140,000 ballot papers rejected
LOR-SAMOVAR6 VETO 08
E-voting: its not a joke
Voting is not a
joke, it is a
privilege and a
responsibility.
It is not to be
taken lightly, it
is not
entertainment,
it should
require some
thought.
LOR-SAMOVAR7 VETO 08
Structure of Talk
The CEV (Commision for electronic voting) in Ireland: verification after system delivery
The EU problem: verifying machines against international “recommendations” that are inadequate
The USA problem: enforcing evolving legal standards for voting machines
Generic solution: verification-based software engineering processes using formal methods (where appropriate)
Work in progress: feature-oriented domain analysis for Software Product Line
LOR-SAMOVAR8 VETO 08
E-voting in Ireland
The Irish government invited tenders for the supply of an electronic voting system, which led to the selection of a system made by UK/Dutch company Nedap/Powervote. [June 2000]
TimeLine
LOR-SAMOVAR9 VETO 08
E-voting in Ireland
Electronic voting was “successfully trialled” in the general election and Nice referendum in seven constituencies in Ireland. [October 2002]
TimeLine
LOR-SAMOVAR10 VETO 08
E-voting in Ireland
Documentation released under the Freedom of Information (FoI) Act reveals that there were “serious inconsistencies”with the counts in two of the constituencies in which e-voting was piloted [late 2002].
TimeLine
LOR-SAMOVAR11 VETO 08
E-voting in Ireland
Widespread Critical Media Coverage of e-voting (in Ireland) [2003]
TimeLine
LOR-SAMOVAR12 VETO 08
E-voting in Ireland
The Independent Commission on Electronic Voting and Counting at Elections (known as the CEV -"Commission on Electronic Voting") was established by the Government of Ireland [March, 2004]
TimeLine
LOR-SAMOVAR13 VETO 08
E-voting in Ireland
The Commission of five members was required by its terms of reference to report on the electronic voting and counting system that has been chosen for use at elections and referenda in Ireland. [April 2004 – September 2006]
TimeLine
LOR-SAMOVAR14 VETO 08
E-voting in Ireland
Reporting: Secrecy, Accuracy & Testing of the Chosen E-Voting System
Interim Report - April 2004
The Commission was not able to satisfy itself as to the accuracy and
secrecy of the system for the following main reasons:
Software Versions – final version “not available”
System Testing - insufficient
Source Code - did not obtain access to the full source code
Accuracy – software …impossible for anyone to certify its accuracy
Secrecy - can let voters identify themselves in context of corruption or
intimidation
TimeLine
LOR-SAMOVAR15 VETO 08
E-voting in Ireland
Reporting: Secrecy, Accuracy & Testing of the Chosen E-Voting System
First Report - December 2004
Recommends the development of a programme for
software assurance and system testing
TimeLine
LOR-SAMOVAR16 VETO 08
E-voting in Ireland
Reporting: Secrecy, Accuracy & Testing of the Chosen E-Voting System
Second Report - July 2006
Unable to recommend the election management software
used to prepare elections and to aggregate and count the
votes.
A need for comprehensive, independent and rigorous end-
to-end testing, verification and certification … of the entire
system as proposed for use in Ireland.
TimeLine
LOR-SAMOVAR17 VETO 08
E-voting in Ireland
Software … has not been developed in accordance
with any recognisable standard process.
Full analysis of the software may not be possible
without a specification of what its behaviour should
be. However, it is not clear whether such a
specification exists…
LEGALLY SPEAKING: The commission emphasises that its conclusion is not based on any finding that the system will not work, but on the finding that it has not been proven at this time to the satisfaction of the commission that it will work.
LOR-SAMOVAR18 VETO 08
E-voting in Ireland
Current Status:
The government has spent €52 million on electronic voting machines and spends approx. €800,000 per annum to store the machines.
Bertie Ahern defended the flawed system and has said in the Dáil, that elections after 2007 should be done without “stupid old pencils”.
Approximately €0.5m is expected to be spent improving the software.
Ahern has defended the system despite public scepticism and opposition from within his own party on the basis that having spent the money, it would cause “loss of national pride” if the system were scrapped.
LOR-SAMOVAR19 VETO 08
E-voting: the EU recommendations
The Multidisciplinary Ad Hoc Group of Specialists on legal, operational and technical standards for e-enabled voting was set up by the Council of Europe in early 2003
“. . . to develop aninter-governmentally agreed set of standards for e-enabled voting, that reflect member states differing circumstances, and can be expected to be followed by the ICT industry.”
LOR-SAMOVAR20 VETO 08
E-voting: the EU recommendations
The document they produced acknowledges that it cannot be judged in isolation.
It states that it should respect: “the obligations and commitments as undertaken within existing internationalinstruments and documents, such as [. . .]”
The list of 12 instruments that follows - though it is clearly not meant to be exhaustive - covers a diverse range of documents, including the Code of Good Practice in Electoral Matters
LOR-SAMOVAR21 VETO 08
E-voting: the EU recommendations
This inter-related set of complex documents is analagous to a software system which has evolved over time, in response to ever changing sets of requirements.
The system depends on a large number of other systems, and the environment of the system is not clearly understood.
We propose a re-engineering of these standards, but note that this needs participation from a wide range of experts.
However, there is currently no better alternative that could be adopted in place of the European standards: “no requirements catalogue existsthat expresses the requirements for e-voting systems with enough precision to be checkable”. [McGaley]
LOR-SAMOVAR22 VETO 08
E-voting: the EU recommendations
Europe provides numerous examples of countries trying to follow the standards but still failing to produce acceptable solutions.
A main problem is that the standards can be more of a hindrance than a help:
•Over and under specification•Incompleteness•Inconsistency and contradictions …
LOR-SAMOVAR23 VETO 08
E-voting: what about the USA?
In the USA, multiple layers of federal, state, and local laws, policies, regulations, and procedures must be followed when running elections.
The recount of votes in Florida during the 2000 presidential election exposed many problems with the traditional voting systems.
LOR-SAMOVAR24 VETO 08
E-voting: what about the USA?
To address the concerns, the Help America Vote Act (HAVA) was signed into law two years after “Florida”.
Currently, most election jurisdictions use systems that are required to conform to the 2002 standards developed by the Federal Election Commission (FEC, 2002).
The standards present a certification procedure involvingtesting by an ITA and most jurisdictions are legally forbidden to use uncertified systems.
Federal government is thus responsible for the testing, certification, decertification, and recertification of voting equipment.
This responsibility has been assigned to the Election Assistance Commission (EAC), an independent commission established in 2003.
LOR-SAMOVAR25 VETO 08
E-voting: what about the USA?
The Election Science Institute Project Director, Steven Hertzberg:
“Help America Vote Act has not stimulated sufficient competition among voting-equipment manufacturers…I don't want legislation to stipulate a solution, I want legislation to stipulate a set of requirements based on the needs of stakeholders. And then I want to be able to go out to private vendors and say, 'I need this. Build it.'”
LOR-SAMOVAR26 VETO 08
E-voting: what about the USA?
Sounds similar to the Irish problem, where the CEV reported:
“In the case of the chosen system, many of these requirements were largely predetermined by the fact that an existing design of electronic voting system was adopted and adapted for use in Ireland and that theirexistence was thus already implicit or explicit in that design.”
LOR-SAMOVAR27 VETO 08
E-voting: what about the USA?
Ray Martinez, former vice chairman of the Election Assistance Commission which administered $3 billion in federal funding under the 2002 Help America Vote Act, summarised the problem by stating:
"When you add so much complexity - federal mandates, state mandates, new equipment, statewide databases - to an endeavor so dependent on human interaction, you're bound to get mistakes.”
LOR-SAMOVAR28 VETO 08
E-voting: what about the USA?
The American standards call for three levels oftests to be performed on voting systems to ensure that the
end product is fit for purpose:
1. Qualification tests to be performed by ITAs designated by the National Association of State Election Directors;
2. Certification tests to be performed by the State; and
3. Acceptance tests to be performed by the jurisdiction acquiring the system.
LOR-SAMOVAR29 VETO 08
E-voting: what about the USA?
So Independent Test Agencies verify the machines before use:
Despite this logical, layered approach to verification, there have been many instances of certified election systems being “broken” (see following slides for some examples)
If systems that meet the standards can be induced to provide inaccurate or unreliable results, is the problem that the standards are poor or is the problem that the verification processes are inadequate?
LOR-SAMOVAR30 VETO 08
E-voting: what about the USA?
North Carolina 2004: Approx. 4,500 votes were lost because officials believed a computer storing ballots electronically could hold more data than it did. Impact: election compromised but results stand
Authorities: There is no way to retrieve the missing data. That is the situation and it's definitely terrible. (But the result would still be the same)
The Media: The point is not whether the votes would have changed things, it's that they didn't get counted at all
Who identified the problem: An election worker noticed that the system’s central controller displayed an error message, “Voter Log Full”, however, the display
continued to increment the number of ballots cast.
Where was the problem: UniLect Corp., the maker of the county's electronic voting system, told them that each storage unit could handle 10,500 votes, but the limit was
actually 3,005 votes
Further Analysis: Non-expert election workers took the incrementing of the number of ballots cast to be evidence that votes were still being recorded.
LOR-SAMOVAR31 VETO 08
E-voting: what about the USA?
Florida 2006: In Sarasota County, machines failed to record 18,000 possible votesImpact: election officials – “probably incorrectly” - declared Buchanan the winner over Jennings, by 369 votes in a race with 238,249 votes cast.
Authorities: Although undervoting in absentee ballots was 2.5 percentfor this race, it was about 15 percent for votes cast on
electronic voting machines.
The Media: electronic machines usually register an undervote of <1%
Who identified the problem: Election officials knew that there were issues with the machines, based upon pre-election day voting, and elections officials called the
problem “critical”. Officials were concerned enough to ask poll workers to caution
voters on Election Day to be careful not to “miss the race”.
Where was the problem: the congressional race was “easy to miss” because of its placement at the top of the second screen of choices, above a colored header
introducing the state office races that followed, and that the ballot layout and design
were thus unclear and confusing.
Further Analysis: "banner blindness" is a well documented HCI problem
LOR-SAMOVAR32 VETO 08
E-voting: what about the USA?
Ohio 2004 - Sandusky County: some ballots in nine precincts were counted twice.
Impact: Legal Action between citizens and state resulted from loss of confidence in e-voting system arising out of this and other problems (including a vote count of -25million!).
Authorities: Many election officials from different counties resignedMedia: Claims of conspiracy and fraud
Who identified the problem: election officials
Where was the problem: What appeared to be an over-vote resulted when a computer disk containing votes was accidentally backed up into the voting machines twice by an
election worker.
Further Analysis: It is entirely wrong to put this down to human error and to say it isn’t a software problem. The requirement that a vote gets counted only once should have
been enforced by the software even after the human error.
LOR-SAMOVAR33 VETO 08
E-voting: How it should be done
Professional Software Engineering on all e-government projects
Formal methods for e-voting, where appropriate:
•Interface design•Vote storage•Feature interactions in requirements models
Software Product Line (feature domain analysis)