[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [ieee] Re: Re: Quick Re: to Dick Johnson's Re: COTS scope

I believe that election systems need to be able to be tested and confirmed as to correctness by ordinary people. I believe that software hash of code systems on lab inspected code and text based configuration that is simple enough to be inspected by the candidates and poll workers as to correctness provide that basic protection of the accuracy of the system.

I believe all code should be hashed. That the encryption should be checked by the poll workers on election morning.

Susan Eustis
WinterGreen Research
6 Raymond Street
Lexington, MA 02421
Tel: (781) 863-5078
Cell: (617)852-7876

----- Original Message ----- From: "Dick Johnson" <dick.johnson@oracle.com>
To: "susan eustis" <susan@wintergreenresearch.com>
Cc: "Andrew Berg" <andrewb@votehere.com>; "Vincent J. Lipsio" <vince@lipsio.com>; <dill@CS.Stanford.edu>; <stds-1583-disc@ieee.org>; <cots@lipsio.com>
Sent: Monday, December 06, 2004 8:34 AM
Subject: Re: [ieee] Re: Re: Quick Re: to Dick Johnson's Re: COTS scope


My intention was to note that configuration itself can produce very different test results (and election behavior), and the complete system must be subject to test in order to predict behavior in production. Most common errors (substitution of one candidate for another) can easily be accomplished through wrong configuration, wrong setup (installation and setup should be automated so that it can be reproduced) and incorrect installation.

Absent Open Test and strict configuration control, absent installation which can be reproduced, we are straining at gnats and swallowing camels. Andrew Berg also points out quite correctly that if you are concerned about the end result, certification of the systems would not hurt. And I believe that automated Open Test is the only way to satisfy this certification need.

I have no problem with Microsoft being mission critical. I prefer Linux, but I just would not exempt any OS from an election application system test, with a lock down before election use of a certified, fully tested system version to include automated installation and configuration along with any other COTS hardware and software. Automated test should allow quick retesting as required for recertification. Without exception, the intended COTS pieces (in the versions and configurations and installation procedures to be used in the election) must be subject to test and certification as a system.

Code inspection is good, but only if it is sufficiently organized. The datapath for election results should be different for each of the electronic recorded vote and the tangible paper ballot at a minimum. Most code inspection I have done focused on critical parts of a system. So, here, I think that the election specific (and not COTS) software should be subject to inspection simply as a matter of priority. Software designed for elections should be open to inspection by public parties, most likely technical proxies for the contenting parties. We have somewhat neglected to role of the poll watcher, which hs certainly changed.

This is my personal and professional opinion, and it does not necessarily reflect the opinions of my employer.

-- Dick

Richard C. Johnson, Ph.D.
Applications Architect
Oracle Corporation

susan eustis wrote:

As an analyst I helped define IBM MQSeries originally which clearly is mission critical in terms of guaranteed delivery and far different from Microsoft or open source e-mail which is also used for mission critical applications, so clearly the definition of mission critical depends on the context. In this case, for election software that is used only a few times a year and which is at risk to be hacked because such large sums of $ are spent to elect candidates, I think we want to adopt a more stringent definition of mission critical than may be the case for a drug store retail application.
While Microsoft is available for some tasks, it seems like there are vulnerabilities that are evident to even the casual observer. probably the most obvious difficulty is the constant upgrades to the system to close back doors. What may work is for a vendor to chose a version of Microsoft, go though some stringent testing at the Lab of their choice, and then put a hash on the version tested and make that what has to be used for the election. Then, to use an updated version of the OS, stringent testing has to happen again.
In this manner, the COTS can be controlled in some manner and the hash would need to be quadruple checked by outside observers.

Susan Eustis
WinterGreen Research
6 Raymond Street
Lexington, MA 02421
Tel: (781) 863-5078
Cell: (617)852-7876
----- Original Message ----- From: "Andrew Berg" <andrewb@votehere.com>
To: "Vincent J. Lipsio" <Vince@lipsio.com>; "Dick Johnson" <dick.johnson@Oracle.com>; "David L. Dill" <dill@CS.Stanford.edu>
Cc: <stds-1583-disc@ieee.org>; <cots@lipsio.com>
Sent: Friday, December 03, 2004 7:10 PM
Subject: Re: [ieee] Re: Re: Quick Re: to Dick Johnson's Re: COTS scope

Perhaps this is another example of "IEEE 1583 Specific Terminology" but are we all working from the same definition of "mission critical" here? Who certifies systems (or even components) for "mission critical" use? Lots of people and orginizations do lots of things which are clearly critical to their success on Microsoft OSs.

In previous jobs, I have had to perform maintanence and troublshooting on a couple of Pont of Sale systems that were built on Microsoft OSs using Microsoft Visual Basic and Microsoft Access. These are pretty clearly mission critical to the stores that use them. Some kind of certification probably would not have hurt them, but then probably would not have helped all that much, either.


On Fri, 3 Dec 2004 13:01:48 -0500 (EST), Vincent J. Lipsio <Vince@LIPSIO.COM> wrote:

My best understanding (and I may be wrong) is that Diebold software runs
on Windows2000 and that its GEMS software uses Access. I intended the
"DRE" reference to be generic and to include the voting system and not

Fair enough; however, 1583 does not in any way apply to the GEMS or to
anything else that is not at the polling place; therefore, it is simply
irrelevant to the COTS scope we are discussing.

Inspection of Windows CE would indeed be a major chore. But without

Which is why it is not used in any mission critical system to the best
of my knowledge. There are commercial RTOSes that are used in such systems,
and their source code has been reviewed so that they could be used in those
systems, and I see RTOSes advertise that they are certified for use in
certain types of systems, but to the best of my knoweldge, no Microsoft
OS has ever been certified for any mission critical use.

inspection, how can we have any assurance that there aren't deliberate
backdoors in the OS, for example?

Even in systems where malicious backdoors are not a concern, standards
for critical systems require all source code to be reviewed.

If one of the requirements of the P1583 standard is to accommodate
existing DRE designs, however flawed, we should say so explicitly in
the document.

As with price considerations, as I argued earlier this year: If the
purpose is to cap development costs, and that concern may preempt the
purposes stated in the scope of 1583, then that, too, should be added to
the scope.

Other industries are required to treat COTS as we are proposing and,
unless we are indeed trying to accommodate existing flawed designs,
we should also; else, we should say so, as Dave wrote, explicitly in
the document.