[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: COTS scope

What was the original of this thread?  I somehow missed it.

Stan Klein

On Sun, 2004-11-21 at 13:02, Vincent J. Lipsio wrote:
> Sandy,
> I agree with the sediment of the last paragraph of what you wrote, and 
> with parts of the first paragraph, but all in all, what you are proposing 
> will largely undo the COTS resolutions that were arrived upon by group 
> consensus involving hundreds, perhaps thousands, of person-hours of 
> discourse. It is my opinion that your concerns would be fully addressed by 
> developing those couple of sections.
> Some specific, albeit brief, comments:
> > This Standard recognizes that firmware is omnipresent in COTS
> > components selected by system designers to facilitate developing voting
> > systems.  For example, firmware exists in devices such as keyboards and
> > is integrated into electronic devices from printers to smart card
> > readers to touch screen controllers, etc.  Furthermore, modern
> > microprocessors are comprised of "microcode" that are components of the
> > CPU's instruction set and such microcode can be considered COTS
> > firmware.
> The problem here is that the devices referred to are COTS hardware and 
> should be tested as such.  The fact that they may contain firmware is not 
> relevant to the software standards of 1583.  The requested change is best 
> accomplished by explicitly stating that the scope of the software 
> specifications applies to the code on any and all processors or logic 
> arrays that contain the voting application or parts thereof.
> > Also, firmware and/or software exists in the main memory of
> > voting systems in the form of device drivers.  Operating systems also
> > contain COTS software.  Security experts are concerned that errors in
> > such COTS software/firmware can render voting systems insecure, the
> > example that is frequently given is the hypothetical printer driver
> > that changes every 10th printing of the number '8' into the number '5.'
> This omits many possible problems, among them, these two:
> 1) The sundry concerns addressed in the threat model.
> 2) The reality that operating systems and device drivers may have erratic 
> behavior, e.g., a run away pointer, that has not been documented, but that 
> may manifest itself when linked with a certain application.
> As an example of how the first item could be affected by a COTS software 
> component, please read the last link in:
> http://grouper.ieee.org/groups/scc38/1583/p1583_-_resource_list.htm
> "Reflections on Trusting Trust, Ken Thompson classic paper that shows how 
> viruses can be concealed even if source code is thoroughly checked"
> > It is impractical to certify that all instances of COTS firmware and
> > software meet the requirements of this Standard as specified below.
> Many real world standards and requirements contradict that sentence; you 
> are contending that class 3 FDA devices, in-flight software conforming to 
> DO-178B, and many DoD devices, among others, are "impractical"!
> > Previous versions of voting system standards provided a blanket
> > exemption of COTS software but this Standard requires certain tests to
> > be performed on any component containing COTS firmware/software.
> > All custom coded software (non-COTS) or firmware are subject to the
> > requirements of the rest of this Section 5.6.  COTS firmware/software
> > is also subject to this Section 5.6 unless at least one of the
> > following conditions applies:
> I am in favor of exempting all software under certain circumstances, or 
> rather, of subjecting software to less stringent standards under certain 
> circumstances, but I see no justification for treating COTS differently 
> than non-COTS.
> > Each COTS device or software program (including operating systems and
> > hardware drivers) that is part of the voting system is submitted with
> > a test plan and automated tests that exercise a set of inputs and
> > verify expected outputs.  The Test Plan shall describe the test and
> > describe the range of inputs and the range of outputs.  The source code
> > for the COTS test is subject to review as described in this Section
> > 5.6.
> I strongly disagree for the reasons mentioned above.  As long as the COTS 
> software components are linked with the application, they can interact in 
> ways that can not be predicted.  What is mandated in other critical 
> systems is necessary in voting systems that are trusted.
> > An ITA may develop its own Test Plan and test software for designed to
> > qualify COTS devices and in that case shall test the COTS device
> > according to that test plan.  A description of the ITA developed COTS
> > test shall be part of the ITA qualification report.
> This is not acceptable.  What you are specifying is that so long as a 
> piece of code is marketed not solely for use in voting equipment, then 
> 1583's software standard does not apply; rather, it is replaced by some 
> unspecified criteria, presumably much less stringent, that an ITA desires 
> to use.
> > The overall voting system includes an end-to-end audit capability that
> > provides each voter the ability to verify that his/her selections have
> > been recorded without error in a human-readable or machine readable
> > form on a permanent media such as a paper ballot.  Systems that provide
> > an audit via an Accessible Voter Verified Paper Audit Trail, or an
> > Automatic Ballot Marker (either an Electronic Ballot Printer or an
> > Electronic Ballot Marker) fall into this category.
> Here I basically agree.  I would be careful about the wording of "machine 
> readable"; it needs to be more verbose as it to leave open significant 
> loopholes as I read it.
> However, please be warned that attempts in the past to use less stringent 
> security or software standards on voter verified systems have been by 1583 
> rejected without significant discussion.  If you can reverse this 
> attitude, I applaud you.
> Type to you later,
> Vince Lipsio
Stanley A. Klein <sklein@cpcug.org>