[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: COTS scope



Sandy,
I agree with the sediment of the last paragraph of what you wrote, and 
with parts of the first paragraph, but all in all, what you are proposing 
will largely undo the COTS resolutions that were arrived upon by group 
consensus involving hundreds, perhaps thousands, of person-hours of 
discourse. It is my opinion that your concerns would be fully addressed by 
developing those couple of sections.

Some specific, albeit brief, comments:

> This Standard recognizes that firmware is omnipresent in COTS
> components selected by system designers to facilitate developing voting
> systems.  For example, firmware exists in devices such as keyboards and
> is integrated into electronic devices from printers to smart card
> readers to touch screen controllers, etc.  Furthermore, modern
> microprocessors are comprised of "microcode" that are components of the
> CPU's instruction set and such microcode can be considered COTS
> firmware.

The problem here is that the devices referred to are COTS hardware and 
should be tested as such.  The fact that they may contain firmware is not 
relevant to the software standards of 1583.  The requested change is best 
accomplished by explicitly stating that the scope of the software 
specifications applies to the code on any and all processors or logic 
arrays that contain the voting application or parts thereof.

> Also, firmware and/or software exists in the main memory of
> voting systems in the form of device drivers.  Operating systems also
> contain COTS software.  Security experts are concerned that errors in
> such COTS software/firmware can render voting systems insecure, the
> example that is frequently given is the hypothetical printer driver
> that changes every 10th printing of the number '8' into the number '5.'

This omits many possible problems, among them, these two:
1) The sundry concerns addressed in the threat model.
2) The reality that operating systems and device drivers may have erratic 
behavior, e.g., a run away pointer, that has not been documented, but that 
may manifest itself when linked with a certain application.

As an example of how the first item could be affected by a COTS software 
component, please read the last link in:
http://grouper.ieee.org/groups/scc38/1583/p1583_-_resource_list.htm
"Reflections on Trusting Trust, Ken Thompson classic paper that shows how 
viruses can be concealed even if source code is thoroughly checked"

> It is impractical to certify that all instances of COTS firmware and
> software meet the requirements of this Standard as specified below.

Many real world standards and requirements contradict that sentence; you 
are contending that class 3 FDA devices, in-flight software conforming to 
DO-178B, and many DoD devices, among others, are "impractical"!

> Previous versions of voting system standards provided a blanket
> exemption of COTS software but this Standard requires certain tests to
> be performed on any component containing COTS firmware/software.
> All custom coded software (non-COTS) or firmware are subject to the
> requirements of the rest of this Section 5.6.  COTS firmware/software
> is also subject to this Section 5.6 unless at least one of the
> following conditions applies:

I am in favor of exempting all software under certain circumstances, or 
rather, of subjecting software to less stringent standards under certain 
circumstances, but I see no justification for treating COTS differently 
than non-COTS.

> Each COTS device or software program (including operating systems and
> hardware drivers) that is part of the voting system is submitted with
> a test plan and automated tests that exercise a set of inputs and
> verify expected outputs.  The Test Plan shall describe the test and
> describe the range of inputs and the range of outputs.  The source code
> for the COTS test is subject to review as described in this Section
> 5.6.

I strongly disagree for the reasons mentioned above.  As long as the COTS 
software components are linked with the application, they can interact in 
ways that can not be predicted.  What is mandated in other critical 
systems is necessary in voting systems that are trusted.

> An ITA may develop its own Test Plan and test software for designed to
> qualify COTS devices and in that case shall test the COTS device
> according to that test plan.  A description of the ITA developed COTS
> test shall be part of the ITA qualification report.

This is not acceptable.  What you are specifying is that so long as a 
piece of code is marketed not solely for use in voting equipment, then 
1583's software standard does not apply; rather, it is replaced by some 
unspecified criteria, presumably much less stringent, that an ITA desires 
to use.

> The overall voting system includes an end-to-end audit capability that
> provides each voter the ability to verify that his/her selections have
> been recorded without error in a human-readable or machine readable
> form on a permanent media such as a paper ballot.  Systems that provide
> an audit via an Accessible Voter Verified Paper Audit Trail, or an
> Automatic Ballot Marker (either an Electronic Ballot Printer or an
> Electronic Ballot Marker) fall into this category.

Here I basically agree.  I would be careful about the wording of "machine 
readable"; it needs to be more verbose as it to leave open significant 
loopholes as I read it.

However, please be warned that attempts in the past to use less stringent 
security or software standards on voter verified systems have been by 1583 
rejected without significant discussion.  If you can reverse this 
attitude, I applaud you.

Type to you later,
Vince Lipsio