Re: Quick Re: to Dick Johnson's Re: COTS scope

[Dick Johnson <dick.johnson@oracle.com>]

OK...the issue at hand  includes only the polling place pieces of the integrated voting system.  Fine.  But I believe the issue is still with us as to (1) whether Microsoft will provide its Windows source code for inspection and (2) whether we are prepared to distinguish between specific voting applications and the COTS software which they may need to run.  Has this been decided, or not?  Certainly the voting system application itself ought to be inspected, in my opinion.

And is it in order to suggest that there be some kind of Open Test opportunity to determine whether a given black box (voting application plus operating system and drivers) performs in a way that meets what will be the IEEE standard?  Anybody care about test coverage/uncoverage?  But maybe IEEE should approve only Open Source operating systems such as Linux  if Windows can't be inspected; maybe Open Test would do.  Somehow, I think that somebody's ox will get gored before this is resolved; either the proprietary OS people or the proprietary test and certification people might be uncomfortable with any solution to this issue.

I can understand the white box not being feasible for COTS, but certainly there should be some open aspect to the tests (at least a subset of them) which assess the performance of the combined polling place system.   Private and proprietary test organizations could be requested to run the Open Tests as well as those proprietary ones hidden from public view.  Again, I would direct you to the IEEE Software Engineering Standards, where these issues are amply addressed.  Boris Belzer writes a very good book on testing for any who are interested.

Finally, I would find it distressing to just paper over the issues we have been discussing and issue a lowest common denominator standard so as to accommodate all existing voting systems of any description.  That would hardly be a contribution to increasing public confidence in our electoral process.  Rather, we should build a standard to which all voting systems, new and yet to be developed, might be fairly compared.  It may not be necessary to inspect voting station operating systems, and it may be beyond our scope to consider any aspect of the voting system outside the polling place, but I feel a responsibility as an IEEE member to do more than just lobby for the lowest possible standard.

This is my personal and professional opinion, and it does not necessarily reflect the opinions of my employer.

-- Dick

Richard C. Johnson, Ph.D.
Applications Architect
Oracle Corporation
631-689-3736

Vincent J. Lipsio wrote:

My best understanding (and I may be wrong) is that Diebold software runs
on Windows2000 and that its GEMS software uses Access.  I intended the
"DRE" reference to be generic and to include the voting system and not
    

Fair enough; however, 1583 does not in any way apply to the GEMS or to
anything else that is not at the polling place; therefore, it is simply
irrelevant to the COTS scope we are discussing.


  

Inspection of Windows CE would indeed be a major chore.  But without
    

Which is why it is not used in any mission critical system to the best
of my knowledge.  There are commercial RTOSes that are used in such systems,
and their source code has been reviewed so that they could be used in those
systems, and I see RTOSes advertise that they are certified for use in
certain types of systems, but to the best of my knoweldge, no Microsoft
OS has ever been certified for any mission critical use.


  

inspection, how can we have any assurance that there aren't deliberate
backdoors in the OS, for example?
    

Even in systems where malicious backdoors are not a concern, standards
for critical systems require all source code to be reviewed.


  

If one of the requirements of the P1583 standard is to accommodate
existing DRE designs, however flawed, we should say so explicitly in
the document.
    

As with price considerations, as I argued earlier this year:  If the
purpose is to cap development costs, and that concern may preempt the
purposes stated in the scope of 1583, then that, too, should be added to
the scope.

Other industries are required to treat COTS as we are proposing and,
unless we are indeed trying to accommodate existing flawed designs,
we should also; else, we should say so, as Dave wrote, explicitly in
the document.


Vince