Resolutions for COTS Comments for Draft 5.0 of IEEE P-1583

--- Sklein-051, wfw - 001, Dill-7, df1, schneidewind - 002, PPLX-001, and Lipsio-6D

Remove "COTS" from the definitions section.

In "Abbreviations and Acronyms" add to the definition of "COTS" this text: "(subsystems or components; software, electrical, electronic, mechanical, et cetera)."

<<<------- Comments requiring changes to Section 4 ------->>>

--- MercuriD50 - 013 (formerly mercuri-034)

At the end of section 4.6, under the bullet "software and firmware", add these sub-bullets:

Description of how an update is to be performed, together with guidelines on how certification may be affected, should there be, declared or discovered, any defect in the voting system's software, hardware, or firmware, or any COTS products used in the development of the system that could compromise its operation as an election device.

Notice that should such a defect as described in the previous bullet be found, the owners of the product should be notified and device should, or at least may, be decertified until the correcting update shall have been applied.

(and some additions to section 7.12.1)

<<<------- Comments requiring changes to Section 5 ------->>>

--- RGH 072

In the second paragraph above clause 5.4.1, change "a documented record of performance under conditions defined in the Standards" to "have been tested by their suppliers to the rigor mandated in these Standards, with documentation of such testing being supplied with the COTS product."

--- Lipsio-12

In paragraph 5 of 5.1.1, change "COTS products may" to "COTS products shall". Before the sentence beginning "Notwithstanding the fact that system certifiers ..." insert this sentence: "In any instance, such evaluations shall comply with section 4.3.11 ("Previously developed or puchased software") of IEEE Std 1228-1994, "IEEE Standard for Software Safety Plans".

--- Simons-002

In 2nd to last paragraph of 5.1.1, replace the sentence, "The security countermeasures implemented by an IT system typically use functions of the underlying products and depend upon the correct operation of those products and their security functions."

with

"All such underlying products, the correct operation of which the system relies upon, shall be thoroughly tested and, because of the potential risk of malicious code, their source code reviewed ."

--- VCW-02

In 2nd to last paragraph of 5.1.1, delete second space before "voting system"; pass on to the editing committee with the suggestion that they excise all redundant spaces within sentences.

--- Corry-023 and RGH 006

Split the last paragraph of section 5.1.1 into 2 paragraphs; begin the 2nd paragraph with the sentence: "Guidance on how to securely configure COTS products requires ...". Note that the same paragraph will be modified because of the following change.

--- RGH 007 and Lipsio-16

In section 5.1.1, on page 20, 3rd paragraph, 3rd sentence, after "properly installed and configured" add the words "with the latest security patches (available at the time of system design or update; not necessarily to be construed as requiring retroactive installation of patches in tested, released versions of code)"

--- Lipsio-7A

In paragraph 7 of section 5.1.1, change "vendors" to "COTS vendors or voting equipment vendors".

--- Lipsio-14

In paragraph 7 of section 5.1.1, replace the text: "The voting system vendor must provide a method to assess the impact of COTS updates on the voting system"

with

'In accordance with Annex D ("V&V of reusable software") of IEEE Std 1012-1998, "IEEE Standard for Software Verification and Validation", using an updated version of a COTS product shall be treated no differently than any other software update, and full regression testing and validation of the software shall be required.'

--- Lipsio-44

At the end of section 5.1.1, insert this paragraph:

The use of COTS software products shall be guided by section 4.3.11 ("Previously developed or purchased software") of IEEE Std 1228-1994, "IEEE Standard for Software Safety Plans" and IEEE Std 982.1-1998, "IEEE Stand Dictionary of Measures to Produce Reliable Software". COTS software products should also be subject to the specifications of IEEE 1008-1987 (R1993). "IEEE Standard for Software Unit Testing".

--- Alice-001, Sklein-007, Sklein-057, and MercuriD50-064

In section 5.6.1.1, replace the paragraph beginning with "Compliance with the requirements ..."

with:

"Compliance with the requirements of the software standards is assessed by several formal tests, including code examination. Source code shall be provided in human readable form; when code is generated by a tool, the term "source code" shall be construed to refer to the input to that tool. Source code inspection shall not be limited to compliance with the explicit standards specified below in subsequent sections; inspection shall also endeavor to preclude the code containing pathogenic functions. Pathogenic functions include:

All testing shall be performed on target hardware executing code compiled or otherwise made from the inspected source, or verified to be exactly binary identical with code so made.

(and some additions to section 7.12.1)

--- Sklein-044

In the second sentence of section 5.6.2.3, after "security requirements defined in" insert "Section 5.1.3.1 and".

--- Lipsio-89

Delete the first sentence of section 5.6.2; begin that section with "This section provides standards for voting systems with regard to:"

--- Lipsio-3E

In section 5.6.2.2, change the second and third sentences to: "Interpreted code is prohibited unless the run-time interpreter and the tool chain used to generate the interpreted code have been validated."

--- Lipsio-45

In section 5.6.2.3, paragraph 1, replace:

"However, COTS software is not required to be inspected for compliance with this requirement, but must be the most recent version ..."

with

"COTS software shall be the most recent version ..."

--- RGH 117, PPLX-035, Sklein-045, Lipsio-43, and schneidewind – 005

Change the first two sentences of section 5.6.2.3 to the following: Voting system application software shall be designed in a modular or object oriented fashion. COTS software components shall incorporate all security patches released as of the time the software is built and shall comply with the security requirements defined in the Software Security sections (5.1.3.1 and 5.1.3.4). The vendor shall, in compliance with the requirements of 5.1.3.1, document how each COTS component has been defended against the threats identified in 5.1.2.3 (A-1), (A-3), (B-1) and (B-2), such as by testing, external controls, et cetera. COTS systems or components shall be documented by their suppliers to have been tested to at least the same rigor as required of voting devices as specified in this Standard. The COTS supplier, or the vendor using the COTS component in their product, shall deliver documentation of this testing. The said documentation may include formal certification (such as Common Criteria) or other recognized independent evaluation at levels appropriate to the voting system evaluation. The COTS systems or components either shall not be changed from documented, certified, tested versions, or, if a software component, shall be able to be built by the ITA from provided COTS modules. COTS components not meeting these criteria shall be tested in a like manner to any other component.

<<<------- Comments requiring changes to Section 6 ------->>>

--- Corry-139

In section 6.4.4.1, delete the first sentence of the second paragraph.

--- schneidewind - 006 and Lipsio-4B

Replace the first sentence of the second paragraph of section 6.4.4.1 with: "Systems designed exclusively with system-level COTS hardware whose manufacturer can document that the system has been tested to at least the same rigor as required of voting devices as specified in this Standard and whose configuration has not been modified in any manner need not be subjected to this segment of hardware testing."

--- Lipsio-4D, MercuriD50 - 022, schneidewind – 007

Eliminate paragraphs 2, 3, and 4 of section 6.6.2, replacing them with: -COTS software components shall comply with section 4.3.11 ("Previously developed or purchased software") of IEEE Std 1228-1994, "IEEE Standard for Software Safety Plans".

<<<------- Comments requiring changes to Section 7 ------->>>

--- MercuriD50 - 013 (formerly mercuri-034)

At the end of section 7.12.1, add these bullets:

f) Include a procedure to inform equipment owners should there be, declared or discovered, any defect in the voting system's software, hardware, or firmware, for any COTS products used in the development of the system that could compromise its operation as an election device.

g) Include a procedure to inform equipment owners should there be discovered subsequent to the certification of the software, any pathogenic function, as defined in section 5.6.1.1.

h) Include a procedure for when a defect of the sort described in the previous bullet is found, to initiate amending the V&V procedures so as to flag the found defect, and to subsequently perform full regression testing.

i) Include a procedure to provide owners of the product with the required update subsequent to the retesting mandated in the previous bullet.

<<<----------- Comments not requiring editing ------------>>>

--- Dill-35

Irrelevant in light of resolution proposed for Lipsio-4D, MercuriD50 - 022, and schneidewind – 007

--- Sklein-056

Pretty much everyone agrees that mandating VVPB is out of scope, so it's being rejected. Please note for the record Dave Dill's comment that it's "... a high-level policy question that should be discussed before line-by-line editing ."

--- Lipsio-80

Reject. The commentators are split, but those who agreed with my comment didn't notice that it was out of scope, which upon looking at it again, I agree it is.

--- Simons-017

Reject. Seems to use "open source" to mean publicly inspectable source, which is dealt with elsewhere.