(i) Vendor responsibility. The vendor shall provide a list of all documentation and data required to be included as part of the independent review, and vendor technical personnel shall be available to the State Board during the performance of the functional configuration audit.

(ii) Technical data. The vendor shall provide the following technical data:

(iii) Audit procedure. The State Board, with the assistance of an independent testing authority, shall subject each voting system to a complete functional test, including but not limited to actual use testing of all components used by voters to enter or review votes. Additionally, the State Board and its independent testing authority shall review the vendor's test procedures and test results. This review shall include an assessment of the adequacy of test cases and input data to exercise all system functions and to detect program logic and data processing errors if such be present. The review shall also include an examination of all test data which is to be used as a basis for qualification.

(i) Vendor responsibility. The vendor shall provide a list of all documentation and data required to be audited by the State Board. Vendor's technical personnel shall be available to the State Board during the performance of the physical configuration audit.

(ii) Technical data. The vendor shall provide the following technical data:

(iii) Audit procedure. Required data items include draft and formal documentation of the vendor's software development program which are relevant to the design and conduct of qualification tests. The vendor shall identify all documents, or portions of documents, which the vendor asserts contain proprietary information not approved for public release. The State Board or its designee shall agree to use any proprietary information contained therein solely for the purpose of analyzing and testing the software and shall refrain from disclosing proprietary information to any other person or agency without the prior written consent of the vendor or a court order. The State Board or its designee shall review the vendor's source code and documentation to verify that the software conforms to the documentation, and that the documentation is sufficient to enable the user to install, validate, operate and maintain the voting system. The review shall also include an inspection of all records of the baseline version against the vendor's release control system to establish that the configuration, being qualified, conforms to the engineering and test data.

(i) Standard commercial, off-the-shelf production models of general purpose data processing equipment (PC's, printers, etc.) shown to be compatible with these requirements and with the voting system.

(ii) Production models of special purpose data processing equipment (scanners, bar code readers, etc.) having successfully performed in elections use and having been shown to be compatible with the voting system.

(i) System overview. The vendor shall identify the system hardware and the environment in which the software will operate and the general design and operational considerations and constraints which have influenced the design of the software.

(ii) Program description. The vendor shall provide descriptions of the software system concept, the array of hardware in which it operates, the intended operating environment, the specific software design objectives and development methodology and the logical structure and algorithms used to accomplish the objectives.

(iii) Standards and conventions. The vendor shall provide information which can be used as a partial basis for code analysis and test design. It should include a description and discussion of the standards and conventions used in the preparation of this specification and in the development of the software.

(iv) Specification standards and conventions. The vendor shall identify all published and private standards and conventions used to document software development and testing. Vendor internal procedures shall be provided as attachments to this software specification.

(v) Test and verification standards. The vendor shall identify any standards or other documents which are applicable to the determination of program correctness and acceptance criteria.

(vi) Quality assurance standards. The vendor shall describe all standards or other documents which are applicable to the examination and testing of the software, including standards for flowcharts, program documentation, test planning and test data acquisition and reporting.

(vii) Operating environment. The vendor shall provide a description of the system and subsystem interfaces at which inputs, outputs and data transformations occur. It shall contain or make reference to all operating environment factors which influence the software design.

(viii) Hardware constraints. The vendor shall identify and describe the hardware characteristics which influence the design of the software, such as:

(ix) Software environment. The vendor shall identify all compilers, assemblers, or other software tools to be used for the generation of executable code and a description of the operating system or system monitor. This section shall also contain an overview of the compile-time interaction of the voting system software with library calls and linking.

(x) Interface characteristics. The vendor shall describe the interfaces between executable code and system input-output and control hardware.

(xi) Software functional specification. The vendor shall provide a description of the overall functions which the software performs in the context of its mode or modes of operation. The vendor shall also describe the capabilities and methods for detecting and handling exceptional conditions, system failure, data input/output errors, error logging and audit record generation and security monitoring and control.

(x) Configurations and operating modes. The vendor shall describe the various software configurations and operating modes of the system; such as preparation for opening of the polling place, vote recording and/or vote processing, closing of the polling place and report generation. For each software function or operating mode, a definition of the inputs (characteristics, tolerances or acceptable ranges) to the function or mode, how the inputs are processed and what outputs are produced (characteristics, tolerances or acceptable ranges) shall be provided.

(xiii) External files. In the event that external files are used for data input or output, the definition of information context and record formats shall be provided. The vendor shall also describe the procedures for file maintenance, access privileges and security.

(xiv) Security. Security requirements and security provisions of the system's software shall be identified for each system function and operating mode. The voting system must be secure against attempts to interfere with correct system operation. The vendor shall identify each potential point of attack. For each potential point of attack, the vendor shall identify the technical safeguards embodied in the voting system to defend against attack, and the procedural safeguards that the vendor has recommended be followed by the election administrators to further defend against that attack. Each defense shall be classified as preventative, if it prevents the attack in the first place; detective if it allows detection of an attack; or corrective if it allows correction of the damage done by an attack. Security requirements and provisions shall include the ability of the system to detect, prevent, log and recover from the broad range of security risks identified. These procedures shall also examine system capabilities and safeguards claimed by the vendor to prevent interference with correct system operations. The State Board, with the assistance of its ITA, shall conduct tests to confirm that the security requirements of this Part have been completely addressed. Notwithstanding any other provisions of this Part, the State Board shall determine whether all or a portion of such security requirements and security provisions shall be available for public inspection, but shall exclude any information which compromises the security of the voting system.

(xv) Programming specifications. The vendor shall provide an overview of the software design, structure and implementation algorithms. Whereas the functional specification of the preceding section provides a description of what functions the software performs and the various modes in which it operates, this section should be prepared so as to facilitate understanding of the internal functioning of the individual software modules. Implementation of functions shall be described in terms of software architecture, algorithms and data structures and all procedures or procedure interfaces which are vulnerable to degradation in data quality or security penetration shall be identified.

(xvi) Test and verification specifications. The vendor shall provide a description of the procedures used during software development to verify logical correctness, data quality and security. This description shall include existing standard test procedures, special purpose test procedures, test criteria and experimental design and validation criteria. In the event that this documentation is not available, the qualification test agency shall design test cases and procedures equivalent to those ordinarily used as a basis for verification (see below).

(xvii) Qualification test specification. The vendor shall provide a description of the specification for verification and validation of overall software performance, including acceptance criteria for control and data input/output, processing accuracy, data quality assessment and maintenance, exceptional handling and security. The specification shall identify specific procedures by means of which the general suitability of the software for elections use can be assessed and demonstrated. The vendor's specification and procedure shall be used to establish the detailed requirements of the tests described in "Laboratory Environmental Test Procedures for Hardware and Software" of this standard.

(xviii) Acceptance test specification. The vendor shall provide a description of the specification for installation, acceptance and readiness verification. This specification shall identify specific procedures by means of which the capability of the software to accommodate actual ballot formats and format logic, and pre-election logic, accuracy and security test requirements of using jurisdictions may be assessed and demonstrated. The vendor's specification shall be used to establish the detailed requirements of the tests described in "Laboratory Environmental Test Procedures for Hardware and Software" of this standard performed to evaluate the adequacy of the vendor's procedures and it shall be suitable for inclusion in the regulations and procedures of user counties when preparing for the conduct of actual elections.

(xix) Appendices. The vendor shall provide descriptive material and data supplementing the various sections of the body of the software specification. The content and arrangement of appendices shall be at the discretion of the vendor. Topics recommended for amplification and treatment in appendix form include:

(i) This documentation shall contain a complete physical and functional description of the equipment and a theory of operation which fully describes the electrical and mechanical function of the equipment, how the processes of ballot handling and reading are performed, how data are handled in the processor and memory sections, how data output is initiated and controlled, how power is converted or conditioned and how test and diagnostic information is acquired and used.

(ii) A complete parts and materials list shall be provided which contains sufficient descriptive information to identify all parts by type, size, value or range and manufacturer's designation.

(iii) Technical illustrations and schematic representations of electronic circuits shall be provided with indications of all test and adjustment points and the nominal value and tolerance or waveform to be measured. Fault detection, isolation and correction procedures or logic diagrams shall be prepared for all operational abnormalities identified by design analysis and operating experiences.

(i) The vendor shall identify all corrective and preventive maintenance tasks, including the calibration of the system, as appropriate, and the level at which they shall be performed. Levels of maintenance shall include operator tasks, maintenance personnel tasks and factory repair.

(ii) Operator tasks shall be limited to the activation of controls to identify irrecoverable error conditions and to the replenishment of consumables such as printer ribbons, paper and the like.

(iii) Maintenance personnel tasks shall include all field maintenance actions which require access to internal portions of the equipment. They shall include the conduct of tests to localize the source of a malfunction; the adjustment, repair or replacement of malfunctioning circuits or components and the conduct of tests to verify restoration to service.

(iv) Factory repair tasks shall be minimized, and repairs shall be made on site whenever reasonably possible. Factory repairs shall only include complex and infrequent maintenance functions which require access to proprietary or to specialized facilities and equipment which cannot be obtained by the county board.

(v) The vendor shall identify by function all personnel required to operate and support the system. For each functional category, the number of personnel and their skills and skill levels shall be specified.

(vi) The vendor shall specify requirements for the training of each category of operating and support personnel, including but not limited to voters, poll workers, and elections staff. The vendor shall prepare all materials required in the training activity and shall provide or otherwise arrange for the provision of as many qualified instructors as are necessary to properly and fully train said personnel in each category.

(vii) The vendor shall recommend a standard complement of supplies, spares and repair parts which will be required to support system operation. This list shall include the identification of these materials and their individual quantities and sources from which they may be obtained. The vendor shall supply, at vendor's expense, any special tools required to repair or maintain the equipment.

(viii) The vendor shall provide complete instructions for all methods of voting which voters may use to cast their vote, including instructions on entering and changing votes, write-in voting, verifying votes and accepting the cast votes. Written and audio instructions shall be provided in each language in which voting shall occur within the State.

(i) The purpose of this test is to provide, in a simulated election day environment, a public demonstration of the usability and accuracy of such systems or machines.

(ii) Vendor must submit, in a quantity to be determined by the State Board, additional voting systems or equipment that have been submitted for certification. These additional systems or equipment will be returned to the vendor upon the completion of voter demonstration testing.

(iii) The State Board shall make available to the public, all non-proprietary documentation submitted by the vendor.

(i) The State Board shall escrow a complete copy of all certified software that is relevant to functionality, setup, configuration, and operation of the voting system, including but not limited to, a complete copy of the source and executable code, build scripts, object libraries, application program interfaces, and complete documentation of all aspects of the system including, but not limited to, compiling instructions, design documentation, technical documentation, user documentation, hardware and software specifications, drawings, records, and data. Documentation shall include a list of programmers responsible for creating the software and a sworn affidavit that the source code includes all relevant program statements in low-level and high-level languages. The State Board may require that additional items be escrowed. If any vendor contracts to escrow additional items, those items shall be subject to the provisions of this section.

(ii) The vendor shall immediately notify the State Board of any change in any item required to be escrowed by subparagraph (i) of this paragraph, and shall provide an updated version for deposit.

(iii) The chief executive officer of the vendor shall sign a sworn affidavit that the source code and other material in escrow is the same being used in its voting systems in the State. The chief executive officer shall have an ongoing obligation to ensure the statement is true.

(iv) The vendor shall promptly notify the State Board and each county board using its voting system of any decertification of the same system in any state, of any defect in the same system known to have occurred anywhere, and of any relevant defect known to have occurred in similar systems.

(v) Upon completion of testing, reports shall be produced by the ITA and State Board staff, and a recommendation either for or against certification shall be made to the State Board's commissioners.

(vi) If the State Board determines that a system meets the requirements of this Part, and is determined to be suitable for use by voters, it shall certify such system. A notice of provisional certification shall be prepared and forwarded to the vendor, forthwith. The vendor shall ensure that the voting system's software has been escrowed as set forth in Election Law, section 7-208, and the vendor has updated any affidavit and complied with the affidavit requirements, as set forth in section 6209.4(h) of this Part.

(vii) Upon compliance with the provisions set forth above, a notice of certification shall be awarded to the vendor. Notice of such certification shall also be provided to all county boards.

(viii) If the State Board fails to certify a system, the vendor shall be so notified.

(ix) Once a certified system is selected for purchase by a county board, that system's software shall be provided to the county board by the State Board, and not the vendor.