CITIZENS' OVERSIGHT PROJECTS (COPs) 
 
Completed on January 15, 2010 
Show Table of Contents Hide Table of Contents 

Table of Contents 
�	COPs Review of the procedures of the San Diego County Registrar of Voters in the November 2008 Election 
�	Executive Summary & Introduction  
�	Status of the Investigation 
�	Methods of the Investigation 
�	Structure of this Document 
�	Next Steps 
�	Acknowledgments 
�	Documents Referenced  
�	Interaction Documents 
�	Abbreviations and Terminology used in this report and by the RoV: 
�	Canvass Procedure Summary  
�	Preparation 
�	Election Day -- At the Polling Place 
�	Tally Center Operations 
�	Vote-By-Mail and Provisional Ballot Processing 
�	Manual Tally Procedure 
�	Canvass Reconciliation 
�	Five Percent Review  
�	Issues of Concern Categories  
�	Lack of Documentation  
�	Issues related to this category: 
�	Unscanned Ballots  
�	Chain of Custody  
�	Audit Log Discrepancies  
�	Manual Tally Procedure  
�	Audit Trail  
�	Issues Summary  
�	Issues related to RoV Procedures 
�	Issues related to our research project 
�	End Notes 

 
Executive Summary & Introduction  
This report is the result of a project initiated by Citizens' Oversight Projects (COPs) in 2008 to document and review the procedures used by San Diego County Registrar of Voters (RoV) to process elections and create their certified result. It addresses chain-of-custody issues, whether adequate in-process records are maintained, and describes serious issues of concern, including the loss of ballots, unexplained central tabulator crashes, and ballot box tampering. 
This document enumerates the extent of this review, summarizes the results, and provides recommendations for improvement in procedures and law. Our data collection effort and the entirety of our interaction with the RoV has been open to the public and can be reviewed on the CitizensOversight.org website. This report may be reviewed on the Internet(1) for the most recent revision. We invite comments, corrections, and enhancements to our review. 
Despite a return to paper ballots, systems used to count and report the vote -- both computerized and paper-based -- are still subject to election fraud vulnerabilities. San Diego County is no exception. There are numerous opportunities to improve the quality and confidence level of the result, as well as to reduce the cost to process each election. Our review exposed a number of procedures are either technically incorrect, do not follow the law, or provide an opportunity for compromised RoV officials to manipulate the outcome of the election. The most obvious deficiency of the RoV is the lack of a comprehensive written procedures document. Such written procedures are always the first step in any quality assurance program. 
There are a large number of voters who remain disenfranchised due to the sloppy procedures of the RoV. Although the RoV is aware of these issues, they continue to maintain that there are no deficiencies. For example, the Post-Election Manual Tally (PEMT) detected problems in 29% of the precincts tested. The RoV ran the ballots back through the scanners and confirmed the errors, but then stated that there were no errors at all, because the rescan cleared them up. However, this only confirmed that the election would likely have 29% of the precincts with errors (478 precincts) and about 500 ballots lost. 
The lead investigator in this work is Raymond Lutz(2) , founder and coordinator of COPs. Mr. Lutz has extensive experience in the document processing field, in quality assurance procedures, has worked with standards organizations in the printer/scanner/fax area and has a Master's of Science degree in Electronics Engineering. Mr. Lutz was a candidate in the November 2008 election for the 77th State Assembly District seat in the California State Legislature and has proposed the Open Canvass method(3) to eliminate the possibility of electronic election fraud. Although related to this report, changes in procedure and law suggested by this report are not directly related to the Open Canvass method, and most are necessary regardless of the method of counting the ballots. 
Status of the Investigation 
Although the RoV was generally cooperative and courteous in complying with our requests, in several cases they refused to answer questions, limited our ability to scan some documents, and did not provide any contact with Premier Election Solutions (formerly Diebold Systems) to follow up on central tabulator crashes and to get relevant central tabulator documentation, per their suggestion. Therefore, we do not view this report as "final" since these and other questions are still outstanding. However, the egregious nature of the deficiencies is such that publishing the report was justified now rather than waiting for the full extent of the research to be completed. This report can also serve as a starting point for future reviews and should be of value as a template for review of other counties' procedures. 
Since our review project was not sanctioned by any county, state, or federal department, we had no ability to demand compliance with our requests for information and therefore, our conclusions may be incomplete in inaccurate to some degree. However, any difficulty we had in reviewing the election system only points out that the process is not open and the RoV only reluctantly returned documents, and then only if it seemed they had no other choice. 
Methods of the Investigation 
The following methods were used in our interaction with the RoV. Videos and documents that relate to this research are listed at the end of this report. 
�	  
�	See: Election-night processing on the Feb 5, 2008 presidential primary election.(4) This video includes the processes used to process the election as well as the Post-Election Manual Tally. This one-hour video is a good starting place for anyone curious about the procedures used by the RoV to process our vote on election night. The procedure is summarized for purposes of this report in a later section, Canvass Procedure Summary. The video does not, however, include every procedure. Mail and provisional ballot processing as well as all procedures prior to the ballots reaching the central Tally Center are not included in the video. 
�	 
�	See: Nov. 21, 2008 Meeting with Deborah Seiler (video)(5) After this initial meeting, the Registrar refused to allow us to record our meetings, and so we discontinued that approach and instead use written requests only. We also found that video recording was less than optimal because of the additional work required to turn the verbal comments of the RoV into written documentation and the perception that we were trying to catch the RoV off-guard, which we were not. 
�	 
�	Responses to these requests were not always within the time period defined by the Act (ten days), and at times, it was necessary to appear before the County Board of Supervisors to request a response. That said, the responses were generally thorough and most of our questions have been answered. RoV staff members were always courteous and respectful. 
�	 
�	These questions were sometimes skirted and avoided with the note that responses were not required under the CPRA, but cooperation improved as we continued our investigation. For example, we asked “Why were the 'lost' ballots not counted by the scanning equipment? What was the source of the error?” and the question was not answered because it is not a CPRA request. (This remains a serious issue covered by this report). 
�	 
�	Central to our project was an independent sample of five percent of the precincts. The exact method of choosing the 5% sample (85 precincts) and the data collected are described in detail later in the report. This sample was used as a means to detect errors by the RoV over and above the Post-Election Manual Tally, which is the first 1% of our 5% sample (comprising 20% of our sample). Thus, the 1% selected by the RoV was 17 precincts and we added an additional 68 precincts, however, we were unable to access the actual ballots, so it was impossible for us to perform a manual tally. 
Structure of this Document 
There are three main components of this document: 
�	Procedure Summary - provides an overview of the entire procedure used by the San Diego County Registrar Of Voters to process the election 
�	Issues of Concern Categories - Issues are grouped into categories to provide context and further detail regarding each category of concern. 
�	Issues Summary - Issues are listed in full in numerical order, with Questions for RoV and actions for the legislature described. Also, each issue is incorporated (redundantly) into the text. When each is first referenced in the textual description, the issue text will be expanded in full. Subsequently, if the same issue is dealt with again, only a brief description of the issue will be provided. 
Each Issue of Concern has several components: 
�	The first section of an Issue includes a number, a title, a brief description, and a longer description. 
�	The second section describes any questions or comments we intend to make to the Registrar Of Voters as a follow-up to this report. 
�	The third section details actions requested for the state legislature so as to improve state law. 
There are two types of issues, the first is numbered starting with 'A' which means it is an actual issue, whereas those starting with 'C' indicate areas where we need to do more work to collect information in our project. 
Next Steps 
It is important to list a number of desired outcomes from the publication of this report. As mentioned, Citizens' Oversight Projects (COPS) has no legal authority to obligate the RoV, the County of San Diego, nor the State of California to take any steps as a result of this document, however,  
Response by the Registar of Voters (RoV) 
We have made quite a number of assertions throughout this document regarding the activities of the RoV. Some of these assertions may be inaccurate, and we invite the RoV to suggest corrections and to comment profusely. 
Essentially all the of the issues, with only a few exceptions, have specific questions to the RoV. Instead of repeating nearly all of the issues here, we will simply suggest that the RoV attempt to address each and every issue enumerated in Issues Summary, particularly with respect to the section "Questions to the RoV". In each case, we urge the RoV to place each issue in context by reading the descriptions of the larger issue in the Procedure Summary and Issues of Concern Categories rather than reading only the text of the specific request to the RoV included with each issue. 
Steps by the Secretary of State 
We have not explicitly detailed steps for the Secretary of State but we encourage them to review the IssuesSummary and comment on each issue, particularly to point out if they can play a role in improving the performance of the RoV. 
Steps by the State Legislature 
For each of the Issue of Concern in the IssuesSummary, we have explicitly detailed any changes in elections code that may be appropriate. We hereby request that the California State Legislature consider our recommendations carefully, and put forward bills to correct the behavior of the San Diego County RoV and apply our results to other counties as well. 
Instead of repeating nearly all of the issues here, we will simply suggest that the those in the Legislature attempting to refine these laws review each and every issue enumerated in Cops Canvass Report Issues Summary, particularly with respect to the section of each issue "Law". In each case, we urge the those reviewers to place each issue in context by reading the descriptions of the larger issue in the Procedure Summary and Issues of Concern Categories rather than reading only the text of the specific request. 
Acknowledgments 
The following people were most helpful in the generation of this report: 
�	Raymond Lutz - primary investigator and report author. 
�	Linda Poniktera - participated in meetings, election night oversight, manual tally oversight 
�	Sal Magellenez - participated in meetings, election night oversight 
�	Brina Rae Schuchman - Save-R-Vote - participated in meetings, election night oversight 
�	Lisa Davis - Cops member, entered almost all the scanner tape data. 
�	Valerie Sanfilippo - Assisted with roster inspection 
�	Joe Sain - Assisted with Ballot Statement data entry 

Documents Referenced  
The following documents were provided by the RoV as a response to our request or obtained from other sources. These documents will be referenced in later sections by name. "RoV Procedure Documents" specify procedures used in each election while "RoV Election Documents" are specific to this election. 
The casual reader of this report may move directly to Procedure Summary while keeping in mind the fact that these documents and abbreviations are defined here. 
�	Voluntary Voting Systems Guidelines(6) were authored by the Technical Guidelines Development Committee (TGDC), a committee authorized under the HELP America Vote Act (HAVA) of 2002, and researchers at the National Institute of Standards and Technology (NIST) for the Election Assistance Commission (EAC). These guidelines were authored primarily to guide voting machine manufacturers in their design of voting systems. 
�	Logic & Accuracy Testing Procedure(7) An RoV Procedure Document that specifies how to confirm basic operation of the electronic scanning equipment. 
�	Ballot Order Procedure(8) An RoV procedure document that specifies how the number of ballots is determined. 
�	Ballot Order Spreadsheet(9) An RoV Election Document that lists the quantity of ballots ordered for each precinct and supplemental orders. Includes Sequence Number to Precinct Number conversion. 
�	Ballot Inventory sheet(10) An RoV Election Document. This is a sample of the document sent to each precinct leader to verify the count of blank ballots received. 
�	Street Indexes(11) An RoV Election Document that lists voters by address and each is checked off as they vote. About half of our five percent sample were scanned but we discovered they were incomplete as most polling places did not continue to complete these after 6pm on election day. 
�	Ballot Statements(12) An RoV Election Document. These were Initially withheld by the RoV due to legal concerns of showing signatures. (Requested in C00022). These have been scanned by the Registrar with the signatures concealed. 
�	Collection Center Logs(13) Reports from 80 collection centers were scanned from RoV on 2009-03-23. Each report handles 21 precincts, on the average. No logging of seal conditions or serial numbers. 
�	Collection Center Seal Report(14) Describes 6 seals that are missing or wrong 
�	Tally Center Log(15) Use by incoming inspectors at the Tally Center. Describes 99 seals broken or wrong, 6% error rate. 
�	Security Seal Standards(16) includes ISO/PAS 17712 - Freight container seal standards. 
�	Batch box label (Used as tracking device when in facility). 
�	Scanner Tapes for our 5% Sample(17) These pdf files contain three scanner tapes each, and were manually scanned by COPs volunteers. 
�	One-Percent Manual Tally Procedure(18) An RoV Procedure Document 
�	San Diego County Manual Tally Report for 2008-11-04 election(19) This five-page report is extremely deficient compared with the recommendation of the Secretary of State, which would result in about 800 pages. This report includes NO vote totals and only a count of the ballots. 
�	Audit Log (of central tabulator)(20) An RoV Election Document produced by the GEMS Central Tabulator. 
Interaction Documents 
The following documents record our requests and responses by the RoV: 
�	C00014: Letter to Registrar of Voters requesting Statement of Votes Cast, 2008-10-03(21) We requested that the RoV create two copies of the scanner tapes, as specified by law, so that we could start our review of the election right away. They refused, forcing us to wait until the election was certified before we could start to review the scanner tapes. 
�	C00015: Letter to Secretary of State requesting support of our request to Registrar of Voters requesting Statement of Votes Cast, 2008-10-08(22) We forwarded our letter to the RoV (C00014) to the Secretary of State in an effort to persuade the county to comply with access to the scanner report tapes prior to the end of the election. This request was refused, with the rationale being that state law applies only in the precinct and they are not required to create a second scanner tape prior to election certification. 
�	C00016: Letter to RoV: Manual Tally Report Questions, 2008-11-20(23) This three-page request was submitted prior to the closing day of the canvass period and pointed out that they were not using the appropriate format for their PEMT report. 
�	C00016 R: Response by County Counsel regarding Nov. 20 Letter to the RoV, 2008-11-25(24) This response was the least cooperative and would not answer simple questions regarding why ballots were missing in the PEMT report. 
�	C00017 - Letter to San Diego Registrar of Voters regarding Election Procedures (2008-12-02)(25) This five-page request was submitted on the closing day of the canvass period. 
�	C00017 R - Response to C00017 by San Diego Registrar of Voters regarding Election Procedures (2008-12-16)(26) Includes Ballot Order Procedure, One Percent Manual Tally Procedure, Collection Center Comments on Carton Seals. 
�	C00019 - Letter to San Diego Registrar of Voters regarding Election Processing (2009-01-08)(27) This 19-page request was a followup to C00017, and was submitted twice, the second time (C00021) in an abbreviated format to aide in processing. 
�	C00019 R - Response to C00019 and C00021 by San Diego Registrar of Voters regarding Election Procedures (2009-02-24)(28) 
�	C00022 - Questions to Registrar of Voters on Nov, 2008 Election (Part 3) (2009-09-01)(29) Request for Ballot Statements, Street Indexes, Question about ballot counts, and Premier contact. 
�	C00022 R: Response to C00022 by Registrar of Voters (2009-09-09)(30) Answered some questions. Ballot Statements took several weeks to finally get in full, no contact to Premier Election Solutions provided. 
Abbreviations and Terminology used in this report and by the RoV: 
If you do not find the definition here, please also see this reference:
http://www.eac.gov/vvsg/g/definitions-of-words-with-special-meanings 
�	Collection Centers - 80 centers throughout the county where ballots are collected from individual precincts, each averaging 21 precincts. 
�	CVR - Cast Vote Record - Archival record of all votes produced by a single voter. Cast vote records may be in electronic, paper, or other form. Electronic cast vote records are also called ballot images. In San Diego County, the durable paper ballot is the CVR. 
�	DRE - Direct Recording Equipment - Voting machines that directly record the vote of the voter without using paper ballots. These machines have superior usability for disabled voters, but are inherently subject to election fraud risks, and therefore are no longer used except for one unit per precinct for use by disabled voters, and their vote is transferred to paper ballots and processed with the rest. 
�	IVVR - independent voter-verifiable records - all voting systems must include a vote-capture device that uses independent voter-verifiable records (IVVR). For example, durable paper ballots that are scanned represents a ballot that can be easily read by the voter and interpreted by machines. 
�	PAV - Permanent Absentee Voter - Deprecated. Now VBM. 
�	PI - Precinct Inspector - the primary official at each polling place. 
�	RoV - San Diego County Registrar Of Voters 
�	Seq# - The precinct sequence number, from 0001 to 1650. This number is used internal to the RoV and relates to one polling place. 
�	SPUW - Supply Pick-Up Weekend - The weekend prior to election day when supplies are distributed to polling places by having polling place workers pick up their supplies. 
�	SoS - California Secretary Of State - This elected office is responsible for overseeing the elections process. 
�	Tally Center - The central office of the RoV where the main task of scanning ballots and compiling the final certified canvass occurs. 
�	VBM - Vote-by-Mail. Previously called PAV, permanent absentee voters. 
�	Voter Data File - Information about each registered voter is maintained in a database, including whether the voter has voted, whether they have requested a VBM ballot, their digitized signature, etc. 

Canvass Procedure Summary  
This section of the Cops Canvass Report describes the overall process used to conduct elections in San Diego County and determine the result. The California Secretary Of State (SoS) made an extensive Top-to-Bottom Review(31) of all elections procedures in California, and this review resulted in changes to the procedures used in San Diego County, including a change from touch-screen machines provided by Diebold Systems (Now Premier Election Solutions) to paper ballots that are scanned in the RoV central office using scanners originally intended for use at polling places (also from Diebold Systems). 
Our description below is our best attempt to document the procedures used by the RoV based on our observations, questions, and the few documents that do exist. Unfortunately, there is no comprehensive document maintained by the RoV that describes their procedures as a whole, although there are several documents that do describe some portions of their process. As a result, we had to determine the procedures used by the RoV using video recordings of the actual process and through meetings and Q&A exchanges. We encourage the RoV or anyone else to submit corrections to this description so that it accurately reflects current practice. We regard the absence of a procedure document as a severe deficiency (See Issue A001: "Lack of Comprehensive Procedure Documentation") and show-stopper for any comprehensive quality assurance program. 
We also encourage the reader to view the Video of the February 5, 2008 Presidential Primary Canvass(32) to see the handling of the ballots once they enter the RoV office. 
The procedure description below does not go into great detail about DRE touchscreen machines because they are a very small part of the total election ballot count, amounting to only about 750 ballots county wide. Those ballots are currently converted to paper ballots by hand by RoV staff prior to scanning. Most precincts did not use touchscreen machines at all, with, in our sample, only 15 of 85 precincts using them at all (17%) and comprising only 49 votes of more than 34,000, or 0.14%. Prior to the top-to-bottom review by the Secretary Of State, this was the most significant vulnerability. Thanks to the changes made in that review, we can essentially ignore this issue and concentrate on other issues. 
 
Because of its importance, the documentation issue is disclosed here. 
�	: Lack of Comprehensive Procedure Documentation -- RoV apparently does not maintain any comprehensive procedures documentation.
COPs requested procedure documentation on a number of occasions but were not successful in obtaining any comprehensive documentation for election procedures. There was inadequate documentation for (at least) the following procedures: 
�	Supply Pickup Weekend (SPUW) 
�	Blank Ballot Inventory Verification 
�	"Office Use" section of Ballot Statement (See Issue A008) 
�	Collection Center operation 
�	Tally Center operations 
�	Incoming Inspection 
�	Memory Card Pairing 
�	Scanning 
�	Memory card Reading 
�	Processing of VBMs & Provisionals 
�	Reconciliation Procedure (See Issue A026) 
Comprehensive procedure documentation is the first step to achieving high quality levels, as required by quality management standards such as ISO-9004-2 (2000) "Quality Management and Quality System Elements - Part 2: Guidelines for Services" as used in private industry. The lack of written procedures by the San Diego County Registrar Of Voters is a significant deficiency of the first step to achieving high quality levels, and points to the fact that local and state government institutions likely have no quality management system standards equivalent to those used by private industry. 

The two most important aspects of this standard for this application is  and  
�	See Lack of Documentation 
Preparation 
Logic Testing Procedures 
�	 Logic & Accuracy Testing Procedure(33) 
�	 (None) 
This procedure is limited to testing the ballot scanners to confirm that each type of ballot and each race are correctly tallied. This is not a stress test nor a quality assurance test but a procedure to verify that the scanners will count the correct candidate or issue on each of the many ballot types. These testing procedures do not test any human elements, such as procedures in the polling place, accuracy of the feeding process, or handling within the walls of the RoV apart from the scanning process. However, this procedure is essential to confirm that a darkened bubble in a specific place on a specific ballot type results in a vote for the correct race. 
 
�	: Logic Testing Witnessing 
Logic Testing Procedures not witnessed nor video recorded by our project team. 
Blank Ballot Printing and Allocation 
�	 Ballot Order Procedure(34) which describes how the number of ballots is determined based on the number of registered voters in that precinct, including vote-by-mail voters, and predicted language variations. Each ballot costs $0.28 and are supplied in groups of 50. 
�	 Ballot Order Spreadsheet 
Ballots are ordered and printed in the quantity as specified by the "Ballot Order Spreadsheet" and according to the Ballot Order Procedure so that there are sufficient ballots at each polling place in each language supported by the RoV. The ballots are counted and grouped by precinct by RoV staff in preparation for pickup. 
 
�	: Ballot Order Chain of Custody 
We did not review the process of maintaining control of blank ballots that are not distributed to precincts and oversight of the printer vendor. There is a concern regarding chain of custody of blank ballots. Is there a vulnerability to fraud based on accessing blank ballots and then removing and replacing completed ballots with those blank ballots which have been voted according to the desires of the instigators? 
Supply Pickup Weekend (SPUW) 
�	 None 
�	 (None) 
Supplies are distributed to the precincts during the weekend prior to election day. These include: 
�	Temporary Voting Booths 
�	Cardboard Ballot Boxes 
�	Touchscreen voting machines 
�	Blank Ballots 
�	Sign-in Rosters 
�	Blue, newly registered voters 
�	White, regular voters marked to indicate if the voter is being sent their ballot by mail. 
�	Pink, voters who have not voted for a while but are still registered. 
�	Peach, blank sheets for provisional voters 
�	Street Indexes 
�	Voters names sorted by address 
�	BallotStatements 
�	Peach envelopes for provisional ballots 
�	Tamper-evident seals 
�	Ballot Inventory Sheet - Two part form to confirm number of ballots distributed (see below). 
 
�	: SPUW Observation 
Supply pickup weekend not witnessed nor video recorded. 
Blank Ballot Inventory Verification 
�	 (None) 
�	 Ballot Inventory sheet(35) which indicates the precinct number, the sequence number for that precinct, and the quantity of ballots sent to that precinct, by language version, completed by RoV staff. 
The Precinct Inspector is instructed to count the ballots, verify the quantities listed, sign and date the two-part "Ballot Inventory sheet", send the white copy to the Registrar of Voters office and retain the yellow copy. 
Although RoV staff attempts to accurately count the blank ballots, mistakes do occur, and this verification by the Precinct Inspector should improve reliability of the count of blank ballots. 
 
�	We received a completed sample Ballot Inventory Sheet. 
�	: No Ballot Inventory Report -- There is no report generated regarding whether any errors occurred during SPUW, including the count of ballots. 
Election Day -- At the Polling Place 
�	 Precinct Worker Instructions -- we never got a copy of this. RoV claimed it was on their website. 
�	 
�	Sign-in Rosters 
�	Street Indexes 
�	Ballot Statements 
 
�	: Precinct Worker Instructions 
RoV indicated that precinct worker instructions were available for review on website but those are not really available to the public. We need to review these instructions. 
Voter Sign-In 
Voter provides name and address to poll worker and they look up the voter in the Blue, White, or Pink rosters. Poll worker validates voter by matching the name and address. 
�	If the voter is found on the Blue, White, or Pink rosters 
�	and they are not marked as a Vote-by-Mail (VBM) voter on the rosters (typically by an "M" or similar designation), then they simply sign the roster. 
�	Otherwise, if they are a VBM voter, then 
�	If they have their VBM ballot to surrender, they sign the Blue, White, or Pink roster over the "M" (or similar) designation. 
�	Otherwise, if they do not have their VBM ballot to surrender, they do not sign the Blue, White or Pink roster and are processed as a provisional voter. 
�	If the voter is not found on the Blue, White, or Pink rosters, 
�	the poll workers is first supposed to try to help the voter find the correct precinct for voting. Sometimes it is an adjacent polling place (even in the same building) or nearby. This is very important because the ballots at the precinct may not be the correct ones for the voter and may not include some races that they are allowed to vote for while the provided ballot may include some races that are inappropriate for that voter. 
�	otherwise, and in the case of a VBM voter with no ballot to surrender, the voter signs the peach roster and complete a provisional ballot, including the envelope. 
Note: Signatures on the Blue, White or Pink rosters are not routinely validated or reviewed. However, since precincts are relatively small and locating in neighborhoods, precinct workers may know the voters. 
Poll workers are also supposed to update the Street Index which lists the voters by address. However, the updating of these sheets is not required after 6pm and therefore many voters, perhaps 30% are not included on the sheets, and it is not useful as an audit tool. 
If the voter has a completed VBM ballot, it can be dropped off at the polling place. There is no logging procedure for these ballots at the precinct level and they are not counted and the voter does not sign any roster. 
 
We reviewed the sign-in rosters for a portion of our 5% sample and counted the signatures. We were unable to scan these documents. 
�	: VBM Ballots Not Logged at Precinct 
VBM Ballots were not logged in at the precinct and apparently not placed in sealed containers. It is not necessary to sign anything to drop off your VBM ballot at the precinct or RoV office, and this is a deficiency. 
Voting 
Each voter that properly signs in is provided with an appropriate ballot according to their language and party designation, if appropriate, and directed to a voting booth. After voting, each voter places the ballot in the ballot box or at least witnesses their ballot being placed into the box. The first voter of the day is supposed to witness an empty box prior to the seal being placed on the box. 
If the voter chooses to use a touchscreen machine, then they use that machine and their vote is later converted to a paper ballot. 
If the voter makes a mistake, they can return the ballot to the poll workers and this ballot is considered "Spoiled" and they are then supplied with a new blank ballot. (Not sure if there is tracking of spoiled ballots, i.e. separate roster where the voters signs to confirm that they spoiled their ballot.) 
End of the Day 
At the end of the day, the precinct workers open the ballot box, count the ballots, and complete the Ballot Statement. The ballot statement includes a worksheet to allow the poll workers to account for the ballots, as follows: 
�	Blank Ballots Received 
�	Total unused Ballots 
�	Total voted paper ballots 
�	Total Provisional ballots 
�	Total spoiled ballots 
�	Sum of the above three items, should equal the ballots received. 
�	Signatures 
�	Blue pages signatures 
�	White pages signatures 
�	Pink pages signatures 
�	Peach Pages Signatures 
�	Total signatures, sum of the above four items. 
�	Number of touchscreen ballots 
�	Voted Ballots 
�	Voted Paper Ballots 
�	Voted Provisional Ballots 
�	Touchscreen ballots 
�	Total voted ballots, sum of the above three items, should equal the total signatures. 
After completing the ballot statement, the ballots are placed in cartons suitable for transporation and sealed with tamper-evident seals. DRE memory cards are placed into plastic envelopes and sealed. 
 
Our oversight project included scanning 5% of the Ballot Statements for the November 2008 election, which can be reviewed here: Ballot Statements(36) These scanned Ballot Statements have had the poll workers signatures redacted. 
�	: No Seal Serial Number on Ballot Statements 
Serial numbers of the tamper-evident seals are not included on Ballot Statements. 
Documenting serial numbers of seals is a requirement for a workable seal management strategy. 
�	: Ballot Statements - Few balanced 
Only a few of the Ballot Statements had figures that balanced, and only one precinct had all the ballots correctly scanned. This is a significant issue that is separately treated in this report as a separate Issues of Concern Category. (See Unscanned Ballots) 

According to federal election guidlines, disenfranchisement of even one voter, or more than one lost ballot in 808,407 ballots cast due to equipment failure is not allowed. Tracking this problem starts with the sign-in rosters and BallotStatements completd at the polling place.

Polling workers are volunteers and are not specifically suspect of malfeasance, but improved training and intuitive reconciliation procedures should be established to minimize the discrepancies in the count of ballots. 
�	: Ballot Statements - Removed Write-ins not Documented -- Ballot Statements did not include a count of the number of write-ins and damaged ballots removed. Precinct workers are requested to look through the ballots and remove all write-ins and damaged ballots and place these into a separate group, which is to be banded. The incoming inspection activity has two checkoffs that relate to this: 1) write-ins removed and 2) write-ins banded. However, no one counts these so we know how many have been removed. This should be recorded on the Ballot Statements. In addition, these ballots (nearly 9,000 ballots) are withheld and not processed for an extended period of time. During this time, the "precincts reporting" figure should not claim that the precinct has been fully reported. 
�	: Ballot Statements - Office Use Section Not Explained -- Ballot Statements have an "Office Use Only" section does not include an explanation to allow the public to understand how it is being used.
The processing of our elections should be an open process with no secrets. The "Office Use" section could be easily explained on the Ballot Statement form. The RoV should explain the use of this section and make that explanation available for anyone to read. 
Collection Centers 
There are 80 collection centers around the county and each one collects ballot cartons and other materials from approximately 21 precincts each. Workers at the collections centers complete log sheets that include the time and number of ballot cartons collected from each precinct. 
 
�	In the November 2008 election, 6 seals were noted as wrong or broken and reported to us. 
�	: Collection Center Observation 
We did not witness nor record activities at any collection centers. 
�	: Collection Center Seal Logs Lack Serial Numbers 
At collection Centers, they are supposed to inspect the tamper-evident seals, but there is no place on the log sheet to document the seal numbers nor to note whether the seals were intact. 
Only 6 tampered seals were detected at collection centers while 99 tampered seals were detected at the tally center. Documenting serial numbers of seals is a requirement for a workable seal management strategy. 
Tally Center Operations 
The collection centers then transport the ballot cartons and other materials to the central RoV facility, known as the "Tally Center". 
Reception of Ballots 
 
Precinct Seq# Tracking Chart 
The tally center receives ballot cartons from vans arriving from collection centers on election night at about 10pm. As the cartons are unloaded, the precinct sequence number is scanned using a bar code reader. Internal to the tally center, each one of the 1650 precincts is designated with a sequence number (Seq#) which ranges from 0001 to 1650 and generally corresponds to a single precinct. 
The RoV maintains a tracking chart which is a matrix of sequence numbers displayed on a computer screen and projected on one wall. Each Sequence number turns green when the ballots have unloaded and physically controlled by the tally center. 
�	: Tally Center Incoming Reception Observation 
We witnessed Tally Center Incoming Reception but did not video record it directly. 
�	: Tally Center Reception Count Misleading 
The count of precincts that have been received by the Tally Center on election night is frequently cited by news media as the number of "Precincts Reporting." However, the ballots may not be even counted yet, and there are a vast number of VBM, provisionals, write-ins, etc. that have not been processed either. Therefore, this number should not be made public. Instead, the figure should be the total number of scanned and accumulated in the central tabulator divided by the total number of ballots received both on election night and as VBM ballots. Also see Issue A007 regarding the removed write-in and damaged ballots that are not included in the count until much later. 	
Incoming Inspection 
 
Incoming Inspection Area at Tally Center 
�	 None. 
�	 Tally Center Log 
The first step in the process is to go through incoming inspection. At this point, the cartons are opened and their contents inspected. The inspectors record the following on log sheets, with one line per precinct: 
�	Seq# 
�	Wrong Seal 
�	Box Not Sealed 
�	Write-Ins Removed 
�	Write-Ins not banded 
�	Damaged Removed 
�	Spoiled Removed 
The inspectors look through the ballots quickly, usually flipping through them to see if write-ins, damaged, and spoiled ballots have been separated. They do not attempt to count the ballots at this point in any fashion, and there is no recording of VBM or Provisional ballots. 
It is important to note at this point that there is an expectation that Write-Ins will be Removed and Banded. The actual processing of these ballots appears, based on other evidence, to be handled separately from the other ballots. In C00022R-2.1.4, the RoV says that if voters vote for write-in candidates their ballots are tabulated separately after a determination is made whether they voted for a qualified write-in candidate. 
 
�	This process was video recorded. 
�	: Tally Center - Write-Ins Removed and Not Counted 
We are not sure why the write-in ballots are removed permanently prior to scanning rather than scanning first and processing later or quickly processing them and uniting them with the rest of the precinct. It makes reconciling the number of ballots problematic because there is no notation as to how many "write-ins" are removed nor what is done with them. Approximately 9,000 ballots appear to be removed at this point. The scanners do tally the number of write-ins and there are many write-ins that were noted on the scanner tapes. We consider this an open issue as we do not have adequate information to understand exactly what happened, and the number of ballots -- 9,000 -- is certainly significant. We are not sure how the write-in ballots, received on election day, are processed. Only 146 ballots were removed and documented as "damaged" and the balance of nearly 9,000 ballots removed at this point are not documented.

Ballots should not be taken from the boxes prior to scanning. Currently, many counts do not match (9,000+ ballots missing), but are magically corrected later. The following figures should all match (or an explanation regarding how they are reconciled should be provided): 
�	The number of nonprovisional and nontouch-screen signatures on the Sign-In Rosters. 
�	The count of physical ballots in the ballot box at the end of election night. 
�	The number "cards cast", as reported on scanner tapes. 
�	The number "# CARDS", as reported by the audit log for that precinct. 
�	The number "Ballots cast, Polling", as reported in the final canvass report. 
See also Issue A007, Issue A013, Issue A015. 
�	: Tally Center - Damaged Ballots 
On the Tally Center Log sheet, there is a checkbox for "Damaged Removed." However, we don't know why any ballot would become damaged and not be treated as "spoiled." 
A spoiled ballot, by definition, is one that is filled out incorrectly or damaged and a new ballot is issued to the voter. Why would there be any ballots that are "damaged" and a new ballot not issued to the voter? However, 146 ballots were separated as "damaged" instead of "spoiled" in the November, 2008, election. 
�	: Tally Center - 99 Seals broken or wrong 
In the November 2008 election, Tally Center Logs documented 99 precincts that had wrong or broken seals, a 6% rate, and there was no investigation. There were no exception documents provided for our review describing why the seals were incorrect or broken. Seal documents do not include a serial number. Only a cursory inspection was performed by elections officials, and there was no attempt to validate the count of ballots as reported by the precinct. Why was it the case that 99 seals were detected as wrong or broken at the Tally Center while only six were detected by the Collection Centers? Documenting serial numbers of seals is a requirement for a workable seal management strategy. See the Chain of Custody for more detail on this issue. 	
Memory Card Pairing 
 
Pairing Memory Card with Precinct Ballots 
All the ballot cartons from each precinct are gathered together and transported to another area where workers find the scanner memory card for that precinct. The first box is opened and the memory card placed inside. 
 
�	This process was video recorded. 
�	Comment: The act of opening and closing the box sometimes takes quite a bit of time and is perhaps superfluous. 	
Scanning 
 
150 manual-feed precinct scanners 
 
Diebold Precinct Scanner 
Once the memory card is mated with the ballots, they are transported to one of 150 workers and 150 scanners. These scanners were originally intended to be used at each precinct, and are therefore manually fed devices with a single memory card appropriate for that precinct. The scanners are outfitted with a printer that prints on a continuous roll of paper tape, about 3 inches wide, using a dot-matrix printer. 
The process of scanning consists of the following steps: 
�	Insert memory card into the scanner. 
�	Zero the scanner and document it on the report tape. 
�	Feed ballots one at a time through the scanner. These can be fed at the rate of about six ballots per minute. 
�	Create a final report on the paper tape, tear off the tape and sign it. 
�	Although a second tape was required by law when these scanners were used in each precinct, that tape being displayed outside the precinct door for anyone to inspect, the printing of the second tape is no longer a practice when they are used in the Tally Center. However, the RoV makes the original tape available for inspection, and we scanned tapes from 5% of the precincts in our review. 
�	Re-box the Ballots and Report Tape, these are moved to the secure archival area. 
�	The report tape, DRE memory, and various seals are later moved to individual manila envelopes and separated from the ballots. 
�	Remove the memory card from the scanner and give it to the worker who transports these to the central tabulator. 
 
�	: No Comparison of Ballot Count 
There is no comparison of the number of ballots scanned, as reported on the report tape, with the number of ballots counted at the precinct nor by the count as read by the central tabulator.
Comparing the count of ballots as counted by polling place workers with the number scanned can eliminate the problems documented by the PEMT - One Percent Manual Tally, i.e. missing ballots. Only 4 of 85 precincts in our 5% sample scanned the number of ballots (Ballots Cast) that were counted by precinct workers. Comparing the count may have eliminated this problem (although this problem is more likely due to the removal of write-ins.) 

Workers do not routinely compare the count of ballots, in terms of the actual count and the count reported on scanner report with the count of ballots input to the GEMS central tabulator. Our project reviewed the count of ballots printed on scanner tapes with the audit report in our 5% sample and found no discrepancy. However, there have been recently reports of problems in Ohio using the same equipment. Ohio Voting Machines Contained Programming Error That Dropped Votes (Washington Post)(37) This article said: A voting system used in 34 states contains a critical programming error that can cause votes to be dropped while being electronically transferred from memory cards to a central tallying point, the manufacturer acknowledges. The problem was identified after complaints from Ohio elections officials following the March primary there, but the logic error that is the root of the problem has been part of the software for 10 years, said Chris Riggall, a spokesman for Premier Election Solutions, formerly known as Diebold. 
�	: Scanner Tapes Not Accessible 
Scanner report tapes are no longer available early in the canvass. According to state election code, a second copy of the scanner report was required to be printed when these scanners were used in each precinct, that report being displayed outside the precinct door for anyone to inspect. However, the printing of the second report is no longer a practice when they are used in the Tally Center. This means we have to wait until the election is certified before we can start to review the election, which is an artificial restriction. The RoV did made the original reports available for inspection, and we scanned the reports from 5% of the precincts in our review.

See also Issue Issue A023: Documents Automatically Processable. 
�	: Scanner Tapes Difficult to Process 
The audit trail provided by the current RoV procedures include cash-register style paper tape which is difficult to process and sometimes impossible to read. Transparency of our RoV procedures demands an audit trail that is much easier to process. Scanner reports should be produced on paper that is easily processable (such as cut-sheet letter paper) and with higher resolution printers that do not suffer from feed errors or light printing. These reports then can be easily scanned by oversight teams. (This suggestion is not essential if the audit log is improved or audit devices installed.) See also Issue Issue A023: Documents Automatically Processable, Issue A024: Scanner Tapes Not Accessible, Issue A018: Audit Log Insufficient, and Issue A029: Audit Devices Should be Allowed. 
�	: Archival Procedures Lacking 
There are no check-in and check-out procedures in the secure archive area and only documentation describing where the materials are in the storage facility. There is no way to know who accessed the records to determine if any may have been modified. There are no seal serial numbers maintained on the documentation. 
See also Issue A001: "Lack of Comprehensive Procedure Documentation". 	
Memory Card Reading 
 
Reading data from the memory cards into the GEMS Central Tabulator 
The memory cards are transported to the central tabulator room. Several scanners are connected with serial communication cables (RS-232 or similar) to the Diebold Systems (now Premier Election Solutions) "GEMS" Central Tabulator. The workers insert a memory card into one of those scanners with the vote captured by one of the 150 scanners and press a button to have the central tabulator read the results of the scanning process. Thus, these scanners are only used to read the memory cards and are not used to scan ballots. The memory card is then sorted by Seq# and stored. These memory cards are not maintained as a permanent record of the election. The audit record available in the Central Tabulator does not provide sufficient detail to reconstruct the data from these cards. 
 
�	: Scanner cards not Archived 
Memory cards used to capture the vote from each scanner are not maintained as a permanent record of the election. 
It is too difficult to access data to reconstruct the election. At present, the only option for oversight groups is to scan the report tapes produced by the ballot scanners, and these do not include any VBM or provisional ballots. If the memory cards were archived, then oversight groups would be able to reconstruct the election results based on those cards without needing to resort to scanning the scanner reports. Such oversight is a check of election manipulation within the GEMS central tabulator. Note: It is preferrable and less expensive for elections officials for oversight groups have access to a complete audit log. (i.e. including all information downloaded from the scanners, not just the count of ballots) and including the VBM and Provisional ballots scanned. (See Issue A018: Audit Log Insufficient and Issue A029: Audit Devices Should be Allowed) 
�	: Audit Log Insufficient 
The audit record available from the GEMS Central Tabulator does not provide sufficient detail to reconstruct the election.
The audit log should be improved to provide an exhaustive list of vote counts for every race and every precinct. This would result in a much larger file, but file size should not be cited as a factor limiting the provision of this information. COPs asked for documentation regarding the availability of additional reporting options, but the RoV said they did not have any documentation.

The following concerns exist: 
1.	The entire election must be included in the audit log. The log we were given did not include initial clearing and pre-election voting intervals. 
2.	The initial clearing command(s) must appear in the log if we are to know that the system was correctly initialized. 
3.	The RoV should have complete documentation for the audit log and for the GEMS central tabulator. 
4.	Every crash and error should be investigated and explained. 
5.	More robust entries are desired to allow an independent reconstructions of the election from the audit log. 
�	Ballot counts should be provided for VBM and Provisional processing. (See Issue Issue A020: No Log of changes to the Election Management System) 
�	Complete (exhaustive) report of each download to the GEMS central tabulator should appear in the log, including the total of votes for each race. 
6.	The log file should have a sequential number on each of the lines to eliminate the possibility that the audit log could be altered by election workers. In the log provided to COPs, we noticed anomalies that indicate that the log may have been tampered with, specifically the intervals when the the log had STOP or START entries without the other associated entry adjacent to it. 
7.	The RoV was apparently in violation of the requirements that they remake all touchscreen ballots into durable paper ballots, since on several occasions, votes were transferred directly from DRE equipment. 
8.	Oversight groups should have access to the audit log as the election is processed, and not be required to wait for final election certification. Nothing in the audit log can change once it is produced, and therefore, it is a public document that should be disclosed immediately, and not withheld until the election is certified, even though the log will become more extensive as the election is processed. 
Providing a complete audit log is the easiest, least expensive way to document the election while providing sufficient detail for oversight groups. 
�	: No Comparison of Ballot Count 
There is no comparison of the number of ballots scanned, as reported on the report tape, with the number of ballots counted at the precinct nor by the count as read by the central tabulator. Details... 	
Vote-By-Mail and Provisional Ballot Processing 
�	 None 
�	 Voter Data File, a database. (No access transaction log available.) 
The Vote-by-Mail (VBM) and provisional ballots are processed similarly but it is substantially different from election-day ballot processing. VBM ballots are completed at home by the voter and mailed in, or may be dropped off at the RoV office or any polling place on election day. Provisional ballots are completed if the voter chooses to vote in a precinct where the voter's name does not appear on the sign-in roster, or if the voter is a VBM voter and wants to vote on election day but does not have a ballot to surrender to polling place workers. 
 
�	: Lack of Comprehensive Procedure Documentation -- RoV apparently does not maintain any comprehensive procedures documentation. Details... 
�	: VBM & Provisional Process Witnessing 
The Vote-By-Mail and Provisional Ballot Processing procedure was not recorded on video and our description was generated solely through Q&A with RoV staff. 
�	: Tally Center - Write-Ins Removed and Not Counted 
We are not sure why the write-in ballots are removed permanently prior to scanning rather than scanning first and processing later or quickly processing them and uniting them with the rest of the precinct. Details... 
RoV Commented on Provisionals in C00022R-2.1:
Provisional ballots are processed according to the type of provisional: i.e., mail ballot voters who did not have their mail ballots to surrender, “visiting voters who are at the wrong precinct: or “fail safe” voters who moved before the close of registration and failed to re-register. The ballots are just regular polls ballots that are placed in a provisional envelope UNLESS the voter is returning a voted mail ballot and did not have their mail ballot ID envelope, in which case they use a provisional envelope.) 	
VBM Incoming Reception 
Envelopes are scanned into the "election management system." The receipt of the mail ballot is recorded, and a daily report is issued. This creates an opportunity for voters to validate the receipt of their ballot, via the RoV website or by telephone. If the voter drops off their voted mail ballot at the polls, the ballot is delivered to the ROV office and processed in the same fashion as mail ballots received prior to the election. 
 
�	: No Log of changes to the Election Management System 
The Election Management Systems is updated when a VBM ballot is received, however, there is no audit log of each transaction. A transaction log of VBM and provisional ballots must be made available for inspection. At this time, we are not aware of such a log. The Voter Data File is modified directly with no audit trail that can be inspected. If we had such an audit log, we could reconstruct the number of ballots actually received and compare that with the number of ballots scanned and entered into the central tabulator. At present, due to this lack of paper trail, such oversight is impossible. 	
VBM Signature Verification 
Each envelope is inspected prior to removing the ballot. The signature is compared with the signature supplied on the Voter Registration Form. Voter Data is also checked to see that voter has not already voted at the precinct. There is a three-level review for non-matching signatures: the operator, a four person board, and a division manager. Records are updated if there is a challenge. 	
VBM Precinct Sorting 
Prior to removing the ballot from the envelope, they are sorted by precinct. 	
VBM Ballot Removal 
Ballots are removed and placed into ballot cartons which have mixed precincts. 
 
�	: VBM Ballots are not counted 
VBM Ballots are not counted after being removed from the envelope and before submitting for scanning, and there is no attempt to confirm the count of ballots as they are scanned. The RoV claims that workers count batches of ballots as they are submitted to the scanner and checking the total scanned, but this was not observed as actual behavior of staff doing the scanning and it would only guard against misfeeds, and would not provide a positive check that the total number of ballots submitted for scanning were actually scanned. This is a chain of custody concern.

We recommend the following procedure: 
�	Maintain a count of ballots that are processed at each step. 
�	A batch of ballots sent to be scanned should have a document that travels with it (a "traveler") that documents the number of ballots in the batch. 
�	The operator of the scanner should be able to start a scanning process and zero the count of ballots. Until the correct number is confirmed, the votes from that batch should not be added to the canvass total. 
�	The operator proceeds to scan the batch. 
�	If the count does not match what is contained in the batch, the discrepancy should be documented and the batch recounted and/or rescanned, and then the reason for the discrepancy resolved. 
�	When the ballot counts match, the batch is approved by the operator, and the totals sent to the central tabulator for accumulation. 
�	The audit log should document the exact number of ballots processed in the batch and the total votes in each race processed. 	
VBM Ballot Scanning 
Each filled ballot carton (mixed precincts) is transported to any one of 15 scanner stations for processing. Ballots are sorted to precinct within small batches. When ballots are scanned they may be or may not be in precinct order. GEMS Central Count option is used, with each scanner directly connected to the central tabulator computer. No zero tape is created and no report tape is created. There is no comparison of ballot count with the count of ballots submitted. However the scanner operator pre-counts the ballots as they run them. During counting they post the count as they go through the scanner. Example: Count 10 ballots, run these through the scanner and at the end 10 ballots will display on the scanner display, count another 10 and scan these and the display will indicated 20 ballot scanned, etc. 
 
�	: VBM Ballots not precounted during scanning 
We were told that the scanner operator pre-counts the ballots as they run them, but this did not occur. The RoV said that during counting of VBM ballots, they post the count as they go through the scanner. Example: Count 10 ballots, run these through the scanner and at the end 10 ballots will display on the scanner display, count another 10 and scan these and the display will indicated 20 ballot scanned, etc. However, our video did not show any counting of ballots in batches to confirm the number counted as they claimed. This procedure is not documented in any procedure document and is not implemented.

Although this is a discrepancy from what we were told, the procedure of precounting ballots in batches does not do enough to provide positive control and chain of custody of those ballots. Instead, positive control should be maintained with a traveler document which proceeds with the ballots documenting the count of ballots in each batch. See Issue A021 for details. 	
VBM Ballot Archival 
Ballots are returned to their cartons and archived in that fashion, and are not united with other ballots from the same precinct. 	
Manual Tally Procedure 
 
Manual Tally 
�	 One-Percent Manual Tally Procedure(38) 
�	 Tally Sheets, Computer report, PEMT Report. 
To validate the requests provided by electronic equipment, the SoS requires that the RoV conduct a Post-Election Manual Tally (PEMT). After the SoS completed the top-to-bottom review of state election systems in 2007, the RoV was directed to conduct a 10% PEMT procedure. However, the County of San Diego declined to do so, and met the SoS in court over this issue. At this point, only the One-Percent Manual Tally is performed. You can view this process in the video of the Feb 5, 2008 presidential primary. 
There are a number of important issues of concern we have with this process. These are detailed in a separate Issues of Concern Category Manual Tally Procedure Review in this document, and will not be further processed here. 	
Canvass Reconciliation 
�	 None (Issue A026: No Reconciliation Procedure) 
�	 None (Issue A027: No Operator Notes, Issue A028: No Reconciliation Report) 
The RoV must somehow decide if the canvass is complete based on whether the counts and vote reconcile. This procedure is mentioned by the RoV but is undocumented. 
 
�	: No Reconciliation Procedure 
The RoV goes through a lengthy process of reconciliation which has been referenced in many conversations with them, but is not documented in any procedure document, nor is there sufficient notes taken regarding what is done. There are many re-scans of precincts all the way to the very last day of the canvass period, but as mentioned, no notes as to why these re-scanning operations are being performed. Because there is no written procedure, no notes kept, and no final report, there is no confidence that the canvass is in fact completed from a technical standpoint. 
See Also Issue Issue A001: Lack of Comprehensive Procedure Documentation, Issue A027: No Operator Notes, Issue A028: No Reconciliation Report 
�	: No Operator Notes 
A log of all operator activities should be maintained, with rationale for re-scans, etc. At present, no operator log is maintained and 133 re-runs occurred with no explanation. 
�	: No Reconciliation Report 
A reconciliation report should be provided. This report should show the number of sign-in signatures, the number of ballots scanned and tallied, the number of provisional ballots accepted and rejected, the number of VBM ballots received at the precinct and the number processed, etc. such that the figures shown above all balance. See also Issue Issue A026: No Reconciliation Procedure. 
�	: No Seal Tampering Report 
A seal report should be generated that explains all tampered seals. At present 99 seals show tampering but there is no report nor any action taken to eliminate the tampering. It is insufficient to simply report the number of seals that were broken or incorrect. The report should also indicate why the tampering occurred and how it can be eliminated in the future. The fact that 99 seals showed tampering is an unbelievable figure, particularly since there was no follow-up or explanation. Also, since no serial numbers are recorded, the number of tampered seals may be much larger. 	

Five Percent Review  
Citizens Oversight conducted a review of RoV procedures by gather all available and relevant documentation on a 5% random sample of the 1650 precincts, i.e. 85 precincts. 
Random Selection of 85 Precincts 
The precincts were chosen by selecting four-digit random numbers from a random number table and keeping values between 1 and 1650 inclusive. The random number table was from William H. Beyer, Ph.D. , 25th Edition, CRC Press (1978)(39) using 'A Table of 14,000 Random Digits' (p 545-548) and using the following procedure: 
1.	Start at the first digit in the table 
2.	Consider the next four digits. 
3.	If the numerical value of the four digits is outside the range from 1 to 1650, inclusive, continue at step 2. 
4.	If the number has been chosen before, continue at step 2. 
5.	Accept the number as a precinct number in the random sample. 
6.	Continue from step 2 until 68 precincts have been chosen. 
These 68 randomly chosen precincts are combined with the 1% sample processed in the "One-Percent Manual Tally" procedure (17 precincts) to result in a total of 85 precincts in the sample. 
Data Collection 
 
An example of overprinting, no way to read the scanner report on this race. 
�	We were only able to gather documentation on ballots processed on election night, including the 1% sample in the RoV One Percent Manual Tally ("1PMT") and another 4% randomly selected. It was not possible to gather data on VBM (Vote-by-Mail) ballots or provisional ballots because these are directly scanned into the central tabulator and do not provide a paper trail. 
�	The ScannerTapes were scanned for all precincts in the sample and posted on our web site. Volunteers entered the data into web-based forms from the scanned tape files so that we could determine how many ballots were scanned for each precinct and could determine the results of the election if only those ballots are considered. 
�	Scanned the BallotStatements for all 85 precincts in our random sample with signatures of precinct workers redacted. This was performed by the RoV, and then we posted these statements for the public to review. 
�	Scanned the Street Indexes for about half of the precincts in our sample. These were not useful from a paper-trail standpoint because they were not always completed by the precinct workers in the 6pm to 8pm time period, thereby leaving off about 30% of the voters. Some of the ballot counts from the Street Indexes were added to our database, but we did not process all of them when we realized that they were incomplete. 
�	Other related documents were also scanned. 
�	We Manually inspected sign-in rosters in about half the sample precincts. We were unable to scan these because the RoV said it was a violation of privacy of the voters signatures. We found almost no discrepancies in the count of signatures on the sign-in rosters and what was reported in the Ballot Statements. 
Precinct Data 
The web page All Precincts(40) provides results of our data collection activity for all the precincts in our 5% sample. (This table is attached to this document in the hardcopy version of the report.) Details and supporting documents can be viewed for each precinct by clicking on the precinct number. 
Races 
We also tabulated the results of races in the election, based on the vote from our 5% sample. The web page All Races(41) provides a link to a separate page for each race in the election and lists the vote count from each precinct in the 5% sample. At the date of this writing, additional review of the data used in the races is required before we are confident of the reliability of these data. In many cases, the exact count was not available from some precincts because the tape was unreadable or the scan was defective. Readers who may be interested in specific races are encouraged to join our review effort and work to validate the count based on the scans of the scanner tapes. 
Because of the other issues we detected in our review, we did not attempt to check the exact count to see if it matches the report from the RoV, although that is our logical next step in the review. Such a comparison could detect some types of election fraud such as count rigging by the central tabulator operator. 
Issues of Concern specific to this process 
�	: Public Posting of Audit Trail -- The paper audit trail should be posted on the Internet by elections officials so anyone can review it.
Currently, oversight groups must access physical documents themselves or pay to have them scanned. Documents such as: Ballot Statements, Audit Log, Chain of Custody Report (disposition of all seals), and the Reconciliation Report are essential. 
�	: Scanner Tapes Not Accessible 
Scanner report tapes are no longer available early in the canvass. Details... 
�	: Scanner Tapes Difficult to Process 
The audit trail provided by the current RoV procedures include cash-register style paper tape which is difficult to process and sometimes impossible to read. Details... 
�	: Ballot Statements Not Scannable 
Ballot Statements must be scannable in their entirety, including signatures of precinct workers who sign them. 
The RoV claims that it is not legal to scan the signatures of volunteers and staff members who complete the Ballot Statements. It is our claim that these documents should be scannable in their entirety and placed on any public website. 
�	: Audit Log Insufficient 
The audit record available from the GEMS Central Tabulator does not provide sufficient detail to reconstruct the election. Details... 
�	: Documents Automatically Processable 
Documents that are considered public, such as BallotStatements, Street Indexes, and others should be automatically processable to minimize oversight costs. Public documents, such as BallotStatements, Street Indexes, and others, should not have additional tabs added to them by poll workers, and they should be of a size appropriate for automatic scanning equipment. For example, tabs effectively limit the ability for automatic document feeding equipment to operate on the documents and thus increase the cost for an outside organization to gather this information. Instead, workers can use separation sheets with tabs to expedite flipping to the correct page. Sheets that are not conventional sizes cannot be easily scanned. Examples of unconventional sizes include scanner tapes and tally sheets. 
�	: Audit Devices Should be Allowed 
If no improvement to the audit log is possible, then audit devices should be placed on the serial interface cables that supply data to the central tabulator to allow a log of all communication from all scanner and DRE equipment to the central tabulator. Data from those audit devices would allow an oversight activity to reconstruct the election and guard against tabulator-based election manipulation by compomised elections workers, and can eliminate the need to scan scanner report tapes and other difficult to process documents. 

Issues of Concern Categories  
Based on our review of the November 2008 presidential election, the Issues of Concern are grouped into the following categories. Each will be considered in a separate section. Some specific Isssues of Concern are duplicated in separate sections, if they apply to both. The Issues Summary contains all the issues in numerical order. 
�	Lack of Documentation - The most prominent concern is the lack of documentation concerning many of the procedures in use. 
�	Unscanned Ballots - The number of signatures does not match the number of ballots scanned (off by 9,000 ballots). 
�	Chain of Custody - The RoV does a poor job of tracking and investigating tampering (99 cases of tampering). 
�	Audit Log Discrepancies - Audit log is insufficient and contains unexplained inconsistencies. 
�	Manual Tally Procedure - Post Election Manual Tally documents that 29% of precincts checked had errors, but they were simply rescanned and deemed fine, when the proper conclusion was that there are serious errors in the election. 
�	Audit Trail - We had difficulty in examining the entire election because the audit trail is insufficient, difficult to process, and only available after the election has been certified. 
Note: 
A number of issues we describe below occur during operations in precincts by precinct volunteers. We DO NOT regard these errors to be due to any fraudulent intention by those workers, but instead by procedures that could be improved. For example, very few precincts (only about 15%) were able to balance the number of ballots received with the number of signatures. We found that the form used and the method of reconciling could be improved to make it feasible for workers who are tired and ready to end their day of service to find the reason the counts do not reconcile. The procedures used in the polling place may also help this issue. 

Lack of Documentation  
The Registrar Of Voters is responsible for processing our election without undue error. The first step in a high-quality result, supported by all quality assurance methodologies, is to establish a full set of comprehensive documentation on the procedures used by the RoV to process the election. Unfortunately, either the documentation does not exist, or the RoV has been unwilling to share it with the public. The processing of our elections should not be considered a private matter, but indeed all steps should be observable and reviewable by oversight groups. 
Indeed, to improve the quality of any operation, this set of procedures is a prerequisite, and should be the first order of business for the RoV, and should be mandated by the Secretary Of State. 
In addition to procedure documents, working documents that detail the results of the election must also be available. This issue is dealt with the issues category, Audit Trail. 
Issues related to this category: 
�	: Lack of Comprehensive Procedure Documentation -- RoV apparently does not maintain any comprehensive procedures documentation. Details... 
�	: Questions should be answered -- Elections officials should be obligated to answer questions from oversight activities.
Elections officials should not have the option to use the "That's not a California Public Records Act Request" or "We don't have any document prepared that addresses that question, and therefore we won't answer it" responses that do nothing to assist oversight groups in their activities. 
�	: No Reconciliation Procedure 
The RoV goes through a lengthy process of reconciliation which has been referenced in many conversations with them, but is not documented in any procedure document, nor is there sufficient notes taken regarding what is done. Details... 
�	: Archival Procedures Lacking 
There are no check-in and check-out procedures in the secure archive area and only documentation describing where the materials are in the storage facility. Details... 

Unscanned Ballots  
The first step of our review is to determine if all the ballots were correctly processed. 
Acceptable Error Rate 
Voluntary Voting Systems Guidelines(42) were authored by the Technical Guidelines Development Committee (TGDC), a committee authorized under the HELP America Vote Act (HAVA) of 2002, and researchers at the National Institute of Standards and Technology (NIST) for the Election Assistance Commission (EAC). These guidelines were authored primarily to guide voting machine manufacturers in their design of voting systems. They were recently modified to require a vote-capture device that uses independent voter-verifiable records (IVVR). For example, durable paper ballots that are scanned represents a ballot that can be easily read by the voter and interpreted by machines. 
This document also describes acceptable error rates. In particular, it defines "disenfranchisement" (Section 6.3.1.3 "Manageable failures per election") as follows: 
Any failure that results in all CVRs pertaining to a given ballot becoming unusable or that makes it impossible to determine whether or not a ballot was cast is called disenfranchisement. It is unacceptable for even one ballot to become unrecoverable or to end up in an unknown state. For example, an optical scanner that shreds a paper ballot, rendering it unreadable by human or machine, is assessed a disenfranchisement type failure; so is a DRE that is observed to "freeze," providing no evidence one way or the other whether the ballot was cast, when the voter attempts to cast the ballot. 
For purposes of San Diego County which primarily uses paper ballots, "CVR" -- Cast Vote Record -- is the paper ballot. 
The guidelines suggest that no error in terms of disenfranchisement is acceptable (error rate=0), and that the central tabulator would exhibit an error rate of 1.237×10-6, that is, one lost ballot in 808,407 ballots cast. In San Diego, with 1,245,947 ballots cast in November 2008, we would expect no more than 1.54 ballots to be lost due to central tabulation error. This can be rounded up to 2, i.e. the election system in San Diego County, according to the federal guidelines, should lose no more than 2 ballots in the November, 2008, election. 
Although the Election Assistance Commission states that there should be zero error rate with regard to disenfranchisement, this requirement, although desired, is not feasible in any real system. Therefore, for the purposes of our analysis, we will use the higher error rate suggested as the maximum for central tabulation, i.e. 2 ballots lost. If we combine the two limits, for the County of San Diego, we are limited to  
The question of disenfranchisement is whether every voter who submits a valid ballot has their vote counted, not whether their vote was counted correctly. Disenfranchisement can occur when elections equipment skips ballots (i.e. misfeeds) and there is no correction or if ballots are lost or stolen, for example. It might be argued that random disenfranchisement will not change the outcome of the election since any truly random change will average out to no change. However, this is absolutely not the statement by the Election Assistance Commission guidelines, which explicitly state that every vote must be counted, and no ballots should be disregarded. 
Method of Analysis 
The question "were all ballots counted" does not consider whether the vote is counted correctly from those ballots, but simply that every ballot that is cast is attempted to be counted and is not lost in the shuffle. This must always be our first order of business in review of any vote counting operation. 
To determine the performance of the San Diego County RoV in this regard, we reviewed data collected in our 5% sample regarding the number of ballots processed. The most significant figures are: 
�	The number of people who signed rosters at the polling place and who received paper ballots. 
�	Total number of signatures is Reported on Ballot Statements and signatures can be counted on the sign-in rosters to verify those figures. 
�	Number of paper ballots is the total number of signatures minus provisional voters and minus touchscreen machine voters. 
�	The number of ballots counted by precinct workers at the close of the polls 
�	Reported on Ballot Statements 
�	The number of ballots scanned by scanner equipment at the Tally Center 
�	Reported on ScannerTapes as "Cards Cast" 
�	The number of ballots reported in the Audit Log of the central tabulator for each precinct. 
�	From the Audit Log as "# CARDS" 
�	The number of ballots reported in the final canvass report by the RoV 
�	From CanvassReports as "Cards Cast" 
Issue 1: Most Precincts Did Not Balance 
We were surprised that very few precincts (15 = 17%) in our sample reported that they could reconcile the count of ballots after their polling place closed. 83% of the precincts reported that their totals did not match the number of signatures on sign-in rosters, and most of those could not identify why the discrepancy occurred. See the column "Ballot Statement Notes" on AllPrecincts and look for "Yes", meaning that the count of signatures matched the count of ballots. Many precincts simply gave up and did not attempt to determine why the discrepancy occurred. 
The following observations are relevant to this issue: 
 
The count of blank ballots received by each polling place is determined by RoV staff in preparation for Supply Pickup Weekend and documented on the Ballot Inventory Sheet. The Precinct Inspector counts the ballots received and confirms the count on the two-part Ballot Inventory Sheet. It is difficult to imagine that this count can be significantly in error with this two-step check. This count can be considered very reliable. 
 
Precinct workers are instructed to count voted, spoiled, unused, and provisional ballots after the polls close. Some reported in their comments that they counted ballots several times, although this is not specifically documented by the Ballot Statement form. These counts can be considered reliable because counting ballots is not a difficult procedure. 
 
We counted the number of signatures on sign-in rosters by manually reviewing the rosters and making notations about the number of signatures on each page. We compared that figure with the number as reported on Ballot Statements. 
We found:
�	no significant variation between the count we made by inspecting sign-in rosters and that reported on Ballot Statements. This implies that this count is relatively reliable 
�	Sometimes it was difficult to know if the signature existed or not, since some signed on top of the notation "Mail Ballot Received", some signatures are not much different from random markings, and some precincts added notations, such as lining-through some signatures in their counting process. 
�	that many precincts allowed VBM voters to sign over "Mail Ballot Received" if they had a mail ballot to surrender, and some allowed VBM voters who did not have a ballot to surrender to sign it as well. 
�	Many provisional voters did not sign the rosters. 
 
Given that the reported counts of signatures and ballots were reasonably reliable, why did so few precincts balance? 
�	Voters did not sign the rosters - Although it is a requirement that voters sign the roster, some did not and still received a ballot. This was particularly true of provisional ballot voters. 
�	VBM voters signed the roster over the "M" or "Mail Ballot Received" designation. There was no place for VBM voters to sign if they had their ballot to surrender. 
�	VBM voters who just turned in their envelopes did not sign anything, so these were untrackable. 
�	VBM voters sometimes signed both the provisional roster and the white/blue/pink rosters if they did not have a ballot to surrender. 
�	Voters sometimes returned their ballot to the wrong precinct, in locations where multiple precincts were conducted in parallel. 
�	RoV staff said it may be the case that voters left with their ballot and did not turn it in. However, this sounds like a very unusual case as it makes no sense for voters to come to the polling place and not vote. 
 
�	Avoid multiple-precinct polling locations or separate them more distinctly so it will be very unlikely that a voter would turn in ballots to the wrong precinct ballot box. 
�	Perhaps use simple ballot scanners that detect the precinct and will cause an alert if a voter attempts to deposit a ballot into the wrong ballot box. 
�	Scanners could also provide a machine count of the ballots. 
�	Provide a worksheet to reconcile smaller aspects. Current reconciliation attempts to reconcile the total number of signatures with the total number of voted ballots. 
�	Separately reconcile provisional ballots and provisional sign-in sheets. 
�	Separate provisional ballots into two groups: 
�	VBM voters who did not surrender their VBM ballots who are listed with that precinct 
�	Other voters who are voting in the "wrong" precinct. 
�	Subtract provisional and touchscreen voters from total signatures to determine total paper ballot signatures and reconcile that with the number of actual ballots. 
�	Maintain sign-in rosters for ALL VBM ballots returned to each polling place. 
�	Currently, separate sign-in rosters are not maintained for VBM voters who do not wish to vote in person. 
�	Sign-in rosters will allow tracking the VBM ballots and provide a way to determine where in the system they are lost, if indeed they are. 
�	Provide sign-in locations for VBM voters who are listed as members of the precinct and designate if mail ballot was surrendered or not. Those who do not surrender their mail ballot should also sign the provisional roster and obtain a provisional ballot. With this designation, duplicate signatures can be easily located. 
�	Avoid the current practice of "signing over" the "MAIL BALLOT RECEIVED" designation in the signature box. 
�	Avoid the practice of putting a line through the signature box, which some precincts did apparently to help count them. 
�	Include a checkbox that can be marked by Precinct workers to designate that the scribble represents the voter's signature. Some signatures are so bad, they are hard to discriminate from random markings. 
�	The RoV should produce a report summarizing the count of signatures, the actual number of ballots returned, and an overall percentage of error. This report should be mandated by the SoS. The Registrar should encourage precincts to do better. Precincts that produce accurate reports that account for all signatures should be acknowledged with certificates of merit or other prizes. Similarly, those precincts that do not do well will may need to additional training or staffing changes. 
�	One suggestion to limit the error is to reduce the number of polling places. The size of precincts (111 to 674 ballots in our 5% sample) is about 5% to 33% of the suggested "typical" precinct size of 2000 ballots, as suggested by the Secretary Of State. On the average, polling places should handle 4 precincts and reduce the number of polling places to 400 instead of 1650. This would reduce the number of polling place workers needing training and allow more well-training workers to be allocated to the same polling places. The number and size of precincts can remain the same, with the vote split out as it is today using scanners that will read the precinct from the ballot. This would also eliminate the current error mode of placing the ballot in the wrong ballot box as well. 
Issue 2: Many Ballots were not scanned 
 
Ballot Statement for Seq# 1563 documenting 565 ballots counted. 

Top of the Scanner tape for Seq# 1563 documenting only 550 ballots scanned -- 15 ballots lost. 
	
Polling place workers count the number of ballots in their attempt to complete the Ballot Statement form and reconcile the numbers. Counting ballots is a relatively simple procedure and many polling places reported that they counted them multiple times. 
However, when the number of ballots was compared with the number of "Cards Cast" from the scanner tapes, we found that 429 ballots were lost in our 5% sample. Extrapolating this over the entirety of the election, a total of over 9,000 ballots were lost, over 5 per precinct. Only 4 precincts had no apparent loss of ballots when they were scanned in our 85 precinct sample (4.7%). 
To cite a clear example, consider Precinct Sequence 1563(43) which is one of the few Ballot Statements that reconciled completely. The precinct had no touchscreen ballots. The ballot statement discloses 565 ballots counted and that matches the number of non-provisional signatures (565). However, the scanner tape from the Tally Center says there were only 550 cards cast. 
Therefore, 15 ballots were lost in this precinct alone. 
 
We asked the Registrar Of Voters this question and they refused to explain it, saying that it was not a record that could be provided in a California Public Records Request. We would like to know what happened to these ballots, as this is the first step to producing a reliable canvass. 
The options are: 
�	Ballots were never in the box - This would imply that workers in over 95% of the precincts cannot count the ballots and that they consistently miscount by missing ballots. Since we did not find any obvious deviation from the number of signatures on rosters and the counts provided by the precinct workers, the concept that they cannot count ballots is viewed as VERY UNLIKELY. 
�	Ballots were removed because they were "Write Ins" - At first glance, this seems to be likely true, despite the fact that the scanner tapes includes values for write-ins. If this is the case, then the removal of these ballots makes it very difficult to track lost ballots in the scanning process.
�	The One Percent Manual Tally procedure also detected lost ballots in 29% of the precincts. The RoV confirmed this loss when they reran the precincts through the scanners, magically finding the ballots. Instead of seeing this as confirmation of a serious problem, the RoV said there was no problem at all because the rescan corrected it, and they refused to answer our question regarding where the ballots went. 
�	Ballots were misfed through the scanner, either accidentally or intentionally. - This is a distinct possibility. The scanners were designed for one-at-a-time manual feeding. If an operator feeds them too quickly, they may skip a ballot, or if the operator sees a vote they do not like, they could skip the ballot and place it directly in the output tray. The RoV confirmed that both of these were possibilities. 
�	Ballots are removed from the ballot box prior to scanning - 6% of the tamper-evident seals are either tampered with or incorrect and the tracking of these seals is minimal, with no followup. However, the number of precincts with tampering was not high enough to account for the large number of precincts with lost ballots (95%). 
In C00022, the RoV mentioned the fact that write-ins and damaged ballots were removed from the rest of the ballots and this may be the explanation for this discrepancy. However, there are no documents (that we are aware of) that track this or report on the number of write-ins and their disposition. 
 
�	: Ballot Statements - Few balanced 
Only a few of the Ballot Statements had figures that balanced, and only one precinct had all the ballots correctly scanned. This is a significant issue that is separately treated in this report as a separate Issues of Concern Category. (See Unscanned Ballots) Details... 
�	: Ballot Statements - Removed Write-ins not Documented -- Ballot Statements did not include a count of the number of write-ins and damaged ballots removed. Details... 
�	: Ballot Statements - Office Use Section Not Explained -- Ballot Statements have an "Office Use Only" section does not include an explanation to allow the public to understand how it is being used. Details... 
�	: Tally Center - Write-Ins Removed and Not Counted 
We are not sure why the write-in ballots are removed permanently prior to scanning rather than scanning first and processing later or quickly processing them and uniting them with the rest of the precinct. Details... 
�	: Tally Center - Damaged Ballots 
On the Tally Center Log sheet, there is a checkbox for "Damaged Removed." However, we don't know why any ballot would become damaged and not be treated as "spoiled." Details... 
�	: No Comparison of Ballot Count 
There is no comparison of the number of ballots scanned, as reported on the report tape, with the number of ballots counted at the precinct nor by the count as read by the central tabulator. Details... 
For coverage of the Chain of Custody and seal management see: Chain of Custody in this report. 

Chain of Custody  
The term "chain of custody" is most commonly used with regard to legal evidence. Chain of custody refers to the chronological documentation, and/or paper trail, showing the seizure, custody, control, transfer, analysis, and disposition of evidence, physical or electronic. In our context, it refers to the active control asserted by the Registrar Of Voters over blank and voted paper ballots, electronic media, and other documents related to the execution of the canvass. 
Robust ballot accounting and secure chain of custody of election materials and equipment are prerequisites for accurate election results and post-election audits. The following conditions must be met: 
1.	There are strict written accounting procedures for paper records to prevent the addition, subtraction, substitution, or alteration of paper records. 
�	Durable Paper Ballots are used as the primary Cast Vote Record (CVR). 
�	Blank ballots must be strictly controlled to avoid ballot substitutions. 
2.	To safeguard the ballots and audit records from loss and tampering, paper records and electronic equipment are fully secured at all times when a breach could adversely affect the integrity of the records including from the time the votes are cast until all audit or recount activity is completed and election results are finalized, including the expiration of all legal recourse to challenge or correct the election. 
�	Procedures regulating access to ballots and equipment could include requiring signatures for access and documenting the reason for it, preventing access by a single person, requiring that access be observed by members of opposing parties, or the use of surveillance cameras to guard storage areas. 
3.	An audit of these procedures should begin as soon as possible after the random selection of audit units, which commences as soon as possible after the initial tallies recorded by the voting system are reported, the audit conducted by an independent group of analysts. 
4.	The secrecy of the ballot is preserved. 
�	It must not be possible to determine how a person voted by inspecting election records. 
�	The order of the votes cast is never compared to the order in which the voters signed in. 
�	Each ballot, although it may be identified with a unique identifier, must not be connected to the identity of the voter. 
5.	There is a reconciliation to ensure that all votes from all audit units are correctly tabulated in the election totals. 
Tamper Evident Seals 
There is some use of tamper evident seals by the RoV. The use of such seals is now common in industry and the best practice for use of these seals is as follows: 
1.	Strict custody of all seals is necessary. 
2.	All seals have unalterable serial numbers. 
3.	A trusted person must have custody of some subset of the seals distributed for use. The serial numbers distributed to that person must be documented. 
4.	The trusted inspector must inspect the item to be sealed, and preferably, witnessed by a second person. The serial number of the seal is then documented by the trusted inspector in separate documents. 
5.	If the item must be opened and resealed, this must be documented as well. 
6.	All remaining seals must be returned. 
7.	At each stage of inspection of the item, the seal number should be checked to validate that the seal has not been changed, and any evident tampering must be documented immediately. 
Review of San Diego County Registrar Of Voters Chain of Custody 
�	: Lack of Comprehensive Procedure Documentation -- RoV apparently does not maintain any comprehensive procedures documentation. Details... 
�	: Public Posting of Audit Trail -- The paper audit trail should be posted on the Internet by elections officials so anyone can review it. Details... 
�	: No Seal Serial Number on Ballot Statements 
Serial numbers of the tamper-evident seals are not included on Ballot Statements. Details... 
�	: Ballot Statements - Few balanced 
Only a few of the Ballot Statements had figures that balanced, and only one precinct had all the ballots correctly scanned. This is a significant issue that is separately treated in this report as a separate Issues of Concern Category. (See Unscanned Ballots) Details... 
�	: VBM Ballots Not Logged at Precinct 
VBM Ballots were not logged in at the precinct and apparently not placed in sealed containers. It is not necessary to sign anything to drop off your VBM ballot at the precinct or RoV office, and this is a deficiency. Details... 
�	: Spoiled Ballots Loosely Controlled 
Spoiled Ballots, both VBM and election day are not under strict control. I.e. it would be a simple matter to checkmark a VBM envelope that the ballot was spoiled and eliminate it from consideration, and to covertly redo a vote and show a spoiled ballot at the polling place. 
To "spoil" a ballot, it should require the signature of the voter. Anyone could checkmark a VBM ballot as spoiled, and easily disenfranchise a voter. All spoiled ballots should be tracked to confirm that an unspoiled ballot is completed by the voter. 
�	: Collection Center Seal Logs Lack Serial Numbers 
At collection Centers, they are supposed to inspect the tamper-evident seals, but there is no place on the log sheet to document the seal numbers nor to note whether the seals were intact. Details... 
�	: Tally Center - Write-Ins Removed and Not Counted 
We are not sure why the write-in ballots are removed permanently prior to scanning rather than scanning first and processing later or quickly processing them and uniting them with the rest of the precinct. Details... 
�	: Tally Center - 99 Seals broken or wrong 
In the November 2008 election, Tally Center Logs documented 99 precincts that had wrong or broken seals, a 6% rate, and there was no investigation. Details... 
�	: Scanner cards not Archived 
Memory cards used to capture the vote from each scanner are not maintained as a permanent record of the election. Details... 
�	: No Comparison of Ballot Count 
There is no comparison of the number of ballots scanned, as reported on the report tape, with the number of ballots counted at the precinct nor by the count as read by the central tabulator. Details... 
�	: VBM Ballots are not counted 
VBM Ballots are not counted after being removed from the envelope and before submitting for scanning, and there is no attempt to confirm the count of ballots as they are scanned. Details... 
�	: VBM Ballots not precounted during scanning 
We were told that the scanner operator pre-counts the ballots as they run them, but this did not occur. Details... 
�	: Documents Automatically Processable 
Documents that are considered public, such as BallotStatements, Street Indexes, and others should be automatically processable to minimize oversight costs. Details... 
�	: Scanner Tapes Difficult to Process 
The audit trail provided by the current RoV procedures include cash-register style paper tape which is difficult to process and sometimes impossible to read. Details... 
�	: No Paper Trail for VBM and Provisionals 
Unlike ballots processed on election night, no scanner reports are produced and ballots are read directly into the central tabulator. VBM ballots and provisionals should be scanned by scanners that produce a report, so these can be accessed later. (May be avoided if the audit log is improved and available.) 

VBM Ballots, once removed from their envelope, have no paper trail, such as a scanner tape, so that the process can be checked for accuracy. The PEMT shows that 29% of precincts had missing ballots. 
�	Although not reviewed in this report, it may be feasible to check the count of VBM ballots received, as reported in the voter database with the count of ballot processed in the entire VBM processing activity. 
�	VBM Ballots could be placed under positive control by including a count of ballots in each batch on documentation that would accompany the ballots. Scanner operators would produce a scanner tape report to confirm the correct number of ballots were processed. Current practice has the operators counting small batches of ballots to see if they are scanned correctly, but this practice does not confirm that ballots were not lost along the way. 
See also Issue Issue A021: VBM Ballots are not counted. 
�	: No Seal Tampering Report 
A seal report should be generated that explains all tampered seals. At present 99 seals show tampering but there is no report nor any action taken to eliminate the tampering. Details... 
�	: Archival Procedures Lacking 
There are no check-in and check-out procedures in the secure archive area and only documentation describing where the materials are in the storage facility. Details... 
�	: Ballot Order Chain of Custody 
We did not review the process of maintaining control of blank ballots that are not distributed to precincts and oversight of the printer vendor. Details... 
�	: SPUW Observation 
Supply pickup weekend not witnessed nor video recorded. Details... 
Chain of Custody -- Concluding Remarks 
The chain of custody of important documents by the San Diego Registrar of Voters is extremely lax and in need of improvement. This issue is severe. There is no sense in using tamper evident seals on ballot cartons if there is no investigation of tampering when there is such evidence in 99 cases. Similarly, we are told that the entire facility is under observation by closed-circuit video camera, however, the RoV did not provide any documented procedures regarding review of the video camera evidence. Therefore, these procedures are nothing more than shams. "The whole place is under video surveillance" means nothing if no one watches the video. "We use tamper-evident seals" means nothing if evidence of tampering goes unnoticed and there is no investigation. 
The RoV must be required to produce a comprehensive chain-of-custody report including rationale for every broken and wrong tamper-evident seal. At this point, there is some logging of seals but no serial numbers, and then no report on the whole situation. 

Audit Log Discrepancies  
Citizens Oversight requested and received the central tabulator audit log. 
We have the following observations regarding the audit log: 
�	The number of cards cast is noted in the log for precincts scanned on election night. No counts are provided for VBM ballots, Provisionals, and Touch Screen entries. 
�	The audit log does not provide any vote information, only the count of ballots, and then only for election night scanning. 
�	The Registrar does not have full documentation for the audit log, such as what all entries mean and any options or configuration settings in terms of what is provided in the audit log. (Their reply to Question C00021-21.1) 
�	The Registrar has no procedures for review of the log. (Their reply to Question C00021-21.2) They said the logs are reviewed by IT staff to verify that an action has take n place such as a batch has been closed and the correct number of cards cast has been captured. But there was no documentation regarding when these reviews took place, if in fact they ever did. 
�	There is no outside review of the audit log, except the one you are reading. 
�	Details of the Audit Log are available in COPs document C00019: Letter to RegistrarOfVoters regarding election processing 
�	There are two cases of transactions of the following form: 
1225838065      Connection Opened on 172.16.1.20 (Port:1027)
1225838074      Uploaded VCenter 17000 Machine 1 DLVersion 1 Copy 0  on 172.16.1.20 (Port:1027) SN 223582 #Ballots 12 12 Precincts
1225838074      Connection Closed on 172.16.1.20 (Port:1027)

1225838148      Connection Opened on 172.16.1.20 (Port:1029)
1225838160      Uploaded VCenter 17000 Machine 0 DLVersion 1 Copy 0 on 172.16.1.20 (Port:1029) SN 208790 #Ballots 20 18 Precincts
1225838160      Connection Closed on 172.16.1.20 (Port:1029)
RoV said (C00021-21.5) this represented voted contest counts for the ballot types at vote center 1700 (early voting location at RoV office), from machine number 1, from the 1st copy of that touch screen card, which is being read from a touch screeen connected to the GEMS server over a secured isolated network on IP address 172.xxx.xxx.xxx (paort: 1027) SN 223582. Precincts. 
We understood that the RoV would be converting all touchscreen ballots to durable paper ballots in the election, so we are confused as to why this was occurring, and  
�	In numerous places throughout the log, we see the transactions for starting and stopping the server. For example, at the start of the log, we see: 
1223283632      Starting AV Server
However, in numerous places (ten), there is no notation that the server was stopped and it is started, or it is started and there is no “Stopping” entry. In addition, during those intervals, there is significant time elapsed in the unaccounted time period. For example, 
1224517959      Connection Closed on COM3
1224517962      Waiting on COM3
1224518202      Starting AV Server  (NOTE NO “STOPPING SERVER”)

1224519519      Stopping AV Server
1224571786      Waiting on COM3     (NOTE NO “STARTING SERVER”)
1224571786      Waiting on COM2

1224586245      Waiting on COM8
1224587341      Starting AV Server  (NOTE NO “STOPPING SERVER”)

1225728881      Waiting on COM1
1225729028      Waiting on COM1
1225817009      Starting AV Server  (NOTE NO “STOPPING SERVER”)

1225837932      Waiting on COM2
1225838035      Starting AV Server  (NOTE NO “STOPPING SERVER”)

1223564989      Stopping AV Server
1224490449      Waiting on COM5     (NOTE NO “STARTING SERVER”)
1224490449      Waiting on COM3

1224519519      Stopping AV Server
1224571786      Waiting on COM3     (NOTE NO “STARTING SERVER”)
1224571786      Waiting on COM2

1225708019      Stopping AV Server
1225712288      Waiting on COM1     (NOTE NO “STARTING SERVER”)
1225712411      Connection Opened on COM1

1225817574      Waiting on COM1
1225817579      Stopping AV Server
1225819478      Waiting on COM1     (NOTE NO “STARTING SERVER”)
1225819489      Waiting on COM2

1228227206      Connection Closed on COM8
1228227210      Waiting on COM8
(END OF THE LOG)                (NOTE NO “STOPPING SERVER”)
The RoV said (C00021-21.7): 
This question assumes an order of processing that is erroneous. As stated above, the AV Server Log is a continuous log file that contains events. 
Events happen before the AV server is started. 
Events happen while the AV server is running. 
Events happen after the AV server is stopped. 
And also said to the question: Why does the log have this inconsistency? (C00021-21.7.1): 
Because it is an Event driven log file. The events themselves can be driven by a myriad of other varying events. 
We find these answers unsatisfactory. The audit log should be a complete accounting of events. Also, we find it interesting that the RoV can even answer this since they have documentation regarding the meaning of the Audit Log (Their reply to Question C00021-21.1) 
�	We asked (C00021-21.7.2): It is possible that some transactions have been deleted from the log, when the central tabulator was still running, and then shut down later, with the “Stopping AV Server” also deleted? And they said: "We are unable to answer this. Perhaps Premier could answer it." But we were never given a contact at Premier Election Solutions (formerly Diebold Systems). 
�	There were a number of errors and system crashes noted in the log: 
1225838402      Error Internal Error
File: DownloadAVSPort.cpp, Line 665, Date: May 13 2005 on COM8
And: 
1225840829      Error Internal Error
And, and invalid Password detected: 
1225850242      Connection Opened on COM2
1225850242      Error "INVALID PASSWORD" on COM2
1225850244      Connection Closed on COM2
RoV officials were unable to explain any of these crashes and error conditions. They referred us to Premier Election Solutions but we were never given a contact (and we did ask for one). 
�	There are a number of times (133 total) where the totals were cleared and a precinct re-run, starting Nov. 20 and occurring through Dec. 2, 2008, as listed below. 
1227198273      Clearing VCenter 5890:0
1227198287      Clearing VCenter 14660:0
1227198295      Clearing VCenter 14810:0
1227198304      Clearing VCenter 14840:0
1227198313      Clearing VCenter 14860:0
1227198320      Clearing VCenter 14870:0
      ...
(121 lines omitted for brevity)
      ...
1228220643      Clearing VCenter 4620:0
1228220657      Clearing VCenter 10880:0
1228220671      Clearing VCenter 11660:0
1228220684      Clearing VCenter 350:0
1228227151      Clearing VCenter 7770:0
1228227169      Clearing VCenter 8230:0
Elections officials could not explain these specifically, as they do not maintain any notes of operation of the election. They said that ballots are rerun for several reasons. There may have been an operator error that allowed one ballot to be entered into the system twice or one that was missed because the system displayed an error that was not caught during processing. There may have been a mail ballot in the deck of polls ballots, which would also require a rerun to remove the ballots. 
�	We asked about the fact that we did not notice any “Clearing Vcenter” entries prior to the initial accumulation of votes. How do we know that the accumulators were zeroed? 
RoV said: 
The initial system is cleared at the beginning of the mail ballot processing, at which time a zero ballots cast report is run. From this point forward, ballots are added to the system. It should be noted that the “clearing Vcenter” entry is for an OS memory card that is used to transfer ballots cast at the polling places (contained in ballot cartons) into GEMS on election night and during the canvass as reruns. 
�	We asked for the log from the earliest date it was maintained. ("If the log includes such an entry at an earlier date, please forward the complete log, including the clearing entry(ies).") However, we were never provided with the complete audit log. Therefore, we must assume it does not exist. The RoV does not maintain an audit log for the complete election process. This is a severe problem. 
Recommendations regarding the Audit Log 
�	: Audit Log Insufficient 
The audit record available from the GEMS Central Tabulator does not provide sufficient detail to reconstruct the election. Details... 
�	: No Comparison of Ballot Count 
There is no comparison of the number of ballots scanned, as reported on the report tape, with the number of ballots counted at the precinct nor by the count as read by the central tabulator. Details... 
�	: No Reconciliation Procedure 
The RoV goes through a lengthy process of reconciliation which has been referenced in many conversations with them, but is not documented in any procedure document, nor is there sufficient notes taken regarding what is done. Details... 
�	: No Operator Notes 
A log of all operator activities should be maintained, with rationale for re-scans, etc. At present, no operator log is maintained and 133 re-runs occurred with no explanation. Details... 
�	: No Reconciliation Report 
A reconciliation report should be provided. This report should show the number of sign-in signatures, the number of ballots scanned and tallied, the number of provisional ballots accepted and rejected, the number of VBM ballots received at the precinct and the number processed, etc. such that the figures shown above all balance. Details... 
�	: Audit Log Discrepancies 
In the November 2008 Election, a number of discrepancies were detected in the Central Tabulator (GEMS) Audit Log. 
�	The Registrar does not have full documentation for the audit log, such as what all entries mean and any options or configuration settings in terms of what is provided in the audit log. (Their reply to Question C00021-21.1) 
�	The Registrar has no procedures for review of the log. (Their reply to Question C00021-21.2) They said the logs are reviewed by IT staff to verify that an action has taken place such as a batch has been closed and the correct number of cards cast has been captured. But there was no documentation regarding when these reviews took place, if in fact they ever did, and no report resulting from any review. 
�	In ten places, the log showed SERVER STOPPED without first seeing SERVER STARTED and vice versa. These should be paired in all cases. If they are not paired, then transactions may have been deleted (C00021-21.7.2). An explanation of this anomaly was requested from the Registrar (C00021-21.7.1) and they did not have an answer to fully explain it other than saying it could be a myriad of reasons. The Registrar suggested that we contact Premier Election Solutions, but no referral was ever provided. 
�	Crashes. There were a number of crashes and other error messages. The Registrar suggested that we contact Premier Election Solutions but no referral was ever provided. 
�	Re-running precincts. In 133 cases, precincts were re-run but no explanation given as to why they were re-run and no report as to why the initial run was in error. (See Issue A027: No Operator Notes) 
�	Clearing not shown. The central tabulator should be fully cleared prior to accumulating votes, but this was not in the log. Apparently, the RoV does not produce a log for the entire election process. 

Manual Tally Procedure  
The RoV maintains the One Percent Manual Tally Procedure to govern this process, which is mandated by the SoS. This is also refered to as the Post-Election Manual Tally (PEMT). 
Manual tally procedure in a nutshell 
1.	Select dates for the random draw of precincts and the actual tally, notify the public by placing a notice at the front counter. (COPs: This should also be placed on the website.) 
2.	Random Draw of Precincts 
1.	Calculate one percent and round up. 
2.	Have scheduled observers attend the draw 
3.	The procedure uses three sets of balls from 0-9 and one set of balls numbered 0 and 1, representing the four digits, (ones, tens, hundreds, and thousands, respectively). 
4.	Shake the containers and draw balls manually. (COPs: Since the containers are clear plastic, this may result in trying to choose specific balls.) 
5.	Draw ones, tens, hundreds digits. 
6.	If the three numbers in step six is 650 or less, the thousands container is then shaken, otherwise if the number is 651 or greater, the selection is complete. 
7.	Probability of selecting any number 650 or less or from 1000 to 1650 = (0.1)*(0.1)*(0.1)*(.65)*(0.5) = 0.0325% 
8.	Probability of selecting any number 651 to 999 = (0.1)*(0.1)*(0.1)*(0.35) = 0.0350% 
9.	The only time this would be correct is when "Cut Off" (i.e. 650 in this case) is 666 so that 666/2 = 1000-666. 
�	Probability of selecting any number 666 or less or from 1000 to 1666 = (0.1)*(0.1)*(0.1)*(.666)*(0.5) = 0.0333% 
�	Probability of selecting any number 667 to 999 = (0.1)*(0.1)*(0.1)*(0.333) = 0.0333% 
10.	Therefore, this procedure does not produce exact results and is not generally applicable, however, results are approximately correct because 650 is almost the same as 666. 
11.	Selecting precincts randomly will not necessarily result in even sampling of the precincts. In the November 2008 election, no precincts were selected in the range from 1 to 453 inclusive while nine precincts were chosen in the highest numbered 450 precincts. The likelihood of that happening is about 1 chance out of 10,000 tries, which is rare indeed. 
3.	Record contests as selected by the random draw. 
4.	Select additional precincts for the manual tally to cover all remaining contests on the ballot. 
5.	Prepare for the manual tally, including: 
1.	Order summary reports for election night and the related ballots. (COPs note: the staff creating these reports will know which ones are being requested, and therefore, could conceivably prepare known good reports for those precincts while the balance of the election may be substantially modified.) 
6.	Conduct the manual tally 
1.	Each precinct as one three-person team, one calls out the vote and two tally on separate sheets. 
2.	Supervisor compares with summary report. 
3.	If there is a variance, they re-tally until they agree, but no more than three times. 
4.	Supervisor records results, variances, and causes (if knoswn) on consolidation log. 
7.	When the tally is complete, return ballots for secure archival 
1.	Compile a memo of the results for the RoV. 
Recommendations to Improve the Post-Election Manual Tally (PEMT) 
�	: PEMT Peformed too early 
The PEMT is performed too early after election night and does not include a complete set of ballots for the precincts under inspection. The PEMT should be performed after all write-ins, VBM and provisional ballots have been processed. The write-ins that were removed on election night certainly should be completed before starting. 
�	: PEMT - Notification on Web Site 
The RoV should notify the public on the RoV website regarding the selection and tally in addition to placing a notice at the front counter. The web site is much more accessible than just the front counter and will incur no significant additional cost for the RoV but will make coordination by the public much easier. 
�	: PEMT Random Selection Not Technically Correct 
The random selection of precincts procedure is incorrect and not generally applicable. COPs recommends that the Registrar Of Voters change this procedure as follows: 
�	Use four ten-sided die (which are easily available on-line for $3.99 for a set of 10 die). 
�	Roll the dice perhaps in separate boxes or use different colors to allow them to be treated as ones, tens, hundreds and thousands. 
�	Declare that on the die representing thousands, even numbers will be considered "0" and odd numbers represent "1". 
�	If the total number shown on the die is greater than the highest numbered precinct sequence number (1650 in this case), disregard the roll and roll again. 
�	: PEMT Computer Report Manipulation Vulnerability 
There is a manipulation vulnerability regarding how the computer reports are generated. Current practice has the RoV staff produce reports which are, because the PEMT is currently performed early in the canvass cycle, tentative at best. The reports used by the PEMT should be the (draft) final canvass report to eliminate the possibility that a compromised member of RoV staff might substitute a known correct report for the precincts to be sampled while allowing the other precincts to be modified. 

If the election was fraudulently manipulated by RoV staff using the central tabulator, then it would be possible to cover that up by producing "good" reports for the precincts in the One Percent Manual Tally but there is sufficient slop between those and the final results to cover it up.
See also Issue Issue A033: PEMT Peformed too early. 
�	: Manual Tally Done Backwards 
Current practice treats the output of the computer scanners as the known good result and the tally teams redo their work until they can match it. This is contrary to the principle that the tally teams should be the most reliable result. Therefore, the PEMT procedure should be modified as follows: 
1.	Tally each precinct by two tally teams, each operating as they are now. 
2.	Compare the result of the two teams. 
3.	If they differ, tally again by a third team 
4.	If possible, determine what causes them to differ and determine the "correct" result. 
5.	Compare this result with the report as generated by the electronic scanners. 
6.	If the scanners produce a different result, rescan the ballots to determine if that is the source of the error. 
7.	If an error is detected, it may require that all ballots be rescanned using different equipment. 
�	: PEMT Rescanning 
There is no provision in the current Post Election Manual Tally procedure to allow that workers rescan precinct ballots if the vote determined by the manual tally differs from the electronic scan. However, the RoV rescans the ballots. They confirm the error (i.e. additional ballots are discovered in the re-scan) but the RoV nevertheless declares that the precinct is error free. The rescanning procedure to confirm an error must be included in the written procedure if it is to be performed, and then the consequences of finding the error must also be described, such as performing a complete election rescan. 
See also Issue Issue A033: PEMT Peformed too early. 
�	: Tally Sheets too big to process automatically 
The Post-Election Manual Tally (PEMT) should use paper that is easily processed by automatic document feeding equipment. Currently, the tally sheets are on over-sized paper and are difficult to scan.
See Issue Issue A023: Documents Automatically Processable. 
�	: PEMT Reports Incomplete 
The PEMT should produce a report for every race and every precinct, as suggested by the SoS. These sheets are easy to prepare if they are prepared when the manual tally is conducted. The resulting number of pages should be at least (Number of Precincts)*(Number of Races). For the November 2008 election, this would have produced about 400 pages for VBM and 400 pages for election day results. (The number of races applicable to any Precinct is less than the total number of races run in the county as many are not county-wide.) The report prepared by the RoV had only five pages, not 800, and did not include any vote counts. By comparison, Los Angeles County produced 1,300 pages in their report. 
�	: PEMT Should Reconcile to Rosters 
The PEMT does not look at the ballot statements and sign-in rosters, and are restricted to the ballots actually run through the scanners on election night. Unfortunately, since there are many (9,000) write-ins removed, the PEMT does not match the number of people who signed the rosters at the polling place. The PEMT should include a complete look at the precincts under consideration and not limit the view to just the operation of the scanners. 
See also Issue A033: PEMT Peformed too early. 
�	: PEMT Errors Not Addressed 
In the November 2008 Election, a number of errors were detected by the Post Election Manual Tally (PEMT) procedure. These issues were not addressed and the RoV simply ignored them. The RoV said the errors were resolved with the reconciliation procedure (which is not documented and does not have any report) but there is no confidence that this procedure actually resolves these issues. 
�	Of the original 17 precincts sampled, 5 had errors, (29% error rate). 
�	The additional 33 precincts combined (50 total) had 10 failing precincts, (20% error rate). 
�	5 ballots were lost in the 1% sample. By extension, 1134 ballots were lost (about 8 per each scanner on the floor). 
�	Apparently, no research into whether the vote was different (Vote Substitution error) unless ballots were missing or total vote count different. 
�	This issue is in addition to the 9,000 "missing" ballots due to removal of "damaged" and write-in ballots. 

See also Issue Issue A033: PEMT Peformed too early. 

Audit Trail  
Fair and accurate elections are essential to the operation of any democracy. To encourage confidence in our elections procedures, it should be possible for an outside organization to audit the results, with minimal cost, difficulty, and delay. Such transparency is far from reality today, with a complete review of the even the count of ballots processed nearly impossible. 
The following suggestions would improve the ability of an organization like Citizens Oversight to provide desperately needed outside oversight of the operation of elections in San Diego County: 
�	: Public Posting of Audit Trail -- The paper audit trail should be posted on the Internet by elections officials so anyone can review it. Details... 
�	: Questions should be answered -- Elections officials should be obligated to answer questions from oversight activities. Details... 
�	: No Ballot Inventory Report -- There is no report generated regarding whether any errors occurred during SPUW, including the count of ballots. Details... 
�	: Ballot Statements Not Scannable 
Ballot Statements must be scannable in their entirety, including signatures of precinct workers who sign them. Details... 
�	: Tally Center Reception Count Misleading 
The count of precincts that have been received by the Tally Center on election night is frequently cited by news media as the number of "Precincts Reporting." However, the ballots may not be even counted yet, and there are a vast number of VBM, provisionals, write-ins, etc. that have not been processed either. Details... 
�	: Tally Center - Write-Ins Removed and Not Counted 
We are not sure why the write-in ballots are removed permanently prior to scanning rather than scanning first and processing later or quickly processing them and uniting them with the rest of the precinct. Details... 
�	: Audit Log Insufficient 
The audit record available from the GEMS Central Tabulator does not provide sufficient detail to reconstruct the election. Details... 
�	: No Log of changes to the Election Management System 
The Election Management Systems is updated when a VBM ballot is received, however, there is no audit log of each transaction. Details... 
�	: Documents Automatically Processable 
Documents that are considered public, such as BallotStatements, Street Indexes, and others should be automatically processable to minimize oversight costs. Details... 
�	: Scanner Tapes Not Accessible 
Scanner report tapes are no longer available early in the canvass. Details... 
�	: Scanner Tapes Difficult to Process 
The audit trail provided by the current RoV procedures include cash-register style paper tape which is difficult to process and sometimes impossible to read. Details... 
�	: No Operator Notes 
A log of all operator activities should be maintained, with rationale for re-scans, etc. At present, no operator log is maintained and 133 re-runs occurred with no explanation. Details... 
�	: No Reconciliation Report 
A reconciliation report should be provided. This report should show the number of sign-in signatures, the number of ballots scanned and tallied, the number of provisional ballots accepted and rejected, the number of VBM ballots received at the precinct and the number processed, etc. such that the figures shown above all balance. Details... 
�	: Audit Devices Should be Allowed 
If no improvement to the audit log is possible, then audit devices should be placed on the serial interface cables that supply data to the central tabulator to allow a log of all communication from all scanner and DRE equipment to the central tabulator. Details... 
�	: No Paper Trail for VBM and Provisionals 
Unlike ballots processed on election night, no scanner reports are produced and ballots are read directly into the central tabulator. VBM ballots and provisionals should be scanned by scanners that produce a report, so these can be accessed later. (May be avoided if the audit log is improved and available.) Details... 
�	: No Seal Tampering Report 
A seal report should be generated that explains all tampered seals. At present 99 seals show tampering but there is no report nor any action taken to eliminate the tampering. Details... 
�	: Tally Sheets too big to process automatically 
The Post-Election Manual Tally (PEMT) should use paper that is easily processed by automatic document feeding equipment. Currently, the tally sheets are on over-sized paper and are difficult to scan. Details... 
�	: PEMT Reports Incomplete 
The PEMT should produce a report for every race and every precinct, as suggested by the SoS. Details... 
We would also like to suggest the actual ballots be completely imaged and provided to the public. For more information on this proposal see Open Canvass(44) A method to completely image the ballots in the election and allow stakeholders to review them. 

Issues Summary  
This part of the report lists all issues in numerical order. There are two major categories of issues. 
�	Issues related to RoV Procedures - These are issues about the procedures used by the RoV to process the election. These issues are numbered Axxx. 
�	Issues related to our research project - These issues are comments about our project in terms of missing oversight or improvements for future work. These issues are numbered Cxxx. 
Issues related to RoV Procedures 
Issue A001 
�	: Lack of Comprehensive Procedure Documentation -- RoV apparently does not maintain any comprehensive procedures documentation.
COPs requested procedure documentation on a number of occasions but were not successful in obtaining any comprehensive documentation for election procedures. There was inadequate documentation for (at least) the following procedures: 
�	Supply Pickup Weekend (SPUW) 
�	Blank Ballot Inventory Verification 
�	"Office Use" section of Ballot Statement (See Issue A008) 
�	Collection Center operation 
�	Tally Center operations 
�	Incoming Inspection 
�	Memory Card Pairing 
�	Scanning 
�	Memory card Reading 
�	Processing of VBMs & Provisionals 
�	Reconciliation Procedure (See Issue A026) 
Comprehensive procedure documentation is the first step to achieving high quality levels, as required by quality management standards such as ISO-9004-2 (2000) "Quality Management and Quality System Elements - Part 2: Guidelines for Services" as used in private industry. The lack of written procedures by the San Diego County Registrar Of Voters is a significant deficiency of the first step to achieving high quality levels, and points to the fact that local and state government institutions likely have no quality management system standards equivalent to those used by private industry. 

The two most important aspects of this standard for this application is  and  
�	: Question to RoV  
Does the RoV have any additional documentation of procedures used, including comprehensive procedure documentation? 
�	: Legislative 
ISO-9004-2 describes a quality management system for services, and can be fruitfully applied to services provided by governmental institutions, and election processing department, such as the Registrar of Voters. The Legislature should consider specifying the use of ISO-9004-2 as a basis for improving the quality of service and/or ISO-9001 (quality management system for products) for use by election districts. 
 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Lack Of Documentation 
�	Cops Canvass Report Procedure Summary 
Issue A002 
�	: Public Posting of Audit Trail -- The paper audit trail should be posted on the Internet by elections officials so anyone can review it.
Currently, oversight groups must access physical documents themselves or pay to have them scanned. Documents such as: Ballot Statements, Audit Log, Chain of Custody Report (disposition of all seals), and the Reconciliation Report are essential. 
�	: Question to RoV  
Please post all elements of the audit trail, such as Ballot Statements, Audit Log, Chain of Custody Report (disposition of all seals), and the Reconciliation Report. We would also like to see all ballots scanned and made available for review (See Open Canvass for details on this request). 
�	: Legislative 
Processing of our elections is integral to our democracy. All elements of the audit trail, such as Ballot Statements, Audit Log, Chain of Custody Report (disposition of all seals), and the Reconciliation Report should be available on the RoV website for review by oversight groups. The Audit Log should be improved, as is mentioned elsewhere in this report, and posted in its entirety. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Five Percent Review 
Issue A003 
�	: Questions should be answered -- Elections officials should be obligated to answer questions from oversight activities.
Elections officials should not have the option to use the "That's not a California Public Records Act Request" or "We don't have any document prepared that addresses that question, and therefore we won't answer it" responses that do nothing to assist oversight groups in their activities. 
�	: Question to RoV  
Please answer questions without attempting to subvert oversight using typical CPRA defenses. 
�	: Legislative 
Elections Officials should be obligated to answer reasonable questions from oversight groups, and not resort to "that is not a CPRA request." 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Lack Of Documentation 
Issue A004 
�	: No Ballot Inventory Report -- There is no report generated regarding whether any errors occurred during SPUW, including the count of ballots. 
�	: Question to RoV  
Please confirm that no Ballot Inventory Report exists, regarding whether any errors occurred during SPUW, including the count of ballots and please create one in the future. 
�	: Legislative 
Make creation of a "Ballot Inventory Report" mandatory, regarding whether any errors occurred during Supply Pickup Weekend, including the count of blank ballots and why any errors occurred, and how these errors can be rectified in the future. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Procedure Summary 
Issue A005 
�	: No Seal Serial Number on Ballot Statements 
Serial numbers of the tamper-evident seals are not included on Ballot Statements. 
Documenting serial numbers of seals is a requirement for a workable seal management strategy. 
�	: Question to RoV  
Please include serial number on ballot statements and add a procedure to note the serial numbers of all seals used on ballot cartons and anywhere else. 
�	: Legislative 
Recording of serial number of any tamper evident seals must be made mandatory. For San Diego, seal serial numbers should be recorded on ballot statements. Also, included in the comprehensive procedure documentation must be a procedure to note the serial numbers of all seals used on ballot cartons and anywhere else, and a procedure to investigate seals that show tampering. 
 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
Issue A006 
�	: Ballot Statements - Few balanced 
Only a few of the Ballot Statements had figures that balanced, and only one precinct had all the ballots correctly scanned. This is a significant issue that is separately treated in this report as a separate Issues of Concern Category. (See Unscanned Ballots) 

According to federal election guidlines, disenfranchisement of even one voter, or more than one lost ballot in 808,407 ballots cast due to equipment failure is not allowed. Tracking this problem starts with the sign-in rosters and BallotStatements completd at the polling place.

Polling workers are volunteers and are not specifically suspect of malfeasance, but improved training and intuitive reconciliation procedures should be established to minimize the discrepancies in the count of ballots. 
�	: Question to RoV  
(See Unscanned Ballots) 

�	Avoid multiple-precinct polling locations or separate them more distinctly so it will be very unlikely that a voter would turn in ballots to the wrong precinct ballot box. 
�	Perhaps use simple ballot scanners that detect the precinct and will cause an alert if a voter attempts to deposit a ballot into the wrong ballot box. 
�	Scanners could also provide a machine count of the ballots. 
�	Provide a worksheet to reconcile smaller aspects. Current reconciliation attempts to reconcile the total number of signatures with the total number of voted ballots. 
�	Separately reconcile provisional ballots and provisional sign-in sheets. 
�	Separate provisional ballots into two groups: 
�	VBM voters who did not surrender their VBM ballots who are listed with that precinct 
�	Other voters who are voting in the "wrong" precinct. 
�	Subtract provisional and touchscreen voters from total signatures to determine total paper ballot signatures and reconcile that with the number of actual ballots. 
�	Maintain sign-in rosters for ALL VBM ballots returned to each polling place. 
�	Currently, separate sign-in rosters are not maintained for VBM voters who do not wish to vote in person. 
�	Sign-in rosters will allow tracking the VBM ballots and provide a way to determine where in the system they are lost, if indeed they are. 
�	Provide sign-in locations for VBM voters who are listed as members of the precinct and designate if mail ballot was surrendered or not. Those who do not surrender their mail ballot should also sign the provisional roster and obtain a provisional ballot. With this designation, duplicate signatures can be easily located. 
�	Avoid the current practice of "signing over" the "MAIL BALLOT RECEIVED" designation in the signature box. 
�	Avoid the practice of putting a line through the signature box, which some precincts did apparently to help count them. 
�	Include a checkbox that can be marked by Precinct workers to designate that the scribble represents the voter's signature. Some signatures are so bad, they are hard to discriminate from random markings. 
�	The RoV should produce a report summarizing the count of signatures, the actual number of ballots returned, and an overall percentage of error. This report should be mandated by the SoS. The Registrar should encourage precincts to do better. Precincts that produce accurate reports that account for all signatures should be acknowledged with certificates of merit or other prizes. Similarly, those precincts that do not do well will may need to additional training or staffing changes. 
�	One suggestion to limit the error is to reduce the number of polling places. The size of precincts (111 to 674 ballots in our 5% sample) is about 5% to 33% of the suggested "typical" precinct size of 2000 ballots, as suggested by the Secretary Of State. On the average, polling places should handle 4 precincts and reduce the number of polling places to 400 instead of 1650. This would reduce the number of polling place workers needing training and allow more well-training workers to be allocated to the same polling places. The number and size of precincts can remain the same, with the vote split out as it is today using scanners that will read the precinct from the ballot. This would also eliminate the current error mode of placing the ballot in the wrong ballot box as well. 
�	: Legislative 
(See Unscanned Ballots) 
 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
�	Cops Canvass Report Unscanned Ballots 
Issue A007 
�	: Ballot Statements - Removed Write-ins not Documented -- Ballot Statements did not include a count of the number of write-ins and damaged ballots removed. Precinct workers are requested to look through the ballots and remove all write-ins and damaged ballots and place these into a separate group, which is to be banded. The incoming inspection activity has two checkoffs that relate to this: 1) write-ins removed and 2) write-ins banded. However, no one counts these so we know how many have been removed. This should be recorded on the Ballot Statements. In addition, these ballots (nearly 9,000 ballots) are withheld and not processed for an extended period of time. During this time, the "precincts reporting" figure should not claim that the precinct has been fully reported. 
�	: Question to RoV  
Please document the number of write-ins and damaged ballots and their disposition for all precincts in future elections. The Ballot Statement may be a good place to record the number. Until these ballots are included in the count, the "precinct reporting" figure should not be 100%. 
�	: Legislative 
Law should mandate that elections officials document the number of write-ins and damaged ballots and their disposition for all precincts in future elections. In addition, the "Precincts Reporting" figure released to the media, should not fully include any precincts that have not been fully counted, including the write-in ballots. 
 
�	Cops Canvass Report Procedure Summary 
�	Cops Canvass Report Unscanned Ballots 
Issue A008 
�	: Ballot Statements - Office Use Section Not Explained -- Ballot Statements have an "Office Use Only" section does not include an explanation to allow the public to understand how it is being used.
The processing of our elections should be an open process with no secrets. The "Office Use" section could be easily explained on the Ballot Statement form. The RoV should explain the use of this section and make that explanation available for anyone to read. 
�	: Question to RoV  
Please explain the office use section and add this explanation to procedure documentation available to the public. 
�	 Legislative 
Our election officials should run an open process, that is available for the public to inspect. Secret "office use only" sections must be avoided in forms and if they are used, how the section is used by elections officials must be explained. This issue can be rectified if the elections officials maintains comprehensive procedure documentation (See Issue A001) 
 
�	Cops Canvass Report Procedure Summary 
�	Cops Canvass Report Unscanned Ballots 
Issue A009 
�	: Ballot Statements Not Scannable 
Ballot Statements must be scannable in their entirety, including signatures of precinct workers who sign them. 
The RoV claims that it is not legal to scan the signatures of volunteers and staff members who complete the Ballot Statements. It is our claim that these documents should be scannable in their entirety and placed on any public website. 
�	: Question to RoV  
Change procedure when this issue is settled and described by elections code. 
�	: Legislative 
This issue needs to be settled in law since the RoV now claims that signatures of elections workers that authenticate the document are private and cannot be disclosed. Please settle this issue by improving elections code to explicity state that the Ballot Statements (and all similar documents) can be scanned and accessed by oversight groups. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Five Percent Review 
Issue A010 
�	: VBM Ballots Not Logged at Precinct 
VBM Ballots were not logged in at the precinct and apparently not placed in sealed containers. It is not necessary to sign anything to drop off your VBM ballot at the precinct or RoV office, and this is a deficiency. 
�	: Question to RoV  
VBM ballots should be counted and logged when dropped off at the RoV office or at precinct locations and placed in sealed containers. 
�	: Legislative 
Improve elections code to mandate logging of VBM Ballots dropped off at precinct locations, and require that they are counted and placed in sealed containers to avoid tampering. 
 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
Issue A011 
�	: Spoiled Ballots Loosely Controlled 
Spoiled Ballots, both VBM and election day are not under strict control. I.e. it would be a simple matter to checkmark a VBM envelope that the ballot was spoiled and eliminate it from consideration, and to covertly redo a vote and show a spoiled ballot at the polling place. 
To "spoil" a ballot, it should require the signature of the voter. Anyone could checkmark a VBM ballot as spoiled, and easily disenfranchise a voter. All spoiled ballots should be tracked to confirm that an unspoiled ballot is completed by the voter. 
�	: Question to RoV  
Modify procedure to tightly control spoiled ballots. 
�	: Legislative 
Mandate that elections officials maintain records of spoiled ballots to allow confirmation that a non-spoiled ballot was eventually turned in, and to avoid the possibility of disenfranchisement by a compromised elections official who could simply checkmark "spoiled" on target ballots. 
 
�	Cops Canvass Report Chain Of Custody 
Issue A012 
�	: Collection Center Seal Logs Lack Serial Numbers 
At collection Centers, they are supposed to inspect the tamper-evident seals, but there is no place on the log sheet to document the seal numbers nor to note whether the seals were intact. 
Only 6 tampered seals were detected at collection centers while 99 tampered seals were detected at the tally center. Documenting serial numbers of seals is a requirement for a workable seal management strategy. 
�	: Question to RoV  
Improve chain of custody reports at collection centers to include status and serial numbers of all security seals. Automated bar-code scanners can expedite the collection of seal serial numbers. 
�	: Legislative 
Mandate that chain of custody reports at collection centers include status and serial numbers of all security seals. 
 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
Issue A013 
�	: Tally Center Reception Count Misleading 
The count of precincts that have been received by the Tally Center on election night is frequently cited by news media as the number of "Precincts Reporting." However, the ballots may not be even counted yet, and there are a vast number of VBM, provisionals, write-ins, etc. that have not been processed either. Therefore, this number should not be made public. Instead, the figure should be the total number of scanned and accumulated in the central tabulator divided by the total number of ballots received both on election night and as VBM ballots. Also see Issue A007 regarding the removed write-in and damaged ballots that are not included in the count until much later. 
�	: Question to RoV  
RoV should discontinue using the "Precincts Reporting" figure and caution the media against using this figure in their reports. Instead, the figure cited should be the total number of scanned and tabulated ballots divided by the total number of ballots received, both those received on election night and those received as VBM ballots. 
�	: Legislative 
Mandate that the RoV should discontinue using the "Precincts Reporting" figure and caution the media against using this figure in their reports. Instead, the figure cited should be the total number of scanned and tabulated ballots divided by the total number of ballots received, both those received on election night and those received as VBM ballots. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Procedure Summary 
Issue A014 
�	: Tally Center - Write-Ins Removed and Not Counted 
We are not sure why the write-in ballots are removed permanently prior to scanning rather than scanning first and processing later or quickly processing them and uniting them with the rest of the precinct. It makes reconciling the number of ballots problematic because there is no notation as to how many "write-ins" are removed nor what is done with them. Approximately 9,000 ballots appear to be removed at this point. The scanners do tally the number of write-ins and there are many write-ins that were noted on the scanner tapes. We consider this an open issue as we do not have adequate information to understand exactly what happened, and the number of ballots -- 9,000 -- is certainly significant. We are not sure how the write-in ballots, received on election day, are processed. Only 146 ballots were removed and documented as "damaged" and the balance of nearly 9,000 ballots removed at this point are not documented.

Ballots should not be taken from the boxes prior to scanning. Currently, many counts do not match (9,000+ ballots missing), but are magically corrected later. The following figures should all match (or an explanation regarding how they are reconciled should be provided): 
�	The number of nonprovisional and nontouch-screen signatures on the Sign-In Rosters. 
�	The count of physical ballots in the ballot box at the end of election night. 
�	The number "cards cast", as reported on scanner tapes. 
�	The number "# CARDS", as reported by the audit log for that precinct. 
�	The number "Ballots cast, Polling", as reported in the final canvass report. 
See also Issue A007, Issue A013, Issue A015. 
�	: Question to RoV  
Why are write-in ballots removed prior to scanning rather than scanning first and processing later or quickly processing them and uniting them with the rest of the precinct. Is there any logging of the count of ballots removed in this manner for each precinct and the disposition of each one? 
�	: Legislative 
Mandate that elections officials maintain a correct count of all ballots from each precinct and maintain these ballots as a group, without removing write-ins prior to scanning. Any ballots removed for inspection by elections officials should be logged. The precinct should be considered fully counted until these ballots are included in the count. The PEMT should included these ballots in the tally (They are not included now). The scanners in use by the RoV can detect and report if a ballot has a write-in for a race, so there is no need to separate these in advance. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
�	Cops Canvass Report Unscanned Ballots 
Issue A015 
�	: Tally Center - Damaged Ballots 
On the Tally Center Log sheet, there is a checkbox for "Damaged Removed." However, we don't know why any ballot would become damaged and not be treated as "spoiled." 
A spoiled ballot, by definition, is one that is filled out incorrectly or damaged and a new ballot is issued to the voter. Why would there be any ballots that are "damaged" and a new ballot not issued to the voter? However, 146 ballots were separated as "damaged" instead of "spoiled" in the November, 2008, election. 
�	: Question to RoV  
146 ballots were separated as "damaged" instead of "spoiled" in the November 2008 election. When are ballots considered damaged and not spoiled? Why were another nearly 9,000 ballots removed that were not accounted for as "damaged" in the Tally Center Log Sheet? 
�	: Legislative 
Law should mandate that ballots for a given precinct should be counted and included in the canvass without separating out write-ins and damaged ballots, and if they are separated, this should be clearly documented. All ballots for the precinct should be included in the PEMT. 
 
�	Cops Canvass Report Procedure Summary 
�	Cops Canvass Report Unscanned Ballots 
Issue A016 
�	: Tally Center - 99 Seals broken or wrong 
In the November 2008 election, Tally Center Logs documented 99 precincts that had wrong or broken seals, a 6% rate, and there was no investigation. There were no exception documents provided for our review describing why the seals were incorrect or broken. Seal documents do not include a serial number. Only a cursory inspection was performed by elections officials, and there was no attempt to validate the count of ballots as reported by the precinct. Why was it the case that 99 seals were detected as wrong or broken at the Tally Center while only six were detected by the Collection Centers? Documenting serial numbers of seals is a requirement for a workable seal management strategy. See the Chain of Custody for more detail on this issue. 
�	: Question to RoV  
Why was it the case that 99 seals were detected as wrong or broken at the Tally Center while only six were detected by the Collection Centers? 
�	: Legislative 
It is not enough to use security seals. An effective security strategy requires that the security seals each have a serial number and that these are recorded at each step of the process. In San Diego, 6% of the seals were noted as tampered or wrong, but there was no investigation or report, and there was no tracking of serial numbers. If serial numbers are not tracked, there is no way to detect if a security seal is replaced with another seal after tampering. Election law must explicitly lay out the requirements for an effective security strategy and mandate compliance. 
 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
Issue A017 
�	: Scanner cards not Archived 
Memory cards used to capture the vote from each scanner are not maintained as a permanent record of the election. 
It is too difficult to access data to reconstruct the election. At present, the only option for oversight groups is to scan the report tapes produced by the ballot scanners, and these do not include any VBM or provisional ballots. If the memory cards were archived, then oversight groups would be able to reconstruct the election results based on those cards without needing to resort to scanning the scanner reports. Such oversight is a check of election manipulation within the GEMS central tabulator. Note: It is preferrable and less expensive for elections officials for oversight groups have access to a complete audit log. (i.e. including all information downloaded from the scanners, not just the count of ballots) and including the VBM and Provisional ballots scanned. (See Issue A018: Audit Log Insufficient and Issue A029: Audit Devices Should be Allowed) 
�	: Question to RoV  
If a  audit log is not available (Issue A018), and if no audit devices are allowed to collect all information transferred to the central tabulator (Issue A029: Audit Devices Should be Allowed) please archive the scanner memory cards so these can be accessed by oversight groups to easily reconstruct the results of the election for a sample without needing to scan the report tapes from the scanners. 
�	: Legislative 
If the complete audit log is not available (Issue A018), and if no audit devices are allowed to collect all information transferred to the central tabulator, elections officials should be required to archive the scanner memory cards as a record of the election so these can be accessed by oversight groups to easily reconstruct the results of the election for a sample without needing to scan the report tapes from the scanners. 
 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
Issue A018 
�	: Audit Log Insufficient 
The audit record available from the GEMS Central Tabulator does not provide sufficient detail to reconstruct the election.
The audit log should be improved to provide an exhaustive list of vote counts for every race and every precinct. This would result in a much larger file, but file size should not be cited as a factor limiting the provision of this information. COPs asked for documentation regarding the availability of additional reporting options, but the RoV said they did not have any documentation.

The following concerns exist: 
1.	The entire election must be included in the audit log. The log we were given did not include initial clearing and pre-election voting intervals. 
2.	The initial clearing command(s) must appear in the log if we are to know that the system was correctly initialized. 
3.	The RoV should have complete documentation for the audit log and for the GEMS central tabulator. 
4.	Every crash and error should be investigated and explained. 
5.	More robust entries are desired to allow an independent reconstructions of the election from the audit log. 
�	Ballot counts should be provided for VBM and Provisional processing. (See Issue Issue A020: No Log of changes to the Election Management System) 
�	Complete (exhaustive) report of each download to the GEMS central tabulator should appear in the log, including the total of votes for each race. 
6.	The log file should have a sequential number on each of the lines to eliminate the possibility that the audit log could be altered by election workers. In the log provided to COPs, we noticed anomalies that indicate that the log may have been tampered with, specifically the intervals when the the log had STOP or START entries without the other associated entry adjacent to it. 
7.	The RoV was apparently in violation of the requirements that they remake all touchscreen ballots into durable paper ballots, since on several occasions, votes were transferred directly from DRE equipment. 
8.	Oversight groups should have access to the audit log as the election is processed, and not be required to wait for final election certification. Nothing in the audit log can change once it is produced, and therefore, it is a public document that should be disclosed immediately, and not withheld until the election is certified, even though the log will become more extensive as the election is processed. 
Providing a complete audit log is the easiest, least expensive way to document the election while providing sufficient detail for oversight groups. 
�	: Question to RoV  
We asked for information about the availablity of other, more robust settings, for the GEMS central tabulator with regard to creation of the Audit Log. These questions were not answered and we were referred to Premier Election Solutions, however, our request for a contact at Premier was not ever answered. Please provide a contact with Premier so we can determine if there exist additional settings in the central tabulator that will allow the Audit Log to include the aspects we require for our audit. 
�	: Legislative 
Election law should be modified to mandate the creation of a exhaustive audit log file. See Issue A018 for details of what we require for our oversight review of the election. 
 
�	Cops Canvass Report Audit Log Discrepancies 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Five Percent Review 
�	Cops Canvass Report Procedure Summary 
Issue A019 
�	: No Comparison of Ballot Count 
There is no comparison of the number of ballots scanned, as reported on the report tape, with the number of ballots counted at the precinct nor by the count as read by the central tabulator.
Comparing the count of ballots as counted by polling place workers with the number scanned can eliminate the problems documented by the PEMT - One Percent Manual Tally, i.e. missing ballots. Only 4 of 85 precincts in our 5% sample scanned the number of ballots (Ballots Cast) that were counted by precinct workers. Comparing the count may have eliminated this problem (although this problem is more likely due to the removal of write-ins.) 

Workers do not routinely compare the count of ballots, in terms of the actual count and the count reported on scanner report with the count of ballots input to the GEMS central tabulator. Our project reviewed the count of ballots printed on scanner tapes with the audit report in our 5% sample and found no discrepancy. However, there have been recently reports of problems in Ohio using the same equipment. Ohio Voting Machines Contained Programming Error That Dropped Votes (Washington Post)(45) This article said: A voting system used in 34 states contains a critical programming error that can cause votes to be dropped while being electronically transferred from memory cards to a central tallying point, the manufacturer acknowledges. The problem was identified after complaints from Ohio elections officials following the March primary there, but the logic error that is the root of the problem has been part of the software for 10 years, said Chris Riggall, a spokesman for Premier Election Solutions, formerly known as Diebold. 
�	: Question to RoV  
In future elections, we request that you maintain an accurate ballot count (i.e do not remove write-ins and damaged ballots without noting the count of ballots removed) and compare that with the count of ballots actually scanned. This should eliminate the 29% failure rate as detected by the PEMT. 
�	: Legislative 
Election law should mandate that election processing procedures include a step that compares the count of ballots as counted manually and the count of ballots scanned for that precinct by electronic scanning equipment as soon as the scan is complete. This should eliminate the 29% failure rate as detected by the PEMT. 
 
�	Cops Canvass Report Audit Log Discrepancies 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
�	Cops Canvass Report Unscanned Ballots 
Issue A020 
�	: No Log of changes to the Election Management System 
The Election Management Systems is updated when a VBM ballot is received, however, there is no audit log of each transaction. A transaction log of VBM and provisional ballots must be made available for inspection. At this time, we are not aware of such a log. The Voter Data File is modified directly with no audit trail that can be inspected. If we had such an audit log, we could reconstruct the number of ballots actually received and compare that with the number of ballots scanned and entered into the central tabulator. At present, due to this lack of paper trail, such oversight is impossible. 
�	: Question to RoV  
Please confirm that no audit log exists that documents each change to the Election Management System when each VBM ballot is received. 
�	: Legislative 
The Audit Log should include entries for each VBM ballot as it is received so that it is possible to determine if all ballots have been scanned. This is equivalent to the Ballot Statement for precinct ballots, but for VBM ballots. A robust audit log (Issue A018) should include this aspect. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Procedure Summary 
Issue A021 
�	: VBM Ballots are not counted 
VBM Ballots are not counted after being removed from the envelope and before submitting for scanning, and there is no attempt to confirm the count of ballots as they are scanned. The RoV claims that workers count batches of ballots as they are submitted to the scanner and checking the total scanned, but this was not observed as actual behavior of staff doing the scanning and it would only guard against misfeeds, and would not provide a positive check that the total number of ballots submitted for scanning were actually scanned. This is a chain of custody concern.

We recommend the following procedure: 
�	Maintain a count of ballots that are processed at each step. 
�	A batch of ballots sent to be scanned should have a document that travels with it (a "traveler") that documents the number of ballots in the batch. 
�	The operator of the scanner should be able to start a scanning process and zero the count of ballots. Until the correct number is confirmed, the votes from that batch should not be added to the canvass total. 
�	The operator proceeds to scan the batch. 
�	If the count does not match what is contained in the batch, the discrepancy should be documented and the batch recounted and/or rescanned, and then the reason for the discrepancy resolved. 
�	When the ballot counts match, the batch is approved by the operator, and the totals sent to the central tabulator for accumulation. 
�	The audit log should document the exact number of ballots processed in the batch and the total votes in each race processed. 
�	: Question to RoV  
To maintain positive control of VBM ballots, it would be preferable to determine the total count of ballots to be scanned based on the number originally submitted, and then check the total number scanned as reported by the scanner. This should reduce the number of rescans necessary during the canvass period, should reduce errors, and should result in lower overall cost to process the VBM ballots. 
�	: Legislative 
Elections officials should be encouraged to maintain "positive control" of the ballots being counted, meaning that the count of ballots in batch boxes are documented in associated "traveler" documents. This allows workers to check that the number of ballots scanned exactly matches the count submitted, and allows rapid correction of errors. 
 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
Issue A022 
�	: VBM Ballots not precounted during scanning 
We were told that the scanner operator pre-counts the ballots as they run them, but this did not occur. The RoV said that during counting of VBM ballots, they post the count as they go through the scanner. Example: Count 10 ballots, run these through the scanner and at the end 10 ballots will display on the scanner display, count another 10 and scan these and the display will indicated 20 ballot scanned, etc. However, our video did not show any counting of ballots in batches to confirm the number counted as they claimed. This procedure is not documented in any procedure document and is not implemented.

Although this is a discrepancy from what we were told, the procedure of precounting ballots in batches does not do enough to provide positive control and chain of custody of those ballots. Instead, positive control should be maintained with a traveler document which proceeds with the ballots documenting the count of ballots in each batch. See Issue A021 for details. 
�	: Question to RoV  
See Issue A021 
�	: Legislative 
See Issue A021 
 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
Issue A023 
�	: Documents Automatically Processable 
Documents that are considered public, such as BallotStatements, Street Indexes, and others should be automatically processable to minimize oversight costs. Public documents, such as BallotStatements, Street Indexes, and others, should not have additional tabs added to them by poll workers, and they should be of a size appropriate for automatic scanning equipment. For example, tabs effectively limit the ability for automatic document feeding equipment to operate on the documents and thus increase the cost for an outside organization to gather this information. Instead, workers can use separation sheets with tabs to expedite flipping to the correct page. Sheets that are not conventional sizes cannot be easily scanned. Examples of unconventional sizes include scanner tapes and tally sheets. 
�	: Question to RoV  
�	: Legislative 
To make oversight of elections processes economically feasible, all public documents should be of appropriate sizes for automatic scanning, and there should be no tabs or other additions to those documents that will make manual scanning necessary. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Five Percent Review 
Issue A024 
�	: Scanner Tapes Not Accessible 
Scanner report tapes are no longer available early in the canvass. According to state election code, a second copy of the scanner report was required to be printed when these scanners were used in each precinct, that report being displayed outside the precinct door for anyone to inspect. However, the printing of the second report is no longer a practice when they are used in the Tally Center. This means we have to wait until the election is certified before we can start to review the election, which is an artificial restriction. The RoV did made the original reports available for inspection, and we scanned the reports from 5% of the precincts in our review.

See also Issue Issue A023: Documents Automatically Processable. 
�	: Question to RoV  
�	: Legislative 
To aid oversight groups to track election processing, the following should be corrected: 
�	Scanner Reports Available as soon as they are produced. 
�	Scanner Reports Easily processable (i.e. not register tape). 
�	Scanner Reports provided in electronic form. 
�	OR: Improved audit log (detailing the exact vote of each precinct, the same as the scanner reports) in electronic format which is available as the election is processed. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Five Percent Review 
�	Cops Canvass Report Procedure Summary 
Issue A025 
�	: Scanner Tapes Difficult to Process 
The audit trail provided by the current RoV procedures include cash-register style paper tape which is difficult to process and sometimes impossible to read. Transparency of our RoV procedures demands an audit trail that is much easier to process. Scanner reports should be produced on paper that is easily processable (such as cut-sheet letter paper) and with higher resolution printers that do not suffer from feed errors or light printing. These reports then can be easily scanned by oversight teams. (This suggestion is not essential if the audit log is improved or audit devices installed.) See also Issue Issue A023: Documents Automatically Processable, Issue A024: Scanner Tapes Not Accessible, Issue A018: Audit Log Insufficient, and Issue A029: Audit Devices Should be Allowed. 
�	: Question to RoV  
�	: Legislative 
Scanner reports must be improved to aid in oversight. See Issue A023 and A024. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Five Percent Review 
�	Cops Canvass Report Procedure Summary 
Issue A026 
�	: No Reconciliation Procedure 
The RoV goes through a lengthy process of reconciliation which has been referenced in many conversations with them, but is not documented in any procedure document, nor is there sufficient notes taken regarding what is done. There are many re-scans of precincts all the way to the very last day of the canvass period, but as mentioned, no notes as to why these re-scanning operations are being performed. Because there is no written procedure, no notes kept, and no final report, there is no confidence that the canvass is in fact completed from a technical standpoint. 
See Also Issue Issue A001: Lack of Comprehensive Procedure Documentation, Issue A027: No Operator Notes, Issue A028: No Reconciliation Report 
�	: Question to RoV  
Please provide the procedure documentation for the "reconciliation procedure" which is used to determine if the election is complete and no additional rescans need to be performed. 
�	: Legislative 
The RoV uses a "reconciliation procedure" that is undocumentated. This "seat of the pants" approach must be discontinued and a fully disclosed and documented procedure introduced with extensive notes taken along the way. Every rescan should be done for a reason, and the reason should be reviewed for possible improvement of the procedure. Without a written procedure and without any exception documents, it is impossible to feel confident that the canvass is complete, it is impossible to know if the procedure needs to be improved, and there is no path of action to improve the procedure. 
 
�	Cops Canvass Report Audit Log Discrepancies 
�	Cops Canvass Report Lack Of Documentation 
�	Cops Canvass Report Procedure Summary 
Issue A027 
�	: No Operator Notes 
A log of all operator activities should be maintained, with rationale for re-scans, etc. At present, no operator log is maintained and 133 re-runs occurred with no explanation. 
�	: Question to RoV  
See Issue A026: No Reconciliation Procedure 
�	: Legislative 
See Issue A026: No Reconciliation Procedure 
 
�	Cops Canvass Report Audit Log Discrepancies 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Procedure Summary 
Issue A028 
�	: No Reconciliation Report 
A reconciliation report should be provided. This report should show the number of sign-in signatures, the number of ballots scanned and tallied, the number of provisional ballots accepted and rejected, the number of VBM ballots received at the precinct and the number processed, etc. such that the figures shown above all balance. See also Issue Issue A026: No Reconciliation Procedure. 
�	: Question to RoV  
Please explain the crashes and program errors encountered by the central tabulator in the November 2008 election with possible assistance by the vendor (Premier Election Solutions in this case). 
�	: Legislative 
It should be the law that crashes and program errors reported in the audit log should be described and resolved in the final canvass report without the need for oversight groups to make requests to have these explained. The reconciliation report is a logical place for these explanations. 
 
�	Cops Canvass Report Audit Log Discrepancies 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Procedure Summary 
Issue A029 
�	: Audit Devices Should be Allowed 
If no improvement to the audit log is possible, then audit devices should be placed on the serial interface cables that supply data to the central tabulator to allow a log of all communication from all scanner and DRE equipment to the central tabulator. Data from those audit devices would allow an oversight activity to reconstruct the election and guard against tabulator-based election manipulation by compomised elections workers, and can eliminate the need to scan scanner report tapes and other difficult to process documents. 
�	: Question to RoV  
�	: Legislative 
If an exhausting audit log is not possible to create, then it should be possible to install audit devices on the serial interface cables on all scanner and other vote counting equipment to allow exhaustive inspection of the vote-count data being sent to the central tabulator. Such audit devices can significantly improve the reliability of the canvass and guard against manipulation of the vote count by compromised elections workers working using the central tabulation equipment or due to programming or innocent ballot handling errors. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Five Percent Review 
Issue A030 
�	: No Paper Trail for VBM and Provisionals 
Unlike ballots processed on election night, no scanner reports are produced and ballots are read directly into the central tabulator. VBM ballots and provisionals should be scanned by scanners that produce a report, so these can be accessed later. (May be avoided if the audit log is improved and available.) 

VBM Ballots, once removed from their envelope, have no paper trail, such as a scanner tape, so that the process can be checked for accuracy. The PEMT shows that 29% of precincts had missing ballots. 
�	Although not reviewed in this report, it may be feasible to check the count of VBM ballots received, as reported in the voter database with the count of ballot processed in the entire VBM processing activity. 
�	VBM Ballots could be placed under positive control by including a count of ballots in each batch on documentation that would accompany the ballots. Scanner operators would produce a scanner tape report to confirm the correct number of ballots were processed. Current practice has the operators counting small batches of ballots to see if they are scanned correctly, but this practice does not confirm that ballots were not lost along the way. 
See also Issue Issue A021: VBM Ballots are not counted. 
�	: Question to RoV  
�	: Legislative 
The law should be changed to require that positive control is maintained over all ballots being processed. See Issue A021. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Chain Of Custody 
Issue A031 
�	: No Seal Tampering Report 
A seal report should be generated that explains all tampered seals. At present 99 seals show tampering but there is no report nor any action taken to eliminate the tampering. It is insufficient to simply report the number of seals that were broken or incorrect. The report should also indicate why the tampering occurred and how it can be eliminated in the future. The fact that 99 seals showed tampering is an unbelievable figure, particularly since there was no follow-up or explanation. Also, since no serial numbers are recorded, the number of tampered seals may be much larger. 
�	: Question to RoV  
�	: Legislative 
Elections officials should be required to prepare a seal tampering report to include at least the following: 
�	Enumeration of each seal broken, incorrect, or missing. 
�	Explanation regarding why each seal was tampered. 
�	Changes to procedure proposed to reduce false positive detections of tampering. 
�	Notices of termination of workers responsible for tampering. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
Issue A032 
�	: Archival Procedures Lacking 
There are no check-in and check-out procedures in the secure archive area and only documentation describing where the materials are in the storage facility. There is no way to know who accessed the records to determine if any may have been modified. There are no seal serial numbers maintained on the documentation. 
See also Issue A001: "Lack of Comprehensive Procedure Documentation". 
�	: Question to RoV  
Please establish rigorous check-in and check-out procedures for material in secure areas. There is no way to know who accessed the records to determine if any have been modified. 
�	: Legislative 
Law should be improved to establish basic requirements for secure archival operation, to include at least: 
�	Check-in and Check out procedures to document who has accessed the materials. 
�	Serial numbers of any security seals used in the operation of the secure archive. 
�	Closed-circuit TV recording, video archival, and review procedures. 
 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Lack Of Documentation 
�	Cops Canvass Report Procedure Summary 
Issue A033 
�	: PEMT Peformed too early 
The PEMT is performed too early after election night and does not include a complete set of ballots for the precincts under inspection. The PEMT should be performed after all write-ins, VBM and provisional ballots have been processed. The write-ins that were removed on election night certainly should be completed before starting. 
�	: Question to RoV  
�	: Legislative 
The Law should be refined to define exactly when the PEMT should be performed and what should be included in it. The law should define: 
�	that the PEMT should include all ballots completed at polling places on election night, including all write-ins and "damaged" ballots, and it also should include all VBM ballots received by the deadline for those precincts. 
�	that the PEMT should also review Ballot Statements and determine irregularities. 
�	that it is not allowed to rescan offending precincts and then inappropriately deem the precincts as error free. 
�	That comparison with a computer report should use the final canvass report, not an interim report. (See Issue A036) 
 
�	Cops Canvass Report Manual Tally Procedure 
Issue A034 
�	: PEMT - Notification on Web Site 
The RoV should notify the public on the RoV website regarding the selection and tally in addition to placing a notice at the front counter. The web site is much more accessible than just the front counter and will incur no significant additional cost for the RoV but will make coordination by the public much easier. 
�	: Question to RoV  
Please provide notification of selection and tally of PEMT precincts on the web site as well as the front counter. 
�	: Legislative 
Include minor revisions to elections code to require that all electing districts provide notices on their web site as well as notices on the front counter of their office. 
 
�	Cops Canvass Report Manual Tally Procedure 
Issue A035 
�	: PEMT Random Selection Not Technically Correct 
The random selection of precincts procedure is incorrect and not generally applicable. COPs recommends that the Registrar Of Voters change this procedure as follows: 
�	Use four ten-sided die (which are easily available on-line for $3.99 for a set of 10 die). 
�	Roll the dice perhaps in separate boxes or use different colors to allow them to be treated as ones, tens, hundreds and thousands. 
�	Declare that on the die representing thousands, even numbers will be considered "0" and odd numbers represent "1". 
�	If the total number shown on the die is greater than the highest numbered precinct sequence number (1650 in this case), disregard the roll and roll again. 
�	: Question to RoV  
�	: Legislative 
The elections code should specify exactly how to randomly select the precincts because the statistics of random selection is frequently not well understood by elections workers. 
 
�	Cops Canvass Report Manual Tally Procedure 
Issue A036 
�	: PEMT Computer Report Manipulation Vulnerability 
There is a manipulation vulnerability regarding how the computer reports are generated. Current practice has the RoV staff produce reports which are, because the PEMT is currently performed early in the canvass cycle, tentative at best. The reports used by the PEMT should be the (draft) final canvass report to eliminate the possibility that a compromised member of RoV staff might substitute a known correct report for the precincts to be sampled while allowing the other precincts to be modified. 

If the election was fraudulently manipulated by RoV staff using the central tabulator, then it would be possible to cover that up by producing "good" reports for the precincts in the One Percent Manual Tally but there is sufficient slop between those and the final results to cover it up.
See also Issue Issue A033: PEMT Peformed too early. 
�	: Question to RoV  
To avoid this vulnerability, the final canvass report should be used, and the actions of the IT staff observed by outsiders during the generation of these reports. 
�	: Legislative 
Elections code should specify that PEMT procedure should use the (draft) final canvass report and not interim reports. The PEMT precincts should be unknown until after the (draft) final report is produced to eliminate the possibility that the report could be generated differently for PEMT precincts than the untested precincts. 
 
�	Cops Canvass Report Manual Tally Procedure 
Issue A037 
�	: Manual Tally Done Backwards 
Current practice treats the output of the computer scanners as the known good result and the tally teams redo their work until they can match it. This is contrary to the principle that the tally teams should be the most reliable result. Therefore, the PEMT procedure should be modified as follows: 
1.	Tally each precinct by two tally teams, each operating as they are now. 
2.	Compare the result of the two teams. 
3.	If they differ, tally again by a third team 
4.	If possible, determine what causes them to differ and determine the "correct" result. 
5.	Compare this result with the report as generated by the electronic scanners. 
6.	If the scanners produce a different result, rescan the ballots to determine if that is the source of the error. 
7.	If an error is detected, it may require that all ballots be rescanned using different equipment. 
�	: Question to RoV  
Please modify the PEMT procedure as described in Issue A037: "Manual Tally Done Backwards" 
�	: Legislative 
Improve elections code law to clearly define how the PEMT tallying should be done with the assumption that the tally teams should produce an independent result, and not rely on the computer report. 
 
�	Cops Canvass Report Manual Tally Procedure 
Issue A038 
�	: PEMT Rescanning 
There is no provision in the current Post Election Manual Tally procedure to allow that workers rescan precinct ballots if the vote determined by the manual tally differs from the electronic scan. However, the RoV rescans the ballots. They confirm the error (i.e. additional ballots are discovered in the re-scan) but the RoV nevertheless declares that the precinct is error free. The rescanning procedure to confirm an error must be included in the written procedure if it is to be performed, and then the consequences of finding the error must also be described, such as performing a complete election rescan. 
See also Issue Issue A033: PEMT Peformed too early. 
�	: Question to RoV  
�	: Legislative 
Elections code law should be enhanced to clearly specify that it is not allowed to simply rescan the offending precincts in the PEMT and fix any problems and then declare everything is resolved. The conclusion is quite the opposite. If there are offending precincts, a full explanation should be included, and if the precinct can be corrected by rescanning, that would imply that the entire canvass may be faulty, and may need to be completely rescanned after corrections are made to scanner operating procedure. 
 
�	Cops Canvass Report Manual Tally Procedure 
Issue A039 
�	: Tally Sheets too big to process automatically 
The Post-Election Manual Tally (PEMT) should use paper that is easily processed by automatic document feeding equipment. Currently, the tally sheets are on over-sized paper and are difficult to scan.
See Issue Issue A023: Documents Automatically Processable. 
�	: Question to RoV  
Please use Tally Sheets that conform to standard sheet sizes so they are easy to automatically scan, and avoid staples, tabs, and other additions that will make scanning difficult. 
�	: Legislative 
See Issue A023: Documents Automatically Processable. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Manual Tally Procedure 
Issue A040 
�	: PEMT Reports Incomplete 
The PEMT should produce a report for every race and every precinct, as suggested by the SoS. These sheets are easy to prepare if they are prepared when the manual tally is conducted. The resulting number of pages should be at least (Number of Precincts)*(Number of Races). For the November 2008 election, this would have produced about 400 pages for VBM and 400 pages for election day results. (The number of races applicable to any Precinct is less than the total number of races run in the county as many are not county-wide.) The report prepared by the RoV had only five pages, not 800, and did not include any vote counts. By comparison, Los Angeles County produced 1,300 pages in their report. 
�	: Question to RoV  
The SoS suggests a format for the PEMT report but the RoV ignores those requirements. We have requested compliance in past elections and have yet to see the RoV comply with the recommended format. We require compliance to help us with our oversight task. 
�	: Legislative 
The SoS suggests a format for the PEMT report but the RoV ignores those requirements. This should be specified in law with penalties to provide a means to force compliance. 
 
�	Cops Canvass Report Audit Trail 
�	Cops Canvass Report Manual Tally Procedure 
Issue A041 
�	: PEMT Should Reconcile to Rosters 
The PEMT does not look at the ballot statements and sign-in rosters, and are restricted to the ballots actually run through the scanners on election night. Unfortunately, since there are many (9,000) write-ins removed, the PEMT does not match the number of people who signed the rosters at the polling place. The PEMT should include a complete look at the precincts under consideration and not limit the view to just the operation of the scanners. 
See also Issue A033: PEMT Peformed too early. 
�	: Question to RoV  
Please modify the procedure used in the PEMT to include review of sign-in rosters and BallotStatements, and include ALL ballots to match sign-in rosters and ballot statements. 
�	: Legislative 
Law should specify that PEMT should reconcile ballot counts to original sign-in rosters, not just to the computer reports. See Issue A033: PEMT Peformed too early. 
 
�	Cops Canvass Report Manual Tally Procedure 
Issue A042 
�	: PEMT Errors Not Addressed 
In the November 2008 Election, a number of errors were detected by the Post Election Manual Tally (PEMT) procedure. These issues were not addressed and the RoV simply ignored them. The RoV said the errors were resolved with the reconciliation procedure (which is not documented and does not have any report) but there is no confidence that this procedure actually resolves these issues. 
�	Of the original 17 precincts sampled, 5 had errors, (29% error rate). 
�	The additional 33 precincts combined (50 total) had 10 failing precincts, (20% error rate). 
�	5 ballots were lost in the 1% sample. By extension, 1134 ballots were lost (about 8 per each scanner on the floor). 
�	Apparently, no research into whether the vote was different (Vote Substitution error) unless ballots were missing or total vote count different. 
�	This issue is in addition to the 9,000 "missing" ballots due to removal of "damaged" and write-in ballots. 

See also Issue Issue A033: PEMT Peformed too early. 
�	: Question to RoV  
You have refused in the past to answer our question as to why 29% of the precincts in the 17-precinct sample had errors of some kind, with 5 ballots lost. This loss exceeds the two-ballot loss allowed for all precincts. 
�	: Legislative 
The RoV simply ignores errors revealed by the PEMT by rescanning those precincts, these failures (29% of the precincts tested) actually proves that there are serious problems with the processing performed by the Registrar of Voters in San Diego County. Election law must be improved to clarify the responsibilities of the RoV with respect to errors resolved by the PEMT procedure. 
 
�	Cops Canvass Report Manual Tally Procedure 
Issue A043 
�	: Audit Log Discrepancies 
In the November 2008 Election, a number of discrepancies were detected in the Central Tabulator (GEMS) Audit Log. 
�	The Registrar does not have full documentation for the audit log, such as what all entries mean and any options or configuration settings in terms of what is provided in the audit log. (Their reply to Question C00021-21.1) 
�	The Registrar has no procedures for review of the log. (Their reply to Question C00021-21.2) They said the logs are reviewed by IT staff to verify that an action has taken place such as a batch has been closed and the correct number of cards cast has been captured. But there was no documentation regarding when these reviews took place, if in fact they ever did, and no report resulting from any review. 
�	In ten places, the log showed SERVER STOPPED without first seeing SERVER STARTED and vice versa. These should be paired in all cases. If they are not paired, then transactions may have been deleted (C00021-21.7.2). An explanation of this anomaly was requested from the Registrar (C00021-21.7.1) and they did not have an answer to fully explain it other than saying it could be a myriad of reasons. The Registrar suggested that we contact Premier Election Solutions, but no referral was ever provided. 
�	Crashes. There were a number of crashes and other error messages. The Registrar suggested that we contact Premier Election Solutions but no referral was ever provided. 
�	Re-running precincts. In 133 cases, precincts were re-run but no explanation given as to why they were re-run and no report as to why the initial run was in error. (See Issue A027: No Operator Notes) 
�	Clearing not shown. The central tabulator should be fully cleared prior to accumulating votes, but this was not in the log. Apparently, the RoV does not produce a log for the entire election process. 
�	: Question to RoV  
�	Please obtain full documentation about the audit log, including configuration settings to allow the GEMS system to provide additional detail. (Ref: C00021-21.1) 
�	Please generate procedures for review of the log and appropriate reports for any anomalies. (Ref. C00021-21.2). 
�	In that review, note time that the log shows SERVER STOPPED without first seeing SERVER STARTED and vice versa, as well as any crashes and error messages, and whether initial clearing is shown. 
�	Please contact Premier Election Solutions and obtain an explanation for why START and STOP are not always paired, and obtain an explanation regarding the crashes and error messages. 
�	Please provide a contact at Premier Election Solutions so we can ask for additional information. 
�	Please utilize written procedures for all operations, including why certain precincts are being re-run. (See Issue A027: No Operator Notes) 
�	The central tabulator should be fully cleared prior to accumulating votes, and this should appear in the log. 
�	Transactions for the entire election should appear in the log. 
�	: Legislative 
The audit log can be our most important tool to provide oversight to the canvass process. However, the log must be much more complete and detailed to be of the most use to oversight groups. 
�	Full documentation should be available to GEMS and the audit log. We request this so we can suggest additional settings that may also be available to increase the level of detail in the log. 
�	Procedures should exist for review of the log, and it should be reviewed and a report created for each election. The review should include an explanation for any and all anomalies, such as: 
�	Places where the log shows SERVER STOPPED without first seeing SERVER STARTED and vice versa. These should be paired in all cases. If they are not paired, then transactions may have been deleted. 
�	Crashes. There were a number of crashes and other error messages. The Registrar suggested that we contact Premier Election Solutions but no referral was ever provided. 
�	Clearing shown. The central tabulator should be fully cleared prior to accumulating votes. 
�	The reconciliation procedure (See Issue A026: "No Reconciliation Procedure") should include an explanation for any and all precincts that are rescanned. In 133 cases, precincts were re-run but no explanation given as to why they were re-run and no report as to why the initial run was in error. (See Issue A027: "No Operator Notes") 
 
�	Cops Canvass Report Manual Tally Procedure 
Issues related to our research project 
These are areas where we could do more work to witness operations or obtain documents for review. 
Issue C001 
�	: Logic Testing Witnessing 
Logic Testing Procedures not witnessed nor video recorded by our project team. 
�	: Question to RoV  
�	: Legislative 
 
�	Cops Canvass Report Procedure Summary 
Issue C002 
�	: Ballot Order Chain of Custody 
We did not review the process of maintaining control of blank ballots that are not distributed to precincts and oversight of the printer vendor. There is a concern regarding chain of custody of blank ballots. Is there a vulnerability to fraud based on accessing blank ballots and then removing and replacing completed ballots with those blank ballots which have been voted according to the desires of the instigators? 
�	: Question to RoV  
�	: Legislative 
 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
Issue C003 
�	: SPUW Observation 
Supply pickup weekend not witnessed nor video recorded. 
�	: Question to RoV  
�	: Legislative 
 
�	Cops Canvass Report Chain Of Custody 
�	Cops Canvass Report Procedure Summary 
Issue C004 
�	: Precinct Worker Instructions 
RoV indicated that precinct worker instructions were available for review on website but those are not really available to the public. We need to review these instructions. 
�	: Question to RoV  
Please provide Precinct Worker Instructions and related procedures documents. 
�	: Legislative 
 
�	Cops Canvass Report Procedure Summary 
Issue C005 
�	: Collection Center Observation 
We did not witness nor record activities at any collection centers. 
�	: Question to RoV  
�	: Legislative 
 
�	Cops Canvass Report Procedure Summary 
Issue C006 
�	: Tally Center Incoming Reception Observation 
We witnessed Tally Center Incoming Reception but did not video record it directly. 
�	: Question to RoV  
�	: Legislative 
 
�	Cops Canvass Report Procedure Summary 
Issue C007 
�	: VBM & Provisional Process Witnessing 
The Vote-By-Mail and Provisional Ballot Processing procedure was not recorded on video and our description was generated solely through Q&A with RoV staff. 
�	: Question to RoV  
�	: Legislative 
 
�	Cops Canvass Report Procedure Summary 

End Notes 
Notes 
 : http://www.copswiki.org/w/bin/view/Common/CopsCanvassReport This report may be reviewed on the Internet - 
 : http://www.copswiki.org/w/bin/view/Common/RaymondLutz Raymond Lutz - 
 : http://www.copswiki.org/w/bin/view/Common/OpenCanvass Open Canvass method - 
 : http://www.copswiki.org/w/bin/view/Common/M434 Election-night processing on the Feb 5, 2008 presidential primary election. - 
 : http://www.copswiki.org/w/bin/view/Common/M688 Nov. 21, 2008 Meeting with Deborah Seiler (video) - After this initial meeting, the Registrar refused to allow us to record our meetings, and so we discontinued that approach and instead use written requests only. 
 ,  : http://www.copswiki.org/w/bin/view/Common/VoluntaryVotingSystemGuidelines Voluntary Voting Systems Guidelines - were authored by the Technical Guidelines Development Committee (TGDC), a committee authorized under the HELP America Vote Act (HAVA) of 2002, and researchers at the National Institute of Standards and Technology (NIST) for the Election Assistance Commission (EAC). These guidelines were authored primarily to guide voting machine manufacturers in their design of voting systems. 
 : http://www.copswiki.org/w/bin/view/Common/M829 Logic & Accuracy Testing Procedure - An RoV Procedure Document that specifies how to confirm basic operation of the electronic scanning equipment. 
 : http://www.copswiki.org/w/pub/Common/M717/COP_PRA_Response_Attach_2.pdf Ballot Order Procedure - An RoV procedure document that specifies how the number of ballots is determined. 
 : http://www.copswiki.org/w/pub/Common/M821/C00019_Response_20.3.3_Ballot_order_and_supplemental_order_for_11-08.xls Ballot Order Spreadsheet - An RoV Election Document that lists the quantity of ballots ordered for each precinct and supplemental orders. Includes Sequence Number to Precinct Number conversion. 
 : http://www.copswiki.org/w/pub/Common/M717/COP_PRA_Response_Attach_1.pdf Ballot Inventory sheet - An RoV Election Document. This is a sample of the document sent to each precinct leader to verify the count of blank ballots received. 
 : http://www.copswiki.org/w/bin/view/Cops/StreetRosters Street Indexes - An RoV Election Document that lists voters by address and each is checked off as they vote. About half of our five percent sample were scanned but we discovered they were incomplete as most polling places did not continue to complete these after 6pm on election day. 
 : http://www.copswiki.org/w/bin/view/Cops/BallotStatements Ballot Statements - An RoV Election Document. These were Initially withheld by the RoV due to legal concerns of showing signatures. (Requested in C00022). These have been scanned by the Registrar with the signatures concealed. 
 : http://www.copswiki.org/w/bin/view/Cops/CollectionCenterLogs Collection Center Logs - Reports from 80 collection centers were scanned from RoV on 2009-03-23. Each report handles 21 precincts, on the average. No logging of seal conditions or serial numbers. 
 : http://www.copswiki.org/w/pub/Common/M717/COP_PRA_Response_Attach_4.pdf Collection Center Seal Report - Describes 6 seals that are missing or wrong 
 : http://www.copswiki.org/w/bin/view/Common/M828 Tally Center Log - Use by incoming inspectors at the Tally Center. Describes 99 seals broken or wrong, 6% error rate. 
 : http://www.copswiki.org/w/bin/view/Common/SecuritySealStandards Security Seal Standards - includes ISO/PAS 17712 - Freight container seal standards. 
 : http://www.copswiki.org/w/bin/view/Cops/ScannerTapes Scanner Tapes for our 5% Sample - These pdf files contain three scanner tapes each, and were manually scanned by COPs volunteers. 
 : http://www.copswiki.org/w/bin/view/Common/M830 One-Percent Manual Tally Procedure - An RoV Procedure Document 
 : http://www.copswiki.org/w/bin/view/Common/M691 San Diego County Manual Tally Report for 2008-11-04 election - This five-page report is extremely deficient compared with the recommendation of the Secretary of State, which would result in about 800 pages. This report includes NO vote totals and only a count of the ballots. 
 : http://www.copswiki.org/w/bin/view/Common/M831 Audit Log (of central tabulator) - An RoV Election Document produced by the GEMS Central Tabulator. 
 : http://www.copswiki.org/w/bin/view/Common/M667 C00014: Letter to Registrar of Voters requesting Statement of Votes Cast, 2008-10-03 - We requested that the RoV create two copies of the scanner tapes, as specified by law, so that we could start our review of the election right away. They refused, forcing us to wait until the election was certified before we could start to review the scanner tapes. 
 : http://www.copswiki.org/w/bin/view/Common/M668 C00015: Letter to Secretary of State requesting support of our request to Registrar of Voters requesting Statement of Votes Cast, 2008-10-08 - We forwarded our letter to the RoV (C00014) to the Secretary of State in an effort to persuade the county to comply with access to the scanner report tapes prior to the end of the election. This request was refused, with the rationale being that state law applies only in the precinct and they are not required to create a second scanner tape prior to election certification. 
 : http://www.copswiki.org/w/bin/view/Common/M691 C00016: Letter to RoV: Manual Tally Report Questions, 2008-11-20 - This three-page request was submitted prior to the closing day of the canvass period and pointed out that they were not using the appropriate format for their PEMT report. 
 : http://www.copswiki.org/w/bin/view/Common/M689 C00016 R: Response by County Counsel regarding Nov. 20 Letter to the RoV, 2008-11-25 - This response was the least cooperative and would not answer simple questions regarding why ballots were missing in the PEMT report. 
 : http://www.copswiki.org/w/bin/view/Common/M716 C00017 - Letter to San Diego Registrar of Voters regarding Election Procedures (2008-12-02) - This five-page request was submitted on the closing day of the canvass period. 
 : http://www.copswiki.org/w/bin/view/Common/M717 C00017 R - Response to C00017 by San Diego Registrar of Voters regarding Election Procedures (2008-12-16) - Includes Ballot Order Procedure, One Percent Manual Tally Procedure, Collection Center Comments on Carton Seals. 
 : http://www.copswiki.org/w/bin/view/Common/M819 C00019 - Letter to San Diego Registrar of Voters regarding Election Processing (2009-01-08) - This 19-page request was a followup to C00017, and was submitted twice, the second time (C00021) in an abbreviated format to aide in processing. 
 : http://www.copswiki.org/w/bin/view/Common/M821 C00019 R - Response to C00019 and C00021 by San Diego Registrar of Voters regarding Election Procedures (2009-02-24) - 
 : http://www.copswiki.org/w/bin/view/Common/M878 C00022 - Questions to Registrar of Voters on Nov, 2008 Election (Part 3) (2009-09-01) - Request for Ballot Statements, Street Indexes, Question about ballot counts, and Premier contact. 
 : http://www.copswiki.org/w/bin/view/Common/M879 C00022 R: Response to C00022 by Registrar of Voters (2009-09-09) - Answered some questions. Ballot Statements took several weeks to finally get in full, no contact to Premier Election Solutions provided. 
 : http://www.copswiki.org/w/bin/view/Common/M860 Top-to-Bottom Review - 
 : http://www.copswiki.org/w/bin/view/Common/M434 Video of the February 5, 2008 Presidential Primary Canvass - 
 : http://www.copswiki.org/w/bin/view/Common/M829 Logic & Accuracy Testing Procedure - 
 : http://www.copswiki.org/w/pub/Common/M717/COP_PRA_Response_Attach_2.pdf Ballot Order Procedure - which describes how the number of ballots is determined based on the number of registered voters in that precinct, including vote-by-mail voters, and predicted language variations. Each ballot costs $0.28 and are supplied in groups of 50. 
 : http://www.copswiki.org/w/pub/Common/M717/COP_PRA_Response_Attach_1.pdf Ballot Inventory sheet - which indicates the precinct number, the sequence number for that precinct, and the quantity of ballots sent to that precinct, by language version, completed by RoV staff. 
 : http://www.copswiki.org/w/bin/view/Cops/BallotStatements Ballot Statements - These scanned Ballot Statements have had the poll workers signatures redacted. 
 ,  : http://www.copswiki.org/w/bin/view/Common/617 Ohio Voting Machines Contained Programming Error That Dropped Votes (Washington Post) - This article said: A voting system used in 34 states contains a critical programming error that can cause votes to be dropped while being electronically transferred from memory cards to a central tallying point, the manufacturer acknowledges. The problem was identified after complaints from Ohio elections officials following the March primary there, but the logic error that is the root of the problem has been part of the software for 10 years, said Chris Riggall, a spokesman for Premier Election Solutions, formerly known as Diebold. 
 : http://www.copswiki.org/w/bin/view/Common/M830 One-Percent Manual Tally Procedure - 
 : http://www.copswiki.org/w/bin/view/Common/M880 William H. Beyer, Ph.D. , 25th Edition, CRC Press (1978) - using 'A Table of 14,000 Random Digits' (p 545-548) 
 : http://www.copswiki.org/w/bin/view/Cops/AllPrecincts All Precincts - provides results of our data collection activity for all the precincts in our 5% sample. (This table is attached to this document in the hardcopy version of the report.) 
 : http://www.copswiki.org/w/bin/view/Cops/AllRaces All Races - provides a link to a separate page for each race in the election and lists the vote count from each precinct in the 5% sample. 
 : http://www.copswiki.org/w/bin/view/Cops/PrecinctSeq1563 Precinct Sequence 1563 - 
 : http://www.copswiki.org/w/bin/view/Common/OpenCanvass Open Canvass - A method to completely image the ballots in the election and allow stakeholders to review them.