Human Factors Public Working Group

Table of ContentsUseful Links

WebHome > ConstituencyGroups


This working group is for the discussion and development of guidance for voting system human factors-related issues, including for accessibility and usability. The guidance will inform the development of requirements for the Election Assistance Commission (EAC) Voluntary Voting System Guidelines (VVSG).


  • Identify gaps/issues with current accessibility and usability requirements for voting including priority gaps/issues that are clearly within the VVSG scope.
  • Collect resource materials and develop recommendations to address priority gaps/issues.
  • Ensure consideration and implementation of accessibility and usability in all election processes identified by other working groups.
  • Ensure integrated and balanced consideration of accessibility and usability within the work of cyber security, interoperability, and testing constituency groups.

VVSG 2.0 Human Factors Deliverables

DRAFT User-Centered Design/Usability Testing Guidance and Templates

DRAFT VVSG 2.0 Requirements Guidance

Supporting Reference Material

Surveys and Reports on Voting Accessibility Issues

  1. "Disability, Voter Turnout, and Voting Difficulties in the 2012 Elections" report by Lisa Schur and Doug Kruse, Rutgers University
  2. "Defining the Barriers to Political Participation for Individuals with Disabilities" report by Thad E. Hall and R. Michael Alvarez
  3. "Experience of Voters with Disabilities in the 2012 Election Cycle", National Council on Disability, October, 2013
  4. "The Blind Voter Experience: A Comparison of the 2008, 2012, and 2014 Elections", National Federation of the Blind, December, 2014
  5. "The Experience of Blind and Visually Impaired Voters Who Used Online Ballot Delivery Systems during the November 2014 General Election", National Federation of the Blind, January, 2015
  6. "Voting Access Barriers: 2016 Survey & Webinar Discussion - Priority Issues" See attachment at the bottom of this web page.
  7. Voter Registration
  8. Vote by Mail

White papers

  1. 2011 TGDC Voluntary High-Level Goals for Remote Electronic Voting Systems

  2. 2011 TGDC Usability and Accessibility Working Group White Paper on VVSG 2.0 and Beyond: Issues and Gaps in the Usability Accessibility Requirements
  3. A roadmap for future usability and accessibility guidance (DRAFT) NIST and Center for Civic Design
  4. Principles for Remote Ballot Marking Systems (DRAFT) NIST and Center for Civic Design
  5. Exploring a revised structure for Usability and Accessibility in the VVSG (DRAFT) NIST and Center for Civic Design


  1. Accessible Voting Technology Portal
  2. EAC-funded Accessible Voting Technology Initiative (AVTI)

Info about usability testing/testing with users for VVSG

  1. Modified CIF Template For Voting System Manufacturers for reporting results of user testing
  2. Guidelines on How to Complete the Modified CIF Template for Voting Manufacturers


Bi-weekly teleconferences to solicit and discuss technical issues for the development of the next iteration of the Voluntary Voting Systems Guidelines begin on Thursday, May 19, 2016, 4:00PM EDT / 1:00PM PDT (1 hour).

Dial: 877-972-8541, Access Code: 9771600

We may have a large number of people on each call, so to make things move along efficiently, please:

  • DO NOT announce your name when joining (we will do a round of introductions)
  • Keep your phone on MUTE at all times unless you are speaking
  • DO NOT put the phone connection on hold, which can cause music to play that makes discussion impossible, resulting in shutdown of the call and redialing
If you cannot attend, you can email the HFCG mailing list at or with your comments relevant to the agenda. We will also archive information on this twiki.

Next Meetings

Biweeky, 4PM EDT/1PM PDT; 877-972-8541, Access Code: 9771600

Meeting Archive

Thursday, June 2, 2016, 4PM EDT/1PM PDT

Goal for this call: Plot course for future calls: identify and prioritize areas where U&A guidance is needed


  1. Introductions (5 minutes)
  2. Update on scope discussion: the EAC is aware of your interest in including standards for electronic pollbooks and remote ballot marking interfaces (5 minutes)
  3. Biweekly meeting goals (3 minutes)
    1. To refine structure for VVSG U&A standard
    2. To identify gaps in previous requirements, and opportunities for new guidance
    3. To address those gaps
  4. Structure for U&A (Principles & Guidelines document attached)
    1. Seeking input on core guidance and system-specific guidance
      1. Core U&A guidance (15 minutes)
  • E.g., color, contrast, font, plain language, plain interaction
  • Universal design (i.e. adjustable/customizable interfaces)
  • Ubiquitous across devices and interfaces
  • Widely accepted usability design
  • Baseline accessibility requirements
  • Includes WCAG as appropriate

  • Voting specific core guidance (5 minutes)

  • Ballot navigation
  • Overvote/undervote handling
  • Review of ballot selections, etc.

  • System-specific guidance (20 minutes)

  • (Based on today’s call we will expand this list and send it to the group for feedback via email.)
  • Devices, Interfaces, interactions
    • Small form factor (e.g., ballot content amount, ballot layout, minimum screen size)
    • Screen orientation (portrait or landscape)
    • Electronic forms (web-based and pdf)
    • Ballot selection representation (e.g., QR code, optical character recognition (OCR))
    • Touchscreen gestures (e.g., navigation, zoom, coordination with screen readers)
    • System feedback (e.g., mobile device vibrations)
    • Personal assistive technology (PAT) (e.g., screen readers, braille readers)>Onscreen keyboard use
    • Magnification and zoom

  • Meta topics: (2 minutes)

  • Over the course of these calls, let’s consider how our vision of the polling place has changed

  • Wrap-up (5 minutes)

  • June 16—update and review gaps and opportunities and prioritize discussion topics
  • June 30—begin topic discussions

Thursday, May 19, 2016 4PM EDT/1PM PDT

Goal for this call: Discuss scope and general approach


  1. Introductions
  2. How these calls will be structured: each call will have a set of issues and questions for participants. Specific topics will be identified for each call. For the next 2 calls we will choose some topics
  3. Brief overview of approach to developing next set of usability and accessibility requirements (see draft attachment at bottom of this page: “Developing Standards for Emerging Voting System Technologies U&A”)
  4. Discussion to help determine overall approach to new VVSG:
    • Scope: Voting systems and any equipment that “touches” the voting system in the polling place. For this group, it is where the voters, pollworkers, and election officials interact with the system.
    • HAVA and VVSG 1.1 call for at least one “accessible” machine per polling place. Currently, this often results in one machine in the corner that pollworkers might not be very familiar with and few voters realize that they could benefit from the accessibility features. We plan to write requirements that make a broader assumption. We would assume that any electronic system in the polling place is designed for all voters, accessible to the greatest extent possible. This would also simplify the organization of the usability and accessibility requirements as they would apply to all electronic voting systems in the polling place.
    • Accessibility standards such as the WCAG 2.0 are more mature than 5-10 years ago. We plan to refer to these in the VVSG. We are thinking about citing them directly. For any web application, WCAG 2.0 applies. We can do more than just cite WCAG in general, we could also look at how they apply in a voting specific context and also point out where they might be more stringent for voting, e.g., high levels of contrast.
    • Emerging technology (mobile devices) and voting research has resulted in numerous possibilities about where and how a voter can mark and cast their ballot. For example, a voter could use personal assistive technology to mark their ballot at home and mail it in or take the marked ballot to a polling place. The marked “ballot” itself could be just a list of selections and/or a QR code. Should we include these other models of voting in the VVSG? How? We are trying to figure out how to include standards for emerging technologies.

Kickoff meeting: FRIDAY JANUARY 29 at 1PM EST/10AM PST


  1. Introductions (25 participants)
  2. EAC TGDC meeting Feb. 8-9
  3. Role of this public working group (PWG)
    1. Constituency group to provide accessibility and usability input to Pre-election/Election/Post-election PWGs
      1. i. Process model being created by above election cycle PWGs
    2. Scope of next generation VVSG
  4. Issues for next generation VVSG
    1. Accessibility for all systems, no separate “accessible system”
    2. Use of existing standards (e.g. WCAG 2.0)
    3. Monolithic set of requirements vs. component-based subsets
    4. Outside of the polling place, personal devices, personal assistive technology, remote ballot marking
    5. Structure and usability of the VVSG itself (e.g. set of core requirements, high level principles)
  5. for more information
  6. Human Factors Twiki



Name: Diane Golden


Affiliation: Association of Assistive Technology Act Programs

Agency Lead

Name: Sharon Laskowski


Affiliation: NIST


Membership Information:

Participation in this Working Group is open to all interested parties. There are no membership fees.

  • List of mailing list members (TBD)

Latest Working Documents

Enter information about latest working documents here.


Enter information about important resources here such as meeting rooms, teleconferencing numbers, access codes, URLs, etc.


Enter important links and websites here.

Wiki URL:

Email List

Email List Name: vvsg-human-factors

To join the list or find more information about list policies and related procedures, please visit the VVSG Working Group Lists page.

Relevant Documents

Reference relevant documents here.



Policies and Procedures:

Reference Architecture:

Work To-Date:

Technology Roadmap:



This Topic Is Referenced By


Topic attachments
I Attachment History Action Size Date Who Comment
Microsoft Word filedocx 2-Principles-and-Guidelines.docx r1 manage 22.2 K 2016-06-01 - 20:45 ShaneeDawkins  
Microsoft Word filedocx 2016_Voting_Access_Barriers_Survey_Summary.docx r1 manage 18.0 K 2016-02-05 - 18:57 SharonLaskowski  
Compressed Zip archivezip r1 manage 511.4 K 2019-06-20 - 16:11 SharonLaskowski  
Microsoft Word filedocx Ballot_secrecy_requirements_from_HF_gap_analysis.docx r1 manage 13.8 K 2017-11-09 - 16:03 SharonLaskowski  
Compressed Zip archivezip r1 manage 6299.1 K 2019-06-20 - 17:10 SharonLaskowski  
PDFpdf Developing_Standards_for_Emerging_Voting_System_Technologies__UA.pdf r1 manage 734.9 K 2016-05-13 - 18:53 ShaneeDawkins Overview of approach to developing next set of usability and accessibility requirements
Microsoft Word filedocx NIST_HF_PWG_meeting__2017-10-05.docx r1 manage 14.9 K 2017-10-19 - 17:44 SharonLaskowski  
Microsoft Word filedocx NIST_HF_PWG_meeting__2017-10-19-final.docx r1 manage 13.7 K 2017-10-25 - 15:17 SharonLaskowski  
Microsoft Word filedocx NIST_HF_PWG_meeting__2017-11-16-final.docx r1 manage 15.7 K 2017-11-28 - 20:15 SharonLaskowski  
Microsoft Word filedocx NIST_HF_PWG_meeting__2017-11-2_final.docx r1 manage 17.3 K 2017-11-09 - 16:03 SharonLaskowski  
Microsoft Word filedocx NIST_HF_PWG_meeting__2017-11-30.docx r1 manage 16.3 K 2017-12-12 - 20:23 SharonLaskowski  
Microsoft Word filedocx NIST_HF_PWG_meeting__2017-12-14_final.docx r1 manage 16.9 K 2018-01-10 - 21:05 SharonLaskowski  
Microsoft Word filedocx NIST_HF_PWG_meeting__2018-1-11_final.docx r1 manage 16.6 K 2018-01-22 - 15:28 SharonLaskowski  
Microsoft Word filedocx NIST_HF_PWG_meeting__2018-1-25-final.docx r1 manage 17.0 K 2018-02-08 - 18:10 SharonLaskowski  
Microsoft Word filedocx NIST_HF_PWG_meeting__2018-2-8-final.docx r1 manage 16.2 K 2018-02-20 - 21:32 SharonLaskowski  
Microsoft Word filedocx NIST_HF_PWG_meeting__2018-3-8-final.docx r1 manage 18.2 K 2018-06-12 - 18:04 SharonLaskowski  
Microsoft Word filedocx NIST_HF_PWG_meeting__2018-4-5-final.docx r1 manage 19.2 K 2018-06-12 - 20:07 SharonLaskowski  
Microsoft Word filedocx NIST_HF_PWG_meeting_notes_20170907-final.docx r1 manage 19.1 K 2017-10-19 - 17:44 SharonLaskowski  
Microsoft Word filedocx NIST_HF_PWG_meeting_notes__20170824-final.docx r1 manage 29.9 K 2017-10-19 - 17:44 SharonLaskowski  
Compressed Zip archivezip r1 manage 570.2 K 2018-02-08 - 18:10 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Glossary-HFP_only-18-01-16a.docx r1 manage 51.2 K 2018-01-25 - 21:28 SharonLaskowski  
Microsoft Word filedocx VVSG_2-GuidanceBrief-Plain_Language-draft-18-01-16.docx r1 manage 1049.7 K 2018-01-25 - 21:27 SharonLaskowski  
Microsoft Word filedocx VVSG_2-GuidanceBrief-Plain_Language-draft-18-01-30pwg.docx r1 manage 1048.4 K 2018-02-12 - 20:52 SharonLaskowski  
Microsoft Word filedocx VVSG_2-GuidanceBrief-Text_Size-draft-18-01-16.docx r1 manage 24.3 K 2018-01-25 - 21:27 SharonLaskowski  
Microsoft Word filedocx VVSG_2-GuidanceBrief-Text_Size-draft-18-01-27pwg.docx r1 manage 24.9 K 2018-02-12 - 20:52 SharonLaskowski  
Microsoft Word filedocx VVSG_2-HFP-Requirements_Summary_for_Commments.docx r1 manage 42.3 K 2018-02-20 - 21:32 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_2.2-draft-18-1-2_PWG.docx r1 manage 22.5 K 2018-01-10 - 21:05 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_5.1-draft_17-10-17.docx r1 manage 27.7 K 2017-10-26 - 16:29 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_5.1-draft_17-10-24.docx r1 manage 27.5 K 2017-10-26 - 16:31 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_5.2-draft_17-10-17.docx r1 manage 26.4 K 2017-10-19 - 17:28 SharonLaskowski Draft VVSG requirements
Microsoft Word filedocx VVSG_2-Principle_5.2-draft_17-10-24.docx r1 manage 28.1 K 2017-10-26 - 16:24 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_6.1-draft_17-10-30-pwg.docx r1 manage 24.2 K 2017-11-03 - 19:25 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_6.1-draft_17-11-9-PWG.docx r1 manage 24.3 K 2017-11-09 - 16:03 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_6.2-draft_17-10-29-pwg.docx r1 manage 19.7 K 2017-11-09 - 16:03 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_7.1-draft_17-11-15-pwg.docx r1 manage 44.0 K 2017-11-15 - 20:23 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_7.1-draft_17-11-27-pwg.docx r1 manage 44.2 K 2017-11-28 - 20:15 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_7.2-draft_17-11-27-pwg.docx r1 manage 42.1 K 2017-11-28 - 20:15 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_7.2-draft_17-12-11.docx r1 manage 42.8 K 2017-12-12 - 20:23 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_7.3-draft-17-12-12_PWG.docx r1 manage 37.8 K 2017-12-12 - 20:23 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_7.3-draft-17-12-19_PWG.docx r1 manage 38.9 K 2018-01-10 - 21:05 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_8.1-draft-18-1-16_PWG.docx r1 manage 39.3 K 2018-01-22 - 15:28 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_8.1-draft-18-1-2__PWG.docx r1 manage 39.2 K 2018-01-11 - 15:37 SharonLaskowski  
Microsoft Word filedocx VVSG_2-Principle_8.2-4-draft-18-1-2_PWG.docx r1 manage 26.1 K 2018-01-10 - 21:05 SharonLaskowski  
Microsoft Word filedocx VVSG_HF_20160519mtgSummary.docx r1 manage 18.6 K 2016-06-01 - 20:45 ShaneeDawkins  
Edit | Attach | Watch | Print version | History: r61 < r60 < r59 < r58 < r57 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r61 - 2019-06-20 - SharonLaskowski
This site is powered by the TWiki collaboration platform Powered by PerlPLEASE NOTE: This wiki is a collaborative website. NIST does not necessarily endorse the views expressed, or concur with the facts presented on these sites. Further, NIST does not endorse any commercial products that may be mentioned on these sites. All the material on this website is in the public domain, including any text, diagrams, or images, unless indicated explicitly. Don't share anything on this site that you do not want to be public. Do not pass any proprietary documents or put any on the TWiki with implied public disclosure. If you do, it shall be deemed to have been disclosed on a non-confidential basis, without any restrictions on use by anyone, except that no valid copyright or patent right shall be deemed to have been waived by such disclosure. Certain commercial equipment, instruments, materials, systems, software, and trade names may be identified throughout this site in order to specify or identify technologies adequately. Such identification is not intended to imply recommendation or endorsement by NIST, nor is it intended to imply that the systems or products identified are necessarily the best available for the purpose. Any data provided on this site is for illustrative purposes only, and does not imply a validation of results by NIST. By selecting external links, you will be leaving NIST webspace. Links to other websites are provided because they may have information that would be of interest to you. No inferences should be drawn on account of other sites being referenced, or not, from this page. There may be other websites that are more appropriate for your purpose.