To: Swami who wrote (45043)
From: Ward Nicholson Tuesday, Nov 14, 2000 7:38 PM ET
Reply # of 45044
Off-topic (election stuff):
I thought that people might find this short paper from our
university interesting.
The Butterfly Ballot Causes Confusion and Systematic Errors in Voting Behavior
Robert C Sinclair
University of Alberta
Melvin M Mark
The Pennsylvania State University
Sean E Moore, Carrie A Lavis, Alexander S Soldat
University of Alberta
Two experiments investigated confusion and bias caused by the butterfly
ballot format used in Palm Beach County in the 2000 US presidential
election. In Study 1, Canadian students voted for Prime Minister of Canada
on a single-column or butterfly ballot. They rated the butterfly ballot as
significantly more confusing than the single-column format; however, they
made no voting errors. Study 2 replicated the confusion effect with a
nonstudent sample. Of greater importance, participants made errors only on
the butterfly ballot. The butterfly ballot causes confusion and systematic
errors in voting.
The issue of systematic bias as a result of ballot format has
become the focus of much controversy surrounding the outcome of the recent
presidential election in the United States. Specifically, people have
argued that the format of the ballot in Palm Beach County led to confusion
and caused people who intended to vote for Al Gore to mistakenly cast votes
for Pat Buchanan or punch two holes resulting in a voided ballot. We
conducted two experimental studies to address this issue.
On Wednesday, November 8, 2000 (the day after the presidential
election), we had Canadian college students vote for Prime Minister of
Canada using a single-column ballot format or a dual-column, butterfly
format (analogous to the Palm Beach County-style ballot). We expected that
students would rate the butterfly style as more confusing than the
single-column format. However, it was unclear whether students, who are
familiar with confusing optical scoring forms, would make errors on the
ballot.
Participants were 324 introductory psychology students from two
classes at University of Alberta. All were volunteers who participated in
order to partially fulfill a course requirement.
Procedure
Ballot Construction. The ballots contained the names of the
leaders of 10 Canadian political parties and space for a write in
candidate. One ballot used a single-column format. The second was designed
to emulate the dual-column, butterfly format used in Palm Beach County (at
the time this study was conducted, to the investigators knowledge the
actual ballot was not available on the web or in print media, and the
ballot was constructed after seeing it displayed for a brief period on
CNN). The butterfly ballot was designed so that the leaders of the 2
predominant parties appeared in the first and second positions in the first
column. Specifically, Stockwell Day, leader of the Canadian Alliance Party,
was in the first position on the ballot, corresponding to George Bush on
the Palm Beach County ballot, and Jean Chretien, leader of the Liberal
Party of Canada, was in the second position, corresponding to Al Gore. The
leader of a third party, expected to receive few votes, was the first name
to appear in the second column. Specifically, Joe Clark, leader of the
Progressive Conservative Party of Canada, was in the position on the ballot
corresponding to Pat Buchanan on the Palm Beach County ballot. The
remaining candidates were also from parties expected to receive few votes.
The data were collected in large classrooms at University of
Alberta at the beginning of class. Participants were told that we were
interested in political issues. They were told that we were holding a mock
election for Prime Minister of Canada (there was, conveniently, a federal
election in Canada within 2 weeks following our data collection).
Participants were randomly assigned to receive one of the two ballot
formats. They were asked to vote for Prime Minister by darkening the circle
beside the preferred candidate s name. Upon turning to the next page, they
were asked to report the degree to which the ballot was confusing (using 2
items on 7-point scales, with high scores indicating greater confusion),
and to write out who they had intended to vote for on the previous page.
Finally, participants were debriefed.
The mean of the two confusion items formed an index of confusion
(Cronbach s alpha = .96). Participants in the butterfly format condition (M
= 3.69) rated the ballot as significantly more confusing than did
participants in the single-column format condition (M = 2.14), t(322) =
8.23, p < .0001. No students made errors on the ballots.
The results of Study 1 demonstrate that the Palm Beach
County-style ballot is perceived as significantly more confusing that the
single-column ballot. Generally, of course, greater confusion is likely to
lead to a higher error rate. This was not the case in Study 1; however, we
were not surprised by the lack of errors given that our sample involved
college students who are quite skilled at completing confusing optical
scoring sheets. Thus, we decided to move our data collection off campus and
conduct a second study.
Participants
Participants were 116 people recruited in a large shopping mall in
Edmonton, Alberta, Canada. There were 51 males and 62 females (three
respondents failed to report their gender). The mean age was 51.10 years
(sd = 19.19, range = 19-86; six respondents failed to report their age).
Procedure
Ballot Construction. The ballots were designed in the same manner
as described above. However, by Thursday, November 9, 2000, we were able to
view, in detail, the Palm Beach County ballot. Thus, the butterfly ballot
was in exactly the same format as that used in Palm Beach County (with the
exception that we did not use punch holes).
Participants were approached individually at a mock polling
station set up in a busy shopping mall and asked to participate in a
Political Issues survey being conducted at the University of Alberta. They
were told that we were holding a mock election for Prime Minister of
Canada. Participants were randomly assigned to one of the two ballot
formats and were directed to one of two polling booths. They voted for
Prime Minister (as above). After turning the page in their ballot, they
reported the degree to which the ballot was confusing (again, on 2 items on
7-point scales, with high scores indicating greater confusion), wrote out
who they had intended to vote for on the previous page, reported their
gender, age, and ethnic background, and placed their ballots in a ballot
box. Finally, participants were given a written debriefing page describing
the basis for the study and were offered a piece of candy.
Confusion Ratings
The mean of the two confusion measures served as an index of
confusion (Cronbach s alpha = .81). The butterfly ballot was rated as
significantly more confusing (M = 3.44) than the single-column ballot (M =
2.28), t(110) = 3.17, p < .003.
Errors
We computed errors as a function of ballot type. There were 4
errors, all of which occurred in the butterfly format, likelihood ratio
X2(1) = 5.27, p < .03. Interestingly, 3 of the 4 errors occurred for the
candidate who was in the same position on the butterfly ballot as was Al
Gore on the Palm Beach County ballot. This candidate s votes were
unintentionally given to the candidate who was in the same position as Pat
Buchanan on our butterfly ballot. Thus, the results suggest that the
butterfly ballot, as used in Palm Beach County, does result in systematic
errors.
Both Studies 1 and 2 demonstrate that the butterfly ballot used in
Palm Beach County Florida is significantly more confusing than a
single-column ballot. Moreover, Study 2 demonstrates that the Palm Beach
County-style ballot systematically causes errors in voting behavior. These
findings call into question the validity of the results from Palm Beach
County in the 2000 US presidential election. With the ballot style in use
there, vote counts will systematically vary from the intention of the
electorate. In addition, the current findings may underestimate the
magnitude of bias. With a punch hole voting system, imperfect alignment of
a ballot in the voting machine might increase the likelihood of errors with
a butterfly ballot, given the proximity of punch holes corresponding to the
two columns. These findings are also likely to underestimate the bias
because the candidate in the first position on our butterfly ballot
(analogous to Bush) received 49.1% of the vote in Study 2 and no errors
occurred in this position (the candidate in the second
position corresponding to Gore received 21.4% of the vote and the
remaining 8 candidates shared 29.5%).
It is not clear whether a biasing ballot format does or should have legal
standing in adjudicating disputes after an election. On the other hand,
given the centrality of elections to the democratic process, it seems
remarkable that biasing formats continue to be used. Low cost application
of social science theory and methods could help avoid such controversies in
the future.
Acknowledgements
This research was supported by a Social Sciences and Humanities
Research Council of Canada operating grant to Robert C Sinclair. We thank
Sheree Kwong See for her assistance and Kelly Sinclair for comments on a
previous draft of this manuscript. Finally, we thank the administrators of
Bonnie Doon Shopping Centre for providing us with the space needed to
conduct Study 2.
http://www.psych.ualberta.ca/people/sinclair.html |
