Knowledge Deserts Quiz Contest

  1. What was the last name of the family on the television show “Duck Dynasty?”

  2. What was the name of the female protagonist in the Netflix Series “Stranger Things?”

According to our pre-tests, Republican respondents are far more likely to answer the first question correctly, whereas Democratic respondents are far more likely to answer the second correctly. We are interested in developing a collection of similar questions that tap into the cultural differences between Republicans and Democrats.

Specifically, our project uses crowdsourcing to assemble trivia questions that reveal differences in cultural knowledge between Republicans and Democrats. As explained in the Overview section, there are two types of “Knowledge Desert” questions, and participants may submit as many questions as they like of each type. The best questions in each category will receive a $100 reward.

Contest Overview

We are accepting submissions on two types of trivia questions.

Democrat “Knowledge Deserts”

These are non-political questions that a person who identifies as a Republican is much more likely to get right than a person who identifies as a Democrat. The Republican should, on average, be much more confident in their answer than a Democrat would be. Questions should be as non-political as possible, and should tap into different lifestyles, media preferences, and consumption habits that differ between Republicans and Democrats.

Pilot testing has shown that the following three questions are particularly good examples:

What is the last name of the Duck Dynasty family?

  • Davis
  • Robertson
  • Wilson
  • Johnson

The restaurant “Cracker Barrel” is known for having what game at each table?

  • The Peg Game
  • Trivial Pursuit
  • Connect 4
  • Mancala

What comedian is known for his routine “Achmed the Dead Terrorist?

  • Jeff Dunham
  • Jeff Foxworthy
  • Larry the Cable Guy
  • Joe Rogan

Republican “Knowledge Deserts”

These are non-political questions that a person who identifies as a Democrat is much more likely to get right than a person who identifies as a Republican. The Democrat should, on average, also be much more confident in their answer than a Republican would be. Questions should be as non-political as possible, and should tap into lifestyles, media preferences, and consumption habits that differ between Democrats and Republicans.

Pilot testing has shown that the following three questions are particularly good examples:

What is Matcha?

  • Tea
  • A comic series
  • A TV show
  • Coffee

Which of the following is a song by Drake?

  • Hotline Bling
  • HUMBLE
  • Gold Digger
  • Drop the World

Which of the following is a flavor of La Croix Sparkling Water?

  • Pamplemousse
  • Chocolate-Mint
  • Cucumber-Melon
  • Peach-Ginger

Judging Criteria

Submission Criteria

  • Submissions must be in English.

  • Participants may submit as many questions as they like, but note that not all questions will pass our initial review. See below for details. Submitting a large number of low-quality questions will not increase your odds of winning a prize.

  • All “Knowledge Desert” questions should be submitted along with four multiple-choice answers, including the correct answer.

  • We will award a $100 prize to anyone who submits a high-quality question (criteria described below); however, regardless of quality, prizes will be awarded to at least the top two submissions for each of the following categories: Democrat “Knowledge Deserts” and Republican “Knowledge Deserts.”

  • The deadline for submitting questions is October 20th, 2019 at 11:59 p.m.

Judging Criteria

To determine winners for each question category, we will do the following:

  1. All questions will be subject to an initial review designed to screen out low-quality questions. (Given that we do not know how many questions we will receive, we cannot commit in advance to testing all submitted questions. It is our intention, however, to test all questions that could plausibly meet our criteria.)

  2. We will collect all of the questions that pass our initial review and ask a large sample of real Republican and Democrat respondents to answer them. If two or more participants submit what is substantively the same question, then only the participant who submitted the question first will be considered for a prize.

  3. To qualify as a valid “Knowledge Desert” question, people from a specific political party (such as Republicans) should, on average, get the answer wrong more often than people from the other political party (such as Democrats). Additionally, only representatives from one political party (such as Republicans) should be confident in their answers, while representatives from the other political party (such as Democrats) should not be confident in their answers. Confidence will be gauged by asking participants to register the degree of confidence they have in their answers.

  4. We will award prizes to the submissions that maximize the difference between the percentage of Republicans answering correctly and the percentage of Democrats answering correctly. (For the best questions we have developed so far, we have found that on average, the difference in the proportions of each party responding correctly exceeds 50%–for example, 76% of Republicans answer correctly, but only 24% of Democrats answer correctly.) We will award at least two prizes in each category, and we will reward any submission beyond the top two that we consider to be “high-quality.”

For Republican “Knowledge Desert” questions (questions Republicans are likely to get wrong), we compute the following:

$$ \mid \text{% Correct for Democrats - % Correct for Republicans} \mid $$

For Democratic “Knowledge Desert” questions (questions Democrats are likely to get wrong), we compute the following:

$$ \mid \text{% Correct for Republicans - % Correct for Democrats} \mid $$

Awards will be given for the questions that maximize the above quantities. We reserve the right to use our discretion to reward questions that are particularly unique or clever, over and above the strict quantitative criteria described above.

We will award $100 to any high-quality “Knowledge Desert” question. In order to qualify as a “high-quality” question, the difference in the proportions of each party responding correctly should exceed 40%. So, for example, a Republican could answer the submitted question correctly 82% of the time, whereas a Democrat could answer correctly 40% of the time, resulting in an accuracy spread of 42%.

Moreover, the submitted question must satisfy the criteria described in the Overview section (e.g. only one party should be confident in their answer; and the question should cover non-political topics such as differences in lifestyles, media preferences, and consumption habits). In the event that a large number of high-quality questions are submitted, we will reward prizes to only the Top 25 “Knowledge Desert” questions.

Prizes

Prizes will be awarded as follows:

  • Two $100 prizes will be awarded for the top two questions in the “Democrat Knowledge Desert” category
  • Two $100 prizes will be awarded for the top two questions in the “Republican Knowledge Desert” category
  • $100 prizes will be awarded to any question that we determine to be of “high-quality” (see Judging page for details)

Submit

To submit to this contest, please click here and fill out the survey. You are free to submit as many trivia questions as you like, although we may discard submissions from participants who submit a large number of questions that in our view have little chance of being high-quality.

The survey above will ask you to pre-specify how many questions you are submitting. For each question, you will be required to submit four multiple choice answers. Please bear in mind that high-quality questions must have well-designed incorrect answers, as the incorrect answers presented have a large effect on accuracy rates.

If you have already participated and would like to participate again, you may do so. But repeat participants will need to re-complete the demographic questions at the end so that we can match it to your previous submission.

About

Who are we?

This project is managed by Evan DeFilippis (PhD student in Psychology and Organizational Behavior) and Joshua Greene (Professor of Psychology) at Harvard University.

Why are we doing this?

We are collecting questions that tap into differences in cultural knowledge and political belief between Republicans and Democrats. We will use these questions as testing materials in our research examining the causes of social/political beliefs and attitudes. These testing materials may eventually be made available to other researchers.

Why crowdsource?

“Knowledge Desert” questions are hard to design because it is difficult to know what one does not know. Furthermore, even if one can identify topics about which Republicans or Democrats are likely to be especially ignorant or uninformed, it is difficult to identify specific questions that expose such differences. As such, we expect that the most talented and creative designers of these questions are not part of our research group. We hope that the best questions will likely come from one of you.

Contact