Ballot measure readability scores, 2023
| 2023 U.S. state ballot measures | |
|---|---|
2024 »
« 2022
| |
 | |
| Overview | |
| Scorecard | |
| Tuesday Count | |
| Deadlines | |
| Requirements | |
| Lawsuits | |
| Readability | |
| Voter guides | |
| Election results | |
| Campaigns | |
| Polls | |
| Media editorials | |
| Filed initiatives | |
| Finances | |
| Contributions | |
| Signature costs | |
| Ballot Measure Monthly | |
| Signature requirements | |
Have you subscribed yet?
Join the hundreds of thousands of readers trusting Ballotpedia to keep them up to date with the latest political news. Sign up for the Daily Brew.
| |
Ballotpedia's readability report analyzes what level of education voters would need to understand the ballot titles and summaries of statewide ballot measures using Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL). A readability score is an estimation of the reading difficulty of a text. Measurements used in calculating readability scores include the number of syllables, words, and sentences in a text. Other factors, such as the complexity of an idea in a text, are not reflected in readability scores.
In 2023, 41 statewide ballot measures were on the ballot in eight states. The average Flesch-Kincaid Grade Level for the measures' ballot titles, also known as ballot questions, was 19 years of education, which is about a third-year graduate school reading level. The average readability score between 2017 and 2022 was 18. The lowest readability score in that period was 15 (college junior reading level) in 2019.
Ballotpedia identified five measures with a ballot summary that was set to appear alongside the ballot question on the ballot. The average grade level for ballot summaries was 22 years of education.
See the sections below for further information on the following topics:
- Readability index details
- Analysis by state
- Analysis by the author of ballot language
- Analysis by type
- Historical readability scores
- 2023 readability scores
Readability index details
Ballotpedia uses two formulas, the Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL), to compute scores for the titles and summaries of ballot measures. The FRE formula produces a score between a negative (-) number and 100, with the highest score (100) representing a 5th-grade equivalent reading level and scores at or below zero representing college graduate-equivalent reading level. Therefore, the higher the score, the easier the text is to read. The FKGL formula produces a score equivalent to the estimated number of years of U.S. education required to understand a text. A score of five estimates that a U.S. 5th grade student would be able to read and comprehend a text, while a score of 20 estimates that a person with 20 years of U.S. formal education would be able to read and comprehend a text. Ballotpedia uses Readable.com to calculate the scores.
- Learn more about these formulas in the formulas section below.
Analysis by state, author, and type
Title and summary grades
The average ballot title grade for the 41 ballot measures is 19. Maine and Ohio were the only states with ballot summaries for some but not all of the measures featured on each statewide ballot. In Maine, ballot summaries were published for the four statewide initiatives and not for the four measures placed on the ballot by the state legislature. In Ohio, a ballot summary was available for Ohio Issue 1 (August 2023) but not for the two measures in November.
The average ballot title word count was 73 words. In 2021, it was 53 words. The ballot measure with the longest ballot title was Ohio Issue 2 with 565 words asking voters to decide on recreational marijuana legalization. The ballot measure with the shortest ballot title was Texas Proposition 13 with 15 words asking voters to increase the mandatory retirement age for state judges.
In the table below, click on the column header to sort by that column. Click on a state to see a list of that state's 2023 measures. "N/A" means that the state did not have a ballot summary.
Title and summary ease
Click on the column header to sort by that column. Click on a state to see a list of that state's 2023 measures. "N/A" means that the state did not have a ballot summary.
Analysis by the author of ballot language
As the text of ballot measures is often multiple pages of statute or constitutional law, a person or group of persons is tasked in each state with writing a shorter title and summary to appear on the ballot for the measures. The person or office responsible for drafting the ballot language for statewide ballot measures varies by state. In some states, the ballot language for different types of measures is drafted by different persons or offices. Moreover, some states require collaboration. For example, the secretary of state might draft the language, but it requires approval by the attorney general.
The average ballot title grade was highest for ballot titles written by state legislatures (21). The ballot language written by state boards had the lowest average ballot title grade of 12.
Analysis by measure type
Ballot measures can be either citizen-initiated or referred to the ballot by the state legislature, a commission, or a constitutionally mandated automatic referral. The table below shows the average readability scores for all 41 measures by type. Citizen-initiated measures received an average title grade of 14 years of education, and referred measures received an average title grade of 20 years.
Historical readability scores
Since 2017, Ballotpedia has tracked the readability scores of statewide ballot measures. The following table shows the average ballot title grade, word counts, summary grade, and the number of measures for each year. The year with the lowest ballot title grade was 2019 with 15 years of education, and the years with the highest were 2017 and 2020 with 20 years of education. The average ballot summary grade was lower than the ballot title grade for every year except 2019, where both were 15 years of education.
Historical ballot title grade
The chart below shows the average ballot title grade from 2017 to 2023.
Historical ballot title word count
The chart below shows the average ballot title word count from 2017 to 2023.
2023 readability scores
Below are the readability scores for all 41 ballot measures that appeared on statewide ballots in 2023. Blank cells mean that the ballot measure did not have a ballot summary.
Educational attainment in the U.S.
In 2022, the U.S. Census Bureau reported on the highest level of education of the population age 25 and older in the United States. Of that population, 14% completed advanced education such as a master’s degree, professional degree, or doctorate. The chart below shows the breakdown by highest level of education.[1]
Formulas
The Flesch Reading Ease and Flesch-Kincaid Grade Level formulas use the same variables and are inversely correlated, meaning that as one increases the other decreases.
Flesch Reading Ease
In the 1940s, Rudolf Flesch developed the Flesch Reading Ease (FRE) test. The U.S. Department of Defense uses the FRE to help craft its documents and manuals.[2] The FRE computes a score based on the number of syllables, the number of words, and the number of sentences in a text. The FRE formula is as follows:[3]
The FRE formula was designed to produce a score between 0 and 100, with the highest score (100) representing a 5th-grade equivalent reading level and the lowest score (0) representing college graduate-equivalent reading level. However, a score can be negative, representing increased difficulty. Therefore, the higher the score, the easier the text is to read. Rudolf Flesch created the following guide to interpreting FRE scores:[3]
| Score | School level |
|---|---|
| 90 to 100 | 5th grade |
| 80 to 90 | 6th grade |
| 70 to 80 | 7th grade |
| 60 to 70 | 8th and 9th grade |
| 50 to 60 | 10th to 12th grade |
| 30 to 50 | College |
| 0 to 30 | College graduate |
Flesch-Kincaid Grade Level
In 1975, J. Peter Kincaid recalculated FRE to give a score in the form of a U.S. school grade level for use by the U.S. Navy. This new formula became known as the Flesch-Kincaid Grade Level (FKGL) test. Like FRE, the FKGL computes a score based on the number of syllables, the number of words, and the number of sentences in a text. The FKGL formula is as follows:[4]
The FKGL produces a score equivalent to the estimated number of years of education required to understand a text. A score of 9 estimates that a U.S. 9th grade student would be able to read and comprehend a text, while a score of 18 estimates that a person with 18 years of U.S. formal education would be able to read and comprehend a text.[5]
Limitations
As the FRE and FKGL, along with other readability tests, do not measure the difficulty or complexity of the ideas expressed in ballot measure titles and summaries, they may underestimate or overestimate the ability of voters to comprehend a text. Political scientist Shauna Reilly, who utilizes readability indices in her research, noted their limitations, stating:[2]
| “ | There are limitations to the value of these measurements. No mathematical formula can tell us how complex the ideas of the passage are nor whether the content is in a logical order. Further, these mathematical equations exist in a vacuum and cannot explain the context of the passage.[6] | ” |
Prior research
Ballot Question Readability and Roll-off: The Impact of Language Complexity
In 2011, political scientists Shauna Reilly and Sean Richey published an article in Political Research Quarterly on research they conducted to answer the question of whether the difficulty or complexity of ballot measure language correlated with voters skipping voting on a ballot measure. The authors referred to voters casting ballots but skipping a ballot measure as voter roll-off. To measure the difficulty or complexity of ballot measure language, Reilly and Richey calculated Flesch-Kincaid Grade Level scores for 1,211 statewide ballot measures from 1997 to 2007. Reilly and Richey concluded that lower readability scores correlated with higher rates of voter roll-off. In their model accounting for state and year variations, Reilly and Richey only found one variable with a stronger influence on voter roll-off than readability—whether or not a ballot measure was on a primary election ballot compared to a special election ballot.[5]
Reilly and Richey calculated the mean Flesch-Kincaid Grade Level score for each state, except Arkansas, Illinois, and West Virginia, with at least one ballot measure during the 10-year period from 1997 to 2007. The state with the highest mean score was New Mexico, which had a mean FKGL score of 28 years of education. The state with the lowest mean score was Oklahoma, which had a mean FKGL score of nine years of education. The following table is from Reilly and Richey's research and contains the number of ballot measures analyzed in each state, the mean, minimum, and maximum readability score of measures in each state, and the standard deviation of the readability scores for measures in each state:[5][7]
| State | Measures | Mean | Mean U.S. equivalent | Standard deviation[7] | Minimum | Maximum |
|---|---|---|---|---|---|---|
| Oklahoma | 38 | 9 | High school | 1.1 | 7 | 12 |
| Connecticut | 1 | 11 | High school | 0 | 11 | 11 |
| North Carolina | 1 | 11 | High school | 0 | 11 | 11 |
| South Dakota | 36 | 12 | High school | 2.1 | 7 | 17 |
| Alaska | 30 | 13 | Associate's degree | 5.3 | 8 | 30 |
| California | 105 | 13 | Associate's degree | 1.8 | 9 | 18 |
| North Dakota | 13 | 13 | Associate's degree | 2.8 | 9 | 18 |
| Idaho | 16 | 14 | Associate's degree | 2.3 | 12 | 20 |
| Iowa | 5 | 14 | Associate's degree | 4 | 11 | 21 |
| Massachusetts | 18 | 14 | Associate's degree | 2.1 | 10 | 19 |
| Michigan | 18 | 14 | Associate's degree | 3.1 | 9 | 21 |
| Mississippi | 3 | 14 | Associate's degree | 5 | 8 | 18 |
| Oregon | 94 | 14 | Associate's degree | 1.7 | 11 | 18 |
| Rhode Island | 35 | 14 | Associate's degree | 6.1 | 6 | 33 |
| Washington | 57 | 15 | Bachelor's degree | 2.8 | 10 | 22 |
| Montana | 29 | 16 | Bachelor's degree | 7.4 | 11 | 52 |
| New Hampshire | 8 | 16 | Bachelor's degree | 5 | 10 | 27 |
| Utah | 6 | 16 | Bachelor's degree | 5.3 | 10 | 24 |
| Arizona | 70 | 17 | Master's degree | 3.1 | 11 | 26 |
| Florida | 40 | 17 | Master's degree | 5 | 8 | 38 |
| Indiana | 6 | 17 | Master's degree | 3.5 | 13 | 23 |
| Louisiana | 61 | 17 | Master's degree | 6.8 | 8 | 44 |
| Ohio | 19 | 17 | Master's degree | 4.9 | 9 | 30 |
| Tennessee | 6 | 17 | Master's degree | 5.8 | 10 | 25 |
| Vermont | 1 | 17 | Master's degree | 0 | 17 | 17 |
| Alabama | 32 | 18 | Master's degree | 6.4 | 12 | 35 |
| Kansas | 4 | 18 | Master's degree | 1.7 | 16 | 20 |
| Maine | 66 | 18 | Master's degree | 6.6 | 8 | 37 |
| Nebraska | 37 | 18 | Master's degree | 3.4 | 11 | 25 |
| Wyoming | 12 | 18 | Master's degree | 12 | 12 | 25 |
| Missouri | 27 | 19 | Ph.D. | 8.2 | 8 | 44 |
| Nevada | 36 | 19 | Ph.D. | 6.4 | 11 | 42 |
| New York | 8 | 19 | Ph.D. | 8.3 | 8 | 35 |
| Maryland | 11 | 20 | Ph.D. | 4.1 | 13 | 26 |
| Texas | 84 | 20 | Ph.D. | 12 | 12 | 45 |
| Wisconsin | 3 | 20 | Ph.D. | 16.6 | 17 | 23 |
| Georgia | 33 | 22 | Ph.D. | 10.4 | 10 | 57 |
| Hawaii | 10 | 22 | Ph.D. | 10.9 | 10 | 44 |
| Kentucky | 7 | 22 | Ph.D. | 6.1 | 14 | 30 |
| Virginia | 3 | 22 | Ph.D. | 3.2 | 19 | 25 |
| New Jersey | 20 | 23 | Ph.D. | 6.6 | 13 | 34 |
| Pennsylvania | 6 | 24 | Ph.D. | 5.4 | 17 | 33 |
| South Carolina | 19 | 25 | N/A | 10.8 | 16 | 63 |
| Minnesota | 1 | 26 | N/A | 0 | 26 | 26 |
| Colorado | 62 | 27 | N/A | 15.2 | 5 | 95 |
| New Mexico | 14 | 28 | N/A | 9.3 | 12 | 39 |
| Arkansas | N/A | N/A | N/A | N/A | N/A | N/A |
| Illinois | N/A | N/A | N/A | N/A | N/A | N/A |
| West Virginia | N/A | N/A | N/A | N/A | N/A | N/A |
See also
Past readability analyses:
- Ballot measure readability scores, 2018
- Ballot measure readability scores, 2017
- Ballot measure readability scores, 2019
- Ballot measure readability scores, 2020
- Ballot measure readability scores, 2021
- Ballot measure readability scores, 2022
External links
Additional reading
Footnotes
- ↑ U.S. Census Bureau, "Census Bureau Releases New Educational Attainment Data," February 16, 2023
- ↑ 2.0 2.1 Reilly, S. (2015). "Language Assistance under the Voting Rights Act: Are Voters Lost in Translation?" Lanham, MD: Lexington Books. (pages 55-56)
- ↑ 3.0 3.1 University of Canterbury, "How to Write Plain English," accessed April 19, 2017
- ↑ U.S. Naval Technical Training Command, "Derivation of new readability formulas (Automated Readability Index, Fog Count, and Flesch Reading Ease Formula) for Navy enlisted personnel," February 1975
- ↑ 5.0 5.1 5.2 Reilly, Shauna, and Sean Richey. "Ballot Question Readability and Roll-off: The Impact of Language Complexity." Political Research Quarterly 64, 1. (2011): 59-67.
- ↑ Note: This text is quoted verbatim from the original source. Any inconsistencies are attributable to the original source.
- ↑ 7.0 7.1 The standard deviation (SD) measures how spread out around the mean the scores of individual measures were. The smaller the standard deviation, the closer the scores of individual measures were to the mean. The larger the standard deviation, the farther apart the scores of individual measures were to the mean.
| |||||||||||||||||||
