Skip to content
Campaign Text Labhome page

Tool 05 · Readability

Reading grade, with its error bars

The Flesch-Kincaid grade combines how long sentences are and how long words are. This page shows how it has moved, with 95% intervals, and how much of it is decided by whoever punctuated the text. A grade describes a transcript or a press release; it says nothing about the quality of what was said.

General and VP debates, per decade

−0.75

95% CI −1.02 to −0.55 · 49 debates in 14 cycles, cycles resampled

Primary debates, per decade

−0.45

95% CI −0.89 to −0.17 · 130 debates in 9 cycles, cycles resampled

Same speaker, transcribed vs written

−5.5

95% CI −6.9 to −4.1 grades (t) · 14 speakers, paired; 14 of 14 lower

Same debate, two transcripts

up to 1.4

grade levels apart for one speaker; 0.56 on average over 7 speakers

Candidates' reading grade in debates, 1960 to 2024

Each dot is the grade of everything the candidates said in one debate. The dashed line is a least-squares trend. Debates in one election cycle share candidates and a transcription source, so its band and the per-decade interval resample whole cycles, not single debates (10,000 resamples, seed 20261010). With 9 to 14 cycles, even these intervals are approximate.

Change per decade: −0.75 grade levels (95% CI −1.02 to −0.55, cycles resampled), 49 debates in 14 cycles.

24681012141960197019801990200020102020
One debate Linear trend, 95% bandNo cycle has five debates, so no cycle means are marked.
Show the numbers as a table
Candidates' mean reading grade per election cycle, with 95% bootstrap intervals over that cycle's debates (five or more)
CycleDebatesGeneral and VP: mean (95% CI)Primary: mean (95% CI)
196049.9 (n = 4, no interval)–
1976410.2 (n = 4, no interval)–
1980211.0 (n = 2, no interval)–
198448.5 (n = 3, no interval)8.7 (n = 1, no interval)
198838.2 (n = 3, no interval)–
199246.7 (n = 4, no interval)–
199647.7 (n = 3, no interval)7.3 (n = 1, no interval)
2000287.2 (n = 4, no interval)7.5 (7.2 to 7.9)
200466.9 (n = 4, no interval)7.6 (n = 2, no interval)
2008397.8 (n = 4, no interval)7.9 (7.6 to 8.2)
2012247.1 (n = 4, no interval)7.2 (7.1 to 7.4)
2016335.9 (n = 4, no interval)6.8 (6.5 to 7.1)
2020165.6 (n = 3, no interval)7.1 (6.7 to 7.6)
202485.7 (n = 3, no interval)6.3 (5.7 to 6.9)

Campaign documents, 2016 to 2024

Written releases against transcribed speech

Mean grade per document (documents with 100 or more words in the candidate's own voice). A campaign's documents share a style, so the 95% intervals resample speakers, each with all of their documents, not single documents; cells with fewer than 5 documents or 10 speakers get no interval. Document types are the scraper's, from title words: releases and statements are written prose; remarks and interviews are transcripts of speech.
Mean Flesch-Kincaid grade by register and election cycle with 95% intervals
  • Written releases and statements
  • 20161,726 docs from 13 speakers · 25.3 words/sentence13.5 (12.7 to 14.7)
  • 20202,855 docs from 24 speakers · 26.7 words/sentence14.6 (14.1 to 14.8)
  • 20241,395 docs from 10 speakers · 19.7 words/sentence10.5 (10.0 to 12.9)
  • Transcribed remarks and interviews
  • 2016191 docs from 14 speakers · 16.6 words/sentence8.1 (7.9 to 8.6)
  • 2020189 docs from 24 speakers · 16.5 words/sentence8.2 (7.4 to 9.2)
  • 2024208 docs from 12 speakers · 13.3 words/sentence6.0 (5.2 to 7.9)
  • Addresses
  • 201616 docs from 8 speakers · 18.6 words/sentence10.1 (no interval)
  • 202036 docs from 20 speakers · 19.5 words/sentence10.2 (8.7 to 11.2)
  • 20249 docs from 6 speakers · 13.7 words/sentence6.7 (no interval)

In every cycle, transcribed remarks grade 4.5 to 6.4 levels below written releases. Both kinds grade lower in 2024, so a trend over cycles depends on the mix of document types as well as on the year: compare one kind of text at a time.

A paired comparison

The same speakers, written and transcribed

For each speaker with at least 5 graded documents of both kinds, the difference between their transcribed and written texts; then the mean over the 14 speakers with a t interval (at this size a percentile bootstrap runs narrow). Because the grade is linear in sentence length and word length, the gap splits exactly into the two parts below.
Within-speaker difference in grade (transcribed minus written) and its two parts, with 95% t intervals
  • Transcribed minus writtenmean over 14 speakers−5.5 (−6.9 to −4.1)
  • … from sentence length0.39 × difference in words per sentence−3.2 (−4.1 to −2.3)
  • … from word length11.8 × difference in syllables per word−2.3 (−2.9 to −1.6)

14 of 14 speakers grade lower when transcribed (exact sign test p = 0.0001); the standardised paired difference is dz = −2.23. A percentile bootstrap over speakers gives −6.7 to −4.2 for the gap, a little narrower than the t interval shown. Most of the gap comes from sentence length, which in a transcript is the transcriber's choice of where to put full stops.

Speakers in this comparison (alphabetical)
  • Amy Klobuchar: 101 written, 10 transcribed, gap −8.6
  • Bernie Sanders: 474 written, 43 transcribed, gap −4.4
  • Cory Booker: 72 written, 8 transcribed, gap −7.6
  • Donald J. Trump (1st Term): 746 written, 139 transcribed, gap −7.0
  • Elizabeth Warren: 122 written, 11 transcribed, gap −7.7
  • Hillary Clinton: 357 written, 66 transcribed, gap −4.8
  • Joseph R. Biden, Jr.: 883 written, 97 transcribed, gap −8.4
  • Kamala Harris: 111 written, 86 transcribed, gap −8.2
  • Kirsten Gillibrand: 19 written, 5 transcribed, gap −4.2
  • Michael Bloomberg: 194 written, 23 transcribed, gap −4.6
  • Nikki Haley: 757 written, 19 transcribed, gap −2.6
  • Pete Buttigieg: 27 written, 5 transcribed, gap −5.0
  • Ron DeSantis: 406 written, 19 transcribed, gap −2.0
  • Tim Scott: 161 written, 7 transcribed, gap −1.4

A natural experiment

Same debate, two transcripts

The archive holds two transcripts of 2 events from January 2000, with almost identical word counts. Each speaker below has within 2% of the same number of words in both, so the difference comes mainly from how the two transcribers punctuated the same speech.

7 speakers from 2 events is too few for an interval; the table is shown as it is. Left out: McCain on 10 Jan 2000 (1,930 against 2,157 words). The two transcripts attribute different turns to this speaker, so a grade difference would mix who said what with how it was punctuated.

Read before comparing

How transcription style moves the grade

Speech has no full stops. A transcriber decides where one sentence ends, whether a run-on answer becomes one sentence or four, and whether false starts and repetitions are kept. Because the grade gives 0.39 grade levels for every extra word per sentence, those decisions move it directly: a transcriber who splits a 30-word answer into three sentences lowers it by almost eight grade levels for that answer.

Written releases are edited prose with long, clause-heavy sentences, so they grade much higher than the same person's transcribed remarks. Debate transcripts from different decades and sources follow different conventions, so part of any trend is a change in transcription. Compare like with like (the same kind of text, ideally the same source), and read grades as a description of the text, never of the speaker. The calculation itself is on the methods page.