The Gauntlet Thrown
In 1989 I had my first academic encounter with the ‘hot hand’ phenomenon, reading ‘The Cold Facts about the “Hot Hand” in Basketball’ by Amos Tversky and Thomas Gilovich.1 By publishing in Chance, a magazine for statisticians, Tversky and Gilovich successfully expanded the audience from their landmark 1985 Cognitive Psychology paper with Robert Vallone, “The hot hand in basketball: On the misperception of random sequences”2 (which I henceforth refer to as ‘GVT’).
It took about thirty years but GVT’s claims have now been thoroughly refuted. See, in particular, “Surprised by the Gambler’s and Hot Hand Fallacies? A Truth in the Law of Small Numbers’ by Joshua Miller and Adam Sanjurjo.3 … Further below I comment on part of why it took so long.
Anecdata
Back in 1989 personal experience had already made me a firm, longtime believer in the hot hand. Two cases in point:
In 1975, my first season as a high school varsity baseball player, I struck out ten times in a row, then immediately followed that with nine consecutive hits. Six years later, I learned, via Sports Illustrated’s “Faces in the Crowd” column that my hit streak had tied what had then been the official high school record,4 smashed by future major leaguer Shane Mack with 16.5
In the mid 80s, I started playing basketball in grad school. As a beginner, I was a terrible shooter. To this day, I don’t recall making five consecutive free throws. But one day I sunk six three-point field goal attempts in a row in an actual game!
Challenge Accepted
So upon reading “The Cold Facts …” I noticed a few unappreciated issues, developed and estimated some substantive mathematical models, and in no time shipped off a 30+ page paper to Chance magazine.
I received a reply from the editor that the magazine had been swamped with responses to “The Cold Facts …” and would not be able to publish any of them as articles. Instead, they requested that respondents submit brief letters to the editor from which they would print a few.
My letter — transcribed in this footnote6 — was one of three selected for publication in the Fall 1990 issue. One key point implicit in my letter is that many psychology studies ‘demonstrate’ cognitive failings of the general public by conflating technical/scientific terms with informal interpretations of those words.
Institutionalized Science
Science as an institution shares with individual humans the cognitive bias of the primacy effect when being first to publish is more important than being correct. One amusing amplification here is that the competitive game of scientific publication can protect particularly egregious research that elicits a firehose of refutation, making it impossible to publish even one peer-reviewed rejoinder — at least for a few decades.
Once More Unto the Breach
In late 2017, during a respite from Lyme disease where not only my brain fog subsided but I was even able to write reasonably correct computer code, I expanded on Boston Celtics free throw shooting data in the 1985 GVT article.
Due to compounding (a) the mistake of employing a woefully inefficient statistical test with (b) the fallacy of accepting an unrejected null hypothesis — statistical “type II error” — GVT7 found “no evidence that the outcome of the second free throw is influenced by the outcome of the first free throw.”8
What I had seen in their data, back in 1989, was that free throw shooting exhibited a large causal effect: “the act of shooting the first free throw, regardless of whether hit or missed causes a typical player to hit 5- to 6- percentage points higher on his 2nd shot than his 1st.” For example, over these two seasons, Larry Bird made 84% of his first free throw attempts and 88% of his second free throw attempts — Larry only improved by 4 percentage points because, as an excellent, i.e. atypical, free throw shooter, he had less room for improvement.9
From this I inferred that repetition heats players up.
In 2017 with much improved access to data, I increased the sample from GVT’s four thousand free throws to 1.65 million free throws via fourteen complete seasons of NBA free throw shooting — 2000 through 2014. With this mountain of data, in addition to bolstering my hypothesis that repetition promotes the hot hand, I was also able to show that interruption cools players down. I further found evidence for fatigue and stress coming into play.
You can find my paper on arXiv.org.10 From its abstract:
If, as seems likely, all four of these effects have comparable impact on field goal shooting, they would justify strategic choices throughout a basketball game that take into account the ‘hot hand’. ... my analysis motivates approaching causal investigation of the variation in the quality of all types of human performance ... Viewing the hot hand as a dynamic, causal process motivates an alternative application of the concept of the ‘hot hand’: instead of trying to detect which player happens to be hot at the moment, promote that which heats up you and your allies.
Postscripts
Coincidentally, Andrew Gelman just posted some big picture thoughts on the sociology of science using the ‘hot hand’ debate as a focal exemplar. Contrasting psychology’s priming11 literature with their treatment of the ‘hot hand’:
They [psychologists and behavioral economists] seem strongly committed to the idea that basketball players can’t be meaningfully influenced by previous shots, even while also being committed to the idea that words associated with old people can slow us down, images of money can make us selfish, and so on.
From my experience, it seems likely that many if not most sports streaks — hot and cold — arise from alterations of physical and mental health. Just before my ten strikeout streak, I’d been hit in the ribs in batting practice by an 80 mph curve when our pitching machine hadn’t been aligned correctly. After my tenth whiff, one of my teammates said to me “Paul, you don’t have to stand so close to the plate.” I backed up a couple of inches and my hit streak began. Even after that streak ended, I stayed hot: for the last 26 plate appearances of my junior year, every time I swung at a pitch I hit a fair ball.
There’s a connection between the hot hand in sports and ‘Measuring the Rate of War Outbreaks’ in my old ‘Feed Your Prodigy’ substack that tipped me toward posting here rather than in the ‘Exceptional Statistics’ section.
Tversky, A. and Gilovich, T. (1989) Chance 2(1):16-21.
Gilovich, T., Vallone, R. and Tversky A. (1985) Cognitive Psychology (17):295-314.
Miller, J. and Sanjurjo A. (11/2018) Econometrica 86(6):2019-2047/
Undoubtedly there had been longer consecutive hit streaks by high schoolers as of 1975, but no one thought to submit them because they figured the record would be (much) higher than the major league record of 12.
[My letter published in Chance.] Reporting a discrepancy between basketball statistics and fans’ intuitions, Tversky and Gilovich contend that the “hot hand” is a “cognitive illusion.” But both statistics and intuitions actually differ from statistically independent sequences. For example, they found that “… aggregate data yielded slightly fewer high and low series than expected by indepen- dence …”, corroborating the fans’ bias against long runs.
Rather than illusion, consonance of fans’ perceptions and basketball statistics may represent evidence of a cognitive aptitude for assessing the balance between “damping forces” that promote alternation between hits and misses (e.g., shot selection and defensive adjustment), and “driving forces” that generate streaks (“hot” and “cold” hands). More generally, their findings suggest that people may be quite adept at assessing and exploiting imbalance between positive and negative feedback when it occurs in real-world processes.
When negative feedback dominates positive feedback, a dynamic system is periodic. The greater the domination, the more accurately we can project the future, facilitating planning and coordination. Positive feedback produces amplification and instability. Even if we recognize a trend, we cannot make accurate projections because small errors are magnified over time. Still, amplification can generate a large return from a small initial “investment.” A balance of positive and negative feedback produces chaotic behavior. This is more difficult to exploit: Planning and coordination are hampered by our inability to make projections; neither are there long- term trends to “ride.”
If the frequency of alternation in a sequence of equal numbers of hits and misses is even slightly below 1/2, then amplification must be present. But if the frequency is just above 1/2, the sequence will not display long-term predictability. Furthermore, amplification may still be present. Only if alternation frequency approaches 1 will the sequence display periodicity. Tversky and Gilovich asked basketball fans to classify sequences as streak, chance, or alter- nating shooting. Their interviews suggest that people accurately interpret streakiness as amplification, chance as chaos, and alternation as periodicity. In the real world, “chance” is not statistical independence; chance is when we can't exploit the universe!
I should make it clear that Amos Tversky is one of my intellectual heroes. Even the best can make mistakes! I highly recommend The Undoing Project by Michael Lewis, the story of Tversky and Nobel Prize winner Daniel Kahneman’s long, successful collaboration; Tversky would certainly have shared that Nobel if he had survived just six more years.
We now know that this failure to find evidence arises in part from poor discrimination between ‘hot’ and ‘cold’ states.
The perspective of missing, rather than making, might give a better idea of Bird’s improvement after shooting his first free throw: he missed 16% of his first attempts but only missed 12% of his second attempts; a relative reduction of 25%. From my 2000 to 2014 data, players missed, respectively, 26.8% and 22.2% of first and second free throws, a relative reduction of 17%.
Some day I hope to update this upload with references I wasn’t aware of that Josh Miller sent me.
Here’s a link to some discussion by Prof. Gelman and me in the comments to his post.