A Brief History of Advanced Hockey Stats
Thinking through the quantitative revolution
I am old enough to remember that there was once a debate about the merits of “advanced stats” in hockey. Not that long ago, maybe around 2012 or so, you could find hockey fans, and even hockey professionals, who didn’t much care for the use of statistical analysis in conversations about which players and teams were actually good. The view basically associated with the old-timers was that statistical analysis would never replace experience, intuition, and observation when it came to assessing players and teams. For them, statistics were a poor replacement for actually watching hockey games, and saying things like “Corsi” or “expected goals for” or “shot suppression” was a sure way to induce eye-rolling and derision. There was a sense in which this sounded a bit like a semi-plausible empirical claim—who in the hockey world was actually qualified to assess if these new statistical tools worked?—but the impression the stat-resistant old-timers gave was also one of severe defensiveness. They were worried, as all established knowledge-brokers are, of being swept away by something new and unstoppably, irrevocably true.
On the other side, back then, were the proponents of a methodology that used to be called “fancystats.” On this side, younger hockey fans and analysts—often people from non-traditional hockey markets, or women, or people who had never played hockey at a high level—developed a set of very basic statistical tools for assessing individual and team performance in hockey. The promise of the fancystats community was to unlock a new way of accessing knowledge about hockey, and a new way of informing hockey-management decisions that divorced truth from experience. You didn’t need to have played hockey, or even necessarily watch the games. There was a vast amount of knowledge to be gained, right under our noses, which the old-timers had voluntarily rejected. It wasn’t that the fancystats community rejected experience itself as a way of gaining knowledge about hockey; it was just that they had a faster, better way of doing things, which didn’t require direct exposure to hockey at all.
There was never very much mystery about how this was going to play out. Rejecting quantitative analysis out of hand is not a hallmark of victorious intellectual movements. And while there were plenty of people who pleasantly and reasonably insisted that experience and statistics could co-exist as ways of gaining knowledge about hockey, that was always just a stop-gap, a proxy. The statistical methods produced measurable, replicable results, and they informed vastly better decision-making on how to construct a winning hockey team. Before very long, coaches and managers who demonstrated a basic hostility to statistical analysis came to seem negligent. They were voluntarily rejecting a source of valuable information. They cared about something other than winning, which was not what they are paid to do. Except for the antique curiosities of John Tortorella’s worldview, they’re all gone now. The NHL is a fancystats league, and NHL management is a fancystats world.
The exact inflection point, the moment in time when the fancystats community defeated the old-timers and quantitative analysis vanquished qualitative analysis in hockey, was the rise and fall of Peter Chiarelli as general manager of the Edmonton Oilers. If you recall, Chiarelli is a former hockey executive (technically, he is currently Vice President of Hockey Operations for the St Louis Blues, but you can’t convince me that this is a real job) who is most famous for being the general manager of the Boston Bruins for their 2011 Stanley Cup win and near miss in the 2013 finals. Coming on board in Boston in 2006, Chiarelli inherited an astonishing number of shrewd moves from his outgoing predecessor, including the Andrew-Raycroft-for-Tuuka-Rask trade, the cheap acquisition of Zdeno Chara in free agency, and the drafting, in a single year, of Phil Kessel, Milan Lucic, and Brad Marchand. Having lucked into this preposterous wealth, Chiarelli shuffled bottom-six pieces, failed to trade most of his stars (though he did turn Kessel into Tyler Seguin and Dougie Hamilton) and rode it through to history. He lost most of his trades, produced one of the worst-ever draft records of any GM in NHL history—remember Zach Hamill, Jordan Caron, and Joe Colborne?—and doubled down repeatedly on the kind of big, old, slow team that was becoming obsolete in that era. In 2015, he was fired.
Because hockey as an industry is uniquely bad at identifying, retaining, and promoting administrative talent, Chiarelli was immediately hired by the Edmonton Oilers, who had just won the 2015 draft lottery, to be the first GM of the Connor McDavid era. A great deal of optimism attended this move. Chiarelli, who is Harvard-educated (lest there be any wavering meritocrats still lurking out there) and speaks confidently, and made various pronouncements about McDavid’s incipient greatness and the forthcoming Albertan empire, though what struck many observers at the time was that Chiarelli, who had demonstrated no empirical ability for hockey management, had lucked into a second consecutive dynasty through no ability of his own. He promptly set about destroying the then-ample resources of the Oilers organization, switching out Taylor Hall for the few remaining ligaments in Milan Lucic’s knees, purging the pick used to select Mat Barzal for a few games of Griffin Reinhart, trading Jordan Eberle for a series of players too obscure to become future trivia questions, succumbing to what can only have been some sort of personal blackmail on the part of Mikko Koskinen, and otherwise staffing the Oilers with a series of players too ancient and cuboid to be of any use in the NHL. He acted, basically, as an undercover agent of the Calgary Flames; a stochastic algorithm could not have performed worse. In 2019, he was fired, much too late, and the McDavid Oilers have been hobbled to this day by the exorbitant effort required to move on from Chiarelli’s final acts. Their emergence as a power over the last few years is a testament to the historical greatness of the pieces Chiarelli first inherited.
Throughout, Chiarelli demonstrated no remorse; and why would he? He had no sense of having done anything wrong, made any mistakes, subscribed to an erroneous paradigm. It was beyond his capacity and his theory of hockey success to understand the enormity of his errors. Yet his service to hockey was great, for it became impossible, thereafter, to believe sincerely in a connection between being good at hockey and being good at hockey management. Chiarelli, the consummate insider, had discredited his fellow-travellers as thoroughly as Bush and Cheney discredited the remnants of Reaganism by invading Iraq. The world-historical danger involved in squandering the prime of Connor McDavid, the greatest player to enter the league since Mario Lemieux, was too much for the industry to stomach.
The confluence of Chiarelli’s unseriousness with the deadly seriousness of being entrusted with Connor McDavid coincided with a number of events in the world that drove home the fact that seriousness was back. Whatever position one happens to take on that 2015-2020 period, phenomena like Donald Trump, Covid-19, and rising authoritarianism were serious, no matter the specific content of one’s views. You could be for or against these things, but they were undeniably important. Being good at one’s job mattered a little bit in a way that had, it seemed at the time, gotten away from us. And so the old-guard, reality-challenged, clownish qualitativeness of Chiarelli and his ilk fell by the wayside, Joe Biden won the 2020 Democratic primary on an electability-only platform, and advanced stats were the order of the day. A fundamental philosophical conflict between ways of accessing knowledge—experiential vs axiomatic, the argument from authority vs the argument from principles—had eroded, at least as far as hockey was concerned.
The term “advanced stats” implies some stats are not advanced, or that which stats count as “advanced” can change over time, or maybe even that stats advance. That is to say, the discipline of “advanced stats” itself has a history, and that as the history of “advanced stats in hockey” unfolds, the set of statistical methods that are considered “advanced” changes. In English: stats get better and better over time. The tools that we once thought were great are now considered simple, and there are yet more advanced stats to come in our future.
This is inevitable, and fine, and simply how methodologies work. Once upon a time, the fancystats guys were obscure, marginal bloggers building a little internet community where they could practice their controversial art. Now, those original bloggers are in NHL management positions, and there is a huge proliferation of hockey statisticians—on Substack, on social media, in the actual media—writing about ever more advanced statistical methods for understanding hockey, and there is obviously a large, if not especially cool, reader base for this stuff. I include myself in that category. Hockey journalism is now, to an extent, about interpreting statistics.
There are few different conservative impulses that Nordiques Fan Corner has tried to play with so far. The fun part of this project, for me, is that I’m not a conservative about politics, but unlike a lot of progressives, I don’t actually think conservative thought is valueless. Conservative ideas can be really fun to experiment with in a laboratory setting. One conservative impulse that I hope I don’t display, and that I certainly won’t be employing here, is an instinctive aversion to the new—neophobia. I think that the application of increasingly advanced statistics in hockey is so inevitable that it doesn’t even matter if we think it might be bad (although I don’t think it is bad). Anyway, I’m a nerd about hockey, so it’s not my place to tell anyone else how to be or not be a hockey nerd. But there is a conservative impulse that I think can be useful here, and that impulse is a predisposition for taking the long view.
There may be no longer view than that of physics. After all, the universe is very old, and there is no consensus among physicists on where the universe came from, or how to properly frame questions about the origin of the universe, or even what shape it is. Nevertheless, people—physicists—have been productively inquiring about the universe for a long time. Without getting into philosophers of science like Popper and Hacking, I think we can basically boil down the history of physics to a series of successive attempts to devise ways of describing the world that are both general and precise. Physicists might want to know how, say, a golf ball in motion behaves, and the project of physics over the centuries is about getting better are saying specifically how golf balls move, while also saying how this applies to other types of balls.
As historian Thomas Kuhn famously argued (and is often misrepresented as having claimed prescriptively) in his 1962 classic The Structure of Scientific Revolutions, the succession of paradigms can follow a “punctuated equilibrium” structure. This means that, instead of a succession of small, accumulative additions to scientific knowledge, ruptures tend to be quite large. A series of contradictions will build up in the dominant model, and will have to be explained away in a patchwork manner. Eventually, the dominant model will become untenable, and will be swept away by a shift in perspective. Kuhn primarily articulated this structure through the transition from Copernican to Newtonian to relativistic (i.e. Einsteinian) physics, and since the book’s publication one could argue that a quantum revolution has been taking place, though from a historical perspective the quantum revolution is presumably far from complete. The celestial motion that Copernicus attempted to explain was superseded and completely transcended by Newton’s laws of motion, which you learned in high school; these in turn were unable to explain the time-space dynamics for which Einstein became famous. Relativity, in turn, was unable to completely explain phenomena that are now considered to be in domain of quantum mechanics. No paradigm could have been explained or made sense of by its predecessor.
Once you read Kuhn, you start seeing this pattern everywhere. Bill Clinton was an unstoppably gifted politician, but Clintonism had to be completely re-invented by Obama to be viable in 2008, and Obamaism could never have stood up to the Trump revolution once Obama himself was gone. Structuralism in social science had to make way for postmodernism, without which its explanatory power would have remained forever feeble. The jet engine is not an improved propeller engine; it is a different thing altogether, and quickly rendered most propeller airplanes obsolete for high-performance tasks. These are not instances of successive accumulation, but of disruptive step-change. The more profound claim implied by Kuhn’s work is that it is okay for science to not have all the answers in a given moment. The current paradigm does not have to be true forever; eventually, all the little things it can’t quite explain will be swept away by the next paradigm. The accretion of small contradictions is valuable insofar as it puts pressure on the currently dominant story for explaining the world, and the same is true of advanced stats in hockey.
The classical battleground between proponents and opponents of fancystats used to be the figure of the enforcer, players specifically employed to fight on behalf of their teammates. The fancystats guys would say that enforcers generally suck at hockey, and the old-timers would say that the value of enforcers is fundamentally intangible, and both sides would basically throw up their hands in epistemological frustration. This was not just a disagreement about reality—enforcers generally don’t score much, although skilled players tend to avow their usefulness nonetheless—but about value-sets. Enforcers, and the enforcer debate, stood in for a broader schism about what theory of hockey, and what theory of hockey knowledge, would prevail. Of course enforcers are not measurably “good at hockey,” but that was never the empirical claim; the claim was that they made their teams better in some other, harder-to-measure way. The issue cannot really be sidestepped with “toughness through the line-up”; there’s only one Tom Wilson, and he makes a lot of money for a reason. And in any case, the viewing public tends to love fights, even if elite hockey pundits often don’t.
But then, anyone who has—and I hate to be that guy—played any kind of hockey will tell you that some players and some teams are nastier than others, that you can’t teach a sociopathic relationship to the rules any more than you can teach being 6’4, and that cruelty confers a competitive advantage on the team willing to deploy it. You could, in theory, measure this; maybe you could devise a statistic to track sneaky slashes after the whistle per 60. But what we currently lack the mathematical tools to track (and please, if you’re going to invent this, name it after me) is the anticipatory compliance of a nasty team that sees Bob Probert or Donald Brashear on the other side, imagines what will happen if they step out of line, and decides not to exercise their competitive advantage in cruelty. This will not show up in anything you can measure about Probert or Brashear, but it will show up in the statistics of the slick, undersized nerd on his off-wing who will have a little more space for saucer passes across the slot. The enforcer thing is real, even if statisticians cannot yet quite describe what it is, any more than Newton could have found the vocabulary for general relativity. And yet it moves.
Shifts in ways of knowing, ways of accessing knowledge, are okay. Advanced hockey stats are good and true, but they are not complete, and they will get better over time. They do not need to be a totalizing paradigm today. As Karl Marx said, philosophers have only attempted to interpret the world, but the point is to change it. That means keeping an open mind about new things. But it means keeping an open mind about old things, too; and if there is room in your philosophy for statistics, great; but there should be room, also, for humility, and for understanding that the world as it is will reveal itself only in droplets, and almost never in a great torrent of sudden understanding. We have made some progress. Take a breath.