For many Capitals fans, Roman Josi winning the Norris Trophy on Monday was a complete surprise. “Robbed,” said one Twitter user. “John Carlson snubbed again,” wrote another.

But the Norris Trophy is “awarded…to the NHL’s top defense player who demonstrates…the greatest all-round ability.” Sassy emphasis mine. Carlson’s dashboard stats were awesome, but below the surface were some concerning defensive numbers that ultimately sunk his candidacy.

First, let’s focus on the delicious nougat of Carlson’s 2019-20 season. In terms of points, Carlson ranked in the top ten for all NHL defenders during the last 15 years.

That’s really good, and he did this in a pandemic-shortened season. Carlson led all defenders in points (75), assists (60), and game-winning goals (6). Per RMNB’s Peter Hassett, much of Carlson’s offensive success was not from just passing it to Alex Ovechkin in the Ovi Spot too. Only six of Carlson’s power-play assists came on Ovi PPGs.

Points wise, Carly’s October was one of the best months by a defender in modern NHL history. He set a franchise defense record for points (23) in a single month. Carlson’s seven goals and 16 assists in 14 October games made him the NHL’s top player of the month.

The Capitals seized on Carlson’s hot start by launching the #Johnny4Norris hashtag and later releasing a a promotional t-shirt that they sold at their team store.

you rock that top @Laughlin18 #Johnny4Norris pic.twitter.com/gvlo8OcWYb

— NBC Sports Capitals (@NBCSCapitals) December 21, 2019

Those offensive stats alone were enough to help Carlson make a decent case for the trophy, and the team’s glowing praise may have overshadowed his more subtle defensive zone struggles. Below the surface, Carlson was a major part of the Capitals’ second-half swoon. For instance, from December 23 until the pause on March 12, Carlson was on the ice for three goals or more in a game seven times, including four goals allowed three times, according to Natural Stat Trick.

Via hockeyviz, you can see Carlson’s impact on defense. The big red blob in front of Washington’s net at the bottom means that’s where opponents get a ton of action, relative to the rest of the league.

Among all Capitals defenders Carlson gave up the most shots and shot attempts per 60. And had the worst expected goals numbers.

Here’s Carlson’s rankings amongst Caps defensemen in key defensive categories (all 5v5):

-Shot attempts against/60: 9th
-SOG against/60: 9th
-Scoring Chances against/60: 8th
-High Danger against/60: 6th
-Expected Goals/60: 9th

So yeah, okay with him not winning.

— Greg Young (@GregY_JR) September 21, 2020

Now I’m just being mean, but there was this highlight goal by Evgeni Malkin too, which I can never unforget.

This is not defense you expect from the best defender in the league and it continued in the playoffs (after voting had already been submitted by the PHWA).

During 5v5, John Carlson has been on ice for 6 Isles goals and 0 Caps goals. If he wins the Norris, then we need to abolish the Norris.

— Peter Hassett (@peterhassett) August 16, 2020

wait, so when Carlson is on the bench, the Caps and Isles are 50% in goals: 2-2.

— Peter Hassett (@peterhassett) August 16, 2020

Peter, in his review of Carlson’s season, put Carlson’s defensive struggles in context best.

Those percentages are all about trade-offs, and Carlson’s really really good at making split-second risk assessments to generate quality offense. He has a great shot, lots of volume, tremendous support from his on-ice partners, and a heaping load of opportunity (Carlson clocked 1700 minutes this season, seventh most among defenders). In order to get a Norris nod, Carlson didn’t even need friendly shooting percentages —  his individual and on-ice shooting during the power-play were both down, and his teammates’ shooting during five-on-five play were down as well.

As much as I want to kvetch about his defense, it’s important that we keep it in context. Carlson’s offense more than makes up for his defensive shortcomings. Evolving Hockey’s Goals Above Replacement model puts Carlson down 4.3 on defense (a bottom-10 number), but up 10.9 on offense (a top-10 number). So, yeah, I worry about Carlson’s play without the puck, but that’s a me problem, not a John problem. For now.

Some fans might point to Carlson losing the trophy and compare it to Mike Green being robbed for the award in 2009. That actually is not a fair comparison.

Both players were weak without the puck and strong with it, but the amount of offense generated by the 2009 Caps with Green was more than three times more dangerous (according to Hockey Viz) than the 2020 Caps with Carlson. And while Carlson racked up points at even strength and the power play, his defense was still a liability. For Norris winner Roman Josi, that’s not the case at all.

While you could argue that Josi has been supported by strong on-ice partners, there’s no arguing with his results:

• 65 points (16 goals and 49 assists), just ten behind Carlson
• 54.5 percent on-ice expected goals percentage, 21st best among full-time defenders
• 2.6 on-ice expected goals per hour, 16th best among full-time defenders
• 62.2 percent on-ice goals percentage, sixth best among full-time defenders.

Here’s how the Predators looked on offense and defense when Josi was on the ice this season, according to HockeyViz.

That’s a lot of team offense (though not Green ’09 levels), individual totals just a bit off Carlson’s, plus a defensive game that’s miles better than Carlson’s and a personal improvement for Josi himself.

None of this is to say Carlson was bad in 2019-20. He wasn’t, and we should appreciate what he accomplished. But defense matters, and PHWA voters got it right.