Win Probability Maintained
Consequentiality vs Value, and a new stat for the playoffs
The Problem With Playoffs:
As someone who has been trying to analyze goalies for a bit now, there’s one problem I consistently run into: an overemphasis on playoff hockey.
‘Overemphasis?!’ You might exclaim. ‘Playoffs are the most important part of the hockey season. It’s when goalies earn their keep, etch their name into hockey history, become legends. Where mediocre regular seasons are forgotten, where 1As and 1Bs seize the seize the reins, where careers are defined!’
Yeah, okay. But there arise some issues when looking at individual playoff seasons that come part-and-parcel with the nature of playoffs.
By design, the playoffs are a series of best-4-of-7 series, each against the same team. Here is how that conflicts with the way we typically look at goalie seasons.
Small samples: half of all playoff teams play between 4 and 7 playoff games in a playoff season. The maximum possible number of games played is 28 games. Only after a goalie has played 2 series can we truly get anything remotely reliable from rate stats. Especially when factoring in the next items on the list.
Lack of variety (overall): the most teams a team can play in a traditional postseason is 4. The fewest is 1. 50% of teams play only 1 team; 25% play only 2 teams; 12.5% play only 3; and another 12.5% play 4 teams. Further consider that the point of playoffs is that the worst teams don’t play in them. In the 2023-24 season, half of all teams in each conference don’t make the playoffs. This leads us to #3.
Match up: It also matters who you play. No matter what style, the most successful teams in the playoffs are the ones who can most successfully dictate how the game is going to be played. Look no further than this past Cup Final1 for an example. Each game was won by whoever established their style of play: the first 3 games, Florida established their forecheck and capitalized on turnovers by Edmonton. The next 3 games, the Oilers were able to open things up and utilize their speed up the ice to pick the Panthers apart. And finally, in Game 7, after a back-and-forth first period and a bit, Florida was able to dictate how the game was going to be play after they took the lead. This kind of thing is always idiosyncratic — no pair of teams are guaranteed to interact the same way and nor will the same two systems produce identical results across series. It’s why they play the games.
Where this comes into goaltending is that better teams tend to be better at dictating play. If you’re playing against an offensive juggernaut, they’re probably gonna generate a lot of chances unless the defensive can shut those down. If you’re playing against an opportunistic team, your save percentage will likely suffer. This leads me to point #4.Lack of variety (series-level): Playing the same team for 4-7 games straight does odd things to one’s save percentage. A way to think of save percentage is as “opponents’ shooting %” (hereafter, oppS%). In the regular season, this way of thinking isn’t very useful, as goalies typically play a variety of teams, and almost never the same team back-to-back. So, the only thing constant to goalies is their play — and that of the team in front. But, anytime you try to measure a goalie’s sv% in a single series, you are simultaneously measuring the other team’s S% in that same series. This actually represents a significant issue, because goalie stats are already always affected by the play of their own team (a reality we have to deal with, unfortunately) — but now, by the nature of the playoff format, we have the play of the other team affecting this as well. With the three factors of goalie’s play, team’s play, and opponent’s play being constant, it becomes increasingly difficult to disentangle the goaltender’s play from all that. Even GSAx, which is best suited to deal with this issue, can’t fully account for this because it counts xGoals include shots that miss the net. There’s always that element of execution by the opposition. According to the eye-test, there is a huge discrepancy between getting robbed and completely whiffing on an open net. This is combatted during the regular season by any two teams in the league sharing 30 offenses that both have to play against.
Playoff overtime: In the playoffs, OT is 5-on-5 until someone scores. It is indefinite. Therefore, playoff OT represents extra game time wherein only 1 goal can be scored. playoff OT lasts anywhere from seconds to longer than full-length games. “Next-goal-wins” win-conditions represent a significant enough variation from the “most-goals-wins” win-conditions of regulation that it is dangerous to so carelessly lump them together.
Consequentiality vs Value and Win Probability
In June, I wrote an article on Stuart Skinner in which I called him the “3rd-to-6th most consequential player on the Oilers on any given night”. I then contrasted consequential with valuable — valuable being production-based and context-neutral and consequential being context-informed.
In baseball, where offence and defence are constrained to 27 outs, there exists this similar divide. Batting Average, doesn’t care about when you get a hit — be it off the first pitch, a walk-off, or sometime in between — it counts the same.
But a 3-run homer to pull ahead and one to extend a lead in a blowout have two very different affects on a ball game — the former gargantuan, the latter miniscule. This is what necessitated the stat win probability added: to measure the effect a play has on game outcome. In baseball, every play in a close game will have an effect on the win probability.
In hockey, using the win probability model from Money Puck, there are only two things that cause a swing in win probability: a goal and a penalty. Here’s an example from the 2018 playoffs — specifically, game 1 of the 2018 quarter-final between Vegas and Los Angeles in which only 10 total goals were scored.
Vegas scores early causing the win probability to leap further in Vegas’s favour (all win probability is home-team relative). Each penalty throughout the game blips the trajectory, but then it ‘corrects’ as the penalty expires. As game time elapses, Vegas’s win probability climbs higher and higher until it reaches 100% when time expires.
Looking at another game — this one game 2 of the 2021 quarter-final between Edmonton and Winnipeg— we see a different trajectory.
This actually beautifully demonstrates a ‘natural’ game trajectory. We start at the pre-game win probability (which is baked into the moneypuck model), slowly descend to an even 50% when regulation ends tied 0-0. The game-winning goal in sudden-death Overtime represents a 50% swing in win probability.
Most games don’t look like these, but it’s good to see the two ‘forces’ that affect a games win probability. A lead will direct things to 100% or 0% (bigger leads later in games will push the win probability further and sooner). Meanwhile, a tied game will approach 50% win probability the closer the game gets to overtime.
Lastly, we’ll see this all come together with our last example — from the very same day as our previous example. This one is a 5-4 game between Carolina and Nashville.
Nashville, the home team in game 3 starts the game with about a 49% win probability. This is due to being the lower seed, but it’s mitigated by home ice advantage. They go up to 66% — a 17% swing— after Ryan Ellis’s goal. The win probability swings back and forth, falling as low as 33%. But we see both of the previous trends in our 1-0 examples represented here. After Nashville ties it up 3-3 in the second, the game hovers at 50%. Once Nashville pulls ahead 4-3, their win probability starts gradually rising until Carolina ties it 4-4. At which point, the rest of the game sits at 50% win probability until Nashvilles wins it in 2OT.
Win Probability Maintained
So, how do we connect this all back to goalies? Well, besides penalties and the onward march of time, the only way to influence win probability is to score goals. Every unblocked shot attempt has three possible outcomes: a goal, a save, or a missed net.2 We’ll just focus on outcomes that come into play with goalies: i.e., goals and saves.3
According to Moneypuck’s xGoals model, every unblocked shot attempt generates an xGoal, which is the estimated probability of a goal being scored off the shot attempt. It accounts for the chance that a puck misses the net (by location of the shot) and the chance that if on net it will be a goal. So, if (a) a goalie is responsible for all shots on goal, (b) all shots on goal generate an xGoal probability, and (c) all goals cause a swing in win probability, we can calculate how much influence a goaltender exercises on the probability of a game, by multiplying the would-be swing of win probability by the xGoals of a shot on net. Since saves do not cause swings in win probability according to the moneypuck model, it cannot be win probability added. But, since saves can be considered all shots on goal that are not goals, they can be thought of as maintaining the current win probability. Thus, the name win probability maintained (WPM).
As an equation, it looks like this:
WPM= |swing| x (xG-G)
or
“win probability maintained is equal to the absolute value of the would-be swing in win probability times the difference of the expected goals and actual goals”.
Allow me to demonstrate:
Let’s take the Duchene OT-winner from our last game. Matt Duchene’s goal carried a 50% swing in win probability, taking the outcome from a coin-flip to a certainty. Duchane’s shot had an xGoals of 0.166 — or a 16.6% chance of scoring. Using our formula above:
WPM= |swing| x (xG-G)
We plug in a swing of 50%, an expected goals of 0.166, and 1 actual goal and we get a WPM of −41.7%. Ouch.
WPM = |50%| x (0.166−1.000)
WPM = |50%| x (−0.834)
WPM = −41.7%
For an example of a save, let’s look to this year’s Cup Final4. In game 7 with 2:18 left to go, Sergei Bobrovsky made this save on Mattias Ekholm.
This shot had an xGoals of 0.502 and would have caused a WP swing of about 46%5. Add in an actual goals of 0 and we get
WPM = |46%| x (0.502-0)
WPM = 46% x 0.502
WPM = 23.1%
With a WPM of 23.1%, this was the single biggest WPM I recorded over 3 SCF games. As fate would have it, however, this game’s game winner had a mirror-image WPM of −23.1%
Carrying an xGoals of 0.014 and causing a WP swing of 23.4%, we converts 0.986 of the WP swing into negative WPM, giving us that aforementioned −23.1% WPM.
Do me/yourself a favour and watch this goal at 0.25 speed. Pay close attention to Draisaitl’s stick. As Reinhart releases the puck, you can see the stick flick as he gets his blade in the path of release, but only enough to interrupt it as it follows through, not to stop interfere with the actual path of the puck.
Compare that to the Bobrovsky save, which, as it turns out isn’t so much a shot by Ekholm as a bank shot by Connor McDavid because is Verhaeghe pushing against Ekholm — ineffectively, in terms of prevent his progress towards the net, but effective in stopping him from getting clean puck possession on a rebound. Ekholm eventually gets another shot off, but it’s not as dangerous according to xGoals (a mere 0.103 for a WPM of 4.7%.
Given these three examples, what have we learned? Well, the OT winner taught us that playoff OT still distorts any stat it sees as the context-dependent WPM will usually leave losing goalies with a negative WPM, but as we can see from some quick and dirty math (thank the hockey Gods for OT’s invariable win probability), Nedeljkovic’s WPM was still positive for the 1.75 OT periods he played. Quite a bit, actually— in the low-20s, which is around the WPM of Sergei Bobrovsky’s biggest save in Game 7 of the Stanley Cup final. The mirror image to that is Stuart Skinner’s WPM on the game-winning goal against at the hands of opponent Sam Reinhart and teammate Leon Draisaitl’s stick. From this perhaps we can preliminarily conclude that any single WPM exceeding 20% in either polarity is a game-breaking moment.
In total, Bobrovsky finished with 42.7% WPM and Skinner with −21.8% WPM. What this truly means, I don’t know. I need more data. But, if nothing else, WPM represents an excellent way to measure a goalie’s impact on a shot-by-shot basis and the impact of those shots on a game.
Does this solve the playoff problems mentioned above? No, not really. We’ll have to keep trying to find the elusive DIG (Defense-independent goaltending, analogous to fielding-independent pitching) which will allow us to estimate future goalie performance. Not even GSAx can get defense out of the picture because — as the Draisaitl Reinhart goal and Bobrovsky save show — defensive misplays and butt-saves are just a part of playing the this God-foresaken position.
Which was 2024, if you’re a freak from the future or just didn’t read the byline.
Posts count as missed nets unless they go in. Truly, a case of “it doesn’t count unless it goes bar-downskies”.
By only looking at unblocked shot attempts, we’ve already accounted for a fourth outcome: a blocked shot, which is the prerogative of the defending team defense, but not the goalie.
See note 1
I was trying to record these stats in more-or-less real time, which proved to be very difficult. Not all my figures for non-goals are going to be precise.