Are Modern Keepers Worth It?

On stopping shots, commanding the box, and the problem with measuring modernity.

Zack Steffen is a modern goalkeeper. I know this because when City loaned him to Fortuna Düsseldorf, Lutz Pfannenstiel said, “Zack is the perfect example of a modern goalkeeper.” I know it because every time I mention that Steffen’s career shotstopping numbers aren’t great—which happens a lot, to be honest, because I’m fascinated by how little it seems to have affected his career—someone tells me he’s a modern goalkeeper. Which, okay, but what does that mean? And are we sure it’s a compliment?

“In modern soccer, goalkeepers are not only expected to distribute the ball well, but to recognize attacking opportunities and play quickly,” Derrick Yam wrote when Statsbomb introduced their keeper radars last year. That jibes with the popular definition of “modern keeper” as “anyone who’s played goal for Pep Guardiola (except you, Joe Hart).” In an old Spielverlagerung post on Manuel Neuer, René Marić proposed six categories of keeper, which maybe work best as Myers-Briggs pairs: proactive (lots of buildup play), reactive (prefers longballs), anticipative (leaves his line), line (stays on it), classic (ugh useless), and libero (Manu Neuer, Fußball Gott). The ideal modern keeper would be a proactive-anticipative-libero type: anything that involves staying home in the six-yard box belongs in the Met, leaving your line and pretending to be a defensive midfielder is pure MoMA.

On this radar, you’d expect a modern keeper to pop in the four stats on the bottom left, while the two on the top right are just boring old shotstopping. But what are all these different metrics worth? “We have little knowledge of the actual value of collecting crosses and distributing the ball well,” Yam wrote. “Therefore, we can say one goalkeeper is more aggressive and better at recognizing the attack than another, but quantitatively, how does that compare to the goals he saved above average?”

In other words, just how modern would a modern keeper have to be to make up for meh shotstopping?

Thanks to American Soccer Analysis’s goals added model, we can start to get some idea of what commanding a keeper’s box is worth. Goals added values every touch by measuring how much it changes a team’s chances of scoring and conceding. Let’s say Trent Alexander-Arnold is laying in a cross. The model would compare Liverpool’s possession to similar ones in the past to estimate how likely the ball is to wind up in the net, then subtract the likelihood that the defense will clear the cross instead and take it all the way to the other end to score on the next possession. That’s the cross’s pre-action value. But just before Firmino can get a head on the ball, Ederson comes out and snatches it away. Now the model calculates City’s chances of building from the back and scoring a goal minus their likelihood of turning it over and conceding. That’s the post-action value. Deduct pre- from post- and voila, now you’ve got some knowledge of the value of collecting a cross.

And that value is … drumroll please … nada. The average claim adds 0.00 goals. Okay, technically it’s 0.002 goals, so not nothing, but all 140 successful claims recorded by every goalkeeper in MLS so far this season amount to a whopping 0.36 goals added. If a single keeper had recorded all 140 of them in a single game, they probably wouldn’t have made the difference between a loss and a draw.

Can that possibly be right? Well, maybe sort of. Without getting too far into the technical weeds, let’s stipulate that it’s a reasonable value for a cross that a keeper nabs successfully. What it doesn’t really tell us is how much difference it would have made if the goalkeeper was one of Marić’s “line goalkeepers” who never came for a cross in the first place. We know the average cross that successfully finds a teammate adds about 3% of a goal to the possession value, but since we don’t have a record of non-events, we don’t know from the raw data how many more successful crosses a passive keeper allows. We would need a model for that.

Lucky for us, that’s exactly what the four o’clock axis on Statsbomb’s keeper radar represents. Yam’s paper modeled how likely crosses were to be collected, then evaluated keepers on how often they came out and got the “claimable” ones. In 2017/18, the most aggressive keepers in the Premier League claimed about 10% more crosses than the league average; the most passive claimed 10% less. But given the small number of claimable crosses per game and the fact that we’re talking about fractional differences in the likelihood of allowing a low-xG chance or earning a zero-value claim, this seems like it’s probably not where modern keepers make their money.

Same goes for all the other box-commanding stuff keepers do. If you add up the value of all of their claims, punches, clearances, recoveries, headers, and so on, the difference between a top keeper and a bottom one in MLS is somewhere in the neighborhood of 0.05 goals added per 96 minutes, or a 5% chance of making a one-goal difference in a game. Again, there’s an unknown unknowns problem where we can’t put a number on the stuff that passive keepers aren’t doing, at least not without more modeling that I don’t have the technical chops for. It’s also an issue that our 5% estimate doesn’t account for the greater number of opportunities the top box commander is getting to punch and claim and clear (it’s Brad Guzan, he plays for Atlanta, they suck). But this at least gives us a sense of what Bill James called “the scale of the elements of the game.” The difference between a keeper who’s successfully commanding his box lots of times a game and one who rarely leaves his line looks to be about on par with the difference between a pretty good winger and an average one.

This is all extremely soggy-bar-napkin math, but hopefully by now you can see why. Measuring the value of what keepers do is hard; valuing what they don’t do is even harder; and knock-on effects from both are hardest of all.

The fog of WAR only gets thicker when we try to bring fancy stats to bear on the quintessential modern keeper skill, building from the back. Again, the model can pretty easily tell us that the difference between a top and bottom keeper is somewhere around 0.05 passing g+ per 96’, but the stuff it’s not capturing is even more important here. Passing in the defensive third is inherently low value, as far as a possession value model is concerned, because even a linebreaking pass to a midfielder still leaves your team with a lot of work to do to get to goal. A keeper doesn’t increase his team’s chances of scoring so much as he tries to avoid turning the ball over in a very dangerous area.

If that’s the problem you’re trying to solve, it seems like the smart thing to do would be to launch the ball as far as you can into the opponent’s half and set up a good structure to win second balls—exactly the tactic the modern keeper was supposed to kill off. Accurate long passing is still a valued skill in the modern game (just type “Ederson” into Youtube and see what pops up) but as part of a completely different game model. If you’ve got fifteen minutes to spare, I highly recommend watching Guardiola’s former goalkeeping coach at Barcelona, Juan Carlos Unzué, explain how a passing keeper creates a 4-v-2 at the back that eventually draws another presser forward into a 4-v-3, at which point you’ve got numbers over the top.

If you’ve only got two minutes, watch how Marc Andre ter Stegen reshaped Barcelona’s buildup against Mallorca in June, setting a La Liga passing record in the process.

I’m not sure how much of that tactical nuance goals added can pick up. I imagine this is the kind of thing a club analyst friend had in mind when he said that a possession value model “restricts your ability to value certain things that aren’t historically valued.” As Victor Valdes recalled thinking the first time Guardiola told him he was supposed to pass to center backs who would split all the way to the baseline, “I didn’t know what he was talking about. It sounded Chinese.”

Which is the whole thing about modernity, isn’t it? It’s hard to know what has real value except in hindsight. Maybe that’s why my default attitude toward modern goalkeeping is skepticism until people figure out a better way to measure all this stuff. Keepers who are heavily involved in the buildup create valuable overloads, but they also concede valuable turnovers. Keepers who are quick off their line catch more crosses, but they also concede more empty net goals. Top teams using modern keepers have been successful at the game’s highest levels, but so have top teams using traditional shotstoppers like Jan Oblak. The theoretical case for modern keepers sounds persuasive, but the empirical one, as far as I know, has yet to be made.

You know what we do know how to value? Stopping shots. And it’s pretty damn valuable: the difference between Turner’s career MLS shotstopping rate and Steffen’s, assuming those numbers were stable enough to predict future shotstopping rates, would come out to more than 11 goals across an average 38-game season. That’s the difference in goals added terms between an average winger and peak fucking Lionel Messi. There’s a ton of work left to do on this stuff, but it does seem at least possible to me that a keeper who can stop shots and send it long might turn out not to be so old fashioned after all. ❧

Further reading:

• Derrick Yam, A Data-Driven Goalkeeper Evaluation Framework (Statsbomb)
• René Marić, In-depth player analysis: Manuel Neuer (Spielverlagerung)
• Sam Tighe, The Evolution of the Goalkeeper: What Makes the Perfect Modern-Day No. 1? (Bleacher Report)
• Ben Lindbergh, Long Before WAR, Nobody Knew What MLB Players Were Worth (The Ringer)
• John Muller, What Are Goals Added (g+)? (American Soccer Analysis)

Image: David Bomberg, The Mud Bath