Manchester City is Turning Soccer into a Video Game (and Maybe Vice Versa)

An AI competition with Google could one day give clubs a way to experiment with innovative tactics. But first, the hard part: learning to pass.

Today Manchester City embarks on another Champions League campaign, and you know what that mea—nah just kidding you have no idea what that means, which is what makes it fun. Pep Guardiola has made a career out of trying wacky shit in high-stakes games. He is, indeed, perhaps unique in his ability to do so, due to particulars like (a) being the world’s best soccer coach, (b) coaching some of the world’s best players, and (c) working for his Barcelona buddies Ferran and Txiki, who (d) have unlimited access to an oil state’s sovereign wealth fund through City Football Group. Not a bad gig if you can get it.

For everyone else, a failed tactical experiment can cost them their livelihood. “Football is a tough environment to perform in and an even tougher environment to learn in. Learning is all about harnessing failure, but failure in football is seldom accepted,” Brian Prestidge, the head of CFG’s analytics department, recently wrote. He was announcing a new competition where teams of coders train “agents” in a video game called the Google Research Football Environment to play soccer, or at least a version of soccer that looks like something you might have played on Sega Dreamcast at the height of the dot com bubble. It’s crude stuff right now but it might matter one day for actual Champions League soccer. The end goal, Prestidge explained, is “to test tactical concepts and refine principles so that they are strong enough for a coach to stake their career on.”

That pitch was catnip to Bruno Dagnino, co-founder of the analytics company Metrica Sports. He’d never done this kind of thing before, but after Metrica spent the last six years working with analytics leaders like FC Barcelona and the Seattle Sounders, he could see the potential of an AI training ground to test out innovative ideas or maybe even discover new ones. “The competition looked like the perfect excuse to learn a little bit about reinforcement learning, to see what we were capable of, to play around with some tactical concepts or things we want to explore,” he said. The learning curve was steep, but in the last week Metrica’s bot has shot up the leaderboard using an approach whose details he wouldn’t divulge because the company is planning to submit a paper about it to FC Barcelona’s Sports Analytics Summit next month.

Watching the game feels like slipping on an old pair of jeans. It’s basically an ancient edition of FIFA, except instead of controlling the player on or nearest to the ball with a joystick and buttons, you do it with blocks of code. Each moment (or “step”) in the game, your code agent gets fresh information about what the ball and players are up to, and it has to decide which one of 19 actions it wants to take: go this way or that way, start sprinting or dribbling, pass short or long, try a slide tackle or a shot. If you’ve ever mashed buttons while talking trash to your little brother, you get the idea.

But trying to translate all the stuff we see and react to while playing video game soccer into clear step-by-step instructions for a computer player is harder than it sounds. Players have the option of starting with a basic rule-based bot they can tell to do stuff like: “If you don't have the ball, go to the ball. If you have the ball, go towards the goal. If you are close to the goal, shoot.” “Here is my best performing bot so far,” wrote the Kaggle user Eugen Keil, who shared that code about a week into the competition to show what things looked like at the top of the leaderboard. “At the time of writing it is number three :-)”

Since then the competition’s gotten bigger and the bots are getting more sophisticated. Of the 757 teams currently competing, a few are veterans of the analytics world like Dagnino, but others are stumbling on the field for the first time in their quest to make their fake soccer players less dumb. “Shouldn't your agent know the probability of scoring with a shot before it shoots?” one user wrote, dropping a link to a 2004 research paper that showed angle and distance to goal and the location of the nearest defender could help you predict a shot’s chance of going in. “This is called expected goals (xG) in football,” another replied. “It would be interesting to introduce xG for Google football, I will try to do it. I think this will really help improve the agent. Thanks.”

Other players are trying to code tactical ideas directly into the game, taking a shortcut to the kind of concept-testing and principle-refining Prestidge had in mind. One gave his agents instructions using zones kind of like the ones on Pep’s training pitch, so that a bot who dribbled into the area beside the six-yard box would know to look for a cutback to a teammate around the penalty spot—classic Sterling-to-Agüero stuff. But writing out tactical instructions as if-then statements isn’t just tedious, it’s borderline impossible in the world of the game. “It’s not like chess, where you say to the computer move this piece two spaces forward,” Dagnino said. “For example, in this Google environment, if you say ‘make a pass’ it won’t be able to make a pass in the direction you want it to make. It’s not really realistic. So if you see a lot of top teams, it’s basically just running straight and optimizing where to shoot from.”

Luckily there’s another way to win: let your bot learn the game for itself. Instead of trying to code rules for what to do in every possible situation, teams can create agents that play via something machine learning specialists refer to as “reinforcement learning” and the rest of us usually just call AI.

“I wanted to build something like a Voronoi tesselation, connected with assessing the amount of free space—everything which is now popular in the soccer analytics field,” said Michał Jaroń, a machine learning engineer who’s worked with top Polish clubs. But though trying to translate academic work by analysts like Barcelona’s Javier Fernández into on-field instructions was a useful exercise for Jaroń, who learned to think less like an analyst and more like a coach working with a player, it wasn’t helping him win simulated games.

“So I started to deal with this problem from the other side, using raw reinforcement learning,” Jaroń said. This approach rewards bots for achieving certain goals and then leaves them to figure out the best way to earn those rewards through trial and error. The challenge for the person doing the coding is figuring out which actions to reward in order to produce good soccer. “The standard reward provided by this Google environment is only goal or conceding a goal. So if you want your agents to learn something useful, you have to give them more rewards. That’s what I’m now implementing,” Jaroń said. “I want to reward them for making good passes using soccer analytics terms like ‘packing.’ I also would like to reward them for a pass that makes free space for their teammates, not only goals.”

These AI bots take longer to train than the rules-based ones, but most competitors expect them to climb to the top of the leaderboards by the time the competition ends next month. The best way to tell if a team is using rewards rather than rules might be to watch it and see whether its agent knows how to pass. The number two team on the leaderboard right now, WeKick, plays an almost Cityesque quick passing game, pulling highly unusual and difficult tricks (in this lobotomized version of soccer, anyway) like sending balls back to the keeper when they’re in a tough spot to reset the buildup and play out of the back again. But WeKick has been getting its ass kicked the last couple days by a competitor called SaltyFish that’s less intricate but excels at counterpressing and direct passing. Watching the games reminded me of a meme Tiago Estêvão once tweeted under the caption “look, its Football in 2018”:

If you squint a little you can start to see how City Football Group hopes to get from a stone age video game to a tool that can help real life coaching staffs at its network of clubs from New York City to Mumbai fine tune their game models. Around the time that they launched the competition, Prestidge’s team posted a job ad for an “AI scientist” who would “research & develop AI models that will evolve the tactical principles utilised by our teams across CFG.” The near-term goal is probably to create a realistic enough environment that coaches can test out new tactics over thousands of simulated games instead of, say, a Champions League quarterfinal tie. But the holy grail for this line of research might one day be a computer that can dream up galaxy brain ways of playing soccer on its own, a sort of Pep Bot 9000.

“Did you see the documentary about DeepMind defeating Go?” Dagnino asked me, referring to an AI program that beat the world’s best players at a Chinese board game that’s more complex than chess. The best human players cope with that complexity by trying to defend a compact space, but the machine developed a probabilistic approach that spread the action around the board in unpredictable ways. “For the human, he didn’t understand what the machine was doing, and that threw him off his game,” Dagnino said.

“I don’t think that’s going to happen with football,” he cautioned. Then he paused for a moment and added: “At least not now.” ❧

Thanks for reading space space space! Please consider becoming a socio to support the project and get the premium Friday letters. The last one was on how Getafe's unique playstyle is proving Guardiola wrong; this week we'll look at some Champions League action.

Further reading:

• Google Research Football with Manchester City FC (Kaggle)
• Karol Kurach et al., Introducing Google Research Football: A Novel Reinforcement Learning Environment (Google AI Blog)
• Mark Thompson, Are Man City muscling in on Liverpool's analytics crown? (Get Goalside!)
• James Somers, How the Artificial-Intelligence Program AlphaZero Mastered Its Games (The New Yorker)
• Grace Robertson, Pep in the Knockouts (Grace on Football)

Image: Jenna Sutela, still from nimiia cétiï