Niantic's new AI model may have been built by unaware Pokémon Go players

Update (11/22/2024): A representative for Niantic confirmed to Game Developer the studio has spent the past five years building AI models that power the VPS needed for its Large Geospatial Model. When players opt-in to sharing their AR scans, that data feeds the VPS, and Niantic assured that players' personal data "[has not] been sold to anyone."

They pointed to the PokéStop scanning function in Pokémon Go as an example of the technology, which lets Niantic "generate accurate, dynamic 3-D maps of real-world objects and their relative locations, and help devices understand the surroundings in AR real-time." A blog published on November 13 that dives into an experimental Pokémon Go feature also explicitly credits Pokémon Go and Ingress players with "supporting a large collection of scans throughout the world that allowed us to deliver a new form of AR gameplay to our users."

Niantic's next step, according to the spokesperson, is to use its small AI models to create a larger, global one that "serves even more players around the world." 

Original story: Niantic recently announced a Large Geospatial Model (LGM), which uses machine learning to "understand a scene and connect it to millions of other scenes globally." But according to 404 Media, the LGM was possibly made by conscripting unaware players into doing the studio's work for years through games like Pokémon Go.

In its blog, Niantic expressed hope the LGM would "implement a shared understanding of geographic locations, and comprehending places yet to be fully scanned." It's built on the studio's Visual Positioning System (VPS), which lets players "position themselves in the world with centimeter-level accuracy" and see or place digital content in their exact location, even after they've left.

Both Pokémon Go and Ingress are augmented reality (AR) titles, and collect geolocated images as players explore locations to find Pokémon or player-made art. By Niantic's admission, its VPS has over 10 million locations scanned globally from the past five years, and "receive[s] about 1 million fresh scans each week," which are collected from its games—one of which has been quite popular for nearly a decade, meaning players may have unknowingly participated in helping build its LGM.

Similar to a Large Language Model (LLM), Niantic's model scrapes data from real-world locations, and it hopes to use the technology to "enable computers not only to perceive and understand physical spaces, but also to interact with them in new ways." And what makes its data more substantial than something like Google Maps or Street is the point of view: as Niantic notes, the data is "taken from a pedestrian perspective and includes places inaccessible to cars."

As of July 15, 2024, Niantic's privacy policy confirms it uses geospatial technology and player recordings to "build a 3D understanding of real-world places, with the goal of offering new types of AR experiences to our users." The feature is opt-in and can be disabled by players at any time, but appears to be a "critical component" of Niantic's goals for AR and its model.

"The path from LLMs to LGMs is another step in AI’s evolution," its blog concluded. "The world’s future operating system will depend on the blending of physical and digital realities to create a system for spatial computing that will put people at the center."

The games industry wants to go in on genAI, for better or worse

Companies like NVIDIA and OpenAI have previously come under criticism for using what or whoever they could to build their AI technology. In August, 404 published a report claiming NVIDIA did extensive scraping of copywritten material across YouTube for its AI tools, and would reportedly dismiss concerns about the practice or say it was "in full compliance with the letter and spirit of copyright law."

Breaching consent has been a major point of criticism regarding generative AI and similar technology for some time: voice actors have talked about how their voices have been used by modders making content for popular games, and expressed similar worry that audio companies are forcing them into letting their performances be used to train AI models.

Some developers have used genAI tech in non-voice acting ways as a means of lightening the development load and finding new fixes to old problems. But top-level executives think the technology can also be put toward concepting new game ideas or other creative means.

Game Developer has reached to Niantic for clarification on how its geospacial data was obtained, and the transparency of that data's use. We will update when a response is given.