How Pokémon Go Paved the Way for Advanced Visual Positioning

When Niantic launched Pokémon Go in 2016, it revolutionized augmented reality (AR) gaming by blending virtual creatures with real-world locations. This innovative AR experience captivated millions globally-from bustling Chicago to scenic Oslo and Japan’s Enoshima-encouraging players to explore their surroundings in search of Pokémon like Jigglypuff, Squirtle, or the elusive Galarian Zapdos. The game’s unprecedented popularity saw over 500 million downloads within just two months, according to Niantic Spatial’s CTO Brian McClendon. Even eight years later, in 2024, Pokémon Go continues to engage more than 100 million active users worldwide, underscoring its lasting impact.

Transforming Crowdsourced Data into Precise Location Intelligence

Niantic Spatial, a spinout from Niantic, is now leveraging the immense volume of geotagged images collected from Pokémon Go and Ingress players to develop a cutting-edge visual positioning system. This technology can determine a user’s exact location within centimeters by analyzing just a few photos of nearby landmarks. The goal is to enhance navigation accuracy in environments where GPS signals falter, such as dense urban areas with tall buildings and complex infrastructure.

By harnessing billions of images tagged with precise spatial metadata-including orientation, movement, and time of capture-Niantic Spatial has created a dynamic “world model” that grounds artificial intelligence in the physical environment. This approach bridges the gap between virtual mapping and real-world navigation, enabling applications beyond gaming.

Robotics Meets AR: Precision Navigation for Delivery Bots

Niantic Spatial’s visual positioning technology recently found a practical application through a partnership with Coco Robotics, a company operating approximately 1,000 autonomous delivery robots across cities like Los Angeles, Chicago, Miami, Jersey City, and Helsinki. These robots, roughly the size of flight cases, can transport up to eight extra-large pizzas or four grocery bags, having completed over 500,000 deliveries across millions of miles in diverse weather conditions.

To rival human couriers, Coco Robotics emphasizes punctuality and reliability. CEO Zach Rash highlights the critical need for robots to arrive exactly when promised, which requires precise navigation to avoid getting lost. However, GPS signals often degrade in urban “canyons” where skyscrapers and overpasses interfere with satellite communication, causing location errors of up to 50 meters-enough to misplace a robot on the wrong street.

Overcoming GPS Limitations with Visual Positioning

Niantic Spatial’s system addresses these challenges by using visual cues from the environment rather than relying solely on GPS. The technology compares real-time images captured by the robots’ multiple cameras to its extensive database of urban landmarks, enabling centimeter-level localization even in GPS-denied zones. This method is akin to how Pokémon Go players’ phones identify their position by recognizing nearby buildings and features.

John Hanke, CEO of Niantic Spatial, explains that the core problem-accurately situating a virtual character like Pikachu or a physical robot in the real world-is fundamentally the same. By repurposing data from millions of AR game users, Niantic has built a robust visual positioning model that can be adapted for robotic navigation.

Building a Vast Visual Database for Enhanced Mapping

According to geospatial expert Konrad Wenzel from ESRI, visual positioning technology is not new, but its effectiveness grows exponentially with the number of cameras and data points available. Niantic Spatial’s model is trained on over 30 billion images collected from key “hot spots” worldwide-locations frequently visited by players, such as Pokémon battle arenas. This results in a rich dataset with thousands of images per site, captured from multiple angles, times of day, and weather conditions.

Each image is accompanied by detailed metadata, including the phone’s exact position, orientation, and movement at the moment of capture. This comprehensive dataset enables the model to accurately infer location even in less-documented areas, extending its utility beyond the original gaming hotspots.

Integrating Visual Positioning into Autonomous Delivery

Coco Robotics equips its delivery bots with four cameras positioned at hip height, providing a 360-degree view of their surroundings. Although the robots’ perspective differs from that of a handheld phone, adapting Niantic’s visual positioning data to their needs has been straightforward. This integration allows the robots to navigate complex urban environments with improved precision, ensuring they stop exactly at designated pickup points and deliver orders right to customers’ doors.

While competitors like Starship Technologies also employ visual mapping-using sensors to create 3D models of their environment-Coco Robotics believes Niantic Spatial’s extensive image database and refined positioning algorithms offer a competitive advantage in accuracy and reliability.

The Growing Role of Visual Positioning in Robotics

Originally developed to enhance AR experiences, Niantic Spatial’s visual positioning system is now fueling a surge in robotics applications. As robots increasingly share spaces with humans-on sidewalks, construction sites, and urban streets-they require sophisticated spatial awareness to operate safely and seamlessly. John Hanke emphasizes that for robots to integrate smoothly into human environments, they must possess a detailed understanding of their surroundings, including the ability to recover from collisions or disruptions.

The collaboration with Coco Robotics marks the beginning of a broader vision: creating a continuously updated “living map” that reflects real-world changes in near real-time. As autonomous machines traverse cities, they will contribute fresh data, enriching digital replicas of the environment and enabling smarter navigation and interaction.

From Static Maps to Intelligent World Models for Machines

Traditional maps have long served humans by representing spatial and temporal information in two, three, or even four dimensions. However, for machines, maps must evolve into comprehensive guidebooks filled with semantic details-descriptions of objects, their properties, and contextual information that humans intuitively understand but machines do not.

Niantic Spatial and companies like ESRI are pioneering this shift by annotating maps with rich metadata, enabling AI systems to interpret and interact with the physical world more effectively. This approach addresses a key limitation of large language models (LLMs), which often lack common sense about everyday environments despite their vast knowledge.

Advancing AI with Real-World Grounding

While some organizations, such as Google DeepMind and World Labs, focus on generating synthetic virtual worlds to train AI agents, Niantic Spatial takes a complementary path by striving to digitally reconstruct the real world in exquisite detail. Brian McClendon expresses a clear ambition: to push mapping technology to the point where it captures the full complexity of the physical environment, providing a solid foundation for AI systems to operate with genuine spatial awareness.

This real-world grounding is essential for the next generation of AI applications, from autonomous delivery and robotics to augmented reality and beyond, promising a future where machines navigate and understand our world as intuitively as humans do.