Geollery.com: Reconstructing a Live Mirrored World from Your Browser

TL;DR

Geollery.com is a breakthrough in mirrored world technology, allowing users to virtually walk through any street in the world with 6 degrees of freedom (6DoF) using only a web browser. By fusion-loading Google Street View data and OpenStreetMap metadata in real-time, it creates an interactive social platform where live social media feeds (Twitter, Yelp) are anchored to their physical 3D locations.

Background: Beyond the Static Panorama

While we have had 360-degree panoramas for years, they often feel like "bubbles"—you can look around, but "walking" requires a jarring teleportation to the next bubble. Conversely, full 3D city models (like those in high-end games) require months of manual labor or heavy offline processing. Geollery bridges this gap by using just-in-time reconstruction, turning flat maps and panoramas into a navigable 3D mesh the moment you arrive.

The Core Challenge: Bandwidth vs. Fidelity

The primary obstacle for web-based VR is the "Data Wall." Downloading dense 3D point clouds or high-poly meshes takes too long. Geollery's insight is to use compressed depth maps (often under 10KB) provided by Google Street View and perform the heavy lifting—the geometry vertex displacement—directly on the user's GPU.

Methodology: The Dual LoD Pipeline

1. Fine Detail (Street Level)

When you are "walking" on the street, the system fetches adjacent 360° panoramas and their corresponding depth maps.

• Geometry Generation: A sphere with ~131k vertices is created. A custom vertex shader samples the depth map and "pushes" the vertices to their correct 3D positions in real-time.
• Seam Smoothing: To fix gaps between different panoramas, the authors use Weighted Alpha Blending and Gap Alignment, ensuring that as you move from one viewpoint to another, the transition is visually seamless.

Figure: The pipeline showing (a) initial sphere, (b) single depth-mapped geometry, (c) merged geometries, and (d-e) the final textured result.

2. Coarse Detail (Bird's Eye View)

When you zoom out to a "God view," the spherical depth maps become inefficient. Instead:

• Building Extrusion: The system queries OpenStreetMap for building footprints and extrudes them into 3D boxes based on height metadata.
• Projective Mapping: The street-level panoramas are "projected" onto these boxes. To prevent "bleeding" (texturing the back of a building with the front view), a Normal-based Occlusion Test is applied in the fragment shader.

Figure: Visualizing the impact of Seam Blending (left) vs. Sharp intersecting edges (right).

Performance and Real-World Impact

Geollery isn't just a tech demo; it's a social hub. By integrating geotagged social media, it populates the 3D world with live human activity.

• Workstations: 90 FPS in VR (Oculus Rift).
• Mobile: 30-40 FPS (Snapdragon 835/iPhone XS).
• Global Reach: Tested by 700+ users across 25 countries during its debut at CHI 2019.

Critical Insight & Future Outlook

The genius of Geollery lies in Inductive Bias—it assumes the world is made of certain shapes (spheres for streets, boxes for buildings) and uses sparse data to "warp" those shapes into reality.

However, the system still struggles with "ghost" geometry caused by vehicles or trees that aren't properly filtered in the depth maps. The future of this technology likely involves Neural Re-rendering or AI-driven in-painting to fill the "shadows" where no camera has looked.

Conclusion

Geollery.com proves that the "Mirrored World" is no longer a sci-fi concept reserved for top-tier research labs. By repurposing the world's existing GIS data, we can democratize 3D telepresence, allowing anyone with a smartphone to "fly" to a remote city and experience it as a live, 3D social space.