The Vision: A “virtual earth” made from millions and millions of images and videos easily uploaded by anyone. These images are merged into true 3D, allowing seamless navigation. These images are tagged and can be filtered. One can walk down the Champs Elysee on a spring morning, or see Times Square in 1900 (via historical photos), or wander around a crowded Death Valley with every person in the database visible at the same time.

In an extreme form, this virtual earth can be overlaid on top of actual scenes in real time, perfectly registered, using augmented display eyewear.


Background: “Traditional” moviemaps like Aspen, Golden Gate, Karlsruhe, and Banff offer photo-realistic virtual travel by building custom camera systems and exhaustively filming a place. This is still a good idea, but two major changes have occurred since the halcyon days of filming the streets of Aspen.

First, the power of the Web is in its numbers, and the most powerful applications, from Flickr and Del.icio.us to Open Source in general, are built and refined little by little by very large numbers of participants.

Second, in a post-9-1-1 world, it’s not clear that a fleet of corporate or government -backed camera cars sweeping a city will be welcome by the community. As summarized by the late geographer Bernard Nietschmann, “Maps are power. You either will map or you will be mapped.”

In the late 1990s, the MIT Media Lab initiated what they called the “WorldWide MovieMap” (was www.wwmm.org). The idea was for the world community, particularly amateurs, to photograph the world as they saw it and to upload the images to an enormous database. The dream was to somehow merge these images to make them contiguously navigable. The project was a still-born.

Today maps are red hot, led by Google Earth. “State of the art” at the moment is high-resolution aerial photographic data mapped onto a topological model with the addition of blank, “shoebox” like representations of known buildings in several urban areas. The obvious next step is to add photograph texture, and perhaps more shape, to these shoeboxes. These will almost certainly be done as a centralized endeavor.

The collective alternative is to allow anyone to upload geographically anchored images. Several such systems currently exist (e.g., Mappr). But 2D images remain 2D in computer 3-space.


Next steps: The trick will be to merge 2D images from arbitrary and numerous sources into something 3D (or 3D-like) allowing navigability, and where the "sense of place-ness" remains. My past work in this area can be found here and here. I have some specific ideas on how to approach this problem.

(last updated 27 March 2006)


News:

Apr 08: "Viewfinder: How to Seamlessly 'Flickrize' Google Earth" report and video are now online.

Jan 08: "Seamless Image Integration Into 3D Models" patent app is now online.

7 August 2007: Recipient, Google Research Award for "Collective Photo Mapping" (details forthcoming).

25 June 2007: Patent filed for “a method and system for seamlessly integrating arbitrary images into pre-existing 3D models” via USC Stevens Institute for Innovation.

14 July 2006: New work on "How to seamlessly 'Flickrize' 'Google Earth': a novel approach to seamlessly integrate arbitrary images into pre-existing 3D models".

                              back to big projects