Google Earth 4.3 3D StreetView Starts To Solve “Immersive Hotdog Problem

Hmmm... This Looks Familiar.....

Courtesy Frank Taylor GEBlog.com.

Kinda like what I proposed December 30th, 2007??

"Can You Solve The 'Immersive Hotdog Problem'?"

While I contact my lawyer about possible Google royalty checks, it is great to see this capability brought to life - and I really like the blend effect that is presented when you click between each immersive sphere to give the sense of warping / traveling between each position.

What would be really fantastic would be for Google to release information on how this is done - particularly if you have an ImmersiveMedia set up of your own - but even if you just have an IPIX system it would be extremely helpful.

However, right now it is still more like a peapod than a hotdog - you can jump from pea to pea and turn the sphere into a mercator projected photo, but the true geometry that is collected in each sphere is not being projected out to the correct positions yet to create the solid hotdog.

How can we make that work? Is a new PhotoOverlay geometry type needed?

Whom do we ask for that? Google or the OGC?

What Could We Do With 20,000+ "Airborne - Immersive Hotdogs" Over The US Per Day?

I just flew in from Miami - and boy did I have an idea.....

On my way down to Miami to watch Virginia Tech hand Kansas University a victory in the 2008 Orange Bowl, I set my GPS logger up near my window seat and periodically snapped some shots out of the window.

(Click here to view or download the 88 Photo dataset for yourself via Google PicasaWeb.)

(Click here to view the interactive Google Maps version.)

(Or click here to view the dataset directly in Google Earth.)

I was pretty pleased with the results, particularly the shots coming in from the Atlantic Ocean, over Miami Beach and into MIA!

Hmm... sort of like Microsoft's BirdsEye....

And, it got me thinking - could you imagine if instead of 88 shots over 2 hours from one window, I had the ability to record 30 frames per second from 12 lenses 360º around the plane for the full 2 hour flight?

That would give me 1,296,000 georeferenced images worth of data .

Then I started thinking about an NPR piece that I heard back in November that mentioned the scale of the number of flights flown each day across the US.

(Watch a beautiful video by Aaron Koblin depicting US Air Traffic Below)

[youtube=http://www.youtube.com/watch?v=dPv8psZsvIU]

So those 1 million + georeferenced images would come from just one of roughly 20,000 flights flown over the US each day...imagine if each of those flights had a similar system!

I think this would be fairly easy to accomplish if you took something like the Immersive Media Dodeca System and distributed the sensors around the airplane (maybe as simple as splitting the camera in half along the equator and mounting one half on top and one half on the bottom of the plane, or splitting the lenses out all around the plane.)

As far as data processing and storage is concerned, I'm not even sure that capability is possible right now - but imagine all of this data available to update Virtual Earth or Google Earth with near real time immersive imagery from multiple angles and altitudes - all time-stamped so that you could observe changes in construction, population movements, traffic, etc over time.

Would the data be collected when the plane lands, or transmitted via some sort of tether to a central processing station?

Apparently the latter idea isn't so far fetched, considering that Boeing has reportedly demonstrated a satellite video tether system to the FAA which is intended to broadcast in-cabin video for security purposes.

The cost of doing something like this at first would seem to be incredibly high, - the electronics and sensors required currently aren't cheap but if purchased on this scale they may not be too outrageous. However, the platform (commercial airliners) are already flying en masse every day - there is practically no cost associated with the actual flight of the sensors - only in their initial equipment purchase and the infrastructure / storage / processing system.

Would it be worth it to the US to try to organize such an effort?

How much is currently spent on similar data collection for the USGS and other national agencies - could this capability provide more accurate and denser data than traditional satellite based sensor systems?

Then again - I'm having a hard enough time figuring out how to process less than 1 mile of an "Immersive Hotdog" along Manasquan, NJ's boardwalk - this "airborne - immersive hotdog" idea probably needs to go on hold for a while :)

Can You Solve the "Immersive Hotdog Problem"?

Alright....what the heck is the Immersive Hotdog Problem??

There probably is a more elegant description for what I am proposing, but to me the best way I can explain what I am envisioning is this:

An Immersive Hotdog is a georeferenced linear collection of immersive photographs which create a seamless immersive environment which can be entered in a Geospatial Exploration System such as Google Earth, Microsoft Virtual Earth, ESRI ArcGIS Explorer, or NASA's WorldWind. The shape of these immersive collections roughly resembles a hotdog.

Imagine if you will, a long hotdog-shaped object lying on the Earth - about 100 feet wide, 100 feet tall, and about a mile long.

Now imagine that hotdog-shaped object is lying on the ground in GOOGLE Earth - and it is textured with real ground photography taken from the center of the hotdog.

This is the Immersive Hotdog - the source can be from a video camera collection system such as Immersive Media, or from standard immersive photograph collection systems such as iPIX.

These collections have existed in stand alone viewers for years, however we haven’t visualized this hotdog yet in a truly 3D environment and I think we should.

So I started a little proof of concept experiment on Christmas Eve.

I didn’t have a fancy Immersive Media video camera, but I did have an iPIX kit which I borrowed to begin exploring this problem.

So, on Christmas Eve I walked down the boardwalk in Manasquan, NJ and took a series of iPIX shots every few houses (and brought along my new trusty QSTAR Solar GPS Bluetooth Data logger to capture the precise location of each shot.)

I then returned home and synced the GPS log from the QSTAR logger up to the iPIX fisheye photos using the awesome Google Picture Sync (GPicSync) application.

Here are the results:

(144 geotagged Raw iPIX Fisheye shots which you can download with full GPS EXIF data here)

((Hint: Use the "Download Album" option on the left side to directly download all images to your own Picasa))

((( Because Google's PicasaWeb supports EXIF data, you can view all of these images on a Google Map here)

Each iPIX shot consists of 2 Fisheye shots (180º opposed to one another) which we need to stitch into one Equirectangular image

Here’s the first problem - each picture is taken at a slightly different time from the corresponding shot so the GPS position wavers a little bit in that time.

Thats actually a good thing, because what we can do is average out the 2 positions for the pair and find an averaged, perhaps more accurate position somewhere “between” both shots to apply to the single equirectangular shot that we get when we stitch the photos together.

For example, the 2 shots “exploded” in Google Earth above are only slightly “off” from each other in Lat and Lon, but we would average their positions and apply it to the resulting image below. (The iPIX software currently does NOT offer this option…so the first challenge is to average both positions from the source images and write the average to the EXIF of the resulting image.)

(72 Equirectangular Images can be downloaded here.)

I am thinking that some script could be created with Phil Harvey's EXIF Tool to automatically strip the 2 source GPS EXIF locations and average them into the corresponding equirectangular shot?

We can then take the Equirectangular image and wrap around the Spherical <PhotoOverlay> object in KML 2.2 - which will form the first “end of the hotdog.”

The best way that I know how to do that today is to use the PhotoOverlay Tool from Digital Urban.

We can continue to do this with the successive scenes….

So, if we continue to do this, we begin to “fill the Hotdog.”

To do this we would need a way to automate the PhotoOverlay tool (it does have a batch process capability), but we also need to be able to define the rotation angle so that the spheres line up properly on the earth.

Also, many of the spheres I create with the PhotoOverlay tool are way to large. i.e the FOV Near setting is too big.

Now, the Immersive Hotdog is limited in Google Earth because we are restricted as of right now to use multiple sphere objects, which turns out something more like a pea-pod than a hotdog.

But, there has been some fantastic development by Microsoft as of late with their photo stitching (HD View, Microsoft Live Photo Gallery, PhotoSynth, Birdseye 3D, etc.)

Maybe their developers would have a solution to not only successively build the sphere objects but actually stitch the scenes together seamlessly and create true immersive hotdog objects?

Or, maybe someone clever out there has an idea on how to do this?

Please let me know!