Abstract

We create a Spherical Panoramic Image generator that allows a user to take photos of a still scene and create a 360 degree panoramic image that can be used in VR to view the scene. To do this, we use Python's Stitching library, which uses SIFT (scale-invariant feature transform) to combine images using common features called keypoints. We then compare our final product with that of Google's Image Sphere software, which creates a 360 degree panoramic view given photos and photo metadata. Based on quantitative survey feedback, our technology is around the same quality as Google's Image Sphere.

Methodology

Our method for generating the panorama from a set of images is as follows:

First, we use a tripod to gather three sets of images at varying pitches and yaws. These three sets of images are stitched together using an open-source Python library called stitching. Since the images are at one of three different heights, we can use homography to stitch them together into one overall image.

We also want the final panorama to be equirectangular. Since our method doesn't involve taking images of the top and bottom of the view (the stitching process is a bit more complicated as the top and bottom are no longer a "ring" and have to be appropriately distorted), we pad the resulting image to be the correct dimensions. To do so, we add bars on the top and bottom of the image to reformat it to the correct ratio. To try to maintain some semblance of uniformity, we color the top bar with the average color of the top row of the image (or the bottom row for the bottom bar). We can then upload our finished equirectangular image into a software like Blender, which will render the perspective and allow the user to view the scene from any angle.

The code and instructions on how to use the software/render it on Blender are found here. We also have some examples of scenes created by our software.

Since our inspiration for this project was making a better version of Google's Image Sphere without necessitating metadata, we will base our metric on how it compares to a similar scene taken with Image Sphere.
We will use a metric based on people's perceptions of our software's results versus Google's. First, we will show the final result (ours rendered using a software like Blender) to various students and get their ratings on a scale of 1-10 based on the following categories: accuracy (how close the result is to the actual scene) and clearness (lack of visible disparities). Our rating for each scene and software would be the average of the peer reviews of the corresponding scene and software.

Results

We ended up taking three panoramic photos: one in our suite, one in the dorm lounge, and one in the dorm courtyard. For each one, we created a panorama using Google Image Sphere and another panorama using our program. We then polled fellow students for their ratings on accuracy and clearness. The results are shown below.

As we see here, our software and Google Image Stitch receive roughly the same average rating, implying that our software is about as good as Google's. We see this as a huge success, especially given that our program doesn't even consider the top and bottom of the scene. It seems many people were very disappointed in the lack of a floor or ceiling in our resulting image, which brought our rating down. There were still many seam errors (some more obvious than others), as shown in the following image:

However, some of our errors persist in more advanced software, like PTGui's panorama creator. Note how the software duplicates the tower due to the source images having different lighting. So, perhaps some of the seam visibility can be attributed to the quality of the images themselves, and not necessarily to the software.