Photographic 360 degree capture devices such as the "360 One VR" manufactured by EyeSee360 and Kaidan are based upon conical mirrors mounted in front of the lens of an ordinary still or video camera. They allow one to capture the whole 360 horizontal field of view and a limited vertical field of view, typically around 100 degrees. The resulting image looks not unlike a panoramic image but with a different radial distortion. Obviously one wants to undistort these images to create more conventional views, in the discussion here that includes panoramic images, and cubic (perspective) maps.
The suppliers of these capture devices typically provide stand alone software or plugins for other commercial image manipulation packages, eg: PhotoShop. They don't generally provide source code so that the undistortion can be integrated into a customers software. The goal of this exercise was to verify the geometric undistortion required was correctly understood and to demonstrate that with efficient and portable algorithms written in C/C++.
Conversion to a cylindrical panoramic
Transformation to a panoramic image is relatively straightforward in theory, lines of longitude in the panoramic are concentric circles in the oneshot image. Lines of latitude in the panoramic are radial lines in the oneshot image. In practice the oneshot image is not necessary centered and the inner and outer ray rims need to be specified.
 Original image |
 Supersampling antialiasing of 3 oneshot2pan -a 3 |
 Specify center of cone oneshot2pan -a 3 -c 330 250 |
 Specify inner and outer radius oneshot2pan -a 3 -c 330 250 -r1 70 -r2 224 |
 Change direction oneshot2pan -a 3 -c 330 250 -r1 70 -r2 224 -d |
 Force the height of the panoramic oneshot2pan -a 3 -c 330 250 -r1 70 -r2 224 -w 800 -h 200 |
 Subset of longitude oneshot2pan -a 3 -c 330 250 -r1 70 -r2 224 -long1 -45 -long2 90 |
Antialising is achieved using supersampling with a rectangular weighted window. The right horizontal axis of the panoramic is taken as 0 degrees longitude. Options are also provided to create a subset of the panoramic image, namely, the panoramic image between two lines of longitude.
Command line usage
oneshot2pan [options] tgafile Options: -long1 n start longitude value -long2 n start longitude value -r1 n inner radius -r2 n outer radius -c x y center of cone -a n set antialias level -w n width of the output image -h n height of the output image -i apply vertical flip -d change direction
Conversion to cubic map
 oneshot2cube -a 3 -c 330 250 -r1 70 -r2 224 -lat1 -60 -lat2 0 |
Command line usage
oneshot2cube [options] tgafile Options: -r1 n inner radius -r2 n outer radius -c x y center of cone -a n set antialias level -w n width of the output image -h n height of the output image -l1 n latitude at r1 -l2 n latitude at r2 -i apply vertical flip
Conversion to cubic map with a range of longitudes
Standard perspective projections can be considered to be longitude subsets (less than 90 degrees wide) taken from the cubic map. Ideally the cubic map would be aligned so that edges didn't not fall within the perspective frustum (not yet implemented). Of course one could derive the perspective views directly without using this cubic map approach.
 oneshot2cube2 -a 3 -c 330 250 -r1 70 -r2 224 -lat1 -60 -lat2 0 -long1 270 -long2 360 |
 oneshot2cube2 -a 3 -c 330 250 -r1 70 -r2 224 -lat1 -60 -lat2 0 -long1 180 -long2 360 |
oneshot2cube2 [options] tgafile Options: -r1 n inner radius -r2 n outer radius -c x y center of cone -a n set antialias level -w n width of the output image -h n height of the output image -lat1 n latitude at r1 -lat2 n latitude at r2 -long1 n start longitude -long2 n stop longitude -i apply vertical flipReferences
A Catadioptric Sensor with Multiple Viewpoints
Libor Spacek
Department of Computer Science, University of Essex
Single viewpoint catadioptric cameras.
Baker, S., Nayar, S., 2001.
PV01. pp. 39-71
Omnidirectional vision: Theory and algorithms.
Daniilidis, K., Geyer, C.,
In: ICPR00. Vol. 1. 2000
A unifying theory for central panoramic systems
and practical applications.
Geyer, C., Daniilidis, K.,
ECCV00. 2000
Nonmetric calibration of wide-angle lenses and polycameras.
Swaminathan, R., Nayar, S. K.,
IEEE Transactions on Pattern Analysis and Machine Intelligence
Evaluation of the lomography fisheye camerafor capturing images for dome/planetarium applicationsApril 2005
Introduction |
 Lomography fisheye camera | |
|
| ||
|
Example The first thing one notices is the fisheye is truncated at the top and bottom of the frame. A real shame since it wouldn't seem like it is a requirement, one would imagine it might be just as easy to arrange the lens so it is inset with the 35mm image frame which would greatly improve the product. Aperture: 170 degrees. |
| |
|
| ||
|
Most of the fisheye tools expect, at least will be easier to use, if the fisheye is centered in the image frame. Unfortunately due to variations in the film processing the fisheye wasn't always in the same place so this process can't readily be automated. The discussion here is targetted at planetarium style domes, for front facing domes (eg: VisionStation) the processing of the panorama will be different since these domes are often truncated at the bottom of the fisheye. |
| |
|
| ||
|
For a uni-direction planetarium (seating mostly pointing in the same direction) it isn't critical if the part of the fisheye at the back of the dome is missing. This means that one can virtually rotate the fisheye so as to create a correct view around the front horizon. Notice that in the image on the right the bottom of the fisheye (camera upright) is truncated more than the top. For fisheye rotating it is better to use the camera upsidedown which means less rotation is necessary. An alternative is to use the camera on its side which means the result will be clipped on the left and right of the dome, less desirable in my opinion. |
| |
|
| ||
|
Many fisheye projection systems truncate the back 1/4 of the fisheye anyway, in those cases this fisheye rotating results in even less image loss. Unfortunately this camera is pretty much limited to daylight (or at least very well lit) spaces, the flash is located behind the lens so the lens casts a shadow into the scene. This is described on the lomography site, indeed they "try" to suggest it gives interesting results. |
| |
|
| ||
|
Panoramic images Using a fisheye camera, it is possible to form a 360 degree panoramic by just capturing 4 images (see below). The aperture of this camera is just enough to get some overlap between the 4 images with the camera in used on its side. The process involves converting the fisheye images into limited angle panoramic images and then stitching them together to form the full panoramic image (see below). The panoramic can then be used in applications such as QuickTime VR, see right. It is easy to find the artifacts in the panoramic shown here, to create these successfully one really needs a good tripod in order to keep the camera level (this camera doesn't have a tripod mount) and one needs good control over the lighting. |
 QuickTime VR | |
|
| ||
 | ||
| The 4 individual fisheye images. | ||
|
| ||
 | ||
| Stitched panoramic image (Melbourne Museum and Royal Exhibition Building). | ||
|
| ||
|
Further examples | ||
 Melbourne Library |
 Melbourne Central | |
Evaluation of the Canon HV20 and Nikon FC-E9 lens
October 2007
|
|
|
The first thing one notices is that the circular fisheye, with the minimum zoom, does not fit within the sensor and it also isn't centered. The details (which may vary slightly with each camera) are shown below. While this may seem like a serious problem, for many applications it isn't. Many small planetarium systems employing a single projector and fisheye lens often only use 3/4 of the fisheye image, the portion at the "back" of the dome isn't used. Alternatively for upright domes it is common for the lower portion of the dome to be truncated (see examples later).
One solution to this is to digitally tilt the fisheye so as to remove the truncation at the bottom of the fisheye image. This technique applied to the image above is shown below, the green line shows the truncated region employed by many fisheye projection systems. Note that the image in the top half of the fisheye now does not fill the circular fisheye area, blue circle outline. This too can be corrected although unlike the tilting of the fisheye, this additional correction introduces distortion to the the upper half of the image which may or may not be objectionable, remembering that it is behind the viewer (at least in a directional dome).
Complete example
Original frame off the Canon HV20 (movie sequence)
Further examples
Two further examples are shown below, single frames extracted from a movie. These are rendered at XGA resolution (1024x768), that is, the lowest resolution fulldome projection employing a fisheye lens. It is my estimation that the quality in this case is limited by the the projection system as much as by the footage. By comparison, the quality on a HD projection system employing a spherical mirror is clearly limited by the quality of the footage. I suspect carefully captured, processed footage would be a good match for a SXGA+ based system.
|
|
Upright dome
It turns out that the camera and fisheye position is more suited to truncated style upright domes. When the camera is operated upsidedown the larger part of the fisheye image missing isn't required anyway since it is below the hemisphere truncation. The smaller missing piece at the other end of the fisheye results in a small section of missing image at the very top of the dome, or it may be filled by using the same fisheye tilting procedure discussed above.
Original frame off the Canon HV20 (movie sequence)
Nikon FC-E8:
This gives a similar circle but doesn't focus towards the rim where
the image fades to black. Also seems the field of view is somewhat less than 180 degrees.
Phoenix Super Fisheye Lens 0.25X:
This gives a smaller circle than the
Nikon FC-E9 and results in a field of view I suspect between 160 and 170 degrees.
Sample image below.
This combination, Nikon D300 and Sunex 185 degree (5.6mm) fisheye lens, gives a full 180 degree fisheye image using more of the full frame than most other fisheye lens tested (notably Sigma lens). After cropping to 180 degree the resulting fisheye is in the order of 2580 pixels square.
|
|
The lens is certainly very solid. Extremely good focus, no manual focus required since it is fixed. I experienced very few problems even when shooting into the sun. It is not a perfect f theta lens, but then what fisheye lens is.
