warpplayer: QuickTime movie player
with realtime warping and navigation

Trial version (Mac OS-X Leopard): WarpPlayer_demo_95.zip. Please check the demo version with your hardware/OS before buying.

Introduction

"warpplayer" is a Mac OS-X application that takes a QuickTime movie and a description of how to distort each frame. The movie is played and distorted on the fly. There are many applications for this and some of them will be discussed below, the main reason for developing this was for movie playback in fulldome (eg: planetarium environments) where a spherical mirror is used to distort the fisheye frames of the movie.

In addition to the realtime warping, the application allows navigation while the movie is playing. The exact navigation that is appropriate depends on the input image projection, for example, for a fisheye image the movie can be rotated about the pole while it is playing. For a movie where each frame is a cylindrical projection the view direction can be rotated horizontally during playback.

The following example is targetted at fisheye movies, the original frame is taken from "Moonlight" by Andrew Quinn. A sample frame is shown below along with various mappings. The power of this approach is immediately obvious, namely that one movie can be shown in multiple environments without being preprocessed for each. Note the grid lines are turned on in order to illustrate the nature of the texture mapping. For all practical purposes any image distortion can be achieved, it only requires that someone can create the correct warp map file.

Single frame from the fisheye movie.	Truncated fisheye as used by many fisheye projector systems.
Warped image for spherical mirror projection, 4x3.	Warped image for spherical mirror projection, 16x9.

Mesh data file structure

The data file that describes the warping is simply a description of a regular mesh with texture coordinates. The format is as follows: the file is a plain (human readable) text file, the first line contains a single digit that indicates the input image format the mesh is assumed to apply to, the second line gives the dimensions of the mesh, the subsequent lines contain 5 numbers per line that describe each node of the mesh. These lines consist of x and y (the coordinate of the mesh in normalised screen coordinates), the u and v texture coordinate of that node (range from 0 to 1), and finally a multiplicative intensity value. Normalised screen coordinates range from -aspect to aspect ratio in the horizontal axis and -1 to 1 in the vertical axis.

• 1: planar image, perspective projection
  
• 2: fisheye projection
  
• 3: cylindrical projection
  
• 4: spherical projection
  
• 5: cubic map
  

Please contact the author for further details on these mesh files and how they might be created for your own distortion requirements. Two mesh files are given here as examples: rect.data.zip doesn't perform any distortion, for a 4:3 movie it plays it back unmodified; warpfish.data.zip warps a fisheye movie for spherical mirror projection (at least for a particular configuration and 4:3 aspect ratio projector).

While the exact form of the projection of the input image and the form of the warping is very general, there are some input and output image projections explicitly supported (tested) and for which example mesh files can be provided. Note also that the exact form of navigation is dependent on the input image projection for example, it doesn't make sense (at least in a hemispherical dome environment) to zoom in on a fisheye image, and only horizontal rotation makes sense for cylindrical projections.

Keyboard control

• Space bar to start/pause movie playback.

• Fast forward and reverse (new) using the "<" and ">" keys.

• Full screen mode (of course) as required for most single projector systems, optionally entered into at launch time.

• Supports all the standard QuickTime video and audio codecs.

• Option to automatically loop movie.

• Drag/drop mode as well as unix scriptable launching.

• Default mesh (loaded at launch time) is called "default.data".

The performance is directly related to the underlying hardware being used, the penalty for the warping part of the pipeline is minimal. Put another way, if the movie plays smoothly in the QuickTime Player then it will most likely play smoothly in this player. There are many factors involved when it comes to the playback of large frame movies, for example, the codec used, the hard disk speed, the CPU type and performance, the graphics card, and so on. At the time of writing the author has not had problems playing 1536x1536 pixel movies at 30fps on a 15" Mac Powerbook. Recent experiments with HD projectors have been successful with a higher configured Powerbook, that is, playing 2000x2000 pixel movies and creating 1920x1080 pixel warped images for spherical mirror projection. The movie size, compression ratio, and performance will only increase over time.

New: stereoscopic projection (March 2008)

The application has been enhanced to both support side-by-side stereoscopic movie playback and to allow fullscreen playback across multiple displays. These are actually related in the case where one is attempting passive stereo projection using a dual display graphics card.

Example

As an example of the generally of this approach to realtime warping and to demonstrate how warp maps might be generated, consider the case of warping a fisheye into half a spherical projection.

#define NX 50 #define NY 50 int main(int argc,char **argv) { int i,j; double x,y,u,v; XYZ p; double theta,phi,r; printf("2\n%d %d\n",NX+1,NY+1); for (j=0;j<=NY;j++) { for (i=0;i<=NX;i++) { x = -1 + 2 * i / (double)NX; y = -1 + 2 * j / (double)NY; theta = PID2 + x * PID2; phi = y * PID2; p.x = cos(phi) * cos(theta); p.y = cos(phi) * sin(theta); p.z = sin(phi); theta = atan2(p.z,p.x); phi = atan2(sqrt(p.x*p.x+p.z*p.z),p.y); r = phi / PID2; u = (1 + r * cos(theta)) / 2; v = (1 + r * sin(theta)) / 2; printf("%g %g %g %g 1\n",x,y,u,v); } } exit(0); }

Sample frame from a movie (left hand side) and resulting spherical projection (right hand side).