Ongoing report from AUC Innovation Grant
High resolution displays using Apple hardware
Written by Paul Bourke
May 2007 - ongoing
Original proposal
Evaluating the creation of a single large digital canvas using Apple technology
The original proposal to the AUC Innovation grant round in 2007 targetted the identification
of capabilities/limitations of the Mac OS-X and related APIs to handle large pixel
displays. Particular attention was to be made of innovative display technology whether
it be high resolution, stereoscopic, or immersive displays.
Some examples of the types of questions asked are:
Which APIs could handle large images/textures (eg: in particular
images larger that texture sizes on graphics cards, 2048 or 4096 pixels).
The author is often interested in large format QuickTime movies, 2048 pixel wide
for XGA stereo, 2800x1050 pixel movies for SXGA+, 3840x1080 pixel movies for HD stereo,
and 2Kx2K movies for fisheye projection. Which are the best codecs for these unusually
large movie formats?
How does one drive large numbers of displays from a single Mac? What are the limits?
There are a number of possible benefits to AUC members of these display possibilties.
Visualisation of geometry rich datasets. These are datasets that one wishes to view in
their entirety in order to detect global structure but for which there is more data than
available pixels.
Viewing/inspection of high resolution images. For example geographic surveys,
Hubble deep field, and remote imaging. Such high resolution images are increasingly
common given the ongoing improvements in imaging CCD chips.
Immersive displays for virtual reality style environments/experiences. There are
applications of such environments for engaged learning and training.
Engaging displays for public education, artistic works, and entertainment.
Revised proposal
It soon became apparent that the original proposal was flawed, in the sense that a
detailed analysis was going to be almost immediately out of date. During the first 6 months
of the project there was the release of Leopard (the original proposal was based upon
Tiger), the QuickTime API went through some significant changes (and continues to do so
on a regular basis), there have been some improvements in the X11 and GLUT libraries, and
there have been releases/support for a number of new graphics cards. All of the above and
the expectation of ongoing change means that the proposal analysis which would occur at
a single point in time would quickly be out-of-date. In particular, any statement as to
limitations would quickly be incorrect.
The solution I suggest is to report on various solutions to various real world "novel"
projection projects I have been engaged in over the duration of of the grant in the form
of case studies.
These demonstrate actual applications employing high resolution, stereoscopic, and immersive
projection environments all leveraging Mac OS-X technology. Readers are encouraged to
contact the author for more indepth implementation details.
Case Studies
These first three case studies are exhibitions that occured before this grant
was awarded. It was the questions raised here that prompted this study into how
to optimally drive novel or challenging displays using Apple hardware. The case studies listed
here illustrate various high resolution activities, test cases, that may extend the
graphics capabilities and thus reveal possible limitations. The lessons learnt here
may be of use to others contemplating something similar, the author is happy to discuss
any of the following further with interested parties.
|
Case study 1
Strange attractors, 2006. Four XGA resolution projections.
3.1 MPixels
|
|
|
Case study 2
Antartica Virtua, 2007. Three XGA resolution stereoscopic projections.
4.7 MPixels
|
|
|
Case Study 3
Boolardy, March 2007. Two 30" Apple displays.
8.2 MPixels
|
|
|
Case Study 4
Sensis HMD, 8 displays, 4 per eye, total of 3200x1200 pixels.
3.8 MPixels
|
|
|
Case Study 5
WA Parliament, August 2007. Three 30" Apple displays, total display is 7680x1600 pixels.
12.2 MPixels
|
|
|
Case Study 6
Driving Sony 4K projector, Four HD (2048x1080) displays, total display is 4096x2160 pixels.
8.8 MPixels
|
|
|
Case Study 7
Driving HD (1920x1080) stereoscopic capable projector.
4.1 MPixels
|
|
|
Case Study 8
Ultra wide tiled display, multiple computers.
Uses network broadcast/receive in Quartz Composer 3.
13.8 MPixels
|
|
|
Case study 9
High resolution dome displays and navigable movies.
2 MPixels
|
|
|
Case study 10
Cosmology gallery, GinGin.
3.9 MPixels
|
|
|
Case study 11 [Yet to be written]
Interactive player for extremely large panoramic images.
|
|
Software Developed
|
Warp mesh patch for Quartz Composer
The Quartz Composer patch discussed here warps an image (or movie) in an
arbitrary way as specified by a user generated mesh stored in a simple text
file. There are many potential applications for this but the motivation for
this capability arose from a very precise requirement, namely the warping of
images for projection into immersive spaces by using a projector and a spherical mirror.
|
|
|
Warp and navigable movie player
Warpplayer is an ongoing project that provides realtime warping of movies according
to a warping description saved in a simple text file, allows navigation within
fisheye, cylindrical, spherical, and planer movies, supports stereoscopic projection,
and a movie player that addresses some of the limitations with the Apple QuickTime player.
|
|
Outstanding issues and comments
As OS and API software releases and updates occur, many of the initially identifed
limitations have disappeared. The following list identify some remaining issues and
includes some comments and other information
for others contemplating demanding digital installations.
|
4096 pixel wide movies
[Last confirmed: March 2008]
There appears to be a bug in the QuickTime API with movies that are exactly 4096 pixels
wide on nVidia video cards. The bug manifests itself as a 1 pixel noisy effect on
the last column, namely column 4096. Doesn't appear with ATI cards. Occurs with software
by the author that uses the QuickTime API as well as the Apple Quicktime player.
Note that this is not related to maximum texture dimensions which are at least
4096 pixels on all cards the bug has been observed. A show stopper when using texture
wrapping of movie frames.
[Update, March 2008]
This bug seems to have been fixed with the current ATI Radeon HD 2600 XT and
NVIDIA GeForce 8800 GT each of which extend the maximum texture size to 8192.
Tiled displays and full screen
It is disappointing that almost all graphics display software on the Mac platform does
not support fullscreen modes across multiple displays. This applies to third party
software as well as to Apple software (QuickTime player, Quartz Composer, screen savers,
etc). An option (as with display drivers on the MSWindows platform) that concatenates
rectangularly bounded tiled displays into a single logical canvas would greatly assist
those working with Macs in large display exhibitions. Currently one is forced to develop
custom applications to support full screen across displays.
Movie playback: nVidia vs ATI
While not quantifed, in the authors experience ATI cards are preferable to nVidia (on the Mac)
for demanding movie playback applications.
Some (standard) large format movie, display, and projection sizes
Projection
- High Definition (1080p): 1920x1080
- Sony 4K native resolution: 4096x2160
- XGA stereoscopic: 2048x768 (horizontal arrangement), 1024x1536 (vertical arrangement)
- SXGA+ stereoscopic: 2560x1024 (horizontal arrangement), 1280x2048 (vertical arrangement)
- HD stereoscopic: 3840x1080 (horizontal arrangement), 1920x2160 (vertical arrangement)
- Planetarium warped fisheye: 1536x1536, 2048x2048 (variable)
Displays
- Apple 24" display (eg: iMac): 1920x1200
- Apple 30" display: 2560x1600
Cameras
- LadyBug camera: 3600x1800 (typical)
- 2 MPixel camera standard: 1600x1200
- 4 MPixel camera standard: 2240x1680
- 11.1 Mpixel camera standard: 4064x2704
- Red camera: 4520x2540
Other
- Matrox dual-head-2-go: 2560x1024
- Matrox triple-head-2-go: 3840x1024
|