From: Denis Pelli (3/17/1995)
To: Apple Developer Support, Apple PCI Bus Development
CC: David Nagel
Subject:Video Drivers 5:06 PM 3/17/1995
March 17, 1995

Dear Apple,

We, the 41 undersigned visual scientists [and another ten who endorsed the letter after it was sent], are writing to suggest the addition of a new "cscSetModeTiming" video driver control call to the "Display Driver Calls" chapter of "Designing PCI Cards & Drivers".

THE NEED:

Today's video cards and monitors are mostly capable of running with a practically infinite range of resolutions and field rates (timings). Current MacOS software, recently enhanced by the addition of the Display Manager, restricts this infinite spectrum to a short list of modes (timings) specified in the video card's ROM and driver. This creates a perennial compatibility problem as old video cards cannot drive new displays with unanticipated timings. (In principle a new driver could be installed, but this would require writing a new driver for each video card that is to support the new display, which is impractical.) New display devices--LCD, projection, HDTV, stereo, etc.--are coming and it seems likely that they will function best with new timings that are hard to anticipate today.

One of the best ways of generating stereo imagery (for virtual reality and cyberspace) is to double the monitor's frame rate (e.g. to 134 Hz) and to use LCD glasses that allow each eye to see alternate frames. Many commercially available video monitors will run at such high frame rates, but, alas, most Mac video drivers have no provision for it, even though the video card is capable.

The availability of custom timings would provide a new opportunity for game developers. Games often achieve exciting effects by pushing the hardware to its limits. Some very interesting effects could be generated by smoothly changing the timing from spatially coarse high frame rate images for fast motion sequences to spatially fine temporally slow (e.g. 67 Hz) frame rates for intricate imagery.

Many vision studies require high frame rates, over 100 Hz, in order to study stereo, motion perception, and retinal phenomena without artifacts. Many monitors and most video cards would run happily at such high frame rates, but hardly any of today's Mac video drivers will allow it.

TESTIMONIALS TO THE NEED:

Patrick Cavanagh, patrick@burrhus.harvard.edu, writes "High-frame-rate video cards are available for the PC, typically for stereo. None are available for the Mac. Many of us and those in computer graphics are being forced to the PC market because of this oversight."

James Dinunzio, dinunzio@scf.usc.edu, writes, "Hi. I am a graduate student at University of Southern Cal in the Computer Science Department. I am a member of a newly formed group working under faculty dedicated to graphics and virtual reality. I am currently working on a project to produce decent stereopsis on the Macintosh, including still stereo images, movies, and if possible "live" walkthroughs. I will be using a pair of LCD shutter glasses controlled through the serial port and running at a nominal 30 fps, 60fps for both eyes. ... Any suggestions?"

Tom Busey, busey@indiana.edu, adds, "I have just finished writing code that displays color stereo 3-d pairs for use with LCD glasses. This code uses sequential-frame presentations synched with LCD glasses controlled via the serial port to provide the illusion of depth. While this technology is old, what is new is my code that takes two pict files and reduces the colors from millions of colors to what amounts to 4 bits per red, green and blue channel for each picture (on monitors with millions of colors) or 2 bits per channel for monitors with thousands of colors. The two pictures are then added together and then I use clut switching to switch between the two pictures. The stereo 3-d code uses calls from (not surprisingly) the VideoToolbox code library [described below]. As with all the code I write, this code is freely available to anyone who wants it. I'm currently working with LCD glasses distributors to have it distributed free to purchasers of LCD glasses. It may be downloaded from:"
ftp://ftp.psych.indiana.edu/pub/busey/AdjustStereoPicts.sea.hqx

"The bottom line is that this approach makes virtual reality a (virtual) reality on macs, for very low cost (about $150 for the glasses, plus some parts for a very simple interface). However, because of the relatively slow frame rate of my 2-page monitor (67 hz), I am only getting about 33 images per eye per second. This causes visible flicker, and I would love to be able to increase the frame rate to 120 hz. Such an increase would not only reduce the flicker, but it would also make the glasses and the mac research-grade for doing experiments. Thus I support your call for Apple to add an adjustable frame rate option."

[The following testimonials were received after the original letter was sent. 3/22/95]

Mats Lind, mats.lind@cmd.uu.se, writes, "3-D Max, a Swedish company, working with a Korean supplier, has just started to deliver attractively priced LCD-shutter glasses and accessories giving binocular stereo capabilities to PCs. Their product received enormous attention at the Ce-Bit fair in Hannover last week. The estimated retail price in the US will be less than 180 USD (including some extra software like 3-D games) and will probably reach a high market share. 3-D Max, is very interested in entering the Apple market segment as well, but to do so requires the usual 120+ Hz."

Chris Tyler, cwt@skivs.ski.org, writes, "We are using the VideoToolbox to develop a two-monitor stereo display system. We would be delighted to know of calls that could control the raster rates and video card synchronization. I think that I can speak for many investigators at the Smith-Kettlewell Institute, which probably owns about 50 Macintoshes, that the features you propose would be of great value to many."

FIVE SUGGESTIONS TO APPLE:

We suggest the following additions to the "Display Driver Calls" chapter of "Designing PCI Cards & Drivers".

1. (MAJOR) Add a cscSetModeTiming control call to the repertoire for video drivers to allow specification of an arbitrary new timing.

Using an extended VDTimingInfoRec struct, or perhaps an entirely new struct, the calling program would specify the desired timing (i.e. pixel rate, line rate, field rate, frame synch width, frame front porch, frame back porch, frame synch polarity, line synch width, line front porch, line back porch, line synch polarity, synch on green (y/n), combined synch polarity, and interlaced/noninterlaced, as well as the desired rowBytes and page [oops, I meant "pageBytes"]) and issue a cscSetModeTiming control call to the video driver. In response, the video driver would change the mode to comply, as nearly as possible, with the request, and, before returning, would update the caller's struct to reflect the achievable timing. The driver would also be responsible for updating the mode's other derivative specifications (bounds rect, etc.) that are used by other control and status calls (e.g. cscSwitchMode, cscGetPages, cscGetModeTiming, cscGetNextResolution, cscGetVideoParams). The effect of this change should be persistent, lasting at least until shut down.

(Timing changes that do not affect rowBytes would not require calling the Display Manager to repaint the screen, so most timing transitions could be done smoothly, without any visible glitches. The application would typically call DMSetDisplayMode only at the end of the special effect, mainly to update the bounds rect.)

(cscSetModeTiming is a powerful call that could create timings that would not effectively drive the monitor, and might even harm it. However, this power can be accessed only through a Control call to the driver, and thus is inaccessible to the casual user. We would expect it to be used only by applications written by the monitor manufacturers and developers of games and virtual reality environments and other special-effects applications.)

cscSetModeTiming would allow an application run at start up to customize ANY video card (whose driver supports the call) to drive a novel display, or drive an old display at novel timings. The new timing would persist until shut down or restart.

2. (MINOR) It would be helpful to enhance the corresponding cscGetModeTiming status call to return the same timing info.

3. (MINOR) Please add a trivial cscGetVBLBitAddress status call to return the location of the VBL bit. As far as we know, every Mac video device has a VBL bit, somewhere, that is true only during blanking. Unfortunately there is no easy way to find it. For many special effects it is necessary to synchronize multiple video monitors. Since every video card insists on being a master, and they have independent clocks, monitors normally run asynchronously of one another. In the past we've used custom hardware and software tricks, specific to particular video cards, to reset multiple cards at once, or to halt the clock on one until it drifts into phase with another. A better, general, solution would be to use cscSetModeTiming to tweak the timing of one video card relative to the other until they come into synch. However, to do this, we need to be able to continuously monitor the VBL bit of each device. (Using a VBL interrupt service routine to set a bit at a known location is much less satisfactory because of the extra uncertainty in timing and because this gives us only the time of onset of VBL, denying us access to the time of offset.)

4. (VERY MINOR) We'd like to request that a clarification be added to the documentation of the cscSetGray control call. Most video drivers implement this call correctly. However, a few video drivers (e.g. PowerMac 7100 vram driving an Apple 2-page monochrome monitor, and the Radius PrecisionColor 8 and 8Xj driving a Radius 2-page monochrome monitor), upon detecting the presence of a monochrome video monitor, lock themselves into the monochrome state (i.e. always do luminance mapping) irrespective of calls to cscSetGray. The cscSetGray calls return without error and calls to cscGetGray misleadingly imply that the mono/color mode is being changed, but the driver stubbornly continues to do luminance mapping. This prevents us from using some very useful special effects that depend on having full control of the CLUT (i.e. color mode) while driving a monochrome display (see Pelli & Zhang, 1991). Please tell the developers of video drivers not to get tricky: don't disable cscSetGray.

5. (VERY MINOR) A surprisingly large number of video drivers (e.g. Mac IIci Built-In Video, Relax 19" Model 200) fail to correctly implement cscGetEntries, even though it is a required status call. Perhaps a sentence could be added to the documentation of cscGetEntries, advising developers to perform a write-then-read (cscSetEntries & cscGetEntries) test of their new driver before releasing it to the world. (TimeVideo, described below, does this test, which revealed the bugs.)

Denis Pelli
Apple Partner
Professor of Neuroscience at Syracuse University
denis_pelli@isr.syr.edu
[September 1995: now Professor Psychology and Neural Science at New York University.]

Kenneth R. Alexander
Eye and Ear Infirmary
University of Illinois
kennalex@tigger.cc.uic.edu

Michael Bach
Dept. of Ophthalmology
University of Freiburg
Germany
bach@sun1.ruf.uni-freiburg.de

Martin S. Banks
School of Optometry
UC Berkeley
marty@john.berkeley.edu

Paul Beckmann
Psychology Department
University of Minnesota
paul@eye.psych.umn.edu

Patrick Bennett
Psychology Department
University of Toronto
bennett@psych.utoronto.ca

Margaret Bradley
NIMH Center for the Study of Emotion and Attention
University of Florida.
BRADLEY@nervm.nerdc.ufl.edu

David Brainard
Psychology Department
UC Santa Barbara
brainard@condor.psych.ucsb.edu

Catherine Burns
Institute for Sensory Research
Syracuse University
katey_burns@isr.syr.edu

Tom Busey
Department of Psychology
Indiana University
busey@indiana.edu

Patrick Cavanagh
Psychology Department
Harvard University
patrick@burrhus.harvard.edu

Frans W. Cornelissen
Laboratory for Experimental Ophthalmology
Center for Behavioural, Cognitive and Neurosciences
University of Groningen
The Netherlands
f.w.cornelissen@bcn.rug.nl

Andrew Derrington
Psychology Dept
Nottingham University
UK
Andrew.Derrington@nott.ac.uk

Michael P. Eckert
School of Electrical Engineering
University of Technology, Sydney
Australia
meckert@ee.uts.edu.au

Rhea T. Eskew, Jr.
Dept. of Psychology
Northeastern University
eskew@neu.edu

Bart Farell
Institute for Sensory Research
Syracuse University
bart_farell@isr.syr.edu

Mark W. Greenlee
Neurologische Universitaetsklinik
University of Freiburg
GERMANY
greenlee@sun1.ruf.uni-freiburg.de

Stevan Harnad
Department of Psychology
University of Southampton
UK
harnad@ecs.soton.ac.uk

Mary Hayhoe
Center for Visual Science
University of Rochester
mary@cvs.rochester.edu

Alun Johns
Psychology Department
Oxford University
UK
ajohns@psy.ox.ac.uk

Keith Karn
Center for Visual Science
University of Rochester
keith@cvs.rochester.edu

Stanley Klein
School of Optometry
UC Berkeley
klein@adage.Berkeley.EDU

Peter Lennie
Center for Visual Science
University of Rochester
pl@cvs.rochester.edu

Dennis Levi
School of Optometry
University of Houston
DLevi@UH.EDU

Donald I. A. MacLeod
Psychology Department
University of California S.D.
dmacleod@ucsd.edu

Tony Norcia
Smith Kettlewell Eye Research Institute
San Francisco, CA
amn@skivs.ski.org

Joel Pokorny
Eye Research Lab.
University of Chicago
s+p@chroma.uchicago.edu

Bertram O. Ploog
Hunter College
CUNY
ploog@envmed.rochester.edu

Russell A. Poldrack
Amnesia Research Lab
Beckman Institute
poldrack@uiuc.edu

Manoj Raghavan
Institute for Sensory Research
Syracuse University
Manoj_Raghavan@isr.syr.edu

Allison B. Sekuler
Psychology Department
University of Toronto
sekuler@psych.toronto.ca

Shui-I Shih
Psychology
Southampton University
UK
SHUI@psy.soton.ac.uk

Ben Singer
Center for Visual Science
U Rochester
bens@cvs.rochester.edu

Vivianne C. Smith
Eye Research Lab.
University of Chicago
s+p@chroma.uchicago.edu

Joshua Solomon
Vision Group
NASA Ames
Joshua_Solomon@qmgate.arc.nasa.gov

Hans Strasburger
Institut fuer Medizinische Psychologie
University of Muenchen
Germany
hans@groucho.imp.med.uni-muenchen.de

Joann M. Taylor
Color Technology Solutions
joannt@teleport.com

Marty Wachter
School of Medicine
Johns Hopkins University
mrw@welchgate.welch.jhu.edu

Andrew B. Watson
Vision Group
NASA Ames
beau@vision.arc.nasa.gov

Lan Zhang
Psychology Dept.
Syracuse University
lzhang@mailbox.syr.edu

Yi-Xiong Zhou
Center for Neural Systems
NYU
zhou@cns.nyu.edu

[In addition to the above 41, the following 10 people also asked
to be included as signatories, but I received their replies too
late for the original letter. 3/27/95]

Lawrence K. Cormack
Psychology Department
University of Texas at Austin
CORMACK@PSY.UTEXAS.EDU

James Dinunzio
Computer Science Department
USC
dinunzio@scf.usc.edu

Lewis O. Harvey Jr.
Department of Psychology
University of Colorado
lharvey@clipr.colorado.edu

Ju Li
Department of Psychology
Columbia University
ta-jl134@columbia.edu

Mats Lind
Center for Human Computer Studies
Uppsala University
Sweden
mats.lind@cmd.uu.se

Walter Makous
Center for Visual Science
University of Rochester
walt@cvs.rochester.edu

David Rose
Department of Psychology
University of Surrey
UK
pss1dr@surrey.ac.uk

Stefan Treue
Div. of Neuroscience
Baylor College of Medicine
streue@bcm.tmc.edu

Christopher Tyler
Smith Kettlewell Eye Research Institute
San Francisco, CA
cwt@skivs.ski.org

Hugh R. Wilson
Visual Science Center
University of Chicago
hrw6@midway.uchicago.edu

BACKGROUND--THE VIDEOTOOLBOX:

Denis Pelli uses Macs to do vision research. A '91 paper of his described a simple electronic circuit that combined the three DAC outputs of a color video card to achieve exquisitely precise grayscale images on a grayscale monitor. A recent paper, on how people see letters, was the cover article of the June '94 issue of Nature. (Reprints are available.) A paper on how we see words is currently under review at Science. Over the years since the Mac II appeared he developed a library of C software (the VideoToolbox) for doing vision research on Macs, and since '91 the complete archive has been available to the public. It is electronically available in source form from info-mac, updated several times a year, and is free for research purposes. We estimate that one hundred vision research labs worldwide are using this software, each with many Macs and students.
ftp://sumex-aim.stanford.edu/info-mac/dev/src/video-toolbox-95-01-14-c.sit

The TimeVideo application included in the VideoToolbox checks out the timing of all video devices in anticipation of their use in critical real-time applications, e.g. movies or lookup table animation. It seems to be the only test program in existence for rigorous testing of Mac video drivers. It uncovered bugs in drivers from Dome, Radius, RasterOps, Relax, SuperMac, TrueVision, and Apple, including the Mac IIci "Macintosh II Built-In Video" (.Display_Video_Apple_RBV1), 4*8 and 8*24 "Macintosh Display Card" (.Display_Video_Apple_MDC), "Macintosh Display Card 8*24 GC" (.Display_Video_Apple_MDCGC), and Quadra 700, 900, and 950 (.Display_Video_Apple_DAFB versions 0 and 2). Version 3 of the .Display_Video_Apple_DAFB driver (e.g. in the Centris 650) is bug-free. All bugs have been reported to the manufacturers. We're happy to report that all of Apple's new drivers published in the last several years are bug free. Perhaps the driver-design teams have added TimeVideo to their testing suite.
ftp://sumex-aim.stanford.edu/info-mac/Configuration/time-video-363.sit

Pelli, D. G. and Zhang, L. (1991) Accurate control of contrast on microcomputer displays. Vision Research, 31, 1337-1350. [pdf]

Solomon, J. A. and Pelli, D. G. (1994) The visual filter mediating letter identification. Nature, 369 (6479), 395-397.

Pelli, D. G., Farell, B., and Moore, D. C. () The role of letters in recognizing words. Under review at Nature.

Burns, C. W., Pelli, D. G., Farell, B., and Moore, D. C. () Identifying letters. Submitted to Vision Research.

TYPOS:

In the "Display Driver Calls" chapter of interim-release "Designing PCI Cards & Drivers" on the MacOS SDK #2.
p. 161 "Too improve ..."
p. 163 "... 32-bit address#."
p. 163 "lumimance"
p. 163 "luminnace-"
p. 167 "The Display Manager. also ..."

Incidentally, the "Display Driver Calls" chapter would be easier to use if it were a complete document, not requiring reference to the old 68k "Designing Cards & Drivers".

ACKNOWLEDGEMENTS:

Thanks to Mike Landy for help with the timing specs.