A Critical History of Computer Graphics and Animation
During the late 1980s and 1990s, virtual reality was touted as a new and emerging application that promised to revolutionize interactivity and man-computer interfaces. In fact, VR is much older than the 1980s, older or nearly as old as the entire computer graphics field itself.
In 1956, Morton Heilig began designing the first multisensory virtual
experiences. Resembling one of today's arcade machines, the Sensorama
combined projected film, audio, vibration, wind, and odors, all designed
to make the user feel as if he were actually in the film rather than simply
watching it. Patented in 1961, the Sensorama placed the viewer in a one
person theater and, for a quarter, the viewer could experience one of
five two-minute 3D full color films with ancillary sensations of motion,
sound, wind in the face and smells. The five "experiences" included
a motorcycle ride through New York, a bicycle ride, a ride on a dune buggy,
a helicopter ride over Century city in 1960 and a dance by a belly dancer.
Since real-time computer graphics were many years away, the entire experience
was prerecorded, and played back for the user.
Comeau and Bryan, employees of Philco Corporation, constructed the first actual fabricated head-mounted display in 1961. Their system, called Headsight featured a single CRT element attached to the helmet and a magnetic tracking system to determine the direction of the head. The HMD was designed to be used with a remote controlled closed circuit video system for remotely viewing dangerous situations. While these devices contributed intellectual ideas for display and virtual experiences, the computer and image generation were yet to be integrated.
The field we now know as virtual reality (VR), a highly multidisciplinary field of computing, emerged from research on three-dimensional interactive graphics and vehicle simulation in the 1960s and 1970s. Not surprisingly, the development of the discipline can be traced to early work at MIT and Utah and none other than Ivan Sutherland.
Two of the necessary foundations of VR were being addressed at MIT by Larry Roberts and Sutherland, among others. The first necessary practical contributions included the research and development that allowed the CRT to serve as an affordable and effective device on which to create a computer generated image, and the interactive interfaces that showed that a user could interact with the CRT image to accomplish some desired task.
As we mentioned in an earlier section, Roberts wrote the first algorithm to eliminate hidden or obscured surfaces from a perspective picture in 1963. His solutions to this and other related problems prompted attempts over the next decade to find faster algorithms for generating hidden surfaces. Among the important activities of Sutherland and his colleagues and students at the University of Utah were efforts to develop fast algorithms for removing hidden surfaces from 3D graphics images, a problem identified as a key computational bottleneck.
Students of the Utah program made two important contributions in this field, including an area search method by Warnock (1969) and a scan-line algorithm that was developed by Watkins (1970) and developed into a hardware system. One of the most important breakthroughs was Henri Gouraud's development of a simple scheme for continuous shading (1971). Unlike polygonal shading, in which an entire polygon (a standard surface representation) was a single level of gray, Gouraud's scheme involved interpolation between points on a surface to describe continuous shading across a single polygon, thus achieving a closer approximation of reality. The effect made a surface composed of discrete polygons appear to be continuous. This ability is essential in the process of generating the quality of visual images necessary to present a believable VR environment.
Each of these efforts provided part of the foundation for early attempts at addressing the concept of a virtual environment. The other part was the earlier work that resulted in the commercial development of the important head mounted display. The Ultimate Display was created by Sutherland in 1965. What made it so important was the fact that it had a stereoscopic display (one CRT element for each eye). The HMD had a mechanical tracking system, and later Sutherland experimented with an ultrasonic tracker. As was discussed in the National Academy of Sciences report Funding a Revoloution: Government Support for Computing Research, the HMD was the central research component of the emerging field.
Click on the images below to view a larger version (when available).
Virtual Reality —
a three dimensional, computer generated simulation in which one can navigate
around, interact with, and be immersed in another environment
Virtual Reality —
the use of computer technology to create the effect of an interactive
three-dimensional world in which the objects have a sense of spatial presence.
The actual term "Virtual Reality"
is attributed to Jaron Lanier of VPL in 1986 in a conversation regarding
the work of Scott Fisher. Fisher, of NASA Ames, had been referring to
the field as "Virtual Environments". Myron Krueger labeled the
activity "Artificial Reality" in 1983, the title of his book,
and a year later, William Gibson coined the term "Cyberspace"
in his book Neuromancer.
VR is one of those fields that Ivan Sutherland would christen "holy grails"--fields involving the synthesis of many separate, expensive, and risky lines of innovation in a future too far distant and with returns too unpredictable to justify the long-term investment.
Ellis, S. (1996). Virtual Environments
and Environmental Instruments, In Simulated and Virtual Realities, Taylor
& Francis: 1996.
"To design a virtual cockpit, we created a very wide field of vision," said Furness, who now directs the University of Washington's Human Interface Technology (HIT) Lab. "About 120 degrees of view on the horizontal as opposed to 60 degrees." In September of 1981, Furness and his team turned on the virtual-cockpit projector for the first time. "I felt like Alexander Graham Bell, demonstrating the telephone," recalled Furness. "We had no idea of the full effect of a wide-angle view display. Until then, we had been on the outside, looking at a picture. Suddenly, it was as if someone reached out and pulled us inside."
The Human Interface Technology Laboratory is a research and development lab in virtual interface technology. HITL was established in 1989 by the Washington Technology Center (WTC) to transform virtual environment concepts and early research into practical, market-driven products and processes. HITL research strengths include interface hardware, virtual environments software, and human factors.
Furness, T. 1986. The super cockpit and its human factors challenges. Proceedings of the Human Factors Society. 30, 48-52.
While multi-million dollar military systems have used head-mounted displays in the years since Sutherland's work, the notion of a personal virtual environment system as a general purpose user-computer interface was generally neglected for almost twenty years. Beginning in 1984, Michael McGreevy created the first of NASA's virtual environment workstations (also known as personal simulators and Virtual Reality systems) for use in human-computer interface research. With contractors Jim Humphries, Saim Eriskin and Joe Deardon, he designed and built the Virtual Visual Environment Display system (VIVED, pronounced "vivid"), the first low-cost, wide field-of-view, stereo, head-tracked, head-mounted display.Clones of this design, and extensions of it, are still predominant in the VR market.
Next, McGreevy configured the workstation hardware: a Digital Equipment Corporation PDP-11/40 computer, an Evans and Sutherland Picture System 2 with two 19" monitors, a Polhemus head and hand tracker, video cameras, custom video circuitry, and the VIVED system. With Amy Wu, McGreevy wrote the software for NASA's first virtual environment workstation. The first demonstrations of this Virtual Reality system at NASA were conducted by McGreevy in early 1985 for local researchers and managers, as well as visitors from universities, industry, and the military. Since that time, over two dozen technical contributors at NASA Ames have worked to develop Virtual Reality for applications including planetary terrain exploration, computational fluid dynamics, and space station telerobotics. In October 1987 Scientific American featured VIVED--a minimal system, but one which demonstrated that a cheap immersive system was possible.
Two other projects from the 80s deserve mention:
In 1978 Andy Lippman and group of researchers from MIT (including Michael Naimark and Scott Fisher) developed what is probably the first true hypermedia system. The Aspen Movie Map was a surrogate travel application that allowed the user to enjoy a simulated ride through the city of Aspen, Colorado.
The system used a set of videodisks containing photographs of all the streets of Aspen. Recording was done by means of four cameras, each pointing in a different direction, and mounted on a truck. Photo's were taken every 3 meters. The user could always continue straight ahead, back up, move left or right.
Each photo was linked to the other relevant photos for supporting these movements. In theory the system could display 30 images per second, simulating a speed of 200 mph (330 km/h). The system was artificially slowed down to at most 10 images per second, or 68 mph (110 km/h).
To make the demo more lively, the user could stop in front of some of the major buildings of Aspen and walk inside. Many buildings had also been filmed inside for the videodisk. The system used two screens, a vertical one for the video and a horizontal one that showed the street map of Aspen. The user could point to a spot on the map and jump directly to it instead of finding her way through the city.
After several other experiments, VIDEOPLACE was created. The computer had control over the relationship between the participant's image and the objects in the graphic scene, and it could coordinate the movement of a graphic object with the actions of the participant. While gravity affected the physical body, it didn't control or confine the image which could float, if needed. A series of simulations could be programmed based on any action. VIDEOPLACE offered over 50 compositions and interactions (including Critter, Individual Medley, Fractal, Finger Painting, Digital Drawing, Body Surfacing, Replay, and others).
In the installation, the participant faced a video-projection screen while the screen behind him was backlit to produce high contrast images for the camera (in front of the projection screen), allowing the computer to distinguish the participant from the background.
The participant's image was then digitized to create silhouettes which were analyzed by specialized processors. The processors could analyze the image's posture, rate of movement, and its relationship to other graphic objects in the system. They could then react to the movement of the participant and create a series of responses, either visual or auditory reactions. Two or more environments could also be linked.
In 1983 Krueger published his now-famous book Artificial Reality, which was updated in 1990.
Krueger, M.W., "Artificial Reality". Reading, Mass., Addison Wesley, 1983
One of the first instrumented gloves described in the literature was the Sayre Glove, developed by Tom Defanti and Daniel Sandin in a 1977 project for the National Endowment for the Arts. (In 1962 Uttal from IBM patented a glove for teaching touch typing, but it was not general purpose enough to be used in VR applications.) The Sayre glove used light based sensors with flexible tubes with a light source at one end and a photocell at the other. As the fingers were bent, the amount of light that hit the photocells varied, thus providing a measure of finger flexion. The glove, based on an idea by colleague Rich Sayre, was an inexpensive, lightweight glove that could monitor hand movements by measuring the metacarpophalangeal joints of the hand. It provided an effective method for multidimensional control, such as mimicking a set of sliders.
The first widely recognized device for measuring hand positions was developed
by Dr. Gary Grimes at Bell Labs. Patented in 1983, Grimes' Digital Data
Entry Glove had finger flex sensors, tactile sensors at the fingertips,
orientation sensing and wrist-positioning sensors. The positions of the
sensors themselves were changeable. It was intended for creating "alpha-numeric''
characters by examining hand positions. It was primarily designed as an
alternative to keyboards, but it also proved to be effective as a tool
for allowing non-vocal users to "finger-spell'' words using such
Sturman, D.J. and Zeltzer, D., A survey of glove-based input, Computer Graphics and Applications, IEEE , V14 #1, Jan. 1994,30 -39
GROPE-I was a 2D system for continuous force fields. GROPE II was expanded to a full six-dimensional (6D) system with three forces and three torques. The computer available for GROPE II in 1976 could produce forces in real time only for very simple world models — a table top; seven child's blocks; and the tongs of the Argonne Remote Manipulator (ARM), a large mechanical device. For real-time evaluation of molecular forces, Brooks and his team estimated that 100 times more computing power would be necessary. After building and testing the GROPE II system, the ARM was mothballed and the project was put on hold for about a decade until 1986, when VAX computers became available. GROPE III, completed in 1988, was a full 6D system. Brooks and his students then went on to build a full-molecular-force-field evaluator and, with 12 experienced biochemists, tested it in GROPE IIIB experiments in 1990. In these experiments, the users changed the structure of a drug molecule to get the best fit to an active site by manipulating up to 12 twistable bonds.
The test results on haptic visualization were extremely promising. The subjects saw the haptic display as a fast way to test many hypotheses in a short time and set up and guide batch computations. The greatest promise of the technique, however, was not in saving time but in improving situational awareness. Chemists using the method reported better comprehension of the force fields in the active site and of exactly why each particular candidate drug docked well or poorly. Based on this improved grasp of the problem, users could form new hypotheses and ideas for new candidate drugs.
The docking station is only one of the projects pursued by Brooks's group at the UNC Graphics Laboratory. The virtual world envisioned by Sutherland would enable scientists or engineers to become immersed in the world rather than simply view a mathematical abstraction through a window from outside. The UNC group has pursued this idea through the development of what Brooks calls "intelligence-amplifying systems." Virtual worlds are a subclass of intelligence-amplifying systems, which are expert systems that tie the mind in with the computer, rather than simply substitute a computer for a human.
In 1970, Brooks's laboratory was designated as an NIH Research Resource in Molecular Graphics, with the goal of developing virtual worlds of technology to help biochemists and molecular biologists visualize and understand their data and models. During the 1990s, UNC has collaborated with industry sponsors such as HP to develop new architectures incorporating 3D graphics and volume-rendering capabilities into desktop computers (HP later decided not to commercialize the technology).
Since 1985, NSF funding has enabled UNC to pursue the Pixel-Planes project, with the goal of constructing an image-generation system capable of rendering 1.8 million polygons per second and a head-mounted display system with a lagtime under 50 milliseconds. This project is connected with GROPE and a large software project for mathematical modeling of molecules, human anatomy, and architecture. It is also linked to VISTANET, in which UNC and several collaborators are testing high-speed network technology for joining a radiologist who is planning cancer therapy with a virtual world system in his clinic, a Cray supercomputer at the North Carolina Supercomputer Center, and the Pixel-Planes graphics engine in Brooks's laboratory.
With Pixel-Planes and the new generation of head-mounted displays, the UNC group has constructed a prototype system that enables the notions explored in GROPE to be transformed into a wearable virtual-world workstation. For example, instead of viewing a drug molecule through a window on a large screen, the chemist wearing a head-mounted display sits at a computer workstation with the molecule suspended in front of him in space. The chemist can pick it up, examine it from all sides, even zoom into remote interior dimensions of the molecule. Instead of an ARM gripper, the chemist wears a force-feedback exoskeleton that enables the right hand to "feel" the spring forces of the molecule being warped and shaped by the left hand.
In a similar use of this technology, a surgeon can work on a simulation of a delicate procedure to be performed remotely. A variation on and modification of the approach taken in the GROPE project is being pursued by UNC medical researcher James Chung, who is designing virtual-world interfaces for radiology. One approach is data fusion, in which a physician wearing a head-mounted display in an examination room could, for example, view a fetus by ultrasound imaging superimposed and projected in 3D by a workstation. The physician would see these data fused with the body of the patient. In related experiments with MRI and CT scan data fusion, a surgeon has been able to plan localized radiation treatment of a tumor.
The term Haptic refers to our sense of touch, and consists of input via mechano-receptors in the skin, neurons which convey information about texture, and the sense of proprioception, which interprets information about the size, weight and shape of objects objects via feedback from muscles and tendons in the hands and other limbs. Haptic Feedback refers to the way we attempt to simulate this haptic sense in our virtual environment, by assigning physical properties to the virtual objects we encounter and designing devices to relay these properties back to the user. Haptic feedback devices use vibrators, air bladders, heat/cold materials, and Titanium-Nickel alloy transducers which provide a minimal sense of touch.
In 1979 F.H Raab and others described the technology behind what has been one of the most widely utilized tracking systems in the VR world — the Polhemus. This six degrees of freedom electromagnetic position tracking was based on the application of orthogonal electromagnetic fields. Two varieties of electromagnetic position trackers have been implemented — one uses alternating current (AC) to generate the magnetic field, and the other uses direct current (DC).
In the Polhemus AC system, mutually perpendicular emitter coils sequentially generate AC magnetic fields that induce currents in the receiving sensor, which consists of three passive mutually perpendicular coils. Sensor location and orientation therefore were computed from the nine induced currents by calculating the small changes in the sensed coordinates and then updating the previous measurements.
In 1964 Bill Polhemus started Polhemus Associates, a 12-person engineering
studies company working on projects related to navigation for the U.S.
Department of Transportation and similar European and Canadian departments,
in Ann Arbor, Michigan. His research was focused on determining an object's
position and orientation in a three-dimensional space.
"The Polhemus system is used to track the orientation of the pilot's helmet," Polhemus said of the electromagnetic technology he pioneered. "The ultimate objective is to optically project an image on the visor of the pilot's helmet so he can look anywhere and have the display that he needs. ... It's critical to know, in a situation like that, where the pilot's helmet is pointed, so you know what kind of a display to put up on the visor," he added before comparing the system to a "heads-up display" or "gun sight," which projects similar data onto an aircraft's windshield.
Polhemus was supported for a few years in the early 1970s by Air Force contracts. But by late 1973, "in the absence of any equity capital to speak of, we just ran dry," in his words. "By that time, however, the device looked attractive to a number of companies, and there were several bids for it. We finally wound up selling to the Austin Company," a large conglomerate with headquarters in Cleveland, Ohio.
The next few years saw the company change hands to McDonnell Douglas Corp. of St. Louis, Mo., and then to the current owner, Kaiser Aerospace and Electronics Corp. of Foster City, Calif., in 1988.
Still convinced that there were commercial possibilities, Blood and Scully started a new company in 1986, which they called Ascension. The first few years were lean years, but Blood improved upon the head-tracking technology for fighter pilots and Scully eventually negotiated a licensing agreement with GEC (General Electric Co. of Great Britain).The contract was put on hold for two years when Polhemus, which had been purchased by Kaiser Aerospace, a direct competitor of GEC, sued Ascension for patent infringement. Polhemus dropped the case shortly before it went to trial, and Ascension, with the financial backing of GEC, was able to stay afloat. The licensing agreement was finalized with GEC, and Ascension sales of equipment based on the technology took off, particularly in the medical field.
When the virtual reality revolution erupted in the early 1990s, Ascension played a part in it, developing motion trackers that could be used in high-priced games. "We decided from the beginning that we were not going to go after a single-segment application," Blood said. "From day one, we've always made sure we were involved in a lot of different markets." As the VR market declined, this philosophy helped Ascension's sales stay constant.
A constant for Ascension has been its work in the field of animation. Scully says Ascension deserves some of the credit for inventing real-time animation, in which sensors capture the motions of performers for the instant animation of computerized characters. Ascension's Flock of Birds product has been used to capture this motion and define the animated characters. Over the years, Ascension technology has been used in the animation of characters in hundreds of television shows (MTV's CyberCindy, Donkey Kong), commercials (the Pillsbury Doughboy, the Keebler elves), video games (Legend, College Hoops Basketball, SONY's The Getaway) and movies (Starship Warriors, pre-animation for Star Wars films).
Ascension Technology serves six markets: animation, medical imaging,
biomechanics, virtual reality, simulation/training and military targeting
systems from its facility. Using DC magnetic, AC magnetic, infrared-optical,
inertial and laser technologies, Ascension provides turnkey motion capture
systems for animated entertainment as well as custom tracking solutions
for original equipment manufacturers to integrate into their products.
F. Raab, E. Blood, T. Steiner, and H. Jones, Magnetic position and orientation tracking system, IEEE Transactions on Aerospace and Electronic Systems, Vol. 15, No. 5, 1979, pp. 709-718.
LEEP Optical System started to develop wide angle lenses for 3-D still photography applications in 1975. The Large Expanse, Extra Perspective (LEEP) optical system was designed by Eric Howlett in 1979 and provides the basis for most of the current virtual reality helmets available today. The combined system gave a very wide field of view stereoscopic image. The users of the system have been impressed by the sensation of depth in the scene and the corresponding realism. The original LEEP system was redesigned for the NASA Ames Research Center in 1985 for their first virtual reality installation, the VIEW (Virtual Interactive Environment Workstation) by Scott Fisher. The system was built according to lessons learned using the LEEP display earlier, and proved to bequite impressive. It already featured many techniques that are used nowadays: a Polhemus tracker, 3D audio output, gesture recognition using VPLs DataGlove, a remote camera, and a BOOM-mounted CRT display.
In 1988 Fakespace began building a telepresence camera system for the Virtual Environment Workstation (View) project at NASA Ames Research Center. The complete system combined a teleoperated camera platform and 3D viewing system.To increase image quality, Fakespace invented the BOOM (Binocular Omni-Orientation Monitor). Very small monitors are mounted on a mechanical arm, and users look into the monitors like they would look into a pair of binoculars. Tracking occurs when the user moves the arm, which changes the perspective. When a user releases the BOOM, another person can look at the same thing from the same perspective, which is an advantage over HMDs. Since real monitors are used, the resolution is quite good.
This environment has been called a "CAVE", (CAVE Automatic Virtual Environment). The first CAVE (as well as the concept) was created by Carolina Cruz-Neira, Dan Sandin, and Tom DeFanti, along with other students and staff of EVL. Since then this back-projection method of virtual reality has gained a strong following.
The CAVE is a surround-screen, surround-sound, projection-based virtual reality (VR) system. The illusion of immersion is created by projecting 3D computer graphics into a 10'x10'x10' cube composed of display screens that completely surround the viewer. It is coupled with head and hand tracking systems to produce the correct stereo perspective and to isolate the position and orientation of a 3D input device. A sound system provides audio feedback. The viewer explores the virtual world by moving around inside the cube and grabbing objects with a three-button, wand-like device.
Unlike users of the video-arcade type of VR system, CAVE dwellers do not wear helmets to experience VR. Instead, they put on lightweight stereo glasses and walk around inside the CAVE as they interact with virtual objects. Multiple viewers often share virtual experiences and easily carry on discussions inside the CAVE, enabling researchers to exchange discoveries and ideas. One user is the active viewer, controlling the stereo projection reference point, while the rest of the users are passive viewers.
The CAVE was designed from the beginning to be a useful tool for scientific visualization; EVL's goal was to help scientists achieve discoveries faster, while matching the resolution, color and flicker-free qualities of high-end workstations. Most importantly, the CAVE can be coupled to remote data sources, supercomputers and scientific instruments via high-speed networks. It has obvious benefits: it is easy for several people to be in the room simultaneously and therefore see images together; and it is easy to mix real and virtual objects in the same environment. Also, because users see for example their own hands and feet as part of the virtual world, they get a heightened sense of being inside that world.
Various CAVE-like environments exist all over the world today. Most of these have up to four projection surfaces; images are then usually projected on three walls and the floor. Adding projection on the ceiling gives a fuller sense of being enclosed in the virtual world. Projection on all six surfaces of a room allows users to turn around and look in all directions. Thus, their perception and experience are never limited, which is necessary for full immersion. The PDC Cube at the Center for Parallel Computers at the Royal Institute of Technology in Stockholm in Sweden is the first fully immersive CAVE.
For a discussion of the current state of the art in VR
|<<Back to Section Index|