A Critical History of Computer Graphics and Animation

Section 4:
Basic and applied research moves the industry


The late 1950s and the decade of the 1960s saw significant development in computer graphics-related computing, displays, and input and output hardware. The nature of the computer at this point in history was that it allowed programs to be written to accomplish different functions. But in the early days of computer graphics, users purchased their hardware, and the burden of developing these reusable programs, often including the development of the underlying algorithms for creating images, fell on the shoulders of the individual user.

Researchers at universities and laboratories around the world began to investigate techniques that could harness the power of the computer and its display and interaction devices to solve problems and provide capabilities that may have been outside of the reach of certain users because of the complexity of this programming process. A software industry was spawned, and turnkey systems were marketed that buffered the individual user from the computer instructions necessary to use the system to its potential.

Click on the images below to view a larger version (when available).

 


 

 

MIT

Some of the early work in algorithm and software development took place at MIT and at Harvard University just up the Charles River.

Tom Stockham was at Lincoln Labs on a project to use computers to process photographic material. His technique was to use a facsimile machine to "digitize" a photograph so that each element of the picture (what would later come to be known as the pixel) could be represented as a number in memory that represented a shade of gray. Once he had this image in digital form, he manipulated properties such as dynamic range and contrast, and then re-photographed the image from the screen of a CRT. Hence, what is now common in software packages such as Photoshop, that is, image processing, was developed.

A professor in the Mechanical Engineering Department at MIT during the 1950s and 1960s, Steven Coons, had a vision of interactive computer graphics as a powerful design tool. During World War II, he worked on the design of aircraft surfaces, developing the mathematics to describe generalized "surface patches." At MIT's Electronic Systems Laboratory he investigated the mathematical formulation for these patches, and in 1967 published one of the most significant contributions to the area of geometric design, a treatise which has become known as "The Little Red Book". His "Coons Patch" was a formulation that presented the notation, mathematical foundation, and intuitive interpretation of an idea that would ultimately become the foundation for surface descriptions that are commonly used today, such as b-spline surfaces, NURB surfaces, etc. His technique for describing a surface was to construct it out of collections of adjacent patches, which had continuity constraints that would allow surfaces to have curvature which was expected by the designer. Each patch was defined by four boundary curves, and a set of "blending functions" that defined how the interior was constructed out of interpolated values of the boundaries.

Two of Coons' students were Ivan Sutherland and Lawrence Roberts, both of whom went on to make numerous contributions to computer graphics and (in Roberts' case) to computer networks.

Lawrence Roberts wrote the first algorithm to eliminate hidden or obscured surfaces from a perspective picture (Ref: Roberts, L G., "Machine Perception of Three Dimensional Solids," MIT Lincoln Lab. Rep., TR 315, May 1963. ). In 1965, Roberts implemented a homogeneous coordinate scheme for transformations and perspective. (Ref: Roberts, Lawrence G. 1965. Homogenous Matrix Representation and Manipulation of N-Dimensional Constructs, MS-1505. MIT Lincoln Laboratory, Lexington, Mass) His solutions to these problems prompted attempts over the next decade to find faster algorithms for generating hidden surfaces, many of which were investigated at the University of Utah (see below).

Harvard

After leaving MIT, Ivan Sutherland went briefly to ARPA, and was then recruited by Harvard. There he engaged in studies to produce pictures with dynamic perspective. The following account of these research activities at Harvard is from FUNDING  A REVOLUTION : Government Support for Computing Research , published by the National Research Council in 1999.

    In 1966, Ivan Sutherland moved from ARPA to Harvard University as an associate professor in applied mathematics. At ARPA, Sutherland had helped implement J.C.R. Licklider's vision of human-computer interaction, and he returned to academe to pursue his own efforts to extend human capabilities. Sutherland and a student, Robert Sproull, turned the "remote reality" vision systems of the Bell Helicopter project into VR by replacing the camera with computer-generated images. The first such computer environment was no more than a wire-frame room with the cardinal directions--north, south, east, and west--initialed on the walls. The viewer could "enter" the room by way of the "west" door and turn to look out windows in the other three directions. What was then called the head-mounted display later became known as VR.
     Sutherland's experiments built on the network of personal and professional contacts he had developed at MIT and ARPA. Funding for Sutherland's project came from a variety of military, academic, and industry sources. The Central Intelligence Agency provided $80,000, and additional funding was provided by ARPA, the Office of Naval Research, and Bell Laboratories. Equipment was provided by Bell Helicopter. A PDP-1 computer was provided by the Air Force and an ultrasonic head-position acoustic sensor was provided by MIT Lincoln Laboratory, also under an ARPA contract.
     Sutherland outlined a number of forms of interactive graphics that later became popular, including augmented reality, in which synthetic, computer-generated images are superimposed on a realistic image of a scene. He used this form of VR in attempting a practical medical application of the head-mounted display. The first published research project deploying the 3D display addressed problems of representing hemodynamic flow in models of prosthetic heart valves. The idea was to generate the results of calculations involving physical laws of fluid mechanics and a variety of numerical analysis techniques to generate a synthetic object that one could walk toward and move into or around (Greenfield et al., 1971).
     As Sutherland later recalled, there was clearly no chance of immediately realizing his initial vision for the head-mounted display. Still, he viewed the project as an important "attention focuser" that "defined a set of problems that motivated people for a number of years." Even though VR was impossible at the time, it provided "a reason to go forward and push the technology as hard as you could. Spin-offs from that kind of pursuit are its greatest value."
     ...Several of the original Harvard group also helped form [Evans and Sutherland], including Charles Seitz, who joined the Utah faculty in 1970 and remained until 1973, when he moved to California Institute of Technology and founded Myricom with Dan Cohen, another Harvard alumnus who contributed to the head-mounted display. The interaction between research on basic problems and development of hardware and software for military projects at Evans & Sutherland was an important feature of work at Utah.

Sutherland's group also developed the first clipping algorithm, eliminating any part of a synthetic environment that was outside the "camera's" field of view, making it less computationally intensive to generate a scene on the screen.

William Newman was also at Harvard during this period. He was interested in the construction of command languages for interactive computer use. His ideas on command language programming have been very important in the evolution of the human-computer graphical interface. Newman was co-author with Robert Sproull of Carnegie Mellon on one of the most influential textbooks in the area of computer graphics, Principles of Interactive Computer Graphics , published by McGraw-Hill in 1973. Newman went on to continue his HCI work at Xerox PARC.

Coons, Steven A. 1967. Surfaces for Computer-aided Design of Space Forms , Project MAC Report MAC-TR-41. Massachusetts Institute of Technology, Cambridge, Mass. (abstract)(full PDF)


Graphic representation of the Coons' patch. The curvature on the interior is controlled by the geometry at the edges and the corners, with the depicted vectors providing the controls.

 

 

 

 

 

 

 

David Rogers wrote a short article on the Roberts hidden surface approach which can be found at

http://web.usna.navy.mil/~dfr/roberts.html

 

Also see Sutherland, I.E., Sproull, R.F., Schumaker, R., A Characterization of Ten Hidden Surface Algorithms, ACM Computing Surveys, 6(1), 1974, pp. 1-55

 

 

 

 

 

 

 


Sutherland's HMD

 

Retrospectives: The Early Years in Computer Graphics at MIT, Lincoln Lab and Harvard
Siggraph '89 Panel Proceedings

 


 

Bell Labs

Bell Telephone Laboratories in Murray Hill, N.J. was a leading research contributor from its founding in 1925, and contributed research in computer graphics, computer animation and electronic music starting in the early 1960s. Initially, researchers were interested in what the computer could be made to do in areas such as speech and communications technology, but the results of the visual work produced by the computer during this period have established people like Michael Noll and Ken Knowlton as pioneering computer artists as well as scientists. . (See The Digital Computer as a Creative Medium, by Michael Noll, IEEE CG&A, 1967)

Physicist Edward Zajac produced one of the first computer generated films in history while at Bell Labs. His work was first publicized in 1963. The animation demonstrated that a satellite could be stabilized to always have a side facing the earth as it orbited. This film was titled A two gyro gravity gradient altitude control system. The composite shown here uses a block to represent the satellite, with each frame showing the positioning relative to the earth.

At about the same time Ken Knowlton and Leon Harmon experimented with human pattern perception and art by perfecting a technique that scanned, fragmented and reconstructed a picture using patterns of dots (such as symbols or printer characters.) Their Reclining Nude (a representation of dancer Deborah Hay) was submitted to one of the earliest computer art exhibitions, The Machine as Seen at the End of the Mechanical Age, curated by K.G. Pontus Hulten, at the Museum of Modern Art in 1968.

Ken Knowlton developed the Beflix (Bell Flicks) animation system in 1963, which was used to produce dozens of artistic films by himself and artists such as Stan VanDerBeek and Lillian Schwartz.

Ruth Weiss created in 1964 (published in 1966) some of the first algorithms for converting equations of surfaces to orthographic views on an output device. Her paper (Ref: Weiss, Ruth E. BE VISION, a Package of IBM 7090 FORTRAN Programs to Drive Views of Combinations of Plane and Quadric Surfaces. Journal of the ACM 13(4) April 1966, p. 194-204.) was selected to be included in  a 1998 compilation by SIGGRAPH of the seminal papers in computer graphics.

The artistic/scientific/educational image making efforts at Bell Labs were some of the first to show that electronic digital processing (using the IBM 7094 computer) could be coupled with electronic film recording (using the Stromberg-Carlson 4020 microfilm recorder) to make exciting, high resolution images. With the dozen or so films made between 1963 and 1967, and the many more films after that, they also showed that computer animation was a viable activity. Zajac's work, Frank Sinden's films (eg, Force, Mass and Motion) and studies by Noll in the area of stereo pairs (eg, Simulated basilar membrane motion) were some of the earliest contributions to what is now known as scientific visualization.

Noll and other Bell Labs researchers contributed some of the earliest computer artwork in the discipline, such as his Gaussian-Quadratic, to the first formal exhibition of computer art in the United States in 1965. The exhibition was called "Computer-Generated Pictures" and was located at the Howard Wise Galleries in New York. Bela Julesz, also from Bell Labs, participated in the exhibition as well, showing his work in random dot stereograms. Later that year, Noll's work from the Wise exhibition was shown at the Fall Joint Computer Conference (FJCC) of the American Federation of Information Processing Societies (AFIPS) in Las Vegas.

Some of Michael Noll's early artwork revolved around an attempt to represent existing fine art on the computer. For example, one of his early computer generated images was a rendition of "Op-artist" Bridget Riley's painting Currents, which Noll mimiced using a set of displaced sine waves. He also "duplicated" Mondrian's Composition with Lines, using visual representations generated with "random" numbers. The circular image was presented, along with a copy of the original, to a group of scientists at Bell Labs as a perception test. (The subjects actually preferred the computer generated version, which they also tagged as the most original.)

Other early "computer artists" (in addition to Noll, Knowlton, Schwartz, VanderBeek, Zajac and Harmon) working at or visiting Bell Labs were Manfred Schroeder, Laurie Spiegel, and Frank Sinden.

 


Excerpt from "The Artist and the Computer"
showing Bell Labs computers

 

Also see: Computing Science Technical Report No. 99 : A History of Computing Research at Bell Laboratories (1937-1975) by Bernard D. Holbrook  and W. Stanley Brown


Composite of frames from Zajac film


Cover of book The Machine as Seen at the End of the Mechanical Age, by K.G. Pontus Hulten. The book has a metal cover.


Knowlton and Harmon


Noll's Mondrian experiment

Noll, Michael. The Beginnings of
Computer Art in the United States: A Memoir

Leonardo V27, #1


Vanderbeek and Schwartz

 

 

LLNL

Pioneering work in software for computer graphics and animation, mostly from an applications perspective, took place at Lawrence Livermore National Laboratories from the early 1950s. Their interest in this technology was related to weapons research and areas such as particle dynamics and heat/fluid flow. They contributed immensely to the evolution of "big" computing, or what is now called supercomputing. George Michael, who started at the Lab in 1953 has put together an interesting oral history (although its coverage is much broader than CGI at LLNL) web page devoted to the history of the lab. It can be accessed at

http://www.computer-history.info

Steve Levine wrote a paper for SIGGRAPH 75 describing the graphics activities there during that period.

In another part of LLNL, Nelson Max worked as a researcher in computer graphics and animation. Max later also became a professor at the University of California, Davis/Livermore. His research interests focused on realism in nature images, molecular graphics, computer animation, and 3D scientific visualization. He served as computer graphics director for the Fujitsu pavilions at Expo 85 and 90 in Japan.

Nelson received his BA in math from Johns Hopkins, and his PhD in mathematics (topology) from Harvard University in 1967. He was previously on the faculty at CMU and Case Western Reserve University in Cleveland (1976) and joined LLNL in 1977. His 1977 CG film Nelson Max, Turning a Sphere Inside Out (International Film Bureau, Chicago, 1977) is one of the classic early films in the discipline. (See Max's remembrances on the making of the film from the Annals of the History of Computing, and read his SIGGRAPH 75 paper here).At LLNL he also produced a series of molecular structure animations that have served to show the role of CGI in scientific visualization. The most famous of these are DNA with Ethidium and Doxorubicin/DNA. He was also instrumental in the success of the IMAX movie The Magic Egg shown at SIGGRAPH 84 in Minneapolis.

 


Nelson Max Movies (coming soon)

 


       
Light Rays
 

University of Utah

The University of Utah established one of the pioneer, and certainly the most influential computer graphics programs in the country when they asked David Evans (who joined Utah in 1965) to establish a program that could advance the state of the art in this new field in 1968. The computer science department had received a large ARPA grant ($5M/yr for 3 yrs) which resulted in the work of many faculty and graduate students who have pushed the CGI discipline to where it is today. In the words of Robert Rivlin in his book The Algorithmic Image: Graphic Visions of the Computer Age, "Almost every influential person in the modern computer-graphics community either passed through the University of Utah or came into contact with it in some way"

Evans joined with Ivan Sutherland, who developed Sketchpad at MIT and later served in a position at the Department of Defense, to create an environment in which new problems in the discipline were proposed, and in which creative solutions were found. They later founded the Evans and Sutherland Computer Company to develop and market CAD/CAM, design, molecular modeling and flight simulators.

Some of the most important algorithms and theoretical results to evolve from the research in the Utah CG group include:

  • Hidden surface (Romney, Warnock, Watkins)
  • scan line coherence (Watkins)
  • Rendering (Crow, Blinn, Newell, Catmull, Clark, etal)
  • z-buffer (Catmull)
  • Patch rendering (Catmull)
  • Texture mapping (Catmull, Blinn, Newell)
  • Shadows (Crow)
  • Antialiasing (Crow)
  • Shading (Phong, Gouraud)
  • Lighting (Phong, Blinn)
  • Atmospheric effects (Blinn)
  • Environment mapping (Blinn, Newell)
  • Blobby surfaces (Blinn)
  • Facial animation (Parke)
  • Procedural modeling (Newell)
  • Splines (Riesenfeld, Lyche, Cohen)
  • Beta-splines (Barsky)

and many others. Many of these algorithms have resulted in the generation of significant hardware implementation, including LDS-1, the SGI Geometry Engine, the Head Mounted Display, the modern frame buffer, flight simulators, etc.

One of the early "motion capture" systems was developed at Utah. Called the Twinklebox, the system used a collection of LEDs, which could be illuminated in rapid succession under computer control, and an elaborate scanning mechanism, consisting of a spinning disk with narrow radial slits and lenses. The scanning discs, positioned in the corners of the room, allowed the system to create a series of one dimensional scans of the environment, including the lights. Using a mathematical process, these scans were combined to determine 3D positions for each of the lights.

The early facilities at Utah included two PDP-10 computers, one of which used the TENEX multi-access system, while the other remained a single access computer. The head mounted display developed by Sutherland at Harvard was interfaced to the single access computer, as was the Twinklebox. A hardware implementation of the Watkins algorithm was used for display, and the LDS-1, developed by E&S was also connected to the PDP-10. This entire environment was connected to the new ARPA network.

A well known contribution of the Utah group was the Utah Raster Toolkit, developed by Spencer Thomas, Rod Bogart and John Peterson. The Utah Raster Toolkit was a set of programs for manipulating and composing raster images. The tools are based on the Unix concepts of pipes and filters, and operate on images in much the same way as the standard Unix tools operate on textual data. The Toolkit used a special run length encoding (RLE) format for storing images and interfacing between the various programs. This reduced the disk space requirements for picture storage and provided a standard header containing descriptive information about an image.

Individuals who were involved in the Utah program have established many leading companies in the graphics industry, including E&S, Silicon Graphics, Adobe, Ashlar, Atari, Pixel Planes, Netscape, Pixar, etc.

 


Halftone Animation - Catmull and Parke
(no sound)

 



Utah shading algorithms


Fred Parke

 


  Digitizing Sutherland's VW



Jim Blinn - bump mapping


Phong shading on transparent glass


Texture maps and reflection maps from Ed Catmull's dissertation

 

 

 

The Utah program is now embodied as the Geometric Design and Computation group, under the supervision of Rich Riesenfeld and Elaine Cohen. The GDC is engaged in both fundamental and applied research in developing methods for representing, specifying, manipulating, and visualizing geometric models. The group has projects ranging from early conceptual design methods to innovative manufacturing processes and from detail modeling applications to large-scale assembly systems. Supporting these applications is fundamental work on surface and model representation, computational geometry, topology, differential geometry, and numerical method.

One of the early premiere GDC efforts was the Alpha_1 modeling environment. Based on GDC project research results, the Alpha_1 system is an advanced research software base, supporting use and research in geometric modeling, high-quality graphics, curve and surface representations and algorithms, engineering design, analysis, visualization, process planning, and computer-integrated manufacturing.

Solid models in Alpha_1 are represented by trimmed B-spline (NURBS) sculptured-surface boundary representations. That is, the surfaces of a solid are represented explicitly, and linked together by shared edges. It is implemented in C++, and provides both command-language and graphical, menu-driven interfaces. Much of the heart of the geometry is based on the pioneering spline work of Riesenfeld (Syracuse University) and the subdivision work of Riesenfeld, Cohen and others.

More about Alpha_1 can be found at

http://www.cs.utah.edu/gdc/projects/alpha1/

 


Cloth draped over a teapot (GDC)

 


Images from Alpha_1

 

Ohio State University

Charles Csuri, an artist at The Ohio State University, started experimenting with the application of computer graphics to art in 1963. (See Computers and Art, by Charles Csuri and James Shaffer, AFIPS Conference Proceedings, V33, FJCC, 1968). His efforts resulted in a prominent CG research laboratory that received funding from the National Science Foundation and other government and private agencies. The work at OSU revolved around animation languages, complex modeling environments, user-centric interfaces, human and creature motion descriptions, and other areas of interest to the discipline.

A complete history of the Ohio State program can be found at

http://design.osu.edu/carlson/history/ACCAD-overview/overview.html

 


Dawn of an Epoch - OSU CGRG/ACCAD

 

 


Sine Curve Man - Charles Csuri

Jet Propulsion Lab (JPL)

Bob Holzman established the JPL CG Lab at the Jet Propulsion Lab in 1977. Working with Ivan Sutherland, who had moved from University of Utah to Cal Tech, he envisioned a group with technology expertise for the purpose of visualizing data being returned from NASA missions. Sutherland recommended a graduate student at Utah named Jim Blinn, whose name has become synonymous with JPL and with graphics in general. (Sutherland once alledgedly commented that "There are about a dozen great computer graphics people, and Jim Blinn is six of them.")

Blinn received his bachelor's degree in physics and communications science from the University of Michigan in 1970, before computer science was offered as a college subject. He went on to earn a master's degree in engineering at Michigan and a Ph.D. in computer science at the University of Utah in 1978.

Blinn had worked with various imaging techniques while at Utah, and had the vision to develop them into a viable system for the visualization task that Holzman outlined. Blinn produced a series of "fly-by" simulations, including the Voyager, Pioneer and Galileo spacecraft fly-bys of Jupiter, Saturn and their moons. Next, Blinn developed CG sequences for a Annenberg/CPB series, The Mechanical Universe, which consisted of over 500 scenes for 52 half hour programs describing physics and mathematics concepts for college students. He worked with Carl Sagan on the PBS Cosmos series.

Due to the overwhelming reception of the images produced for The Mechanical Universe, Blinn began production of another series devoted to advanced mathematical concepts. Originally titled Mathematica, the title had to be changed because of a software program called Mathematica for mathematics visualization. The series is now called Project Mathematics!

Blinn wrote a series for IEEE Computer Graphics and Applications (for which he received the IEEE Service Award) and is the author of many influential papers, including

Texture and Reflection In Computer Generated Images,
CACM, 19(10), October 1976, pp 542-547.
(The original teapot paper. Introduces environment mapping.)
Models of Light Reflection for Computer Synthesized Pictures,

SIGGRAPH 77, pp 192-198.
(Introduces the Torrance-Sparrow highlight model.)
Simulation of Wrinkled Surfaces,
SIGGRAPH 78, pp 286-292.
(Introduces Bump Mapping.)
A Generalization of Algebraic Surface Drawing,
ACM Transactions on Graphics, 1(3), July 1982, pp 235-256.
(Introduces Blobby Modeling.)
Light Reflection Functions for the Simulation of Clouds and Dusty Surfaces,
SIGGRAPH 82, pp 21-29.
(Lighting model for rings of Saturn.)

Blinn left JPL for Cal Tech, and later Microsoft, where he is involved with the Direct3D project. He received the SIGGRAPH Computer Graphics Achievement Award in 1983, the NASA Exceptional Service Medal and the prestigious MacArthur Foundation Fellowship in 1991, and the Coons award from ACM-SIGGRAPH in 1999.

 


Galileo fly-by

Mimas fly-by

Rings of Saturn

Pioneer 11 fly-by

Blobby Man
 

Pythagorean formula

The Story of Pi

Mechanical Universe 1

Mechanical Universe 2
 

National Research Council of Canada

National Research Council of Canada scientist Nestor Burtnyk started Canada's first substantive computer graphics research project in the 1960s. Marceli Wein, who joined this same project in 1966, had been exposed to the potential of computer imaging while studying at McGill University. He teamed up with Burtnyk to pursue the promising field of applying evolving computer techniques to animation.

The Division of Radio and Electrical Engineering's Data Systems Group wanted to develop ways to make computers easier to use, and it settled on computer animation as the application to pursue after Burtnyk returned from a 1969 conference and heard an animator from Disney studios talk about how cartoons are made. In the traditional process, a head animator draws the key cels or pictures that demonstrate the actions. Assistants then draw the fill in pictures that carry the image from one key picture to the next.

The work of the artist's assistant seemed like the ideal demonstration vehicle for computer animation. Within a year, Burtnyk had programmed a complete "key frame animation" package that allowed the creation of animated sequences by providing only the key frames. The National Film Board in Montreal was contacted, and a project to allow artists to experiment with computer animation was started.

The first experimental film involving freehand drawings, called Metadata, was made by artist and animator Peter Foldes. This led to a more substantial collaboration on a 10-minute feature called Hunger/La Faim about world hunger and about rich and poor countries.

It took Foldes and his NRC partners a year and a half to make, and in 1974 it became the first computer-animated movie to be nominated for an Academy Award as best short. It received other honors, including the Prix du Jury at the Cannes Film Festival and other international film awards.

(The above text was taken in part from a press release from the NRC announcing that Wein and Burtnyk were recognized as the "Fathers of Computer Animation Technology in Canada by the Festival of Computer Animation in Toronto.)

In 1996, Burtnyk and Wein received an Academy Award for Technical Achievement for their key-framing animation work.

 


Metadata Movie

Interactive Skeleton Techniques for Enhancing Motion Dynamics in Key Frame Animation by Nestor Burtnyk and Marceli Wein in Communications of the ACM , October 1976, volume 19 #10, pp. 564-569


Hunger Part1 
   

 

 

 

Of course, the above discussions are not by any stretch of the imagination reflective of all of the research activities taking place in the U.S. or abroad during this early period. Work in geometric modeling was being conducted at the Cambridge University CAD lab, with Robin Forrest and others; Appel and his team of researchers at IBM were developing software and hardware that would not only contribute to the future CAD industry, but which affected the CG discipline at large. For example, he contributed to the rendering literature with his research into hidden line solutions; automotive and aerospace companies , such as Ford, Renault in France (Bezier), Boeing (Ferguson), McDonnel Douglas, and Lockheed Georgia (Chasen) were investigating software solutions for corraling the power of the digital computer for design activities; Giloi and Encarnacao were developing 3D programming systems, such as PRADIS, and investigating issues of rendering in Berlin beginning in 1965 (See an accounting of this activity in an article on the Giloi school).

The early 70s saw an increase in the funding for basic research in computer graphics and interactive processes. As a result, universities around the world established research programs to tackle many of these new projects.

 
   

 


Next: University research activities grow rapidly
<<Previous
Next>>
<<Back to Section Index