Once and Future Graphics Pioneer

by B.J. Novitski

In the glitzy world of computer-generated visualizations that dominate movies and magazines today, it's easy to take for granted the photographic quality that architects are able to give their renderings of proposed buildings.

But behind the scenes, there have been have been four decades of grueling, dedicated, and inspired research to make possible these synthetic images that are indistinguishable from photographs.

One of the world's leading laboratories in this field is the Program of Computer Graphics (PCG) at Cornell University, in Ithaca, New York. Director Donald P. Greenberg has led the program since its founding in 1974. Greenberg, his staff, and students have developed the theoretical basis for many of the practical applications that architects now use routinely.

For example, research performed at Cornell's PCG led to the development of Lightscape, a rendering program capable of creating very realistic lighting effects by calculating the precise amount of light reflected from surfaces and materials within a scene.

A Unique Research Program

Greenberg was educated in architecture and engineering at Cornell and Columbia universities. As a consulting engineer with Severud Associates, an architecture and engineering firm, he was involved with the design of the St. Louis Arch, Madison Square Garden, and other projects.

During the 1960s, Greenberg became intrigued with the design potential of computers but was impatient with the numeric output provided by engineering applications. His efforts to write software to display results graphically led to the establishment of the multidisciplinary Program of Computer Graphics, with significant funding from the National Science Foundation.

Today, the research center is still funded by NSF and also by Intel, Hewlett-Packard, and Autodesk. Because the PCG is independent of any one academic department, its students and faculty enjoy an unusual opportunity for multidisciplinary research.

Most individuals are from the departments of architecture and computer science. Others have come from the fields of engineering, art, perception psychology, and theater arts. This mix results in a rich research environment that values human perception and the aesthetics of light as much as physics and precision computation.

Currently, research performed at PCG focuses on three major areas:

- improving the user interfaces for architectural applications to make them more suitable for designers; - simulating the behavior of light in space and understanding the human visual perception system to refine the rendering algorithms (computational procedures); and
- developing methods for improving image capture and the quality of image-based rendering.

For the architecture profession specifically, the PCG is concentrating on developing conceptual design tools, enabling architects to design in context, and enabling collaboration over the Internet.

Rethinking the Medium

In Greenberg's opinion, the typical user interface for architectural software, with the familiar mouse and monitor, is badly suited to the hand-eye collaboration that characterizes architectural design. He and research associate, visiting professor Moreno Piccolotto are developing a system that works with a drawing-board sized device from Toronto-based Input Technologies Inc., which functions as both sketch pad and display device.

The equipment consists of a transparent digitizing surface, a cordless pen, and a high-resolution, rear-projection display, driven by a powerful microprocessor (or several working in parallel). Using this device, designers can sketch comfortably with the pen on the large, gently tilted surface.

But unlike with conventional 2D paint software, the plane on which the sketch first appears can be rotated and navigated in three dimensions and placed into an underlying 3D scene.

The system can also enable two or more designers to work on the same sketch collaboratively on the Internet. At $55,000 per unit, this equipment is beyond the budgets of many architects, but already experiments are being conducted with $3000 Wacom tablets and hand-held devices such as the Palm.

The software used in this system is under development and is based on work done by Piccolotto and Michael Malone, now a software developer at Autodesk, while they were graduate students at Cornell. The software is unique in its ability to move smoothly between the realms of rough sketch, precision rendering, and real-time walkthroughs.

The ability to sketch naturally and create accurate architectural drawings as well as 3D models connects the art of design directly with the science of architectural evaluation and development. The PCG is collaborating on a prototype with a commercial software developer and plans call for a program to be available for public use in a few years.

Greenberg notes, "We make tools for a profession where the designers already know how to draw, think spatially, conceive of designs in three-space, and have great appreciation for light. If we are successful, the word 'computer' will disappear from our jargon and we will only be providing, in an electronic way, the opportunity for architects to express their design ideas."

Physics Meets Art

In the PCG's second major research area, Greenberg and his colleagues are studying the behavior of light to improve the algorithms used to generate 3D renderings. The well funded light measurement laboratory at the PCG is one of the best equipped in academia.

Just as acoustical engineers study sound in a theater at a variety of frequencies, researchers here study the physics of light at the wavelength level. No other architectural research lab in the world is studying light at this level of precision.

Cornell's lab, under the direction of mechanical engineering professor Kenneth Torrance, has elaborate instrumentation to measure light within physical models as it reflects from surfaces and moves through various media such as air and glass. These measurements are then compared to the simulated light calculated by existing algorithms, which are then further refined according to the real world models. As a result, the software's ability to imitate visible reality increases in precision.

However, unlike a physicist's approach to this study, the PCG lab also employs perceptual psychologists to determine how and where mathematical improvements in rendering calculations affect the communication of meaningful visual information to the human eye and brain. When the improvement in precision is not perceptible, the researchers can modify the software to maximize speed without detracting from the realism of the computer models.

Precision in light simulation, as in Lightscape, is important because it gives predictive credibility to the resulting renderings. For example, if an architect models an interior space that is supposed to be illuminated by a clerestory, a precise rendering will show whether the space does indeed receive enough light with that window configuration and orientation. If the space looks too dark, the architect using conventional renderers could simply modify the software settings to make the model look brighter.

With a physically precise simulation, the architect must adjust the window size, shape, or the position of the glazing or the color or reflectivity of the interior surfaces to improve the quality of light in the space. In other words, the problem won't be solved until the architectural elements are correctly designed.

In conventional rendering packages, light is often simulated by the "ray-tracing" method. This works best with direct lighting and highly reflective materials. Another method, "radiosity," works best with indirect lighting and diffuse surfaces.

The simulation methods currently under development at Cornell combine the advantages of both ray-tracing and radiosity to produce the most realistic possible rendition of a lighted scene. Even with today's best hardware, these computation-intensive renderings are time-consuming.

Given the pace of technology advances in hardware and software, Greenberg and his fellow researchers believe that in just a few years they will be able to generate high-resolution, precisely predictive images at a rate sufficient for real-time animation, that is, 30 frames per second.

Looking Forward to the Past

"It is fascinating to ponder the implications of exponential digital growth. In a world of infinite bandwidth, where processing, storage, and memory are essentially free, what does an architect do?

"With display devices limited only by our own visual acuity, we can create virtual worlds that are physically accurate and perceptually indistinguishable from real world scenes. Data for these predictive models, including all geometry, materials, and lighting characteristics, will be easily obtainable over the Internet, provided by manufacturers selling their wares.

"But we still need our design tools! Ones that are not restrictive and are easy to use and very comfortable. That allow me to create. In a sense, I want to go back to where I started, with pen and ink and yellow trace. Where I am free to compose in silence or to classical music or alone with nature in the outdoors. Where I am free and yet connected and able to digitally doodle and sketch. The electronic future may rapidly be approaching the potential of the environments of the past."

—Donald P. Greenberg

At the PCG, the combination of interface design, research into the physics of light, and pedagogical work demonstrates the balance with which Cornell is approaching architectural applications The PCG is improving the fit between traditional processes and new technologies.

This article will continue next week with more about this research, a peek into an undergraduate design studio, and a look back at the Cornell legacy.

B.J. Novitski is managing editor for ArchitectureWeek and author of Rendering Real and Imagined Buildings.

This article first appeared in Architectural Record, June, 2000.



In 1984, "The Cornell Box," by Cindy Goral, Kenneth Torrance, and Donald Greenberg made history as the first radiosity image. Computations for this simple environment were measured in tens of minutes.
Image: Program of Computer Graphics

In 1985, Michael Cohen's radiosity image of "The Magritte Studio" was computed using the hemi-cube algorithm. It took four hours on a VAX 11/780.
Image: Program of Computer Graphics

IN 1987, "Steel Factory," by Stuart Feldman, Michael Cohen, and John Wallace was the lab's first complex radiosity image. The model consisted of 30,000 patches. The radiosity solution was followed by a post-process radiosity gathering algorithm requiring another 190 hours of computation.
Image: Program of Computer Graphics

Moreno Piccolotto demonstrates sketching on the ITI VisionMaker PS digital drafting table. The sketch routines, conceived in 1998 by Donald Greenberg, Piccolotto, and Michael Malone, operate at interactive rates.
Photo: Program of Computer Graphics

Sketching on the digital drafting table can be superimposed on a projected radiosity image of a virtual space. The sketch's 2D plane can then be manipulated in the same 3D space as the underlying perspective.
Image: Program of Computer Graphics

The sketching routines are also being implemented on a relatively low-cost WACOM PL-400 tablet.
Photo: Program of Computer Graphics

In 1989, the "Stair Tower, Rhodes Hall, Cornell University," by Keith Howie and Ben Trumbore, was rendered using distributed ray-tracing to create soft shadows. It took one hour to compute on eight HP835 workstations.
Image: Program of Computer Graphics

In 1993, "The Green Room," by Dani Lischinski and Filippo Tampieri, was computed using discontinuity meshing so that accurate radiosity solutions could be obtained at shadow edges. Total rendering time: 42 minutes on a single HP720 workstation.
Image: Program of Computer Graphics


Once and Future Graphics Pioneer, Part II

by B.J. Novitski

The Program of Computer Graphics (PCG) at Cornell University, which recently celebrated its 25th anniversary, continues to set the highest standards for innovation in architectural design technology. Director Donald P. Greenberg has led the program since its founding in 1974.

Though perhaps better known for its work in developing algorithms for photorealistic rendering, the PCG is also bringing such sophisticated technologies to the earliest years of design education. And, as one alumnus of the program reports, the influence of Cornell's program reaches back far and stretches wide.

As described in the previous article, the PCG's research focuses on improving the designer interfaces for architectural applications and refining the rendering algorithms by simulating the behavior of light and understanding the human visual perception system.

From Real Photograph to Synthetic Image

A third area of research is in "image-based" techniques. These techniques are already familiar through currently available technologies. For example, a digital photograph of an object or material can be "texture-mapped" onto the surface of a geometric model, giving the rendering the appearance of realism without requiring much geometric complexity.

Another common application is in the animation technology pioneered by Apple Computer with the QuickTime VR format. Using QuickTime VR, several still photos taken at regular intervals for 360-degrees around a stationary viewpoint can be stitched together to create a panorama. Viewers can "look" around a 360-degree space by moving the mouse. The application is becoming popular for displaying architectural spaces on the Web.

These technologies, as currently used today, have their drawbacks. Because QuickTime VR, for instance, samples only a few images, the resulting animation may suffer from geometric distortions or blank spots, or they may fail to capture specular highlights.

To overcome these shortcomings, PCG researchers are comparing image-based renderings of scenes with physically accurate images. They are identifying the most important sources of error in image-based rendering and refining algorithms to alleviate those problems.

Technology in the Design Studio

For the past three years, select undergraduate design students at Cornell's Department of Architecture, have benefited from the program's technologically rich environment. In the fall of 1999, for example, a third-year design studio used computers to develop projects for an Indian Culture Museum in Chaco Canyon, New Mexico.

To celebrate the many facets of Anasazi, Hopi, and Pueblo cultures, the museum wanted to focus both inward on the spaces that exhibited cultural artifacts and outward to the beautiful surrounding desert. As in many other undergraduate studios, the students were asked to study issues of light, scale, and environment, but the similarity ended there.

Although these students had minimal experience with computer-aided design (CAD) before the semester began, they were able to exploit it fully to design and present their projects. They created their models of the museum using the Autodesk packages, 3D Studio Viz and Lightscape. The physically accurate rendering software informed students about such issues as whether their skylights were illuminating the exhibits at the times and in the amounts that the students intended.

With software available at the PCG, the students placed their models within a photographic, 360-degree panorama of the real site. Thus, any view from inside the model out through a window automatically displayed the correct portion of the surrounding landscape.

Also, being able to apply material textures to the surfaces in their renderings gave an aura of reality that improved the students' ability to empathize with the inhabitants in the space.

Indeed, the overall effect of the imagery gave a much clearer sense of feedback than architecture students normally experience. This allowed them to advance more quickly in their design development than is common among third-year students.

According to Moreno Piccolotto, who with Greenberg was their studio instructor, the students' success can also be attributed to the fact that computer applications are not taught as separate courses. Instead, students learn to model and render within the context of design thinking.

This provides them with the motivation to learn the applications quickly and thoroughly, and it teaches them subliminally the importance of taking a designerly approach to using the machines.

Piccolotto notes that students are also encouraged to blend traditional, manual media with their computer work whenever it is appropriate or more comfortable, such as with initial idea sketches.

He says that the sophisticated technology not only allows the students to present their work interactively, but it also enables average students to achieve a much higher level of expression than they would otherwise be able to.

During reviews throughout the semester, the students displayed their rendered projects on the PCG's nearly room-size display screen. For those sitting at an optimum distance from it, this screen gives a 120-degree panorama, making the viewer feel virtually immersed in the scene.

The triple-wide display from three projectors was particularly effective in portraying the vastness of the Chaco Canyon environment. Looking at nearly life-sized renderings made it easy for students and teachers to evaluate the designs.

Because they work with the most advanced possible technologies, Cornell students are getting an education using technologies they can expect to work with a few years from now rather than on today's technologies, many of which will be obsolete by then. Greenberg says, "We have always emphasized the teaching of concepts so that the students can ride the technological wave and never be outdated."

Looking Ahead

Indeed, PCG's work is characterized by a forward-looking philosophy. "We are creating the user-interfaces," Greenberg explains, "for tomorrow's technologies knowing with great confidence what will be available three years from now. We can demonstrate to the profession how their design tools might change over the next decade.

"Perhaps this is the most appropriate role that a research university can play, and we have been fortunate in being able to combine both the computer science and architecture professions in one interdisciplinary laboratory."

Cornell's Long Arm of Influence Early in the history of computer-aided design systems, many students trained at Cornell University went on to create software that has greatly influenced the architecture profession's adoption of technology. In addition to research work that led to the widely used Lightscape rendering software, the program's graduates have also been leaders in developing 3D modelers.

Four Cornell alumni created Wavefront, which is now sold by Silicon Graphics subsidiary, Alias/Wavefront (now called Alias, Inc.) John Pittman led the development work at Hellmuth, Obata & Kassabaum (HOK) to create a pioneering, in-house system of design and drafting software, some of which is still in use. Many of the key innovators at Autodesk, including Kevin Weiler and Carl Bass, came out of Cornell's program.

One of the leaders in computer-aided design development at Skidmore Owings & Merrill (SOM) was Nicholas Weingarten, also from Cornell. He now works at J.D. Edwards (recently merged with PeopleSoft) in Chicago on "CustomWorks," a system for configuring made-to-order and engineered-to-order products. They have modeled everything from custom windows, which can be specified in an infinite variety, to entire houses for a modular home builder.

Weingarten studied under Greenberg in the early to mid 1970s, during the early days of computer graphics research. Long before most architects had even heard of computers, Greenberg's team was borrowing state-of-the-art technology from the aerospace industry.

Although primitive by today's standards, and painfully difficult to use, these early computers were put to work generating architectural images and, more important, the underlying algorithms. The Cornell group created an animation of their campus; its publication in Scientific American in 1974 helped to popularize computer graphics for architectural applications.

Looking back at his days at Cornell, Weingarten credits Greenberg with fueling a keen curiosity about the science behind computer graphics. He also created an environment for fostering an architectural problem-solving approach and a designer's eye for the quality and usefulness of the resulting imagery.

"Greenberg is one of the world's most respected experts in this field," Weingarten asserts, "but he's also pursued a lot of things from the aesthetic side. That's why his passion for trying to render the beauty of the real world is so keen. He's combined that passion with a deep understanding of both math and physics."

During Weingarten's time at Cornell, he witnessed the evolution of animation, the use of 3D topological maps for resource analysis, the development of innovative structural analysis of inflatables and large-scale domes, and the graphing of energy flows through buildings. He remembers the PCG as "a very exciting place."

After graduation, Weingarten took his experience in structural analysis and visual simulations to SOM, where he joined Doug Stoker's computer group that created the 3D design system that later became IBM AES. This software, used extensively for production at SOM, was developed to design high-rise buildings, and it integrates the disciplines of architecture and engineering.

Although AES was never widely used in this country, it was influential in establishing a high standard for the potential of architectural software. Its creators have gone on to become the leaders in software development today.

B.J. Novitski is managing editor for ArchitectureWeek and author of Rendering Real and Imagined Buildings.

This article first appeared in Architectural Record, June, 2000




In 1994, Cornell student Jason Ardizzone proposed a renovation of a "Theater in New York City" by combining photographs of an existing building with a ray-traced model and post-production lighting effects. Image: Program of Computer Graphics

Moreno Piccolotto demonstrates to his students sketching on the ITI VIsionMaker PS digital drafting table. The sketch routines, conceived in 1998 by Donald Greenberg, Piccolotto, and Michael Malone, operate at interactive rates.
Image: Program of Computer Graphics

View of the south transept and the south façade of the Cathedral of St. John the Divine, in New York City, with a bioshelter addition designed and rendered by student Layth Madi.
Image: Program of Computer Graphics

Professor Greenberg discusses the site location for the Anasazi Museum project with student Jessica Allen. The background is the triple-screen panoramic scene in the classroom.
Image: Program of Computer Graphics

The Indian Culture Museum by Domingos Macedo Garcia demonstrates a transparency through diverse exhibition settings to the landscape beyond. A rhythmic sequence moves the visitor through these spaces both horizontally and vertically. Image: Program of Computer Graphics

In her design of the Indian Culture Museum, Theresa Seung Bak uses transparency in an exhibit of a sculpture group to illustrate the storytelling tradition in Native American cultures. Image: Program of Computer Graphics

In a recent studio, Casey Cadwallader created a conceptual model for a media center for Ithaca, New York.
Image: Program of Computer Graphics

Back in 1971, "The Arts Quad, Cornell University," was created by Donald Greenberg and his students at General Electric's Visual Simulation Laboratory as part of a movie entitled "Cornell in Perspective." Image: Program of Computer Graphics

In 1987, "The Vermeer Studio," modeled by John Wallace and Michael Cohen, was inspired by the painting "Lady and Gentleman at the Virginals." Computed on a VAX 11/780, it was the first example of the two-pass algorithm combining radiosity and ray-tracing effects.
Image: Program of Computer Graphics

In 1995, "The Pyramid at the Louvre by I. M. Pei," modeled by Sean Barry and Ben Trumbore, contained 32 million triangles. It required 18 hours on eight HP7000 workstations to compute. Image: Program of Computer Graphics