Computer Graphics

Over the past 10 years an explosion of activity has occurred in computer graphics. Far more than making pretty pictures, computer graphics is one part of computer science with tremendous potential for general use. For example, one of the earliest computer graphics research problems solved in the 1970s was representing and printing text on a television‑like screen. Now we take computer terminals and word processing for granted. And even though it may seem like science fiction now, in the not‑so-distant future computers will be able to make color animated pictures as easily as text. We’ll take this for granted too, expecting even the smallest computer to have this capability.

Computer graphics as a field began when Ivan Sutherland (MS 1960) combined the new concept of computer data structures with vector line drawing capability in his MIT PhD thesis in 1963. Activity in graphics started at Caltech only a few years ago, but the group is already establishing an international reputation.

Al Barr, Jim Blinn, and Jim Kajiya form the core of the Caltech research group in computer graphics, which is probably the most mathematically sophisticated computer graphics group in the country. The group is developing fundamental mathematical approaches for computationally simulated physical objects. “It’s like Gottingen in the 1920s in physics,” says Kajiya, assistant professor of computer science. “Exciting new ideas are surfacing almost daily.”

Although the three represent a wide diversity of interests, their basic research creates and uses similar mathematics. They feel that the future direction of the Caltech computer graphics group is to lay a new mathematical foundation for computer graphics. “The graphics field is maturing,” says Barr, who is also assistant professor of computer science. “The next really big advances in computer graphics are going to involve a new level of mathematical knowledge and skill.”

Barr’s work in graphics is dedicated to creating a unified mathematical formalism for representing the shape and the behavior of objects. “Whether the task is making new computer images, creating the next generation of CAD/CAM systems, or studying form and function in biology, the idea is to work out the mathematics, which will be the language to express the new ideas,” says Barr.

Kajiya is currently working toward connecting computer graphics principles to the basic equations of electromagnetism that govern the behavior of light. “Every time we apply a deeper understanding of the physics of light,” says Kajiya, “a startling advance in the quality of computer‑generated images is made. The graphics field is progressing in quantum jumps; each year the new images make last year’s look primitive.”

Of the three, Blinn has been involved in computer graphics the longest. He’s been a major influence in the field (a “giant,” according to everyone who knows his research) over the past 15 years. He’s made significant contributions to a number of areas, the most important, perhaps, being his work in surface modeling and texture mapping. Currently Blinn is mainly interested in the animation of physics, the uses of animation in education, and exploring how the computer can best be used to produce quality animation. “Animation is a venture where computer graphics is especially relevant,” says Blinn, “because it can reproduce unchanging parts and mechanize the movement of the parts that do change.”

Blinn’s interest goes back to his undergraduate days at the University of Michigan in 1967, shortly after the birth of computer graphics. After his 1978 PhD in computer science from the University of Utah, Blinn was recruited to Caltech as a half‑time research fellow (later lecturer) by Ivan Sutherland, who was the Fletcher Jones Professor of Computer Science.

But his primary interest was in the space program, and Blinn spent the other half of his time at the Jet Propulsion Laboratory producing animated simulations of the Voyager missions to Jupiter and Saturn. His simulations, which always have to be specifically pointed out as simulations and not the real thing, won him NASA’s Exceptional Service Medal in 1983 as well as a prominent national reputation in computer graphics. His research contributions were also recognized with the first Computer Graphics Achievement Award last year from the ACM SIGGRAPH organization (the special interest group in graphics of the Association for Computing Machinery). He has also produced computer graphics effects for several education programs, such as the PBS series “Cosmos”—notably the sequences on evolution and on the DNA double helix.

Kajiya was recruited by Sutherland in 1979. He also had a Utah PhD, but his fields of interest were very high‑level programming languages, theoretical computer science, and signal processing. His interest in computer graphics began in 1981, after he presented a paper at the national SIGGRAPH conference on different ways of manipulating pixels (individual picture elements) to get a sharper image in the display of characters on CRT screens.

While Kajiya was in the process of becoming an enthusiastic convert, interest in computer graphics was growing rapidly on campus. Blinn’s introductory course was suddenly swamped. Interest was exploding nationally as well, perhaps because of the wider availability of computers. Kajiya met Al Barr in the summer of 1983, when they were both speakers at the SIGGRAPH seminar on the state of the art in computer graphics. At that time Barr was senior research scientist at Raster Technologies, Inc. and was finishing his thesis at Rensselaer Polytechnic Institute. They hit it off quite well—so well, in fact, that Barr, who was about to accept a faculty position at MIT, ended up last January at Caltech instead. He’s also one of the four Caltech recipients of IBM Faculty Development Awards—two-year renewable $30,000 grants to outstanding young faculty.

Barr had been here before—with Professor of Biology Charles Brokaw, to study the biophysics of the flagellar tails of microorganisms, to see how they move in order to swim. The purpose of Barr’s PhD thesis in mathematics was to shed some light on a problem in theoretical evolutionary biology, developing generic mathematical modeling tools in the process.

“Microscopic observations suggest that speed could be an evolutionary selection criterion that maintains the form of swimming sperm cells,” Barr says. “I needed to model the shape of realistic cells as a function of time, to mathematically set up and solve the differential equations of the swimming motion of these shapes in a viscous fluid, and then race the different sperm cell designs to find optimal shapes that maximize the net forward speed. The numerical experiments provided evidence that speed is an important selection criterion affecting the evolution of sperm cell shapes.”

Barr realized that the easiest and most convincing way to show that his mathematics worked and that it was correctly implemented on a computer was to make computer graphics movies of hypothetical sperm cells swimming. “Making plots and charts of the various swimming speeds was not enough,” he says. “Graphics made a visual confirmation possible, to verify that the swimming simulation observed on the computer screen matched the real swimming under the microscope.”

This work developed into a general interest in shapes—how to represent mathematically shapes that deform (bend, twist) over time. The same mathematics can be applied to biophysics, to CAD/CAM (computer‑aided design/computer‑aided manufacture), and to robotics. Barr can generate a whole family of round and square solid shapes he calls superquadrics with “just a few sines, cosines, and exponents,” and put a hole through the middle by adding a constant to one of the cosine terms.

Fractals are mathematical shapes generated out of a process of constrained randomness; they’re good for modeling a wide variety of natural phenomena, such as trees, mountains, and clouds. Kajiya’s most recent work is on clouds, simulating how light scatters in an inhomogeneous medium. Coauthor with Kajiya on the cloud paper was graduate student Brian Von Herzen. He made the first computer graphics movie of the time evolution of clouds, rendered with Kajiya’s algorithms, using a simplified meteorological model. Currently he is finishing his master’s thesis with Barr on new representation of curved surfaces.

Kajiya is also interested in anisotropic reflection, that is, reflection from surfaces such as cloth, hair, or fur. What Kajiya calls the “fuzzy object problem,” an open problem proposed by Blinn seven years ago, is still unsolved. Barr’s geometric modeling of shape, together with Kajiya’s sophisticated models of light, has the potential to set new standards in the state of the art in computer graphics realism. They are working on the mathematical methods used for the simulation of hair, fire, fabric, and splashing water, as well as simulating the shapes and appearance of plants and animals. “Computer graphics is currently able to make pictures of only a small fraction of the kinds of natural objects we encounter every day,” Kajiya says. Realism is one of the major goals of the Caltech group.

Realism has evolved in stages over the history of computer graphics in the last 15 years. It was a problem just to define and make pictures of surfaces in the first place. First, all the hidden lines had to go, so that the lines defining the front surface obscured those in back. “This took about five years,” says Kajiya. “Solving light reflections was at first simply a matter of picking up a freshman physics book and using Lambert’s cosine law.” New methods were then invented to simulate specular reflection. Highlights and shading eventually yielded realistic goblets and teapots, but all computer graphics objects still looked as though they were made of plastic. A new model, incorporating Fresnel’s laws of reflection from conductors, renders different surface properties—different kinds of metals and so on. (Blinn was involved in the development of many of the reflection models.) Eventually a sophisticated, timeconsuming set of algorithms called ray tracing incorporated refraction, multiple reflections, and shadows. Ray tracing basically follows the path of each photon as it bounces off surfaces. In the past year researchers at Lucasfilm have made startling advances by applying Monte Carlo techniques to graphics.

Kajiya’s favorite observation about the field was made by graduate student John Platt, who came up with the dictum: “Computer graphics is just applied electricity and magnetism.”

But that’s not all it is. Computer graphics also offers a new medium to artists. This has generated some interesting interactions at Caltech between artists and scientists. Vibeke Sorensen is currently a visiting associate at Caltech, active in the computer graphics group. She is also on the faculty of the School of Film and Video at the California Institute of the Arts, where she teaches computer animation. For the past two years, she has led a tutorial at the SIGGRAPH conference on computer art and design, and has won a number of awards for computer animation. Sorensen is also interested in participating in Caltech’s computer music group as well, to create music to accompany her digital images. This group, led by Carver Mead, the Gordon and Betty Moore Professor of Computer Science, is translating the underlying physics and differential equations of musical instruments into VLSI circuits—but that’s another story.

Blinn has also been interested in bringing the art world and the computer world together and in 1982 started a course at Pasadena’s Art Center College of Design. A year later this evolved into a projects course for students from both institutions. Caltech and Art Center have a reciprocal exchange program, but Blinn’s course was a unique cooperation, combining the particular complementary talents of each: Art Center students mostly designed the artistic objectives, and the Caltech students came up with the software to implement them.

Blinn isn’t teaching the introductory Caltech‑Art Center course this year, but Sorensen and Bob Schaff (BS 1984) are continuing it. Instead of teaching just a few students, Blinn is currently devoting most of his time to reaching hundreds of thousands through computer graphics. He is creating animated computer graphics sequences for “The Mechanical Universe,” Caltech’s TV course in basic physics, developed in cooperation with the Corporation for Community College Television and funded by the Annenberg/CPB Project. Blinn’s creations, which include such scenes as animated equations and physics experiments, orbiting planets, and an ion going through a particle accelerator, will take up approximately five minutes of each of the 60 half‑hour programs. The series is scheduled for release in the fall of 1985.

This year the SIGGRAPH conference sponsored the world’s first computer-generated Omnimax film, where very large-format images were projected inside an eight-story hemispherical dome. Out of 19 contributions produced nationally, 4 were from Caltech. Barr animated a giant school of graceful sperm cells swimming toward a looming, undulating egg cell, using a rendering program written for the purpose by an RPI colleague. JPL’s Jeff Goldsmith, with software by Kajiya and Blinn, contributed a fly‑by of Saturn. The third was Sorensen’s sequence based on the constellations, with software written by Blinn. Kajiya, with computer science grad students Tim Kay and Brian Von Herzen, together with help from the Art Center students, contributed a 30second animation of a flight into a space colony (which just happened to house the Caltech campus).

“Every frame of the space colony picture represents a state‑ of‑the‑art advance in ray tracing. Several new algorithms and techniques were invented just to make this movie possible,” says Kajiya. “It’s one of the most complex ray‑tracing movies ever made.” It was also one of the most expensive computer graphics movies ever made.

Cray Research in Mendota Heights, Minnesota, donated the computer time for three of the four sequences. It was winter when the Caltech researchers did the calculations (“One day it was nearly 40 degrees below zero,” Barr recalls.) But despite the climatic and logistic difficulties in Minnesota, the Cray was absolutely essential because it’s 60 times faster than the VAX 780. The projects could not have been done at Caltech.

Barr and Kajiya need a surprising amount of computer time—last year the group used about 500 CPU hours from Cray Research just to do the Omnimax sequences. They are in the process of creating an industrial consortium to help support the laboratory and obtain access to fast computers to continue their work. They’re investigating the design of specialized hardware to perform large‑scale computation in graphics, particularly looking at methods suited for VLSI technology. A number of industrial firms have shown a strong interest in their work.

Since graphics is the natural interface between computers and people, eventually it will become the dominant mode of interaction. Therein lies the motivation to make graphics as real as the technology allows.