The Stanford team is applying the latest in advanced electronics including pixel-level processing, as well as optics and image science to design advanced imaging chips. The industry partners will assist in design and prototyping, and will fabricate chips for test purposes.
Current electronic imaging systems, called charge-coupled devices or CCDs, produce analog signals that are digitized converted to strings of 1's and 0's by a special purpose chip. The new imaging system, by contrast, integrates digitization with the image-capture process by moving it to the pixel level. Stanford has taken out four patents on different aspects of pixel-level processing.
"Pixel-level processing provides a number of potential advantages," says Abbas El Gamal, associate professor of electrical engineering, who is the project's principal investigator. Those advantages include a dynamic range large enough to capture details of objects in bright sunlight and deep shade at the same time; reduction in noise; and pixel-level programmability, which could aid in processes like automatic image recognition.
In addition, the new imaging chip is made from CMOS, the same technology used to make low-power computer chips. This allows engineers to combine the imaging sensors with computer circuitry, reducing the chip count and cutting production costs. Also, because the pixels in CMOS imaging arrays are "read out" in parallel, row by row, they are much faster than CCD arrays that read out pixels sequentially. A number of companies are actively developing analog CMOS imaging chips.
El Gamal sees a range of applications for the programmable imaging chip that extend beyond basic photography, applications in which digital imaging has more intrinsic advantages. One example might be a recognition system for automatic teller machines. The imaging chip could be programmed to capture just those aspects of an individual's face that provide positive identification, he says.
Industry interest
The range of companies that are supporting the Stanford effort suggests how widespread its impact may be, Gibbons says. Hewlett-Packard sees digital image capture as a natural complement to their printer business. Kodak recognizes that it will have a significant impact on its digital and conventional photo operations. Intel envisions it as one of the next large users of silicon. Interval is interested in video applications.
Last July, HP's chief executive, Lewis Platt, told Business Week magazine that the company is betting that digital imagery "will fundamentally change the way people think about photography" and that this will allow the electronics company to carve a chunk out of the $40 billion per year photography market. The potential for producing substantially improved digital cameras was what attracted the company to the Stanford project, says Jim Duley, strategic analysis manager for HP Labs. "We are looking at this as a potentially revolutionary way to capture images, and we expect to supply the world with the printers to print them," he says.
"Stanford's approach is intriguing," says Les Moore, manager of digital camera advanced development at Eastman Kodak. "They are attacking the problem of designing a single-chip digital camera from a very broad systems perspective. They have one group actively designing silicon chips and another group modeling the entire imaging system. That's unique."
David E. Liddle, president and CEO of Interval Research, says, "We believe that video will play a key role in the development of intelligent systems in the coming decades. To that end, we are conducting research designed to improve the capability to process video data in real time as it is captured. This dovetails with one of the goals of the Stanford Digital Camera Project, which is to develop architectures and algorithms for silicon implementation that combine image capture and image processing."
Intel Corp. recently created a new business division specifically focused on digital imaging and video. "Intel believes that there is a strong future in digital imaging and video, and we offer a variety of technologies and products in this area," says Ray Hirt, who oversees Intel's participation in the Stanford project. "Our objective is to provide people with easy-to-use and affordable methods to capture, enhance, store and deliver high-quality digital imagery. El Gamal's research is exploring an area that the industry needs to understand better to move forward on this front."
Started in 1993
The origins of the programmable digital imaging chip date back to 1993, when El Gamal and graduate student Boyd Fowler, now a research associate at Stanford, were looking for something interesting to do with CMOS imaging sensors. After considering a number of different projects, they hit on the idea of moving the process of digitizing the electrical signals produced by the image sensors (a process called "analog to digital conversion," or ADC) onto the chip itself. Currently, the signals from the thousands or millions of individual sensors on the chip are collected into a single stream before they are digitized.
"At first this seemed like a crazy idea because ADC chips are very complex and require a large number of transistors," says El Gamal. "But we persisted and figured out a way to do it using very few transistors per pixel."
After completing a prototype chip, however, the researchers thought they had reached a dead end. So they turned their attention to learning more about CMOS imaging sensors themselves. Much research has been done on these devices, but almost all of it behind closed doors in industry laboratories. The scientists could find very little information about the sensors in the technical literature, so they decided to duplicate all of the commercial sensors on a single chip. This would allow them to easily compare their physical and electrical characteristics.
About two years ago, industry interest in image sensors surged. "When corporate engineers began searching the web looking for people doing research on image sensors they found us. So we started to get a number of unsolicited inquiries," El Gamal says.
Soon these inquiries were followed by offers of funding. Several companies, including Intel, Hewlett-Packard, Rockwell and Analog Devices, gave the researchers substantial grants in return for use of their image sensor test set. This allowed the researchers to support their work for several years without having to apply for government funding.
Industry interest also spurred them to revisit the issue of pixel-level digitization. Joined by graduate student David Yang, they came up with a new architecture that promised a number of advantages. "We realized that we could get enhanced dynamic range and could also get control of all the parameters, instead of being limited to the few adjustments you can make with CCD arrays," El Gamal says.
El Gamal met Brian Wandell, professor of psychology, and Joseph Goodman, the William E. Ayer Professor of Electrical Engineering, who are key participants in the new effort, through Stanford's new Image Systems Engineering program.
"The thing that excites me about this project," says Wandell, "is that it could lead to a future where cameras are everywhere, like pencils and notepads are today. Imagine giving your child a small camera, worn as a button on her shirt, so she can send or save images of her school day. Or imagine making computer-readable notes of important documents or business presentations by pointing and clicking a device the size of a penlight."
Wandell is an expert on human vision. Over the last five years, with funding from Hewlett-Packard, he has developed a set of metrics that measure how accurately a digital image will appear to the naked eye. By simulating the imaging system and the display system, he can predict the quality of the pictures that an imaging chip with certain characteristics will produce without having to build the chip. He has been using this capability to advise El Gamal and his students on the design of their prototype chips.
At the same time, Goodman, who is an expert on optics, and several of his students have been investigating ways to use the chip's programmability to compensate for the imperfections caused by optical lenses.
With their assistance, El Gamal's group has finished and begun testing the second version of a third-generation chip. The 1.2 million transistor integrated circuit has pixel-level digitization that can be programmed to improve its performance in different environments, such as outdoor scenes, indoor scenes, pages of text or graphics displayed on computer screens.
With support from Stanford's Center for Integrated Systems, El Gamal and his students set up a new sensor characterization laboratory in the CIS annex. The project caught the eye of Gibbons when he approved the establishment of the laboratory in his previous position as dean. When he moved to his current position as special counsel for industry relations, he volunteered his services to help line up significant industry support. "This project has exactly the potential that I was looking for," he says.
By David Salisbury