Research

A Sense of Collaboration

Dr. Terry Boult, Computer Science

Dr. Terry Boult, Computer Science - Funded by Office of Naval Research, Dr. Boult is collaborating with Columbia University researching/developing novel sensors that address the development of technologies for adaptive dynamic range imaging and programmable imaging. The fundamental questions started with how we can increase the dynamic range of sensors, i.e. the ability to see very bright and very dark things at the same time. While the human eye can see four orders of magnitude of brightness, modern CCD/CMOS cameras can only see two or three. The project mixes research in optics, imaging, control, and computer vision in some very novel ways.

The core innovations center around using a computer controlled reflective digital spatial light modulator (SLM) and re-imaging it with a video sensor. The initial modulator designed used a digital micro-mirror device (DMD) from Texas Instruments, the core component of commonly used digital light processing (DLP) projectors. The DMD is a micro-machine with millions of tiny mirrors that flip ± 10 degrees in four nanoseconds. The optical system was redesigned so that if the mirror is "on" (at -10 degrees), it reflects light into the camera and when the mirror was at "off" (at +10 degrees) the camera sees nothing (black). If the mirror was on half of the camera's frame time, then the measured image was only half the scene brightness. By controlling the time "on", the measured brightness can be "optically" modulated and, by combining the control signal and measured value, produce a much larger dynamic range. With millions of mirrors in the DMD, the overall spatial modulation can adapt some parts of the image to moderately bright lighting, while others see in the deep shadows, and yet other parts see direct reflections of light sources.

Because of the complexity of the DMD control signals combined with TI's lack of suitable development tools or documentation/support for non-projector applications, the group built the prototype by reverse engineering it from a DLP projector.

Once they developed the control algorithms and demonstrated the adaptive dynamic range abilities, they began looking at other applications of the spatial light modulation concept. Since the mirror effectively provides an optical "point-wise multiply" of the incident scene energy with the control signal, they realized they could use them for optical computations requiring convolution or dot-products with the input image. If the SLM has sufficient resolution, it can have multiple modulations per camera pixel, and perform sub-pixel optical filtering. They demonstrated the use of the SLM approach for real-time edge detection. By replicating an input image on a LCD, and then re-imaging it with the Programmable Imaging System, they were also about to optically implement the "product" computation of a PCA-based face detector, and developed a simple real-time face detection system.

The project's first paper, "Programmable Imaging using a Digital Micromirror Array" by S. Nayar and V. Branzoi (Columbia University) and T. Boult (UCCS) won the Best paper award at the 2004 IEEE Conference on Computer Vision and Pattern Recognition. The paper beat out over 870 other submissions to win the top national honor in computer vision.

Dr. Boult has a wide range of other research interests, ranging from his recently completed DARPA funded studies on biometrics and statistical analysis, to his NISSC funded work on networking security, to joint projects supporting Dr. Chip Benight's work on trauma treatment.