The U.S. Defense Advanced Research Projects Agency (DARPA) has launched a program that will ultimately develop multi-purpose imaging sensors with "smarter pixels" that behave like many types of eyes working all at the same time.
DARPA's new Reconfigurable Imaging (ReImagine) program envisions millions of sensors (each about the size of a red blood cell) combined in an array about as large as a human thumbnail.
Each of these sensors will have over 1,000 integrated transistors, which provide each and every pixel with a tiny reprogrammable brain of its own.
"What we are aiming for is a single, multi-talented camera sensor that can detect visual scenes as still and video imagers do, but that also can adapt and change their personality and effectively morph into the type of imager that provides the most useful information for a given situation," said Jay Lewis, ReImagine program manager.
This could mean selecting among different thermal (infrared) emissions or different resolutions or frame rates, or even collecting 3-D LIDAR data for mapping and other jobs that increase situational awareness.
This camera with thousands upon thousands of eyes will ultimately rely on machine learning to autonomously take notice of what's happening in its field of view, and reconfigure the imaging sensor based on the context of the situation.
The future sensor Lewis has in mind will be able to perform many of these functions simultaneously because different patches of the sensor's carpet of pixels could be reconfigured by way of software to work in different imaging modes.
That same reconfigurability should enable the same sensor to toggle between different sensor modes from one lightning-quick frame to the next. No single camera can do that now.
A primary driver here is the shrinking size and cost of militarily important platforms. With multi-functional sensors like the ones that will come out of a successful ReImagine program, these smaller and cheaper platforms would provide a degree of situational awareness that today can only come from suites of single-purpose sensors fitted onto larger airborne, ground, space-based, and naval vehicles and platforms. More extensive situational awareness means more informed decision-making.
DARPA posted a Special Notice (DARPA-SN-16-68) on FedBizOpps.gov with instructions for those that might want to attend a Proposers Day on September 30 in Arlington, Virginia as a step toward possibly participating in the ReImagine program.
One key feature of the ReImagine program is that teams will be asked to develop software-configurable applications based on a common digital circuit and software platform.