The US Defense Advanced Research Projects Agency (DARPA) has selected three teams of researchers to develop event-based (neuromorphic) infrared (IR) camera technologies.
The three teams are being led by Raytheon, BAE Systems, and Northrop Grumman. The development will be carried out under the Fast Event-based Neuromorphic Camera and Electronics (FENCE) programme.
DARPA’s FENCE programme is aimed at developing a low-latency, low-power, event-based IR focal plane array (FPA) and a new class of digital signal processing (DSP) and machine learning (ML) algorithms.
The development of these neuromorphic camera technologies will enable intelligent sensors that can handle more dynamic scenes and aid future military applications.
Event-based imaging sensors operate asynchronously and only transmit information from pixels that have changed.
This leads to much lower latency and lower power.
FENCE programme manager Whitney Mason said: “Neuromorphic refers to silicon circuits that mimic brain operation; they offer sparse output, low latency, and high energy efficiency.
“Event-based cameras operate under these same principles when dealing with sparse scenes, but currently lack advanced ‘intelligence’ to perform more difficult perception and control tasks.”
Researchers from Raytheon, BAE, and Northrop will work to develop an asynchronous read-out integrated circuit (ROIC) with low-latency and a processing layer that integrates with the ROIC to detect relevant ‘spatial and temporal signals’.
According to DARPA, the ROIC and processing layer will jointly enable an integrated FENCE sensor to operate on less than 1.5W of power.
Mason added: “The goal is to develop a ‘smart’ sensor that can intelligently reduce the amount of information that is transmitted from the camera, narrowing down the data for consideration to only the most relevant pixels.”
In February, DARPA awarded preliminary Phase I design contracts for its LongShot unmanned air vehicle (UAV) programme.