High encoding-sensitivity vision sensor with complementary nonlinear neuromorphic computing

Building a neuromorphic vision sensor capable of signal processing and spike generation is essential for developing hardware tailored to brain-inspired spiking neural networks. A critical challenge, however, is the constrained adaptive sensitivity when dealing with expanding ranges of light intensity. Here we show a neuromorphic vision sensor built on a one-transistor-one-memristor pixel structure, attaining high encoding-sensitivity over a broad intensity range by fusing complementary superlinear and sublinear encoding. Specifically, the superlinear intensity-to-spike firing is based on the plasmonic volatile Ag/hBN/Au memristor, which has the high time-to-first-spike (TTFS)- and rate- encoding sensitivity in high-brightness. And the sublinear firing behavior is based on the MoS2 synaptic photodetector and volatile Ag/hBN/Au memristor neuron, which has the high TTFS- and rate- encoding sensitivity in dim light. When deployed in polar environments (intense brightness to darkness), the complementary vision sensor achieves high-quality imaging, segmenting ice/land regions and predicting thickness—showcasing its robustness under challenging conditions. Neuromorphic vision sensors lack adaptive sensitivity across wide light intensities. Yang et al. report a complementary vision sensor that fuses superlinear and sublinear light-to-spike encoding in bright and dim environments, respectively, thereby enabling high robustness under challenging conditions.