Flexible computational photodetectors for self-powered activity sensing
Project Overview:
Conventional vision-based systems, such as cameras, have demonstrated their enormous versatility in sensing human activities and developing interactive environments. However, these systems have long been criticized for incurring privacy, power, and latency issues due to their underlying structure of pixel-wise analog signal acquisition, computation, and communication. In this research, we overcome these limitations by introducing in-sensor analog computation through the distribution of interconnected photodetectors in space, having a weighted responsivity, to create what we call a computational photodetector. Computational photodetectors can be used to extract mid-level vision features as a single continuous analog signal measured via a two-pin connection. We develop computational photodetectors using thin and flexible low-noise organic photodiode arrays coupled with a self-powered wireless system to demonstrate a set of designs that capture position, orientation, direction, speed, and identification information, in a range of applications from explicit interactions on everyday surfaces to implicit activity detection.
Our principles and design strategies：
a Computational photodetector with a pattern design produces a motion-incited output photocurrent, which corresponds to the convolution between the signed heterogeneous responsivity distribution and the incident optical power along the time axis. b The equivalent circuit of a typical computational photodetector without or with one photodiode blocked, and the corresponding current output (mean and standard deviation) versus the number of photodiodes connected in parallel with a single photodiode blocked at any given time. c Design space of a computational photodetector, which connects the target information, to relevant design parameters, and finally the pattern design. In this work, we demonstrate and evaluate the linear, crossing, and grid patterns as representatives.
Pattern 1: linear pattern
a Design and prototype of a computational photodetector with a linear pattern. b Simulation of responses from a linear pattern with alternating-polarity photodiodes and an interspace of 4 cm when an object hovers over with varying object sizes or sensing distances. c Experiment setup of technical evaluation using a linear actuator and laser-cut wooden bars with variable widths for light blocking. d Sensor output in various conditions of ambient light, sensing distance, object size, and object speed. e Speed detection result for the above conditions.
Pattern 2: crossing pattern
a Design of a computational photodetector with a crossing pattern. b A prototype with an interspace of 1 cm and overall size of 8 × 8 cm. c Simulation of the computational photodetector responding to an object of 10 and 20 cm width moving in the horizontal and vertical direction at a height of 0.25 and 0.5 mm. d Sensor output from all the motion events with varying object speed, under 250 lux illumination. Under 500 and 750 lux illumination, the signal has a similar envelope with slightly larger magnitudes. The object size is fixed at 16 cm and the sensing range is 0.25 cm for all conditions. e Speed detection result for the various conditions of ambient light and object speed.
Pattern 3: crossing pattern
a Design of a computational photodetector with a crossing pattern. b A prototype with an interspace of 1 cm and overall size of 8 × 8 cm. c Simulation of the computational photodetector responding to an object of 10 and 20 cm width moving in the horizontal and vertical direction at a height of 0.25 and 0.5 mm. d Sensor output from all the motion events with varying object speed, under 250 lux illumination. Under 500 and 750 lux illumination, the signal has a similar envelope with slightly larger magnitudes. The object size is fixed at 16 cm and the sensing range is 0.25 cm for all conditions. e Speed detection result for the various conditions of ambient light and object speed.
System and applications
a Schematics of the self-powered wireless sensing system compatible with various computational photodetector designs. b The wireless sensing circuit and a rechargeable battery can hide behind a solar cell. c Power consumption and harvesting characteristics: 100 Hertz sensor measurement frequency and 160 millisecond wireless communication interval for all applications can be powered solely by the solar cell under 250 lux. d Music player: a linear pattern can sense the finger swipe to switch track or adjust the volume. e Board game and buttons: a grid pattern can sense the location of a chess piece or a finger touch. f Braille: a combination of linear and grid patterns can be used to encode braille alphabets from finger motion. g Package monitoring: a grid pattern deployed on a package can sense its orientation and monitor its handling. h Inventory management: a crossing pattern deployed on a shelf rack can sense the motion direction of an item.
