Inhalt und Ziel des Forschungsprojekts
Unser Projektziel ist die Entwicklung eines visuellen Wahrnehmungssystems mit extrem geringem Stromverbrauch (ULP) für batteriebetriebene oder energieautarke Anwendungen. Um dieses Ziel zu erreichen, werden wir eine Architektur für “Deep Neural Networks” (DNN) entwickeln, welche für ULP-Anwendungen optimiert ist. Diese Architektur wird in Form einer dedizierten integrierten Schaltung (ASIC) implementiert, welche eine Schnittstelle zu Bildsensoren mit sehr geringem Verbrauch besitzt. Die finalen Demonstratoren werden Gesichtserkennung, Ortung von Leuten sowie Menschen-Zählungen durchführen können indem sie den entwickelten DNN Chip mit ULP Bildsensoren kombinieren.
Wissenschaftlicher und sozialer Kontext des Forschungsprojekts
VIPS hat das offenkundige technologische Ziel, ULP Hardware für ML in die Schweizer Elektronikindustrie zu bringen. Gleichzeitig werden damit grundlegende Ansätze von Tieren erforscht, zwei lebenswichtige aber gegensätzliche Ziele zu verfolgen: Kostbare Energie zu sparen und gleichzeitig ständig auf der Hut zu sein, um schnell auf die Umgebung reagieren zu können. Im Kontext der heutigen Welt, in der wir zunehmend von intelligenten vernetzten Sensoren umgeben sind, wird VIPS intelligente Bildverarbeitung mit geringem Stromverbrauch in batteriebetriebenen Geräten ermöglichen. Die Fähigkeit, die Batterielebensdauer auf lange Zeiträume von Monaten oder sogar Jahren zu erstrecken wird viele nützliche Anwendungen ermöglichen, die derzeit noch nicht realisierbar sind.
There is increasing opportunity for pervasive sensing in internet of things (IoT) applications. Nearly all modern developments of advanced visual perception are based on deep convolutional neural networks (CNNs), reaching close to human accuracy or even surpassing it in specific tasks, but by consuming many orders of magnitude more energy than biology. Current state of the art (SOA) real time vision systems are mainly aiming at high-throughput high-sample rate applications in autonomous driving, manufacturing, quality control, etc. These systems combine conventional image sensors and processors, which are mainly high performance graphics processing units (GPUs), adapted from gaming. Because of the large power consumption tied to such systems, there are currently no integrated solutions for ultra-low power visual perception using deep convolutional neural networks on the market. If these were available, they would open a large number of application areas that are currently not possible. Employing a sub-mW vision sensor with a mW-range deep CNN perception processor would enable always-on object detection and localization in small battery-powered devices, allowing intelligent systems to be easily set up and run for extended periods of time in environments without access to power supply or a dedicated network infrastructure. The goal of the VIPS project is to develop an ultra-low power visual perception system for battery powered or fully self-sustainable applications with visual scene analysis and decision making ability; its intelligent sensing will be extremely parsimonious in waking up expensive post processing or communication.
This VIPS system will allow applications that have previously not been possible without setting up costly power supply and communication infrastructures. It will make enable reliable people counting for building automation, allow for active advertisement interaction and eye contact analysis, and help to increase security and safety in public transportation platforms in train stations and pedestrian crossings without infringing privacy by sending sensitive data to the cloud (everything is processed onboard the device). In homes, it could detect falling and fallen elderly people, and could enable smarter robot vacuum cleaners that avoid rooms with people in them, and cables, paper, and clothing on the floor. For context-aware surveillance systems such as automatic door systems, to save valuable heating energy, it could more intelligently detect the intent of the pedestrians. On mobile devices to assist blind people, it would enable devices like smart canes.
In order to successfully accomplish this challenging project, a highly qualified consortium has been established between the Institute of Neuroinformatics (INI) at UZH/ETH, a leader in event sensors and efficiently accelerating deep neural network inference, and CSEM, contributing its expertise in the design of highly integrated chips and vision sensors at ultralow power consumption for battery powered and autonomous applications.
Last updated:23.04.2022
