Dynamic Vision Sensor (DVS) - asynchronous temporal contrast silicon retina

Application Areas Advantages Functionality Temporal resolution and latency Dynamic Range System Integration Simulation

Technology Briefing

Conventional vision sensors see the world as a series of frames. Successive frames contain enormous amounts of redundant information, wasting memory access, RAM, disk space, energy, computational power and time. In addition, each frame imposes the same exposure time on every pixel, making it difficult to deal with scenes containing very dark and very bright regions.

The Dynamic Vision Sensor (DVS) solves these problems by using patented technology that works like your own retina. Instead of wastefully sending entire images at fixed frame rates, only the local pixel-level changes caused by movement in a scene are transmitted – at the time they occur. The result is a stream of events at microsecond time resolution, equivalent to or better than conventional high-speed vision sensors running at thousands of frames per second. Power, data storage and computational requirements are also drastically reduced, and sensor dynamic range is increased by orders of magnitude due to the local processing.

Application Areas

  • Surveillance and ambient sensing
  • Fast Robotics: mobile, fixed (e.g. RoboGoalie)
  • Factory automation
  • Microscopy
  • Motion analysis, e.g. human or animal motion
  • Hydrodynamics
  • Sleep research and chronobiology
  • Fluorescent imaging
  • Particle Tracking

Advantages

Conventional high-speed vision systemsDVSDVS Benefits
Requires fast PC Works with any laptop Lower costs
Lower power consumption
Extremely large data storage (often several TB)
Highly redundant data
Low storage requirements
No redundant data
Lower costs
More portable
Easier and faster data management
Custom interface cards Webcam-sized, USB2.0
Java API
More portable
Easier programming
Batch-mode acquisition
Off-line post-processing
Real-time acquisition
Extremely low latency
Continuous processing
No downtime, lower costs
Low dynamic range, ordinary sensitivity
Needs special bright lighting (lasers, strobes, etc.) for short exposure times
High sensitivity
No special lighting needed
Lower costs
Simpler data acquisition
Limited dynamic range, typically 50 dB Very high dynamic range (120 dB) Usable in more real-world situations

Functionality

The DVS functionality is achieved by having pixels that respond with precisely-timed events to temporal contrast. Movement of the scene or of an object with constant reflectance and illumination causes relative intensity change; thus the pixels are intrinsically invariant to scene illumination and directly encode scene reflectance change .

Principle of operation 

Temporal resolution and latency

The events are output asynchronously and nearly instantaneously on an Address-Event bus, so they have much higher timing precision than the frame rate of a frame-based imager. This is shown by these recording from a spinning disk painted with wedges of various contrasts. The disk spins at 17 rev/sec, and the events are painted with colored-time in the right image. Our measurements show that we can often achieve a timing precision of 1 us and a latency of 15 us with bright illumination. Because there are no frames, the events can be played back at any desired rate, as shown in the right video. The low latency is very useful for robotic systems, such as the pencil balancing robot.

Temporal resolution High speed video

Dynamic range

Because the pixels locally respond to relative change of intensity, the device has a large intra-scene dynamic range. This wide dynamic range is demonstrated by the Edmund gray scale chart, which is differentially illuminated by a ratio of 135:1 – a 42dB illumination ratio, which means a normal high-quality CCD based device like the Nikon 995 used below must either expose for the bright or dark part of the image to obtain sensible data. Most of the vision sensor pixels still respond to the 10% contrast steps in both halves of the scene. The rightmost data is captured under 3/4 moon with a high contrast scene. Under these conditions the photocurrent is <20% of the photodiode leakage current, but the low threshold mismatch still allows a good response.

DVS Wide dynamic range  DVS Wide dynamic range
DVS operating under moonlight

 

System integration

The DVS is integrated with a USB2.0 high-speed interface that plugs into any PC or laptop. The host software presently stands at >200 Java classes. The open source jAER software project lets you render events in a variety of formats, capture them, replay them, and most important, process them using events and their precise timing.

Simulation

The DVS128 is modeled in the robotic simulation software V-REP (since release 3.0.4) by Coppelia Robotics.