4D Camera to Improve Machine Vision for Robots and Virtual Reality

Image credit: Stanford Computational Imaging Lab and Photonic Systems Integration Laboratory at UC San Diego
Image credit: Stanford Computational Imaging Lab and Photonic Systems Integration Laboratory at UC San Diego

A new type of camera built by Stanford engineers and funded by the NSF and Intel generates a four dimensional image that is capable of capturing nearly 140 degrees of information.

The 4D camera, built by Donald Dansereau, a postdoctoral fellow in electrical engineering and Gordon Wetzstein, assistant professor of electrical engineering at Stanford, along with colleagues from the University of California, San Diego is the first single-lens, wide field of view, light field camera ever made.

With current cameras robots have to change position to get multiple perspectives of their surroundings in order to maneuver in complex environments and understand the objects within those environments.

robot vision
Image credit: Stanford Computational Imaging Lab and Photonic Systems Integration Laboratory at UC San Diego

The 4D camera could let robots collect the same information as multiple perspectives would in a single image.

The camera produces an extremely wide field of view, nearly a third of the circle around the camera, which is enabled by a specially designed spherical lens.

The engineers the faced a problem of translating a spherical image produced by the lens onto a flat sensor.

Earlier attempts to solve this problem had been heavy and error prone, but combining the optics and fabrication expertise of UCSD and the signal processing and algorithmic expertise of Wetzstein’s lab resulted in a digital solution to this problem that not only leads to the creation of these extra-wide images but enhances them.

Image credit: Stanford Computational Imaging Lab and Photonic Systems Integration Laboratory at UC San Diego

The engineers write that the main contributions of their paper [PDF] are:

• Optical design and prototype of a novel single-lenswide-FOV LF camera

• A parameterization that closely matches the properties of the camera and permits conventional LF processing

• A field flattening method that efficiently couples planar sensors and the spherical focal surface of the monocentric
lens

• Panoramic capture in realistic scenarios showing depth estimation, post-capture refocus, and LF resolution enhancement for panoramas with 15 × 15 × 1600 × 200 pixels (72 MPix) and a 138° FOV

“This work presents the first steps towards practical, compact, wide-FOV LF capture.”

The engineers see their new camera significantly improving the vision of autonomous vehicles such as cars and drones and aiding augmented and virtual reality technologies.

The next step for the engineers is to create a compact multi-sensor rig to to achieve contiguous image capture.

Facebook Comments