Jon Barron

I am a staff research scientist at Google Research, where I work on computer vision and machine learning.

At Google I've worked on Lens Blur, HDR+, Jump, Portrait Mode, and Glass. I did my PhD at UC Berkeley, where I was advised by Jitendra Malik and funded by the NSF GRFP. I did my bachelors at the University of Toronto.

Email  /  CV  /  Biography  /  Google Scholar  /  Social

I'm interested in computer vision, machine learning, optimization, and image processing. Much of my research is about inferring the physical world (shape, depth, motion, paint, light, colors, etc) from images. Representative papers are highlighted.

Training a tiny non-convolutional neural network to reproduce a scene using volume rendering achieves photorealistic view synthesis.

Networks can be trained to remove shadows cast on human faces and to soften harsh lighting.

Machine learning can be used to train cameras to autofocus (which is not the same problem as "depth from defocus").

We predict a volume from an input stereo pair that can be used to calculate incident lighting at any 3D point within a scene.

By rethinking metering, white balance, and tone mapping, we can take pictures in places too dark for humans to see clearly.

Variational auto-encoders can be used to disentangle a characters style from its content.

Considering the optics of dual-pixel image sensors improves monocular depth estimation techniques.

Training a neural network on light stage scans and environment maps produces an effective relighting method.

A single robust loss function is a superset of many other common robust loss functions, and allows training to automatically adapt the robustness of its own loss.

View extrapolation with multiplane images works better if you reason about disocclusions and disparity sampling frequencies.

We can learn a better denoising model by processing and unprocessing images the same way a camera does.

Frame interpolation techniques can be used to train a network that directly synthesizes linear blur kernels.

By making one camera in a stereo pair hyperspectral we can multiplex dark flash pairs in space instead of time.

Depth cues from camera motion allow for real-time occlusion effects in augmented reality applications.

Dual pixel cameras and semantic segmentation algorithms can be used for shallow depth of field effects.

This system is the basis for "Portrait Mode" on the Google Pixel 2 smartphones

Varying a camera's aperture provides a supervisory signal that can teach a neural network to do monocular depth estimation.

We train a network to predict linear kernels that denoise noisy bursts from cellphone cameras.

A reformulation of the bilateral solver can be implemented efficiently on GPUs and FPGAs.

By training a deep network in bilateral space we can learn a model for high-resolution and real-time image enhancement.

Color space can be aliased, allowing white balance models to be learned and evaluated in the frequency domain. This improves accuracy by 13-20% and speed by 250-3000x.

This technology is used by Google Pixel, Google Photos, and Google Maps.

Using computer vision and a ring of cameras, we can make video for virtual reality headsets that is both stereo and 360°.

This technology is used by Jump.

Mobile phones can take beautiful photographs in low-light or high dynamic range environments by aligning and merging a burst of images.

This technology is used by the Nexus HDR+ feature.

Our solver smooths things better than other filters and faster than other optimization algorithms, and you can backprop through it.

Standard techniques for stereo calibration don't work for cheap mobile cameras.

By integrating an edge-aware filter into a convolutional neural network we can learn an edge-detector while improving semantic segmentation.

By framing white balance as a chroma localization task we can discriminatively learn a color constancy model that beats the state-of-the-art by 40%.

The monocular depth estimates produced by fully convolutional networks can be used to inform intrinsic image estimation.

By embedding a stereo optimization problem in "bilateral-space" we can very quickly solve for an edge-aware depth map, letting us render beautiful depth-of-field effects.

This technology is used by the Google Camera "Lens Blur" feature.

We produce state-of-the-art contours, regions and object candidates, and we compute normalized-cuts eigenvectors 20× faster.

This paper subsumes our CVPR 2014 paper.

Shape, Illumination, and Reflectance from Shading
Jonathan T. Barron, Jitendra Malik
TPAMI, 2015
supplement / bibtex / keynote (or powerpoint, PDF) / video / code & data / kudos

We present SIRFS, which can estimate shape, chromatic illumination, reflectance, and shading from a single image of an masked object.

This paper subsumes our CVPR 2011, CVPR 2012, and ECCV 2012 papers.

This paper is subsumed by our journal paper.

We present a technique for efficient per-voxel linear classification, which enables accurate and fast semantic segmentation of volumetric Drosophila imagery.

Our system allows users to create textured 3D models of themselves in arbitrary poses using only a single 3D sensor.

By embedding mixtures of shapes & lights into a soft segmentation of an image, and by leveraging the output of the Kinect, we can extend SIRFS to scenes.

TPAMI Journal version: version / bibtex

Boundary cues (like occlusions and folds) can be used for shape reconstruction, which improves object recognition for humans and computers.

This paper is subsumed by SIRFS.

This paper is subsumed by SIRFS.

We present a large RGB-D dataset of indoor scenes and investigate ways to improve object detection using depth information.

This paper is subsumed by SIRFS.

A model and feature representation that allows for sub-linear coarse-to-fine semantic segmentation.

Markov Decision Problems which lie in a low-dimensional latent space can be decomposed, allowing modified RL algorithms to run orders of magnitude faster in parallel.

Using the relative motions of stars we can accurately estimate the date of origin of historical astronomical images.

We use computer vision techniques to identify and remove diffraction spikes and reflection halos in the USNO-B Catalog.

In use at Astrometry.net

Feel free to steal this website's source code, just add a link back to my website. If you'd like your new page linked to from here, submit a pull request adding yourself or send me an email with a link. ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩
Also, consider using Leonid Keselman's Jekyll fork of this page.