The process of manually colorizing an image is usually cumbersome, time consuming and requires significant skill, which makes it a prime target for automation. Researchers at UC Berkeley led by Alexei A. Efros, Professor of Electrical Engineering and Computer Sciences, have released a paper that outlines a new method for using deep neural networks to help aid in the colorization of images.
Aided by these deep neural networks almost anyone, even those completely lacking any artistic ability, can easily create convincing colorized images.
The group’s paper, entitled “Real-Time User Guided Colorization with Learned Deep Priors,” will be presented at SIGGRAPH 2017, a conference which highlights computer graphics research.
The researchers had previously trained a deep neural network on a million images to colorize images without any human input. They then set up a company around their work called Algorithmia. You can try out their Algroithmia neural network here.
This is an example using their previous automatic algorithm without user input
Algorithmia is capable of producing good colorizations, such as the image above I tested it on, but it also often struggles with complex images. The goal of their previous project was to “just get a single, plausible colorization,” says researcher Richard Zhang, quoted by Phys.org “If the user didn’t like the result, or wanted to change something, they were out of luck. We realized that empowering the user and adding them in the loop was actually a necessary component for obtaining desirable results.”
Prof. Efros and his team’s new deep neural network is trained on grayscale images and simulated user inputs. The new work allows the user, in real-time, to easily correct and change the colorization. The user gives their deep neural network guidance by providing it with hints about an object or areas color that the deep neural network can then apply to the rest of the image.
The team’s deep neural network is able to learn commonly used colors for different objects and recommend actions to the user.
The results are impressive. Users without any experience in image colorization were able to colorize images –within one minute– that fooled real human judges as to whether they were original color images or not. The user study they conducted gave 28 users just a 2 minute explanation of how use the system beforehand.
Here’s a few examples that were colorized by novice users within one minute per image