Nvidia debuted a deep learning method that can edit or reconstruct an image that is missing pixels or has holes via a process called “image inpainting.” The model can handle holes of “any shape, size, location or distance from image borders,” and could be integrated in photo editing software to remove undesirable imagery and replace it with a realistic digital image – instantly and with great accuracy. Previous AI-based approaches focused on rectangular regions in the image’s center and required post processing.

Nvidia notes that the project, led by Guilin Liu, enables the model to “gracefully handle holes of increasing size.” Liu’s team first created 55,116 “masks of random streaks and holes of arbitrary shapes and sizes” to train the neural network, and “also generated nearly 25,000 for testing.” Then they were “further categorized into six categories based on sizes relative to the input image, in order to improve reconstruction accuracy.”

Researchers trained the neural network by “applying the generated masks to images from the ImageNet, Places2 and CelebA-HQ datasets … using Nvidia Tesla V100 GPUs and the cuDNN-accelerated PyTorch deep learning framework.”

“To the best of our knowledge, we are the first to demonstrate the efficacy of deep learning image inpainting models on irregularly shaped holes,” said the researchers. Inpainting has thus far been limited because “the outputs for missing pixels necessarily depend on the value of the input that must be supplied to the neural network for those missing pixels,” leading to “color discrepancy and [image] blurriness” among other artifacts.

This team “developed a method that guarantees the output for missing pixels does not depend on the input value supplied for those pixels,” by ensuring that “the value of the output is independent of the values of the missing pixels in each receptive field.”

The Next Web reports that Nvidia’s “newest AI breakthrough” is a “mind-blowing computer vision technique that can ‘inpaint’ parts of an image that have been deleted or modified.” “If you’re thinking Photoshop already does this, think again,” it declared. “This is something you have to see to believe.”

In contrast to earlier techniques for filling in missing spaces in images, Nvidia’s new technique shows “no granular degradation or blurred edges which require fiddling with different brushes and levels of smoothness or opacity.” “Nvidia’s AI does, almost instantly, what could take even a seasoned graphic designer minutes or even hours to accomplish,” it concluded.