Adobe’s research team is working on a visual effects tool, codenamed Cloak, for easy and economical removal of rigs, power lines and other unwanted parts of an image. The tool is similar to Photoshop’s content-aware fill feature that lets the user select and delete unwanted elements in the image, with “intelligent” software filling in the missing background behind them. Doing the same thing with video, however, is more challenging, which is why Cloak is still in an experimental stage, with no release date slated.

Engadget, which spoke with Adobe research engineer Geoffrey Oxholm and product manager for video graphics/VFX Victoria Nece, reports that Cloak would have uses for “every video producer,” and will also be “indispensable for VR, AR, and other types of new video tech.”

“With 360 degree video, the removal of objects, the crew and the camera rig becomes virtually mandatory,” said Nece.

Removing objects from photos is challenging “because the computer has to figure out what was behind the deleted object based on the pixels around it,” and video makes that task even more challenging, “because you have to track any moving objects you want to erase,” and “the fill has to look the same from frame to frame or it will be a glitchy mess.”

“It’s a fascinating problem,” said Oxholm. “Everything is moving, so even if you nail one frame, you have to be consistent. The saving grace is that we can see behind the thing we want to remove. If you’ve got a microphone to remove, you can see behind the microphone.”

One fact that has made this easier to do is the improvements surrounding motion tracking. “We can do really dense tracking, using parts of the scene as they become visible,” said Oxholm. “That gives you something you can use to fill in.” A demonstration of Cloak showed an automatic, one-process removal of erasing cars from a freeway, deleting a pole in front of a cathedral and, likewise, a hiking couple from a cave scene.

But the process is not perfect: “shadow traces are visible on the cave floor, and the cathedral is blurred in spots where the pole used to be.” Still, the Engadget reviewer opines that, “even at this early stage, though, the tool could do much of the grunt-work, making it easier for a human user to do the final touch-ups.”

Other recent experimental Adobe technology includes last year’s VoCo, which lets the user Photoshop voiceovers or podcasts, and Smartpic, “which eventually became part of Adobe’s Experience Manager.”