(images via BBC) Before and after: a pesky rooftop removed and some boats on a pristine bay added with the help of this algorithm.

Speaking, as I was last week, of photo retouching, BBC news has just reported on a new tool that takes image doctoring much further than a little dodge and burn. Developed by James Hays and Alexei Efros from Carnegie Mellon University, this new algorithm replaces unwanted bits of photos with preferable scenery cherry-picked from other images.

So, you come back from your European vacation with that perfect shot of a tranquil country road winding up the English countryside…except for the front of your rented minivan poking into view. Ugh. This tool works the way a photo editor’s eye would in this scenario, trying to find an image with similar composition, lighting, and colors from which you could sample to substitute for your car. But it’s working on a massive scale, searching through a 2.3 million-image database gathered from Flickr to narrow down an image with the perfect fit.

The area of the image slated for removal is isolated. Then, to quote the BBC article, here’s where the algorithm comes in:

“We search for other scenes that share as closely as possible the same semantic scene data,” says Mr. Hays. “The algorithm then picks the closest 200 for further analysis. Next the algorithm searches the 200 to see if they have elements, such as hillsides or even buildings, the right size and colours for the hole to be filled. The useful parts of the 20 best scenes are then cropped, added to the image being edited so the best fit can be chosen.”

According to the tool’s creators, it has a 70 percent success rate of creating images with undetectable changes. Their goal is to create a community-generated library of images that can be used with this algorithm.

Stock photographers would surely benefit from this; it’s bound to be of use to photographers with a slew of almost-perfect shots that would have otherwise not been commercially viable. The developers also highlight the tool for personal use, as it offers a more sophisticated version of the old “cutting the ex-boyfriend/ex-girlfriend out of a photo.” However, this tool leaves an unblemished background—a revised memory—where an obvious excision would have been. But could this tool be put to mischievous use? I’m picturing a whole new level of tabloid “celebrity scandal” fabrications made possible by undetectable image doctoring. But that’s just too far to go even for the tabloids…or is it?

This entry was posted on Friday, August 10th, 2007 at 5:56 pm and is filed under Photography. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.