Researchers from Adobe, the Indian Institute of Technology and Stanford University teamed up to create SieveNet, an AI-powered technology that allows a user to virtually try on clothing. The “image-based virtual try-on” maps the item to the virtual body, retaining its characteristics without creating blurry or bleeding textures. According to banking company Klarna, 29 percent of shoppers want to virtually try on apparel, accessories and cosmetics and 49 percent would like solutions that take their measurements into account.

VentureBeat reports that, “SieveNet’s objective is to take an image of clothing and a body model image and generate a new image of the model wearing the clothing with the original body shape, pose, and other details preserved.”

Its “multi-stage technique …  involves warping a garment to align with the body model’s pose and shape before transferring the warped texture onto the model … [which] requires accounting for variations in shape or pose between the image of clothing, as well as occlusions in the model image (for example, long hair or crossed arms).”

SieveNet has “specialized modules” that can “predict coarse-level transformations and fine-level corrections on top of the earlier coarse transformations … [and] another module computes the rendered image and a mask atop the body model.”

The researchers trained SieveNet with a dataset of about 19,000 images of front-facing female models and upper-clothing product images, on a PC with 16GB of RAM and four Nvidia 1080Ti graphics cards. The system, they reported, “handled occlusion, variation in poses, bleeding, geometric warping, and overall quality preservation better than baselines.”

The system, they continued, also “achieved state-of-the-art results across qualitative metrics, including Fréchet Inception Distance (FID), which takes photos from both the target distribution and the system being evaluated (in this case SieveNet) and uses an AI object recognition system to capture important features and suss out similarities.”

Other, similar apps include L’Oréal’s ModiFace, which allows customers to “test different shades of lipstick on live pics and videos of themselves,” and Vue.ai’s AI system, which “susses out clothing characteristics and learns to produce realistic poses, skin colors, and other features, generating model images in every size up to 5 times faster than a traditional photo shoot.”

Gucci and Nike also have apps that allow a virtual try-on of shoes. SieveNet’s researchers, however, said their app offers “significant …  improvement[s] over the current state-of-the-art methods for image-based virtual try-on.”