How does AI virtual try-on work?

You upload two photos and twenty seconds later you're wearing a coat you've never touched. Nothing is pasted, cut out or overlaid — the image is generated from scratch. Here's what the system is doing in those twenty seconds, without the research-paper vocabulary.

Last updated 16 August 2026 · 8 min read

What it is not: cut and paste

The intuitive guess is that the app cuts the coat out of the product photo and drops it over your torso, scaled to fit. That approach was tried, and it looks exactly as bad as you'd expect: the lighting doesn't match, the sleeves end in mid-air, the shoulders are the wrong width, and any pose that isn't perfectly frontal falls apart.

What actually happens is that the model paints a new photograph. Every pixel of the garment in the output is generated, not copied. That is why the folds fall correctly around your arms and the shadow under the collar sits where your lighting says it should.

The four stages

1. Garment understandingThe model reads the product image and separates the item from everything around it — hanger, mannequin, studio background, and the model wearing it. It builds an internal description covering shape, colour, texture, print scale, closure type and how the fabric behaves.
2. Person understandingYour photo is analysed for a pose skeleton (where shoulders, elbows, hips and knees sit), body proportions, camera angle, and the direction, colour and hardness of the light on you.
3. Conditioned generationThe two descriptions are fed into an image generator as conditions: draw this person, in this pose, wearing this garment, under this light. The output region covering the clothing is synthesised fresh, with folds, drape, occlusion where your arm crosses your body, and perspective consistent with the camera.
4. Identity preservationYour face, hair, hands and background are carried through unchanged, or reconstructed to match closely. This is what separates a try-on from "a picture of somebody in that coat" — and it is the stage most likely to be visibly imperfect.

Why diffusion models made this work

Try-on research goes back well over a decade; the results were unusable until recently. Two things changed.

First, diffusion models. Instead of trying to warp a garment image into place with a geometric transform, a diffusion model starts from noise and repeatedly refines it toward an image consistent with its conditions. Physically plausible drape emerges from the refinement rather than having to be modelled explicitly — the model has seen enough photographs of fabric on bodies to know what wool does at an elbow.

Second, multi-image conditioning. Recent image models take several reference images at once and hold each one's identity. That is precisely the try-on problem: keep the person from image A, keep the garment from image B, produce a single coherent scene. A general-purpose model with that capability does try-on well without a purpose-built try-on architecture — which is why the field moved so quickly in 2024–2026.

The parts that are genuinely hard

• Hands and fingers. The long-standing weak spot of image generation, and clothing try-on puts hands right next to the region being redrawn.
• Occlusion. When an arm crosses the body, the model must decide what the garment does behind it — visible under a sleeve, or hidden.
• Print scale and placement. A logo has to land on the chest at plausible size, not stretched across the whole front.
• Layering. An open jacket implies something underneath. The model has to invent it consistently.
• Identity drift. Faces regenerate slightly differently each run. Good systems mask the face out of the generation entirely.
• Lighting mismatch. A garment shot under cool studio light onto a person shot in warm evening light needs the fabric relit, not recoloured.

What the architecture cannot do

The same limitation explains two other things people expect and don't get: it can't tell you whether a fabric will feel scratchy, and it can't show you the garment moving. Both need information a still photograph doesn't contain.

Why it runs in the cloud, not on your phone

Diffusion at this quality needs a datacentre GPU and several gigabytes of model weights. Phones can run small image models, but not ones that hold two reference identities at once at usable fidelity. So the two photos go to a server, the generated image comes back, and that round trip is most of the ten-to-thirty seconds you wait.

The practical consequence is a privacy question, and it's a fair one to ask of any try-on app: which images leave the device, what is retained afterwards, and whether they're used for training. Hoilo sends the garment photo and your active photo for the single generation you requested, stores results in the app on your phone, and requires no account — details in the privacy policy.