Earring virtual try-on: how it works and what to upload

What the shopper does

The shopper opens the product and takes a photo with the try-on's own camera. On a desktop the page shows a QR code and the phone takes the picture; the photo appears on the computer within seconds.

Small red dots mark the usual piercing spots on the ear. The shopper taps where the earring is worn, and the earring appears there at its real size. A tap on the cheek or the background places nothing; the earring can be dragged afterwards.

• A pair, when both ears show. On a frontal photo the earring appears on both ears, mirrored. On a turned head only the visible ear is dressed.
• Hair stays in front. Strands that cross the ear cover the earring, as they would in life. When the ear is fully under the hair, the earring is shown anyway and the shopper is told to tuck the hair back.
• Save and share. The finished picture can be saved or sent from the phone's share sheet.

What the brand uploads

Everything comes from your own product photography. Three things make it work:

1. One earring, hanging as it is worn. A single piece on a transparent or white background, not the pair and not the box. The background is removed automatically on upload and the picture is cropped to the piece.
2. The real width in millimetres. This is the whole scale contract: the earring is drawn on the shopper's ear from this number and the size of the shopper's iris. Nothing is drawn until it is typed.
3. How it hangs and where it is worn. A stud sits on the tap; a hoop or a drop hangs from it. The placement — lobe, helix, tragus — decides which spot on the ear is emphasised and seeds a sensible width.

Optional, and worth doing: a profile image of the earring, shown when the head faces the camera and the ear is seen edge-on; colourways, each with its own front image, which the shopper flicks between on one photo; and a few model portraits in the store's settings, so the try-on opens on a model before anyone takes a selfie.

How the placement is decided

The face mesh gives 478 points on the face but none on the ear, and we do not add an ear detector: hair, angle and lighting change from photo to photo, and a wrong automatic guess is worse than no guess. Instead the shopper's tap decides, and the system makes that tap easy to get right.

• A ruler from the iris. The iris is 11.7 mm across in almost every adult, so the photo's millimetres come from it. The ear sits deeper than the eyes on a frontal photo, and the drawn size is corrected for that from the head pose.
• An ear zone, not an ear. A generous area behind and below the ear's front edge is where an earring can plausibly be. A tap outside it is refused with a short note; a tap inside places.
• Spots from the ear itself. When the photo's segmentation shows the ear clearly, the piercing spots are placed from the ear's own outline — the lobe's bottom, the rim — rather than from averages. When it does not, anatomical averages stand in and the tap still decides.

What happens to the photo

The shopper's photograph is uploaded to our servers to build the composite and is used for nothing else. It is never shown to the store or to other shoppers, and the shopper can delete it from inside the try-on. The store sees counts — how many opened the try-on, placed an earring, saved a picture — and never a photograph.

Where to start

Create an earring scene, upload one earring and its width, publish it to the product. If you sell several colours of the same design, add them as colourways on the same scene rather than as separate products.