Fallnn.
An iOS visual-search app that finds buyable matches for any outfit you photograph, screenshot, or paste from TikTok.
fallnn.vercel.app — embedded below.
Product screenshots, the visual-search story, and the pre-launch waitlist live here. The iOS app itself is in TestFlight review.
Most lenses see one image. Fallnn sees each garment.
Pinterest Lens and Google Lens both flatten "find the clothes" into whole-image similarity, which collapses on a busy street photo into "items that share a color." Fallnn segments each garment first, embeds each crop separately, and returns matches per-garment.
A single photo produces distinct top, pants, and shoes sections — each with its own ranked list of buyable alternatives across a curated catalog of independent fashion brands. The user takes a picture; the app does the parsing.
Four engines, one retrieval loop.
Per-garment segmentation
SegFormer-B2 trained on ATR isolates each piece in the frame — top, pants, shoes, outerwear — so a single photo produces distinct, separately-searchable crops instead of one collapsed silhouette.
Domain-specific embeddings
FashionCLIP — a CLIP variant fine-tuned on Farfetch product imagery — embeds each garment crop. Vanilla CLIP knows the world; FashionCLIP knows the difference between a barrel jean and a wide-leg.
Vector retrieval
FAISS IndexFlatIP over L2-normalized vectors, so cosine similarity is the inner product. ~34k products from 70+ DTC brands — Stussy, Aimé Leon Dore, Reformation, Mejuri — indexed and queryable in milliseconds.
Personalized home feed
A style centroid computed over the user's recently-matched products seeds a 'Based on your style' feed via nearest-neighbor lookup in CLIP space. The home screen sharpens with every search.
From a TikTok cover to a ranked buy list.
~4–6 second per-garment round-trip on shared CPU. Models are pre-baked into the Docker image so cold starts are bounded.
- Step 01
Capture
Take a photo from the camera, paste a TikTok or Pinterest link, or import a screenshot. URL mode pulls TikTok cover frames via the public oEmbed endpoint.
- Step 02
Segment
Image is resized + compressed client-side to cut cellular latency by 5–10s, then sent to the FastAPI backend. SegFormer returns per-garment masks and crops.
- Step 03
Embed & retrieve
Each crop is embedded with FashionCLIP and queried against the FAISS index. The response is a per-garment list of nearest products with brand, price, and direct buy links.
- Step 04
Re-rank by taste
Results can be re-ranked against the user's style centroid. The home feed uses the same primitive — call it user-conditioned retrieval, not personalization theatre.
SwiftUI-styled React Native.
React Native + Expo with Supabase for auth, saved looks, and style preferences. Custom typography system pairing Playfair Display, Inter, and Didact Gothic. The camera ships with a grid + framing overlay; the URL mode handles TikTok and Pinterest links transparently.
Per-garment results render as collapsible sections with chevron-driven reordering, so users push the looks they care about to the top. A full onboarding flow captures gender, style, brand, and budget preferences and seeds the home feed before the user has any search history.
Bounded cold starts, sub-second retrieval.
Backend deployed on Fly.io — Docker image, shared-cpu-2x with 4 GB RAM, models pre-baked so the container is hot at boot. Catalog ingestion pulls from public Shopify Storefront /products.json endpoints across 70+ DTC brands and is normalized into a shared fashion taxonomy.
A /products/similar endpoint computes the centroid of the user's recently-matched products and returns nearest neighbors in CLIP space — the same primitive that powers personalized search re-ranking. Client-side resize + compression cuts upload latency by 5–10s on cellular.
The flow, in motion.
Camera — grid + framing overlay
Results — per-garment sections
Home — style centroid feed
Picked for the job, not the résumé.
- ●TypeScript
- ●Python
- ●SQL
- ●React Native
- ●Expo
- ●Expo Router
- ●Supabase JS SDK
- ●FastAPI
- ●SQLAlchemy
- ●FAISS
- ●Pillow
- ●SegFormer
- ●FashionCLIP
- ●PyTorch
- ●Hugging Face Transformers
- ●Vector retrieval
- ●Fly.io
- ●Docker
- ●Supabase (Postgres + RLS)
- ●AsyncStorage
- ●Shopify Storefront API
- ●oEmbed
- ●Multi-stage Docker builds
"Most fashion-search projects use vanilla CLIP and call it done. Fallnn picks FashionCLIP because it's domain-fine-tuned, and segments first because whole-image embedding is a known failure mode."
— Engineering notes, Fallnn