Abstract:
The eyewear industry has long struggled with a fundamental challenge: customers cannot accurately visualize how a pair of glasses will look on their face before purchasing, leading to high return rates and customer dissatisfaction. TryNStyle addresses this problem through an AI-powered, web-based personalized eyewear try-on platform that combines real-time computer vision, 3D rendering, and intelligent face analysis.
The system provides two distinct try-on modalities: a 2D overlay mode that uses MediaPipe facial landmark detection to precisely position glasses frames on the user's face through the webcam in real-time, and a 3D try-on mode powered by Three.js with WebGL that renders actual GLB-format 3D glasses models with full six-degree-of-freedom head tracking, including roll, yaw, and pitch rotation. In addition, a contact lens try-on feature overlays photorealistic lens PNG textures directly onto the detected iris region of both eyes simultaneously using MediaPipe iris landmark detection.
Beyond the try-on functionality, TryNStyle incorporates a face shape classification model built on MobileNetV2 through transfer learning, trained on a dataset of 9,326 facial images across five categories: Heart, Oblong, Oval, Round, and Square. The model achieves an overall test accuracy of 81.9% with per-class accuracies ranging from 78.2% to 88.8%. A skin tone analyzer using LAB colour space analysis provides personalized frame colour recommendations.
The full-stack application is built with a Next.js frontend, NestJS backend, PostgreSQL database, and FastAPI AI microservice, all containerized with Docker for seamless deployment. The platform features a comprehensive admin panel for product management including 3D model uploads, lens PNG management, and multi-color variant support. Customer-facing features include a product catalog with color-switching, wishlist, cart, order management, and Stripe payment integration. Results demonstrate that TryNStyle successfully delivers a real-time, accurate, and user-friendly virtual eyewear try-on experience that can meaningfully reduce purchase uncertainty in the online eyewear retail space.
