Computer Vision

Designing an AR-Powered Personalized Hairstyle Recommendation App

A proposed pipeline combining facial landmark detection, generative hairstyle prediction, and augmented reality try-on into a single mobile styling experience.

August 21, 2026

OUTLINE

ENGAGEMENT SNAPSHOT

Figure 1 - Key figures from this engagement, at a glance.

EXECUTIVE SUMMARY

Our client wanted to help users choose a hairstyle that genuinely suits them, rather than browsing generic style galleries with no connection to their actual face shape or hair type - a personalization gap that static style-inspiration content can't close.

Pfactorial Technologies proposed a mobile AI pipeline that detects a user's facial landmarks, generates candidate hairstyles matched to those landmarks with a generative model, predicts precise hairstyle placement, and lets the user preview and apply the result through augmented reality on their own device.

The design sequences three distinct AI tasks - facial understanding, style generation, and placement prediction - as a deliberate pipeline rather than one model attempting personalization, generation, and rendering all at once, with the AR layer handling only presentation once the styling decision has already been made.

Why this engagement is representative This engagement demonstrates Pfactorial's approach to consumer AI/AR proposals: sequencing distinct AI capabilities - detection, generation, and placement - deliberately rather than collapsing them into one model, and keeping AR strictly as the presentation layer once a styling decision is made.

THE CHALLENGE

Recommending a hairstyle that actually suits a person is a personalization problem, not a content-browsing problem. Several distinct pieces needed to work together.

1. Generic style galleries ignore the user's own face

Style inspiration content shown without reference to a user's face shape, hair type, or history isn't a recommendation - it's just browsing, and it doesn't tell a user whether a style would actually suit them.

2. Generating a hairstyle needs the user's real geometry

A convincing generated hairstyle has to be conditioned on the user's actual facial landmarks, not applied as a generic overlay, which means facial understanding has to happen before any generation step.

3. A generated style still needs precise placement

Producing a hairstyle image is not the same as knowing exactly where it sits relative to the user's head - that requires a second, dedicated step to predict hairstyle landmarks from the generated result.

4. Trying a style needs to feel real, not flat

A 2D preview image doesn't let a user see how a style looks from their own angle, in their own space - that requires positioning the styled result accurately in augmented reality.

The real brief Not a static style gallery, but a pipeline that understands a specific user's face, generates a style genuinely matched to it, and lets them see it on themselves in real time.

THE SOLUTION

Pfactorial proposed a sequenced, three-model pipeline: facial landmark detection first, then generative hairstyle prediction, with augmented reality handling presentation once the styling decision is made.

Figure 1 - Proposed pipeline: facial landmark detection, generative hairstyle prediction, hairstyle landmark prediction, and AR try-on.

Architectural principles

Facial understanding happens once, up front - the Face Landmarker pipeline - face detection, landmark location, then expression/feature identification - runs as a one-time process per user, with the extracted landmarks reused across every generation and placement step that follows.
Generation and placement are two separate models - a style-based GAN generates a candidate hairstyle matched to the user's landmarks, and a second, purpose-built model then predicts precise hairstyle landmarks from that generated result - rather than one model doing both jobs.
AR is presentation, not decision-making - augmented reality (Google ARCore) is used specifically to position the already-generated, already-placed hairstyle accurately in the user's own space - the styling decision is made upstream, not inside the AR layer.
A proven mobile-AR stack, not custom rendering - ARCore's device-tracking and real-world understanding is used directly rather than building custom spatial tracking, keeping engineering effort focused on the styling pipeline itself.

CAPABILITIES DELIVERED

The proposed system's capabilities span the full path from a user's face to a styled, previewable result.

CAPABILITY	WHAT IT DOES
Facial Landmark Detection	Extracts facial landmarks and expressions from an image or video using the Google Face Landmarker (MediaPipe) task.
Generative Hairstyle Prediction	A style-based GAN generates candidate hairstyles matched to the user's extracted facial landmarks.
Hairstyle Landmark Prediction	A custom-architecture model (e.g. MobileNetV2) predicts precise hairstyle landmarks from the generated result.
Braid Feature Customization	Braid landmark coordinates support customizable length and spacing, feeding directly into the AR object.
AR Try-On Experience	Google ARCore, integrated with a Flutter application, positions the generated hairstyle on the user in their own space.
Backend & API Layer	A Django REST API backend connects the mobile application to web servers, supporting the end-to-end styling experience.

Figure 2 - Two-phase delivery plan: MVP design and development, followed by deployment, integration, and testing.

Design note A separate, real-time braid-centering feature is proposed as a companion capability: continuous camera monitoring with live feedback on whether a hair braid is centered, displaying precisely centered braid lines for the user to align to.

ENGINEERING FOR SCALE AND RELIABILITY

Several design choices in the proposal reflect deliberate sequencing and technology selection for a personalized, on-device styling experience.

Facial landmark detection is treated as a one-time, reusable step

landmarks are extracted once per user and reused across generation and placement, rather than being recomputed at every stage of the pipeline.

A style-based GAN was selected specifically for style diversity

Style-based GANs are proposed specifically because they support mixing and generating multiple candidate hairstyles for a user to choose between, rather than committing to a single generated result.

Placement prediction uses a lightweight, purpose-built model

MobileNetV2, or a comparable lightweight architecture, is proposed for hairstyle landmark prediction specifically because it needs to run efficiently on the generated image, not for raw capacity.

AR object creation is landmark-driven

the exact braid landmark coordinates - not a generic template - drive the resulting VR/AR object, so the try-on result reflects the specific generated style rather than an approximation of it.

A proven mobile AR framework over custom tracking

Google ARCore's device-position tracking and real-world understanding is used directly, avoiding the cost and risk of building custom spatial tracking for a consumer mobile app.

Backend and mobile are cleanly separated

a Django REST API backend is proposed specifically to connect the Flutter/React Native mobile application with web servers, keeping the AI pipeline and the mobile presentation layer independently deployable.

DELIVERY APPROACH

The proposal lays out a two-phase delivery plan moving from MVP design through deployment and testing.

1. Design and develop the Facemesh and hair-line models - build and deploy the facial landmark detection and hairstyle-landmark prediction models as the pipeline's foundation.

2. Design and develop the application - build the mobile application shell that will host the styling pipeline and AR try-on experience.

3. Develop and integrate the AR object pipeline - build the AR object generation and integration layer that positions generated hairstyles on the user via ARCore.

4. Deploy, integrate, and test the MVP - bring the facial detection, generation, and AR layers together into a single deployed, tested MVP application.

RESULTS AND IMPACT

Figure - Key outcomes from this engagement.

As a proposal-stage engagement, no production results exist yet; the proposal instead lays out a phased delivery plan - Facemesh and hair-line model design and deployment, application design and development, and AR object development and integration - each scoped with its own build duration ahead of a final MVP deployment and testing phase.

The pipeline's sequencing - facial landmark detection once, generation and placement as two distinct models, and AR strictly for presentation - is designed specifically so each component can be built, tested, and improved independently rather than as one monolithic styling model.

What it enabled commercially

If built as proposed, the app would let the client's users see a hairstyle recommendation genuinely matched to their own face shape and hair type, previewed on themselves in real time through AR, rather than browsing generic style inspiration disconnected from how it would actually look on them.

WHY PFACTORIAL

This engagement draws on Pfactorial's AI product engineering capability: sequencing detection, generation, and placement as distinct, purpose-built steps, and using augmented reality strictly for presentation once the styling decision has already been made.

Figure - Service lines this engagement draws on.

Engagement enquiries Pfactorial Technologies works with consumer app teams looking to build genuinely personalized AI/AR experiences without over-building the underlying pipeline. If you are evaluating whether a personalization or AR try-on feature is worth building properly, we are happy to give you an honest read on scope, cost, and risk before anyone commits to anything. · pfactorial.ai

APPENDIX A - TECHNOLOGY STACK

The technology stack underpinning the system, grouped by the layer it serves.

Result and Analysis

ENGAGEMENT SNAPSHOT

A proposed pipeline combining facial landmark detection, generative hairstyle prediction, and augmented reality try-on into a single mobile styling experience.