Computer Vision

A Hugging Face Inference-Backed Handwriting Transcription Tool for Single-Line Document Digitization

How Pfactorial Technologies built a Flask application that turns photographs of handwritten text into digital text using Microsoft's TrOCR model, without hosting the model itself.

August 21, 2026

OUTLINE

ENGAGEMENT SNAPSHOT

Pfactorial_Case_Study_HandwritingRecognition image 1

Figure 1 - Key figures from this engagement, at a glance.

EXECUTIVE SUMMARY

Our client needed a way to digitize handwritten text - for document archival, accessibility, and data capture - without standing up and hosting a transformer model themselves.

A naive upload-and-predict flow wasn't reliable enough on its own: real photographs of handwriting arrive tilted, at inconsistent resolutions, and an inference call to an external API can fail or rate-limit, so the pipeline needed to correct the image before prediction and retry gracefully after it.

Pfactorial built a Flask application that deskews and resizes each uploaded image, then calls Microsoft's TrOCR handwritten-text model through the Hugging Face Inference API, returning transcribed text to the browser.

Why this engagement is representative This engagement demonstrates Pfactorial's ability to wrap a hosted third-party model in the image preprocessing and retry logic that makes it usable in a real product, without taking on the cost of hosting the model in-house.

THE CHALLENGE

Turning a photograph of handwriting into reliable text meant handling the gap between an ideal input and what a camera or scanner actually produces.

1. Handwritten photographs arrive tilted, not upright

A skewed image degrades transcription accuracy, so every upload needed automatic deskewing to a horizontal baseline before it reached the model.

2. The model expects a fixed input shape, not whatever a camera produces

Uploaded images vary in resolution and aspect ratio, so every image needed resizing to the model's standard 384x384 input dimension before prediction.

3. An external inference API can fail or rate-limit mid-request

Relying on a hosted API instead of a self-hosted model meant the app had to retry a failed prediction call rather than surface a single failure straight to the user.

The real brief Not "call a hosted OCR model and show the result" but "correct the image for what a hosted model actually needs, and retry gracefully when the API it depends on doesn't respond the first time."

THE SOLUTION

Pfactorial built a Flask application around a resize-deskew-predict pipeline that calls Microsoft's TrOCR handwritten-text model via the Hugging Face Inference API.

Pfactorial_Case_Study_HandwritingRecognition image 2

Figure 1 - Every image is corrected for skew and resized before it ever reaches the model, and prediction retries rather than fails outright.

Architectural principles

Use a hosted inference API instead of hosting the model - Calling TrOCR through Hugging Face's serverless inference API was chosen specifically to optimise resource usage, avoiding the cost of hosting a transformer model for a lightweight use case.
Fix the image before you trust the model - Deskewing and resizing run before every prediction call, because the model's own accuracy depends on receiving an upright, correctly-dimensioned image rather than an unprocessed photograph.
Treat a third-party API as unreliable by default - The prediction function retries up to 15 times with a delay between attempts, because a dependency the app doesn't operate itself can be slow or briefly unavailable without warning.

CAPABILITIES DELIVERED

The application's capabilities center on getting a raw handwritten photo into a shape the model can transcribe reliably.

CAPABILITY	WHAT IT DOES
Image upload & format validation	Accepts PNG, JPG, or JPEG uploads through a simple web form and rejects unsupported formats before processing.
Automatic deskewing	Corrects tilted handwriting images to a horizontal baseline to improve recognition accuracy.
Standardized resizing	Resizes every uploaded image to the 384x384 dimension the TrOCR model expects.
Resilient model inference	Calls the hosted TrOCR model with retry logic so a transient API issue doesn't surface as a hard failure to the user.

Pfactorial_Case_Study_HandwritingRecognition image 3

Figure 2 - A handwritten photograph becomes transcribed text in three steps, with retries absorbing API instability.

Design note The app calls TrOCR through a hosted inference API rather than hosting the model itself - a deliberate cost/latency tradeoff that means the app's availability is bounded by a dependency it doesn't operate, which is exactly why the retry logic exists.

ENGINEERING FOR SCALE AND RELIABILITY

Four decisions kept the app reliable despite depending entirely on a hosted, third-party model.

Hosted inference over self-hosted model serving

Using Hugging Face's serverless inference API for TrOCR was chosen to optimise resource usage rather than stand up GPU-backed model hosting for a single-model use case.

Deskew before predict, every time

The deskew library converts the image to grayscale, determines the skew angle, and rotates it to correct alignment as a mandatory preprocessing step ahead of every prediction call.

Fixed-size preprocessing to match model expectations

Every image is resized to 384x384 regardless of its original dimensions, so the model always receives the input shape it was trained on.

Retry with bounded attempts instead of failing on the first error

The prediction function retries the inference call up to 15 times with a delay between attempts before giving up, absorbing the API's transient errors instead of surfacing them immediately.

DELIVERY APPROACH

The application was built around the image-processing steps the model needed, then wired to the frontend last.

1. Flask app scaffolding - setting up routing, file uploads, form submissions, and template rendering.

2. Image preprocessing functions - implementing deskew correction and the resize step ahead of prediction.

3. Model integration - wiring the prediction function to call the TrOCR model through the Hugging Face Inference API with retry handling.

4. Frontend & delivery - building the HTML/CSS/JS interface for image upload, processing status, and displaying the transcribed result.

RESULTS AND IMPACT

Pfactorial_Case_Study_HandwritingRecognition image 4

- Key outcomes from this engagement.

The application is built and functional: users upload a single-line handwritten image, and the deskew-resize-predict pipeline returns the transcribed text through Microsoft's TrOCR model.

Because deskewing and resizing run ahead of every prediction, and the inference call retries up to 15 times before failing, the tool holds up against the realistic inconsistency of camera-captured handwriting rather than only working on ideal, pre-cleaned images.

What it enabled commercially

The client can offer handwriting-to-text digitization for document archival, accessibility, or data-capture workflows without the cost and complexity of hosting a transformer model themselves.

WHY PFACTORIAL

This engagement reflects Pfactorial's approach to building on top of hosted, best-in-class models - wrapping them in the preprocessing and retry logic that turns a research model into something a real product can depend on.

Pfactorial_Case_Study_HandwritingRecognition image 5

- Service lines this engagement draws on.

Engagement enquiries Pfactorial Technologies works with organisations that need document digitization, OCR, or handwriting-to-text capability built around best-in-class hosted models rather than in-house model hosting. If you're evaluating a digitization or accessibility feature, we're happy to give you an honest read on scope and risk before anyone commits to anything. · pfactorial.ai

APPENDIX A - TECHNOLOGY STACK

The technology stack underpinning the system, grouped by the layer it serves.

Pfactorial_Case_Study_HandwritingRecognition image 6

Result and Analysis

ENGAGEMENT SNAPSHOT

How Pfactorial Technologies built a Flask application that turns photographs of handwritten text into digital text using Microsoft's TrOCR model, without hosting the model itself.