Multi-Agent & Agentic Systems

A Modular Pipeline for Multilingual Manuscript Narration

A proposed document-to-audiobook architecture that cleans and structures manuscript text, then converts it to natural, standards-compliant narration through a fine-tunable, voice-cloning text-to-speech model.

August 21, 2026

OUTLINE

ENGAGEMENT SNAPSHOT

CS-007_Multilingual_AI_Narration_System image 1

Figure 1 - Key figures from this engagement, at a glance.

EXECUTIVE SUMMARY

Our client needed to convert manuscripts - PDFs, Word documents, web content, and scanned or handwritten pages - into narrated audio that meets professional audiobook distribution standards, without a fully agentic system that would add complexity the task does not need.

Pfactorial Technologies proposed a modular, four-part pipeline instead: document processing, text-to-speech conversion, model fine-tuning, and API management, each handling one well-defined stage of the manuscript-to-narration process rather than one system trying to do everything.

The design explicitly separates deciding what to build (modular components with clear responsibilities) from picking the best tool for each stage - a purpose-built voice-cloning model for speech generation, rule-based checks for narration-quality text preparation, and continuous fine-tuning to improve voice quality over time.

Why this engagement is representative This engagement demonstrates Pfactorial's approach to media AI pipelines: choosing a modular, purpose-built architecture over a fully agentic system where the task does not require one, and treating industry compliance standards (ACX) as a rule-based verification step rather than an afterthought.

THE CHALLENGE

Converting a manuscript into professionally narrated, standards-compliant audio surfaced four distinct technical problems.

1. Manuscripts arrive in every format

Documents can be PDFs, Word files, or web content, and some contain scanned images or handwritten pages that must be treated with a separate reading path before any narration work can begin.

2. Raw manuscript text does not read aloud well

Text needs structural cleanup - pauses inserted around titles and author names, repeated headers removed - before it produces a natural listening experience, work a generic text-to-speech call will not do on its own.

3. Audiobook distribution has hard technical standards

ACX narration standards require the output audio to sit within an allowed volume range, and explicitly prohibit AI-generated voices unless separately negotiated - both constraints the pipeline has to verify, not assume.

4. A single voice sample is not enough data to fine-tune from scratch

Producing a natural, human-like narration voice for a client requires a voice-cloning approach that works from a short reference clip, since collecting hours of dedicated training audio per voice is not practical.

The real brief Not "build an agentic narration assistant" but "break the manuscript-to-audiobook process into modular, well-defined components, each handled by the right tool - text cleanup, speech generation, quality verification, and continuous voice improvement."

THE SOLUTION

Pfactorial proposed a four-module architecture, deliberately not fully agentic, so that each stage of manuscript narration - reading, cleaning, converting, and verifying - is a modular, independently testable component.

CS-007_Multilingual_AI_Narration_System image 2

Figure 1 - The proposed pipeline: document processing and language detection, text-to-speech conversion with quality verification, continuous fine-tuning, and API-managed delivery.

Architectural principles

Modular components, not one agentic system - the narration process is broken into separate, well-defined components rather than a single agent attempting every task - document processing, TTS, fine-tuning, and API management each own their stage.
Text is prepared for listening, not just reading - the document processor uses an LLM to structure text for clarity and speaker quality - adding pauses around titles and removing repeated headers - before it ever reaches the speech model.
Voice cloning from a short reference clip - the recommended text-to-speech model clones a voice from roughly six seconds of reference audio rather than requiring hours of dedicated per-voice training data.
Compliance is a rule-based check, not an assumption - the TTS module verifies output against ACX volume-range requirements through rule-based checks, rather than assuming a generated voice meets distribution standards.

CAPABILITIES DELIVERED

The proposed system covers the full path from a raw manuscript upload to distribution-ready narrated audio.

CAPABILITY	WHAT IT DOES
Multi-Format Document Loading	Accepts PDFs, Word documents, and web URLs, with scanned and handwritten documents routed to a separate processing path.
Language Identification & Text Cleaning	Identifies the manuscript's language and cleans and processes the text before conversion, using an LLM to structure it for narration clarity.
Voice-Cloning Text-to-Speech	Converts cleaned text to natural-sounding speech via a voice-cloning model, working from a short reference clip per voice.
ACX Standards Verification	Checks generated audio against required volume-range and other distribution standards through rule-based verification before it is accepted.
Continuous Model Fine-Tuning	Retrains and updates the production text-to-speech model over time to improve voice quality and naturalness for a long-term user experience.
API-Managed Delivery	Exposes upload, processing, and narration output through a REST API built for handling file uploads and managing the workflow end to end.

CS-007_Multilingual_AI_Narration_System image 3

Figure 2 - Where the proposed pipeline's tooling choices sit: pretrained voice cloning for speech generation, with fine-tuning and post-processing reserved for standards compliance.

Design note In initial testing, the recommended voice-cloning model's output exceeded the ACX-allowed volume range, while a comparison text-to-speech model's output sat inside the allowed range by design. The gap is expected to close through fine-tuning and post-processing, and is treated as a known, actively managed constraint rather than a blocker.

ENGINEERING FOR SCALE AND RELIABILITY

Several design decisions shape whether a narration pipeline produces genuinely listenable, standards-compliant audio rather than a technically correct but unpolished result.

Document handling and speech generation are cleanly separated

the document processor handles format identification, language detection, and text cleanup entirely independently of the text-to-speech module, so either stage can be improved or swapped without touching the other.

Text structuring is purpose-built for listening, not reading

specific, tested interventions - inserting pauses before and after titles and author names, stripping repeated headers - are applied because they measurably improve the listening experience, not as generic text cleanup.

The recommended TTS model is chosen for practical cloning economics

requiring only a short reference clip rather than hours of training audio per voice makes voice cloning practical at the number of distinct manuscripts and narrators the pipeline needs to support.

Fine-tuning is a continuous process, not a one-time step

the fine-tuning module is designed to update or switch the production TTS model to the best-trained version over time, rather than freezing quality at initial launch.

Standards compliance is verified programmatically

audio quality checks run as rule-based verification against ACX volume and format requirements, so a non-compliant output is caught before delivery rather than discovered after submission.

The architecture stays deliberately non-agentic

the design explicitly rejects a fully agentic approach for this task, on the basis that separate, modular components handle the well-defined narration steps more reliably and more testably than a single autonomous agent would.

DELIVERY APPROACH

The proposed engagement builds the four core modules in sequence, from document intake through to a verified, continuously improving narration output.

1. Build the document processor - implement multi-format loading, language identification, and LLM-assisted text cleaning and structuring for narration readiness.

2. Integrate the text-to-speech module - wire up the voice-cloning model for speech generation, speaker profiling, and audio output.

3. Build ACX standards verification - implement rule-based checks against required volume range and other distribution standards before audio is accepted.

4. Build the fine-tuning pipeline - implement the continuous fine-tuning and model-switching process that improves voice quality and naturalness over time.

5. Expose the API layer - build the FastAPI backend for file uploads, manuscript processing, and workflow management end to end.

RESULTS AND IMPACT

CS-007_Multilingual_AI_Narration_System image 4

Figure - Key outcomes from this engagement.

The proposed pipeline's recommended voice-cloning model demonstrated multilingual narration capability across 17 languages from a roughly six-second reference clip, with a comparison text-to-speech model's output confirmed to sit within the required ACX volume range by design, giving a concrete baseline for the fine-tuning work ahead.

By separating document processing from speech generation and treating ACX compliance as an explicit, rule-based verification step, the design gives the client a clear path to distribution-ready narration output rather than audio that has to be manually checked and reprocessed after the fact.

What it enabled commercially

If delivered as scoped, the pipeline lets the client convert manuscripts across formats and languages into narrated audio that is verified against distribution standards before delivery, with a continuous fine-tuning process designed to improve voice quality over time rather than staying fixed at launch quality.

WHY PFACTORIAL

This engagement draws on Pfactorial's AI product engineering capability: choosing a modular architecture matched to the actual complexity of the task, rather than defaulting to a fully agentic system, and treating industry compliance requirements as engineering constraints from the start.

CS-007_Multilingual_AI_Narration_System image 5

Figure - Service lines this engagement draws on.

Engagement enquiries Pfactorial Technologies works with media and publishing teams looking to automate narration, dubbing, or voice production without sacrificing distribution-standard quality. If you are evaluating a text-to-speech or voice-cloning pipeline, we are happy to give you an honest read on scope, cost, and risk before anyone commits to anything. · pfactorial.ai

APPENDIX A - TECHNOLOGY STACK

The technology stack underpinning the system, grouped by the layer it serves.