Healthcare & Clinical

A Multi-Tool Extraction Pipeline for Structured Clinical Variables from Unstructured Notes

How Pfactorial Technologies combined NLP tools, custom numeric extraction, targeted regex and LLM-based parsing to pull structured clinical variables out of narrative reports standard tools couldn't fully handle.

August 21, 2026

OUTLINE

ENGAGEMENT SNAPSHOT

Pfactorial_Case_Study_Clinical_Data_Extraction_Pipeline image 1

Figure 1 - Key figures from this engagement, at a glance.

EXECUTIVE SUMMARY

Our client needed key clinical variables - pulmonary pressures, walk-test distances, functional classifications, pulmonary function test results - pulled out of unstructured clinical notes and reports for research use, but no single extraction tool covered every variable reliably.

Standard clinical NLP tools handle common medical concepts well, but specific numeric measurements from catheterization and echocardiogram reports, and complex, inconsistently-formatted pulmonary function test tables, consistently fell through the gaps of a single-tool approach.

Pfactorial built a tiered extraction stack - a standard clinical NLP tool, a custom Python numeric-extraction tool, targeted regex for holdout variables, and a two-stage LLM pipeline for complex tables - with each tool covering what the ones before it couldn't reliably extract.

Why this engagement is representative This engagement shows Pfactorial's approach to a common clinical-data-engineering reality: no single extraction tool is enough, and the real work is knowing which tool to reach for when the previous one hits its limit.

THE CHALLENGE

Getting complete, accurate clinical variables out of narrative reports required solving for the specific ways each extraction approach falls short.

1. Standard clinical NLP misses key numeric values

Variables like mean Pulmonary Artery Pressure and Pulmonary Arterial Wedge Pressure were not successfully extracted from catheterization and echocardiogram reports by a standard clinical NLP tool alone.

2. Some variables resist automated tools entirely

Six-Minute Walk Distance and WHO Functional Classification weren't consistently captured by any automated approach, requiring manual review of format variations before a targeted solution could be built.

3. Pulmonary function test reports defy rule-based parsing

Complex tables, inconsistent formatting, and free-text interpretations in PFT reports are difficult to parse reliably with regex or rule-based methods alone.

4. LLM extraction needs guardrails to be trustworthy

Using a large language model to extract structured clinical data introduces a risk of hallucination and formatting errors that has to be actively controlled for, not assumed away.

The real brief Not “run an NLP tool over the notes” but “build the layered extraction approach that gets every variable, including the ones a single tool will always miss.”

THE SOLUTION

Pfactorial layered four extraction approaches, each addressing what the previous one couldn't reliably capture, with the most complex case - PFT reports - handled by a validated two-stage LLM pipeline.

Pfactorial_Case_Study_Clinical_Data_Extraction_Pipeline image 2

Figure 2 - Four tools, layered to cover what each one alone would miss.

Architectural principles

Start with standard clinical NLP - Apache cTAKES handles tokenization, entity recognition and vocabulary mapping to SNOMED-CT and RxNorm for the majority of standard medical concepts, including negation and uncertainty detection.
Fill numeric gaps with a custom tool - A Python-based numeric extraction tool, using a structured seven-step process, targets specific values like mPAP and PAWP that cTAKES didn't successfully extract from catheterization and echo reports.
Reach for targeted regex only where needed - Regular expressions, manually developed after reviewing real report format variations, extract variables like 6MWD and WHO Functional Class that automated tools consistently missed.
Reserve the LLM for genuinely complex cases - The two-stage LLM pipeline is applied specifically to pulmonary function test reports, where table complexity and formatting inconsistency exceed what rule-based approaches can handle.

CAPABILITIES DELIVERED

Each capability targets a specific extraction gap the previous tool in the stack couldn't close.

CAPABILITY	WHAT IT DOES
Standard clinical concept extraction	cTAKES-based identification of diseases, symptoms, medications and lab tests, mapped to standardized vocabularies.
Custom numeric extraction	Python-based tool for catheterization and echocardiogram values missed by standard NLP.
Targeted regex extraction	Manually developed patterns for 6MWD, WHO Functional Class, and PFT tabular data.
LLM-based PFT extraction	Two-stage pipeline converting complex pulmonary function reports into structured, validated JSON.
Multi-layer output validation	Schema validation, model validation, and source-text verification to catch AI-generated errors.
Enterprise-scale readiness	GPU-based inference designed to scale from pilot studies to enterprise-scale processing.

Pfactorial_Case_Study_Clinical_Data_Extraction_Pipeline image 3

Figure 3 - Section identification first, LLM extraction second, validation throughout.

Design note Isolating just the PFT section of a report before running the LLM - rather than feeding it the entire document - cuts both processing cost and the surface area for extraction errors. Scoping the input is as important as the extraction method itself.

ENGINEERING FOR SCALE AND RELIABILITY

Five decisions keep the LLM-based extraction stage trustworthy enough for research use.

Section identification before extraction

A lightweight rule-based and NLP segmentation module isolates only the PFT-relevant sections of a report, reducing unnecessary processing and improving downstream extraction accuracy.

Schema-constrained LLM prompting

Carefully engineered prompts paired with predefined JSON schemas guide the model to extract clinically relevant variables in a consistent structured format.

Three-layer output validation

JSON schema validation, Pydantic model validation, and source-text verification each check the LLM's output before it's accepted, reducing hallucinations and formatting errors.

Automated reprocessing on failure

Failed or incomplete outputs are automatically fed back through the pipeline rather than silently accepted or requiring manual intervention.

Benchmarked accuracy before scaling

Initial benchmarking against representative sample datasets confirmed approximately 85-88% extraction accuracy before the pipeline was considered for enterprise-scale deployment.

DELIVERY APPROACH

The engagement layered extraction tools in order of increasing complexity, validating coverage at each stage before adding the next.

1. Standard NLP extraction baseline - deploying cTAKES for common medical concept extraction and identifying its coverage gaps.

2. Custom numeric extraction tool - building the Python-based seven-step process for values cTAKES missed from cath and echo reports.

3. Targeted regex development - manual review of report format variations and development of patterns for 6MWD, WHO FC and PFT tables.

4. PFT section identification - building the rule-based and NLP segmentation module to isolate relevant report sections.

5. LLM extraction & validation - schema-constrained prompting with three-layer validation for complex PFT tables.

6. Benchmarking & scale readiness - accuracy benchmarking against sample data and GPU-based inference for enterprise-scale volume.

RESULTS AND IMPACT

Pfactorial_Case_Study_Clinical_Data_Extraction_Pipeline image 4

Figure 4 - Key outcomes from this engagement.

The layered extraction stack recovered clinical variables that any single tool alone would have missed, producing structured, validated pulmonary function datasets suitable for research use.

The structured PFT datasets now support downstream applications including clinical research datasets, disease registries, cohort discovery and machine learning model development.

What it enabled commercially

The client's research team gained access to structured clinical variables that were previously trapped in narrative text and unreachable by any single extraction tool, directly expanding the dataset available for downstream research and analytics.

WHY PFACTORIAL

This engagement reflects Pfactorial's clinical data engineering service line: layering extraction tools deliberately, each one scoped to what it's actually good at, rather than over-relying on any single method - including the LLM.

Pfactorial_Case_Study_Clinical_Data_Extraction_Pipeline image 5

Figure 5 - Service lines this engagement draws on.

Engagement enquiries Pfactorial Technologies works with life-sciences and clinical research teams that need structured variables extracted reliably from unstructured medical text. If you're evaluating a clinical data extraction initiative, we're happy to give you an honest read on scope and risk before anyone commits to anything. · pfactorial.ai

APPENDIX A - TECHNOLOGY STACK

The technology stack underpinning the system, grouped by the layer it serves.

Pfactorial_Case_Study_Clinical_Data_Extraction_Pipeline image 6

© 2026 Pfactorial Technologies. Client identity and product-specific implementation detail are withheld or generalized; no client data, credentials, source code, or infrastructure detail is included in this document.