Healthcare & Clinical

Standardizing Clinical Trial Entity Names at the Source

A proposed hybrid entity-resolution pipeline that links every variant name for a drug, sponsor, or condition to one canonical record - built on rule matching, fuzzy matching, and AI for the genuinely ambiguous cases.

August 21, 2026

OUTLINE

ENGAGEMENT SNAPSHOT

CS-023_MasterKey_Clinical_Trial_Entity_Resolution image 1

Figure 1 - Key figures from this engagement, at a glance.

EXECUTIVE SUMMARY

Our client's clinical trial analysis was blocked by a basic data problem: the same drug, company, or disease appears under different names across trials - the drug Imatinib is also Gleevec or STI571, and a sponsor like Novartis appears as Novartis AG, Novartis Pharma, and other variants - making comprehensive cross-trial analysis effectively impossible.

Pfactorial Technologies proposed MasterKey, a system that automatically standardizes entity names by creating one official version for each entity and linking every variation to it, combining ClinicalTrials.gov data with official medical vocabularies (RxNorm, WHO INN, FDA, MeSH, ICD-10, SNOMED) through a hybrid matching approach - exact rule matching for certainty, fuzzy matching for typos and abbreviations, and AI for the genuinely ambiguous edge cases.

The proposed output is three clean, analysis-ready reference tables - Sponsors, Investigational Products, and Indications - each carrying official names, all known synonyms, parent-subsidiary relationships, and medical codes, refreshed on a weekly schedule.

Why this engagement is representative This engagement demonstrates Pfactorial's approach to data standardization proposals: combining cheap, certain techniques with AI only where matching genuinely can't be resolved otherwise, and testing real integration assumptions - like cross-vocabulary coverage - before committing to a delivery estimate.

THE CHALLENGE

Entity name variation is a foundational data problem that blocks any downstream analysis until it's solved. Several distinct difficulties needed to be addressed together.

1. The same entity has many names across trials

A drug, sponsor, or condition can appear under several different names, abbreviations, or corporate variants across different trial records, making a simple exact-match join useless for comprehensive analysis.

2. External vocabularies have incomplete coverage

WHO INN drug data via PubChem, and sponsor hierarchy data via the GLEIF API, each have partial coverage - many drugs lack an INN equivalent, and many trial sponsors are absent from the GLEIF registry entirely.

3. Textual similarity alone risks incorrect matches

Fuzzy text matching can align terms that look similar but mean something different - for example, the condition "5q35 Duplications" can incorrectly align with the unrelated MeSH term "15q24 Microdeletion" despite having no real textual resemblance.

4. The MVP scope needed clear boundaries

The clinical trial database spans Behavioral, Biological, Device, and Genetic intervention categories, and the proposal needed to scope the MVP specifically to the Drug category rather than attempting universal coverage from day one.

The real brief Not "deduplicate some records" but "produce three canonical, analysis-ready reference tables that every future cross-trial analysis can join against with confidence."

THE SOLUTION

Pfactorial proposed a hybrid entity-resolution pipeline that ingests ClinicalTrials.gov data alongside external medical vocabularies, applies matching techniques in order of cost and certainty, and outputs standardized reference tables on a weekly schedule.

CS-023_MasterKey_Clinical_Trial_Entity_Resolution image 2

Figure 1 - Proposed system flow: raw trial data ingestion, per-entity normalization, and standardized reference table output.

Architectural principles

Cheapest, most certain matching runs first - exact rule matching is applied before fuzzy matching, and fuzzy matching before AI, so the most expensive technique is reserved specifically for genuinely ambiguous edge cases rather than run against every record.
Known external vocabularies anchor the canonical identity - RxNorm, WHO INN, FDA, MeSH, ICD-10, and SNOMED are combined with ClinicalTrials.gov data to ground each canonical entity in recognized medical and regulatory vocabularies rather than an internally invented naming scheme.
Business rules for canonical IDs come from the client, not assumed - governance rules for how new entities are added or merged are explicitly a client approval step, keeping the canonicalization logic aligned with how the client's own teams already think about entity identity.
MVP scope is explicit and bounded - the proposal deliberately scopes the MVP to Drug-category interventions rather than attempting to cover Behavioral, Biological, and Device interventions from the outset, keeping delivery focused and estimable.

CAPABILITIES DELIVERED

The proposed system's capabilities span ingestion, transformation, resolution, and delivery of standardized reference data.

CAPABILITY	WHAT IT DOES
Scheduled Multi-Source Ingestion	Ingests ClinicalTrials.gov (AACT schema) alongside RxNorm, WHO INN, FDA, MeSH, ICD-10, SNOMED, and a client-provided company reference dataset.
Hybrid Entity Resolution	Combines exact rule matching, fuzzy matching (pg_trgm), and AI-assisted resolution for ambiguous cases into one canonicalization pipeline.
Canonical ID Generation	Generates a single official identifier per Sponsor, Investigational Product, and Indication, with all known name variants linked to it.
Automated Validation & Reporting	Runs automated data validation and completeness checks, producing a summary report of coverage and accuracy metrics.
Weekly Automated Delivery	Exports CSV and JSON reference datasets on a weekly schedule to client-controlled cloud storage.
Documented Governance & Handover	Includes entity resolution logic documentation, known-limitations reporting, and structured knowledge-transfer sessions.

CS-023_MasterKey_Clinical_Trial_Entity_Resolution image 3

Figure 2 - Entity normalization by type: Sponsors, Investigational Products, and Indications, each resolved against their relevant external vocabularies.

Design note The proposal is explicit about coverage limits already identified during initial review: WHO INN data via PubChem is incomplete for many drugs, and GLEIF sponsor hierarchy data is partial - both documented as known gaps requiring supplementary sources rather than assumed solved.

ENGINEERING FOR SCALE AND RELIABILITY

Several engineering findings from the proposal's initial technical review shaped how the matching pipeline is scoped.

Cross-vocabulary linking was tested before being assumed

initial tests joining vocabularies via Common Unique Identifiers (CUIs) from BioPortal produced approximately 1,708 successful matches, indicating limited direct overlap and confirming the need for hybrid matching logic rather than ID-based joins alone.

MeSH and RxNorm matching has a known, tested runtime

text similarity-based mappings for MeSH and RxNorm were found to typically complete within one hour at a similarity threshold of 0.7, giving a concrete performance baseline for the production pipeline rather than an estimate.

Fuzzy matching runs at the database layer

PostgreSQL's pg_trgm extension is proposed specifically for fuzzy matching with a configurable similarity threshold, keeping approximate matching close to the data rather than in a separate application-layer process.

Unmatched entities are routed to manual or AI-assisted review, not dropped

combining MeSH mappings, keyword-based matching, and external vocabulary alignment improves coverage but doesn't resolve every entity - the proposal routes the remainder to manual validation or AI-assisted resolution rather than leaving them unmatched.

Infrastructure is scoped to a known, modest footprint

the proposed server specification (4 vCPUs, 16 GB RAM, 500 GB SSD, Ubuntu 22.04 LTS) is sized specifically for the ETL and entity-resolution workload described, provisioned in the client's own cloud account.

Client validation is a defined checkpoint, not an afterthought

the client is scoped to validate sample entity reference tables and sign off on mapping rules before full pipeline deployment, with subject matter experts reviewing entity mappings and canonicalization accuracy periodically.

DELIVERY APPROACH

The proposed Phase 1 delivery is scoped across four stages totalling 345 hours over roughly two to three months.

1. Planning and setup - gather clinical trial and medical database data, study and confirm the technical approach, design the system structure, and get client sign-off on the technology stack - 85 hours.

2. Data preparation - process and clean raw data, import external medical databases and the reference company dataset, add error tracking/logging, and run quality checks - 90 hours.

3. Building the matching system - build the exact rule matching, fuzzy matching, and LLM-assisted resolution logic, construct the sponsor hierarchy and indications transformation pipelines, deduplicate, and add semantic search - 115 hours.

4. Delivery and handover - build the automated export pipeline, run three knowledge-transfer sessions, and provide up to 15 days / 20 hours of post-handover support - 55 hours.

RESULTS AND IMPACT

CS-023_MasterKey_Clinical_Trial_Entity_Resolution image 4

Figure - Key outcomes from this engagement.

As a proposal-stage engagement, no production accuracy results exist yet; the sample outputs shown in the proposal are explicitly marked for demonstration purposes only and not finalized. The proposal instead commits to a specific validation checkpoint: the client reviews and signs off on sample reference tables and mapping rules before full pipeline deployment.

Initial technical review already surfaced concrete, tested findings that shape the delivery plan - approximately 1,708 successful CUI-based cross-vocabulary matches, a roughly one-hour runtime for MeSH/RxNorm similarity matching at a 0.7 threshold, and documented coverage gaps in WHO INN and GLEIF data - giving the proposed 345-hour Phase 1 estimate a grounding in real, tested behavior rather than an unverified assumption.

What it enabled commercially

If built as proposed, the pipeline would let the client's research and analysis teams query clinical trial data by canonical drug, sponsor, and condition identity, closing cross-trial analyses that are effectively blocked today by inconsistent entity naming.

WHY PFACTORIAL

This engagement draws on Pfactorial's data and pipeline infrastructure capability: applying the cheapest, most certain matching technique first and reserving AI specifically for the ambiguous cases it's actually needed for, while testing real coverage assumptions before committing to an estimate.

CS-023_MasterKey_Clinical_Trial_Entity_Resolution image 5

Figure - Service lines this engagement draws on.

Engagement enquiries Pfactorial Technologies works with research and life-sciences teams whose analysis is blocked by inconsistent entity naming across data sources. If you are evaluating whether an entity resolution pipeline is worth building properly, we are happy to give you an honest read on scope, cost, and risk before anyone commits to anything. · pfactorial.ai

APPENDIX A - TECHNOLOGY STACK

The technology stack underpinning the system, grouped by the layer it serves.