Healthcare & Clinical

A Layered Safety Pipeline for a Healthcare Patient Companion

A proposed conversation architecture that routes every patient message through crisis detection, intent, boundary and sentiment checks before - and after - a language model ever generates a reply.

August 21, 2026

OUTLINE

ENGAGEMENT SNAPSHOT

Figure 1 - Key figures from this engagement, at a glance.

EXECUTIVE SUMMARY

Our client needed a conversational AI pipeline for a healthcare patient companion application that could support therapeutic conversations and continuous patient engagement, without a single high-capability language model call standing between a distressed patient and a response with no safety net around it.

Pfactorial Technologies proposed a layered inference pipeline: every patient message passes through fast, purpose-trained safety classifiers - crisis detection, intent classification, clinical boundary detection, sentiment tracking, and behavioral anomaly scoring - before it reaches a high-capability model, and every generated response passes a second, post-generation safety check before it reaches the patient.

Behind the real-time pipeline sits a structured training pipeline for continuously improving the safety classifiers themselves, with reinforcement learning, automated evaluation, and a model registry that only promotes a new model version once it demonstrably outperforms the one currently in production.

Why this engagement is representative This engagement demonstrates Pfactorial's approach to AI systems in sensitive healthcare contexts: safety checks positioned before and after generation rather than relying on the language model alone, high recall prioritized over precision specifically for crisis detection, and every model promotion gated on outperforming the current production version.

THE CHALLENGE

Supporting therapeutic conversation safely, at low latency, and at a defensible cost surfaced four distinct design problems.

1. A single model call is not a safety architecture

Sending a patient message straight to a high-capability language model with no upstream risk assessment leaves crisis signals - self-harm or suicidal ideation - to be caught only by the model's own judgment in the moment.

2. Crisis detection has to run before generation, at very low latency

A classifier that flags risk after a response has already been generated and shown is too late; the check has to sit ahead of generation and return in well under the time a full conversational turn takes.

3. Cost and quality pull in opposite directions

Routing every message - a two-word check-in and a genuine crisis disclosure alike - to the most capable model available is not sustainable at continuous-engagement volume, but under-routing a complex, emotionally nuanced message is not acceptable either.

4. A model update cannot be trusted on its own say-so

Improving the safety classifiers over time requires a way to prove a newly trained model is actually better than the one currently live, not just different, before it is allowed to replace it.

The real brief Not "add a safety filter around a chatbot" but "architect a layered pipeline where crisis, boundary and sentiment signals are caught before generation, verified again after generation, and where the classifiers themselves keep improving under a controlled, evaluated release process."

THE SOLUTION

Pfactorial proposed a pipeline where fast, purpose-trained small models screen every message before it reaches a high-capability LLM, and a second safety layer checks the LLM's own output before it is shown to the patient.

Figure 1 - The proposed inference pipeline: safety pre-check, risk evaluation, context assembly, model routing, generation, and post-generation safety validation.

Architectural principles

Safety runs before generation, not just after - crisis detection, intent classification, clinical boundary detection, sentiment tracking, and behavioral anomaly scoring all run on fast small language models before any content reaches the conversational LLM.
A second check verifies what the model actually said - even after the LLM generates a response, a post-generation layer verifies it does not provide diagnosis or treatment advice, contain unsafe suggestions, or breach conversational boundaries - modifying or replacing it with a safer fallback if it does.
Model routing follows conversation complexity, not habit - a router evaluates message complexity, emotional intensity, and context length to select between a fast, cost-efficient model for routine check-ins and a stronger model for emotionally nuanced conversations.
New models earn production status, they are not assumed - a newly trained classifier only replaces the current production model through the registry once it demonstrates better performance and passes defined evaluation checks - not on the basis of a single promising metric.

CAPABILITIES DELIVERED

The proposed pipeline spans real-time safety screening, model-tiered generation, and a continuous training loop for the safety classifiers themselves.

CAPABILITY	WHAT IT DOES
Crisis Detection	Identifies potential self-harm or suicide risk signals in a patient message before any response is generated.
Clinical Boundary Detection	Detects messages requesting medical diagnosis, prescriptions, or treatment advice the system should not provide.
Behavioral Anomaly Scoring	Uses conversation patterns and behavioral signals to flag unusual changes that may indicate worsening mental health.
Complexity-Based Model Routing	Routes short, routine check-ins to a fast model and emotionally complex conversations to a stronger reasoning model.
Post-Generation Safety Validation	Verifies every generated response against clinical and safety guardrail policies before it reaches the patient, substituting a safe fallback on violation.
Continuous, Gated Model Improvement	Trains, evaluates, and promotes new classifier versions only after they outperform the current production model on defined thresholds.

Figure 2 - The proposed training and evaluation loop: data ingestion, processing, model training, fine-tuning, evaluation against the current production model, and gated deployment via the model registry.

Design note The training pipeline specifies four specialized models trained independently - crisis detection, sentiment analysis, clinical boundary detection, and behavioral anomaly detection - each fine-tuned via LoRA or QLoRA and evaluated on classification accuracy, precision/recall, crisis-detection recall rate, false-negative rate, and latency before being eligible for deployment.

ENGINEERING FOR SCALE AND RELIABILITY

Several engineering decisions shape whether a layered safety pipeline actually holds up in a real, continuously operating patient-facing system.

Crisis detection prioritizes recall over precision, by design

for a crisis-detection classifier, minimizing false negatives is the primary validation metric, since failing to detect a genuine crisis case carries materially more risk than an occasional false positive routed for extra review.

Context is summarized, not replayed in full

rather than sending the entire conversation history to the model on every turn, the context builder selects the last several messages plus a summarized version of earlier context, reducing token usage while preserving continuity.

Only structured signal reaches the risk engine

the safety pre-check layer returns structured outputs - not free text - which the risk evaluation engine combines to decide whether the conversation proceeds normally or triggers an escalation protocol.

Adversarial and edge-case testing is a first-class evaluation step

model evaluation includes curated adversarial test cases and indirect expressions of distress alongside standard validation datasets, so subtle or evolving crisis signals are tested for explicitly, not assumed to be caught.

The classifier architecture is chosen for latency, not just accuracy

a compact transformer-based encoder with a task-specific classification head is proposed specifically because it balances contextual understanding against the sub-50-millisecond latency the safety layer needs to run ahead of every message.

The approach is grounded in a related, delivered production system

Pfactorial's related delivered voice AI system for a recruitment client - a distilled Llama 3.2 1B model serving roughly 50 concurrent conversations at 1000-1200 millisecond end-to-end latency on an H200 GPU cluster - demonstrates the same low-latency, distilled-model approach this design proposes for the safety classifiers.

DELIVERY APPROACH

The proposed engagement builds the inference pipeline and its supporting training pipeline as two coordinated systems.

1. Build the safety pre-check layer - implement crisis detection, intent classification, boundary detection, sentiment tracking, and anomaly scoring as fast, low-latency classifiers.

2. Build the risk evaluation and escalation engine - combine classifier outputs into an escalation decision, routing high-risk conversations to crisis-specific response flows.

3. Build context assembly and model routing - implement the context builder and the complexity-based router between the fast and deep-reasoning model tiers.

4. Build post-generation validation - implement the second safety layer that checks every generated response against clinical and guardrail policy before delivery.

5. Stand up the training and evaluation pipeline - build data ingestion, processing, model training, fine-tuning, evaluation against the current production model, and the gated model registry.

6. Establish continuous improvement - wire the pipeline for ongoing retraining and evaluation as new data becomes available, without disrupting the live system.

RESULTS AND IMPACT

Figure - Key outcomes from this engagement.

The proposed architecture gives the client a pipeline where every patient message is screened for risk before generation and every generated response is checked again before delivery - two independent safety layers rather than reliance on a single model's judgment in the moment.

The model-tiering approach, validated in principle by Pfactorial's related delivered distillation-based voice system supporting roughly 50 concurrent conversations at sub-1200-millisecond latency, gives the client a concrete, evidence-based path to a responsive pipeline rather than an unproven cost and latency assumption.

What it enabled commercially

If delivered as scoped, the client gains a patient companion pipeline where safety review is structural rather than a single point of failure, and where the underlying classifiers are designed to keep improving under a controlled, evaluated release process rather than being frozen at initial training quality.

WHY PFACTORIAL

This engagement draws on Pfactorial's applied research and compliance capability: layered safety architecture for sensitive conversational AI, evaluation-gated model deployment, and a distillation approach already proven in a related delivered production voice system.

Figure - Service lines this engagement draws on.

Engagement enquiries Pfactorial Technologies works with healthcare and digital health teams building patient-facing conversational AI that has to be safe by architecture, not just by prompt. If you are evaluating a clinical or mental-health-adjacent conversational system, we are happy to give you an honest read on scope, cost, and risk before anyone commits to anything. · pfactorial.ai

APPENDIX A - TECHNOLOGY STACK

The technology stack underpinning the system, grouped by the layer it serves.

Result and Analysis

ENGAGEMENT SNAPSHOT

A proposed conversation architecture that routes every patient message through crisis detection, intent, boundary and sentiment checks before - and after - a language model ever generates a reply.