Curio Digital Therapeutics (Current Role)

HIPAA-Compliant Real-Time AI Inference Engine

Architecture Lead

The Problem

The clinical platform required real-time AI processing capabilities to support digital therapeutics, but integrating Large Language Models natively posed extreme risks of leaking Protected Health Information (PHI) to third-party endpoints.

The Constraints

Strict HIPAA compliance. Zero tolerance for PHI leaks. The system had to support burst traffic without degrading the core application performance.

Architecture & Solution

Designed a privacy-first Retrieve-and-Generate (RAG) pipeline. Architected a critical PII/PHI Redaction Layer using Microsoft Presidio, executing before any LLM inference calls to guarantee zero data leakage. Furthermore, I decoupled the heavy inference operations from the main application thread using an asynchronous, event-driven pattern via Apache Kafka, ensuring the UI remained completely responsive during processing.

System architecture of the HIPAA-compliant AI inference engine: requests pass through a Presidio PHI redaction layer before Kafka-decoupled LangChain inference; PHI never reaches the LLM provider. — System architecture — PHI is redacted at the boundary before any Kafka-decoupled LLM inference.

Sequence diagram: PHI is tokenized by Presidio before publishing to Kafka; the LLM only ever receives redacted context; re-identification happens after inference inside the trust boundary. — Request sequence — the LLM only ever receives redacted context; re-identification happens inside the trust boundary.

How It Works

The core design decision was to treat the LLM provider as an untrusted boundary. Nothing that crosses that boundary may contain PHI — not the prompt, not the retrieved context, not the metadata. That framing turns "AI compliance" from a policy question into an architecture question: where exactly is the trust boundary, and what enforces it?

The enforcement point is a Presidio-based redaction layer that sits in front of every inference call. Incoming text is analyzed for PHI entities — names, dates, contact details, record identifiers — and each entity is replaced with a stable placeholder token before the request is allowed to proceed. The mapping between tokens and original values never leaves the platform. After the model responds, a re-identification step swaps the placeholders back inside the trust boundary, so clinicians see natural output while the LLM only ever saw redacted context.

The second problem was load isolation. Clinical traffic is bursty, and synchronous LLM calls on the request path would have coupled core application latency to a third-party service. Inference requests are instead published to Apache Kafka and processed by dedicated consumer workers. The main application acknowledges immediately and delivers results asynchronously, so a spike in AI workload cannot degrade the interactive experience — and a slow or failing model provider degrades only the AI feature, never the platform.

Retrieval follows the same privacy rules as generation: documents are chunked and embedded behind the redaction layer, so vector search operates on sanitized content. This closed the subtle leak path where raw clinical notes end up verbatim inside a vector store or a retrieved-context window.

Outcome & Impact

Deployed a fully compliant AI engine that met all regulatory clinical constraints. The event-driven architecture successfully absorbed 100% of burst traffic volatility, maintaining core system stability.

Impact: Zero-PHI leakage; 400ms inference time.

LangChainPresidioApache KafkaAWSPythonTypeScript