Curio Digital Therapeutics (Current Role)

HIPAA-Compliant Real-Time AI Inference Engine

Architecture Lead

The Problem

The clinical platform required real-time AI processing capabilities to support digital therapeutics, but integrating Large Language Models natively posed extreme risks of leaking Protected Health Information (PHI) to third-party endpoints.

The Constraints

Strict HIPAA obligations: PHI must not reach third-party model endpoints. The system had to support burst traffic without degrading the core application performance.

Architecture & Solution

Designed a privacy-first Retrieve-and-Generate (RAG) pipeline. Architected a PII/PHI redaction layer using Microsoft Presidio, executing before any LLM inference call so that unredacted PHI is not sent to external model endpoints. I also decoupled the heavy inference operations from the main application thread using an asynchronous, event-driven pattern via Apache Kafka, keeping the interactive experience independent of AI processing load.

System architecture of the HIPAA-compliant AI inference engine: requests pass through a Presidio PHI redaction layer before Kafka-decoupled LangChain inference; PHI never reaches the LLM provider. — System architecture — PHI is redacted at the boundary before any Kafka-decoupled LLM inference.

Sequence diagram: PHI is tokenized by Presidio before publishing to Kafka; the LLM only ever receives redacted context; re-identification happens after inference inside the trust boundary. — Request sequence — the LLM only ever receives redacted context; re-identification happens inside the trust boundary.

How It Works

The core design decision was to treat the LLM provider as an untrusted boundary. Nothing that crosses that boundary may contain PHI — not the prompt, not the retrieved context, not the metadata. That framing turns "AI compliance" from a policy question into an architecture question: where exactly is the trust boundary, and what enforces it?

The enforcement point is a Presidio-based redaction layer that sits in front of every inference call. Incoming text is analyzed for PHI entities — names, dates, contact details, record identifiers — and each entity is replaced with a stable placeholder token before the request is allowed to proceed. The mapping between tokens and original values is stored only inside the platform. After the model responds, a re-identification step swaps the placeholders back inside the trust boundary, so clinicians see natural output while the LLM only ever saw redacted context.

The second problem was load isolation. Clinical traffic is bursty, and synchronous LLM calls on the request path would have coupled core application latency to a third-party service. Inference requests are instead published to Apache Kafka and processed by dedicated consumer workers. The main application acknowledges immediately and delivers results asynchronously, so spikes in AI workload land on the queue rather than the request path, and a slow or failing model provider affects the AI feature rather than the core platform.

Retrieval follows the same privacy rules as generation: documents are chunked and embedded behind the redaction layer, so vector search operates on sanitized content. This closed the subtle leak path where raw clinical notes end up verbatim inside a vector store or a retrieved-context window.

Outcome & Impact

Deployed to production with the redaction and audit controls in place. The event-driven design routes spikes in AI workload onto the queue rather than the core application, so inference load is isolated from the interactive request path.

Impact: PHI redaction enforced at the inference boundary; inference load isolated from the core request path.

LangChainPresidioApache KafkaAWSPythonTypeScript