Decouple transform from persistence
We chose Pub/Sub between transformation and FHIR Store so backpressure and replay do not block the transform stage.
CASE STUDYHEALTHCARE · INTEROPERABILITY
We delivered a FHIR transformation pipeline on GCP that turns clinic EHR exports (CSV, JSON, XML) into standards-compliant FHIR resources (R4/R6) — predictable throughput, full replay and backfill, and an end-to-end audit trail from source to persisted resource.
Role
Scope
Scale
Services
Tags
The client operated on legacy EHR exports with no interoperability pathway. Data arrived in various formats; manual triage of clinical records was slow, error-prone, and blocked downstream analytics.
The existing formats could not integrate with modern EHR systems or FHIR-compatible endpoints. A zero-downtime migration was required — the clinical environment could not tolerate outages.
A pipeline converting EHR source data into FHIR resources (Patient, Observation, Encounter, and related types) with schema validation against target profiles.
Conditional create-or-update enabling safe replay and backfill without duplicates.
Rule sets pinned to releases so rollback becomes a config change, not a code change.
Cloud Healthcare NLP extracting structured entities (e.g. ICD-10, SNOMED CT) from free-text notes before mapping.
Every transformed resource written as JSON for traceability, partitioned by date and source batch.
FHIR validation, search, and compliance-friendly persistence as the system of record.
We designed the system to support both steady-state ingestion and large backfills with the same pipeline code. Autoscaling handles both modes without a separate batch system.
Validation gates catch schema drift early: we validate resources against target profiles before persistence, and keep audit copies for post-hoc inspection and reconciliation.
Replays and backfills are normal — design for them.
We chose Pub/Sub between transformation and FHIR Store so backpressure and replay do not block the transform stage.
We evaluated batch-only runs and went with a streaming-capable architecture so the client can move toward real-time ingestion later.
Mapping rules are versioned and pinned to releases so changes are auditable and rollbacks are predictable.
A repeatable interoperability pathway: predictable throughput, safe replay/backfill, and an end-to-end audit trail — delivered without downtime in a clinical environment.
Healthcare pipelines must tolerate reruns. Idempotent persistence makes replays safe and routine.
Treat transformation rules like a product: versions, fixtures, and rollback paths.
Traceability from raw inputs to persisted resources shortens diagnosis time and cuts manual triage.
Schema drift, policy updates, and backfills are expected — decoupled stages keep the system operable.