back_office_ops · healthcare · workflow
Vivian Health connects healthcare professionals to their dream job with dbt Cloud and Metaplane
Vivian Health's data team faced recurring data incidents with downstream impacts that were hard to identify and fix. Manual pipeline testing was slow (one hour to a full day per pipeline), test thresholds required constant updates as data evolved, and new objects from product growth went unmonitored.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · Multi-source data ingestion
Data from PostgreSQL databases, business applications, and Segment events is funneled into Snowflake via Stitch, Airflow, Kinesis Firehose, and in-house builds.
Tools used
dbt CloudMetaplaneSnowflakeStitchAirflowKinesis FirehoseSegmentLookerCensusSnowpark
Outcome
Vivian Health tripled the number of data pipeline contributors, moved 100% of data models into dbt, achieved their data quality OKR, and saved weeks of effort on creating and maintaining custom tests.
What failed first
Using Persistent Derived Tables in Looker for data transformation required specialized LookML knowledge and produced fragile, hard-to-maintain code. Manual unit tests were too slow to set up and could not keep pace with scale.
Results
Volume3x increase
Running sinceat least a year and a half
Grounding & classification
Source type: vendor customer story
32 fields verified against source quotes.
recommendation systemknowledge basefailure mode describedmetric backednamed customerproduction runtime claimedsource backedtools describedworkflow describedhealthcareemployee productivityerror reductiontime savedvendor customer storyback office opsdata sync enrichmentmonitor detect alert