back_office_ops · workflow
Griffin: How Instacart's ML platform tripled ML applications in a year
Instacart's previous ML framework Lore became a bottleneck as the number, diversity, and complexity of machine learning applications grew — its monolithic architecture required refactoring the core design for every new feature.
How it works
Common implementation structure
How this type of workflow is generally built, generalized across documented cases — not tied to any one vendor's stack. Click any stage to read what happens there. Specific products that implement these stages appear in “Tools commonly seen” below.
Stage 1 · MLE generates workflow code
MLEs generate ML workflow code from base templates and update it for their application.
Tools used
GriffinLoreMLCLIMLFlowRayKubeflowFeastSnowflakeAWSDatabricksSagemakerAirflowDockerRedisScyllaS3SparkFlinkTensorflowPytorchSklearnXGBoostFastTextFaissTwirpAWS ECSKerasScikit-learnPostgres
Outcome
Griffin enabled Instacart to triple the number of ML applications in one year by providing an extensible, self-serving platform that lets MLEs quickly iterate on models, effortlessly manage product releases, and closely track production applications.
What failed first
Lore's monolithic architecture became untenable at scale — when the company only had a few ML applications it fulfilled requirements, but as applications multiplied the framework required core refactoring for every new capability.
Grounding & classification
Source type: technical build writeup
42 fields verified against source quotes.
recommendation systemproduct catalogbuilder submittedfailure mode describedmetric backednamed customerproduction runtime claimedtools describedworkflow describedecommerceemployee productivitythroughput increasetechnical build writeupback office ops