Back office ops · Production

Neoway builds ML platform with team-first approach and product thinking

The problem

Neoway's functional team structure created silos that prevented data scientists from accessing historical data in the proper format and from deploying model predictions integrated with the company's architecture; internal ML tooling suffered from poor developer experience and low user adoption.

First attempt

The original features framework had terrible user experience and was abandoned; the functional organizational structure created dependency bottlenecks that slowed business value delivery.

Workflow diagram · grounded in source
1
Raw data pipeline preparation
integration
“Data pipelines are created to prepare the raw data that is produced by other applications in a structured format to be used by data science tasks. Every dataset schema is registered and it's available to users in an organized way for eas…”
2
Feature Store feature engineering
integration
“The Feature Store facilitates the creation and consumption of features used in data science models. It maintains consistency between development and production by using the same feature code in both environments. It encourages the reusab…”
3
Distributed model training
ai_action
“The Development Environment enables data scientists to provision the infrastructure of distributed spark clusters with a set of tools installed and configured in an easy way. The users can do any kind of data science tasks like data expl…”
4
Model productization and CI/CD
validation
“Model productization templates and tools include a data science template to package the source code. It has batteries included to create python modules, run unit and integration tests on CI pipeline, publish docker images in the registry”
5
Airflow batch pipeline orchestration
integration
“Pipeline orchestration is the part responsible for running batch pipelines for model training and scoring. We use Airflow to create the DAGs that run periodically, but we use the concept of DAG as a configuration.”
6
Model API prediction serving
output
“Model APIs can be used to serve models. Binary artifacts from the models created in the productization step are loaded during initialization and used for making predictions by key.”
Reported outcome

Neoway redesigned into cross-functional cells in early 2020 and built an ML platform covering data pipelines, a Feature Store, distributed development environments, model productization tooling, Airflow-based pipeline orchestration, and model serving APIs.

Reported metrics
user adoption of internal ML toolsbad experience and low user adoption
Software delivery speedaccelerating software delivery
Reported stack
AirflowGitlabDocker
Source
https://mlops.community/blog/building-neoways-ml-platform-with-a-team-first-approach-and-product-thinking
Read source ↗

Frequently asked questions

What did this team achieve with this AI workflow?

Neoway redesigned into cross-functional cells in early 2020 and built an ML platform covering data pipelines, a Feature Store, distributed development environments, model productization tooling, Airflow-based pipeline…

What tools did this team use?

Airflow, Gitlab, Docker.

What results were reported?

user adoption of internal ML tools: bad experience and low user adoption; Software delivery speed: accelerating software delivery (source-reported, not independently verified).

What failed first in this deployment?

The original features framework had terrible user experience and was abandoned; the functional organizational structure created dependency bottlenecks that slowed business value delivery.

How is this back office ops AI workflow structured?

Raw data pipeline preparation → Feature Store feature engineering → Distributed model training → Model productization and CI/CD → Airflow batch pipeline orchestration → Model API prediction serving.

Related back office ops cases