SOTA MLOps Tools

Another post on the proliferation of tools. I'm writing this to just keep up-to-date on what the cool kids are talking about.

Feature Store

The most popular offering is Feast (FOSS and mostly Python)  but it's batch only.  There is also Tecton (closed source, managed patform) that claims to do streaming.

Features stores basically store features that are hard or expensive to compute. For example, in Edge ML (where smart devices process locally), some data may be derived from an aggregation of data the device does not otherwise have access to.

"Feature store is an increasingly loaded term that can be used by different people to refer to very different things... There are three main problems that a feature store can help address

• Feature management: "features used for one model can be used in another"
• Feature computation: "feature engineering logic, after being defined, needs to be computed... If the computation of this features isn't too expeisnive, it might be acceptable computing this feature each time [or] you might want to execute it only once the first time it is required, then store it"
• Feature consistency: "feature definitions written in Python during development might need to be converted into the language used in production... Modern feature stores... unify the logic for both batch features and streaming."

"Some features stores only manage feature definitions without computing features from data; some feature stores do both.

"Out of 95 companies I surveyed in January 2022, only around 40% of them use a features store. Out of those who use a feature store, half of them build their own" [1]

However, note that some say Feature Stores are not worth the complexity [TowardsDataScience].

Model Serving

MLFlow is an open source project from Databricks. By putting its dependencies in your code, you can serve models via HTTP. However, you must still create a Docker container for its server. 

Metaflow was open sourced by Netflix. It's a framework that creates a DAG of the pipeline a data scientist may use.

Kubeflow is an open source offering from Google.

Seldon Core helps to deploy pickled SKLearn models. Because the models are pickled, any pre-processing of the input must be done before calling it. For example, you'd call the served model with something like:

curl -s -d '{"data": {"ndarray":[[1.0, 2.0, 5.0, 6.0]]}}' \ -X POST http://localhost:8003/seldon/seldon/sklearn/api/v1.0/predictions \ -H "Content-Type: application/json"

(from here).

Development Tools

Apache TVM is an "An end to end Machine Learning Compiler Framework for CPUs, GPUs and accelerators".

DVC is a Git based tool for versioning data.

[1] Designing Machine Learning Systems, Chip Huyen