The entire lifecycle of Machine Learning from preparing data to deploying and monitoring the model artifacts for training and inference is called MLOps.
What is MLOps and Why is it needed?
Let's take an analogy of the development of Computer Science. Computers are a marvel of human inventions and it is almost impossible to understand and work on all of its associated components at the same time. But all of the advancement in hardware and software and other collaborative fields is a huge success in itself in the form of abstraction layers. For each component there is an abstraction layer that is arranged in hierarchical order or other topologies.
Similarly the level of complexity in whole machine learning is increasing in a way that it is further divided into various roles and fields like data engineering,data science, ml engineering. One among such fields which is new and growing is MLOps. The goal is to simplify the pipeline of the whole ML workflow.
Components of MLOps:-
As individual fields; software engineering, machine learning and devops are well established in the industry with the standard tools and technology for each of them. Integrating all of these technologies and tools for the purpose of ML workflow lies at the heart of MLOps.
Key Challenges in deployment of machine learning:-
- Models blocked before deployment
- Wasting time
- Efficient collaboration
- Manual Tracking
- No reproducibility or provenance
- Unmonitored models
Difference between perception and reality of Machine Learning models:-
It is a common perception that machine learning is about building complicated models and rigorous mathematics involved underneath it but the reality is that model building is just one of the tasks among many to take it before serving to customers. Few of these tasks are Configuration, Data Validation, Monitoring, Analysis tools, Resource management ,Serving infrastructure and Data collection. On an average the time consumed in building a machine learning model takes just 15-20% of whole ml pipeline time.
How is it different from DevOps?
The challenges faced while building a software that include machine learning model vs software t are quite different in nature. Software development is focused on code commits but in machine learning there are many parts in momentum like data, models and metrics. There is a need to continuously train the models on new data and keep tracking the changes in code as well for which git is commonly used. It is a distributed version control system designed for small to big projects. There are many other tools for version control but ML pipeline is more than version control of code itself .The following diagram explain the difference:-
Machine learning lifecycle:- On a high level the lifecycle can be divided into three components:-
- Data Engineering:- Data gathering, storing data and preparing data
- Machine learning model engineering:- Building ML model and serving.
- Code Engineering:- Using ML model inside the business application.
Deployment Modes:-
There are few common modes of ML model deployment based upon how they are going to serve the end user.
- Embedded Architecture:-The model is embedded inside a software code . The model is invoked by software to infer as and when required.
- Dedicated model API:- The model is saved as an api . The end user can use the api and integrate it into his application.
- Model published as Data:- Various models for the same task are served as data so that they can be run by the end user in their device only and used. This is particularly useful in edge deployment.
- Offline/Batch Prediction:- This deployment model is typically used in scientific applications and research where the model is giving inference on a batch. It does not predict on the fly and has very high latency.
Model monitoring:-
The ML model deployment is not a one time task. Depending upon the factors like data quality and model quality the models are monitored to ensure that the quality of inferences are up to the mark. There are several factors which can deteriorate the data and model performance like Data drift, broken pipelines, schema changes, data outage, underperforming segments, model bias, concept drift, model accuracy.
Let's take an example of data drift: the distribution of data over which it is trained and over what it is inferred should be the same but if the distribution underlying the data changes then the model needs to be retrained on new data. This is called data drift.
The change in relationship between input and output data can cause concept drift .For example the sales of loungewear suddenly hike when a national lockdown happens. So in some situations where the new parameters can affect the outcomes model will perform poorly.
Choosing the right performance metric for model monitoring:- The measure of performance to find how good a model is performing depends upon the problem. In some cases the 99% accuracy is an excellent measure while in another it is unacceptable.
E2E Cloud is here to help you in deploying your ML models. We have kubernetes as a service cluster and GPU’s which can give a cost saving up to 50% than hyperscalers.
Want more information? Contact us: sales@e2enetworks.com
Request for a free trial:
