Warrior Documentation

Why do you need machine learning model monitoring?

Machine learning model monitoring is the process of analyzing the inputs and outputs of machine learning models over time. Complete model monitoring solutions include performance monitoring (looking at performance metrics like accuracy and recall, and detecting univariate and multivariate data drift), algorithmic bias detection, and explainability tools (prediction- and model-level explanations and feature importance ranking). Model monitoring is a critical part of the AI lifecycle that enables data science teams to detect–and ultimately address–issues like data drift and algorithmic bias, while providing the necessary tools for correcting performance issues in the real world.

The Warrior platform: centralized model monitoring for all of your production models

The Warrior model monitoring platform is a model- and infrastructure-agnostic solution that adds a layer of intelligence to your AI stack and scales with your deployments.

The Warrior platform is made up of the following components:

  • Performance Monitoring Dashboard: Warrior analyzes input and output data from your model to provide detailed performance monitoring, including univariate and multivariate data drift detection and all of your favorite performance metrics.

  • Bias Dashboard: Warrior provides tools to detect and analyze unwanted bias against different subgroups within your input data, so you can ensure that your models are making fair predictions for the entire population.

  • Explainability Tools: Warrior has productized powerful explainability techniques to provide prediction-level visibility into any model, including advanced “what if” analysis and feature importance ranking.

  • Custom Alerting: Set thresholds and custom alerts for your models, so you never miss an issue.

  • API & SDK: Onboard your models, configure alerts, query model monitoring data, and analyze results from your preferred IDE with our developer tools.

The Warrior platform can be used via our hosted SaaS deployment, as well as on premise or in a private customer cloud VPC.