Use the Model Monitor to configure monitoring for models that aren’t deployed on Model APIs or for those models deployed outside Domino.
The following topics explain the steps:
-
Install packages for Model Monitoring. Domino 5.0 requires packages that are specific to the 5.0 release.
-
Connect a Data Source. Connect to external Data Sources to access your training, prediction, and ground truth data.
-
Register a Model. Register external models so that Domino can capture information about them through their Monitoring Config JSON files.
-
Set up Drift Detection. Add data that was trained on historical data to your model’s code to start ingesting and storing prediction data for monitoring.
-
Optional: Configure notifications or change the scheduled checks.
-
-
Analyze data drift. See the divergence value for features and experiment with test types, thresholds, and other conditions.
-
Optional: Configure notifications or change the scheduled checks.
-
-
Set up Model Quality Monitoring. Ingest ground truth data to monitor the quality of the model’s predictions.
-
Optional: Configure notifications or change the scheduled checks.
-
-
Set up Cohort Analysis. Get insights into model quality so you can find underperforming data hotspots for model remediation.