Offline evaluation of ML systems
In this tutorial, we will practice selected techniques for evaluating machine learning systems, and then monitoring them in production. It is one of a 3-part series:
- Offline evaluation of ML systems (this part!)
- Online evaluation of ML systems
- Evaluation of ML systems by closing the feedback loop
In this particular section, we will practice evaluation in the offline testing stage - when the system is not yet serving real users.
Follow along at Offline evaluation of ML systems.
This tutorial uses: one m1.medium VM at KVM@TACC, and one floating IP.
This material is based upon work supported by the National Science Foundation under Grant No. 2230079.
Launching this artifact will open it within Chameleon’s shared Jupyter experiment environment, which is accessible to all Chameleon users with an active allocation.
Download ArchiveDownload an archive containing the files of this artifact.
Download with git
Clone the git repository for this artifact, and checkout the version's commit
git clone https://github.com/teaching-on-testbeds/eval-offline-chi
# cd into the created directory
git checkout 31a35bb19e63771027671decc73094a4a1016b24Submit feedback through GitHub issues