Online evaluation of ML systems
In this tutorial, we will practice selected techniques for evaluating machine learning systems, and then monitoring them in production. It is one of a 3-part series:
- Offline evaluation of ML systems
- Online evaluation of ML systems (this part!)
- Evaluation of ML systems by closing the feedback loop
In this particular section, we will practice evaluation and monitoring in the online stage - when a system is serving some or all real users.
Follow along at Online evaluation of ML systems.
This tutorial uses: one m1.medium VM at KVM@TACC, and one floating IP.
This material is based upon work supported by the National Science Foundation under Grant No. 2230079.
Launching this artifact will open it within Chameleon’s shared Jupyter experiment environment, which is accessible to all Chameleon users with an active allocation.
Download ArchiveDownload an archive containing the files of this artifact.
Download with git
Clone the git repository for this artifact, and checkout the version's commit
git clone https://github.com/teaching-on-testbeds/eval-online-chi
# cd into the created directory
git checkout 4c7078cbd22790ff650145ece32a52354889746bSubmit feedback through GitHub issues