AI Experiment and Optimization Tutorial
Last updated
Last updated
We'll walk through an example of instrumenting a model in order to run a model parameter optimization with SigOpt. In this tutorial, you will learn how to:
Install the SigOpt Python client
Set your SigOpt API token
Set the project
Instrument your model
Configure your AI Experiment
Create your first Experiment and optimize your model metric with SigOpt
Visualize your Experiment results
Before starting, make sure you have and installed on your environment.
Install the SigOpt Python package and the libraries required to run the model used for this tutorial.
Runs are created within projects. The project allows you to sort and filter through your Runs and AI Experiments and view useful charts to gain insights into everything you've tried.
The code below is a sample model instrumented with SigOpt where we highlight how to use SigOpt methods to log and track key model information.
Save the lines below in a script called model.py
.
The names of the parameters are expected to match the names of the properties/attributes on sigopt.params
. Similarly the metrics should match the names of the metrics passed to sigopt.log_metric
calls. The budget defines how many Runs you will create for your AI Experiment.
A SigOpt AI Experiment can be configured using a YAML configuration file. Save the lines below in a YAML file called experiment.yml
.
Run the following command to start an experiment using the model from Step 4 and the experiment file from Step 5.
SigOpt will conveniently output links to the AI Experiment and Runs pages on our web application.
Open the AI Experiment link to view your AI Experiment in our web application. Here's a view of the Experiment page once the Experiment is completed.
From the Experiment page, open the History tab to see the list of Runs for your AI Experiment. Click on any individual run ID link to view any completed Run. Here's a view of a Run page:
In this tutorial, we covered the recommended way to instrument and optimize your model, and visualize your results with SigOpt. You learned that experiments are collections of runs that search through a defined parameter space to satisfy the experiment search criteria.
Once you've installed SigOpt, you need to get your API token in order to use the SigOpt API and later explore your Runs and AI Experiments in the SigOpt app. To find your API token, go directly to the .
If you do not have an account, for a free account and get started with SigOpt today.
AI Experiments are created in folders, your AI Experiment will automatically be created in the folder you set in Step 3. The experiment definition also includes a name, parameters (variables that SigOpt will suggest) and metrics. You can also set
Check out our tutorial, , for a closer look at a single Run, and see how to track one-off runs without creating an experiment.