Tracking a Run

Set and Get Run Parameters

There are a few ways you are able to set parameter values for a SigOpt Run. The sigopt.params object is used to track chosen parameter values and to get suggested parameter values for optimization. The sigopt.params object can be used like the Python dict class, allowing you to use methods like .keys(), .items(), .values() and access values using string indexing.

Set Parameter Values

sigopt.params.learning_rate = 0.1
sigopt.params["num_hidden_layers"] = 200
sigopt.params.update({"activation": "sigmoid", "num_epochs": 12})

# set default values for your SigOpt AI Experiment
sigopt.params.setdefault("batch_size", 32)
sigopt.params.setdefaults({"momentum": 0.9, "decay": 0.99})

>>>print(sigopt.params)
{‘learning_rate’:0.1, ‘num_hidden_layers’:200,‘activation’:’sigmoid’, ‘batch_size’:32, ‘momentum’:0.9, ‘decay’:0.99, ‘batch_size’:32, ‘num_epochs’:12}

Get Parameter Values

>>>sigopt.params.learning_rate
0.1

>>>sigopt.params["batch_size"]
32

SigOpt Run Attribute Tracking Methods

sigopt.log_dataset(name)

Logs the dataset name for your Run

Name	Type	Required?	Description
name	String	Yes	The name of the dataset you would like to log.

---

sigopt.log_model(model_type)

Logs the model type for your Run

Name	Type	Required?	Description
name	String	Yes	The name of the model type you would like to log.

---

sigopt.log_checkpoint(checkpoint_values)

Logs metric values for a single checkpoint. To see a chart of checkpoints on the Runs Page, log checkpoints across each epoch of your model's training.

Name	Type	Required?	Description
checkpoint_values	dictionary of {string: number}	Yes	The values of your checkpoints. The keys of each entry are the names of your checkpoints and the values are the numerical values that you would like to log.

---

sigopt.log_metric(name, value, stddev=None)

Logs a metric value for your Run. A metric should be a scalar artifact of your model's training and evaluation. You may repeat this call with unique metric names to log values for many metrics. If you log the same metric multiple times then we will only keep the most recent value.

Name	Type	Required?	Description
name	string	Yes	The name of the metric that you would like to log.
value	number	Yes	The value of the metric to log.
stddev	number	No	The standard deviation of the metric to log.

---

sigopt.log_failure()

Indicates that the Run has failed for any reason. When performing optimization, you will see that a Run for your AI Experiment has also been marked as failed

---

sigopt.log_image(image, name=None)

Uploads an image artifact for your run. See the image argument description for a list of compatible inputs.

Name	Type	Required?	Description
image	string, PIL.Image.Image, matplotlib.figure.Figure or numpy.ndarray	Yes	The image artifact that you would like to log. See notes below for more details.
name	string	Yes	A name for the uploaded image.

Additional notes for image argument:

• If a string is provided then this argument will be treated like a filesystem path and the image will be opened and uploaded. The image type will be inferred from the file extension.

• If a PIL Image is provided then it will be converted to PNG and uploaded.

• If a matplotlib Figure is provided then it will be converted to SVG and uploaded.

• If a numpy array is provided then the values will be clamped to the range [0, 255] and then cast to unsigned 8-bit integers. The resulting array will be converted to PNG and then uploaded.

  • 2D arrays will be interpreted as grayscale images.

  • 3D arrays with a single channel in the last dimension will also be interpreted as grayscale images.

  • 3D arrays with 3 channels in the last dimension will be interpreted as RGB images with the 3 channels representing the red, green and blue intensities respectively.

  • 3D arrays with 4 channels in the last dimension will be interpreted as RGBA images with the 4 channels representing the red, green, blue and alpha intensities respectively.

---

sigopt.log_metadata(key, value)

This stores any extra information about your Run.

Name	Type	Required?	Description
key	string	Yes	The key for the metadata that you would like to log.
value	number,object	Yes	The value of the metadata that you would like to log. If value is not a number then it will be logged as a string.