AI Experiment Client Calls

sigopt.create_experiment(...)

Creates a new AI Experiment.

Example for creating an AI Experiment:

experiment = sigopt.create_experiment(
    name="Keras Model Optimization (Python)",
    type="offline",
    parameters=[
        dict(name="hidden_layer_size", type="int", bounds=dict(min=32, max=128)),
        dict(
            name="activation_fn",
            type="categorical",
            categorical_values=["relu", "tanh"],
        ),
    ],
    metrics=[dict(name="holdout_accuracy", objective="maximize")],
    parallel_bandwidth=1,
    budget=30,
)

Once you’ve created an AI Experiment, you are able to loop through an Experiment in two ways:

for run in experiment.loop():
  with run:
    ...
while not experiment.is_finished():
  with experiment.create_run() as run:
    ...

sigopt.get_experiment(experiment_id)

Retrieves an existing AI Experiment.

experiment.create_run()

Creates a new Run in the AI Experiment. Returns a RunContext object to use for tracking Run attributes.

experiment.loop()

Start an AI Experiment loop. Returns an iterator of RunContext objects, used for tracking attributes of each Run in the AI Experiment. The iterator will terminate when the AI Experiment has consumed its entire budget.

experiment.is_finished()

Check if the AI Experiment has consumed its entire budget.

experiment.refresh()

Refresh the AI Experiment attributes.

experiment.get_runs()

Returns an iterator of all the TrainingRuns for an AI Experiment. Method applied to an instance of an AI Experiment object.

experiment.get_best_runs()

Returns an iterator of the best TrainingRuns for an AI Experiment. Method applied to an instance of an AI Experiment object.

experiment.update()

Update experiment parameters during execution.

Example for updating parameter bounds within an AI Experiment

experiment = sigopt.create_experiment(…)
parameters = experiment.parameters
parameters[0].bounds.max = 100 
experiment.update(parameters=parameters)

experiment.archive()

Archives the AI Experiment. All associated Runs will not be archived and can be found on the Project Runs page. Method applied to an instance of an AI Experiment object.

Last updated