Experiment Create

https://api.sigopt.com/v1/experiments
Creates a new Experiment.
Request Method: POST
Parameters
Name
Type
Required?
Description
name
string
Y
A user-specified name for this experiment.
parameters
array<Parameter>
Y
An array of Parameter objects.
conditionals
array<Conditional>
N
See conditionals.
linear_constraints
array<Parameter Constraint>
N
See constraints.
metadata
​Metadata​
N
Optional user-provided object. See Using Metadata for more information.
metrics
array<Metric>
N
An array of Metric objects to be optimized or stored for analysis. If the array is of length one, the standard optimization problem is conducted. This array can have no more than 2 optimized entries and no more than 50 entries in total.
num_solutions
int
N
The number of (diverse) solutions SigOpt will search for. This feature is only available for special plans, and does not need to be set unless the desired number of solutions is greater than 1. An observation budget is required if the number of solutions is greater than 1. No categorical variables are allowed in multiple solution experiments.
observation_budget
int
N
The number of Observations you plan to create for this experiment. This is required when the length of metrics is greater than 1, and optional for a single metric experiment. Deviating from this value, especially by failing to reach it, may result in suboptimal performance for your experiment.
parallel_bandwidth
int
N
The number of simultaneously open Suggestions you plan to maintain during this experiment. The default value for this is 1, i.e., a sequential experiment. The maximum value for this is dependent on your plan. This field is optional, but setting it correctly may improve performance.
type
string
N
A type for this experiment. Used for experimental and alpha features only.
Deprecated Parameters
These parameters should no longer be used because there are better alternatives.
Name
Type
Required?
Description
metric
​Metric​
N
Information about the metric that this experiment is optimizing.
Response
​Experiment object.
Examples
All types of parameters, experiment types, and metrics
Python
Bash
Java
experiment = conn.experiments().create(
  name="Support Vector Classifier Accuracy",
  parameters=[
    dict(
      name="degree",
      bounds=dict(
        min=1,
        max=5
        ),
      type="int"
      ),
    dict(
      name="gamma",
      bounds=dict(
        min=0.001,
        max=1
        ),
      type="double"
      ),
    dict(
      name="kernel",
      categorical_values=[
        dict(
          name="rbf"
          ),
        dict(
          name="poly"
          ),
        dict(
          name="sigmoid"
          )
        ],
      type="categorical"
      )
    ],
  metrics=[
    dict(
      name="Accuracy",
      objective="maximize",
      strategy="optimize"
      )
    ],
  observation_budget=60,
  parallel_bandwidth=1,
  type="offline"
  )
EXPERIMENT=curl -s -X POST https://api.sigopt.com/v1/experiments -u "$SIGOPT_API_TOKEN": \
    -H "Content-Type: application/json" -d "`cat experiment_meta.json`"
JSON file defining the Experiment:
experiment_meta.json
{
  "name": "Support Vector Classifier Accuracy",
  "parameters": [
    {
      "name": "degree",
      "bounds": {
        "min": 1,
        "max": 5
      },
      "type": "int"
    },
    {
      "name": "gamma",
      "bounds": {
        "min": 0.001,
        "max": 1
      },
      "type": "double"
    },
    {
      "name": "kernel",
      "categorical_values": [
        {
          "name": "rbf"
        },
        {
          "name": "poly"
        },
        {
          "name": "sigmoid"
        }
      ],
      "type": "categorical"
    }
  ],
  "metrics": [
    {
      "name": "Accuracy",
      "objective": "maximize",
      "strategy": "optimize"
    }
  ],
  "observation_budget": 60,
  "parallel_bandwidth": 1,
  "type": "offline"
}
Experiment experiment = Experiment.create()
  .data(
    new Experiment.Builder()
      .name("Support Vector Classifier Accuracy")
      .parameters(java.util.Arrays.asList(
        new Parameter.Builder()
          .name("degree")
          .bounds(new Bounds.Builder()
            .min(1)
            .max(5)
            .build())
          .type("int")
          .build(),
        new Parameter.Builder()
          .name("gamma")
          .bounds(new Bounds.Builder()
            .min(0.001)
            .max(1)
            .build())
          .type("double")
          .build(),
        new Parameter.Builder()
          .name("kernel")
          .categoricalValues(java.util.Arrays.asList(
            new CategoricalValue(
              "rbf"
              ),
            new CategoricalValue(
              "poly"
              ),
            new CategoricalValue(
              "sigmoid"
              )
            ))
          .type("categorical")
          .build()
        ))
      .metrics(java.util.Arrays.asList(
        new Metric.Builder()
          .name("Accuracy")
          .objective("maximize")
          .strategy("optimize")
          .build()
        ))
      .observationBudget(60)
      .parallelBandwidth(1)
      .type("offline")
      .build()
    )
  .call();
Response
{
  "client": "1",
  "conditionals": [],
  "created": 1414800000,
  "development": false,
  "id": "1",
  "linear_constraints": [],
  "metadata": null,
  "metric": {
    "name": "Accuracy",
    "object": "metric",
    "objective": "maximize",
    "strategy": "optimize",
    "threshold": null
  },
  "metrics": [
    {
      "name": "Accuracy",
      "object": "metric",
      "objective": "maximize",
      "strategy": "optimize",
      "threshold": null
    }
  ],
  "name": "Support Vector Classifier Accuracy",
  "num_solutions": null,
  "object": "experiment",
  "observation_budget": 60,
  "parallel_bandwidth": null,
  "parameters": [
    {
      "bounds": {
        "max": 5,
        "min": 1,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "degree",
      "object": "parameter",
      "precision": null,
      "prior": null,
      "transformation": null,
      "tunable": true,
      "type": "int"
    },
    {
      "bounds": {
        "max": 1,
        "min": 0.001,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "gamma",
      "object": "parameter",
      "precision": null,
      "prior": null,
      "transformation": null,
      "tunable": true,
      "type": "double"
    },
    {
      "bounds": null,
      "categorical_values": [
        {
          "enum_index": 1,
          "name": "rbf",
          "object": "categorical_value"
        },
        {
          "enum_index": 2,
          "name": "poly",
          "object": "categorical_value"
        },
        {
          "enum_index": 3,
          "name": "sigmoid",
          "object": "categorical_value"
        }
      ],
      "conditions": {},
      "default_value": null,
      "name": "kernel",
      "object": "parameter",
      "precision": null,
      "prior": null,
      "transformation": null,
      "tunable": true,
      "type": "categorical"
    }
  ],
  "progress": null,
  "project": "classification-models",
  "runs_only": false,
  "state": "active",
  "type": "offline",
  "updated": 1446422400,
  "user": null
}
​
Two metrics
Python
Bash
Java
experiment = conn.experiments().create(
  name="Profit vs. Robustness",
  parameters=[
    dict(
      name="mixing speed",
      bounds=dict(
        min=0,
        max=3
        ),
      type="double"
      ),
    dict(
      name="personnel",
      bounds=dict(
        min=10,
        max=25
        ),
      type="int"
      )
    ],
  metrics=[
    dict(
      name="profit",
      objective="maximize",
      strategy="optimize"
      ),
    dict(
      name="robustness",
      objective="maximize",
      strategy="optimize"
      )
    ],
  observation_budget=120,
  parallel_bandwidth=1
  )
EXPERIMENT=curl -s -X POST https://api.sigopt.com/v1/experiments -u "$SIGOPT_API_TOKEN": \
    -H "Content-Type: application/json" -d "`cat experiment_meta.json`"
JSON file defining the Experiment:
experiment_meta.json
{
  "name": "Profit vs. Robustness",
  "parameters": [
    {
      "name": "mixing speed",
      "bounds": {
        "min": 0,
        "max": 3
      },
      "type": "double"
    },
    {
      "name": "personnel",
      "bounds": {
        "min": 10,
        "max": 25
      },
      "type": "int"
    }
  ],
  "metrics": [
    {
      "name": "profit",
      "objective": "maximize",
      "strategy": "optimize"
    },
    {
      "name": "robustness",
      "objective": "maximize",
      "strategy": "optimize"
    }
  ],
  "observation_budget": 120,
  "parallel_bandwidth": 1
}
Experiment experiment = Experiment.create()
  .data(
    new Experiment.Builder()
      .name("Profit vs. Robustness")
      .parameters(java.util.Arrays.asList(
        new Parameter.Builder()
          .name("mixing speed")
          .bounds(new Bounds.Builder()
            .min(0)
            .max(3)
            .build())
          .type("double")
          .build(),
        new Parameter.Builder()
          .name("personnel")
          .bounds(new Bounds.Builder()
            .min(10)
            .max(25)
            .build())
          .type("int")
          .build()
        ))
      .metrics(java.util.Arrays.asList(
        new Metric.Builder()
          .name("profit")
          .objective("maximize")
          .strategy("optimize")
          .build(),
        new Metric.Builder()
          .name("robustness")
          .objective("maximize")
          .strategy("optimize")
          .build()
        ))
      .observationBudget(120)
      .parallelBandwidth(1)
      .build()
    )
  .call();
Response
{
  "client": "1",
  "conditionals": [],
  "created": 1414800000,
  "development": false,
  "id": "2",
  "linear_constraints": [],
  "metadata": null,
  "metric": null,
  "metrics": [
    {
      "name": "profit",
      "object": "metric",
      "objective": "maximize",
      "strategy": "optimize",
      "threshold": null
    },
    {
      "name": "robustness",
      "object": "metric",
      "objective": "maximize",
      "strategy": "optimize",
      "threshold": null
    }
  ],
  "name": "Profit vs. Robustness",
  "num_solutions": null,
  "object": "experiment",
  "observation_budget": 120,
  "parallel_bandwidth": 1,
  "parameters": [
    {
      "bounds": {
        "max": 3,
        "min": 0,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "mixing speed",
      "object": "parameter",
      "precision": null,
      "prior": null,
      "transformation": null,
      "tunable": true,
      "type": "double"
    },
    {
      "bounds": {
        "max": 25,
        "min": 10,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "personnel",
      "object": "parameter",
      "precision": null,
      "prior": null,
      "transformation": null,
      "tunable": true,
      "type": "int"
    }
  ],
  "progress": null,
  "project": "classification-models",
  "runs_only": false,
  "state": "active",
  "type": "offline",
  "updated": 1446422400,
  "user": null
}
​
Two solutions
Python
Bash
Java
experiment = conn.experiments().create(
  name="Classifier Accuracy",
  parameters=[
    dict(
      name="layer_size",
      bounds=dict(
        min=10,
        max=100
        ),
      type="int"
      ),
    dict(
      name="learning_rate",
      bounds=dict(
        min=0.001,
        max=1
        ),
      type="double"
      )
    ],
  metrics=[
    dict(
      name="Accuracy",
      objective="maximize",
      strategy="optimize"
      )
    ],
  num_solutions=2,
  observation_budget=60,
  parallel_bandwidth=1
  )
EXPERIMENT=curl -s -X POST https://api.sigopt.com/v1/experiments -u "$SIGOPT_API_TOKEN": \
    -H "Content-Type: application/json" -d "`cat experiment_meta.json`"
JSON file defining the Experiment:
{
  "name": "Classifier Accuracy",
  "parameters": [
    {
      "name": "layer_size",
      "bounds": {
        "min": 10,
        "max": 100
      },
      "type": "int"
    },
    {
      "name": "learning_rate",
      "bounds": {
        "min": 0.001,
        "max": 1
      },
      "type": "double"
    }
  ],
  "metrics": [
    {
      "name": "Accuracy",
      "objective": "maximize",
      "strategy": "optimize"
    }
  ],
  "num_solutions": 2,
  "observation_budget": 60,
  "parallel_bandwidth": 1
}
Experiment experiment = Experiment.create()
  .data(
    new Experiment.Builder()
      .name("Classifier Accuracy")
      .parameters(java.util.Arrays.asList(
        new Parameter.Builder()
          .name("layer_size")
          .bounds(new Bounds.Builder()
            .min(10)
            .max(100)
            .build())
          .type("int")
          .build(),
        new Parameter.Builder()
          .name("learning_rate")
          .bounds(new Bounds.Builder()
            .min(0.001)
            .max(1)
            .build())
          .type("double")
          .build()
        ))
      .metrics(java.util.Arrays.asList(
        new Metric.Builder()
          .name("Accuracy")
          .objective("maximize")
          .strategy("optimize")
          .build()
        ))
      .numSolutions(2)
      .observationBudget(60)
      .parallelBandwidth(1)
      .build()
    )
  .call();
Response
{
  "client": "1",
  "conditionals": [],
  "created": 1414800000,
  "development": false,
  "id": "2",
  "linear_constraints": [],
  "metadata": null,
  "metric": {
    "name": "Accuracy",
    "object": "metric",
    "objective": "maximize",
    "strategy": "optimize",
    "threshold": null
  },
  "metrics": [
    {
      "name": "Accuracy",
      "object": "metric",
      "objective": "maximize",
      "strategy": "optimize",
      "threshold": null
    }
  ],
  "name": "Classifier Accuracy",
  "num_solutions": 2,
  "object": "experiment",
  "observation_budget": 60,
  "parallel_bandwidth": 1,
  "parameters": [
    {
      "bounds": {
        "max": 100,
        "min": 10,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "layer_size",
      "object": "parameter",
      "precision": null,
      "prior": null,
      "transformation": null,
      "tunable": true,
      "type": "int"
    },
    {
      "bounds": {
        "max": 1,
        "min": 0.001,
        "object": "bounds"
      },
      "categorical_values": null,
      "conditions": {},
      "default_value": null,
      "name": "learning_rate",
      "object": "parameter",
      "precision": null,
      "prior": null,
      "transformation": null,
      "tunable": true,
      "type": "double"
    }
  ],
  "progress": null,
  "project": "classification-models",
  "runs_only": false,
  "state": "active",
  "type": "offline",
  "updated": 1446422400,
  "user": null
}
​
​

Name	Type	Required?	Description
`name`	`string`	Y	A user-specified name for this experiment.
`parameters`	`array<Parameter>`	Y	An array of Parameter objects.
`conditionals`	`array<Conditional>`	N	See conditionals.
`linear_constraints`	`array<Parameter Constraint>`	N	See constraints.
`metadata`	`Metadata`	N	Optional user-provided object. See Using Metadata for more information.
`metrics`	`array<Metric>`	N	An array of Metric objects to be optimized or stored for analysis. If the array is of length one, the standard optimization problem is conducted. This array can have no more than 2 optimized entries and no more than 50 entries in total.
`num_solutions`	`int`	N	The number of (diverse) solutions SigOpt will search for. This feature is only available for special plans, and does not need to be set unless the desired number of solutions is greater than 1. An observation budget is required if the number of solutions is greater than 1. No categorical variables are allowed in multiple solution experiments.
`observation_budget`	`int`	N	The number of Observations you plan to create for this experiment. This is required when the length of metrics is greater than 1, and optional for a single metric experiment. Deviating from this value, especially by failing to reach it, may result in suboptimal performance for your experiment.
`parallel_bandwidth`	`int`	N	The number of simultaneously open Suggestions you plan to maintain during this experiment. The default value for this is 1, i.e., a sequential experiment. The maximum value for this is dependent on your plan. This field is optional, but setting it correctly may improve performance.
`type`	`string`	N	A type for this experiment. Used for experimental and alpha features only.

Name	Type	Required?	Description
`metric`	`Metric`	N	Information about the metric that this experiment is optimizing.

Experiment Best Assignments

Experiment Delete

Last modified 9mo ago