Observation Create

https://api.sigopt.com/v1/experiments/EXPERIMENT_ID/observations

Creates a new Observation. Every observation reports on a set of parameters which can come from a Suggestion or manually specified Assignments.

Request Method: POST

Parameters

NameTypeRequired?Description

assignments

N

Manually specify values that the parameters held during this trial. You must specify exactly one value for each parameter in the experiment. Prefer using suggestion.

failed

boolean

N

A boolean indicating whether this observation failed for some reason. SigOpt takes this into consideration when optimizing. It is invalid for both failed to be true, and any of value, value_stddev, or values to be present. The default value is false.

metadata

N

Optional user-provided object. See Using Metadata for more information.

suggestion

string

N

The id of the Suggestion you are reporting on. Preferred over assignments.

values

N

An array of Metric Evaluation objects. For experiments with more than one Metric, you must use this instead of value and value_stddev.

Deprecated Parameters

These parameters should no longer be used because there are better alternatives.

NameTypeRequired?Description

value

float

N

The observed metric value from this trial.

value_stddev

float

N

The standard deviation in value that was observed in this trial.

Response

Observation object.

Examples

Create an Observation

observation = conn.experiments(EXPERIMENT_ID).observations().create(
  suggestion="1",
  values=[
    dict(
      name="accuracy",
      value=0.95
      ),
    dict(
      name="loss",
      value=123.45
      )
    ]
  )
Response
{
  "assignments": {
    "degree": 2,
    "gamma": 3.6,
    "kernel": "rbf"
  },
  "created": 1414800000,
  "experiment": "1",
  "failed": false,
  "id": "1",
  "metadata": null,
  "object": "observation",
  "suggestion": "1",
  "value": 1,
  "value_stddev": null,
  "values": [
    {
      "name": "Accuracy",
      "object": "metric_evaluation",
      "value": 1,
      "value_stddev": null
    }
  ]
}

Create an Observation with custom Assignments

observation = conn.experiments(EXPERIMENT_ID).observations().create(
  assignments=dict(
    degree=2,
    gamma=3.6,
    kernel="rbf"
    ),
  values=[
    dict(
      name="Accuracy",
      value=1
      )
    ]
  )
Response
{
  "assignments": {
    "degree": 2,
    "gamma": 3.6,
    "kernel": "rbf"
  },
  "created": 1414800000,
  "experiment": "1",
  "failed": false,
  "id": "1",
  "metadata": null,
  "object": "observation",
  "suggestion": "1",
  "value": 1,
  "value_stddev": null,
  "values": [
    {
      "name": "Accuracy",
      "object": "metric_evaluation",
      "value": 1,
      "value_stddev": null
    }
  ]
}

Create a Failed Observation

observation = conn.experiments(EXPERIMENT_ID).observations().create(
  failed=True,
  suggestion="2"
  )
Response
{
  "assignments": {
    "degree": 2,
    "gamma": 3.6,
    "kernel": "rbf"
  },
  "created": 1414800000,
  "experiment": "1",
  "failed": true,
  "id": "3",
  "metadata": null,
  "object": "observation",
  "suggestion": "2",
  "value": null,
  "value_stddev": null,
  "values": []
}

Create an Observation with Metadata

observation = conn.experiments(EXPERIMENT_ID).observations().create(
  metadata=dict(
    estimated_training_time_s=302
    ),
  suggestion="1",
  values=[
    dict(
      name="Accuracy",
      value=1,
      value_stddev=0.1
      )
    ]
  )
Response
{
  "assignments": {
    "degree": 2,
    "gamma": 3.6,
    "kernel": "rbf"
  },
  "created": 1414800000,
  "experiment": "1",
  "failed": false,
  "id": "2",
  "metadata": {
    "estimated_training_time_s": 302
  },
  "object": "observation",
  "suggestion": "1",
  "value": 1,
  "value_stddev": 0.1,
  "values": [
    {
      "name": "Accuracy",
      "object": "metric_evaluation",
      "value": 1,
      "value_stddev": 0.1
    }
  ]
}

Last updated