Define and Set Up Parameter Space

You can define the parameter space for any SigOpt Experiment using four types of parameters: double, integers, categorical, and grid value parameters. Each of the parameters is defined as its own object and the entire parameter space is defined as a list of Parameter objects. You can define up to 100 parameters.

Continuous Parameters

A continuous parameter, also referred to as double, can take on any value between the minimum and maximum values specified by the bounds.

Python

dict(
  name="parameter",
  bounds=dict(
    min=0,
    max=1
  ),
  type="double"
)

YAML

parameters:
  - name: parameter
    bounds:
      min: 0
      max: 1
    type: double

Log Transformation of Continuous Parameter

It is also possible to have the optimizer search any continuous parameter in a log space by invoking logarithmic transformation. SigOpt assumes base 10 when applying the transformation.

Python

dict(
  name="log_parameter",
  bounds=dict(
    min=0.001,
    max=1
  ),
  transformation="log",
  type="double"
)

YAML

parameters:
  - name: log_parameter
    bounds:
      min: 0.001
      max: 1
    transformation: log
    type: double

Integer Parameters

An integer parameter is similar to a double parameter, but it can only take on integer values in-between the bounds.

Python

dict(
  name="parameter",
  bounds=dict(
    min=0,
    max=10
  ),
  type="int"
)

YAML

parameters:
  - name: parameter
    bounds:
      min: 0
      max: 10
    type: int

Categorical Parameters

A categorical parameter is a finite set of options to choose from. Usually the categorical values have non-ordinal relationship.

Python

dict(
  name="parameter",
  categorical_values=[
    "val_a",
    "val_b",
    "val_c"
  ],
  type="categorical"
)

YAML

parameters:
  - name: parameter
    categorical_values:
      - val_a
      - val_b
      - val_c
    type: categorical

Limitations: A SigOpt Experiment can only have a total of 10 categorical values across all categorical parameters.

Grid Values

Grid values are any continuous and integer parameter being discretized into a set of values. Grid values can be any set of numerical values and do not need to be evenly spaced.

Example 1

Python

dict(
  name="parameter",
  type="double",
  grid=[
    0.00001,
    0.001,
    0.33,
    0.999
  ]
)

YAML

parameters:
  - name: parameter
    type: double
    grid:
      - 0.00001
      - 0.001
      - 0.33
      - 0.999

Example 2

Python

dict(
  name="parameter",
  type="int",
  grid=[
    3,
    4,
    7,
    10
  ]
)

YAML

parameters:
  - name: parameter
    type: int
    grid:
      - 3
      - 4
      - 7
      - 10

Specifying the grid values is not the same as conducting grid search. Unlike grid search, every grid value is not guaranteed to appear in the suggestions generated by SigOpt. In other words, instead of exhaustively searching through all possible grid values, SigOpt intelligently suggests the most promising grid value. This is especially efficient when the user defines a grid on many Parameters.

Log Transformation of Grid Values

Grid values can be used in conjunction with log transformation for parameters that should be searched in the log space.

Python

dict(
  name="parameter",
  type="double",
  grid=[
    0.00001,
    0.001,
    0.33,
    0.999
  ],
  transformation="log"
)

YAML

parameters:
  - name: parameter
    type: double
    grid:
      - 0.00001
      - 0.001
      - 0.33
      - 0.999
    transformation: log

Define Prior Distributions

When defining a continuous parameter, it is possible to specify it with a prior distribution of the likely "good" parameter values. When the prior belief is set for a parameter, SigOpt is more likely to generate suggestions from regions of this parameter with high probability density (pdf) value. For more information on how to define these distributions visit the following documentation on prior beliefs.

Example 1: Beta distribution

Python

dict(
  name="name",
  bounds=dict(
    min=0,
    max=1
  ),
  prior=dict(
    name="beta",
    shape_a=2,
    shape_b=4.5
  ),
  type="double"
)

YAML

parameters:
  - name: name
    bounds:
      min: 0
      max: 1
    prior:
      name: beta
      shape_a: 2
      shape_b: 4.5
    type: double

Example 2: Normal distribution

Python

dict(
  name="name",
  bounds=dict(
    min=0,
    max=1
  ),
  prior=dict(
    name="normal",
    mean=0.6,
    scale=0.15
  ),
  type="double"
)

YAML

parameters:
  - name: name
    bounds:
      min: 0
      max: 1
    prior:
      name: normal
      mean: 0.6
      scale: 0.15
    type: double

Define Conditional Parameters

Conditional parameters allow you to define dependencies between parameters. When a dependency is satisfied, SigOpt will recognize this and provide parameter assignments for dependent parameters. Here's how to define conditional parameters in your Experiment:

Python

dict(
  conditionals=[
    dict(
      name="num_conv_layers",
      values=[
        "1",
        "2",
        "3"
      ]
    )
  ],
  parameters=[
    dict(
      name="layer_2_num_filters",
      conditions=dict(
        num_conv_layers=[
          "2",
          "3"
        ]
      ),
      type="int",
      bounds=dict(
        min=100,
        max=300
      )
    )
  ]
)

YAML

conditionals:
  - name: num_conv_layers
    values:
      - "1"
      - "2"
      - "3"
parameters:
  - name: layer_2_num_filters
    conditions:
      num_conv_layers:
        - "2"
        - "3"
    type: int
    bounds:
      min: 100
      max: 300

In the above example, SigOpt will provide parameter assignments for layer_2_num_filters when num_conv_layers == 2 or num_conv_layers == 3.