Define and Set Up Parameter Space
You can define the hyperparameter space for any SigOpt Experiment using four types of parameters: double, integers, categorical and grid value parameters. You're able to use up to 100 different hyperparameters to define the hyperparameter space.
Each of the hyperparameters are defined as its own object and the entire hyperparameter space is defined in the experiment create call as a list of objects.
If you have a prior probability for any of the hyperparameters, it is possible to give this information to the SigOpt Optimizer as part of the SigOpt Experiment create call.
Continuous Parameters
A continuous parameter also referred to as double can take on any value between the minimum and maximum values specified by the bounds.
Python
YAML
1
dict(
2
name="parameter",
3
bounds=dict(
4
min=0,
5
max=1
6
),
7
type="double"
8
)
Copied!
1
parameters:
2
- name: parameter
3
bounds:
4
min: 0
5
max: 1
6
type: double
Copied!
Log Transformation of Continuous Parameter
It is also possible to have the optimizer search any continuous parameter in a log-space by invoking log-transformation as part of the experiment create call.
Python
YAML
1
dict(
2
name="log_parameter",
3
bounds=dict(
4
min=0.001,
5
max=1
6
),
7
transformation="log",
8
type="double"
9
)
Copied!
1
parameters:
2
- name: log_parameter
3
bounds:
4
min: 0.001
5
max: 1
6
transformation: log
7
type: double
Copied!
Integer Parameters
An Integer parameter is similar to a double parameter, but it can only take on integer values in-between the bounds.
Python
YAML
1
dict(
2
name="parameter",
3
bounds=dict(
4
min=0,
5
max=10
6
),
7
type="int"
8
)
Copied!
1
parameters:
2
- name: parameter
3
bounds:
4
min: 0
5
max: 10
6
type: int
Copied!
Categorical Parameters
A categorical parameter is defined as any parameter with a non-ordinal relation, meaning that the user has to define a finite set of values which the parameter can take.
Python
YAML
1
dict(
2
name="parameter",
3
categorical_values=[
4
"val_a",
5
"val_b",
6
"val_c"
7
],
8
type="categorical"
9
)
Copied!
1
parameters:
2
- name: parameter
3
categorical_values:
4
- val_a
5
- val_b
6
- val_c
7
type: categorical
Copied!
Limitations: A SigOpt Experiment can only have a total of 10 categorical values across all categorical parameters.
Grid Values
Grid values are any continuous and integer parameter being discretized into a set of values.
You are free to define any set of values and it is not necessary for these values to be evenly spaced out.
Example 1
Python
YAML
1
dict(
2
name="parameter",
3
type="double",
4
grid=[
5
0.00001,
6
0.001,
7
0.33,
8
0.999
9
]
10
)
Copied!
1
parameters:
2
- name: parameter
3
type: double
4
grid:
5
- 0.00001
6
- 0.001
7
- 0.33
8
- 0.999
Copied!
Example 2
Python
YAML
1
dict(
2
name="parameter",
3
type="int",
4
grid=[
5
3,
6
4,
7
7,
8
10
9
]
10
)
Copied!
1
parameters:
2
- name: parameter
3
type: int
4
grid:
5
- 3
6
- 4
7
- 7
8
- 10
Copied!
Specifying the grid values is not the same as conducting grid search. Unlike grid search, every grid value is not guaranteed to appear in the suggestions generated by SigOpt. In other words, instead of exhaustively searching through all possible grid values, SigOpt intelligently suggests the most promising grid value. This is especially efficient when the user defines a grid on many Parameters.
Log Transformation of Grid Values
It is possible to log-transform continuous Grid Values.
Python
YAML
1
dict(
2
name="parameter",
3
type="double",
4
grid=[
5
0.00001,
6
0.001,
7
0.33,
8
0.999
9
],
10
transformation="log"
11
)
Copied!
1
parameters:
2
- name: parameter
3
type: double
4
grid:
5
- 0.00001
6
- 0.001
7
- 0.33
8
- 0.999
9
transformation: log
Copied!
Define Prior Distributions
When defining a continuous hyperparameter it is possible to seed this parameter using a prior probability. The parameter is seeded during the SigOpt Experiment create call using either a normal- or beta-distribution. For more information on how to define these distributions visit the following documentation.
Example 1
Python
YAML
1
dict(
2
name="name",
3
bounds=dict(
4
min=0,
5
max=1
6
),
7
prior=dict(
8
name="beta",
9
shape_a=2,
10
shape_b=4.5
11
),
12
type="double"
13
)
Copied!
1
parameters:
2
- name: name
3
bounds:
4
min: 0
5
max: 1
6
prior:
7
name: beta
8
shape_a: 2
9
shape_b: 4.5
10
type: double
Copied!
Example 2
Python
YAML
1
dict(
2
name="name",
3
bounds=dict(
4
min=0,
5
max=1
6
),
7
prior=dict(
8
name="normal",
9
mean=0.6,
10
scale=0.15
11
),
12
type="double"
13
)
Copied!
1
parameters:
2
- name: name
3
bounds:
4
min: 0
5
max: 1
6
prior:
7
name: normal
8
mean: 0.6
9
scale: 0.15
10
type: double
Copied!
Define Conditional Parameters
Conditional parameters allow you to define dependencies between parameters. When a dependency is satisfied, SigOpt will recognize this and provide parameter assignments for dependent parameters. Here's how to define conditional parameters in your Experiment:
Python
YAML
1
dict(
2
conditionals=[
3
dict(
4
name="num_conv_layers",
5
values=[
6
"1",
7
"2",
8
"3"
9
]
10
)
11
],
12
parameters=[
13
dict(
14
name="layer_2_num_filters",
15
conditions=dict(
16
num_hidden_layers=[
17
"2",
18
"3"
19
]
20
),
21
type="int",
22
bounds=dict(
23
min=100,
24
max=300
25
)
26
)
27
]
28
)
Copied!
1
conditionals:
2
- name: num_conv_layers
3
values:
4
- "1"
5
- "2"
6
- "3"
7
parameters:
8
- name: layer_2_num_filters
9
conditions:
10
num_hidden_layers:
11
- "2"
12
- "3"
13
type: int
14
bounds:
15
min: 100
16
max: 300
Copied!
In the above example, SigOpt will provide parameter assignments for
layer_2_num_filters when num_conv_layers == 2 or num_conv_layers == 3.Last modified 1mo ago