sigopt.com
Log In / Sign Up
Search…
Welcome to SigOpt!
Quick Start Tutorials
Tracking Your Training Runs
Optimization Experiments
XGBoost Integration
HyperOpt
Advanced Experimentation
SigOpt Orchestrate
Install SigOpt
Orchestrate a Tracked Training Run
Orchestrate an HPO Experiment
AWS Cluster Create and Manage
SigOpt: Bring Your Own Cluster
Dockerfile: Define Your Environment
Debugging
CLI Reference
API Reference
Support
Archived Docs
Powered By
GitBook
Debugging
When initially setting up your environment, it can be helpful to have a few debugging commands handy.
Jobs
Firstly, check to see if your Kubernetes Jobs started correctly:
1
$ sigopt cluster kubectl get
jobs
Copied!
Output:
1
NAME. COMPLETIONS DURATION AGE
2
experiment-controller-999999 0/1 2m22s
Copied!
If you see Jobs that have
0/1
Completions and no Duration, there's likely an issue with the Job.
You can then get more detailed information about the specific job:
1
$ sigopt cluster kubectl describe jobs/ experiment-controller-999999
Copied!
This will contain a lot of information about the Job, and list any errors at the bottom.
If the Job has started successfully but you're still not getting results, next check the Pod(s) with a similar process.
Pods
1
$ sigopt cluster kubectl get pods
Copied!
Output:
1
NAME READY STATUS RESTARTS AGE
2
experiment-controller-999999-xxxxx 0/1 ImagePullBackOff 0 5m
Copied!
A Pod Status such as
ImagePullBackOff
might indicate an issue with credentials not being properly loaded into Kubernetes, or trying to access the wrong registry.
More detailed information about the Pod and any errors can be queried:
1
$ sigopt cluster kubectl describe pods/ experiment-controller-999999-xxxxx
Copied!
Previous
Dockerfile: Define Your Environment
Next
CLI Reference
Last modified
1mo ago
Copy link
Contents
Jobs
Pods