LogoLogo
sigopt.comLog In / Sign Up
  • Welcome to SigOpt!
    • SigOpt API Modules
    • Main Concepts
      • Define and Set Up Parameter Space
      • Define and Set Up Metric Space
      • Alternative Experiment Types
  • Advanced Experimentation
    • Multimetric Optimization
    • Metric Thresholds
    • Metric Constraints
    • Metric Failure
    • All Constraints Experiment
    • Multisolution
    • Parallelism
    • Parameter Constraints
    • Prior Beliefs
    • Multitask Experiments
  • Core Module API References
    • Installation and Setup
    • Quick Start
    • API Topics
      • API Tokens and Authentication
      • API Errors
      • Manage Open Suggestions
      • Metadata
      • Pagination
    • API Endpoints
      • Client Detail
      • Experiment Best Assignments
      • Experiment Create
      • Experiment Delete
      • Experiment Detail
      • Experiment Metric Importances
      • Experiment Stopping Criteria
      • Experiment Update
      • Observation Create
      • Observation Batch Create
      • Observation Delete
      • Observation Detail
      • Observation List Delete
      • Observation List
      • Observation Update
      • Queued Suggestion Create
      • Queued Suggestion Delete
      • Queued Suggestion Detail
      • Queued Suggestion List
      • Suggestion Create
      • Suggestion Delete
      • Suggestion Detail
      • Suggestion List Delete
      • Suggestion List
      • Experiment Token Create
      • Organization Detail
      • Suggestion Update
    • API Objects
      • Assignments Object
      • Best Assignments Object
      • Bounds Object
      • Categorical Value Object
      • Client Object
      • Conditional Object
      • Conditions Object
      • Constraint Term Object
      • Experiment Object
      • Metadata Object
      • Metric Object
      • Metric Evaluation Object
      • Metric Importances Object
      • Observation Object
      • Organization Object
      • Pagination Object
      • Parameter Object
      • Parameter Constraint Object
      • Plan Object
      • Plan Period Object
      • Plan Rules Object
      • Prior Object
      • Progress Object
      • Queued Suggestion Object
      • Stopping Criteria Object
      • Suggestion Object
      • Token Object
  • AI MODULE API REFERENCES
    • Installation and Setup
    • Quick Start Tutorials
      • Run Tutorial
      • Project Tutorial
      • AI Experiment and Optimization Tutorial
    • Tracking Your Training Runs
      • Set Up for Example Code
      • Record SigOpt Runs in Jupyter
      • Record SigOpt Runs with Python IDE and SigOpt CLI
      • Record SigOpt Runs with Python IDE
      • View and Analyze the Recorded SigOpt Run
      • Enable Optimization for your SigOpt Runs
    • AI Experiments
      • AI Experiment Set Up
    • Bring Your Own Optimizer
    • XGBoost Integration
      • Installation
      • Tracking XGBoost Training
      • Tuning XGBoost Models
    • HyperOpt
      • Installation
      • User Case
      • API
    • SigOpt Orchestrate
      • Install SigOpt Orchestrate
      • Orchestrate a Tracked Training Run
      • Orchestrate an AI Experiment
      • AWS Cluster Create and Manage
      • SigOpt: Bring Your Own Cluster
      • Dockerfile: Define Your Environment
      • Debugging
      • CLI Reference
    • API Reference
      • Manually create a SigOpt Run
      • Tracking a Run
      • AI Experiment Client Calls
      • Project Client Calls
      • Objects
        • Training Run Object
        • AI Experiment Object
        • Parameter Object
        • Metric Object
      • SigOpt CLI Commands
      • Orchestrate CLI Commands
  • Support
    • Support
    • FAQ
    • Best Practices
      • Setting an experiment budget
      • Reproducibility in SigOpt
      • Uploading data as a Training Run artifact
      • Navigating multiple metrics
Powered by GitBook
On this page
  • Jobs
  • Pods
  1. AI MODULE API REFERENCES
  2. SigOpt Orchestrate

Debugging

When initially setting up your environment, it can be helpful to have a few debugging commands handy.

Jobs

Firstly, check to see if your Kubernetes Jobs started correctly:

$ sigopt cluster kubectl get jobs

Output:

NAME.                          COMPLETIONS   DURATION   AGE
experiment-controller-999999   0/1                      2m22s

If you see Jobs that have 0/1 Completions and no Duration, there's likely an issue with the Job.

You can then get more detailed information about the specific job:

$ sigopt cluster kubectl describe jobs/ experiment-controller-999999

This will contain a lot of information about the Job, and list any errors at the bottom.

If the Job has started successfully but you're still not getting results, next check the Pod(s) with a similar process.

Pods

$ sigopt cluster kubectl get pods

Output:

NAME                                 READY   STATUS               RESTARTS   AGE
experiment-controller-999999-xxxxx   0/1     ImagePullBackOff     0          5m

A Pod Status such as ImagePullBackOff might indicate an issue with credentials not being properly loaded into Kubernetes, or trying to access the wrong registry.

More detailed information about the Pod and any errors can be queried:

$ sigopt cluster kubectl describe pods/ experiment-controller-999999-xxxxx
PreviousDockerfile: Define Your EnvironmentNextCLI Reference

Last updated 2 years ago