trains.automation.hpbandster.bandster.
OptimizerBOHB
¶Initialize a BOHB search strategy optimizer BOHB performs robust and efficient hyperparameter optimization at scale by combining the speed of Hyperband searches with the guidance and guarantees of convergence of Bayesian Optimization. Instead of sampling new configurations at random, BOHB uses kernel density estimators to select promising candidates.
For reference:
@InProceedings{falkner-icml-18,
title = {{BOHB}: Robust and Efficient Hyperparameter Optimization at Scale},
author = {Falkner, Stefan and Klein, Aaron and Hutter, Frank},
booktitle = {Proceedings of the 35th International Conference on Machine Learning},
pages = {1436--1445},
year = {2018},
}
base_task_id (str) – Task ID (str)
hyper_parameters (list) – The list of Parameter objects to optimize over.
objective_metric (Objective) – Objective metric to maximize / minimize.
execution_queue (str) – The execution queue to use for launching Tasks (experiments).
num_concurrent_workers (int) – The maximum number of concurrent running Tasks (machines).
min_iteration_per_job (int) – The minimum number of iterations for a job to run, where iterations are the reported iterations for the specified objective, not the maximum reported iteration of the Task.
max_iteration_per_job (int) – The number of iterations per job, where iterations are the reported iterations for the specified objective, not the maximum reported iteration of the Task.
total_max_jobs (int) – The total maximum number of jobs for the optimization process.
Must be provided in order to calculate the total budget for the optimization process.
The total budget is measured by “iterations” (see above), and will be set to max_iteration_per_job
*
total_max_jobs
. This means more than total_max_jobs
could be created, as long as the cumulative
iterations (summed over all created jobs) will not exceed max_iteration_per_job
* total_max_jobs
.
pool_period_min (float) – The time in minutes between two consecutive pools.
time_limit_per_job (float) – Optional, maximum execution time per single job in minutes, when time limit is exceeded job is aborted.
local_port (int) – Default port 9090 tcp, this is a must for the BOHB workers to communicate, even locally.
bohb_kwargs – Arguments passed directly to the BOHB object.
create_job
()¶Abstract helper function. Implementation is not required. Default use in process_step
default implementation
Create a new job if needed. return the newly created job. If no job needs to be created, return None
.
A Newly created TrainsJob object, or None if no TrainsJob created.
get_created_jobs_ids
()¶Return a Task IDs dict created by this optimizer until now, including completed and running jobs. The values of the returned dict are the parameters used in the specific job
dict of task IDs (str) as keys, and their parameters dict as values.
get_created_jobs_tasks
()¶Return a Task IDs dict created by this optimizer until now. The values of the returned dict are the TrainsJob.
dict of task IDs (str) as keys, and their TrainsJob as values.
get_objective_metric
()¶Return the metric title, series pair of the objective.
(title, series)
get_random_seed
()¶Get the global seed for all hyper-parameter strategy random number sampling.
The random seed.
get_running_jobs
()¶Return the current running TrainsJobs.
List of TrainsJob objects.
get_top_experiments
(top_k)¶Return a list of Tasks of the top performing experiments, based on the controller Objective
object.
top_k (int) – The number of Tasks (experiments) to return.
A list of Task objects, ordered by performance, where index 0 is the best performing Task.
helper_create_job
(base_task_id, parameter_override=None, task_overrides=None, tags=None, parent=None, **kwargs)¶Create a Job using the specified arguments, TrainsJob
for details.
A newly created Job instance.
monitor_job
(job)¶Helper function, Implementation is not required. Default use in process_step
default implementation.
Check if the job needs to be aborted or already completed.
If returns False
, the job was aborted / completed, and should be taken off the current job list
If there is a budget limitation, this call should update
self.budget.compute_time.update
/ self.budget.iterations.update
job (TrainsJob) – A TrainsJob
object to monitor.
False, if the job is no longer relevant.
process_step
()¶Abstract helper function. Implementation is not required. Default use in start default implementation
Main optimization loop, called from the daemon thread created by start
.
Call monitor job on every TrainsJob
in jobs:
Check the performance or elapsed time, and then decide whether to kill the jobs.
Call create_job:
Check if spare job slots exist, and if they do call create a new job based on previous tested experiments.
True, if continue the optimization. False, if immediately stop.
set_job_class
(job_class)¶Set the class to use for the helper_create_job
function.
job_class (TrainsJob) – The Job Class type.
set_job_default_parent
(job_parent_task_id)¶Set the default parent for all Jobs created by the helper_create_job
method.
job_parent_task_id (str) – The parent Task ID.
set_job_naming_scheme
(naming_function)¶Set the function used to name a newly created job.
naming_function (callable) –
naming_functor(base_task_name, argument_dict) -> str
set_optimization_args
(eta=3, min_budget=None, max_budget=None, min_points_in_model=None, top_n_percent=15, num_samples=None, random_fraction=0.3333333333333333, bandwidth_factor=3, min_bandwidth=0.001)¶Defaults copied from BOHB constructor, see details in BOHB.__init__
BOHB performs robust and efficient hyperparameter optimization at scale by combining the speed of Hyperband searches with the guidance and guarantees of convergence of Bayesian Optimization. Instead of sampling new configurations at random, BOHB uses kernel density estimators to select promising candidates.
For reference:
@InProceedings{falkner-icml-18, title = {{BOHB}: Robust and Efficient Hyperparameter Optimization at Scale}, author = {Falkner, Stefan and Klein, Aaron and Hutter, Frank}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {1436--1445}, year = {2018}, }
- etafloat (3)
In each iteration, a complete run of sequential halving is executed. In it, after evaluating each configuration on the same subset size, only a fraction of 1/eta of them ‘advances’ to the next round. Must be greater or equal to 2.
- min_budgetfloat (0.01)
The smallest budget to consider. Needs to be positive!
- max_budgetfloat (1)
The largest budget to consider. Needs to be larger than min_budget! The budgets will be geometrically distributed
\(a^2 + b^2 = c^2 /sim /eta^k\) for \(k/in [0, 1, ... , num/_subsets - 1]\).
- min_points_in_model: int (None)
number of observations to start building a KDE. Default ‘None’ means dim+1, the bare minimum.
- top_n_percent: int (15)
percentage ( between 1 and 99, default 15) of the observations that are considered good.
- num_samples: int (64)
number of samples to optimize EI (default 64)
- random_fraction: float (1/3.)
fraction of purely random configurations that are sampled from the prior without the model.
- bandwidth_factor: float (3.)
to encourage diversity, the points proposed to optimize EI, are sampled from a ‘widened’ KDE where the bandwidth is multiplied by this factor (default: 3)
- min_bandwidth: float (1e-3)
to keep diversity, even when all (good) samples have the same value for one of the parameters, a minimum bandwidth (Default: 1e-3) is used instead of zero.
set_optimizer_task
(task)¶Set the optimizer task object to be used to store/generate reports on the optimization process. Usually this is the current task of this process.
task (Task) – The optimizer’s current Task.
set_random_seed
(seed=1337)¶Set global seed for all hyper-parameter strategy random number sampling.
seed (int) – The random seed.