automation.optimization.RandomSearch

class trains.automation.optimization.RandomSearch

Random search strategy controller. Random uniform sampling of hyper-parameters.

Initialize a random search optimizer.

Parameters
  • base_task_id (str) – The Task ID.

  • hyper_parameters (list) – The list of Parameter objects to optimize over.

  • objective_metric (Objective) – The Objective metric to maximize / minimize.

  • execution_queue (str) – The execution queue to use for launching Tasks (experiments).

  • num_concurrent_workers (int) – The maximum umber of concurrent running machines.

  • pool_period_min (float) – The time between two consecutive pools (minutes).

  • time_limit_per_job (float) – The maximum execution time per single job in minutes, when time limit is exceeded job is aborted. (Optional)

  • compute_time_limit (float) – The maximum compute time in minutes. When time limit is exceeded, all jobs aborted. (Optional)

  • max_iteration_per_job (int) – The maximum iterations (of the Objective metric) per single job. When exceeded, the job is aborted.

  • total_max_jobs (int) – The total maximum number of jobs for the optimization process. The default is None, for unlimited.

create_job()

Create a new job if needed. Return the newly created job. If no job needs to be created, return None.

Returns

A newly created TrainsJob object, or None if no TrainsJob created

get_created_jobs_ids()

Return a Task IDs dict created by this optimizer until now, including completed and running jobs. The values of the returned dict are the parameters used in the specific job

Returns

dict of task IDs (str) as keys, and their parameters dict as values.

get_created_jobs_tasks()

Return a Task IDs dict created by this optimizer until now. The values of the returned dict are the TrainsJob.

Returns

dict of task IDs (str) as keys, and their TrainsJob as values.

get_objective_metric()

Return the metric title, series pair of the objective.

Returns

(title, series)

get_running_jobs()

Return the current running TrainsJobs.

Returns

List of TrainsJob objects.

get_top_experiments(top_k)

Return a list of Tasks of the top performing experiments, based on the controller Objective object.

Parameters

top_k (int) – The number of Tasks (experiments) to return.

Returns

A list of Task objects, ordered by performance, where index 0 is the best performing Task.

helper_create_job(base_task_id, parameter_override=None, task_overrides=None, tags=None, parent=None, **kwargs)

Create a Job using the specified arguments, TrainsJob for details.

Returns

A newly created Job instance.

monitor_job(job)

Helper function, Implementation is not required. Default use in process_step default implementation. Check if the job needs to be aborted or already completed.

If returns False, the job was aborted / completed, and should be taken off the current job list

If there is a budget limitation, this call should update self.budget.compute_time.update / self.budget.iterations.update

Parameters

job (TrainsJob) – A TrainsJob object to monitor.

Returns

False, if the job is no longer relevant.

process_step()

Abstract helper function. Implementation is not required. Default use in start default implementation Main optimization loop, called from the daemon thread created by start.

  • Call monitor job on every TrainsJob in jobs:

    • Check the performance or elapsed time, and then decide whether to kill the jobs.

  • Call create_job:

    • Check if spare job slots exist, and if they do call create a new job based on previous tested experiments.

Returns

True, if continue the optimization. False, if immediately stop.

set_job_class(job_class)

Set the class to use for the helper_create_job function.

Parameters

job_class (TrainsJob) – The Job Class type.

set_job_default_parent(job_parent_task_id)

Set the default parent for all Jobs created by the helper_create_job method.

Parameters

job_parent_task_id (str) – The parent Task ID.

set_job_naming_scheme(naming_function)

Set the function used to name a newly created job.

Parameters

naming_function (callable) –

naming_functor(base_task_name, argument_dict) -> str
 

set_optimizer_task(task)

Set the optimizer task object to be used to store/generate reports on the optimization process. Usually this is the current task of this process.

Parameters

task (Task) – The optimizer’s current Task.

start()

Start the Optimizer controller function loop(). If the calling process is stopped, the controller will stop as well.

Important

This function returns only after the optimization is completed or stop was called.

stop()

Stop the current running optimization loop. Called from a different thread than the start.