Task Module

class trains.task.Task

The Task class is a code template for a Task object which, together with its connected experiment components, represents the current running experiment. These connected components include hyperparameters, loggers, configuration, label enumeration, models, and other artifacts.

The term “main execution Task” refers to the Task context for current running experiment. Python experiment scripts can create one, and only one, main execution Task. It is traceable, and after a script runs and Trains stores the Task in the Trains Server (backend), it is modifiable, reproducible, executable by a worker, and you can duplicate it for further experimentation.

The Task class and its methods allow you to create and manage experiments, as well as perform advanced experimentation functions, such as autoML.

For detailed information about creating Task objects, see the following methods:

  • Create a new reproducible Task - Task.init

    Important

    In some cases, Task.init may return a Task object which is already stored in Trains Server (already initialized), instead of creating a new Task. For a detailed explanation of those cases, see the Task.init method.

  • Create a new non-reproducible Task - Task.create

  • Get the current running Task - Task.current_task

  • Get another (different) Task - Task.get_task

Warning

Do not construct Task manually! Please use Task.init or Task.get_task

classmethod current_task()

Get the current running Task (experiment). This is the main execution Task (task context) returned as a Task object.

Returns

The current running Task (experiment).

classmethod init(project_name=None, task_name=None, task_type=<TaskTypes.training: 'training'>, reuse_last_task_id=True, output_uri=None, auto_connect_arg_parser=True, auto_connect_frameworks=True, auto_resource_monitoring=True)

A new Task is initialized if one of the following is true:

  • No Task with the same task_name and project_name is stored in Trains Server.

  • A Task with the same task_name and project_name is stored, and the Task stored models and / or artifacts.

  • A Task with the same task_name and project_name is stored, and the Task’s status is Published or Archived.

  • A new Task is forced by calling Task.init with reuse_last_task_id=False.

Otherwise, the already initialized Task object for the same task_name and project_name is returned.

Note

To reference another Task, instead of initializing the same Task more than once, call Task.get_task. For example, to “share” the same experiment in more than one script, call Task.get_task. See the Task.get_task method for an example.

For example:

The first time the following code runs, it will create a new Task. The status will be Completed.

from trains import Task
  task = Task.init('myProject', 'myTask')
  

If this code runs again, it will not create a new Task. It does not store a model or artifact, it is not Published (its status Completed) , it was not Archived, and a new Task is not forced.

If the Task is Published or Archived, and run again, it will create a new Task with a new Task ID.

The following code will create a new Task every time it runs, because it stores an artifact.

task = Task.init('myProject', 'myOtherTask')
  
  d = {'a': '1'}
  task.upload_artifact('myArtifact', d)
  
Parameters
  • project_name (str) – The name of the project in which the experiment will be created. If the project does not exist, it is created. If project_name is None, the repository name is used. (Optional)

  • task_name (str) – The name of Task (experiment). If task_name is None, the Python experiment script’s file name is used. (Optional)

  • task_type (TaskTypes) –

    The task type.

    Valid task types:

    • TaskTypes.training (default)

    • TaskTypes.testing

    • TaskTypes.inference

    • TaskTypes.data_processing

    • TaskTypes.application

    • TaskTypes.monitor

    • TaskTypes.controller

    • TaskTypes.optimizer

    • TaskTypes.service

    • TaskTypes.qc

    • TaskTypes.custom

  • reuse_last_task_id (bool) –

    Force a new Task (experiment) with a new Task ID, but the same project and Task names.

    Note

    Trains creates the new Task ID using the previous Id, which is stored in the data cache folder.

    The values are:

    • True - Reuse the last Task ID. (default)

    • False - Force a new Task (experiment).

    • A string - In addition to a boolean, you can use a string to set a specific value for Task ID (instead of the system generated UUID).

  • output_uri (str) –

    The default location for output models and other artifacts. In the default location, Trains creates a subfolder for the output. The subfolder structure is the following:

    <output destination name> / <project name> / <task name>.< Task ID>

    The following are examples of output_uri values for the supported locations:

    • A shared folder: /mnt/share/folder

    • S3: s3://bucket/folder

    • Google Cloud Storage: gs://bucket-name/folder

    • Azure Storage: azure://company.blob.core.windows.net/folder/

    Important

    For cloud storage, you must install the Trains package for your cloud storage type, and then configure your storage credentials. For detailed information, see Trains Python Client Extras in the “Trains Python Client Reference” section.

  • auto_connect_arg_parser

    Automatically connect an argparse object to the Task?

    The values are:

    • True - Automatically connect. (default)

    • False - Do not automatically connect.

    • A dictionary - In addition to a boolean, you can use a dictionary for fined grained control of connected arguments. The dictionary keys are argparse variable names and the values are booleans. The False value excludes the specified argument from the Task’s parameter section. Keys missing from the dictionary default to True, and an empty dictionary defaults to False.

    For example:

    auto_connect_arg_parser={'do_not_include_me': False, }
      

    Note

    To manually connect an argparse, use Task.connect.

  • auto_connect_frameworks

    Automatically connect frameworks? This includes patching MatplotLib, XGBoost, scikit-learn, Keras callbacks, and TensorBoard/X to serialize plots, graphs, and the model location to the Trains Server (backend), in addition to original output destination.

    The values are:

    • True - Automatically connect (default)

    • False - Do not automatically connect

    • A dictionary - In addition to a boolean, you can use a dictionary for fined grained control of connected frameworks. The dictionary keys are frameworks and the values are booleans. Keys missing from the dictionary default to True, and an empty dictionary defaults to False.

    For example:

    auto_connect_frameworks={'matplotlib': True, 'tensorflow': True, 'pytorch': True,
           'xgboost': True, 'scikit': True}
      

  • auto_resource_monitoring (bool) –

    Automatically create machine resource monitoring plots? These plots appear in in the Trains Web-App (UI), RESULTS tab, SCALARS sub-tab, with a title of :resource monitor:.

    The values are:

    • True - Automatically create resource monitoring plots. (default)

    • False - Do not automatically create.

Returns

The main execution Task (Task context).

classmethod create(project_name=None, task_name=None, task_type=<TaskTypes.training: 'training'>)

Create a new, non-reproducible Task (experiment). This is called a sub-task.

Note

This method always creates a new, non-reproducible Task. To create a reproducible Task, call the Task.init method. To reference another Task, call the Task.get_task method .

Parameters
  • project_name (str) – The name of the project in which the experiment will be created. If project_name is None, and the main execution Task is initialized (see Task.init), then the main execution Task’s project is used. Otherwise, if the project does not exist, it is created. (Optional)

  • task_name (str) – The name of Task (experiment).

  • task_type (TaskTypes) –

    The task type.

    Valid task types:

    • TaskTypes.training (default)

    • TaskTypes.testing

    • TaskTypes.inference

    • TaskTypes.data_processing

    • TaskTypes.application

    • TaskTypes.monitor

    • TaskTypes.controller

    • TaskTypes.optimizer

    • TaskTypes.service

    • TaskTypes.qc

    • TaskTypes.custom

Returns

A new experiment.

classmethod get_task(task_id=None, project_name=None, task_name=None)

Get a Task by Id, or project name / task name combination.

For example:

The following code demonstrates calling Task.get_task to report a scalar to another Task. The output of Logger.report_scalar from testing is associated with the Task named training. It allows training and testing to run concurrently, because they initialized different Tasks (see Task.init for information about initializing Tasks).

The training script:

# initialize the training Task
  task = Task.init('myProject', 'training')
  
  # do some training
  

The testing script:

# initialize the testing Task
  task = Task.init('myProject', 'testing')
  
  # get the training Task
  train_task = Task.get_task(project_name='myProject', task_name='training')
  
  # report metrics in the training Task
  for x in range(10):
      train_task.get_logger().report_scalar('title', 'series', value=x * 2, iteration=x)
  
Parameters
  • task_id (str) – The Id (system UUID) of the experiment to get. If specified, project_name and task_name are ignored.

  • project_name (str) – The project name of the Task to get.

  • task_name (str) – The name of the Task within project_name to get.

Returns

The Task specified by ID, or project name / experiment name combination.

classmethod get_tasks(task_ids=None, project_name=None, task_name=None, task_filter=None)

Get a list of Tasks by one of the following:

  • A list of specific Task IDs.

  • All Tasks in a project matching a full or partial Task name.

  • All Tasks in any project matching a full or partial Task name.

Parameters
  • task_ids (list(str)) – The Ids (system UUID) of experiments to get. If task_ids specified, then project_name and task_name are ignored.

  • project_name (str) – The project name of the Tasks to get. To get the experiment in all projects, use the default value of None. (Optional)

  • task_name (str) – The full name or partial name of the Tasks to match within the specified project_name (or all projects if project_name is None). This method supports regular expressions for name matching. (Optional)

  • task_ids – list of unique task id string (if exists other parameters are ignored)

  • project_name – project name (str) the task belongs to (use None for all projects)

  • task_name – task name (str) in within the selected project Return any partial match of task_name, regular expressions matching is also supported If None is passed, returns all tasks within the project

  • task_filter (dict) – filter and order Tasks. See service.tasks.GetAllRequest for details

Returns

The Tasks specified by the parameter combinations (see the parameters).

property artifacts

A read-only dictionary of Task artifacts (name, artifact).

Returns

The artifacts.

property models

Read-only dictionary of the Task’s loaded/stored models

Returns

A dictionary of models loaded/stored {‘input’: list(Model), ‘output’: list(Model)}.

classmethod clone(source_task=None, name=None, comment=None, parent=None, project=None)

Create a duplicate (a clone) of a Task (experiment). The status of the cloned Task is Draft and modifiable.

Use this method to manage experiments and for autoML.

Parameters
  • source_task (str) – The Task to clone. Specify a Task object or a Task ID. (Optional)

  • name (str) – The name of the new cloned Task. (Optional)

  • comment (str) – A comment / description for the new cloned Task. (Optional)

  • parent (str) –

    The Id of the parent Task of the new Task.

    • If parent is not specified, then parent is set to source_task.parent.

    • If parent is not specified and source_task.parent is not available, then parent set to to source_task.

  • project (str) – The Id of the project in which to create the new Task. If None, the new task inherits the original Task’s project. (Optional)

Returns

The new cloned Task (experiment).

classmethod enqueue(task, queue_name=None, queue_id=None)

Enqueue a Task for execution, by adding it to an execution queue.

Note

A worker daemon must be listening at the queue for the worker to fetch the Task and execute it, see Use Case Examples on the “Trains Agent Reference page.

Parameters
  • task (Task/str) – The Task to enqueue. Specify a Task object or Task ID.

  • queue_name (str) – The name of the queue. If not specified, then queue_id must be specified.

  • queue_id (str) – The Id of the queue. If not specified, then queue_name must be specified.

Returns

An enqueue JSON response.

{
       "queued": 1,
       "updated": 1,
       "fields": {
           "status": "queued",
           "status_reason": "",
           "status_message": "",
           "status_changed": "2020-02-24T15:05:35.426770+00:00",
           "last_update": "2020-02-24T15:05:35.426770+00:00",
           "execution.queue": "2bd96ab2d9e54b578cc2fb195e52c7cf"
           }
   }
  
  • queued - The number of Tasks enqueued (an integer or null).

  • updated - The number of Tasks updated (an integer or null).

  • fields

    • status - The status of the experiment.

    • status_reason - The reason for the last status change.

    • status_message - Information about the status.

    • status_changed - The last status change date and time (ISO 8601 format).

    • last_update - The last Task update time, including Task creation, update, change, or events for this task (ISO 8601 format).

    • execution.queue - The Id of the queue where the Task is enqueued. null indicates not enqueued.

classmethod dequeue(task)

Dequeue (remove) a Task from an execution queue.

Parameters

task (Task/str) – The Task to dequeue. Specify a Task object or Task ID.

Returns

A dequeue JSON response.

{
       "dequeued": 1,
       "updated": 1,
       "fields": {
           "status": "created",
           "status_reason": "",
           "status_message": "",
           "status_changed": "2020-02-24T16:43:43.057320+00:00",
           "last_update": "2020-02-24T16:43:43.057320+00:00",
           "execution.queue": null
           }
   }
  
  • dequeued - The number of Tasks enqueued (an integer or null).

  • fields

    • status - The status of the experiment.

    • status_reason - The reason for the last status change.

    • status_message - Information about the status.

    • status_changed - The last status change date and time in ISO 8601 format.

    • last_update - The last time the Task was created, updated,

      changed or events for this task were reported.

    • execution.queue - The Id of the queue where the Task is enqueued. null indicates not enqueued.

  • updated - The number of Tasks updated (an integer or null).

add_tags(tags)

Add Tags to this task. Old tags are not deleted. When executing a Task (experiment) remotely, this method has no effect).

Parameters

tags – A list of tags which describe the Task to add.

connect(mutable)

Connect an object to a Task object. This connects an experiment component (part of an experiment) to the experiment. For example, connect hyperparameters or models.

Parameters

mutable (object) –

The experiment component to connect. The object can be any object Task supports integrating, including:

  • argparse - An argparse object for parameters.

  • dict - A dictionary for parameters.

  • TaskParameters - A TaskParameters object.

  • model - A model object for initial model warmup, or for model update/snapshot uploading.

Returns

The result returned when connecting the object, if supported.

Raise

Raise an exception on unsupported objects.

connect_configuration(configuration)

Connect a configuration dictionary or configuration file (pathlib.Path / str) to a Task object. This method should be called before reading the configuration file.

Later, when creating an output model, the model will include the contents of the configuration dictionary or file.

For example, a local file:

config_file = task.connect_configuration(config_file)
  my_params = json.load(open(config_file,'rt'))
  

A parameter dictionary:

my_params = task.connect_configuration(my_params)
  
Parameters

configuration

The configuration. This is usually the configuration used in the model training process. Specify one of the following:

  • A dictionary - A dictionary containing the configuration. Trains stores the configuration in the Trains Server (backend), in a HOCON format (JSON-like format) which is editable.

  • A pathlib2.Path string - A path to the configuration file. Trains stores the content of the file. A local path must be relative path. When executing a Task remotely in a worker, the contents brought from the Trains Server (backend) overwrites the contents of the file.

Returns

If a dictionary is specified, then a dictionary is returned. If pathlib2.Path / string is specified, then a path to a local configuration file is returned. Configuration object.

connect_label_enumeration(enumeration)

Connect a label enumeration dictionary to a Task (experiment) object.

Later, when creating an output model, the model will include the label enumeration dictionary.

Parameters

enumeration (dict) –

A label enumeration dictionary of string (label) to integer (value) pairs.

For example:

{
       'background': 0,
       'person': 1
  }
  

Returns

The label enumeration dictionary (JSON).

get_logger()

Get a Logger object for reporting, for this task context. You can view all Logger report output associated with the Task for which this method is called, including metrics, plots, text, tables, and images, in the Trains Web-App (UI).

Returns

The Logger for the Task (experiment).

mark_started()

Manually mark a Task as started (happens automatically)

mark_stopped()

Manually mark a Task as stopped (also used in _at_exit)

flush(wait_for_uploads=False)

Flush any outstanding reports or console logs.

Parameters

wait_for_uploads (bool) –

Wait for all outstanding uploads to complete before existing the flush?

  • True - Wait

  • False - Do not wait (default)

reset(set_started_on_success=False, force=False)

Reset a Task. Trains reloads a Task after a successful reset. When a worker executes a Task remotely, the Task does not reset unless the force parameter is set to True (this avoids accidentally clearing logs and metrics).

Parameters
  • set_started_on_success (bool) –

    If successful, automatically set the Task to started?

    • True - If successful, set to started.

    • False - If successful, do not set to started. (default)

  • force (bool) –

    Force a Task reset, even when executing the Task (experiment) remotely in a worker?

    • True - Force

    • False - Do not force (default)

close()

Close the current Task. Enables you to manually shutdown the task.

Warning

Only call Task.close if you are certain the Task is not needed.

register_artifact(name, artifact, metadata=None, uniqueness_columns=True)

Register (add) an artifact for the current Task. Registered artifacts are dynamically synchronized with the Trains Server (backend). If a registered artifact is updated, the update is stored in the Trains Server (backend). Registered artifacts are primarily used for Data Audition.

The currently supported registered artifact object type is a pandas.DataFrame.

See also Task.unregister_artifact and Task.get_registered_artifacts.

Note

Trains also supports uploaded artifacts which are one-time uploads of static artifacts that are not dynamically synchronized with the Trains Server (backend). These static artifacts include additional object types. For more information, see Task.upload_artifact.

Parameters
  • name (str) –

    The name of the artifact.

    Warning

    If an artifact with the same name was previously registered, it is overwritten.

  • artifact (object) – The artifact object.

  • metadata (dict) – A dictionary of key-value pairs for any metadata. This dictionary appears with the experiment in the Trains Web-App (UI), ARTIFACTS tab.

  • uniqueness_columns – A Sequence of columns for artifact uniqueness comparison criteria, or the default value of True. If True, the artifact uniqueness comparison criteria is all the columns, which is the same as artifact.columns.

unregister_artifact(name)

Unregister (remove) a registered artifact. This removes the artifact from the watch list that Trains uses to synchronize artifacts with the Trains Server (backend).

Important

  • Calling this method does not remove the artifact from a Task. It only stops Trains from monitoring the artifact.

  • When this method is called, Trains immediately takes the last snapshot of the artifact.

get_registered_artifacts()

Get a dictionary containing the Task’s registered (dynamically synchronized) artifacts (name, artifact object).

Note

After calling get_registered_artifacts, you can still modify the registered artifacts.

Returns

The registered (dynamically synchronized) artifacts.

upload_artifact(name, artifact_object, metadata=None, delete_after_upload=False)

Upload (add) a static artifact to a Task object. The artifact is uploaded in the background.

The currently supported upload (static) artifact types include:

  • string / pathlib2.Path - A path to artifact file. If a wildcard or a folder is specified, then Trains creates and uploads a ZIP file.

  • dict - Trains stores a dictionary as .json file and uploads it.

  • pandas.DataFrame - Trains stores a pandas.DataFrame as .csv.gz (compressed CSV) file and uploads it.

  • numpy.ndarray - Trains stores a numpy.ndarray as .npz file and uploads it.

  • PIL.Image - Trains stores a PIL.Image as .png file and uploads it.

Parameters
  • name (str) –

    The artifact name.

    Warning

    If an artifact with the same name was previously uploaded, then it is overwritten.

  • artifact_object (object) – The artifact object.

  • metadata (dict) – A dictionary of key-value pairs for any metadata. This dictionary appears with the experiment in the Trains Web-App (UI), ARTIFACTS tab.

  • delete_after_upload (bool) –

    After the upload, delete the local copy of the artifact?

    • True - Delete the local copy of the artifact.

    • False - Do not delete. (default)

Returns

The status of the upload.

  • True - Upload succeeded.

  • False - Upload failed.

Raise

If the artifact object type is not supported, raise a ValueError.

get_models()

Return a dictionary with {‘input’: [], ‘output’: []} loaded/stored models of the current Task Input models are files loaded in the task, either manually or automatically logged Output models are files stored in the task, either manually or automatically logged Automatically logged frameworks are for example: TensorFlow, Keras, PyTorch, ScikitLearn(joblib) etc.

Returns

A dictionary with keys input/output, each is list of Model objects.

Example:

{'input': [trains.Model()], 'output': [trains.Model()]}
  

is_current_task()

Deprecated since version 0.13.0: This method is deprecated. Use Task.is_main_task instead.

Is this Task object the main execution Task (initially returned by Task.init)?

Returns

Is this Task object the main execution Task?

  • True - Is the main execution Task.

  • False - Is not the main execution Task.

is_main_task()

Is this Task object the main execution Task (initially returned by Task.init)?

Note

If Task.init was never called, this method will not create it, making this test more efficient than:

Task.init() == task
  
Returns

Is this Task object the main execution Task?

  • True - Is the main execution Task.

  • False - Is not the main execution Task.

set_model_config(config_text=None, config_dict=None)

Deprecated since version 0.14.1: Use Task.connect_configuration instead.

get_model_config_text()

Deprecated since version 0.14.1: Use Task.connect_configuration instead.

get_model_config_dict()

Deprecated since version 0.14.1: Use Task.connect_configuration instead.

set_model_label_enumeration(enumeration=None)

Set the label enumeration for the Task object before creating an output model. Later, when creating an output model, the model will inherit these properties.

Parameters

enumeration (dict) –

A label enumeration dictionary of string (label) to integer (value) pairs.

For example:

{
       'background': 0,
       'person': 1
  }
  

get_last_iteration()

Get the last reported iteration, which is the last iteration for which the Task reported a metric.

Note

The maximum reported iteration is not in the local cache. This method sends a request to the Trains Server (backend).

Returns

The last reported iteration number.

set_last_iteration(last_iteration)

Forcefully set the last reported iteration, which is the last iteration for which the Task reported a metric.

Parameters

last_iteration (int) – The last reported iteration number.

set_initial_iteration(offset=0)

Set initial iteration, instead of zero. Useful when continuing training from previous checkpoints

Parameters

offset (int) – Initial iteration (at starting point)

Returns

Newly set initial offset.

get_initial_iteration()

Return the initial iteration offset, default is 0 Useful when continuing training from previous checkpoints

Returns

Initial iteration offset.

get_last_scalar_metrics()

Get the last scalar metrics which the Task reported. This is a nested dictionary, ordered by title and series.

For example:

{
   'title': {
       'series': {
           'last': 0.5,
           'min': 0.1,
           'max': 0.9
           }
       }
   }
  
Returns

The last scalar metrics.

get_parameters_as_dict()

Get the Task parameters as a raw nested dictionary.

Note

The values are not parsed. They are returned as is.

set_parameters_as_dict(dictionary)

Set the parameters for the Task object from a dictionary. The dictionary can be nested. This does not link the dictionary to the Task object. It does a one-time update. This is the same behavior as the Task.connect method.

execute_remotely(queue_name=None, clone=False, exit_process=True)

If task is running locally (i.e., not by trains-agent), then clone the Task and enqueue it for remote execution; or, stop the execution of the current Task, reset its state, and enqueue it. If exit==True, exit this process.

Note

If the task is running remotely (i.e., trains-agent is executing it), this call is a no-op (i.e., does nothing).

Parameters
  • queue_name – The queue name used for enqueueing the task. If None, this call exits the process without enqueuing the task.

  • clone

    Clone the Task and execute the newly cloned Task?

    The values are:

    • True - A cloned copy of the Task will be created, and enqueued, instead of this Task.

    • False - The Task will be enqueued.

  • exit_process

    The function call will leave the calling process at the end?

    • True - Exit the process (exit(0)).

    • False - Do not exit the process.

    Warning

    If clone==False, then exit_process must be True.

wait_for_status(status=(<TaskStatusEnum.completed: 'completed'>, <TaskStatusEnum.stopped: 'stopped'>, <TaskStatusEnum.closed: 'closed'>), raise_on_status=(<TaskStatusEnum.failed: 'failed'>, ), check_interval_sec=60.0)

Wait for a task to reach a defined status.

Parameters
  • status – Status to wait for. Defaults to (‘completed’, ‘stopped’, ‘closed’, )

  • raise_on_status – Raise RuntimeError if the status of the tasks matches one of these values. Defaults to (‘failed’).

  • check_interval_sec – Interval in seconds between two checks. Defaults to 60 seconds.

Raise

RuntimeError if the status is one of {raise_on_status}.

classmethod set_credentials(api_host=None, web_host=None, files_host=None, key=None, secret=None, host=None)

Set new default Trains Server (backend) host and credentials.

These credentials will be overridden by either OS environment variables, or the Trains configuration file, trains.conf.

Warning

Credentials must be set before initializing a Task object.

For example, to set credentials for a remote computer:

Task.set_credentials(api_host='http://localhost:8008', web_host='http://localhost:8080',
   files_host='http://localhost:8081',  key='optional_credentials',  secret='optional_credentials')
   task = Task.init('project name', 'experiment name')
  
Parameters
  • api_host (str) – The API server url. For example, host='http://localhost:8008'

  • web_host (str) – The Web server url. For example, host='http://localhost:8080'

  • files_host (str) – The file server url. For example, host='http://localhost:8081'

  • key (str) – The user key (in the key/secret pair). For example, key='thisisakey123'

  • secret (str) – The user secret (in the key/secret pair). For example, secret='thisisseceret123'

  • host (str) – The host URL (overrides api_host). For example, host='http://localhost:8008'

class TaskStatusEnum(value)

An enumeration.

classmethod add_requirements(package_name, package_version=None)

Force the adding of a package to the requirements list. If package_version is not specified, use the installed package version, if found.

Parameters
  • package_name (str) – The package name to add to the “Installed Packages” section of the task.

  • package_version – The package version requirements. If None, then use the installed version.

property cache_dir

The cache directory which is used to store the Task related files.

completed(ignore_errors=True)

The signal indicating that this Task completed.

classmethod get_all(session=None, log=None, **kwargs)

List all the Tasks based on specific projection.

Parameters
  • session (Session) – The session object used for sending requests to the API.

  • log (logging.Logger) – The Log object.

  • kwargs (dict) –

    Keyword args passed to the GetAllRequest (see backend_api.services.v2_5.tasks.GetAllRequest)

    For example:

    status='completed', 'search_text'='specific_word', 'user'='user_id', 'project'='project_id'
      

Returns

The API response.

get_base_docker()

Get the base Docker command (image) that is set for this experiment.

get_label_num_description()

Get a dictionary of label number to a string pairs representing all labels associated with this number on the model labels.

get_labels_enumeration()

Get the label enumeration dictionary label enumeration dictionary of string (label) to integer (value) pairs.

Returns

A dictionary containing the label enumeration.

get_model_design()

Get the model configuration as blob of text.

Returns

The model configuration as blob of text.

get_num_of_classes()

number of classes based on the task’s labels

get_output_destination(extra_path=None, **kwargs)

Get the task’s output destination, with an optional suffix

get_output_log_web_page()

Return the Task results & outputs web page address. For example: https://demoapp.trains.allegro.ai/projects/216431/experiments/60763e04/output/log

Returns

http/s URL link.

get_parameter(name, default=None)

Get a value for a parameter.

Parameters
  • name – Parameter name

  • default – Default value

Returns

The Parameter value (or default value if parameter is not defined).

get_reported_console_output(number_of_reports=1)

Return a list of console outputs reported by the Task. Retrieved outputs are the most updated console outputs.

Parameters

number_of_reports (int) – The number of reports to return. The default value is 1, indicating the last (most updated) console output

Returns

A list of strings, each entry corresponds to one report.

get_reported_scalars(max_samples=0, x_axis='iter')

Return a nested dictionary for the scalar graphs, where the first key is the graph title and the second is the series name. Value is a dict with ‘x’: values and ‘y’: values

Note

This call is not cached, any call will retrieve all the scalar reports from the back-end. If the Task has many scalars reported, it might take long for the call to return.

Example:

{'title': {'series': {
              'x': [0, 1 ,2],
              'y': [10, 11 ,12],
  }}}
  
Parameters
  • max_samples (int) – Maximum samples per series to return. Default is 0 returning all scalars. With sample limit, average scalar values inside sampling window.

  • x_axis (str) – scalar x_axis, possible values: ‘iter’: iteration (default), ‘timestamp’: seconds from start, ‘iso_time’: absolute time

Returns

dict: Nested scalar graphs: dict[title(str), dict[series(str), dict[axis(str), list(float)]]]

get_status()

Return The task status without refreshing the entire Task object object (only the status property)

TaskStatusEnum: [“created”, “in_progress”, “stopped”, “closed”, “failed”, “completed”,

“queued”, “published”, “publishing”, “unknown”]

Returns

str: Task status as string (TaskStatusEnum)

property input_model

A model manager used to handle the input model object

property labels_stats

Get accumulated label stats for the current/last frames iteration

mark_failed(ignore_errors=True, status_reason=None, status_message=None)

The signal that this Task stopped.

property metrics_manager

A metrics manager used to manage the metrics related to this task

property output_model

A model manager used to manage the output model object

publish(ignore_errors=True)

The signal that this Task will be published

property reporter

Returns a simple metrics reporter instance.

static running_locally()

Is the task running locally (i.e., trains-agent is not executing it)?

Returns

True, if the task is running locally. False, if the task is not running locally.

save_exec_model_design_file(filename='model_design.txt', use_cache=False)

Save execution model design to file

set_artifacts(artifacts_list=None)

List of artifacts (tasks.Artifact) to update the task

Parameters

artifacts_list (list) – list of artifacts (type tasks.Artifact)

set_base_docker(docker_cmd)

Set the base docker image for this experiment If provided, this value will be used by trains-agent to execute this experiment inside the provided docker image.

set_comment(comment)

Set a comment / description for the Task.

Parameters

comment (str) – The comment / description for the Task.

set_input_model(model_id=None, model_name=None, update_task_design=True, update_task_labels=True)

Set a new input model for the Task. The model must be “ready” (status is Published) to be used as the Task’s input model.

Parameters
  • model_id – The Id of the model on the Trains Server (backend). If model_name is not specified, then model_id must be specified.

  • model_name – The model name. The name is used to locate an existing model in the Trains Server (backend). If model_id is not specified, then model_name must be specified.

  • update_task_design

    Update the Task’s design?

    • True - Trains copies the Task’s model design from the input model.

    • False - Trains does not copy the Task’s model design from the input model.

  • update_task_labels

    Update the Task’s label enumeration?

    • True - Trains copies the Task’s label enumeration from the input model.

    • False - Trains does not copy the Task’s label enumeration from the input model.

set_name(name)

Set the Task name.

Parameters

name (str) – The name of the Task.

set_parameter(name, value, description=None)

Set a single Task parameter. This overrides any previous value for this parameter.

Parameters
  • name – The parameter name.

  • value – The parameter value.

  • description

    The parameter description.

    Note

    The description is not yet in use.

set_parameters(*args, **kwargs)

Set the parameters for a Task. This method sets a complete group of key-value parameter pairs, but does not support parameter descriptions (the input is a dictionary of key-value pairs).

Parameters
  • args – Positional arguments, which are one or more dictionary or (key, value) iterable. They are merged into a single key-value pair dictionary.

  • kwargs – Key-value pairs, merged into the parameters dictionary created from args.

set_parent(parent)

Set the parent task for the Task. :param parent: The parent task id (or parent Task object) for the Task. Set None for no parent. :type parent: str or Task

started(ignore_errors=True)

The signal that this Task started.

property status

The Task’s status. To keep the Task updated. Trains reloads the Task status information only, when this value is accessed.

return str: TaskStatusEnum status

stopped(ignore_errors=True)

The signal that this Task stopped.

update_model_desc(new_model_desc_file=None)

Change the Task’s model description.

update_output_model(model_uri, name=None, comment=None, tags=None)

Update the Task’s output model. Use this method to update the output model when you have a local model URI, for example, storing the weights file locally, and specifying a file://path/to/file URI)

Important

This method only updates the model’s metadata using the API. It does not upload any data.

Parameters
  • model_uri (str) – The URI of the updated model weights file.

  • name (str) – The updated model name. (Optional)

  • comment (str) – The updated model description. (Optional)

  • tags ([str]) – The updated model tags. (Optional)

update_output_model_and_upload(model_file, name=None, comment=None, tags=None, async_enable=False, cb=None, iteration=None)

Update the Task’s output model weights file. First, Trains uploads the file to the preconfigured output destination (see the Task’s output.destination property or call the setup_upload method), then Trains updates the model object associated with the Task an API call. The API call uses with the URI of the uploaded file, and other values provided by additional arguments.

Parameters
  • model_file (str) – The path to the updated model weights file.

  • name (str) – The updated model name. (Optional)

  • comment (str) – The updated model description. (Optional)

  • tags (list) – The updated model tags. (Optional)

  • async_enable (bool) –

    Request asynchronous upload?

    • True - The API call returns immediately, while the upload and update are scheduled in another thread.

    • False - The API call blocks until the upload completes, and the API call updating the model returns. (default)

  • cb (callable) – Asynchronous callback. A callback. If async_enable is set to True, this is a callback that is invoked once the asynchronous upload and update complete.

  • iteration (int) – iteration number for the current stored model (Optional)

Returns

The URI of the uploaded weights file. If async_enable is set to True, this is the expected URI, as the upload is probably still in progress.

update_parameters(*args, **kwargs)

Update the parameters for a Task. This method updates a complete group of key-value parameter pairs, but does not support parameter descriptions (the input is a dictionary of key-value pairs).

Parameters
  • args – Positional arguments, which are one or more dictionary or (key, value) iterable. They are merged into a single key-value pair dictionary.

  • kwargs – Key-value pairs, merged into the parameters dictionary created from args.