Task Piping

The task_piping_example.py example demonstrates creating an instance of a Task from a template Task, customizing that instance, and enqueuing the customized instance to execute. The customization is changing the value of a parameter.

This example accomplishes a task pipe by doing the following:

  1. Creating the template Task which is named Toy Base Task. It must be stored in ClearML Server before instances of it can be created. To create it, run another ClearML example script, toy_base_task.py.

  2. Creating a parameter dictionary, which we connect to the Task by calling Task.connect so that the parameters are logged by ClearML. The dictionary contains the parameter in the template to customize, Example_Param, its value of 1, and other parameters for the Task pipe execution.

  3. Creating a Task object referencing the template Task.

  4. Creating an instance of the template Task by cloning it.

  5. Getting the newly cloned Task’s parameters, see Task.get_parameters.

  6. Setting the newly cloned Task’s parameters to the search values in the parameter dictionary (Step 2), see Task.set_parameters.

  7. Enqueuing the newly cloned Task to execute, see Task.enqueue.

When the example script runs, it creates an instance of the template experiment, named Auto generated cloned task which is associated with the examples project. In the instance, the value of the customized parameter, Example_Param changed to 3. You can see it in CONFIGURATIONS > HYPER PARAMETERS.