Trains Python Package Reference
Trains is now ClearML
This documentation applies to the legacy Trains versions. For the latest documentation, see ClearML.
Overview
This reference section provides detailed information about the Trains Python Package classes and their usage. This section also includes Trains Python Package Extras which are additional Trains components that require installation and configuration separate from the Trains Python Package. These classes and extras allow you to implement the many features of Trains for enhanced experimentation and automation. This is in addition to our automagical capturing of experiment inputs and outputs.
Task Module
The Task Module reference page provides detailed information for task.py in the trains
repository.
The Task
class contains methods to manage all the parts of experiments and integrate them with Trains for tracking, storage, coding, analysis, comparison, and experiment management, including:
Task.init
- Initialize a Task (experiment).connect
- Connect the parts of an experiment to the experiment in Trains, including hyperparameter dictionaries, input and output models, configuration dictionaries, and class enumeration labels.get_logger
- Create a logger object which you can use for explicit reporting.get_task
- Get a reference to any experiment in Trains by Task Id, task name and project name, or task name only.register_artifact
- Register an artifact, and Trains logs changes to it.get_registered_artifacts
- After you register an artifact, retrieve a reference to it, manipulate the artifact in your code, and the changes are updated in Trains.upload_artifact
- Upload an artifact, but Trains does not logs changes to it.get_model_config_dict
- Get the configuration as a dictionary and then easily work with it in your code.set_model_config
- Set the model configuration before creating an output model so that the output model inherits the configuration properties.set_model_label_enumeration
- Similar to set_model_config(), but for class label enumeration.clone
- Create an editable copy of an experiment. For example, clone an experiment for hyperparameter tuning or multiple times for automation.enqueue
- Add an experiment to a queue. Similar to dequeue() above, enqueue() allows you to develop scripts which can automate your experimentation.dequeue
- Remove an enqueued experiment from a queue.reset
- Make an experiment editable and allow remote execution of the experiment. This deletes output of the previous.set_initial_iteration
- To continue a previous experiment run, set the initial iteration (set to a non-zero).get_initial_iteration
- Get the value of the initial iteration.
Logger Module
The Logger Module reference page provides detailed information for logger.py in the trains
repository.
Use the Logger
class and methods to extend Trains automagical capturing of stdout and stderr with Trains' own explicit reporting for scalar metrics, any plot data in a variety of chart types, text messages, errors, warnings, debug messages, uploading tables, and uploading debug samples (images, audio, and video), as well as other methods.
Logger.current_logger
Logger.report_text
Logger.report_scalar
Logger.report_line_plot
Logger.report_scatter2d
Logger.report_scatter3d
Logger.report_histogram
Logger.report_table
Logger.report_confusion_matrix
Logger.set_upload_destination
Logger.report_image
Logger.report_media
Model Module
The Model Module reference page provides detailed information for model.py in the trains
repository.
Use the Model class's InputModel
and OutputModel
classes and methods to augment control of input models and output using methods such as:
InputModel.import_model
- Import a pre-trained model for your experiment.InputModel.connect
- Connect the current model the Task (experiment).InputModel.config_dict
- Get the input model's configuration as a dictionary.InputModel.get_weights
- Download the base model.OutputModel.connect
- Connect the output model to the Task (experiment).OutputModel.set_upload_destination
- Specify a destination for uploading debug samples, including images, audio, and video.
Automation Module
The Automation Module reference page provides detailed information for the following:
- optimization classes
- parameters classes
Storage Module
The Storage Module reference page provides detailed information for manager.py in the trains
repository.
To list, download, and upload files to / from storage, as well as manage cache, use the StorageManager
and StoreHelper
class methods.
Trains Python Package Extras
Trains Python Package Extras are components of the Trains Python Package which provide additional features, but are not installed when you install Trains (meaning, they are not installed when you execute, pip install trains
). They are referred to as extras, because they are defined in the Trains Python Package setup.py extras_require
dictionary. Python package components in the extras_require
dictionary require separate installation.
The Trains Python Package Extras include extras for experiment storage of model checkpoints (snapshots) and other artifacts. The supported Cloud storage types include:
- AWS S3
- Google Cloud Storage
- Azure Storage
These extras for storage also require that you add your storage bucket names, credentials, and related information to your Trains configuration file.
To install and configure any of these, follow the instructions on the Trains Python Package Extras page.
After installing and configuring, you can use the Cloud storage in your Python experiment scripts by calling the Task.init
method, and specifying the output_uri parameter.