To help you learn and use ClearML, we provide example scripts and describe their Web UI tracking, results visualizations, and workflow execution.
Examples scripts are in examples folder the GitHub
clearml repository, they are pre-loaded in ClearML Server, and include:
Ready to run examples - These demonstrate ClearML Python Package, ClearML Web UI, and ClearML Server features, including ClearML automatic logging, explicit reporting, integrating ClearML into code with frameworks and visualization tools, automation, and optimization. They are associated with the
ClearML Examplesproject. Their status is Published. You can clone, edit, and enqueue them.
Configurable services examples - These perform various functions (for example, Task status alerts and old Task cleanup) which continue running on ClearML Server. They execute in ClearML Agent services mode. Their status is Draft (editable), you configure them, and then enqueue them to the
Each examples folder in the GitHub
clearml repository contains a
requirements.txt file for example scripts in that folder.
Manual Random Parameter Search - Executing an experiment multiple times, each time with different sets of random hyperparameters.
Task Piping - Creating an instance of a Task from a template Task, customizing that instance, and enqueuing the customized instance to execute.
PyTorch Distributed - Integrating ClearML into code that uses the
torch.distributed. Spawn Tasks in subprocesses which train a network, and report artifacts, scalars, hyperparameters to the main Task.
Subprocess - Multiple subprocesses interacting and reporting to a main Task.
Explicit Reporting - Jupyter Notebook - Several explicit reporting examples running in a Jupyter Notebook, including scalars, plots, media (audio, HTML, images, and video), and text.
2D Plots Reporting - Reporting series as 2D plots in histogram, confusion matrix, and 2D scatter plot formats.
3D Plots Reporting - Reporting series as a surface plot and as a 3D scatter plot.
Artifacts Reporting - Uploading objects (other than models) to storage as experiment artifacts.
Configuring Models - Configuring a model and defining label enumeration.
HTML Reporting - Reporting local HTML files and HTML by URL.
Hyperparameters Reporting - The example hyper_parameters.py demonstrates automatic logging of command line options from argparse, TensorFlow Definitions, and parameter dictionaries which are explicitly connected to Tasks.
Images Reporting - Reporting (uploading) images in several formats, including NumPy arrays, uint8, uint8 RGB, PIL Image objects, and local files.
Matplotlib Reporting - Reporting Matplotlib in ClearML by calling the Logger.report_matplotlib_figure method.
Media Reporting - Reporting images, audio, and video. Upload from a local path, provide a BytesIO stream, or provide the URL of media already uploaded to some storage.
Plotly Reporting - Reporting Plotly plots in ClearML by calling the Logger.report_plotly method, and passing it a complex Plotly figure using the figure parameter.
Scalars Reporting - Reporting scalars.
Tables Reporting (Pandas and CSV Files) - Reporting tabular data from Pandas DataFrames and CSV files as tables.
Text Reporting - Explicitly reporting (as compared to automatic logging) text.
Keras and TensorFlow examples include legacy examples for versions of TensorFlow older than v2.0.
Keras with Matplotlib - Jupyter Notebook - ClearML running in Jupyter Notebook with Keras, Matplotlib, and automatic logging.
Keras with TensorBoard - Integrating ClearML into code which uses Keras and TensorBoard.
Keras with TensorBoard - Jupyter Notebook - The “Keras with TensorBoard” example (the preceding example as a Jupyter Notebook).
Manual Model Upload - ClearML tracking of a manually configured model created with Keras, including model checkpoints (snapshots), hyperparameters, and output to the console.
Keras Tuner - Integrating ClearML into code which uses the Keras Tuner
Hyperbandtuner to optimize hyperparameters for training a network on a CIFAR10 dataset. This example is described in the “Integration” section, on the “Keras Tuner” page.
Matplotlib - Integrating ClearML into code which uses Matplotlib to plot scatter diagrams, and show images.
Matplotlib - Jupyter Notebook - The same “Matplotlib” example (as the preceding example) running in a Jupyter Notebook.
These examples demonstration integrating ClearML into code that uses PyTorch.
Manual Model Upload - ClearML tracking of a manually configured model created with PyTorch, including model checkpoints (snapshots), and output to the console.
PyTorch TensorBoard Toy - ClearML with PyTorch and TensorBoard to log debug sample images.
PyTorch with Matplotlib - ClearML with PyTorch and Matplotlib.
PyTorch with TensorBoard - ClearML with PyTorch and TensorBoard.
PyTorch with TensorBoardX - ClearML with PyTorch and TensorBoardX.
Audio Preprocessing - Jupyter Notebook - Integrating ClearML into a Jupyter Notebook which uses PyTorch and preprocesses audio samples.
Audio Classification - Jupyter Notebooks - Integrating ClearML into a Jupyter Notebook which uses PyTorch, TensorBoard, and TorchVision to train a neural network on the UrbanSound8K dataset for audio classification.
Hyperparameter Optimization - Jupyter Notebook - Integrating ClearML into a Jupyter Notebook which performs automated hyperparameter optimization.
Image Classification - Jupyter Notebook - Integrating ClearML into a Jupyter Notebook which uses PyTorch, TensorBoard, and TorchVision to train a neural network on the UrbanSound8K dataset for image classification.
Tabular Data Preprocessing - ClearML stores downloaded training as artifacts.
See the pipeline example using tabular data, Pipeline with Concurrent Steps - Tabular Data.
Text Classification - Jupyter Notebook for ClearML, and the integration of ClearML into code which trains a network to classify text in the
torchtextAG_NEWS dataset, and then applies the model to predict the classification of sample text.
scikit-learn with Joblib - Integrating ClearML into code which uses
joblibto store a model and model snapshot, and Matplotlib to create a scatter diagram.
scikit-learn with Matplotlib - Integrating ClearML into code which uses
scikit-learnto determine cross-validated training and test scores, and
matplotlibto plot the learning curves. ClearML automatically logs the scatter diagrams for the learning curves.
TensorBoardX - Integrating ClearML into code which uses PyTorch and TensorBoardX.
Manual Model Upload - ClearML tracking of a manually configured model created with TensorFlow, including model checkpoints (snapshots), hyperparameters, and output to the console.
TensorBoard PR Curve - Integrating ClearML into code which uses TensorFlow and TensorBoard.
TensorBoard Toy - ClearML automatic logging of TensorBoard scalars, histograms, images, and text, as well as all other console output and TensorFlow Definitions.
Hyperparameter Optimization - ClearML hyperparameter optimization automation.
Basic Pipeline - Serialized Data - A basic pipeline to download data, process it, and a train a network. The data is serialized.
Pipeline with Concurrent Steps - Tabular Data - A pipeline with nodes that run concurrently to preprocess two sets of data, train on each, and select the better model. The data is tabular.
ClearML AWS Autoscaler - Optimizes AWS EC2 instance scaling according to the budget you configure.
Cleanup Service - Deletes Archived Tasks, and their associated artifacts and debug samples, based on configurable parameter criteria.
Jupyter Notebook Server Service - A Jupyter Notebook server.
Monitoring Service Posting Slack Alerts - Monitors Task completion/failure based on configurable parameter criteria and post alerts to your Slack channel.