Examples

To help you learn and use ClearML, we provide example scripts and describe their Web UI tracking, results visualizations, and workflow execution.

Examples scripts are in examples folder the GitHub clearml repository, they are pre-loaded in ClearML Server, and include:

  • Ready to run examples - These demonstrate ClearML Python Package, ClearML Web UI, and ClearML Server features, including ClearML automatic logging, explicit reporting, integrating ClearML into code with frameworks and visualization tools, automation, and optimization. They are associated with the ClearML Examples project. Their status is Published. You can clone, edit, and enqueue them.

  • Configurable services examples - These perform various functions (for example, Task status alerts and old Task cleanup) which continue running on ClearML Server. They execute in ClearML Agent services mode. Their status is Draft (editable), you configure them, and then enqueue them to the services queue.

Each examples folder in the GitHub clearml repository contains a requirements.txt file for example scripts in that folder.

Automation

  • Manual Random Parameter Search - Executing an experiment multiple times, each time with different sets of random hyperparameters.

  • Task Piping - Creating an instance of a Task from a template Task, customizing that instance, and enqueuing the customized instance to execute.

Distributed

  • PyTorch Distributed - Integrating ClearML into code that uses the torch.distributed. Spawn Tasks in subprocesses which train a network, and report artifacts, scalars, hyperparameters to the main Task.

  • Subprocess - Multiple subprocesses interacting and reporting to a main Task.

Explicit Reporting

  • Explicit Reporting - Jupyter Notebook - Several explicit reporting examples running in a Jupyter Notebook, including scalars, plots, media (audio, HTML, images, and video), and text.

  • 2D Plots Reporting - Reporting series as 2D plots in histogram, confusion matrix, and 2D scatter plot formats.

  • 3D Plots Reporting - Reporting series as a surface plot and as a 3D scatter plot.

  • Artifacts Reporting - Uploading objects (other than models) to storage as experiment artifacts.

  • Configuring Models - Configuring a model and defining label enumeration.

  • HTML Reporting - Reporting local HTML files and HTML by URL.

  • Hyperparameters Reporting - The example hyper_parameters.py demonstrates automatic logging of command line options from argparse, TensorFlow Definitions, and parameter dictionaries which are explicitly connected to Tasks.

  • Images Reporting - Reporting (uploading) images in several formats, including NumPy arrays, uint8, uint8 RGB, PIL Image objects, and local files.

  • Matplotlib Reporting - Reporting Matplotlib in ClearML by calling the Logger.report_matplotlib_figure method.

  • Media Reporting - Reporting images, audio, and video. Upload from a local path, provide a BytesIO stream, or provide the URL of media already uploaded to some storage.

  • Plotly Reporting - Reporting Plotly plots in ClearML by calling the Logger.report_plotly method, and passing it a complex Plotly figure using the figure parameter.

  • Scalars Reporting - Reporting scalars.

  • Tables Reporting (Pandas and CSV Files) - Reporting tabular data from Pandas DataFrames and CSV files as tables.

  • Text Reporting - Explicitly reporting (as compared to automatic logging) text.

Frameworks

Note

Keras and TensorFlow examples include legacy examples for versions of TensorFlow older than v2.0.

AutoKeras

Fastai

  • Fastai - Integrating ClearML into code which uses fastai and TensorBoard.

Keras

Keras Tuner

  • Keras Tuner - Integrating ClearML into code which uses the Keras Tuner Hyperband tuner to optimize hyperparameters for training a network on a CIFAR10 dataset. This example is described in the “Integration” section, on the “Keras Tuner” page.

Matplotlib

  • Matplotlib - Integrating ClearML into code which uses Matplotlib to plot scatter diagrams, and show images.

  • Matplotlib - Jupyter Notebook - The same “Matplotlib” example (as the preceding example) running in a Jupyter Notebook.

PyTorch

These examples demonstration integrating ClearML into code that uses PyTorch.

Notebooks

Audio
Images
Tables
Text
  • Text Classification - Jupyter Notebook for ClearML, and the integration of ClearML into code which trains a network to classify text in the torchtext AG_NEWS dataset, and then applies the model to predict the classification of sample text.

scikit-learn

  • scikit-learn with Joblib - Integrating ClearML into code which uses scikit-learn, joblib to store a model and model snapshot, and Matplotlib to create a scatter diagram.

  • scikit-learn with Matplotlib - Integrating ClearML into code which uses scikit-learn to determine cross-validated training and test scores, and matplotlib to plot the learning curves. ClearML automatically logs the scatter diagrams for the learning curves.

TensorBoardX

  • TensorBoardX - Integrating ClearML into code which uses PyTorch and TensorBoardX.

TensorFlow

  • Manual Model Upload - ClearML tracking of a manually configured model created with TensorFlow, including model checkpoints (snapshots), hyperparameters, and output to the console.

  • TensorBoard PR Curve - Integrating ClearML into code which uses TensorFlow and TensorBoard.

  • TensorBoard Toy - ClearML automatic logging of TensorBoard scalars, histograms, images, and text, as well as all other console output and TensorFlow Definitions.

  • TensorFlow MNIST - Integrating ClearML into code which uses TensorFlow and Keras to trains a neural network on the Keras built-in MNIST handwritten digits dataset.

XGBoost

  • XGBoost - Integrating ClearML into code that trains a network on the scikit-learn iris classification dataset, and XGBoost.

Optimization

Pipeline

Storage

Services