Trains Python Package Extras
Trains is now ClearML
This documentation applies to the legacy Trains versions. For the latest documentation, see ClearML.
This page describes how to install and configure Trains Python Package Extras. Currently, the Trains Python Package Extras include extras for experiment storage of model checkpoints (snapshots) and other artifacts. The supported Cloud storage types include:
- AWS S3
- Azure Storage
- Google Cloud Storage
In addition to installing a Trains extra for storage, you must add your storage bucket names, credentials, and related information to the Trains configuration file.
Step 1. Install Trains extras for storage
To install a Trains extra for storage, do the following:
For your Cloud storage, execute one of the following commands:
-
AWS S3
pip install trains[s3]
-
Google Cloud Storage
pip install trains[gs]
-
Azure Storage
pip install trains[azure]
Step 2. Initializing a new Trains configuration file
Use our trains-init
setup wizard to initialize a new Trains configuration file.
If you already have a configuration file, the setup wizard will notify you. You can then skip to the Step 3. Add storage credentials to your Trains configuration file, where you add Trains settings to an existing Trains configuration file.
Why would a Trains configuration file already exist?
If you previously configured Trains for the demo Trains Server, a self-hosted Trains Server, or installed and configure Trains Agent, you may have an existing configuration file. This is because Trains and Trains Agent use the same configuration file, but not all the same configuration options.
To initialize a new Trains configuration file, do the following:
- Open a terminal session.
-
In your terminal session, run the setup wizard.
trains-init
If the setup wizard's response indicates you already have a configuration file, then go to Step 3. Add storage credentials to your Trains configuration file and add your storage credentials. The wizard does not edit or overwrite existing configuration files.
Additional Trains configuration files
You can create additional Trains configuration files using the
--file
option oftrains-init
.For example, create a configuration file to use inside Docker containers when executing a Task:
trains-init --file MyOtherTrains.conf
and then specify it using the
TRAINS_CONFIG_FILE
environment variable inside the container:TRAINS_CONFIG_FILE = MyOtherTrains.conf
For more information about running experiments inside Docker containers, see Trains Agent Use Case Examples and Trains Agent Reference pages.
The setup begins.
TRAINS SDK setup process Please create new trains credentials through the profile page in your trains web app (e.g. http://localhost:8080/profile) In the profile page, press "Create new credentials", then press "Copy to clipboard". Paste copied configuration here:
-
At the command prompt
Paste copied configuration here:
, copy and paste the Trains credentials you create as follows:- Open the Trains Web-App (UI) in your browser (if you are using are demo Trains Server, go to https://demoapp.trains.allegro.ai/dashboard).
- Click the PROFILE page.
- Click Create new credentials.
- Click Copy to clipboard.
- In your terminal session, paste your credentials and press Enter.
-
Enter the API server (press Enter to accept the default value which is based on your Trains credentials):
Detected credentials key="********************" secret="*******" API Host configured to: [https://demoapi.trains.allegro.ai]
-
Enter your Trains Server web server (press Enter to accept the default value which is based on your previous response):
demoapi.trains.allegro.ai is the api server, we need the web server. Replacing 'demoapi.' with 'demoapp.' Web Application Host configured to: [https://demoapp.trains.allegro.ai]
-
Enter your Trains file server (press Enter to accept the default value which is based on your previous response):
File Store Host configured to: [https://demofiles.trains.allegro.ai]
-
Your Trains configuration file is verified and created.
TRAINS Hosts configuration: Web App: https://demoapp.trains.allegro.ai API: https://demoapi.trains.allegro.ai File Store: https://demofiles.trains.allegro.ai Verifying credentials ... Credentials verified! New configuration stored in /home/<username>/trains.conf TRAINS setup completed successfully.
The configuration file's location depends upon your operating system:
- Linux - ~/trains.conf`
- Mac -
$HOME/trains.conf
- Windows -
\User\<username>\trains.conf
Step 3. Add storage credentials to your Trains configuration file
AWS S3
For AWS S3, you can use default credentials for all buckets, or specify credentials per individual buckets.
See the AWS documentation for restrictions and limitations on bucket naming.
-
S3 default credentials - In your configuration file
aws.s3
section, insert yourkey
,secret
, andregion
default credentials.aws { s3 { # S3 credentials, used for read/write access by various SDK elements # default, used for any bucket not specified below key: "my-access-key" secret: "my-secret-key" region: "my-region" credentials: [ # specifies key/secret credentials to use when handling s3 urls (read or write) # { # bucket: "my-bucket-name" # key: "my-access-key" # secret: "my-secret-key" # }, # { # # This will apply to all buckets in this host (unless key/value is specifically provided for a given bucket) # host: "my-minio-host:9000" # key: "12345678" # secret: "12345678" # multipart: false # secure: false # } ] } boto3 { pool_connections: 512 max_multipart_concurrency: 16 } }
-
S3 credentials per individual bucket - In the
credentials
section, uncomment the lines for a set of bucket
credentials and insert yourbucket
,key
, andsecret
.aws { s3 { # S3 credentials, used for read/write access by various SDK elements # default, used for any bucket not specified below key: "" secret: "" region: "" credentials: [ # specifies key/secret credentials to use when handling s3 urls (read or write) { bucket: "my-bucket-name" key: "my-access-key" secret: "my-secret-key" }, # { # # This will apply to all buckets in this host (unless key/value is specifically provided for a given bucket) # host: "my-minio-host:9000" # key: "12345678" # secret: "12345678" # multipart: false # secure: false # } ] } boto3 { pool_connections: 512 max_multipart_concurrency: 16 } }
Azure Storage
- Azure Storage credentials - Create an
azure.storage.containers
section and specify your credentials.azure.storage { containers: [ { # Replace YourAccountName with your Azure Storage account name. account_name: "YourAccountName" # Replace YourAccountKey with your Azure Storage account key. account_key: "YourAccountKey" # Container is optional, replace YourContainer with your container name. container_name: "YourContainer" } ] } }
Google Cloud Storage
For Google Cloud Storage, you can use default project credentials for all buckets, or specify credentials per individual buckets and sub-directory.
-
Google Cloud Storage credentials for a project - In the
google.storage
section, modifytrains.conf
by specifying yourproject
andcredentials_json
file:google.storage { # # Default project and credentials file # # Will be used when no bucket configuration is found project: "trains" credentials_json: "/path/to/credentials.json" # Specific credentials per bucket and sub directory # credentials = [ # { # bucket: "my-bucket" # subdir: "path/in/bucket" # Not required # project: "trains" # credentials_json: "/path/to/credentials.json" # }, # ] }
-
Google Cloud Storage credentials for each bucket and sub-directory, specify a set of credentials as follows (use
subdir
for sub-directories in your bucket, only):google.storage { # # Default project and credentials file # # Will be used when no bucket configuration is found # project: "trains" # credentials_json: "/path/to/credentials.json" # # Specific credentials per bucket and sub directory credentials = [ { bucket: "my-bucket" subdir: "path/in/bucket" # Not required project: "trains" credentials_json: "/path/to/credentials.json" }, ] }
Advanced Google Cloud Storage users
Advanced users can configure Google Cloud Storage templates for the Trains configuration.