Deploying Trains Server: Google Cloud Platform

Trains is now ClearML

This documentation applies to the legacy Trains versions. For the latest documentation, see ClearML.

Deploy Trains Server on the Google Cloud Platform (GCP) using use one of our pre-built GCP Custom Images. Trains provides custom images for each released version of Trains Server. For a list of our pre-built custom images, see Trains Server GCP Custom Image.

After deploying Trains Server, configure the Trains Python Package for it, see Configuring Trains for Trains Server.

Default Trains Server service ports

After deploying Trains Server, the services expose the following node ports:

Web server on 8080
API server on 8008
File Server on 8081

Default Trains Server storage paths

The persistent storage configuration:

MongoDB: /opt/trains/data/mongo/
Elasticsearch: /opt/trains/data/elastic_7/
File Server: /mnt/fileserver/

Importing the Custom Image to your GCP account

Before launching an instance using a Trains Server GCP Custom Image, import the image to your custom images list.

No upload of the image file is required. We provide links to image files stored in Google Storage.

To import the image to your custom images list:

In the Cloud Console, go to the Images page.
At the top of the page, click Create image.
In Name, specify a unique name for the image.
Optionally, specify an image family for your new image, or configure specific encryption settings for the image.
In the Source menu, select Cloud Storage file.
Enter the Trains Server image bucket path (see Trains Server GCP Custom Image), for example: allegro-files/trains-server/trains-server.tar.gz.
Click Create to import the image. The process can take several minutes depending on the size of the boot disk image.

For more information see Import the image to your custom images list in the Compute Engine Documentation.

Launching

Securing deployment

By default, Trains Server deploys as an open network. To restrict Trains Server access, follow the instructions in the Network and security section, on the "Configuring Trains Server" page.

To launch Trains Server using a GCP Custom Image, see the Manually importing virtual disks in the "Google Cloud Storage" documentation, Compute Engine documentation. For more information on Custom Images, see Custom Images in the "Compute Engine documentation".

The minimum requirements for Trains Server are:

2 vCPUs
7.5GB RAM

Restarting

To restart your Trains Server Docker deployment:

To restart Trains Server, first stop and then restart the Docker containers by executing the following commands:
```
docker-compose -f /opt/trains/docker-compose.yml down
docker-compose -f /opt/trains/docker-compose.yml up -d
```

Upgrading

To upgrade your Trains Server Docker deployment:

Shut down the docker containers. Execute the following command:
```
docker-compose -f docker-compose.yml down
```
If you are upgrading from Trains Server version 0.15 or older, a data migration is required before you upgrade. First follow these data migration instructions, and then continue this upgrade.
We recommend backing up your data and, if your configuration folder is not empty, backing up your configuration.

Download the latest docker-compose.yml file.

curl https://raw.githubusercontent.com/allegroai/trains-server/master/docker-compose.yml -o /opt/trains/docker-compose.yml

Startup Trains Server. This automatically pulls the latest Trains Server build.

docker-compose -f /opt/trains/docker-compose.yml pull
docker-compose -f /opt/trains/docker-compose.yml up -d

If issues arise during your upgrade, see the FAQ page, How do I fix Docker upgrade errors?.

Backing up and restoring data and configuration

The commands in this section are an example of how to backup and restore your data and configuration .

If your data and configuration folders are in /opt/trains, then archive all data into ~/trains_backup_data.tgz, and your configuration into ~/trains_backup_config.tgz:

sudo tar czvf ~/trains_backup_data.tgz -C /opt/trains/data .
sudo tar czvf ~/trains_backup_config.tgz -C /opt/trains/config .

If you need to restore your data and configuration:

Verify you have the backup files.

Execute the following commands:

sudo rm -fR /opt/trains/data/* /opt/trains/config/*
sudo tar -xzf ~/trains_backup_data.tgz -C /opt/trains/data
sudo tar -xzf ~/trains_backup_config.tgz -C /opt/trains/config

Examples and use cases

For examples and use cases, see the Examples section in this User Guide documentation, and the examples folder of the Trains repository.

Trains Server GCP Custom Image

The following a section contains a list of Custom Image URLs (exported in different formats) for each released Trains Server version.