Deploying Trains Server: Google Cloud Platform
Trains is now ClearML
This documentation applies to the legacy Trains versions. For the latest documentation, see ClearML.
Deploy Trains Server on the Google Cloud Platform (GCP) using use one of our pre-built GCP Custom Images. Trains provides custom images for each released version of Trains Server. For a list of our pre-built custom images, see Trains Server GCP Custom Image.
After deploying Trains Server, configure the Trains Python Package for it, see Configuring Trains for Trains Server.
Default Trains Server service ports
After deploying Trains Server, the services expose the following node ports:
- Web server on
8080
- API server on
8008
- File Server on
8081
Default Trains Server storage paths
The persistent storage configuration:
- MongoDB:
/opt/trains/data/mongo/
- Elasticsearch:
/opt/trains/data/elastic_7/
- File Server:
/mnt/fileserver/
Importing the Custom Image to your GCP account
Before launching an instance using a Trains Server GCP Custom Image, import the image to your custom images list.
No upload of the image file is required. We provide links to image files stored in Google Storage.
To import the image to your custom images list:
- In the Cloud Console, go to the Images page.
- At the top of the page, click Create image.
- In Name, specify a unique name for the image.
- Optionally, specify an image family for your new image, or configure specific encryption settings for the image.
- In the Source menu, select Cloud Storage file.
-
Enter the Trains Server image bucket path (see Trains Server GCP Custom Image), for example:
allegro-files/trains-server/trains-server.tar.gz
. -
Click Create to import the image. The process can take several minutes depending on the size of the boot disk image.
For more information see Import the image to your custom images list in the Compute Engine Documentation.
Launching
Securing deployment
By default, Trains Server deploys as an open network. To restrict Trains Server access, follow the instructions in the Network and security section, on the "Configuring Trains Server" page.
To launch Trains Server using a GCP Custom Image, see the Manually importing virtual disks in the "Google Cloud Storage" documentation, Compute Engine documentation. For more information on Custom Images, see Custom Images in the "Compute Engine documentation".
The minimum requirements for Trains Server are:
- 2 vCPUs
- 7.5GB RAM
Restarting
To restart your Trains Server Docker deployment:
- To restart Trains Server, first stop and then restart the Docker containers by executing the following commands:
docker-compose -f /opt/trains/docker-compose.yml down docker-compose -f /opt/trains/docker-compose.yml up -d
Upgrading
To upgrade your Trains Server Docker deployment:
-
Shut down the docker containers. Execute the following command:
docker-compose -f docker-compose.yml down
-
If you are upgrading from Trains Server version 0.15 or older, a data migration is required before you upgrade. First follow these data migration instructions, and then continue this upgrade.
-
We recommend backing up your data and, if your configuration folder is not empty, backing up your configuration.
-
Download the latest
docker-compose.yml
file.curl https://raw.githubusercontent.com/allegroai/trains-server/master/docker-compose.yml -o /opt/trains/docker-compose.yml
-
Startup Trains Server. This automatically pulls the latest Trains Server build.
docker-compose -f /opt/trains/docker-compose.yml pull docker-compose -f /opt/trains/docker-compose.yml up -d
If issues arise during your upgrade, see the FAQ page, How do I fix Docker upgrade errors?.
Backing up and restoring data and configuration
The commands in this section are an example of how to backup and restore your data and configuration .
If your data and configuration folders are in /opt/trains
, then archive all data into ~/trains_backup_data.tgz
, and your configuration into ~/trains_backup_config.tgz
:
sudo tar czvf ~/trains_backup_data.tgz -C /opt/trains/data .
sudo tar czvf ~/trains_backup_config.tgz -C /opt/trains/config .
If you need to restore your data and configuration:
- Verify you have the backup files.
- Execute the following commands:
sudo rm -fR /opt/trains/data/* /opt/trains/config/* sudo tar -xzf ~/trains_backup_data.tgz -C /opt/trains/data sudo tar -xzf ~/trains_backup_config.tgz -C /opt/trains/config
Examples and use cases
For examples and use cases, see the Examples section in this User Guide documentation, and the examples folder of the Trains repository.
Trains Server GCP Custom Image
The following a section contains a list of Custom Image URLs (exported in different formats) for each released Trains Server version.
Latest version - v0.16.1
All release versions
- v0.16.1 - https://storage.googleapis.com/allegro-files/trains-server/trains-server-0-16-1.tar.gz
- v0.15.1 - https://storage.googleapis.com/allegro-files/trains-server/trains-server-0-15-1.tar.gz
- v0.15.0 - https://storage.googleapis.com/allegro-files/trains-server/trains-server-0-15-0.tar.gz
- v0.14.1 - https://storage.googleapis.com/allegro-files/trains-server/trains-server-0-14-1.tar.gz