Deploying Trains Server: Linux and macOS
Trains is now ClearML
This documentation applies to the legacy Trains versions. For the latest documentation, see ClearML.
Trains is now ClearML
This documentation applies to the legacy Trains versions. For the latest documentation, see ClearML.
Deploy the Trains Server in Linux or macOS using our prebuilt Docker image.
Trains on Docker Hub
For Trains docker images, including previous versions, see Trains in Docker Hub. However, pulling the Trains Docker image directly is not required. We provide a docker compose YAML file that does this for you. The docker compose file is included in the instructions on this page.
Prerequisites
For Linux users only:
- Your Linux distribution must support Docker. For more information, see this explanation in the Docker documentation.
- You must be logged in as a user with
sudo
privileges. - Use
bash
for all command-line instructions in this installation. - The ports
8080
,8081
, and8008
must be available for the Trains Server services.
Deploying
Securing deployment
By default, Trains Server deploys as an open network. To restrict Trains Server access, follow the instructions in the Network and security section, on the "Configuring Trains Server" page.
To launch your Trains Server on Linux or macOS:
-
Install Docker. The instructions depend upon your operating system:
- Linux - see Docker for Ubuntu.
- macOS - see Docker for OS X.
-
Verify the Docker CE installation. Execute the command:
docker run hello-world
The expected is output is:
Hello from Docker! This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. (amd64) 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal.
-
For macOS, only, increase the memory allocation in Docker Desktop to
8GB
.- In your top status bar, click the Docker icon.
- Click Preferences, Resources, Advanced, and then set the memory to at least
8192
. - Click Apply.
-
For Linux, only, install
docker-compose
. Execute the following commands (for more information, see Install Docker Compose in the Docker documentation):sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose
-
Increase
vm.max_map_count
for Elasticsearch in Docker. Execute the following commands, depending upon your operating system:-
Linux:
echo "vm.max_map_count=262144" > /tmp/99-trains.conf sudo mv /tmp/99-trains.conf /etc/sysctl.d/99-trains.conf sudo sysctl -w vm.max_map_count=262144 sudo service docker restart
-
macOS:
screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty sysctl -w vm.max_map_count=262144
-
-
Remove any previous installation of Trains Server.
This clears all existing Trains SDK databases.
sudo rm -R /opt/trains/
-
Create local directories for the databases and storage.
sudo mkdir -p /opt/trains/data/elastic_7 sudo mkdir -p /opt/trains/data/mongo/db sudo mkdir -p /opt/trains/data/mongo/configdb sudo mkdir -p /opt/trains/data/redis sudo mkdir -p /opt/trains/logs sudo mkdir -p /opt/trains/config sudo mkdir -p /opt/trains/data/fileserver
-
For macOS, only, do the following:
-
Open the Docker app.
-
Select Preferences.
-
On the File Sharing tab, add
/opt/trains
.
-
-
Grant access to the Dockers, depending upon your operating system.
-
Linux:
sudo chown -R 1000:1000 /opt/trains
-
macOS:
sudo chown -R $(whoami):staff /opt/trains
-
-
Download the Trains Server docker-compose YAML file.
sudo curl https://raw.githubusercontent.com/allegroai/trains-server/master/docker-compose.yml -o /opt/trains/docker-compose.yml
-
For Linux only, configure the Trains Agent Services. If
TRAINS_HOST_IP
is not provided, then Trains Agent Services will use the external public address of the Trains Server. IfTRAINS_AGENT_GIT_USER
/TRAINS_AGENT_GIT_PASS
are not provided, then Trains Agent Services will not be able to access any private repositories for running service tasks.export TRAINS_HOST_IP=server_host_ip_here export TRAINS_AGENT_GIT_USER=git_username_here export TRAINS_AGENT_GIT_PASS=git_password_here
-
Run
docker-compose
with the downloaded configuration file.docker-compose -f /opt/trains/docker-compose.yml up -d
Your server is now running on http://localhost:8080.
Port mapping
After deploying Trains Server, the services expose the following ports:
- Web server on port
8080
- API server on port
8008
- File server on port
8081
Restarting
To restart your Trains Server Docker deployment:
- To restart Trains Server, first stop and then restart the Docker containers by executing the following commands:
docker-compose -f /opt/trains/docker-compose.yml down docker-compose -f /opt/trains/docker-compose.yml up -d
Upgrading
Important: Upgrading from v0.14 or older
For Linux only, if you are upgrading from Trains Server v0.14 or older, configure the Trains Agent Services
- If
TRAINS_HOST_IP
is not provided, then Trains Agent Services will use the external public address of the Trains Server. - If
TRAINS_AGENT_GIT_USER
/TRAINS_AGENT_GIT_PASS
are not provided, then Trains Agent Services will not be able to access any private repositories for running service tasks.export TRAINS_HOST_IP=server_host_ip_here export TRAINS_AGENT_GIT_USER=git_username_here export TRAINS_AGENT_GIT_PASS=git_password_here
To upgrade your Trains Server Docker deployment:
-
Shutdown Trains Server. Executing the following command (which assumes the configuration file is in the environment path).
docker-compose -f docker-compose.yml down
-
If you are upgrading from Trains Server version 0.15 or older, a data migration is required before you upgrade. First follow these data migration instructions, and then continue this upgrade.
-
We recommend backing up your data and, if your configuration folder is not empty, backing up your configuration.
-
Download the latest
docker-compose.yml
file.curl https://raw.githubusercontent.com/allegroai/trains-server/master/docker-compose.yml -o /opt/trains/docker-compose.yml
-
Startup Trains Server. This automatically pulls the latest Trains Server build.
docker-compose -f /opt/trains/docker-compose.yml pull docker-compose -f /opt/trains/docker-compose.yml up -d
If issues arise during your upgrade, see the FAQ page, How do I fix Docker upgrade errors?.
Backing up and restoring data and configuration
The commands in this section are an example of how to backup and restore your data and configuration .
If your data and configuration folders are in /opt/trains
, then archive all data into ~/trains_backup_data.tgz
, and your configuration into ~/trains_backup_config.tgz
:
sudo tar czvf ~/trains_backup_data.tgz -C /opt/trains/data .
sudo tar czvf ~/trains_backup_config.tgz -C /opt/trains/config .
If you need to restore your data and configuration:
- Verify you have the backup files.
- Execute the following commands:
sudo rm -fR /opt/trains/data/* /opt/trains/config/* sudo tar -xzf ~/trains_backup_data.tgz -C /opt/trains/data sudo tar -xzf ~/trains_backup_config.tgz -C /opt/trains/config