Version 0.15

Version 0.15.1

Trains

Features

  • Add direct plotly figure reporting (GitHub Issue #136).
  • Improve matplotlib integration (GitHub Issue #140).
  • Add Task.set_parent() (GitHub Issue #144).
  • Add Task.wait_for_status() (GitHub Issue #145).
  • Enhance framework save / load binding callbacks (pytorch/ignite Issue #1056).
  • Add TrainsJob.is_failed().
  • Add APIClient support using trains.backend_api.session.APIClient.
  • Add URL tracking when using StorageManager.get_local_copy() (log remote URL instead of local file copy).
  • Improve and expand examples.

Bug Fixes

  • Fix pytorch_tensorboard.py example (GitHub Issue #148).
  • Fix ssh://user@domain@server.com/ in Git repository (GitHub Issue #149).
  • Fix do not call make_deterministic() on import.
  • Fix initialization wizard (allow at most two verification retries, then print error) (GitHub trains-agent Issue #16).
  • Fix Google CoLab code/package detection.
  • Improve pytorch-ignite integration.
  • Fix TensorFlow version 2 and later histogram binding.
  • Fix Logger.tensorboard_single_series_per_graph.
  • Fix anonymous named models.
  • Fix incorrect entry point detection when called from Trains wrapper (e.g. TrainsLogger in Pytorch Ignite/Lightning).

Trains Server

Bug Fixes

  • Fix experiment clone dialog resets while typing (GitHub trains-server Issue 44).
  • Fix image-only plots are not displayed (GitHub Issue #140).
  • Fix last item in plots list is hidden.
  • Fix input model edit view contains a redundant save button.
  • Fix credentials popup shows incorrect URL for fileserver.
  • Fix experiment description does not allow multiple lines.
  • Fix tags filter ("No tags" selection, display only tags available for the project).
  • Fix cannot save empty model configuration.
  • Fix add tag dialog does not close after adding a tag.
  • Show active column filters for values even when values are unavailable in current project.
  • Fix project selection resets when cloning an experiment.
  • Fix several broken icons.
  • Fix cannot add tags in Results view.
  • Add warning when deleting a model/experiment tag.
  • New icons for experiment/task types.

Trains Agent

Features

  • Add Trains Agent Daemon and Services docker files.

Bug Fixes

Version 0.15.0

Trains

Features

  • Add automation support including hyper-parameters optimization (see example here)
  • Task.init() auto_connect_arg_parser argument can accept a dictionary disabling specific keys from the argparser (Trains Slack channel thread)
  • Allow worker_id override using TRAINS_WORKER_NAME environment variable (Trains Slack channel thread)
  • Support layout configuration for plotly objects using extra_layout argument in all Logger reporting methods (GitHub Issue #136).
  • Add Task.execute_remotely() to allow cloning and enqueuing a locally executed task (or stopping and re-enqueuing a remotely executed task) (GitHub Issue #128).
  • Add Parquet framework and model type
  • Support recursive model folder packaging
  • Add Task.get_reported_console_output() and Task.get_reported_scalars() to allow retrieval of reported output and scalar metrics
  • Add Task.add_requirements() to force requirement package into "installed packages"
  • Improve task reuse responsiveness
  • Add raise_on_error (default False) argument to Model.get_local_copy() and Artifact.get_local_copy() (GitHub trains-agent Issue #17).
  • Support Task.get_task() without project name (i.e. all projects)
  • Support using the file calling Task.init() as the task's script in case sys.argv doesn't point to a git repository
  • Support detecting and remotely executing code running from a module (i.e. -m module)
  • Add callback for framework save / load binding for better integration with pytorch/ignite (GitHub pytorch/ignite Issue #1056)
  • Support new task types provided in Trains Server v0.15.0
  • Add automation and distributed examples
  • Upgrade default pip version to <20.2

Bug Fixes

  • Fix exact_match_regex() in case of empty pattern (GitHub Issue #138).
  • Address deprecation warning and newer attrs versions in MetricsEventAdapter (GitHub Issue #134).
  • Fix issues with plotly support (Trains Slack channel thread and thread)
  • Fix default argument behavior to match argparse behavior
  • Fix OutputModel with task=None should use current task, if exists
  • Fix Task.get_task() to raise proper error on incorrect task_id
  • Fix Task.enqueue() to use an exact queue name match
  • Fix NaN, Inf and -Inf values display in reported table (not supported by JSON)
  • Limit max requirement size to 0.5MB
  • Fix issues with repository analysis
  • Fix StorageManager should only try to extract .zip files, Model should not auto extract package (GitHub trains-agent Issue #17).

Trains Server

Features

  • Add support for debug media (images, audio, video) (GitHub Issue #120).
  • Add support for previewing HTML debug samples.
  • Add experiment and model tag management (GitHub Issue #46).
  • Add support for more task types (see Trains v0.15.0 and Trains Agent v0.15.0 for related support and services mode).
  • Experiment comparison:
    • Add Last / Max (maximal) / Min (minimal) scalar values to parallel coordinates in experiment comparison (GitHub Issue #131).
    • Add deep comparison support to experiment comparision, including uncommitted changes by file with line diff and search of all text.
  • Enhance table columns:
    • Add dynamic column ordering to the experiments and models tables.
    • Add the User column to the experiments table.
    • Add the Description to experiments and models tables.
    • Add double click to table column resizers for column width reset.
    • Add column customization to the models table.
  • Add support for saving and sharing customized model table URLs by embedding table customizations in the URL.
  • Add a popup text editor for all text areas.
  • Move resource monitoring plots after the metrics plots in the experiment results SCALARS tab.
  • Add support for the clear_all parameter in tasks.reset to improve Trains SDK Task reuse.
  • Unify experiments table and comparison auto-refresh (switching between table and comparison auto refresh setting did not change).
  • Add a cancel button to all text searches.
  • Improve the experiment / details panel split to 75% / 25%.
  • Infrastructure:
    • Update pymongo and mongoengine to their latest versions.
    • Improve mongo performance by eliminating non-indexed queries.
    • Upgrade WebApp to Angular 9.
    • Use native javascript instead of date-fns.
  • Add survey invitation popup.

Bug Fixes

  • Fix init wizard, correctly display the input servers (GitHub Issue #19).
  • Improve docker host-mount support, use TRAINS_AGENT_DOCKER_HOST_MOUNT environment variable.
  • Support pip v20.1 local/http package reference in pip freeze.
  • Fix detached mode to correctly use cache folder slots.
  • Fix CUDA_VISIBLE_DEVICES should never be set to "all" (from the Trains Slack channel).
  • Do not monitor GPU when running with --cpu-only
  • Fix experiment comparison issues:
    • Scalar comparison type does not reflect displayed comparison (GitHub Issue #40).
    • Comparison animations overlapping other fields.
    • Cannot choose a metric after clearing it in experiment comparison parallel coordinates.
    • Adding a dimension without a value removes all lines in experiment comparison parallel coordinates.
  • Fix experiment view issues:
    • Custom URL gets erased when clicking on any tab on experiment.
    • All experiments re-fetched when navigating through tabs of an experiment.
    • Input and output models configuration box sizes differs in the experiments ARTIFACTS tab.
    • Changing experiment table / details pane split redraws scalars and plots.
    • Wrong download URL shown for S3 artifact links with special characters.
  • Fix experiment results view issues:
    • Browser zoom breaks labels (Ctrl + / Ctrl - / mouse wheel) in experiment results SCALARS tab (GitHub Issue #127).
    • Debug image name not shown if file cannot be loaded.
    • minio images with spaces in the path not displaying correctly
    • Plots should use the full panel space.
    • Plot relative time displaying incorrectly in experiment results SCALARS tab.
    • Wall time display in local time in experiment results SCALARS tab.
  • Fix experiment table issues:
    • Hyper parameters sorted as strings, not as numbers (GitHub Issue #124).
    • Prevent auto refresh reordering when the details panel is open.
    • Table resizing made the preferences wheel to jump over titles.
  • Fix experiment reset when both the input and output model are the same causes model deletion.
  • Fix experiment clone not working if the input model was deleted.
  • Fix missing trash icon (delete) if a custom column title is too long.
  • Fix display of aborted tasks in the recent experiments on the Homepage.

Trains Agent

Features

  • Add daemon Services Mode (daemon --services-mode), where the daemon spins a task in its own Docker container, and verifies start-up and shut-down. This allows multiple tasks to be launched simultaneously on the same machine (currently in CPU mode only), where each task service will register itself as a worker for the lifetime of the task.
  • Enhance build --docker mode.
    • Add --install-globally option to install required packages in the Docker's system Python.
    • Add --entry-point option to allow automatic task cloning when running the Docker container.
  • Support PyTorch Nightly builds using the agent.torch_nightly configuration flag. If true, the agent looks for a nightly build when a stable torch wheel is not found.
  • Add environment variables support for git user/password
    • Using TRAINS_AGENT_GIT_USER / TRAINS_AGENT_GIT_PASS.
    • Pass git credentials to dockerized experiment execution.
  • Support running code from module (i.e. -m in execution entry point).
  • Add daemon --create-queue to automatically create a queue and use it, if the queue name doesn't exist in the server.
  • Move --gpus and --cpu-only to worker args (used by daemon, execute and build).

Bug Fixes

  • Fix init wizard, correctly display the input servers (GitHub Issue #19).
  • Improve docker host-mount support, use TRAINS_AGENT_DOCKER_HOST_MOUNT environment variable.
  • Support pip v20.1 local/http package reference in pip freeze.
  • Fix detached mode to correctly use cache folder slots.
  • Fix CUDA_VISIBLE_DEVICES should never be set to "all" (from the Trains Slack channel).
  • Do not monitor GPU when running with --cpu-only