Hyperparameters Reporting

The example hyper_parameters.py demonstrates:

  • Trains automatic logging of:
    • argparse command line arguments.
  • absl-py flags DEFINEs (e.g., DEFINE_string, DEFINE_integer, and DEFINE_boolean).
  • Trains support of explicit hyperparameter reporting:
    • Defining your own hyperparameters, using a parameter dictionary and connecting it to a Task.

Trains tracks hyperparameters with the experiment, including changes to them. Trains reports these hyperparameters in the Trains Web (UI), experiment details, HYPER PARAMETERS tab. When the script runs, it creates an experiment named hyper-parameters example, which is associated with the examples project.

argparse command line arguments

By creating an ArgumentParser() object, adding arguments, and parsing the object, Trains automatically logs those command line arguments.

parser = ArgumentParser()
parser.add_argument('--argparser_int_value', help='integer value', type=int, default=1)
parser.add_argument('--argparser_disabled', action='store_true', default=False, help='disables something')
parser.add_argument('--argparser_str_value', help='string value', default='a string')

args = parser.parse_args()

absl-py flags DEFINES

Trains automatically logs absl-py flags DEFINEs, whether they are defined before or after the Task is initialized.

flags.DEFINE_string('echo', None, 'Text to echo.')
flags.DEFINE_string('another_str', 'My string', 'A string', module_name='test')

task = Task.init(project_name='examples', task_name='hyper-parameters example')

flags.DEFINE_integer('echo3', 3, 'Text to echo.')

flags.DEFINE_string('echo5', '5', 'Text to echo.', module_name='test')

Hyperparameter dictionaries

To log a dictionary of parameters, create the dictionary and then call the Task.connect method. After connecting the dictionary, Trains updates changes to it for tracking with the experiment.

parameters = {
    'list': [1, 2, 3],
    'dict': {'a': 1, 'b': 2},
    'tuple': (1, 2, 3),
    'int': 3,
    'float': 2.2,
    'string': 'my string',
}

parameters = task.connect(parameters)

# adding new parameter after connect (will be logged as well)
parameters['new_param'] = 'this is new'

# changing the value of a parameter (new value will be stored instead of previous one)
parameters['float'] = '9.9'