Table Of Contents

  1. Docs
  2. Model Module

Model Module

class trains.model.InputModel

Load an existing model in the system, search by model id. The Model will be read-only and can be used to pre initialize a network We can connect the model to a task as input model, then when running remotely override it with the UI.

Parameters

model_id (str) – The Trains Id (system UUID) of the input model whose metadata the Trains Server (backend) stores.

classmethod import_model(weights_url, config_text=None, config_dict=None, label_enumeration=None, name=None, tags=None, comment=None, is_package=False, create_as_published=False, framework=None)

Create an InputModel object from a pre-trained model by specifying the URL of an initial weight files. Optionally, input a configuration, label enumeration, name for the model, tags describing the model, comment as a description of the model, indicate whether the model is a package, specify the model’s framework, and indicate whether to immediately set the model’s status to Published. The model is read-only.

The Trains Server (backend) may already store the model’s URL. If the input model’s URL is not stored, meaning the model is new, then it is imported and Trains stores its metadata. If the URL is already stored, the import process stops, Trains issues a warning message, and Trains reuses the model.

In your Python experiment script, after importing the model, you can connect it to the main execution Task as an input model using InputModel.connect() or Task.connect(). That initializes the network.

Note

Using the Trains Web-App (user interface), you can reuse imported models and switch models in experiments.

Parameters

  • weights_url (str) –

    A valid URL for the initial weights file. If the Trains Web-App (backend) already stores the metadata of a model with the same URL, that existing model is returned and Trains ignores all other parameters.

    For example:

    • https://domain.com/file.bin

    • s3://bucket/file.bin

    • file:///home/user/file.bin

  • config_text (unconstrained text string) – The configuration as a string. This is usually the content of a configuration dictionary file. Specify config_text or config_dict, but not both.

  • config_dict (dict) – The configuration as a dictionary. Specify config_text or config_dict, but not both.

  • label_enumeration (dict) –

    The label enumeration dictionary of string (label) to integer (value) pairs. (Optional)

    For example:

    {
     'background': 0,
     'person': 1
}

  • name (str) – The name of the newly imported model. (Optional)

  • tags (list(str)) – The list of tags which describe the model. (Optional)

  • comment (str) – A comment / description for the model. (Optional)

  • is_package (bool) –

    Is the imported weights file is a package? (Optional)

    • True - Is a package. Add a package tag to the model.

    • False - Is not a package. Do not add a package tag. (Default)

  • create_as_published (bool) –

    Set the model’s status to Published? (Optional)

    • True - Set the status to Published.

    • False - Do not set the status to Published. The status will be Draft. (Default)

  • framework (str or Framework object) – The framework of the model. (Optional)

Returns

The imported model or existing model (see above).

Return type

A model object.

classmethod load_model(weights_url, load_archived=False)

Load an already registered model based on a pre-existing model file (link must be valid).

If the url to the weights file already exists, the returned object is a Model representing the loaded Model If there could not be found any registered model Model with the specified url, None is returned.

Parameters

  • weights_url – valid url for the weights file (string). examples: “https://domain.com/file.bin” or “s3://bucket/file.bin” or “file:///home/user/file.bin”. NOTE: if a model with the exact same URL exists, it will be used, and all other arguments will be ignored.

  • load_archived (bool) – If True return registered Model with even if they are archived, otherwise archived models are ignored,

Return Model

InputModel object or None if no model could be found

classmethod empty(config_text=None, config_dict=None, label_enumeration=None)

Create an empty model object. Later, you can assign a model to the empty model object.

Parameters

  • config_text (unconstrained text string) – The model configuration as a string. This is usually the content of a configuration dictionary file. Specify config_text or config_dict, but not both.

  • config_dict (dict) – The model configuration as a dictionary. Specify config_text or config_dict, but not both.

  • label_enumeration (dict) –

    The label enumeration dictionary of string (label) to integer (value) pairs. (Optional)

    For example:

    {
     'background': 0,
     'person': 1
}

property id

The Id (system UUID) of the model.

Returns

The model id.

Return type

str

connect(task)

Connect the current model to a Task object, if the model is preexisting. Preexisting models include:

  • Imported models (InputModel objects created using the import_model() method).

  • Models whose metadata is already in the Trains platform, meaning the InputModel object is instantiated from the InputModel class specifying the the model’s Trains Id as an argument.

  • Models whose origin is not Trains that are used to create an InputModel object. For example, models created using TensorFlow models.

When the experiment is executed remotely in a worker, the input model already specified in the experiment is used.

Note

The Trains Web-App allows you to switch one input model for another and then enqueue the experiment to execute in a worker.

Parameters

task (object) – A Task object.

property comment

The comment for the model. Also, use for a model description.

Returns

The model comment / description.

Return type

str

property config_dict

The configuration as a dictionary, parsed from the design text. This usually represents the model configuration. For example, prototxt, an ini file, or Python code to evaluate.

Returns

The configuration.

Return type

dict

property config_text

The configuration as a string. For example, prototxt, an ini file, or Python code to evaluate.

Returns

The configuration.

Return type

str

get_local_copy(extract_archive=True)

Retrieve a valid link to the model file(s). If the model URL is a file system link, it will be returned directly. If the model URL is points to a remote location (http/s3/gs etc.), it will download the file(s) and return the temporary location of the downloaded model.

Parameters

extract_archive (bool) – If True and the model is of type ‘packaged’ (e.g. TensorFlow compressed folder) The returned path will be a temporary folder containing the archive content

Return str

a local path to the model (or a downloaded copy of it)

get_weights()

Download the base model and return the locally stored filename.

Returns

The locally stored file.

Return type

str

get_weights_package(return_path=False)

Download the base model package into a temporary directory (extract the files), or return a list of the locally stored filenames.

Parameters

return_path (bool) –

Return the model weights or a list of filenames? (Optional)

  • True - Download the model weights into a temporary directory, and return the temporary directory path.

  • False - Return a list of the locally stored filenames. (Default)

Returns

The model weights, or a list of the locally stored filenames.

Return type

package or path

property labels

The label enumeration of string (label) to integer (value) pairs.

Returns

A dictionary containing labels enumeration, where the keys are labels and the values as integers.

Return type

dict

property name

The name of the model.

Returns

The model name.

Return type

str

publish()

Set the model to the status published and for public use. If the model’s status is already published, then this method is a no-op.

property tags

A list of tags describing the model.

Returns

The list of tags.

Return type

list(str)

property task

Return the creating task id (str)

Return str

Task ID

property url

Return the url of the model file (or archived files)

Return str

Model file URL

class trains.model.OutputModel

Create an output model for a Task (experiment) to store the training results.

The OutputModel object is always connected to a Task object, because it is instantiated with a Task object as an argument. It is, therefore, automatically registered as the Task’s (experiment’s) output model.

The OutputModel object is read-write.

A common use case is to reuse the OutputModel object, and override the weights after storing a model snapshot. Another use case is to create multiple OutputModel objects for a Task (experiment), and after a new high score is found, store a model snapshot.

If the model configuration and / or the model’s label enumeration are None, then the output model is initialized with the values from the Task object’s input model.

Note

When executing a Task (experiment) remotely in a worker, you can modify the model configuration and / or model’s label enumeration using the Trains Web-App.

Create a new model and immediately connect it to a task.

We do not allow for Model creation without a task, so we always keep track on how we created the models In remote execution, Model parameters can be overridden by the Task (such as model configuration & label enumerator)

Parameters

  • task (Task) – The Task object with which the OutputModel object is associated.

  • config_text (unconstrained text string) – The configuration as a string. This is usually the content of a configuration dictionary file. Specify config_text or config_dict, but not both.

  • config_dict (dict) – The configuration as a dictionary. Specify config_dict or config_text, but not both.

  • label_enumeration (dict) –

    The label enumeration dictionary of string (label) to integer (value) pairs. (Optional)

    For example:

    {
     'background': 0,
     'person': 1
}

  • name (str) – The name for the newly created model. (Optional)

  • tags (list(str)) – A list of strings which are tags for the model. (Optional)

  • comment (str) – A comment / description for the model. (Optional)

  • framework (str or Framework object) – The framework of the model or a Framework object. (Optional)

property config_text

Get the configuration as a string. For example, prototxt, an ini file, or Python code to evaluate.

Returns

The configuration.

Return type

str

property config_dict

Get the configuration as a dictionary parsed from the config_text text. This usually represents the model configuration. For example, from prototxt to ini file or python code to evaluate.

Returns

The configuration.

Return type

dict

property labels

Get the label enumeration as a dictionary of string (label) to integer (value) pairs.

For example:

{
     'background': 0,
     'person': 1
}
Returns

The label enumeration.

Return type

dict

connect(task)

Connect the current model to a Task object, if the model is a preexisting model. Preexisting models include:

  • Imported models.

  • Models whose metadata the Trains Server (backend) is already storing.

  • Models from another source, such as frameworks like TensorFlow.

Parameters

task (object) – A Task object.

set_upload_destination(uri)

Set the URI of the storage destination for uploaded model weight files. Supported storage destinations include S3, Google Cloud Storage), and file locations.

Using this method, files uploads are separate and then a link to each is stored in the model object.

Note

For storage requiring credentials, the credentials are stored in the Trains configuration file, ~/trains/conf.

Parameters

uri (str) –

The URI of the upload storage destination.

For example:

  • s3://bucket/directory/

  • file:///tmp/debug/

Returns

The status of whether the storage destination schema is supported.

  • True - The storage destination scheme is supported.

  • False - The storage destination scheme is not supported.

Return type

bool

update_weights(weights_filename=None, upload_uri=None, target_filename=None, auto_delete_file=True, register_uri=None, iteration=None, update_comment=True)

Update the model weights from a locally stored model filename.

Note

Uploading the model is a background process. The call to the method update_weights() returns immediately.

Parameters

  • weights_filename (str) – The name of the locally stored weights file to upload. Specify weights_filename or register_uri, but not both.

  • upload_uri (str) – The URI of the storage destination for model weights upload. The default value is the previously used URI. (Optional)

  • target_filename (str) – The newly created filename in the storage destination location. The default value is the weights_filename value. (Optional)

  • auto_delete_file (bool) –

    Delete the temporary file after uploading? (Optional)

    • True - Delete (Default)

    • False - Do not delete

  • register_uri (str) – The URI of an already uploaded weights file. The URI must be valid. Specify register_uri or weights_filename, but not both.

  • update_comment (bool) –

    Update the model comment with the local weights file name (to maintain provenance)? (Optional)

    • True - Update model comment (Default)

    • False - Do not update

Returns

The uploaded URI.

Return type

str

update_weights_package(weights_filenames=None, weights_path=None, upload_uri=None, target_filename=None, auto_delete_file=True, iteration=None)

Update the model weights from locally stored model files, or from directory containing multiple files.

Note

Uploading the model weights is a background process. The call to the method update_weights_package() returns immediately.

Parameters

  • weights_filenames (list(str)) – The file names of the locally stored model files. Specify weights_filenames or weights_path, but not both.

  • weights_path (str) – The directory path to a package. All the files in the directory will be uploaded. Specify weights_path or weights_filenames, but not both.

  • upload_uri (str) – The URI of the storage destination for the model weights upload. The default is the previously used URI. (Optional)

  • target_filename (str) – The newly created filename in the storage destination URI location. The default is the value specified in the weights_filename parameter. (Optional)

  • auto_delete_file (bool) –

    Delete temporary file after uploading? (Optional)

    • True - Delete (Default)

    • False - Do not delete

Returns

The uploaded URI for the weights package.

Return type

str

update_design(config_text=None, config_dict=None)

Update the model configuration. Store a blob of text for custom usage.

Note

The update_design() method’s behavior is lazy. The design update is only forced when the weights are updated.

Parameters

  • config_text (unconstrained text string) – The configuration as a string. This is usually the content of a configuration dictionary file. Specify config_text or config_dict, but not both.

  • config_dict (dict) – The configuration as a dictionary. Specify config_text or config_dict, but not both.

Returns

The status of the update.

  • True - Update successful.

  • False - Update not successful.

Return type

bool

property comment

The comment for the model. Also, use for a model description.

Returns

The model comment / description.

Return type

str

get_weights()

Download the base model and return the locally stored filename.

Returns

The locally stored file.

Return type

str

get_weights_package(return_path=False)

Download the base model package into a temporary directory (extract the files), or return a list of the locally stored filenames.

Parameters

return_path (bool) –

Return the model weights or a list of filenames? (Optional)

  • True - Download the model weights into a temporary directory, and return the temporary directory path.

  • False - Return a list of the locally stored filenames. (Default)

Returns

The model weights, or a list of the locally stored filenames.

Return type

package or path

property id

The Id (system UUID) of the model.

Returns

The model id.

Return type

str

property name

The name of the model.

Returns

The model name.

Return type

str

publish()

Set the model to the status published and for public use. If the model’s status is already published, then this method is a no-op.

property tags

A list of tags describing the model.

Returns

The list of tags.

Return type

list(str)

property task

Return the creating task id (str)

Return str

Task ID

update_labels(labels)

Update the label enumeration.

Parameters

labels (dict) –

The label enumeration dictionary of string (label) to integer (value) pairs.

For example:

{
     'background': 0,
     'person': 1
}

Returns

property url

Return the url of the model file (or archived files)

Return str

Model file URL

classmethod wait_for_uploads(timeout=None, max_num_uploads=None)

Wait for any pending or in-progress model uploads to complete. If no uploads are pending or in-progress, then the wait_for_uploads returns immediately.

Parameters

  • timeout (float) – The timeout interval to wait for uploads (seconds). (Optional).

  • max_num_uploads (int) – The maximum number of uploads to wait for. (Optional).

Logger Module
© 2020 • Allegro AI. All Rights Reserved.