idmtools.entities.experiment module#

Our Experiment class definition.

Experiments can be thought of as a metadata object analogous to a folder on a filesystem. An experiment is a container that contains one or more simulations. Before creations, experiment.simulations can be either a list of a TemplatedSimulations. TemplatedSimulations are useful for building large numbers of similar simulations such as sweeps.

Copyright 2021, Bill & Melinda Gates Foundation. All rights reserved.

class idmtools.entities.experiment.Experiment(_uid: str = None, _IItem__pre_creation_hooks: ~typing.List[~typing.Callable[[IItem, IPlatform], None]] = <factory>, _IItem__post_creation_hooks: ~typing.List[~typing.Callable[[IItem, IPlatform], None]] = <factory>, platform_id: str = None, _platform: IPlatform = None, parent_id: str = None, _parent: IEntity = None, status: ~idmtools.core.enums.EntityStatus = None, tags: ~typing.Dict[str, ~typing.Any] = <factory>, _platform_object: ~typing.Any = None, _IRunnableEntity__pre_run_hooks: ~typing.List[~typing.Callable[[IRunnableEntity, IPlatform], None]] = <factory>, _IRunnableEntity__post_run_hooks: ~typing.List[~typing.Callable[[IRunnableEntity, IPlatform], None]] = <factory>, name: str = None, assets: ~idmtools.assets.asset_collection.AssetCollection = <factory>, suite_id: str = None, task_type: str = 'idmtools.entities.command_task.CommandTask', platform_requirements: ~typing.Set[~idmtools.entities.platform_requirements.PlatformRequirements] = <factory>, simulations: dataclasses.InitVar[typing.Union[idmtools.core.interfaces.entity_container.EntityContainer, typing.Generator[ForwardRef('Simulation'), NoneType, NoneType], idmtools.entities.templated_simulation.TemplatedSimulations, typing.Iterator[ForwardRef('Simulation')]]] = <property object>, _Experiment__simulations: ~idmtools.core.interfaces.entity_container.EntityContainer | ~typing.Generator[Simulation, None, None] | ~idmtools.entities.templated_simulation.TemplatedSimulations | ~typing.Iterator[Simulation] = <factory>, gather_common_assets_from_task: bool = None, disable_default_pre_create: bool = False)[source]#

Bases: IAssetsEnabled, INamedEntity, IRunnableEntity

Class that represents a generic experiment.

This class needs to be implemented for each model type with specifics.

Parameters:

  • name – The experiment name.

  • assets – The asset collection for assets global to this experiment.

suite_id: str = None#

Suite ID

item_type: ItemType = 'Experiment'#

Item Item(always an experiment)

task_type: str = 'idmtools.entities.command_task.CommandTask'#

Task Type(defaults to command)

platform_requirements: Set[PlatformRequirements]#

List of Requirements for the task that a platform must meet to be able to run

frozen: bool = False#

Is the Experiment Frozen

gather_common_assets_from_task: bool = None#

Determines if we should gather assets from the first task. Only use when not using TemplatedSimulations

disable_default_pre_create: bool = False#

Determines if we should gather assets from the first task. Only use when not using TemplatedSimulations

post_creation(platform: IPlatform) → None[source]#

Post creation of experiments.

Parameters:

platform – Platform the experiment was created on

Returns:

None

property status#

Get status of experiment. Experiment status is based in simulations.

The first rule to be true is used. The rules are: * If simulations is a TemplatedSimulations we assume status is None if _platform_object is not set. * If simulations is a TemplatedSimulations we assume status is CREATED if _platform_object is set. * If simulations length is 0 or all simulations have a status of None, experiment status is none * If any simulation has a running status, experiment is considered running. * If any simulation has a created status and any other simulation has a FAILED or SUCCEEDED status, experiment is considered running. * If any simulation has a None status and any other simulation has a FAILED or SUCCEEDED status, experiment is considered Created. * If any simulation has a status of failed, experiment is considered failed. * If any simulation has a status of SUCCEEDED, experiment is considered SUCCEEDED. * If any simulation has a status of CREATED, experiment is considered CREATED.

Returns:

Status

property suite#

Suite the experiment belongs to.

Returns:

Suite

property parent#

Return parent object for item.

Returns:

Parent entity if set

display()[source]#

Display the experiment.

Returns:

None

pre_creation(platform: IPlatform, gather_assets=True) → None[source]#

Experiment pre_creation callback.

Parameters:

  • platform – Platform experiment is being created on

  • gather_assets – Determines if an experiment will try to gather the common assets or defer. It most cases, you want this enabled but when modifying existing experiments you may want to disable if there are new assets and the platform has performance hits to determine those assets

Returns:

None

Raises:

ValueError - If simulations length is 0 –

property done#

Return if an experiment has finished executing.

Returns:

True if all simulations have ran, False otherwise

property succeeded: bool#

Return if an experiment has succeeded. An experiment is succeeded when all simulations have succeeded.

Returns:

True if all simulations have succeeded, False otherwise

property any_failed: bool#

Return if an experiment has any simulation in failed state.

Returns:

True if all simulations have succeeded, False otherwise

property simulations: ExperimentParentIterator#

Simulation in this experiment

property simulation_count: int#

Return the total simulations.

Returns:

Length of simulations

refresh_simulations() → NoReturn[source]#

Refresh the simulations from the platform.

Returns:

None

refresh_simulations_status()[source]#

Refresh the simulation status.

Returns:

None

pre_getstate()[source]#

Return default values for pickle_ignore_fields().

Call before pickling.

gather_assets() -> AssetCollection(_uid=None, _IItem__pre_creation_hooks=[], _IItem__post_creation_hooks=[], platform_id=None, _platform=None, parent_id=None, _parent=None, status=None, tags={}, item_type=<ItemType.ASSETCOLLECTION: 'Asset Collection'>, _platform_object=None)[source]#

Gather all our assets for our experiment.

Returns:

Assets

classmethod from_task(task, name: str | None = None, tags: Dict[str, Any] | None = None, assets: AssetCollection | None = None, gather_common_assets_from_task: bool = True) → Experiment[source]#

Creates an Experiment with one Simulation from a task.

Parameters:

  • task – Task to use

  • assets – Asset collection to use for common tasks. Defaults to gather assets from task

  • name – Name of experiment

  • tags – Tags for the items

  • gather_common_assets_from_task – Whether we should attempt to gather assets from the Task object for the experiment. With large amounts of tasks, this can be expensive as we loop through all

Returns:

classmethod from_builder(builders: SimulationBuilder | List[SimulationBuilder], base_task: ITask, name: str | None = None, assets: AssetCollection | None = None, tags: Dict[str, Any] | None = None) → Experiment[source]#

Creates an experiment from a SimulationBuilder object(or list of builders.

Parameters:

  • builders – List of builder to create experiment from

  • base_task – Base task to use as template

  • name – Experiment name

  • assets – Experiment level assets

  • tags – Experiment tags

Returns:

Experiment object from the builders

classmethod from_template(template: TemplatedSimulations, name: str | None = None, assets: AssetCollection | None = None, tags: Dict[str, Any] | None = None) → Experiment[source]#

Creates an Experiment from a TemplatedSimulation object.

Parameters:

  • template – TemplatedSimulation object

  • name – Experiment name

  • assets – Experiment level assets

  • tags – Tags

Returns:

Experiment object from the TemplatedSimulation object

list_static_assets(children: bool = False, platform: IPlatform = None, **kwargs) → List[Asset][source]#

List assets that have been uploaded to a server already.

Parameters:

  • children – When set to true, simulation assets will be loaded as well

  • platform – Optional platform to load assets list from

  • **kwargs –

Returns:

List of assets

run(wait_until_done: bool = False, platform: IPlatform = None, regather_common_assets: bool = None, wait_on_done_progress: bool = True, **run_opts) → NoReturn[source]#

Runs an experiment on a platform.

Parameters:

  • wait_until_done – Whether we should wait on experiment to finish running as well. Defaults to False

  • platform – Platform object to use. If not specified, we first check object for platform object then the current context

  • regather_common_assets – Triggers gathering of assets for existing experiments. If not provided, we use the platforms default behaviour. See platform details for performance implications of this. For most platforms, it should be ok but for others, it could decrease performance when assets are not changing. It is important to note that when using this feature, ensure the previous simulations have finished provisioning. Failure to do so can lead to unexpected behaviour

  • wait_on_done_progress – Should experiment status be shown when waiting

  • **run_opts – Options to pass to the platform

Returns:

None

to_dict()[source]#

Convert experiment to dictionary.

Returns:

Dictionary of experiment.

classmethod from_id(item_id: str, platform: IPlatform = None, copy_assets: bool = False, **kwargs) → Experiment[source]#

Helper function to provide better intellisense to end users.

Parameters:

  • item_id – Item id to load

  • platform – Optional platform. Fallbacks to context

  • copy_assets – Allow copying assets on load. Makes modifying experiments easier when new assets are involved.

  • **kwargs – Optional arguments to be passed on to the platform

Returns:

Experiment loaded with ID

print(verbose: bool = False)[source]#

Print summary of experiment.

Parameters:

verbose – Verbose printing

Returns:

None

add_simulation(item: Simulation)[source]#

Adds a simulation to an experiment. :param item: Item to add

Returns:

None

add_simulations(item: List[Simulation] | TemplatedSimulations)[source]#

Extends experiment’s simulations. :param item: Item to extend

Returns:

None

__init__(_uid: str = None, _IItem__pre_creation_hooks: ~typing.List[~typing.Callable[[IItem, IPlatform], None]] = <factory>, _IItem__post_creation_hooks: ~typing.List[~typing.Callable[[IItem, IPlatform], None]] = <factory>, platform_id: str = None, _platform: IPlatform = None, parent_id: str = None, _parent: IEntity = None, status: ~idmtools.core.enums.EntityStatus = None, tags: ~typing.Dict[str, ~typing.Any] = <factory>, _platform_object: ~typing.Any = None, _IRunnableEntity__pre_run_hooks: ~typing.List[~typing.Callable[[IRunnableEntity, IPlatform], None]] = <factory>, _IRunnableEntity__post_run_hooks: ~typing.List[~typing.Callable[[IRunnableEntity, IPlatform], None]] = <factory>, name: str = None, assets: ~idmtools.assets.asset_collection.AssetCollection = <factory>, suite_id: str = None, task_type: str = 'idmtools.entities.command_task.CommandTask', platform_requirements: ~typing.Set[~idmtools.entities.platform_requirements.PlatformRequirements] = <factory>, simulations: dataclasses.InitVar[typing.Union[idmtools.core.interfaces.entity_container.EntityContainer, typing.Generator[ForwardRef('Simulation'), NoneType, NoneType], idmtools.entities.templated_simulation.TemplatedSimulations, typing.Iterator[ForwardRef('Simulation')]]] = <property object>, _Experiment__simulations: ~idmtools.core.interfaces.entity_container.EntityContainer | ~typing.Generator[Simulation, None, None] | ~idmtools.entities.templated_simulation.TemplatedSimulations | ~typing.Iterator[Simulation] = <factory>, gather_common_assets_from_task: bool = None, disable_default_pre_create: bool = False) → None#
class idmtools.entities.experiment.ExperimentSpecification[source]#

Bases: ExperimentPluginSpecification

ExperimentSpecification is the spec for Experiment plugins.

get_description() → str[source]#

Description of our plugin.

Returns:

Description

get(configuration: dict) → Experiment[source]#

Get experiment with configuration.

get_type() → Type[Experiment][source]#

Return the experiment type.

Returns:

Experiment type.