idmtools_platform_comps.comps_operations.simulation_operations module

idmtools comps simulation operations.

Copyright 2021, Bill & Melinda Gates Foundation. All rights reserved.

idmtools_platform_comps.comps_operations.simulation_operations.comps_batch_worker(simulations: List[Simulation], interface: CompsPlatformSimulationOperations, executor, num_cores: Optional[int] = None, priority: Optional[str] = None, asset_collection_id: Optional[Union[str, UUID]] = None, min_time_between_commissions: int = 10, **kwargs) List[Simulation]

Run batch worker.

Parameters:
  • simulations – Batch of simulation to process

  • interface – SimulationOperation Interface

  • executor – Thread/process executor

  • num_cores – Optional Number of core to allocate for MPI

  • priority – Optional Priority to set to

  • asset_collection_id – Override asset collection id

  • min_time_between_commissions – Minimum amount of time(in seconds) between calls to commission on an experiment

  • for (extra info) –

Returns:

List of Comps Simulations

class idmtools_platform_comps.comps_operations.simulation_operations.CompsPlatformSimulationOperations(platform: COMPSPlatform, platform_type: ~typing.Type = <class 'COMPS.Data.Simulation.Simulation'>)

Bases: IPlatformSimulationOperations

Provides simuilation operations to COMPSPlatform.

platform: COMPSPlatform
platform_type

alias of Simulation

get(simulation_id: UUID, columns: Optional[List[str]] = None, load_children: Optional[List[str]] = None, query_criteria: Optional[QueryCriteria] = None, **kwargs) Simulation

Get Simulation from Comps.

Parameters:
  • simulation_id – ID

  • columns – Optional list of columns to load. Defaults to “id”, “name”, “experiment_id”, “state”

  • load_children – Optional children to load. Defaults to “tags”, “configuration”

  • query_criteria – Optional query_criteria object to use your own custom criteria object

  • **kwargs

Returns:

COMPSSimulation

platform_create(simulation: Simulation, num_cores: Optional[int] = None, priority: Optional[str] = None, enable_platform_task_hooks: bool = True, asset_collection_id: Optional[Union[str, UUID]] = None, **kwargs) Simulation

Create Simulation on COMPS.

Parameters:
  • simulation – Simulation to create

  • num_cores – Optional number of MPI Cores to allocate

  • priority – Priority to load

  • enable_platform_task_hooks – Should platform task hoooks be ran

  • asset_collection_id – Override for asset collection id on sim

  • **kwargs – Expansion fields

Returns:

COMPS Simulation

to_comps_sim(simulation: Simulation, num_cores: Optional[int] = None, priority: Optional[str] = None, config: Optional[Configuration] = None, asset_collection_id: Optional[Union[str, UUID]] = None, **kwargs)

Covert IDMTools object to COMPS Object.

Parameters:
  • simulation – Simulation object to convert

  • num_cores – Optional Num of MPI Cores to allocate

  • priority – Optional Priority

  • config – Optional Configuration object

  • asset_collection_id

  • comps (**kwargs additional option for) –

Returns:

COMPS Simulation

get_simulation_config_from_simulation(simulation: Simulation, num_cores: Optional[int] = None, priority: Optional[str] = None, asset_collection_id: Optional[UUID] = None, **kwargs) Configuration

Get the comps configuration for a Simulation Object.

Parameters:
  • simulation – Simulation

  • num_cores – Optional Num of core for MPI

  • priority – Optional Priority

  • asset_collection_id – Override simulation asset_collection_id

  • comps (**kwargs additional option for) –

Returns:

Configuration

batch_create(simulations: List[Simulation], num_cores: Optional[int] = None, priority: Optional[str] = None, asset_collection_id: Optional[Union[str, UUID]] = None, **kwargs) List[Simulation]

Perform batch creation of Simulations.

Parameters:
  • simulations – Simulation to create

  • num_cores – Optional MPI Cores to allocate per simulation

  • priority – Optional Priority

  • asset_collection_id – Asset collection id for sim(overide experiment)

  • **kwargs – Future expansion

Returns:

List of COMPSSimulations that were created

get_parent(simulation: Any, **kwargs) Experiment

Get the parent of the simulation.

Parameters:
  • simulation – Simulation to load parent for

  • **kwargs

Returns:

COMPSExperiment

platform_run_item(simulation: Simulation, **kwargs)

For simulations, there is no running for COMPS.

send_assets(simulation: Simulation, comps_sim: Optional[Simulation] = None, add_metadata: bool = False, **kwargs)

Send assets to Simulation.

Parameters:
  • simulation – Simulation to send asset for

  • comps_sim – Optional COMPSSimulation object to prevent reloading it

  • add_metadata – Add idmtools metadata object

  • **kwargs

Returns:

None

refresh_status(simulation: Simulation, additional_columns: Optional[List[str]] = None, **kwargs)

Refresh status of a simulation.

Parameters:
  • simulation – Simulation to refresh

  • additional_columns – Optional additional columns to load from COMPS

  • **kwargs

Returns:

None

to_entity(simulation: Simulation, load_task: bool = False, parent: Optional[Experiment] = None, load_parent: bool = False, load_metadata: bool = False, load_cli_from_workorder: bool = False, **kwargs) Simulation

Convert COMPS simulation object to IDM Tools simulation object.

Parameters:
  • simulation – Simulation object

  • load_task – Should we load tasks. Defaults to No. This can increase the load items on fetchs

  • parent – Optional parent object to prevent reloads

  • load_parent – Force load of parent(Beware, This could cause loading loops)

  • load_metadata – Should we load metadata by default. If load task is enabled, this is also enabled

  • load_cli_from_workorder – Used with COMPS scheduling where the CLI is defined in our workorder

  • **kwargs

Returns:

Simulation object

get_asset_collection_from_comps_simulation(simulation: Simulation) Optional[AssetCollection]

Get assets from COMPS Simulation.

Parameters:

simulation – Simulation to get assets from

Returns:

Simulation Asset Collection, if any.

get_assets(simulation: Simulation, files: List[str], include_experiment_assets: bool = True, **kwargs) Dict[str, bytearray]

Fetch the files associated with a simulation.

Parameters:
  • simulation – Simulation

  • files – List of files to download

  • include_experiment_assets – Should we also load experiment assets?

  • **kwargs

Returns:

Dictionary of filename -> ByteArray

list_assets(simulation: Simulation, common_assets: bool = False, **kwargs) List[Asset]

List assets for a simulation.

Parameters:
  • simulation – Simulation to load data for

  • common_assets – Should we load asset files

  • **kwargs

Returns:

AssetCollection

retrieve_output_files(simulation: Simulation)

Retrieve the output files for a simulation.

Parameters:

simulation – Simulation to fetch files for

Returns:

List of output files for simulation

all_files(simulation: Simulation, common_assets: bool = False, outfiles: bool = True, **kwargs) List[Asset]

Returns all files for a specific simulation including experiments or non-assets.

Parameters:
  • simulation – Simulation all files

  • common_assets – Include experiment assets

  • outfiles – Include output files

  • **kwargs

Returns:

AssetCollection

create_sim_directory_map(simulation_id: str) Dict

Build simulation working directory mapping. :param simulation_id: simulation id

Returns:

Dict of simulation id as key and working dir as value

__init__(platform: COMPSPlatform, platform_type: ~typing.Type = <class 'COMPS.Data.Simulation.Simulation'>) None