idmtools_platform_comps.utils.general module

idmtools general status.

Copyright 2021, Bill & Melinda Gates Foundation. All rights reserved.

idmtools_platform_comps.utils.general.fatal_code(e: Exception) → bool

Uses to determine if we should stop retrying based on request status code.

Parameters:

e – Exception to check

Returns:

True is exception is a request and status code matches 404

idmtools_platform_comps.utils.general.convert_comps_status(comps_status: COMPS.Data.Simulation.SimulationState) → idmtools.core.enums.EntityStatus

Convert status from COMPS to IDMTools.

Parameters:

comps_status – Status in Comps

Returns:

EntityStatus

idmtools_platform_comps.utils.general.convert_comps_workitem_status(comps_status: COMPS.Data.WorkItem.WorkItemState) → idmtools.core.enums.EntityStatus

Convert status from COMPS to IDMTools.

Created = 0 # WorkItem has been saved to the database CommissionRequested = 5 # WorkItem is ready to be processed by the next available worker of the correct type Commissioned = 10 # WorkItem has been commissioned to a worker of the correct type and is beginning execution Validating = 30 # WorkItem is being validated Running = 40 # WorkItem is currently running Waiting = 50 # WorkItem is waiting for dependent items to complete ResumeRequested = 60 # Dependent items have completed and WorkItem is ready to be processed by the next available worker of the correct type CancelRequested = 80 # WorkItem cancellation was requested Canceled = 90 # WorkItem was successfully canceled Resumed = 100 # WorkItem has been claimed by a worker of the correct type and is resuming Canceling = 120 # WorkItem is in the process of being canceled by the worker Succeeded = 130 # WorkItem completed successfully Failed = 140 # WorkItem failed

Parameters:

comps_status – Status in Comps

Returns:

EntityStatus

idmtools_platform_comps.utils.general.clean_experiment_name(experiment_name: str) → str

Enforce any COMPS-specific demands on experiment names.

Parameters:

experiment_name – name of the experiment

Returns:the experiment name allowed for use

idmtools_platform_comps.utils.general.get_file_from_collection(platform: idmtools.entities.iplatform.IPlatform, collection_id: uuid.UUID, file_path: str) → bytearray

Retrieve a file from an asset collection.

Parameters:

  • platform – Platform object to use

  • collection_id – Asset Collection ID

  • file_path – Path within collection

Examples:: >>> import uuid >>> get_file_from_collection(platform, uuid.UUID(“fc461146-3b2a-441f-bc51-0bff3a9c1ba0”), “StdOut.txt”)

Returns:

Object Byte Array

idmtools_platform_comps.utils.general.get_file_as_generator(file: Union[COMPS.Data.SimulationFile.SimulationFile, COMPS.Data.AssetCollectionFile.AssetCollectionFile, COMPS.Data.AssetFile.AssetFile, COMPS.Data.WorkItemFile.WorkItemFile, COMPS.Data.OutputFileMetadata.OutputFileMetadata], chunk_size: int = 128, resume_byte_pos: Optional[int] = None) → Generator[bytearray, None, None]

Get file as a generator.

Parameters:

  • file – File to stream contents through a generator

  • chunk_size – Size of chunks to load

  • resume_byte_pos – Optional start of download

Returns:

Generator for file content

class idmtools_platform_comps.utils.general.Workitem

Bases: object

SimpleItem to define workitem for proxy purposes.

Notes

  • TODO deprecate this if possible

idmtools_platform_comps.utils.general.get_asset_for_comps_item(platform: idmtools.entities.iplatform.IPlatform, item: idmtools.core.interfaces.ientity.IEntity, files: List[str], cache=None, load_children: Optional[List[str]] = None, comps_item: Optional[Union[COMPS.Data.Experiment.Experiment, idmtools_platform_comps.utils.general.Workitem, COMPS.Data.Simulation.Simulation]] = None) → Dict[str, bytearray]

Retrieve assets from an Entity(Simulation, Experiment, WorkItem).

Parameters:

  • platform – Platform Object to use

  • item – Item to fetch assets from

  • files – List of file names to retrieve

  • cache – Cache object to use

  • load_children – Optional Load children fields

  • comps_item – Optional comps item

Returns:

Dictionary in structure of filename -> bytearray

idmtools_platform_comps.utils.general.update_item(platform: idmtools.entities.iplatform.IPlatform, item_id: str, item_type: idmtools.core.enums.ItemType, tags: Optional[dict] = None, name: Optional[str] = None)

Utility function to update existing COMPS experiment/simulation/workitem’s tags.

For example, you can add/update simulation’s tag once its post-process is done to mark the simulation with more meaningful text with tag/name :param platform: Platform :param item_id: experiment/simulation/workitem id :param item_type: The type of the object to be retrieved :param tags: tags dict for update :param name: name of experiment/simulation/workitem

Returns:

None