idmtools_platform_comps.utils.general module#

idmtools general status.

Copyright 2021, Bill & Melinda Gates Foundation. All rights reserved.

idmtools_platform_comps.utils.general.fatal_code(e: Exception) → bool[source]#

Uses to determine if we should stop retrying based on request status code.

Parameters:

e – Exception to check

Returns:

True is exception is a request and status code matches 404

idmtools_platform_comps.utils.general.convert_comps_status(comps_status: SimulationState) → EntityStatus[source]#

Convert status from COMPS to IDMTools.

Parameters:

comps_status – Status in Comps

Returns:

EntityStatus

idmtools_platform_comps.utils.general.convert_comps_workitem_status(comps_status: WorkItemState) → EntityStatus[source]#

Convert status from COMPS to IDMTools.

Created = 0 # WorkItem has been saved to the database CommissionRequested = 5 # WorkItem is ready to be processed by the next available worker of the correct type Commissioned = 10 # WorkItem has been commissioned to a worker of the correct type and is beginning execution Validating = 30 # WorkItem is being validated Running = 40 # WorkItem is currently running Waiting = 50 # WorkItem is waiting for dependent items to complete ResumeRequested = 60 # Dependent items have completed and WorkItem is ready to be processed by the next available worker of the correct type CancelRequested = 80 # WorkItem cancellation was requested Canceled = 90 # WorkItem was successfully canceled Resumed = 100 # WorkItem has been claimed by a worker of the correct type and is resuming Canceling = 120 # WorkItem is in the process of being canceled by the worker Succeeded = 130 # WorkItem completed successfully Failed = 140 # WorkItem failed

Parameters:

comps_status – Status in Comps

Returns:

EntityStatus

idmtools_platform_comps.utils.general.clean_experiment_name(experiment_name: str) → str[source]#

Enforce any COMPS-specific demands on experiment names.

Parameters:

experiment_name – name of the experiment

Returns:the experiment name allowed for use

idmtools_platform_comps.utils.general.get_file_from_collection(platform: IPlatform, collection_id: UUID, file_path: str) → bytearray[source]#

Retrieve a file from an asset collection.

Parameters:

  • platform – Platform object to use

  • collection_id – Asset Collection ID

  • file_path – Path within collection

Examples:: >>> import uuid >>> get_file_from_collection(platform, uuid.UUID(“fc461146-3b2a-441f-bc51-0bff3a9c1ba0”), “StdOut.txt”)

Returns:

Object Byte Array

idmtools_platform_comps.utils.general.get_file_as_generator(file: SimulationFile | AssetCollectionFile | AssetFile | WorkItemFile | OutputFileMetadata, chunk_size: int = 128, resume_byte_pos: int | None = None) → Generator[bytearray, None, None][source]#

Get file as a generator.

Parameters:

  • file – File to stream contents through a generator

  • chunk_size – Size of chunks to load

  • resume_byte_pos – Optional start of download

Returns:

Generator for file content

class idmtools_platform_comps.utils.general.Workitem[source]#

Bases: object

SimpleItem to define workitem for proxy purposes.

Notes

  • TODO deprecate this if possible

idmtools_platform_comps.utils.general.get_asset_for_comps_item(platform: IPlatform, item: IEntity, files: List[str], cache=None, load_children: List[str] | None = None, comps_item: Experiment | Workitem | Simulation | None = None) → Dict[str, bytearray][source]#

Retrieve assets from an Entity(Simulation, Experiment, WorkItem).

Parameters:

  • platform – Platform Object to use

  • item – Item to fetch assets from

  • files – List of file names to retrieve

  • cache – Cache object to use

  • load_children – Optional Load children fields

  • comps_item – Optional comps item

Returns:

Dictionary in structure of filename -> bytearray

idmtools_platform_comps.utils.general.update_item(platform: IPlatform, item_id: str, item_type: ItemType, tags: dict | None = None, name: str | None = None)[source]#

Utility function to update existing COMPS experiment/simulation/workitem’s tags.

For example, you can add/update simulation’s tag once its post-process is done to mark the simulation with more meaningful text with tag/name :param platform: Platform :param item_id: experiment/simulation/workitem id :param item_type: The type of the object to be retrieved :param tags: tags dict for update :param name: name of experiment/simulation/workitem

Returns:

None

idmtools_platform_comps.utils.general.generate_ac_from_asset_md5(file_name: str, asset_md5: [<class 'str'>, <class 'uuid.UUID'>], platform: ~idmtools.entities.iplatform.IPlatform | None = None, tags: dict | None = None)[source]#

Get an asset collection by asset id(md5). :param file_name: file name string :param asset_md5: asset md5 string :param platform: Platform object :param tags: tags dict for asset collection

Returns:

COMPS AssetCollection

idmtools_platform_comps.utils.general.generate_ac_from_asset_md5_file(file_path: str)[source]#

Get an asset collection by file path. :param file_path: file path

Returns:

COMPS AssetCollection

idmtools_platform_comps.utils.general.save_sif_asset_md5_from_ac_id(ac_id: str)[source]#

Get the md5 of the asset in the asset collection of singularity. :param ac_id: asset collection id