idmtools_platform_comps.utils.assetize_output.assetize_output module

exception idmtools_platform_comps.utils.assetize_output.assetize_output.CrossEnvironmentAssetizeNotSupport

Bases: Exception

exception idmtools_platform_comps.utils.assetize_output.assetize_output.AtLeastOneItemToWatch

Bases: Exception

class idmtools_platform_comps.utils.assetize_output.assetize_output.AssetizeOutput(_uid: uuid.UUID = None, _IItem__pre_creation_hooks: List[Callable[[ForwardRef('IItem'), ForwardRef('IPlatform')], NoneType]] = <factory>, _IItem__post_creation_hooks: List[Callable[[ForwardRef('IItem'), ForwardRef('IPlatform')], NoneType]] = <factory>, platform_id: uuid.UUID = None, _platform: 'IPlatform' = None, parent_id: uuid.UUID = None, _parent: 'IEntity' = None, status: idmtools.core.enums.EntityStatus = None, tags: Dict[str, Any] = <factory>, _platform_object: Any = None, _IRunnableEntity__pre_run_hooks: List[Callable[[ForwardRef('IRunnableEntity'), ForwardRef('IPlatform')], NoneType]] = <factory>, _IRunnableEntity__post_run_hooks: List[Callable[[ForwardRef('IRunnableEntity'), ForwardRef('IPlatform')], NoneType]] = <factory>, name: str = 'idmtools workflow item', assets: idmtools.assets.asset_collection.AssetCollection = <factory>, item_name: dataclasses.InitVar = <property object at 0x7f0a388110b0>, asset_collection_id: dataclasses.InitVar = <property object at 0x7f0a38800f50>, transient_assets: idmtools.assets.asset_collection.AssetCollection = <factory>, asset_files: dataclasses.InitVar = <property object at 0x7f0a38800fb0>, user_files: dataclasses.InitVar = <property object at 0x7f0a38811050>, task: idmtools.entities.itask.ITask = None, related_experiments: list = <factory>, related_simulations: list = <factory>, related_suites: list = <factory>, related_work_items: list = <factory>, related_asset_collections: list = <factory>, work_item_type: str = None, work_order: dict = <factory>, plugin_key: str = '1.0.0.0_RELEASE', docker_image: str = None, command: dataclasses.InitVar = <property object at 0x7f0a36b31410>, file_patterns: List[str] = <factory>, exclude_patterns: List[str] = <factory>, include_assets: bool = False, simulation_prefix_format_str: str = '{simulation.id}', work_item_prefix_format_str: str = None, no_simulation_prefix: bool = False, verbose: bool = False, pre_run_functions: List[Callable] = <factory>, entity_filter_function: Callable[[COMPS.Data.CommissionableEntity.CommissionableEntity], bool] = None, asset_tags: Dict[str, str] = <factory>, asset_collection: idmtools.assets.asset_collection.AssetCollection = None, dry_run: bool = False)

Bases: idmtools_platform_comps.ssmt_work_items.comps_workitems.SSMTWorkItem

file_patterns: List[str]

//docs.python.org/3.7/library/glob.html for details on the patterns

Type

List of glob patterns. See https

exclude_patterns: List[str]
include_assets: bool = False

Include Assets directories. This allows patterns to also include items from the assets directory

simulation_prefix_format_str: str = '{simulation.id}'

Formatting pattern for directory names. Simulations tend to have similar outputs so Assetize puts those in directories using the simulation id by default as the directory name

work_item_prefix_format_str: str = None

WorkFlowItem outputs will not have a folder prefix by default. If you are assetizing multiple work items, you may want to set this to “{workflow_item.id}”

no_simulation_prefix: bool = False

Simulations outputs will not have a folder. Useful when you are assetizing a single simulation

verbose: bool = False

Enable verbose

pre_run_functions: List[Callable]

Python Functions that will be ran before Assetizing script. The function must be named

entity_filter_function: Callable[[COMPS.Data.CommissionableEntity.CommissionableEntity], bool] = None

Python Function to filter entities. This Function should receive a Comps CommissionableEntity. Return True if item should be assetize, False otherwise

asset_tags: Dict[str, str]
asset_collection: idmtools.assets.asset_collection.AssetCollection = None

The asset collection created by Assetize

dry_run: bool = False

Enables running jobs without creating assets. It instead produces a file list of what would be includes in the final assets

create_command() → str

Builds our command line for the SSMT Job

Returns

Command string

clear_exclude_patterns()

Clear Exclude Patterns will remove all current rules.

Returns

None

pre_creation(platform: idmtools.entities.iplatform.IPlatform) → None

Pre-Creation

Parameters

platform – Platform

Returns:

total_items_watched() → int

Returns the number of items watched

Returns

Total number of items watched

run_after_by_id(item_id: str, item_type: idmtools.core.enums.ItemType, platform: idmtools.entities.iplatform.IPlatform = None)
from_items(item: Union[idmtools.entities.experiment.Experiment, idmtools.entities.simulation.Simulation, idmtools.entities.iworkflow_item.IWorkflowItem, List[Union[idmtools.entities.experiment.Experiment, idmtools.entities.simulation.Simulation, idmtools.entities.iworkflow_item.IWorkflowItem]]])

Add items to load assets from

Parameters

item – Item or list of items to watch.

Returns:

run(wait_until_done: bool = False, platform: Optional[idmtools.entities.iplatform.IPlatform] = None, wait_on_done_progress: bool = True, wait_on_done: bool = True, **run_opts) → Optional[idmtools.assets.asset_collection.AssetCollection]

Run the AssetizeOutput

Parameters

  • wait_until_done – Wait until Done will wait for the workitem to complet

  • platform – Platform Object

  • wait_on_done_progress – When set to true, a progress bar will be shown from the item

  • wait_on_done – Wait for item to be done. This will first wait on any dependencies

  • **run_opts – Additional options to pass to Run on platform

Returns

AssetCollection created if item succeeds

wait(wait_on_done_progress: bool = True, timeout: int = None, refresh_interval=None, platform: Optional[idmtools.entities.iplatform.IPlatform] = None) → Optional[idmtools.assets.asset_collection.AssetCollection]

Waits on Assetize Workitem to finish. This first waits on any dependent items to finish(Experiment/Simulation/WorkItems)

Parameters

  • wait_on_done_progress – When set to true, a progress bar will be shown from the item

  • timeout – Timeout for waiting on item. If none, wait will be forever

  • refresh_interval – How often to refresh progress

  • platform – Platform

Returns

AssetCollection created if item succeeds

fetch_error(print_error: bool = True) → Dict

Fetches the error from a WorkItem

Parameters

print_error – Should error be printed. If false, error will be returned

Returns

Error info