idmtools_platform_comps.utils.singularity_build module

idmtools singularity build workitem.

Notes

  • TODO add examples here.

Copyright 2021, Bill & Melinda Gates Foundation. All rights reserved.

class idmtools_platform_comps.utils.singularity_build.SingularityBuildWorkItem(_uid: ~uuid.UUID = None, _IItem__pre_creation_hooks: ~typing.List[~typing.Callable[[IItem, IPlatform], None]] = <factory>, _IItem__post_creation_hooks: ~typing.List[~typing.Callable[[IItem, IPlatform], None]] = <factory>, platform_id: ~uuid.UUID = None, _platform: IPlatform = None, parent_id: ~uuid.UUID = None, _parent: IEntity = None, status: ~idmtools.core.enums.EntityStatus = None, tags: ~typing.Dict[str, ~typing.Any] = <factory>, _platform_object: ~typing.Any = None, _IRunnableEntity__pre_run_hooks: ~typing.List[~typing.Callable[[IRunnableEntity, IPlatform], None]] = <factory>, _IRunnableEntity__post_run_hooks: ~typing.List[~typing.Callable[[IRunnableEntity, IPlatform], None]] = <factory>, name: str = None, assets: ~idmtools.assets.asset_collection.AssetCollection = <factory>, item_name: dataclasses.InitVar[str] = None, asset_collection_id: dataclasses.InitVar[UUID] = None, transient_assets: ~idmtools.assets.asset_collection.AssetCollection = <factory>, asset_files: dataclasses.InitVar[FileList] = None, user_files: dataclasses.InitVar[FileList] = None, task: ~idmtools.entities.itask.ITask = None, related_experiments: list = <factory>, related_simulations: list = <factory>, related_suites: list = <factory>, related_work_items: list = <factory>, related_asset_collections: list = <factory>, work_item_type: str = None, work_order: dict = <factory>, plugin_key: str = '1.0.0.0_RELEASE', definition_file: ~typing.Union[~os.PathLike, str] = None, definition_content: str = None, is_template: bool = False, template_args: ~typing.Dict[str, str] = <factory>, image_url: dataclasses.InitVar[str] = <property object>, image_name: str = None, image_tags: ~typing.Dict[str, str] = <factory>, library: str = None, section: ~typing.List[str] = <factory>, fix_permissions: bool = False, asset_collection: ~idmtools.assets.asset_collection.AssetCollection = None, additional_mounts: ~typing.List[str] = <factory>, environment_variables: ~typing.Dict[str, str] = <factory>, force: bool = False, disable_default_tags: bool = None, run_id: ~uuid.UUID = <factory>, _SingularityBuildWorkItem__digest: ~typing.Dict[str, str] = None, _SingularityBuildWorkItem__image_tag: str = None, _SingularityBuildWorkItem__rendered_template: str = None)

Bases: InputDataWorkItem

Provides a wrapper to build utilizing the COMPS build server.

Notes

  • TODO add references to examples

definition_file: Union[PathLike, str] = None

Path to definition file

definition_content: str = None

definition content. Alternative to file

is_template: bool = False

Enables Jinja parsing of the definition file or content

template_args: Dict[str, str]

template_args

image_name: str = None

Destination image name

name: str = None

Name of the workitem

image_tags: Dict[str, str]

Tages to add to container asset collection

library: str = None

//sylabs.io/guides/3.5/user-guide/cli/singularity_build.html

Type:

Allows you to set a different library. (The default library is “https

Type:

//library.sylabs.io”). See https

section: List[str]

only run specific section(s) of definition file (setup, post, files, environment, test, labels, none) (default [all])

fix_permissions: bool = False

build using user namespace to fake root user (requires a privileged installation)

asset_collection: AssetCollection = None
additional_mounts: List[str]

Additional Mounts

environment_variables: Dict[str, str]

Environment vars for remote build

force: bool = False

Force build

disable_default_tags: bool = None

Don’t include default tags

run_id: UUID
get_container_info() Dict[str, str]

Get container info.

Notes

  • TODO remove this

property image_url

Image Url

context_checksum() str

Calculate the context checksum of a singularity build.

The context is the checksum of all the assets defined for input, the singularity definition file, and the environment variables

Returns:

Conext checksum.

render_template() Optional[str]

Render template. Only applies when is_template is True. When true, it renders the template using Jinja to a cache value.

Returns:

Rendered Template

static find_existing_container(sbi: SingularityBuildWorkItem, platform: IPlatform = None) Optional[AssetCollection]

Find existing container.

Parameters:
  • sbi – SingularityBuildWorkItem to find existing container matching config

  • platform – Platform To load the object from

Returns:

Existing Asset Collection

pre_creation(platform: IPlatform) None

Pre-Creation item.

Parameters:

platform – Platform object

Returns:

None

run(wait_until_done: bool = True, platform: IPlatform = None, wait_on_done_progress: bool = True, wait_on_done: bool = True, **run_opts) Optional[AssetCollection]

Run the build.

Parameters:
  • wait_until_done – Wait until build completes

  • platform – Platform to run on

  • wait_on_done_progress – Show progress while waiting

  • wait_on_done – Overload of wait_until_done

  • **run_opts – Extra run options

Returns:

Asset collection that was created if successful

wait(wait_on_done_progress: bool = True, timeout: int = None, refresh_interval=None, platform: IPlatform = None, wait_progress_desc: str = None) Optional[AssetCollection]

Waits on Singularity Build Work item to finish and fetches the resulting asset collection.

Parameters:
  • wait_on_done_progress – When set to true, a progress bar will be shown from the item

  • timeout – Timeout for waiting on item. If none, wait will be forever

  • refresh_interval – How often to refresh progress

  • platform – Platform

  • wait_progress_desc – Wait Progress Description Text

Returns:

AssetCollection created if item succeeds

__init__(_uid: ~uuid.UUID = None, _IItem__pre_creation_hooks: ~typing.List[~typing.Callable[[IItem, IPlatform], None]] = <factory>, _IItem__post_creation_hooks: ~typing.List[~typing.Callable[[IItem, IPlatform], None]] = <factory>, platform_id: ~uuid.UUID = None, _platform: IPlatform = None, parent_id: ~uuid.UUID = None, _parent: IEntity = None, status: ~idmtools.core.enums.EntityStatus = None, tags: ~typing.Dict[str, ~typing.Any] = <factory>, _platform_object: ~typing.Any = None, _IRunnableEntity__pre_run_hooks: ~typing.List[~typing.Callable[[IRunnableEntity, IPlatform], None]] = <factory>, _IRunnableEntity__post_run_hooks: ~typing.List[~typing.Callable[[IRunnableEntity, IPlatform], None]] = <factory>, name: str = None, assets: ~idmtools.assets.asset_collection.AssetCollection = <factory>, item_name: dataclasses.InitVar[str] = None, asset_collection_id: dataclasses.InitVar[UUID] = None, transient_assets: ~idmtools.assets.asset_collection.AssetCollection = <factory>, asset_files: dataclasses.InitVar[FileList] = None, user_files: dataclasses.InitVar[FileList] = None, task: ~idmtools.entities.itask.ITask = None, related_experiments: list = <factory>, related_simulations: list = <factory>, related_suites: list = <factory>, related_work_items: list = <factory>, related_asset_collections: list = <factory>, work_item_type: str = None, work_order: dict = <factory>, plugin_key: str = '1.0.0.0_RELEASE', definition_file: ~typing.Union[~os.PathLike, str] = None, definition_content: str = None, is_template: bool = False, template_args: ~typing.Dict[str, str] = <factory>, image_url: dataclasses.InitVar[str] = <property object>, image_name: str = None, image_tags: ~typing.Dict[str, str] = <factory>, library: str = None, section: ~typing.List[str] = <factory>, fix_permissions: bool = False, asset_collection: ~idmtools.assets.asset_collection.AssetCollection = None, additional_mounts: ~typing.List[str] = <factory>, environment_variables: ~typing.Dict[str, str] = <factory>, force: bool = False, disable_default_tags: bool = None, run_id: ~uuid.UUID = <factory>, _SingularityBuildWorkItem__digest: ~typing.Dict[str, str] = None, _SingularityBuildWorkItem__image_tag: str = None, _SingularityBuildWorkItem__rendered_template: str = None) None
get_id_filename(prefix: Optional[str] = None) str

Determine the id filename. Mostly used when use does not provide one.

The logic is combine prefix and either * definition file minus extension * image url using with parts filtered out of the name.

Parameters:

prefix – Optional prefix.

Returns:

id file name

Raises:

ValueError - When the filename cannot be calculated

to_id_file(filename: Optional[Union[str, PathLike]] = None, save_platform: bool = False)

Create an ID File.

If the filename is not provided, it will be calculate for definition files or for docker pulls

Parameters:
  • filename – Filename

  • save_platform – Save Platform info to file as well

Returns:

None