idmtools_platform_comps.utils.python_requirements_ac.requirements_to_asset_collection module

idmtools requirements to asset collection.

This is the entry point for users to use RequirementsToAssetCollection tool.

Copyright 2021, Bill & Melinda Gates Foundation. All rights reserved.

class idmtools_platform_comps.utils.python_requirements_ac.requirements_to_asset_collection.RequirementsToAssetCollection(platform: COMPSPlatform | None = None, name: str = 'install custom requirements', requirements_path: str | None = None, pkg_list: list | None = None, local_wheels: list | None = None, asset_tags: dict | None = None)[source]

Bases: object

RequirementsToAssetCollection provides a utility to install python packages into an asset collection.

Notes

  • TODO - Incorporate example in this docs

platform: COMPSPlatform = None

Platform object

name: str = 'install custom requirements'

Name of experiment when installing requirements

requirements_path: str = None

Path to requirements file

pkg_list: list = None

list of packages

local_wheels: list = None

list of wheel files locally to upload and install

asset_tags: dict = None
property checksum

Calculate checksum on the requirements file.

Returns:

The md5 of the requirements.

property md5_tag

Get unique key for our requirements + target.

Returns:

The md5 tag.

property requirements

Requirements property. We calculate this using consolidate_requirements.

Returns:

Consolidated requirements.

init_platform()[source]

Initialize the platform.

run(rerun=False)[source]

Run our utility.

The working logic of this utility:

  1. check if asset collection exists for given requirements, return ac id if exists

  2. create an Experiment to install the requirements on COMPS

  3. create a WorkItem to create a Asset Collection

Returns: return ac id based on the requirements if Experiment and WorkItem Succeeded

Raises:

Exception - If an error happens on workitem –

Notes

  • TODO The exceptions here should be rewritten to parse errors from remote system like AssetizeOutputs

save_updated_requirements()[source]

Save consolidated requirements to a file requirements_updated.txt.

Returns:

None

retrieve_ac_by_tag(md5_check=None)[source]

Retrieve comps asset collection given ac tag.

Parameters:

md5_check – also can use custom md5 string as search tag

Returns: comps asset collection

retrieve_ac_from_wi(wi)[source]

Retrieve ac id from file ac_info.txt saved by WI.

Parameters:

wi – SSMTWorkItem (which was used to create ac from library)

Returns: COMPS asset collection

add_wheels_to_assets(experiment)[source]

Add wheels to assets of our experiment.

Parameters:

experiment – Experiment to add assets to

Returns:

None

run_experiment_to_install_lib()[source]

Create an Experiment which will run another py script to install requirements.

Returns: Experiment created

run_wi_to_create_ac(exp_id)[source]

Create an WorkItem which will run another py script to create new asset collection.

Parameters:

exp_id – the Experiment id (which installed requirements)

Returns: work item created

consolidate_requirements()[source]

Combine requirements and dynamic requirements (a list).

We do the following:

  • get the latest version of package if version is not provided

  • dynamic requirements will overwrites the requirements file

Returns: the consolidated requirements (as a list)

__init__(platform: COMPSPlatform | None = None, name: str = 'install custom requirements', requirements_path: str | None = None, pkg_list: list | None = None, local_wheels: list | None = None, asset_tags: dict | None = None) → None