idmtools.assets.asset_collection module

idmtools assets collection package.

Copyright 2021, Bill & Melinda Gates Foundation. All rights reserved.

class idmtools.assets.asset_collection.AssetCollection(assets: Optional[Union[List[str], List[TAsset], AssetCollection]] = None, tags=None)

Bases: IEntity

A class that represents a collection of assets.

__init__(assets: Optional[Union[List[str], List[TAsset], AssetCollection]] = None, tags=None)

A constructor.

Args: assets: An optional list of assets to create the collection with. tags: dict: tags associated with asset collection

item_type: ItemType = 'Asset Collection'

ItemType so platform knows how to handle item properly

assets: List[Asset] = None

Assets for collection

classmethod from_id(item_id: Union[str, UUID], platform: IPlatform = None, as_copy: bool = False, **kwargs) → AssetCollection

Loads a AssetCollection from id.

Parameters:

  • item_id – Asset Collection ID

  • platform – Platform Object

  • as_copy – Should you load the object as a copy. When True, the contents of AC are copied, but not the id. Useful when editing ACs

  • **kwargs –

Returns:

AssetCollection

classmethod from_directory(assets_directory: str, recursive: bool = True, flatten: bool = False, filters: Optional[List[Union[Callable[[TAsset], bool], Callable]]] = None, filters_mode: FilterMode = FilterMode.OR, relative_path: Optional[str] = None) → TAssetCollection

Fill up an AssetCollection from the specified directory.

See assets_from_directory() for arguments.

Returns:

A created AssetCollection object.

static assets_from_directory(assets_directory: Union[str, PathLike], recursive: bool = True, flatten: bool = False, filters: Optional[List[Union[Callable[[TAsset], bool], Callable]]] = None, filters_mode: FilterMode = FilterMode.OR, forced_relative_path: Optional[str] = None, no_ignore: bool = False) → List[Asset]

Create assets for files in a given directory.

Parameters:

  • assets_directory – The root directory of the assets.

  • recursive – True to recursively traverse the subdirectory.

  • flatten – Put all the files in root regardless of whether they were in a subdirectory or not.

  • filters – A list of filters to apply to the assets. The filters are functions taking an Asset as argument and returning true or false. True adds the asset to the collection; False filters it out. See asset_filters().

  • filters_mode – When given multiple filters, either OR or AND the results.

  • forced_relative_path – Prefix a relative path to the path created from the root directory.

  • no_ignore – Should we not ignore common directories(.git, .svn. etc) The full list is defined in IGNORE_DIRECTORIES

Examples

For relative_path, given the following folder structure root/a/1,txt root/b.txt and relative_path=”test”. Will return assets with relative path: test/a/1,txt and test/b.txt

Given the previous example, if flatten is also set to True, the following relative_path will be set: /1.txt and /b.txt

Returns:

A list of assets.

copy() → AssetCollection

Copy our Asset Collection, removing ID and tags.

Returns:

New AssetCollection containing Assets from other AssetCollection

add_directory(assets_directory: Union[str, PathLike], recursive: bool = True, flatten: bool = False, filters: Optional[List[Union[Callable[[TAsset], bool], Callable]]] = None, filters_mode: FilterMode = FilterMode.OR, relative_path: Optional[str] = None, no_ignore: bool = False)

Retrieve assets from the specified directory and add them to the collection.

See assets_from_directory() for arguments.

is_editable(error=False) → bool

Checks whether Item is editable.

Parameters:

error – Throw error is not

Returns:

True if editable, False otherwise.

add_asset(asset: Union[Asset, str, PathLike], fail_on_duplicate: bool = True, fail_on_deep_comparison: bool = False, **kwargs)

Add an asset to the collection.

Parameters:

  • asset – A string or an Asset object to add. If a string, the string will be used as the absolute_path and any kwargs will be passed to the Asset constructor

  • fail_on_duplicate – Raise a DuplicateAssetError if an asset is duplicated. If not, simply replace it.

  • fail_on_deep_comparison – Fails only if deep comparison differs

  • **kwargs – Arguments to pass to Asset constructor when asset is a string

Raises:

DuplicatedAssetError - If fail_on_duplicate is true and the asset is already part of the collection –

add_assets(assets: Union[List[TAsset], AssetCollection], fail_on_duplicate: bool = True, fail_on_deep_comparison: bool = False)

Add assets to a collection.

Parameters:

  • assets – An list of assets as either list or a collection

  • fail_on_duplicate – Raise a DuplicateAssetError if an asset is duplicated. If not, simply replace it.

  • fail_on_deep_comparison – Fail if relative path/file is same but contents differ

Returns:

None

add_or_replace_asset(asset: Union[Asset, str, PathLike], fail_on_deep_comparison: bool = False)

Add or replaces an asset in a collection.

Parameters:

  • asset – Asset to add or replace

  • fail_on_deep_comparison – Fail replace if contents differ

Returns:

None.

get_one(**kwargs)

Get an asset out of the collection based on the filers passed.

Examples:

>>> a = AssetCollection()
>>> a.get_one(filename="filename.txt")
Parameters:

**kwargs – keyword argument representing the filters.

Returns:

None or Asset if found.

remove(**kwargs) → NoReturn

Remove an asset from the AssetCollection based on keywords attributes.

Parameters:

**kwargs – Filter for the asset to remove.

pop(**kwargs) → Asset

Get and delete an asset based on keywords.

Parameters:

**kwargs – Filter for the asset to pop.

extend(assets: List[Asset], fail_on_duplicate: bool = True) → NoReturn

Extend the collection with new assets.

Parameters:

  • assets – Which assets to add

  • fail_on_duplicate – Fail if duplicated asset is included.

clear()

Clear the asset collection.

Returns:

None

set_all_persisted()

Set all persisted.

Returns:

None

property count

Number of assets in collections.

Returns:

Total assets

property uid

Uid of Asset Collection.

Returns:

Asset Collection UID.

has_asset(absolute_path: Optional[str] = None, filename: Optional[str] = None, relative_path: Optional[str] = None, checksum: Optional[str] = None) → bool

Search for asset by absolute_path or by filename.

Parameters:

  • absolute_path – Absolute path of source file

  • filename – Destination filename

  • relative_path – Relative path of asset

  • checksum – Checksum of asset(optional)

Returns:

True if asset exists, False otherwise

find_index_of_asset(other: Asset, deep_compare: bool = False) → Optional[int]

Finds the index of asset by path or filename.

Parameters:

  • other – Other asset

  • deep_compare – Should content as well as path be compared

Returns:

Index number if found. None if not found.

pre_creation(platform: IPlatform) → None

Pre-Creation hook for the asset collection.

Parameters:

platform – Platform object we are create asset collection on

Returns:

None

Notes

TODO - Make default tags optional

set_tags(tags: Dict[str, Any])

Set the tags on the asset collection.

Parameters:

tags – Tags to set on the item

Returns:

None

add_tags(tags: Dict[str, Any])

Add tags to the Asset collection.

Parameters:

tags – Tags to add

Returns:

None