idmtools_platform_comps.utils.package_version module¶

idmtools Tools to filter versions of packages for requriements for asset collections.

class idmtools_platform_comps.utils.package_version.PackageHTMLParser¶

Bases: html.parser.HTMLParser, abc.ABC

Base Parser for our other parsers.

previous_tag = None¶

__init__()¶: Constructor.

pkg_version = None¶

class idmtools_platform_comps.utils.package_version.LinkHTMLParser¶

Bases: idmtools_platform_comps.utils.package_version.PackageHTMLParser

Parse hrefs from links.

handle_starttag(tag, attrs)¶: Parse links and extra hrefs.

class idmtools_platform_comps.utils.package_version.LinkNameParser¶

Bases: idmtools_platform_comps.utils.package_version.PackageHTMLParser

Provides parsing of packages from pypi/arfifactory.

We parse links that match versions patterns

in_link = False¶

ver_pattern = re.compile('^[\\d\\.brcdev\\+nightly]+$')¶

handle_starttag(tag, attrs)¶: Handle begin of links.

handle_endtag(tag)¶: End link tags.

handle_data(data)¶: Process links.

idmtools_platform_comps.utils.package_version.get_latest_package_version_from_pypi(pkg_name, display_all=False)¶

Utility to get the latest version for a given package name.

Parameters:

pkg_name – package name given
display_all – determine if output all package releases

Returns: the latest version of ven package

idmtools_platform_comps.utils.package_version.get_latest_pypi_package_version_from_artifactory(pkg_name, display_all=False, base_version: Optional[str] = None)¶

Utility to get the latest version for a given package name.

Parameters:

pkg_name – package name given
display_all – determine if output all package releases
base_version – Base version

Returns: the latest version of ven package

idmtools_platform_comps.utils.package_version.get_pypi_package_versions_from_artifactory(pkg_name, display_all=False, base_version: Optional[str] = None, exclude_pre_release: bool = True)¶

Utility to get versions of a package in artifactory.

Parameters:

pkg_name – package name given
display_all – determine if output all package releases
base_version – Base version
exclude_pre_release – Exclude any prerelease versions

Returns: the latest version of ven package

idmtools_platform_comps.utils.package_version.get_latest_ssmt_image_version_from_artifactory(pkg_name='comps_ssmt_worker', base_version: Optional[str] = None, display_all=False)¶

Utility to get the latest version for a given package name.

Parameters:

pkg_name – package name given
base_version – Optional base version. Versions above this will not be added.
display_all – determine if output all package releases

Returns: the latest version of ven package

idmtools_platform_comps.utils.package_version.get_docker_manifest(image_path='idmtools/comps_ssmt_worker', repo_base='https://packages.idmod.org/artifactory/list/docker-production')¶

Get docker manifest from IDM Artifactory. It mimics latest even when user has no latest tag defined.

Parameters:

image_path – Path of docker image we want
repo_base – Base of the repo

Returns:

None

Raises:

ValueError - When the manifest cannot be found –

idmtools_platform_comps.utils.package_version.get_digest_from_docker_hub(repo, tag='latest')¶

Get the digest for image from docker.

Parameters:

repo – string, repository (e.g. ‘library/fedora’)
tag – string, tag of the repository (e.g. ‘latest’)

idmtools_platform_comps.utils.package_version.fetch_versions_from_server(pkg_url: str, parser: typing.Type[idmtools_platform_comps.utils.package_version.PackageHTMLParser] = <class 'idmtools_platform_comps.utils.package_version.LinkHTMLParser'>) → List[str]¶

Fetch all versions from server.

Parameters:

pkg_url – Url to fetch
parser – Parser tp use

Returns:

All the releases for a package

idmtools_platform_comps.utils.package_version.fetch_versions_from_artifactory(pkg_name: str, parser: typing.Type[idmtools_platform_comps.utils.package_version.PackageHTMLParser] = <class 'idmtools_platform_comps.utils.package_version.LinkHTMLParser'>) → List[str]¶

Fetch all versions from server.

Parameters:

pkg_name – Url to fetch
parser – Parser tp use

Returns:

Available releases

idmtools_platform_comps.utils.package_version.get_versions_from_site(pkg_url, base_version: typing.Optional[str] = None, display_all=False, parser: typing.Type[idmtools_platform_comps.utils.package_version.PackageHTMLParser] = <class 'idmtools_platform_comps.utils.package_version.LinkNameParser'>, exclude_pre_release: bool = True)¶

Utility to get the the available versions for a package.

The default properties filter out pre releases. You can also include a base version to only list items starting with a particular version

Parameters:

pkg_url – package name given
base_version – Optional base version. Versions above this will not be added. For example, to get versions 1.18.5, 1.18.4, 1.18.3, 1.18.2 pass 1.18
display_all – determine if output all package releases
parser – Parser needs to be a HTMLParser that returns a pkg_versions
exclude_pre_release – Exclude prerelease versions

Returns: the latest version of ven package

Raises:: ValueError - If a latest versions cannot be determined –

idmtools_platform_comps.utils.package_version.get_latest_version_from_site(pkg_url, base_version: typing.Optional[str] = None, display_all=False, parser: typing.Type[idmtools_platform_comps.utils.package_version.PackageHTMLParser] = <class 'idmtools_platform_comps.utils.package_version.LinkNameParser'>, exclude_pre_release: bool = True)¶

Utility to get the latest version for a given package name.

Parameters:

pkg_url – package name given
base_version – Optional base version. Versions above this will not be added.
display_all – determine if output all package releases
parser – Parser needs to be a HTMLParser that returns a pkg_versions
exclude_pre_release – Exclude pre-release versions

Returns: the latest version of ven package

idmtools_platform_comps.utils.package_version.fetch_package_versions_from_pypi(pkg_name)¶

Utility to get the latest version for a given package name.

Parameters:: pkg_name – package name given

Returns: the latest version of ven package

idmtools_platform_comps.utils.package_version.fetch_package_versions(pkg_name, is_released=True, sort=True, display_all=False)¶

Utility to get the latest version for a given package name.

Parameters:

pkg_name – package name given
is_released – get released version only
sort – make version sorted or not
display_all – determine if output all package releases

Returns: the latest version of ven package

idmtools_platform_comps.utils.package_version.get_pkg_match_version(pkg_name, base_version=None, test='==', validate=True)¶

Utility to get the latest version for a given package name.

Parameters:

pkg_name – package name given
base_version – Optional base version. Versions above this will not be added.
test – default ==, a filter to find version
validate – bool, if True, will validate base_version

Returns: the latest version of ven package

Raises:: Exception - if we cannot find version –

Notes

TODO - Make custom exception or use ValueError

idmtools_platform_comps.utils.package_version.get_latest_version(pkg_name)¶

Utility to get the latest version for a given package name.

Parameters:: pkg_name – package name given

Returns: the latest version of package

Raises:: Exception if package could not be found. –

Notes

TODO - Make custom exception or use ValueError

idmtools_platform_comps.utils.package_version.get_latest_compatible_version(pkg_name, base_version=None, versions=None, validate=True)¶

Utility to get the latest compatible version from a given version list.

Parameters:

base_version – Optional base version. Versions above this will not be added.
pkg_name – package name given
versions – user input of version list
validate – bool, if True, will validate base_version

Returns: the latest compatible version from versions

Raises:: Exception - If we cannot find version –

Notes

TODO - Make custom exception or use ValueError