idmtools_platform_local.infrastructure.service_manager module

idmtools service manager. Manages all our local platform services.

Copyright 2021, Bill & Melinda Gates Foundation. All rights reserved.

class idmtools_platform_local.infrastructure.service_manager.DockerServiceManager(client: DockerClient, host_data_directory: str = '/home/docs/.local_data', network: str = 'idmtools', redis_image: str = 'redis:5.0.4-alpine', heartbeat_timeout: int = 15, redis_port: int = 6379, runtime: str | None = 'runc', redis_mem_limit: str = '256m', redis_mem_reservation: str = '32m', postgres_image: str = 'postgres:11.4', postgres_mem_limit: str = '128m', postgres_mem_reservation: str = '32m', postgres_port: str | None = 5432, workers_image: str | None = None, workers_ui_port: int = 5000, workers_mem_limit: str | None = None, workers_mem_reservation: str = '64m', run_as: str | None = None, enable_singularity_support: bool = False, _services: Dict[str, BaseServiceContainer] | None = None)

Bases: object

Provides single interface to manage all the local platform services.

client: DockerClient

host_data_directory: str = '/home/docs/.local_data'

network: str = 'idmtools'

redis_image: str = 'redis:5.0.4-alpine'

heartbeat_timeout: int = 15

redis_port: int = 6379

runtime: str | None = 'runc'

redis_mem_limit: str = '256m'

redis_mem_reservation: str = '32m'

postgres_image: str = 'postgres:11.4'

postgres_mem_limit: str = '128m'

postgres_mem_reservation: str = '32m'

postgres_port: str | None = 5432

workers_image: str = None

workers_ui_port: int = 5000

workers_mem_limit: str = None

workers_mem_reservation: str = '64m'

run_as: str | None = None

enable_singularity_support: bool = False

init_services()

Start all the containers we should have running.

cleanup(delete_data: bool = False, tear_down_brokers: bool = False) → NoReturn

Stops the containers and removes the network.

Optionally the postgres data container can be deleted as well as closing any active Redis connections.

Parameters:

• delete_data – Delete postgres data

• tear_down_brokers – True to close redis brokers, false otherwise

Returns:

NoReturn

static setup_broker(heartbeat_timeout)

Start the broker to send data to workers.

static restart_brokers(heartbeat_timeout)

Restart brokers talking to workers.

create_services(spinner=None) → NoReturn

Create all the components of local platform.

Our architecture is as depicted in the UML diagram below

Returns:

(NoReturn)

wait_on_ports_to_open(ports: List[str], wait_between_tries: int | float = 0.2, max_retries: int = 5, sleep_after: int | float = 0.5) → bool

Polls list of port attributes(eg postgres_port, redis_port and checks if they are currently open.

We use this to verify postgres/redis are ready for our workers

Parameters:

• ports – List of port attributes

• wait_between_tries – Time between port checks

• max_retries – Max checks

• sleep_after – Sleep after all our found open(Postgres starts accepting connections before actually ready)

Returns:

True if ports are ready

stop_services(spinner=None) → NoReturn

Stops all running IDM Tools services.

Returns:

(NoReturn)

get(container_name: str, create=True) → Container

Get the server with specified name.

Parameters:

• container_name – Name of container

• create – Create if it doesn’t exists

Returns:

Container

get_container_config(service: BaseServiceContainer, opts=None)

Get the container config for the service.

Parameters:

• service – Service to get config for

• opts – Opts to Extract. Should be a fields object

Returns:

Container config

restart_all(spinner=None) → NoReturn

Restart all the services IDM-Tools services.

Returns:

(NoReturn)

static is_port_open(host: str, port: int) → bool

Check if a port is open.

Parameters:

• host – Host to check

• port – Port to check

Returns:

True if port is open, False otherwise

static stop_service_and_wait(service) → bool

Stop server and wait.

Parameters:

service – Service to stop

Returns:

True

get_network() → Network

Fetches the IDM Tools network.

Returns:

(Network) Return Docker network object

__init__(client: DockerClient, host_data_directory: str = '/home/docs/.local_data', network: str = 'idmtools', redis_image: str = 'redis:5.0.4-alpine', heartbeat_timeout: int = 15, redis_port: int = 6379, runtime: str | None = 'runc', redis_mem_limit: str = '256m', redis_mem_reservation: str = '32m', postgres_image: str = 'postgres:11.4', postgres_mem_limit: str = '128m', postgres_mem_reservation: str = '32m', postgres_port: str | None = 5432, workers_image: str | None = None, workers_ui_port: int = 5000, workers_mem_limit: str | None = None, workers_mem_reservation: str = '64m', run_as: str | None = None, enable_singularity_support: bool = False, _services: Dict[str, BaseServiceContainer] | None = None) → None