idmtools.entities.iplatform_ops.utils module

Utils for platform operations.

Here we have mostly utilities to handle batch operations which tend to overlap across different item types.

Copyright 2021, Bill & Melinda Gates Foundation. All rights reserved.

idmtools.entities.iplatform_ops.utils.batch_items(items: Union[Iterable, Generator], batch_size=16)

Batch items.

Parameters:

  • items – Items to batch

  • batch_size – Size of the batch

Returns:

Generator

Raises:

StopIteration –

idmtools.entities.iplatform_ops.utils.item_batch_worker_thread(create_func: Callable, items: List, **kwargs) → List

Default batch worker thread function. It just calls create on each item.

Parameters:

  • create_func – Create function for item

  • items – Items to create

Returns:

List of items created

idmtools.entities.iplatform_ops.utils.batch_create_items(items: Union[Iterable, Generator], batch_worker_thread_func: Optional[Callable[[List], List]] = None, create_func: Optional[Callable[[...], Any]] = None, display_progress: bool = True, progress_description: str = 'Commissioning items', unit: Optional[str] = None, **kwargs)

Batch create items. You must specify either batch_worker_thread_func or create_func.

Parameters:

  • items – Items to create

  • batch_worker_thread_func – Optional Function to execute. Should take a list and return a list

  • create_func – Optional Create function

  • display_progress – Enable progress bar

  • progress_description – Description to show in progress bar

  • unit – Unit for progress bar

  • **kwargs –

Returns:

Batches crated results

idmtools.entities.iplatform_ops.utils.show_progress_of_batch(progress_bar: tqdm, futures: List[Future]) → List

Show progress bar for batch.

Parameters:

  • progress_bar – Progress bar

  • futures – List of futures that are still running/queued

Returns:

Returns results