idmtools_platform_slurm.slurm_platform module#
Here we implement the SlurmPlatform object.
Copyright 2021, Bill & Melinda Gates Foundation. All rights reserved.
- class idmtools_platform_slurm.slurm_platform.SlurmPlatform(_uid: str = None, _IItem__pre_creation_hooks: List[Callable[[ForwardRef('IItem'), ForwardRef('IPlatform')], NoneType]] = <factory>, _IItem__post_creation_hooks: List[Callable[[ForwardRef('IItem'), ForwardRef('IPlatform')], NoneType]] = <factory>, _platform_defaults: List[idmtools.entities.iplatform_default.IPlatformDefault] = <factory>, _config_block: str = None, job_directory: str = None, bridged_jobs_directory: str = PosixPath('/home/docs/.idmtools/singularity-bridge'), bridged_results_directory: str = PosixPath('/home/docs/.idmtools/singularity-bridge/results'), mode: idmtools_platform_slurm.slurm_operations.slurm_constants.SlurmOperationalMode = <SlurmOperationalMode.LOCAL: 'local'>, mail_type: Optional[str] = None, mail_user: Optional[str] = None, nodes: Optional[int] = None, ntasks: Optional[int] = None, cpus_per_task: Optional[int] = None, ntasks_per_core: Optional[int] = None, max_running_jobs: Optional[int] = None, mem: Optional[int] = None, mem_per_cpu: Optional[int] = None, partition: Optional[str] = None, constraint: Optional[str] = None, time: str = None, account: str = None, exclusive: bool = False, requeue: bool = True, retries: int = 1, sbatch_custom: Optional[str] = None, modules: list = <factory>, dir_exist_ok: bool = False, array_batch_size: int = None, run_on_slurm: bool = False)[source]#
Bases:
IPlatform
- bridged_jobs_directory: str = PosixPath('/home/docs/.idmtools/singularity-bridge')#
Needed for bridge mode
- mode: SlurmOperationalMode = 'local'#
- post_setstate()[source]#
Function called after restoring the state if additional initialization is required.
- property slurm_fields#
Get list of fields that have metadata sbatch. :returns: Set of fields that have sbatch metadata
- get_slurm_configs(**kwargs) Dict[str, Any] [source]#
Identify the Slurm config parameters from the fields. :param kwargs: additional parameters
- Returns:
slurm config dict
- flatten_item(item: IEntity, raw=False, **kwargs) List[object] [source]#
Flatten an item: resolve the children until getting to the leaves.
For example, for an experiment, will return all the simulations. For a suite, will return all the simulations contained in the suites experiments.
- Parameters:
item – Which item to flatten
raw – bool
kwargs – extra parameters
- Returns:
List of leaves
- validate_item_for_analysis(item: Simulation | SlurmSimulation, analyze_failed_items=False)[source]#
Check if item is valid for analysis.
- Parameters:
item – which item to verify status
analyze_failed_items – bool
Returns: bool
- get_directory(item: Suite | Experiment | Simulation) Path [source]#
Get item’s path. :param item: Suite, Experiment, Simulation
- Returns:
item file directory
- get_directory_by_id(item_id: str, item_type: ItemType) Path [source]#
Get item’s path. :param item_id: entity id (Suite, Experiment, Simulation) :param item_type: the type of items (Suite, Experiment, Simulation)
- Returns:
item file directory
- __init__(_uid: str = None, _IItem__pre_creation_hooks: ~typing.List[~typing.Callable[[IItem, IPlatform], None]] = <factory>, _IItem__post_creation_hooks: ~typing.List[~typing.Callable[[IItem, IPlatform], None]] = <factory>, _platform_defaults: ~typing.List[~idmtools.entities.iplatform_default.IPlatformDefault] = <factory>, _config_block: str = None, job_directory: str = None, bridged_jobs_directory: str = PosixPath('/home/docs/.idmtools/singularity-bridge'), bridged_results_directory: str = PosixPath('/home/docs/.idmtools/singularity-bridge/results'), mode: ~idmtools_platform_slurm.slurm_operations.slurm_constants.SlurmOperationalMode = SlurmOperationalMode.LOCAL, mail_type: str | None = None, mail_user: str | None = None, nodes: int | None = None, ntasks: int | None = None, cpus_per_task: int | None = None, ntasks_per_core: int | None = None, max_running_jobs: int | None = None, mem: int | None = None, mem_per_cpu: int | None = None, partition: str | None = None, constraint: str | None = None, time: str = None, account: str = None, exclusive: bool = False, requeue: bool = True, retries: int = 1, sbatch_custom: str | None = None, modules: list = <factory>, dir_exist_ok: bool = False, array_batch_size: int = None, run_on_slurm: bool = False) None #