toil.batchSystems.slurm¶
Attributes¶
Classes¶
A partial implementation of BatchSystemSupport for batch systems run on a |
Functions¶
|
Parse a Slurm-style time duration like 7-00:00:00 to a number of seconds. |
Module Contents¶
- toil.batchSystems.slurm.logger¶
- toil.batchSystems.slurm.TERMINAL_STATES: dict[str, toil.batchSystems.abstractBatchSystem.BatchJobExitReason]¶
- toil.batchSystems.slurm.parse_slurm_time(slurm_time)[source]¶
Parse a Slurm-style time duration like 7-00:00:00 to a number of seconds.
Raises ValueError if not parseable.
- class toil.batchSystems.slurm.SlurmBatchSystem(config, maxCores, maxMemory, maxDisk)[source]¶
Bases:
toil.batchSystems.abstractGridEngineBatchSystem.AbstractGridEngineBatchSystem
A partial implementation of BatchSystemSupport for batch systems run on a standard HPC cluster. By default auto-deployment is not implemented.
- Parameters:
config (toil.common.Config)
maxCores (float)
maxMemory (int)
maxDisk (int)
- class PartitionSet[source]¶
Set of available partitions detected on the slurm batch system
- default_gpu_partition: SlurmBatchSystem | None¶
- all_partitions: list[SlurmBatchSystem]¶
- class GridEngineThread(newJobsQueue, updatedJobsQueue, killQueue, killedJobsQueue, boss)[source]¶
Bases:
toil.batchSystems.abstractGridEngineBatchSystem.AbstractGridEngineBatchSystem.GridEngineThread
A class that represents a thread of control.
This class can be safely subclassed in a limited fashion. There are two ways to specify the activity: by passing a callable object to the constructor, or by overriding the run() method in a subclass.
- Parameters:
newJobsQueue (queue.Queue)
updatedJobsQueue (queue.Queue)
killQueue (queue.Queue)
killedJobsQueue (queue.Queue)
- boss: SlurmBatchSystem¶
- getRunningJobIDs()[source]¶
Get a list of running job IDs. Implementation-specific; called by boss AbstractGridEngineBatchSystem implementation via AbstractGridEngineBatchSystem.getRunningBatchJobIDs()
- Return type:
- killJob(jobID)[source]¶
Kill specific job with the Toil job ID. Implementation-specific; called by GridEngineThread.killJobs()
- Parameters:
jobID (string) – Toil job ID
- Return type:
None
- prepareSubmission(cpu, memory, jobID, command, jobName, job_environment=None, gpus=None)[source]¶
Preparation in putting together a command-line string for submitting to batch system (via submitJob().)
- Param:
int cpu
- Param:
int memory
- Param:
int jobID: Toil job ID
- Param:
string subLine: the command line string to be called
- Param:
string jobName: the name of the Toil job, to provide metadata to batch systems if desired
- Param:
dict job_environment: the environment variables to be set on the worker
- Return type:
List[str]
- Parameters:
- submitJob(subLine)[source]¶
Wrapper routine for submitting the actual command-line call, then processing the output to get the batch system job ID
- coalesce_job_exit_codes(batch_job_id_list)[source]¶
Collect all job exit codes in a single call. :param batch_job_id_list: list of Job ID strings, where each string has the form “<job>[.<task>]”. :return: list of job exit codes or exit code, exit reason pairs associated with the list of job IDs.
- partitions¶
- issueBatchJob(command, job_desc, job_environment=None)[source]¶
Issues a job with the specified command to the batch system and returns a unique job ID number.
- Parameters:
command (str) – the command to execute somewhere to run the Toil worker process
job_desc (toil.job.JobDescription) – the JobDescription for the job being run
job_environment (dict[str, str] | None) – a collection of job-specific environment variables to be set on the worker.
- Returns:
a unique job ID number that can be used to reference the newly issued job
- Return type:
- classmethod add_options(parser)[source]¶
If this batch system provides any command line options, add them to the given parser.
- Parameters:
parser (argparse.ArgumentParser | argparse._ArgumentGroup)
- Return type:
None
- OptionType¶
- classmethod setOptions(setOption)[source]¶
Process command line or configuration options relevant to this batch system.
- Parameters:
setOption (toil.batchSystems.options.OptionSetter) – A function with signature setOption(option_name, parsing_function=None, check_function=None, default=None, env=None) returning nothing, used to update run configuration as a side effect.
- Return type:
None