toil.batchSystems.torque

Module Contents

Classes

TorqueBatchSystem

A partial implementation of BatchSystemSupport for batch systems run on a

Attributes

logger

toil.batchSystems.torque.logger
class toil.batchSystems.torque.TorqueBatchSystem(config, maxCores, maxMemory, maxDisk)[source]

Bases: toil.batchSystems.abstractGridEngineBatchSystem.AbstractGridEngineBatchSystem

Inheritance diagram of toil.batchSystems.torque.TorqueBatchSystem

A partial implementation of BatchSystemSupport for batch systems run on a standard HPC cluster. By default auto-deployment is not implemented.

class Worker(newJobsQueue, updatedJobsQueue, killQueue, killedJobsQueue, boss)[source]

Bases: toil.batchSystems.abstractGridEngineBatchSystem.AbstractGridEngineBatchSystem.Worker

Inheritance diagram of toil.batchSystems.torque.TorqueBatchSystem.Worker

A class that represents a thread of control.

This class can be safely subclassed in a limited fashion. There are two ways to specify the activity: by passing a callable object to the constructor, or by overriding the run() method in a subclass.

getRunningJobIDs()[source]

Get a list of running job IDs. Implementation-specific; called by boss AbstractGridEngineBatchSystem implementation via AbstractGridEngineBatchSystem.getRunningBatchJobIDs()

Return type

list

getUpdatedBatchJob(maxWait)[source]
killJob(jobID)[source]

Kill specific job with the Toil job ID. Implementation-specific; called by AbstractGridEngineWorker.killJobs()

Parameters

jobID (string) – Toil job ID

prepareSubmission(cpu, memory, jobID, command, jobName, job_environment=None, gpus=None)[source]

Preparation in putting together a command-line string for submitting to batch system (via submitJob().)

Param

int cpu

Param

int memory

Param

int jobID: Toil job ID

Param

string subLine: the command line string to be called

Param

string jobName: the name of the Toil job, to provide metadata to batch systems if desired

Param

dict job_environment: the environment variables to be set on the worker

Return type

List[str]

Parameters

  • cpu (int) –

  • memory (int) –

  • jobID (int) –

  • command (str) –

  • jobName (str) –

  • job_environment (Optional[Dict[str, str]]) –

  • gpus (Optional[int]) –

submitJob(subLine)[source]

Wrapper routine for submitting the actual command-line call, then processing the output to get the batch system job ID

Param

string subLine: the literal command line string to be called

Return type

string: batch system job ID, which will be stored internally

getJobExitCode(torqueJobID)[source]

Returns job exit code or an instance of abstractBatchSystem.BatchJobExitReason. if something else happened other than the job exiting. Implementation-specific; called by AbstractGridEngineWorker.checkOnJobs()

Parameters

batchjobID (string) – batch system job ID

Return type

int|toil.batchSystems.abstractBatchSystem.BatchJobExitReason: exit code int or BatchJobExitReason if something else happened other than job exiting.

prepareQsub(cpu, mem, jobID, job_environment)[source]
Parameters

  • cpu (int) –

  • mem (int) –

  • jobID (int) –

  • job_environment (Optional[Dict[str, str]]) –

Return type

List[str]

generateTorqueWrapper(command, jobID)[source]

A very simple script generator that just wraps the command given; for now this goes to default tempdir