toil.batchSystems.gridengine

Module Contents

Classes

GridEngineBatchSystem

A partial implementation of BatchSystemSupport for batch systems run on a

Attributes

logger

toil.batchSystems.gridengine.logger
class toil.batchSystems.gridengine.GridEngineBatchSystem(config, maxCores, maxMemory, maxDisk)[source]

Bases: toil.batchSystems.abstractGridEngineBatchSystem.AbstractGridEngineBatchSystem

Inheritance diagram of toil.batchSystems.gridengine.GridEngineBatchSystem

A partial implementation of BatchSystemSupport for batch systems run on a standard HPC cluster. By default auto-deployment is not implemented.

class Worker(newJobsQueue, updatedJobsQueue, killQueue, killedJobsQueue, boss)[source]

Bases: toil.batchSystems.abstractGridEngineBatchSystem.AbstractGridEngineBatchSystem.Worker

Inheritance diagram of toil.batchSystems.gridengine.GridEngineBatchSystem.Worker

Grid Engine-specific AbstractGridEngineWorker methods

Parameters
getRunningJobIDs()[source]

Get a list of running job IDs. Implementation-specific; called by boss AbstractGridEngineBatchSystem implementation via AbstractGridEngineBatchSystem.getRunningBatchJobIDs()

Return type

list

killJob(jobID)[source]

Kill specific job with the Toil job ID. Implementation-specific; called by AbstractGridEngineWorker.killJobs()

Parameters

jobID (string) – Toil job ID

prepareSubmission(cpu, memory, jobID, command, jobName, job_environment=None, gpus=None)[source]

Preparation in putting together a command-line string for submitting to batch system (via submitJob().)

Param

int cpu

Param

int memory

Param

int jobID: Toil job ID

Param

string subLine: the command line string to be called

Param

string jobName: the name of the Toil job, to provide metadata to batch systems if desired

Param

dict job_environment: the environment variables to be set on the worker

Return type

List[str]

Parameters

  • cpu (int) –

  • memory (int) –

  • jobID (int) –

  • command (str) –

  • jobName (str) –

  • job_environment (Optional[Dict[str, str]]) –

  • gpus (Optional[int]) –

submitJob(subLine)[source]

Wrapper routine for submitting the actual command-line call, then processing the output to get the batch system job ID

Param

string subLine: the literal command line string to be called

Return type

string: batch system job ID, which will be stored internally

getJobExitCode(sgeJobID)[source]

Get job exist code, checking both qstat and qacct. Return None if still running. Higher level should retry on CalledProcessErrorStderr, for the case the job has finished and qacct result is stale.

prepareQsub(cpu, mem, jobID, job_environment=None)[source]
Parameters

  • cpu (int) –

  • mem (int) –

  • jobID (int) –

  • job_environment (Optional[Dict[str, str]]) –

Return type

List[str]

classmethod getWaitDuration()[source]