Module Contents



The interface for Parasol.



class toil.batchSystems.parasol.ParasolBatchSystem(config, maxCores, maxMemory, maxDisk)[source]

Bases: toil.batchSystems.abstractBatchSystem.BatchSystemSupport

digraph inheritance6c01dbfb68 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "ABC" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="Helper class that provides a standard way to create an ABC using"]; "AbstractBatchSystem" [URL="../abstractBatchSystem/index.html#toil.batchSystems.abstractBatchSystem.AbstractBatchSystem",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="An abstract base class to represent the interface the batch system must provide to Toil."]; "ABC" -> "AbstractBatchSystem" [arrowsize=0.5,style="setlinewidth(0.5)"]; "BatchSystemSupport" [URL="../abstractBatchSystem/index.html#toil.batchSystems.abstractBatchSystem.BatchSystemSupport",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Partial implementation of AbstractBatchSystem, support methods."]; "AbstractBatchSystem" -> "BatchSystemSupport" [arrowsize=0.5,style="setlinewidth(0.5)"]; "ParasolBatchSystem" [URL="#toil.batchSystems.parasol.ParasolBatchSystem",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="The interface for Parasol."]; "BatchSystemSupport" -> "ParasolBatchSystem" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

The interface for Parasol.

classmethod supportsWorkerCleanup()[source]

Indicates whether this batch system invokes BatchSystemSupport.workerCleanup() after the last job for a particular workflow invocation finishes. Note that the term worker refers to an entire node, not just a worker process. A worker process may run more than one job sequentially, and more than one concurrent worker process may exist on a worker node, for the same workflow. The batch system is said to shut down after the last worker process terminates.

classmethod supportsAutoDeployment()[source]

Whether this batch system supports auto-deployment of the user script itself.

If it does, the setUserScript() can be invoked to set the resource object representing the user script.

Note to implementors: If your implementation returns True here, it should also override

issueBatchJob(jobDesc, job_environment=None)[source]

Issue parasol with job commands.


job_environment (Optional[Dict[str, str]]) –

setEnv(name, value=None)[source]

Set an environment variable for the worker process before it is launched. The worker process will typically inherit the environment of the machine it is running on but this method makes it possible to override specific variables in that inherited environment before the worker is launched. Note that this mechanism is different to the one used by the worker internally to set up the environment of a job. A call to this method affects all jobs issued after this method returns. Note to implementors: This means that you would typically need to copy the variables before enqueuing a job.

If no value is provided it will be looked up from the current environment.

  • name – the environment variable to be set on the worker.

  • value – if given, the environment variable given by name will be set to this value.

if None, the variable’s current value will be used as the value on the worker


RuntimeError – if value is None and the name cannot be found in the environment


Kills the given jobs, represented as Job ids, then checks they are dead by checking they are not in the list of issued jobs.


Get all queued and running jobs for a results file.


Gets the list of jobs issued to parasol in all results files, but not including jobs created by other users.


Returns map of running jobIDs and the time they have been running.


Returns information about job that has updated its status (i.e. ceased running, either successfully or with an error). Each such job will be returned exactly once.

Does not return info for jobs killed by killBatchJobs, although they may cause None to be returned earlier than maxWait.


maxWait – the number of seconds to block, waiting for a result


If a result is available, returns UpdatedBatchJobInfo. Otherwise it returns None. wallTime is the number of seconds (a strictly positive float) in wall-clock time the job ran for, or None if this batch system does not support tracking wall time.


We use the parasol results to update the status of jobs, adding them to the list of updated jobs.

Results have the following structure.. (thanks Mark D!)

int status;

Job status - wait() return format. 0 is good.

char host;

Machine job ran on.

char jobId;

Job queuing system job ID

char exe;

Job executable file (no path)

int usrTicks;

‘User’ CPU time in ticks.

int sysTicks;

‘System’ CPU time in ticks.

unsigned submitTime;

Job submission time in seconds since 1/1/1970

unsigned startTime;

Job start time in seconds since 1/1/1970

unsigned endTime;

Job end time in seconds since 1/1/1970

char user;

User who ran job

char errFile;

Location of stderr file on host

Plus you finally have the command name.


Called at the completion of a toil invocation. Should cleanly terminate all worker threads.

Return type


classmethod add_options(parser)[source]

If this batch system provides any command line options, add them to the given parser.


parser (Union[argparse.ArgumentParser, argparse._ArgumentGroup]) –

Return type


classmethod setOptions(setOption)[source]

Process command line or configuration options relevant to this batch system.


setOption (toil.batchSystems.options.OptionSetter) – A function with signature setOption(option_name, parsing_function=None, check_function=None, default=None, env=None) returning nothing, used to update run configuration as a side effect.