toil.test.batchSystems.batch_system_plugin_test

Attributes

logger

Classes

FakeBatchSystem

Adds cleanup support when the last running job leaves a node, for batch

BatchSystemPluginTest

A common base class for Toil tests.

Module Contents

toil.test.batchSystems.batch_system_plugin_test.logger
class toil.test.batchSystems.batch_system_plugin_test.FakeBatchSystem(config, maxCores, maxMemory, maxDisk)[source]

Bases: toil.batchSystems.cleanup_support.BatchSystemCleanupSupport

Adds cleanup support when the last running job leaves a node, for batch systems that can’t provide it using the backing scheduler.

Parameters:
classmethod supportsAutoDeployment()[source]

Whether this batch system supports auto-deployment of the user script itself.

If it does, the setUserScript() can be invoked to set the resource object representing the user script.

Note to implementors: If your implementation returns True here, it should also override

Return type:

bool

issueBatchJob(command, job_desc, job_environment=None)[source]

Issues a job with the specified command to the batch system and returns a unique job ID number.

Parameters:
  • command (str) – the command to execute somewhere to run the Toil worker process

  • job_desc (toil.job.JobDescription) – the JobDescription for the job being run

  • job_environment (Optional[Dict[str, str]]) – a collection of job-specific environment variables to be set on the worker.

Returns:

a unique job ID number that can be used to reference the newly issued job

Return type:

int

killBatchJobs(jobIDs)[source]

Kills the given job IDs. After returning, the killed jobs will not appear in the results of getRunningBatchJobIDs. The killed job will not be returned from getUpdatedBatchJob.

Parameters:

jobIDs (List[int]) – list of IDs of jobs to kill

Return type:

None

getIssuedBatchJobIDs()[source]

Gets all currently issued jobs

Returns:

A list of jobs (as job ID numbers) currently issued (may be running, or may be waiting to be run). Despite the result being a list, the ordering should not be depended upon.

Return type:

List[int]

getRunningBatchJobIDs()[source]

Gets a map of jobs as job ID numbers that are currently running (not just waiting) and how long they have been running, in seconds.

Returns:

dictionary with currently running job ID number keys and how many seconds they have been running as the value

Return type:

Dict[int, float]

getUpdatedBatchJob(maxWait)[source]

Returns information about job that has updated its status (i.e. ceased running, either successfully or with an error). Each such job will be returned exactly once.

Does not return info for jobs killed by killBatchJobs, although they may cause None to be returned earlier than maxWait.

Parameters:

maxWait (int) – the number of seconds to block, waiting for a result

Returns:

If a result is available, returns UpdatedBatchJobInfo. Otherwise it returns None. wallTime is the number of seconds (a strictly positive float) in wall-clock time the job ran for, or None if this batch system does not support tracking wall time.

Return type:

Optional[toil.batchSystems.abstractBatchSystem.UpdatedBatchJobInfo]

shutdown()[source]

Called at the completion of a toil invocation. Should cleanly terminate all worker threads.

Return type:

None

classmethod add_options(parser)[source]

If this batch system provides any command line options, add them to the given parser.

Parameters:

parser (configargparse.ArgumentParser)

Return type:

None

classmethod setOptions(setOption)[source]

Process command line or configuration options relevant to this batch system.

Parameters:

setOption (toil.batchSystems.options.OptionSetter) – A function with signature setOption(option_name, parsing_function=None, check_function=None, default=None, env=None) returning nothing, used to update run configuration as a side effect.

Return type:

None

class toil.test.batchSystems.batch_system_plugin_test.BatchSystemPluginTest(methodName='runTest')[source]

Bases: toil.test.ToilTest

A common base class for Toil tests.

Please have every test case directly or indirectly inherit this one.

When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.

test_batchsystem_plugin_installable()[source]

Test that installing a batch system plugin works. :return: