toil.test.batchSystems.test_slurm

Attributes

EXIT_STATUS_UNAVAILABLE_VALUE

Exceptions

CalledProcessErrorStderr

Version of CalledProcessError that include stderr in the error message if it is set

Classes

BatchJobExitReason	Enum where members are also (and must be) ints
Config	Class to represent configuration operations for a toil workflow run.
ToilTest	A common base class for Toil tests.
FakeBatchSystem	Class that implements a minimal Batch System, needed to create a Worker (see below).
SlurmTest	Class for unit-testing SlurmBatchSystem

Functions

call_sacct(args, **_)	The arguments passed to call_command when executing sacct are:
call_scontrol(args, **_)	The arguments passed to call_command when executing scontrol are:
call_sacct_raises(*_)	Fake that the sacct command fails by raising a CalledProcessErrorStderr

Module Contents

class toil.test.batchSystems.test_slurm.BatchJobExitReason

Bases: enum.IntEnum

Enum where members are also (and must be) ints

FINISHED: int = 1

Successfully finished.

FAILED: int = 2

Job finished, but failed.

LOST: int = 3

Preemptable failure (job’s executing host went away).

KILLED: int = 4

Job killed before finishing.

ERROR: int = 5

Internal error.

MEMLIMIT: int = 6

Job hit batch system imposed memory limit.

MISSING: int = 7

Job disappeared from the scheduler without actually stopping, so Toil killed it.

MAXJOBDURATION: int = 8

Job ran longer than –maxJobDuration, so Toil killed it.

PARTITION: int = 9

Job was not able to talk to the leader via the job store, so Toil declared it failed.

classmethod to_string(value)

Convert to human-readable string.

Given an int that may be or may be equal to a value from the enum, produce the string value of its matching enum entry, or a stringified int.

Parameters:

value (int)

Return type:

str

toil.test.batchSystems.test_slurm.EXIT_STATUS_UNAVAILABLE_VALUE = 255

class toil.test.batchSystems.test_slurm.Config

Class to represent configuration operations for a toil workflow run.

logFile: str | None

logRotating: bool

cleanWorkDir: str

max_jobs: int

max_local_jobs: int

manualMemArgs: bool

run_local_jobs_on_workers: bool

coalesceStatusCalls: bool

mesos_endpoint: str | None

mesos_framework_id: str | None

mesos_role: str | None

mesos_name: str

kubernetes_host_path: str | None

kubernetes_owner: str | None

kubernetes_service_account: str | None

kubernetes_pod_timeout: float

kubernetes_privileged: bool

tes_endpoint: str

tes_user: str

tes_password: str

tes_bearer_token: str

aws_batch_region: str | None

aws_batch_queue: str | None

aws_batch_job_role_arn: str | None

scale: float

batchSystem: str

batch_logs_dir: str | None

The backing scheduler will be instructed, if possible, to save logs to this directory, where the leader can read them.

statePollingWait: int

state_polling_timeout: int

disableAutoDeployment: bool

workflowID: str | None

This attribute uniquely identifies the job store and therefore the workflow. It is necessary in order to distinguish between two consecutive workflows for which self.jobStore is the same, e.g. when a job store name is reused after a previous run has finished successfully and its job store has been clean up.

workflowAttemptNumber: int

jobStore: str

logLevel: str

colored_logs: bool

workDir: str | None

coordination_dir: str | None

noStdOutErr: bool

stats: bool

clean: str | None

clusterStats: str

restart: bool

caching: bool | None

symlinkImports: bool

moveOutputs: bool

provisioner: str | None

nodeTypes: List[Tuple[Set[str], float | None]]

minNodes: List[int]

maxNodes: List[int]

targetTime: float

betaInertia: float

scaleInterval: int

preemptibleCompensation: float

nodeStorage: int

nodeStorageOverrides: List[str]

metrics: bool

assume_zero_overhead: bool

maxPreemptibleServiceJobs: int

maxServiceJobs: int

deadlockWait: float | int

deadlockCheckInterval: float | int

defaultMemory: int

defaultCores: float | int

defaultDisk: int

defaultPreemptible: bool

defaultAccelerators: List[toil.job.AcceleratorRequirement]

maxCores: int

maxMemory: int

maxDisk: int

retryCount: int

enableUnlimitedPreemptibleRetries: bool

doubleMem: bool

maxJobDuration: int

rescueJobsFrequency: int

job_store_timeout: float

maxLogFileSize: int

writeLogs: str

writeLogsGzip: str

writeLogsFromAllJobs: bool

write_messages: str | None

realTimeLogging: bool

environment: Dict[str, str]

disableChaining: bool

disableJobStoreChecksumVerification: bool

sseKey: str | None

servicePollingInterval: int

useAsync: bool

forceDockerAppliance: bool

statusWait: int

disableProgress: bool

readGlobalFileMutableByDefault: bool

debugWorker: bool

disableWorkerOutputCapture: bool

badWorker: float

badWorkerFailInterval: float

kill_polling_interval: int

cwl: bool

set_from_default_config()

Return type:

None

prepare_start()

After options are set, prepare for initial start of workflow.

Return type:

None

prepare_restart()

Before restart options are set, prepare for a restart of a workflow. Set up any execution-specific parameters and clear out any stale ones.

Return type:

None

setOptions(options)

Creates a config object from the options object.

Parameters:

options (argparse.Namespace)

Return type:

None

check_configuration_consistency()

Old checks that cannot be fit into an action class for argparse

Return type:

None

__eq__(other)

Return self==value.

Parameters:

other (object)

Return type:

bool

__hash__()

Return hash(self).

Return type:

int

exception toil.test.batchSystems.test_slurm.CalledProcessErrorStderr(returncode, cmd, output=None, stderr=None)

Bases: subprocess.CalledProcessError

Version of CalledProcessError that include stderr in the error message if it is set

__str__()

Return str(self).

Return type:

str

class toil.test.batchSystems.test_slurm.ToilTest(methodName='runTest')

Bases: unittest.TestCase

A common base class for Toil tests.

Please have every test case directly or indirectly inherit this one.

When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.

setup_method(method)

Parameters:

method (Any)

Return type:

None

classmethod setUpClass()

Hook method for setting up class fixture before running tests in the class.

Return type:

None

classmethod tearDownClass()

Hook method for deconstructing the class fixture after running all tests in the class.

Return type:

None

setUp()

Hook method for setting up the test fixture before exercising it.

Return type:

None

tearDown()

Hook method for deconstructing the test fixture after testing it.

Return type:

None

classmethod awsRegion()

Pick an appropriate AWS region.

Use us-west-2 unless running on EC2, in which case use the region in which the instance is located

Return type:

str

toil.test.batchSystems.test_slurm.call_sacct(args, **_)

The arguments passed to call_command when executing sacct are: [‘sacct’, ‘-n’, ‘-j’, ‘<comma-separated list of job-ids>’, ‘–format’, ‘JobIDRaw,State,ExitCode’, ‘-P’, ‘-S’, ‘1970-01-01’] The multi-line output is something like:

1234|COMPLETED|0:0
1234.batch|COMPLETED|0:0
1235|PENDING|0:0
1236|FAILED|0:2
1236.extern|COMPLETED|0:0

Return type:

str

toil.test.batchSystems.test_slurm.call_scontrol(args, **_)

The arguments passed to call_command when executing scontrol are: ['scontrol', 'show', 'job'] or ['scontrol', 'show', 'job', '<job-id>']

Return type:

str

toil.test.batchSystems.test_slurm.call_sacct_raises(*_)

Fake that the sacct command fails by raising a CalledProcessErrorStderr

class toil.test.batchSystems.test_slurm.FakeBatchSystem

Class that implements a minimal Batch System, needed to create a Worker (see below).

getWaitDuration()

class toil.test.batchSystems.test_slurm.SlurmTest(methodName='runTest')

Bases: toil.test.ToilTest

Class for unit-testing SlurmBatchSystem

setUp()

Hook method for setting up the test fixture before exercising it.

test_getJobDetailsFromSacct_one_exists()

test_getJobDetailsFromSacct_one_not_exists()

test_getJobDetailsFromSacct_many_all_exist()

test_getJobDetailsFromSacct_many_some_exist()

test_getJobDetailsFromSacct_many_none_exist()

test_getJobDetailsFromScontrol_one_exists()

test_getJobDetailsFromScontrol_one_not_exists()

Asking for the job details of a single job that scontrol doesn’t know about should raise an exception.

test_getJobDetailsFromScontrol_many_all_exist()

test_getJobDetailsFromScontrol_many_some_exist()

test_getJobDetailsFromScontrol_many_none_exist()

test_getJobExitCode_job_exists()

test_getJobExitCode_job_not_exists()

test_getJobExitCode_sacct_raises_job_exists()

This test forces the use of scontrol to get job information, by letting sacct raise an exception.

test_getJobExitCode_sacct_raises_job_not_exists()

This test forces the use of scontrol to get job information, by letting sacct raise an exception. Next, scontrol should also raise because it doesn’t know the job.

test_coalesce_job_exit_codes_one_exists()

test_coalesce_job_exit_codes_one_not_exists()

test_coalesce_job_exit_codes_many_all_exist()

test_coalesce_job_exit_codes_some_exists()

test_coalesce_job_exit_codes_sacct_raises_job_exists()

This test forces the use of scontrol to get job information, by letting sacct raise an exception.

test_coalesce_job_exit_codes_sacct_raises_job_not_exists()

This test forces the use of scontrol to get job information, by letting sacct raise an exception. Next, scontrol should also raise because it doesn’t know the job.