toil.test.src.promisedRequirementTest

Attributes

log

Classes

AbstractPromisedRequirementsTest

An abstract base class for testing Toil workflows with promised requirements.

TestSingleMachinePromisedRequirements

Tests against the SingleMachine batch system

TestMesosPromisedRequirements

Tests against the Mesos batch system

Functions

maxConcurrency(job, cpuCount, filename, coresPerJob)

Returns the max number of concurrent tasks when using a PromisedRequirement instance

getOne()

getThirtyTwoMb()

logDiskUsage(job, funcName[, sleep])

Logs the job's disk usage to master and sleeps for specified amount of time.

Module Contents

toil.test.src.promisedRequirementTest.log
class toil.test.src.promisedRequirementTest.AbstractPromisedRequirementsTest

Bases: toil.test.batchSystems.batchSystemTest.AbstractBatchSystemJobTest

An abstract base class for testing Toil workflows with promised requirements.

testConcurrencyDynamic(tmp_path)

Asserts that promised core resources are allocated properly using a dynamic Toil workflow

Parameters:

tmp_path (pathlib.Path)

Return type:

None

testConcurrencyStatic(tmp_path)

Asserts that promised core resources are allocated properly using a static DAG

Parameters:

tmp_path (pathlib.Path)

Return type:

None

getOptions(tempDir, caching=True)

Configures options for Toil workflow and makes job store. :param str tempDir: path to test directory :return: Toil options object

Parameters:
Return type:

argparse.Namespace

getCounterPath(tempDir)

Returns path to a counter file :param tempDir: path to test directory :return: path to counter file

Parameters:

tempDir (pathlib.Path)

Return type:

pathlib.Path

testPromisesWithJobStoreFileObjects(tmp_path, caching=True)

Check whether FileID objects are being pickled properly when used as return values of functions. Then ensure that lambdas of promised FileID objects can be used to describe the requirements of a subsequent job. This type of operation will be used commonly in Toil scripts. :return: None

Parameters:
Return type:

None

testPromisesWithNonCachingFileStore(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

testPromiseRequirementRaceStatic(tmp_path)

Checks for a race condition when using promised requirements and child job functions.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.src.promisedRequirementTest.maxConcurrency(job, cpuCount, filename, coresPerJob)

Returns the max number of concurrent tasks when using a PromisedRequirement instance to allocate the number of cores per job.

Parameters:

  • cpuCount (int) – number of available cpus

  • filename (pathlib.Path) – path to counter file

  • coresPerJob (int) – number of cores assigned to each job

  • job (toil.job.Job)

Returns:

max concurrency value per CPU

Return type:

list[toil.job.Promise]

toil.test.src.promisedRequirementTest.getOne()
Return type:

int

toil.test.src.promisedRequirementTest.getThirtyTwoMb()
Return type:

str

toil.test.src.promisedRequirementTest.logDiskUsage(job, funcName, sleep=0)

Logs the job’s disk usage to master and sleeps for specified amount of time.

Returns:

job function’s disk usage

Parameters:
Return type:

int

class toil.test.src.promisedRequirementTest.TestSingleMachinePromisedRequirements

Bases: AbstractPromisedRequirementsTest

Tests against the SingleMachine batch system

getBatchSystemName()
Return type:

(str, AbstractBatchSystem)

tearDown()
Return type:

None

class toil.test.src.promisedRequirementTest.TestMesosPromisedRequirements

Bases: AbstractPromisedRequirementsTest, toil.batchSystems.mesos.test.MesosTestSupport

Tests against the Mesos batch system

mesos_support()
Return type:

collections.abc.Generator[None]

getOptions(tempDir, caching=True)

Configures options for Toil workflow and makes job store. :param str tempDir: path to test directory :return: Toil options object

Parameters:
Return type:

argparse.Namespace

getBatchSystemName()
Return type:

(str, AbstractBatchSystem)