toil.test.src.promisedRequirementTest

Module Contents

Classes

hidden

Hide abstract base class from unittest's test case loader.

SingleMachinePromisedRequirementsTest

Tests against the SingleMachine batch system

MesosPromisedRequirementsTest

Tests against the Mesos batch system

Functions

maxConcurrency(job, cpuCount, filename, coresPerJob)

Returns the max number of concurrent tasks when using a PromisedRequirement instance

getOne()

getThirtyTwoMb()

logDiskUsage(job, funcName[, sleep])

Logs the job's disk usage to master and sleeps for specified amount of time.

Attributes

log

toil.test.src.promisedRequirementTest.log[source]
class toil.test.src.promisedRequirementTest.hidden[source]

Hide abstract base class from unittest’s test case loader.

http://stackoverflow.com/questions/1323455/python-unit-test-with-base-and-sub-class#answer-25695512

class AbstractPromisedRequirementsTest(methodName='runTest')[source]

Bases: toil.test.batchSystems.batchSystemTest.hidden.AbstractBatchSystemJobTest

digraph inheritancee8f3914c1c { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AbstractBatchSystemJobTest" [URL="../../batchSystems/batchSystemTest/index.html#toil.test.batchSystems.batchSystemTest.hidden.AbstractBatchSystemJobTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="An abstract base class for batch system tests that use a full Toil workflow rather"]; "ToilTest" -> "AbstractBatchSystemJobTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractPromisedRequirementsTest" [URL="#toil.test.src.promisedRequirementTest.hidden.AbstractPromisedRequirementsTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="An abstract base class for testing Toil workflows with promised requirements."]; "AbstractBatchSystemJobTest" -> "AbstractPromisedRequirementsTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

An abstract base class for testing Toil workflows with promised requirements.

testConcurrencyDynamic()[source]

Asserts that promised core resources are allocated properly using a dynamic Toil workflow

testConcurrencyStatic()[source]

Asserts that promised core resources are allocated properly using a static DAG

getOptions(tempDir, caching=True)[source]

Configures options for Toil workflow and makes job store. :param str tempDir: path to test directory :return: Toil options object

getCounterPath(tempDir)[source]

Returns path to a counter file :param str tempDir: path to test directory :return: path to counter file

testPromisesWithJobStoreFileObjects(caching=True)[source]

Check whether FileID objects are being pickled properly when used as return values of functions. Then ensure that lambdas of promised FileID objects can be used to describe the requirements of a subsequent job. This type of operation will be used commonly in Toil scripts. :return: None

testPromisesWithNonCachingFileStore()[source]
testPromiseRequirementRaceStatic()[source]

Checks for a race condition when using promised requirements and child job functions.

toil.test.src.promisedRequirementTest.maxConcurrency(job, cpuCount, filename, coresPerJob)[source]

Returns the max number of concurrent tasks when using a PromisedRequirement instance to allocate the number of cores per job.

Parameters:

  • cpuCount (int) – number of available cpus

  • filename (str) – path to counter file

  • coresPerJob (int) – number of cores assigned to each job

Return int max concurrency value:

toil.test.src.promisedRequirementTest.getOne()[source]
toil.test.src.promisedRequirementTest.getThirtyTwoMb()[source]
toil.test.src.promisedRequirementTest.logDiskUsage(job, funcName, sleep=0)[source]

Logs the job’s disk usage to master and sleeps for specified amount of time.

Returns:

job function’s disk usage

class toil.test.src.promisedRequirementTest.SingleMachinePromisedRequirementsTest(methodName='runTest')[source]

Bases: hidden

digraph inheritancebeaeb573b5 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AbstractBatchSystemJobTest" [URL="../../batchSystems/batchSystemTest/index.html#toil.test.batchSystems.batchSystemTest.hidden.AbstractBatchSystemJobTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="An abstract base class for batch system tests that use a full Toil workflow rather"]; "ToilTest" -> "AbstractBatchSystemJobTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractPromisedRequirementsTest" [URL="#toil.test.src.promisedRequirementTest.hidden.AbstractPromisedRequirementsTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="An abstract base class for testing Toil workflows with promised requirements."]; "AbstractBatchSystemJobTest" -> "AbstractPromisedRequirementsTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "SingleMachinePromisedRequirementsTest" [URL="#toil.test.src.promisedRequirementTest.SingleMachinePromisedRequirementsTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Tests against the SingleMachine batch system"]; "AbstractPromisedRequirementsTest" -> "SingleMachinePromisedRequirementsTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Tests against the SingleMachine batch system

getBatchSystemName()[source]
Return type:

(str, AbstractBatchSystem)

tearDown()[source]

Hook method for deconstructing the test fixture after testing it.

class toil.test.src.promisedRequirementTest.MesosPromisedRequirementsTest(methodName='runTest')[source]

Bases: hidden, toil.batchSystems.mesos.test.MesosTestSupport

digraph inheritance8c5ea6b240 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AbstractBatchSystemJobTest" [URL="../../batchSystems/batchSystemTest/index.html#toil.test.batchSystems.batchSystemTest.hidden.AbstractBatchSystemJobTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="An abstract base class for batch system tests that use a full Toil workflow rather"]; "ToilTest" -> "AbstractBatchSystemJobTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractPromisedRequirementsTest" [URL="#toil.test.src.promisedRequirementTest.hidden.AbstractPromisedRequirementsTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="An abstract base class for testing Toil workflows with promised requirements."]; "AbstractBatchSystemJobTest" -> "AbstractPromisedRequirementsTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "MesosPromisedRequirementsTest" [URL="#toil.test.src.promisedRequirementTest.MesosPromisedRequirementsTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Tests against the Mesos batch system"]; "AbstractPromisedRequirementsTest" -> "MesosPromisedRequirementsTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "MesosTestSupport" -> "MesosPromisedRequirementsTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "MesosTestSupport" [URL="../../../batchSystems/mesos/test/index.html#toil.batchSystems.mesos.test.MesosTestSupport",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Mixin for test cases that need a running Mesos master and agent on the local host."]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Tests against the Mesos batch system

getOptions(tempDir, caching=True)[source]

Configures options for Toil workflow and makes job store. :param str tempDir: path to test directory :return: Toil options object

getBatchSystemName()[source]
Return type:

(str, AbstractBatchSystem)

tearDown()[source]

Hook method for deconstructing the test fixture after testing it.