toil.test.src.jobServiceTest

Module Contents

Classes

JobServiceTest

Tests testing the Job.Service class

PerfectServiceTest

Tests testing the Job.Service class

ToyService

Abstract class used to define the interface to a service.

ToySerializableService

Abstract class used to define the interface to a service.

Functions

serviceTest(job, outFile, messageInt)

Creates one service and one accessing job, which communicate with two files to establish

serviceTestRecursive(job, outFile, messages)

Creates a chain of services and accessing jobs, each paired together.

serviceTestParallelRecursive(job, outFiles, messageBundles)

Creates multiple chains of services and accessing jobs.

serviceAccessor(job, communicationFiles, outFile, randInt)

Writes a random integer iinto the inJobStoreFileID file, then tries 10 times reading

fnTest(strings, outputFile)

Function concatenates the strings together and writes them to the output file

Attributes

logger

toil.test.src.jobServiceTest.logger
class toil.test.src.jobServiceTest.JobServiceTest(methodName='runTest')[source]

Bases: toil.test.ToilTest

digraph inheritance4e4bdc3617 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "JobServiceTest" [URL="#toil.test.src.jobServiceTest.JobServiceTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Tests testing the Job.Service class"]; "ToilTest" -> "JobServiceTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Tests testing the Job.Service class

testServiceSerialization()[source]

Tests that a service can receive a promise without producing a serialization error.

testService(checkpoint=False)[source]

Tests the creation of a Job.Service with random failures of the worker.

testServiceDeadlock()[source]

Creates a job with more services than maxServices, checks that deadlock is detected.

testServiceWithCheckpoints()[source]

Tests the creation of a Job.Service with random failures of the worker, making the root job use checkpointing to restart the subtree.

testServiceRecursive(checkpoint=True)[source]

Tests the creation of a Job.Service, creating a chain of services and accessing jobs. Randomly fails the worker.

testServiceParallelRecursive(checkpoint=True)[source]

Tests the creation of a Job.Service, creating parallel chains of services and accessing jobs. Randomly fails the worker.

runToil(rootJob, retryCount=1, badWorker=0.5, badWorkedFailInterval=0.1, maxServiceJobs=sys.maxsize, deadlockWait=60)[source]
class toil.test.src.jobServiceTest.PerfectServiceTest(methodName='runTest')[source]

Bases: JobServiceTest

digraph inheritance00806b4df8 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "JobServiceTest" [URL="#toil.test.src.jobServiceTest.JobServiceTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Tests testing the Job.Service class"]; "ToilTest" -> "JobServiceTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "PerfectServiceTest" [URL="#toil.test.src.jobServiceTest.PerfectServiceTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "JobServiceTest" -> "PerfectServiceTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Tests testing the Job.Service class

runToil(rootJob, retryCount=1, badWorker=0, badWorkedFailInterval=1000, maxServiceJobs=sys.maxsize, deadlockWait=60)[source]

Let us run all the tests in the other service test class, but without worker failures.

toil.test.src.jobServiceTest.serviceTest(job, outFile, messageInt)[source]

Creates one service and one accessing job, which communicate with two files to establish that both run concurrently.

toil.test.src.jobServiceTest.serviceTestRecursive(job, outFile, messages)[source]

Creates a chain of services and accessing jobs, each paired together.

toil.test.src.jobServiceTest.serviceTestParallelRecursive(job, outFiles, messageBundles)[source]

Creates multiple chains of services and accessing jobs.

class toil.test.src.jobServiceTest.ToyService(messageInt, *args, **kwargs)[source]

Bases: toil.job.Job.Service

digraph inheritance3310dddd50 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "Requirer" [URL="../../../job/index.html#toil.job.Requirer",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Base class implementing the storage and presentation of requirements."]; "Service" [URL="../../../job/index.html#toil.job.Job.Service",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Abstract class used to define the interface to a service."]; "Requirer" -> "Service" [arrowsize=0.5,style="setlinewidth(0.5)"]; "ToyService" [URL="#toil.test.src.jobServiceTest.ToyService",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "Service" -> "ToyService" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Abstract class used to define the interface to a service.

Should be subclassed by the user to define services.

Is not executed as a job; runs within a ServiceHostJob.

start(job)[source]

Start the service.

Parameters:

job – The underlying host job that the service is being run in. Can be used to register deferred functions, or to access the fileStore for creating temporary files.

Returns:

An object describing how to access the service. The object must be pickleable and will be used by jobs to access the service (see toil.job.Job.addService()).

stop(job)[source]

Stops the service. Function can block until complete.

Parameters:

job – The underlying host job that the service is being run in. Can be used to register deferred functions, or to access the fileStore for creating temporary files.

check()[source]

Checks the service is still running.

Raises:

exceptions.RuntimeError – If the service failed, this will cause the service job to be labeled failed.

Returns:

True if the service is still running, else False. If False then the service job will be terminated, and considered a success. Important point: if the service job exits due to a failure, it should raise a RuntimeError, not return False!

static serviceWorker(jobStore, terminate, error, inJobStoreID, outJobStoreID, messageInt)[source]
toil.test.src.jobServiceTest.serviceAccessor(job, communicationFiles, outFile, randInt)[source]

Writes a random integer iinto the inJobStoreFileID file, then tries 10 times reading from outJobStoreFileID to get a pair of integers, the first equal to i the second written into the outputFile.

class toil.test.src.jobServiceTest.ToySerializableService(messageInt, *args, **kwargs)[source]

Bases: toil.job.Job.Service

digraph inheritancec5cf9b034d { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "Requirer" [URL="../../../job/index.html#toil.job.Requirer",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Base class implementing the storage and presentation of requirements."]; "Service" [URL="../../../job/index.html#toil.job.Job.Service",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Abstract class used to define the interface to a service."]; "Requirer" -> "Service" [arrowsize=0.5,style="setlinewidth(0.5)"]; "ToySerializableService" [URL="#toil.test.src.jobServiceTest.ToySerializableService",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "Service" -> "ToySerializableService" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Abstract class used to define the interface to a service.

Should be subclassed by the user to define services.

Is not executed as a job; runs within a ServiceHostJob.

start(job)[source]

Start the service.

Parameters:

job – The underlying host job that the service is being run in. Can be used to register deferred functions, or to access the fileStore for creating temporary files.

Returns:

An object describing how to access the service. The object must be pickleable and will be used by jobs to access the service (see toil.job.Job.addService()).

stop(job)[source]

Stops the service. Function can block until complete.

Parameters:

job – The underlying host job that the service is being run in. Can be used to register deferred functions, or to access the fileStore for creating temporary files.

check()[source]

Checks the service is still running.

Raises:

exceptions.RuntimeError – If the service failed, this will cause the service job to be labeled failed.

Returns:

True if the service is still running, else False. If False then the service job will be terminated, and considered a success. Important point: if the service job exits due to a failure, it should raise a RuntimeError, not return False!

toil.test.src.jobServiceTest.fnTest(strings, outputFile)[source]

Function concatenates the strings together and writes them to the output file