toil.test.src.jobTest

Module Contents

Classes

JobTest	Tests the job class.
TrivialService	Abstract class used to define the interface to a service.

Functions

simpleJobFn(job, value)
fn1Test(string, outputFile)	Function appends the next character after the last character in the given
fn2Test(pStrings, s, outputFile)	Function concatenates the strings in pStrings and s, in that order, and writes the result to
trivialParent(job)
parent(job)
diamond(job)
child(job)
errorChild(job)

Attributes

logger

toil.test.src.jobTest.logger

class toil.test.src.jobTest.JobTest(methodName='runTest')

Bases: toil.test.ToilTest

digraph inheritance5cba658aa8 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "JobTest" [URL="#toil.test.src.jobTest.JobTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Tests the job class."]; "ToilTest" -> "JobTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Tests the job class.

classmethod setUpClass()

Hook method for setting up class fixture before running tests in the class.

testStatic()

Create a DAG of jobs non-dynamically and run it. DAG is:

A -> F -—— B -> D

——- C -> E

Follow on is marked by ->

testStatic2()

Create a DAG of jobs non-dynamically and run it. DAG is:

A -> F -—— B -> D

——- C -> E

Follow on is marked by ->

testTrivialDAGConsistency()

testDAGConsistency()

testSiblingDAGConsistency()

Slightly more complex case. The stranded job’s predecessors are siblings instead of parent/child.

testDeadlockDetection()

Randomly generate job graphs with various types of cycle in them and check they cause an exception properly. Also check that multiple roots causes a deadlock exception.

testNewCheckpointIsLeafVertexNonRootCase()

Test for issue #1465: Detection of checkpoint jobs that are not leaf vertices identifies leaf vertices incorrectly

Test verification of new checkpoint jobs being leaf verticies, starting with the following baseline workflow:

Parent

Child # Checkpoint=True

testNewCheckpointIsLeafVertexRootCase()

Test for issue #1466: Detection of checkpoint jobs that are not leaf vertices

omits the workflow root job

Test verification of a new checkpoint job being leaf vertex, starting with a baseline workflow of a single, root job:

Root # Checkpoint=True

runNewCheckpointIsLeafVertexTest(createWorkflowFn)

Test verification that a checkpoint job is a leaf vertex using both valid and invalid cases.

Parameters:

createWorkflowFn –

function to create and new workflow and return a tuple of:

0. the workflow root job

1. a checkpoint job to test within the workflow

runCheckpointVertexTest(workflowRootJob, checkpointJob, checkpointJobService=None, checkpointJobChild=None, checkpointJobFollowOn=None, expectedException=None)

Modifies the checkpoint job according to the given parameters then runs the workflow, checking for the expected exception, if any.

testEvaluatingRandomDAG()

Randomly generate test input then check that the job graph can be run successfully, using the existence of promises to validate the run.

static getRandomEdge(nodeNumber)

static makeRandomDAG(nodeNumber)

Makes a random dag with “nodeNumber” nodes in which all nodes are connected. Return value is list of edges, each of form (a, b), where a and b are integers >= 0 < nodeNumber referring to nodes and the edge is from a to b.

static getAdjacencyList(nodeNumber, edges)

Make adjacency list representation of edges

reachable(node, adjacencyList, followOnAdjacencyList=None)

Find the set of nodes reachable from this node (including the node). Return is a set of integers.

addRandomFollowOnEdges(childAdjacencyList)

Adds random follow on edges to the graph, represented as an adjacency list. The follow on edges are returned as a set and their augmented edges are added to the adjacency list.

makeJobGraph(nodeNumber, childEdges, followOnEdges, outPath, addServices=True)

Converts a DAG into a job graph. childEdges and followOnEdges are the lists of child and followOn edges.

isAcyclic(adjacencyList)

Returns true if there are any cycles in the graph, which is represented as an adjacency list.

toil.test.src.jobTest.simpleJobFn(job, value)

toil.test.src.jobTest.fn1Test(string, outputFile)

Function appends the next character after the last character in the given string to the string, writes the string to a file, and returns it. For example, if string is “AB”, we will write and return “ABC”.

toil.test.src.jobTest.fn2Test(pStrings, s, outputFile)

Function concatenates the strings in pStrings and s, in that order, and writes the result to the output file. Returns s.

toil.test.src.jobTest.trivialParent(job)

toil.test.src.jobTest.parent(job)

toil.test.src.jobTest.diamond(job)

toil.test.src.jobTest.child(job)

toil.test.src.jobTest.errorChild(job)

class toil.test.src.jobTest.TrivialService(message, *args, **kwargs)

Bases: toil.job.Job.Service

digraph inheritance44e29e17c2 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "Requirer" [URL="../../../job/index.html#toil.job.Requirer",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Base class implementing the storage and presentation of requirements."]; "Service" [URL="../../../job/index.html#toil.job.Job.Service",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Abstract class used to define the interface to a service."]; "Requirer" -> "Service" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TrivialService" [URL="#toil.test.src.jobTest.TrivialService",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "Service" -> "TrivialService" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Abstract class used to define the interface to a service.

Should be subclassed by the user to define services.

Is not executed as a job; runs within a ServiceHostJob.

start(job)

Start the service.

Parameters:

job – The underlying host job that the service is being run in. Can be used to register deferred functions, or to access the fileStore for creating temporary files.

Returns:

An object describing how to access the service. The object must be pickleable and will be used by jobs to access the service (see toil.job.Job.addService()).

stop(job)

Stops the service. Function can block until complete.

Parameters:

job – The underlying host job that the service is being run in. Can be used to register deferred functions, or to access the fileStore for creating temporary files.

check()

Checks the service is still running.

Raises:

exceptions.RuntimeError – If the service failed, this will cause the service job to be labeled failed.

Returns:

True if the service is still running, else False. If False then the service job will be terminated, and considered a success. Important point: if the service job exits due to a failure, it should raise a RuntimeError, not return False!