toil.test.src.jobTest¶
Classes¶
Tests the job class. |
|
Abstract class used to define the interface to a service. |
Functions¶
|
|
|
Function appends the next character after the last character in the given |
|
Function concatenates the strings in pStrings and s, in that order, and writes the result to |
|
|
|
|
|
|
|
|
|
Module Contents¶
- class toil.test.src.jobTest.TestJob¶
Tests the job class.
- testStatic(tmp_path)¶
Create a DAG of jobs non-dynamically and run it. DAG is:
A -> F \------- B -> D \ \ \ ------- C -> E
Follow on is marked by
->
- Parameters:
tmp_path (pathlib.Path)
- Return type:
None
- testStatic2(tmp_path)¶
Create a DAG of jobs non-dynamically and run it. DAG is:
A -> F \------- B -> D \ \ \ ------- C -> E
Follow on is marked by
->
- Parameters:
tmp_path (pathlib.Path)
- Return type:
None
- testTrivialDAGConsistency(tmp_path)¶
- Parameters:
tmp_path (pathlib.Path)
- Return type:
None
- testDAGConsistency(tmp_path)¶
- Parameters:
tmp_path (pathlib.Path)
- Return type:
None
- testSiblingDAGConsistency(tmp_path)¶
Slightly more complex case. The stranded job’s predecessors are siblings instead of parent/child.
- Parameters:
tmp_path (pathlib.Path)
- Return type:
None
- testDeadlockDetection()¶
Randomly generate job graphs with various types of cycle in them and check they cause an exception properly. Also check that multiple roots causes a deadlock exception.
- Return type:
None
- testNewCheckpointIsLeafVertexNonRootCase(tmp_path, subtests)¶
Test for issue #1465: Detection of checkpoint jobs that are not leaf vertices identifies leaf vertices incorrectly
Test verification of new checkpoint jobs being leaf vertices, starting with the following baseline workflow:
Parent | Child # Checkpoint=True
- Parameters:
tmp_path (pathlib.Path)
subtests (pytest_subtests.SubTests)
- Return type:
None
- testNewCheckpointIsLeafVertexRootCase(tmp_path, subtests)¶
- Test for issue #1466: Detection of checkpoint jobs that are not leaf vertices
omits the workflow root job
Test verification of a new checkpoint job being leaf vertex, starting with a baseline workflow of a single, root job:
Root # Checkpoint=True
- Parameters:
tmp_path (pathlib.Path)
subtests (pytest_subtests.SubTests)
- Return type:
None
- runNewCheckpointIsLeafVertexTest(tmp_path, subtests, createWorkflowFn)¶
Test verification that a checkpoint job is a leaf vertex using both valid and invalid cases.
- Parameters:
createWorkflowFn (collections.abc.Callable[[], tuple[toil.job.Job, toil.job.Job]]) –
function to create and new workflow and return a tuple of:
the workflow root job
a checkpoint job to test within the workflow
tmp_path (pathlib.Path)
subtests (pytest_subtests.SubTests)
- Return type:
None
- runCheckpointVertexTest(workflowRootJob, checkpointJob, tmp_path, checkpointJobService=None, checkpointJobChild=None, checkpointJobFollowOn=None, expectedException=None)¶
Modifies the checkpoint job according to the given parameters then runs the workflow, checking for the expected exception, if any.
- Parameters:
workflowRootJob (toil.job.Job)
checkpointJob (toil.job.Job)
tmp_path (pathlib.Path)
checkpointJobService (Optional[toil.job.Job.Service])
checkpointJobChild (Optional[toil.job.Job])
checkpointJobFollowOn (Optional[toil.job.Job])
- Return type:
None
- testEvaluatingRandomDAG(tmp_path)¶
Randomly generate test input then check that the job graph can be run successfully, using the existence of promises to validate the run.
- Parameters:
tmp_path (pathlib.Path)
- Return type:
None
- static makeRandomDAG(nodeNumber)¶
Makes a random dag with “nodeNumber” nodes in which all nodes are connected. Return value is list of edges, each of form (a, b), where a and b are integers >= 0 < nodeNumber referring to nodes and the edge is from a to b.
- static getAdjacencyList(nodeNumber, edges)¶
Make adjacency list representation of edges
- reachable(node, adjacencyList)¶
Find the set of nodes reachable from this node (including the node). Return is a set of integers.
- addRandomFollowOnEdges(childAdjacencyList)¶
Adds random follow on edges to the graph, represented as an adjacency list. The follow on edges are returned as a set and their augmented edges are added to the adjacency list.
- makeJobGraph(nodeNumber, childEdges, followOnEdges, outPath, addServices=True)¶
Converts a DAG into a job graph. childEdges and followOnEdges are the lists of child and followOn edges.
- toil.test.src.jobTest.simpleJobFn(job, value)¶
- Parameters:
job (toil.job.ServiceHostJob)
value (str)
- Return type:
None
- toil.test.src.jobTest.fn1Test(string, outputFile)¶
Function appends the next character after the last character in the given string to the string, writes the string to a file, and returns it. For example, if string is “AB”, we will write and return “ABC”.
- toil.test.src.jobTest.fn2Test(pStrings, s, outputFile)¶
Function concatenates the strings in pStrings and s, in that order, and writes the result to the output file. Returns s.
- Parameters:
s (str)
outputFile (pathlib.Path)
- Return type:
- toil.test.src.jobTest.trivialParent(job)¶
- Parameters:
job (toil.job.Job)
- Return type:
None
- toil.test.src.jobTest.parent(job)¶
- Parameters:
job (toil.job.Job)
- Return type:
None
- toil.test.src.jobTest.diamond(job)¶
- Parameters:
job (toil.job.Job)
- Return type:
None
- toil.test.src.jobTest.child(job)¶
- Parameters:
job (toil.job.Job)
- Return type:
None
- toil.test.src.jobTest.errorChild(job)¶
- Parameters:
job (toil.job.Job)
- Return type:
NoReturn
- class toil.test.src.jobTest.TrivialService(message, *args, **kwargs)¶
Bases:
toil.job.Job.Service
Abstract class used to define the interface to a service.
Should be subclassed by the user to define services.
Is not executed as a job; runs within a ServiceHostJob.
- Parameters:
message (str)
args (Any)
kwargs (Any)
- message¶
- start(job)¶
Start the service.
- Parameters:
job (toil.job.ServiceHostJob) – The underlying host job that the service is being run in. Can be used to register deferred functions, or to access the fileStore for creating temporary files.
- Returns:
An object describing how to access the service. The object must be pickleable and will be used by jobs to access the service (see
toil.job.Job.addService()
).- Return type:
- stop(job)¶
Stops the service. Function can block until complete.
- Parameters:
job (toil.job.ServiceHostJob) – The underlying host job that the service is being run in. Can be used to register deferred functions, or to access the fileStore for creating temporary files.
- Return type:
None
- check()¶
Checks the service is still running.
- Raises:
exceptions.RuntimeError – If the service failed, this will cause the service job to be labeled failed.
- Returns:
True if the service is still running, else False. If False then the service job will be terminated, and considered a success. Important point: if the service job exits due to a failure, it should raise a RuntimeError, not return False!
- Return type: