toil.test.utils.utilsTest

Attributes

logger

Classes

TestUtils

Tests the utilities that toil ships with, e.g. stats and status, in conjunction with restart

RunTwoJobsPerWorker

Runs child job with same resources as self in an attempt to chain the jobs on the same worker

Functions

unsortedFile(tmp_path)

correctSort(unsortedFile)

printUnicodeCharacter()

Module Contents

toil.test.utils.utilsTest.logger
toil.test.utils.utilsTest.unsortedFile(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

collections.abc.Generator[pathlib.Path]

toil.test.utils.utilsTest.correctSort(unsortedFile)
Parameters:

unsortedFile (pathlib.Path)

Return type:

list[str]

class toil.test.utils.utilsTest.TestUtils

Tests the utilities that toil ships with, e.g. stats and status, in conjunction with restart functionality.

N: int = 1000
property toilMain: str
Return type:

str

cleanCommand(jobstore)
Parameters:

jobstore (pathlib.Path)

Return type:

list[str]

statsCommand(jobstore)
Parameters:

jobstore (pathlib.Path)

Return type:

list[str]

statusCommand(jobstore, failIfNotComplete=False)
Parameters:
Return type:

list[str]

test_config_functionality(tmp_path)

Ensure that creating and reading back the config file works

Parameters:

tmp_path (pathlib.Path)

Return type:

None

testAWSProvisionerUtils()

Runs a number of the cluster utilities in sequence.

Launches a cluster with custom tags. Verifies the tags exist. ssh’s into the cluster. Does some weird string comparisons. Makes certain that TOIL_WORKDIR is set as expected in the ssh’ed cluster. Rsyncs a file and verifies it exists on the leader. Destroys the cluster.

Returns:

Return type:

None

testUtilsSort(tmp_path, unsortedFile, correctSort)

Tests the status and stats commands of the toil command line utility using the sort example with the –restart flag.

Parameters:
Return type:

None

testUtilsStatsSort(tmp_path, unsortedFile, correctSort)

Tests the stats commands on a complete run of the stats test.

Parameters:
Return type:

None

testUnicodeSupport(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

testMultipleJobsPerWorkerStats(tmp_path)

Tests case where multiple jobs are run on 1 worker to ensure that all jobs report back their data

Parameters:

tmp_path (pathlib.Path)

Return type:

None

check_status(jobstore, status, status_fn, process=None, seconds=20)
Parameters:
Return type:

None

testGetPIDStatus(tmp_path, unsortedFile)

Test that ToilStatus.getPIDStatus() behaves as expected.

Parameters:
Return type:

None

testGetStatusFailedToilWF(tmp_path, unsortedFile)

Test that ToilStatus.getStatus() behaves as expected with a failing Toil workflow. While this workflow could be called by importing and evoking its main function, doing so would remove the opportunity to test the ‘RUNNING’ functionality of getStatus().

Parameters:
Return type:

None

testGetStatusFailedCWLWF(tmp_path)

Test that ToilStatus.getStatus() behaves as expected with a failing CWL workflow.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

testGetStatusSuccessfulCWLWF(tmp_path)

Test that ToilStatus.getStatus() behaves as expected with a successful CWL workflow.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

testPrintJobLog(tmp_path, monkeypatch)

Test that ToilStatus.printJobLog() reads the log from a failed command without error.

Parameters:
Return type:

None

testRestartAttribute(tmp_path)

Test that the job store is only destroyed when we observe a successful workflow run. The following simulates a failing workflow that attempts to resume without restart(). In this case, the job store should not be destroyed until restart() is called.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.utils.utilsTest.printUnicodeCharacter()
Return type:

None

class toil.test.utils.utilsTest.RunTwoJobsPerWorker(memory=None, cores=None, disk=None, accelerators=None, preemptible=None, preemptable=None, unitName='', checkpoint=False, displayName='', descriptionClass=None, local=None, files=None)

Bases: toil.job.Job

Runs child job with same resources as self in an attempt to chain the jobs on the same worker

Parameters:

  • memory (Optional[ParseableIndivisibleResource])

  • cores (Optional[ParseableDivisibleResource])

  • disk (Optional[ParseableIndivisibleResource])

  • accelerators (Optional[ParseableAcceleratorRequirement])

  • preemptible (Optional[ParseableFlag])

  • preemptable (Optional[ParseableFlag])

  • unitName (Optional[str])

  • checkpoint (Optional[bool])

  • displayName (Optional[str])

  • descriptionClass (Optional[type])

  • local (Optional[bool])

  • files (Optional[set[toil.fileStores.FileID]])

run(fileStore)

Override this function to perform work and dynamically create successor jobs.

Parameters:

fileStore (toil.fileStores.abstractFileStore.AbstractFileStore) – Used to create local and globally sharable temporary files and to send log messages to the leader process.

Returns:

The return value of the function can be passed to other jobs by means of toil.job.Job.rv().

Return type:

None