toil.test.cwl.cwlTest

Attributes

pkg_root
DirectoryStructure
log
CONFORMANCE_TEST_TIMEOUT
TesterFuncType

Classes

FileID	A small wrapper around Python's builtin string class.
AbstractFileStore	Interface used to allow user code run by Toil to read and write files.
ToilTest	A common base class for Toil tests.
CWLWorkflowTest	CWL tests included in Toil that don't involve the whole CWL conformance
CWLv10Test	Run the CWL 1.0 conformance tests in various environments.
CWLv11Test	Run the CWL 1.1 conformance tests in various environments.
CWLv12Test	Run the CWL 1.2 conformance tests in various environments.

Functions

download_structure(file_store, index, existing, ...)	Download nested dictionary from the Toil file store to a local path.
visit_cwl_class_and_reduce(rec, classes, op_down, op_up)	Apply the given operations to all CWL objects with the given named CWL class.
visit_top_cwl_class(rec, classes, op)	Apply the given operation to all top-level CWL objects with the given named CWL class.
cpu_count()	Get the rounded-up integer number of whole CPUs available.
cluster_factory(provisioner[, clusterName, ...])	Find and instantiate the appropriate provisioner instance to make clusters in the given cloud.
needs_aws_ec2(test_item)	Use as a decorator before test classes or methods to run only if AWS EC2 is usable.
needs_aws_s3(test_item)	Use as a decorator before test classes or methods to run only if AWS S3 is usable.
needs_cwl(test_item)	Use as a decorator before test classes or methods to only run them if CWLTool is installed
needs_docker(test_item)	Use as a decorator before test classes or methods to only run them if
needs_docker_cuda(test_item)	Use as a decorator before test classes or methods to only run them if
needs_env_var(var_name[, comment])	Use as a decorator before test classes or methods to run only if the given
needs_fetchable_appliance(test_item)	Use as a decorator before test classes or methods to only run them if
needs_gridengine(test_item)	Use as a decorator before test classes or methods to run only if GridEngine is installed.
needs_kubernetes(test_item)	Use as a decorator before test classes or methods to run only if Kubernetes is installed and configured.
needs_local_cuda(test_item)	Use as a decorator before test classes or methods to only run them if
needs_lsf(test_item)	Use as a decorator before test classes or methods to only run them if LSF is installed.
needs_mesos(test_item)	Use as a decorator before test classes or methods to run only if Mesos is installed.
needs_online(test_item)	Use as a decorator before test classes or methods to run only if we are meant to talk to the Internet.
needs_slurm(test_item)	Use as a decorator before test classes or methods to run only if Slurm is installed.
needs_torque(test_item)	Use as a decorator before test classes or methods to run only if PBS/Torque is installed.
needs_wes_server(test_item)	Use as a decorator before test classes or methods to run only if a WES
slow(test_item)	Use this decorator to identify tests that are slow and not critical.
run_conformance_tests(workDir, yml[, runner, caching, ...])	Run the CWL conformance tests.
test_workflow_echo_string_scatter_stderr_log_dir(tmp_path)
test_log_dir_echo_no_output(tmp_path)
test_log_dir_echo_stderr(tmp_path)
test_filename_conflict_resolution(tmp_path)
test_filename_conflict_detection(tmp_path)	Make sure we don't just stage files over each other when using a container.
test_filename_conflict_detection_at_root(tmp_path)	Make sure we don't just stage files over each other.
test_pick_value_with_one_null_value(caplog)	Make sure toil-cwl-runner does not false log a warning when pickValue is
test_workflow_echo_string()
test_workflow_echo_string_scatter_capture_stdout()
test_visit_top_cwl_class()
test_visit_cwl_class_and_reduce()
test_download_structure(tmp_path)	Make sure that download_structure makes the right calls to what it thinks is the file store.

Module Contents

toil.test.cwl.cwlTest.pkg_root

toil.test.cwl.cwlTest.DirectoryStructure

toil.test.cwl.cwlTest.download_structure(file_store, index, existing, dir_dict, into_dir)

Download nested dictionary from the Toil file store to a local path.

Guaranteed to fill the structure with real files, and not symlinks out of it to elsewhere. File URIs may be toilfile: URIs or any other URI that Toil’s job store system can read.

Parameters:

• file_store (toil.fileStores.abstractFileStore.AbstractFileStore) – The Toil file store to download from.

• index (Dict[str, str]) – Maps from downloaded file path back to input URI.

• existing (Dict[str, str]) – Maps from file_store_id URI to downloaded file path.

• dir_dict (DirectoryStructure) – a dict from string to string (for files) or dict (for subdirectories) describing a directory structure.

• into_dir (str) – The directory to download the top-level dict’s files into.

Return type:

None

toil.test.cwl.cwlTest.visit_cwl_class_and_reduce(rec, classes, op_down, op_up)

Apply the given operations to all CWL objects with the given named CWL class.

Applies the down operation top-down, and the up operation bottom-up, and passes the down operation’s result and a list of the up operation results for all child keys (flattening across lists and collapsing nodes of non-matching classes) to the up operation.

Returns:

The flattened list of up operation results from all calls.

Parameters:

• rec (Any)

• classes (Iterable[str])

• op_down (Callable[[Any], DownReturnType])

• op_up (Callable[[Any, DownReturnType, List[UpReturnType]], UpReturnType])

Return type:

List[UpReturnType]

toil.test.cwl.cwlTest.visit_top_cwl_class(rec, classes, op)

Apply the given operation to all top-level CWL objects with the given named CWL class.

Like cwltool’s visit_class but doesn’t look inside any object visited.

Parameters:

• rec (Any)

• classes (Iterable[str])

• op (Callable[[Any], Any])

Return type:

None

class toil.test.cwl.cwlTest.FileID(fileStoreID, size, executable=False)

Bases: str

A small wrapper around Python’s builtin string class.

It is used to represent a file’s ID in the file store, and has a size attribute that is the file’s size in bytes. This object is returned by importFile and writeGlobalFile.

Calls into the file store can use bare strings; size will be queried from the job store if unavailable in the ID.

Parameters:

• fileStoreID (str)

• size (int)

• executable (bool)

pack()

Pack the FileID into a string so it can be passed through external code.

Return type:

str

classmethod forPath(fileStoreID, filePath)

Parameters:

• fileStoreID (str)

• filePath (str)

Return type:

FileID

classmethod unpack(packedFileStoreID)

Unpack the result of pack() into a FileID object.

Parameters:

packedFileStoreID (str)

Return type:

FileID

class toil.test.cwl.cwlTest.AbstractFileStore(jobStore, jobDesc, file_store_dir, waitForPreviousCommit)

Bases: abc.ABC

Interface used to allow user code run by Toil to read and write files.

Also provides the interface to other Toil facilities used by user code, including:

• normal (non-real-time) logging

• finding the correct temporary directory for scratch work

• importing and exporting files into and out of the workflow

Stores user files in the jobStore, but keeps them separate from actual jobs.

May implement caching.

Passed as argument to the toil.job.Job.run() method.

Access to files is only permitted inside the context manager provided by toil.fileStores.abstractFileStore.AbstractFileStore.open().

Also responsible for committing completed jobs back to the job store with an update operation, and allowing that commit operation to be waited for.

Parameters:

• jobStore (toil.jobStores.abstractJobStore.AbstractJobStore)

• jobDesc (toil.job.JobDescription)

• file_store_dir (str)

• waitForPreviousCommit (Callable[[], Any])

static createFileStore(jobStore, jobDesc, file_store_dir, waitForPreviousCommit, caching)

Create a concreate FileStore.

Parameters:

• jobStore (toil.jobStores.abstractJobStore.AbstractJobStore)

• jobDesc (toil.job.JobDescription)

• file_store_dir (str)

• waitForPreviousCommit (Callable[[], Any])

• caching (Optional[bool])

Return type:

Union[toil.fileStores.nonCachingFileStore.NonCachingFileStore, toil.fileStores.cachingFileStore.CachingFileStore]

static shutdownFileStore(workflowID, config_work_dir, config_coordination_dir)

Carry out any necessary filestore-specific cleanup.

This is a destructive operation and it is important to ensure that there are no other running processes on the system that are modifying or using the file store for this workflow.

This is the intended to be the last call to the file store in a Toil run, called by the batch system cleanup function upon batch system shutdown.

Parameters:

• workflowID (str) – The workflow ID for this invocation of the workflow

• config_work_dir (Optional[str]) – The path to the work directory in the Toil Config.

• config_coordination_dir (Optional[str]) – The path to the coordination directory in the Toil Config.

Return type:

None

open(job)

Create the context manager around tasks prior and after a job has been run.

File operations are only permitted inside the context manager.

Implementations must only yield from within with super().open(job):.

Parameters:

job (toil.job.Job) – The job instance of the toil job to run.

Return type:

Generator[None, None, None]

get_disk_usage()

Get the number of bytes of disk used by the last job run under open().

Disk usage is measured at the end of the job. TODO: Sample periodically and record peak usage.

Return type:

Optional[int]

getLocalTempDir()

Get a new local temporary directory in which to write files.

The directory will only persist for the duration of the job.

Returns:

The absolute path to a new local temporary directory. This directory will exist for the duration of the job only, and is guaranteed to be deleted once the job terminates, removing all files it contains recursively.

Return type:

str

getLocalTempFile(suffix=None, prefix=None)

Get a new local temporary file that will persist for the duration of the job.

Parameters:

• suffix (Optional[str]) – If not None, the file name will end with this string. Otherwise, default value “.tmp” will be used

• prefix (Optional[str]) – If not None, the file name will start with this string. Otherwise, default value “tmp” will be used

Returns:

The absolute path to a local temporary file. This file will exist for the duration of the job only, and is guaranteed to be deleted once the job terminates.

Return type:

str

getLocalTempFileName(suffix=None, prefix=None)

Get a valid name for a new local file. Don’t actually create a file at the path.

Parameters:

• suffix (Optional[str]) – If not None, the file name will end with this string. Otherwise, default value “.tmp” will be used

• prefix (Optional[str]) – If not None, the file name will start with this string. Otherwise, default value “tmp” will be used

Returns:

Path to valid file

Return type:

str

abstract writeGlobalFile(localFileName, cleanup=False)

Upload a file (as a path) to the job store.

If the file is in a FileStore-managed temporary directory (i.e. from toil.fileStores.abstractFileStore.AbstractFileStore.getLocalTempDir()), it will become a local copy of the file, eligible for deletion by toil.fileStores.abstractFileStore.AbstractFileStore.deleteLocalFile().

If an executable file on the local filesystem is uploaded, its executability will be preserved when it is downloaded again.

Parameters:

• localFileName (str) – The path to the local file to upload. The last path component (basename of the file) will remain associated with the file in the file store, if supported by the backing JobStore, so that the file can be searched for by name or name glob.

• cleanup (bool) – if True then the copy of the global file will be deleted once the job and all its successors have completed running. If not the global file must be deleted manually.

Returns:

an ID that can be used to retrieve the file.

Return type:

toil.fileStores.FileID

writeGlobalFileStream(cleanup=False, basename=None, encoding=None, errors=None)

Similar to writeGlobalFile, but allows the writing of a stream to the job store. The yielded file handle does not need to and should not be closed explicitly.

Parameters:

• encoding (Optional[str]) – The name of the encoding used to decode the file. Encodings are the same as for decode(). Defaults to None which represents binary mode.

• errors (Optional[str]) – Specifies how encoding errors are to be handled. Errors are the same as for open(). Defaults to ‘strict’ when an encoding is specified.

• cleanup (bool) – is as in toil.fileStores.abstractFileStore.AbstractFileStore.writeGlobalFile().

• basename (Optional[str]) – If supported by the backing JobStore, use the given file basename so that when searching the job store with a query matching that basename, the file will be detected.

Returns:

A context manager yielding a tuple of 1) a file handle which can be written to and 2) the toil.fileStores.FileID of the resulting file in the job store.

Return type:

Iterator[Tuple[toil.lib.io.WriteWatchingStream, toil.fileStores.FileID]]

logAccess(fileStoreID, destination=None)

Record that the given file was read by the job.

(to be announced if the job fails)

If destination is not None, it gives the path that the file was downloaded to. Otherwise, assumes that the file was streamed.

Must be called by readGlobalFile() and readGlobalFileStream() implementations.

Parameters:

• fileStoreID (Union[toil.fileStores.FileID, str])

• destination (Union[str, None])

Return type:

None

abstract readGlobalFile(fileStoreID, userPath=None, cache=True, mutable=False, symlink=False)

Make the file associated with fileStoreID available locally.

If mutable is True, then a copy of the file will be created locally so that the original is not modified and does not change the file for other jobs. If mutable is False, then a link can be created to the file, saving disk resources. The file that is downloaded will be executable if and only if it was originally uploaded from an executable file on the local filesystem.

If a user path is specified, it is used as the destination. If a user path isn’t specified, the file is stored in the local temp directory with an encoded name.

The destination file must not be deleted by the user; it can only be deleted through deleteLocalFile.

Implementations must call logAccess() to report the download.

Parameters:

• fileStoreID (str) – job store id for the file

• userPath (Optional[str]) – a path to the name of file to which the global file will be copied or hard-linked (see below).

• cache (bool) – Described in toil.fileStores.CachingFileStore.readGlobalFile()

• mutable (bool) – Described in toil.fileStores.CachingFileStore.readGlobalFile()

• symlink (bool) – True if caller can accept symlink, False if caller can only accept a normal file or hardlink

Returns:

An absolute path to a local, temporary copy of the file keyed by fileStoreID.

Return type:

str

readGlobalFileStream(fileStoreID: str, encoding: Literal[None] = None, errors: str | None = None) → ContextManager[IO[bytes]]

readGlobalFileStream(fileStoreID: str, encoding: str, errors: str | None = None) → ContextManager[IO[str]]

Read a stream from the job store; similar to readGlobalFile.

The yielded file handle does not need to and should not be closed explicitly.

Parameters:

• encoding – the name of the encoding used to decode the file. Encodings are the same as for decode(). Defaults to None which represents binary mode.

• errors – an optional string that specifies how encoding errors are to be handled. Errors are the same as for open(). Defaults to ‘strict’ when an encoding is specified.

Implementations must call logAccess() to report the download.

Returns:

a context manager yielding a file handle which can be read from.

getGlobalFileSize(fileStoreID)

Get the size of the file pointed to by the given ID, in bytes.

If a FileID or something else with a non-None ‘size’ field, gets that.

Otherwise, asks the job store to poll the file’s size.

Note that the job store may overestimate the file’s size, for example if it is encrypted and had to be augmented with an IV or other encryption framing.

Parameters:

fileStoreID (Union[toil.fileStores.FileID, str]) – File ID for the file

Returns:

File’s size in bytes, as stored in the job store

Return type:

int

abstract deleteLocalFile(fileStoreID)

Delete local copies of files associated with the provided job store ID.

Raises an OSError with an errno of errno.ENOENT if no such local copies exist. Thus, cannot be called multiple times in succession.

The files deleted are all those previously read from this file ID via readGlobalFile by the current job into the job’s file-store-provided temp directory, plus the file that was written to create the given file ID, if it was written by the current job from the job’s file-store-provided temp directory.

Parameters:

fileStoreID (Union[toil.fileStores.FileID, str]) – File Store ID of the file to be deleted.

Return type:

None

abstract deleteGlobalFile(fileStoreID)

Delete local files and then permanently deletes them from the job store.

To ensure that the job can be restarted if necessary, the delete will not happen until after the job’s run method has completed.

Parameters:

fileStoreID (Union[toil.fileStores.FileID, str]) – the File Store ID of the file to be deleted.

Return type:

None

importFile(srcUrl, sharedFileName=None)

Parameters:

• srcUrl (str)

• sharedFileName (Optional[str])

Return type:

Optional[toil.fileStores.FileID]

import_file(src_uri, shared_file_name=None)

Parameters:

• src_uri (str)

• shared_file_name (Optional[str])

Return type:

Optional[toil.fileStores.FileID]

exportFile(jobStoreFileID, dstUrl)

Parameters:

• jobStoreFileID (toil.fileStores.FileID)

• dstUrl (str)

Return type:

None

abstract export_file(file_id, dst_uri)

Parameters:

• file_id (toil.fileStores.FileID)

• dst_uri (str)

Return type:

None

log_to_leader(text, level=logging.INFO)

Send a logging message to the leader. The message will also be logged by the worker at the same level.

Parameters:

• text (str) – The string to log.

• level (int) – The logging level.

Return type:

None

logToMaster(text, level=logging.INFO)

Parameters:

• text (str)

• level (int)

Return type:

None

log_user_stream(name, stream)

Send a stream of UTF-8 text to the leader as a named log stream.

Useful for things like the error logs of Docker containers. The leader will show it to the user or organize it appropriately for user-level log information.

Parameters:

• name (str) – A hierarchical, .-delimited string.

• stream (IO[bytes]) – A stream of encoded text. Encoding errors will be tolerated.

Return type:

None

abstract startCommit(jobState=False)

Update the status of the job on the disk.

May bump the version number of the job.

May start an asynchronous process. Call waitForCommit() to wait on that process. You must waitForCommit() before committing any further updates to the job. During the asynchronous process, it is safe to modify the job; modifications after this call will not be committed until the next call.

Parameters:

jobState (bool) – If True, commit the state of the FileStore’s job, and file deletes. Otherwise, commit only file creates/updates.

Return type:

None

abstract waitForCommit()

Blocks while startCommit is running.

This function is called by this job’s successor to ensure that it does not begin modifying the job store until after this job has finished doing so.

Might be called when startCommit is never called on a particular instance, in which case it does not block.

Returns:

Always returns True

Return type:

bool

classmethod shutdown(shutdown_info)

Abstractmethod:

Parameters:

shutdown_info (Any)

Return type:

None

Shutdown the filestore on this node.

This is intended to be called on batch system shutdown.

Parameters:

shutdown_info (Any) – The implementation-specific shutdown information, for shutting down the file store and removing all its state and all job local temp directories from the node.

Return type:

None

toil.test.cwl.cwlTest.cpu_count()

Get the rounded-up integer number of whole CPUs available.

Counts hyperthreads as CPUs.

Uses the system’s actual CPU count, or the current v1 cgroup’s quota per period, if the quota is set.

Ignores the cgroup’s cpu shares value, because it’s extremely difficult to interpret. See https://github.com/kubernetes/kubernetes/issues/81021.

Caches result for efficiency.

Returns:

Integer count of available CPUs, minimum 1.

Return type:

int

toil.test.cwl.cwlTest.cluster_factory(provisioner, clusterName=None, clusterType='mesos', zone=None, nodeStorage=50, nodeStorageOverrides=None, sseKey=None, enable_fuse=False)

Find and instantiate the appropriate provisioner instance to make clusters in the given cloud.

Raises ClusterTypeNotSupportedException if the given provisioner does not implement clusters of the given type.

Parameters:

• provisioner (str) – The cloud type of the cluster.

• clusterName (Optional[str]) – The name of the cluster.

• clusterType (str) – The type of cluster: ‘mesos’ or ‘kubernetes’.

• zone (Optional[str]) – The cloud zone

• nodeStorage (int)

• nodeStorageOverrides (Optional[List[str]])

• sseKey (Optional[str])

• enable_fuse (bool)

Returns:

A cluster object for the the cloud type.

Return type:

Union[aws.awsProvisioner.AWSProvisioner, gceProvisioner.GCEProvisioner]

class toil.test.cwl.cwlTest.ToilTest(methodName='runTest')

Bases: unittest.TestCase

A common base class for Toil tests.

Please have every test case directly or indirectly inherit this one.

When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.

setup_method(method)

Parameters:

method (Any)

Return type:

None

classmethod setUpClass()

Hook method for setting up class fixture before running tests in the class.

Return type:

None

classmethod tearDownClass()

Hook method for deconstructing the class fixture after running all tests in the class.

Return type:

None

setUp()

Hook method for setting up the test fixture before exercising it.

Return type:

None

tearDown()

Hook method for deconstructing the test fixture after testing it.

Return type:

None

classmethod awsRegion()

Pick an appropriate AWS region.

Use us-west-2 unless running on EC2, in which case use the region in which the instance is located

Return type:

str

toil.test.cwl.cwlTest.needs_aws_ec2(test_item)

Use as a decorator before test classes or methods to run only if AWS EC2 is usable.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_aws_s3(test_item)

Use as a decorator before test classes or methods to run only if AWS S3 is usable.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_cwl(test_item)

Use as a decorator before test classes or methods to only run them if CWLTool is installed and configured.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_docker(test_item)

Use as a decorator before test classes or methods to only run them if docker is installed and docker-based tests are enabled.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_docker_cuda(test_item)

Use as a decorator before test classes or methods to only run them if a CUDA setup is available through Docker.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_env_var(var_name, comment=None)

Use as a decorator before test classes or methods to run only if the given environment variable is set. Can include a comment saying what the variable should be set to.

Parameters:

• var_name (str)

• comment (Optional[str])

Return type:

Callable[[MT], MT]

toil.test.cwl.cwlTest.needs_fetchable_appliance(test_item)

Use as a decorator before test classes or methods to only run them if the Toil appliance Docker image is able to be downloaded from the Internet.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_gridengine(test_item)

Use as a decorator before test classes or methods to run only if GridEngine is installed.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_kubernetes(test_item)

Use as a decorator before test classes or methods to run only if Kubernetes is installed and configured.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_local_cuda(test_item)

Use as a decorator before test classes or methods to only run them if a CUDA setup legible to cwltool (i.e. providing userspace nvidia-smi) is present.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_lsf(test_item)

Use as a decorator before test classes or methods to only run them if LSF is installed.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_mesos(test_item)

Use as a decorator before test classes or methods to run only if Mesos is installed.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_online(test_item)

Use as a decorator before test classes or methods to run only if we are meant to talk to the Internet.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_slurm(test_item)

Use as a decorator before test classes or methods to run only if Slurm is installed.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_torque(test_item)

Use as a decorator before test classes or methods to run only if PBS/Torque is installed.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.needs_wes_server(test_item)

Use as a decorator before test classes or methods to run only if a WES server is available to run against.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.slow(test_item)

Use this decorator to identify tests that are slow and not critical. Skip if TOIL_TEST_QUICK is true.

Parameters:

test_item (MT)

Return type:

MT

toil.test.cwl.cwlTest.log

toil.test.cwl.cwlTest.CONFORMANCE_TEST_TIMEOUT = 10000

toil.test.cwl.cwlTest.run_conformance_tests(workDir, yml, runner=None, caching=False, batchSystem=None, selected_tests=None, selected_tags=None, skipped_tests=None, extra_args=None, must_support_all_features=False, junit_file=None)

Run the CWL conformance tests.

Parameters:

• workDir (str) – Directory to run tests in.

• yml (str) – CWL test list YML to run tests from.

• runner (Optional[str]) – If set, use this cwl runner instead of the default toil-cwl-runner.

• caching (bool) – If True, use Toil file store caching.

• batchSystem (Optional[str]) – If set, use this batch system instead of the default single_machine.

• selected_tests (Optional[str]) – If set, use this description of test numbers to run (comma-separated numbers or ranges)

• selected_tags (Optional[str]) – As an alternative to selected_tests, run tests with the given tags.

• skipped_tests (Optional[str]) – Comma-separated string labels of tests to skip.

• extra_args (Optional[List[str]]) – Provide these extra arguments to runner for each test.

• must_support_all_features (bool) – If set, fail if some CWL optional features are unsupported.

• junit_file (Optional[str]) – JUnit XML file to write test info to.

Return type:

None

toil.test.cwl.cwlTest.TesterFuncType

class toil.test.cwl.cwlTest.CWLWorkflowTest(methodName='runTest')

Bases: toil.test.ToilTest

CWL tests included in Toil that don’t involve the whole CWL conformance test suite. Tests Toil-specific functions like URL types supported for inputs.

setUp()

Runs anew before each test to create farm fresh temp dirs.

Return type:

None

tearDown()

Clean up outputs.

Return type:

None

test_cwl_cmdline_input()

Test that running a CWL workflow with inputs specified on the command line passes.

Return type:

None

revsort(cwl_filename, tester_fn)

Parameters:

• cwl_filename (str)

• tester_fn (TesterFuncType)

Return type:

None

revsort_no_checksum(cwl_filename, tester_fn)

Parameters:

• cwl_filename (str)

• tester_fn (TesterFuncType)

Return type:

None

download(inputs, tester_fn)

Parameters:

• inputs (str)

• tester_fn (TesterFuncType)

Return type:

None

load_contents(inputs, tester_fn)

Parameters:

• inputs (str)

• tester_fn (TesterFuncType)

Return type:

None

download_directory(inputs, tester_fn)

Parameters:

• inputs (str)

• tester_fn (TesterFuncType)

Return type:

None

download_subdirectory(inputs, tester_fn)

Parameters:

• inputs (str)

• tester_fn (TesterFuncType)

Return type:

None

test_mpi()

Return type:

None

test_s3_as_secondary_file()

Return type:

None

test_run_revsort()

Return type:

None

test_run_revsort_nochecksum()

Return type:

None

test_run_revsort2()

Return type:

None

test_run_revsort_debug_worker()

Return type:

None

test_run_colon_output()

Return type:

None

test_glob_dir_bypass_file_store()

Return type:

None

test_download_s3()

Return type:

None

test_download_http()

Return type:

None

test_download_https()

Return type:

None

test_download_https_reference()

Return type:

None

test_download_file()

Return type:

None

test_download_directory_s3()

Return type:

None

test_download_directory_s3_reference()

Return type:

None

test_download_directory_file()

Return type:

None

test_download_subdirectory_s3()

Return type:

None

test_download_subdirectory_file()

Return type:

None

test_load_contents_s3()

Return type:

None

test_load_contents_http()

Return type:

None

test_load_contents_https()

Return type:

None

test_load_contents_file()

Return type:

None

test_bioconda()

Return type:

None

test_default_args()

Return type:

None

test_biocontainers()

Return type:

None

test_cuda()

Return type:

None

test_restart()

Enable restarts with toil-cwl-runner – run failing test, re-run correct test. Only implemented for single machine.

Return type:

None

test_streamable(extra_args=None)

Test that a file with ‘streamable’=True is a named pipe. This is a CWL1.2 feature.

Parameters:

extra_args (Optional[List[str]])

Return type:

None

test_streamable_reference()

Test that a streamable file is a stream even when passed around by URI.

Return type:

None

test_preemptible()

Tests that the http://arvados.org/cwl#UsePreemptible extension is supported.

Return type:

None

test_preemptible_expression()

Tests that the http://arvados.org/cwl#UsePreemptible extension is validated.

Return type:

None

class toil.test.cwl.cwlTest.CWLv10Test(methodName='runTest')

Bases: toil.test.ToilTest

Run the CWL 1.0 conformance tests in various environments.

setUp()

Runs anew before each test to create farm fresh temp dirs.

Return type:

None

tearDown()

Clean up outputs.

Return type:

None

test_run_conformance_with_caching()

Return type:

None

test_run_conformance(batchSystem=None, caching=False, selected_tests=None, skipped_tests=None, extra_args=None)

Parameters:

• batchSystem (Optional[str])

• caching (bool)

• selected_tests (Optional[str])

• skipped_tests (Optional[str])

• extra_args (Optional[List[str]])

Return type:

None

test_lsf_cwl_conformance(caching=False)

Parameters:

caching (bool)

Return type:

None

test_slurm_cwl_conformance(caching=False)

Parameters:

caching (bool)

Return type:

None

test_torque_cwl_conformance(caching=False)

Parameters:

caching (bool)

Return type:

None

test_gridengine_cwl_conformance(caching=False)

Parameters:

caching (bool)

Return type:

None

test_mesos_cwl_conformance(caching=False)

Parameters:

caching (bool)

Return type:

None

test_kubernetes_cwl_conformance(caching=False)

Parameters:

caching (bool)

Return type:

None

test_lsf_cwl_conformance_with_caching()

Return type:

None

test_slurm_cwl_conformance_with_caching()

Return type:

None

test_torque_cwl_conformance_with_caching()

Return type:

None

test_gridengine_cwl_conformance_with_caching()

Return type:

None

test_mesos_cwl_conformance_with_caching()

Return type:

None

test_kubernetes_cwl_conformance_with_caching()

Return type:

None

class toil.test.cwl.cwlTest.CWLv11Test(methodName='runTest')

Bases: toil.test.ToilTest

Run the CWL 1.1 conformance tests in various environments.

rootDir: str

cwlSpec: str

test_yaml: str

classmethod setUpClass()

Runs anew before each test.

Return type:

None

tearDown()

Clean up outputs.

Return type:

None

test_run_conformance(caching=False, batchSystem=None, skipped_tests=None, extra_args=None)

Parameters:

• caching (bool)

• batchSystem (Optional[str])

• skipped_tests (Optional[str])

• extra_args (Optional[List[str]])

Return type:

None

test_run_conformance_with_caching()

Return type:

None

test_kubernetes_cwl_conformance(caching=False)

Parameters:

caching (bool)

Return type:

None

test_kubernetes_cwl_conformance_with_caching()

Return type:

None

class toil.test.cwl.cwlTest.CWLv12Test(methodName='runTest')

Bases: toil.test.ToilTest

Run the CWL 1.2 conformance tests in various environments.

rootDir: str

cwlSpec: str

test_yaml: str

classmethod setUpClass()

Runs anew before each test.

Return type:

None

tearDown()

Clean up outputs.

Return type:

None

test_run_conformance(runner=None, caching=False, batchSystem=None, selected_tests=None, skipped_tests=None, extra_args=None, must_support_all_features=False, junit_file=None)

Parameters:

• runner (Optional[str])

• caching (bool)

• batchSystem (Optional[str])

• selected_tests (Optional[str])

• skipped_tests (Optional[str])

• extra_args (Optional[List[str]])

• must_support_all_features (bool)

• junit_file (Optional[str])

Return type:

None

test_run_conformance_with_caching()

Return type:

None

test_run_conformance_with_in_place_update()

Make sure that with –bypass-file-store we properly support in place update on a single node, and that this doesn’t break any other features.

Return type:

None

test_kubernetes_cwl_conformance(caching=False, junit_file=None)

Parameters:

• caching (bool)

• junit_file (Optional[str])

Return type:

None

test_kubernetes_cwl_conformance_with_caching()

Return type:

None

test_wes_server_cwl_conformance()

Run the CWL conformance tests via WES. TOIL_WES_ENDPOINT must be specified. If the WES server requires authentication, set TOIL_WES_USER and TOIL_WES_PASSWORD.

To run manually:

TOIL_WES_ENDPOINT=http://localhost:8080 TOIL_WES_USER=test TOIL_WES_PASSWORD=password python -m pytest src/toil/test/cwl/cwlTest.py::CWLv12Test::test_wes_server_cwl_conformance -vv –log-level INFO –log-cli-level INFO

Return type:

None

toil.test.cwl.cwlTest.test_workflow_echo_string_scatter_stderr_log_dir(tmp_path)

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_log_dir_echo_no_output(tmp_path)

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_log_dir_echo_stderr(tmp_path)

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_resolution(tmp_path)

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_detection(tmp_path)

Make sure we don’t just stage files over each other when using a container.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_detection_at_root(tmp_path)

Make sure we don’t just stage files over each other.

Specifically, when using a container and the files are at the root of the work dir.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_pick_value_with_one_null_value(caplog)

Make sure toil-cwl-runner does not false log a warning when pickValue is used but outputSource only contains one null value. See: #3991.

Parameters:

caplog (pytest.LogCaptureFixture)

Return type:

None

toil.test.cwl.cwlTest.test_workflow_echo_string()

Return type:

None

toil.test.cwl.cwlTest.test_workflow_echo_string_scatter_capture_stdout()

Return type:

None

toil.test.cwl.cwlTest.test_visit_top_cwl_class()

Return type:

None

toil.test.cwl.cwlTest.test_visit_cwl_class_and_reduce()

Return type:

None

toil.test.cwl.cwlTest.test_download_structure(tmp_path)

Make sure that download_structure makes the right calls to what it thinks is the file store.

Parameters:

tmp_path (pathlib.Path)

Return type:

None