toil.test.cwl.cwlTest

Attributes

pkg_root

log

CONFORMANCE_TEST_TIMEOUT

TesterFuncType

Classes

TestCWLWorkflow

CWL tests included in Toil that don't involve the whole CWL conformance

TestCWLv10Conformance

Run the CWL 1.0 conformance tests in various environments.

TestCWLv11Conformance

Run the CWL 1.1 conformance tests in various environments.

TestCWLv12Conformance

Run the CWL 1.2 conformance tests in various environments.

ImportWorkersMessageHandler

Detect whether any WorkerImportJob jobs ran during a workflow.

Functions

run_conformance_tests(workDir, yml[, runner, caching, ...])

Run the CWL conformance tests.

cwl_v1_0_spec(tmp_path)

cwl_v1_1_spec(tmp_path)

cwl_v1_2_spec(tmp_path)

test_workflow_echo_string_scatter_stderr_log_dir(tmp_path)

test_log_dir_echo_no_output(tmp_path)

test_log_dir_echo_stderr(tmp_path)

test_filename_conflict_resolution(tmp_path)

test_filename_conflict_resolution_3_or_more(tmp_path)

test_filename_conflict_detection(tmp_path)

Make sure we don't just stage files over each other when using a container.

test_filename_conflict_detection_at_root(tmp_path)

Make sure we don't just stage files over each other.

test_pick_value_with_one_null_value(caplog, tmp_path)

Make sure toil-cwl-runner does not false log a warning when pickValue is

test_workflow_echo_string(tmp_path)

test_workflow_echo_string_scatter_capture_stdout(tmp_path)

test_visit_top_cwl_class()

test_visit_cwl_class_and_reduce()

test_trim_mounts_op_nonredundant()

Make sure we don't remove all non-duplicate listings

test_trim_mounts_op_redundant()

Make sure we remove all duplicate listings

test_trim_mounts_op_partially_redundant()

Make sure we remove only the redundant listings in the CWL object and leave nonredundant listings intact

test_trim_mounts_op_mixed_urls_and_paths()

Ensure we remove redundant listings in certain edge cases

test_trim_mounts_op_decodable_paths()

test_trim_mounts_op_multiple_encodings()

test_download_structure(tmp_path)

Make sure that download_structure makes the right calls to what it thinks is the file store.

test_import_on_workers()

test_missing_tmpdir_and_tmp_outdir(tmp_path)

tmpdir_prefix and tmp_outdir_prefix do not need to exist prior to running the workflow

Module Contents

toil.test.cwl.cwlTest.pkg_root = b'.'
toil.test.cwl.cwlTest.log
toil.test.cwl.cwlTest.CONFORMANCE_TEST_TIMEOUT = 10000
toil.test.cwl.cwlTest.run_conformance_tests(workDir, yml, runner=None, caching=False, batchSystem=None, selected_tests=None, selected_tags=None, skipped_tests=None, extra_args=None, must_support_all_features=False, junit_file=None)

Run the CWL conformance tests.

Parameters:

  • workDir (str) – Directory to run tests in.

  • yml (str) – CWL test list YML to run tests from.

  • runner (str | None) – If set, use this cwl runner instead of the default toil-cwl-runner.

  • caching (bool) – If True, use Toil file store caching.

  • batchSystem (str | None) – If set, use this batch system instead of the default single_machine.

  • selected_tests (str | None) – If set, use this description of test numbers to run (comma-separated numbers or ranges)

  • selected_tags (str | None) – As an alternative to selected_tests, run tests with the given tags.

  • skipped_tests (str | None) – Comma-separated string labels of tests to skip.

  • extra_args (list[str] | None) – Provide these extra arguments to runner for each test.

  • must_support_all_features (bool) – If set, fail if some CWL optional features are unsupported.

  • junit_file (str | None) – JUnit XML file to write test info to.

Return type:

None

toil.test.cwl.cwlTest.TesterFuncType
class toil.test.cwl.cwlTest.TestCWLWorkflow

CWL tests included in Toil that don’t involve the whole CWL conformance test suite. Tests Toil-specific functions like URL types supported for inputs.

test_cwl_cmdline_input()

Test that running a CWL workflow with inputs specified on the command line passes.

Return type:

None

revsort(cwl_filename, tester_fn, out_dir)
Parameters:
Return type:

None

revsort_no_checksum(cwl_filename, tester_fn, out_dir)
Parameters:
Return type:

None

download(inputs, tester_fn, out_dir)

Run a generic download test with a tester function and check the result.

Ther test is the download.cwl workflow.

The result has to match _expected_download_output on the output directory, so it must contain an empty “output.txt” file.

Parameters:

  • inputs (str) – Relative path to the inputs file within the Toil source tree’s src/toil/test/cwl directory.

  • tester_fn (TesterFuncType) – The tester function to use to run the workflow and check the result.

  • out_dir (pathlib.Path) – Path to the output directory to save the workflow output in.

Return type:

None

load_contents(inputs, tester_fn, out_dir)
Parameters:
Return type:

None

download_directory(inputs, tester_fn, out_dir)
Parameters:
Return type:

None

download_subdirectory(inputs, tester_fn, out_dir)
Parameters:
Return type:

None

test_mpi(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_s3_as_secondary_file(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_revsort(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_revsort_nochecksum(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_revsort_no_container(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_revsort2(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_revsort_debug_worker(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_colon_output(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_dockstore_trs(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_glob_dir_bypass_file_store(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_required_input_condition_protection(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_slurm_node_memory(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_s3(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_http(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_https(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_https_reference(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_file(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_file_worker_import(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_file_uri(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_file_uri_worker_import(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_file_uri_no_hostname(tmp_path)

Test if CWL handles file: URIs without even empty hostnames.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_file_uri_no_hostname_worker_import(tmp_path)

Test if CWL handles file: URIs without even empty hostnames, with worker import.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_directory_s3(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_directory_s3_reference(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_directory_file(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_subdirectory_s3(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_subdirectory_file(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_load_contents_s3(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_load_contents_http(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_load_contents_https(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_load_contents_file(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_bioconda(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_default_args(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_biocontainers(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_cuda(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_restart(tmp_path)

Enable restarts with toil-cwl-runner – run failing test, re-run correct test. Only implemented for single machine.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_caching(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_streamable(tmp_path, extra_args=None)

Test that a file with ‘streamable’=True is a named pipe. This is a CWL1.2 feature.

Parameters:
Return type:

None

test_streamable_reference(tmp_path)

Test that a streamable file is a stream even when passed around by URI.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_preemptible(tmp_path)

Tests that the http://arvados.org/cwl#UsePreemptible extension is supported.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_preemptible_expression(tmp_path)

Tests that the http://arvados.org/cwl#UsePreemptible extension is validated.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_missing_import(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_optional_secondary_files_exists(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_optional_secondary_files_missing(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.cwl_v1_0_spec(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

collections.abc.Generator[pathlib.Path]

class toil.test.cwl.cwlTest.TestCWLv10Conformance

Run the CWL 1.0 conformance tests in various environments.

test_run_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

test_run_conformance(cwl_v1_0_spec, batchSystem=None, caching=False, selected_tests=None, skipped_tests=None, extra_args=None)
Parameters:
Return type:

None

test_lsf_cwl_conformance(cwl_v1_0_spec, caching=False)
Parameters:
Return type:

None

test_slurm_cwl_conformance(cwl_v1_0_spec, caching=False)
Parameters:
Return type:

None

test_torque_cwl_conformance(cwl_v1_0_spec, caching=False)
Parameters:
Return type:

None

test_gridengine_cwl_conformance(cwl_v1_0_spec, caching=False)
Parameters:
Return type:

None

test_mesos_cwl_conformance(cwl_v1_0_spec, caching=False)
Parameters:
Return type:

None

test_kubernetes_cwl_conformance(cwl_v1_0_spec, caching=False)
Parameters:
Return type:

None

test_lsf_cwl_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

test_slurm_cwl_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

test_torque_cwl_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

test_gridengine_cwl_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

test_mesos_cwl_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

test_kubernetes_cwl_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.cwl_v1_1_spec(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

collections.abc.Generator[pathlib.Path]

class toil.test.cwl.cwlTest.TestCWLv11Conformance

Run the CWL 1.1 conformance tests in various environments.

test_run_conformance(cwl_v1_1_spec, caching=False, batchSystem=None, skipped_tests=None, extra_args=None)
Parameters:
Return type:

None

test_run_conformance_with_caching(cwl_v1_1_spec)
Parameters:

cwl_v1_1_spec (pathlib.Path)

Return type:

None

test_kubernetes_cwl_conformance(cwl_v1_1_spec, caching=False)
Parameters:
Return type:

None

test_kubernetes_cwl_conformance_with_caching(cwl_v1_1_spec)
Parameters:

cwl_v1_1_spec (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.cwl_v1_2_spec(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

collections.abc.Generator[pathlib.Path]

class toil.test.cwl.cwlTest.TestCWLv12Conformance

Run the CWL 1.2 conformance tests in various environments.

test_run_conformance(cwl_v1_2_spec, runner=None, caching=False, batchSystem=None, selected_tests=None, skipped_tests=None, extra_args=None, must_support_all_features=False, junit_file=None)
Parameters:

  • cwl_v1_2_spec (pathlib.Path)

  • runner (str | None)

  • caching (bool)

  • batchSystem (str | None)

  • selected_tests (str | None)

  • skipped_tests (str | None)

  • extra_args (list[str] | None)

  • must_support_all_features (bool)

  • junit_file (str | None)

Return type:

None

test_run_conformance_with_caching(cwl_v1_2_spec)
Parameters:

cwl_v1_2_spec (pathlib.Path)

Return type:

None

test_run_conformance_with_task_caching(cwl_v1_2_spec, tmp_path)
Parameters:
Return type:

None

test_run_conformance_with_in_place_update(cwl_v1_2_spec)

Make sure that with –bypass-file-store we properly support in place update on a single node, and that this doesn’t break any other features.

Parameters:

cwl_v1_2_spec (pathlib.Path)

Return type:

None

test_kubernetes_cwl_conformance(cwl_v1_2_spec, caching=False, junit_file=None)
Parameters:
Return type:

None

test_kubernetes_cwl_conformance_with_caching(cwl_v1_2_spec)
Parameters:

cwl_v1_2_spec (pathlib.Path)

Return type:

None

test_wes_server_cwl_conformance(cwl_v1_2_spec)

Run the CWL conformance tests via WES. TOIL_WES_ENDPOINT must be specified. If the WES server requires authentication, set TOIL_WES_USER and TOIL_WES_PASSWORD.

To run manually:

TOIL_WES_ENDPOINT=http://localhost:8080 TOIL_WES_USER=test TOIL_WES_PASSWORD=password python -m pytest src/toil/test/cwl/cwlTest.py::TestCWLv12Conformance::test_wes_server_cwl_conformance -vv –log-level INFO –log-cli-level INFO

Parameters:

cwl_v1_2_spec (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_workflow_echo_string_scatter_stderr_log_dir(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_log_dir_echo_no_output(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_log_dir_echo_stderr(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_resolution(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_resolution_3_or_more(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_detection(tmp_path)

Make sure we don’t just stage files over each other when using a container.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_detection_at_root(tmp_path)

Make sure we don’t just stage files over each other.

Specifically, when using a container and the files are at the root of the work dir.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_pick_value_with_one_null_value(caplog, tmp_path)

Make sure toil-cwl-runner does not false log a warning when pickValue is used but outputSource only contains one null value. See: #3991.

Parameters:
Return type:

None

toil.test.cwl.cwlTest.test_workflow_echo_string(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_workflow_echo_string_scatter_capture_stdout(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_visit_top_cwl_class()
Return type:

None

toil.test.cwl.cwlTest.test_visit_cwl_class_and_reduce()
Return type:

None

toil.test.cwl.cwlTest.test_trim_mounts_op_nonredundant()

Make sure we don’t remove all non-duplicate listings

Return type:

None

toil.test.cwl.cwlTest.test_trim_mounts_op_redundant()

Make sure we remove all duplicate listings

Return type:

None

toil.test.cwl.cwlTest.test_trim_mounts_op_partially_redundant()

Make sure we remove only the redundant listings in the CWL object and leave nonredundant listings intact

Return type:

None

toil.test.cwl.cwlTest.test_trim_mounts_op_mixed_urls_and_paths()

Ensure we remove redundant listings in certain edge cases

Return type:

None

toil.test.cwl.cwlTest.test_trim_mounts_op_decodable_paths()
Return type:

None

toil.test.cwl.cwlTest.test_trim_mounts_op_multiple_encodings()
Return type:

None

toil.test.cwl.cwlTest.test_download_structure(tmp_path)

Make sure that download_structure makes the right calls to what it thinks is the file store.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_import_on_workers()
Return type:

None

toil.test.cwl.cwlTest.test_missing_tmpdir_and_tmp_outdir(tmp_path)

tmpdir_prefix and tmp_outdir_prefix do not need to exist prior to running the workflow

Parameters:

tmp_path (pathlib.Path)

Return type:

None

class toil.test.cwl.cwlTest.ImportWorkersMessageHandler

Bases: _stream_handler

Detect whether any WorkerImportJob jobs ran during a workflow.

detected = False
emit(record)

Emit a record.

If a formatter is specified, it is used to format the record. The record is then written to the stream with a trailing newline. If exception information is present, it is formatted using traceback.print_exception and appended to the stream. If the stream has an ‘encoding’ attribute, it is used to determine how to do the output to the stream.

Parameters:

record (logging.LogRecord)

Return type:

None