toil.test.cwl.cwlTest

Attributes

pkg_root

log

CONFORMANCE_TEST_TIMEOUT

TesterFuncType

Classes

TestCWLWorkflow

CWL tests included in Toil that don't involve the whole CWL conformance

TestCWLv10

Run the CWL 1.0 conformance tests in various environments.

TestCWLv11

Run the CWL 1.1 conformance tests in various environments.

TestCWLv12

Run the CWL 1.2 conformance tests in various environments.

ImportWorkersMessageHandler

Detect the import workers log message and set a flag.

Functions

run_conformance_tests(workDir, yml[, runner, caching, ...])

Run the CWL conformance tests.

cwl_v1_0_spec(tmp_path)

cwl_v1_1_spec(tmp_path)

cwl_v1_2_spec(tmp_path)

test_workflow_echo_string_scatter_stderr_log_dir(tmp_path)

test_log_dir_echo_no_output(tmp_path)

test_log_dir_echo_stderr(tmp_path)

test_filename_conflict_resolution(tmp_path)

test_filename_conflict_resolution_3_or_more(tmp_path)

test_filename_conflict_detection(tmp_path)

Make sure we don't just stage files over each other when using a container.

test_filename_conflict_detection_at_root(tmp_path)

Make sure we don't just stage files over each other.

test_pick_value_with_one_null_value(caplog, tmp_path)

Make sure toil-cwl-runner does not false log a warning when pickValue is

test_workflow_echo_string(tmp_path)

test_workflow_echo_string_scatter_capture_stdout(tmp_path)

test_visit_top_cwl_class()

test_visit_cwl_class_and_reduce()

test_download_structure(tmp_path)

Make sure that download_structure makes the right calls to what it thinks is the file store.

test_import_on_workers()

Module Contents

toil.test.cwl.cwlTest.pkg_root = b'.'
toil.test.cwl.cwlTest.log
toil.test.cwl.cwlTest.CONFORMANCE_TEST_TIMEOUT = 10000
toil.test.cwl.cwlTest.run_conformance_tests(workDir, yml, runner=None, caching=False, batchSystem=None, selected_tests=None, selected_tags=None, skipped_tests=None, extra_args=None, must_support_all_features=False, junit_file=None)

Run the CWL conformance tests.

Parameters:

  • workDir (str) – Directory to run tests in.

  • yml (str) – CWL test list YML to run tests from.

  • runner (Optional[str]) – If set, use this cwl runner instead of the default toil-cwl-runner.

  • caching (bool) – If True, use Toil file store caching.

  • batchSystem (Optional[str]) – If set, use this batch system instead of the default single_machine.

  • selected_tests (Optional[str]) – If set, use this description of test numbers to run (comma-separated numbers or ranges)

  • selected_tags (Optional[str]) – As an alternative to selected_tests, run tests with the given tags.

  • skipped_tests (Optional[str]) – Comma-separated string labels of tests to skip.

  • extra_args (Optional[list[str]]) – Provide these extra arguments to runner for each test.

  • must_support_all_features (bool) – If set, fail if some CWL optional features are unsupported.

  • junit_file (Optional[str]) – JUnit XML file to write test info to.

Return type:

None

toil.test.cwl.cwlTest.TesterFuncType
class toil.test.cwl.cwlTest.TestCWLWorkflow

CWL tests included in Toil that don’t involve the whole CWL conformance test suite. Tests Toil-specific functions like URL types supported for inputs.

test_cwl_cmdline_input()

Test that running a CWL workflow with inputs specified on the command line passes.

Return type:

None

revsort(cwl_filename, tester_fn, out_dir)
Parameters:
Return type:

None

revsort_no_checksum(cwl_filename, tester_fn, out_dir)
Parameters:
Return type:

None

download(inputs, tester_fn, out_dir)
Parameters:
Return type:

None

load_contents(inputs, tester_fn, out_dir)
Parameters:
Return type:

None

download_directory(inputs, tester_fn, out_dir)
Parameters:
Return type:

None

download_subdirectory(inputs, tester_fn, out_dir)
Parameters:
Return type:

None

test_mpi(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_s3_as_secondary_file(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_revsort(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_revsort_nochecksum(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_revsort_no_container(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_revsort2(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_revsort_debug_worker(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_colon_output(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_run_dockstore_trs(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_glob_dir_bypass_file_store(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_required_input_condition_protection(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_slurm_node_memory(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_s3(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_http(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_https(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_https_reference(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_file(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_directory_s3(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_directory_s3_reference(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_directory_file(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_subdirectory_s3(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_download_subdirectory_file(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_load_contents_s3(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_load_contents_http(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_load_contents_https(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_load_contents_file(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_bioconda(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_default_args(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_biocontainers(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_cuda(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_restart(tmp_path)

Enable restarts with toil-cwl-runner – run failing test, re-run correct test. Only implemented for single machine.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_caching(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_streamable(tmp_path, extra_args=None)

Test that a file with ‘streamable’=True is a named pipe. This is a CWL1.2 feature.

Parameters:
Return type:

None

test_streamable_reference(tmp_path)

Test that a streamable file is a stream even when passed around by URI.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_preemptible(tmp_path)

Tests that the http://arvados.org/cwl#UsePreemptible extension is supported.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_preemptible_expression(tmp_path)

Tests that the http://arvados.org/cwl#UsePreemptible extension is validated.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_missing_import(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_optional_secondary_files_exists(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

test_optional_secondary_files_missing(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.cwl_v1_0_spec(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

collections.abc.Generator[pathlib.Path]

class toil.test.cwl.cwlTest.TestCWLv10

Run the CWL 1.0 conformance tests in various environments.

test_run_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

test_run_conformance(cwl_v1_0_spec, batchSystem=None, caching=False, selected_tests=None, skipped_tests=None, extra_args=None)
Parameters:

  • cwl_v1_0_spec (pathlib.Path)

  • batchSystem (Optional[str])

  • caching (bool)

  • selected_tests (Optional[str])

  • skipped_tests (Optional[str])

  • extra_args (Optional[list[str]])

Return type:

None

test_lsf_cwl_conformance(cwl_v1_0_spec, caching=False)
Parameters:
Return type:

None

test_slurm_cwl_conformance(cwl_v1_0_spec, caching=False)
Parameters:
Return type:

None

test_torque_cwl_conformance(cwl_v1_0_spec, caching=False)
Parameters:
Return type:

None

test_gridengine_cwl_conformance(cwl_v1_0_spec, caching=False)
Parameters:
Return type:

None

test_mesos_cwl_conformance(cwl_v1_0_spec, caching=False)
Parameters:
Return type:

None

test_kubernetes_cwl_conformance(cwl_v1_0_spec, caching=False)
Parameters:
Return type:

None

test_lsf_cwl_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

test_slurm_cwl_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

test_torque_cwl_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

test_gridengine_cwl_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

test_mesos_cwl_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

test_kubernetes_cwl_conformance_with_caching(cwl_v1_0_spec)
Parameters:

cwl_v1_0_spec (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.cwl_v1_1_spec(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

collections.abc.Generator[pathlib.Path]

class toil.test.cwl.cwlTest.TestCWLv11

Run the CWL 1.1 conformance tests in various environments.

test_run_conformance(cwl_v1_1_spec, caching=False, batchSystem=None, skipped_tests=None, extra_args=None)
Parameters:
Return type:

None

test_run_conformance_with_caching(cwl_v1_1_spec)
Parameters:

cwl_v1_1_spec (pathlib.Path)

Return type:

None

test_kubernetes_cwl_conformance(cwl_v1_1_spec, caching=False)
Parameters:
Return type:

None

test_kubernetes_cwl_conformance_with_caching(cwl_v1_1_spec)
Parameters:

cwl_v1_1_spec (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.cwl_v1_2_spec(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

collections.abc.Generator[pathlib.Path]

class toil.test.cwl.cwlTest.TestCWLv12

Run the CWL 1.2 conformance tests in various environments.

test_run_conformance(cwl_v1_2_spec, runner=None, caching=False, batchSystem=None, selected_tests=None, skipped_tests=None, extra_args=None, must_support_all_features=False, junit_file=None)
Parameters:

  • cwl_v1_2_spec (pathlib.Path)

  • runner (Optional[str])

  • caching (bool)

  • batchSystem (Optional[str])

  • selected_tests (Optional[str])

  • skipped_tests (Optional[str])

  • extra_args (Optional[list[str]])

  • must_support_all_features (bool)

  • junit_file (Optional[str])

Return type:

None

test_run_conformance_with_caching(cwl_v1_2_spec)
Parameters:

cwl_v1_2_spec (pathlib.Path)

Return type:

None

test_run_conformance_with_task_caching(cwl_v1_2_spec, tmp_path)
Parameters:
Return type:

None

test_run_conformance_with_in_place_update(cwl_v1_2_spec)

Make sure that with –bypass-file-store we properly support in place update on a single node, and that this doesn’t break any other features.

Parameters:

cwl_v1_2_spec (pathlib.Path)

Return type:

None

test_kubernetes_cwl_conformance(cwl_v1_2_spec, caching=False, junit_file=None)
Parameters:
Return type:

None

test_kubernetes_cwl_conformance_with_caching(cwl_v1_2_spec)
Parameters:

cwl_v1_2_spec (pathlib.Path)

Return type:

None

test_wes_server_cwl_conformance(cwl_v1_2_spec)

Run the CWL conformance tests via WES. TOIL_WES_ENDPOINT must be specified. If the WES server requires authentication, set TOIL_WES_USER and TOIL_WES_PASSWORD.

To run manually:

TOIL_WES_ENDPOINT=http://localhost:8080 TOIL_WES_USER=test TOIL_WES_PASSWORD=password python -m pytest src/toil/test/cwl/cwlTest.py::TestCWLv12::test_wes_server_cwl_conformance -vv –log-level INFO –log-cli-level INFO

Parameters:

cwl_v1_2_spec (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_workflow_echo_string_scatter_stderr_log_dir(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_log_dir_echo_no_output(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_log_dir_echo_stderr(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_resolution(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_resolution_3_or_more(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_detection(tmp_path)

Make sure we don’t just stage files over each other when using a container.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_detection_at_root(tmp_path)

Make sure we don’t just stage files over each other.

Specifically, when using a container and the files are at the root of the work dir.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_pick_value_with_one_null_value(caplog, tmp_path)

Make sure toil-cwl-runner does not false log a warning when pickValue is used but outputSource only contains one null value. See: #3991.

Parameters:
Return type:

None

toil.test.cwl.cwlTest.test_workflow_echo_string(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_workflow_echo_string_scatter_capture_stdout(tmp_path)
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_visit_top_cwl_class()
Return type:

None

toil.test.cwl.cwlTest.test_visit_cwl_class_and_reduce()
Return type:

None

toil.test.cwl.cwlTest.test_download_structure(tmp_path)

Make sure that download_structure makes the right calls to what it thinks is the file store.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_import_on_workers()
Return type:

None

class toil.test.cwl.cwlTest.ImportWorkersMessageHandler

Bases: _stream_handler

Detect the import workers log message and set a flag.

detected = False
emit(record)

Emit a record.

If a formatter is specified, it is used to format the record. The record is then written to the stream with a trailing newline. If exception information is present, it is formatted using traceback.print_exception and appended to the stream. If the stream has an ‘encoding’ attribute, it is used to determine how to do the output to the stream.

Parameters:

record (logging.LogRecord)

Return type:

None