toil.test.cwl.cwlTest

Module Contents

Classes

CWLWorkflowTest	CWL tests included in Toil that don't involve the whole CWL conformance
CWLv10Test	Run the CWL 1.0 conformance tests in various environments.
CWLv11Test	Run the CWL 1.1 conformance tests in various environments.
CWLv12Test	Run the CWL 1.2 conformance tests in various environments.
CWLOnARMTest	Run the CWL 1.2 conformance tests on ARM specifically.

Functions

run_conformance_tests(workDir, yml[, runner, caching, ...])	Run the CWL conformance tests.
test_workflow_echo_string_scatter_stderr_log_dir(tmp_path)
test_log_dir_echo_no_output(tmp_path)
test_log_dir_echo_stderr(tmp_path)
test_filename_conflict_resolution(tmp_path)
test_filename_conflict_detection(tmp_path)	Make sure we don't just stage files over each other when using a container.
test_filename_conflict_detection_at_root(tmp_path)	Make sure we don't just stage files over each other.
test_pick_value_with_one_null_value(caplog)	Make sure toil-cwl-runner does not false log a warning when pickValue is
test_workflow_echo_string()
test_workflow_echo_string_scatter_capture_stdout()
test_visit_top_cwl_class()
test_visit_cwl_class_and_reduce()
test_download_structure(tmp_path)	Make sure that download_structure makes the right calls to what it thinks is the file store.

Attributes

pkg_root
log
CONFORMANCE_TEST_TIMEOUT

toil.test.cwl.cwlTest.pkg_root

toil.test.cwl.cwlTest.log

toil.test.cwl.cwlTest.CONFORMANCE_TEST_TIMEOUT = 10000

toil.test.cwl.cwlTest.run_conformance_tests(workDir, yml, runner=None, caching=False, batchSystem=None, selected_tests=None, selected_tags=None, skipped_tests=None, extra_args=None, must_support_all_features=False, junit_file=None)

Run the CWL conformance tests.

Parameters:

• workDir (str) – Directory to run tests in.

• yml (str) – CWL test list YML to run tests from.

• runner (Optional[str]) – If set, use this cwl runner instead of the default toil-cwl-runner.

• caching (bool) – If True, use Toil file store caching.

• batchSystem (str) – If set, use this batch system instead of the default single_machine.

• selected_tests (str) – If set, use this description of test numbers to run (comma-separated numbers or ranges)

• selected_tags (str) – As an alternative to selected_tests, run tests with the given tags.

• skipped_tests (str) – Comma-separated string labels of tests to skip.

• extra_args (Optional[List[str]]) – Provide these extra arguments to runner for each test.

• must_support_all_features (bool) – If set, fail if some CWL optional features are unsupported.

• junit_file (Optional[str]) – JUnit XML file to write test info to.

class toil.test.cwl.cwlTest.CWLWorkflowTest(methodName='runTest')

Bases: toil.test.ToilTest

CWL tests included in Toil that don’t involve the whole CWL conformance test suite. Tests Toil-specific functions like URL types supported for inputs.

setUp()

Runs anew before each test to create farm fresh temp dirs.

tearDown()

Clean up outputs.

test_cwl_cmdline_input()

Test that running a CWL workflow with inputs specified on the command line passes.

revsort(cwl_filename, tester_fn)

revsort_no_checksum(cwl_filename, tester_fn)

download(inputs, tester_fn)

load_contents(inputs, tester_fn)

download_directory(inputs, tester_fn)

download_subdirectory(inputs, tester_fn)

test_mpi()

test_s3_as_secondary_file()

test_run_revsort()

test_run_revsort_nochecksum()

test_run_revsort2()

test_run_revsort_debug_worker()

test_run_colon_output()

test_glob_dir_bypass_file_store()

test_download_s3()

test_download_http()

test_download_https()

test_download_https_reference()

test_download_file()

test_download_directory_s3()

test_download_directory_s3_reference()

test_download_directory_file()

test_download_subdirectory_s3()

test_download_subdirectory_file()

test_load_contents_s3()

test_load_contents_http()

test_load_contents_https()

test_load_contents_file()

test_bioconda()

test_biocontainers()

test_cuda()

test_restart()

Enable restarts with toil-cwl-runner – run failing test, re-run correct test. Only implemented for single machine.

test_streamable(extra_args=None)

Test that a file with ‘streamable’=True is a named pipe. This is a CWL1.2 feature.

Parameters:

extra_args (List[str]) –

test_streamable_reference()

Test that a streamable file is a stream even when passed around by URI.

test_preemptible()

Tests that the http://arvados.org/cwl#UsePreemptible extension is supported.

test_preemptible_expression()

Tests that the http://arvados.org/cwl#UsePreemptible extension is validated.

class toil.test.cwl.cwlTest.CWLv10Test(methodName='runTest')

Bases: toil.test.ToilTest

Run the CWL 1.0 conformance tests in various environments.

setUp()

Runs anew before each test to create farm fresh temp dirs.

tearDown()

Clean up outputs.

test_run_conformance_with_caching()

test_run_conformance(batchSystem=None, caching=False, selected_tests=None)

test_lsf_cwl_conformance(**kwargs)

test_slurm_cwl_conformance(**kwargs)

test_torque_cwl_conformance(**kwargs)

test_gridengine_cwl_conformance(**kwargs)

test_mesos_cwl_conformance(**kwargs)

test_kubernetes_cwl_conformance(**kwargs)

test_lsf_cwl_conformance_with_caching()

test_slurm_cwl_conformance_with_caching()

test_torque_cwl_conformance_with_caching()

test_gridengine_cwl_conformance_with_caching()

test_mesos_cwl_conformance_with_caching()

test_kubernetes_cwl_conformance_with_caching()

class toil.test.cwl.cwlTest.CWLv11Test(methodName='runTest')

Bases: toil.test.ToilTest

Run the CWL 1.1 conformance tests in various environments.

classmethod setUpClass()

Runs anew before each test.

tearDown()

Clean up outputs.

test_run_conformance(**kwargs)

test_run_conformance_with_caching()

test_kubernetes_cwl_conformance(**kwargs)

test_kubernetes_cwl_conformance_with_caching()

class toil.test.cwl.cwlTest.CWLv12Test(methodName='runTest')

Bases: toil.test.ToilTest

Run the CWL 1.2 conformance tests in various environments.

classmethod setUpClass()

Runs anew before each test.

tearDown()

Clean up outputs.

test_run_conformance(**kwargs)

test_run_conformance_with_caching()

test_run_conformance_with_in_place_update()

Make sure that with –bypass-file-store we properly support in place update on a single node, and that this doesn’t break any other features.

test_kubernetes_cwl_conformance(**kwargs)

test_kubernetes_cwl_conformance_with_caching()

test_wes_server_cwl_conformance()

Run the CWL conformance tests via WES. TOIL_WES_ENDPOINT must be specified. If the WES server requires authentication, set TOIL_WES_USER and TOIL_WES_PASSWORD.

To run manually:

TOIL_WES_ENDPOINT=http://localhost:8080 TOIL_WES_USER=test TOIL_WES_PASSWORD=password python -m pytest src/toil/test/cwl/cwlTest.py::CWLv12Test::test_wes_server_cwl_conformance -vv –log-level INFO –log-cli-level INFO

class toil.test.cwl.cwlTest.CWLOnARMTest(methodName)

Bases: toil.test.provisioners.clusterTest.AbstractClusterTest

Run the CWL 1.2 conformance tests on ARM specifically.

setUp()

Set up for the test. Must be overridden to call this method and set self.jobStore.

test_cwl_on_arm()

toil.test.cwl.cwlTest.test_workflow_echo_string_scatter_stderr_log_dir(tmp_path)

Parameters:

tmp_path (pathlib.Path) –

toil.test.cwl.cwlTest.test_log_dir_echo_no_output(tmp_path)

Parameters:

tmp_path (pathlib.Path) –

Return type:

None

toil.test.cwl.cwlTest.test_log_dir_echo_stderr(tmp_path)

Parameters:

tmp_path (pathlib.Path) –

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_resolution(tmp_path)

Parameters:

tmp_path (pathlib.Path) –

toil.test.cwl.cwlTest.test_filename_conflict_detection(tmp_path)

Make sure we don’t just stage files over each other when using a container.

Parameters:

tmp_path (pathlib.Path) –

toil.test.cwl.cwlTest.test_filename_conflict_detection_at_root(tmp_path)

Make sure we don’t just stage files over each other.

Specifically, when using a container and the files are at the root of the work dir.

Parameters:

tmp_path (pathlib.Path) –

toil.test.cwl.cwlTest.test_pick_value_with_one_null_value(caplog)

Make sure toil-cwl-runner does not false log a warning when pickValue is used but outputSource only contains one null value. See: #3991.

toil.test.cwl.cwlTest.test_workflow_echo_string()

toil.test.cwl.cwlTest.test_workflow_echo_string_scatter_capture_stdout()

toil.test.cwl.cwlTest.test_visit_top_cwl_class()

toil.test.cwl.cwlTest.test_visit_cwl_class_and_reduce()

toil.test.cwl.cwlTest.test_download_structure(tmp_path)

Make sure that download_structure makes the right calls to what it thinks is the file store.

Return type:

None