toil.test.cwl.cwlTest

Module Contents

Classes

CWLWorkflowTest

CWL tests included in Toil that don't involve the whole CWL conformance

CWLv10Test

Run the CWL 1.0 conformance tests in various environments.

CWLv11Test

Run the CWL 1.1 conformance tests in various environments.

CWLv12Test

Run the CWL 1.2 conformance tests in various environments.

CWLOnARMTest

Run the CWL 1.2 conformance tests on ARM specifically.

Functions

run_conformance_tests(workDir, yml[, runner, caching, ...])

Run the CWL conformance tests.

test_workflow_echo_string_scatter_stderr_log_dir(tmp_path)

test_log_dir_echo_no_output(tmp_path)

test_log_dir_echo_stderr(tmp_path)

test_filename_conflict_resolution(tmp_path)

test_filename_conflict_detection(tmp_path)

Make sure we don't just stage files over each other when using a container.

test_filename_conflict_detection_at_root(tmp_path)

Make sure we don't just stage files over each other.

test_pick_value_with_one_null_value(caplog)

Make sure toil-cwl-runner does not false log a warning when pickValue is

test_workflow_echo_string()

test_workflow_echo_string_scatter_capture_stdout()

test_visit_top_cwl_class()

test_visit_cwl_class_and_reduce()

test_download_structure(tmp_path)

Make sure that download_structure makes the right calls to what it thinks is the file store.

Attributes

pkg_root

log

CONFORMANCE_TEST_TIMEOUT

TesterFuncType

toil.test.cwl.cwlTest.pkg_root
toil.test.cwl.cwlTest.log[source]
toil.test.cwl.cwlTest.CONFORMANCE_TEST_TIMEOUT = 10000
toil.test.cwl.cwlTest.run_conformance_tests(workDir, yml, runner=None, caching=False, batchSystem=None, selected_tests=None, selected_tags=None, skipped_tests=None, extra_args=None, must_support_all_features=False, junit_file=None)[source]

Run the CWL conformance tests.

Parameters:

  • workDir (str) – Directory to run tests in.

  • yml (str) – CWL test list YML to run tests from.

  • runner (Optional[str]) – If set, use this cwl runner instead of the default toil-cwl-runner.

  • caching (bool) – If True, use Toil file store caching.

  • batchSystem (Optional[str]) – If set, use this batch system instead of the default single_machine.

  • selected_tests (Optional[str]) – If set, use this description of test numbers to run (comma-separated numbers or ranges)

  • selected_tags (Optional[str]) – As an alternative to selected_tests, run tests with the given tags.

  • skipped_tests (Optional[str]) – Comma-separated string labels of tests to skip.

  • extra_args (Optional[List[str]]) – Provide these extra arguments to runner for each test.

  • must_support_all_features (bool) – If set, fail if some CWL optional features are unsupported.

  • junit_file (Optional[str]) – JUnit XML file to write test info to.

Return type:

None

toil.test.cwl.cwlTest.TesterFuncType
class toil.test.cwl.cwlTest.CWLWorkflowTest(methodName='runTest')[source]

Bases: toil.test.ToilTest

CWL tests included in Toil that don’t involve the whole CWL conformance test suite. Tests Toil-specific functions like URL types supported for inputs.

setUp()[source]

Runs anew before each test to create farm fresh temp dirs.

Return type:

None

tearDown()[source]

Clean up outputs.

Return type:

None

test_cwl_cmdline_input()[source]

Test that running a CWL workflow with inputs specified on the command line passes.

Return type:

None

revsort(cwl_filename, tester_fn)[source]
Parameters:

  • cwl_filename (str)

  • tester_fn (TesterFuncType)

Return type:

None

revsort_no_checksum(cwl_filename, tester_fn)[source]
Parameters:

  • cwl_filename (str)

  • tester_fn (TesterFuncType)

Return type:

None

download(inputs, tester_fn)[source]
Parameters:

  • inputs (str)

  • tester_fn (TesterFuncType)

Return type:

None

load_contents(inputs, tester_fn)[source]
Parameters:

  • inputs (str)

  • tester_fn (TesterFuncType)

Return type:

None

download_directory(inputs, tester_fn)[source]
Parameters:

  • inputs (str)

  • tester_fn (TesterFuncType)

Return type:

None

download_subdirectory(inputs, tester_fn)[source]
Parameters:

  • inputs (str)

  • tester_fn (TesterFuncType)

Return type:

None

test_mpi()[source]
Return type:

None

test_s3_as_secondary_file()[source]
Return type:

None

test_run_revsort()[source]
Return type:

None

test_run_revsort_nochecksum()[source]
Return type:

None

test_run_revsort2()[source]
Return type:

None

test_run_revsort_debug_worker()[source]
Return type:

None

test_run_colon_output()[source]
Return type:

None

test_glob_dir_bypass_file_store()[source]
Return type:

None

test_download_s3()[source]
Return type:

None

test_download_http()[source]
Return type:

None

test_download_https()[source]
Return type:

None

test_download_https_reference()[source]
Return type:

None

test_download_file()[source]
Return type:

None

test_download_directory_s3()[source]
Return type:

None

test_download_directory_s3_reference()[source]
Return type:

None

test_download_directory_file()[source]
Return type:

None

test_download_subdirectory_s3()[source]
Return type:

None

test_download_subdirectory_file()[source]
Return type:

None

test_load_contents_s3()[source]
Return type:

None

test_load_contents_http()[source]
Return type:

None

test_load_contents_https()[source]
Return type:

None

test_load_contents_file()[source]
Return type:

None

test_bioconda()[source]
Return type:

None

test_default_args()[source]
Return type:

None

test_biocontainers()[source]
Return type:

None

test_cuda()[source]
Return type:

None

test_restart()[source]

Enable restarts with toil-cwl-runner – run failing test, re-run correct test. Only implemented for single machine.

Return type:

None

test_streamable(extra_args=None)[source]

Test that a file with ‘streamable’=True is a named pipe. This is a CWL1.2 feature.

Parameters:

extra_args (Optional[List[str]])

Return type:

None

test_streamable_reference()[source]

Test that a streamable file is a stream even when passed around by URI.

Return type:

None

test_preemptible()[source]

Tests that the http://arvados.org/cwl#UsePreemptible extension is supported.

Return type:

None

test_preemptible_expression()[source]

Tests that the http://arvados.org/cwl#UsePreemptible extension is validated.

Return type:

None

class toil.test.cwl.cwlTest.CWLv10Test(methodName='runTest')[source]

Bases: toil.test.ToilTest

Run the CWL 1.0 conformance tests in various environments.

setUp()[source]

Runs anew before each test to create farm fresh temp dirs.

Return type:

None

tearDown()[source]

Clean up outputs.

Return type:

None

test_run_conformance_with_caching()[source]
Return type:

None

test_run_conformance(batchSystem=None, caching=False, selected_tests=None, skipped_tests=None, extra_args=None)[source]
Parameters:

  • batchSystem (Optional[str])

  • caching (bool)

  • selected_tests (Optional[str])

  • skipped_tests (Optional[str])

  • extra_args (Optional[List[str]])

Return type:

None

test_lsf_cwl_conformance(caching=False)[source]
Parameters:

caching (bool)

Return type:

None

test_slurm_cwl_conformance(caching=False)[source]
Parameters:

caching (bool)

Return type:

None

test_torque_cwl_conformance(caching=False)[source]
Parameters:

caching (bool)

Return type:

None

test_gridengine_cwl_conformance(caching=False)[source]
Parameters:

caching (bool)

Return type:

None

test_mesos_cwl_conformance(caching=False)[source]
Parameters:

caching (bool)

Return type:

None

test_kubernetes_cwl_conformance(caching=False)[source]
Parameters:

caching (bool)

Return type:

None

test_lsf_cwl_conformance_with_caching()[source]
Return type:

None

test_slurm_cwl_conformance_with_caching()[source]
Return type:

None

test_torque_cwl_conformance_with_caching()[source]
Return type:

None

test_gridengine_cwl_conformance_with_caching()[source]
Return type:

None

test_mesos_cwl_conformance_with_caching()[source]
Return type:

None

test_kubernetes_cwl_conformance_with_caching()[source]
Return type:

None

class toil.test.cwl.cwlTest.CWLv11Test(methodName='runTest')[source]

Bases: toil.test.ToilTest

Run the CWL 1.1 conformance tests in various environments.

rootDir: str
cwlSpec: str
test_yaml: str
classmethod setUpClass()[source]

Runs anew before each test.

Return type:

None

tearDown()[source]

Clean up outputs.

Return type:

None

test_run_conformance(caching=False, batchSystem=None, skipped_tests=None, extra_args=None)[source]
Parameters:

  • caching (bool)

  • batchSystem (Optional[str])

  • skipped_tests (Optional[str])

  • extra_args (Optional[List[str]])

Return type:

None

test_run_conformance_with_caching()[source]
Return type:

None

test_kubernetes_cwl_conformance(caching=False)[source]
Parameters:

caching (bool)

Return type:

None

test_kubernetes_cwl_conformance_with_caching()[source]
Return type:

None

class toil.test.cwl.cwlTest.CWLv12Test(methodName='runTest')[source]

Bases: toil.test.ToilTest

Run the CWL 1.2 conformance tests in various environments.

rootDir: str
cwlSpec: str
test_yaml: str
classmethod setUpClass()[source]

Runs anew before each test.

Return type:

None

tearDown()[source]

Clean up outputs.

Return type:

None

test_run_conformance(runner=None, caching=False, batchSystem=None, selected_tests=None, skipped_tests=None, extra_args=None, must_support_all_features=False, junit_file=None)[source]
Parameters:

  • runner (Optional[str])

  • caching (bool)

  • batchSystem (Optional[str])

  • selected_tests (Optional[str])

  • skipped_tests (Optional[str])

  • extra_args (Optional[List[str]])

  • must_support_all_features (bool)

  • junit_file (Optional[str])

Return type:

None

test_run_conformance_with_caching()[source]
Return type:

None

test_run_conformance_with_in_place_update()[source]

Make sure that with –bypass-file-store we properly support in place update on a single node, and that this doesn’t break any other features.

Return type:

None

test_kubernetes_cwl_conformance(caching=False, junit_file=None)[source]
Parameters:

  • caching (bool)

  • junit_file (Optional[str])

Return type:

None

test_kubernetes_cwl_conformance_with_caching()[source]
Return type:

None

test_wes_server_cwl_conformance()[source]

Run the CWL conformance tests via WES. TOIL_WES_ENDPOINT must be specified. If the WES server requires authentication, set TOIL_WES_USER and TOIL_WES_PASSWORD.

To run manually:

TOIL_WES_ENDPOINT=http://localhost:8080 TOIL_WES_USER=test TOIL_WES_PASSWORD=password python -m pytest src/toil/test/cwl/cwlTest.py::CWLv12Test::test_wes_server_cwl_conformance -vv –log-level INFO –log-cli-level INFO

Return type:

None

class toil.test.cwl.cwlTest.CWLOnARMTest(methodName)[source]

Bases: toil.test.provisioners.clusterTest.AbstractClusterTest

Run the CWL 1.2 conformance tests on ARM specifically.

Parameters:

methodName (str)

setUp()[source]

Set up for the test. Must be overridden to call this method and set self.jobStore.

Return type:

None

test_cwl_on_arm()[source]
Return type:

None

toil.test.cwl.cwlTest.test_workflow_echo_string_scatter_stderr_log_dir(tmp_path)[source]
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_log_dir_echo_no_output(tmp_path)[source]
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_log_dir_echo_stderr(tmp_path)[source]
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_resolution(tmp_path)[source]
Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_detection(tmp_path)[source]

Make sure we don’t just stage files over each other when using a container.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_filename_conflict_detection_at_root(tmp_path)[source]

Make sure we don’t just stage files over each other.

Specifically, when using a container and the files are at the root of the work dir.

Parameters:

tmp_path (pathlib.Path)

Return type:

None

toil.test.cwl.cwlTest.test_pick_value_with_one_null_value(caplog)[source]

Make sure toil-cwl-runner does not false log a warning when pickValue is used but outputSource only contains one null value. See: #3991.

Parameters:

caplog (pytest.LogCaptureFixture)

Return type:

None

toil.test.cwl.cwlTest.test_workflow_echo_string()[source]
Return type:

None

toil.test.cwl.cwlTest.test_workflow_echo_string_scatter_capture_stdout()[source]
Return type:

None

toil.test.cwl.cwlTest.test_visit_top_cwl_class()[source]
Return type:

None

toil.test.cwl.cwlTest.test_visit_cwl_class_and_reduce()[source]
Return type:

None

toil.test.cwl.cwlTest.test_download_structure(tmp_path)[source]

Make sure that download_structure makes the right calls to what it thinks is the file store.

Parameters:

tmp_path (pathlib.Path)

Return type:

None