toil.test.cwl.cwlTest
¶
Module Contents¶
Classes¶
CWL tests included in Toil that don't involve the whole CWL conformance |
|
Run the CWL 1.0 conformance tests in various environments. |
|
Run the CWL 1.1 conformance tests in various environments. |
|
Run the CWL 1.2 conformance tests in various environments. |
|
Run the CWL 1.2 conformance tests on ARM specifically. |
Functions¶
|
Run the CWL conformance tests. |
|
|
|
|
|
|
|
Make sure we don't just stage files over each other when using a container. |
|
Make sure we don't just stage files over each other. |
Make sure toil-cwl-runner does not false log a warning when pickValue is |
|
|
Make sure that download_structure makes the right calls to what it thinks is the file store. |
Attributes¶
- toil.test.cwl.cwlTest.pkg_root¶
- toil.test.cwl.cwlTest.log¶
- toil.test.cwl.cwlTest.CONFORMANCE_TEST_TIMEOUT = 10000¶
- toil.test.cwl.cwlTest.run_conformance_tests(workDir, yml, runner=None, caching=False, batchSystem=None, selected_tests=None, selected_tags=None, skipped_tests=None, extra_args=None, must_support_all_features=False, junit_file=None)¶
Run the CWL conformance tests.
- Parameters:
workDir (str) – Directory to run tests in.
yml (str) – CWL test list YML to run tests from.
runner (Optional[str]) – If set, use this cwl runner instead of the default toil-cwl-runner.
caching (bool) – If True, use Toil file store caching.
batchSystem (str) – If set, use this batch system instead of the default single_machine.
selected_tests (str) – If set, use this description of test numbers to run (comma-separated numbers or ranges)
selected_tags (str) – As an alternative to selected_tests, run tests with the given tags.
skipped_tests (str) – Comma-separated string labels of tests to skip.
extra_args (Optional[List[str]]) – Provide these extra arguments to runner for each test.
must_support_all_features (bool) – If set, fail if some CWL optional features are unsupported.
junit_file (Optional[str]) – JUnit XML file to write test info to.
- class toil.test.cwl.cwlTest.CWLWorkflowTest(methodName='runTest')¶
Bases:
digraph inheritancedf5f0821be { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "CWLWorkflowTest" [URL="#toil.test.cwl.cwlTest.CWLWorkflowTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="CWL tests included in Toil that don't involve the whole CWL conformance"]; "ToilTest" -> "CWLWorkflowTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }toil.test.ToilTest
CWL tests included in Toil that don’t involve the whole CWL conformance test suite. Tests Toil-specific functions like URL types supported for inputs.
- setUp()¶
Runs anew before each test to create farm fresh temp dirs.
- tearDown()¶
Clean up outputs.
- test_cwl_cmdline_input()¶
Test that running a CWL workflow with inputs specified on the command line passes.
- revsort(cwl_filename, tester_fn)¶
- revsort_no_checksum(cwl_filename, tester_fn)¶
- download(inputs, tester_fn)¶
- load_contents(inputs, tester_fn)¶
- download_directory(inputs, tester_fn)¶
- download_subdirectory(inputs, tester_fn)¶
- test_mpi()¶
- test_s3_as_secondary_file()¶
- test_run_revsort()¶
- test_run_revsort_nochecksum()¶
- test_run_revsort2()¶
- test_run_revsort_debug_worker()¶
- test_run_colon_output()¶
- test_glob_dir_bypass_file_store()¶
- test_download_s3()¶
- test_download_http()¶
- test_download_https()¶
- test_download_https_reference()¶
- test_download_file()¶
- test_download_directory_s3()¶
- test_download_directory_s3_reference()¶
- test_download_directory_file()¶
- test_download_subdirectory_s3()¶
- test_download_subdirectory_file()¶
- test_load_contents_s3()¶
- test_load_contents_http()¶
- test_load_contents_https()¶
- test_load_contents_file()¶
- test_bioconda()¶
- test_biocontainers()¶
- test_cuda()¶
- test_restart()¶
Enable restarts with toil-cwl-runner – run failing test, re-run correct test. Only implemented for single machine.
- test_streamable(extra_args=None)¶
Test that a file with ‘streamable’=True is a named pipe. This is a CWL1.2 feature.
- Parameters:
extra_args (List[str]) –
- test_streamable_reference()¶
Test that a streamable file is a stream even when passed around by URI.
- test_preemptible()¶
Tests that the http://arvados.org/cwl#UsePreemptible extension is supported.
- test_preemptible_expression()¶
Tests that the http://arvados.org/cwl#UsePreemptible extension is validated.
- class toil.test.cwl.cwlTest.CWLv10Test(methodName='runTest')¶
Bases:
digraph inheritance82d1f4d8c0 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "CWLv10Test" [URL="#toil.test.cwl.cwlTest.CWLv10Test",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Run the CWL 1.0 conformance tests in various environments."]; "ToilTest" -> "CWLv10Test" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }toil.test.ToilTest
Run the CWL 1.0 conformance tests in various environments.
- setUp()¶
Runs anew before each test to create farm fresh temp dirs.
- tearDown()¶
Clean up outputs.
- test_run_conformance_with_caching()¶
- test_run_conformance(batchSystem=None, caching=False, selected_tests=None)¶
- test_lsf_cwl_conformance(**kwargs)¶
- test_slurm_cwl_conformance(**kwargs)¶
- test_torque_cwl_conformance(**kwargs)¶
- test_gridengine_cwl_conformance(**kwargs)¶
- test_mesos_cwl_conformance(**kwargs)¶
- test_kubernetes_cwl_conformance(**kwargs)¶
- test_lsf_cwl_conformance_with_caching()¶
- test_slurm_cwl_conformance_with_caching()¶
- test_torque_cwl_conformance_with_caching()¶
- test_gridengine_cwl_conformance_with_caching()¶
- test_mesos_cwl_conformance_with_caching()¶
- test_kubernetes_cwl_conformance_with_caching()¶
- class toil.test.cwl.cwlTest.CWLv11Test(methodName='runTest')¶
Bases:
digraph inheritance383d8d360f { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "CWLv11Test" [URL="#toil.test.cwl.cwlTest.CWLv11Test",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Run the CWL 1.1 conformance tests in various environments."]; "ToilTest" -> "CWLv11Test" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }toil.test.ToilTest
Run the CWL 1.1 conformance tests in various environments.
- classmethod setUpClass()¶
Runs anew before each test.
- tearDown()¶
Clean up outputs.
- test_run_conformance(**kwargs)¶
- test_run_conformance_with_caching()¶
- test_kubernetes_cwl_conformance(**kwargs)¶
- test_kubernetes_cwl_conformance_with_caching()¶
- class toil.test.cwl.cwlTest.CWLv12Test(methodName='runTest')¶
Bases:
digraph inheritance1fae53748c { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "CWLv12Test" [URL="#toil.test.cwl.cwlTest.CWLv12Test",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Run the CWL 1.2 conformance tests in various environments."]; "ToilTest" -> "CWLv12Test" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }toil.test.ToilTest
Run the CWL 1.2 conformance tests in various environments.
- classmethod setUpClass()¶
Runs anew before each test.
- tearDown()¶
Clean up outputs.
- test_run_conformance(**kwargs)¶
- test_run_conformance_with_caching()¶
- test_run_conformance_with_in_place_update()¶
Make sure that with –bypass-file-store we properly support in place update on a single node, and that this doesn’t break any other features.
- test_kubernetes_cwl_conformance(**kwargs)¶
- test_kubernetes_cwl_conformance_with_caching()¶
- test_wes_server_cwl_conformance()¶
Run the CWL conformance tests via WES. TOIL_WES_ENDPOINT must be specified. If the WES server requires authentication, set TOIL_WES_USER and TOIL_WES_PASSWORD.
To run manually:
TOIL_WES_ENDPOINT=http://localhost:8080 TOIL_WES_USER=test TOIL_WES_PASSWORD=password python -m pytest src/toil/test/cwl/cwlTest.py::CWLv12Test::test_wes_server_cwl_conformance -vv –log-level INFO –log-cli-level INFO
- class toil.test.cwl.cwlTest.CWLOnARMTest(methodName)¶
Bases:
digraph inheritance1d0c418345 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AbstractClusterTest" [URL="../../provisioners/clusterTest/index.html#toil.test.provisioners.clusterTest.AbstractClusterTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "ToilTest" -> "AbstractClusterTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "CWLOnARMTest" [URL="#toil.test.cwl.cwlTest.CWLOnARMTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Run the CWL 1.2 conformance tests on ARM specifically."]; "AbstractClusterTest" -> "CWLOnARMTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }toil.test.provisioners.clusterTest.AbstractClusterTest
Run the CWL 1.2 conformance tests on ARM specifically.
- setUp()¶
Set up for the test. Must be overridden to call this method and set self.jobStore.
- test_cwl_on_arm()¶
- toil.test.cwl.cwlTest.test_workflow_echo_string_scatter_stderr_log_dir(tmp_path)¶
- Parameters:
tmp_path (pathlib.Path) –
- toil.test.cwl.cwlTest.test_log_dir_echo_no_output(tmp_path)¶
- Parameters:
tmp_path (pathlib.Path) –
- Return type:
None
- toil.test.cwl.cwlTest.test_log_dir_echo_stderr(tmp_path)¶
- Parameters:
tmp_path (pathlib.Path) –
- Return type:
None
- toil.test.cwl.cwlTest.test_filename_conflict_resolution(tmp_path)¶
- Parameters:
tmp_path (pathlib.Path) –
- toil.test.cwl.cwlTest.test_filename_conflict_detection(tmp_path)¶
Make sure we don’t just stage files over each other when using a container.
- Parameters:
tmp_path (pathlib.Path) –
- toil.test.cwl.cwlTest.test_filename_conflict_detection_at_root(tmp_path)¶
Make sure we don’t just stage files over each other.
Specifically, when using a container and the files are at the root of the work dir.
- Parameters:
tmp_path (pathlib.Path) –
- toil.test.cwl.cwlTest.test_pick_value_with_one_null_value(caplog)¶
Make sure toil-cwl-runner does not false log a warning when pickValue is used but outputSource only contains one null value. See: #3991.
- toil.test.cwl.cwlTest.test_workflow_echo_string()¶
- toil.test.cwl.cwlTest.test_workflow_echo_string_scatter_capture_stdout()¶
- toil.test.cwl.cwlTest.test_visit_top_cwl_class()¶
- toil.test.cwl.cwlTest.test_visit_cwl_class_and_reduce()¶
- toil.test.cwl.cwlTest.test_download_structure(tmp_path)¶
Make sure that download_structure makes the right calls to what it thinks is the file store.
- Return type:
None