WDL Conformance Testing¶
The Toil team maintains a set of WDL Conformance Tests. Much like the CWL Conformance Tests for CWL, the WDL Conformance Tests are useful for determining if a WDL implementation actually follows the WDL specification.
The WDL Conformance Tests include a runner harness that is able to test
toil-wdl-runner, as well as Cromwell and MiniWDL, and supports testing
conformance with the 1.1, 1.0, and
draft-2 versions of WDL.
If you would like to evaluate Toil’s WDL conformance for yourself, first make
sure that you have
toil-wdl-runner installed. It comes with the
extra; see Installing Toil with Extra Features.
Then, you can check out the test repository:
$ git clone https://github.com/DataBiosphere/wdl-conformance-tests
$ cd wdl-conformance-tests
Most tests will need a Docker daemon available, so make sure yours is working properly:
$ docker info
$ docker run --rm docker/whalesay cowsay "Docker is working"
Then, you can test
toil-wdl-runner against a particular WDL spec version,
$ python3 run.py --runner toil-wdl-runner --versions 1.1
For any failed tests, the test number and the log of the failing test will be reported.
After the tests run, you can clean up intermediate files with:
$ make clean
For more options, see:
$ python3 run.py --help
Or, consult the conformance test documentation.