toil.server.cli.wes_cwl_runner

Module Contents

Classes

WESClientWithWorkflowEngineParameters

A modified version of the WESClient from the wes-service package that

Functions

generate_attachment_path_names(paths)

Take in a list of path names and return a list of names with the common path

get_deps_from_cwltool(cwl_file[, input_file])

Return a list of dependencies of the given workflow from cwltool.

submit_run(client, cwl_file[, input_file, engine_options])

Given a CWL file, its input files, and an optional list of engine options,

poll_run(client, run_id)

Return True if the given workflow run is in a finished state.

print_logs_and_exit(client, run_id)

Fetch the workflow logs from the WES server, print the results, then exit

main()

Attributes

logger

toil.server.cli.wes_cwl_runner.logger
toil.server.cli.wes_cwl_runner.generate_attachment_path_names(paths)

Take in a list of path names and return a list of names with the common path name stripped out, while preserving the input order. This guarantees that there are no relative paths that traverse up.

For example, for the following CWL workflow where “hello.yaml” references a file “message.txt”,

~/toil/workflows/hello.cwl ~/toil/input_files/hello.yaml ~/toil/input_files/message.txt

This may be run with the command:

toil-wes-cwl-runner hello.cwl ../input_files/hello.yaml

Where “message.txt” is resolved to “../input_files/message.txt”.

We’d send the workflow file as “workflows/hello.cwl”, and send the inputs as “input_files/hello.yaml” and “input_files/message.txt”.

Parameters:

paths (List[str]) – A list of absolute or relative path names. Relative paths are interpreted as relative to the current working directory.

Returns:

The common path name and a list of minimal path names.

Return type:

Tuple[str, List[str]]

class toil.server.cli.wes_cwl_runner.WESClientWithWorkflowEngineParameters(endpoint, auth=None)

Bases: wes_client.util.WESClient

A modified version of the WESClient from the wes-service package that includes workflow_engine_parameters support.

TODO: Propose a PR in wes-service to include workflow_engine_params.

Parameters:

  • endpoint (str)

  • auth (Optional[Tuple[str, str]])

get_version(extension, workflow_file)

Determines the version of a .py, .wdl, or .cwl file.

Parameters:

  • extension (str)

  • workflow_file (str)

Return type:

str

parse_params(workflow_params_file)

Parse the CWL input file into a dictionary to be attached to the body of the WES run request.

Parameters:

workflow_params_file (str) – The URL or path to the CWL input file.

Return type:

Dict[str, Any]

modify_param_paths(base_dir, workflow_params)

Modify the file paths in the input workflow parameters to be relative to base_dir.

Parameters:

  • base_dir (str) – The base directory to make the file paths relative to. This should be the common ancestor of all attached files, which will become the root of the execution folder.

  • workflow_params (Dict[str, Any]) – A dict containing the workflow parameters.

Return type:

None

build_wes_request(workflow_file, workflow_params_file, attachments, workflow_engine_parameters=None)

Build the workflow run request to submit to WES.

Parameters:

  • workflow_file (str) – The path or URL to the CWL workflow document. Only file:// URL supported at the moment.

  • workflow_params_file (Optional[str]) – The path or URL to the CWL input file.

  • attachments (Optional[List[str]]) – A list of local paths to files that will be uploaded to the server.

  • workflow_engine_parameters (Optional[List[str]]) – A list of engine parameters to set along with this workflow run.

Returns:

A dictionary of parameters as the body of the request, and an iterable for the pairs of filename and file contents to upload to the server.

Return type:

Tuple[Dict[str, str], Iterable[Tuple[str, Tuple[str, io.BytesIO]]]]

run_with_engine_options(workflow_file, workflow_params_file, attachments, workflow_engine_parameters)

Composes and sends a post request that signals the WES server to run a workflow.

Parameters:

  • workflow_file (str) – The path to the CWL workflow document.

  • workflow_params_file (Optional[str]) – The path to the CWL input file.

  • attachments (Optional[List[str]]) – A list of local paths to files that will be uploaded to the server.

  • workflow_engine_parameters (Optional[List[str]]) – A list of engine parameters to set along with this workflow run.

Returns:

The body of the post result as a dictionary.

Return type:

Dict[str, Any]

toil.server.cli.wes_cwl_runner.get_deps_from_cwltool(cwl_file, input_file=None)

Return a list of dependencies of the given workflow from cwltool.

Parameters:

  • cwl_file (str) – The CWL file.

  • input_file (Optional[str]) – Omit to get the dependencies from the CWL file. If set, this returns the dependencies from the input file.

Return type:

List[str]

toil.server.cli.wes_cwl_runner.submit_run(client, cwl_file, input_file=None, engine_options=None)

Given a CWL file, its input files, and an optional list of engine options, submit the CWL workflow to the WES server via the WES client.

This function also attempts to find the attachments from the CWL workflow and its input file, and attach them to the WES run request.

Parameters:

  • client (WESClientWithWorkflowEngineParameters) – The WES client.

  • cwl_file (str) – The path to the CWL workflow document.

  • input_file (Optional[str]) – The path to the CWL input file.

  • engine_options (Optional[List[str]]) – A list of engine parameters to set along with this workflow run.

Return type:

str

toil.server.cli.wes_cwl_runner.poll_run(client, run_id)

Return True if the given workflow run is in a finished state.

Parameters:
Return type:

bool

toil.server.cli.wes_cwl_runner.print_logs_and_exit(client, run_id)

Fetch the workflow logs from the WES server, print the results, then exit the program with the same exit code as the workflow run.

Parameters:
Return type:

None

toil.server.cli.wes_cwl_runner.main()
Return type:

None