toil.server.wes.toil_backend

Attributes

logger

Classes

ToilWorkflow

ToilBackend

WES backend implemented for Toil to run CWL, WDL, or Toil workflows. This

Module Contents

toil.server.wes.toil_backend.logger
class toil.server.wes.toil_backend.ToilWorkflow(base_work_dir, state_store_url, run_id)
Parameters:

  • base_work_dir (str)

  • state_store_url (str)

  • run_id (str)

base_scratch_dir
state_store_url
run_id
scratch_dir
exec_dir
store
state_machine
fetch_state(key: str, default: str) → str
fetch_state(key: str, default: None = None) → str | None

Return the contents of the given key in the workflow’s state store. If the key does not exist, the default value is returned.

fetch_scratch(filename)

Get a context manager for either a stream for the given file from the workflow’s scratch directory, or None if it isn’t there.

Parameters:

filename (str)

Return type:

collections.abc.Generator[Optional[TextIO], None, None]

exists()

Return True if the workflow run exists.

Return type:

bool

get_state()

Return the state of the current run.

Return type:

str

check_on_run(task_runner)

Check to make sure nothing has gone wrong in the task runner for this workflow. If something has, log, and fail the workflow with an error.

Parameters:

task_runner (type[toil.server.wes.tasks.TaskRunner])

Return type:

None

set_up_run()

Set up necessary directories for the run.

Return type:

None

clean_up()

Clean directory and files related to the run.

Return type:

None

queue_run(task_runner, request, options)

This workflow should be ready to run. Hand this to the task system.

Parameters:
Return type:

None

get_output_files()

Return a collection of output files that this workflow generated.

Return type:

Any

get_stdout_path()

Return the path to the standard output log, relative to the run’s scratch_dir, or None if it doesn’t exist.

Return type:

Optional[str]

get_stderr_path()

Return the path to the standard output log, relative to the run’s scratch_dir, or None if it doesn’t exist.

Return type:

Optional[str]

get_messages_path()

Return the path to the bus message log, relative to the run’s scratch_dir, or None if it doesn’t exist.

Return type:

Optional[str]

get_task_logs(filter_function=None)

Return all the task log objects for the individual tasks in the workflow.

Task names will be the job_type values from issued/completed/failed messages, with annotations from JobAnnotationMessage messages if available.

Parameters:

filter_function (Optional[Callable[[toil.server.wes.abstract_backend.TaskLog, toil.bus.JobStatus], Optional[toil.server.wes.abstract_backend.TaskLog]]]) – If set, will be called with each task log and its job annotations. Returns a modified copy of the task log to actually report, or None if the task log should be omitted.

Return type:

list[dict[str, Union[str, int, None]]]

class toil.server.wes.toil_backend.ToilBackend(work_dir, state_store, options, dest_bucket_base, bypass_celery=False, wes_dialect='standard')

Bases: toil.server.wes.abstract_backend.WESBackend

WES backend implemented for Toil to run CWL, WDL, or Toil workflows. This class is responsible for validating and executing submitted workflows.

Parameters:

  • work_dir (str)

  • state_store (Optional[str])

  • options (list[str])

  • dest_bucket_base (Optional[str])

  • bypass_celery (bool)

  • wes_dialect (str)

run_id_prefix = 'run-'
task_runner
wes_dialect = 'standard'
dest_bucket_base
work_dir = b'.'
supported_versions
get_runs()

A generator of a list of run ids and their state.

Return type:

collections.abc.Generator[tuple[str, str], None, None]

get_state(run_id)

Return the state of the workflow run with the given run ID. May raise an error if the workflow does not exist.

Parameters:

run_id (str)

Return type:

str

get_service_info()

Get information about the Workflow Execution Service.

Return type:

dict[str, Any]

list_runs(page_size=None, page_token=None)

List the workflow runs.

Parameters:

  • page_size (Optional[int])

  • page_token (Optional[str])

Return type:

dict[str, Any]

run_workflow()

Run a workflow.

Return type:

dict[str, str]

get_run_log(run_id)

Get detailed info about a workflow run.

Parameters:

run_id (str)

Return type:

dict[str, Any]

cancel_run(run_id)

Cancel a running workflow.

Parameters:

run_id (str)

Return type:

dict[str, str]

get_run_status(run_id)

Get quick status info about a workflow run, returning a simple result with the overall state of the workflow run.

Parameters:

run_id (str)

Return type:

dict[str, str]

get_stdout(run_id)

Get the stdout of a workflow run as a static file.

Parameters:

run_id (str)

Return type:

Any

get_stderr(run_id)

Get the stderr of a workflow run as a static file.

Parameters:

run_id (str)

Return type:

Any

get_health()

Return successfully if the server is healthy.

Return type:

werkzeug.wrappers.response.Response

get_homepage()

Provide a sensible result for / other than 404.

Return type:

werkzeug.wrappers.response.Response