toil.cwl.utils

Utility functions used for Toil’s CWL interpreter.

Module Contents

Functions

visit_top_cwl_class(rec, classes, op)

Apply the given operation to all top-level CWL objects with the given named CWL class.

visit_cwl_class_and_reduce(rec, classes, op_down, op_up)

Apply the given operations to all CWL objects with the given named CWL class.

get_from_structure(dir_dict, path)

Given a relative path, follow it in the given directory structure.

download_structure(file_store, index, existing, ...)

Download nested dictionary from the Toil file store to a local path.

Attributes

logger

CWL_UNSUPPORTED_REQUIREMENT_EXIT_CODE

CWL_UNSUPPORTED_REQUIREMENT_EXCEPTION

DownReturnType

UpReturnType

DirectoryStructure

toil.cwl.utils.logger
toil.cwl.utils.CWL_UNSUPPORTED_REQUIREMENT_EXIT_CODE = 33
exception toil.cwl.utils.CWLUnsupportedException[source]

Bases: Exception

Fallback exception.

toil.cwl.utils.CWL_UNSUPPORTED_REQUIREMENT_EXCEPTION: Type[cwltool.errors.UnsupportedRequirement] | Type[CWLUnsupportedException]
toil.cwl.utils.visit_top_cwl_class(rec, classes, op)[source]

Apply the given operation to all top-level CWL objects with the given named CWL class.

Like cwltool’s visit_class but doesn’t look inside any object visited.

Parameters:

  • rec (Any)

  • classes (Iterable[str])

  • op (Callable[[Any], Any])

Return type:

None

toil.cwl.utils.DownReturnType
toil.cwl.utils.UpReturnType
toil.cwl.utils.visit_cwl_class_and_reduce(rec, classes, op_down, op_up)[source]

Apply the given operations to all CWL objects with the given named CWL class.

Applies the down operation top-down, and the up operation bottom-up, and passes the down operation’s result and a list of the up operation results for all child keys (flattening across lists and collapsing nodes of non-matching classes) to the up operation.

Returns:

The flattened list of up operation results from all calls.

Parameters:

  • rec (Any)

  • classes (Iterable[str])

  • op_down (Callable[[Any], DownReturnType])

  • op_up (Callable[[Any, DownReturnType, List[UpReturnType]], UpReturnType])

Return type:

List[UpReturnType]

toil.cwl.utils.DirectoryStructure
toil.cwl.utils.get_from_structure(dir_dict, path)[source]

Given a relative path, follow it in the given directory structure.

Return the string URI for files, the directory dict for subdirectories, or None for nonexistent things.

Parameters:

  • dir_dict (DirectoryStructure)

  • path (str)

Return type:

Union[str, DirectoryStructure, None]

toil.cwl.utils.download_structure(file_store, index, existing, dir_dict, into_dir)[source]

Download nested dictionary from the Toil file store to a local path.

Guaranteed to fill the structure with real files, and not symlinks out of it to elsewhere. File URIs may be toilfile: URIs or any other URI that Toil’s job store system can read.

Parameters:

  • file_store (toil.fileStores.abstractFileStore.AbstractFileStore) – The Toil file store to download from.

  • index (Dict[str, str]) – Maps from downloaded file path back to input URI.

  • existing (Dict[str, str]) – Maps from file_store_id URI to downloaded file path.

  • dir_dict (DirectoryStructure) – a dict from string to string (for files) or dict (for subdirectories) describing a directory structure.

  • into_dir (str) – The directory to download the top-level dict’s files into.

Return type:

None