toil.cwl.utils

Utility functions used for Toil’s CWL interpreter.

Attributes

logger
CWL_UNSUPPORTED_REQUIREMENT_EXIT_CODE
CWL_UNSUPPORTED_REQUIREMENT_EXCEPTION
DownReturnType
UpReturnType
DirectoryStructure

Exceptions

CWLUnsupportedException

Fallback exception.

Functions

visit_top_cwl_class(rec, classes, op)	Apply the given operation to all top-level CWL objects with the given named CWL class.
visit_cwl_class_and_reduce(rec, classes, op_down, op_up)	Apply the given operations to all CWL objects with the given named CWL class.
get_from_structure(dir_dict, path)	Given a relative path, follow it in the given directory structure.
download_structure(file_store, index, existing, ...)	Download nested dictionary from the Toil file store to a local path.
trim_mounts_op_down(file_or_directory)	No-op function for mount-point trimming.
sniff_location(file_or_directory)	Get the local bare path for a CWL file or directory, or None.
trim_mounts_op_up(file_or_directory, op_down_ret, ...)	Remove subtrees of the CWL file or directory object tree that only have redundant stuff in them.
remove_redundant_mounts(cwljob)	Remove any redundant mount points from the listing. Modifies the CWL object in place.

Module Contents

toil.cwl.utils.logger

toil.cwl.utils.CWL_UNSUPPORTED_REQUIREMENT_EXIT_CODE = 33

exception toil.cwl.utils.CWLUnsupportedException

Bases: Exception

Fallback exception.

toil.cwl.utils.CWL_UNSUPPORTED_REQUIREMENT_EXCEPTION: type[cwltool.errors.UnsupportedRequirement] | type[CWLUnsupportedException]

toil.cwl.utils.visit_top_cwl_class(rec, classes, op)

Apply the given operation to all top-level CWL objects with the given named CWL class.

Like cwltool’s visit_class but doesn’t look inside any object visited.

Parameters:

• rec (Any)

• classes (collections.abc.Iterable[str])

• op (collections.abc.Callable[[Any], Any])

Return type:

None

toil.cwl.utils.DownReturnType

toil.cwl.utils.UpReturnType

toil.cwl.utils.visit_cwl_class_and_reduce(rec, classes, op_down, op_up)

Apply the given operations to all CWL objects with the given named CWL class.

Applies the down operation top-down, and the up operation bottom-up, and passes the down operation’s result and a list of the up operation results for all child keys (flattening across lists and collapsing nodes of non-matching classes) to the up operation.

Returns:

The flattened list of up operation results from all calls.

Parameters:

• rec (Any)

• classes (collections.abc.Iterable[str])

• op_down (collections.abc.Callable[[Any], DownReturnType])

• op_up (collections.abc.Callable[[Any, DownReturnType, list[UpReturnType]], UpReturnType])

Return type:

list[UpReturnType]

toil.cwl.utils.DirectoryStructure

toil.cwl.utils.get_from_structure(dir_dict, path)

Given a relative path, follow it in the given directory structure.

Return the string URI for files, the directory dict for subdirectories, or None for nonexistent things.

Parameters:

• dir_dict (DirectoryStructure)

• path (str)

Return type:

str | DirectoryStructure | None

toil.cwl.utils.download_structure(file_store, index, existing, dir_dict, into_dir)

Download nested dictionary from the Toil file store to a local path.

Guaranteed to fill the structure with real files, and not symlinks out of it to elsewhere. File URIs may be toilfile: URIs or any other URI that Toil’s job store system can read.

Parameters:

• file_store (toil.fileStores.abstractFileStore.AbstractFileStore) – The Toil file store to download from.

• index (dict[str, str]) – Maps from downloaded file path back to input URI.

• existing (dict[str, str]) – Maps from file_store_id URI to downloaded file path.

• dir_dict (DirectoryStructure) – a dict from string to string (for files) or dict (for subdirectories) describing a directory structure.

• into_dir (str) – The directory to download the top-level dict’s files into.

Return type:

None

toil.cwl.utils.trim_mounts_op_down(file_or_directory)

No-op function for mount-point trimming.

Parameters:

file_or_directory (cwl_utils.types.CWLObjectType)

Return type:

None

toil.cwl.utils.sniff_location(file_or_directory)

Get the local bare path for a CWL file or directory, or None.

Returns:

None if we don’t have a local path or file URI

Parameters:

file_or_directory (cwl_utils.types.CWLObjectType)

Return type:

str | None

toil.cwl.utils.trim_mounts_op_up(file_or_directory, op_down_ret, child_results)

Remove subtrees of the CWL file or directory object tree that only have redundant stuff in them.

Nonredundant for something in a directory means its path or location is not within the parent directory or doesn’t match its basename Nonredundant for something in a secondary file means its path or location is not adjacent to the primary file or doesn’t match its basename

If on a File: Returns True if anything in secondary files is nonredundant or has nonredundant children to this file, false otherwise If on a Directory: Returns True if anything in top level listing is nonredundant or has nonredundant children, otherwise false. If something in the listing is redundant and all children are redundant, then delete it :param file_or_directory: CWL file or CWL directory type :return: boolean

Parameters:

• file_or_directory (cwl_utils.types.CWLObjectType)

• op_down_ret (None)

• child_results (list[bool])

Return type:

bool

toil.cwl.utils.remove_redundant_mounts(cwljob)

Remove any redundant mount points from the listing. Modifies the CWL object in place.

Parameters:

cwljob (cwl_utils.types.CWLObjectType)

Return type:

None