toil.lib.interpreter

Shared code for workflow language interpreters implemented with Toil.

Attributes

logger
INJECTED_MESSAGE_DIR

Functions

command_line_to_shell_script(command_line)	Extract or synthesize the inner shell script from a cwltool argv list.
shell_script_to_command_line(script)	Wrap a shell script as a list of arguments for launchign a process.
add_injections(command_string, file_mounts, message_dir)	Add resource usage monitoring and file mount checking code to a command.
handle_message_file(file_path)	Handle a message file received from injected code from add_injections().
handle_injection_messages_from(message_dir)	Handle any message files in the given directory.

Module Contents

toil.lib.interpreter.logger

toil.lib.interpreter.INJECTED_MESSAGE_DIR = '.toil_runtime'

toil.lib.interpreter.command_line_to_shell_script(command_line)

Extract or synthesize the inner shell script from a cwltool argv list.

We don’t want to disturb user CWL command line binding lists that explicitly ask for things like [“bash”, “-c”].

Parameters:

command_line (list[str])

Return type:

str

toil.lib.interpreter.shell_script_to_command_line(script)

Wrap a shell script as a list of arguments for launchign a process.

The resulting command only requires a POSIX shell as sh.

Parameters:

script (str)

Return type:

list[str]

toil.lib.interpreter.add_injections(command_string, file_mounts, message_dir)

Add resource usage monitoring and file mount checking code to a command.

The command is expected to be about to run in a container, under a container system that does not itself attribute resource usage to the calling Toil process (such as Docker, which uses a daemon).

All shell code added is compatible with POSIX sh.

Parameters:

• command_string (str) – shell command or script to modify

• file_mounts (Iterable[tuple[str, str]]) – collection of (host path, container path) tuples for files mounted into the container. Code will be added to require that the container sees the complete file that the host sees.

• message_dir (str) – directory, absolute or relative to the working directory that the command should record resource usage to

Returns:

shell command string (possibly containing multiple commands) that runs the original command and reports resource usage.

Return type:

str

toil.lib.interpreter.handle_message_file(file_path)

Handle a message file received from injected code from add_injections().

Records the described usage as if it had occurred within a child process, by talking to the toil.lib.resources.ResourceMonitor system.

Parameters:

file_path (str) – the host-side path of the message file.

Return type:

None

toil.lib.interpreter.handle_injection_messages_from(message_dir)

Handle any message files in the given directory.

Files would have been left by injected code from add_injections().

Parameters:

message_dir (str)

Return type:

None