toil.lib.misc

Attributes

logger
StrPath
FileDescriptorOrPath

Exceptions

CalledProcessErrorStderr

Version of CalledProcessError that include stderr in the error message if it is set

Functions

get_public_ip()	Get the IP that this machine uses to contact the internet.
get_user_name()	Get the current user name, or a suitable substitute string if the user name
utc_now()	Return a datetime in the UTC timezone corresponding to right now.
unix_now_ms()	Return the current time in milliseconds since the Unix epoch.
unix_seconds_to_timestamp(timestamp)	Convert a time in seconds since the Unix epoch to an ISO 8601 string.
unix_seconds_to_local_time(timestamp)	Returns a local time corresponding to the given number of seconds since the Unix epoch.
seconds_to_duration(time_difference)	Convert a time difference in seconds to an ISO 8601 duration string.
slow_down(seconds)	Toil jobs that have completed are not allowed to have taken 0 seconds, but
printq(msg, quiet[, log])	This is for functions used simultaneously in Toil proper and in the admin scripts.
truncExpBackoff()
call_command(cmd, *args[, input, timeout, useCLocale, ...])	Simplified calling of external commands.

Module Contents

toil.lib.misc.logger

type toil.lib.misc.StrPath = str | os.PathLike[str]

type toil.lib.misc.FileDescriptorOrPath = int | bytes | os.PathLike[bytes] | StrPath

toil.lib.misc.get_public_ip()

Get the IP that this machine uses to contact the internet.

If behind a NAT, this will still be this computer’s IP, and not the router’s.

Return type:

str

toil.lib.misc.get_user_name()

Get the current user name, or a suitable substitute string if the user name is not available.

Return type:

str

toil.lib.misc.utc_now()

Return a datetime in the UTC timezone corresponding to right now.

Return type:

datetime.datetime

toil.lib.misc.unix_now_ms()

Return the current time in milliseconds since the Unix epoch.

Return type:

float

toil.lib.misc.unix_seconds_to_timestamp(timestamp)

Convert a time in seconds since the Unix epoch to an ISO 8601 string.

Parameters:

timestamp (float)

Return type:

str

toil.lib.misc.unix_seconds_to_local_time(timestamp)

Returns a local time corresponding to the given number of seconds since the Unix epoch.

Parameters:

timestamp (float)

Return type:

datetime.datetime

toil.lib.misc.seconds_to_duration(time_difference)

Convert a time difference in seconds to an ISO 8601 duration string.

Parameters:

time_difference (float)

Return type:

str

toil.lib.misc.slow_down(seconds)

Toil jobs that have completed are not allowed to have taken 0 seconds, but Kubernetes timestamps round things to the nearest second. It is possible in some batch systems for a pod to have identical start and end timestamps.

This function takes a possibly 0 job length in seconds and enforces a minimum length to satisfy Toil.

Parameters:

seconds (float) – Timestamp difference

Returns:

seconds, or a small positive number if seconds is 0

Return type:

float

toil.lib.misc.printq(msg, quiet, log=False)

This is for functions used simultaneously in Toil proper and in the admin scripts.

Our admin scripts “print” to stdout, while Toil proper uses logging. For a script that, for example, cleans up IAM, EC2, etc. cruft leftover after failed CI runs, we can call an AWS delete IAM role function, and this prints or logs progress (unless quiet is True), depending on whether the function is called in, say, the jobstore or a script.

Parameters:

• msg (str) – The string to print or log to stdout.

• quiet (bool) – Silent output to stdout.

• log (bool) – Use logging (else “print” to the screen).

Return type:

None

toil.lib.misc.truncExpBackoff()

Return type:

collections.abc.Iterator[float]

exception toil.lib.misc.CalledProcessErrorStderr(returncode, cmd, output=None, stderr=None)

Bases: subprocess.CalledProcessError

Version of CalledProcessError that include stderr in the error message if it is set

__str__()

Return str(self).

Return type:

str

toil.lib.misc.call_command(cmd, *args, input=None, timeout=None, useCLocale=True, env=None, quiet=False)

Simplified calling of external commands.

If the process fails, CalledProcessErrorStderr is raised.

The captured stderr is always printed, regardless of if an exception occurs, so it can be logged.

Always logs the command at debug log level.

Parameters:

• quiet (Optional[bool]) – If True, do not log the command output. If False (the default), do log the command output at debug log level.

• useCLocale (bool) – If True, C locale is forced, to prevent failures that can occur in some batch systems when using UTF-8 locale.

• cmd (list[str])

• args (str)

• input (Optional[str])

• timeout (Optional[float])

• env (Optional[dict[str, str]])

Returns:

Command standard output, decoded as utf-8.

Return type:

str