toil.statsAndLogging
¶
Module Contents¶
Classes¶
A thread to aggregate statistics and logging. |
Functions¶
|
Sets the root logger level to a given string level (like "INFO"). |
|
Add logging options to set the global log level. |
Set up the root logger with handlers and formatting. |
|
|
|
|
|
|
Attempts to suppress the loggers of all non-Toil packages by setting them to CRITICAL. |
Attributes¶
- toil.statsAndLogging.logger¶
- toil.statsAndLogging.root_logger¶
- toil.statsAndLogging.toil_logger¶
- toil.statsAndLogging.DEFAULT_LOGLEVEL¶
- class toil.statsAndLogging.StatsAndLogging(jobStore, config)[source]¶
A thread to aggregate statistics and logging.
- Parameters
jobStore (toil.jobStores.abstractJobStore.AbstractJobStore) –
config (toil.common.Config) –
- classmethod formatLogStream(stream, job_name=None)[source]¶
Given a stream of text or bytes, and the job name, job itself, or some other optional stringifyable identity info for the job, return a big text string with the formatted job log, suitable for printing for the user.
We don’t want to prefix every line of the job’s log with our own logging info, or we get prefixes wider than any reasonable terminal and longer than the messages.
- classmethod writeLogFiles(jobNames, jobLogList, config, failed=False)[source]¶
- Parameters
jobNames (List[str]) –
jobLogList (List[str]) –
config (toil.common.Config) –
failed (bool) –
- Return type
None
- classmethod statsAndLoggingAggregator(jobStore, stop, config)[source]¶
The following function is used for collating stats/reporting log messages from the workers. Works inside of a thread, collates as long as the stop flag is not True.
- Parameters
jobStore (toil.jobStores.abstractJobStore.AbstractJobStore) –
stop (threading.Event) –
config (toil.common.Config) –
- Return type
None
- toil.statsAndLogging.set_log_level(level, set_logger=None)[source]¶
Sets the root logger level to a given string level (like “INFO”).
- Parameters
level (str) –
set_logger (Optional[logging.Logger]) –
- Return type
None
- toil.statsAndLogging.add_logging_options(parser)[source]¶
Add logging options to set the global log level.
- Parameters
parser (argparse.ArgumentParser) –
- Return type
None
- toil.statsAndLogging.configure_root_logger()[source]¶
Set up the root logger with handlers and formatting.
Should be called before any entry point tries to log anything, to ensure consistent formatting.
- Return type
None
- toil.statsAndLogging.set_logging_from_options(options)[source]¶
- Parameters
options (Union[toil.common.Config, argparse.Namespace]) –
- Return type
None
- toil.statsAndLogging.suppress_exotic_logging(local_logger)[source]¶
Attempts to suppress the loggers of all non-Toil packages by setting them to CRITICAL.
For example: ‘requests_oauthlib’, ‘google’, ‘boto’, ‘websocket’, ‘oauthlib’, etc.
This will only suppress loggers that have already been instantiated and can be seen in the environment, except for the list declared in “always_suppress”.
This is important because some packages, particularly boto3, are not always instantiated yet in the environment when this is run, and so we create the logger and set the level preemptively.
- Parameters
local_logger (str) –
- Return type
None