toil.realtimeLogger

Implements a real-time UDP-based logging system that user scripts can use for debugging.

Attributes

logger

Classes

LoggingDatagramHandler

Receive logging messages from the jobs and display them on the leader.

JSONDatagramHandler

Send logging records over UDP serialized as JSON.

RealtimeLoggerMetaclass

Metaclass for RealtimeLogger that lets add logging methods.

RealtimeLogger

Provide a logger that logs over UDP to the leader.

Functions

get_public_ip()

Get the IP that this machine uses to contact the internet.

set_log_level(level[, set_logger])

Sets the root logger level to a given string level (like "INFO").

Module Contents

toil.realtimeLogger.get_public_ip()[source]

Get the IP that this machine uses to contact the internet.

If behind a NAT, this will still be this computer’s IP, and not the router’s.

Return type:

str

toil.realtimeLogger.set_log_level(level, set_logger=None)[source]

Sets the root logger level to a given string level (like “INFO”).

Parameters:
Return type:

None

toil.realtimeLogger.logger
class toil.realtimeLogger.LoggingDatagramHandler(request, client_address, server)[source]

Bases: socketserver.BaseRequestHandler

Receive logging messages from the jobs and display them on the leader.

Uses bare JSON message encoding.

handle()[source]

Handle a single message. SocketServer takes care of splitting out the messages.

Messages are JSON-encoded logging module records.

Return type:

None

class toil.realtimeLogger.JSONDatagramHandler(host, port)[source]

Bases: logging.handlers.DatagramHandler

Send logging records over UDP serialized as JSON.

They have to fit in a single UDP datagram, so don’t try to log more than 64kb at once.

makePickle(record)[source]

Actually, encode the record as bare JSON instead.

Parameters:

record (logging.LogRecord)

Return type:

bytes

class toil.realtimeLogger.RealtimeLoggerMetaclass[source]

Bases: type

Metaclass for RealtimeLogger that lets add logging methods.

Like RealtimeLogger.warning(), RealtimeLogger.info(), etc.

__getattr__(name)[source]

Fallback to attributes on the logger.

Parameters:

name (str)

Return type:

Any

class toil.realtimeLogger.RealtimeLogger(batchSystem, level=defaultLevel)[source]

Provide a logger that logs over UDP to the leader.

To use in a Toil job, do:

>>> from toil.realtimeLogger import RealtimeLogger
>>> RealtimeLogger.info("This logging message goes straight to the leader")

That’s all a user of Toil would need to do. On the leader, Job.Runner.startToil() automatically starts the UDP server by using an instance of this class as a context manager.

Parameters:
envPrefix = 'TOIL_RT_LOGGING_'
defaultLevel = 'INFO'
lock
loggingServer = None
serverThread = None
initialized = 0
logger = None
classmethod getLogger()[source]

Get the logger that logs real-time to the leader.

Note that if the returned logger is used on the leader, you will see the message twice, since it still goes to the normal log handlers, too.

Return type:

logging.Logger

__enter__()[source]
Return type:

None

__exit__(exc_type, exc_val, exc_tb)[source]
Parameters:
Return type:

None