toil.realtimeLogger

Implements a real-time UDP-based logging system that user scripts can use for debugging.

Attributes

logger

Classes

LoggingDatagramHandler	Receive logging messages from the jobs and display them on the leader.
JSONDatagramHandler	Send logging records over UDP serialized as JSON.
RealtimeLoggerMetaclass	Metaclass for RealtimeLogger that lets add logging methods.
RealtimeLogger	Provide a logger that logs over UDP to the leader.

Module Contents

toil.realtimeLogger.logger

class toil.realtimeLogger.LoggingDatagramHandler(request, client_address, server)

Bases: socketserver.BaseRequestHandler

Receive logging messages from the jobs and display them on the leader.

Uses bare JSON message encoding.

handle()

Handle a single message. SocketServer takes care of splitting out the messages.

Messages are JSON-encoded logging module records.

Return type:

None

class toil.realtimeLogger.JSONDatagramHandler(host, port)

Bases: logging.handlers.DatagramHandler

Send logging records over UDP serialized as JSON.

They have to fit in a single UDP datagram, so don’t try to log more than 64kb at once.

makePickle(record)

Actually, encode the record as bare JSON instead.

Parameters:

record (logging.LogRecord)

Return type:

bytes

class toil.realtimeLogger.RealtimeLoggerMetaclass

Bases: type

Metaclass for RealtimeLogger that lets add logging methods.

Like RealtimeLogger.warning(), RealtimeLogger.info(), etc.

__getattr__(name)

Fallback to attributes on the logger.

Parameters:

name (str)

Return type:

Any

class toil.realtimeLogger.RealtimeLogger(batchSystem, level=defaultLevel)

Provide a logger that logs over UDP to the leader.

To use in a Toil job, do:

>>> from toil.realtimeLogger import RealtimeLogger
>>> RealtimeLogger.info("This logging message goes straight to the leader")

That’s all a user of Toil would need to do. On the leader, Job.Runner.startToil() automatically starts the UDP server by using an instance of this class as a context manager.

Parameters:

• batchSystem (toil.batchSystems.abstractBatchSystem.AbstractBatchSystem)

• level (str)

envPrefix = 'TOIL_RT_LOGGING_'

defaultLevel = 'INFO'

lock

loggingServer = None

serverThread = None

initialized = 0

logger = None

classmethod getLogger()

Get the logger that logs real-time to the leader.

Note that if the returned logger is used on the leader, you will see the message twice, since it still goes to the normal log handlers, too.

Return type:

logging.Logger

__enter__()

Return type:

None

__exit__(exc_type, exc_val, exc_tb)

Parameters:

• exc_type (Optional[Type[BaseException]])

• exc_val (Optional[BaseException])

• exc_tb (Optional[types.TracebackType])

Return type:

None