toil.realtimeLogger

Implements a real-time UDP-based logging system that user scripts can use for debugging.

Module Contents

Classes

LoggingDatagramHandler	Receive logging messages from the jobs and display them on the leader.
JSONDatagramHandler	Send logging records over UDP serialized as JSON.
RealtimeLoggerMetaclass	Metaclass for RealtimeLogger that lets add logging methods.
RealtimeLogger	Provide a logger that logs over UDP to the leader.

Attributes

logger

toil.realtimeLogger.logger

class toil.realtimeLogger.LoggingDatagramHandler(request, client_address, server)

Bases: socketserver.BaseRequestHandler

Receive logging messages from the jobs and display them on the leader.

Uses bare JSON message encoding.

handle()

Handle a single message. SocketServer takes care of splitting out the messages.

Messages are JSON-encoded logging module records.

Return type

None

class toil.realtimeLogger.JSONDatagramHandler(host, port)

Bases: logging.handlers.DatagramHandler

Send logging records over UDP serialized as JSON.

They have to fit in a single UDP datagram, so don’t try to log more than 64kb at once.

makePickle(record)

Actually, encode the record as bare JSON instead.

Parameters

record (logging.LogRecord) –

Return type

bytes

class toil.realtimeLogger.RealtimeLoggerMetaclass

Bases: type

Metaclass for RealtimeLogger that lets add logging methods.

Like RealtimeLogger.warning(), RealtimeLogger.info(), etc.

__getattr__(name)

Fallback to attributes on the logger.

Parameters

name (str) –

Return type

Any

class toil.realtimeLogger.RealtimeLogger(batchSystem, level=defaultLevel)

Provide a logger that logs over UDP to the leader.

To use in a Toil job, do:

>>> from toil.realtimeLogger import RealtimeLogger
>>> RealtimeLogger.info("This logging message goes straight to the leader")

That’s all a user of Toil would need to do. On the leader, Job.Runner.startToil() automatically starts the UDP server by using an instance of this class as a context manager.

Parameters

• batchSystem (toil.batchSystems.abstractBatchSystem.AbstractBatchSystem) –

• level (str) –

envPrefix = 'TOIL_RT_LOGGING_'

defaultLevel = 'INFO'

lock

loggingServer

serverThread

initialized = 0

logger

classmethod getLogger()

Get the logger that logs real-time to the leader.

Note that if the returned logger is used on the leader, you will see the message twice, since it still goes to the normal log handlers, too.

Return type

logging.Logger

__enter__()

Return type

None

__exit__(exc_type, exc_val, exc_tb)

Parameters

• exc_type (Optional[Type[BaseException]]) –

• exc_val (Optional[BaseException]) –

• exc_tb (Optional[types.TracebackType]) –

Return type

None