toil.lib.checksum

Attributes

logger

hashers

Exceptions

ChecksumError

Raised when a download does not contain the correct data.

Classes

Etag

A hasher for s3 etags.

Functions

compute_checksum_for_file(local_file_path[, algorithm])

compute_checksum_for_content(fh[, algorithm])

Note: Chunk size matters for s3 etags, and must be the same to get the same hash from the same object.

Module Contents

toil.lib.checksum.logger
exception toil.lib.checksum.ChecksumError

Bases: Exception

Raised when a download does not contain the correct data.

class toil.lib.checksum.Etag(chunk_size)

A hasher for s3 etags.

Parameters:

chunk_size (int)

etag_bytes: int = 0
etag_parts: List[bytes] = []
etag_hasher: hashlib._Hash
chunk_size: int
update(chunk)
Parameters:

chunk (bytes)

Return type:

None

hexdigest()
Return type:

str

toil.lib.checksum.hashers
toil.lib.checksum.compute_checksum_for_file(local_file_path, algorithm='sha1')
Parameters:
  • local_file_path (str)

  • algorithm (str)

Return type:

str

toil.lib.checksum.compute_checksum_for_content(fh, algorithm='sha1')

Note: Chunk size matters for s3 etags, and must be the same to get the same hash from the same object. Therefore this buffer is not modifiable throughout Toil.

Parameters:
Return type:

str