toil.utils.toilStats

Reports statistical data about a given Toil workflow.

Attributes

logger

CATEGORIES

CATEGORY_UNITS

TITLES

TIME_CATEGORIES

SPACE_CATEGORIES

COMPUTED_CATEGORIES

LONG_FORMS

sort_category_choices

sort_field_choices

Classes

ColumnWidths

Convenience object that stores the width of columns for printing. Helps make things pretty.

Functions

pad_str(s[, field])

Pad the beginning of a string with spaces, if necessary.

pretty_space(k[, field, alone])

Given input k as kibibytes, return a nicely formatted string.

pretty_time(t[, field, unit, alone])

Given input t as seconds, return a nicely formatted string.

report_unit(unit)

Format a unit name for display.

report_time(t, options[, field, unit, alone])

Given t seconds, report back the correct format as string.

report_space(k, options[, field, unit, alone])

Given k kibibytes, report back the correct format as string.

report_number(n[, field, nan_value])

Given a number, report back the correct format as string.

report(v, category, options[, field, alone])

Report a value of the given category formatted as a string.

sprint_tag(key, tag, options[, columnWidths])

Generate a pretty-print ready string from a JTTag().

decorate_title(category, title, options)

Add extra parts to the category titles.

decorate_subheader(category, columnWidths, options)

Add a marker to the correct field if the TITLE is sorted on.

get(tree, name)

Return a float value attribute NAME from TREE.

sort_jobs(jobTypes, options)

Return a jobTypes all sorted.

report_pretty_data(root, worker, job, job_types, options)

Print the important bits out.

compute_column_widths(job_types, worker, job, options)

Return a ColumnWidths() object with the correct max widths.

update_column_widths(tag, cw, options)

Update the column width attributes for this tag's fields.

build_element(element, items, item_name, defaults)

Create an element for output.

create_summary(element, containingItems, ...)

Figure out how many jobs (or contained items) ran on each worker (or containing item).

get_stats(jobStore)

Sum together all the stats information in the job store.

process_data(config, stats)

Collate the stats and report

report_data(tree, options)

add_stats_options(parser)

main()

Reports stats on the workflow, use with --stats option to toil.

Module Contents

toil.utils.toilStats.logger
toil.utils.toilStats.CATEGORIES = ['time', 'clock', 'wait', 'memory', 'disk']
toil.utils.toilStats.CATEGORY_UNITS
toil.utils.toilStats.TITLES
toil.utils.toilStats.TIME_CATEGORIES
toil.utils.toilStats.SPACE_CATEGORIES
toil.utils.toilStats.COMPUTED_CATEGORIES
toil.utils.toilStats.LONG_FORMS
class toil.utils.toilStats.ColumnWidths[source]

Convenience object that stores the width of columns for printing. Helps make things pretty.

title(category)[source]

Return the total printed length of this category item.

Parameters:

category (str)

Return type:

int

get_width(category, field)[source]
Parameters:
Return type:

int

set_width(category, field, width)[source]
Parameters:
Return type:

None

report()[source]
Return type:

None

toil.utils.toilStats.pad_str(s, field=None)[source]

Pad the beginning of a string with spaces, if necessary.

Parameters:
  • s (str)

  • field (Optional[int])

Return type:

str

toil.utils.toilStats.pretty_space(k, field=None, alone=False)[source]

Given input k as kibibytes, return a nicely formatted string.

Parameters:
Return type:

str

toil.utils.toilStats.pretty_time(t, field=None, unit='s', alone=False)[source]

Given input t as seconds, return a nicely formatted string.

Parameters:
Return type:

str

toil.utils.toilStats.report_unit(unit)[source]

Format a unit name for display.

Parameters:

unit (str)

Return type:

str

toil.utils.toilStats.report_time(t, options, field=None, unit='s', alone=False)[source]

Given t seconds, report back the correct format as string.

Parameters:
Return type:

str

toil.utils.toilStats.report_space(k, options, field=None, unit='KiB', alone=False)[source]

Given k kibibytes, report back the correct format as string.

If unit is set to B, convert to KiB first.

Parameters:
Return type:

str

toil.utils.toilStats.report_number(n, field=None, nan_value='NaN')[source]

Given a number, report back the correct format as string.

If it is a NaN or None, use nan_value to represent it instead.

Parameters:
Return type:

str

toil.utils.toilStats.report(v, category, options, field=None, alone=False)[source]

Report a value of the given category formatted as a string.

Uses the given field width if set.

If alone is set, the field is being formatted outside a table and might need a unit.

Parameters:
Return type:

str

toil.utils.toilStats.sprint_tag(key, tag, options, columnWidths=None)[source]

Generate a pretty-print ready string from a JTTag().

Parameters:
Return type:

str

toil.utils.toilStats.decorate_title(category, title, options)[source]

Add extra parts to the category titles.

Add units to title if they won’t appear in the formatted values. Add a marker to TITLE if the TITLE is sorted on.

Parameters:
Return type:

str

toil.utils.toilStats.decorate_subheader(category, columnWidths, options)[source]

Add a marker to the correct field if the TITLE is sorted on.

Parameters:
Return type:

str

toil.utils.toilStats.get(tree, name)[source]

Return a float value attribute NAME from TREE.

Parameters:
Return type:

float

toil.utils.toilStats.sort_jobs(jobTypes, options)[source]

Return a jobTypes all sorted.

Parameters:
Return type:

List[Any]

toil.utils.toilStats.report_pretty_data(root, worker, job, job_types, options)[source]

Print the important bits out.

Parameters:
Return type:

str

toil.utils.toilStats.compute_column_widths(job_types, worker, job, options)[source]

Return a ColumnWidths() object with the correct max widths.

Parameters:
Return type:

ColumnWidths

toil.utils.toilStats.update_column_widths(tag, cw, options)[source]

Update the column width attributes for this tag’s fields.

Parameters:
Return type:

None

toil.utils.toilStats.build_element(element, items, item_name, defaults)[source]

Create an element for output.

Parameters:
Return type:

toil.lib.expando.Expando

toil.utils.toilStats.create_summary(element, containingItems, containingItemName, count_contained)[source]

Figure out how many jobs (or contained items) ran on each worker (or containing item).

Stick a bunch of xxx_number_per_xxx stats into element to describe this.

Parameters:
Return type:

None

toil.utils.toilStats.get_stats(jobStore)[source]

Sum together all the stats information in the job store.

Produces one object containing lists of the values from all the summed objects.

Parameters:

jobStore (toil.jobStores.abstractJobStore.AbstractJobStore)

Return type:

toil.lib.expando.Expando

toil.utils.toilStats.process_data(config, stats)[source]

Collate the stats and report

Parameters:
Return type:

toil.lib.expando.Expando

toil.utils.toilStats.report_data(tree, options)[source]
Parameters:
Return type:

None

toil.utils.toilStats.sort_category_choices
toil.utils.toilStats.sort_field_choices = ['min', 'med', 'ave', 'max', 'total']
toil.utils.toilStats.add_stats_options(parser)[source]
Parameters:

parser (argparse.ArgumentParser)

Return type:

None

toil.utils.toilStats.main()[source]

Reports stats on the workflow, use with –stats option to toil.

Return type:

None