toil.utils.toilStats

Reports statistical data about a given Toil workflow.

Attributes

logger
CATEGORIES
CATEGORY_UNITS
TITLES
TIME_CATEGORIES
SPACE_CATEGORIES
COMPUTED_CATEGORIES
LONG_FORMS
sort_category_choices
sort_field_choices

Classes

ColumnWidths

Convenience object that stores the width of columns for printing. Helps make things pretty.

Functions

pad_str(s[, field])	Pad the beginning of a string with spaces, if necessary.
pretty_space(k[, field, alone])	Given input k as kibibytes, return a nicely formatted string.
pretty_time(t[, field, unit, alone])	Given input t as seconds, return a nicely formatted string.
report_unit(unit)	Format a unit name for display.
report_time(t, options[, field, unit, alone])	Given t seconds, report back the correct format as string.
report_space(k, options[, field, unit, alone])	Given k kibibytes, report back the correct format as string.
report_number(n[, field, nan_value])	Given a number, report back the correct format as string.
report(v, category, options[, field, alone])	Report a value of the given category formatted as a string.
sprint_tag(key, tag, options[, columnWidths])	Generate a pretty-print ready string from a JTTag().
decorate_title(category, title, options)	Add extra parts to the category titles.
decorate_subheader(category, columnWidths, options)	Add a marker to the correct field if the TITLE is sorted on.
get(tree, name)	Return a float value attribute NAME from TREE.
sort_jobs(jobTypes, options)	Return a jobTypes all sorted.
report_pretty_data(root, worker, job, job_types, options)	Print the important bits out.
compute_column_widths(job_types, worker, job, options)	Return a ColumnWidths() object with the correct max widths.
update_column_widths(tag, cw, options)	Update the column width attributes for this tag's fields.
build_element(element, items, item_name, defaults)	Create an element for output.
create_summary(element, containingItems, ...)	Figure out how many jobs (or contained items) ran on each worker (or containing item).
get_stats(jobStore)	Sum together all the stats information in the job store.
process_data(config, stats)	Collate the stats and report
report_data(tree, options)
add_stats_options(parser)
main()	Reports stats on the workflow, use with --stats option to toil.

Module Contents

toil.utils.toilStats.logger

toil.utils.toilStats.CATEGORIES = ['time', 'clock', 'wait', 'memory', 'disk']

toil.utils.toilStats.CATEGORY_UNITS

toil.utils.toilStats.TITLES

toil.utils.toilStats.TIME_CATEGORIES

toil.utils.toilStats.SPACE_CATEGORIES

toil.utils.toilStats.COMPUTED_CATEGORIES

toil.utils.toilStats.LONG_FORMS

class toil.utils.toilStats.ColumnWidths

Convenience object that stores the width of columns for printing. Helps make things pretty.

categories

fields_count = ['count', 'min', 'med', 'ave', 'max', 'total']

fields = ['min', 'med', 'ave', 'max', 'total']

data: Dict[str, int]

title(category)

Return the total printed length of this category item.

Parameters:

category (str)

Return type:

int

get_width(category, field)

Parameters:

• category (str)

• field (str)

Return type:

int

set_width(category, field, width)

Parameters:

• category (str)

• field (str)

• width (int)

Return type:

None

report()

Return type:

None

toil.utils.toilStats.pad_str(s, field=None)

Pad the beginning of a string with spaces, if necessary.

Parameters:

• s (str)

• field (Optional[int])

Return type:

str

toil.utils.toilStats.pretty_space(k, field=None, alone=False)

Given input k as kibibytes, return a nicely formatted string.

Parameters:

• k (float)

• field (Optional[int])

• alone (bool)

Return type:

str

toil.utils.toilStats.pretty_time(t, field=None, unit='s', alone=False)

Given input t as seconds, return a nicely formatted string.

Parameters:

• t (float)

• field (Optional[int])

• unit (str)

• alone (bool)

Return type:

str

toil.utils.toilStats.report_unit(unit)

Format a unit name for display.

Parameters:

unit (str)

Return type:

str

toil.utils.toilStats.report_time(t, options, field=None, unit='s', alone=False)

Given t seconds, report back the correct format as string.

Parameters:

• t (float)

• options (argparse.Namespace)

• field (Optional[int])

• unit (str)

• alone (bool)

Return type:

str

toil.utils.toilStats.report_space(k, options, field=None, unit='KiB', alone=False)

Given k kibibytes, report back the correct format as string.

If unit is set to B, convert to KiB first.

Parameters:

• k (float)

• options (argparse.Namespace)

• field (Optional[int])

• unit (str)

• alone (bool)

Return type:

str

toil.utils.toilStats.report_number(n, field=None, nan_value='NaN')

Given a number, report back the correct format as string.

If it is a NaN or None, use nan_value to represent it instead.

Parameters:

• n (Union[int, float, None])

• field (Optional[int])

• nan_value (str)

Return type:

str

toil.utils.toilStats.report(v, category, options, field=None, alone=False)

Report a value of the given category formatted as a string.

Uses the given field width if set.

If alone is set, the field is being formatted outside a table and might need a unit.

Parameters:

• v (float)

• category (str)

• options (argparse.Namespace)

• field (Optional[int])

Return type:

str

toil.utils.toilStats.sprint_tag(key, tag, options, columnWidths=None)

Generate a pretty-print ready string from a JTTag().

Parameters:

• key (str)

• tag (toil.lib.expando.Expando)

• options (argparse.Namespace)

• columnWidths (Optional[ColumnWidths])

Return type:

str

toil.utils.toilStats.decorate_title(category, title, options)

Add extra parts to the category titles.

Add units to title if they won’t appear in the formatted values. Add a marker to TITLE if the TITLE is sorted on.

Parameters:

• category (str)

• title (str)

• options (argparse.Namespace)

Return type:

str

toil.utils.toilStats.decorate_subheader(category, columnWidths, options)

Add a marker to the correct field if the TITLE is sorted on.

Parameters:

• category (str)

• columnWidths (ColumnWidths)

• options (argparse.Namespace)

Return type:

str

toil.utils.toilStats.get(tree, name)

Return a float value attribute NAME from TREE.

Parameters:

• tree (toil.lib.expando.Expando)

• name (str)

Return type:

float

toil.utils.toilStats.sort_jobs(jobTypes, options)

Return a jobTypes all sorted.

Parameters:

• jobTypes (List[Any])

• options (argparse.Namespace)

Return type:

List[Any]

toil.utils.toilStats.report_pretty_data(root, worker, job, job_types, options)

Print the important bits out.

Parameters:

• root (toil.lib.expando.Expando)

• worker (toil.lib.expando.Expando)

• job (toil.lib.expando.Expando)

• job_types (List[Any])

• options (argparse.Namespace)

Return type:

str

toil.utils.toilStats.compute_column_widths(job_types, worker, job, options)

Return a ColumnWidths() object with the correct max widths.

Parameters:

• job_types (List[Any])

• worker (toil.lib.expando.Expando)

• job (toil.lib.expando.Expando)

• options (argparse.Namespace)

Return type:

ColumnWidths

toil.utils.toilStats.update_column_widths(tag, cw, options)

Update the column width attributes for this tag’s fields.

Parameters:

• tag (toil.lib.expando.Expando)

• cw (ColumnWidths)

• options (argparse.Namespace)

Return type:

None

toil.utils.toilStats.build_element(element, items, item_name, defaults)

Create an element for output.

Parameters:

• element (toil.lib.expando.Expando)

• items (List[toil.job.Job])

• item_name (str)

• defaults (Dict[str, float])

Return type:

toil.lib.expando.Expando

toil.utils.toilStats.create_summary(element, containingItems, containingItemName, count_contained)

Figure out how many jobs (or contained items) ran on each worker (or containing item).

Stick a bunch of xxx_number_per_xxx stats into element to describe this.

Parameters:

• count_contained (Callable[[toil.lib.expando.Expando], int]) – function that maps from containing item to number of contained items.

• element (toil.lib.expando.Expando)

• containingItems (List[toil.lib.expando.Expando])

• containingItemName (str)

Return type:

None

toil.utils.toilStats.get_stats(jobStore)

Sum together all the stats information in the job store.

Produces one object containing lists of the values from all the summed objects.

Parameters:

jobStore (toil.jobStores.abstractJobStore.AbstractJobStore)

Return type:

toil.lib.expando.Expando

toil.utils.toilStats.process_data(config, stats)

Collate the stats and report

Parameters:

• config (toil.common.Config)

• stats (toil.lib.expando.Expando)

Return type:

toil.lib.expando.Expando

toil.utils.toilStats.report_data(tree, options)

Parameters:

• tree (toil.lib.expando.Expando)

• options (argparse.Namespace)

Return type:

None

toil.utils.toilStats.sort_category_choices

toil.utils.toilStats.sort_field_choices = ['min', 'med', 'ave', 'max', 'total']

toil.utils.toilStats.add_stats_options(parser)

Parameters:

parser (argparse.ArgumentParser)

Return type:

None

toil.utils.toilStats.main()

Reports stats on the workflow, use with –stats option to toil.

Return type:

None