toil.toilState

Attributes

logger

Classes

ToilState

Holds the leader's scheduling information.

Module Contents

toil.toilState.logger
class toil.toilState.ToilState(jobStore)[source]

Holds the leader’s scheduling information.

But onlt that which does not need to be persisted back to the JobStore (such as information on completed and outstanding predecessors)

Holds the true single copies of all JobDescription objects that the Leader and ServiceManager will use. The leader and service manager shouldn’t do their own load() and update() calls on the JobStore; they should go through this class.

Everything in the leader should reference JobDescriptions by ID.

Only holds JobDescription objects, not Job objects, and those JobDescription objects only exist in single copies.

Parameters:

jobStore (toil.jobStores.abstractJobStore.AbstractJobStore)

bus
successor_to_predecessors: dict[str, set[str]]
successorCounts: dict[str, int]
service_to_client: dict[str, str]
servicesIssued: dict[str, set[str]]
jobs_issued: set[str]
totalFailedJobs: set[str]
hasFailedSuccessors: set[str]
failedSuccessors: set[str]
jobsToBeScheduledWithMultiplePredecessors: set[str]
load_workflow(rootJob, jobCache=None)[source]

Load the workflow rooted at the given job.

If jobs are loaded that have updated and need to be dealt with by the leader, JobUpdatedMessage messages will be sent to the message bus.

The jobCache is a map from jobStoreID to JobDescription or None. Is used to speed up the building of the state when loading initially from the JobStore, and is not preserved.

Parameters:
Return type:

None

job_exists(job_id)[source]

Test if the givin job exists now.

Returns True if the given job exists right now, and false if it hasn’t been created or it has been deleted elsewhere.

Doesn’t guarantee that the job will or will not be gettable, if racing another process, or if it is still cached.

Parameters:

job_id (str)

Return type:

bool

get_job(job_id)[source]

Get the one true copy of the JobDescription with the given ID.

Parameters:

job_id (str)

Return type:

toil.job.JobDescription

commit_job(job_id)[source]

Save back any modifications made to a JobDescription.

(one retrieved from get_job())

Parameters:

job_id (str)

Return type:

None

delete_job(job_id)[source]

Destroy a JobDescription.

May raise an exception if the job could not be cleaned up (i.e. files belonging to it failed to delete).

Parameters:

job_id (str)

Return type:

None

reset_job(job_id)[source]

Discard any local modifications to a JobDescription.

Will make modifications from other hosts visible.

Parameters:

job_id (str)

Return type:

None

reset_job_expecting_change(job_id, timeout)[source]

Discard any local modifications to a JobDescription.

Will make modifications from other hosts visible.

Will wait for up to timeout seconds for a modification (or deletion) from another host to actually be visible.

Always replaces the JobDescription with what is stored in the job store, even if no modification ends up being visible.

Returns True if an update was detected in time, and False otherwise.

Parameters:
Return type:

bool

successors_pending(predecessor_id, count)[source]

Remember that the given job has the given number more pending successors.

(that have not yet succeeded or failed.)

Parameters:

  • predecessor_id (str)

  • count (int)

Return type:

None

successor_returned(predecessor_id)[source]

Remember that the given job has one fewer pending successors.

(because one has succeeded or failed.)

Parameters:

predecessor_id (str)

Return type:

None

count_pending_successors(predecessor_id)[source]

Count number of pending successors of the given job.

Pending successors are those which have not yet succeeded or failed.

Parameters:

predecessor_id (str)

Return type:

int