toil.provisioners

Submodules

Attributes

logger

Exceptions

NoSuchClusterException

Indicates that the specified cluster does not exist.

NoSuchZoneException

Indicates that a valid zone could not be found.

ClusterTypeNotSupportedException

Indicates that a provisioner does not support a given cluster type.

ClusterCombinationNotSupportedException

Indicates that a provisioner does not support making a given type of cluster with a given architecture.

Functions

cluster_factory(provisioner[, clusterName, ...])

Find and instantiate the appropriate provisioner instance to make clusters in the given cloud.

add_provisioner_options(parser)

parse_node_types(node_type_specs)

Parse a specification for zero or more node types.

check_valid_node_types(provisioner, node_types)

Raises if an invalid nodeType is specified for aws or gce.

Package Contents

toil.provisioners.logger
toil.provisioners.cluster_factory(provisioner, clusterName=None, clusterType='mesos', zone=None, nodeStorage=50, nodeStorageOverrides=None, sseKey=None, enable_fuse=False)[source]

Find and instantiate the appropriate provisioner instance to make clusters in the given cloud.

Raises ClusterTypeNotSupportedException if the given provisioner does not implement clusters of the given type.

Parameters:

  • provisioner (str) – The cloud type of the cluster.

  • clusterName (Optional[str]) – The name of the cluster.

  • clusterType (str) – The type of cluster: ‘mesos’ or ‘kubernetes’.

  • zone (Optional[str]) – The cloud zone

  • nodeStorage (int)

  • nodeStorageOverrides (Optional[list[str]])

  • sseKey (Optional[str])

  • enable_fuse (bool)

Returns:

A cluster object for the the cloud type.

Return type:

Union[aws.awsProvisioner.AWSProvisioner, gceProvisioner.GCEProvisioner]

toil.provisioners.add_provisioner_options(parser)[source]
Parameters:

parser (argparse.ArgumentParser)

Return type:

None

toil.provisioners.parse_node_types(node_type_specs)[source]

Parse a specification for zero or more node types.

Takes a comma-separated list of node types. Each node type is a slash-separated list of at least one instance type name (like ‘m5a.large’ for AWS), and an optional bid in dollars after a colon.

Raises ValueError if a node type cannot be parsed.

Inputs should look something like this:

>>> parse_node_types('c5.4xlarge/c5a.4xlarge:0.42,t2.large')
[({'c5.4xlarge', 'c5a.4xlarge'}, 0.42), ({'t2.large'}, None)]
Parameters:

node_type_specs (Optional[str]) – A string defining node types

Returns:

a list of node types, where each type is the set of instance types, and the float bid, or None.

Return type:

list[tuple[set[str], Optional[float]]]

toil.provisioners.check_valid_node_types(provisioner, node_types)[source]

Raises if an invalid nodeType is specified for aws or gce.

Parameters:

  • provisioner (str) – ‘aws’ or ‘gce’ to specify which cloud provisioner used.

  • node_types (list[tuple[set[str], Optional[float]]]) – A list of node types. Example: [({‘t2.micro’}, None), ({‘t2.medium’}, 0.5)]

Returns:

Nothing. Raises if any instance type in the node type isn’t real.

exception toil.provisioners.NoSuchClusterException(cluster_name)[source]

Bases: Exception

Indicates that the specified cluster does not exist.

Parameters:

cluster_name (str)

exception toil.provisioners.NoSuchZoneException[source]

Bases: Exception

Indicates that a valid zone could not be found.

exception toil.provisioners.ClusterTypeNotSupportedException(provisioner_class, cluster_type)[source]

Bases: Exception

Indicates that a provisioner does not support a given cluster type.

exception toil.provisioners.ClusterCombinationNotSupportedException(provisioner_class, cluster_type, architecture, reason=None)[source]

Bases: Exception

Indicates that a provisioner does not support making a given type of cluster with a given architecture.

Parameters:

  • provisioner_class (type)

  • cluster_type (str)

  • architecture (str)

  • reason (Optional[str])