toil.utils.toilLaunchCluster

Launches a toil leader instance with the specified provisioner.

Attributes

logger

Functions

applianceSelf([forceDockerAppliance])

Return the fully qualified name of the Docker image to start Toil appliance containers from.

parser_with_common_options([provisioner_options, ...])

build_tag_dict_from_env([environment])

opt_strtobool(b)

Convert an optional string representation of bool to None or bool

check_valid_node_types(provisioner, node_types)

Raises if an invalid nodeType is specified for aws or gce.

cluster_factory(provisioner[, clusterName, ...])

Find and instantiate the appropriate provisioner instance to make clusters in the given cloud.

parse_node_types(node_type_specs)

Parse a specification for zero or more node types.

set_logging_from_options(options)

create_tags_dict(tags)

main()

Module Contents

toil.utils.toilLaunchCluster.applianceSelf(forceDockerAppliance=False)

Return the fully qualified name of the Docker image to start Toil appliance containers from.

The result is determined by the current version of Toil and three environment variables: TOIL_DOCKER_REGISTRY, TOIL_DOCKER_NAME and TOIL_APPLIANCE_SELF.

TOIL_DOCKER_REGISTRY specifies an account on a publicly hosted docker registry like Quay or Docker Hub. The default is UCSC’s CGL account on Quay.io where the Toil team publishes the official appliance images. TOIL_DOCKER_NAME specifies the base name of the image. The default of toil will be adequate in most cases. TOIL_APPLIANCE_SELF fully qualifies the appliance image, complete with registry, image name and version tag, overriding both TOIL_DOCKER_NAME and TOIL_DOCKER_REGISTRY` as well as the version tag of the image. Setting TOIL_APPLIANCE_SELF will not be necessary in most cases.

Parameters:

forceDockerAppliance (bool)

Return type:

str

toil.utils.toilLaunchCluster.parser_with_common_options(provisioner_options=False, jobstore_option=True, prog=None, default_log_level=None)
Parameters:

  • provisioner_options (bool)

  • jobstore_option (bool)

  • prog (Optional[str])

  • default_log_level (Optional[int])

Return type:

configargparse.ArgParser

toil.utils.toilLaunchCluster.build_tag_dict_from_env(environment=os.environ)
Parameters:

environment (MutableMapping[str, str])

Return type:

Dict[str, str]

toil.utils.toilLaunchCluster.opt_strtobool(b)

Convert an optional string representation of bool to None or bool

Parameters:

b (Optional[str])

Return type:

Optional[bool]

toil.utils.toilLaunchCluster.check_valid_node_types(provisioner, node_types)

Raises if an invalid nodeType is specified for aws or gce.

Parameters:

  • provisioner (str) – ‘aws’ or ‘gce’ to specify which cloud provisioner used.

  • node_types (List[Tuple[Set[str], Optional[float]]]) – A list of node types. Example: [({‘t2.micro’}, None), ({‘t2.medium’}, 0.5)]

Returns:

Nothing. Raises if any instance type in the node type isn’t real.

toil.utils.toilLaunchCluster.cluster_factory(provisioner, clusterName=None, clusterType='mesos', zone=None, nodeStorage=50, nodeStorageOverrides=None, sseKey=None, enable_fuse=False)

Find and instantiate the appropriate provisioner instance to make clusters in the given cloud.

Raises ClusterTypeNotSupportedException if the given provisioner does not implement clusters of the given type.

Parameters:

  • provisioner (str) – The cloud type of the cluster.

  • clusterName (Optional[str]) – The name of the cluster.

  • clusterType (str) – The type of cluster: ‘mesos’ or ‘kubernetes’.

  • zone (Optional[str]) – The cloud zone

  • nodeStorage (int)

  • nodeStorageOverrides (Optional[List[str]])

  • sseKey (Optional[str])

  • enable_fuse (bool)

Returns:

A cluster object for the the cloud type.

Return type:

Union[aws.awsProvisioner.AWSProvisioner, gceProvisioner.GCEProvisioner]

toil.utils.toilLaunchCluster.parse_node_types(node_type_specs)

Parse a specification for zero or more node types.

Takes a comma-separated list of node types. Each node type is a slash-separated list of at least one instance type name (like ‘m5a.large’ for AWS), and an optional bid in dollars after a colon.

Raises ValueError if a node type cannot be parsed.

Inputs should look something like this:

>>> parse_node_types('c5.4xlarge/c5a.4xlarge:0.42,t2.large')
[({'c5.4xlarge', 'c5a.4xlarge'}, 0.42), ({'t2.large'}, None)]
Parameters:

node_type_specs (Optional[str]) – A string defining node types

Returns:

a list of node types, where each type is the set of instance types, and the float bid, or None.

Return type:

List[Tuple[Set[str], Optional[float]]]

toil.utils.toilLaunchCluster.set_logging_from_options(options)
Parameters:

options (Union[toil.common.Config, argparse.Namespace])

Return type:

None

toil.utils.toilLaunchCluster.logger
toil.utils.toilLaunchCluster.create_tags_dict(tags)
Parameters:

tags (List[str])

Return type:

Dict[str, str]

toil.utils.toilLaunchCluster.main()
Return type:

None