toil.lib.ec2

Attributes

a_short_time

a_long_time

logger

INCONSISTENCY_ERRORS

iam_client

Exceptions

UserError

Unspecified run-time error.

UnexpectedResourceState

Common base class for all non-exit exceptions.

Functions

is_base64(value)

Return True if value is base64-decodeable, and False otherwise.

not_found(e)

inconsistencies_detected(e)

retry_ec2([t, retry_for, retry_while])

wait_transition(boto3_ec2, resource, from_states, to_state)

Wait until the specified EC2 resource (instance, image, volume, ...) transitions from any

wait_instances_running(boto3_ec2, instances)

Wait until no instance in the given iterable is 'pending'. Yield every instance that

wait_spot_requests_active(boto3_ec2, requests[, ...])

Wait until no spot request in the given iterator is in the 'open' state or, optionally,

create_spot_instances(boto3_ec2, price, image_id, spec)

Create instances on the spot market.

create_ondemand_instances(boto3_ec2, image_id, spec[, ...])

Requests the RunInstances EC2 API call but accounts for the race between recently created

increase_instance_hop_limit(boto3_ec2, boto_instance_list)

Increase the default HTTP hop limit, as we are running Toil and Kubernetes inside a Docker container, so the default

prune(bushy)

Prune entries in the given dict with false-y values.

wait_until_instance_profile_arn_exists(...)

create_instances(ec2_resource, image_id, key_name, ...)

Replaces create_ondemand_instances. Uses boto3 and returns a list of Boto3 instance dicts.

create_launch_template(ec2_client, template_name, ...)

Creates a launch template with the given name for launching instances with the given parameters.

create_auto_scaling_group(autoscaling_client, ...[, ...])

Create a new Auto Scaling Group with the given name (which is also its

Module Contents

toil.lib.ec2.a_short_time = 5
toil.lib.ec2.a_long_time = 3600
toil.lib.ec2.logger
toil.lib.ec2.is_base64(value)

Return True if value is base64-decodeable, and False otherwise.

Parameters:

value (str)

Return type:

bool

exception toil.lib.ec2.UserError(message=None, cause=None)

Bases: RuntimeError

Unspecified run-time error.

toil.lib.ec2.not_found(e)
toil.lib.ec2.inconsistencies_detected(e)
toil.lib.ec2.INCONSISTENCY_ERRORS
toil.lib.ec2.retry_ec2(t=a_short_time, retry_for=10 * a_short_time, retry_while=not_found)
exception toil.lib.ec2.UnexpectedResourceState(resource, to_state, state)

Bases: Exception

Common base class for all non-exit exceptions.

toil.lib.ec2.wait_transition(boto3_ec2, resource, from_states, to_state, state_getter=lambda x: ...)

Wait until the specified EC2 resource (instance, image, volume, …) transitions from any of the given ‘from’ states to the specified ‘to’ state. If the instance is found in a state other that the to state or any of the from states, an exception will be thrown.

Parameters:

  • resource (mypy_boto3_ec2.type_defs.InstanceTypeDef) – the resource to monitor

  • from_states (collections.abc.Iterable[str]) – a set of states that the resource is expected to be in before the transition occurs

  • to_state (str) – the state of the resource when this method returns

  • boto3_ec2 (mypy_boto3_ec2.client.EC2Client)

  • state_getter (collections.abc.Callable[[mypy_boto3_ec2.type_defs.InstanceTypeDef], str])

toil.lib.ec2.wait_instances_running(boto3_ec2, instances)

Wait until no instance in the given iterable is ‘pending’. Yield every instance that entered the running state as soon as it does.

Parameters:

  • boto3_ec2 (mypy_boto3_ec2.client.EC2Client) – the EC2 connection to use for making requests

  • instances (collections.abc.Iterable[mypy_boto3_ec2.type_defs.InstanceTypeDef]) – the instances to wait on

Return type:

collections.abc.Generator[mypy_boto3_ec2.type_defs.InstanceTypeDef, None, None]

toil.lib.ec2.wait_spot_requests_active(boto3_ec2, requests, timeout=None, tentative=False)

Wait until no spot request in the given iterator is in the ‘open’ state or, optionally, a timeout occurs. Yield spot requests as soon as they leave the ‘open’ state.

Parameters:

  • boto3_ec2 (mypy_boto3_ec2.client.EC2Client) – ec2 client

  • requests (collections.abc.Iterable[mypy_boto3_ec2.type_defs.SpotInstanceRequestTypeDef]) – The requests to wait on.

  • timeout (float) – Maximum time in seconds to spend waiting or None to wait forever. If a timeout occurs, the remaining open requests will be cancelled.

  • tentative (bool) – if True, give up on a spot request at the earliest indication of it not being fulfilled immediately

Return type:

collections.abc.Iterable[list[mypy_boto3_ec2.type_defs.SpotInstanceRequestTypeDef]]

toil.lib.ec2.create_spot_instances(boto3_ec2, price, image_id, spec, num_instances=1, timeout=None, tentative=False, tags=None)

Create instances on the spot market.

The “UserData” field in “LaunchSpecification” in spec MUST ALREADY BE base64-encoded. It will NOT be automatically encoded.

Parameters:

  • tags (dict[str, str]) – Dict from tag key to tag value of tags to apply to the request.

  • boto3_ec2 (mypy_boto3_ec2.client.EC2Client)

  • spec (dict[Literal['LaunchSpecification'], dict[str, Any]])

Return type:

collections.abc.Generator[mypy_boto3_ec2.type_defs.DescribeInstancesResultTypeDef, None, None]

toil.lib.ec2.create_ondemand_instances(boto3_ec2, image_id, spec, num_instances=1)

Requests the RunInstances EC2 API call but accounts for the race between recently created instance profiles, IAM roles and an instance creation that refers to them.

The “UserData” field in spec MUST NOT be base64 encoded; it will be base64-encoded by boto3 automatically. See <https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2/client/run_instances.html>.

Replaced by create_instances.

Parameters:
Return type:

list[mypy_boto3_ec2.type_defs.InstanceTypeDef]

toil.lib.ec2.increase_instance_hop_limit(boto3_ec2, boto_instance_list)

Increase the default HTTP hop limit, as we are running Toil and Kubernetes inside a Docker container, so the default hop limit of 1 will not be enough when grabbing metadata information with ec2_metadata

Must be called after the instances are guaranteed to be running.

Parameters:

  • boto_instance_list (list[mypy_boto3_ec2.type_defs.InstanceTypeDef]) – List of boto instances to modify

  • boto3_ec2 (mypy_boto3_ec2.client.EC2Client)

Returns:

Return type:

None

toil.lib.ec2.prune(bushy)

Prune entries in the given dict with false-y values. Boto3 may not like None and instead wants no key.

Parameters:

bushy (dict)

Return type:

dict

toil.lib.ec2.iam_client
toil.lib.ec2.wait_until_instance_profile_arn_exists(instance_profile_arn)
Parameters:

instance_profile_arn (str)

toil.lib.ec2.create_instances(ec2_resource, image_id, key_name, instance_type, num_instances=1, security_group_ids=None, user_data=None, block_device_map=None, instance_profile_arn=None, placement_az=None, subnet_id=None, tags=None)

Replaces create_ondemand_instances. Uses boto3 and returns a list of Boto3 instance dicts.

See “create_instances” (returns a list of ec2.Instance objects):

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.ServiceResource.create_instances

Not to be confused with “run_instances” (same input args; returns a dictionary):

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.run_instances

Parameters:

  • user_data (str | bytes | None) – non-base64-encoded user data to control instance startup.

  • tags (dict[str, str] | None) – if given, these tags are applied to the instances, and all volumes.

  • ec2_resource (mypy_boto3_ec2.service_resource.EC2ServiceResource)

  • image_id (str)

  • key_name (str)

  • instance_type (str)

  • num_instances (int)

  • security_group_ids (list | None)

  • block_device_map (list[dict] | None)

  • instance_profile_arn (str | None)

  • placement_az (str | None)

  • subnet_id (str)

Return type:

list[mypy_boto3_ec2.service_resource.Instance]

toil.lib.ec2.create_launch_template(ec2_client, template_name, image_id, key_name, instance_type, security_group_ids=None, user_data=None, block_device_map=None, instance_profile_arn=None, placement_az=None, subnet_id=None, tags=None)

Creates a launch template with the given name for launching instances with the given parameters.

We only ever use the default version of any launch template.

Internally calls https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html?highlight=create_launch_template#EC2.Client.create_launch_template

Parameters:

  • tags (dict[str, str] | None) – Tags, if given, are applied to the template itself, all instances, and all volumes.

  • user_data (str | bytes | None) – non-base64-encoded user data to pass to the instances.

  • ec2_client (mypy_boto3_ec2.client.EC2Client)

  • template_name (str)

  • image_id (str)

  • key_name (str)

  • instance_type (str)

  • security_group_ids (list | None)

  • block_device_map (list[dict] | None)

  • instance_profile_arn (str | None)

  • placement_az (str | None)

  • subnet_id (str | None)

Returns:

the ID of the launch template.

Return type:

str

toil.lib.ec2.create_auto_scaling_group(autoscaling_client, asg_name, launch_template_ids, vpc_subnets, min_size, max_size, instance_types=None, spot_bid=None, spot_cheapest=False, tags=None)

Create a new Auto Scaling Group with the given name (which is also its unique identifier).

Parameters:

  • autoscaling_client (mypy_boto3_autoscaling.client.AutoScalingClient) – Boto3 client for autoscaling.

  • asg_name (str) – Unique name for the autoscaling group.

  • launch_template_ids (dict[str, str]) – ID of the launch template to make instances from, for each instance type.

  • vpc_subnets (list[str]) – One or more subnet IDs to place instances in the group into. Determine the availability zone(s) instances will launch into.

  • min_size (int) – Minimum number of instances to have in the group at all times.

  • max_size (int) – Maximum number of instances to allow in the group at any time.

  • instance_types (collections.abc.Iterable[str] | None) – Use a pool over the given instance types, instead of the type given in the launch template. For on-demand groups, this is a prioritized list. For spot groups, we let AWS balance according to spot_strategy. Must be 20 types or shorter.

  • spot_bid (float | None) – If set, the ASG will be a spot market ASG. Bid is in dollars per instance hour. All instance types in the group are bid on equivalently.

  • spot_cheapest (bool) – If true, use the cheapest spot instances available out of instance_types, instead of the spot instances that minimize eviction probability.

  • tags (dict[str, str] | None) – Tags to apply to the ASG only. Tags for the instances should be added to the launch template instead.

Return type:

None

The default version of the launch template is used.