toil.test.provisioners.aws.awsProvisionerTest
¶
Module Contents¶
Classes¶
Tests for the AWS provisioner that don't actually provision anything. |
|
A common base class for Toil tests. |
|
A common base class for Toil tests. |
|
Runs the tests on a statically provisioned cluster with autoscaling enabled. |
|
Runs the tests on a self-scaling Kubernetes cluster. |
|
A common base class for Toil tests. |
|
This test insures autoscaling works on a restarted Toil run. |
|
A common base class for Toil tests. |
Attributes¶
- toil.test.provisioners.aws.awsProvisionerTest.log¶
- class toil.test.provisioners.aws.awsProvisionerTest.AWSProvisionerBenchTest(methodName='runTest')¶
Bases:
digraph inheritance3215268d81 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AWSProvisionerBenchTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AWSProvisionerBenchTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Tests for the AWS provisioner that don't actually provision anything."]; "ToilTest" -> "AWSProvisionerBenchTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }toil.test.ToilTest
Tests for the AWS provisioner that don’t actually provision anything.
- test_AMI_finding()¶
- test_read_write_global_files()¶
Make sure the _write_file_to_cloud() and _read_file_from_cloud() functions of the AWS provisioner work as intended.
- class toil.test.provisioners.aws.awsProvisionerTest.AbstractAWSAutoscaleTest(methodName)¶
Bases:
digraph inheritancef601c76a70 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AbstractAWSAutoscaleTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AbstractAWSAutoscaleTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AbstractClusterTest" -> "AbstractAWSAutoscaleTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractClusterTest" [URL="../../clusterTest/index.html#toil.test.provisioners.clusterTest.AbstractClusterTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "ToilTest" -> "AbstractClusterTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }toil.test.provisioners.clusterTest.AbstractClusterTest
A common base class for Toil tests.
Please have every test case directly or indirectly inherit this one.
When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.
- script()¶
Return the full path to the user script on the leader.
- data(filename)¶
Return the full path to the data file with the given name on the leader.
- rsyncUtil(src, dest)¶
- getRootVolID()¶
- class toil.test.provisioners.aws.awsProvisionerTest.AWSAutoscaleTest(name)¶
Bases:
digraph inheritance73acbe27d7 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AWSAutoscaleTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AWSAutoscaleTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AbstractAWSAutoscaleTest" -> "AWSAutoscaleTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractAWSAutoscaleTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AbstractAWSAutoscaleTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AbstractClusterTest" -> "AbstractAWSAutoscaleTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractClusterTest" [URL="../../clusterTest/index.html#toil.test.provisioners.clusterTest.AbstractClusterTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "ToilTest" -> "AbstractClusterTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }AbstractAWSAutoscaleTest
A common base class for Toil tests.
Please have every test case directly or indirectly inherit this one.
When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.
- setUp()¶
Set up for the test. Must be overridden to call this method and set self.jobStore.
- launchCluster()¶
- getRootVolID()¶
Adds in test to check that EBS volume is build with adequate size. Otherwise is functionally equivalent to parent. :return: volumeID
- testAutoScale()¶
- testSpotAutoScale()¶
- testSpotAutoScaleBalancingTypes()¶
- class toil.test.provisioners.aws.awsProvisionerTest.AWSStaticAutoscaleTest(name)¶
Bases:
digraph inheritance5897dd5eea { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AWSAutoscaleTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AWSAutoscaleTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AbstractAWSAutoscaleTest" -> "AWSAutoscaleTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AWSStaticAutoscaleTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AWSStaticAutoscaleTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Runs the tests on a statically provisioned cluster with autoscaling enabled."]; "AWSAutoscaleTest" -> "AWSStaticAutoscaleTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractAWSAutoscaleTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AbstractAWSAutoscaleTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AbstractClusterTest" -> "AbstractAWSAutoscaleTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractClusterTest" [URL="../../clusterTest/index.html#toil.test.provisioners.clusterTest.AbstractClusterTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "ToilTest" -> "AbstractClusterTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }AWSAutoscaleTest
Runs the tests on a statically provisioned cluster with autoscaling enabled.
- launchCluster()¶
- class toil.test.provisioners.aws.awsProvisionerTest.AWSManagedAutoscaleTest(name)¶
Bases:
digraph inheritance6138a3b761 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AWSAutoscaleTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AWSAutoscaleTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AbstractAWSAutoscaleTest" -> "AWSAutoscaleTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AWSManagedAutoscaleTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AWSManagedAutoscaleTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="Runs the tests on a self-scaling Kubernetes cluster."]; "AWSAutoscaleTest" -> "AWSManagedAutoscaleTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractAWSAutoscaleTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AbstractAWSAutoscaleTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AbstractClusterTest" -> "AbstractAWSAutoscaleTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractClusterTest" [URL="../../clusterTest/index.html#toil.test.provisioners.clusterTest.AbstractClusterTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "ToilTest" -> "AbstractClusterTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }AWSAutoscaleTest
Runs the tests on a self-scaling Kubernetes cluster.
- launchCluster()¶
- class toil.test.provisioners.aws.awsProvisionerTest.AWSAutoscaleTestMultipleNodeTypes(name)¶
Bases:
digraph inheritance20c3cc241f { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AWSAutoscaleTestMultipleNodeTypes" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AWSAutoscaleTestMultipleNodeTypes",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AbstractAWSAutoscaleTest" -> "AWSAutoscaleTestMultipleNodeTypes" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractAWSAutoscaleTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AbstractAWSAutoscaleTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AbstractClusterTest" -> "AbstractAWSAutoscaleTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractClusterTest" [URL="../../clusterTest/index.html#toil.test.provisioners.clusterTest.AbstractClusterTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "ToilTest" -> "AbstractClusterTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }AbstractAWSAutoscaleTest
A common base class for Toil tests.
Please have every test case directly or indirectly inherit this one.
When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.
- setUp()¶
Set up for the test. Must be overridden to call this method and set self.jobStore.
- testAutoScale()¶
- class toil.test.provisioners.aws.awsProvisionerTest.AWSRestartTest(name)¶
Bases:
digraph inheritance80d3478d81 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AWSRestartTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AWSRestartTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="This test insures autoscaling works on a restarted Toil run."]; "AbstractAWSAutoscaleTest" -> "AWSRestartTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractAWSAutoscaleTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AbstractAWSAutoscaleTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AbstractClusterTest" -> "AbstractAWSAutoscaleTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractClusterTest" [URL="../../clusterTest/index.html#toil.test.provisioners.clusterTest.AbstractClusterTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "ToilTest" -> "AbstractClusterTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }AbstractAWSAutoscaleTest
This test insures autoscaling works on a restarted Toil run.
- setUp()¶
Set up for the test. Must be overridden to call this method and set self.jobStore.
- testAutoScaledCluster()¶
- class toil.test.provisioners.aws.awsProvisionerTest.PreemptibleDeficitCompensationTest(name)¶
Bases:
digraph inheritance0442664848 { bgcolor=transparent; rankdir=LR; size="8.0, 12.0"; "AbstractAWSAutoscaleTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.AbstractAWSAutoscaleTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AbstractClusterTest" -> "AbstractAWSAutoscaleTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "AbstractClusterTest" [URL="../../clusterTest/index.html#toil.test.provisioners.clusterTest.AbstractClusterTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "ToilTest" -> "AbstractClusterTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "PreemptibleDeficitCompensationTest" [URL="#toil.test.provisioners.aws.awsProvisionerTest.PreemptibleDeficitCompensationTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top"]; "AbstractAWSAutoscaleTest" -> "PreemptibleDeficitCompensationTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; "TestCase" [fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",tooltip="A class whose instances are single test cases."]; "ToilTest" [URL="../../../index.html#toil.test.ToilTest",fillcolor=white,fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5),filled",target="_top",tooltip="A common base class for Toil tests."]; "TestCase" -> "ToilTest" [arrowsize=0.5,style="setlinewidth(0.5)"]; }AbstractAWSAutoscaleTest
A common base class for Toil tests.
Please have every test case directly or indirectly inherit this one.
When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.
- setUp()¶
Set up for the test. Must be overridden to call this method and set self.jobStore.
- test()¶