toil.test.jobStores.jobStoreTest

Module Contents

Classes

AbstractJobStoreTest	Hide abstract base class from unittest's test case loader
AbstractEncryptedJobStoreTest
FileJobStoreTest	A common base class for Toil tests.
GoogleJobStoreTest	A common base class for Toil tests.
AWSJobStoreTest	A common base class for Toil tests.
InvalidAWSJobStoreTest	A common base class for Toil tests.
EncryptedAWSJobStoreTest	A common base class for Toil tests.
StubHttpRequestHandler	Simple HTTP request handler with GET and HEAD commands.

Functions

google_retry(x)
tearDownModule()

Attributes

logger

toil.test.jobStores.jobStoreTest.google_retry(x)

toil.test.jobStores.jobStoreTest.logger

toil.test.jobStores.jobStoreTest.tearDownModule()

class toil.test.jobStores.jobStoreTest.AbstractJobStoreTest

Hide abstract base class from unittest’s test case loader

http://stackoverflow.com/questions/1323455/python-unit-test-with-base-and-sub-class#answer-25695512

class Test(methodName='runTest')

Bases: toil.test.ToilTest

A common base class for Toil tests.

Please have every test case directly or indirectly inherit this one.

When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.

externalStoreCache

mpTestPartSize

classmethod setUpClass()

Hook method for setting up class fixture before running tests in the class.

setUp()

Hook method for setting up the test fixture before exercising it.

tearDown()

Hook method for deconstructing the test fixture after testing it.

testInitialState()

Ensure proper handling of nonexistant files.

testJobCreation()

Test creation of a job.

Does the job exist in the jobstore it is supposed to be in? Are its attributes what is expected?

testConfigEquality()

Ensure that the command line configurations are successfully loaded and stored.

In setUp() self.jobstore1 is created and initialized. In this test, after creating newJobStore, .resume() will look for a previously instantiated job store and load its config options. This is expected to be equal but not the same object.

testJobLoadEquality()

Tests that a job created via one JobStore instance can be loaded from another.

testChildLoadingEquality()

Test that loading a child job operates as expected.

testPersistantFilesToDelete()

Make sure that updating a job persists filesToDelete.

The following demonstrates the job update pattern, where files to be deleted atomically with a job update are referenced in “filesToDelete” array, which is persisted to disk first. If things go wrong during the update, this list of files to delete is used to ensure that the updated job and the files are never both visible at the same time.

testUpdateBehavior()

Tests the proper behavior during updating jobs.

testJobDeletions()

Tests the consequences of deleting jobs.

testSharedFiles()

Tests the sharing of files.

testReadWriteSharedFilesTextMode()

Checks if text mode is compatible for shared file streams.

testReadWriteFileStreamTextMode()

Checks if text mode is compatible for file streams.

testPerJobFiles()

Tests the behavior of files on jobs.

testStatsAndLogging()

Tests behavior of reading and writting stats and logging.

testWriteLogFiles()

Test writing log files.

testBatchCreate()

Test creation of many jobs.

testGrowingAndShrinkingJob()

Make sure jobs update correctly if they grow/shrink.

classmethod cleanUpExternalStores()

classmethod makeImportExportTests()

testImportHttpFile()

Test importing a file over HTTP.

testImportFtpFile()

Test importing a file over FTP

testFileDeletion()

Intended to cover the batch deletion of items in the AWSJobStore, but it doesn’t hurt running it on the other job stores.

testMultipartUploads()

This test is meant to cover multi-part uploads in the AWSJobStore but it doesn’t hurt running it against the other job stores as well.

testZeroLengthFiles()

Test reading and writing of empty files.

testLargeFile()

Test the reading and writing of large files.

fetch_url(url)

Fetch the given URL. Throw an error if it cannot be fetched in a reasonable number of attempts.

Parameters:

url (str)

Return type:

None

assertUrl(url)

testCleanCache()

testPartialReadFromStream()

Test whether readFileStream will deadlock on a partial read.

testDestructionOfCorruptedJobStore()

testDestructionIdempotence()

testEmptyFileStoreIDIsReadable()

Simply creates an empty fileStoreID and attempts to read from it.

class toil.test.jobStores.jobStoreTest.AbstractEncryptedJobStoreTest

class Test(methodName='runTest')

Bases: AbstractJobStoreTest

A test of job stores that use encryption

setUp()

Hook method for setting up the test fixture before exercising it.

tearDown()

Hook method for deconstructing the test fixture after testing it.

testEncrypted()

Create an encrypted file. Read it in encrypted mode then try with encryption off to ensure that it fails.

class toil.test.jobStores.jobStoreTest.FileJobStoreTest(methodName='runTest')

Bases: AbstractJobStoreTest

A common base class for Toil tests.

Please have every test case directly or indirectly inherit this one.

When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.

testPreserveFileName()

Check that the fileID ends with the given file name.

test_jobstore_init_preserves_symlink_path()

Test that if we provide a fileJobStore with a symlink to a directory, it doesn’t de-reference it.

test_jobstore_does_not_leak_symlinks()

Test that if we link imports into the FileJobStore, we can’t get hardlinks to symlinks.

test_file_link_imports()

Test that imported files are symlinked when when expected

class toil.test.jobStores.jobStoreTest.GoogleJobStoreTest(methodName='runTest')

Bases: AbstractJobStoreTest

A common base class for Toil tests.

Please have every test case directly or indirectly inherit this one.

When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.

projectID

headers

class toil.test.jobStores.jobStoreTest.AWSJobStoreTest(methodName='runTest')

Bases: AbstractJobStoreTest

A common base class for Toil tests.

Please have every test case directly or indirectly inherit this one.

When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.

testSDBDomainsDeletedOnFailedJobstoreBucketCreation()

This test ensures that SDB domains bound to a jobstore are deleted if the jobstore bucket failed to be created. We simulate a failed jobstore bucket creation by using a bucket in a different region with the same name.

testInlinedFiles()

testOverlargeJob()

testMultiThreadImportFile()

Tests that importFile is thread-safe.

Return type:

None

class toil.test.jobStores.jobStoreTest.InvalidAWSJobStoreTest(methodName='runTest')

Bases: toil.test.ToilTest

A common base class for Toil tests.

Please have every test case directly or indirectly inherit this one.

When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.

testInvalidJobStoreName()

class toil.test.jobStores.jobStoreTest.EncryptedAWSJobStoreTest(methodName='runTest')

Bases: AWSJobStoreTest, AbstractEncryptedJobStoreTest

A common base class for Toil tests.

Please have every test case directly or indirectly inherit this one.

When running tests you may optionally set the TOIL_TEST_TEMP environment variable to the path of a directory where you want temporary test files be placed. The directory will be created if it doesn’t exist. The path may be relative in which case it will be assumed to be relative to the project root. If TOIL_TEST_TEMP is not defined, temporary files and directories will be created in the system’s default location for such files and any temporary files or directories left over from tests will be removed automatically removed during tear down. Otherwise, left-over files will not be removed.

class toil.test.jobStores.jobStoreTest.StubHttpRequestHandler(*args, directory=None, **kwargs)

Bases: http.server.SimpleHTTPRequestHandler

Simple HTTP request handler with GET and HEAD commands.

This serves files from the current directory and any of its subdirectories. The MIME type for files is determined by calling the .guess_type() method.

The GET and HEAD requests are identical except that the HEAD request omits the actual contents of the file.

fileContents = 'A good programmer looks both ways before crossing a one-way street'

do_GET()

Serve a GET request.