Toil Documentation¶
Toil is an open-source pure-Python workflow engine that lets people write better pipelines.
Check out our website for a comprehensive list of Toil’s features and read our paper to learn what Toil can do in the real world. Please subscribe to our low-volume announce mailing list and feel free to also join us on GitHub and Gitter.
If using Toil for your research, please cite
Vivian, J., Rao, A. A., Nothaft, F. A., Ketchum, C., Armstrong, J., Novak, A., … Paten, B. (2017). Toil enables reproducible, open source, big biomedical data analyses. Nature Biotechnology, 35(4), 314–316. http://doi.org/10.1038/nbt.3772
Getting Started
Running Toil
- Introduction
- Commandline Options
- Toil Debugging
- Running in the Cloud
- Cloud Platforms
- Running on Kubernetes
- Running in AWS
- Running in Google Compute Engine (GCE)
- Cluster Utilities
- Stats Command
- Status Command
- Clean Command
- Launch-Cluster Command
- Ssh-Cluster Command
- Rsync-Cluster Command
- Destroy-Cluster Command
- Kill Command
- HPC Environments
- CWL in Toil
- WDL in Toil
- Workflow Execution Service (WES)
Developing Toil Workflows
- Developing a Workflow
- Scripting Quick Start
- Job Basics
- Invoking a Workflow
- Specifying Commandline Arguments
- Resuming a Workflow
- Functions and Job Functions
- Workflows with Multiple Jobs
- Dynamic Job Creation
- Promises
- Promised Requirements
- FileID
- Managing files within a workflow
- Using Docker Containers in Toil
- Services
- Checkpoints
- Encapsulation
- Depending on Toil
- Best Practices for Dockerizing Toil Workflows
- Toil Class API
- Job Store API
- Toil Job API
- Job Methods API
- Job.Runner API
- job.fileStore API
- Batch System API
- Job.Service API
- Exceptions API
Contributing to Toil
Appendices
- Toil Architecture
- Minimum AWS IAM permissions
- Auto-Deployment
- Environment Variables
- API Reference
toil
- Subpackages
toil.batchSystems
- Subpackages
toil.batchSystems.mesos
- Subpackages
- Submodules
toil.batchSystems.mesos.batchSystem
- Module Contents
- Classes
- Attributes
log
MesosBatchSystem
MesosBatchSystem.ExecutorInfo
MesosBatchSystem.userScript
MesosBatchSystem.supportsAutoDeployment()
MesosBatchSystem.supportsWorkerCleanup()
MesosBatchSystem.setUserScript()
MesosBatchSystem.ignoreNode()
MesosBatchSystem.unignoreNode()
MesosBatchSystem.issueBatchJob()
MesosBatchSystem.killBatchJobs()
MesosBatchSystem.getIssuedBatchJobIDs()
MesosBatchSystem.getRunningBatchJobIDs()
MesosBatchSystem.getUpdatedBatchJob()
MesosBatchSystem.nodeInUse()
MesosBatchSystem.getWaitDuration()
MesosBatchSystem.shutdown()
MesosBatchSystem.registered()
MesosBatchSystem.resourceOffers()
MesosBatchSystem.statusUpdate()
MesosBatchSystem.frameworkMessage()
MesosBatchSystem.getNodes()
MesosBatchSystem.reregistered()
MesosBatchSystem.executorLost()
MesosBatchSystem.get_default_mesos_endpoint()
MesosBatchSystem.add_options()
MesosBatchSystem.setOptions()
- Module Contents
toil.batchSystems.mesos.conftest
toil.batchSystems.mesos.executor
- Package Contents
- Submodules
toil.batchSystems.abstractBatchSystem
- Module Contents
- Classes
- Attributes
logger
EXIT_STATUS_UNAVAILABLE_VALUE
BatchJobExitReason
UpdatedBatchJobInfo
WorkerCleanupInfo
AbstractBatchSystem
AbstractBatchSystem.supportsAutoDeployment()
AbstractBatchSystem.supportsWorkerCleanup()
AbstractBatchSystem.setUserScript()
AbstractBatchSystem.set_message_bus()
AbstractBatchSystem.issueBatchJob()
AbstractBatchSystem.killBatchJobs()
AbstractBatchSystem.getIssuedBatchJobIDs()
AbstractBatchSystem.getRunningBatchJobIDs()
AbstractBatchSystem.getUpdatedBatchJob()
AbstractBatchSystem.getSchedulingStatusMessage()
AbstractBatchSystem.shutdown()
AbstractBatchSystem.setEnv()
AbstractBatchSystem.add_options()
AbstractBatchSystem.setOptions()
AbstractBatchSystem.getWorkerContexts()
BatchSystemSupport
NodeInfo
AbstractScalableBatchSystem
InsufficientSystemResources
AcquisitionTimeoutException
ResourcePool
ResourceSet
- Module Contents
toil.batchSystems.abstractGridEngineBatchSystem
- Module Contents
- Classes
- Attributes
logger
JobTuple
AbstractGridEngineBatchSystem
AbstractGridEngineBatchSystem.Worker
AbstractGridEngineBatchSystem.Worker.getBatchSystemID()
AbstractGridEngineBatchSystem.Worker.forgetJob()
AbstractGridEngineBatchSystem.Worker.createJobs()
AbstractGridEngineBatchSystem.Worker.killJobs()
AbstractGridEngineBatchSystem.Worker.checkOnJobs()
AbstractGridEngineBatchSystem.Worker.run()
AbstractGridEngineBatchSystem.Worker.coalesce_job_exit_codes()
AbstractGridEngineBatchSystem.Worker.prepareSubmission()
AbstractGridEngineBatchSystem.Worker.submitJob()
AbstractGridEngineBatchSystem.Worker.getRunningJobIDs()
AbstractGridEngineBatchSystem.Worker.killJob()
AbstractGridEngineBatchSystem.Worker.getJobExitCode()
AbstractGridEngineBatchSystem.supportsWorkerCleanup()
AbstractGridEngineBatchSystem.supportsAutoDeployment()
AbstractGridEngineBatchSystem.issueBatchJob()
AbstractGridEngineBatchSystem.killBatchJobs()
AbstractGridEngineBatchSystem.getIssuedBatchJobIDs()
AbstractGridEngineBatchSystem.getRunningBatchJobIDs()
AbstractGridEngineBatchSystem.getUpdatedBatchJob()
AbstractGridEngineBatchSystem.shutdown()
AbstractGridEngineBatchSystem.setEnv()
AbstractGridEngineBatchSystem.getWaitDuration()
AbstractGridEngineBatchSystem.sleepSeconds()
AbstractGridEngineBatchSystem.with_retries()
- Module Contents
toil.batchSystems.awsBatch
- Module Contents
- Classes
- Attributes
logger
STATE_TO_EXIT_REASON
MAX_POLL_COUNT
MIN_REQUESTABLE_MIB
MIN_REQUESTABLE_CORES
AWSBatchBatchSystem
AWSBatchBatchSystem.supportsAutoDeployment()
AWSBatchBatchSystem.setUserScript()
AWSBatchBatchSystem.issueBatchJob()
AWSBatchBatchSystem.getUpdatedBatchJob()
AWSBatchBatchSystem.shutdown()
AWSBatchBatchSystem.getIssuedBatchJobIDs()
AWSBatchBatchSystem.getRunningBatchJobIDs()
AWSBatchBatchSystem.killBatchJobs()
AWSBatchBatchSystem.add_options()
AWSBatchBatchSystem.setOptions()
- Module Contents
toil.batchSystems.cleanup_support
toil.batchSystems.contained_executor
toil.batchSystems.gridengine
toil.batchSystems.htcondor
- Module Contents
- Classes
- Attributes
logger
JobTuple
schedd_lock
HTCondorBatchSystem
HTCondorBatchSystem.Worker
HTCondorBatchSystem.Worker.createJobs()
HTCondorBatchSystem.Worker.prepareSubmission()
HTCondorBatchSystem.Worker.submitJob()
HTCondorBatchSystem.Worker.getRunningJobIDs()
HTCondorBatchSystem.Worker.killJob()
HTCondorBatchSystem.Worker.getJobExitCode()
HTCondorBatchSystem.Worker.connectSchedd()
HTCondorBatchSystem.Worker.duplicate_quotes()
HTCondorBatchSystem.Worker.getEnvString()
HTCondorBatchSystem.issueBatchJob()
- Module Contents
toil.batchSystems.kubernetes
- Module Contents
- Classes
- Functions
- Attributes
logger
retryable_kubernetes_errors
is_retryable_kubernetes_error()
KeyValuesList
KubernetesBatchSystem
KubernetesBatchSystem.DecoratorWrapper
KubernetesBatchSystem.Placement
KubernetesBatchSystem.KubernetesConfig
KubernetesBatchSystem.ItemT
KubernetesBatchSystem.CovItemT
KubernetesBatchSystem.P
KubernetesBatchSystem.R
KubernetesBatchSystem.OptionType
KubernetesBatchSystem.supportsAutoDeployment()
KubernetesBatchSystem.setUserScript()
KubernetesBatchSystem.issueBatchJob()
KubernetesBatchSystem.getUpdatedBatchJob()
KubernetesBatchSystem.shutdown()
KubernetesBatchSystem.getIssuedBatchJobIDs()
KubernetesBatchSystem.getRunningBatchJobIDs()
KubernetesBatchSystem.killBatchJobs()
KubernetesBatchSystem.get_default_kubernetes_owner()
KubernetesBatchSystem.add_options()
KubernetesBatchSystem.setOptions()
- Module Contents
toil.batchSystems.local_support
toil.batchSystems.lsf
- Module Contents
- Classes
- Attributes
logger
LSFBatchSystem
LSFBatchSystem.Worker
LSFBatchSystem.Worker.getRunningJobIDs()
LSFBatchSystem.Worker.fallbackRunningJobIDs()
LSFBatchSystem.Worker.killJob()
LSFBatchSystem.Worker.prepareSubmission()
LSFBatchSystem.Worker.submitJob()
LSFBatchSystem.Worker.coalesce_job_exit_codes()
LSFBatchSystem.Worker.getJobExitCode()
LSFBatchSystem.Worker.parse_bjobs_record()
LSFBatchSystem.Worker.getJobExitCodeBACCT()
LSFBatchSystem.Worker.fallbackGetJobExitCode()
LSFBatchSystem.Worker.prepareBsub()
LSFBatchSystem.Worker.parseBjobs()
LSFBatchSystem.Worker.parseMaxMem()
LSFBatchSystem.getWaitDuration()
- Module Contents
toil.batchSystems.lsfHelper
- Module Contents
- Functions
- Attributes
LSB_PARAMS_FILENAME
LSF_CONF_FILENAME
LSF_CONF_ENV
DEFAULT_LSF_UNITS
DEFAULT_RESOURCE_UNITS
LSF_JSON_OUTPUT_MIN_VERSION
logger
find()
find_first_match()
get_conf_file()
apply_conf_file()
per_core_reserve_from_stream()
get_lsf_units_from_stream()
tokenize_conf_stream()
apply_bparams()
apply_lsadmin()
get_lsf_units()
parse_mem_and_cmd_from_output()
get_lsf_version()
check_lsf_json_output_supported()
parse_memory()
per_core_reservation()
- Module Contents
toil.batchSystems.options
toil.batchSystems.parasol
- Module Contents
- Classes
- Attributes
logger
ParasolBatchSystem
ParasolBatchSystem.parasolOutputPattern
ParasolBatchSystem.runningPattern
ParasolBatchSystem.supportsWorkerCleanup()
ParasolBatchSystem.supportsAutoDeployment()
ParasolBatchSystem.issueBatchJob()
ParasolBatchSystem.setEnv()
ParasolBatchSystem.killBatchJobs()
ParasolBatchSystem.getJobIDsForResultsFile()
ParasolBatchSystem.getIssuedBatchJobIDs()
ParasolBatchSystem.getRunningBatchJobIDs()
ParasolBatchSystem.getUpdatedBatchJob()
ParasolBatchSystem.updatedJobWorker()
ParasolBatchSystem.shutdown()
ParasolBatchSystem.add_options()
ParasolBatchSystem.setOptions()
- Module Contents
toil.batchSystems.registry
- Module Contents
- Functions
- Attributes
logger
aws_batch_batch_system_factory()
gridengine_batch_system_factory()
parasol_batch_system_factory()
lsf_batch_system_factory()
single_machine_batch_system_factory()
mesos_batch_system_factory()
slurm_batch_system_factory()
tes_batch_system_factory()
torque_batch_system_factory()
htcondor_batch_system_factory()
kubernetes_batch_system_factory()
BATCH_SYSTEM_FACTORY_REGISTRY
BATCH_SYSTEMS
DEFAULT_BATCH_SYSTEM
addBatchSystemFactory()
save_batch_system_plugin_state()
restore_batch_system_plugin_state()
- Module Contents
toil.batchSystems.singleMachine
- Module Contents
- Classes
- Attributes
logger
SingleMachineBatchSystem
SingleMachineBatchSystem.numCores
SingleMachineBatchSystem.minCores
SingleMachineBatchSystem.physicalMemory
SingleMachineBatchSystem.supportsAutoDeployment()
SingleMachineBatchSystem.supportsWorkerCleanup()
SingleMachineBatchSystem.daddy()
SingleMachineBatchSystem.getSchedulingStatusMessage()
SingleMachineBatchSystem.check_resource_request()
SingleMachineBatchSystem.issueBatchJob()
SingleMachineBatchSystem.killBatchJobs()
SingleMachineBatchSystem.getIssuedBatchJobIDs()
SingleMachineBatchSystem.getRunningBatchJobIDs()
SingleMachineBatchSystem.shutdown()
SingleMachineBatchSystem.getUpdatedBatchJob()
SingleMachineBatchSystem.add_options()
SingleMachineBatchSystem.setOptions()
Info
- Module Contents
toil.batchSystems.slurm
- Module Contents
- Classes
- Attributes
logger
SlurmBatchSystem
SlurmBatchSystem.Worker
SlurmBatchSystem.Worker.getRunningJobIDs()
SlurmBatchSystem.Worker.killJob()
SlurmBatchSystem.Worker.prepareSubmission()
SlurmBatchSystem.Worker.submitJob()
SlurmBatchSystem.Worker.coalesce_job_exit_codes()
SlurmBatchSystem.Worker.getJobExitCode()
SlurmBatchSystem.Worker.prepareSbatch()
SlurmBatchSystem.Worker.parse_elapsed()
SlurmBatchSystem.OptionType
SlurmBatchSystem.add_options()
SlurmBatchSystem.setOptions()
- Module Contents
toil.batchSystems.tes
- Module Contents
- Classes
- Attributes
logger
STATE_TO_EXIT_REASON
TESBatchSystem
TESBatchSystem.supportsAutoDeployment()
TESBatchSystem.get_default_tes_endpoint()
TESBatchSystem.setUserScript()
TESBatchSystem.issueBatchJob()
TESBatchSystem.getUpdatedBatchJob()
TESBatchSystem.shutdown()
TESBatchSystem.getIssuedBatchJobIDs()
TESBatchSystem.getRunningBatchJobIDs()
TESBatchSystem.killBatchJobs()
TESBatchSystem.add_options()
TESBatchSystem.setOptions()
- Module Contents
toil.batchSystems.torque
- Module Contents
- Classes
- Attributes
logger
TorqueBatchSystem
TorqueBatchSystem.Worker
TorqueBatchSystem.Worker.getRunningJobIDs()
TorqueBatchSystem.Worker.getUpdatedBatchJob()
TorqueBatchSystem.Worker.killJob()
TorqueBatchSystem.Worker.prepareSubmission()
TorqueBatchSystem.Worker.submitJob()
TorqueBatchSystem.Worker.getJobExitCode()
TorqueBatchSystem.Worker.prepareQsub()
TorqueBatchSystem.Worker.generateTorqueWrapper()
- Module Contents
- Package Contents
- Subpackages
toil.cwl
- Submodules
toil.cwl.conftest
toil.cwl.cwltoil
- Module Contents
- Classes
- Functions
- Attributes
logger
DEFAULT_TMPDIR
DEFAULT_TMPDIR_PREFIX
cwltoil_was_removed()
UnresolvedDict
SkipNull
filter_skip_null()
ensure_no_collisions()
Conditional
ResolveSource
StepValueFrom
DefaultWithSource
JustAValue
resolve_dict_w_promises()
simplify_list()
ToilPathMapper
ToilSingleJobExecutor
ToilTool
ToilCommandLineTool
ToilExpressionTool
toil_make_tool()
DirectoryContents
check_directory_dict_invariants()
decode_directory()
encode_directory()
ToilFsAccess
toil_get_file()
write_file()
path_to_loc()
import_files()
upload_directory()
upload_file()
writeGlobalFileWrapper()
remove_empty_listings()
CWLNamedJob
ResolveIndirect
toilStageFiles()
CWLJobWrapper
CWLJob
get_container_engine()
makeJob()
CWLScatter
CWLGather
SelfJob
ProcessType
remove_pickle_problems()
CWLWorkflow
visitSteps()
rm_unprocessed_secondary_files()
filtered_secondary_files()
scan_for_unsupported_requirements()
determine_load_listing()
NoAvailableJobStoreException
generate_default_job_store()
usage_message
main()
find_default_container()
- Module Contents
toil.cwl.utils
- Package Contents
- Submodules
toil.fileStores
- Submodules
toil.fileStores.abstractFileStore
- Module Contents
- Classes
- Attributes
logger
AbstractFileStore
AbstractFileStore.createFileStore()
AbstractFileStore.shutdownFileStore()
AbstractFileStore.open()
AbstractFileStore.getLocalTempDir()
AbstractFileStore.getLocalTempFile()
AbstractFileStore.getLocalTempFileName()
AbstractFileStore.writeGlobalFile()
AbstractFileStore.writeGlobalFileStream()
AbstractFileStore.logAccess()
AbstractFileStore.readGlobalFile()
AbstractFileStore.readGlobalFileStream()
AbstractFileStore.getGlobalFileSize()
AbstractFileStore.deleteLocalFile()
AbstractFileStore.deleteGlobalFile()
AbstractFileStore.importFile()
AbstractFileStore.import_file()
AbstractFileStore.exportFile()
AbstractFileStore.export_file()
AbstractFileStore.logToMaster()
AbstractFileStore.startCommit()
AbstractFileStore.waitForCommit()
AbstractFileStore.shutdown()
- Module Contents
toil.fileStores.cachingFileStore
- Module Contents
- Classes
- Attributes
logger
SQLITE_TIMEOUT_SECS
CacheError
CacheUnbalancedError
IllegalDeletionCacheError
InvalidSourceCacheError
CachingFileStore
CachingFileStore.getCacheLimit()
CachingFileStore.getCacheUsed()
CachingFileStore.getCacheExtraJobSpace()
CachingFileStore.getCacheAvailable()
CachingFileStore.getSpaceUsableForJobs()
CachingFileStore.getCacheUnusedJobRequirement()
CachingFileStore.adjustCacheLimit()
CachingFileStore.fileIsCached()
CachingFileStore.getFileReaderCount()
CachingFileStore.cachingIsFree()
CachingFileStore.open()
CachingFileStore.writeGlobalFile()
CachingFileStore.readGlobalFile()
CachingFileStore.readGlobalFileStream()
CachingFileStore.deleteLocalFile()
CachingFileStore.deleteGlobalFile()
CachingFileStore.exportFile()
CachingFileStore.export_file()
CachingFileStore.waitForCommit()
CachingFileStore.startCommit()
CachingFileStore.startCommitThread()
CachingFileStore.shutdown()
CachingFileStore.__del__()
- Module Contents
toil.fileStores.nonCachingFileStore
- Module Contents
- Classes
- Attributes
logger
NonCachingFileStore
NonCachingFileStore.check_for_coordination_corruption()
NonCachingFileStore.check_for_state_corruption()
NonCachingFileStore.open()
NonCachingFileStore.writeGlobalFile()
NonCachingFileStore.readGlobalFile()
NonCachingFileStore.readGlobalFileStream()
NonCachingFileStore.exportFile()
NonCachingFileStore.export_file()
NonCachingFileStore.deleteLocalFile()
NonCachingFileStore.deleteGlobalFile()
NonCachingFileStore.waitForCommit()
NonCachingFileStore.startCommit()
NonCachingFileStore.__del__()
NonCachingFileStore.shutdown()
- Module Contents
- Package Contents
- Submodules
toil.jobStores
- Subpackages
toil.jobStores.aws
- Submodules
toil.jobStores.aws.jobStore
- Module Contents
- Classes
- Attributes
boto3_session
s3_boto3_resource
s3_boto3_client
logger
CONSISTENCY_TICKS
CONSISTENCY_TIME
ChecksumError
AWSJobStore
AWSJobStore.FileInfo
AWSJobStore.FileInfo.fileID
AWSJobStore.FileInfo.ownerID
AWSJobStore.FileInfo.version
AWSJobStore.FileInfo.previousVersion
AWSJobStore.FileInfo.content
AWSJobStore.FileInfo.checksum
AWSJobStore.FileInfo.outer
AWSJobStore.FileInfo.create()
AWSJobStore.FileInfo.presenceIndicator()
AWSJobStore.FileInfo.exists()
AWSJobStore.FileInfo.load()
AWSJobStore.FileInfo.loadOrCreate()
AWSJobStore.FileInfo.loadOrFail()
AWSJobStore.FileInfo.fromItem()
AWSJobStore.FileInfo.toItem()
AWSJobStore.FileInfo.maxInlinedSize()
AWSJobStore.FileInfo.save()
AWSJobStore.FileInfo.upload()
AWSJobStore.FileInfo.uploadStream()
AWSJobStore.FileInfo.copyFrom()
AWSJobStore.FileInfo.copyTo()
AWSJobStore.FileInfo.download()
AWSJobStore.FileInfo.downloadStream()
AWSJobStore.FileInfo.delete()
AWSJobStore.FileInfo.getSize()
AWSJobStore.FileInfo.__repr__()
AWSJobStore.sseKeyPath
AWSJobStore.bucketNameRe
AWSJobStore.minBucketNameLen
AWSJobStore.maxBucketNameLen
AWSJobStore.maxNameLen
AWSJobStore.nameSeparator
AWSJobStore.jobsPerBatchInsert
AWSJobStore.itemsPerBatchDelete
AWSJobStore.sharedFileOwnerID
AWSJobStore.statsFileOwnerID
AWSJobStore.readStatsFileOwnerID
AWSJobStore.versionings
AWSJobStore.initialize()
AWSJobStore.resume()
AWSJobStore.batch()
AWSJobStore.assign_job_id()
AWSJobStore.create_job()
AWSJobStore.job_exists()
AWSJobStore.jobs()
AWSJobStore.load_job()
AWSJobStore.update_job()
AWSJobStore.delete_job()
AWSJobStore.get_empty_file_store_id()
AWSJobStore.get_size()
AWSJobStore.write_file()
AWSJobStore.write_file_stream()
AWSJobStore.write_shared_file_stream()
AWSJobStore.update_file()
AWSJobStore.update_file_stream()
AWSJobStore.file_exists()
AWSJobStore.get_file_size()
AWSJobStore.read_file()
AWSJobStore.read_file_stream()
AWSJobStore.read_shared_file_stream()
AWSJobStore.delete_file()
AWSJobStore.write_logs()
AWSJobStore.read_logs()
AWSJobStore.get_public_url()
AWSJobStore.get_shared_public_url()
AWSJobStore.destroy()
aRepr
custom_repr
BucketLocationConflictException
- Module Contents
toil.jobStores.aws.utils
- Module Contents
- Submodules
- Submodules
toil.jobStores.abstractJobStore
- Module Contents
- Classes
- Attributes
logger
ProxyConnectionError
InvalidImportExportUrlException
UnimplementedURLException
NoSuchJobException
ConcurrentFileModificationException
NoSuchFileException
NoSuchJobStoreException
JobStoreExistsException
AbstractJobStore
AbstractJobStore.config
AbstractJobStore.locator
AbstractJobStore.rootJobStoreIDFileName
AbstractJobStore.publicUrlExpiration
AbstractJobStore.sharedFileNameRegex
AbstractJobStore.initialize()
AbstractJobStore.writeConfig()
AbstractJobStore.write_config()
AbstractJobStore.resume()
AbstractJobStore.setRootJob()
AbstractJobStore.set_root_job()
AbstractJobStore.loadRootJob()
AbstractJobStore.load_root_job()
AbstractJobStore.createRootJob()
AbstractJobStore.create_root_job()
AbstractJobStore.getRootJobReturnValue()
AbstractJobStore.get_root_job_return_value()
AbstractJobStore.importFile()
AbstractJobStore.import_file()
AbstractJobStore.exportFile()
AbstractJobStore.export_file()
AbstractJobStore.list_url()
AbstractJobStore.get_is_directory()
AbstractJobStore.read_from_url()
AbstractJobStore.getSize()
AbstractJobStore.get_size()
AbstractJobStore.destroy()
AbstractJobStore.getEnv()
AbstractJobStore.get_env()
AbstractJobStore.clean()
AbstractJobStore.assignID()
AbstractJobStore.assign_job_id()
AbstractJobStore.batch()
AbstractJobStore.create()
AbstractJobStore.create_job()
AbstractJobStore.exists()
AbstractJobStore.job_exists()
AbstractJobStore.getPublicUrl()
AbstractJobStore.get_public_url()
AbstractJobStore.getSharedPublicUrl()
AbstractJobStore.get_shared_public_url()
AbstractJobStore.load()
AbstractJobStore.load_job()
AbstractJobStore.update()
AbstractJobStore.update_job()
AbstractJobStore.delete()
AbstractJobStore.delete_job()
AbstractJobStore.jobs()
AbstractJobStore.writeFile()
AbstractJobStore.write_file()
AbstractJobStore.writeFileStream()
AbstractJobStore.write_file_stream()
AbstractJobStore.getEmptyFileStoreID()
AbstractJobStore.get_empty_file_store_id()
AbstractJobStore.readFile()
AbstractJobStore.read_file()
AbstractJobStore.readFileStream()
AbstractJobStore.read_file_stream()
AbstractJobStore.deleteFile()
AbstractJobStore.delete_file()
AbstractJobStore.fileExists()
AbstractJobStore.file_exists()
AbstractJobStore.getFileSize()
AbstractJobStore.get_file_size()
AbstractJobStore.updateFile()
AbstractJobStore.update_file()
AbstractJobStore.updateFileStream()
AbstractJobStore.update_file_stream()
AbstractJobStore.writeSharedFileStream()
AbstractJobStore.write_shared_file_stream()
AbstractJobStore.readSharedFileStream()
AbstractJobStore.read_shared_file_stream()
AbstractJobStore.writeStatsAndLogging()
AbstractJobStore.write_logs()
AbstractJobStore.readStatsAndLogging()
AbstractJobStore.read_logs()
AbstractJobStore.write_leader_pid()
AbstractJobStore.read_leader_pid()
AbstractJobStore.write_leader_node_id()
AbstractJobStore.read_leader_node_id()
AbstractJobStore.write_kill_flag()
AbstractJobStore.read_kill_flag()
AbstractJobStore.default_caching()
JobStoreSupport
- Module Contents
toil.jobStores.conftest
toil.jobStores.fileJobStore
- Module Contents
- Classes
- Attributes
logger
FileJobStore
FileJobStore.validDirs
FileJobStore.validDirsSet
FileJobStore.JOB_DIR_PREFIX
FileJobStore.JOB_NAME_DIR_PREFIX
FileJobStore.BUFFER_SIZE
FileJobStore.default_caching()
FileJobStore.__repr__()
FileJobStore.initialize()
FileJobStore.resume()
FileJobStore.destroy()
FileJobStore.assign_job_id()
FileJobStore.create_job()
FileJobStore.batch()
FileJobStore.job_exists()
FileJobStore.get_public_url()
FileJobStore.get_shared_public_url()
FileJobStore.load_job()
FileJobStore.update_job()
FileJobStore.delete_job()
FileJobStore.jobs()
FileJobStore.optional_hard_copy()
FileJobStore.get_size()
FileJobStore.write_file()
FileJobStore.write_file_stream()
FileJobStore.get_empty_file_store_id()
FileJobStore.update_file()
FileJobStore.read_file()
FileJobStore.delete_file()
FileJobStore.file_exists()
FileJobStore.get_file_size()
FileJobStore.update_file_stream()
FileJobStore.read_file_stream()
FileJobStore.write_shared_file_stream()
FileJobStore.read_shared_file_stream()
FileJobStore.write_logs()
FileJobStore.read_logs()
- Module Contents
toil.jobStores.googleJobStore
- Module Contents
- Classes
- Functions
- Attributes
log
GOOGLE_STORAGE
MAX_BATCH_SIZE
google_retry_predicate()
google_retry()
GoogleJobStore
GoogleJobStore.nodeServiceAccountJson
GoogleJobStore.initialize()
GoogleJobStore.resume()
GoogleJobStore.destroy()
GoogleJobStore.assign_job_id()
GoogleJobStore.batch()
GoogleJobStore.create_job()
GoogleJobStore.job_exists()
GoogleJobStore.get_public_url()
GoogleJobStore.get_shared_public_url()
GoogleJobStore.load_job()
GoogleJobStore.update_job()
GoogleJobStore.delete_job()
GoogleJobStore.get_env()
GoogleJobStore.jobs()
GoogleJobStore.write_file()
GoogleJobStore.write_file_stream()
GoogleJobStore.get_empty_file_store_id()
GoogleJobStore.read_file()
GoogleJobStore.read_file_stream()
GoogleJobStore.delete_file()
GoogleJobStore.file_exists()
GoogleJobStore.get_file_size()
GoogleJobStore.update_file()
GoogleJobStore.update_file_stream()
GoogleJobStore.write_shared_file_stream()
GoogleJobStore.read_shared_file_stream()
GoogleJobStore.get_size()
GoogleJobStore.write_logs()
GoogleJobStore.read_logs()
- Module Contents
toil.jobStores.utils
- Subpackages
toil.lib
- Subpackages
toil.lib.aws
- Submodules
toil.lib.aws.ami
toil.lib.aws.iam
- Module Contents
- Functions
- Attributes
logger
CLUSTER_LAUNCHING_PERMISSIONS
AllowedActionCollection
init_action_collection()
add_to_action_collection()
policy_permissions_allow()
permission_matches_any()
get_actions_from_policy_document()
allowed_actions_attached()
allowed_actions_roles()
allowed_actions_users()
get_policy_permissions()
get_aws_account_num()
- Module Contents
toil.lib.aws.session
toil.lib.aws.utils
- Module Contents
- Functions
- Attributes
BotoServerError
logger
THROTTLED_ERROR_CODES
delete_iam_role()
delete_iam_instance_profile()
delete_sdb_domain()
connection_reset()
retryable_s3_errors()
retry_s3()
delete_s3_bucket()
create_s3_bucket()
enable_public_objects()
get_bucket_region()
region_to_bucket_location()
bucket_location_to_region()
get_object_for_url()
list_objects_for_url()
flatten_tags()
- Module Contents
- Package Contents
- Submodules
toil.lib.encryption
- Submodules
toil.lib.accelerators
toil.lib.bioio
toil.lib.compatibility
toil.lib.conversions
toil.lib.docker
toil.lib.ec2
- Module Contents
- Functions
- Attributes
a_short_time
a_long_time
logger
UserError
not_found()
inconsistencies_detected()
INCONSISTENCY_ERRORS
retry_ec2()
UnexpectedResourceState
wait_transition()
wait_instances_running()
wait_spot_requests_active()
create_spot_instances()
create_ondemand_instances()
prune()
iam_client
wait_until_instance_profile_arn_exists()
create_instances()
create_launch_template()
create_auto_scaling_group()
- Module Contents
toil.lib.ec2nodes
toil.lib.exceptions
toil.lib.expando
toil.lib.generatedEC2Lists
toil.lib.humanize
toil.lib.io
toil.lib.iterables
toil.lib.memoize
toil.lib.misc
toil.lib.objects
toil.lib.resources
toil.lib.retry
- Module Contents
- Classes
- Functions
- Attributes
SUPPORTED_HTTP_ERRORS
kubernetes
botocore
logger
ErrorCondition
retry()
return_status_code()
get_error_code()
get_error_message()
get_error_status()
get_error_body()
meets_error_message_condition()
meets_error_code_condition()
meets_boto_error_code_condition()
error_meets_conditions()
DEFAULT_DELAYS
DEFAULT_TIMEOUT
old_retry()
retry_flaky_test
- Module Contents
toil.lib.threading
toil.lib.throttle
- Subpackages
toil.provisioners
- Subpackages
toil.provisioners.aws
- Submodules
toil.provisioners.aws.awsProvisioner
- Module Contents
- Classes
- Functions
- Attributes
logger
awsRetryPredicate()
expectedShutdownErrors()
awsRetry()
awsFilterImpairedNodes()
InvalidClusterStateException
AWSProvisioner
AWSProvisioner.supportedClusterTypes()
AWSProvisioner.createClusterSettings()
AWSProvisioner.readClusterSettings()
AWSProvisioner.launchCluster()
AWSProvisioner.toil_service_env_options()
AWSProvisioner.getKubernetesAutoscalerSetupCommands()
AWSProvisioner.getKubernetesCloudProvider()
AWSProvisioner.getNodeShape()
AWSProvisioner.retryPredicate()
AWSProvisioner.destroyCluster()
AWSProvisioner.terminateNodes()
AWSProvisioner.addNodes()
AWSProvisioner.addManagedNodes()
AWSProvisioner.getProvisionedWorkers()
AWSProvisioner.getLeader()
AWSProvisioner.full_policy()
AWSProvisioner.kubernetes_policy()
- Module Contents
- Package Contents
- Submodules
- Submodules
toil.provisioners.abstractProvisioner
- Module Contents
- Classes
- Attributes
a_short_time
logger
ManagedNodesNotSupportedException
Shape
AbstractProvisioner
AbstractProvisioner.InstanceConfiguration
AbstractProvisioner.LEADER_HOME_DIR
AbstractProvisioner.cloud
AbstractProvisioner.supportedClusterTypes()
AbstractProvisioner.createClusterSettings()
AbstractProvisioner.readClusterSettings()
AbstractProvisioner.setAutoscaledNodeTypes()
AbstractProvisioner.hasAutoscaledNodeTypes()
AbstractProvisioner.getAutoscaledInstanceShapes()
AbstractProvisioner.retryPredicate()
AbstractProvisioner.launchCluster()
AbstractProvisioner.addNodes()
AbstractProvisioner.addManagedNodes()
AbstractProvisioner.terminateNodes()
AbstractProvisioner.getLeader()
AbstractProvisioner.getProvisionedWorkers()
AbstractProvisioner.getNodeShape()
AbstractProvisioner.destroyCluster()
AbstractProvisioner.getBaseInstanceConfiguration()
AbstractProvisioner.addVolumesService()
AbstractProvisioner.addNodeExporterService()
AbstractProvisioner.toil_service_env_options()
AbstractProvisioner.add_toil_service()
AbstractProvisioner.getKubernetesValues()
AbstractProvisioner.addKubernetesServices()
AbstractProvisioner.getKubernetesAutoscalerSetupCommands()
AbstractProvisioner.getKubernetesCloudProvider()
AbstractProvisioner.addKubernetesLeader()
AbstractProvisioner.addKubernetesWorker()
- Module Contents
toil.provisioners.clusterScaler
- Module Contents
- Classes
- Functions
- Attributes
logger
EVICTION_THRESHOLD
RESERVE_SMALL_LIMIT
RESERVE_SMALL_AMOUNT
RESERVE_BREAKPOINTS
RESERVE_FRACTIONS
OS_SIZE
FailedConstraint
BinPackedFit
NodeReservation
adjustEndingReservationForJob()
split()
binPacking()
ClusterScaler
ClusterScaler.getAverageRuntime()
ClusterScaler.addCompletedJob()
ClusterScaler.setStaticNodes()
ClusterScaler.getStaticNodes()
ClusterScaler.smoothEstimate()
ClusterScaler.getEstimatedNodeCounts()
ClusterScaler.updateClusterSize()
ClusterScaler.setNodeCount()
ClusterScaler.filter_out_static_nodes()
ClusterScaler.getNodes()
ClusterScaler.shutDown()
JobTooBigError
ScalerThread
ClusterStats
- Module Contents
toil.provisioners.gceProvisioner
- Module Contents
- Classes
- Attributes
logger
GCEProvisioner
GCEProvisioner.NODE_BOTO_PATH
GCEProvisioner.SOURCE_IMAGE
GCEProvisioner.DEFAULT_TASK_COMPLETION_TIMEOUT
GCEProvisioner.supportedClusterTypes()
GCEProvisioner.createClusterSettings()
GCEProvisioner.readClusterSettings()
GCEProvisioner.launchCluster()
GCEProvisioner.getNodeShape()
GCEProvisioner.retryPredicate()
GCEProvisioner.destroyCluster()
GCEProvisioner.terminateNodes()
GCEProvisioner.addNodes()
GCEProvisioner.getProvisionedWorkers()
GCEProvisioner.getLeader()
GCEProvisioner.ex_create_multiple_nodes()
- Module Contents
toil.provisioners.node
- Package Contents
- Subpackages
toil.server
- Subpackages
toil.server.api_spec
toil.server.cli
toil.server.wes
- Submodules
toil.server.wes.abstract_backend
toil.server.wes.amazon_wes_utils
toil.server.wes.tasks
toil.server.wes.toil_backend
- Module Contents
- Classes
- Attributes
logger
ToilWorkflow
ToilWorkflow.fetch_state()
ToilWorkflow.fetch_scratch()
ToilWorkflow.exists()
ToilWorkflow.get_state()
ToilWorkflow.check_on_run()
ToilWorkflow.set_up_run()
ToilWorkflow.clean_up()
ToilWorkflow.queue_run()
ToilWorkflow.get_output_files()
ToilWorkflow.get_stdout_path()
ToilWorkflow.get_stderr_path()
ToilWorkflow.get_messages_path()
ToilWorkflow.get_task_logs()
ToilBackend
ToilBackend.get_runs()
ToilBackend.get_state()
ToilBackend.get_service_info()
ToilBackend.list_runs()
ToilBackend.run_workflow()
ToilBackend.get_run_log()
ToilBackend.cancel_run()
ToilBackend.get_run_status()
ToilBackend.get_stdout()
ToilBackend.get_stderr()
ToilBackend.get_health()
ToilBackend.get_homepage()
- Module Contents
- Submodules
- Submodules
toil.server.app
toil.server.celery_app
toil.server.utils
- Module Contents
- Classes
- Functions
- Attributes
HAVE_S3
logger
get_iso_time()
link_file()
download_file_from_internet()
download_file_from_s3()
get_file_class()
safe_read_file()
safe_write_file()
MemoryStateCache
AbstractStateStore
MemoryStateStore
FileStateStore
S3StateStore
state_store_cache
connect_to_state_store()
WorkflowStateStore
connect_to_workflow_state_store()
TERMINAL_STATES
MAX_CANCELING_SECONDS
WorkflowStateMachine
WorkflowStateMachine.send_enqueue()
WorkflowStateMachine.send_initialize()
WorkflowStateMachine.send_run()
WorkflowStateMachine.send_cancel()
WorkflowStateMachine.send_canceled()
WorkflowStateMachine.send_complete()
WorkflowStateMachine.send_executor_error()
WorkflowStateMachine.send_system_error()
WorkflowStateMachine.get_current_state()
- Module Contents
toil.server.wsgi_app
- Subpackages
toil.test
- Subpackages
toil.test.batchSystems
- Submodules
toil.test.batchSystems.batchSystemTest
- Module Contents
- Classes
- Functions
- Attributes
logger
numCores
preemptible
defaultRequirements
BatchSystemPluginTest
hidden
hidden.AbstractBatchSystemTest
hidden.AbstractBatchSystemTest.createBatchSystem()
hidden.AbstractBatchSystemTest.supportsWallTime()
hidden.AbstractBatchSystemTest.createConfig()
hidden.AbstractBatchSystemTest.setUpClass()
hidden.AbstractBatchSystemTest.setUp()
hidden.AbstractBatchSystemTest.tearDown()
hidden.AbstractBatchSystemTest.get_max_startup_seconds()
hidden.AbstractBatchSystemTest.test_available_cores()
hidden.AbstractBatchSystemTest.test_run_jobs()
hidden.AbstractBatchSystemTest.test_set_env()
hidden.AbstractBatchSystemTest.test_set_job_env()
hidden.AbstractBatchSystemTest.testCheckResourceRequest()
hidden.AbstractBatchSystemTest.testScalableBatchSystem()
hidden.AbstractBatchSystemJobTest
hidden.AbstractBatchSystemJobTest.cpuCount
hidden.AbstractBatchSystemJobTest.allocatedCores
hidden.AbstractBatchSystemJobTest.sleepTime
hidden.AbstractBatchSystemJobTest.getBatchSystemName()
hidden.AbstractBatchSystemJobTest.getOptions()
hidden.AbstractBatchSystemJobTest.setUp()
hidden.AbstractBatchSystemJobTest.tearDown()
hidden.AbstractBatchSystemJobTest.testJobConcurrency()
hidden.AbstractBatchSystemJobTest.test_omp_threads()
hidden.AbstractGridEngineBatchSystemTest
KubernetesBatchSystemTest
KubernetesBatchSystemBenchTest
TESBatchSystemTest
AWSBatchBatchSystemTest
MesosBatchSystemTest
write_temp_file()
SingleMachineBatchSystemTest
MaxCoresSingleMachineBatchSystemTest
parentJob()
childJob()
grandChildJob()
greatGrandChild()
Service
ParasolBatchSystemTest
GridEngineBatchSystemTest
SlurmBatchSystemTest
LSFBatchSystemTest
TorqueBatchSystemTest
HTCondorBatchSystemTest
SingleMachineBatchSystemJobTest
MesosBatchSystemJobTest
measureConcurrency()
count()
getCounters()
resetCounters()
get_omp_threads()
- Module Contents
toil.test.batchSystems.parasolTestSupport
toil.test.batchSystems.test_lsf_helper
toil.test.batchSystems.test_slurm
- Module Contents
- Classes
- Functions
call_sacct()
call_scontrol()
call_sacct_raises()
FakeBatchSystem
SlurmTest
SlurmTest.setUp()
SlurmTest.test_getJobDetailsFromSacct_one_exists()
SlurmTest.test_getJobDetailsFromSacct_one_not_exists()
SlurmTest.test_getJobDetailsFromSacct_many_all_exist()
SlurmTest.test_getJobDetailsFromSacct_many_some_exist()
SlurmTest.test_getJobDetailsFromSacct_many_none_exist()
SlurmTest.test_getJobDetailsFromScontrol_one_exists()
SlurmTest.test_getJobDetailsFromScontrol_one_not_exists()
SlurmTest.test_getJobDetailsFromScontrol_many_all_exist()
SlurmTest.test_getJobDetailsFromScontrol_many_some_exist()
SlurmTest.test_getJobDetailsFromScontrol_many_none_exist()
SlurmTest.test_getJobExitCode_job_exists()
SlurmTest.test_getJobExitCode_job_not_exists()
SlurmTest.test_getJobExitCode_sacct_raises_job_exists()
SlurmTest.test_getJobExitCode_sacct_raises_job_not_exists()
SlurmTest.test_coalesce_job_exit_codes_one_exists()
SlurmTest.test_coalesce_job_exit_codes_one_not_exists()
SlurmTest.test_coalesce_job_exit_codes_many_all_exist()
SlurmTest.test_coalesce_job_exit_codes_some_exists()
SlurmTest.test_coalesce_job_exit_codes_sacct_raises_job_exists()
SlurmTest.test_coalesce_job_exit_codes_sacct_raises_job_not_exists()
- Module Contents
- Submodules
toil.test.cwl
- Submodules
toil.test.cwl.conftest
toil.test.cwl.cwlTest
- Module Contents
- Classes
- Functions
- Attributes
pkg_root
log
CONFORMANCE_TEST_TIMEOUT
run_conformance_tests()
CWLWorkflowTest
CWLWorkflowTest.setUp()
CWLWorkflowTest.tearDown()
CWLWorkflowTest.revsort()
CWLWorkflowTest.revsort_no_checksum()
CWLWorkflowTest.download()
CWLWorkflowTest.load_contents()
CWLWorkflowTest.download_directory()
CWLWorkflowTest.download_subdirectory()
CWLWorkflowTest.test_mpi()
CWLWorkflowTest.test_s3_as_secondary_file()
CWLWorkflowTest.test_run_revsort()
CWLWorkflowTest.test_run_revsort_nochecksum()
CWLWorkflowTest.test_run_revsort2()
CWLWorkflowTest.test_run_revsort_debug_worker()
CWLWorkflowTest.test_run_colon_output()
CWLWorkflowTest.test_download_s3()
CWLWorkflowTest.test_download_http()
CWLWorkflowTest.test_download_https()
CWLWorkflowTest.test_download_file()
CWLWorkflowTest.test_download_directory_s3()
CWLWorkflowTest.test_download_directory_file()
CWLWorkflowTest.test_download_subdirectory_s3()
CWLWorkflowTest.test_download_subdirectory_file()
CWLWorkflowTest.test_load_contents_s3()
CWLWorkflowTest.test_load_contents_http()
CWLWorkflowTest.test_load_contents_https()
CWLWorkflowTest.test_load_contents_file()
CWLWorkflowTest.test_bioconda()
CWLWorkflowTest.test_biocontainers()
CWLWorkflowTest.test_cuda()
CWLWorkflowTest.test_restart()
CWLWorkflowTest.test_streamable()
CWLv10Test
CWLv10Test.setUp()
CWLv10Test.tearDown()
CWLv10Test.test_run_conformance_with_caching()
CWLv10Test.test_run_conformance()
CWLv10Test.test_lsf_cwl_conformance()
CWLv10Test.test_slurm_cwl_conformance()
CWLv10Test.test_torque_cwl_conformance()
CWLv10Test.test_gridengine_cwl_conformance()
CWLv10Test.test_mesos_cwl_conformance()
CWLv10Test.test_parasol_cwl_conformance()
CWLv10Test.test_kubernetes_cwl_conformance()
CWLv10Test.test_lsf_cwl_conformance_with_caching()
CWLv10Test.test_slurm_cwl_conformance_with_caching()
CWLv10Test.test_torque_cwl_conformance_with_caching()
CWLv10Test.test_gridengine_cwl_conformance_with_caching()
CWLv10Test.test_mesos_cwl_conformance_with_caching()
CWLv10Test.test_parasol_cwl_conformance_with_caching()
CWLv10Test.test_kubernetes_cwl_conformance_with_caching()
CWLv11Test
CWLv12Test
CWLv12Test.setUpClass()
CWLv12Test.tearDown()
CWLv12Test.test_run_conformance()
CWLv12Test.test_run_conformance_with_caching()
CWLv12Test.test_run_conformance_with_in_place_update()
CWLv12Test.test_kubernetes_cwl_conformance()
CWLv12Test.test_kubernetes_cwl_conformance_with_caching()
CWLv12Test.test_wes_server_cwl_conformance()
CWLOnARMTest
test_workflow_echo_string_scatter_stderr_log_dir()
test_log_dir_echo_no_output()
test_log_dir_echo_stderr()
test_filename_conflict_resolution()
test_filename_conflict_detection()
test_filename_conflict_detection_at_root()
test_pick_value_with_one_null_value()
test_usage_message()
test_workflow_echo_string()
test_workflow_echo_string_scatter_capture_stdout()
test_visit_top_cwl_class()
test_visit_cwl_class_and_reduce()
test_download_structure()
- Module Contents
- Submodules
toil.test.docs
- Submodules
toil.test.docs.scriptsTest
- Module Contents
- Classes
- Attributes
pkg_root
ToilDocumentationTest
ToilDocumentationTest.setUpClass()
ToilDocumentationTest.tearDown()
ToilDocumentationTest.checkExitCode()
ToilDocumentationTest.checkExpectedOut()
ToilDocumentationTest.checkExpectedPattern()
ToilDocumentationTest.testCwlexample()
ToilDocumentationTest.testDiscoverfiles()
ToilDocumentationTest.testDynamic()
ToilDocumentationTest.testEncapsulation()
ToilDocumentationTest.testEncapsulation2()
ToilDocumentationTest.testHelloworld()
ToilDocumentationTest.testInvokeworkflow()
ToilDocumentationTest.testInvokeworkflow2()
ToilDocumentationTest.testJobFunctions()
ToilDocumentationTest.testManaging()
ToilDocumentationTest.testManaging2()
ToilDocumentationTest.testMultiplejobs()
ToilDocumentationTest.testMultiplejobs2()
ToilDocumentationTest.testMultiplejobs3()
ToilDocumentationTest.testPromises2()
ToilDocumentationTest.testQuickstart()
ToilDocumentationTest.testRequirements()
ToilDocumentationTest.testArguments()
ToilDocumentationTest.testDocker()
ToilDocumentationTest.testPromises()
ToilDocumentationTest.testServices()
ToilDocumentationTest.testStaging()
- Module Contents
- Submodules
toil.test.jobStores
- Submodules
toil.test.jobStores.jobStoreTest
- Module Contents
- Classes
- Functions
- Attributes
google_retry()
logger
tearDownModule()
AbstractJobStoreTest
AbstractJobStoreTest.Test
AbstractJobStoreTest.Test.externalStoreCache
AbstractJobStoreTest.Test.mpTestPartSize
AbstractJobStoreTest.Test.setUpClass()
AbstractJobStoreTest.Test.setUp()
AbstractJobStoreTest.Test.tearDown()
AbstractJobStoreTest.Test.testInitialState()
AbstractJobStoreTest.Test.testJobCreation()
AbstractJobStoreTest.Test.testConfigEquality()
AbstractJobStoreTest.Test.testJobLoadEquality()
AbstractJobStoreTest.Test.testChildLoadingEquality()
AbstractJobStoreTest.Test.testPersistantFilesToDelete()
AbstractJobStoreTest.Test.testUpdateBehavior()
AbstractJobStoreTest.Test.testJobDeletions()
AbstractJobStoreTest.Test.testSharedFiles()
AbstractJobStoreTest.Test.testReadWriteSharedFilesTextMode()
AbstractJobStoreTest.Test.testReadWriteFileStreamTextMode()
AbstractJobStoreTest.Test.testPerJobFiles()
AbstractJobStoreTest.Test.testStatsAndLogging()
AbstractJobStoreTest.Test.testWriteLogFiles()
AbstractJobStoreTest.Test.testBatchCreate()
AbstractJobStoreTest.Test.testGrowingAndShrinkingJob()
AbstractJobStoreTest.Test.cleanUpExternalStores()
AbstractJobStoreTest.Test.makeImportExportTests()
AbstractJobStoreTest.Test.testImportHttpFile()
AbstractJobStoreTest.Test.testImportFtpFile()
AbstractJobStoreTest.Test.testFileDeletion()
AbstractJobStoreTest.Test.testMultipartUploads()
AbstractJobStoreTest.Test.testZeroLengthFiles()
AbstractJobStoreTest.Test.testLargeFile()
AbstractJobStoreTest.Test.fetch_url()
AbstractJobStoreTest.Test.assertUrl()
AbstractJobStoreTest.Test.testCleanCache()
AbstractJobStoreTest.Test.testPartialReadFromStream()
AbstractJobStoreTest.Test.testDestructionOfCorruptedJobStore()
AbstractJobStoreTest.Test.testDestructionIdempotence()
AbstractJobStoreTest.Test.testEmptyFileStoreIDIsReadable()
AbstractEncryptedJobStoreTest
FileJobStoreTest
GoogleJobStoreTest
AWSJobStoreTest
InvalidAWSJobStoreTest
EncryptedAWSJobStoreTest
StubHttpRequestHandler
- Module Contents
- Submodules
toil.test.lib
- Subpackages
- Submodules
toil.test.lib.dockerTest
- Module Contents
- Classes
- Attributes
logger
DockerTest
DockerTest.setUp()
DockerTest.testDockerClean()
DockerTest.testDockerClean_CRx_FORGO()
DockerTest.testDockerClean_CRx_STOP()
DockerTest.testDockerClean_CRx_RM()
DockerTest.testDockerClean_CRx_None()
DockerTest.testDockerClean_CxD_FORGO()
DockerTest.testDockerClean_CxD_STOP()
DockerTest.testDockerClean_CxD_RM()
DockerTest.testDockerClean_CxD_None()
DockerTest.testDockerClean_Cxx_FORGO()
DockerTest.testDockerClean_Cxx_STOP()
DockerTest.testDockerClean_Cxx_RM()
DockerTest.testDockerClean_Cxx_None()
DockerTest.testDockerClean_xRx_FORGO()
DockerTest.testDockerClean_xRx_STOP()
DockerTest.testDockerClean_xRx_RM()
DockerTest.testDockerClean_xRx_None()
DockerTest.testDockerClean_xxD_FORGO()
DockerTest.testDockerClean_xxD_STOP()
DockerTest.testDockerClean_xxD_RM()
DockerTest.testDockerClean_xxD_None()
DockerTest.testDockerClean_xxx_FORGO()
DockerTest.testDockerClean_xxx_STOP()
DockerTest.testDockerClean_xxx_RM()
DockerTest.testDockerClean_xxx_None()
DockerTest.testDockerPipeChain()
DockerTest.testDockerPipeChainErrorDetection()
DockerTest.testNonCachingDockerChain()
DockerTest.testNonCachingDockerChainErrorDetection()
DockerTest.testDockerLogs()
DockerTest.testDockerLogs_Stream()
DockerTest.testDockerLogs_Demux()
DockerTest.testDockerLogs_Demux_Stream()
- Module Contents
toil.test.lib.test_conversions
toil.test.lib.test_ec2
toil.test.lib.test_misc
toil.test.mesos
toil.test.provisioners
- Subpackages
toil.test.provisioners.aws
- Submodules
toil.test.provisioners.clusterScalerTest
- Module Contents
- Classes
- Attributes
logger
c4_8xlarge_preemptible
c4_8xlarge
r3_8xlarge
r5_2xlarge
r5_4xlarge
t2_micro
BinPackingTest
BinPackingTest.setUp()
BinPackingTest.testPackingOneShape()
BinPackingTest.testSorting()
BinPackingTest.testAddingInitialNode()
BinPackingTest.testLowTargetTime()
BinPackingTest.testHighTargetTime()
BinPackingTest.testZeroResourceJobs()
BinPackingTest.testLongRunningJobs()
BinPackingTest.run1000JobsOnMicros()
BinPackingTest.testPathologicalCase()
BinPackingTest.testJobTooLargeForAllNodes()
ClusterScalerTest
ClusterScalerTest.setUp()
ClusterScalerTest.testRounding()
ClusterScalerTest.testMaxNodes()
ClusterScalerTest.testMinNodes()
ClusterScalerTest.testPreemptibleDeficitResponse()
ClusterScalerTest.testPreemptibleDeficitIsSet()
ClusterScalerTest.testNoLaunchingIfDeltaAlreadyMet()
ClusterScalerTest.testBetaInertia()
ClusterScalerTest.test_overhead_accounting_large()
ClusterScalerTest.test_overhead_accounting_small()
ClusterScalerTest.test_overhead_accounting_observed()
ScalerThreadTest
MockBatchSystemAndProvisioner
MockBatchSystemAndProvisioner.start()
MockBatchSystemAndProvisioner.shutDown()
MockBatchSystemAndProvisioner.nodeInUse()
MockBatchSystemAndProvisioner.ignoreNode()
MockBatchSystemAndProvisioner.unignoreNode()
MockBatchSystemAndProvisioner.supportedClusterTypes()
MockBatchSystemAndProvisioner.createClusterSettings()
MockBatchSystemAndProvisioner.readClusterSettings()
MockBatchSystemAndProvisioner.setAutoscaledNodeTypes()
MockBatchSystemAndProvisioner.getProvisionedWorkers()
MockBatchSystemAndProvisioner.terminateNodes()
MockBatchSystemAndProvisioner.remainingBillingInterval()
MockBatchSystemAndProvisioner.addJob()
MockBatchSystemAndProvisioner.getNumberOfJobsIssued()
MockBatchSystemAndProvisioner.getJobs()
MockBatchSystemAndProvisioner.getNodes()
MockBatchSystemAndProvisioner.addNodes()
MockBatchSystemAndProvisioner.getNodeShape()
MockBatchSystemAndProvisioner.getWorkersInCluster()
MockBatchSystemAndProvisioner.launchCluster()
MockBatchSystemAndProvisioner.destroyCluster()
MockBatchSystemAndProvisioner.getLeader()
MockBatchSystemAndProvisioner.getNumberOfNodes()
- Module Contents
toil.test.provisioners.clusterTest
toil.test.provisioners.gceProvisionerTest
- Module Contents
- Classes
- Attributes
log
AbstractGCEAutoscaleTest
AbstractGCEAutoscaleTest.projectID
AbstractGCEAutoscaleTest.sshUtil()
AbstractGCEAutoscaleTest.rsyncUtil()
AbstractGCEAutoscaleTest.destroyClusterUtil()
AbstractGCEAutoscaleTest.createClusterUtil()
AbstractGCEAutoscaleTest.cleanJobStoreUtil()
AbstractGCEAutoscaleTest.setUp()
AbstractGCEAutoscaleTest.tearDown()
AbstractGCEAutoscaleTest.launchCluster()
GCEAutoscaleTest
GCEStaticAutoscaleTest
GCEAutoscaleTestMultipleNodeTypes
GCERestartTest
- Module Contents
toil.test.provisioners.provisionerTest
toil.test.provisioners.restartScript
- Subpackages
toil.test.server
- Submodules
toil.test.server.serverTest
- Module Contents
- Classes
- Attributes
logger
ToilServerUtilsTest
hidden
FileStateStoreTest
FileStateStoreURLTest
BucketUsingTest
AWSStateStoreTest
AbstractToilWESServerTest
ToilWESServerBenchTest
ToilWESServerWorkflowTest
ToilWESServerWorkflowTest.run_zip_workflow()
ToilWESServerWorkflowTest.test_run_workflow_relative_url_no_attachments_fails()
ToilWESServerWorkflowTest.test_run_workflow_relative_url()
ToilWESServerWorkflowTest.test_run_workflow_https_url()
ToilWESServerWorkflowTest.test_run_workflow_single_file_zip()
ToilWESServerWorkflowTest.test_run_workflow_multi_file_zip()
ToilWESServerWorkflowTest.test_run_workflow_manifest_zip()
ToilWESServerWorkflowTest.test_run_workflow_inputs_zip()
ToilWESServerWorkflowTest.test_run_workflow_manifest_and_inputs_zip()
ToilWESServerWorkflowTest.test_run_workflow_no_params_zip()
ToilWESServerWorkflowTest.test_run_and_cancel_workflows()
ToilWESServerCeleryWorkflowTest
ToilWESServerCeleryS3StateWorkflowTest
- Module Contents
- Submodules
toil.test.sort
- Submodules
toil.test.sort.restart_sort
toil.test.sort.sort
toil.test.sort.sortTest
- Module Contents
- Classes
- Functions
- Attributes
logger
defaultLineLen
defaultLines
defaultN
runMain()
SortTest
SortTest.testNo
SortTest.setUp()
SortTest.tearDown()
SortTest.testAwsSingle()
SortTest.testAwsMesos()
SortTest.testFileMesos()
SortTest.testGoogleSingle()
SortTest.testGoogleMesos()
SortTest.testFileSingle()
SortTest.testFileSingleNonCaching()
SortTest.testFileSingleCheckpoints()
SortTest.testFileSingle10000()
SortTest.testFileGridEngine()
SortTest.testFileTorqueEngine()
SortTest.testFileParasol()
SortTest.testSort()
SortTest.testMerge()
SortTest.testCopySubRangeOfFile()
SortTest.testGetMidPoint()
- Module Contents
- Submodules
toil.test.src
- Submodules
toil.test.src.autoDeploymentTest
- Module Contents
- Classes
- Attributes
logger
AutoDeploymentTest
AutoDeploymentTest.sitePackages
AutoDeploymentTest.setUp()
AutoDeploymentTest.testRestart()
AutoDeploymentTest.testSplitRootPackages()
AutoDeploymentTest.testUserTypesInJobFunctionArgs()
AutoDeploymentTest.testDeferralWithConcurrentEncapsulation()
AutoDeploymentTest.testDeferralWithFailureAndEncapsulation()
- Module Contents
toil.test.src.busTest
toil.test.src.checkpointTest
toil.test.src.deferredFunctionTest
- Module Contents
- Classes
- Attributes
logger
DeferredFunctionTest
DeferredFunctionTest.jobStoreType
DeferredFunctionTest.setUp()
DeferredFunctionTest.testDeferredFunctionRunsWithMethod()
DeferredFunctionTest.testDeferredFunctionRunsWithClassMethod()
DeferredFunctionTest.testDeferredFunctionRunsWithLambda()
DeferredFunctionTest.testDeferredFunctionRunsWithFailures()
DeferredFunctionTest.testNewJobsCanHandleOtherJobDeaths()
DeferredFunctionTest.testBatchSystemCleanupCanHandleWorkerDeaths()
- Module Contents
toil.test.src.dockerCheckTest
- Module Contents
- Classes
DockerCheckTest
DockerCheckTest.testOfficialUbuntuRepo()
DockerCheckTest.testBroadDockerRepo()
DockerCheckTest.testBroadDockerRepoBadTag()
DockerCheckTest.testNonexistentRepo()
DockerCheckTest.testToilQuayRepo()
DockerCheckTest.testBadQuayRepoNTag()
DockerCheckTest.testBadQuayRepo()
DockerCheckTest.testBadQuayTag()
DockerCheckTest.testGoogleRepo()
DockerCheckTest.testBadGoogleRepo()
DockerCheckTest.testApplianceParser()
- Classes
- Module Contents
toil.test.src.fileStoreTest
- Module Contents
- Classes
- Attributes
testingIsAutomatic
logger
hidden
hidden.AbstractFileStoreTest
hidden.AbstractFileStoreTest.jobStoreType
hidden.AbstractFileStoreTest.setUp()
hidden.AbstractFileStoreTest.create_file()
hidden.AbstractFileStoreTest.testToilIsNotBroken()
hidden.AbstractFileStoreTest.testFileStoreLogging()
hidden.AbstractFileStoreTest.testFileStoreOperations()
hidden.AbstractFileStoreTest.testWriteReadGlobalFilePermissions()
hidden.AbstractFileStoreTest.testWriteExportFileCompatibility()
hidden.AbstractFileStoreTest.testImportReadFileCompatibility()
hidden.AbstractFileStoreTest.testReadWriteFileStreamTextMode()
hidden.AbstractNonCachingFileStoreTest
hidden.AbstractCachingFileStoreTest
hidden.AbstractCachingFileStoreTest.setUp()
hidden.AbstractCachingFileStoreTest.testExtremeCacheSetup()
hidden.AbstractCachingFileStoreTest.testCacheEvictionPartialEvict()
hidden.AbstractCachingFileStoreTest.testCacheEvictionTotalEvict()
hidden.AbstractCachingFileStoreTest.testCacheEvictionFailCase()
hidden.AbstractCachingFileStoreTest.testAsyncWriteWithCaching()
hidden.AbstractCachingFileStoreTest.testWriteNonLocalFileToJobStore()
hidden.AbstractCachingFileStoreTest.testWriteLocalFileToJobStore()
hidden.AbstractCachingFileStoreTest.testReadCacheMissFileFromJobStoreWithoutCachingReadFile()
hidden.AbstractCachingFileStoreTest.testReadCacheMissFileFromJobStoreWithCachingReadFile()
hidden.AbstractCachingFileStoreTest.testReadCachHitFileFromJobStore()
hidden.AbstractCachingFileStoreTest.testMultipleJobsReadSameCacheHitGlobalFile()
hidden.AbstractCachingFileStoreTest.testMultipleJobsReadSameCacheMissGlobalFile()
hidden.AbstractCachingFileStoreTest.testFileStoreExportFile()
hidden.AbstractCachingFileStoreTest.testReturnFileSizes()
hidden.AbstractCachingFileStoreTest.testReturnFileSizesWithBadWorker()
hidden.AbstractCachingFileStoreTest.testControlledFailedWorkerRetry()
hidden.AbstractCachingFileStoreTest.testRemoveLocalMutablyReadFile()
hidden.AbstractCachingFileStoreTest.testRemoveLocalImmutablyReadFile()
hidden.AbstractCachingFileStoreTest.testDeleteLocalFile()
hidden.AbstractCachingFileStoreTest.testSimultaneousReadsUncachedStream()
NonCachingFileStoreTestWithFileJobStore
CachingFileStoreTestWithFileJobStore
NonCachingFileStoreTestWithAwsJobStore
CachingFileStoreTestWithAwsJobStore
NonCachingFileStoreTestWithGoogleJobStore
CachingFileStoreTestWithGoogleJobStore
- Module Contents
toil.test.src.helloWorldTest
toil.test.src.importExportFileTest
toil.test.src.jobDescriptionTest
toil.test.src.jobEncapsulationTest
toil.test.src.jobFileStoreTest
toil.test.src.jobServiceTest
toil.test.src.jobTest
- Module Contents
- Classes
- Functions
- Attributes
logger
JobTest
JobTest.setUpClass()
JobTest.testStatic()
JobTest.testStatic2()
JobTest.testTrivialDAGConsistency()
JobTest.testDAGConsistency()
JobTest.testSiblingDAGConsistency()
JobTest.testDeadlockDetection()
JobTest.testNewCheckpointIsLeafVertexNonRootCase()
JobTest.testNewCheckpointIsLeafVertexRootCase()
JobTest.runNewCheckpointIsLeafVertexTest()
JobTest.runCheckpointVertexTest()
JobTest.testEvaluatingRandomDAG()
JobTest.getRandomEdge()
JobTest.makeRandomDAG()
JobTest.getAdjacencyList()
JobTest.reachable()
JobTest.addRandomFollowOnEdges()
JobTest.makeJobGraph()
JobTest.isAcyclic()
simpleJobFn()
fn1Test()
fn2Test()
trivialParent()
parent()
diamond()
child()
errorChild()
TrivialService
- Module Contents
toil.test.src.miscTests
toil.test.src.promisedRequirementTest
- Module Contents
- Classes
- Functions
- Attributes
log
hidden
hidden.AbstractPromisedRequirementsTest
hidden.AbstractPromisedRequirementsTest.testConcurrencyDynamic()
hidden.AbstractPromisedRequirementsTest.testConcurrencyStatic()
hidden.AbstractPromisedRequirementsTest.getOptions()
hidden.AbstractPromisedRequirementsTest.getCounterPath()
hidden.AbstractPromisedRequirementsTest.testPromisesWithJobStoreFileObjects()
hidden.AbstractPromisedRequirementsTest.testPromisesWithNonCachingFileStore()
hidden.AbstractPromisedRequirementsTest.testPromiseRequirementRaceStatic()
maxConcurrency()
getOne()
getThirtyTwoMb()
logDiskUsage()
SingleMachinePromisedRequirementsTest
MesosPromisedRequirementsTest
- Module Contents
toil.test.src.promisesTest
toil.test.src.realtimeLoggerTest
toil.test.src.regularLogTest
toil.test.src.resourceTest
toil.test.src.restartDAGTest
toil.test.src.resumabilityTest
toil.test.src.retainTempDirTest
toil.test.src.systemTest
toil.test.src.threadingTest
toil.test.src.toilContextManagerTest
toil.test.src.userDefinedJobArgTypeTest
toil.test.src.workerTest
- Submodules
toil.test.utils
- Submodules
toil.test.utils.toilDebugTest
toil.test.utils.toilKillTest
toil.test.utils.utilsTest
- Module Contents
- Classes
- Functions
- Attributes
pkg_root
logger
UtilsTest
UtilsTest.toilMain
UtilsTest.cleanCommand
UtilsTest.statsCommand
UtilsTest.setUp()
UtilsTest.tearDown()
UtilsTest.statusCommand()
UtilsTest.testAWSProvisionerUtils()
UtilsTest.testUtilsSort()
UtilsTest.testUtilsStatsSort()
UtilsTest.testUnicodeSupport()
UtilsTest.testMultipleJobsPerWorkerStats()
UtilsTest.check_status()
UtilsTest.testGetPIDStatus()
UtilsTest.testGetStatusFailedToilWF()
UtilsTest.testGetStatusFailedCWLWF()
UtilsTest.testGetStatusSuccessfulCWLWF()
UtilsTest.testPrintJobLog()
UtilsTest.testRestartAttribute()
printUnicodeCharacter()
RunTwoJobsPerWorker
- Module Contents
- Submodules
toil.test.wdl
- Submodules
toil.test.wdl.builtinTest
- Module Contents
- Classes
WdlStandardLibraryFunctionsTest
WdlStandardLibraryFunctionsTest.setUp()
WdlStandardLibraryFunctionsTest.setUpClass()
WdlStandardLibraryFunctionsTest.tearDown()
WdlStandardLibraryFunctionsTest.testFn_Sub()
WdlStandardLibraryFunctionsTest.testFn_Ceil()
WdlStandardLibraryFunctionsTest.testFn_Floor()
WdlStandardLibraryFunctionsTest.testFn_ReadLines()
WdlStandardLibraryFunctionsTest.testFn_ReadTsv()
WdlStandardLibraryFunctionsTest.testFn_ReadJson()
WdlStandardLibraryFunctionsTest.testFn_ReadMap()
WdlStandardLibraryFunctionsTest.testFn_ReadInt()
WdlStandardLibraryFunctionsTest.testFn_ReadString()
WdlStandardLibraryFunctionsTest.testFn_ReadFloat()
WdlStandardLibraryFunctionsTest.testFn_ReadBoolean()
WdlStandardLibraryFunctionsTest.testFn_WriteLines()
WdlStandardLibraryFunctionsTest.testFn_WriteTsv()
WdlStandardLibraryFunctionsTest.testFn_WriteJson()
WdlStandardLibraryFunctionsTest.testFn_WriteMap()
WdlStandardLibraryFunctionsTest.testFn_Transpose()
WdlStandardLibraryFunctionsTest.testFn_Length()
WdlStandardLibraryFunctionsTest.testFn_Zip()
WdlStandardLibraryFunctionsTest.testFn_Cross()
WdlWorkflowsTest
WdlLanguageSpecWorkflowsTest
WdlStandardLibraryWorkflowsTest
WdlStandardLibraryWorkflowsTest.setUpClass()
WdlStandardLibraryWorkflowsTest.test_sub()
WdlStandardLibraryWorkflowsTest.test_size()
WdlStandardLibraryWorkflowsTest.test_ceil()
WdlStandardLibraryWorkflowsTest.test_floor()
WdlStandardLibraryWorkflowsTest.test_round()
WdlStandardLibraryWorkflowsTest.test_stdout()
WdlStandardLibraryWorkflowsTest.test_read()
WdlStandardLibraryWorkflowsTest.test_write()
WdlStandardLibraryWorkflowsTest.test_range()
WdlStandardLibraryWorkflowsTest.test_transpose()
WdlStandardLibraryWorkflowsTest.test_length()
WdlStandardLibraryWorkflowsTest.test_zip()
WdlStandardLibraryWorkflowsTest.test_cross()
WdlStandardLibraryWorkflowsTest.test_as_pairs()
WdlStandardLibraryWorkflowsTest.test_as_map()
WdlStandardLibraryWorkflowsTest.test_keys()
WdlStandardLibraryWorkflowsTest.test_collect_by_key()
WdlStandardLibraryWorkflowsTest.test_flatten()
- Classes
- Module Contents
toil.test.wdl.conftest
toil.test.wdl.toilwdlTest
- Module Contents
- Classes
- Functions
BaseToilWdlTest
ToilWdlTest
ToilWDLLibraryTest
ToilWDLLibraryTest.testFn_SelectFirst()
ToilWDLLibraryTest.testFn_Size()
ToilWDLLibraryTest.testFn_Basename()
ToilWDLLibraryTest.testFn_Glob()
ToilWDLLibraryTest.testFn_ParseMemory()
ToilWDLLibraryTest.testFn_ParseCores()
ToilWDLLibraryTest.testFn_ParseDisk()
ToilWDLLibraryTest.testPrimitives()
ToilWDLLibraryTest.testCSV()
ToilWDLLibraryTest.testTSV()
ToilWdlIntegrationTest
ToilWdlIntegrationTest.gatk_data
ToilWdlIntegrationTest.gatk_data_dir
ToilWdlIntegrationTest.encode_data
ToilWdlIntegrationTest.encode_data_dir
ToilWdlIntegrationTest.wdl_data
ToilWdlIntegrationTest.wdl_data_dir
ToilWdlIntegrationTest.setUpClass()
ToilWdlIntegrationTest.tearDownClass()
ToilWdlIntegrationTest.testTut01()
ToilWdlIntegrationTest.testTut02()
ToilWdlIntegrationTest.testTut03()
ToilWdlIntegrationTest.testTut04()
ToilWdlIntegrationTest.testENCODE()
ToilWdlIntegrationTest.testPipe()
ToilWdlIntegrationTest.testJSON()
ToilWdlIntegrationTest.test_size_large()
ToilWdlIntegrationTest.fetch_and_unzip_from_s3()
compare_runs()
compare_vcf_files()
- Module Contents
toil.test.wdl.wdltoil_test
- Submodules
- Package Contents
- Classes
- Functions
- Attributes
ApplianceImageNotFound
applianceSelf()
toilPackageDirPath()
have_working_nvidia_docker_runtime()
have_working_nvidia_smi()
running_on_ec2()
concat
memoize
ExceptionalThread
cpu_count()
distVersion
logger
ToilTest
MT
get_temp_file()
needs_env_var()
needs_rsync3()
needs_aws_s3()
needs_aws_ec2()
needs_aws_batch()
needs_google()
needs_gridengine()
needs_torque()
needs_tes()
needs_kubernetes_installed()
needs_kubernetes()
needs_mesos()
needs_parasol()
needs_slurm()
needs_htcondor()
needs_lsf()
needs_java()
needs_docker()
needs_singularity()
needs_local_cuda()
needs_docker_cuda()
needs_encryption()
needs_cwl()
needs_server()
needs_celery_broker()
needs_wes_server()
needs_local_appliance()
needs_fetchable_appliance()
integrative()
slow()
methodNamePartRegex
timeLimit()
make_tests()
ApplianceTestSupport
ApplianceTestSupport.Appliance
ApplianceTestSupport.Appliance.lock
ApplianceTestSupport.Appliance.__enter__()
ApplianceTestSupport.Appliance.__exit__()
ApplianceTestSupport.Appliance.tryRun()
ApplianceTestSupport.Appliance.runOnAppliance()
ApplianceTestSupport.Appliance.writeToAppliance()
ApplianceTestSupport.Appliance.deployScript()
ApplianceTestSupport.LeaderThread
ApplianceTestSupport.WorkerThread
- Subpackages
toil.utils
- Submodules
toil.utils.toilClean
toil.utils.toilDebugFile
toil.utils.toilDebugJob
toil.utils.toilDestroyCluster
toil.utils.toilKill
toil.utils.toilLaunchCluster
toil.utils.toilMain
toil.utils.toilRsyncCluster
toil.utils.toilServer
toil.utils.toilSshCluster
toil.utils.toilStats
- Module Contents
- Classes
- Functions
- Attributes
logger
ColumnWidths
padStr()
prettyMemory()
prettyTime()
reportTime()
reportMemory()
reportNumber()
sprintTag()
decorateTitle()
decorateSubHeader()
get()
sortJobs()
reportPrettyData()
computeColumnWidths()
updateColumnWidths()
buildElement()
createSummary()
getStats()
processData()
reportData()
category_choices
sort_category_choices
sort_field_choices
add_stats_options()
main()
- Module Contents
toil.utils.toilStatus
- Module Contents
- Classes
- Functions
- Attributes
logger
ToilStatus
ToilStatus.print_dot_chart()
ToilStatus.printJobLog()
ToilStatus.printJobChildren()
ToilStatus.printAggregateJobStats()
ToilStatus.report_on_jobs()
ToilStatus.getPIDStatus()
ToilStatus.getStatus()
ToilStatus.print_bus_messages()
ToilStatus.fetchRootJob()
ToilStatus.fetchUserJobs()
ToilStatus.traverseJobGraph()
main()
- Module Contents
toil.utils.toilUpdateEC2Instances
- Submodules
toil.wdl
- Subpackages
toil.wdl.versions
- Submodules
toil.wdl.versions.dev
- Module Contents
- Classes
- Attributes
logger
AnalyzeDevelopmentWDL
AnalyzeDevelopmentWDL.version
AnalyzeDevelopmentWDL.analyze()
AnalyzeDevelopmentWDL.visit_document()
AnalyzeDevelopmentWDL.visit_document_element()
AnalyzeDevelopmentWDL.visit_call()
AnalyzeDevelopmentWDL.visit_string_expr_part()
AnalyzeDevelopmentWDL.visit_wdl_type()
AnalyzeDevelopmentWDL.visit_expr_core()
- Module Contents
toil.wdl.versions.draft2
- Module Contents
- Classes
- Attributes
logger
AnalyzeDraft2WDL
AnalyzeDraft2WDL.version
AnalyzeDraft2WDL.analyze()
AnalyzeDraft2WDL.write_AST()
AnalyzeDraft2WDL.find_asts()
AnalyzeDraft2WDL.create_tasks_dict()
AnalyzeDraft2WDL.parse_task()
AnalyzeDraft2WDL.parse_task_rawcommand_attributes()
AnalyzeDraft2WDL.parse_task_rawcommand()
AnalyzeDraft2WDL.modify_cmd_expr_w_attributes()
AnalyzeDraft2WDL.parse_task_runtime_key()
AnalyzeDraft2WDL.parse_task_runtime()
AnalyzeDraft2WDL.parse_task_outputs()
AnalyzeDraft2WDL.translate_wdl_string_to_python_string()
AnalyzeDraft2WDL.create_workflows_dict()
AnalyzeDraft2WDL.parse_workflow()
AnalyzeDraft2WDL.parse_workflow_body()
AnalyzeDraft2WDL.parse_workflow_if()
AnalyzeDraft2WDL.parse_workflow_if_expression()
AnalyzeDraft2WDL.parse_workflow_scatter()
AnalyzeDraft2WDL.parse_workflow_scatter_item()
AnalyzeDraft2WDL.parse_workflow_scatter_collection()
AnalyzeDraft2WDL.parse_declaration()
AnalyzeDraft2WDL.parse_declaration_name()
AnalyzeDraft2WDL.parse_declaration_type()
AnalyzeDraft2WDL.parse_declaration_expressn()
AnalyzeDraft2WDL.parse_declaration_expressn_logicalnot()
AnalyzeDraft2WDL.parse_declaration_expressn_arraymaplookup()
AnalyzeDraft2WDL.parse_declaration_expressn_memberaccess()
AnalyzeDraft2WDL.parse_declaration_expressn_ternaryif()
AnalyzeDraft2WDL.parse_declaration_expressn_tupleliteral()
AnalyzeDraft2WDL.parse_declaration_expressn_arrayliteral()
AnalyzeDraft2WDL.parse_declaration_expressn_operator()
AnalyzeDraft2WDL.parse_declaration_expressn_fncall()
AnalyzeDraft2WDL.parse_declaration_expressn_fncall_normalparams()
AnalyzeDraft2WDL.parse_workflow_call_taskname()
AnalyzeDraft2WDL.parse_workflow_call_taskalias()
AnalyzeDraft2WDL.parse_workflow_call_body_declarations()
AnalyzeDraft2WDL.parse_workflow_call_body_io()
AnalyzeDraft2WDL.parse_workflow_call_body_io_map()
AnalyzeDraft2WDL.parse_workflow_call_body()
AnalyzeDraft2WDL.parse_workflow_call()
- Module Contents
toil.wdl.versions.v1
- Module Contents
- Classes
- Functions
- Attributes
logger
is_context()
AnalyzeV1WDL
AnalyzeV1WDL.version
AnalyzeV1WDL.analyze()
AnalyzeV1WDL.visit_document()
AnalyzeV1WDL.visit_document_element()
AnalyzeV1WDL.visit_workflow()
AnalyzeV1WDL.visit_workflow_input()
AnalyzeV1WDL.visit_workflow_output()
AnalyzeV1WDL.visit_inner_workflow_element()
AnalyzeV1WDL.visit_call()
AnalyzeV1WDL.visit_scatter()
AnalyzeV1WDL.visit_conditional()
AnalyzeV1WDL.visit_task()
AnalyzeV1WDL.visit_task_input()
AnalyzeV1WDL.visit_task_output()
AnalyzeV1WDL.visit_task_command()
AnalyzeV1WDL.visit_task_command_string_part()
AnalyzeV1WDL.visit_task_command_expr_with_string()
AnalyzeV1WDL.visit_task_command_expr_part()
AnalyzeV1WDL.visit_task_runtime()
AnalyzeV1WDL.visit_any_decls()
AnalyzeV1WDL.visit_unbound_decls()
AnalyzeV1WDL.visit_bound_decls()
AnalyzeV1WDL.visit_wdl_type()
AnalyzeV1WDL.visit_primitive_literal()
AnalyzeV1WDL.visit_number()
AnalyzeV1WDL.visit_string()
AnalyzeV1WDL.visit_string_expr_with_string_part()
AnalyzeV1WDL.visit_string_expr_part()
AnalyzeV1WDL.visit_string_part()
AnalyzeV1WDL.visit_expression_placeholder_option()
AnalyzeV1WDL.visit_expr()
AnalyzeV1WDL.visit_infix0()
AnalyzeV1WDL.visit_lor()
AnalyzeV1WDL.visit_infix1()
AnalyzeV1WDL.visit_land()
AnalyzeV1WDL.visit_infix2()
AnalyzeV1WDL.visit_infix3()
AnalyzeV1WDL.visit_infix4()
AnalyzeV1WDL.visit_infix5()
AnalyzeV1WDL.visit_expr_core()
AnalyzeV1WDL.visit_apply()
AnalyzeV1WDL.visit_array_literal()
AnalyzeV1WDL.visit_pair_literal()
AnalyzeV1WDL.visit_ifthenelse()
AnalyzeV1WDL.visit_expression_group()
AnalyzeV1WDL.visit_at()
AnalyzeV1WDL.visit_get_name()
AnalyzeV1WDL.visit_negate()
AnalyzeV1WDL.visit_unarysigned()
AnalyzeV1WDL.visit_primitives()
- Module Contents
- Submodules
- Submodules
toil.wdl.toilwdl
toil.wdl.utils
toil.wdl.wdl_analysis
toil.wdl.wdl_functions
- Module Contents
- Classes
- Functions
- Attributes
logger
WDLRuntimeError
WDLJSONEncoder
generate_docker_bashscript_file()
process_single_infile()
process_infile()
sub()
defined()
process_single_outfile()
process_outfile()
abspath_single_file()
abspath_file()
read_single_file()
read_file()
process_and_read_file()
generate_stdout_file()
parse_memory()
parse_cores()
parse_disk()
is_number()
size()
select_first()
combine_dicts()
basename()
heredoc_wdl()
floor()
ceil()
read_lines()
read_tsv()
read_csv()
read_json()
read_map()
read_int()
read_string()
read_float()
read_boolean()
write_lines()
write_tsv()
write_json()
write_map()
wdl_range()
transpose()
length()
wdl_zip()
cross()
as_pairs()
as_map()
keys()
collect_by_key()
flatten()
- Module Contents
toil.wdl.wdl_synthesis
- Module Contents
- Classes
- Attributes
logger
SynthesizeWDL
SynthesizeWDL.write_modules()
SynthesizeWDL.write_main()
SynthesizeWDL.write_main_header()
SynthesizeWDL.write_main_jobwrappers()
SynthesizeWDL.write_main_jobwrappers_declaration()
SynthesizeWDL.write_main_destbucket()
SynthesizeWDL.fetch_ignoredifs()
SynthesizeWDL.fetch_ignoredifs_chain()
SynthesizeWDL.write_main_jobwrappers_if()
SynthesizeWDL.write_main_jobwrappers_scatter()
SynthesizeWDL.fetch_scatter_outputs()
SynthesizeWDL.fetch_scatter_inputs()
SynthesizeWDL.fetch_scatter_inputs_chain()
SynthesizeWDL.write_main_jobwrappers_call()
SynthesizeWDL.fetch_call_outputs()
SynthesizeWDL.write_functions()
SynthesizeWDL.write_scatterfunctions_within_if()
SynthesizeWDL.write_scatterfunction()
SynthesizeWDL.write_scatterfunction_header()
SynthesizeWDL.write_scatterfunction_outputreturn()
SynthesizeWDL.write_scatterfunction_lists()
SynthesizeWDL.write_scatterfunction_loop()
SynthesizeWDL.write_scatter_callwrapper()
SynthesizeWDL.write_function()
SynthesizeWDL.write_function_header()
SynthesizeWDL.json_var()
SynthesizeWDL.needs_file_import()
SynthesizeWDL.write_declaration_type()
SynthesizeWDL.write_function_bashscriptline()
SynthesizeWDL.write_function_dockercall()
SynthesizeWDL.write_function_cmdline()
SynthesizeWDL.write_function_subprocesspopen()
SynthesizeWDL.write_function_outputreturn()
SynthesizeWDL.indent()
SynthesizeWDL.needsdocker()
SynthesizeWDL.write_python_file()
- Module Contents
toil.wdl.wdl_types
toil.wdl.wdltoil
- Module Contents
- Classes
- Functions
- Attributes
logger
potential_absolute_uris()
toil_read_source()
WDLBindings
combine_bindings()
log_bindings()
get_supertype()
for_each_node()
recursive_dependencies()
TOIL_URI_SCHEME
pack_toil_uri()
unpack_toil_uri()
NonDownloadingSize
ToilWDLStdLibBase
ToilWDLStdLibTaskOutputs
evaluate_named_expression()
evaluate_decl()
evaluate_call_inputs()
evaluate_defaultable_decl()
devirtualize_files()
virtualize_files()
import_files()
drop_missing_files()
get_file_paths_in_bindings()
map_over_typed_files_in_bindings()
map_over_files_in_bindings()
map_over_typed_files_in_binding()
map_over_typed_files_in_value()
WDLBaseJob
WDLTaskJob
WDLWorkflowNodeJob
WDLCombineBindingsJob
WDLNamespaceBindingsJob
WDLSectionJob
WDLScatterJob
WDLArrayBindingsJob
WDLConditionalJob
WDLWorkflowJob
WDLOutputsJob
WDLRootJob
main()
- Module Contents
- Subpackages
- Submodules
toil.bus
- Module Contents
- Classes
- Functions
- Attributes
logger
JobIssuedMessage
JobUpdatedMessage
JobCompletedMessage
JobFailedMessage
JobMissingMessage
JobAnnotationMessage
ExternalBatchIdMessage
QueueSizeMessage
ClusterSizeMessage
ClusterDesiredSizeMessage
message_to_bytes()
MessageType
bytes_to_message()
MessageBus
MessageBusClient
MessageInbox
MessageOutbox
MessageBusConnection
JobStatus
replay_message_bus()
gen_message_bus_path()
- Module Contents
toil.common
- Module Contents
- Classes
- Functions
- Attributes
defaultTargetTime
SYS_MAX_SIZE
UUID_LENGTH
logger
Config
Config.logFile
Config.logRotating
Config.cleanWorkDir
Config.max_jobs
Config.max_local_jobs
Config.run_local_jobs_on_workers
Config.tes_endpoint
Config.tes_user
Config.tes_password
Config.tes_bearer_token
Config.jobStore
Config.batchSystem
Config.batch_logs_dir
Config.workflowAttemptNumber
Config.disableAutoDeployment
Config.workflowID
Config.prepare_start()
Config.prepare_restart()
Config.setOptions()
Config.__eq__()
Config.__hash__()
JOBSTORE_HELP
parser_with_common_options()
addOptions()
parseBool()
getNodeID()
Toil
Toil.config
Toil.__enter__()
Toil.__exit__()
Toil.start()
Toil.restart()
Toil.getJobStore()
Toil.parseLocator()
Toil.buildLocator()
Toil.resumeJobStore()
Toil.createBatchSystem()
Toil.importFile()
Toil.import_file()
Toil.exportFile()
Toil.export_file()
Toil.normalize_uri()
Toil.getToilWorkDir()
Toil.get_toil_coordination_dir()
Toil.getLocalWorkflowDir()
Toil.get_local_workflow_coordination_dir()
ToilRestartException
ToilContextManagerException
ToilMetrics
ToilMetrics.startDashboard()
ToilMetrics.add_prometheus_data_source()
ToilMetrics.log()
ToilMetrics.logClusterSize()
ToilMetrics.logClusterDesiredSize()
ToilMetrics.logQueueSize()
ToilMetrics.logMissingJob()
ToilMetrics.logIssuedJob()
ToilMetrics.logFailedJob()
ToilMetrics.logCompletedJob()
ToilMetrics.shutdown()
parseSetEnv()
iC()
fC()
parse_accelerator_list()
cacheDirName()
getDirSizeRecursively()
getFileSystemSize()
safeUnpickleFromStream()
- Module Contents
toil.deferred
toil.exceptions
toil.job
- Module Contents
- Classes
- Functions
- Attributes
logger
JobPromiseConstraintError
ConflictingPredecessorError
TemporaryID
AcceleratorRequirement
parse_accelerator()
accelerator_satisfies()
accelerators_fully_satisfy()
RequirementsDict
REQUIREMENT_NAMES
ParsedRequirement
ParseableIndivisibleResource
ParseableDivisibleResource
ParseableFlag
ParseableAcceleratorRequirement
ParseableRequirement
Requirer
JobDescription
JobDescription.services
JobDescription.remainingTryCount
JobDescription.serviceHostIDsInBatches()
JobDescription.successorsAndServiceHosts()
JobDescription.allSuccessors()
JobDescription.successors_by_phase()
JobDescription.nextSuccessors()
JobDescription.filterSuccessors()
JobDescription.filterServiceHosts()
JobDescription.clear_nonexistent_dependents()
JobDescription.clear_dependents()
JobDescription.is_subtree_done()
JobDescription.replace()
JobDescription.addChild()
JobDescription.addFollowOn()
JobDescription.addServiceHostJob()
JobDescription.hasChild()
JobDescription.hasFollowOn()
JobDescription.hasServiceHostJob()
JobDescription.renameReferences()
JobDescription.addPredecessor()
JobDescription.onRegistration()
JobDescription.setupJobAfterFailure()
JobDescription.getLogFileHandle()
JobDescription.clearRemainingTryCount()
JobDescription.__str__()
JobDescription.__repr__()
JobDescription.pre_update_hook()
JobDescription.get_job_kind()
ServiceJobDescription
CheckpointJobDescription
Job
Job.Runner
Job.Service
Job.jobStoreID
Job.description
Job.disk
Job.memory
Job.cores
Job.accelerators
Job.preemptible
Job.checkpoint
Job.tempDir
Job.__str__()
Job.preemptable()
Job.assignConfig()
Job.run()
Job.addChild()
Job.hasChild()
Job.addFollowOn()
Job.hasPredecessor()
Job.hasFollowOn()
Job.addService()
Job.hasService()
Job.addChildFn()
Job.addFollowOnFn()
Job.addChildJobFn()
Job.addFollowOnJobFn()
Job.log()
Job.wrapFn()
Job.wrapJobFn()
Job.encapsulate()
Job.rv()
Job.registerPromise()
Job.prepareForPromiseRegistration()
Job.checkJobGraphForDeadlocks()
Job.getRootJobs()
Job.checkJobGraphConnected()
Job.checkJobGraphAcylic()
Job.checkNewCheckpointsAreLeafVertices()
Job.defer()
Job.getUserScript()
Job.getTopologicalOrderingOfJobs()
Job.saveBody()
Job.saveAsRootJob()
Job.loadJob()
JobException
JobGraphDeadlockException
FunctionWrappingJob
JobFunctionWrappingJob
PromisedRequirementFunctionWrappingJob
PromisedRequirementJobFunctionWrappingJob
EncapsulatedJob
- Module Contents
- Subpackages