Toil Documentation¶
Toil is an open-source pure-Python workflow engine that lets people write better pipelines.
Check out our website for a comprehensive list of Toil’s features and read our paper to learn what Toil can do in the real world. Please subscribe to our low-volume announce mailing list and feel free to also join us on GitHub and Gitter.
If using Toil for your research, please cite
Vivian, J., Rao, A. A., Nothaft, F. A., Ketchum, C., Armstrong, J., Novak, A., … Paten, B. (2017). Toil enables reproducible, open source, big biomedical data analyses. Nature Biotechnology, 35(4), 314–316. http://doi.org/10.1038/nbt.3772
Getting Started
Common Workflow Language (CWL)
Workflow Description Language (WDL)
Advanced Toil Usage
- Introduction
- Commandline Options
- Toil Utilities
- Toil Debugging
- Running in the Cloud
- Cloud Platforms
- HPC Environments
- Workflow Execution Service (WES)
Toil Python API
- Developing a Python Workflow
- Scripting Quick Start
- Job Basics
- Invoking a Workflow
- Specifying Commandline Arguments
- Resuming a Workflow
- Functions and Job Functions
- Workflows with Multiple Jobs
- Dynamic Job Creation
- Promises
- Promised Requirements
- FileID
- Managing files within a workflow
- Using Docker Containers in Toil
- Services
- Checkpoints
- Encapsulation
- Depending on Toil
- Best Practices for Dockerizing Toil Workflows
- Toil Class API
- Job Store API
- Toil Job API
- Job Methods API
- Job.Runner API
- job.fileStore API
- Batch System API
- Job.Service API
- Exceptions API
Contributing to Toil
Appendices
- Toil Architecture
- Minimum AWS IAM permissions
- Auto-Deployment
- Environment Variables
- API Reference
- toil
- Submodules
- toil.batchSystems
- Submodules
- toil.batchSystems.abstractBatchSystem
- Attributes
- Exceptions
- Classes
- Module Contents
logger
EXIT_STATUS_UNAVAILABLE_VALUE
BatchJobExitReason
UpdatedBatchJobInfo
WorkerCleanupInfo
AbstractBatchSystem
AbstractBatchSystem.supportsAutoDeployment()
AbstractBatchSystem.supportsWorkerCleanup()
AbstractBatchSystem.setUserScript()
AbstractBatchSystem.set_message_bus()
AbstractBatchSystem.issueBatchJob()
AbstractBatchSystem.killBatchJobs()
AbstractBatchSystem.getIssuedBatchJobIDs()
AbstractBatchSystem.getRunningBatchJobIDs()
AbstractBatchSystem.getUpdatedBatchJob()
AbstractBatchSystem.getSchedulingStatusMessage()
AbstractBatchSystem.shutdown()
AbstractBatchSystem.setEnv()
AbstractBatchSystem.add_options()
AbstractBatchSystem.setOptions()
AbstractBatchSystem.getWorkerContexts()
BatchSystemSupport
BatchSystemSupport.config
BatchSystemSupport.maxCores
BatchSystemSupport.maxMemory
BatchSystemSupport.maxDisk
BatchSystemSupport.environment
BatchSystemSupport.check_resource_request()
BatchSystemSupport.setEnv()
BatchSystemSupport.set_message_bus()
BatchSystemSupport.get_batch_logs_dir()
BatchSystemSupport.format_std_out_err_path()
BatchSystemSupport.format_std_out_err_glob()
BatchSystemSupport.workerCleanup()
NodeInfo
AbstractScalableBatchSystem
InsufficientSystemResources
AcquisitionTimeoutException
ResourcePool
ResourceSet
- toil.batchSystems.abstractGridEngineBatchSystem
- Attributes
- Exceptions
- Classes
- Module Contents
logger
JobTuple
ExceededRetryAttempts
AbstractGridEngineBatchSystem
AbstractGridEngineBatchSystem.GridEngineThreadException
AbstractGridEngineBatchSystem.GridEngineThread
AbstractGridEngineBatchSystem.GridEngineThread.boss
AbstractGridEngineBatchSystem.GridEngineThread.newJobsQueue
AbstractGridEngineBatchSystem.GridEngineThread.updatedJobsQueue
AbstractGridEngineBatchSystem.GridEngineThread.killQueue
AbstractGridEngineBatchSystem.GridEngineThread.killedJobsQueue
AbstractGridEngineBatchSystem.GridEngineThread.waitingJobs
AbstractGridEngineBatchSystem.GridEngineThread.runningJobs
AbstractGridEngineBatchSystem.GridEngineThread.runningJobsLock
AbstractGridEngineBatchSystem.GridEngineThread.batchJobIDs
AbstractGridEngineBatchSystem.GridEngineThread.exception
AbstractGridEngineBatchSystem.GridEngineThread.getBatchSystemID()
AbstractGridEngineBatchSystem.GridEngineThread.forgetJob()
AbstractGridEngineBatchSystem.GridEngineThread.createJobs()
AbstractGridEngineBatchSystem.GridEngineThread.killJobs()
AbstractGridEngineBatchSystem.GridEngineThread.checkOnJobs()
AbstractGridEngineBatchSystem.GridEngineThread.run()
AbstractGridEngineBatchSystem.GridEngineThread.coalesce_job_exit_codes()
AbstractGridEngineBatchSystem.GridEngineThread.prepareSubmission()
AbstractGridEngineBatchSystem.GridEngineThread.submitJob()
AbstractGridEngineBatchSystem.GridEngineThread.getRunningJobIDs()
AbstractGridEngineBatchSystem.GridEngineThread.killJob()
AbstractGridEngineBatchSystem.GridEngineThread.getJobExitCode()
AbstractGridEngineBatchSystem.config
AbstractGridEngineBatchSystem.currentJobs
AbstractGridEngineBatchSystem.newJobsQueue
AbstractGridEngineBatchSystem.updatedJobsQueue
AbstractGridEngineBatchSystem.killQueue
AbstractGridEngineBatchSystem.killedJobsQueue
AbstractGridEngineBatchSystem.background_thread
AbstractGridEngineBatchSystem.supportsAutoDeployment()
AbstractGridEngineBatchSystem.count_needed_gpus()
AbstractGridEngineBatchSystem.issueBatchJob()
AbstractGridEngineBatchSystem.killBatchJobs()
AbstractGridEngineBatchSystem.getIssuedBatchJobIDs()
AbstractGridEngineBatchSystem.getRunningBatchJobIDs()
AbstractGridEngineBatchSystem.getUpdatedBatchJob()
AbstractGridEngineBatchSystem.shutdown()
AbstractGridEngineBatchSystem.setEnv()
AbstractGridEngineBatchSystem.getWaitDuration()
AbstractGridEngineBatchSystem.sleepSeconds()
AbstractGridEngineBatchSystem.with_retries()
- toil.batchSystems.awsBatch
- Attributes
- Classes
- Module Contents
logger
STATE_TO_EXIT_REASON
MAX_POLL_COUNT
MIN_REQUESTABLE_MIB
MIN_REQUESTABLE_CORES
AWSBatchBatchSystem
AWSBatchBatchSystem.supportsAutoDeployment()
AWSBatchBatchSystem.region
AWSBatchBatchSystem.client
AWSBatchBatchSystem.queue
AWSBatchBatchSystem.job_role_arn
AWSBatchBatchSystem.owner_tag
AWSBatchBatchSystem.worker_work_dir
AWSBatchBatchSystem.user_script
AWSBatchBatchSystem.docker_image
AWSBatchBatchSystem.job_definition
AWSBatchBatchSystem.bs_id_to_aws_id
AWSBatchBatchSystem.aws_id_to_bs_id
AWSBatchBatchSystem.killed_job_aws_ids
AWSBatchBatchSystem.setUserScript()
AWSBatchBatchSystem.issueBatchJob()
AWSBatchBatchSystem.getUpdatedBatchJob()
AWSBatchBatchSystem.shutdown()
AWSBatchBatchSystem.getIssuedBatchJobIDs()
AWSBatchBatchSystem.getRunningBatchJobIDs()
AWSBatchBatchSystem.killBatchJobs()
AWSBatchBatchSystem.add_options()
AWSBatchBatchSystem.setOptions()
- toil.batchSystems.cleanup_support
- toil.batchSystems.contained_executor
- toil.batchSystems.gridengine
- Attributes
- Classes
- Module Contents
logger
GridEngineBatchSystem
GridEngineBatchSystem.GridEngineThread
GridEngineBatchSystem.GridEngineThread.getRunningJobIDs()
GridEngineBatchSystem.GridEngineThread.killJob()
GridEngineBatchSystem.GridEngineThread.prepareSubmission()
GridEngineBatchSystem.GridEngineThread.submitJob()
GridEngineBatchSystem.GridEngineThread.getJobExitCode()
GridEngineBatchSystem.GridEngineThread.prepareQsub()
GridEngineBatchSystem.getWaitDuration()
- toil.batchSystems.htcondor
- Attributes
- Classes
- Module Contents
logger
JobTuple
schedd_lock
HTCondorBatchSystem
HTCondorBatchSystem.GridEngineThread
HTCondorBatchSystem.GridEngineThread.createJobs()
HTCondorBatchSystem.GridEngineThread.prepareSubmission()
HTCondorBatchSystem.GridEngineThread.submitJob()
HTCondorBatchSystem.GridEngineThread.getRunningJobIDs()
HTCondorBatchSystem.GridEngineThread.killJob()
HTCondorBatchSystem.GridEngineThread.getJobExitCode()
HTCondorBatchSystem.GridEngineThread.connectSchedd()
HTCondorBatchSystem.GridEngineThread.duplicate_quotes()
HTCondorBatchSystem.GridEngineThread.getEnvString()
HTCondorBatchSystem.issueBatchJob()
- toil.batchSystems.kubernetes
- Attributes
- Classes
- Functions
- Module Contents
logger
retryable_kubernetes_errors
is_retryable_kubernetes_error()
KeyValuesList
KubernetesBatchSystem
KubernetesBatchSystem.supportsAutoDeployment()
KubernetesBatchSystem.credential_time
KubernetesBatchSystem.namespace
KubernetesBatchSystem.host_path
KubernetesBatchSystem.service_account
KubernetesBatchSystem.pod_timeout
KubernetesBatchSystem.unique_id
KubernetesBatchSystem.job_prefix
KubernetesBatchSystem.finished_job_ttl
KubernetesBatchSystem.user_script
KubernetesBatchSystem.docker_image
KubernetesBatchSystem.worker_work_dir
KubernetesBatchSystem.aws_secret_name
KubernetesBatchSystem.enable_watching
KubernetesBatchSystem.run_id
KubernetesBatchSystem.resource_sources
KubernetesBatchSystem.schedulingThread
KubernetesBatchSystem.DecoratorWrapper
KubernetesBatchSystem.ItemT
KubernetesBatchSystem.CovItemT
KubernetesBatchSystem.P
KubernetesBatchSystem.R
KubernetesBatchSystem.setUserScript()
KubernetesBatchSystem.Placement
KubernetesBatchSystem.FakeResponse
KubernetesBatchSystem.T
KubernetesBatchSystem.issueBatchJob()
KubernetesBatchSystem.getUpdatedBatchJob()
KubernetesBatchSystem.shutdown()
KubernetesBatchSystem.getIssuedBatchJobIDs()
KubernetesBatchSystem.getRunningBatchJobIDs()
KubernetesBatchSystem.killBatchJobs()
KubernetesBatchSystem.get_default_kubernetes_owner()
KubernetesBatchSystem.KubernetesConfig
KubernetesBatchSystem.add_options()
KubernetesBatchSystem.OptionType
KubernetesBatchSystem.setOptions()
- toil.batchSystems.local_support
- Attributes
- Classes
- Module Contents
logger
BatchSystemLocalSupport
BatchSystemLocalSupport.localBatch
BatchSystemLocalSupport.handleLocalJob()
BatchSystemLocalSupport.killLocalJobs()
BatchSystemLocalSupport.getIssuedLocalJobIDs()
BatchSystemLocalSupport.getRunningLocalJobIDs()
BatchSystemLocalSupport.getUpdatedLocalJob()
BatchSystemLocalSupport.getNextJobID()
BatchSystemLocalSupport.shutdownLocal()
- toil.batchSystems.lsf
- Attributes
- Classes
- Module Contents
logger
LSFBatchSystem
LSFBatchSystem.GridEngineThread
LSFBatchSystem.GridEngineThread.getRunningJobIDs()
LSFBatchSystem.GridEngineThread.fallbackRunningJobIDs()
LSFBatchSystem.GridEngineThread.killJob()
LSFBatchSystem.GridEngineThread.prepareSubmission()
LSFBatchSystem.GridEngineThread.submitJob()
LSFBatchSystem.GridEngineThread.coalesce_job_exit_codes()
LSFBatchSystem.GridEngineThread.getJobExitCode()
LSFBatchSystem.GridEngineThread.parse_bjobs_record()
LSFBatchSystem.GridEngineThread.getJobExitCodeBACCT()
LSFBatchSystem.GridEngineThread.fallbackGetJobExitCode()
LSFBatchSystem.GridEngineThread.prepareBsub()
LSFBatchSystem.GridEngineThread.parseBjobs()
LSFBatchSystem.GridEngineThread.parseMaxMem()
LSFBatchSystem.getWaitDuration()
- toil.batchSystems.lsfHelper
- Attributes
- Functions
- Module Contents
LSB_PARAMS_FILENAME
LSF_CONF_FILENAME
LSF_CONF_ENV
DEFAULT_LSF_UNITS
DEFAULT_RESOURCE_UNITS
LSF_JSON_OUTPUT_MIN_VERSION
logger
find()
find_first_match()
get_conf_file()
apply_conf_file()
per_core_reserve_from_stream()
get_lsf_units_from_stream()
tokenize_conf_stream()
apply_bparams()
apply_lsadmin()
get_lsf_units()
parse_mem_and_cmd_from_output()
get_lsf_version()
check_lsf_json_output_supported()
parse_memory()
per_core_reservation()
- toil.batchSystems.mesos
- Submodules
- toil.batchSystems.mesos.batchSystem
- Attributes
- Classes
- Module Contents
log
MesosBatchSystem
MesosBatchSystem.supportsAutoDeployment()
MesosBatchSystem.supportsWorkerCleanup()
MesosBatchSystem.ExecutorInfo
MesosBatchSystem.userScript
MesosBatchSystem.jobQueues
MesosBatchSystem.mesos_endpoint
MesosBatchSystem.mesos_name
MesosBatchSystem.killedJobIds
MesosBatchSystem.killJobIds
MesosBatchSystem.intendedKill
MesosBatchSystem.hostToJobIDs
MesosBatchSystem.nodeFilter
MesosBatchSystem.runningJobMap
MesosBatchSystem.taskResources
MesosBatchSystem.updatedJobsQueue
MesosBatchSystem.driver
MesosBatchSystem.frameworkId
MesosBatchSystem.executors
MesosBatchSystem.agentsByID
MesosBatchSystem.nonPreemptibleNodes
MesosBatchSystem.executor
MesosBatchSystem.lastTimeOfferLogged
MesosBatchSystem.logPeriod
MesosBatchSystem.ignoredNodes
MesosBatchSystem.setUserScript()
MesosBatchSystem.ignoreNode()
MesosBatchSystem.unignoreNode()
MesosBatchSystem.issueBatchJob()
MesosBatchSystem.killBatchJobs()
MesosBatchSystem.getIssuedBatchJobIDs()
MesosBatchSystem.getRunningBatchJobIDs()
MesosBatchSystem.getUpdatedBatchJob()
MesosBatchSystem.nodeInUse()
MesosBatchSystem.getWaitDuration()
MesosBatchSystem.shutdown()
MesosBatchSystem.registered()
MesosBatchSystem.resourceOffers()
MesosBatchSystem.statusUpdate()
MesosBatchSystem.frameworkMessage()
MesosBatchSystem.getNodes()
MesosBatchSystem.reregistered()
MesosBatchSystem.executorLost()
MesosBatchSystem.get_default_mesos_endpoint()
MesosBatchSystem.add_options()
MesosBatchSystem.setOptions()
- toil.batchSystems.mesos.conftest
- toil.batchSystems.mesos.executor
- Attributes
- Classes
- Functions
- Module Contents
log
MesosExecutor
MesosExecutor.popenLock
MesosExecutor.runningTasks
MesosExecutor.workerCleanupInfo
MesosExecutor.address
MesosExecutor.id
MesosExecutor.registered()
MesosExecutor.reregistered()
MesosExecutor.disconnected()
MesosExecutor.killTask()
MesosExecutor.shutdown()
MesosExecutor.error()
MesosExecutor.launchTask()
MesosExecutor.frameworkMessage()
main()
- toil.batchSystems.mesos.test
- toil.batchSystems.mesos.batchSystem
- Classes
- Package Contents
- Submodules
- toil.batchSystems.options
- toil.batchSystems.registry
- Attributes
- Functions
- Module Contents
logger
add_batch_system_factory()
get_batch_systems()
get_batch_system()
DEFAULT_BATCH_SYSTEM
aws_batch_batch_system_factory()
gridengine_batch_system_factory()
lsf_batch_system_factory()
single_machine_batch_system_factory()
mesos_batch_system_factory()
slurm_batch_system_factory()
torque_batch_system_factory()
htcondor_batch_system_factory()
kubernetes_batch_system_factory()
__getattr__()
addBatchSystemFactory()
save_batch_system_plugin_state()
restore_batch_system_plugin_state()
- toil.batchSystems.singleMachine
- Attributes
- Classes
- Module Contents
logger
SingleMachineBatchSystem
SingleMachineBatchSystem.supportsAutoDeployment()
SingleMachineBatchSystem.supportsWorkerCleanup()
SingleMachineBatchSystem.numCores
SingleMachineBatchSystem.minCores
SingleMachineBatchSystem.physicalMemory
SingleMachineBatchSystem.config
SingleMachineBatchSystem.physicalDisk
SingleMachineBatchSystem.scale
SingleMachineBatchSystem.debugWorker
SingleMachineBatchSystem.jobIndex
SingleMachineBatchSystem.jobIndexLock
SingleMachineBatchSystem.jobs
SingleMachineBatchSystem.inputQueue
SingleMachineBatchSystem.outputQueue
SingleMachineBatchSystem.runningJobs
SingleMachineBatchSystem.children
SingleMachineBatchSystem.childToJob
SingleMachineBatchSystem.accelerator_identities
SingleMachineBatchSystem.resource_sources
SingleMachineBatchSystem.schedulingStatusMessage
SingleMachineBatchSystem.shuttingDown
SingleMachineBatchSystem.daddyThread
SingleMachineBatchSystem.daddyException
SingleMachineBatchSystem.daddy()
SingleMachineBatchSystem.getSchedulingStatusMessage()
SingleMachineBatchSystem.check_resource_request()
SingleMachineBatchSystem.issueBatchJob()
SingleMachineBatchSystem.killBatchJobs()
SingleMachineBatchSystem.getIssuedBatchJobIDs()
SingleMachineBatchSystem.getRunningBatchJobIDs()
SingleMachineBatchSystem.shutdown()
SingleMachineBatchSystem.getUpdatedBatchJob()
SingleMachineBatchSystem.add_options()
SingleMachineBatchSystem.setOptions()
Info
- toil.batchSystems.slurm
- Attributes
- Classes
- Functions
- Module Contents
logger
TERMINAL_STATES
NONTERMINAL_STATES
parse_slurm_time()
option_detector()
any_option_detector()
SlurmBatchSystem
SlurmBatchSystem.PartitionInfo
SlurmBatchSystem.PartitionSet
SlurmBatchSystem.GridEngineThread
SlurmBatchSystem.GridEngineThread.boss
SlurmBatchSystem.GridEngineThread.getRunningJobIDs()
SlurmBatchSystem.GridEngineThread.killJob()
SlurmBatchSystem.GridEngineThread.prepareSubmission()
SlurmBatchSystem.GridEngineThread.submitJob()
SlurmBatchSystem.GridEngineThread.coalesce_job_exit_codes()
SlurmBatchSystem.GridEngineThread.getJobExitCode()
SlurmBatchSystem.GridEngineThread.prepareSbatch()
SlurmBatchSystem.partitions
SlurmBatchSystem.issueBatchJob()
SlurmBatchSystem.add_options()
SlurmBatchSystem.OptionType
SlurmBatchSystem.setOptions()
- toil.batchSystems.torque
- Attributes
- Classes
- Module Contents
logger
TorqueBatchSystem
TorqueBatchSystem.GridEngineThread
TorqueBatchSystem.GridEngineThread.getRunningJobIDs()
TorqueBatchSystem.GridEngineThread.getUpdatedBatchJob()
TorqueBatchSystem.GridEngineThread.killJob()
TorqueBatchSystem.GridEngineThread.prepareSubmission()
TorqueBatchSystem.GridEngineThread.submitJob()
TorqueBatchSystem.GridEngineThread.getJobExitCode()
TorqueBatchSystem.GridEngineThread.prepareQsub()
TorqueBatchSystem.GridEngineThread.generateTorqueWrapper()
- toil.batchSystems.abstractBatchSystem
- Exceptions
- Package Contents
- Submodules
- toil.bus
- Attributes
- Classes
- Functions
- Module Contents
logger
Names
get_job_kind()
JobIssuedMessage
JobUpdatedMessage
JobCompletedMessage
JobFailedMessage
JobMissingMessage
JobAnnotationMessage
ExternalBatchIdMessage
QueueSizeMessage
ClusterSizeMessage
ClusterDesiredSizeMessage
message_to_bytes()
MessageType
bytes_to_message()
MessageBus
MessageBusClient
MessageInbox
MessageOutbox
MessageBusConnection
JobStatus
replay_message_bus()
gen_message_bus_path()
- toil.common
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
UUID_LENGTH
logger
get_default_config_path()
Config
Config.logFile
Config.logRotating
Config.cleanWorkDir
Config.max_jobs
Config.max_local_jobs
Config.manualMemArgs
Config.run_local_jobs_on_workers
Config.coalesceStatusCalls
Config.mesos_endpoint
Config.mesos_framework_id
Config.mesos_role
Config.mesos_name
Config.kubernetes_host_path
Config.kubernetes_owner
Config.kubernetes_service_account
Config.kubernetes_pod_timeout
Config.kubernetes_privileged
Config.kubernetes_pod_security_context
Config.kubernetes_security_context
Config.tes_endpoint
Config.tes_user
Config.tes_password
Config.tes_bearer_token
Config.aws_batch_region
Config.aws_batch_queue
Config.aws_batch_job_role_arn
Config.scale
Config.batchSystem
Config.batch_logs_dir
Config.statePollingWait
Config.state_polling_timeout
Config.disableAutoDeployment
Config.workflowID
Config.workflowAttemptNumber
Config.jobStore
Config.logLevel
Config.colored_logs
Config.workDir
Config.coordination_dir
Config.noStdOutErr
Config.stats
Config.clean
Config.clusterStats
Config.restart
Config.caching
Config.symlinkImports
Config.moveOutputs
Config.symlink_job_store_reads
Config.provisioner
Config.nodeTypes
Config.minNodes
Config.maxNodes
Config.targetTime
Config.betaInertia
Config.scaleInterval
Config.preemptibleCompensation
Config.nodeStorage
Config.nodeStorageOverrides
Config.metrics
Config.assume_zero_overhead
Config.maxPreemptibleServiceJobs
Config.maxServiceJobs
Config.deadlockWait
Config.deadlockCheckInterval
Config.defaultMemory
Config.defaultCores
Config.defaultDisk
Config.defaultPreemptible
Config.defaultAccelerators
Config.maxCores
Config.maxMemory
Config.maxDisk
Config.retryCount
Config.stop_on_first_failure
Config.enableUnlimitedPreemptibleRetries
Config.doubleMem
Config.maxJobDuration
Config.rescueJobsFrequency
Config.job_store_timeout
Config.maxLogFileSize
Config.writeLogs
Config.writeLogsGzip
Config.writeLogsFromAllJobs
Config.write_messages
Config.realTimeLogging
Config.publish_workflow_metrics
Config.environment
Config.disableChaining
Config.disableJobStoreChecksumVerification
Config.sseKey
Config.servicePollingInterval
Config.useAsync
Config.forceDockerAppliance
Config.statusWait
Config.disableProgress
Config.readGlobalFileMutableByDefault
Config.debugWorker
Config.disableWorkerOutputCapture
Config.badWorker
Config.badWorkerFailInterval
Config.kill_polling_interval
Config.cwl
Config.memory_is_product
Config.set_from_default_config()
Config.prepare_start()
Config.prepare_restart()
Config.setOptions()
Config.check_configuration_consistency()
Config.__eq__()
Config.__hash__()
ensure_config()
generate_config()
update_config()
parser_with_common_options()
addOptions()
getNodeID()
Toil
Toil.config
Toil.options
Toil.__enter__()
Toil.__exit__()
Toil.start()
Toil.restart()
Toil.JOB_STORE_TYPES
Toil.getJobStore()
Toil.parseLocator()
Toil.buildLocator()
Toil.canonical_locator()
Toil.resumeJobStore()
Toil.createBatchSystem()
Toil.url_exists()
Toil.importFile()
Toil.import_file()
Toil.exportFile()
Toil.export_file()
Toil.normalize_uri()
Toil.getToilWorkDir()
Toil.get_toil_coordination_dir()
Toil.get_workflow_path_component()
Toil.getLocalWorkflowDir()
Toil.get_local_workflow_coordination_dir()
ToilRestartException
ToilContextManagerException
ToilMetrics
ToilMetrics.mtailImage
ToilMetrics.grafanaImage
ToilMetrics.prometheusImage
ToilMetrics.nodeExporterProc
ToilMetrics.startDashboard()
ToilMetrics.add_prometheus_data_source()
ToilMetrics.log()
ToilMetrics.logClusterSize()
ToilMetrics.logClusterDesiredSize()
ToilMetrics.logQueueSize()
ToilMetrics.logMissingJob()
ToilMetrics.logIssuedJob()
ToilMetrics.logFailedJob()
ToilMetrics.logCompletedJob()
ToilMetrics.shutdown()
cacheDirName()
getDirSizeRecursively()
getFileSystemSize()
safeUnpickleFromStream()
- toil.cwl
- Submodules
- toil.cwl.conftest
- toil.cwl.cwltoil
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
logger
DEFAULT_TMPDIR
DEFAULT_TMPDIR_PREFIX
cwltoil_was_removed()
UnresolvedDict
SkipNull
filter_skip_null()
ensure_no_collisions()
try_prepull()
Conditional
ResolveSource
StepValueFrom
DefaultWithSource
JustAValue
resolve_dict_w_promises()
simplify_list()
ToilPathMapper
ToilSingleJobExecutor
ToilTool
ToilCommandLineTool
ToilExpressionTool
toil_make_tool()
MISSING_FILE
DirectoryContents
check_directory_dict_invariants()
decode_directory()
encode_directory()
ToilFsAccess
toil_get_file()
convert_file_uri_to_toil_uri()
path_to_loc()
extract_file_uri_once()
V
VisitFunc
visit_files()
upload_directory()
extract_and_convert_file_to_toil_uri()
writeGlobalFileWrapper()
remove_empty_listings()
CWLNamedJob
ResolveIndirect
toilStageFiles()
CWLJobWrapper
CWLJob
get_container_engine()
makeRootJob()
makeJob()
CWLScatter
CWLGather
SelfJob
ProcessType
remove_pickle_problems()
CWLWorkflow
CWLInstallImportsJob
CWLImportWrapper
CWLStartJob
extract_workflow_inputs()
import_workflow_inputs()
T
visitSteps()
rm_unprocessed_secondary_files()
filtered_secondary_files()
scan_for_unsupported_requirements()
determine_load_listing()
NoAvailableJobStoreException
generate_default_job_store()
usage_message
get_options()
main()
find_default_container()
- toil.cwl.utils
- Functions
- Package Contents
- Submodules
- toil.deferred
- Attributes
- Classes
- Module Contents
logger
DeferredFunction
DeferredFunctionManager
DeferredFunctionManager.STATE_DIR_STEM
DeferredFunctionManager.PREFIX
DeferredFunctionManager.WIP_SUFFIX
DeferredFunctionManager.stateDir
DeferredFunctionManager.stateFileName
DeferredFunctionManager.stateFileOut
DeferredFunctionManager.stateFileIn
DeferredFunctionManager.__del__()
DeferredFunctionManager.open()
DeferredFunctionManager.cleanupWorker()
- toil.exceptions
- toil.fileStores
- Submodules
- toil.fileStores.abstractFileStore
- Attributes
- Classes
- Module Contents
logger
AbstractFileStore
AbstractFileStore.jobStore
AbstractFileStore.jobDesc
AbstractFileStore.localTempDir
AbstractFileStore.workflow_dir
AbstractFileStore.coordination_dir
AbstractFileStore.jobName
AbstractFileStore.waitForPreviousCommit
AbstractFileStore.logging_messages
AbstractFileStore.logging_user_streams
AbstractFileStore.filesToDelete
AbstractFileStore.createFileStore()
AbstractFileStore.shutdownFileStore()
AbstractFileStore.open()
AbstractFileStore.get_disk_usage()
AbstractFileStore.getLocalTempDir()
AbstractFileStore.getLocalTempFile()
AbstractFileStore.getLocalTempFileName()
AbstractFileStore.writeGlobalFile()
AbstractFileStore.writeGlobalFileStream()
AbstractFileStore.logAccess()
AbstractFileStore.readGlobalFile()
AbstractFileStore.readGlobalFileStream()
AbstractFileStore.getGlobalFileSize()
AbstractFileStore.deleteLocalFile()
AbstractFileStore.deleteGlobalFile()
AbstractFileStore.importFile()
AbstractFileStore.import_file()
AbstractFileStore.exportFile()
AbstractFileStore.export_file()
AbstractFileStore.log_to_leader()
AbstractFileStore.logToMaster()
AbstractFileStore.log_user_stream()
AbstractFileStore.startCommit()
AbstractFileStore.waitForCommit()
AbstractFileStore.shutdown()
- toil.fileStores.cachingFileStore
- Attributes
- Exceptions
- Classes
- Module Contents
logger
SQLITE_TIMEOUT_SECS
CacheError
CacheUnbalancedError
IllegalDeletionCacheError
InvalidSourceCacheError
CachingFileStore
CachingFileStore.forceNonFreeCaching
CachingFileStore.forceDownloadDelay
CachingFileStore.contentionBackoff
CachingFileStore.localCacheDir
CachingFileStore.jobName
CachingFileStore.jobID
CachingFileStore.jobDiskBytes
CachingFileStore.workflowAttemptNumber
CachingFileStore.dbPath
CachingFileStore.process_identity_lock
CachingFileStore.commitThread
CachingFileStore.as_process()
CachingFileStore.con
CachingFileStore.cur
CachingFileStore.getCacheLimit()
CachingFileStore.getCacheUsed()
CachingFileStore.getCacheExtraJobSpace()
CachingFileStore.getCacheAvailable()
CachingFileStore.getSpaceUsableForJobs()
CachingFileStore.getCacheUnusedJobRequirement()
CachingFileStore.adjustCacheLimit()
CachingFileStore.fileIsCached()
CachingFileStore.getFileReaderCount()
CachingFileStore.cachingIsFree()
CachingFileStore.open()
CachingFileStore.writeGlobalFile()
CachingFileStore.readGlobalFile()
CachingFileStore.readGlobalFileStream()
CachingFileStore.deleteLocalFile()
CachingFileStore.deleteGlobalFile()
CachingFileStore.exportFile()
CachingFileStore.export_file()
CachingFileStore.waitForCommit()
CachingFileStore.startCommit()
CachingFileStore.startCommitThread()
CachingFileStore.shutdown()
CachingFileStore.__del__()
- toil.fileStores.nonCachingFileStore
- Attributes
- Classes
- Module Contents
logger
NonCachingFileStore
NonCachingFileStore.jobStateFile
NonCachingFileStore.localFileMap
NonCachingFileStore.check_for_coordination_corruption()
NonCachingFileStore.check_for_state_corruption()
NonCachingFileStore.open()
NonCachingFileStore.writeGlobalFile()
NonCachingFileStore.readGlobalFile()
NonCachingFileStore.readGlobalFileStream()
NonCachingFileStore.exportFile()
NonCachingFileStore.export_file()
NonCachingFileStore.deleteLocalFile()
NonCachingFileStore.deleteGlobalFile()
NonCachingFileStore.waitForCommit()
NonCachingFileStore.startCommit()
NonCachingFileStore.__del__()
NonCachingFileStore.shutdown()
- toil.fileStores.abstractFileStore
- Classes
- Package Contents
- Submodules
- toil.job
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
logger
JobPromiseConstraintError
ConflictingPredecessorError
DebugStoppingPointReached
FilesDownloadedStoppingPointReached
TemporaryID
AcceleratorRequirement
parse_accelerator()
accelerator_satisfies()
accelerators_fully_satisfy()
RequirementsDict
REQUIREMENT_NAMES
ParsedRequirement
ParseableIndivisibleResource
ParseableDivisibleResource
ParseableFlag
ParseableAcceleratorRequirement
ParseableRequirement
Requirer
JobBodyReference
JobDescription
JobDescription.local
JobDescription.jobName
JobDescription.unitName
JobDescription.displayName
JobDescription.jobStoreID
JobDescription.filesToDelete
JobDescription.predecessorNumber
JobDescription.predecessorsFinished
JobDescription.childIDs
JobDescription.followOnIDs
JobDescription.successor_phases
JobDescription.serviceTree
JobDescription.logJobStoreFileID
JobDescription.files_to_use
JobDescription.get_names()
JobDescription.get_chain()
JobDescription.serviceHostIDsInBatches()
JobDescription.successorsAndServiceHosts()
JobDescription.allSuccessors()
JobDescription.successors_by_phase()
JobDescription.services
JobDescription.has_body()
JobDescription.attach_body()
JobDescription.detach_body()
JobDescription.get_body()
JobDescription.nextSuccessors()
JobDescription.filterSuccessors()
JobDescription.filterServiceHosts()
JobDescription.clear_nonexistent_dependents()
JobDescription.clear_dependents()
JobDescription.is_subtree_done()
JobDescription.replace()
JobDescription.assert_is_not_newer_than()
JobDescription.is_updated_by()
JobDescription.addChild()
JobDescription.addFollowOn()
JobDescription.addServiceHostJob()
JobDescription.hasChild()
JobDescription.hasFollowOn()
JobDescription.hasServiceHostJob()
JobDescription.renameReferences()
JobDescription.addPredecessor()
JobDescription.onRegistration()
JobDescription.setupJobAfterFailure()
JobDescription.getLogFileHandle()
JobDescription.remainingTryCount
JobDescription.clearRemainingTryCount()
JobDescription.__str__()
JobDescription.__repr__()
JobDescription.reserve_versions()
JobDescription.pre_update_hook()
ServiceJobDescription
CheckpointJobDescription
Job
Job.userModule
Job.__str__()
Job.check_initialized()
Job.jobStoreID
Job.description
Job.disk
Job.memory
Job.cores
Job.accelerators
Job.preemptible
Job.preemptable()
Job.checkpoint
Job.files_to_use
Job.add_to_files_to_use()
Job.remove_from_files_to_use()
Job.assignConfig()
Job.run()
Job.addChild()
Job.hasChild()
Job.addFollowOn()
Job.hasPredecessor()
Job.hasFollowOn()
Job.addService()
Job.hasService()
Job.addChildFn()
Job.addFollowOnFn()
Job.addChildJobFn()
Job.addFollowOnJobFn()
Job.tempDir
Job.log()
Job.wrapFn()
Job.wrapJobFn()
Job.encapsulate()
Job.rv()
Job.registerPromise()
Job.prepareForPromiseRegistration()
Job.checkJobGraphForDeadlocks()
Job.getRootJobs()
Job.checkJobGraphConnected()
Job.checkJobGraphAcylic()
Job.checkNewCheckpointsAreLeafVertices()
Job.defer()
Job.Runner
Job.Service
Job.getUserScript()
Job.getTopologicalOrderingOfJobs()
Job.saveBody()
Job.saveAsRootJob()
Job.loadJob()
Job.set_debug_flag()
Job.has_debug_flag()
Job.files_downloaded_hook()
JobException
JobGraphDeadlockException
FunctionWrappingJob
JobFunctionWrappingJob
PromisedRequirementFunctionWrappingJob
PromisedRequirementJobFunctionWrappingJob
EncapsulatedJob
ServiceHostJob
FileMetadata
potential_absolute_uris()
get_file_sizes()
CombineImportsJob
WorkerImportJob
ImportsJob
Promise
T
Promised
unwrap()
unwrap_all()
PromisedRequirement
UnfulfilledPromiseSentinel
- toil.jobStores
- Submodules
- toil.jobStores.abstractJobStore
- Attributes
- Exceptions
- Classes
- Module Contents
logger
ProxyConnectionError
LocatorException
InvalidImportExportUrlException
NoSuchJobException
ConcurrentFileModificationException
NoSuchFileException
NoSuchJobStoreException
JobStoreExistsException
AbstractJobStore
AbstractJobStore.initialize()
AbstractJobStore.writeConfig()
AbstractJobStore.write_config()
AbstractJobStore.resume()
AbstractJobStore.config
AbstractJobStore.locator
AbstractJobStore.rootJobStoreIDFileName
AbstractJobStore.setRootJob()
AbstractJobStore.set_root_job()
AbstractJobStore.loadRootJob()
AbstractJobStore.load_root_job()
AbstractJobStore.createRootJob()
AbstractJobStore.create_root_job()
AbstractJobStore.getRootJobReturnValue()
AbstractJobStore.get_root_job_return_value()
AbstractJobStore.importFile()
AbstractJobStore.import_file()
AbstractJobStore.exportFile()
AbstractJobStore.export_file()
AbstractJobStore.url_exists()
AbstractJobStore.get_size()
AbstractJobStore.get_is_directory()
AbstractJobStore.list_url()
AbstractJobStore.read_from_url()
AbstractJobStore.open_url()
AbstractJobStore.destroy()
AbstractJobStore.getEnv()
AbstractJobStore.get_env()
AbstractJobStore.clean()
AbstractJobStore.assignID()
AbstractJobStore.assign_job_id()
AbstractJobStore.batch()
AbstractJobStore.create()
AbstractJobStore.create_job()
AbstractJobStore.exists()
AbstractJobStore.job_exists()
AbstractJobStore.publicUrlExpiration
AbstractJobStore.getPublicUrl()
AbstractJobStore.get_public_url()
AbstractJobStore.getSharedPublicUrl()
AbstractJobStore.get_shared_public_url()
AbstractJobStore.load()
AbstractJobStore.load_job()
AbstractJobStore.update()
AbstractJobStore.update_job()
AbstractJobStore.delete()
AbstractJobStore.delete_job()
AbstractJobStore.jobs()
AbstractJobStore.writeFile()
AbstractJobStore.write_file()
AbstractJobStore.writeFileStream()
AbstractJobStore.write_file_stream()
AbstractJobStore.getEmptyFileStoreID()
AbstractJobStore.get_empty_file_store_id()
AbstractJobStore.readFile()
AbstractJobStore.read_file()
AbstractJobStore.readFileStream()
AbstractJobStore.read_file_stream()
AbstractJobStore.deleteFile()
AbstractJobStore.delete_file()
AbstractJobStore.fileExists()
AbstractJobStore.file_exists()
AbstractJobStore.getFileSize()
AbstractJobStore.get_file_size()
AbstractJobStore.updateFile()
AbstractJobStore.update_file()
AbstractJobStore.updateFileStream()
AbstractJobStore.update_file_stream()
AbstractJobStore.sharedFileNameRegex
AbstractJobStore.writeSharedFileStream()
AbstractJobStore.write_shared_file_stream()
AbstractJobStore.readSharedFileStream()
AbstractJobStore.read_shared_file_stream()
AbstractJobStore.writeStatsAndLogging()
AbstractJobStore.write_logs()
AbstractJobStore.readStatsAndLogging()
AbstractJobStore.read_logs()
AbstractJobStore.write_leader_pid()
AbstractJobStore.read_leader_pid()
AbstractJobStore.write_leader_node_id()
AbstractJobStore.read_leader_node_id()
AbstractJobStore.write_kill_flag()
AbstractJobStore.read_kill_flag()
AbstractJobStore.default_caching()
JobStoreSupport
- toil.jobStores.aws
- Submodules
- toil.jobStores.aws.jobStore
- Attributes
- Exceptions
- Classes
- Module Contents
boto3_session
s3_boto3_resource
s3_boto3_client
logger
CONSISTENCY_TICKS
CONSISTENCY_TIME
ChecksumError
DomainDoesNotExist
AWSJobStore
AWSJobStore.bucketNameRe
AWSJobStore.minBucketNameLen
AWSJobStore.maxBucketNameLen
AWSJobStore.maxNameLen
AWSJobStore.nameSeparator
AWSJobStore.region
AWSJobStore.name_prefix
AWSJobStore.part_size
AWSJobStore.jobs_domain_name
AWSJobStore.files_domain_name
AWSJobStore.files_bucket
AWSJobStore.db
AWSJobStore.s3_resource
AWSJobStore.s3_client
AWSJobStore.initialize()
AWSJobStore.sseKeyPath
AWSJobStore.resume()
AWSJobStore.jobsPerBatchInsert
AWSJobStore.batch()
AWSJobStore.assign_job_id()
AWSJobStore.create_job()
AWSJobStore.job_exists()
AWSJobStore.jobs()
AWSJobStore.load_job()
AWSJobStore.update_job()
AWSJobStore.itemsPerBatchDelete
AWSJobStore.delete_job()
AWSJobStore.get_empty_file_store_id()
AWSJobStore.write_file()
AWSJobStore.write_file_stream()
AWSJobStore.write_shared_file_stream()
AWSJobStore.update_file()
AWSJobStore.update_file_stream()
AWSJobStore.file_exists()
AWSJobStore.get_file_size()
AWSJobStore.read_file()
AWSJobStore.read_file_stream()
AWSJobStore.read_shared_file_stream()
AWSJobStore.delete_file()
AWSJobStore.write_logs()
AWSJobStore.read_logs()
AWSJobStore.get_public_url()
AWSJobStore.get_shared_public_url()
AWSJobStore.sharedFileOwnerID
AWSJobStore.statsFileOwnerID
AWSJobStore.readStatsFileOwnerID
AWSJobStore.FileInfo
AWSJobStore.FileInfo.outer
AWSJobStore.FileInfo.encrypted
AWSJobStore.FileInfo.fileID
AWSJobStore.FileInfo.ownerID
AWSJobStore.FileInfo.version
AWSJobStore.FileInfo.previousVersion
AWSJobStore.FileInfo.content
AWSJobStore.FileInfo.checksum
AWSJobStore.FileInfo.create()
AWSJobStore.FileInfo.presenceIndicator()
AWSJobStore.FileInfo.exists()
AWSJobStore.FileInfo.load()
AWSJobStore.FileInfo.loadOrCreate()
AWSJobStore.FileInfo.loadOrFail()
AWSJobStore.FileInfo.fromItem()
AWSJobStore.FileInfo.toItem()
AWSJobStore.FileInfo.maxInlinedSize()
AWSJobStore.FileInfo.save()
AWSJobStore.FileInfo.upload()
AWSJobStore.FileInfo.uploadStream()
AWSJobStore.FileInfo.copyFrom()
AWSJobStore.FileInfo.copyTo()
AWSJobStore.FileInfo.download()
AWSJobStore.FileInfo.downloadStream()
AWSJobStore.FileInfo.delete()
AWSJobStore.FileInfo.getSize()
AWSJobStore.FileInfo.__repr__()
AWSJobStore.versionings
AWSJobStore.destroy()
aRepr
custom_repr
BucketLocationConflictException
- toil.jobStores.aws.utils
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
- toil.jobStores.aws.jobStore
- Submodules
- toil.jobStores.conftest
- toil.jobStores.fileJobStore
- Attributes
- Classes
- Module Contents
logger
FileJobStore
FileJobStore.validDirs
FileJobStore.validDirsSet
FileJobStore.JOB_DIR_PREFIX
FileJobStore.JOB_NAME_DIR_PREFIX
FileJobStore.BUFFER_SIZE
FileJobStore.LOG_TEMP_SUFFIX
FileJobStore.LOG_PREFIX
FileJobStore.default_caching()
FileJobStore.jobStoreDir
FileJobStore.jobsDir
FileJobStore.statsDir
FileJobStore.stats_inbox
FileJobStore.stats_archive
FileJobStore.filesDir
FileJobStore.jobFilesDir
FileJobStore.sharedFilesDir
FileJobStore.fanOut
FileJobStore.linkImports
FileJobStore.moveExports
FileJobStore.symlink_job_store_reads
FileJobStore.__repr__()
FileJobStore.initialize()
FileJobStore.resume()
FileJobStore.destroy()
FileJobStore.assign_job_id()
FileJobStore.create_job()
FileJobStore.batch()
FileJobStore.job_exists()
FileJobStore.get_public_url()
FileJobStore.get_shared_public_url()
FileJobStore.load_job()
FileJobStore.update_job()
FileJobStore.delete_job()
FileJobStore.jobs()
FileJobStore.write_file()
FileJobStore.write_file_stream()
FileJobStore.get_empty_file_store_id()
FileJobStore.update_file()
FileJobStore.read_file()
FileJobStore.delete_file()
FileJobStore.file_exists()
FileJobStore.get_file_size()
FileJobStore.update_file_stream()
FileJobStore.read_file_stream()
FileJobStore.write_shared_file_stream()
FileJobStore.read_shared_file_stream()
FileJobStore.list_all_file_names()
FileJobStore.write_logs()
FileJobStore.read_logs()
- toil.jobStores.googleJobStore
- Attributes
- Classes
- Functions
- Module Contents
log
GOOGLE_STORAGE
MAX_BATCH_SIZE
google_retry_predicate()
google_retry()
permission_error_reporter()
GoogleJobStore
GoogleJobStore.nodeServiceAccountJson
GoogleJobStore.projectID
GoogleJobStore.bucketName
GoogleJobStore.bucket
GoogleJobStore.statsBaseID
GoogleJobStore.statsReadPrefix
GoogleJobStore.readStatsBaseID
GoogleJobStore.sseKey
GoogleJobStore.create_client()
GoogleJobStore.initialize()
GoogleJobStore.resume()
GoogleJobStore.destroy()
GoogleJobStore.assign_job_id()
GoogleJobStore.batch()
GoogleJobStore.create_job()
GoogleJobStore.job_exists()
GoogleJobStore.get_public_url()
GoogleJobStore.get_shared_public_url()
GoogleJobStore.load_job()
GoogleJobStore.update_job()
GoogleJobStore.delete_job()
GoogleJobStore.get_env()
GoogleJobStore.jobs()
GoogleJobStore.write_file()
GoogleJobStore.write_file_stream()
GoogleJobStore.get_empty_file_store_id()
GoogleJobStore.read_file()
GoogleJobStore.read_file_stream()
GoogleJobStore.delete_file()
GoogleJobStore.file_exists()
GoogleJobStore.get_file_size()
GoogleJobStore.update_file()
GoogleJobStore.update_file_stream()
GoogleJobStore.write_shared_file_stream()
GoogleJobStore.read_shared_file_stream()
GoogleJobStore.write_logs()
GoogleJobStore.read_logs()
- toil.jobStores.utils
- toil.jobStores.abstractJobStore
- Submodules
- toil.leader
- Attributes
- Classes
- Module Contents
logger
Leader
Leader.config
Leader.jobStore
Leader.jobStoreLocator
Leader.toilState
Leader.batchSystem
Leader.issued_jobs_by_batch_system_id
Leader.preemptibleJobsIssued
Leader.serviceJobsIssued
Leader.serviceJobsToBeIssued
Leader.preemptibleServiceJobsIssued
Leader.preemptibleServiceJobsToBeIssued
Leader.timeSinceJobsLastRescued
Leader.reissueMissingJobs_missingHash
Leader.provisioner
Leader.clusterScaler
Leader.serviceManager
Leader.statsAndLogging
Leader.potentialDeadlockedJobs
Leader.potentialDeadlockTime
Leader.toilMetrics
Leader.debugJobNames
Leader.deadlockThrottler
Leader.statusThrottler
Leader.kill_throttler
Leader.progress_overall
Leader.progress_failed
Leader.GOOD_COLOR
Leader.BAD_COLOR
Leader.PROGRESS_BAR_FORMAT
Leader.recommended_fail_exit_code
Leader.run()
Leader.create_status_sentinel_file()
Leader.innerLoop()
Leader.checkForDeadlocks()
Leader.feed_deadlock_watchdog()
Leader.issueJob()
Leader.issueJobs()
Leader.issueServiceJob()
Leader.issueQueingServiceJobs()
Leader.getNumberOfJobsIssued()
Leader.removeJob()
Leader.getJobs()
Leader.killJobs()
Leader.reissueOverLongJobs()
Leader.reissueMissingJobs()
Leader.processRemovedJob()
Leader.process_finished_job()
Leader.process_finished_job_description()
Leader.getSuccessors()
Leader.processTotallyFailedJob()
- toil.lib
- Submodules
- toil.lib.accelerators
- toil.lib.aws
- Submodules
- toil.lib.aws.ami
- toil.lib.aws.iam
- Attributes
- Functions
- Module Contents
logger
CLUSTER_LAUNCHING_PERMISSIONS
AllowedActionCollection
delete_iam_instance_profile()
delete_iam_role()
create_iam_role()
init_action_collection()
add_to_action_collection()
policy_permissions_allow()
permission_matches_any()
get_actions_from_policy_document()
allowed_actions_attached()
allowed_actions_roles()
collect_policy_actions()
allowed_actions_user()
allowed_actions_group()
get_policy_permissions()
get_aws_account_num()
- toil.lib.aws.s3
- toil.lib.aws.session
- toil.lib.aws.utils
- Attributes
- Exceptions
- Functions
- Module Contents
logger
THROTTLED_ERROR_CODES
delete_sdb_domain()
connection_reset()
connection_error()
retryable_s3_errors()
retry_s3()
delete_s3_bucket()
create_s3_bucket()
enable_public_objects()
NoBucketLocationError
get_bucket_region()
get_bucket_region_if_available()
region_to_bucket_location()
bucket_location_to_region()
get_object_for_url()
list_objects_for_url()
flatten_tags()
boto3_pager()
get_item_from_attributes()
- Attributes
- Functions
- Package Contents
- Submodules
- toil.lib.bioio
- toil.lib.compatibility
- toil.lib.conversions
- toil.lib.docker
- toil.lib.dockstore
- toil.lib.ec2
- Attributes
- Exceptions
- Functions
- Module Contents
a_short_time
a_long_time
logger
UserError
not_found()
inconsistencies_detected()
INCONSISTENCY_ERRORS
retry_ec2()
UnexpectedResourceState
wait_transition()
wait_instances_running()
wait_spot_requests_active()
create_spot_instances()
create_ondemand_instances()
increase_instance_hop_limit()
prune()
iam_client
wait_until_instance_profile_arn_exists()
create_instances()
create_launch_template()
create_auto_scaling_group()
- toil.lib.ec2nodes
- toil.lib.encryption
- toil.lib.exceptions
- toil.lib.expando
- toil.lib.ftp_utils
- toil.lib.generatedEC2Lists
- toil.lib.history
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
logger
HistoryDatabaseSchemaTooNewError
WorkflowSummary
WorkflowAttemptSummary
WorkflowAttemptSummary.workflow_id
WorkflowAttemptSummary.attempt_number
WorkflowAttemptSummary.succeeded
WorkflowAttemptSummary.start_time
WorkflowAttemptSummary.runtime
WorkflowAttemptSummary.submitted_to_dockstore
WorkflowAttemptSummary.batch_system
WorkflowAttemptSummary.caching
WorkflowAttemptSummary.toil_version
WorkflowAttemptSummary.python_version
WorkflowAttemptSummary.platform_system
WorkflowAttemptSummary.platform_machine
WorkflowAttemptSummary.workflow_job_store
WorkflowAttemptSummary.workflow_trs_spec
JobAttemptSummary
RT
db_retry()
HistoryManager
HistoryManager.enabled()
HistoryManager.enabled_job()
HistoryManager.database_path_override
HistoryManager.database_path()
HistoryManager.connection()
HistoryManager.no_transaction()
HistoryManager.ensure_tables()
HistoryManager.record_workflow_creation()
HistoryManager.record_workflow_metadata()
HistoryManager.record_job_attempt()
HistoryManager.record_workflow_attempt()
HistoryManager.summarize_workflows()
HistoryManager.get_submittable_workflow_attempts()
HistoryManager.get_workflow_attempts_with_submittable_job_attempts()
HistoryManager.get_workflow_attempt()
HistoryManager.get_unsubmitted_job_attempts()
HistoryManager.mark_workflow_attempt_submitted()
HistoryManager.mark_job_attempts_submitted()
HistoryManager.count_workflows()
HistoryManager.count_workflow_attempts()
HistoryManager.count_job_attempts()
HistoryManager.get_fully_submitted_workflow_ids()
HistoryManager.get_oldest_workflow_ids()
HistoryManager.delete_workflow()
HistoryManager.get_database_byte_size()
HistoryManager.compact_database()
HistoryManager.enforce_byte_size_limit()
HistoryManager.database_dump_lines()
- toil.lib.history_submission
- Attributes
- Classes
- Functions
- Module Contents
logger
workflow_execution_id()
workflow_task_set_execution_id()
job_execution_id()
get_parsed_trs_spec()
Submission
create_history_submission()
create_current_submission()
KeyType
dialog_tkinter()
display_dialog_tkinter()
dialog_tui()
Decision
DIALOG_TITLE
DIALOG_TEXT
DIALOG_OPTIONS
DIALOG_TIMEOUT
ask_user_about_publishing_metrics()
- toil.lib.humanize
- toil.lib.io
- Attributes
- Classes
- Functions
- Module Contents
logger
get_toil_home()
TOIL_URI_SCHEME
STANDARD_SCHEMES
REMOTE_SCHEMES
ALL_SCHEMES
is_standard_url()
is_remote_url()
is_any_url()
is_url_with_scheme()
is_toil_url()
is_file_url()
mkdtemp()
robust_rmtree()
atomic_tmp_file()
atomic_install()
AtomicFileCreate()
atomic_copy()
atomic_copyobj()
make_public_dir()
try_path()
WriteWatchingStream
ReadableFileObj
file_digest()
- toil.lib.iterables
- toil.lib.memoize
- toil.lib.misc
- toil.lib.objects
- toil.lib.resources
- toil.lib.retry
- Attributes
- Classes
- Functions
- Module Contents
SUPPORTED_HTTP_ERRORS
kubernetes
botocore
logger
ErrorCondition
RT
retry()
return_status_code()
get_error_code()
get_error_message()
get_error_status()
get_error_body()
meets_error_message_condition()
meets_error_code_condition()
meets_boto_error_code_condition()
error_meets_conditions()
DEFAULT_DELAYS
DEFAULT_TIMEOUT
E
old_retry()
retry_flaky_test
- toil.lib.threading
- Attributes
- Classes
- Functions
- Module Contents
- toil.lib.throttle
- toil.lib.trs
- toil.lib.web
- Submodules
- toil.options
- toil.provisioners
- Submodules
- toil.provisioners.abstractProvisioner
- Attributes
- Exceptions
- Classes
- Module Contents
a_short_time
logger
ManagedNodesNotSupportedException
Shape
AbstractProvisioner
AbstractProvisioner.LEADER_HOME_DIR
AbstractProvisioner.cloud
AbstractProvisioner.clusterName
AbstractProvisioner.clusterType
AbstractProvisioner.enable_fuse
AbstractProvisioner.supportedClusterTypes()
AbstractProvisioner.createClusterSettings()
AbstractProvisioner.readClusterSettings()
AbstractProvisioner.setAutoscaledNodeTypes()
AbstractProvisioner.hasAutoscaledNodeTypes()
AbstractProvisioner.getAutoscaledInstanceShapes()
AbstractProvisioner.retryPredicate()
AbstractProvisioner.launchCluster()
AbstractProvisioner.addNodes()
AbstractProvisioner.addManagedNodes()
AbstractProvisioner.terminateNodes()
AbstractProvisioner.getLeader()
AbstractProvisioner.getProvisionedWorkers()
AbstractProvisioner.getNodeShape()
AbstractProvisioner.destroyCluster()
AbstractProvisioner.InstanceConfiguration
AbstractProvisioner.InstanceConfiguration.files
AbstractProvisioner.InstanceConfiguration.units
AbstractProvisioner.InstanceConfiguration.sshPublicKeys
AbstractProvisioner.InstanceConfiguration.addFile()
AbstractProvisioner.InstanceConfiguration.addUnit()
AbstractProvisioner.InstanceConfiguration.addSSHRSAKey()
AbstractProvisioner.InstanceConfiguration.toIgnitionConfig()
AbstractProvisioner.getBaseInstanceConfiguration()
AbstractProvisioner.addVolumesService()
AbstractProvisioner.addNodeExporterService()
AbstractProvisioner.toil_service_env_options()
AbstractProvisioner.add_toil_service()
AbstractProvisioner.getKubernetesValues()
AbstractProvisioner.addKubernetesServices()
AbstractProvisioner.getKubernetesAutoscalerSetupCommands()
AbstractProvisioner.getKubernetesCloudProvider()
AbstractProvisioner.addKubernetesLeader()
AbstractProvisioner.addKubernetesWorker()
- toil.provisioners.aws
- Submodules
- toil.provisioners.aws.awsProvisioner
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
logger
awsRetryPredicate()
expectedShutdownErrors()
F
awsRetry()
awsFilterImpairedNodes()
InvalidClusterStateException
collapse_tags()
AWSProvisioner
AWSProvisioner.cloud
AWSProvisioner.aws
AWSProvisioner.s3_bucket_name
AWSProvisioner.supportedClusterTypes()
AWSProvisioner.createClusterSettings()
AWSProvisioner.readClusterSettings()
AWSProvisioner.launchCluster()
AWSProvisioner.toil_service_env_options()
AWSProvisioner.getKubernetesAutoscalerSetupCommands()
AWSProvisioner.getKubernetesCloudProvider()
AWSProvisioner.getNodeShape()
AWSProvisioner.retryPredicate()
AWSProvisioner.destroyCluster()
AWSProvisioner.terminateNodes()
AWSProvisioner.addNodes()
AWSProvisioner.addManagedNodes()
AWSProvisioner.getProvisionedWorkers()
AWSProvisioner.getLeader()
AWSProvisioner.full_policy()
AWSProvisioner.kubernetes_policy()
- toil.provisioners.aws.awsProvisioner
- Attributes
- Classes
- Functions
- Package Contents
- Submodules
- toil.provisioners.clusterScaler
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
logger
EVICTION_THRESHOLD
RESERVE_SMALL_LIMIT
RESERVE_SMALL_AMOUNT
RESERVE_BREAKPOINTS
RESERVE_FRACTIONS
OS_SIZE
FailedConstraint
BinPackedFit
NodeReservation
adjustEndingReservationForJob()
split()
binPacking()
ClusterScaler
ClusterScaler.provisioner
ClusterScaler.leader
ClusterScaler.config
ClusterScaler.static
ClusterScaler.on_too_big
ClusterScaler.jobNameToAvgRuntime
ClusterScaler.jobNameToNumCompleted
ClusterScaler.totalAvgRuntime
ClusterScaler.totalJobsCompleted
ClusterScaler.targetTime
ClusterScaler.betaInertia
ClusterScaler.nodeShapeToType
ClusterScaler.instance_types
ClusterScaler.nodeShapes
ClusterScaler.ignoredNodes
ClusterScaler.preemptibleNodeDeficit
ClusterScaler.previousWeightedEstimate
ClusterScaler.minNodes
ClusterScaler.maxNodes
ClusterScaler.node_shapes_after_overhead
ClusterScaler.without_overhead
ClusterScaler.getAverageRuntime()
ClusterScaler.addCompletedJob()
ClusterScaler.setStaticNodes()
ClusterScaler.getStaticNodes()
ClusterScaler.smoothEstimate()
ClusterScaler.getEstimatedNodeCounts()
ClusterScaler.updateClusterSize()
ClusterScaler.setNodeCount()
ClusterScaler.filter_out_static_nodes()
ClusterScaler.getNodes()
ClusterScaler.shutDown()
JobTooBigError
ScalerThread
ClusterStats
- toil.provisioners.gceProvisioner
- Attributes
- Classes
- Module Contents
logger
GCEProvisioner
GCEProvisioner.NODE_BOTO_PATH
GCEProvisioner.SOURCE_IMAGE
GCEProvisioner.cloud
GCEProvisioner.supportedClusterTypes()
GCEProvisioner.createClusterSettings()
GCEProvisioner.readClusterSettings()
GCEProvisioner.launchCluster()
GCEProvisioner.getNodeShape()
GCEProvisioner.retryPredicate()
GCEProvisioner.destroyCluster()
GCEProvisioner.terminateNodes()
GCEProvisioner.addNodes()
GCEProvisioner.getProvisionedWorkers()
GCEProvisioner.getLeader()
GCEProvisioner.DEFAULT_TASK_COMPLETION_TIMEOUT
GCEProvisioner.ex_create_multiple_nodes()
- toil.provisioners.node
- Attributes
- Classes
- Module Contents
a_short_time
logger
Node
Node.maxWaitTime
Node.publicIP
Node.privateIP
Node.name
Node.nodeType
Node.preemptible
Node.tags
Node.__str__()
Node.__repr__()
Node.__hash__()
Node.remainingBillingInterval()
Node.waitForNode()
Node.copySshKeys()
Node.injectFile()
Node.extractFile()
Node.sshAppliance()
Node.sshInstance()
Node.coreSSH()
Node.coreRsync()
- toil.provisioners.abstractProvisioner
- Attributes
- Exceptions
- Functions
- Package Contents
- Submodules
- toil.realtimeLogger
- toil.resource
- Attributes
- Exceptions
- Classes
- Module Contents
logger
Resource
FileResource
DirectoryResource
VirtualEnvResource
ModuleDescriptor
ModuleDescriptor.dirPath
ModuleDescriptor.name
ModuleDescriptor.forModule()
ModuleDescriptor.belongsToToil
ModuleDescriptor.saveAsResourceTo()
ModuleDescriptor.localize()
ModuleDescriptor.globalize()
ModuleDescriptor.toCommand()
ModuleDescriptor.fromCommand()
ModuleDescriptor.makeLoadable()
ModuleDescriptor.load()
ResourceException
- toil.server
- Submodules
- toil.server.api_spec
- toil.server.app
- toil.server.celery_app
- toil.server.cli
- toil.server.utils
- Attributes
- Classes
- Functions
- Module Contents
HAVE_S3
logger
get_iso_time()
link_file()
download_file_from_internet()
download_file_from_s3()
get_file_class()
safe_read_file()
safe_write_file()
MemoryStateCache
AbstractStateStore
MemoryStateStore
FileStateStore
S3StateStore
state_store_cache
connect_to_state_store()
WorkflowStateStore
connect_to_workflow_state_store()
TERMINAL_STATES
MAX_CANCELING_SECONDS
WorkflowStateMachine
WorkflowStateMachine.send_enqueue()
WorkflowStateMachine.send_initialize()
WorkflowStateMachine.send_run()
WorkflowStateMachine.send_cancel()
WorkflowStateMachine.send_canceled()
WorkflowStateMachine.send_complete()
WorkflowStateMachine.send_executor_error()
WorkflowStateMachine.send_system_error()
WorkflowStateMachine.get_current_state()
- toil.server.wes
- Submodules
- toil.server.wes.abstract_backend
- toil.server.wes.amazon_wes_utils
- toil.server.wes.tasks
- Attributes
- Classes
- Functions
- Module Contents
logger
WAIT_FOR_DEATH_TIMEOUT
ToilWorkflowRunner
ToilWorkflowRunner.scratch_dir
ToilWorkflowRunner.store
ToilWorkflowRunner.state_machine
ToilWorkflowRunner.request
ToilWorkflowRunner.engine_options
ToilWorkflowRunner.wf_type
ToilWorkflowRunner.version
ToilWorkflowRunner.exec_dir
ToilWorkflowRunner.out_dir
ToilWorkflowRunner.default_job_store
ToilWorkflowRunner.job_store
ToilWorkflowRunner.write_scratch_file()
ToilWorkflowRunner.get_state()
ToilWorkflowRunner.write_workflow()
ToilWorkflowRunner.sort_options()
ToilWorkflowRunner.initialize_run()
ToilWorkflowRunner.call_cmd()
ToilWorkflowRunner.run()
ToilWorkflowRunner.write_output_files()
run_wes_task()
run_wes
cancel_run()
TaskRunner
MultiprocessingTaskRunner
- toil.server.wes.toil_backend
- Attributes
- Classes
- Module Contents
logger
ToilWorkflow
ToilWorkflow.base_scratch_dir
ToilWorkflow.state_store_url
ToilWorkflow.run_id
ToilWorkflow.scratch_dir
ToilWorkflow.exec_dir
ToilWorkflow.store
ToilWorkflow.state_machine
ToilWorkflow.fetch_state()
ToilWorkflow.fetch_scratch()
ToilWorkflow.exists()
ToilWorkflow.get_state()
ToilWorkflow.check_on_run()
ToilWorkflow.set_up_run()
ToilWorkflow.clean_up()
ToilWorkflow.queue_run()
ToilWorkflow.get_output_files()
ToilWorkflow.get_stdout_path()
ToilWorkflow.get_stderr_path()
ToilWorkflow.get_messages_path()
ToilWorkflow.get_task_logs()
ToilBackend
ToilBackend.run_id_prefix
ToilBackend.task_runner
ToilBackend.wes_dialect
ToilBackend.dest_bucket_base
ToilBackend.work_dir
ToilBackend.supported_versions
ToilBackend.get_runs()
ToilBackend.get_state()
ToilBackend.get_service_info()
ToilBackend.list_runs()
ToilBackend.run_workflow()
ToilBackend.get_run_log()
ToilBackend.cancel_run()
ToilBackend.get_run_status()
ToilBackend.get_stdout()
ToilBackend.get_stderr()
ToilBackend.get_health()
ToilBackend.get_homepage()
- Submodules
- toil.server.wsgi_app
- Submodules
- toil.serviceManager
- Attributes
- Classes
- Module Contents
logger
ServiceManager
ServiceManager.services_are_starting()
ServiceManager.get_job_count()
ServiceManager.start()
ServiceManager.put_client()
ServiceManager.get_ready_client()
ServiceManager.get_unservable_client()
ServiceManager.get_startable_service()
ServiceManager.kill_services()
ServiceManager.is_active()
ServiceManager.is_running()
ServiceManager.check()
ServiceManager.shutdown()
- toil.statsAndLogging
- toil.test
- Submodules
- toil.test.batchSystems
- Submodules
- toil.test.batchSystems.batchSystemTest
- Attributes
- Classes
- Functions
- Module Contents
logger
numCores
preemptible
defaultRequirements
BatchSystemPluginTest
hidden
hidden.AbstractBatchSystemTest
hidden.AbstractBatchSystemTest.createBatchSystem()
hidden.AbstractBatchSystemTest.supportsWallTime()
hidden.AbstractBatchSystemTest.createConfig()
hidden.AbstractBatchSystemTest.setUpClass()
hidden.AbstractBatchSystemTest.setUp()
hidden.AbstractBatchSystemTest.tearDown()
hidden.AbstractBatchSystemTest.get_max_startup_seconds()
hidden.AbstractBatchSystemTest.test_available_cores()
hidden.AbstractBatchSystemTest.test_run_jobs()
hidden.AbstractBatchSystemTest.test_set_env()
hidden.AbstractBatchSystemTest.test_set_job_env()
hidden.AbstractBatchSystemTest.testCheckResourceRequest()
hidden.AbstractBatchSystemTest.testScalableBatchSystem()
hidden.AbstractGridEngineBatchSystemTest
AbstractBatchSystemJobTest
KubernetesBatchSystemTest
KubernetesBatchSystemBenchTest
AWSBatchBatchSystemTest
MesosBatchSystemTest
write_temp_file()
SingleMachineBatchSystemTest
MaxCoresSingleMachineBatchSystemTest
parentJob()
childJob()
grandChildJob()
greatGrandChild()
Service
GridEngineBatchSystemTest
SlurmBatchSystemTest
LSFBatchSystemTest
TorqueBatchSystemTest
HTCondorBatchSystemTest
TestSingleMachineBatchSystemJob
TestMesosBatchSystemJob
measureConcurrency()
count()
getCounters()
resetCounters()
get_omp_threads()
- toil.test.batchSystems.batch_system_plugin_test
- Attributes
- Classes
- Module Contents
logger
FakeBatchSystem
FakeBatchSystem.supportsAutoDeployment()
FakeBatchSystem.issueBatchJob()
FakeBatchSystem.killBatchJobs()
FakeBatchSystem.getIssuedBatchJobIDs()
FakeBatchSystem.getRunningBatchJobIDs()
FakeBatchSystem.getUpdatedBatchJob()
FakeBatchSystem.shutdown()
FakeBatchSystem.add_options()
FakeBatchSystem.setOptions()
BatchSystemPluginTest
- toil.test.batchSystems.test_gridengine
- toil.test.batchSystems.test_lsf_helper
- toil.test.batchSystems.test_slurm
- Attributes
- Classes
- Functions
- Module Contents
logger
call_sacct()
call_scontrol()
call_sacct_raises()
call_sinfo()
FakeBatchSystem
SlurmTest
SlurmTest.setUp()
SlurmTest.test_getJobDetailsFromSacct_one_exists()
SlurmTest.test_getJobDetailsFromSacct_one_not_exists()
SlurmTest.test_getJobDetailsFromSacct_many_all_exist()
SlurmTest.test_getJobDetailsFromSacct_many_some_exist()
SlurmTest.test_getJobDetailsFromSacct_many_none_exist()
SlurmTest.test_getJobDetailsFromSacct_argument_list_too_big()
SlurmTest.test_getJobDetailsFromScontrol_one_exists()
SlurmTest.test_getJobDetailsFromScontrol_one_not_exists()
SlurmTest.test_getJobDetailsFromScontrol_many_all_exist()
SlurmTest.test_getJobDetailsFromScontrol_many_some_exist()
SlurmTest.test_getJobDetailsFromScontrol_many_none_exist()
SlurmTest.test_getJobExitCode_job_exists()
SlurmTest.test_getJobExitCode_job_not_exists()
SlurmTest.test_getJobExitCode_sacct_raises_job_exists()
SlurmTest.test_getJobExitCode_sacct_raises_job_not_exists()
SlurmTest.test_coalesce_job_exit_codes_one_exists()
SlurmTest.test_coalesce_job_exit_codes_one_not_exists()
SlurmTest.test_coalesce_job_exit_codes_many_all_exist()
SlurmTest.test_coalesce_job_exit_codes_some_exists()
SlurmTest.test_coalesce_job_exit_codes_sacct_raises_job_exists()
SlurmTest.test_coalesce_job_exit_codes_sacct_raises_job_not_exists()
SlurmTest.test_PartitionSet_get_partition()
SlurmTest.test_PartitionSet_default_gpu_partition()
SlurmTest.test_prepareSbatch_partition()
SlurmTest.test_prepareSbatch_time()
SlurmTest.test_prepareSbatch_export()
SlurmTest.test_option_detector()
SlurmTest.test_any_option_detector()
- toil.test.batchSystems.batchSystemTest
- Submodules
- toil.test.cactus
- toil.test.conftest
- toil.test.cwl
- Submodules
- toil.test.cwl.conftest
- toil.test.cwl.cwlTest
- Attributes
- Classes
- Functions
- Module Contents
pkg_root
log
CONFORMANCE_TEST_TIMEOUT
run_conformance_tests()
TesterFuncType
TestCWLWorkflow
TestCWLWorkflow.test_cwl_cmdline_input()
TestCWLWorkflow.revsort()
TestCWLWorkflow.revsort_no_checksum()
TestCWLWorkflow.download()
TestCWLWorkflow.load_contents()
TestCWLWorkflow.download_directory()
TestCWLWorkflow.download_subdirectory()
TestCWLWorkflow.test_mpi()
TestCWLWorkflow.test_s3_as_secondary_file()
TestCWLWorkflow.test_run_revsort()
TestCWLWorkflow.test_run_revsort_nochecksum()
TestCWLWorkflow.test_run_revsort_no_container()
TestCWLWorkflow.test_run_revsort2()
TestCWLWorkflow.test_run_revsort_debug_worker()
TestCWLWorkflow.test_run_colon_output()
TestCWLWorkflow.test_run_dockstore_trs()
TestCWLWorkflow.test_glob_dir_bypass_file_store()
TestCWLWorkflow.test_required_input_condition_protection()
TestCWLWorkflow.test_slurm_node_memory()
TestCWLWorkflow.test_download_s3()
TestCWLWorkflow.test_download_http()
TestCWLWorkflow.test_download_https()
TestCWLWorkflow.test_download_https_reference()
TestCWLWorkflow.test_download_file()
TestCWLWorkflow.test_download_directory_s3()
TestCWLWorkflow.test_download_directory_s3_reference()
TestCWLWorkflow.test_download_directory_file()
TestCWLWorkflow.test_download_subdirectory_s3()
TestCWLWorkflow.test_download_subdirectory_file()
TestCWLWorkflow.test_load_contents_s3()
TestCWLWorkflow.test_load_contents_http()
TestCWLWorkflow.test_load_contents_https()
TestCWLWorkflow.test_load_contents_file()
TestCWLWorkflow.test_bioconda()
TestCWLWorkflow.test_default_args()
TestCWLWorkflow.test_biocontainers()
TestCWLWorkflow.test_cuda()
TestCWLWorkflow.test_restart()
TestCWLWorkflow.test_caching()
TestCWLWorkflow.test_streamable()
TestCWLWorkflow.test_streamable_reference()
TestCWLWorkflow.test_preemptible()
TestCWLWorkflow.test_preemptible_expression()
TestCWLWorkflow.test_missing_import()
TestCWLWorkflow.test_optional_secondary_files_exists()
TestCWLWorkflow.test_optional_secondary_files_missing()
cwl_v1_0_spec()
TestCWLv10
TestCWLv10.test_run_conformance_with_caching()
TestCWLv10.test_run_conformance()
TestCWLv10.test_lsf_cwl_conformance()
TestCWLv10.test_slurm_cwl_conformance()
TestCWLv10.test_torque_cwl_conformance()
TestCWLv10.test_gridengine_cwl_conformance()
TestCWLv10.test_mesos_cwl_conformance()
TestCWLv10.test_kubernetes_cwl_conformance()
TestCWLv10.test_lsf_cwl_conformance_with_caching()
TestCWLv10.test_slurm_cwl_conformance_with_caching()
TestCWLv10.test_torque_cwl_conformance_with_caching()
TestCWLv10.test_gridengine_cwl_conformance_with_caching()
TestCWLv10.test_mesos_cwl_conformance_with_caching()
TestCWLv10.test_kubernetes_cwl_conformance_with_caching()
cwl_v1_1_spec()
TestCWLv11
cwl_v1_2_spec()
TestCWLv12
TestCWLv12.test_run_conformance()
TestCWLv12.test_run_conformance_with_caching()
TestCWLv12.test_run_conformance_with_task_caching()
TestCWLv12.test_run_conformance_with_in_place_update()
TestCWLv12.test_kubernetes_cwl_conformance()
TestCWLv12.test_kubernetes_cwl_conformance_with_caching()
TestCWLv12.test_wes_server_cwl_conformance()
test_workflow_echo_string_scatter_stderr_log_dir()
test_log_dir_echo_no_output()
test_log_dir_echo_stderr()
test_filename_conflict_resolution()
test_filename_conflict_resolution_3_or_more()
test_filename_conflict_detection()
test_filename_conflict_detection_at_root()
test_pick_value_with_one_null_value()
test_workflow_echo_string()
test_workflow_echo_string_scatter_capture_stdout()
test_visit_top_cwl_class()
test_visit_cwl_class_and_reduce()
test_download_structure()
test_import_on_workers()
ImportWorkersMessageHandler
- Submodules
- toil.test.docs
- Submodules
- toil.test.docs.scriptsTest
- Attributes
- Classes
- Module Contents
pkg_root
ToilDocumentationTest
ToilDocumentationTest.setUpClass()
ToilDocumentationTest.tearDown()
ToilDocumentationTest.checkExitCode()
ToilDocumentationTest.checkExpectedOut()
ToilDocumentationTest.checkExpectedPattern()
ToilDocumentationTest.testStats()
ToilDocumentationTest.testDynamic()
ToilDocumentationTest.testEncapsulation()
ToilDocumentationTest.testEncapsulation2()
ToilDocumentationTest.testHelloworld()
ToilDocumentationTest.testInvokeworkflow()
ToilDocumentationTest.testInvokeworkflow2()
ToilDocumentationTest.testJobFunctions()
ToilDocumentationTest.testManaging()
ToilDocumentationTest.testManaging2()
ToilDocumentationTest.testMultiplejobs()
ToilDocumentationTest.testMultiplejobs2()
ToilDocumentationTest.testMultiplejobs3()
ToilDocumentationTest.testPromises2()
ToilDocumentationTest.testQuickstart()
ToilDocumentationTest.testRequirements()
ToilDocumentationTest.testArguments()
ToilDocumentationTest.testDocker()
ToilDocumentationTest.testPromises()
ToilDocumentationTest.testServices()
ToilDocumentationTest.testStaging()
- toil.test.docs.scriptsTest
- Submodules
- toil.test.jobStores
- Submodules
- toil.test.jobStores.jobStoreTest
- Attributes
- Classes
- Functions
- Module Contents
google_retry()
logger
tearDownModule()
AbstractJobStoreTest
AbstractJobStoreTest.Test
AbstractJobStoreTest.Test.setUpClass()
AbstractJobStoreTest.Test.setUp()
AbstractJobStoreTest.Test.tearDown()
AbstractJobStoreTest.Test.testInitialState()
AbstractJobStoreTest.Test.testJobCreation()
AbstractJobStoreTest.Test.testConfigEquality()
AbstractJobStoreTest.Test.testJobLoadEquality()
AbstractJobStoreTest.Test.testChildLoadingEquality()
AbstractJobStoreTest.Test.testPersistantFilesToDelete()
AbstractJobStoreTest.Test.testUpdateBehavior()
AbstractJobStoreTest.Test.testJobDeletions()
AbstractJobStoreTest.Test.testSharedFiles()
AbstractJobStoreTest.Test.testReadWriteSharedFilesTextMode()
AbstractJobStoreTest.Test.testReadWriteFileStreamTextMode()
AbstractJobStoreTest.Test.testPerJobFiles()
AbstractJobStoreTest.Test.testStatsAndLogging()
AbstractJobStoreTest.Test.testWriteLogFiles()
AbstractJobStoreTest.Test.testBatchCreate()
AbstractJobStoreTest.Test.testGrowingAndShrinkingJob()
AbstractJobStoreTest.Test.externalStoreCache
AbstractJobStoreTest.Test.cleanUpExternalStores()
AbstractJobStoreTest.Test.mpTestPartSize
AbstractJobStoreTest.Test.makeImportExportTests()
AbstractJobStoreTest.Test.testImportHttpFile()
AbstractJobStoreTest.Test.testImportFtpFile()
AbstractJobStoreTest.Test.testFileDeletion()
AbstractJobStoreTest.Test.testMultipartUploads()
AbstractJobStoreTest.Test.testZeroLengthFiles()
AbstractJobStoreTest.Test.testLargeFile()
AbstractJobStoreTest.Test.fetch_url()
AbstractJobStoreTest.Test.assertUrl()
AbstractJobStoreTest.Test.testCleanCache()
AbstractJobStoreTest.Test.testPartialReadFromStream()
AbstractJobStoreTest.Test.testDestructionOfCorruptedJobStore()
AbstractJobStoreTest.Test.testDestructionIdempotence()
AbstractJobStoreTest.Test.testEmptyFileStoreIDIsReadable()
AbstractEncryptedJobStoreTest
FileJobStoreTest
GoogleJobStoreTest
AWSJobStoreTest
InvalidAWSJobStoreTest
EncryptedAWSJobStoreTest
StubHttpRequestHandler
- toil.test.jobStores.jobStoreTest
- Submodules
- toil.test.lib
- Submodules
- toil.test.lib.aws
- toil.test.lib.dockerTest
- Attributes
- Classes
- Module Contents
logger
TestDocker
TestDocker.dockerTestLogLevel
TestDocker.testDockerClean()
TestDocker.testDockerClean_CRx_FORGO()
TestDocker.testDockerClean_CRx_STOP()
TestDocker.testDockerClean_CRx_RM()
TestDocker.testDockerClean_CRx_None()
TestDocker.testDockerClean_CxD_FORGO()
TestDocker.testDockerClean_CxD_STOP()
TestDocker.testDockerClean_CxD_RM()
TestDocker.testDockerClean_CxD_None()
TestDocker.testDockerClean_Cxx_FORGO()
TestDocker.testDockerClean_Cxx_STOP()
TestDocker.testDockerClean_Cxx_RM()
TestDocker.testDockerClean_Cxx_None()
TestDocker.testDockerClean_xRx_FORGO()
TestDocker.testDockerClean_xRx_STOP()
TestDocker.testDockerClean_xRx_RM()
TestDocker.testDockerClean_xRx_None()
TestDocker.testDockerClean_xxD_FORGO()
TestDocker.testDockerClean_xxD_STOP()
TestDocker.testDockerClean_xxD_RM()
TestDocker.testDockerClean_xxD_None()
TestDocker.testDockerClean_xxx_FORGO()
TestDocker.testDockerClean_xxx_STOP()
TestDocker.testDockerClean_xxx_RM()
TestDocker.testDockerClean_xxx_None()
TestDocker.testDockerPipeChain()
TestDocker.testDockerPipeChainErrorDetection()
TestDocker.testNonCachingDockerChain()
TestDocker.testNonCachingDockerChainErrorDetection()
TestDocker.testDockerLogs()
TestDocker.testDockerLogs_Stream()
TestDocker.testDockerLogs_Demux()
TestDocker.testDockerLogs_Demux_Stream()
- toil.test.lib.test_conversions
- toil.test.lib.test_ec2
- toil.test.lib.test_history
- toil.test.lib.test_misc
- toil.test.lib.test_trs
- Attributes
- Classes
- Module Contents
logger
DockstoreLookupTest
DockstoreLookupTest.read_result()
DockstoreLookupTest.test_lookup_from_page_url()
DockstoreLookupTest.test_lookup_from_trs_with_version()
DockstoreLookupTest.test_lookup_from_trs_no_version()
DockstoreLookupTest.test_get()
DockstoreLookupTest.test_get_from_trs_cached()
DockstoreLookupTest.test_lookup_from_trs_nonexistent_workflow()
DockstoreLookupTest.test_lookup_from_trs_nonexistent_workflow_bad_format()
DockstoreLookupTest.test_lookup_from_trs_nonexistent_version()
DockstoreLookupTest.test_get_nonexistent_workflow()
DockstoreLookupTest.test_get_nonexistent_version()
DockstoreLookupTest.test_get_nonexistent_workflow_bad_format()
- Submodules
- toil.test.mesos
- toil.test.options
- toil.test.provisioners
- Submodules
- toil.test.provisioners.aws
- Submodules
- toil.test.provisioners.aws.awsProvisionerTest
- Attributes
- Classes
- Module Contents
log
AWSProvisionerBenchTest
AbstractAWSAutoscaleTest
AbstractAWSAutoscaleTest.instanceTypes
AbstractAWSAutoscaleTest.clusterName
AbstractAWSAutoscaleTest.numWorkers
AbstractAWSAutoscaleTest.numSamples
AbstractAWSAutoscaleTest.spotBid
AbstractAWSAutoscaleTest.scriptDir
AbstractAWSAutoscaleTest.venvDir
AbstractAWSAutoscaleTest.dataDir
AbstractAWSAutoscaleTest.scriptName
AbstractAWSAutoscaleTest.script()
AbstractAWSAutoscaleTest.data()
AbstractAWSAutoscaleTest.rsyncUtil()
AbstractAWSAutoscaleTest.getRootVolID()
AbstractAWSAutoscaleTest.putScript()
AWSAutoscaleTest
AWSAutoscaleTest.clusterName
AWSAutoscaleTest.requestedLeaderStorage
AWSAutoscaleTest.scriptName
AWSAutoscaleTest.setUp()
AWSAutoscaleTest.launchCluster()
AWSAutoscaleTest.getRootVolID()
AWSAutoscaleTest.testAutoScale()
AWSAutoscaleTest.testSpotAutoScale()
AWSAutoscaleTest.testSpotAutoScaleBalancingTypes()
AWSStaticAutoscaleTest
AWSManagedAutoscaleTest
AWSAutoscaleTestMultipleNodeTypes
AWSRestartTest
PreemptibleDeficitCompensationTest
- toil.test.provisioners.aws.awsProvisionerTest
- Submodules
- toil.test.provisioners.clusterScalerTest
- Attributes
- Classes
- Module Contents
logger
c4_8xlarge_preemptible
c4_8xlarge
r3_8xlarge
r5_2xlarge
r5_4xlarge
t2_micro
BinPackingTest
BinPackingTest.setUp()
BinPackingTest.testPackingOneShape()
BinPackingTest.testSorting()
BinPackingTest.testAddingInitialNode()
BinPackingTest.testLowTargetTime()
BinPackingTest.testHighTargetTime()
BinPackingTest.testZeroResourceJobs()
BinPackingTest.testLongRunningJobs()
BinPackingTest.run1000JobsOnMicros()
BinPackingTest.testPathologicalCase()
BinPackingTest.testJobTooLargeForAllNodes()
ClusterScalerTest
ClusterScalerTest.setUp()
ClusterScalerTest.testRounding()
ClusterScalerTest.testMaxNodes()
ClusterScalerTest.testMinNodes()
ClusterScalerTest.testPreemptibleDeficitResponse()
ClusterScalerTest.testPreemptibleDeficitIsSet()
ClusterScalerTest.testNoLaunchingIfDeltaAlreadyMet()
ClusterScalerTest.testBetaInertia()
ClusterScalerTest.test_overhead_accounting_large()
ClusterScalerTest.test_overhead_accounting_small()
ClusterScalerTest.test_overhead_accounting_observed()
ScalerThreadTest
MockBatchSystemAndProvisioner
MockBatchSystemAndProvisioner.config
MockBatchSystemAndProvisioner.secondsPerJob
MockBatchSystemAndProvisioner.provisioner
MockBatchSystemAndProvisioner.batchSystem
MockBatchSystemAndProvisioner.jobQueue
MockBatchSystemAndProvisioner.updatedJobsQueue
MockBatchSystemAndProvisioner.jobBatchSystemIDToIssuedJob
MockBatchSystemAndProvisioner.totalJobs
MockBatchSystemAndProvisioner.totalWorkerTime
MockBatchSystemAndProvisioner.toilMetrics
MockBatchSystemAndProvisioner.nodesToWorker
MockBatchSystemAndProvisioner.workers
MockBatchSystemAndProvisioner.maxWorkers
MockBatchSystemAndProvisioner.running
MockBatchSystemAndProvisioner.leaderThread
MockBatchSystemAndProvisioner.toilState
MockBatchSystemAndProvisioner.start()
MockBatchSystemAndProvisioner.shutDown()
MockBatchSystemAndProvisioner.nodeInUse()
MockBatchSystemAndProvisioner.ignoreNode()
MockBatchSystemAndProvisioner.unignoreNode()
MockBatchSystemAndProvisioner.supportedClusterTypes()
MockBatchSystemAndProvisioner.createClusterSettings()
MockBatchSystemAndProvisioner.readClusterSettings()
MockBatchSystemAndProvisioner.setAutoscaledNodeTypes()
MockBatchSystemAndProvisioner.getProvisionedWorkers()
MockBatchSystemAndProvisioner.terminateNodes()
MockBatchSystemAndProvisioner.remainingBillingInterval()
MockBatchSystemAndProvisioner.addJob()
MockBatchSystemAndProvisioner.getNumberOfJobsIssued()
MockBatchSystemAndProvisioner.getJobs()
MockBatchSystemAndProvisioner.getNodes()
MockBatchSystemAndProvisioner.addNodes()
MockBatchSystemAndProvisioner.getNodeShape()
MockBatchSystemAndProvisioner.getWorkersInCluster()
MockBatchSystemAndProvisioner.launchCluster()
MockBatchSystemAndProvisioner.destroyCluster()
MockBatchSystemAndProvisioner.getLeader()
MockBatchSystemAndProvisioner.getNumberOfNodes()
- toil.test.provisioners.clusterTest
- Attributes
- Classes
- Module Contents
log
AbstractClusterTest
AbstractClusterTest.keyName
AbstractClusterTest.clusterName
AbstractClusterTest.leaderNodeType
AbstractClusterTest.clusterType
AbstractClusterTest.zone
AbstractClusterTest.region
AbstractClusterTest.aws
AbstractClusterTest.venvDir
AbstractClusterTest.python()
AbstractClusterTest.pip()
AbstractClusterTest.destroyCluster()
AbstractClusterTest.setUp()
AbstractClusterTest.tearDown()
AbstractClusterTest.sshUtil()
AbstractClusterTest.rsync_util()
AbstractClusterTest.createClusterUtil()
AbstractClusterTest.launchCluster()
CWLOnARMTest
- toil.test.provisioners.gceProvisionerTest
- Attributes
- Classes
- Module Contents
log
AbstractGCEAutoscaleTest
AbstractGCEAutoscaleTest.projectID
AbstractGCEAutoscaleTest.sshUtil()
AbstractGCEAutoscaleTest.rsyncUtil()
AbstractGCEAutoscaleTest.destroyClusterUtil()
AbstractGCEAutoscaleTest.createClusterUtil()
AbstractGCEAutoscaleTest.cleanJobStoreUtil()
AbstractGCEAutoscaleTest.keyName
AbstractGCEAutoscaleTest.botoDir
AbstractGCEAutoscaleTest.googleZone
AbstractGCEAutoscaleTest.leaderInstanceType
AbstractGCEAutoscaleTest.instanceTypes
AbstractGCEAutoscaleTest.numWorkers
AbstractGCEAutoscaleTest.numSamples
AbstractGCEAutoscaleTest.spotBid
AbstractGCEAutoscaleTest.setUp()
AbstractGCEAutoscaleTest.tearDown()
AbstractGCEAutoscaleTest.launchCluster()
GCEAutoscaleTest
GCEStaticAutoscaleTest
GCEAutoscaleTestMultipleNodeTypes
GCERestartTest
- toil.test.provisioners.provisionerTest
- toil.test.provisioners.restartScript
- toil.test.provisioners.aws
- Submodules
- toil.test.server
- Submodules
- toil.test.server.serverTest
- Attributes
- Classes
- Module Contents
logger
ToilServerUtilsTest
hidden
FileStateStoreTest
FileStateStoreURLTest
BucketUsingTest
AWSStateStoreTest
AbstractToilWESServerTest
ToilWESServerBenchTest
ToilWESServerWorkflowTest
ToilWESServerWorkflowTest.run_zip_workflow()
ToilWESServerWorkflowTest.test_run_workflow_relative_url_no_attachments_fails()
ToilWESServerWorkflowTest.test_run_workflow_relative_url()
ToilWESServerWorkflowTest.test_run_workflow_https_url()
ToilWESServerWorkflowTest.test_run_workflow_single_file_zip()
ToilWESServerWorkflowTest.test_run_workflow_multi_file_zip()
ToilWESServerWorkflowTest.test_run_workflow_manifest_zip()
ToilWESServerWorkflowTest.test_run_workflow_inputs_zip()
ToilWESServerWorkflowTest.test_run_workflow_manifest_and_inputs_zip()
ToilWESServerWorkflowTest.test_run_workflow_no_params_zip()
ToilWESServerWorkflowTest.test_run_and_cancel_workflows()
ToilWESServerCeleryWorkflowTest
ToilWESServerCeleryS3StateWorkflowTest
- toil.test.server.serverTest
- Submodules
- toil.test.sort
- Submodules
- toil.test.sort.restart_sort
- toil.test.sort.sort
- toil.test.sort.sortTest
- Attributes
- Classes
- Functions
- Module Contents
logger
defaultLineLen
defaultLines
defaultN
runMain()
SortTest
SortTest.setUp()
SortTest.tearDown()
SortTest.testAwsSingle()
SortTest.testAwsMesos()
SortTest.testFileMesos()
SortTest.testGoogleSingle()
SortTest.testGoogleMesos()
SortTest.testFileSingle()
SortTest.testFileSingleNonCaching()
SortTest.testFileSingleCheckpoints()
SortTest.testFileSingle10000()
SortTest.testFileGridEngine()
SortTest.testFileTorqueEngine()
SortTest.testNo
SortTest.testSort()
SortTest.testMerge()
SortTest.testCopySubRangeOfFile()
SortTest.testGetMidPoint()
- Submodules
- toil.test.src
- Submodules
- toil.test.src.autoDeploymentTest
- Attributes
- Classes
- Module Contents
logger
AutoDeploymentTest
AutoDeploymentTest.setUp()
AutoDeploymentTest.sitePackages
AutoDeploymentTest.testRestart()
AutoDeploymentTest.testSplitRootPackages()
AutoDeploymentTest.testUserTypesInJobFunctionArgs()
AutoDeploymentTest.testDeferralWithConcurrentEncapsulation()
AutoDeploymentTest.testDeferralWithFailureAndEncapsulation()
- toil.test.src.busTest
- toil.test.src.checkpointTest
- toil.test.src.deferredFunctionTest
- Attributes
- Classes
- Functions
- Module Contents
logger
options()
TestDeferredFunction
TestDeferredFunction.testDeferredFunctionRunsWithMethod()
TestDeferredFunction.testDeferredFunctionRunsWithClassMethod()
TestDeferredFunction.testDeferredFunctionRunsWithLambda()
TestDeferredFunction.testDeferredFunctionRunsWithFailures()
TestDeferredFunction.testNewJobsCanHandleOtherJobDeaths()
TestDeferredFunction.testBatchSystemCleanupCanHandleWorkerDeaths()
- toil.test.src.dockerCheckTest
- Classes
- Module Contents
DockerCheckTest
DockerCheckTest.testOfficialUbuntuRepo()
DockerCheckTest.testBroadDockerRepo()
DockerCheckTest.testBroadDockerRepoBadTag()
DockerCheckTest.testNonexistentRepo()
DockerCheckTest.testToilQuayRepo()
DockerCheckTest.testBadQuayRepoNTag()
DockerCheckTest.testBadQuayRepo()
DockerCheckTest.testBadQuayTag()
DockerCheckTest.testGoogleRepo()
DockerCheckTest.testBadGoogleRepo()
DockerCheckTest.testApplianceParser()
- toil.test.src.environmentTest
- toil.test.src.fileStoreTest
- Attributes
- Classes
- Module Contents
testingIsAutomatic
logger
hidden
hidden.AbstractFileStoreTest
hidden.AbstractFileStoreTest.jobStoreType
hidden.AbstractFileStoreTest.setUp()
hidden.AbstractFileStoreTest.create_file()
hidden.AbstractFileStoreTest.testToilIsNotBroken()
hidden.AbstractFileStoreTest.testFileStoreLogging()
hidden.AbstractFileStoreTest.testFileStoreOperations()
hidden.AbstractFileStoreTest.testWriteReadGlobalFilePermissions()
hidden.AbstractFileStoreTest.testWriteExportFileCompatibility()
hidden.AbstractFileStoreTest.testImportReadFileCompatibility()
hidden.AbstractFileStoreTest.testReadWriteFileStreamTextMode()
hidden.AbstractNonCachingFileStoreTest
hidden.AbstractCachingFileStoreTest
hidden.AbstractCachingFileStoreTest.setUp()
hidden.AbstractCachingFileStoreTest.testExtremeCacheSetup()
hidden.AbstractCachingFileStoreTest.testCacheEvictionPartialEvict()
hidden.AbstractCachingFileStoreTest.testCacheEvictionTotalEvict()
hidden.AbstractCachingFileStoreTest.testCacheEvictionFailCase()
hidden.AbstractCachingFileStoreTest.testAsyncWriteWithCaching()
hidden.AbstractCachingFileStoreTest.testWriteNonLocalFileToJobStore()
hidden.AbstractCachingFileStoreTest.testWriteLocalFileToJobStore()
hidden.AbstractCachingFileStoreTest.testReadCacheMissFileFromJobStoreWithoutCachingReadFile()
hidden.AbstractCachingFileStoreTest.testReadCacheMissFileFromJobStoreWithCachingReadFile()
hidden.AbstractCachingFileStoreTest.testReadCachHitFileFromJobStore()
hidden.AbstractCachingFileStoreTest.testMultipleJobsReadSameCacheHitGlobalFile()
hidden.AbstractCachingFileStoreTest.testMultipleJobsReadSameCacheMissGlobalFile()
hidden.AbstractCachingFileStoreTest.testFileStoreExportFile()
hidden.AbstractCachingFileStoreTest.testReturnFileSizes()
hidden.AbstractCachingFileStoreTest.testReturnFileSizesWithBadWorker()
hidden.AbstractCachingFileStoreTest.testControlledFailedWorkerRetry()
hidden.AbstractCachingFileStoreTest.testRemoveLocalMutablyReadFile()
hidden.AbstractCachingFileStoreTest.testRemoveLocalImmutablyReadFile()
hidden.AbstractCachingFileStoreTest.testDeleteLocalFile()
hidden.AbstractCachingFileStoreTest.testSimultaneousReadsUncachedStream()
NonCachingFileStoreTestWithFileJobStore
CachingFileStoreTestWithFileJobStore
NonCachingFileStoreTestWithAwsJobStore
CachingFileStoreTestWithAwsJobStore
NonCachingFileStoreTestWithGoogleJobStore
CachingFileStoreTestWithGoogleJobStore
- toil.test.src.helloWorldTest
- toil.test.src.importExportFileTest
- toil.test.src.jobDescriptionTest
- toil.test.src.jobEncapsulationTest
- toil.test.src.jobFileStoreTest
- toil.test.src.jobServiceTest
- toil.test.src.jobTest
- Classes
- Functions
- Module Contents
TestJob
TestJob.testStatic()
TestJob.testStatic2()
TestJob.testTrivialDAGConsistency()
TestJob.testDAGConsistency()
TestJob.testSiblingDAGConsistency()
TestJob.testDeadlockDetection()
TestJob.testNewCheckpointIsLeafVertexNonRootCase()
TestJob.testNewCheckpointIsLeafVertexRootCase()
TestJob.runNewCheckpointIsLeafVertexTest()
TestJob.runCheckpointVertexTest()
TestJob.testEvaluatingRandomDAG()
TestJob.getRandomEdge()
TestJob.makeRandomDAG()
TestJob.getAdjacencyList()
TestJob.reachable()
TestJob.addRandomFollowOnEdges()
TestJob.makeJobGraph()
TestJob.isAcyclic()
simpleJobFn()
fn1Test()
fn2Test()
trivialParent()
parent()
diamond()
child()
errorChild()
TrivialService
- toil.test.src.miscTests
- toil.test.src.promisedRequirementTest
- Attributes
- Classes
- Functions
- Module Contents
log
AbstractPromisedRequirementsTest
AbstractPromisedRequirementsTest.testConcurrencyDynamic()
AbstractPromisedRequirementsTest.testConcurrencyStatic()
AbstractPromisedRequirementsTest.getOptions()
AbstractPromisedRequirementsTest.getCounterPath()
AbstractPromisedRequirementsTest.testPromisesWithJobStoreFileObjects()
AbstractPromisedRequirementsTest.testPromisesWithNonCachingFileStore()
AbstractPromisedRequirementsTest.testPromiseRequirementRaceStatic()
maxConcurrency()
getOne()
getThirtyTwoMb()
logDiskUsage()
TestSingleMachinePromisedRequirements
TestMesosPromisedRequirements
- toil.test.src.promisesTest
- toil.test.src.realtimeLoggerTest
- toil.test.src.regularLogTest
- toil.test.src.resourceTest
- toil.test.src.restartDAGTest
- toil.test.src.resumabilityTest
- toil.test.src.retainTempDirTest
- toil.test.src.systemTest
- toil.test.src.threadingTest
- toil.test.src.toilContextManagerTest
- toil.test.src.userDefinedJobArgTypeTest
- toil.test.src.workerTest
- toil.test.src.autoDeploymentTest
- Submodules
- toil.test.utils
- Submodules
- toil.test.utils.toilDebugTest
- toil.test.utils.toilKillTest
- toil.test.utils.utilsTest
- Attributes
- Classes
- Functions
- Module Contents
logger
unsortedFile()
correctSort()
TestUtils
TestUtils.N
TestUtils.toilMain
TestUtils.cleanCommand()
TestUtils.statsCommand()
TestUtils.statusCommand()
TestUtils.test_config_functionality()
TestUtils.testAWSProvisionerUtils()
TestUtils.testUtilsSort()
TestUtils.testUtilsStatsSort()
TestUtils.testUnicodeSupport()
TestUtils.testMultipleJobsPerWorkerStats()
TestUtils.check_status()
TestUtils.testGetPIDStatus()
TestUtils.testGetStatusFailedToilWF()
TestUtils.testGetStatusFailedCWLWF()
TestUtils.testGetStatusSuccessfulCWLWF()
TestUtils.testPrintJobLog()
TestUtils.testRestartAttribute()
printUnicodeCharacter()
RunTwoJobsPerWorker
- Submodules
- toil.test.wdl
- Submodules
- toil.test.wdl.wdltoil_test
- Attributes
- Classes
- Functions
- Module Contents
logger
WDL_CONFORMANCE_TEST_REPO
WDL_CONFORMANCE_TEST_COMMIT
WDL_CONFORMANCE_TESTS_UNSUPPORTED_BY_TOIL
WDL_UNIT_TESTS_UNSUPPORTED_BY_TOIL
wdl_conformance_test_repo()
TestWDLConformance
TestWDL
TestWDL.base_command
TestWDL.test_MD5sum()
TestWDL.test_url_to_file()
TestWDL.test_string_file_coercion()
TestWDL.test_wait()
TestWDL.test_workflow_file_deletion()
TestWDL.test_all_call_outputs()
TestWDL.test_croo_detection()
TestWDL.test_caching()
TestWDL.test_url_to_optional_file()
TestWDL.test_missing_output_directory()
TestWDL.test_miniwdl_self_test()
TestWDL.test_miniwdl_self_test_by_reference()
TestWDL.test_dockstore_trs()
TestWDL.test_dockstore_metrics_publication()
TestWDL.test_giraffe_deepvariant()
TestWDL.test_giraffe()
TestWDL.test_gs_uri()
TestWDL.test_check()
TestWDLToilBench
- toil.test.wdl.wdltoil_test_kubernetes
- toil.test.wdl.wdltoil_test
- Submodules
- toil.test.batchSystems
- Attributes
- Exceptions
- Classes
- Functions
- Package Contents
ProxyConnectionError
logger
get_data()
ToilTest
MT
get_temp_file()
needs_env_var()
needs_rsync3()
pneeds_rsync3
needs_online()
pneeds_online
needs_aws_s3()
pneeds_aws_s3
needs_aws_ec2()
pneeds_aws_ec2
needs_aws_batch()
needs_google_storage()
needs_google_project()
needs_gridengine()
pneeds_gridengine
needs_torque()
pneeds_torque
needs_kubernetes_installed()
needs_kubernetes()
pneeds_kubernetes
needs_mesos()
pneeds_mesos
needs_slurm()
pneeds_slurm
needs_htcondor()
needs_lsf()
pneeds_lsf
needs_java()
needs_docker()
pneeds_docker
needs_singularity()
needs_singularity_or_docker()
needs_local_cuda()
pneeds_local_cuda
needs_docker_cuda()
pneeds_docker_cuda
needs_encryption()
needs_cwl()
pneeds_cwl
needs_wdl()
pneeds_wdl
needs_server()
needs_celery_broker()
needs_wes_server()
pneeds_wes_server
needs_local_appliance()
needs_fetchable_appliance()
integrative()
pintegrative
slow()
pslow
methodNamePartRegex
timeLimit()
make_tests()
ApplianceTestSupport
ApplianceTestSupport.Appliance
ApplianceTestSupport.Appliance.lock
ApplianceTestSupport.Appliance.outer
ApplianceTestSupport.Appliance.mounts
ApplianceTestSupport.Appliance.cleanMounts
ApplianceTestSupport.Appliance.containerName
ApplianceTestSupport.Appliance.popen
ApplianceTestSupport.Appliance.__enter__()
ApplianceTestSupport.Appliance.__exit__()
ApplianceTestSupport.Appliance.tryRun()
ApplianceTestSupport.Appliance.runOnAppliance()
ApplianceTestSupport.Appliance.writeToAppliance()
ApplianceTestSupport.Appliance.deployScript()
ApplianceTestSupport.LeaderThread
ApplianceTestSupport.WorkerThread
- Submodules
- toil.toilState
- Attributes
- Classes
- Module Contents
logger
ToilState
ToilState.bus
ToilState.successor_to_predecessors
ToilState.successorCounts
ToilState.service_to_client
ToilState.servicesIssued
ToilState.jobs_issued
ToilState.totalFailedJobs
ToilState.hasFailedSuccessors
ToilState.failedSuccessors
ToilState.jobsToBeScheduledWithMultiplePredecessors
ToilState.load_workflow()
ToilState.job_exists()
ToilState.get_job()
ToilState.commit_job()
ToilState.delete_job()
ToilState.reset_job()
ToilState.reset_job_expecting_change()
ToilState.successors_pending()
ToilState.successor_returned()
ToilState.count_pending_successors()
- toil.utils
- Submodules
- toil.utils.toilClean
- toil.utils.toilConfig
- toil.utils.toilDebugFile
- toil.utils.toilDebugJob
- toil.utils.toilDestroyCluster
- toil.utils.toilKill
- toil.utils.toilLaunchCluster
- toil.utils.toilMain
- toil.utils.toilRsyncCluster
- toil.utils.toilServer
- toil.utils.toilSshCluster
- toil.utils.toilStats
- Attributes
- Classes
- Functions
- Module Contents
logger
CATEGORIES
CATEGORY_UNITS
TITLES
TIME_CATEGORIES
SPACE_CATEGORIES
COMPUTED_CATEGORIES
LONG_FORMS
ColumnWidths
pad_str()
pretty_space()
pretty_time()
report_unit()
report_time()
report_space()
report_number()
report()
sprint_tag()
decorate_title()
decorate_subheader()
get()
sort_jobs()
report_pretty_data()
compute_column_widths()
update_column_widths()
build_element()
create_summary()
get_stats()
process_data()
report_data()
sort_category_choices
sort_field_choices
add_stats_options()
main()
- toil.utils.toilStatus
- Attributes
- Classes
- Functions
- Module Contents
logger
ToilStatus
ToilStatus.jobStoreName
ToilStatus.jobStore
ToilStatus.message_bus_path
ToilStatus.print_dot_chart()
ToilStatus.printJobLog()
ToilStatus.printJobChildren()
ToilStatus.printAggregateJobStats()
ToilStatus.report_on_jobs()
ToilStatus.getPIDStatus()
ToilStatus.getStatus()
ToilStatus.print_running_jobs()
ToilStatus.fetchRootJob()
ToilStatus.fetchUserJobs()
ToilStatus.traverseJobGraph()
main()
- toil.utils.toilUpdateEC2Instances
- Submodules
- toil.version
- toil.wdl
- Submodules
- toil.wdl.utils
- toil.wdl.wdltoil
- Attributes
- Exceptions
- Classes
- Functions
- Module Contents
logger
ReadableFileObj
FileDigester
file_digest
WDLContext
InsufficientMountDiskSpace
wdl_error_reporter()
F
report_wdl_errors()
remove_common_leading_whitespace()
toil_read_source()
virtualized_equal()
WDLBindings
combine_bindings()
log_bindings()
get_supertype()
for_each_node()
recursive_dependencies()
parse_disks()
pack_toil_uri()
unpack_toil_uri()
SHARED_PATH_ATTR
clone_metadata()
set_file_value()
set_file_nonexistent()
get_file_nonexistent()
set_file_virtualized_value()
get_file_virtualized_value()
get_shared_fs_path()
set_shared_fs_path()
view_shared_fs_paths()
poll_execution_cache()
fill_execution_cache()
DirectoryNamingStateDict
choose_human_readable_directory()
evaluate_decls_to_bindings()
NonDownloadingSize
extract_file_values()
extract_file_virtualized_values()
convert_files()
convert_remote_files()
ToilWDLStdLibBase
ToilWDLStdLibWorkflow
ToilWDLStdLibTaskCommand
ToilWDLStdLibTaskOutputs
evaluate_named_expression()
evaluate_decl()
evaluate_call_inputs()
evaluate_defaultable_decl()
devirtualize_files()
virtualize_files()
delete_dead_files()
add_paths()
drop_if_missing()
drop_missing_files()
get_file_paths_in_bindings()
map_over_files_in_bindings()
map_over_files_in_binding()
map_over_typed_files_in_value()
ensure_null_files_are_nullable()
WDLBaseJob
WDLTaskWrapperJob
WDLTaskJob
WDLWorkflowNodeJob
WDLWorkflowNodeListJob
WDLCombineBindingsJob
WDLWorkflowGraph
WDLSectionJob
WDLScatterJob
WDLArrayBindingsJob
WDLConditionalJob
WDLWorkflowJob
WDLOutputsJob
WDLStartJob
WDLInstallImportsJob
WDLImportWrapper
make_root_job()
main()
- Submodules
- toil.worker
- toil.batchSystems
- Attributes
- Exceptions
- Functions
- Package Contents
log
which()
toilPackageDirPath()
inVirtualEnv()
resolveEntryPoint()
physicalMemory()
physicalDisk()
applianceSelf()
customDockerInitCmd()
customInitCmd()
lookupEnvVar()
checkDockerImageExists()
parseDockerAppliance()
checkDockerSchema()
ApplianceImageNotFound
KNOWN_EXTANT_IMAGES
requestCheckRegularDocker()
requestCheckDockerIo()
logProcessContext()
- Submodules
- toil