Toil Documentation¶
Toil is an open-source pure-Python workflow engine that lets people write better pipelines.
Check out our website for a comprehensive list of Toil’s features and read our paper to learn what Toil can do in the real world. Please subscribe to our low-volume announce mailing list and feel free to also join us on GitHub and Gitter.
If using Toil for your research, please cite
Vivian, J., Rao, A. A., Nothaft, F. A., Ketchum, C., Armstrong, J., Novak, A., … Paten, B. (2017). Toil enables reproducible, open source, big biomedical data analyses. Nature Biotechnology, 35(4), 314–316. http://doi.org/10.1038/nbt.3772
Getting Started
Running Toil
- Introduction
- Commandline Options
- Toil Debugging
- Running in the Cloud
- Cloud Platforms
- Running on Kubernetes
- Running in AWS
- Running in Google Compute Engine (GCE)
- Cluster Utilities
- Stats Command
- Status Command
- Clean Command
- Launch-Cluster Command
- Ssh-Cluster Command
- Rsync-Cluster Command
- Destroy-Cluster Command
- Kill Command
- HPC Environments
- CWL in Toil
- WDL in Toil
- Workflow Execution Service (WES)
Developing Toil Workflows
- Developing a Workflow
- Scripting Quick Start
- Job Basics
- Invoking a Workflow
- Specifying Commandline Arguments
- Resuming a Workflow
- Functions and Job Functions
- Workflows with Multiple Jobs
- Dynamic Job Creation
- Promises
- Promised Requirements
- FileID
- Managing files within a workflow
- Using Docker Containers in Toil
- Services
- Checkpoints
- Encapsulation
- Depending on Toil
- Best Practices for Dockerizing Toil Workflows
- Toil Class API
- Job Store API
- Toil Job API
- Job Methods API
- Job.Runner API
- job.fileStore API
- Batch System API
- Job.Service API
- Exceptions API
Contributing to Toil
Appendices
- Toil Architecture
- Minimum AWS IAM permissions
- Auto-Deployment
- Environment Variables
- API Reference
toil- Subpackages
toil.batchSystems- Subpackages
toil.batchSystems.mesos- Subpackages
- Submodules
toil.batchSystems.mesos.batchSystem- Module Contents
- Classes
- Attributes
logMesosBatchSystemMesosBatchSystem.ExecutorInfoMesosBatchSystem.userScriptMesosBatchSystem.supportsAutoDeployment()MesosBatchSystem.supportsWorkerCleanup()MesosBatchSystem.setUserScript()MesosBatchSystem.ignoreNode()MesosBatchSystem.unignoreNode()MesosBatchSystem.issueBatchJob()MesosBatchSystem.killBatchJobs()MesosBatchSystem.getIssuedBatchJobIDs()MesosBatchSystem.getRunningBatchJobIDs()MesosBatchSystem.getUpdatedBatchJob()MesosBatchSystem.nodeInUse()MesosBatchSystem.getWaitDuration()MesosBatchSystem.shutdown()MesosBatchSystem.registered()MesosBatchSystem.resourceOffers()MesosBatchSystem.statusUpdate()MesosBatchSystem.frameworkMessage()MesosBatchSystem.getNodes()MesosBatchSystem.reregistered()MesosBatchSystem.executorLost()MesosBatchSystem.get_default_mesos_endpoint()MesosBatchSystem.add_options()MesosBatchSystem.setOptions()
- Module Contents
toil.batchSystems.mesos.conftesttoil.batchSystems.mesos.executor
- Package Contents
- Submodules
toil.batchSystems.abstractBatchSystem- Module Contents
- Classes
- Attributes
loggerEXIT_STATUS_UNAVAILABLE_VALUEBatchJobExitReasonUpdatedBatchJobInfoWorkerCleanupInfoAbstractBatchSystemAbstractBatchSystem.supportsAutoDeployment()AbstractBatchSystem.supportsWorkerCleanup()AbstractBatchSystem.setUserScript()AbstractBatchSystem.set_message_bus()AbstractBatchSystem.issueBatchJob()AbstractBatchSystem.killBatchJobs()AbstractBatchSystem.getIssuedBatchJobIDs()AbstractBatchSystem.getRunningBatchJobIDs()AbstractBatchSystem.getUpdatedBatchJob()AbstractBatchSystem.getSchedulingStatusMessage()AbstractBatchSystem.shutdown()AbstractBatchSystem.setEnv()AbstractBatchSystem.add_options()AbstractBatchSystem.setOptions()AbstractBatchSystem.getWorkerContexts()
BatchSystemSupportNodeInfoAbstractScalableBatchSystemInsufficientSystemResourcesAcquisitionTimeoutExceptionResourcePoolResourceSet
- Module Contents
toil.batchSystems.abstractGridEngineBatchSystem- Module Contents
- Classes
- Attributes
loggerJobTupleAbstractGridEngineBatchSystemAbstractGridEngineBatchSystem.WorkerAbstractGridEngineBatchSystem.Worker.getBatchSystemID()AbstractGridEngineBatchSystem.Worker.forgetJob()AbstractGridEngineBatchSystem.Worker.createJobs()AbstractGridEngineBatchSystem.Worker.killJobs()AbstractGridEngineBatchSystem.Worker.checkOnJobs()AbstractGridEngineBatchSystem.Worker.run()AbstractGridEngineBatchSystem.Worker.coalesce_job_exit_codes()AbstractGridEngineBatchSystem.Worker.prepareSubmission()AbstractGridEngineBatchSystem.Worker.submitJob()AbstractGridEngineBatchSystem.Worker.getRunningJobIDs()AbstractGridEngineBatchSystem.Worker.killJob()AbstractGridEngineBatchSystem.Worker.getJobExitCode()
AbstractGridEngineBatchSystem.supportsWorkerCleanup()AbstractGridEngineBatchSystem.supportsAutoDeployment()AbstractGridEngineBatchSystem.issueBatchJob()AbstractGridEngineBatchSystem.killBatchJobs()AbstractGridEngineBatchSystem.getIssuedBatchJobIDs()AbstractGridEngineBatchSystem.getRunningBatchJobIDs()AbstractGridEngineBatchSystem.getUpdatedBatchJob()AbstractGridEngineBatchSystem.shutdown()AbstractGridEngineBatchSystem.setEnv()AbstractGridEngineBatchSystem.getWaitDuration()AbstractGridEngineBatchSystem.sleepSeconds()AbstractGridEngineBatchSystem.with_retries()
- Module Contents
toil.batchSystems.awsBatch- Module Contents
- Classes
- Attributes
loggerSTATE_TO_EXIT_REASONMAX_POLL_COUNTMIN_REQUESTABLE_MIBMIN_REQUESTABLE_CORESAWSBatchBatchSystemAWSBatchBatchSystem.supportsAutoDeployment()AWSBatchBatchSystem.setUserScript()AWSBatchBatchSystem.issueBatchJob()AWSBatchBatchSystem.getUpdatedBatchJob()AWSBatchBatchSystem.shutdown()AWSBatchBatchSystem.getIssuedBatchJobIDs()AWSBatchBatchSystem.getRunningBatchJobIDs()AWSBatchBatchSystem.killBatchJobs()AWSBatchBatchSystem.add_options()AWSBatchBatchSystem.setOptions()
- Module Contents
toil.batchSystems.cleanup_supporttoil.batchSystems.contained_executortoil.batchSystems.gridenginetoil.batchSystems.htcondor- Module Contents
- Classes
- Attributes
loggerJobTupleschedd_lockHTCondorBatchSystemHTCondorBatchSystem.WorkerHTCondorBatchSystem.Worker.createJobs()HTCondorBatchSystem.Worker.prepareSubmission()HTCondorBatchSystem.Worker.submitJob()HTCondorBatchSystem.Worker.getRunningJobIDs()HTCondorBatchSystem.Worker.killJob()HTCondorBatchSystem.Worker.getJobExitCode()HTCondorBatchSystem.Worker.connectSchedd()HTCondorBatchSystem.Worker.duplicate_quotes()HTCondorBatchSystem.Worker.getEnvString()
HTCondorBatchSystem.issueBatchJob()
- Module Contents
toil.batchSystems.kubernetes- Module Contents
- Classes
- Functions
- Attributes
loggerretryable_kubernetes_errorsis_retryable_kubernetes_error()KeyValuesListKubernetesBatchSystemKubernetesBatchSystem.DecoratorWrapperKubernetesBatchSystem.PlacementKubernetesBatchSystem.KubernetesConfigKubernetesBatchSystem.ItemTKubernetesBatchSystem.CovItemTKubernetesBatchSystem.PKubernetesBatchSystem.RKubernetesBatchSystem.OptionTypeKubernetesBatchSystem.supportsAutoDeployment()KubernetesBatchSystem.setUserScript()KubernetesBatchSystem.issueBatchJob()KubernetesBatchSystem.getUpdatedBatchJob()KubernetesBatchSystem.shutdown()KubernetesBatchSystem.getIssuedBatchJobIDs()KubernetesBatchSystem.getRunningBatchJobIDs()KubernetesBatchSystem.killBatchJobs()KubernetesBatchSystem.get_default_kubernetes_owner()KubernetesBatchSystem.add_options()KubernetesBatchSystem.setOptions()
- Module Contents
toil.batchSystems.local_supporttoil.batchSystems.lsf- Module Contents
- Classes
- Attributes
loggerLSFBatchSystemLSFBatchSystem.WorkerLSFBatchSystem.Worker.getRunningJobIDs()LSFBatchSystem.Worker.fallbackRunningJobIDs()LSFBatchSystem.Worker.killJob()LSFBatchSystem.Worker.prepareSubmission()LSFBatchSystem.Worker.submitJob()LSFBatchSystem.Worker.coalesce_job_exit_codes()LSFBatchSystem.Worker.getJobExitCode()LSFBatchSystem.Worker.parse_bjobs_record()LSFBatchSystem.Worker.getJobExitCodeBACCT()LSFBatchSystem.Worker.fallbackGetJobExitCode()LSFBatchSystem.Worker.prepareBsub()LSFBatchSystem.Worker.parseBjobs()LSFBatchSystem.Worker.parseMaxMem()
LSFBatchSystem.getWaitDuration()
- Module Contents
toil.batchSystems.lsfHelper- Module Contents
- Functions
- Attributes
LSB_PARAMS_FILENAMELSF_CONF_FILENAMELSF_CONF_ENVDEFAULT_LSF_UNITSDEFAULT_RESOURCE_UNITSLSF_JSON_OUTPUT_MIN_VERSIONloggerfind()find_first_match()get_conf_file()apply_conf_file()per_core_reserve_from_stream()get_lsf_units_from_stream()tokenize_conf_stream()apply_bparams()apply_lsadmin()get_lsf_units()parse_mem_and_cmd_from_output()get_lsf_version()check_lsf_json_output_supported()parse_memory()per_core_reservation()
- Module Contents
toil.batchSystems.optionstoil.batchSystems.parasol- Module Contents
- Classes
- Attributes
loggerParasolBatchSystemParasolBatchSystem.parasolOutputPatternParasolBatchSystem.runningPatternParasolBatchSystem.supportsWorkerCleanup()ParasolBatchSystem.supportsAutoDeployment()ParasolBatchSystem.issueBatchJob()ParasolBatchSystem.setEnv()ParasolBatchSystem.killBatchJobs()ParasolBatchSystem.getJobIDsForResultsFile()ParasolBatchSystem.getIssuedBatchJobIDs()ParasolBatchSystem.getRunningBatchJobIDs()ParasolBatchSystem.getUpdatedBatchJob()ParasolBatchSystem.updatedJobWorker()ParasolBatchSystem.shutdown()ParasolBatchSystem.add_options()ParasolBatchSystem.setOptions()
- Module Contents
toil.batchSystems.registry- Module Contents
- Functions
- Attributes
loggeraws_batch_batch_system_factory()gridengine_batch_system_factory()parasol_batch_system_factory()lsf_batch_system_factory()single_machine_batch_system_factory()mesos_batch_system_factory()slurm_batch_system_factory()tes_batch_system_factory()torque_batch_system_factory()htcondor_batch_system_factory()kubernetes_batch_system_factory()BATCH_SYSTEM_FACTORY_REGISTRYBATCH_SYSTEMSDEFAULT_BATCH_SYSTEMaddBatchSystemFactory()save_batch_system_plugin_state()restore_batch_system_plugin_state()
- Module Contents
toil.batchSystems.singleMachine- Module Contents
- Classes
- Attributes
loggerSingleMachineBatchSystemSingleMachineBatchSystem.numCoresSingleMachineBatchSystem.minCoresSingleMachineBatchSystem.physicalMemorySingleMachineBatchSystem.supportsAutoDeployment()SingleMachineBatchSystem.supportsWorkerCleanup()SingleMachineBatchSystem.daddy()SingleMachineBatchSystem.getSchedulingStatusMessage()SingleMachineBatchSystem.check_resource_request()SingleMachineBatchSystem.issueBatchJob()SingleMachineBatchSystem.killBatchJobs()SingleMachineBatchSystem.getIssuedBatchJobIDs()SingleMachineBatchSystem.getRunningBatchJobIDs()SingleMachineBatchSystem.shutdown()SingleMachineBatchSystem.getUpdatedBatchJob()SingleMachineBatchSystem.add_options()SingleMachineBatchSystem.setOptions()
Info
- Module Contents
toil.batchSystems.slurm- Module Contents
- Classes
- Attributes
loggerSlurmBatchSystemSlurmBatchSystem.WorkerSlurmBatchSystem.Worker.getRunningJobIDs()SlurmBatchSystem.Worker.killJob()SlurmBatchSystem.Worker.prepareSubmission()SlurmBatchSystem.Worker.submitJob()SlurmBatchSystem.Worker.coalesce_job_exit_codes()SlurmBatchSystem.Worker.getJobExitCode()SlurmBatchSystem.Worker.prepareSbatch()SlurmBatchSystem.Worker.parse_elapsed()
SlurmBatchSystem.OptionTypeSlurmBatchSystem.add_options()SlurmBatchSystem.setOptions()
- Module Contents
toil.batchSystems.tes- Module Contents
- Classes
- Attributes
loggerSTATE_TO_EXIT_REASONTESBatchSystemTESBatchSystem.supportsAutoDeployment()TESBatchSystem.get_default_tes_endpoint()TESBatchSystem.setUserScript()TESBatchSystem.issueBatchJob()TESBatchSystem.getUpdatedBatchJob()TESBatchSystem.shutdown()TESBatchSystem.getIssuedBatchJobIDs()TESBatchSystem.getRunningBatchJobIDs()TESBatchSystem.killBatchJobs()TESBatchSystem.add_options()TESBatchSystem.setOptions()
- Module Contents
toil.batchSystems.torque- Module Contents
- Classes
- Attributes
loggerTorqueBatchSystemTorqueBatchSystem.WorkerTorqueBatchSystem.Worker.getRunningJobIDs()TorqueBatchSystem.Worker.getUpdatedBatchJob()TorqueBatchSystem.Worker.killJob()TorqueBatchSystem.Worker.prepareSubmission()TorqueBatchSystem.Worker.submitJob()TorqueBatchSystem.Worker.getJobExitCode()TorqueBatchSystem.Worker.prepareQsub()TorqueBatchSystem.Worker.generateTorqueWrapper()
- Module Contents
- Package Contents
- Subpackages
toil.cwl- Submodules
toil.cwl.conftesttoil.cwl.cwltoil- Module Contents
- Classes
- Functions
- Attributes
loggerDEFAULT_TMPDIRDEFAULT_TMPDIR_PREFIXcwltoil_was_removed()UnresolvedDictSkipNullfilter_skip_null()ensure_no_collisions()ConditionalResolveSourceStepValueFromDefaultWithSourceJustAValueresolve_dict_w_promises()simplify_list()ToilPathMapperToilSingleJobExecutorToilToolToilCommandLineToolToilExpressionTooltoil_make_tool()DirectoryContentscheck_directory_dict_invariants()decode_directory()encode_directory()ToilFsAccesstoil_get_file()write_file()path_to_loc()import_files()upload_directory()upload_file()writeGlobalFileWrapper()remove_empty_listings()CWLNamedJobResolveIndirecttoilStageFiles()CWLJobWrapperCWLJobget_container_engine()makeJob()CWLScatterCWLGatherSelfJobProcessTyperemove_pickle_problems()CWLWorkflowvisitSteps()rm_unprocessed_secondary_files()filtered_secondary_files()scan_for_unsupported_requirements()determine_load_listing()NoAvailableJobStoreExceptiongenerate_default_job_store()usage_messagemain()find_default_container()
- Module Contents
toil.cwl.utils
- Package Contents
- Submodules
toil.fileStores- Submodules
toil.fileStores.abstractFileStore- Module Contents
- Classes
- Attributes
loggerAbstractFileStoreAbstractFileStore.createFileStore()AbstractFileStore.shutdownFileStore()AbstractFileStore.open()AbstractFileStore.getLocalTempDir()AbstractFileStore.getLocalTempFile()AbstractFileStore.getLocalTempFileName()AbstractFileStore.writeGlobalFile()AbstractFileStore.writeGlobalFileStream()AbstractFileStore.logAccess()AbstractFileStore.readGlobalFile()AbstractFileStore.readGlobalFileStream()AbstractFileStore.getGlobalFileSize()AbstractFileStore.deleteLocalFile()AbstractFileStore.deleteGlobalFile()AbstractFileStore.importFile()AbstractFileStore.import_file()AbstractFileStore.exportFile()AbstractFileStore.export_file()AbstractFileStore.logToMaster()AbstractFileStore.startCommit()AbstractFileStore.waitForCommit()AbstractFileStore.shutdown()
- Module Contents
toil.fileStores.cachingFileStore- Module Contents
- Classes
- Attributes
loggerSQLITE_TIMEOUT_SECSCacheErrorCacheUnbalancedErrorIllegalDeletionCacheErrorInvalidSourceCacheErrorCachingFileStoreCachingFileStore.getCacheLimit()CachingFileStore.getCacheUsed()CachingFileStore.getCacheExtraJobSpace()CachingFileStore.getCacheAvailable()CachingFileStore.getSpaceUsableForJobs()CachingFileStore.getCacheUnusedJobRequirement()CachingFileStore.adjustCacheLimit()CachingFileStore.fileIsCached()CachingFileStore.getFileReaderCount()CachingFileStore.cachingIsFree()CachingFileStore.open()CachingFileStore.writeGlobalFile()CachingFileStore.readGlobalFile()CachingFileStore.readGlobalFileStream()CachingFileStore.deleteLocalFile()CachingFileStore.deleteGlobalFile()CachingFileStore.exportFile()CachingFileStore.export_file()CachingFileStore.waitForCommit()CachingFileStore.startCommit()CachingFileStore.startCommitThread()CachingFileStore.shutdown()CachingFileStore.__del__()
- Module Contents
toil.fileStores.nonCachingFileStore- Module Contents
- Classes
- Attributes
loggerNonCachingFileStoreNonCachingFileStore.check_for_coordination_corruption()NonCachingFileStore.check_for_state_corruption()NonCachingFileStore.open()NonCachingFileStore.writeGlobalFile()NonCachingFileStore.readGlobalFile()NonCachingFileStore.readGlobalFileStream()NonCachingFileStore.exportFile()NonCachingFileStore.export_file()NonCachingFileStore.deleteLocalFile()NonCachingFileStore.deleteGlobalFile()NonCachingFileStore.waitForCommit()NonCachingFileStore.startCommit()NonCachingFileStore.__del__()NonCachingFileStore.shutdown()
- Module Contents
- Package Contents
- Submodules
toil.jobStores- Subpackages
toil.jobStores.aws- Submodules
toil.jobStores.aws.jobStore- Module Contents
- Classes
- Attributes
boto3_sessions3_boto3_resources3_boto3_clientloggerCONSISTENCY_TICKSCONSISTENCY_TIMEChecksumErrorAWSJobStoreAWSJobStore.FileInfoAWSJobStore.FileInfo.fileIDAWSJobStore.FileInfo.ownerIDAWSJobStore.FileInfo.versionAWSJobStore.FileInfo.previousVersionAWSJobStore.FileInfo.contentAWSJobStore.FileInfo.checksumAWSJobStore.FileInfo.outerAWSJobStore.FileInfo.create()AWSJobStore.FileInfo.presenceIndicator()AWSJobStore.FileInfo.exists()AWSJobStore.FileInfo.load()AWSJobStore.FileInfo.loadOrCreate()AWSJobStore.FileInfo.loadOrFail()AWSJobStore.FileInfo.fromItem()AWSJobStore.FileInfo.toItem()AWSJobStore.FileInfo.maxInlinedSize()AWSJobStore.FileInfo.save()AWSJobStore.FileInfo.upload()AWSJobStore.FileInfo.uploadStream()AWSJobStore.FileInfo.copyFrom()AWSJobStore.FileInfo.copyTo()AWSJobStore.FileInfo.download()AWSJobStore.FileInfo.downloadStream()AWSJobStore.FileInfo.delete()AWSJobStore.FileInfo.getSize()AWSJobStore.FileInfo.__repr__()
AWSJobStore.sseKeyPathAWSJobStore.bucketNameReAWSJobStore.minBucketNameLenAWSJobStore.maxBucketNameLenAWSJobStore.maxNameLenAWSJobStore.nameSeparatorAWSJobStore.jobsPerBatchInsertAWSJobStore.itemsPerBatchDeleteAWSJobStore.sharedFileOwnerIDAWSJobStore.statsFileOwnerIDAWSJobStore.readStatsFileOwnerIDAWSJobStore.versioningsAWSJobStore.initialize()AWSJobStore.resume()AWSJobStore.batch()AWSJobStore.assign_job_id()AWSJobStore.create_job()AWSJobStore.job_exists()AWSJobStore.jobs()AWSJobStore.load_job()AWSJobStore.update_job()AWSJobStore.delete_job()AWSJobStore.get_empty_file_store_id()AWSJobStore.get_size()AWSJobStore.write_file()AWSJobStore.write_file_stream()AWSJobStore.write_shared_file_stream()AWSJobStore.update_file()AWSJobStore.update_file_stream()AWSJobStore.file_exists()AWSJobStore.get_file_size()AWSJobStore.read_file()AWSJobStore.read_file_stream()AWSJobStore.read_shared_file_stream()AWSJobStore.delete_file()AWSJobStore.write_logs()AWSJobStore.read_logs()AWSJobStore.get_public_url()AWSJobStore.get_shared_public_url()AWSJobStore.destroy()
aReprcustom_reprBucketLocationConflictException
- Module Contents
toil.jobStores.aws.utils- Module Contents
- Submodules
- Submodules
toil.jobStores.abstractJobStore- Module Contents
- Classes
- Attributes
loggerProxyConnectionErrorInvalidImportExportUrlExceptionUnimplementedURLExceptionNoSuchJobExceptionConcurrentFileModificationExceptionNoSuchFileExceptionNoSuchJobStoreExceptionJobStoreExistsExceptionAbstractJobStoreAbstractJobStore.configAbstractJobStore.locatorAbstractJobStore.rootJobStoreIDFileNameAbstractJobStore.publicUrlExpirationAbstractJobStore.sharedFileNameRegexAbstractJobStore.initialize()AbstractJobStore.writeConfig()AbstractJobStore.write_config()AbstractJobStore.resume()AbstractJobStore.setRootJob()AbstractJobStore.set_root_job()AbstractJobStore.loadRootJob()AbstractJobStore.load_root_job()AbstractJobStore.createRootJob()AbstractJobStore.create_root_job()AbstractJobStore.getRootJobReturnValue()AbstractJobStore.get_root_job_return_value()AbstractJobStore.importFile()AbstractJobStore.import_file()AbstractJobStore.exportFile()AbstractJobStore.export_file()AbstractJobStore.list_url()AbstractJobStore.get_is_directory()AbstractJobStore.read_from_url()AbstractJobStore.getSize()AbstractJobStore.get_size()AbstractJobStore.destroy()AbstractJobStore.getEnv()AbstractJobStore.get_env()AbstractJobStore.clean()AbstractJobStore.assignID()AbstractJobStore.assign_job_id()AbstractJobStore.batch()AbstractJobStore.create()AbstractJobStore.create_job()AbstractJobStore.exists()AbstractJobStore.job_exists()AbstractJobStore.getPublicUrl()AbstractJobStore.get_public_url()AbstractJobStore.getSharedPublicUrl()AbstractJobStore.get_shared_public_url()AbstractJobStore.load()AbstractJobStore.load_job()AbstractJobStore.update()AbstractJobStore.update_job()AbstractJobStore.delete()AbstractJobStore.delete_job()AbstractJobStore.jobs()AbstractJobStore.writeFile()AbstractJobStore.write_file()AbstractJobStore.writeFileStream()AbstractJobStore.write_file_stream()AbstractJobStore.getEmptyFileStoreID()AbstractJobStore.get_empty_file_store_id()AbstractJobStore.readFile()AbstractJobStore.read_file()AbstractJobStore.readFileStream()AbstractJobStore.read_file_stream()AbstractJobStore.deleteFile()AbstractJobStore.delete_file()AbstractJobStore.fileExists()AbstractJobStore.file_exists()AbstractJobStore.getFileSize()AbstractJobStore.get_file_size()AbstractJobStore.updateFile()AbstractJobStore.update_file()AbstractJobStore.updateFileStream()AbstractJobStore.update_file_stream()AbstractJobStore.writeSharedFileStream()AbstractJobStore.write_shared_file_stream()AbstractJobStore.readSharedFileStream()AbstractJobStore.read_shared_file_stream()AbstractJobStore.writeStatsAndLogging()AbstractJobStore.write_logs()AbstractJobStore.readStatsAndLogging()AbstractJobStore.read_logs()AbstractJobStore.write_leader_pid()AbstractJobStore.read_leader_pid()AbstractJobStore.write_leader_node_id()AbstractJobStore.read_leader_node_id()AbstractJobStore.write_kill_flag()AbstractJobStore.read_kill_flag()AbstractJobStore.default_caching()
JobStoreSupport
- Module Contents
toil.jobStores.conftesttoil.jobStores.fileJobStore- Module Contents
- Classes
- Attributes
loggerFileJobStoreFileJobStore.validDirsFileJobStore.validDirsSetFileJobStore.JOB_DIR_PREFIXFileJobStore.JOB_NAME_DIR_PREFIXFileJobStore.BUFFER_SIZEFileJobStore.default_caching()FileJobStore.__repr__()FileJobStore.initialize()FileJobStore.resume()FileJobStore.destroy()FileJobStore.assign_job_id()FileJobStore.create_job()FileJobStore.batch()FileJobStore.job_exists()FileJobStore.get_public_url()FileJobStore.get_shared_public_url()FileJobStore.load_job()FileJobStore.update_job()FileJobStore.delete_job()FileJobStore.jobs()FileJobStore.optional_hard_copy()FileJobStore.get_size()FileJobStore.write_file()FileJobStore.write_file_stream()FileJobStore.get_empty_file_store_id()FileJobStore.update_file()FileJobStore.read_file()FileJobStore.delete_file()FileJobStore.file_exists()FileJobStore.get_file_size()FileJobStore.update_file_stream()FileJobStore.read_file_stream()FileJobStore.write_shared_file_stream()FileJobStore.read_shared_file_stream()FileJobStore.write_logs()FileJobStore.read_logs()
- Module Contents
toil.jobStores.googleJobStore- Module Contents
- Classes
- Functions
- Attributes
logGOOGLE_STORAGEMAX_BATCH_SIZEgoogle_retry_predicate()google_retry()GoogleJobStoreGoogleJobStore.nodeServiceAccountJsonGoogleJobStore.initialize()GoogleJobStore.resume()GoogleJobStore.destroy()GoogleJobStore.assign_job_id()GoogleJobStore.batch()GoogleJobStore.create_job()GoogleJobStore.job_exists()GoogleJobStore.get_public_url()GoogleJobStore.get_shared_public_url()GoogleJobStore.load_job()GoogleJobStore.update_job()GoogleJobStore.delete_job()GoogleJobStore.get_env()GoogleJobStore.jobs()GoogleJobStore.write_file()GoogleJobStore.write_file_stream()GoogleJobStore.get_empty_file_store_id()GoogleJobStore.read_file()GoogleJobStore.read_file_stream()GoogleJobStore.delete_file()GoogleJobStore.file_exists()GoogleJobStore.get_file_size()GoogleJobStore.update_file()GoogleJobStore.update_file_stream()GoogleJobStore.write_shared_file_stream()GoogleJobStore.read_shared_file_stream()GoogleJobStore.get_size()GoogleJobStore.write_logs()GoogleJobStore.read_logs()
- Module Contents
toil.jobStores.utils
- Subpackages
toil.lib- Subpackages
toil.lib.aws- Submodules
toil.lib.aws.amitoil.lib.aws.iam- Module Contents
- Functions
- Attributes
loggerCLUSTER_LAUNCHING_PERMISSIONSAllowedActionCollectioninit_action_collection()add_to_action_collection()policy_permissions_allow()permission_matches_any()get_actions_from_policy_document()allowed_actions_attached()allowed_actions_roles()allowed_actions_users()get_policy_permissions()get_aws_account_num()
- Module Contents
toil.lib.aws.sessiontoil.lib.aws.utils- Module Contents
- Functions
- Attributes
BotoServerErrorloggerTHROTTLED_ERROR_CODESdelete_iam_role()delete_iam_instance_profile()delete_sdb_domain()connection_reset()retryable_s3_errors()retry_s3()delete_s3_bucket()create_s3_bucket()enable_public_objects()get_bucket_region()region_to_bucket_location()bucket_location_to_region()get_object_for_url()list_objects_for_url()flatten_tags()
- Module Contents
- Package Contents
- Submodules
toil.lib.encryption
- Submodules
toil.lib.acceleratorstoil.lib.bioiotoil.lib.compatibilitytoil.lib.conversionstoil.lib.dockertoil.lib.ec2- Module Contents
- Functions
- Attributes
a_short_timea_long_timeloggerUserErrornot_found()inconsistencies_detected()INCONSISTENCY_ERRORSretry_ec2()UnexpectedResourceStatewait_transition()wait_instances_running()wait_spot_requests_active()create_spot_instances()create_ondemand_instances()prune()iam_clientwait_until_instance_profile_arn_exists()create_instances()create_launch_template()create_auto_scaling_group()
- Module Contents
toil.lib.ec2nodestoil.lib.exceptionstoil.lib.expandotoil.lib.generatedEC2Liststoil.lib.humanizetoil.lib.iotoil.lib.iterablestoil.lib.memoizetoil.lib.misctoil.lib.objectstoil.lib.resourcestoil.lib.retry- Module Contents
- Classes
- Functions
- Attributes
SUPPORTED_HTTP_ERRORSkubernetesbotocoreloggerErrorConditionretry()return_status_code()get_error_code()get_error_message()get_error_status()get_error_body()meets_error_message_condition()meets_error_code_condition()meets_boto_error_code_condition()error_meets_conditions()DEFAULT_DELAYSDEFAULT_TIMEOUTold_retry()retry_flaky_test
- Module Contents
toil.lib.threadingtoil.lib.throttle
- Subpackages
toil.provisioners- Subpackages
toil.provisioners.aws- Submodules
toil.provisioners.aws.awsProvisioner- Module Contents
- Classes
- Functions
- Attributes
loggerawsRetryPredicate()expectedShutdownErrors()awsRetry()awsFilterImpairedNodes()InvalidClusterStateExceptionAWSProvisionerAWSProvisioner.supportedClusterTypes()AWSProvisioner.createClusterSettings()AWSProvisioner.readClusterSettings()AWSProvisioner.launchCluster()AWSProvisioner.toil_service_env_options()AWSProvisioner.getKubernetesAutoscalerSetupCommands()AWSProvisioner.getKubernetesCloudProvider()AWSProvisioner.getNodeShape()AWSProvisioner.retryPredicate()AWSProvisioner.destroyCluster()AWSProvisioner.terminateNodes()AWSProvisioner.addNodes()AWSProvisioner.addManagedNodes()AWSProvisioner.getProvisionedWorkers()AWSProvisioner.getLeader()AWSProvisioner.full_policy()AWSProvisioner.kubernetes_policy()
- Module Contents
- Package Contents
- Submodules
- Submodules
toil.provisioners.abstractProvisioner- Module Contents
- Classes
- Attributes
a_short_timeloggerManagedNodesNotSupportedExceptionShapeAbstractProvisionerAbstractProvisioner.InstanceConfigurationAbstractProvisioner.LEADER_HOME_DIRAbstractProvisioner.cloudAbstractProvisioner.supportedClusterTypes()AbstractProvisioner.createClusterSettings()AbstractProvisioner.readClusterSettings()AbstractProvisioner.setAutoscaledNodeTypes()AbstractProvisioner.hasAutoscaledNodeTypes()AbstractProvisioner.getAutoscaledInstanceShapes()AbstractProvisioner.retryPredicate()AbstractProvisioner.launchCluster()AbstractProvisioner.addNodes()AbstractProvisioner.addManagedNodes()AbstractProvisioner.terminateNodes()AbstractProvisioner.getLeader()AbstractProvisioner.getProvisionedWorkers()AbstractProvisioner.getNodeShape()AbstractProvisioner.destroyCluster()AbstractProvisioner.getBaseInstanceConfiguration()AbstractProvisioner.addVolumesService()AbstractProvisioner.addNodeExporterService()AbstractProvisioner.toil_service_env_options()AbstractProvisioner.add_toil_service()AbstractProvisioner.getKubernetesValues()AbstractProvisioner.addKubernetesServices()AbstractProvisioner.getKubernetesAutoscalerSetupCommands()AbstractProvisioner.getKubernetesCloudProvider()AbstractProvisioner.addKubernetesLeader()AbstractProvisioner.addKubernetesWorker()
- Module Contents
toil.provisioners.clusterScaler- Module Contents
- Classes
- Functions
- Attributes
loggerEVICTION_THRESHOLDRESERVE_SMALL_LIMITRESERVE_SMALL_AMOUNTRESERVE_BREAKPOINTSRESERVE_FRACTIONSOS_SIZEFailedConstraintBinPackedFitNodeReservationadjustEndingReservationForJob()split()binPacking()ClusterScalerClusterScaler.getAverageRuntime()ClusterScaler.addCompletedJob()ClusterScaler.setStaticNodes()ClusterScaler.getStaticNodes()ClusterScaler.smoothEstimate()ClusterScaler.getEstimatedNodeCounts()ClusterScaler.updateClusterSize()ClusterScaler.setNodeCount()ClusterScaler.filter_out_static_nodes()ClusterScaler.getNodes()ClusterScaler.shutDown()
JobTooBigErrorScalerThreadClusterStats
- Module Contents
toil.provisioners.gceProvisioner- Module Contents
- Classes
- Attributes
loggerGCEProvisionerGCEProvisioner.NODE_BOTO_PATHGCEProvisioner.SOURCE_IMAGEGCEProvisioner.DEFAULT_TASK_COMPLETION_TIMEOUTGCEProvisioner.supportedClusterTypes()GCEProvisioner.createClusterSettings()GCEProvisioner.readClusterSettings()GCEProvisioner.launchCluster()GCEProvisioner.getNodeShape()GCEProvisioner.retryPredicate()GCEProvisioner.destroyCluster()GCEProvisioner.terminateNodes()GCEProvisioner.addNodes()GCEProvisioner.getProvisionedWorkers()GCEProvisioner.getLeader()GCEProvisioner.ex_create_multiple_nodes()
- Module Contents
toil.provisioners.node
- Package Contents
- Subpackages
toil.server- Subpackages
toil.server.api_spectoil.server.clitoil.server.wes- Submodules
toil.server.wes.abstract_backendtoil.server.wes.amazon_wes_utilstoil.server.wes.taskstoil.server.wes.toil_backend- Module Contents
- Classes
- Attributes
loggerToilWorkflowToilWorkflow.fetch_state()ToilWorkflow.fetch_scratch()ToilWorkflow.exists()ToilWorkflow.get_state()ToilWorkflow.check_on_run()ToilWorkflow.set_up_run()ToilWorkflow.clean_up()ToilWorkflow.queue_run()ToilWorkflow.get_output_files()ToilWorkflow.get_stdout_path()ToilWorkflow.get_stderr_path()ToilWorkflow.get_messages_path()ToilWorkflow.get_task_logs()
ToilBackendToilBackend.get_runs()ToilBackend.get_state()ToilBackend.get_service_info()ToilBackend.list_runs()ToilBackend.run_workflow()ToilBackend.get_run_log()ToilBackend.cancel_run()ToilBackend.get_run_status()ToilBackend.get_stdout()ToilBackend.get_stderr()ToilBackend.get_health()ToilBackend.get_homepage()
- Module Contents
- Submodules
- Submodules
toil.server.apptoil.server.celery_apptoil.server.utils- Module Contents
- Classes
- Functions
- Attributes
HAVE_S3loggerget_iso_time()link_file()download_file_from_internet()download_file_from_s3()get_file_class()safe_read_file()safe_write_file()MemoryStateCacheAbstractStateStoreMemoryStateStoreFileStateStoreS3StateStorestate_store_cacheconnect_to_state_store()WorkflowStateStoreconnect_to_workflow_state_store()TERMINAL_STATESMAX_CANCELING_SECONDSWorkflowStateMachineWorkflowStateMachine.send_enqueue()WorkflowStateMachine.send_initialize()WorkflowStateMachine.send_run()WorkflowStateMachine.send_cancel()WorkflowStateMachine.send_canceled()WorkflowStateMachine.send_complete()WorkflowStateMachine.send_executor_error()WorkflowStateMachine.send_system_error()WorkflowStateMachine.get_current_state()
- Module Contents
toil.server.wsgi_app
- Subpackages
toil.test- Subpackages
toil.test.batchSystems- Submodules
toil.test.batchSystems.batchSystemTest- Module Contents
- Classes
- Functions
- Attributes
loggernumCorespreemptibledefaultRequirementsBatchSystemPluginTesthiddenhidden.AbstractBatchSystemTesthidden.AbstractBatchSystemTest.createBatchSystem()hidden.AbstractBatchSystemTest.supportsWallTime()hidden.AbstractBatchSystemTest.createConfig()hidden.AbstractBatchSystemTest.setUpClass()hidden.AbstractBatchSystemTest.setUp()hidden.AbstractBatchSystemTest.tearDown()hidden.AbstractBatchSystemTest.get_max_startup_seconds()hidden.AbstractBatchSystemTest.test_available_cores()hidden.AbstractBatchSystemTest.test_run_jobs()hidden.AbstractBatchSystemTest.test_set_env()hidden.AbstractBatchSystemTest.test_set_job_env()hidden.AbstractBatchSystemTest.testCheckResourceRequest()hidden.AbstractBatchSystemTest.testScalableBatchSystem()
hidden.AbstractBatchSystemJobTesthidden.AbstractBatchSystemJobTest.cpuCounthidden.AbstractBatchSystemJobTest.allocatedCoreshidden.AbstractBatchSystemJobTest.sleepTimehidden.AbstractBatchSystemJobTest.getBatchSystemName()hidden.AbstractBatchSystemJobTest.getOptions()hidden.AbstractBatchSystemJobTest.setUp()hidden.AbstractBatchSystemJobTest.tearDown()hidden.AbstractBatchSystemJobTest.testJobConcurrency()hidden.AbstractBatchSystemJobTest.test_omp_threads()
hidden.AbstractGridEngineBatchSystemTest
KubernetesBatchSystemTestKubernetesBatchSystemBenchTestTESBatchSystemTestAWSBatchBatchSystemTestMesosBatchSystemTestwrite_temp_file()SingleMachineBatchSystemTestMaxCoresSingleMachineBatchSystemTestparentJob()childJob()grandChildJob()greatGrandChild()ServiceParasolBatchSystemTestGridEngineBatchSystemTestSlurmBatchSystemTestLSFBatchSystemTestTorqueBatchSystemTestHTCondorBatchSystemTestSingleMachineBatchSystemJobTestMesosBatchSystemJobTestmeasureConcurrency()count()getCounters()resetCounters()get_omp_threads()
- Module Contents
toil.test.batchSystems.parasolTestSupporttoil.test.batchSystems.test_lsf_helpertoil.test.batchSystems.test_slurm- Module Contents
- Classes
- Functions
call_sacct()call_scontrol()call_sacct_raises()FakeBatchSystemSlurmTestSlurmTest.setUp()SlurmTest.test_getJobDetailsFromSacct_one_exists()SlurmTest.test_getJobDetailsFromSacct_one_not_exists()SlurmTest.test_getJobDetailsFromSacct_many_all_exist()SlurmTest.test_getJobDetailsFromSacct_many_some_exist()SlurmTest.test_getJobDetailsFromSacct_many_none_exist()SlurmTest.test_getJobDetailsFromScontrol_one_exists()SlurmTest.test_getJobDetailsFromScontrol_one_not_exists()SlurmTest.test_getJobDetailsFromScontrol_many_all_exist()SlurmTest.test_getJobDetailsFromScontrol_many_some_exist()SlurmTest.test_getJobDetailsFromScontrol_many_none_exist()SlurmTest.test_getJobExitCode_job_exists()SlurmTest.test_getJobExitCode_job_not_exists()SlurmTest.test_getJobExitCode_sacct_raises_job_exists()SlurmTest.test_getJobExitCode_sacct_raises_job_not_exists()SlurmTest.test_coalesce_job_exit_codes_one_exists()SlurmTest.test_coalesce_job_exit_codes_one_not_exists()SlurmTest.test_coalesce_job_exit_codes_many_all_exist()SlurmTest.test_coalesce_job_exit_codes_some_exists()SlurmTest.test_coalesce_job_exit_codes_sacct_raises_job_exists()SlurmTest.test_coalesce_job_exit_codes_sacct_raises_job_not_exists()
- Module Contents
- Submodules
toil.test.cwl- Submodules
toil.test.cwl.conftesttoil.test.cwl.cwlTest- Module Contents
- Classes
- Functions
- Attributes
pkg_rootlogCONFORMANCE_TEST_TIMEOUTrun_conformance_tests()CWLWorkflowTestCWLWorkflowTest.setUp()CWLWorkflowTest.tearDown()CWLWorkflowTest.revsort()CWLWorkflowTest.revsort_no_checksum()CWLWorkflowTest.download()CWLWorkflowTest.load_contents()CWLWorkflowTest.download_directory()CWLWorkflowTest.download_subdirectory()CWLWorkflowTest.test_mpi()CWLWorkflowTest.test_s3_as_secondary_file()CWLWorkflowTest.test_run_revsort()CWLWorkflowTest.test_run_revsort_nochecksum()CWLWorkflowTest.test_run_revsort2()CWLWorkflowTest.test_run_revsort_debug_worker()CWLWorkflowTest.test_run_colon_output()CWLWorkflowTest.test_download_s3()CWLWorkflowTest.test_download_http()CWLWorkflowTest.test_download_https()CWLWorkflowTest.test_download_file()CWLWorkflowTest.test_download_directory_s3()CWLWorkflowTest.test_download_directory_file()CWLWorkflowTest.test_download_subdirectory_s3()CWLWorkflowTest.test_download_subdirectory_file()CWLWorkflowTest.test_load_contents_s3()CWLWorkflowTest.test_load_contents_http()CWLWorkflowTest.test_load_contents_https()CWLWorkflowTest.test_load_contents_file()CWLWorkflowTest.test_bioconda()CWLWorkflowTest.test_biocontainers()CWLWorkflowTest.test_cuda()CWLWorkflowTest.test_restart()CWLWorkflowTest.test_streamable()
CWLv10TestCWLv10Test.setUp()CWLv10Test.tearDown()CWLv10Test.test_run_conformance_with_caching()CWLv10Test.test_run_conformance()CWLv10Test.test_lsf_cwl_conformance()CWLv10Test.test_slurm_cwl_conformance()CWLv10Test.test_torque_cwl_conformance()CWLv10Test.test_gridengine_cwl_conformance()CWLv10Test.test_mesos_cwl_conformance()CWLv10Test.test_parasol_cwl_conformance()CWLv10Test.test_kubernetes_cwl_conformance()CWLv10Test.test_lsf_cwl_conformance_with_caching()CWLv10Test.test_slurm_cwl_conformance_with_caching()CWLv10Test.test_torque_cwl_conformance_with_caching()CWLv10Test.test_gridengine_cwl_conformance_with_caching()CWLv10Test.test_mesos_cwl_conformance_with_caching()CWLv10Test.test_parasol_cwl_conformance_with_caching()CWLv10Test.test_kubernetes_cwl_conformance_with_caching()
CWLv11TestCWLv12TestCWLv12Test.setUpClass()CWLv12Test.tearDown()CWLv12Test.test_run_conformance()CWLv12Test.test_run_conformance_with_caching()CWLv12Test.test_run_conformance_with_in_place_update()CWLv12Test.test_kubernetes_cwl_conformance()CWLv12Test.test_kubernetes_cwl_conformance_with_caching()CWLv12Test.test_wes_server_cwl_conformance()
CWLOnARMTesttest_workflow_echo_string_scatter_stderr_log_dir()test_log_dir_echo_no_output()test_log_dir_echo_stderr()test_filename_conflict_resolution()test_filename_conflict_detection()test_filename_conflict_detection_at_root()test_pick_value_with_one_null_value()test_usage_message()test_workflow_echo_string()test_workflow_echo_string_scatter_capture_stdout()test_visit_top_cwl_class()test_visit_cwl_class_and_reduce()test_download_structure()
- Module Contents
- Submodules
toil.test.docs- Submodules
toil.test.docs.scriptsTest- Module Contents
- Classes
- Attributes
pkg_rootToilDocumentationTestToilDocumentationTest.setUpClass()ToilDocumentationTest.tearDown()ToilDocumentationTest.checkExitCode()ToilDocumentationTest.checkExpectedOut()ToilDocumentationTest.checkExpectedPattern()ToilDocumentationTest.testCwlexample()ToilDocumentationTest.testDiscoverfiles()ToilDocumentationTest.testDynamic()ToilDocumentationTest.testEncapsulation()ToilDocumentationTest.testEncapsulation2()ToilDocumentationTest.testHelloworld()ToilDocumentationTest.testInvokeworkflow()ToilDocumentationTest.testInvokeworkflow2()ToilDocumentationTest.testJobFunctions()ToilDocumentationTest.testManaging()ToilDocumentationTest.testManaging2()ToilDocumentationTest.testMultiplejobs()ToilDocumentationTest.testMultiplejobs2()ToilDocumentationTest.testMultiplejobs3()ToilDocumentationTest.testPromises2()ToilDocumentationTest.testQuickstart()ToilDocumentationTest.testRequirements()ToilDocumentationTest.testArguments()ToilDocumentationTest.testDocker()ToilDocumentationTest.testPromises()ToilDocumentationTest.testServices()ToilDocumentationTest.testStaging()
- Module Contents
- Submodules
toil.test.jobStores- Submodules
toil.test.jobStores.jobStoreTest- Module Contents
- Classes
- Functions
- Attributes
google_retry()loggertearDownModule()AbstractJobStoreTestAbstractJobStoreTest.TestAbstractJobStoreTest.Test.externalStoreCacheAbstractJobStoreTest.Test.mpTestPartSizeAbstractJobStoreTest.Test.setUpClass()AbstractJobStoreTest.Test.setUp()AbstractJobStoreTest.Test.tearDown()AbstractJobStoreTest.Test.testInitialState()AbstractJobStoreTest.Test.testJobCreation()AbstractJobStoreTest.Test.testConfigEquality()AbstractJobStoreTest.Test.testJobLoadEquality()AbstractJobStoreTest.Test.testChildLoadingEquality()AbstractJobStoreTest.Test.testPersistantFilesToDelete()AbstractJobStoreTest.Test.testUpdateBehavior()AbstractJobStoreTest.Test.testJobDeletions()AbstractJobStoreTest.Test.testSharedFiles()AbstractJobStoreTest.Test.testReadWriteSharedFilesTextMode()AbstractJobStoreTest.Test.testReadWriteFileStreamTextMode()AbstractJobStoreTest.Test.testPerJobFiles()AbstractJobStoreTest.Test.testStatsAndLogging()AbstractJobStoreTest.Test.testWriteLogFiles()AbstractJobStoreTest.Test.testBatchCreate()AbstractJobStoreTest.Test.testGrowingAndShrinkingJob()AbstractJobStoreTest.Test.cleanUpExternalStores()AbstractJobStoreTest.Test.makeImportExportTests()AbstractJobStoreTest.Test.testImportHttpFile()AbstractJobStoreTest.Test.testImportFtpFile()AbstractJobStoreTest.Test.testFileDeletion()AbstractJobStoreTest.Test.testMultipartUploads()AbstractJobStoreTest.Test.testZeroLengthFiles()AbstractJobStoreTest.Test.testLargeFile()AbstractJobStoreTest.Test.fetch_url()AbstractJobStoreTest.Test.assertUrl()AbstractJobStoreTest.Test.testCleanCache()AbstractJobStoreTest.Test.testPartialReadFromStream()AbstractJobStoreTest.Test.testDestructionOfCorruptedJobStore()AbstractJobStoreTest.Test.testDestructionIdempotence()AbstractJobStoreTest.Test.testEmptyFileStoreIDIsReadable()
AbstractEncryptedJobStoreTestFileJobStoreTestGoogleJobStoreTestAWSJobStoreTestInvalidAWSJobStoreTestEncryptedAWSJobStoreTestStubHttpRequestHandler
- Module Contents
- Submodules
toil.test.lib- Subpackages
- Submodules
toil.test.lib.dockerTest- Module Contents
- Classes
- Attributes
loggerDockerTestDockerTest.setUp()DockerTest.testDockerClean()DockerTest.testDockerClean_CRx_FORGO()DockerTest.testDockerClean_CRx_STOP()DockerTest.testDockerClean_CRx_RM()DockerTest.testDockerClean_CRx_None()DockerTest.testDockerClean_CxD_FORGO()DockerTest.testDockerClean_CxD_STOP()DockerTest.testDockerClean_CxD_RM()DockerTest.testDockerClean_CxD_None()DockerTest.testDockerClean_Cxx_FORGO()DockerTest.testDockerClean_Cxx_STOP()DockerTest.testDockerClean_Cxx_RM()DockerTest.testDockerClean_Cxx_None()DockerTest.testDockerClean_xRx_FORGO()DockerTest.testDockerClean_xRx_STOP()DockerTest.testDockerClean_xRx_RM()DockerTest.testDockerClean_xRx_None()DockerTest.testDockerClean_xxD_FORGO()DockerTest.testDockerClean_xxD_STOP()DockerTest.testDockerClean_xxD_RM()DockerTest.testDockerClean_xxD_None()DockerTest.testDockerClean_xxx_FORGO()DockerTest.testDockerClean_xxx_STOP()DockerTest.testDockerClean_xxx_RM()DockerTest.testDockerClean_xxx_None()DockerTest.testDockerPipeChain()DockerTest.testDockerPipeChainErrorDetection()DockerTest.testNonCachingDockerChain()DockerTest.testNonCachingDockerChainErrorDetection()DockerTest.testDockerLogs()DockerTest.testDockerLogs_Stream()DockerTest.testDockerLogs_Demux()DockerTest.testDockerLogs_Demux_Stream()
- Module Contents
toil.test.lib.test_conversionstoil.test.lib.test_ec2toil.test.lib.test_misc
toil.test.mesostoil.test.provisioners- Subpackages
toil.test.provisioners.aws
- Submodules
toil.test.provisioners.clusterScalerTest- Module Contents
- Classes
- Attributes
loggerc4_8xlarge_preemptiblec4_8xlarger3_8xlarger5_2xlarger5_4xlarget2_microBinPackingTestBinPackingTest.setUp()BinPackingTest.testPackingOneShape()BinPackingTest.testSorting()BinPackingTest.testAddingInitialNode()BinPackingTest.testLowTargetTime()BinPackingTest.testHighTargetTime()BinPackingTest.testZeroResourceJobs()BinPackingTest.testLongRunningJobs()BinPackingTest.run1000JobsOnMicros()BinPackingTest.testPathologicalCase()BinPackingTest.testJobTooLargeForAllNodes()
ClusterScalerTestClusterScalerTest.setUp()ClusterScalerTest.testRounding()ClusterScalerTest.testMaxNodes()ClusterScalerTest.testMinNodes()ClusterScalerTest.testPreemptibleDeficitResponse()ClusterScalerTest.testPreemptibleDeficitIsSet()ClusterScalerTest.testNoLaunchingIfDeltaAlreadyMet()ClusterScalerTest.testBetaInertia()ClusterScalerTest.test_overhead_accounting_large()ClusterScalerTest.test_overhead_accounting_small()ClusterScalerTest.test_overhead_accounting_observed()
ScalerThreadTestMockBatchSystemAndProvisionerMockBatchSystemAndProvisioner.start()MockBatchSystemAndProvisioner.shutDown()MockBatchSystemAndProvisioner.nodeInUse()MockBatchSystemAndProvisioner.ignoreNode()MockBatchSystemAndProvisioner.unignoreNode()MockBatchSystemAndProvisioner.supportedClusterTypes()MockBatchSystemAndProvisioner.createClusterSettings()MockBatchSystemAndProvisioner.readClusterSettings()MockBatchSystemAndProvisioner.setAutoscaledNodeTypes()MockBatchSystemAndProvisioner.getProvisionedWorkers()MockBatchSystemAndProvisioner.terminateNodes()MockBatchSystemAndProvisioner.remainingBillingInterval()MockBatchSystemAndProvisioner.addJob()MockBatchSystemAndProvisioner.getNumberOfJobsIssued()MockBatchSystemAndProvisioner.getJobs()MockBatchSystemAndProvisioner.getNodes()MockBatchSystemAndProvisioner.addNodes()MockBatchSystemAndProvisioner.getNodeShape()MockBatchSystemAndProvisioner.getWorkersInCluster()MockBatchSystemAndProvisioner.launchCluster()MockBatchSystemAndProvisioner.destroyCluster()MockBatchSystemAndProvisioner.getLeader()MockBatchSystemAndProvisioner.getNumberOfNodes()
- Module Contents
toil.test.provisioners.clusterTesttoil.test.provisioners.gceProvisionerTest- Module Contents
- Classes
- Attributes
logAbstractGCEAutoscaleTestAbstractGCEAutoscaleTest.projectIDAbstractGCEAutoscaleTest.sshUtil()AbstractGCEAutoscaleTest.rsyncUtil()AbstractGCEAutoscaleTest.destroyClusterUtil()AbstractGCEAutoscaleTest.createClusterUtil()AbstractGCEAutoscaleTest.cleanJobStoreUtil()AbstractGCEAutoscaleTest.setUp()AbstractGCEAutoscaleTest.tearDown()AbstractGCEAutoscaleTest.launchCluster()
GCEAutoscaleTestGCEStaticAutoscaleTestGCEAutoscaleTestMultipleNodeTypesGCERestartTest
- Module Contents
toil.test.provisioners.provisionerTesttoil.test.provisioners.restartScript
- Subpackages
toil.test.server- Submodules
toil.test.server.serverTest- Module Contents
- Classes
- Attributes
loggerToilServerUtilsTesthiddenFileStateStoreTestFileStateStoreURLTestBucketUsingTestAWSStateStoreTestAbstractToilWESServerTestToilWESServerBenchTestToilWESServerWorkflowTestToilWESServerWorkflowTest.run_zip_workflow()ToilWESServerWorkflowTest.test_run_workflow_relative_url_no_attachments_fails()ToilWESServerWorkflowTest.test_run_workflow_relative_url()ToilWESServerWorkflowTest.test_run_workflow_https_url()ToilWESServerWorkflowTest.test_run_workflow_single_file_zip()ToilWESServerWorkflowTest.test_run_workflow_multi_file_zip()ToilWESServerWorkflowTest.test_run_workflow_manifest_zip()ToilWESServerWorkflowTest.test_run_workflow_inputs_zip()ToilWESServerWorkflowTest.test_run_workflow_manifest_and_inputs_zip()ToilWESServerWorkflowTest.test_run_workflow_no_params_zip()ToilWESServerWorkflowTest.test_run_and_cancel_workflows()
ToilWESServerCeleryWorkflowTestToilWESServerCeleryS3StateWorkflowTest
- Module Contents
- Submodules
toil.test.sort- Submodules
toil.test.sort.restart_sorttoil.test.sort.sorttoil.test.sort.sortTest- Module Contents
- Classes
- Functions
- Attributes
loggerdefaultLineLendefaultLinesdefaultNrunMain()SortTestSortTest.testNoSortTest.setUp()SortTest.tearDown()SortTest.testAwsSingle()SortTest.testAwsMesos()SortTest.testFileMesos()SortTest.testGoogleSingle()SortTest.testGoogleMesos()SortTest.testFileSingle()SortTest.testFileSingleNonCaching()SortTest.testFileSingleCheckpoints()SortTest.testFileSingle10000()SortTest.testFileGridEngine()SortTest.testFileTorqueEngine()SortTest.testFileParasol()SortTest.testSort()SortTest.testMerge()SortTest.testCopySubRangeOfFile()SortTest.testGetMidPoint()
- Module Contents
- Submodules
toil.test.src- Submodules
toil.test.src.autoDeploymentTest- Module Contents
- Classes
- Attributes
loggerAutoDeploymentTestAutoDeploymentTest.sitePackagesAutoDeploymentTest.setUp()AutoDeploymentTest.testRestart()AutoDeploymentTest.testSplitRootPackages()AutoDeploymentTest.testUserTypesInJobFunctionArgs()AutoDeploymentTest.testDeferralWithConcurrentEncapsulation()AutoDeploymentTest.testDeferralWithFailureAndEncapsulation()
- Module Contents
toil.test.src.busTesttoil.test.src.checkpointTesttoil.test.src.deferredFunctionTest- Module Contents
- Classes
- Attributes
loggerDeferredFunctionTestDeferredFunctionTest.jobStoreTypeDeferredFunctionTest.setUp()DeferredFunctionTest.testDeferredFunctionRunsWithMethod()DeferredFunctionTest.testDeferredFunctionRunsWithClassMethod()DeferredFunctionTest.testDeferredFunctionRunsWithLambda()DeferredFunctionTest.testDeferredFunctionRunsWithFailures()DeferredFunctionTest.testNewJobsCanHandleOtherJobDeaths()DeferredFunctionTest.testBatchSystemCleanupCanHandleWorkerDeaths()
- Module Contents
toil.test.src.dockerCheckTest- Module Contents
- Classes
DockerCheckTestDockerCheckTest.testOfficialUbuntuRepo()DockerCheckTest.testBroadDockerRepo()DockerCheckTest.testBroadDockerRepoBadTag()DockerCheckTest.testNonexistentRepo()DockerCheckTest.testToilQuayRepo()DockerCheckTest.testBadQuayRepoNTag()DockerCheckTest.testBadQuayRepo()DockerCheckTest.testBadQuayTag()DockerCheckTest.testGoogleRepo()DockerCheckTest.testBadGoogleRepo()DockerCheckTest.testApplianceParser()
- Classes
- Module Contents
toil.test.src.fileStoreTest- Module Contents
- Classes
- Attributes
testingIsAutomaticloggerhiddenhidden.AbstractFileStoreTesthidden.AbstractFileStoreTest.jobStoreTypehidden.AbstractFileStoreTest.setUp()hidden.AbstractFileStoreTest.create_file()hidden.AbstractFileStoreTest.testToilIsNotBroken()hidden.AbstractFileStoreTest.testFileStoreLogging()hidden.AbstractFileStoreTest.testFileStoreOperations()hidden.AbstractFileStoreTest.testWriteReadGlobalFilePermissions()hidden.AbstractFileStoreTest.testWriteExportFileCompatibility()hidden.AbstractFileStoreTest.testImportReadFileCompatibility()hidden.AbstractFileStoreTest.testReadWriteFileStreamTextMode()
hidden.AbstractNonCachingFileStoreTesthidden.AbstractCachingFileStoreTesthidden.AbstractCachingFileStoreTest.setUp()hidden.AbstractCachingFileStoreTest.testExtremeCacheSetup()hidden.AbstractCachingFileStoreTest.testCacheEvictionPartialEvict()hidden.AbstractCachingFileStoreTest.testCacheEvictionTotalEvict()hidden.AbstractCachingFileStoreTest.testCacheEvictionFailCase()hidden.AbstractCachingFileStoreTest.testAsyncWriteWithCaching()hidden.AbstractCachingFileStoreTest.testWriteNonLocalFileToJobStore()hidden.AbstractCachingFileStoreTest.testWriteLocalFileToJobStore()hidden.AbstractCachingFileStoreTest.testReadCacheMissFileFromJobStoreWithoutCachingReadFile()hidden.AbstractCachingFileStoreTest.testReadCacheMissFileFromJobStoreWithCachingReadFile()hidden.AbstractCachingFileStoreTest.testReadCachHitFileFromJobStore()hidden.AbstractCachingFileStoreTest.testMultipleJobsReadSameCacheHitGlobalFile()hidden.AbstractCachingFileStoreTest.testMultipleJobsReadSameCacheMissGlobalFile()hidden.AbstractCachingFileStoreTest.testFileStoreExportFile()hidden.AbstractCachingFileStoreTest.testReturnFileSizes()hidden.AbstractCachingFileStoreTest.testReturnFileSizesWithBadWorker()hidden.AbstractCachingFileStoreTest.testControlledFailedWorkerRetry()hidden.AbstractCachingFileStoreTest.testRemoveLocalMutablyReadFile()hidden.AbstractCachingFileStoreTest.testRemoveLocalImmutablyReadFile()hidden.AbstractCachingFileStoreTest.testDeleteLocalFile()hidden.AbstractCachingFileStoreTest.testSimultaneousReadsUncachedStream()
NonCachingFileStoreTestWithFileJobStoreCachingFileStoreTestWithFileJobStoreNonCachingFileStoreTestWithAwsJobStoreCachingFileStoreTestWithAwsJobStoreNonCachingFileStoreTestWithGoogleJobStoreCachingFileStoreTestWithGoogleJobStore
- Module Contents
toil.test.src.helloWorldTesttoil.test.src.importExportFileTesttoil.test.src.jobDescriptionTesttoil.test.src.jobEncapsulationTesttoil.test.src.jobFileStoreTesttoil.test.src.jobServiceTesttoil.test.src.jobTest- Module Contents
- Classes
- Functions
- Attributes
loggerJobTestJobTest.setUpClass()JobTest.testStatic()JobTest.testStatic2()JobTest.testTrivialDAGConsistency()JobTest.testDAGConsistency()JobTest.testSiblingDAGConsistency()JobTest.testDeadlockDetection()JobTest.testNewCheckpointIsLeafVertexNonRootCase()JobTest.testNewCheckpointIsLeafVertexRootCase()JobTest.runNewCheckpointIsLeafVertexTest()JobTest.runCheckpointVertexTest()JobTest.testEvaluatingRandomDAG()JobTest.getRandomEdge()JobTest.makeRandomDAG()JobTest.getAdjacencyList()JobTest.reachable()JobTest.addRandomFollowOnEdges()JobTest.makeJobGraph()JobTest.isAcyclic()
simpleJobFn()fn1Test()fn2Test()trivialParent()parent()diamond()child()errorChild()TrivialService
- Module Contents
toil.test.src.miscTeststoil.test.src.promisedRequirementTest- Module Contents
- Classes
- Functions
- Attributes
loghiddenhidden.AbstractPromisedRequirementsTesthidden.AbstractPromisedRequirementsTest.testConcurrencyDynamic()hidden.AbstractPromisedRequirementsTest.testConcurrencyStatic()hidden.AbstractPromisedRequirementsTest.getOptions()hidden.AbstractPromisedRequirementsTest.getCounterPath()hidden.AbstractPromisedRequirementsTest.testPromisesWithJobStoreFileObjects()hidden.AbstractPromisedRequirementsTest.testPromisesWithNonCachingFileStore()hidden.AbstractPromisedRequirementsTest.testPromiseRequirementRaceStatic()
maxConcurrency()getOne()getThirtyTwoMb()logDiskUsage()SingleMachinePromisedRequirementsTestMesosPromisedRequirementsTest
- Module Contents
toil.test.src.promisesTesttoil.test.src.realtimeLoggerTesttoil.test.src.regularLogTesttoil.test.src.resourceTesttoil.test.src.restartDAGTesttoil.test.src.resumabilityTesttoil.test.src.retainTempDirTesttoil.test.src.systemTesttoil.test.src.threadingTesttoil.test.src.toilContextManagerTesttoil.test.src.userDefinedJobArgTypeTesttoil.test.src.workerTest
- Submodules
toil.test.utils- Submodules
toil.test.utils.toilDebugTesttoil.test.utils.toilKillTesttoil.test.utils.utilsTest- Module Contents
- Classes
- Functions
- Attributes
pkg_rootloggerUtilsTestUtilsTest.toilMainUtilsTest.cleanCommandUtilsTest.statsCommandUtilsTest.setUp()UtilsTest.tearDown()UtilsTest.statusCommand()UtilsTest.testAWSProvisionerUtils()UtilsTest.testUtilsSort()UtilsTest.testUtilsStatsSort()UtilsTest.testUnicodeSupport()UtilsTest.testMultipleJobsPerWorkerStats()UtilsTest.check_status()UtilsTest.testGetPIDStatus()UtilsTest.testGetStatusFailedToilWF()UtilsTest.testGetStatusFailedCWLWF()UtilsTest.testGetStatusSuccessfulCWLWF()UtilsTest.testPrintJobLog()UtilsTest.testRestartAttribute()
printUnicodeCharacter()RunTwoJobsPerWorker
- Module Contents
- Submodules
toil.test.wdl- Submodules
toil.test.wdl.builtinTest- Module Contents
- Classes
WdlStandardLibraryFunctionsTestWdlStandardLibraryFunctionsTest.setUp()WdlStandardLibraryFunctionsTest.setUpClass()WdlStandardLibraryFunctionsTest.tearDown()WdlStandardLibraryFunctionsTest.testFn_Sub()WdlStandardLibraryFunctionsTest.testFn_Ceil()WdlStandardLibraryFunctionsTest.testFn_Floor()WdlStandardLibraryFunctionsTest.testFn_ReadLines()WdlStandardLibraryFunctionsTest.testFn_ReadTsv()WdlStandardLibraryFunctionsTest.testFn_ReadJson()WdlStandardLibraryFunctionsTest.testFn_ReadMap()WdlStandardLibraryFunctionsTest.testFn_ReadInt()WdlStandardLibraryFunctionsTest.testFn_ReadString()WdlStandardLibraryFunctionsTest.testFn_ReadFloat()WdlStandardLibraryFunctionsTest.testFn_ReadBoolean()WdlStandardLibraryFunctionsTest.testFn_WriteLines()WdlStandardLibraryFunctionsTest.testFn_WriteTsv()WdlStandardLibraryFunctionsTest.testFn_WriteJson()WdlStandardLibraryFunctionsTest.testFn_WriteMap()WdlStandardLibraryFunctionsTest.testFn_Transpose()WdlStandardLibraryFunctionsTest.testFn_Length()WdlStandardLibraryFunctionsTest.testFn_Zip()WdlStandardLibraryFunctionsTest.testFn_Cross()
WdlWorkflowsTestWdlLanguageSpecWorkflowsTestWdlStandardLibraryWorkflowsTestWdlStandardLibraryWorkflowsTest.setUpClass()WdlStandardLibraryWorkflowsTest.test_sub()WdlStandardLibraryWorkflowsTest.test_size()WdlStandardLibraryWorkflowsTest.test_ceil()WdlStandardLibraryWorkflowsTest.test_floor()WdlStandardLibraryWorkflowsTest.test_round()WdlStandardLibraryWorkflowsTest.test_stdout()WdlStandardLibraryWorkflowsTest.test_read()WdlStandardLibraryWorkflowsTest.test_write()WdlStandardLibraryWorkflowsTest.test_range()WdlStandardLibraryWorkflowsTest.test_transpose()WdlStandardLibraryWorkflowsTest.test_length()WdlStandardLibraryWorkflowsTest.test_zip()WdlStandardLibraryWorkflowsTest.test_cross()WdlStandardLibraryWorkflowsTest.test_as_pairs()WdlStandardLibraryWorkflowsTest.test_as_map()WdlStandardLibraryWorkflowsTest.test_keys()WdlStandardLibraryWorkflowsTest.test_collect_by_key()WdlStandardLibraryWorkflowsTest.test_flatten()
- Classes
- Module Contents
toil.test.wdl.conftesttoil.test.wdl.toilwdlTest- Module Contents
- Classes
- Functions
BaseToilWdlTestToilWdlTestToilWDLLibraryTestToilWDLLibraryTest.testFn_SelectFirst()ToilWDLLibraryTest.testFn_Size()ToilWDLLibraryTest.testFn_Basename()ToilWDLLibraryTest.testFn_Glob()ToilWDLLibraryTest.testFn_ParseMemory()ToilWDLLibraryTest.testFn_ParseCores()ToilWDLLibraryTest.testFn_ParseDisk()ToilWDLLibraryTest.testPrimitives()ToilWDLLibraryTest.testCSV()ToilWDLLibraryTest.testTSV()
ToilWdlIntegrationTestToilWdlIntegrationTest.gatk_dataToilWdlIntegrationTest.gatk_data_dirToilWdlIntegrationTest.encode_dataToilWdlIntegrationTest.encode_data_dirToilWdlIntegrationTest.wdl_dataToilWdlIntegrationTest.wdl_data_dirToilWdlIntegrationTest.setUpClass()ToilWdlIntegrationTest.tearDownClass()ToilWdlIntegrationTest.testTut01()ToilWdlIntegrationTest.testTut02()ToilWdlIntegrationTest.testTut03()ToilWdlIntegrationTest.testTut04()ToilWdlIntegrationTest.testENCODE()ToilWdlIntegrationTest.testPipe()ToilWdlIntegrationTest.testJSON()ToilWdlIntegrationTest.test_size_large()ToilWdlIntegrationTest.fetch_and_unzip_from_s3()
compare_runs()compare_vcf_files()
- Module Contents
toil.test.wdl.wdltoil_test
- Submodules
- Package Contents
- Classes
- Functions
- Attributes
ApplianceImageNotFoundapplianceSelf()toilPackageDirPath()have_working_nvidia_docker_runtime()have_working_nvidia_smi()running_on_ec2()concatmemoizeExceptionalThreadcpu_count()distVersionloggerToilTestMTget_temp_file()needs_env_var()needs_rsync3()needs_aws_s3()needs_aws_ec2()needs_aws_batch()needs_google()needs_gridengine()needs_torque()needs_tes()needs_kubernetes_installed()needs_kubernetes()needs_mesos()needs_parasol()needs_slurm()needs_htcondor()needs_lsf()needs_java()needs_docker()needs_singularity()needs_local_cuda()needs_docker_cuda()needs_encryption()needs_cwl()needs_server()needs_celery_broker()needs_wes_server()needs_local_appliance()needs_fetchable_appliance()integrative()slow()methodNamePartRegextimeLimit()make_tests()ApplianceTestSupportApplianceTestSupport.ApplianceApplianceTestSupport.Appliance.lockApplianceTestSupport.Appliance.__enter__()ApplianceTestSupport.Appliance.__exit__()ApplianceTestSupport.Appliance.tryRun()ApplianceTestSupport.Appliance.runOnAppliance()ApplianceTestSupport.Appliance.writeToAppliance()ApplianceTestSupport.Appliance.deployScript()
ApplianceTestSupport.LeaderThreadApplianceTestSupport.WorkerThread
- Subpackages
toil.utils- Submodules
toil.utils.toilCleantoil.utils.toilDebugFiletoil.utils.toilDebugJobtoil.utils.toilDestroyClustertoil.utils.toilKilltoil.utils.toilLaunchClustertoil.utils.toilMaintoil.utils.toilRsyncClustertoil.utils.toilServertoil.utils.toilSshClustertoil.utils.toilStats- Module Contents
- Classes
- Functions
- Attributes
loggerColumnWidthspadStr()prettyMemory()prettyTime()reportTime()reportMemory()reportNumber()sprintTag()decorateTitle()decorateSubHeader()get()sortJobs()reportPrettyData()computeColumnWidths()updateColumnWidths()buildElement()createSummary()getStats()processData()reportData()category_choicessort_category_choicessort_field_choicesadd_stats_options()main()
- Module Contents
toil.utils.toilStatus- Module Contents
- Classes
- Functions
- Attributes
loggerToilStatusToilStatus.print_dot_chart()ToilStatus.printJobLog()ToilStatus.printJobChildren()ToilStatus.printAggregateJobStats()ToilStatus.report_on_jobs()ToilStatus.getPIDStatus()ToilStatus.getStatus()ToilStatus.print_bus_messages()ToilStatus.fetchRootJob()ToilStatus.fetchUserJobs()ToilStatus.traverseJobGraph()
main()
- Module Contents
toil.utils.toilUpdateEC2Instances
- Submodules
toil.wdl- Subpackages
toil.wdl.versions- Submodules
toil.wdl.versions.dev- Module Contents
- Classes
- Attributes
loggerAnalyzeDevelopmentWDLAnalyzeDevelopmentWDL.versionAnalyzeDevelopmentWDL.analyze()AnalyzeDevelopmentWDL.visit_document()AnalyzeDevelopmentWDL.visit_document_element()AnalyzeDevelopmentWDL.visit_call()AnalyzeDevelopmentWDL.visit_string_expr_part()AnalyzeDevelopmentWDL.visit_wdl_type()AnalyzeDevelopmentWDL.visit_expr_core()
- Module Contents
toil.wdl.versions.draft2- Module Contents
- Classes
- Attributes
loggerAnalyzeDraft2WDLAnalyzeDraft2WDL.versionAnalyzeDraft2WDL.analyze()AnalyzeDraft2WDL.write_AST()AnalyzeDraft2WDL.find_asts()AnalyzeDraft2WDL.create_tasks_dict()AnalyzeDraft2WDL.parse_task()AnalyzeDraft2WDL.parse_task_rawcommand_attributes()AnalyzeDraft2WDL.parse_task_rawcommand()AnalyzeDraft2WDL.modify_cmd_expr_w_attributes()AnalyzeDraft2WDL.parse_task_runtime_key()AnalyzeDraft2WDL.parse_task_runtime()AnalyzeDraft2WDL.parse_task_outputs()AnalyzeDraft2WDL.translate_wdl_string_to_python_string()AnalyzeDraft2WDL.create_workflows_dict()AnalyzeDraft2WDL.parse_workflow()AnalyzeDraft2WDL.parse_workflow_body()AnalyzeDraft2WDL.parse_workflow_if()AnalyzeDraft2WDL.parse_workflow_if_expression()AnalyzeDraft2WDL.parse_workflow_scatter()AnalyzeDraft2WDL.parse_workflow_scatter_item()AnalyzeDraft2WDL.parse_workflow_scatter_collection()AnalyzeDraft2WDL.parse_declaration()AnalyzeDraft2WDL.parse_declaration_name()AnalyzeDraft2WDL.parse_declaration_type()AnalyzeDraft2WDL.parse_declaration_expressn()AnalyzeDraft2WDL.parse_declaration_expressn_logicalnot()AnalyzeDraft2WDL.parse_declaration_expressn_arraymaplookup()AnalyzeDraft2WDL.parse_declaration_expressn_memberaccess()AnalyzeDraft2WDL.parse_declaration_expressn_ternaryif()AnalyzeDraft2WDL.parse_declaration_expressn_tupleliteral()AnalyzeDraft2WDL.parse_declaration_expressn_arrayliteral()AnalyzeDraft2WDL.parse_declaration_expressn_operator()AnalyzeDraft2WDL.parse_declaration_expressn_fncall()AnalyzeDraft2WDL.parse_declaration_expressn_fncall_normalparams()AnalyzeDraft2WDL.parse_workflow_call_taskname()AnalyzeDraft2WDL.parse_workflow_call_taskalias()AnalyzeDraft2WDL.parse_workflow_call_body_declarations()AnalyzeDraft2WDL.parse_workflow_call_body_io()AnalyzeDraft2WDL.parse_workflow_call_body_io_map()AnalyzeDraft2WDL.parse_workflow_call_body()AnalyzeDraft2WDL.parse_workflow_call()
- Module Contents
toil.wdl.versions.v1- Module Contents
- Classes
- Functions
- Attributes
loggeris_context()AnalyzeV1WDLAnalyzeV1WDL.versionAnalyzeV1WDL.analyze()AnalyzeV1WDL.visit_document()AnalyzeV1WDL.visit_document_element()AnalyzeV1WDL.visit_workflow()AnalyzeV1WDL.visit_workflow_input()AnalyzeV1WDL.visit_workflow_output()AnalyzeV1WDL.visit_inner_workflow_element()AnalyzeV1WDL.visit_call()AnalyzeV1WDL.visit_scatter()AnalyzeV1WDL.visit_conditional()AnalyzeV1WDL.visit_task()AnalyzeV1WDL.visit_task_input()AnalyzeV1WDL.visit_task_output()AnalyzeV1WDL.visit_task_command()AnalyzeV1WDL.visit_task_command_string_part()AnalyzeV1WDL.visit_task_command_expr_with_string()AnalyzeV1WDL.visit_task_command_expr_part()AnalyzeV1WDL.visit_task_runtime()AnalyzeV1WDL.visit_any_decls()AnalyzeV1WDL.visit_unbound_decls()AnalyzeV1WDL.visit_bound_decls()AnalyzeV1WDL.visit_wdl_type()AnalyzeV1WDL.visit_primitive_literal()AnalyzeV1WDL.visit_number()AnalyzeV1WDL.visit_string()AnalyzeV1WDL.visit_string_expr_with_string_part()AnalyzeV1WDL.visit_string_expr_part()AnalyzeV1WDL.visit_string_part()AnalyzeV1WDL.visit_expression_placeholder_option()AnalyzeV1WDL.visit_expr()AnalyzeV1WDL.visit_infix0()AnalyzeV1WDL.visit_lor()AnalyzeV1WDL.visit_infix1()AnalyzeV1WDL.visit_land()AnalyzeV1WDL.visit_infix2()AnalyzeV1WDL.visit_infix3()AnalyzeV1WDL.visit_infix4()AnalyzeV1WDL.visit_infix5()AnalyzeV1WDL.visit_expr_core()AnalyzeV1WDL.visit_apply()AnalyzeV1WDL.visit_array_literal()AnalyzeV1WDL.visit_pair_literal()AnalyzeV1WDL.visit_ifthenelse()AnalyzeV1WDL.visit_expression_group()AnalyzeV1WDL.visit_at()AnalyzeV1WDL.visit_get_name()AnalyzeV1WDL.visit_negate()AnalyzeV1WDL.visit_unarysigned()AnalyzeV1WDL.visit_primitives()
- Module Contents
- Submodules
- Submodules
toil.wdl.toilwdltoil.wdl.utilstoil.wdl.wdl_analysistoil.wdl.wdl_functions- Module Contents
- Classes
- Functions
- Attributes
loggerWDLRuntimeErrorWDLJSONEncodergenerate_docker_bashscript_file()process_single_infile()process_infile()sub()defined()process_single_outfile()process_outfile()abspath_single_file()abspath_file()read_single_file()read_file()process_and_read_file()generate_stdout_file()parse_memory()parse_cores()parse_disk()is_number()size()select_first()combine_dicts()basename()heredoc_wdl()floor()ceil()read_lines()read_tsv()read_csv()read_json()read_map()read_int()read_string()read_float()read_boolean()write_lines()write_tsv()write_json()write_map()wdl_range()transpose()length()wdl_zip()cross()as_pairs()as_map()keys()collect_by_key()flatten()
- Module Contents
toil.wdl.wdl_synthesis- Module Contents
- Classes
- Attributes
loggerSynthesizeWDLSynthesizeWDL.write_modules()SynthesizeWDL.write_main()SynthesizeWDL.write_main_header()SynthesizeWDL.write_main_jobwrappers()SynthesizeWDL.write_main_jobwrappers_declaration()SynthesizeWDL.write_main_destbucket()SynthesizeWDL.fetch_ignoredifs()SynthesizeWDL.fetch_ignoredifs_chain()SynthesizeWDL.write_main_jobwrappers_if()SynthesizeWDL.write_main_jobwrappers_scatter()SynthesizeWDL.fetch_scatter_outputs()SynthesizeWDL.fetch_scatter_inputs()SynthesizeWDL.fetch_scatter_inputs_chain()SynthesizeWDL.write_main_jobwrappers_call()SynthesizeWDL.fetch_call_outputs()SynthesizeWDL.write_functions()SynthesizeWDL.write_scatterfunctions_within_if()SynthesizeWDL.write_scatterfunction()SynthesizeWDL.write_scatterfunction_header()SynthesizeWDL.write_scatterfunction_outputreturn()SynthesizeWDL.write_scatterfunction_lists()SynthesizeWDL.write_scatterfunction_loop()SynthesizeWDL.write_scatter_callwrapper()SynthesizeWDL.write_function()SynthesizeWDL.write_function_header()SynthesizeWDL.json_var()SynthesizeWDL.needs_file_import()SynthesizeWDL.write_declaration_type()SynthesizeWDL.write_function_bashscriptline()SynthesizeWDL.write_function_dockercall()SynthesizeWDL.write_function_cmdline()SynthesizeWDL.write_function_subprocesspopen()SynthesizeWDL.write_function_outputreturn()SynthesizeWDL.indent()SynthesizeWDL.needsdocker()SynthesizeWDL.write_python_file()
- Module Contents
toil.wdl.wdl_typestoil.wdl.wdltoil- Module Contents
- Classes
- Functions
- Attributes
loggerpotential_absolute_uris()toil_read_source()WDLBindingscombine_bindings()log_bindings()get_supertype()for_each_node()recursive_dependencies()TOIL_URI_SCHEMEpack_toil_uri()unpack_toil_uri()NonDownloadingSizeToilWDLStdLibBaseToilWDLStdLibTaskOutputsevaluate_named_expression()evaluate_decl()evaluate_call_inputs()evaluate_defaultable_decl()devirtualize_files()virtualize_files()import_files()drop_missing_files()get_file_paths_in_bindings()map_over_typed_files_in_bindings()map_over_files_in_bindings()map_over_typed_files_in_binding()map_over_typed_files_in_value()WDLBaseJobWDLTaskJobWDLWorkflowNodeJobWDLCombineBindingsJobWDLNamespaceBindingsJobWDLSectionJobWDLScatterJobWDLArrayBindingsJobWDLConditionalJobWDLWorkflowJobWDLOutputsJobWDLRootJobmain()
- Module Contents
- Subpackages
- Submodules
toil.bus- Module Contents
- Classes
- Functions
- Attributes
loggerJobIssuedMessageJobUpdatedMessageJobCompletedMessageJobFailedMessageJobMissingMessageJobAnnotationMessageExternalBatchIdMessageQueueSizeMessageClusterSizeMessageClusterDesiredSizeMessagemessage_to_bytes()MessageTypebytes_to_message()MessageBusMessageBusClientMessageInboxMessageOutboxMessageBusConnectionJobStatusreplay_message_bus()gen_message_bus_path()
- Module Contents
toil.common- Module Contents
- Classes
- Functions
- Attributes
defaultTargetTimeSYS_MAX_SIZEUUID_LENGTHloggerConfigConfig.logFileConfig.logRotatingConfig.cleanWorkDirConfig.max_jobsConfig.max_local_jobsConfig.run_local_jobs_on_workersConfig.tes_endpointConfig.tes_userConfig.tes_passwordConfig.tes_bearer_tokenConfig.jobStoreConfig.batchSystemConfig.batch_logs_dirConfig.workflowAttemptNumberConfig.disableAutoDeploymentConfig.workflowIDConfig.prepare_start()Config.prepare_restart()Config.setOptions()Config.__eq__()Config.__hash__()
JOBSTORE_HELPparser_with_common_options()addOptions()parseBool()getNodeID()ToilToil.configToil.__enter__()Toil.__exit__()Toil.start()Toil.restart()Toil.getJobStore()Toil.parseLocator()Toil.buildLocator()Toil.resumeJobStore()Toil.createBatchSystem()Toil.importFile()Toil.import_file()Toil.exportFile()Toil.export_file()Toil.normalize_uri()Toil.getToilWorkDir()Toil.get_toil_coordination_dir()Toil.getLocalWorkflowDir()Toil.get_local_workflow_coordination_dir()
ToilRestartExceptionToilContextManagerExceptionToilMetricsToilMetrics.startDashboard()ToilMetrics.add_prometheus_data_source()ToilMetrics.log()ToilMetrics.logClusterSize()ToilMetrics.logClusterDesiredSize()ToilMetrics.logQueueSize()ToilMetrics.logMissingJob()ToilMetrics.logIssuedJob()ToilMetrics.logFailedJob()ToilMetrics.logCompletedJob()ToilMetrics.shutdown()
parseSetEnv()iC()fC()parse_accelerator_list()cacheDirName()getDirSizeRecursively()getFileSystemSize()safeUnpickleFromStream()
- Module Contents
toil.deferredtoil.exceptionstoil.job- Module Contents
- Classes
- Functions
- Attributes
loggerJobPromiseConstraintErrorConflictingPredecessorErrorTemporaryIDAcceleratorRequirementparse_accelerator()accelerator_satisfies()accelerators_fully_satisfy()RequirementsDictREQUIREMENT_NAMESParsedRequirementParseableIndivisibleResourceParseableDivisibleResourceParseableFlagParseableAcceleratorRequirementParseableRequirementRequirerJobDescriptionJobDescription.servicesJobDescription.remainingTryCountJobDescription.serviceHostIDsInBatches()JobDescription.successorsAndServiceHosts()JobDescription.allSuccessors()JobDescription.successors_by_phase()JobDescription.nextSuccessors()JobDescription.filterSuccessors()JobDescription.filterServiceHosts()JobDescription.clear_nonexistent_dependents()JobDescription.clear_dependents()JobDescription.is_subtree_done()JobDescription.replace()JobDescription.addChild()JobDescription.addFollowOn()JobDescription.addServiceHostJob()JobDescription.hasChild()JobDescription.hasFollowOn()JobDescription.hasServiceHostJob()JobDescription.renameReferences()JobDescription.addPredecessor()JobDescription.onRegistration()JobDescription.setupJobAfterFailure()JobDescription.getLogFileHandle()JobDescription.clearRemainingTryCount()JobDescription.__str__()JobDescription.__repr__()JobDescription.pre_update_hook()JobDescription.get_job_kind()
ServiceJobDescriptionCheckpointJobDescriptionJobJob.RunnerJob.ServiceJob.jobStoreIDJob.descriptionJob.diskJob.memoryJob.coresJob.acceleratorsJob.preemptibleJob.checkpointJob.tempDirJob.__str__()Job.preemptable()Job.assignConfig()Job.run()Job.addChild()Job.hasChild()Job.addFollowOn()Job.hasPredecessor()Job.hasFollowOn()Job.addService()Job.hasService()Job.addChildFn()Job.addFollowOnFn()Job.addChildJobFn()Job.addFollowOnJobFn()Job.log()Job.wrapFn()Job.wrapJobFn()Job.encapsulate()Job.rv()Job.registerPromise()Job.prepareForPromiseRegistration()Job.checkJobGraphForDeadlocks()Job.getRootJobs()Job.checkJobGraphConnected()Job.checkJobGraphAcylic()Job.checkNewCheckpointsAreLeafVertices()Job.defer()Job.getUserScript()Job.getTopologicalOrderingOfJobs()Job.saveBody()Job.saveAsRootJob()Job.loadJob()
JobExceptionJobGraphDeadlockExceptionFunctionWrappingJobJobFunctionWrappingJobPromisedRequirementFunctionWrappingJobPromisedRequirementJobFunctionWrappingJobEncapsulatedJobServiceHostJobPromiseTPromisedunwrap()unwrap_all()PromisedRequirementUnfulfilledPromiseSentinel
- Module Contents
toil.leader- Module Contents
- Classes
- Attributes
loggerLeaderLeader.run()Leader.create_status_sentinel_file()Leader.innerLoop()Leader.checkForDeadlocks()Leader.feed_deadlock_watchdog()Leader.issueJob()Leader.issueJobs()Leader.issueServiceJob()Leader.issueQueingServiceJobs()Leader.getNumberOfJobsIssued()Leader.removeJob()Leader.getJobs()Leader.killJobs()Leader.reissueOverLongJobs()Leader.reissueMissingJobs()Leader.processRemovedJob()Leader.process_finished_job()Leader.process_finished_job_description()Leader.getSuccessors()Leader.processTotallyFailedJob()
- Module Contents
toil.realtimeLoggertoil.resource- Module Contents
- Classes
- Attributes
loggerResourceFileResourceDirectoryResourceVirtualEnvResourceModuleDescriptorModuleDescriptor.belongsToToilModuleDescriptor.dirPathModuleDescriptor.nameModuleDescriptor.forModule()ModuleDescriptor.saveAsResourceTo()ModuleDescriptor.localize()ModuleDescriptor.globalize()ModuleDescriptor.toCommand()ModuleDescriptor.fromCommand()ModuleDescriptor.makeLoadable()ModuleDescriptor.load()
ResourceException
- Module Contents
toil.serviceManager- Module Contents
- Classes
- Attributes
loggerServiceManagerServiceManager.services_are_starting()ServiceManager.get_job_count()ServiceManager.start()ServiceManager.put_client()ServiceManager.get_ready_client()ServiceManager.get_unservable_client()ServiceManager.get_startable_service()ServiceManager.kill_services()ServiceManager.is_active()ServiceManager.is_running()ServiceManager.check()ServiceManager.shutdown()
- Module Contents
toil.statsAndLoggingtoil.toilStatetoil.versiontoil.worker
- Package Contents
- Functions
- Attributes
memoizeretry()currentCommitlogwhich()toilPackageDirPath()inVirtualEnv()resolveEntryPoint()physicalMemory()physicalDisk()applianceSelf()customDockerInitCmd()customInitCmd()lookupEnvVar()checkDockerImageExists()parseDockerAppliance()checkDockerSchema()ApplianceImageNotFoundKNOWN_EXTANT_IMAGESrequestCheckRegularDocker()requestCheckDockerIo()logProcessContext()cache_path
- Subpackages
tutorial_dockertutorial_managing2tutorial_helloworldtutorial_discoverfilestutorial_multiplejobs2tutorial_dynamictutorial_invokeworkflow2tutorial_jobfunctionstutorial_managingexample_alwaysfailexample_cachingbenchmarktutorial_quickstarttutorial_encapsulation2tutorial_multiplejobs3tutorial_cwlexampletutorial_encapsulationtutorial_invokeworkflowtutorial_requirementstutorial_stagingtutorial_promisestutorial_servicestutorial_promises2tutorial_multiplejobstutorial_argumentsmkFiledebugWorkflowfake_mpi_run
