Running in GCE

After setting up Toil on Installation, Toil scripts can be run just by designating a job store location as shown in Running a basic workflow.

Note

Google Cloud Storage is available in Toil for experimental purposes. Only AWS is currently supported in Toil.

If you wish to use the Google Storage job store, install Toil with the google extra (Installing extra features). Then, create a file named .boto with your credentials and some configuration:

[Credentials]
gs_access_key_id = KEY_ID
gs_secret_access_key = SECRET_KEY

[Boto]
https_validate_certificates = True

[GSUtil]
content_language = en
default_api_version = 2

gs_access_key_id and gs_secret_access_key can be generated by navigating to your Google Cloud Storage console and clicking on Settings. On the Settings page, navigate to the Interoperability tab and click Enable interoperability access. On this page you can now click Create a new key to generate an access key and a matching secret. Insert these into their respective places in the .boto file and you will be able to use a Google job store when invoking a Toil script, as in the following example:

$ python HelloWorld.py google:projectID:jobStore

The projectID component of the job store argument above refers your Google Cloud Project ID in the Google Cloud Console, and will be visible in the console’s banner at the top of the screen. The jobStore component is a name of your choosing that you will use to refer to this job store.