broadinstitute/terra-tools repository overview

⁠Tools for use with Terra

To run a script locally:

1. Clone the repository and cd into the terra-tools directory
2. Create a virtual environment by running python3 -m venv venv
3. Activate the virtual environment by running source venv/bin/activate
4. Install the required packages by running pip install -r requirements.txt
5. Authenticate with Google Cloud by running gcloud auth application-default login
6. Run the script by running python3 scripts/path_to_script/<script name.py> <arguments>

To run a script using Docker:

docker run --rm -it -v "$HOME"/.config:/.config -v "$HOME"/Documents/:/data broadinstitute/terra-tools:latest bash -c "cd data; python3 /scripts/path_to_script/<script name.py> <arguments>"

1. -v "$HOME"/.config:/.config - allows for authentication within the Docker of your Google credentials in your local $HOME directory where they are stored by default
2. -v "$HOME"/Documents/:/data - mounts the directory named "Documents" to a folder named "data" within the Docker - files generated by scripts will be written to your "Documents" directory - change "Documents" to the path of any location on your local machine
3. bash -c - directs Docker to run the following bash commands (within quotes and separated by semi-colon)

For example, to run the import_large_tsv.py script:

1. move input file (<tsv_name>) to Documents/ directory locally
2. run docker run --rm -v "$HOME"/.config:/.config -v "$HOME"/Documents/:/data broadinstitute/terra-tools:latest bash -c "cd data; python3 scripts/import_large_tsv/import_large_tsv.py --tsv data/<tsv_name> --project <terra_project> --workspace <terra_workspace>"

⁠Prerequisites

• Install the Google Cloud SDK from https://cloud.google.com/sdk/downloads⁠
• Set the Application Default Credentials (run gcloud auth application-default login)
• Python 3.7

When running without the docker, check the packages in requirements.txt.