Download a file securely from GCS on an untrusted system
We publish some of our build artifacts to Google Cloud Storage, and users need to download these to the target installation system. But, this target system is not always trusted and can have shared local users, so we don't want to store long-lived credentials.
As a user, I can download the artifact on my (secure) laptop and transfer it to the target system. But, the artifact can be large (several GBs). So, downloading and uploading again makes it cumbersome and slow.
Option 1: use gcloud CLI on the target system
Log in to the target system, install gcloud CLI, authenticate, and then download the file:
This has two problems:
- The user must install (and maybe update) gcloud CLI on the target system.
- The user needs to store their credentials on the target system. These credentials have full access to whatever resources the user has. So, it's a huge security risk, especially if we don't trust the target system.
To mitigate (2), the user can log out of gcloud CLI after downloading. But, this is a manual step they might miss.
Option 2: use gcloud CLI with a service account
This is a variation of the above solution - we log in using a service account instead of the user account. This service account can have restricted access to only the resources needed.
This partially mitigates problem (2) above. If the user forgets to log out of gcloud CLI, the damage will be restricted to the resources accessible by the service account.
Option 3: Short-lived access token
Gcloud CLI supports creating short-lived credentials for the end-user account or any service account.
This credential can be used to download the artifact using wget with an authorization header - no need to install gcloud CLI.
Here's a small script that asks for the auth token as input, parses various GCS bucket URL formats, and downloads the requested artifact directly using wget:
Option 4: Signed URLs
Google Cloud Storage also supports signed URLs - which give time-limited access to a specific Cloud Storage resource. Anyone possessing the signed URL can use it while it's active without any further credentials. This fits our use case brilliantly.
To do this, first we need to give ourselves the
iam.serviceAccountTokenCreator role so that we can impersonate a service account.
Then, we can generate a signed URL:
And we can use wget to download the artifact from this URL without any further authentication.