Cloud storage (S3, GCS, Azure Blob)
Terra can dump raw data payloads directly into your cloud storage bucket. Supported providers: AWS S3, Google Cloud Storage (GCS), and Azure Blob Storage.
Authentication
Create an IAM user with write access to your target bucket. Attach a policy granting at minimum s3:PutObject on your bucket.
Enter the bucket name, region, Access Key ID, and Secret Access Key in the Terra Dashboard when adding the S3 destination.
Create a service account and grant it the Storage Object Admin role (roles/storage.objectAdmin) on your target bucket. This role is required because Terra overwrites existing files on redelivery, which needs both storage.objects.create and storage.objects.delete permissions. Download the JSON credentials file.
You can grant the role via the console or with:
gsutil iam ch serviceAccount:YOUR_SA@YOUR_PROJECT.iam.gserviceaccount.com:objectAdmin gs://YOUR_BUCKETEnter the bucket name and upload the credentials JSON in the Terra Dashboard when adding the GCS destination.
Generate a SAS URL for your storage account that grants write access to your target container.
Enter the SAS URL and container name in the Terra Dashboard when adding the Azure Blob destination.
Data Structure
All cloud storage destinations use the same folder structure:
2022-03-16/
{event_type}/
{user_id}-{timestamp}.json # Auth/misc events
{data_type}/
{user_id}-{start_time}.json # Data events (activity, sleep, etc.)Examples:
2022-03-16/auth/abc123-2026-03-12T14:30:00Z.json
2022-03-16/activity/abc123-2026-03-12T06:00:00Z.json
2022-03-16/sleep/abc123-2026-03-11T22:00:00Z.jsonData events (activity, sleep, body, etc.) are placed under a folder named after the data type, with each file named
{user_id}-{start_time}.json.Auth and other events are placed under a folder named after the event type, with each file named
{user_id}-{timestamp}.json.The
2022-03-16prefix corresponds to the API version.
Last updated
Was this helpful?