Request Historical data

Learn why and how to request for historical data to be sent to your destination through the REST API

Prerequisites

Before requesting historical data:

1. Verify Data Sources:

   • Ensure the required data sources and permissions (e.g., activity, sleep) are enabled.

2. Connect the User:

   • The User must be connected via Terra.

   • The user_id is needed to make requests for historical data

When to request historical data

Upon creating a User

When an end user first connects their wearable device, you will by default only receive data from that point onwards.

If you would like to build a historical health & fitness profile of the user, to establish baselines for statistics and other calculations you might want to perform, you will need to make a request to Terra for data from prior to the user's authentication

Historical data limits

Some providers impose limits on how far back a developer can request data. Those are:

• Garmin: 5 years into the past, from the current point in time

• Polar: no historical data retrieval permitted

• Health Connect: 30 days into the past, from the moment the user granted permission

As a "pull down to refresh" functionality

As a fallback, you should implement a "pull down to refresh" functionality or alternative (like a sync now button, etc...). Hitting this button should trigger a data pull of the last e.g. 14 days of data for the user, for all the data types you ingest

Using the dashboard to request historical data

You may want your customer support team to use the Terra dashboard to debug by re-sending data, or verifying whether or not Terra fetches the data correctly. See the guide below for a step by step tutorial on how to do so.

Practical example

Taking Python FastAPI as an example framework, you may set up your server code to look like the following:

@app.post("/webhook")
async def webhook_handler(request: Request):
    payload = await request.json()
    
    if payload.get("type") == "auth" and payload.get("status") == "success":
        # Calculate date range, to get data from 28 days ago to today
        start_date = (datetime.utcnow() - timedelta(days=28)).strftime('%Y-%m-%d')
        end_date = (datetime.utcnow() + timedelta(days=1)).strftime('%Y-%m-%d')
        headers = {
            "x-api-key": os.getenv("TERRA_API_KEY"), 
            "dev-id": os.getenv("TERRA_DEVELOPER_ID")
        }
        endpoints = [
            "/activity", 
            "/daily", 
            "/sleep", 
            "/body"
        ]

        # Fetch data from each endpoint
        results = {}
        for endpoint in endpoints:
            try:
                response = requests.get(
                    f"{TERRA_API_URL}{endpoint}", 
                    headers=headers, 
                    params={
                        "start_date": start_date, 
                        "end_date": end_date,
                        "to_webhook": True
                        }
                    )
                response.raise_for_status()
                results[endpoint] = response.json()
            except requests.HTTPError as e:
                raise HTTPException(status_code=response.status_code, detail=f"Error fetching data from {endpoint}")

        return {"message": "Data fetched successfully", "data": results}
    
    return {"message": "Webhook type or status did not match criteria"}

Refer to the API endpoints section for further details on usage.

In the example above, to_webhook is set to true. As a result, any request made to Terra will be handled asynchronously, and the data retrieved will be sent to your destination as soon as it is fetched. For synchronous data retrieval (if your use case is time-sensitive), you may set this parameter to false.

Handling asynchronous data retrievals

When making a to_webhook=true request, keep track of the value of the terra-reference header. When the event is eventually sent to your webhook, the event's terra-signature will match that of the initial request, allowing you to mark off the data transfer as successfully completed

Request longer time ranges

For time ranges longer than 28 days, Terra will default to sending the data asynchronously, even if to_webhook is set to false. This is due to the large amounts of data being transferred, which would hang the request making/handling process for >30 seconds.

When a large request is made, Terra will send the data to you in chunks of

• at most 10MB

• at most 10 objects in the data array

ALL of which will contain the same terra-reference header value.

The chunk size will be limited by whichever of the two limits gets hit first.

Following a large request, you will immediately receive a #large-request-processing event. This indicates that Terra has acknowledged the large request and is currently processing the data retrieval from the Provider.

Once data retrieval has finished, Terra will organize the data into chunks as mentioned above, and send a #large-request-sending payload, indicating the number of payloads to expect.

Once the number of payloads indicated with the same terra-reference header value have been received, you can consider the data transfer request complete.