> For the complete documentation index, see [llms.txt](https://docs.tryterra.co/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.tryterra.co/reference/admin-api/environment-doctor.md).

# Environment doctor

Check an environment for problems across every product, and what each finding means.

The environment doctor checks one environment and returns one finding, called a lint, for each problem you can fix yourself. Problems on Terra's side are handled by Terra and not reported.

## Run the doctor

{% tabs %}
{% tab title="CLI" %}

```bash
terra doctor check --env <DEV_ID>
```

`<DEV_ID>` is the environment's dev ID or name. Without `--env`, the CLI uses `TERRA_ENV` or your default environment.
{% endtab %}

{% tab title="API" %}

```bash
curl https://access.tryterra.co/api/v3/admin/environments/<DEV_ID>/doctor \
  -H "Authorization: Bearer <TOKEN>"
```

{% endtab %}
{% endtabs %}

Checks run on each call and nothing is stored. The same token gets the same report for the same environment for 30 seconds. Checks for a product the environment does not have, such as Vantage, are skipped and do not appear. Many checks count over a window, such as the last hour, 24 hours or 30 days, so their finding stays after you fix the cause until the window moves past the problem.

### Filter checks

Every check runs by default.

| Parameter  | CLI flag     | Takes                                                                    |
| ---------- | ------------ | ------------------------------------------------------------------------ |
| `name`     | `--name`     | Check names, such as `token_leaked`.                                     |
| `category` | `--category` | `security`, `performance` or `health`. No check is in `performance` yet. |

Pass several values comma-separated or by repeating the parameter. A check runs if it matches any value of a parameter, and must match both parameters when you pass both. An unknown name or category returns `400`.

```bash
terra doctor check --env <DEV_ID> --category security
terra doctor check --env <DEV_ID> --name destination_failing,destination_degraded
```

### Permissions

The doctor needs `destinations:read`, plus the scope below for each check. A check whose scope your token lacks is reported as [could not run](#check_unavailable), never as a pass.

| Checks                                                                                                                                                                           | Scope                |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
| `environment_inactive`, `api_version_behind`                                                                                                                                     | `account:read`       |
| `billing_unlinked`, `subscription_past_due`                                                                                                                                      | `billing:read`       |
| `team_invitation_expiring`                                                                                                                                                       | `team:read`          |
| `token_leaked`, `token_never_expires`, `token_unused`, `token_cap_near`                                                                                                          | `tokens:admin`       |
| `api_key_missing`, `data_token_expiring`                                                                                                                                         | `tokens:read`        |
| `testing_environment_live_users`, `user_limit_near`, `user_limit_reached`, `connections_stale`, `synthetic_users_running`, `synthetic_capacity_reached`, `streaming_disconnects` | `connections:read`   |
| `sources_none_enabled`, `source_credentials_missing`, `source_deprecated`, `source_not_enableable`, `app_subscription_failed`, `source_domain_unhealthy`                         | `providers:read`     |
| `connections_insufficient_permissions`, `connections_access_revoked`                                                                                                             | `events:read`        |
| `data_types_none_selected`, `data_type_not_delivered`, `destination_events_unfed`                                                                                                | `dataconfig:read`    |
| `vantage_markets_not_chosen`, `vantage_catalogue_empty`, `vantage_webhook_not_configured`                                                                                        | `vantageconfig:read` |
| The other Vantage checks                                                                                                                                                         | `vantage:read`       |
| `lab_reports_delivery_failed`, `lab_reports_processing_failed`                                                                                                                   | `labreports:read`    |
| `model_runs_failing`                                                                                                                                                             | `models:read`        |
| `planned_workouts_unsynced`                                                                                                                                                      | `workouts:read`      |
| Every other check                                                                                                                                                                | No other scope       |

## Response

```json
{
  "lints": [
    {
      "name": "destination_failing",
      "title": "Destination is failing",
      "level": "error",
      "facing": "external",
      "categories": ["health"],
      "description": "Events are not reaching the destination: …",
      "detail": "The endpoint answered 429 Too Many Requests.",
      "remediation": {
        "text": "List the destination's failed events …",
        "action": "getDestinationsHealth",
        "docs_url": "https://docs.tryterra.co/reference/admin-api/environment-doctor#destination_failing"
      },
      "metadata": {
        "type": "destination",
        "id": "1937…",
        "name": "webhook",
        "reason": "rate_limited",
        "since": "2026-11-05T09:12:00Z",
        "observed": { "delivered_last_hour": 0, "failed_last_hour": 12 }
      },
      "cache_key": "destination_failing:1937…",
      "observed_at": "2026-11-05T12:00:00Z"
    }
  ]
}
```

| Field                          | Meaning                                                                            |
| ------------------------------ | ---------------------------------------------------------------------------------- |
| `name`                         | Check that found the problem.                                                      |
| `title`, `description`         | What the check looks for. Same for every lint from one check.                      |
| `level`                        | Urgency. See [Levels](#levels).                                                    |
| `categories`                   | `security`, `performance` or `health`.                                             |
| `facing`                       | Always `external`.                                                                 |
| `detail`                       | What this run found, such as which token or how many users.                        |
| `remediation.text`             | How to fix it.                                                                     |
| `remediation.action`           | operationId of the Admin API call that inspects or fixes it, or `null`.            |
| `remediation.docs_url`         | The check's section on this page.                                                  |
| `metadata.type`                | What the lint is about, such as `environment`, `destination`, `source` or `token`. |
| `metadata.id`, `metadata.name` | Which one, when the lint is about a single thing.                                  |
| `metadata.reason`              | Machine-readable reason, when the check has one.                                   |
| `metadata.since`               | When the problem started, or will happen, such as a revocation date.               |
| `metadata.observed`            | Counts behind the finding.                                                         |
| `cache_key`                    | Same for the same problem across runs. See [Accept a finding](#accept-a-finding).  |
| `observed_at`                  | When the checks ran.                                                               |

An empty `lints` array means every check that ran found nothing. Lints are sorted by level, errors first. A check reports at most 25 lints; if it found more, its last lint has `metadata.observed.omitted` set to the number left out.

### Levels

| Level   | Meaning                                                                                         |
| ------- | ----------------------------------------------------------------------------------------------- |
| `error` | Broken now: data is not arriving, users cannot connect, or a credential is exposed. Fix it now. |
| `warn`  | Will break, or is getting worse. Fix it soon.                                                   |
| `info`  | Worth knowing. Nothing is broken.                                                               |

There is no overall pass or fail. A few checks report a higher level in a production environment, where the same setting does more harm with real users.

## Use the doctor in CI

With `TERRA_ADMIN_TOKEN` set to a [personal access token](https://docs.tryterra.co/reference/admin-api/personal-access-tokens.md) holding the scopes your checks need and `TERRA_ENV` set to the dev ID, this script fails on any `error` lint and prints each with its fix:

{% code title="doctor-gate.sh" %}

```bash
#!/usr/bin/env bash
set -euo pipefail

errors=$(terra doctor check --env "$TERRA_ENV" \
  --jq '[.lints[] | select(.level == "error")]')

if [ "$(jq length <<<"$errors")" -gt 0 ]; then
  jq -r '.[] | "\(.name): \(.detail)\n  fix: \(.remediation.text)\n  docs: \(.remediation.docs_url)"' <<<"$errors"
  exit 1
fi
```

{% endcode %}

`check_unavailable` is `info`, so the script passes it. To block a deploy when a check could not run, also fail on `.name == "check_unavailable"`.

## Accept a finding

The doctor has no setting to hide a finding you chose on purpose, such as a pinned API version. `cache_key` stays the same for the same problem across runs, so filter out the ones you accept:

{% code title="filter-accepted.sh" %}

```bash
# accepted.txt holds one cache_key per line
terra doctor check --env <DEV_ID> --jq '.lints[]' \
  | jq -c --rawfile accepted accepted.txt \
      'select(.cache_key as $k | ($accepted | split("\n") | index($k)) | not)'
```

{% endcode %}

A different destination, token or source gets a different `cache_key`, so accepting one never hides a new problem.

## Checks that could not run <a href="#check_unavailable" id="check_unavailable"></a>

A check that cannot run is reported, never treated as a pass. The response has one `info` `check_unavailable` lint per cause. `metadata.checks` lists the affected checks and `metadata.reason` gives the cause:

| Reason                   | Meaning                                              | Fix                                                                                                          |
| ------------------------ | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| `missing_scope`          | The token lacks the scope in `metadata.scope`.       | Use a token that holds it, or narrow the run with `--name` or `--category`. See [Permissions](#permissions). |
| `dependency_unavailable` | A Terra service the checks read from did not answer. | Run the doctor again shortly.                                                                                |

## Checks at a glance

| Check                                                                           | Level                         | Category   | Meaning                                         |
| ------------------------------------------------------------------------------- | ----------------------------- | ---------- | ----------------------------------------------- |
| [`environment_inactive`](#environment_inactive)                                 | `error`                       | `health`   | Environment is turned off                       |
| [`testing_environment_live_users`](#testing_environment_live_users)             | `warn`                        | `health`   | Testing environment has many users              |
| [`api_version_behind`](#api_version_behind)                                     | `info`                        | `health`   | API version is not the latest                   |
| [`deprecated_api_usage`](#deprecated_api_usage)                                 | `info` or `warn`              | `health`   | Deprecated API in use                           |
| [`billing_unlinked`](#billing_unlinked)                                         | `error`                       | `health`   | Billing is not set up                           |
| [`subscription_past_due`](#subscription_past_due)                               | `error`                       | `health`   | Subscription payment is overdue                 |
| [`team_invitation_expiring`](#team_invitation_expiring)                         | `info`                        | `health`   | Team invitation about to expire                 |
| [`api_key_missing`](#api_key_missing)                                           | `error`                       | `health`   | No live API key                                 |
| [`data_token_expiring`](#data_token_expiring)                                   | `warn`                        | `security` | Unused API key will be revoked                  |
| [`token_leaked`](#token_leaked)                                                 | `error`                       | `security` | A token was found in public                     |
| [`token_never_expires`](#token_never_expires)                                   | `warn`                        | `security` | Admin token never expires                       |
| [`token_unused`](#token_unused)                                                 | `info`                        | `security` | Admin token unused for 90 days                  |
| [`token_cap_near`](#token_cap_near)                                             | `info`                        | `health`   | Close to the token limit                        |
| [`sources_none_enabled`](#sources_none_enabled)                                 | `warn`                        | `health`   | No sources enabled                              |
| [`source_credentials_missing`](#source_credentials_missing)                     | `error`                       | `health`   | Source needs your own credentials               |
| [`source_deprecated`](#source_deprecated)                                       | `warn`                        | `health`   | Source is retired                               |
| [`source_not_enableable`](#source_not_enableable)                               | `warn`                        | `health`   | Source is not available to this environment     |
| [`app_subscription_failed`](#app_subscription_failed)                           | `warn`                        | `health`   | Source app webhooks are not registered          |
| [`source_domain_unhealthy`](#source_domain_unhealthy)                           | `error` or `warn`             | `health`   | Custom auth domain is not serving               |
| [`user_limit_near`](#user_limit_near)                                           | `warn`                        | `health`   | Close to the user limit                         |
| [`user_limit_reached`](#user_limit_reached)                                     | `error`                       | `health`   | User limit reached                              |
| [`connections_insufficient_permissions`](#connections_insufficient_permissions) | `warn`                        | `health`   | Users connected without the permissions needed  |
| [`connections_access_revoked`](#connections_access_revoked)                     | `warn`                        | `health`   | Users revoked access                            |
| [`connections_stale`](#connections_stale)                                       | `warn`                        | `health`   | Connections stopped updating                    |
| [`synthetic_users_running`](#synthetic_users_running)                           | `info`, `warn` in production  | `health`   | Test users are running                          |
| [`synthetic_capacity_reached`](#synthetic_capacity_reached)                     | `warn`                        | `health`   | Test user limit reached                         |
| [`destinations_none_active`](#destinations_none_active)                         | `info`                        | `health`   | No active destination                           |
| [`destination_failing`](#destination_failing)                                   | `error`                       | `health`   | Destination is failing                          |
| [`destination_degraded`](#destination_degraded)                                 | `warn`                        | `health`   | Some deliveries to a destination are failing    |
| [`destination_auto_disabled`](#destination_auto_disabled)                       | `error`                       | `health`   | Destination was turned off after failing        |
| [`destination_insecure_http`](#destination_insecure_http)                       | `warn`, `error` in production | `security` | Destination is not encrypted                    |
| [`destination_url_credentials`](#destination_url_credentials)                   | `error`                       | `security` | Destination URL contains a password             |
| [`data_types_none_selected`](#data_types_none_selected)                         | `error`                       | `health`   | No data types selected                          |
| [`data_type_not_delivered`](#data_type_not_delivered)                           | `warn`                        | `health`   | Data type selected but not delivered            |
| [`destination_events_unfed`](#destination_events_unfed)                         | `warn`                        | `health`   | Destination waits for data that is not selected |
| [`destination_events_unsupported`](#destination_events_unsupported)             | `warn`                        | `health`   | Destination is sent events it cannot store      |
| [`vantage_markets_not_chosen`](#vantage_markets_not_chosen)                     | `warn`                        | `health`   | No Vantage markets chosen                       |
| [`vantage_catalogue_empty`](#vantage_catalogue_empty)                           | `warn`                        | `health`   | Vantage catalogue is empty                      |
| [`vantage_orders_delayed`](#vantage_orders_delayed)                             | `warn`                        | `health`   | Orders are delayed                              |
| [`vantage_payment_failed_orders`](#vantage_payment_failed_orders)               | `error`                       | `health`   | Order payments failed                           |
| [`vantage_kits_not_activated`](#vantage_kits_not_activated)                     | `warn`                        | `health`   | Kits not activated                              |
| [`vantage_results_failed`](#vantage_results_failed)                             | `error`                       | `health`   | Results failed                                  |
| [`vantage_results_unacknowledged`](#vantage_results_unacknowledged)             | `warn`                        | `health`   | Results are waiting to be acknowledged          |
| [`vantage_webhook_not_configured`](#vantage_webhook_not_configured)             | `warn`, `error` in production | `health`   | No Vantage webhook                              |
| [`vantage_webhook_failures`](#vantage_webhook_failures)                         | `warn`                        | `health`   | Vantage webhooks failed in the last 24 hours    |
| [`lab_reports_delivery_failed`](#lab_reports_delivery_failed)                   | `error`                       | `health`   | Lab reports were not delivered                  |
| [`lab_reports_processing_failed`](#lab_reports_processing_failed)               | `warn`                        | `health`   | Lab reports failed processing                   |
| [`model_runs_failing`](#model_runs_failing)                                     | `warn`                        | `health`   | Model runs are failing                          |
| [`planned_workouts_unsynced`](#planned_workouts_unsynced)                       | `warn`                        | `health`   | Planned workouts never reached the device       |
| [`streaming_disconnects`](#streaming_disconnects)                               | `warn`                        | `health`   | Streaming sessions are dropping                 |
| [`check_unavailable`](#check_unavailable)                                       | `info`                        | Any        | A check could not run                           |

Below, each check shows its level and scope. `…` stands for `/environments/{dev_id}`.

## Account

### `environment_inactive` <a href="#environment_inactive" id="environment_inactive"></a>

`error` · `account:read`

The environment is turned off: it receives no data, every call with its API key is refused, and nothing from your users' connections is delivered.

Fix: check its state with `terra environments retrieve --env <DEV_ID>` (`GET …`). Pay any open invoice ([`subscription_past_due`](#subscription_past_due)), otherwise ask Terra support to turn the environment back on.

### `testing_environment_live_users` <a href="#testing_environment_live_users" id="testing_environment_live_users"></a>

`warn` · `connections:read`

A testing or staging environment has 50 or more connected users and no development cap, so nothing stops real users building up in it. These environments are normally capped at 50 users, so this covers those with no limit or a limit above 50. Capped ones are covered by [`user_limit_near`](#user_limit_near) and [`user_limit_reached`](#user_limit_reached).

Fix: find your production environment with `terra environments list` (`GET /environments`), point your app's production build at its dev ID and API key, and ask real users to connect again there.

[Accept it](#accept-a-finding) while a load test or a large QA team needs many test users.

### `api_version_behind` <a href="#api_version_behind" id="api_version_behind"></a>

`info` · `account:read`

The environment pins the Vantage or Lab Reports API to a version older than the latest, and the lint names both. Nothing breaks while pinned, but you miss later changes, and the further behind you are, the more a later upgrade has to absorb.

Fix: read the changes between the two versions in the changelog, update your integration, then move the pin with `PUT …/vantage/api-version` or `PUT …/lab-reports/api-version`:

```bash
terra vantage api-version update --env <DEV_ID> --version <VERSION>
terra lab-reports api-version update --env <DEV_ID> --version <VERSION>
```

[Accept it](#accept-a-finding) when you pinned on purpose and are not ready to upgrade. A newer release gives a different finding.

### `deprecated_api_usage` <a href="#deprecated_api_usage" id="deprecated_api_usage"></a>

`info`, `warn` once the API has a removal date · `destinations:read`

The environment called a deprecated API in the last 24 hours. Its responses carry a `Deprecation: true` header, plus a `Sunset` header once a removal date is set. On that date every call fails, and the calls often come from code nobody has touched in a while, such as an old SDK in a released app. `metadata.observed.minutes_last_24h` is the number of minutes in which it was called. The finding clears 24 hours after the last call.

Fix: move to the replacement.

| Deprecated                        | Replacement                                                                                                   |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| `POST /v2/auth/generateAuthToken` | `POST /v2/auth/tokens`, which behaves the same. If an older Terra mobile SDK makes the call, upgrade the SDK. |
| `/v2/lab-reports` routes          | The matching `/v2/reports` route.                                                                             |

### `billing_unlinked` <a href="#billing_unlinked" id="billing_unlinked"></a>

`error` · `billing:read`

Billing is not set up. Users can connect, but Terra delivers no data for a billable environment until its account has billing set up.

Fix: subscribe to a plan with `terra billing subscriptions create` (`POST /billing/subscriptions`), or from the billing page in the dashboard.

### `subscription_past_due` <a href="#subscription_past_due" id="subscription_past_due"></a>

`error` · `billing:read`

A subscription payment is overdue. If it stays unpaid, the account is suspended and its environments are turned off ([`environment_inactive`](#environment_inactive)).

Fix: pay the open invoice with `terra billing invoices pay` (`POST /billing/invoices/pay`), or update the payment method in the dashboard so the next attempt succeeds.

### `team_invitation_expiring` <a href="#team_invitation_expiring" id="team_invitation_expiring"></a>

`info` · `team:read`

A team invitation expires within 3 days and has not been accepted. Once it expires, it has to be sent again before the person can join.

Fix: ask the person to accept it, or send it again with `terra team invitations create --email <EMAIL> --role developer` (`POST /team/invitations`).

### `api_key_missing` <a href="#api_key_missing" id="api_key_missing"></a>

`error` · `tokens:read`

The environment has no live API key or data token, so your backend cannot call the data API to authenticate users, read data or request historical data.

Fix: rotate the API key with `terra environments rotate-api-key --env <DEV_ID>` (`POST …/api-key/rotate`) and store the new key where your backend reads it, or mint a data token with `terra data-tokens create` (`POST …/tokens`).

### `data_token_expiring` <a href="#data_token_expiring" id="data_token_expiring"></a>

`warn` · `tokens:read`

Terra revokes a data token after 365 days without use, and this one will be revoked within 30 days, on `metadata.since`. Something that holds it but calls rarely, such as a yearly job, fails after that date.

Fix: find the token in `metadata.id` with `terra data-tokens list --env <DEV_ID>` (`GET …/tokens`). If something needs it, use it before the revocation date to keep it. Otherwise, revoke it with `terra data-tokens delete <token_id> --env <DEV_ID>` (`DELETE …/tokens/{token_id}`).

### `token_leaked` <a href="#token_leaked" id="token_leaked"></a>

`error` · `tokens:admin`

A token was found in public and revoked in the last 30 days. Terra revokes a token as soon as it is reported in a public repository, so it cannot be used against your account, but any script, CI job or service that used it now fails.

Fix: remove the token from where it was published, including the repository's history. Create a new token and switch everything that used the old one to it. Check `terra tokens list` (`GET /tokens`) for other tokens stored in the same place.

[Accept it](#accept-a-finding) once you have replaced the token, since the finding stays for 30 days after the revocation.

### `token_never_expires` <a href="#token_never_expires" id="token_never_expires"></a>

`warn` · `tokens:admin`

A personal access token has no expiry. It stays valid until revoked, so if it leaks it keeps working long after the person or job that held it is gone.

Fix: create a replacement that expires, switch whatever uses the old token to it, then revoke the old one with `terra tokens delete <token_id>` (`DELETE /tokens/{token_id}`).

### `token_unused` <a href="#token_unused" id="token_unused"></a>

`info` · `tokens:admin`

A personal access token has not been used for 90 days. It is still a way into your account.

Fix: if nothing uses it, revoke it with `terra tokens delete <token_id>` (`DELETE /tokens/{token_id}`).

[Accept it](#accept-a-finding) for a break-glass token kept for emergencies, which goes unused by design.

### `token_cap_near` <a href="#token_cap_near" id="token_cap_near"></a>

`info` · `tokens:admin`

The account holds 80% or more of its limit of live tokens. `metadata.observed` gives `live` and `limit`. Creating a token fails once the limit is reached.

Fix: list tokens with `terra tokens list` (`GET /tokens`) and revoke the ones nothing uses, starting with the [`token_unused`](#token_unused) findings.

## Unified API

### `sources_none_enabled` <a href="#sources_none_enabled" id="sources_none_enabled"></a>

`warn` · `providers:read`

The environment enables no sources. Users can connect only to enabled sources, so no user can connect.

Fix: enable the sources your users connect with, with `terra unified-api sources enable <provider> --env <DEV_ID>` (`PUT …/unified-api/sources/{provider}`).

### `source_credentials_missing` <a href="#source_credentials_missing" id="source_credentials_missing"></a>

`error` · `providers:read`

Terra has no app of its own for some sources, and this enabled source has none of yours either. Users see the source, but every attempt to connect to it fails.

Fix: register an app with the source, then add its credentials with `PUT …/unified-api/sources/{provider}/credentials`. If you do not plan to support the source, disable it instead.

```bash
terra unified-api sources credentials replace <provider> --env <DEV_ID> \
  --client-id <CLIENT_ID> --client-secret <CLIENT_SECRET>
```

### `source_deprecated` <a href="#source_deprecated" id="source_deprecated"></a>

`warn` · `providers:read`

The source is retired. Existing connections keep working but have to move eventually, and no new user can connect to it. The lint names its successor if it has one.

Fix: enable the successor, or another source, and ask your users to connect to it. Then disable the retired source with `terra unified-api sources disable <provider> --env <DEV_ID>` (`DELETE …/unified-api/sources/{provider}`).

### `source_not_enableable` <a href="#source_not_enableable" id="source_not_enableable"></a>

`warn` · `providers:read`

The environment enables a source that is in preview or for internal use. Users see the source but cannot connect to it.

Fix: disable it with `terra unified-api sources disable <provider> --env <DEV_ID>` (`DELETE …/unified-api/sources/{provider}`), or ask Terra for access.

### `app_subscription_failed` <a href="#app_subscription_failed" id="app_subscription_failed"></a>

`warn` · `providers:read`

Registering your app's webhooks with a source that pushes data failed in the last 24 hours. Users can connect, but the source does not tell Terra about new data, so it does not arrive.

Fix: check the app's credentials at the source, then save them again with `PATCH …/unified-api/sources/{provider}/credentials` to retry the registration:

```bash
terra unified-api sources credentials update <provider> --env <DEV_ID> \
  --client-id <CLIENT_ID> --client-secret <CLIENT_SECRET>
```

### `source_domain_unhealthy` <a href="#source_domain_unhealthy" id="source_domain_unhealthy"></a>

`error` when the certificate failed to issue, `warn` otherwise · `providers:read`

A source's custom auth domain is not serving: its certificate failed to issue, has been provisioning for over an hour, or expires within 14 days without renewing. Users can sign in through the domain only once its certificate is issued and in date, and it is issued only while the domain's DNS points at Terra. Without it, sign-in shows a certificate error.

Fix: check the domain's state with `GET …/unified-api/sources/{provider}/domain`, check that its DNS record points at Terra, then set the domain again with `PUT` on the same path to restart issuing:

```bash
terra unified-api sources domain retrieve <provider> --env <DEV_ID>
terra unified-api sources domain set <provider> --env <DEV_ID> --hostname <HOSTNAME>
```

### `user_limit_near` <a href="#user_limit_near" id="user_limit_near"></a>

`warn` · `connections:read`

90% or more of the environment's user limit is in use. `metadata.observed` gives `users` and `limit`. New connections are refused once the limit is reached ([`user_limit_reached`](#user_limit_reached)).

Fix: in a production environment, remove users you no longer need, or ask Terra to raise the limit. Testing and staging environments are capped for development, so connect real users to your production environment. `terra users list --env <DEV_ID>` (`GET …/users`) lists connected users.

### `user_limit_reached` <a href="#user_limit_reached" id="user_limit_reached"></a>

`error` · `connections:read`

The environment is at its user limit. New users cannot connect. Existing connections keep working.

Fix: in a production environment, remove users you no longer need, or ask Terra to raise the limit. Testing and staging environments are capped for development, so connect real users to your production environment. `terra users list --env <DEV_ID>` (`GET …/users`) lists connected users.

### `connections_insufficient_permissions` <a href="#connections_insufficient_permissions" id="connections_insufficient_permissions"></a>

`warn` · `events:read`

A source's connections reported an error in the last 24 hours, most often a permission the user did not grant. The connection works, but the lint's data type never arrives for these users. There is one lint per source and data type.

Fix: ask the affected users to connect again and grant every permission. `terra users list --env <DEV_ID> --provider <PROVIDER>` (`GET …/users`) lists the connections.

### `connections_access_revoked` <a href="#connections_access_revoked" id="connections_access_revoked"></a>

`warn` · `events:read`

Users of a source removed Terra's access in the last 24 hours. `metadata.observed` gives `events_last_24h`. A user who removed access sends no more data until they connect again.

Fix: ask the users to connect again if they still want to share their data.

Users leave on purpose, so a few of these are normal. [Accept the finding](#accept-a-finding) if that is all it shows.

### `connections_stale` <a href="#connections_stale" id="connections_stale"></a>

`warn` · `connections:read`

Terra reads some sources on a schedule, and connections to this source have not been read for over twice their polling interval, so their data is behind. `metadata.observed` gives `stale` and `active`.

Fix: check whether the source is having problems. If only some users are stale, they may need to connect again. `terra users list --env <DEV_ID> --provider <PROVIDER>` (`GET …/users`) lists the connections.

### `synthetic_users_running` <a href="#synthetic_users_running" id="synthetic_users_running"></a>

`info`, `warn` in a production environment · `connections:read`

Test users are running and sending made-up data to the environment's destinations. In production, it mixes with your real users' data.

Fix: stop the test users once you are done testing, with `POST …/synthetic-users/{user_id}/stop`:

```bash
terra synthetic-users list --env <DEV_ID> --status generating
terra synthetic-users stop <user_id> --env <DEV_ID>
```

[Accept it](#accept-a-finding) in a testing environment, where running test users is the point.

### `synthetic_capacity_reached` <a href="#synthetic_capacity_reached" id="synthetic_capacity_reached"></a>

`warn` · `connections:read`

The environment has reached its limit of running test users. No more can start until a running one stops.

Fix: stop a running test user with `terra synthetic-users stop <user_id> --env <DEV_ID>` (`POST …/synthetic-users/{user_id}/stop`).

### `destinations_none_active` <a href="#destinations_none_active" id="destinations_none_active"></a>

`info` · `destinations:read`

The environment has no active destination. Data from connected users is delivered only to active destinations.

Fix: create a destination with `POST …/unified-api/destinations`, or turn one on with `terra unified-api destinations update <destination_id> --env <DEV_ID> --active` (`PATCH …/unified-api/destinations/{destination_id}`).

[Accept it](#accept-a-finding) if you only read data on request through the data API and never take deliveries.

### `destination_failing` <a href="#destination_failing" id="destination_failing"></a>

`error` · `destinations:read`

Events are not reaching the destination: every delivery that finished in the last hour, or in the last 24 hours, failed; deliveries have waited on automatic retries for an hour with none delivered; or retries ran out and nothing has been delivered since. `metadata.reason` and `detail` say why, and `metadata.since` is when the failures started. After 24 hours of failing, Terra turns the destination off and the finding becomes [`destination_auto_disabled`](#destination_auto_disabled). The finding clears once deliveries succeed again.

Fix: check the destination's recent counts with `GET …/unified-api/destinations/health`, fix the endpoint or the destination's config, then resend the failed events with `POST …/events/resend`:

```bash
terra unified-api destinations health --env <DEV_ID> --destination-id <DESTINATION_ID>
terra events list --env <DEV_ID> --destination <DESTINATION_ID> --status failed
terra events resend --env <DEV_ID> --event-id <EVENT_ID> --destination-id <DESTINATION_ID>
```

### `destination_degraded` <a href="#destination_degraded" id="destination_degraded"></a>

`warn` · `destinations:read`

Events reach the destination late, or not at all: some deliveries in the last hour failed; a delivery has waited on automatic retries for 15 minutes; a resend failed in the last hour and is still retrying; or most deliveries in the last hour needed a retry. `metadata.reason` names the cause, such as `rate_limited`. A destination often degrades before it fails.

Fix: follow [`destination_failing`](#destination_failing). If retries are the problem, return a `2xx` as soon as you receive an event and process it afterwards.

### `destination_auto_disabled` <a href="#destination_auto_disabled" id="destination_auto_disabled"></a>

`error` · `destinations:read`

Terra turned the destination off after deliveries to it failed for 24 hours. Nothing is delivered to it while it is off.

Fix: fix the endpoint or the destination's config, turn it back on with `terra unified-api destinations update <destination_id> --env <DEV_ID> --active` (`PATCH …/unified-api/destinations/{destination_id}`), then resend the events it missed as in [`destination_failing`](#destination_failing).

### `destination_insecure_http` <a href="#destination_insecure_http" id="destination_insecure_http"></a>

`warn`, `error` in a production environment · `destinations:read`

An active webhook's URL uses `http`, so your users' health data crosses the internet unencrypted, where it can be read and changed.

Fix: serve the endpoint over `https`, then update the destination's URL with `PATCH …/unified-api/destinations/{destination_id}`:

```bash
terra unified-api destinations update <destination_id> --env <DEV_ID> \
  --config '{"url": "https://<HOST>/<PATH>"}'
```

[Accept it](#accept-a-finding) for a local tunnel or a test receiver in a testing environment, never in production.

### `destination_url_credentials` <a href="#destination_url_credentials" id="destination_url_credentials"></a>

`error` · `destinations:read`

The destination's stored URL contains a user and password, which shows up anywhere the URL is logged or displayed.

Fix: update the destination with `terra unified-api destinations update <destination_id> --env <DEV_ID> --config '<CONFIG_JSON>'` (`PATCH …/unified-api/destinations/{destination_id}`) so its URL has no user or password, and set them as credentials in its config instead. Then change the password at the receiving end, since it has been visible wherever the URL was shown.

### `data_types_none_selected` <a href="#data_types_none_selected" id="data_types_none_selected"></a>

`error` · `dataconfig:read`

The environment selects no data types. Terra delivers only selected data types, so nothing is delivered.

Fix: list the data types you can select with `GET …/unified-api/data-types`, then select the ones you want with `PUT …/unified-api/scopes`. It replaces the whole selection, so send every data type you want.

```bash
terra unified-api data data-types list --env <DEV_ID>
terra unified-api data scopes replace --env <DEV_ID> --scopes ACTIVITY --scopes SLEEP
```

### `data_type_not_delivered` <a href="#data_type_not_delivered" id="data_type_not_delivered"></a>

`warn` · `dataconfig:read`

No active destination accepts a data type the environment selects. Terra processes it for your users, but it never reaches you.

Fix: add the data type to a destination's event types, or clear its event types so it accepts everything, with `terra unified-api destinations update <destination_id> --env <DEV_ID> --event-types <EVENT_TYPE>` (`PATCH …/unified-api/destinations/{destination_id}`). If you do not need the data type, deselect it instead.

### `destination_events_unfed` <a href="#destination_events_unfed" id="destination_events_unfed"></a>

`warn` · `dataconfig:read`

The destination accepts only event types the environment does not select, so it receives nothing.

Fix: select one of the destination's data types with `PUT …/unified-api/scopes`, or change its event types with `terra unified-api destinations update <destination_id> --event-types <EVENT_TYPE>` (`PATCH …/unified-api/destinations/{destination_id}`).

### `destination_events_unsupported` <a href="#destination_events_unsupported" id="destination_events_unsupported"></a>

`warn` · `destinations:read`

Terra held back events from the destination in the last 24 hours because its type, such as a database, cannot store them. Those events are never delivered there.

Fix: limit the destination's event types to data it can store with `PATCH …/unified-api/destinations/{destination_id}`, and send the rest to a webhook.

## Vantage

### `vantage_markets_not_chosen` <a href="#vantage_markets_not_chosen" id="vantage_markets_not_chosen"></a>

`warn` · `vantageconfig:read`

The environment has no Vantage markets. The catalogue lists, and orders accept, only products sold in the environment's markets, so the catalogue is empty and every order is refused.

Fix: choose the markets with `terra vantage markets update --env <DEV_ID> --markets <MARKET>` (`PUT …/vantage/markets`). Terra sets the markets for a production environment, so ask Terra to set those.

### `vantage_catalogue_empty` <a href="#vantage_catalogue_empty" id="vantage_catalogue_empty"></a>

`warn` · `vantageconfig:read`

The Vantage catalogue enables no products, so nothing can be ordered.

Fix: enable the products you offer with `terra vantage products enable <product_id> --env <DEV_ID>` (`PUT …/vantage/products/{product_id}`).

### `vantage_orders_delayed` <a href="#vantage_orders_delayed" id="vantage_orders_delayed"></a>

`warn` · `vantage:read`

Some orders are taking longer than expected to progress, so the patient waits longer for their kit or result. `metadata.observed.count` is how many.

Fix: list the orders with `terra vantage orders list --env <DEV_ID>` (`GET …/vantage/orders`) and check the delayed ones for a fulfilment or sample problem.

### `vantage_payment_failed_orders` <a href="#vantage_payment_failed_orders" id="vantage_payment_failed_orders"></a>

`error` · `vantage:read`

An order's payment failed in the last 7 days. The order is not fulfilled, so the patient is waiting for a kit that will not ship.

Fix: list the orders with `terra vantage orders list --env <DEV_ID>` (`GET …/vantage/orders`) and retry their payment.

### `vantage_kits_not_activated` <a href="#vantage_kits_not_activated" id="vantage_kits_not_activated"></a>

`warn` · `vantage:read`

A kit has waited more than 14 days for its sample. A kit is processed only once the patient activates it and sends the sample back.

Fix: remind the patients to activate their kit and send the sample back. `terra vantage results list --env <DEV_ID>` (`GET …/vantage/results`) lists them.

### `vantage_results_failed` <a href="#vantage_results_failed" id="vantage_results_failed"></a>

`error` · `vantage:read`

A result failed in the last 30 days: a rejected sample, a lab processing error or an escalation. The patient needs a new test. The finding clears 30 days after the last failure.

Fix: list the results with `terra vantage results list --env <DEV_ID>` (`GET …/vantage/results`), contact the patients and reorder their tests.

### `vantage_results_unacknowledged` <a href="#vantage_results_unacknowledged" id="vantage_results_unacknowledged"></a>

`warn` · `vantage:read`

Results are waiting to be acknowledged. Acknowledging a result is a compliance step, and the patient cannot see a result until you do it.

Fix: list the waiting results with `terra vantage results list --env <DEV_ID> --is-acknowledged=false` (`GET …/vantage/results`), review them and acknowledge each.

### `vantage_webhook_not_configured` <a href="#vantage_webhook_not_configured" id="vantage_webhook_not_configured"></a>

`warn`, `error` in a production environment · `vantageconfig:read`

The environment has no Vantage webhook URL. Order and result events are delivered only to the Vantage webhook, so none are delivered.

Fix: set the URL with `terra vantage webhook update --env <DEV_ID> --url <URL>` (`PUT …/vantage/webhook`).

### `vantage_webhook_failures` <a href="#vantage_webhook_failures" id="vantage_webhook_failures"></a>

`warn` · `vantage:read`

Some Vantage events failed delivery to the Vantage webhook in the last 24 hours, so your backend missed order and result updates.

Fix: list the failed events with `GET …/vantage/events` and read their attempts for the cause, fix the webhook endpoint, then resend each event with `POST …/vantage/events/resend`:

```bash
terra vantage events list --env <DEV_ID> --status failed
terra vantage events resend --env <DEV_ID> --event-id <EVENT_ID>
```

## Lab reports

### `lab_reports_delivery_failed` <a href="#lab_reports_delivery_failed" id="lab_reports_delivery_failed"></a>

`error` · `labreports:read`

A lab report from the last 30 days was processed but not delivered, so its results have not reached any destination.

Fix: find the failing destination with `terra unified-api destinations list --env <DEV_ID>` (`GET …/unified-api/destinations`) and check its findings, such as [`destination_failing`](#destination_failing). Fix it, then reprocess the reports.

### `lab_reports_processing_failed` <a href="#lab_reports_processing_failed" id="lab_reports_processing_failed"></a>

`warn` · `labreports:read`

A lab report from the last 30 days failed processing and produced no results.

Fix: check each report's error. Reprocess the reports that can be retried, and upload the rest again.

## Models

### `model_runs_failing` <a href="#model_runs_failing" id="model_runs_failing"></a>

`warn` · `models:read`

A model ran at least 10 times in the last 24 hours, and at least 10% of those runs failed, producing no result. There is one lint per model. `metadata.observed` gives `failed` and `total`.

Fix: list the failed runs and read their errors with `terra models runs list --env <DEV_ID> --model <MODEL> --status error` (`GET …/models/runs`).

## Planned workouts

### `planned_workouts_unsynced` <a href="#planned_workouts_unsynced" id="planned_workouts_unsynced"></a>

`warn` · `workouts:read`

A planned workout from the last 30 days passed its date without syncing to the device, so it was never available to the user.

Fix: check that the affected connections support planned workouts and are still connected. `terra users list --env <DEV_ID>` (`GET …/users`) lists the connections.

## Streaming

### `streaming_disconnects` <a href="#streaming_disconnects" id="streaming_disconnects"></a>

`warn` · `connections:read`

At least 10 streaming sessions closed in the last 24 hours, and at least 20% of them were closed by a client fault. Data sent while the client reconnects is lost. Closes caused by Terra, such as a redeploy or an expiring token, never count. `metadata.observed` gives the count for each code as `close_<code>`, and `metadata.reason` is the most frequent one.

| Code   | Client fault       |
| ------ | ------------------ |
| `4005` | Heartbeat missed   |
| `4000` | Identify timeout   |
| `4002` | Duplicate session  |
| `4003` | Already identified |
| `4004` | Invalid opcode     |
| `1003` | Malformed frame    |

Fix, by the most frequent code:

* `4005`: check the devices' network, keep sending heartbeats while the app is in the background, and reconnect with backoff.
* `4002`: close a user's old session before opening a new one.
* Any other code: check that the client follows the streaming protocol. It should identify within the timeout, send valid frames and opcodes, and identify each session only once.

## See also

* [Personal access tokens](https://docs.tryterra.co/reference/admin-api/personal-access-tokens.md)
* [Scopes](https://docs.tryterra.co/reference/admin-api/scopes.md)
* [Webhook signing secrets](https://docs.tryterra.co/reference/admin-api/webhook-signing-secrets.md)
