> ## Documentation Index
> Fetch the complete documentation index at: https://docs.formal.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Satellites

> Optional containers for advanced policy enforcement, data discovery, and classification

## What are Satellites?

Formal Satellites are optional specialized containers you deploy alongside Connectors to enable advanced capabilities like PII detection, schema discovery, and custom policy data loading.

Satellites extend Formal's core functionality without adding complexity to the Connector itself. They're deployed in your infrastructure and communicate with Connectors and the Control Plane.

## Satellite Types

<CardGroup cols={3}>
  <Card title="AI" icon="magnifying-glass">
    Detects PII and PHI and sensitive data in real-time for automatic redaction and
    classification.
    Enables real-time threat detection and mitigation for SSH and Kubernetes sessions
  </Card>

  <Card title="Data Discovery" icon="map">
    Catalogs database schemas, tables, and columns across your data
    infrastructure
  </Card>

  <Card title="Policy Data Loader" icon="code">
    Loads external data into policies using custom code in Python or Node.js
  </Card>
</CardGroup>

## AI Satellite

Identifies Personally Identifiable Information (PII) and Protected Health Information (PHI) in database responses, enabling automatic data masking and classification policies. Also enables real-time threat detection and mitigation for SSH and Kubernetes sessions.

### Features

* **Real-time PII/PHI detection** on query responses
* **Threat detection and mitigation** for SSH and Kubernetes sessions
* **Automatic labeling** of columns and fields with PII/PHI types
* **Integration with policies** for conditional masking
* **GPU acceleration** for high-throughput processing (optional, recommended for production)

### Configuration

**Required Environment Variables:**

* `FORMAL_CONTROL_PLANE_API_KEY`: Satellite authentication token

**GPU Support:**

The AI Satellite can leverage NVIDIA GPUs for improved performance. Use `--gpus all` to enable GPU acceleration:

```bash theme={null}
# Run with GPU support (necessary to parallelize across GPUs)
export FORMAL_AI_SATELLITE_IMAGE="654654333078.dkr.ecr.us-east-1.amazonaws.com/formalco-prod-ai-satellite:latest"
# Or use GCP Artifact Registry:
# export FORMAL_AI_SATELLITE_IMAGE="us-docker.pkg.dev/formal-public-assets/formalco-prod-ai-satellite/formalco-prod-ai-satellite:latest"

docker run -d \
  --gpus all \
  -e FORMAL_CONTROL_PLANE_API_KEY=<your-api-key> \
  "$FORMAL_AI_SATELLITE_IMAGE"
```

The satellite automatically detects and configures available GPUs for optimal performance.

## Data Discovery Satellite

Automatically discovers and catalogs your database schemas, tables, columns, and relationships.

### Features

* **Scheduled schema discovery** across all resources
* **AI-powered column classification** when linked to an AI Satellite
* **Schema change tracking** with deletion policies

### AI Satellite Integration

The Data Discovery Satellite can leverage an AI Satellite for intelligent column classification during schema discovery. When linked, discovered columns are automatically classified with PII/PHI labels (e.g., `email_address`, `phone_number`, `ssn`).

**To enable AI classification:**

1. Deploy both a Data Discovery Satellite and an AI Satellite
2. Link the Data Discovery Satellite to the AI Satellite via the Formal console or Terraform

<Tabs>
  <Tab title="Terraform">
    ```hcl theme={null}
    resource "formal_satellite" "data_discovery" {
      name           = "data-discovery-satellite"
      satellite_type = "data_discovery"
    }

    resource "formal_satellite" "ai" {
      name           = "ai-satellite"
      satellite_type = "ai"
    }

    resource "formal_satellite_link" "discovery_to_ai" {
      source_satellite_id = formal_satellite.data_discovery.id
      target_satellite_id = formal_satellite.ai.id
    }
    ```
  </Tab>

  <Tab title="Console">
    1. Navigate to [Satellites](https://app.formal.ai/satellites)
    2. Select your Data Discovery Satellite
    3. Under "Linked Satellites", click "Link Satellite"
    4. Select your AI Satellite as the target
  </Tab>
</Tabs>

<Note>
  Without an AI Satellite link, schema discovery still works but columns will not be automatically classified. You can add classification labels manually or link an AI Satellite later.
</Note>

### Configuration

Environment variables:

* `FORMAL_CONTROL_PLANE_API_KEY`: Satellite authentication token

### Schema Discovery Jobs

Configure discovery schedules per resource:

* **Frequency**: None, every 6/12/18/24 hours, or custom cron
* **Deletion policy**: Mark for deletion or auto-delete removed schemas
* **Native user**: Which credentials to use for discovery

## Policy Data Loader Satellite

Enables custom code to load data from external sources into your policies, extending policy evaluation with dynamic business logic.

### Features

* **Custom code execution** in Python 3.11 or Node.js 18
* **Scheduled runs** with cron expressions
* **External API calls** to fetch data
* **JSON output** accessible in policies via `data` object

### Supported Runtimes

| Runtime     | Identifier   | Available Libraries |
| ----------- | ------------ | ------------------- |
| Python 3.11 | `python3.11` | `requests`, `httpx` |
| Node.js 18  | `nodejs18.x` | `lodash`, `axios`   |

### Example: Load Zendesk Tickets for Contextual Data

This example fetches open Zendesk tickets and enriches them with user information for use in policies:

```python theme={null}
import asyncio
import json
import os

import httpx

ZENDESK_SUBDOMAIN = os.environ["ZENDESK_SUBDOMAIN"]
ZENDESK_EMAIL = os.environ["ZENDESK_EMAIL"]
ZENDESK_API_TOKEN = os.environ["ZENDESK_API_TOKEN"]

auth = httpx.BasicAuth(ZENDESK_EMAIL, ZENDESK_API_TOKEN)
base_url = f"https://{ZENDESK_SUBDOMAIN}.zendesk.com"

users_cache: dict[str, dict | None] = {}

async def get_user(user_id) -> dict | None:
    if user := users_cache.get(user_id):
        return user

    async with httpx.AsyncClient(base_url=base_url, auth=auth) as client:
        response = await client.get(f"/api/v2/users/{user_id}.json")

    response.raise_for_status()
    user = response.json()["user"]
    users_cache[user_id] = user
    return user

async def get_tickets() -> list[dict]:
    async with httpx.AsyncClient(base_url=base_url, auth=auth) as client:
        params = {"query": "status:new status:open status:pending"}
        response = await client.get("/api/v2/search.json", params=params)

    response.raise_for_status()
    return response.json()["results"]

async def main():
    tickets = await get_tickets()

    for ticket in tickets:
        if requester_id := ticket.get("requester_id"):
            ticket["requester"] = await get_user(requester_id)
        if submitter_id := ticket.get("submitter_id"):
            ticket["submitter"] = await get_user(submitter_id)
        if assignee_id := ticket.get("assignee_id"):
            ticket["assignee"] = await get_user(assignee_id)

        ticket["url"] = f"{base_url}/agent/tickets/{ticket['id']}"

    print(json.dumps(tickets))

asyncio.run(main())
```

### Using in Policies

The Policy Data Loader outputs JSON data that becomes available in policies via the `data` object. Here's how to use the Zendesk tickets data in a policy:

```rego theme={null}
package formal.v2

import future.keywords.if

default response := {"action": "filter", "reason": "No tickets are open for the given row"}

response := {"action": "allow", "contextual_data": filteredTickets, "reason": 
  "At least one ticket is open for the given row"} if {
    col := input.row[_]
    col["path"] == "postgres.public.pii.email"
    filteredTickets := [obj | obj := data.zendesk_tickets[_]
    obj.requester.email == col.value]
    count(filteredTickets) > 0
}
```

This policy:

* Filters database rows by checking if there are open Zendesk tickets for the email address
* Allows access with contextual ticket data when tickets exist
* Blocks access when no tickets are found for the email

### Schedule Format

Policy Data Loaders use second-based cron expressions:

| Expression       | Description      |
| ---------------- | ---------------- |
| `* * * * * *`    | Every second     |
| `*/30 * * * * *` | Every 30 seconds |
| `0 * * * * *`    | Every minute     |
| `0 */5 * * * *`  | Every 5 minutes  |
| `0 30 8 * * *`   | Daily at 8:30 AM |

Format: `second minute hour day month year`

### Configuration

Environment variables:

* `FORMAL_CONTROL_PLANE_API_KEY`: Satellite authentication token
* **Custom variables**: Available to your code

<Note>
  The Satellite passes all its environment variables to worker processes, so you
  can use environment variables in your code (e.g., API keys, endpoints).
</Note>

## Deployment

Satellites are Docker containers deployed in your infrastructure, similar to Connectors.

### Container Image Registries

You can pull satellite images from either AWS ECR or GCP Artifact Registry:

| Satellite Type               | AWS ECR                                                                                        | GCP Artifact Registry                                                                                                                |
| ---------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| AI Satellite                 | `654654333078.dkr.ecr.<region>.amazonaws.com/formalco-prod-ai-satellite:<tag>`                 | `us-docker.pkg.dev/formal-public-assets/formalco-prod-ai-satellite/formalco-prod-ai-satellite:<tag>`                                 |
| Data Discovery Satellite     | `654654333078.dkr.ecr.<region>.amazonaws.com/formalco-prod-data-discovery-satellite:<tag>`     | `us-docker.pkg.dev/formal-public-assets/formalco-prod-data-discovery-satellite/formalco-prod-data-discovery-satellite:<tag>`         |
| Policy Data Loader Satellite | `654654333078.dkr.ecr.<region>.amazonaws.com/formalco-prod-policy-data-loader-satellite:<tag>` | `us-docker.pkg.dev/formal-public-assets/formalco-prod-policy-data-loader-satellite/formalco-prod-policy-data-loader-satellite:<tag>` |

### Prerequisites

1. Create the Satellite in the Formal console
2. Copy the API token
3. Deploy the container with appropriate environment variables

### Recommended Deployment

<Tabs>
  <Tab title="AWS ECS Fargate">
    See the [AWS Satellite deployment example](https://github.com/formalco/terraform-provider-formal/tree/main/examples/deployments/aws/data-discovery-satellite) for Terraform configuration.
  </Tab>

  <Tab title="Kubernetes">
    ```yaml theme={null}
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: formal-ai-satellite
    spec:
      replicas: 1
      template:
        spec:
          containers:
          - name: satellite
            image: 654654333078.dkr.ecr.us-east-1.amazonaws.com/formalco-prod-ai-satellite:latest
            # Or use GCP Artifact Registry:
            # image: us-docker.pkg.dev/formal-public-assets/formalco-prod-ai-satellite/formalco-prod-ai-satellite:latest
            env:
            - name: FORMAL_CONTROL_PLANE_API_KEY
              valueFrom:
                secretKeyRef:
                  name: formal-satellite
                  key: api-key
    ```
  </Tab>

  <Tab title="Docker">
    ```bash theme={null}
    export FORMAL_SATELLITE_IMAGE="654654333078.dkr.ecr.us-east-1.amazonaws.com/formalco-prod-ai-satellite:latest"
    # Or use GCP Artifact Registry:
    # export FORMAL_SATELLITE_IMAGE="us-docker.pkg.dev/formal-public-assets/formalco-prod-ai-satellite/formalco-prod-ai-satellite:latest"

    docker run -d \
      --name formal-ai-satellite \
      -e FORMAL_CONTROL_PLANE_API_KEY=satellite_abc123 \
      "$FORMAL_SATELLITE_IMAGE"
    ```
  </Tab>
</Tabs>

<Tip>
  By default, the satellite communicates with the Connectors using a TLS certificate issued by the Control Plane.
</Tip>

## Spaces and Satellites

Like Connectors and Resources, Satellites can be assigned to [Spaces](/docs/guides/core-concepts/spaces):

* **Satellite with a Space**: Only communicates with Connectors and Resources in the same Space
* **Satellite without a Space**: Can communicate with any Connector or Resource

<Warning>
  Changing a Satellite's Space requires restarting the Satellite container.
</Warning>

## Managing Satellites

### Creating a Satellite

<Steps>
  <Step title="Navigate to Satellites">
    Go to [Satellites](https://app.formal.ai/satellites) in the console
  </Step>

  <Step title="Select type">
    Choose AI Satellite, Data Discovery, or Policy Data Loader
  </Step>

  <Step title="Configure settings">
    * **Name**: Friendly identifier - **Space**: (Optional) Logical grouping
  </Step>

  <Step title="Copy API token">Save the token for deployment</Step>

  <Step title="Deploy container">
    Use the token in your deployment (ECS, Kubernetes, Docker)
  </Step>
</Steps>

### Policy Data Loader Status

* **Draft**: Not running; code is being edited
* **Active**: Running and loading data on schedule

Activate after testing your code to make data available to policies.

## Next Steps

<CardGroup cols={2}>
  <Card title="Deploy a Satellite" icon="rocket" href="#deployment">
    Run Satellites in your infrastructure
  </Card>

  <Card title="Use in Policies" icon="shield-check" href="/docs/guides/policies/introduction">
    Reference Satellite data in policy rules
  </Card>

  <Card title="Configure Spaces" icon="layer-group" href="/docs/guides/core-concepts/spaces">
    Segment Satellite access
  </Card>

  <Card title="View Logs" icon="file-lines" href="/docs/guides/observability/logs">
    Monitor Satellite activity in logs
  </Card>
</CardGroup>