Log Schemas - Formal Documentation

Overview

Log schemas modify matching Connector and Endpoint logs before Formal stores or exports them. Use schemas to remove, encrypt, truncate, or sanitize sensitive fields with field-level precision. Configure them with the formal_log_schema Terraform resource. Each schema contains:

A CEL scope that selects logs.
One or more protobuf field paths.
One or more actions for each path.
An optional asymmetric encryption key.

When a log matches a schema, it overrides Log Configuration content settings for that log. Policy evaluation input retention settings still apply.

Configure a Log Schema

This example removes SQL literals from queries for one resource:

resource "formal_log_schema" "strip_customer_queries" {
  name = "strip-customer-queries"

  scope_cel = "log.source == \"connector\" && log.resource.id == \"${formal_resource.customers.id}\""

  path {
    name      = "log.request.query"
    strip_sql = true
  }
}

Run terraform apply, then query the resource through Formal. Verify: Open Logs and inspect the resulting request log. The query has values_stripped: true and does not contain its original literals.

Scope Fields

The scope_cel expression must return a boolean. It can use these fields:

Field	Values
`log.source`	`connector` or `endpoint`
`log.event_type`	The log event type, such as `request`
`log.resource`	`id`, `name`, and `technology`
`log.connector`	`id` and `name`
`log.space`	`id` and `name`
`log.user`	`id`, `username`, `type`, and `groups`
`log.end_user`	`id`, `username`, `type`, and `groups`

The scope cannot inspect payload fields, timestamps, or the complete log. Use paths to select payload fields after the scope matches.

Endpoint schemas use log.source == "endpoint". Stored Endpoint logs use source:desktop when you search them.

Schema Actions

Action	Behavior
`drop`	Removes the selected field. Selecting `log` drops the entire log.
`encrypt`	Encrypts one string field with the schema’s asymmetric key.
`truncate`	Limits one string field to a maximum UTF-8 byte length.
`strip_sql`	Removes literal values and parameters from a datastore query.
`encrypt_sql`	Encrypts query text, normalized text, and parameters.

encrypt and encrypt_sql require encryption_key_id. The key must use an asymmetric algorithm. See Encryption Keys. You can combine compatible actions on one path. drop takes precedence over every other action. Truncation runs before encryption. SQL stripping runs before SQL encryption.

resource "formal_log_schema" "encrypt_http_body" {
  name              = "encrypt-http-body"
  encryption_key_id = formal_encryption_key.logs_key.id
  scope_cel         = "log.source == \"endpoint\" && log.event_type == \"request\""

  path {
    name    = "log.request.http.body.received"
    encrypt = true
  }
}

Supported Paths

Every path starts with log. The following path families support schema actions:

Path family	Supported actions
`log`	`drop`
`log.request.query`	`drop`, `strip_sql`, `encrypt_sql`
`log.request.http.*`	`drop`, `encrypt`, `truncate`
`log.request.s3.*`	`drop`, `encrypt`, `truncate`
`log.request.grpc.*`	`drop`, `encrypt`, `truncate`
`log.request.dynamodb.*`	`drop`, `encrypt`, `truncate`
`log.response.datastore.*`	`drop`, `encrypt`, `truncate`
`log.response.http.*`	`drop`, `encrypt`, `truncate`
`log.response.s3.*`	`drop`, `encrypt`, `truncate`
`log.response.mcp.*`	`drop`, `encrypt`, `truncate`
`log.stream.message.*`	`drop`, `encrypt`, `truncate`
`log.llm.*`	`drop`, `encrypt`, `truncate`

encrypt and truncate only support singular string fields. drop can select a supported parent field, string, list, or map. Paths use protobuf field names, not search aliases. For example, use log.request.http.body.received, not request_body.

Path Restrictions

A schema cannot contain both a parent path and its child. For example, you cannot combine log.request.http.body with log.request.http.body.received. Do not configure overlapping paths across separate schemas. When any schema matches, its path actions replace legacy log content settings for that log. Existing policy evaluation input retention settings still apply. If no schema matches, Formal applies the legacy log configuration.

Failure Behavior

Formal clears a selected sensitive field if its encryption action fails. It then continues applying actions to other paths. A failed SQL encryption clears the complete selected query object. A scope evaluation failure skips only that schema. Other matching schemas continue to apply.

Next Steps

Configure Encryption Keys

Log Configuration

Configure default logging and retention behavior

​Overview

​Configure a Log Schema

​Scope Fields

​Schema Actions

​Supported Paths

​Path Restrictions

​Failure Behavior

​Next Steps

Configure Encryption Keys

Log Configuration

Overview

Configure a Log Schema

Scope Fields

Schema Actions

Supported Paths

Path Restrictions

Failure Behavior

Next Steps