Skip to main content

​
Overview

LLM resources are only supported on macOS.
LLM resources work differently from other Formal resource types. Rather than routing traffic through a running in your infrastructure (a reverse proxy), LLM traffic is proxied directly by the Formal Desktop App acting as a local forward proxy. When you run formal connect for an LLM resource, the Desktop App starts a local HTTPS listener and forwards requests directly to the upstream LLM provider (e.g. api.openai.com, api.anthropic.com). No Connector is involved in the data path. 
your app  →  localhost (Desktop App forward proxy)  →  LLM provider API

​
Creating LLM Resources

You must create an LLM resource in Formal for each LLM API you want to intercept. The resource tells the Desktop App which upstream hostname and port to forward traffic to. 
resource "formal_resource" "anthropic" {
  name       = "anthropic"
  technology = "llm"
  hostname   = "api.anthropic.com"
  port       = 443
}

resource "formal_resource" "chatgpt" {
  name       = "chatgpt"
  technology = "llm"
  hostname   = "chatgpt.com"
  port       = 443
}

​
Connecting

formal connect --transparent --local-tls anthropic

​
Observability

​
Log source

Because LLM traffic does not pass through a Connector, logs appear with source: desktop rather than source: connector. You can filter for these logs in the Logs page with: 
source:desktop

​
What is captured

The Desktop App captures the full content of each LLM request and response, including:
  • Model and provider — the model name extracted from the request body
  • Messages — the full prompt and completion text
  • Tool calls — any tool/function calls the model makes, including their names, arguments, and results
  • Token usage — prompt tokens, completion tokens, and total tokens
  • Streaming responses — individual chunks are reassembled into the complete response before logging

​
Session replay

Each formal connect invocation for an LLM resource creates a session. Full session replay is available in the Sessions page. The session replay shows every request sent and response received during the connection, including all tool call exchanges.

​
Intercepting HTTPS Traffic

LLM APIs are served over HTTPS. Intercepting them requires both --local-tls and --transparent — the Desktop App uses the transparent proxy to intercept outbound traffic and local TLS to terminate and re-encrypt it.
Both flags are required and only supported on macOS. The Formal network extension must be enabled (formal transparent-proxy enable).

​
Trusting the Formal CA

The Desktop App terminates TLS using a CA it manages at ~/.formal/ca/formal-org-ca.cer. Clients must trust this CA. On login, the Desktop App automatically configures Claude Code to trust the Formal CA by injecting NODE_EXTRA_CA_CERTS into ~/.claude/settings.json (if the file exists): 
{
  "env": {
    "NODE_EXTRA_CA_CERTS": "/Users/you/.formal/ca/formal-org-ca.cer"
  }
}
For other clients, trust the CA manually:
  1. Open ~/.formal/ca/formal-org-ca.cer (double-click or open from Finder on macOS).
  2. In Keychain Access, select the Formal certificate.
  3. Open Trust, expand Secure Sockets Layer (SSL), and choose Always Trust.
Or set the environment variable for any Node.js-based tool: 
export NODE_EXTRA_CA_CERTS="$HOME/.formal/ca/formal-org-ca.cer"

​
Agent-Scoped Interception

Pass --only-agents to restrict interception to recognized AI coding agents (Claude Code, Codex, Cursor, OpenCode). All other processes that use these APIs will not have TLS interception nor LLM observability. 
formal connect --local-tls --only-agents anthropic