> ## Documentation Index > Fetch the complete documentation index at: https://docs.formal.ai/llms.txt > Use this file to discover all available pages before exploring further. # Braintrust > Query Braintrust logs and experiments with SQL and summarize experiment results. Developer Tools Publisher: Braintrust Route Braintrust MCP traffic through Formal to control and log what AI agents do in Braintrust: * **Block** tools you don't want agents to call, such as `sql_query`. * **Mask** sensitive data in tool results, such as emails and phone numbers in `sql_query` responses. * **Keep credentials at the Connector**, so users and agents never see Braintrust tokens. * **Log** every tool call with the user, the MCP client and its parameters. Cloud agents Laptop / VM Endpoint [https://api.braintrust.dev/mcp](https://api.braintrust.dev/mcp) Connector ![](https://docs.formal.ai/static/brands/mcp/braintrust.png) ## 0. Prepare your Connector The Connector needs an MCP listener and a hostname with wildcard DNS and TLS. Set them up once by following [Deploy the Connector](https://docs.formal.ai/docs/guides/core-concepts/connectors/mcp-gateway.md#deploy-the-connector). Every MCP server reuses them. ## 1. Add Braintrust MCP server to your Resources > [tab] UI > > 1. Open the [Admin Catalog](https://app.formal.ai/catalog) and click **Add MCP**. > 2. Select the **Braintrust** template. > 3. Enter a DNS-label **Name**, such as `braintrust`. > 4. Choose the **Space** containing your Connector, such as `default`. > 5. Configure upstream authentication as described below, then save. > > If no Connector is in the selected Space, the **Configuration** tab shows **MCP clients can't connect directly yet**. > The reason reads: `No Connector is in this server's Space.` > > Adding Braintrust to the Admin Catalog > [tab] Terraform > > Set `space_id` to the ID of the Space containing your Connector. > > ```hcl theme={"languages":{"custom":["/languages/cel.json","/languages/rego.json"]}} > resource "formal_resource" "braintrust" { > name = "braintrust" > technology = "mcp" > hostname = "api.braintrust.dev" > space_id = "" > native_users_v3_enabled = true > port = 443 > tags = { > "mcp-endpoint" = "/mcp" > } > } > > resource "formal_resource_oauth_settings" "braintrust" { > resource_id = formal_resource.braintrust.id > } > ``` > [tab] Pulumi > > Set `spaceId` to the ID of the Space containing your Connector. > > ```ts theme={"languages":{"custom":["/languages/cel.json","/languages/rego.json"]}} > import * as formal from "@formalco/pulumi"; > > const resource = new formal.Resource("mcp", { > name: "braintrust", > technology: "mcp", > hostname: "api.braintrust.dev", > port: 443, > spaceId: "", > tags: { "mcp-endpoint": "/mcp" }, > nativeUsersV3Enabled: true, > }); > > new formal.ResourceOauthSettings("mcpOAuth", { > resourceId: resource.id, > }); > ``` > [tab] API > > Create a Resource with the MCP endpoint and its Space ID. > > ```bash theme={"languages":{"custom":["/languages/cel.json","/languages/rego.json"]}} > export FORMAL_API_KEY="" > RESOURCE_RESPONSE=$(curl --fail --silent --show-error \ > --request POST \ > https://api.formal.ai/core.v1.ResourceService/CreateResource \ > --header "x-api-key: ${FORMAL_API_KEY}" \ > --header "Content-Type: application/json" \ > --data-binary @- < { > "name": "braintrust", > "hostname": "api.braintrust.dev", > "port": 443, > "technology": "mcp", > "spaceId": "", > "tags": [ > { > "key": "mcp-endpoint", > "value": "/mcp" > } > ] > } > JSON > ) > RESOURCE_ID=$(printf '%s' "${RESOURCE_RESPONSE}" | jq -r '.resource.id') > curl --fail --silent --show-error \ > --request POST \ > https://api.formal.ai/core.v1.ResourceService/CreateResourceOAuthSettings \ > --header "x-api-key: ${FORMAL_API_KEY}" \ > --header "Content-Type: application/json" \ > --data-binary @- < { > "resourceId": "${RESOURCE_ID}" > } > JSON > ``` ## 2. Setup Authentication > [tab] Linked Accounts (Recommended) > > Each user links their own Braintrust account once in the Employee Catalog, and the Connector sends that user's token to Braintrust. > > ![Linked Accounts selected under How does Formal authenticate to this resource?](https://mintcdn.com/formal/vzPMpIcsF4BYmdx8/assets/images/mcp-servers/auth-linked-accounts.png?fit=max\&auto=format\&n=vzPMpIcsF4BYmdx8\&q=85\&s=dc448db2a373faa6e1bf49fd5ffaf7c4) > > Under **OAuth app**, keep **Formal's App**. > > #### Formal's App (default) > > Formal has a registered OAuth app with Braintrust, so you register nothing. > Users see Formal on Braintrust's consent screen. > [tab] Shared Credentials > > An administrator stores one Braintrust token in Formal, and the Connector sends it on every user's requests without showing it to them. > > ![Shared Credentials selected under How does Formal authenticate to this resource?](https://mintcdn.com/formal/R-3CwVgs8IU6liji/assets/images/mcp-servers/auth-shared-credentials.png?fit=max\&auto=format\&n=R-3CwVgs8IU6liji\&q=85\&s=91032e202518088292ee1cf39451b9ef) > > Braintrust's MCP endpoint accepts an [API key](https://www.braintrust.dev/docs/integrations/developer-tools/mcp) as a Bearer token. > Create one at [www.braintrust.dev/app/\~/configuration/api-keys](https://www.braintrust.dev/app/~/configuration/api-keys). > Use a key owned by an account limited to the projects and resources the MCP client needs. > > 1. Open this server from the [Admin Catalog](https://app.formal.ai/catalog). > 2. Select **Configuration**, then **Authentication**. > 3. Choose **Shared Credentials**, then select **Add credential**. > 4. Enter the credential, make it the default, and select **Save authentication**. > > See [Native Users](https://docs.formal.ai/docs/guides/core-concepts/resources/native-users.md). > > For HTTP Bearer authentication, enter the raw token. The Connector adds the `Bearer ` prefix. > > In the Terraform tab, set `native_users_v3_enabled = true` on the Resource and `upstream_oauth_mode = "none"` on its OAuth settings. > > ```hcl theme={"languages":{"custom":["/languages/cel.json","/languages/rego.json"]}} > variable "mcp_shared_credential" { > type = string > sensitive = true > } > > resource "formal_native_user_v3" "mcp_shared_credential" { > resource_id = formal_resource.braintrust.id > label = "shared-credential" > > http_bearer { > header = "Authorization" > token { > literal_wo = var.mcp_shared_credential > literal_wo_version = 1 > } > } > } > > resource "formal_resource_native_user_selection" "mcp_shared_credential" { > resource_id = formal_resource.braintrust.id > cel = "\"${formal_native_user_v3.mcp_shared_credential.id}\"" > } > ``` > [tab] Pass Through > > Each user sends their own Braintrust sign-in or token from their MCP client, and Formal forwards it unchanged without storing it. > > ![Pass Through selected under How does Formal authenticate to this resource?](https://mintcdn.com/formal/R-3CwVgs8IU6liji/assets/images/mcp-servers/auth-pass-through.png?fit=max\&auto=format\&n=R-3CwVgs8IU6liji\&q=85\&s=c14ae61de2dba8b567626cba03aae2cf) > > Nothing in the request identifies the Formal user. Formal attributes these requests only when they come through the [Formal Endpoint](https://docs.formal.ai/docs/guides/core-concepts/connectors/mcp-gateway.md#identity-headers). Pass-through servers have no Connector address, so cloud agents can't use them. > > For OAuth, each MCP client signs in to Braintrust itself. ## 3. Route Endpoint traffic Auth is managed at the Connector level when you use Linked Accounts or Shared Credentials, so the Endpoint needs to forward the MCP traffic to it. Laptops and VMs with the Formal Endpoint keep using `https://api.braintrust.dev/mcp`, and a network rule sends that traffic to the Connector. > [tab] UI > > 1. Go to [Network Rules](https://app.formal.ai/network-rules) and create a rule named `braintrust-mcp`. > 2. Paste this expression, set the rule to **Active**, then save: > > ```javascript theme={"languages":{"custom":["/languages/cel.json","/languages/rego.json"]}} > { > "condition": { > "pre_tcp": process.is_agent, > // Every MCP server added to Formal: process.is_agent && resource.technology == "mcp" > "pre_tls": process.is_agent && hostname == "api.braintrust.dev", > "post_tls": true > }, > "outputs": { > "forward_to_connector": true > } > } > ``` > [tab] Terraform > > ```hcl theme={"languages":{"custom":["/languages/cel.json","/languages/rego.json"]}} > resource "formal_network_rule" "braintrust_mcp" { > name = "braintrust-mcp" > description = "Forward Braintrust MCP traffic from agents to the Connector" > status = "active" > > # Every MCP server added to Formal: set pre_tls to process.is_agent && resource.technology == "mcp" > cel_expression = <<-EOT > { > "condition": { > "pre_tcp": process.is_agent, > "pre_tls": process.is_agent && hostname == "api.braintrust.dev", > "post_tls": true > }, > "outputs": { > "forward_to_connector": true > } > } > EOT > } > ``` > [tab] Pulumi > > ```ts theme={"languages":{"custom":["/languages/cel.json","/languages/rego.json"]}} > import * as formal from "@formalco/pulumi"; > > new formal.NetworkRule("braintrustMcp", { > name: "braintrust-mcp", > description: "Forward Braintrust MCP traffic from agents to the Connector", > status: "active", > // Every MCP server added to Formal: set pre_tls to process.is_agent && resource.technology == "mcp" > celExpression: ` > { > "condition": { > "pre_tcp": process.is_agent, > "pre_tls": process.is_agent && hostname == "api.braintrust.dev", > "post_tls": true > }, > "outputs": { > "forward_to_connector": true > } > } > `, > }); > ``` > [tab] API > > ```bash theme={"languages":{"custom":["/languages/cel.json","/languages/rego.json"]}} > export FORMAL_API_KEY="" > # Every MCP server added to Formal: set pre_tls to process.is_agent && resource.technology == "mcp" > CEL_EXPRESSION=$(cat <<'CEL' > { > "condition": { > "pre_tcp": process.is_agent, > "pre_tls": process.is_agent && hostname == "api.braintrust.dev", > "post_tls": true > }, > "outputs": { > "forward_to_connector": true > } > } > CEL > ) > jq -n --arg cel "${CEL_EXPRESSION}" '{ > name: "braintrust-mcp", > description: "Forward Braintrust MCP traffic from agents to the Connector", > status: "active", > celExpression: $cel > }' | curl --fail --silent --show-error \ > --request POST \ > https://api.formal.ai/core.v1.DesktopService/CreateDesktopRoutingRule \ > --header "x-api-key: ${FORMAL_API_KEY}" \ > --header "Content-Type: application/json" \ > --data-binary @- > ``` The rule matches [known agents](https://docs.formal.ai/docs/guides/network-rules.md#known-agent-identities) only. See [Identity headers](https://docs.formal.ai/docs/guides/core-concepts/connectors/mcp-gateway.md#identity-headers). Cloud agents such as Claude, ChatGPT and Cursor Cloud Agents don't run the Endpoint. Point them at this server's Connector address, `https://braintrust./mcp`. Find it in the [Admin Catalog](https://app.formal.ai/catalog): open Braintrust, then copy **Connector address** on the **Configuration** tab. ## 4. Create policies ### Tools The Admin Catalog lists these tools for Braintrust: | Tool | Description | | - | - | | `sql_query` | Query logs, experiments, and datasets with SQL. | | `infer_schema` | Infer the schema of a project's logs or an experiment. | | `summarize_experiment` | Summarize an experiment's scores and results. | ### Policies examples > [tab] Mask a tool's output > > This policy replaces email addresses in results returned by `sql_query` and `infer_schema`. > > ```rego theme={"languages":{"custom":["/languages/cel.json","/languages/rego.json"]}} > package formal.v2 > > import future.keywords.if > import future.keywords.in > > email_pattern := `[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}` > > response := { > "action": "rewrite", > "body": regex.replace(input.http.body, email_pattern, "[REDACTED-EMAIL]"), > "reason": "Mask emails in Braintrust results", > } if { > input.resource.name == "braintrust" > input.mcp.tool_name in {"sql_query", "infer_schema"} > regex.match(email_pattern, input.http.body) > } > ``` See [MCP resources](https://docs.formal.ai/docs/guides/policies/evaluation.md#mcp-resources) for the `input.mcp` fields and [Rewrite Action](https://docs.formal.ai/docs/guides/policies/enforcement.md#rewrite-action) for body rewrites. ## 5. Check the logs Open [Logs](https://app.formal.ai/logs) and search for `resource.name:braintrust`. Each Braintrust tool call is one `request` log with the user, the MCP client and the tool. Open a log to see its tool parameters and the policies it triggered. A masked search shows the rewrite policy. Braintrust tool calls in Logs, including a masked response and a blocked write See [Logs](https://docs.formal.ai/docs/guides/observability/logs.md) for search syntax and filters. ## Troubleshooting > [accordion] The agent says Formal has no current sign-in > > The agent relays: `Formal has no current braintrust sign-in for you. Link or re-link your braintrust account in Formal…` > > **Cause:** The user never linked Braintrust, or the token expired and couldn't be refreshed. > > **Fix:** Open the server in the [Employee Catalog](https://catalog.formal.ai) and link Braintrust again. > [accordion] The agent says Braintrust rejected your credentials > > The agent relays: `braintrust rejected your credentials. Link or re-link your braintrust account in Formal…` > > **Cause:** Braintrust no longer accepts the linked token, for example after the user revoked access in Braintrust. > > **Fix:** Link Braintrust again from the [Employee Catalog](https://catalog.formal.ai). > > With Shared Credentials, the message asks the user to contact an administrator. Replace the token in the server's **Shared Credentials**. > [accordion] Laptop traffic doesn't appear in Logs > > **Cause:** The Endpoint isn't running, or the network rule doesn't match the client. > > **Fix:** Check the Endpoint is connected. The rule above only matches [known agents](https://docs.formal.ai/docs/guides/network-rules.md#known-agent-identities). Other clients, such as a browser, reach Braintrust directly unless the rule also matches them. See [Network rules](https://docs.formal.ai/docs/guides/network-rules.md). > [accordion] MCP clients can't connect directly yet > > **Cause:** The **Configuration** tab lists the reason, most often `No Connector is in this server's Space.` > > **Fix:** Edit the server and choose the Space containing its Connector. ## Next Steps > [card] [MCP Gateway](https://docs.formal.ai/docs/guides/core-concepts/connectors/mcp-gateway.md) > > Configure Connector routing and downstream access > [card] [All MCP servers](https://docs.formal.ai/docs/guides/core-concepts/connectors/mcp-servers/index.md) > > Browse every MCP server template This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.