Skip to main content

Trigger Background Agents

Causely pushes structured diagnoses to your agents and automation through webhooks as Issues are detected, not only through MCP queries. A background agent doesn't need to poll or wait for someone to ask a question. It starts working as soon as Causely identifies the problem.

This page describes how to set up a Generic Webhook notification that targets the endpoint a background agent is listening on.

How it works​

  1. Causely detects an Issue and determines its underlying diagnosis.
  2. The Causely Mediator sends a Generic Webhook notification to your agent's endpoint. The request body is the Causely notification payload, which includes the Issue, the affected entity, its severity, and the diagnosis.
  3. The agent receives the notification and starts an investigation.
  4. The agent calls the Causely MCP server to retrieve the causal chain, logs, topology, and other evidence, then resolves the Issue or recommends a remediation.

The webhook tells the agent that something happened and what Causely concluded. MCP gives the agent the context to verify the diagnosis and act on it. This keeps the webhook payload small and lets the agent pull only the evidence it needs.

Prerequisites​

  • A background agent with an HTTP endpoint that accepts POST requests with a JSON body, reachable from the cluster where the Causely Mediator runs.
  • The agent is connected to the Causely MCP server. See Causely MCP Server for connection details, and Advanced Authentication for credentials that don't require a browser, which is what a non-interactive agent needs.
  • Permission to manage notifications in Causely.

Set up the notification​

  1. Go to Settings → Notifications and click Create.

  2. Complete the configuration:

    FieldValue
    NameA descriptive name, for example background-agent
    TypeGeneric
    URLThe endpoint your agent listens on, for example https://agent.example.com/trigger
    TokenThe full Authorization header value your agent expects, for example Bearer <token>

  3. Choose the cluster or clusters that handle processing and delivery.

  4. Add filters so the agent is only triggered by the Issues you want it to handle.

  5. Click Create.

The token is sent verbatim as the Authorization header, so include the scheme keyword yourself. See Authentication for details. To define the destination with a Kubernetes secret or environment variables instead of the UI, see Mediator-based configuration.

Control what wakes the agent​

Every notification that reaches your agent can start an investigation, and every investigation consumes agent time and model tokens. Filter on the Causely side so the agent only receives Issues it should act on.

For example, forward only High and Critical Issues:

notif_config_filters: '[{"field":"severity","operator":"in","value":["High","Critical"]}]'
notif_config_filters_enabled: "true"

You can also filter by cluster, namespace, entity type, or whether the Issue impacts an SLO. See Filters for the full list of fields and operators.

Start in a read-only mode

When you first connect an agent, have it investigate and report without making changes. Switch to automated remediation once you have confirmed that the diagnoses and the agent's conclusions match your expectations.

What the agent receives​

The request is an HTTP POST with Content-Type: application/json. The fields most useful to an agent are:

  • type: ProblemDetected when the Issue is detected, or ProblemCleared when it resolves.
  • objectId: The identifier of the Issue, which the agent can pass to MCP tools to retrieve its full details.
  • name and severity: What Causely found and how serious it is.
  • entity: The affected entity, with its id, name, and type.
  • labels: Cluster, namespace, and workload context.
  • description: A summary of the diagnosis.

See Notification Payload for a complete example and field reference.

Write the agent's handler​

Your agent's handler should:

  1. Authenticate the request. Reject calls that don't carry the expected Authorization header.
  2. Acknowledge quickly. Return a 2xx response immediately and run the investigation asynchronously.
  3. Ignore ProblemCleared events, or use them to close out an investigation in progress.
  4. De-duplicate on objectId, so a repeated notification for the same Issue doesn't start a second investigation.
  5. Investigate through MCP. Use the Causely MCP tools to retrieve the Issue, its root cause, supporting evidence, and affected services before deciding what to do. Treat the diagnosis in the payload as the starting point and verify it against live evidence.

Examples​

The causely-oss/background-agents repository contains runnable examples of agents that are triggered by Causely and use the MCP server to investigate:

  • causely-background-agent: A standalone reference agent that receives Causely's webhook push, investigates with the Causely MCP tools, and either opens a pull request with a fix or recommends a remediation. It starts in an observe mode that never makes changes.
  • open-inspect: Connects Open-Inspect to the Causely MCP server so its background investigations cite root cause and log evidence in the pull requests they open.
  • aws-devops-agent: Wakes the AWS DevOps Agent with a Causely notification through a signed-webhook relay. The agent calls back into the Causely MCP server for the causal chain.
  • claude-managed-agents: A Claude Managed Agent investigating a live Kubernetes cluster through remote MCP servers.

Learn more​