# Causely documentation: getting-started > Full content of the "getting-started" section of the Causely documentation. This file contains all documentation content in a single document following the llmstxt.org standard. ## Architecture Causely is built on a split-architecture model that balances local control with a dedicated reasoning backend that can run in your cloud or Causely-managed infrastructure. This design keeps overhead low, keeps raw telemetry in your environment, and integrates with the tools you already run. This page provides detailed information about Causely's deployment architecture and component structure. For a high-level overview of how Causely's causal reasoning engine works, see [How Causely Works](/getting-started/how-causely-works). ### System Architecture {(() => { const { colorMode } = useColorMode(); const src = colorMode === 'dark' ? '/img/how_causely_works_diagramLight1.svg' : '/img/how_causely_works_diagramDark1.svg'; return ; })()} ### Deployment Architecture Causely can be deployed across various environments, including Kubernetes clusters, standalone Docker hosts, Nomad clusters, and more. The deployment architecture consists of several components that work together to provide a real-time Diagnosis. :::info Deployment model Causely is designed for flexible deployment. The mediation layer and agents always run in your environment. The Causely backend (causal engine) can run in your cloud (BYOC) or in Causely-managed infrastructure. For requirements and data flows in a self-hosted deployment, see [BYOC (Self-Hosted) Deployment](/getting-started/byoc). ::: #### Mediation Layer The mediation layer is deployed locally in your infrastructure and processes telemetry data to extract only the signals needed for reasoning. It performs: - **Signal Detection**: Converts telemetry from Prometheus, CloudWatch, Datadog, OpenTelemetry (including eBPF), and other sources into a binary stream of active/inactive [signals](/reference/symptoms/). - **Topology Discovery and Ingestion**: Leverages integrated telemetry sources to discover entities and dependencies and ingest topology from systems such as OpenTelemetry, cloud provider APIs and other sources. - **Local Processing**: Processes telemetry locally to minimize data transfer, control cost, and ensure data stays within your environment. Most raw telemetry remains local, with only distilled insights and targeted evidence sent to the Causely backend (deployed in your cloud or Causely-managed) for analysis. The mediation layer primarily sends distilled insights to the cloud. After a Diagnosis is made, a targeted subset of relevant telemetry (metrics, traces, and log-derived errors/events) may be sent as evidence to enhance its clarity. For more detail on supported telemetry sources, see [Supported Telemetry](/telemetry-sources/). The mediation layer consists of the following components: ##### Mediator The Mediator is the core component that runs locally in your environment. It discovers your services, infrastructure and dependencies, receives OpenTelemetry data on port 4317, pulls from your existing telemetry sources, and evaluates signals locally. It sends the Causely backend (deployed in your cloud or Causely-managed) an abstract representation of your environment, plus targeted evidence after a Diagnosis is made. For what the Mediator does with each type of telemetry, what leaves your environment, and how it scales, see [Mediator Details](/getting-started/mediator). :::info Mediator scale behavior Mediator memory scales with the **number of entities under management**, not with trace or metric throughput, and the Mediator works from a statistically significant sample of traces. See [How it scales](/getting-started/mediator#how-it-scales) for details, and [Sizing the Mediator for large environments](/installation/customize#sizing-the-mediator-for-large-environments) for configuration. ::: ##### Agents Agents are deployed across your infrastructure to gather node and container level metrics. The deployment method varies depending on your environment: - **Kubernetes**: Agents are deployed as a DaemonSet across all nodes in the cluster - **Docker**: Agents run as containers on standalone Docker hosts - **Nomad**: Agents are deployed as Nomad jobs across the cluster Agents leverage [eBPF](/telemetry-sources/ebpf) technology, which requires privileged access to the host system. This enables automatic instrumentation without code changes. The eBPF instrumentation uses **[uprobes](https://docs.kernel.org/trace/uprobetracer.html) exclusively** to intercept specific user-space functions within your applications. It does not hook into kernel networking callbacks or the packet datapath, and does not act as a [Container Network Interface (CNI)](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/) or network infrastructure component. The instrumentation may inject trace context headers for distributed tracing, but does not intercept, block, or route network traffic at the kernel level. Agents don't establish any outbound connections to the internet or any other service apart from the Mediator and VictoriaMetrics. The agents periodically forward the topology and manifestation data to the Mediator, which, in turn, sends it to the Causely backend for analysis (running in your cloud or Causely-managed). If an agent fails or is removed, your applications and network continue to function normally. ###### Agent Architecture {(() => { const { colorMode } = useColorMode(); const src = colorMode === 'dark' ? '/img/architecture/architecture-detailed-dark.svg' : '/img/architecture/architecture-detailed-light.svg'; return ; })()} ##### Executor The Executor is an optional component responsible for executing remediation actions within your infrastructure. The Executor can be enabled as part of the deployment process. The specific permissions required depend on your deployment environment: - **Kubernetes**: The Executor's ServiceAccount is granted the `cluster-admin` role - **Docker/Nomad**: The Executor requires appropriate permissions to execute remediation actions ##### VictoriaMetrics VictoriaMetrics is a timeseries database used by the agents and mediator (on `port: 8428`) to store additional timeseries data locally in your environment. If you observe gaps in metrics graphs, VictoriaMetrics may be under-resourced for your environment. See [Sizing VictoriaMetrics for high-volume environments](/installation/customize#sizing-victoriametrics-for-high-volume-environments). #### Causal Engine The Causal Engine runs in the Causely backend, which can be deployed in your cloud or in Causely-managed infrastructure. It receives the stream of signal states and performs real-time analysis using probabilistic modeling, system graphs, and causal inference. It produces Diagnoses, evaluates blast radius, validates constraints, and prioritizes remediation, all without requiring manual correlation. #### Telemetry Sources Causely supports a wide range of telemetry sources, including OpenTelemetry, Prometheus, CloudWatch, Datadog, and more. For a full list of supported telemetry sources, see [Supported Telemetry](/telemetry-sources/). Causely Agents are deployed in your infrastructure and are responsible for collecting the telemetry data from those sources. By default Causely will automatically instrument your applications to receive [OpenTelemetry](/telemetry-sources/opentelemetry) traces. This allows Causely to discover service dependencies, monitor sync and async communication signals. Additionally you can export traces to Causely from your existing OpenTelemetry Collectors. We recommend that you always send OpenTelemetry traces to Causely, as this allows Causely to provide cross-service insights. #### Agent Integration Causely exposes its causal reasoning (Diagnoses, blast radius, dependencies) to AI agents via an MCP server. Any MCP-compatible agent, including Claude Code, Cursor, HolmesGPT, or a custom agent, gets deterministic access to root causes, service health, dependency maps, and blast radius. Instead of reasoning over raw telemetry, agents query structured, causal system knowledge. This reduces token consumption, eliminates guesswork, and lets agents take action with auditable evidence. For details on connecting agents to Causely, see [Agent Integration](/agent-integration/). #### Workflow Integration Causely integrates directly into your tools of choice, delivering causal insights into Slack, Alertmanager, Opsgenie, Grafana, and more. For details on how to connect Causely to your workflows, see [Supported Workflows](/workflows/). This architecture allows Causely to deliver precise, real-time insights without burdening your data pipelines or violating privacy requirements. ### Security Considerations For detailed information about security, permissions, and data handling, see the [Security](/security/) documentation. --- ## BYOC (Self-Hosted) Deployment Causely can run entirely inside your own cloud account or data center. In a bring your own cloud (BYOC) deployment, also called self-hosted or on-premises, both the mediation layer and the Causely backend (causal engine, API, UI and MCP server) run on infrastructure you control. Nothing is sent to Causely. This page explains what a BYOC deployment includes, what infrastructure it requires, and how data flows. It is not an installation guide. :::info Deploying Causely in your environment To deploy Causely as BYOC, contact your Causely support representative. They will provide deployment instructions, the Helm chart, and access to the container images. ::: ### BYOC compared with SaaS (Causely-hosted) The mediation layer always runs in your environment. The difference between the two models is where the backend runs and whether anything leaves your network. | | SaaS (Causely-hosted) | BYOC (self-hosted) | | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | Mediation layer (mediator, agents, local metrics store) | Your environment | Your environment | | Causely backend (causal engine, API, UI) | Causely-managed infrastructure | Your cloud account or data center | | MCP server for AI agents | Causely-hosted endpoint (`api.causely.app/mcp`) | Part of the API service, served from your own portal endpoint | | What leaves your environment | An abstract representation of your environment and targeted evidence. See [What leaves your environment](/security#what-leaves-your-environment). | Nothing. All traffic stays within your network. | | Outbound connections to Causely | Mediator connects to the Causely backend over outbound TLS | None. There is no license check, phone-home, or update channel to Causely. | | Causely's own health telemetry | Sent to Causely for support by default. Can be turned off. | If turned on, sent only to your self-hosted backend. Can be turned off. | | User authentication | You configure single sign-on (SSO) with any identity provider over OpenID Connect or SAML from the admin portal. See [Single Sign-On (SSO)](/manage/administration/sso). | Optional OIDC through your identity provider (for example Microsoft Entra ID, Okta, or Keycloak) | | Updates | Causely updates and manages the backend. You manage and upgrade the mediation layer (mediator and agents). | You manage and upgrade both the backend and the mediation layer, pulling new versions from Causely's registry or your private mirror on your own schedule | | Encryption | Managed by Causely. See [Encryption](/security#encryption). | Governed by your own infrastructure and policies | | LLM usage | Not used for causal inference. See [LLM usage](#llm-usage). | None by default. Optional features use your own LLM provider. | ### Architecture A BYOC deployment uses the same components as SaaS. The only difference is where the backend runs. Mediators are deployed next to your workloads, one per cluster or environment, as described in [Architecture](/getting-started/architecture) and [Mediator Details](/getting-started/mediator). The backend runs in a Kubernetes cluster you operate and is reached through two endpoints on a domain you choose: - **`gw.`**: the gateway that receives data from your mediators. - **`portal.`**: the UI, API and MCP server for users and AI agents. ```mermaid flowchart LR subgraph ENV["Your cloud or data center"] direction LR subgraph WL["Monitored environments"] MED["Mediators
one per cluster or environment"] end subgraph BE["Causely backend (Kubernetes)"] ING["Ingress
gw. and portal. endpoints"] SVC["Gateway, API + MCP,
Analysis (causal engine), UI"] ING --> SVC end PG[("PostgreSQL
managed service")] RD[("Redis or Valkey
managed service")] USERS["Users, AI agents,
workflow integrations"] IDP["Your identity provider
(OIDC, optional)"] MED -- "TLS" --> ING USERS -- "TLS" --> ING SVC --> PG SVC --> RD SVC -. "login" .-> IDP end ``` #### Backend components The backend is installed with a single Helm chart and includes the following services: | Component | Role | | ----------------------------- | ---------------------------------------------------------------------------------------------------------------- | | Gateway | Receives topology, symptom states and evidence from your mediators | | Analysis | Runs the causal engine that builds the causality graph and infers root causes. Includes a machine learning sidecar. | | API | Serves the Causely API, including the [MCP server](/agent-integration/) for AI agents | | UI | Serves the Causely portal | | Background | Runs scheduled and asynchronous jobs | | Image generation | Renders metric charts and dependency graphs as images | | Database migrations | Kubernetes job that applies database schema changes during installs and upgrades | | Copilot controller (optional) | Generates LLM-based diagnosis summaries using your own AI gateway. Disabled by default. | #### Supporting infrastructure You provide or install the following alongside the backend: - **Ingress controller**, such as NGINX Ingress, to expose the `gw.` and `portal.` endpoints. We recommend an internal-only load balancer that users and mediators can reach over your private network. - **cert-manager** to provision and renew TLS certificates. - **PostgreSQL**, the primary data store for configuration, analysis results and system metadata. - **Redis or Valkey**, used by the API, Gateway and Analysis services for caching and coordination. ### Resource requirements | Requirement | Specification | | ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Kubernetes cluster for the backend | At least 3 nodes, each with 4 vCPU and 16 GB RAM | | Tooling | Helm 3.x and `kubectl` with access to the cluster | | DNS | Two records under one base domain: `portal.` and `gw.` | | TLS | Certificates for both endpoints, typically issued by cert-manager | | Network | Mediators must reach the `gw.` endpoint. Users and AI agents must reach the `portal.` endpoint. | | Container images | Access to Causely's container registry with credentials from Causely, or a private registry where you mirror the Causely backend, mediation and supporting open source images | | PostgreSQL | A managed PostgreSQL instance (see [Managed services](#managed-services)) | | Redis | A managed Redis or Valkey instance (see [Managed services](#managed-services)) | | Identity provider (optional) | An OIDC provider to protect access to the portal and API | These requirements cover the backend. Mediation layer resources depend on the size of each monitored environment. See [Sizing the Mediator for large environments](/installation/customize#sizing-the-mediator-for-large-environments) and [Sizing VictoriaMetrics for high-volume environments](/installation/customize#sizing-victoriametrics-for-high-volume-environments). ### Managed services For production BYOC deployments, we recommend running PostgreSQL and Redis as managed services from your cloud provider or database team rather than inside the Kubernetes cluster. Managed services provide backups, high availability and operational support that in-cluster databases do not. In-cluster PostgreSQL and Redis are supported only for evaluation and other non-production environments. #### PostgreSQL - Supported services include Amazon RDS and Aurora, Google Cloud SQL, Azure Database for PostgreSQL, or an equivalent offering. - The `pg_stat_statements` and `uuid-ossp` extensions are required. They are installed automatically if the database user can create extensions. Otherwise, your DBA must install them. - A non-default schema is supported. - On Amazon RDS and Aurora, Causely can authenticate with IAM database authentication instead of a password. #### Redis - Supported services include Amazon ElastiCache, Google Memorystore, Azure Cache for Redis, or any Redis-compatible (Redis or Valkey) service. - TLS connections are supported. - On Amazon ElastiCache, including serverless caches, Causely can authenticate with IAM instead of a password. ### Data and network flows In a BYOC deployment, every Causely component runs within your network boundary: - Mediators send an abstract representation of your environment and targeted evidence to your own backend over TLS. The content is the same as described in [What leaves your environment](/security#what-leaves-your-environment), but it never leaves your network. - The backend does not connect to Causely for licensing, authorization, updates or telemetry. - Causely's own health telemetry, if turned on, is sent only to your self-hosted backend. - User and API access can be protected with OIDC through your own identity provider. - The MCP server is served from your own `portal.` endpoint. When you connect AI agents, the [agent integration](/agent-integration/) instructions in your portal point to that endpoint. ### LLM usage Causely does not use LLMs for topology discovery, symptom detection, causal inference or diagnosis. These are performed by Causely's causal reasoning engine. In the default BYOC deployment, Causely sends no input to and receives no output from any LLM. Two optional capabilities let you bring LLMs in under your own control: - **LLM-based diagnosis summaries.** The optional Copilot controller generates log summaries and descriptions for each diagnosis. It uses an AI gateway and model that you configure. - **AI agents through MCP.** You can connect your own agents, such as Claude Code, Cursor or a custom agent, to the MCP server in your deployment. In both cases, data flows directly between your deployment and the LLM provider you choose, under your own agreement with that provider. It does not pass through Causely. ### Getting started Contact your Causely support representative to plan a BYOC deployment. They will provide deployment instructions, the Helm chart, registry credentials, and guidance on sizing for your environment. For more on permissions and data handling in the mediation layer, see [Security](/security). --- ## First-Time Usage Guide This short guide walks you through the core actions to take right [after setup](/getting-started/quick-setup). No deep configuration or prior knowledge needed, just follow along and start uncovering real insights from your system. ### Understanding the Summary Insight Panel Your journey begins at the [Causely Portal](https://portal.causely.app/welcome), where you'll find the **Summary Insight Panel**. This is your command center for monitoring system health and identifying critical issues. Unlike traditional monitoring systems that bombard you with alerts, Causely focuses on delivering the **underlying cause** through our innovative **Issue Cards**. Each card represents a verified issue in your system, backed by our extensive knowledge base of cloud native failure patterns. :::tip Pro Tip To maximize efficiency, [integrate Causely with your existing tools](/workflows) for seamless workflow integration. ::: ### Exploring Issue Details When you click on an Issue Card, you'll access the **Issue Details View**, your comprehensive dashboard for understanding and resolving system issues. This view is organized into three main sections: #### 1. Summary Tab The Summary tab provides essential information: - **Overview**: Current status, affected services, SLO violations, and timeline - **Diagnosis Description**: Clear explanation of the issue - **Impact Analysis**: Detailed assessment of system effects - **Evidence Collection**: - Observed [signals](/reference/symptoms/) - System exceptions - Relevant logs - Related events - **Remediation Steps**: Actionable solutions to resolve the issue #### 2. Causality Tab The Causality tab presents a visual representation of the issue's causal path, helping you understand how different system components interact. #### 3. Timeline Tab The Timeline tab provides a visualization of all Diagnoses, Signals, events, and SLO breaches associated with the Issue, from first detection to resolution. :::tip Best Practice The Issue Details View is your primary investigation tool. In many cases, the suggested remediation steps provide a complete solution to your system issues. ::: ### Connect your AI Agent to Causely Our [MCP Server](/agent-integration/mcp-server/) exposes Causely's causal reasoning (Diagnoses, blast radius, dependencies) to any AI agent: Claude, Cursor, or your own tooling. Once connected, your agent can answer questions like: - "What's causing this Issue?" - "How's this affecting our services?" - "What's the timeline of an Issue?" - "What steps should we take to fix this?" [Setup the MCP Server](/agent-integration/mcp-server/) ### Next Steps for reducing MTTR Now that you've mastered the basics, we recommend: 1. [Connecting agents](/agent-integration/) for faster issue resolution and proactive remediation 2. [Adding additional telemetry sources](/telemetry-sources/) for more comprehensive analysis 3. [Setting up workflow integrations](/workflows) with your existing tools --- ## How Causely Works Causely transforms raw telemetry into a live, queryable causality graph of your system providing the semantic and causal foundation your agents need to diagnose, evaluate impact, and act safely in production. It works alongside your existing telemetry sources, interpreting the meaning behind the metrics, logs, and traces they provide, rather than replacing them. This page explains how Causely works: the **ontology-first approach** that turns raw telemetry into higher abstractions, then how the **causal reasoning engine** uses those abstractions to infer a Diagnosis and compute the blast radius that follows from it. For deployment architecture, components, and infrastructure, see [Architecture](/getting-started/architecture). ### Issue, Diagnosis, and Signal Before going into the mechanics, it helps to know the three terms used to describe what Causely finds: an **Issue** is the persistent problem you acknowledge and resolve; a **Diagnosis** is Causely's current best explanation for it, produced by the causal reasoning engine described below; and a **Signal** is an observed anomaly a Diagnosis explains, usually a threshold violation on a metric or SLI, and sometimes an alert, log line, or event. > An Issue causes anomalies that surface as Signals. Causely infers a Diagnosis, the best explanation of those Signals at that moment. As Signals change, the Diagnosis updates. The Issue stays stable across those changes. If two Diagnoses share even one Signal, they are treated as the same Issue. The rest of this page describes the causal reasoning engine that produces a Diagnosis from Signals, using the terms root cause and symptom, the formal names for these concepts defined in [Terminology](/reference/terminology/#issue-diagnosis-and-signal). ### How the Causal Engine Works At the core of Causely is a probabilistic reasoning engine that maps symptoms to root causes using its causal models and a causality graph of your environment. It maintains three core data structures: the **Topology Graph**, the **Causality Graph** (with its Codebook), and the **Attribute Dependency Graph**, and uses them for analysis. The engine is composed of six interdependent components, described below. {(() => { const { colorMode } = useColorMode(); const src = colorMode === 'dark' ? '/img/how_causely_works_diagramLight1.svg' : '/img/how_causely_works_diagramDark1.svg'; return ; })()} #### 1. From telemetry to semantic understanding Instead of feeding raw telemetry straight into analysis, Causely uses an **ontology-first approach**: **mediation** distills telemetry (logs, metrics, traces) and other data locally into a structured layer of: - **[Entities](/reference/entity-types)**: services, pods, databases, queues - **Relations**: how they connect - **[Signals](/reference/symptoms/)**: observable anomalies such as "high latency here" or "errors increasing there"; detected when thresholds or patterns are met and used for reasoning even when no alert is fired That distillation turns the vast volumes of data teams commonly ingest for observability into higher-quality abstractions. This distilled data is sent to the backend, where the causal reasoning engine infers the [causes](/reference/root-causes/) explaining those Signals. :::tip Local processing and privacy All distillation runs in your environment. The **mediation layer** turns telemetry into **symptom states** and **topology**; only this **distilled data** is sent to the causal reasoning engine. Raw logs, full traces, and bulk metrics stay in your datacenter. Only semantic state and, when needed, minimal evidence leave your environment. This design keeps sensitive and high-volume telemetry local and preserves **privacy**. ::: #### 2. Ontology: causal and attribute models The engine’s **ontology** is the formal representation of entities, behaviors, and failure modes used for inference. It is composed of two foundational parts: the **Causal Models** and the **Attribute Dependency Model**. Together, they define what root causes and symptoms exist and how they relate, forming the semantic backbone of the causal reasoning engine. - **Causal Models**: Causely includes a built-in library of causal knowledge that captures root causes capable of degrading application performance. These models require no configuration and enable the system to begin identifying root causes as soon as it is deployed. Causal models are engineered knowledge of how failures propagate. Causely calibrates them to your environment; no LLM is trained on your data. - Cover a broad range of entities, including applications, databases, caches, messaging systems, load balancers, DNS, compute, and storage. - Encode how each root cause propagates through the environment and the symptoms it may produce. - Are designed to be environment-agnostic and applicable across Kubernetes, cloud managed services, VMs, and bare metal. - **Attribute Dependency Model**: The attribute model extends the causal models by capturing how performance-related attributes (for example, latency, throughput, utilization) are interdependent across entities. It also encodes the operational constraints those attributes must satisfy to meet performance goals. - Represents attribute dependencies across a wide range of services and infrastructure layers. - Supports both predefined and learned functional relationships between attributes. - Defines the desired state of the system based on application goals and constraints. - Like the causal models, it is fully environment-independent and generalizable across architectures. #### 3. Topology Graph Causely continuously discovers and maintains the topology graph of services, infrastructure, and their interconnections via its integrations with [your existing telemetry sources](/telemetry-sources/). It ingests and reconciles topology from any source, including OpenTelemetry traces, cloud provider APIs and other inventories. Cloud native environments are highly dynamic, composed of applications, services, databases, caches, messaging systems, load balancers, compute, storage, and more. Through continuous discovery and ingestion from these sources, Causely determines for each entity: - **Connectivity**: which other entities it communicates with horizontally - **Layering**: which entities it is built upon or supports vertically - **Composition**: the internal components or resources that make up the entity These relationships are stitched together into a continuously updated, real-time graph that represents the full system [topology](/reference/terminology/#topology), regardless of whether data comes from OpenTelemetry, cloud APIs or other integrated sources. This graph forms the foundation for blast radius analysis, root cause attribution, and cross-service impact modeling. #### 4. Causality Mapping Causely automatically generates a [Bayesian network](/reference/terminology/#bayesian-network) that models how root causes lead to observable symptoms, based on its built-in Causal Models and the real-time Topology Graph. This causal mapping encodes probabilistic cause-effect relationships, allowing the system to infer the most likely root cause from a given set of active symptoms. It reflects both the structural dependencies in your environment and learned patterns of failure propagation. Causely represents this mapping through two core data structures: - **[Causality Graph](/reference/terminology/#causality-graph-cg)**: A directed acyclic graph (DAG) where nodes represent root causes and symptoms, and edges denote potential causal relationships. Each edge is weighted with a probability, indicating the likelihood that one node (a root cause) leads to another (a symptom). - **[Codebook](/reference/terminology/#codebook)**: A table where each column corresponds to a root cause and each row to a symptom. Each column is a vector of probabilities defining a unique signature of the root cause. A cell in the vector represents the probability that the root cause may cause the symptom. Together, these structures power Causely’s ability to perform real-time probabilistic inference and deliver explainable, high-confidence root cause insights. #### 5. Attribute Dependency Graph Causely generates this graph using its built-in Attribute Dependency Model and the live Topology Graph. The result is a directed acyclic graph (DAG) that models functional dependencies between system attributes. In this graph: - Nodes represent individual attributes, for example CPU usage of a service or queue length of a messaging system. - Edges represent dependency relationships, for example an edge from attribute A to attribute B means that B is a function of A. - Edge labels define these functions. Some may be explicitly defined in the Attribute Dependency Model, while others are learned automatically from observed behavior in your environment. - Nodes representing attributes that must satisfy a constraint are decorated with the constraint the attribute must satisfy. The Attribute Dependency Graph enables Causely to reason about how changes in one part of the system cascade across others, identify emerging bottlenecks, and validate whether the environment remains within defined performance bounds. #### 6. Analysis The analysis automatically pinpoints causes in real time based on observed symptoms, using the Codebook described above. No configuration is required. Causely immediately identifies a broad set of issues, ranging from application malfunctions to service congestion to infrastructure bottlenecks, using the causality graph built from your environment, typically thousands to tens of thousands of cause-to-symptom mappings. In any given environment, there can be tens of thousands of different causes that may cause hundreds of thousands of symptoms. Causely prevents service degradation by detangling this mess and pinpointing the cause putting your SLOs at risk and driving remediation actions before SLOs are violated. With the ontology and causal layers in place [agents connected to Causely](/agent-integration/) can query diagnosis, blast radius, and ownership directly, answering production questions in seconds with a small fraction of the tokens querying raw telemetry would consume. ### Deployment architecture For deployment architecture, component structure, and workflow integration, see [Architecture](/getting-started/architecture). For what the mediator does with your telemetry and how it scales, see [Mediator Details](/getting-started/mediator). --- ## Mediator Details The mediator is a lightweight container you run inside your own environment. It turns your telemetry into a live, abstract representation of your system (entities, their relationships and their symptom states), and sends only that representation to Causely's reasoning system. Your raw data stays with you. That design is why Causely works at large-scale telemetry volumes. - **It processes telemetry where it lives.** Metrics, traces, logs, events and alerts are analyzed inside your own cluster or VPC. - **It sends an abstract representation, not your data.** What goes upstream is a compact, compressed snapshot of entities, their relationships and their symptom states (healthy or anomalous). That's the input causal reasoning needs. - **It scales out, not just up.** Heavy trace volumes can be spread across horizontally scaled trace workers. Large estates run many mediators, one per cluster, VPC or business domain. The reasoning system joins them into one live causality graph. - **It works with the telemetry you already have.** The mediator combines data from all of your existing sources into one picture. You never have to choose one source of truth. For how the mediator fits with the other components Causely deploys, see [Architecture](/getting-started/architecture). ### Where the mediator sits The mediator runs next to your workloads: one per Kubernetes cluster, ECS or Nomad environment, or VM estate. It is the only Causely component that processes your telemetry. It connects to the Causely backend over a single outbound TLS connection. That backend can be Causely-hosted or run in your own cloud (bring your own cloud, BYOC). ```mermaid flowchart LR subgraph ENV["Your environment"] direction LR SRC["Existing tools
Prometheus, Datadog, Dynatrace,
Grafana, Splunk, Elastic"] OTEL["OpenTelemetry
collectors and SDKs"] PLAT["Platform APIs
Kubernetes, AWS, Azure, GCP, CMDB"] MED["Mediator
discover, distill, detect"] TSDB[("Local metrics store
1-day retention")] SRC --> MED OTEL --> MED PLAT --> MED MED <--> TSDB end MED -- "topology + symptom states
outbound TLS only" --> BE["Causely reasoning system
SaaS or BYOC"] BE -. "targeted evidence requests" .-> MED ``` Telemetry arrives in two ways: sources push it to the mediator, or the mediator pulls it. - **Pushed to the mediator:** [OpenTelemetry](/telemetry-sources/opentelemetry) traces and metrics (OTLP on port 4317), Datadog APM traffic, and alert webhooks. - **Pulled by the mediator:** it queries your existing tools and platform APIs on a schedule (Prometheus, Datadog, Dynatrace, cloud provider APIs, databases, message brokers, CMDB and inventory APIs). Nothing needs to be re-instrumented. See [Supported Telemetry](/telemetry-sources/) for the full list. - **Local store:** entity metrics are written to a small time series store in your environment, kept for one day by default. You can point this at [your own Prometheus-compatible store](/installation/customize#using-an-external-time-series-database) instead. ### What the mediator does, continuously The mediator is always on. It keeps its representation of your system current in real time. It runs six steps in a loop. 1. **Discover.** It builds a live graph of your entities (services, workloads, pods, nodes, databases, queues, topics, HTTP routes, cloud resources) and their relationships. It uses Kubernetes and cloud APIs, your APM and monitoring systems, and CMDB or inventory data. 2. **Ingest.** It receives OpenTelemetry traces and metrics, scrapes your existing tools on a schedule (as often as every 10 seconds), and scans container logs on a schedule for errors and known failure patterns. 3. **Distill.** Each span is reduced as soon as it arrives: - Request, error and duration counters, plus compact latency sketches. - URL paths collapsed into route templates, so `/users/123` becomes `/users/{id}`. - SQL statements normalized, with literal values replaced by placeholders. Once distilled, the raw span is discarded. 4. **Detect locally.** [Symptoms](/reference/symptoms/) such as high latency, error spikes, saturation, restarts and consumer lag are evaluated inside your environment. They use default thresholds, your overrides, or thresholds the mediator learns from your own recent history. Learned thresholds are recalculated hourly from the last 24 hours. 5. **Sync.** Approximately every 30 seconds, the mediator sends its current representation to the reasoning system: entities, relationships and symptom states. The snapshot is compressed and streamed in chunks. 6. **Answer.** When the reasoning system is building or explaining a Diagnosis, it can ask for specific evidence. Examples are one metric series for one entity, or a short, scrubbed log excerpt for one pod. The mediator fetches just that item and returns it. This is how Causely can reason over your entire system continuously without first copying all of your data to a central store. ### What happens to each type of telemetry Each type of telemetry becomes structure or state in the mediator's abstract representation, and is then discarded or kept locally. Only that representation leaves your environment continuously. Anything more is sent only when the reasoning system asks for a specific piece of evidence. | Telemetry | Where it comes from | What the mediator does with it | What stays in your environment | What goes to the reasoning system | | --- | --- | --- | --- | --- | | Traces | OpenTelemetry SDKs and collectors, Datadog APM and other APM agents | Discovers dependencies (HTTP, gRPC, database, queue and topic, GenAI and MCP calls). Computes request rate, error rate and latency per service and endpoint. | Nothing. Spans are discarded once analyzed. | Relationships, plus symptom states such as high latency or errors | | Metrics | Prometheus, Datadog, Dynatrace, cloud monitoring, OTLP metrics, Causely agents | Maps them onto the entities they describe. Evaluates symptoms against thresholds. | Entity metrics in the local store (one-day default retention) | Symptom states. A specific series only on request. | | Logs | Container logs, read through the Kubernetes API or from Loki, Elasticsearch or Splunk | Scans new lines on a schedule. Counts error lines and known failure patterns (out-of-memory, deadlocks, connection-pool exhaustion, and similar) and turns them into symptoms. | Your log store stays the system of record. Causely keeps only counts. | Symptom states. A short, PII-scrubbed excerpt only on request. | | Events and alerts | Kubernetes events, Alertmanager, Prometheus, Grafana, Datadog, Dynatrace, incident management systems | Maps them to symptoms on the affected entity. You can map your own alerts. | Nothing extra | Alert and symptom states | | Topology and config | Kubernetes, AWS, Azure, GCP, service mesh, APM topology, CMDB, topology files | Builds and maintains the entity graph. Merges overlapping sources. | Configuration details | Entities and relationships. Config only on request. | #### Logs Causely doesn't copy, index, or store your logs. The mediator queries them where they already live and pulls only what it needs: it retrieves the relevant logs as supporting evidence when it identifies an active issue and scans on a schedule for defined error patterns (such as Java null pointer exceptions) that become symptoms for Causely's inference system. - **Scheduled, incremental scanning.** The mediator works through every container on a rolling cycle, reading only the lines written since its last pass. It reads from the Kubernetes API, or from [Loki](/telemetry-sources/grafana), [Elasticsearch](/telemetry-sources/elasticsearch) or [Splunk](/telemetry-sources/splunk) if you use them. Scans run with bounded concurrency, so your log store isn't hit by a constant stream of heavy queries. - **Patterns become symptoms.** Each pass counts error lines and matches a built-in library of known failure signatures, such as out-of-memory errors, deadlocks, connection-pool exhaustion, full disks, open circuit breakers and thread-pool exhaustion. You can add your own patterns. Matches activate symptoms on the affected entity, alongside symptoms from metrics and traces. - **Only counts are kept.** Log lines are discarded once they're counted. Nothing is indexed, and no log stream goes upstream. - **Evidence on demand, scrubbed.** When the reasoning system needs to show evidence for a specific entity, the mediator retrieves a short, scoped slice of recent lines. It removes emails, phone numbers, card numbers, national IDs, IP addresses, tokens, API keys and credentials before anything leaves. Today, service dependencies come from traces, the service mesh, and cloud and APM topology rather than from log text, because those sources record who calls whom directly. If your logs carry signals those sources miss, custom patterns are the simplest way to bring them in. The result is that Causely uses the signal in your logs without you needing to move them or depend on storing and indexing them, however large your log volume is. ### Combine data from multiple sources You don't have to choose one source of truth. The mediator merges every source it receives into a single entity graph, so each source adds to the same picture instead of producing its own partial one. For example, one team instruments its services with OpenTelemetry and another uses Datadog APM. Both send traces to the mediator. It recognizes the same services and endpoints in each, so a call from an OpenTelemetry-instrumented service to a Datadog-instrumented one becomes one dependency in one graph, not two disconnected maps. - **Same thing, same entity.** Entities get stable IDs derived from what they are, such as their Kubernetes identity, cloud resource or service name. When several sources describe the same service, they update one entity, not several. - **Conflicts resolve per attribute.** When two sources disagree on a value, the mediator applies a defined source priority for that type of data. Each attribute comes from the highest-priority source available, and gaps are filled from the rest. - **Nothing to rip out.** Keep your collectors, vendors and dashboards. Causely adds a causal layer on top of them. Where a service has no instrumentation at all, Causely's [automatic instrumentation](/telemetry-sources/ebpf) can add coverage without code changes, and its data merges the same way. :::info Deploy mediators close to your workloads Telemetry from a service must reach the same mediator that discovers it. The mediator places incoming traces into the entity graph by matching them to workloads it already knows about, so a trace from a service it can't see has nowhere to go. That's why mediators run close to the workloads they observe, not as a single central sink at the network edge. ::: ### How it scales The mediator's work grows with the number of entities it manages, not with the volume of telemetry flowing through it. The answer to "can it handle petabytes a day?" therefore doesn't depend on any single component. Volume is absorbed where the data is produced, and capacity is added by adding trace workers and mediators. Every request to the same endpoint updates the same counters and the same latency sketch. Ten requests a second and ten thousand a second use roughly the same memory. What adds memory is new entities: services, pods, routes, queues, database queries. A cluster with more services needs more mediator memory; a high-traffic cluster with the same number of services does not. The mediator works from a **statistically significant sample** of traces and metrics. It does not need 100% of raw telemetry to perform accurate analysis. When incoming trace volume exceeds what the mediator can handle under memory pressure, it drops the excess before decoding it and keeps processing. This is intentional, not data loss: it means the mediator already has enough data for analysis, and symptom detection stays accurate. It doesn't fall over, and it doesn't back-pressure your collectors. Request counts you see in Causely may therefore differ from totals in other monitoring systems. #### Three ways to add capacity ```mermaid flowchart LR subgraph A["Cluster / VPC A"] OA["OTel collectors"] --> TC["Trace controller"] TC --> W1["Trace worker 1"] TC --> W2["Trace worker 2"] TC --> W3["Trace worker N"] W1 --> MA["Mediator A"] W2 --> MA W3 --> MA end subgraph B["Cluster / VPC B"] OB["OpenTelemetry"] --> MB["Mediator B"] end subgraph C["Business domain C"] OC["OTel + existing APM"] --> MC["Mediator C"] end MA --> BE["Causely reasoning system
one causality graph"] MB --> BE MC --> BE ``` 1. **Scale up the mediator.** It starts small and grows with the environment. See [Recommended Sizing Guidelines](/installation#recommended-sizing-guidelines) for typical requests and limits, and [Sizing the Mediator for large environments](/installation/customize#sizing-the-mediator-for-large-environments) to raise the default 16Gi memory limit. 2. **Scale out trace processing.** For high trace volumes, turn on the [scalable trace processor](/installation/scalable-trace-processor). A stateless trace controller receives traces and routes each service to a dedicated trace worker. Workers run as a horizontally scaled set and do the CPU-heavy span analysis. The mediator only receives their results. Routing can use `service.name` or a coarser key, such as a business application, to keep related services together. 3. **Run more mediators.** Large estates deploy one mediator per cluster, VPC or business domain. Each mediator owns a subset of services. The reasoning system joins their entity graphs into one causality graph, including dependencies that cross mediator boundaries. If your traces flow through a central OpenTelemetry Collector gateway, its load-balancing exporter can route each service's traces to the right mediator. #### Egress at scale What streams upstream continuously is an abstract representation of your environment: entity IDs, relationships and symptom states, compressed and sent approximately every 30 seconds. Its size depends on how many entities you have, not how much telemetry they produce, and growth in data volume does not become growth in egress. ### Why run the mediator in your environment Running the mediator inside your environment means the analysis happens where the data lives, which pays off in five ways: - **Egress stays small.** Only the abstract representation and targeted evidence leave, not 100% of traces, metrics and logs. - **The footprint stays light.** The mediator focuses on real-time analysis and keeps no long-term history. - **Your data stays local.** Raw telemetry remains in your datacenter, and mediators can run at the edge of your infrastructure, so data doesn't cross regions or clouds. - **The picture is always current.** Topology and symptom states are up to date before an incident begins. - **Detection sees all traffic.** Symptoms are evaluated in the mediator, before any sampling your pipeline applies downstream. It also means Causely runs continuously inside your environment, rather than as a destination you ship data to. Three familiar approaches show what that avoids: - **Sending all telemetry to a central store.** Ingest, indexing and cloud egress costs rise with every new service and every traffic spike. Teams sample aggressively to control that cost, and the sampled-away requests become blind spots. Raw telemetry also leaves your environment, which triggers data-sovereignty and compliance reviews. What you get is stored data, not a causality graph. - **Querying your tools when an incident starts.** Topology and dependencies are rebuilt, or guessed, at the start of every investigation. Each investigation makes many API and query calls, which adds latency, runs into rate limits and costs money in query fees and LLM tokens. It only sees what your tools kept after sampling and retention limits. - **Building on a data lake.** You pay to store everything to analyze a small fraction of it. Analysis happens at query time, which is slowest when an incident is in progress. Correlation rules need constant upkeep and still produce correlation, not causation. The outcome is causal answers from a live picture of your entire system, built where your data already lives. ### Data handling Raw telemetry stays in your environment. The abstract representation leaves continuously, and targeted, scrubbed evidence leaves only when the reasoning system asks for it. The mediator connects outbound over TLS only; Causely never opens inbound connections into your environment. For the full breakdown of what is sent, what is never sent and how personal data is scrubbed, see [What leaves your environment](/security#what-leaves-your-environment). ### Frequently asked questions **Do we need to send Causely all of our logs?** No. The mediator scans your logs where they live and keeps only counts of errors and failure patterns, which become symptoms. Short, scrubbed excerpts are retrieved only as evidence for a specific entity. **What happens if we send more traces than a mediator can process?** The mediator sheds the excess before decoding it and keeps running on a statistically significant sample. If that happens regularly, turn on the [scalable trace processor](/installation/scalable-trace-processor) or split the workload across more mediators. **Do we need one mediator for the whole company?** No, and we recommend against it. Deploy one mediator per cluster, VPC or business domain. The reasoning system combines them into one causality graph. **Do we have to replace our collectors or observability vendors?** No. Keep them. The mediator pulls from the tools you have and accepts standard OpenTelemetry, and it merges every source into one entity graph. **How much network egress should we expect?** It depends on the size of your topology, not your data volume. The continuous stream is a compressed snapshot of entities, relationships and symptom states approximately every 30 seconds, plus occasional evidence requests. **Can we run everything on our own infrastructure?** Yes. The reasoning system can run in your own cloud (bring your own cloud, BYOC). With Causely's own health telemetry turned off, nothing leaves your network. See [BYOC (Self-Hosted) Deployment](/getting-started/byoc). --- ## Start with Causely asdf this goes that way --- ## Quick Setup Guide for Causely export const CAUSELY_VERSION = versionHistory?.versions?.[0]?.imageVersion ?? versionHistory?.versions?.[0]?.version ?? ''; Get Causely running in your environment and connected to your agents in just a few minutes. This guide covers planning your telemetry setup, deploying the mediator, connecting data sources, verifying discovery, and giving AI agents access to Causely's causal reasoning via MCP. After completing this guide, you are able to use Causely with your AI agents to identify root causes of issues in your environment, enabling faster problem resolution and improved system reliability. ## Plan Your Deployment Before installing Causely, it's important to understand what telemetry sources you'll need for a successful deployment. Causely requires two types of data sources to be most effective: - [Services](#services) - [Service Connections](#service-connections) :::warning Why those two are most important for your success **Service discovery** tells Causely what exists. **Service connections** shows how services interact with each other. **Both together** provide a complete picture, enabling accurate causal inference and impact assessment. Make sure you have at least one of these sources to be most effective, or rely on the built-in eBPF-based OpenTelemetry instrumentation, which is included by default with every installation. ::: #### Services Service discovery helps Causely understand what services and infrastructure exist in your environment. These sources create entities like services, pods, containers, databases, and other infrastructure components. **You need at least one of these**: For Kubernetes, Docker and Nomad, the service discovery is enabled during the installation of the Causely mediator on those platforms. For your cloud provider you can add the corresponding integration [via the UI](https://portal.causely.app/integrations). #### Service Connections Service connections reveal how services communicate with each other. These sources create connections between services, showing dependencies, request flows, and communication patterns. This is critical for building the service dependency graph and understanding how issues propagate. **You need at least one of these**: The default Causely installation includes [OpenTelemetry with eBPF](/telemetry-sources/ebpf) instrumentation automatically. ## Deploy the Causely Mediator Causely can be installed on any platform, from virtual machines and bare metal servers to container orchestration platforms like Kubernetes, Nomad, and Docker. **Kubernetes** Navigate to the [Mediators page](https://portal.causely.app/agents) in the Causely UI and click **"Add New"** to get your personalized installation command. Follow the instructions provided, which will look similar to: {`helm upgrade --install causely \\ --create-namespace oci://us-docker.pkg.dev/public-causely/public/causely \\ --version ${CAUSELY_VERSION} \\ --namespace=causely \\ --set image.tag=${CAUSELY_VERSION} \\ --set global.cluster_name=CHANGE_ME_CLUSTER_NAME \\ --set mediator.gateway.token=YOUR_ACCESS_TOKEN`} Replace `YOUR_ACCESS_TOKEN` with your access token and `CHANGE_ME_CLUSTER_NAME` with your cluster name. The Causely mediator is installed in the `causely` namespace. To verify the installation, run: ```bash kubectl get pods -n causely ``` By default, the applications in your cluster will be instrumented automatically using OpenTelemetry eBPF instrumentation. This means that Causely can use [OpenTelemetry](/telemetry-sources/opentelemetry) traces to discover service dependencies, monitor sync and async communication signals and diagnose issues in your environment. **Nomad** For detailed instructions on installing Causely on HashiCorp Nomad, see the [Nomad installation guide](/installation/nomad). The guide covers prerequisites, configuration, and deployment steps for running Causely on Nomad clusters. **Docker** For detailed instructions on installing Causely on standalone Docker hosts, see the [Docker installation guide](/installation/docker). The guide covers prerequisites, configuration, and deployment steps for running Causely on Docker hosts outside of Kubernetes. **VM / Bare Metal** For installation on virtual machines or bare metal servers, please [reach out to Causely support](mailto:support@causely.ai) for assistance with your specific environment and requirements. ## Connect Telemetry Sources With the mediator deployed, you can now connect your planned telemetry sources to Causely. As outlined above, begin with sources that provide you with discovery of services and service connections. To detect a wide range of [Signals](/reference/symptoms/) and causes in your environment, you can configure additional data sources. Visit the [telemetry sources](/telemetry-sources/) page to learn more about the data sources that Causely supports. ## Review Discovery You have successfully installed Causely! Navigate to https://portal.causely.app to verify your environment has been discovered. You should see entities populated in the [Topology](/in-action/topology) view: If the entities and connections you are expecting to see are not appearing, go to the [Integrations](https://portal.causely.app/integrations) page and check for any errors for the telemetry sources you have configured. Errors in the integrations page will prevent Causely from discovering entities and connections in your environment. Follow the [troubleshooting steps](/telemetry-sources/troubleshooting) to resolve the errors. ## Connect Causely to AI Agents Give your agents access to Causely's causal reasoning via the MCP server. Any MCP-compatible agent or assistant including Claude Code, Cursor, VS Code, HolmesGPT, or your own custom agent, can query root causes, service health, dependency maps, and reliability reports directly. Visit the [agent integration](/agent-integration) page to get started. --- ## Supported Technologies Causely integrates with your existing infrastructure, observability stack and AI agents. This page provides a complete reference of all supported technologies across four areas: - **[Agents](/agent-integration)**: Give MCP-compatible agents like Claude Code, Cursor, and HolmesGPT deterministic causal context. - **[Installation](/installation)**: Deploy Causely's mediator in Kubernetes, Docker, or other environments - **[Telemetry Sources](/telemetry-sources)**: Connect metrics, traces, logs, and infrastructure data from platforms you already use - **[Workflows](/workflows)**: Route insights to Slack, Microsoft Teams, Grafana, and other tools your team relies on Use the search and filters below to find what you need. :::tip Missing something? [Let us know](mailto:support@causely.app). We're always expanding our integrations. :::