# Causely documentation: installation > Full content of the "installation" section of the Causely documentation. This file contains all documentation content in a single document following the llmstxt.org standard. ## Argo CD Installation export const CAUSELY_VERSION = versionHistory?.versions?.[0]?.imageVersion ?? versionHistory?.versions?.[0]?.version ?? 'latest'; This installation method enables Causely on [**Kubernetes clusters**](https://kubernetes.io/) using [**Argo CD**](https://argo-cd.readthedocs.io/) for GitOps-based deployment and management. ### Prerequisites - Kubernetes cluster running - [Argo CD](https://argo-cd.readthedocs.io/en/stable/getting_started/) installed and configured - Git repository for storing configuration (if you manage Applications via Git) - Linux kernel with eBPF support (5.4+ recommended) ### 1. Install Causely using Argo CD #### Retrieve Your Access Token 1. Visit https://portal.causely.app and log in. 2. Click `Mediators` on the side navigation. 3. Click the `Add ➕` button and copy your access token to use in the next step. #### Get the Argo CD manifests Download the Argo CD manifests and copy them to your GitOps repository: ```bash # Clone the causely-deploy repository to get the manifests git clone https://github.com/causely-oss/causely-deploy.git cd causely-deploy/kubernetes/argocd/components/applications/causely # Copy the manifests to your GitOps repository cp *.yaml /path/to/your/gitops-repo/causely/ ``` Alternatively, you can copy the files directly from the [causely-deploy repository](https://github.com/causely-oss/causely-deploy/tree/main/kubernetes/argocd/components/applications/causely) into your GitOps repository. The base [`causely.yaml`](https://github.com/causely-oss/causely-deploy/blob/main/kubernetes/argocd/components/applications/causely/causely.yaml) Application deploys the public Causely Helm chart from `us-docker.pkg.dev/public-causely/public` into the `causely` namespace with automated sync, pruning, and self-heal enabled. #### Configure your environment ##### Create the Kubernetes Secret for your token For production and GitOps workflows, store your gateway token in a Kubernetes Secret instead of committing it to Git. See [Using a Kubernetes Secret for the Access Token](/installation/customize#using-a-kubernetes-secret-for-the-access-token) for details. ```bash kubectl create namespace causely kubectl create secret generic causely-token \ --from-literal=gateway-token= \ -n causely ``` ##### Create an Argo CD AppProject The Application references Argo CD project `causely`. Create an `AppProject` that allows the Helm chart source and the `causely` namespace (and `kube-system` if your chart installs cluster-scoped resources such as PriorityClasses): {`apiVersion: argoproj.io/v1alpha1 kind: AppProject metadata: name: causely namespace: argocd spec: description: Causely agent deployment clusterResourceWhitelist: - group: '*' kind: ClusterRole - group: '*' kind: ClusterRoleBinding - group: scheduling.k8s.io kind: PriorityClass destinations: - namespace: causely server: https://kubernetes.default.svc - namespace: kube-system server: https://kubernetes.default.svc sourceRepos: - us-docker.pkg.dev/public-causely/public `} :::note AppProject name Ensure the `AppProject` metadata name matches `spec.project` in your Application (`causely`). If you use the bundled `causely-project.yaml` from causely-deploy, rename the project or update the Application so both resources use the same project name. ::: ##### Configure the Application Update `causely.yaml` for your cluster name, chart version, and token reference. The chart version should match the image tag you want to run: {`apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: causely namespace: argocd spec: project: causely destination: namespace: causely server: https://kubernetes.default.svc source: repoURL: us-docker.pkg.dev/public-causely/public chart: causely targetRevision: "${CAUSELY_VERSION}" helm: releaseName: causely parameters: - name: image.tag value: "${CAUSELY_VERSION}" - name: global.cluster_name value: - name: mediator.secretName value: causely-token ignoreDifferences: - group: apps kind: Deployment jqPathExpressions: - .spec.template.spec.containers[].image - group: apps kind: DaemonSet jqPathExpressions: - .spec.template.spec.containers[].image syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true - RespectIgnoreDifferences=true `} The `ignoreDifferences` entries tell Argo CD to ignore container image fields on Deployments and DaemonSets. That avoids endless OutOfSync states when image tags are updated outside the Application manifest (for example by the Causely Operator or a release pipeline). You can find the latest chart version in the [Causely Portal](https://portal.causely.app) under **Gear Icon → Integrations → Agents**, or use the version shown above (`${CAUSELY_VERSION}`). For additional Helm values (integrations, notifications, operator settings, and more), add a `values.yaml` next to the Application and reference it with `helm.valueFiles`, as in the [production overlay example](https://github.com/causely-oss/causely-deploy/tree/main/kubernetes/argocd/clusters/production/components/applications/causely) in causely-deploy. #### Deploy via GitOps Commit the manifests to the repository Argo CD watches, or apply them directly if you bootstrap Applications with `kubectl`: ```bash # Navigate to your GitOps repository cd /path/to/your/gitops-repo # Add the Causely configuration files git add causely/ git commit -m "Add Causely Argo CD configuration" git push origin main ``` Monitor the deployment: ```bash # Check Application sync status kubectl get application causely -n argocd # Check if Causely pods are running kubectl get pods -n causely ``` In the Argo CD UI, open the **causely** Application and confirm it is **Synced** and **Healthy**. #### Review Discovery To verify the agent is successfully installed navigate to the [Integrations Agents view](https://portal.causely.app/integrations?tab=agents). Once the agent has appeared you have successfully installed Causely! Navigate to https://portal.causely.app to verify your environment has been discovered. You should see entities populated in the `Topology` view. As Diagnoses are made, they will appear in the `Root Cause` view. ### Next Steps Visit the [Customize your Installation](/installation/customize) page to learn more about customizing your installation. We recommend that you connect additional telemetry sources to help Causely infer Diagnoses in your environment. Visit the [Telemetry Sources](/telemetry-sources/) page to learn more about the data sources that Causely supports. To push insights into your workflows, visit the [Workflow Integrations](/workflows/) page. --- ## CLI Installation ### Prerequisites Before you begin, make sure your environment meets the following requirements: - [Kubernetes](https://kubernetes.io/releases/) 1.18+ - [Helm](https://github.com/helm/helm/releases) 3.8+ :::info **Persistent Volume Access:** Causely requires the ability to create and attach PersistentVolumeClaims (PVCs) for Mediator and VictoriaMetrics storage. - Ensure your account can create PersistentVolume, PersistentVolumeClaim, and StorageClass resources. - Check that a default StorageClass is configured (kubectl get storageclass). - If dynamic provisioning is disabled, pre-create a PersistentVolume and bind it to the PVC. If you are using OpenShift, you need to use the group id from the uid-range assigned to the project. Review the section on [OpenShift Deployment](/installation/customize#deploying-on-openshift) for more details. If you are using a custom StorageClass, you can specify the StorageClass to use for persistent volumes. Review the section on [Using a custom StorageClass instead of the default one](/installation/customize#using-a-custom-storageclass-instead-of-the-default-one) for more details. ::: ### 1. Install the Causely CLI Run the following command to download and install the Causely CLI: ```bash bash -c "$(curl -fksSL https://install.causely.ai/install.sh)" ``` :::info The Causely CLI simply wraps Helm. If you prefer, you can alternatively run Helm commands directly. See [Helm Installation](/installation/helm) ::: ### 2. Install the Causely Agent #### Step 1: Retrieve Your Access Token 1. Visit https://portal.causely.app and log in. 2. Click `Mediators` on the side navigation. 3. Click the `Add ➕` button and copy your access token to use in the next step. #### Step 2: Install via CLI Run the following command by replacing `` with your access token. ```bash causely agent install --token ``` :::note If you have not installed the CLI in your PATH, make sure to run the command from absolute path, for example `~/bin/causely`. This installs Causely agents into the cluster defined by your current `kubectx` context. To specify a custom cluster name, see [CLI Options](#cli-options) below. ::: #### Step 3: Review Discovery To verify the agent is successfully installed navigate to the [Integrations Agents view](https://portal.causely.app/integrations?tab=agents). Once the agent has appeared you have successfully installed Causely! Navigate to https://portal.causely.app to verify your environment has been discovered. You should see entities populated in the `Topology` view. As Diagnoses are made, they will appear in the `Root Cause` view. ### CLI Options You can see a complete list of options for customizing the installation by using the `--help` option. ```bash causely agent --help ``` Here's a list of possible options: ```bash causely agent install --namespace --repository --tag --cluster-name --kube-context --values ``` ### Next Steps Visit the [Customize your installation](/installation/customize) page to learn more about customizing your installation. We recommend that you connect additional telemetry sources to help Causely infer Diagnoses in your environment. Visit the [Telemetry Sources](/telemetry-sources/) page to learn more about the data sources that Causely supports. To push insights into your workflows, visit the [Workflow Integrations](/workflows/) page. --- ## Customize your installation After you have installed the Causely agent, you can customize your installation to meet your needs. ### Use a Custom Values File To specify additional configurations - like a specific image tag, integrations, webhook notifications, etc - you can do this by creating a `causely-values.yaml` file. Below is an example of common values used for installation. ```yaml global: cluster_name: image: tag: # Locate latest version in the Causely Portal > Gear Icon > Integrations > Agents mediator: gateway: token: # Locate your token in the Causely Portal > Gear Icon > Integrations > Details ``` Now add the `--values` parameter to the command you used to install the Causely agent. **helm** ```bash helm upgrade --install causely \ --create-namespace oci://us-docker.pkg.dev/public-causely/public/causely \ --version \ --namespace=causely \ --values ``` **cli** ```bash causely agent install --token --values ``` ### Using a Kubernetes Secret for the Access Token Instead of passing your access token directly in the Helm values, you can store it in a Kubernetes secret and reference it using `mediator.secretName`. This approach is recommended for production environments and GitOps workflows. #### Step 1: Create the Kubernetes Secret Create a secret containing your Causely access token: ```bash kubectl create secret --namespace causely generic causely-token \ --from-literal=gateway-token= ``` :::note The secret must contain a key named `gateway-token` with your access token value. ::: #### Step 2: Reference the Secret in Your Values File Update your `causely-values.yaml` to reference the secret instead of the token directly: ```yaml mediator: secretName: causely-token # Name of the secret created above ``` When `mediator.secretName` is set, Causely will automatically read the token from the referenced secret instead of expecting it in `mediator.gateway.token`. ### Connect Additional Telemetry Sources You can add additional data sources to Causely which expands Causely's causality map to help infer Diagnoses. Some data sources require you to update your `causely-values.yaml` file. For more details on Telemetry Sources, see [Telemetry Sources](/telemetry-sources/). ### Push Insights into Your Workflows Causely can automatically send identified Diagnoses to your existing notification and observability tools. To explore integrations with Slack, Grafana, Opsgenie, and more, visit our [Workflow Integrations](/workflows/) page. You'll find setup instructions for webhooks (via `causely-values.yaml`) and details on our Grafana plugin, with additional native integrations coming soon. ### Custom Labels for Scope configuration You can [scope the Causely interface](/configuration/scopes/) to specific components out-of-the-box (clusters, namespaces, services). Causely can also use labels to provide users with additional scopes - this can be accomplished by adding the following to your `causely-values.yaml`. ```yaml label_semconv: scopes: geography: - 'app.kubernetes.io/geography' environment: - 'app.kubernetes.io/environment' customer: - 'app.kubernetes.io/customer' team: - 'app.kubernetes.io/team' product: - 'app.kubernetes.io/product' project: - 'app.kubernetes.io/project' service: - 'app.kubernetes.io/service' ``` When configured, Causely will automatically detect entities with these labels and make them available as scope options in the UI. ### Sizing the Mediator for large environments The Mediator defaults to a memory limit of 16Gi. Mediator memory scales with the **number of entities under management**: the more services, pods, and infrastructure components in your environment, the more memory the Mediator requires. It does not scale with trace or metric throughput. For why, see [How it scales](/getting-started/mediator#how-it-scales). If the Mediator pod is OOMKilled or you are running a large cluster, increase the memory limits in your `causely-values.yaml`: ```yaml mediator: resources: limits: memory: "32Gi" ``` Apply the change with: ```bash helm upgrade causely \ oci://us-docker.pkg.dev/public-causely/public/causely \ --namespace=causely \ --values ``` :::tip If the Mediator is dropping traces due to memory pressure, that is expected and intentional: it means the Mediator already has a statistically sufficient sample for analysis. An OOMKill is different: it means the Mediator process itself ran out of memory and needs more headroom. Check which situation you are in by inspecting pod events: ```bash kubectl describe pod -n causely -l app=mediator ``` ::: ### Sizing VictoriaMetrics for high-volume environments VictoriaMetrics defaults to a memory limit of 8Gi, which is sufficient for most environments. In high-volume environments, such as large clusters, high service counts, or high cardinality metrics, VictoriaMetrics may hit that ceiling, causing **gaps in metrics graphs** within Causely. If you observe missing or inconsistent metrics, increase the VictoriaMetrics memory limits in your `causely-values.yaml`: ```yaml victoriametrics: resources: limits: memory: "16Gi" ``` Apply the change with: ```bash helm upgrade causely \ oci://us-docker.pkg.dev/public-causely/public/causely \ --namespace=causely \ --values ``` :::note Metrics visible in Causely may not exactly match totals in external monitoring tools even when VictoriaMetrics is healthy. The Mediator processes a statistically significant sample of telemetry by design. See [Mediator scale behavior](/getting-started/architecture#mediator) for details. ::: ### Using an external time series database By default, Causely deploys VictoriaMetrics in-cluster to store time series data. If you already operate a Prometheus-compatible store (such as Prometheus, Thanos, Mimir, Cortex, or VictoriaMetrics), you can point Causely at it instead of running the bundled VictoriaMetrics instance. The external store must support: - **Remote write** for ingesting metrics that Causely collects. - A **PromQL query API** for Causely to read metrics back. To use an external store, set `timeseries.external.enabled=true` in your `causely-values.yaml`. The bundled VictoriaMetrics is automatically skipped when external storage is enabled, so you do not need to disable it explicitly. ```yaml timeseries: external: enabled: true # Prometheus API base URL used for both PromQL queries and remote write. endpoint: "http://prometheus-operated.monitoring.svc.cluster.local:9090" ``` When only `endpoint` is set, Causely uses it for PromQL queries and derives the remote write URL as `{endpoint}/api/v1/write`. If your store exposes the query and remote write APIs at different URLs, set them explicitly: ```yaml timeseries: external: enabled: true endpoint: "http://prometheus.monitoring.svc.cluster.local:9090" # Override the PromQL query API base URL (defaults to endpoint when empty). queryEndpoint: "http://thanos-query.monitoring.svc.cluster.local:9090" # Override the remote write URL (defaults to "{endpoint}/api/v1/write" when empty). remoteWriteEndpoint: "http://thanos-receive.monitoring.svc.cluster.local:19291/api/v1/receive" ``` #### Query concurrency When Causely fetches entity metrics, it runs PromQL queries concurrently. The default is 16, which is appropriate for most stores. Kafka-backed stores such as Grafana Mimir can take a few seconds to serve a single query, but they can answer a batch of in-flight queries together. Raise `timeseries.external.queryConcurrency` if metric graphs in Causely are slow against Mimir or a similar store: ```yaml timeseries: external: enabled: true endpoint: "http://mimir.monitoring.svc.cluster.local:8080/prometheus" queryConcurrency: 32 ``` #### Authentication If your external store requires HTTP Basic Auth, create a Kubernetes secret with `username` and `password` keys and reference it with `timeseries.external.secretName`: ```bash kubectl create secret --namespace causely generic timeseries-credentials \ --from-literal=username= \ --from-literal=password= ``` ```yaml timeseries: external: enabled: true endpoint: "http://prometheus-operated.monitoring.svc.cluster.local:9090" secretName: timeseries-credentials ``` ::: Apply the change with: ```bash helm upgrade causely \ oci://us-docker.pkg.dev/public-causely/public/causely \ --namespace=causely \ --values ``` ### Installing in a large Kubernetes cluster (>1000 nodes) If you install Causely in a large Kubernetes cluster, you can enable centralized caching of the api server responses by adding to the `causely-values.yaml` file: ``` k8sCache: enabled: true ``` ### Connecting to a remote Kubernetes cluster If you are connecting to a remote Kubernetes API server, you need to create a secret with the kubeconfig credentials: ```bash kubectl create secret --namespace causely generic kubeconfig --from-file=kubeconfig=path_to_kubeconfig ``` and include in the `causely-values.yaml` file: ``` scrapers: kubernetes: # remote k8s cluster credentials secretName: kubeconfig agent: enabled: false ``` ### Deploying on Openshift If you are deploying on Openshift, you need to use the group id from the uid-range assigned to the project: ```bash oc new-project causely oc get ns causely -o yaml|grep uid-range openshift.io/sa.scc.uid-range: 1000630000/10000 ``` and include in the `causely-values.yaml` file: ``` global: securityContext: fsGroup: 1000630000 ``` Or you can change the security context of the project to the 'anyuid' SCC ```bash oc adm policy add-scc-to-group anyuid system:serviceaccounts:causely ``` In both cases, you need to assign the `privileged` SCC to the `causely-agent` service account used by the Causely agents: ```bash oc adm policy add-scc-to-user privileged -z causely-agent -n causely ``` ### Using a custom StorageClass instead of the default one If you are deploying into a cluster, where there is no default StorageClass defined, you can specify the StorageClass to use for persistent volumes: ``` global: storageClass: ocs-storagecluster-ceph-rbd ``` Alternatively you can annotate a default StorageClass: ``` kubectl patch storageclass ocs-storagecluster-ceph-rbd -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}' ``` --- ## Standalone Docker Host Installation This installation method extends Causely beyond Kubernetes clusters to **standalone Docker hosts**, enabling telemetry collection and root‑cause analysis for services running outside Kubernetes. ### Prerequisites - Docker Engine installed on the host - Linux kernel with eBPF support (5.4+ recommended) - Ability to run privileged containers and use `pid: host` ### 1. Install the Causely Agent on a Docker Host #### Step 1: Retrieve Your Access Token 1. Visit https://portal.causely.app and log in. 2. Click `Mediators` on the side navigation. 3. Click the `Add ➕` button and copy your access token to use in the next step. #### Step 2: Download the Docker bundle via `install.sh` ```bash curl -fsSL https://raw.githubusercontent.com/causely-oss/causely-deploy/main/docker/install.sh | bash ``` This script creates a `causely-docker/` directory, downloads `docker-compose.yaml`, config files, and a `.env` file. #### Step 3: Edit environment variables Set the following environment variables in your `.env` file. Note that `` is the token retrieved in step 1 and `` is the friendly name you'd like displayed in the Causely UI. ```env CAUSELY_GATEWAY_TOKEN= DOCKER_HOST_NAME= ``` #### Step 4: Start the stack Run the following command to start the Causely stack: ```bash docker compose up -d ``` #### Step 5: Review Discovery To verify the agent is successfully installed navigate to the [Integrations Agents view](https://portal.causely.app/integrations?tab=agents). Once the agent has appeared you have successfully installed Causely! Navigate to https://portal.causely.app to verify your environment has been discovered. You should see entities populated in the `Topology` view. As Diagnoses are made, they will appear in the `Root Cause` view. ### Next Steps Visit the [Customize your Installation](/installation/customize) page to learn more about customizing your installation. We recommend that you connect additional telemetry sources to help Causely infer Diagnoses in your environment. Visit the [Telemetry Sources](/telemetry-sources/) page to learn more about the data sources that Causely supports. To push insights into your workflows, visit the [Workflow Integrations](/workflows/) page. --- ## FluxCD Installation export const CAUSELY_VERSION = versionHistory?.versions?.[0]?.imageVersion ?? versionHistory?.versions?.[0]?.version ?? 'latest'; This installation method enables Causely on [**Kubernetes clusters**](https://kubernetes.io/) using [**FluxCD**](https://fluxcd.io/) for GitOps-based deployment and management. ### Prerequisites - Kubernetes cluster running - FluxCD installed and configured - Git repository for storing configuration - Linux kernel with eBPF support (5.4+ recommended) ### 1. Install Causely using FluxCD #### Retrieve Your Access Token 1. Visit https://portal.causely.app and log in. 2. Click `Mediators` on the side navigation. 3. Click the `Add ➕` button and copy your access token to use in the next step. #### Get the FluxCD manifests Download the FluxCD manifests and copy them to your GitOps repository: ```bash # Clone the causely-deploy repository to get the manifests git clone https://github.com/causely-oss/causely-deploy.git cd causely-deploy/kubernetes/fluxcd/causely # Copy the manifests to your GitOps repository cp *.yaml /path/to/your/gitops-repo/causely/ ``` Alternatively, you can copy the files directly from the [causely-deploy repository](https://github.com/causely-oss/causely-deploy/tree/main/kubernetes/fluxcd/causely) into your GitOps repository. #### Configure your environment ##### Create the Kubernetes Secret for your token For production and GitOps workflows, store your gateway token in a Kubernetes Secret instead of committing it to Git. See [Using a Kubernetes Secret for the Access Token](/installation/customize#using-a-kubernetes-secret-for-the-access-token) for setup instructions. ##### Optionally, update the configuration files Edit the file `kustomization.yaml` to match your environment. You can optionally update the version to your desired version: {`apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - namespace.yaml - release.yaml - repository.yaml postBuild: substitute: CAUSELY_VERSION: "${CAUSELY_VERSION}" `} #### Deploy via GitOps Navigate to your GitOps repository and commit the changes: ```bash # Navigate to your GitOps repository cd /path/to/your/gitops-repo # Add the Causely configuration files git add causely/ git commit -m "Add Causely FluxCD configuration" git push origin main ``` FluxCD will automatically detect the changes and deploy Causely to your cluster. You can monitor the deployment status: ```bash # Check FluxCD reconciliation status kubectl get helmreleases -n causely # Check if Causely pods are running kubectl get pods -n causely ``` #### Review Discovery To verify the agent is successfully installed navigate to the [Integrations Agents view](https://portal.causely.app/integrations?tab=agents). Once the agent has appeared you have successfully installed Causely! Navigate to https://portal.causely.app to verify your environment has been discovered. You should see entities populated in the `Topology` view. As Diagnoses are made, they will appear in the `Root Cause` view. ### Next Steps Visit the [Customize your Installation](/installation/customize) page to learn more about customizing your installation. We recommend that you connect additional telemetry sources to help Causely infer Diagnoses in your environment. Visit the [Telemetry Sources](/telemetry-sources/) page to learn more about the data sources that Causely supports. To push insights into your workflows, visit the [Workflow Integrations](/workflows/) page. --- ## Helm Installation export const CAUSELY_VERSION = versionHistory?.versions?.[0]?.imageVersion ?? versionHistory?.versions?.[0]?.version ?? ''; ### Prerequisites Before you begin, make sure your environment meets the following requirements: - [Kubernetes](https://kubernetes.io/releases/) 1.18+ - [Helm](https://github.com/helm/helm/releases) 3.8+ :::info **Persistent Volume Access:** Causely requires the ability to create and attach PersistentVolumeClaims (PVCs) for Mediator and VictoriaMetrics storage. - Ensure your account can create PersistentVolume, PersistentVolumeClaim, and StorageClass resources. - Check that a default StorageClass is configured (kubectl get storageclass). - If dynamic provisioning is disabled, pre-create a PersistentVolume and bind it to the PVC. If you are using OpenShift, you need to use the group id from the uid-range assigned to the project. Review the section on [OpenShift Deployment](/installation/customize#deploying-on-openshift) for more details. If you are using a custom StorageClass, you can specify the StorageClass to use for persistent volumes. Review the section on [Using a custom StorageClass instead of the default one](/installation/customize#using-a-custom-storageclass-instead-of-the-default-one) for more details. ::: ### 1. Install the Causely Agent #### Step 1: Retrieve Your Access Token 1. Visit https://portal.causely.app and log in. 2. Click `Mediators` on the side navigation. 3. Click the `Add ➕` button and copy your access token to use in the next step. #### Step 2: Install via Helm Run the following command by replacing `` with your access token.\ You can find the latest version in the [Causely Portal](https://portal.causely.app) > Gear Icon > Integrations > Agents. {` export CAUSELY_TOKEN= export CAUSELY_CLUSTER_NAME= export CAUSELY_VERSION=${CAUSELY_VERSION} helm upgrade --install causely \\ --create-namespace oci://us-docker.pkg.dev/public-causely/public/causely \\ --version "${CAUSELY_VERSION}" \\ --namespace=causely \\ --set image.tag="${CAUSELY_VERSION}" \\ --set global.cluster_name="\${CAUSELY_CLUSTER_NAME}" \\ --set mediator.gateway.token="\${CAUSELY_TOKEN}" `} #### Step 3: Wait for the Causely Agent to be Ready Run the following command to wait for the Causely agent to be ready: ```bash kubectl wait --for=condition=Ready pod -l app.kubernetes.io/part-of=causely -n causely --timeout=300s ``` :::info If all pods are ready, but you still cannot see any entities in the Causely UI, please check the logs of the mediator pod for errors: ```bash kubectl logs -n causely \ -l app.kubernetes.io/name=mediator \ -c mediator \ --tail=-1 | grep ERROR ``` For example the following error indicates that the credentials are invalid: ```json { "error": "failed to send update: rpc error: code = Unknown desc = unauthorized" } ``` For other errors, please reach out to us at [support@causely.app](mailto:support@causely.app) for help. ::: #### Step 4: Review Discovery To verify the agent is successfully installed navigate to the [Integrations Agents view](https://portal.causely.app/integrations?tab=agents). Once the agent has appeared you have successfully installed Causely! Navigate to https://portal.causely.app to verify your environment has been discovered. You should see entities populated in the `Topology` view. As Diagnoses are made, they will appear in the `Root Cause` view. ### Uninstall the Causely Agent Run the following command to uninstall the Causely agent: ```bash helm uninstall causely -n causely ``` After uninstalling the Causely agent, you can delete the PVCs created by the Causely agent: ```bash kubectl delete pvc -n causely -l app=victoriametrics ``` ### Next Steps Visit the [Customize your installation](/installation/customize) page to learn more about customizing your installation. We recommend that you connect additional telemetry sources to help Causely infer Diagnoses in your environment. Visit the [Telemetry Sources](/telemetry-sources/) page to learn more about the data sources that Causely supports. To push insights into your workflows, visit the [Workflow Integrations](/workflows/) page. --- ## Network Allowlisting This page outlines the URLs that need to be allowlisted in your firewall to enable communication between your Causely deployment and the Causely SaaS platform. ### Required URLs #### For Mediator Component The mediator component requires access to the following URLs to communicate with the Causely SaaS platform: | URL | Protocol | IP Endpoint & Port | Purpose | | -------------------------- | -------- | -------------------- | ---------------------------------------- | | `https://gw.causely.app` | gRPC | `35.186.230.176:443` | Gateway for mediator communication | | `https://otlp.causely.app` | gRPC | `35.186.230.176:443` | OTLP endpoint for mediator communication | #### For End Users The end user interfaces (browser, API clients, etc.) require access to the following URLs to communicate with the Causely SaaS platform: | URL | Purpose | | ---------------------------- | ------------------------------ | | `https://portal.causely.app` | Causely Portal (web interface) | | `https://auth.causely.app` | Authentication | | `https://api.causely.app` | API access | --- ## Nomad Installation This installation method enables Causely on [**HashiCorp Nomad clusters**](https://www.nomadproject.io/), providing telemetry collection and root‑cause analysis for services running on Nomad. ### Prerequisites - HashiCorp Nomad cluster running - Consul cluster for service discovery - NFS server for persistent storage - Docker driver enabled in Nomad - Linux kernel with eBPF support (5.4+ recommended) ### 1. Install Causely on Nomad #### Step 1: Retrieve Your Access Token 1. Visit https://portal.causely.app and log in. 2. Click `Mediators` on the side navigation. 3. Click the `Add ➕` button and copy your access token to use in the next step. #### Step 2: Download the Nomad bundle ```bash git clone https://github.com/causely-oss/causely-deploy.git cd causely-deploy/nomad ``` This directory contains the Nomad job specifications and configuration files needed to run Causely on Nomad. #### Step 3: Configure your environment Set your Causely gateway token as an environment variable: ```bash export CAUSELY_GATEWAY_TOKEN="your-gateway-token-here" ``` Edit the configuration files to match your environment. You'll need to update the datacenters, NFS settings, and Causely specific variables: ```hcl name="executor.hcl" datacenters = ["dc1"] # Update to match your Nomad datacenters ``` ```hcl name="mediator.hcl" # Token can be provided via environment variable $CAUSELY_GATEWAY_TOKEN # If not set, you can specify it here: # gateway_token = "your-gateway-token-here" nfs_server = "nfs.example.com" # Update to your NFS server nfs_path = "/exported/path" # Update to your NFS path ``` ```hcl name="victoriametrics.hcl" datacenters = ["dc1"] # Update to match your Nomad datacenters nfs_server = "nfs.example.com" # Update to your NFS server nfs_path = "/exported/path" # Update to your NFS path ``` #### Step 4: Deploy the jobs Deploy the Causely jobs to your Nomad cluster: ```bash # Deploy VictoriaMetrics first nomad job run -var-file=victoriametrics.hcl victoriametrics.nomad # Deploy the mediator nomad job run -var-file=mediator.hcl mediator.nomad # Deploy the executor nomad job run -var-file=executor.hcl executor.nomad ``` #### Step 5: Review Discovery To verify the agent is successfully installed navigate to the [Integrations Agents view](https://portal.causely.app/integrations?tab=agents). Once the agent has appeared you have successfully installed Causely! Navigate to https://portal.causely.app to verify your environment has been discovered. You should see entities populated in the `Topology` view. As Diagnoses are made, they will appear in the `Root Cause` view. ### Next Steps Visit the [Customize your Installation](/installation/customize) page to learn more about customizing your installation. We recommend that you connect additional telemetry sources to help Causely infer Diagnoses in your environment. Visit the [Telemetry Sources](/telemetry-sources/) page to learn more about the data sources that Causely supports. To push insights into your workflows, visit the [Workflow Integrations](/workflows/) page. --- ## Causely Operator The Causely Operator manages version updates for Causely components running in your Kubernetes cluster. It periodically checks for version updates from the Causely backend and, when a new version is available, presents an update button in the UI. You must click the button to apply the update. ### Overview The operator runs as a Kubernetes deployment and performs the following tasks: - **Version Synchronization**: Polls the Causely Gateway every 60 seconds to check for version updates - **Update Notifications**: Surfaces an update button in the UI when a new version is available, you click it to apply the update to the `mediator` Deployment and `agent` DaemonSet - **Zero Downtime**: Uses Kubernetes rolling updates to minimize service disruption ### When to Use the Operator The operator is useful in scenarios where: - You prefer centralized version control from the Causely backend - You want a guided, one-click update experience rather than managing Helm upgrades manually :::note The operator is disabled by default. Most users manage versions through Helm upgrades or GitOps workflows. ::: ### Enabling the Operator #### Using Helm Values Add the following to your `causely-values.yaml`: ```yaml operator: enabled: true ``` #### Using --set Flag You can also enable the operator directly via the `--set` flag: ```bash helm upgrade causely \ oci://us-docker.pkg.dev/public-causely/public/causely \ --namespace=causely \ --set operator.enabled=true ``` ### Verifying the Operator After enabling, verify the operator is running: ```bash kubectl get deployment operator -n causely kubectl get pods -n causely -l app=causely-operator ``` ### How It Works 1. **Polling**: The operator periodically connects to the Causely Gateway via gRPC 2. **Version Check**: It retrieves the target version configured for your repository 3. **Update Available**: If the target version differs from the current deployment, an update button appears in the Causely UI. Clicking it triggers the update for: - The `mediator` Deployment (mediator and ML containers) - The `agent` DaemonSet (agent container) 4. **Rolling Update**: Kubernetes handles the rolling update process automatically ### Troubleshooting #### Operator Pod Not Starting Check if the mediator ConfigMap exists: ```bash kubectl get configmap mediator -n causely ``` The operator shares configuration with the mediator. Ensure the mediator is properly installed first. #### "Version is empty" in Logs This indicates no target version is configured in the Causely backend. The operator will continue polling until a version is set. #### Permission Errors The operator requires permissions to update Deployments and DaemonSets. Verify the RBAC resources are created: ```bash kubectl get clusterrole causely-operator kubectl get clusterrolebinding causely-operator ``` ### Disabling the Operator To disable the operator, set `operator.enabled: false` in your values file and upgrade: ```bash helm upgrade causely \ oci://us-docker.pkg.dev/public-causely/public/causely \ --namespace=causely \ --set operator.enabled=false ``` This removes the operator Deployment but does not affect the mediator or agent versions. --- ## Installation Overview Causely agents are lightweight components that can be installed on your servers in minutes to provide a real-time Diagnosis. This page covers what gets deployed when you install Causely, the key benefits of its distributed architecture, and how to get started with the installation. Causely works alongside your existing observability tools, processing telemetry data locally while keeping your raw data secure in your datacenter. The platform automatically discovers service dependencies, builds topology maps, and infers Diagnoses without requiring you to send terabytes of telemetry data to the cloud, resulting in dramatically lower costs and stronger privacy compared to traditional observability platforms. ### Installation Options Choose the installation method that best fits your infrastructure and deployment preferences: :::note Telemetry Sources After deployment, telemetry data can be ingested from any source. Causely's mediator listens for [OpenTelemetry Protocol (OTLP)](/telemetry-sources/opentelemetry/) data on port 4317, making it easy to send data from existing OpenTelemetry Collectors or any OTLP-compatible source. ::: ### What Gets Deployed When you install Causely, the following components are deployed in your infrastructure: - [Mediator](#mediator) - [eBPF Instrumentation](#ebpf-instrumentation) - [Executor](#executor) - [VictoriaMetrics](#victoriametrics) To learn more about how causely works, see [How Causely Works](/getting-started/how-causely-works). The relevant documentation for your security considerations can be found in the [Security](/security/) section. #### Mediator The core component that runs locally in your environment and serves as the data processing layer: - **Signal Detection**: Converts telemetry from various sources into binary signal states - **Topology Discovery**: Automatically discovers services, infrastructure, and dependencies - **Local Processing**: Keeps raw telemetry local; only distilled insights and targeted evidence are sent to the Causely backend - **OTLP Endpoint**: Listens on port 4317 for OpenTelemetry Protocol data The mediator handles secure communication with Causely's causal reasoning system. To learn what the mediator does with each type of telemetry and how it scales, see [Mediator Details](/getting-started/mediator). #### eBPF Instrumentation (Default) {#ebpf-instrumentation} Causely leverages [**OpenTelemetry eBPF instrumentation**](/telemetry-sources/ebpf), powered by [Grafana Beyla](https://grafana.com/docs/beyla/latest/), to automatically collect the following telemetry data: - Service dependencies and communication patterns - Synchronous and asynchronous communication signals - Performance metrics without code changes :::note eBPF Configuration eBPF instrumentation can be disabled if you already have comprehensive telemetry coverage. However, we recommend keeping it enabled as it serves as telemetry quality assurance, ensuring you have complete visibility into service dependencies and communication patterns. If your existing [OpenTelemetry](/telemetry-sources/opentelemetry/), [Prometheus](/telemetry-sources/prometheus/), or other telemetry sources already provide excellent coverage, you can [disable eBPF](/telemetry-sources/ebpf#configuration) to reduce overhead. ::: #### Executor The Executor is an optional component and can be enabled as part of the deployment process. The Executor is responsible for executing remediation actions. #### VictoriaMetrics VictoriaMetrics is a timeseries database used by the agents and mediator to store additional timeseries data. ### Data Sources and Integration While the components are deployed via Helm or CLI, data can be sent to the Causely mediator from multiple sources. See [all available telemetry sources](/telemetry-sources/) for a complete list, but make sure you have at least one source providing traces for the platform to be most effective. Without traces, Causely's ability to discover service dependencies and perform root cause analysis is severely limited. Any source that generates OpenTelemetry traces is perfect - Causely comes with auto-instrumentation for that, but if you already have another solution like [groundcover](/telemetry-sources/groundcover), [Odigos](/telemetry-sources/odigos), [Grafana Beyla](/telemetry-sources/grafana), an existing OpenTelemetry Collector, you can use those as well. Additionally, Causely can leverage data from [datadog](/telemetry-sources/datadog) and [dynatrace](/telemetry-sources/dynatrace) to discover service dependencies. ### Key Benefits of Causely's Architecture Because the mediator processes telemetry inside your environment, only distilled insights and targeted evidence leave it. That keeps egress small, keeps raw telemetry in your datacenter, and lets mediators run at the edge of your infrastructure. With [eBPF instrumentation](#ebpf-instrumentation), Causely also starts collecting traces without code changes. For the full picture, see [Why run the mediator in your environment](/getting-started/mediator#why-run-the-mediator-in-your-environment). ### Recommended Sizing Guidelines Below are the recommended resource requests and limits for the main Causely components as typically deployed. You may need or want to adjust these values based on your environment, workload, or scaling requirements. **Mediator** - CPU: minimum 100m, up to 2 - Memory: minimum 128Mi, up to 8Gi - Storage: 10Gi **Executor** - CPU: minimum 100m, up to 2 - Memory: minimum 128Mi, up to 8Gi - Storage: not required **VictoriaMetrics** - CPU: minimum 100m, up to 4 - Memory: minimum 128Mi, up to 8Gi - Storage: 50Gi ### Next Steps After installation, we recommend: 1. **Integrate Agents**: Give MCP-compatible agents like Claude Code, Cursor, and HolmesGPT deterministic causal context see [Agent Integration](/agent-integration) 2. **Connect Additional Data Sources**: Visit [Telemetry Sources](/telemetry-sources/) to learn about supported integrations 3. **Customize Your Setup**: Learn about advanced configuration options in [Customize Installation](/installation/customize) For a detailed explanation of how Causely works, see [How Causely Works](/getting-started/how-causely-works). --- ## Scalable Trace Processor By default, Causely processes OpenTelemetry traces inline inside the mediator. For most environments this is sufficient. When trace volume grows or you want to scale trace processing independently from the rest of the mediator, you can enable the scalable trace processor. For how this fits with the other ways to scale the mediator, see [How it scales](/getting-started/mediator#how-it-scales). ### What gets deployed Enabling the scalable trace processor adds two components to your Causely installation: - **Trace controller**: a stateless router that receives OTLP traces from your collectors and distributes work across workers. Exposes gRPC on port `4317`. - **Trace workers**: a StatefulSet of workers that analyze spans and forward results to the mediator. Workers are stateful; each one accumulates trace context for the services assigned to it. Your OTel collectors send traces to the trace controller instead of the mediator directly. The mediator continues to handle all other telemetry sources unchanged. ### When to enable it Consider enabling the scalable trace processor when: - Trace volume has grown to the point where the mediator's CPU is consistently elevated due to span processing - You want to scale trace throughput by adding worker replicas without resizing the mediator ### Enabling the scalable trace processor Add the following to your `causely-values.yaml`: ```yaml traceController: enabled: true ``` Then upgrade your Helm release: ```bash helm upgrade causely \ oci://us-docker.pkg.dev/public-causely/public/causely \ --version \ --namespace=causely \ --values ``` ### Updating your trace sources #### Beyla (eBPF auto-instrumentation) If you are using Causely's built-in Beyla instrumentation, no changes are needed. When `traceController.enabled: true`, the Helm chart automatically reconfigures Beyla to send traces to the trace controller. The `helm upgrade` will trigger a rolling restart of the agent DaemonSet to pick up the new configuration. #### OpenTelemetry Collector If you have an external OTel collector sending traces to Causely, update its exporter endpoint: ```yaml exporters: otlp/causely: endpoint: trace-controller.causely:4317 compression: none tls: insecure: true ``` Replace `mediator.causely:4317` with `trace-controller.causely:4317`. All other collector settings remain unchanged. See [Integrate OpenTelemetry Traces](/telemetry-sources/opentelemetry) for full collector configuration examples. :::info Gradual migration The mediator continues to accept traces on port `4317` even after the scalable trace processor is enabled. External collectors that have not yet been updated will continue to work via the inline path. You can migrate collectors one at a time without any gap in trace ingestion. ::: ### Sizing The defaults are suitable for most environments. Use the replica table below as a starting point based on your service count. #### Default resources | | CPU request | CPU limit | Memory request | Memory limit | |---|---|---|---|---| | Trace controller | 200m | 1000m | 128Mi | 4Gi | | Trace worker | 200m | 1000m | 256Mi | 4Gi | #### Replica starting points | Services monitored per mediator | Worker replicas | Controller replicas | |---|---|---| | < 100 | 2 | 1 | | 100–500 | 3–5 | 2 | | 500–1,000 | 5–10 | 2 | | > 1,000 | 10+ | 3+ | :::info Never run fewer than 2 worker replicas in production. A single worker replica means all trace processing stops if that pod is lost; trace context is lost until the next export. ::: To override replicas in your values file: ```yaml traceController: enabled: true replicas: 2 traceWorker: replicas: 3 ``` ### Scaling signals Monitor these four signals to know when to add replicas. | Signal | Meaning | Action | |---|---|---| | `causely_tc_inflight_rejected_total` > 0 | Controller dispatch queue is full | Check controller CPU. If high → add controller replicas. If low → add worker replicas. | | `causely_tc_dispatch_drops_total` or `causely_tc_consume_drops_total` growing | Exports being dropped | Check per-worker `causely_tw_analyze_inflight`. If one pod is high → add workers. If all pods are high → add workers. | | `causely_tw_analyze_inflight` > 2 sustained (per pod) | Work queuing inside a worker | Add worker replicas. | | Worker RSS > 75% of memory limit | Memory pressure | Raise the worker memory limit; also check whether your service count has grown significantly. | **Scale workers horizontally, not vertically.** Adding worker replicas increases parallel processing capacity. Increasing pod size does not.