Skip to main content

Data Pipeline

Series of processes that move and transform data between systems for analysis, storage, or further processing.

Root Causes​

HTTP Path​

RPC Method​

Database Table​

HTTP Path​

Congested​

The HTTP path is experiencing congestion, resulting in high latency for clients. This suggests that the system is unable to handle the current load efficiently, causing delays in response times.
Congestion often occurs when the service receives more requests than it can handle within its capacity, leading to bottlenecks in processing. This may be due to insufficient resources (for example, CPU, memory, or bandwidth), unoptimized code, or a surge in traffic (for example, due to a sudden increase in demand or DDoS attack).

Malfunction​

The HTTP path is experiencing a high rate of errors, causing disruptions for clients. This can lead to degraded performance, failed requests, or complete service unavailability, significantly affecting the user experience.

RPC Method​

Congested​

The RPC method is experiencing congestion, resulting in high latency for clients. This suggests that the system is unable to handle the current load efficiently, causing delays in response times.
Congestion often occurs when the service receives more requests than it can handle within its capacity, leading to bottlenecks in processing. This may be due to insufficient resources (for example, CPU, memory, or bandwidth), unoptimized code, or a surge in traffic (for example, due to a sudden increase in demand or DDoS attack).

Malfunction​

The RPC Method is experiencing a high rate of errors, causing disruptions for clients. This can lead to degraded performance, failed requests, or complete service unavailability, significantly affecting the user experience.

Database Table​

Database tables are critical components of any data-driven application, and their performance issues can have widespread impacts across your system. Causely helps identify several types of database table impacting root causes that can affect various database systems including MySQL and PostgreSQL.

Malfunction​

The database table is experiencing performance degradation or errors, causing disruptions for client applications. This results in slow query response times, potential errors, and degraded service performance for systems that depend on this table.

  • High query duration: The table's queries are taking significantly longer than normal to execute, often due to inefficient query patterns, lack of proper indexing, or database resource constraints.
  • Database engine issues: The underlying database server might be experiencing problems that affect this specific table's performance.
  • Schema design issues: Poor table design, missing indexes, or improper constraints might be causing performance problems.
  • Data growth: The table may have grown too large for its current configuration, causing slower query execution.

Excessive Lock​

The database table is experiencing an unusually high rate of Data Dictionary Lock(DDL), which block both read and write operations during schema modifications. This significantly impacts all client applications, causing service disruptions and performance degradation.

DDL Excessive Locking occurs when database schema changes (like ALTER TABLE, CREATE INDEX) are holding exclusive locks on the table for extended periods. This issue is particularly severe because:

  • Complete access blocking: Unlike regular DML locks which may allow some concurrent operations, DDL locks typically block all access to the affected table.
  • Cascading effects: DDL locks on a table can also affect dependent tables and objects, creating widespread performance issues.
  • Extended duration: Schema changes on large tables can take significant time to complete, extending the lock duration.
  • Application timeout: Client applications may time out while waiting for access to the locked table, resulting in errors and failed operations.
  • Resource consumption: Blocked transactions can accumulate, consuming database resources and potentially causing deadlocks.

DDL Excessive Lock​

The database table is experiencing an unusually high rate of Data Dictionary Lock(DDL), which block both read and write operations during schema modifications. This significantly impacts all client applications, causing service disruptions and performance degradation.