Broker Monitoring Overview

Broker Monitoring Overview

The Cluster Monitoring Page provides comprehensive insights into the status and performance of your Apache Kafka cluster. This guide will help you understand the metrics and visualizations on this page.

Time windows:

The metrics are all relative to the selected time window on the top right (Last hour, Last day, Last week).

The following time windows can be selected:

  • Last hour: depending on the configured interval (default = 60 seconds), the most fine-grained view over the last 60 minutes. By default, 60 data points are displayed (each data point represents one minute). Th maximum value is 3600 seconds (1 hour) and that will display a single data point in all cases.
  • Last day: shows 24 data points, each represents one hour. This value cannot be changed.
  • Last week: shows 14 data points, each represents 12 hours. This value cannot be changed.

Monitoring Broker:

  • Broker Select: Choose a specific broker or view aggregated data for the full cluster.
    • Top 5 Write Activity: Selects the top 5 brokers with the highest write activity.
    • Top 5 Read Activity: Selects the top 5 brokers with the most read activity.
    • Top 5 Largest Brokers: Selects the top 5 brokers by disk space.
    • Top 5 Smallest Brokers : Selects the 5 smallest brokers by disk space.
  • Health Assistant: Indicates the overall health status of the broker.

Write and Read Metrics:

  • Written per Broker (msg/s): Displays the rate of messages written to each broker.
  • Read per Broker (msg/s): Shows the rate of messages read from each broker.
  • Total Written: Cumulative count of messages written to the cluster.
  • Total Read: Cumulative count of messages read from the cluster.

Storage Metrics:

  • Total Disk Used: Graph depicting disk space used over time.
  • Total Disk Used (GB): Shows the current total disk usage in GB.
  • Skewness: Indicates the distribution of data across brokers. A higher skew means one broker is storing more data than others. A negative skew means one broker is storing less data than others.

Replica Metrics:

  • Ratio In-Sync Replicas / Total Replicas: The percentage of in-sync replicas (ISRs) to total replicas. A value below 100% indicates some replicas are lagging.
  • Total Messages: Graph displaying the total number of messages in the cluster over time.

Broker Leader Disk:

Visual representation of the leader disk usage of each broker.

Broker Replica Disk:

Displays how replica data (messages) for different topic partitions is distributed across the replica disks of each broker.

Key Recommendations for Users:

  1. Monitor Skewness: Ensure data is balanced across brokers. High skewness may lead to performance bottlenecks.
  2. Maintain High ISRs: A low ratio of in-sync replicas can risk data loss. Aim to keep this close to 100%.
  3. Watch Disk Usage: Ensure brokers have adequate storage. High disk usage can affect performance and durability.

    • Related Articles

    • Introduction to Monitoring & Health

      We are excited to introduce Kadeck's new Monitoring Module, which makes monitoring and troubleshooting Apache Kafka infrastructure easier than ever and raises the bar in the industry. The new monitoring module includes four new pages, Monitoring ...
    • Monitoring Overview

      The Monitoring Overview page is your infrastructure command post and concisely summarizes all activity in your Apache Kafka cluster. Cluster Overview AI Health Assistant Cluster Status: Get insights on the health status of your cluster, as monitored ...
    • Topic Monitoring Overview

      This guide provides a detailed overview of the Topic Monitoring view within Kadeck's Apache Kafka UI, allowing users to monitor and analyze various metrics associated with Kafka topics. Overview Topic select: Dropdown to select specific topics or ...
    • Getting Started

      The Kadeck Team says: Thank you! Kadeck is the result of experienced developers, operations, architects and project managers from the IT field, who were looking for an easier and better way to implement, communicate and operate data streaming ...
    • Topic Overview & Documentation

      Kadeck’s Topic Overview, provides a consolidated glance at all topic details and documentation. This article will guide you through what information you can find in this view and how to document your topic. Overview Page When you open the Data ...