The Cluster Monitoring Page provides comprehensive insights into the status and performance of your Apache Kafka cluster. This guide will help you understand the metrics and visualizations on this page.
Time windows:
The metrics are all relative to the selected time window on the top right (Last hour, Last day, Last week).
The following time windows can be selected:
- Last hour: depending on the configured interval (default = 60 seconds), the most fine-grained view over the last 60 minutes. By default, 60 data points are displayed (each data point represents one minute). Th maximum value is 3600 seconds (1 hour) and that will display a single data point in all cases.
- Last day: shows 24 data points, each represents one hour. This value cannot be changed.
- Last week: shows 14 data points, each represents 12 hours. This value cannot be changed.
Monitoring Broker:
- Broker Select: Choose a specific broker or view aggregated data for the full cluster.
- Top 5 Write Activity: Selects the top 5 brokers with the highest write activity.
- Top 5 Read Activity: Selects the top 5 brokers with the most read activity.
- Top 5 Largest Brokers: Selects the top 5 brokers by disk space.
- Top 5 Smallest Brokers : Selects the 5 smallest brokers by disk space.
- Health Assistant: Indicates the overall health status of the broker.
Write and Read Metrics:
- Written per Broker (msg/s): Displays the rate of messages written to each broker.
- Read per Broker (msg/s): Shows the rate of messages read from each broker.
- Total Written: Cumulative count of messages written to the cluster.
- Total Read: Cumulative count of messages read from the cluster.
Storage Metrics:
- Total Disk Used: Graph depicting disk space used over time.
- Total Disk Used (GB): Shows the current total disk usage in GB.
- Skewness: Indicates the distribution of data across brokers. A higher skew means one broker is storing more data than others. A negative skew means one broker is storing less data than others.
Replica Metrics:
- Ratio In-Sync Replicas / Total Replicas: The percentage of in-sync replicas (ISRs) to total replicas. A value below 100% indicates some replicas are lagging.
- Total Messages: Graph displaying the total number of messages in the cluster over time.
Broker Leader Disk:
Visual representation of the leader disk usage of each broker.
Broker Replica Disk:
Displays how replica data (messages) for different topic partitions is distributed across the replica disks of each broker.
Key Recommendations for Users:
- Monitor Skewness: Ensure data is balanced across brokers. High skewness may lead to performance bottlenecks.
- Maintain High ISRs: A low ratio of in-sync replicas can risk data loss. Aim to keep this close to 100%.
- Watch Disk Usage: Ensure brokers have adequate storage. High disk usage can affect performance and durability.