Release 5.X

Release 5.X

This article describes the changes of the version 5.x update.

Latest image tag: xeotek/kadeck:5.1.13

Activating the new monitoring section can lead to slightly higher memory and thread consumption. A new right is required to access the monitoring section. Learn more

Release 5.1

Kadeck 5.1 brings a brand new Alert Notifications section to Kadeck. This way you will be informed about problems in your Kafka system and can fix them before others are affected. Receive notifications in your favorite tools such as Slack, Microsoft Teams, Pagerduty or via email.

The AI-based Health Assistant has been enhanced and is now smarter than ever. Improve the availability and reliability of your Kafka system with intelligent support from Kadeck.

Security is an important component: whether for companies with 5 users or 800+ users, with version 5.1. we take Kadeck security to a new level. This is our commitment as a partner to the largest banks, critical infrastructure and logistics companies that depend on security without compromise.

Unfortunately, some bugs have crept into the gigantic Kadeck 5 release, which have now also been fixed with Kadeck 5.1.
We look forward to your feedback and hope you enjoy the new features!

Your Kadeck team

Patch notes

This section contains all patch notes for Kadeck 5.1‍

Patch 5.1.1 (Day one)

  • Adds temporary DNS lookup configuration via environment variable xeotek_kadeck_connection_mode (set to "confluent" for use_all_dns_ips lookup).
  • Incidents weren't resolved if connection details changed.

Patch 5.1.2

  • Add custom properties when creating connections for greater flexibility
  • Use all dns ips on kafka clients by default apart from SASL Kerberos
  • Bug fix for failure to file upload
  • Bug fix for NPE when ldap fetches a user with no groups
  • Bug fix concurrency issues when deserialising avro

Patch 5.1.3

  • Small custom properties fix

Patch 5.1.4

  • Fixes schemas deleted outside Kadeck were still visible in Kadeck
  • Fixes calculation of in sync replicas count
  • Fixes incident being generated of Apicurio schema registries being down when it wasn't

Patch 5.1.5

  1. Schemas named <topic>-key and <topic>-value will show up in the data catalog and topic details automatically
  2. Topic, schemas, consumers, etc.. can now be open in separate windows and/or tabs
  3. Fixes floating windows on linux showing in the wrong monitor
  4. Fixes floating windows lingering after main windows is closed
  5. Fixes auto-restart entries not being cleaned up after a connect cluster is removed
  6. Fixes partition selection drop-down in Data Browser

Patch 5.1.6

  1. Added feature to import and export topics
  2. Added feature to disable Hazelcast metrics
  3. Enhanced exception catching on consumer synchronizer
  4. Added restrictions to disallow roles with the same name
  5. Added restrictions to disallow groups with the same name
  6. Fixed the decimal places in sync replicas
  7. Resolved scrolling issue on Kafka Connect screen

Patch 5.1.7

  1. Fixes consumer not showing up on consumer page in certain scenarios.
  2. ​Fixes in-sync calculation.

Patch 5.1.8

  1. New JMX metrics for the monitoring section:

    1. Bytes In (MB/Sec)

    2. Bytes Out (MB/Sec)

    3. Producers Req Rate (/s) In

    4. Consumers Req Rate (/s) Out

Patch 5.1.9

  1. Fixes missing top padding on some cards
  2. Fixes consumer groups not being shown on refresh

Patch 5.1.10

  1. Improves consumer groups performance.
  2. Fixes missing schema registry user info for some codecs and connection test.
  3. Adds xeotek_kadeck_monitoring_mode configuration to limit the number of concurrent monitoring scans per connection to 1 when set to "flex". Defaults to "strict".

Patch 5.1.11

  1. Improves wording & fixes grammar on the UI.
  2. Upgrades vulnerable dependencies.

Patch 5.1.12

  1. Fixes scrollbars
  2. Adds resilience for consumer groups & data browser
  3. Adds resilience for monitoring
  4. Fixes consumer group visibility for restricted users
  5. Adds codec information to key and value table headers in data browser
  6. Improves data browser UX
  7. Improves create topic dialog UX
  8. Improves colors
  9. Adds "open in data catalog" option to data browser
  10. Fixes hidden multi-select status bar on consumer groups page
  11. Fixes missing header bar on Kadeck Desktop floating windows on some pages

Patch 5.1.13

  1. Fixed a JsonParsing error on the consumer groups page, which previously rendered Kadeck inoperative for some users.
  2. Resolved a ConcurrentModification error in Kadeck Teams that could sporadically occur during the synchronization of monitoring data.
  3. Improved memory efficiency in Kadeck Teams by optimizing the serialization process for monitoring data.
  4. Corrected an issue where filtering by time would omit the latest record.
  5. Fixed an issue where expanded rows were not rendered correctly in tables on Firefox.
  6. Addressed a rendering issue with upgrade icons across the platform.

‍

New features

  1. Added a new alert notification section with integrations for Slack, Microsoft Teams, Pagerduty and email.
  2. Improved Health Assistant to detect more issues.
  3. Added incidents to the Health Assistant in combination with alerts.
  4. Show more data in the data catalog with additional columns such as record count, retention size, retention period and retention policy.

Security improvements

  1. Resets log in form on password failures
  2. Brute force password discovery protection
  3. Tighten up CORS
  4. More secure content security policy
  5. Stricter content-type API policy
  6. Improved cross site request forgery protection
  7. Updates dependency version to remove security vulnerabilities
  8. Bug fix for disk space calculations in metrics
  9. Bug Fix for topic metrics calculations that were generating NPEs
  10. Bug Fix for SSL issues with the schema registry
  11. Bug Fix for ‘search from the beginning’ button on empty data browser search
  12. Bug Fix for content on empty data browser search

Bug fixes & improvements

  • Bug fix for disk space calculations in metrics.
  • Bug Fix for topic metrics calculations that were generating NPEs.
  • Bug Fix for SSL issues with the schema registry.
  • Start a search from the beginning of the stream with one click.



Release 5

Find out all about our new intelligent Kafka ops platform here: Kadeck - Intelligent Kafka Ops.
As a team, there is nothing more exciting than releasing something truly new and revolutionary.
Kadeck 5.0 revolutionizes Apache Kafka Monitoring in many ways and raises the bar in the industry even higher: Kadeck 5 comes with a groundbreaking new Kafka Monitoring section and the industry's first AI-based Health Assistant. This release is all about the Kafka infrastructure.
We, the team behind Kadeck, hope that you have as much fun with the new version as we had while working on it.

AI-Powered Kafka Health Assistant

Kadeck 5.0 introduces a groundbreaking AI-powered Kafka Health Assistant. Whether you are new to Kafka or an experienced pro, the AI Assistant will help keep your Kafka infrastructure operating smoothly 24 x 7. It continuously assesses the health of your Kafka clusters, topics, consumers, and partitions and automatically recommends changes that will improve performance and avoid downtime.

Kafka monitoring section

Kadeck 5.0 also introduces a Kafka monitoring module. The Kafka monitoring module contains three new dashboards: Monitoring Overview, Broker monitoring and Topic monitoring that visualize real-time and historic Kafka key performance metrics such as:
  1. I/O
  2. Throughput
  3. Storage
  4. Replicas
  5. Consumer group status
  6. Partition and leader skew
  7. And many more
The Kafka Monitoring module is the perfect complement to Kadeck. It makes Kadeck the industry’s most complete Kafka performance solution, allowing you to not only monitor, but also analyze and troubleshoot Kafka performance quickly and confidently from a single platform.

Consumer Monitoring Enhancements

Kadeck contains many features for analyzing and reconfiguring Kafka consumers and consumer groups. Kadeck 5.0 introduces a new ability to view consumers and consumer lag grouped by topic. In this view, the total lag of all consumer groups is aggregated for each topic, which makes it easy to see whether certain topics are experiencing lag problems.
The new Healt
h Assistant monitors your consumer groups and notifies you, if, for example, a consumer group is not consuming from all partitions of a topic which can lead to data loss.
Kafka Connect auto-restart
To make it easier for infrastructure and operations teams to ensure higher uptime, the new Health Assistant can monitor and restart crashed Kafka Connect tasks. This helps ensure data streaming continuity 24x7 with less manual oversight.

Other Kadeck Enhancements

  1. Navigate records in the data browser using the arrow keys
  2. Topic details view no longer closes if you click a record twice
  3. Protobuf Codec improvements
  4. Table shows total items now
  5. Security improvements
  6. The consumer rights are now more fine-granular. Read our article about the new consumer right management.
  7. The drop-down list box in the data browser has been replaced with the Maximum Results drop-down list box and no longer limits the number of records scanned, but the actual records returned. This has an impact when a search or filter is applied.

Patch notes

This section contains all patch notes for Kadeck 5

Version 5.0.1

Features
  1. Added UTC option: toggle between local time and UTC on monitoring pages. The option is accessible through the new drop-down on the top right on each monitoring page.
  2. Add baseline adjustment option to zoom into diagrams using the min value as the "zero" baseline.  The option is accessible through the new drop-down on the top right on each monitoring page.
  3. Force update: Force the collection of data outside the interval. The option is accessible through the new drop-down on the top right on each monitoring page.
  4. Fullscreen mode for charts.
Security
  • Security: Updated dependencies
    • CVE-2022-25647  com.google.code.gson:gson     2.8.6  -> 2.8.9
    • CVE-2022-31197  org.postgresql:postgresql     42.4.0 -> 42.4.1
    • CVE-2022-36944  org.scala-lang:scala-library  2.13.1 -> 2.13.9
Fixes
  • Connection switch on monitoring pages sometimes disappeared
  • Totals weren't visible on tree maps
  • Y-labels got cut off on windows
  • Codecs were not reloaded when connection details change

Patch 5.0.2

Features
  1. The selected time window on the monitoring pages is now retained when switching pages.
Fixes
  1. Delta on monitoring pages is now calculated correctly depending on the selected time window.
  2. Errors when migrating older versions to 5.0 have now been fixed.
  3. Long bootstrap server entries are now supported.
  4. The diagrams in full screen mode now take up the full vertical height so that details are easier to see.

Patch 5.0.3

Features
  • Data Browser: Filter component now suggests available object attributes for filtering, accelerating the process of adding filters for attributes.
  • Connectors, including inactive ones, are now visible in cluster monitoring.
  • Monitoring: the time when the URP incident occurred is now displayed
Fixes
  • Monitoring: Resolved an issue where missing producer and consumer data erroneously displayed as NaN; it now correctly shows as 0.
  • Monitoring: message count was not accurate in certain scenarios
  • Monitoring: improves the precision of smaller numbers (more digits are shown)
  • Consumer groups: change offset dialog was not correctly showing the topic names
  • Migration: Kadeck startup and database migration are now robust against paths containing non-standard characters and spaces.
  • Improved user interface: cursor keys now scroll selections without affecting tables under the selection overlay.

Patch 5.0.4

Fixes
  • Consumer groups: resetting consumer group based on a date is now working again.
  • Improved user interface: improved light mode

Patch 5.0.5 (Hotfix)

Fixes
  • Data browser: fixes the endless spinning when using Avro or custom codecs.

Patch 5.0.6

Features
  • New UI! Experience our new ultra-dark mode with a sleek, modern twist! :-)
  • New time distribution chart! Our new time distribution chart visualizes message volumes over time with more details.
Fixes
  • Monitoring: the broker count displayed on monitoring and broker overview was not correct.
  • Monitoring: the Kafka API sometimes returned null values which caused the monitoring scan to fail.
  • Consumer groups: gathering the offsets took longer due to a change in the API - fixed now.
  • Topic details: if a topic has a compact and delete retention policy, the compaction is now displayed correctly.
  • General: sometimes the log level setting had no effect.

Patch 5.0.7 (Hotfix - Desktop only)

Fixes
  • Fixes a problem that prevented migration from older Kadeck Desktop versions to Kadeck Desktop 5. 

    • Related Articles

    • Release 4.3

      This article describes the changes of the version 4.3.x update. New image tag: xeotek/kadeck:4.3.7 Kadeck Teams is no longer run as root as of version 4.3.5. Instead, a separate user kadeck is used. Depending on the container service, this requires ...

    • Release 4.2.9

      This article describes the changes of the version 4.2.x update. Note: We have released a 4.2.9 hotfix that solves a performance problem in connection with the schema registry, which led to an extremely long connection process or missing schemas for ...

    • General upgrade information

      The process of upgrading a single, distributed, or clustered Kadeck Teams instance is straightforward. In many cases, you upgrade the software by installing the latest distribution over your existing installation. The following requirements must be ...

    • Startup parameters for Desktop

      You can pass any parameter to Kadeck's Java backend by adding them to the KA_DECK_OPTS environment variable. Windows cmd /c "SET KA_DECK_OPTS='-Xmx6G'&KaDeck.exe MacOS launchctl setenv KA_DECK_OPTS "-Xms6G" && open Kadeck.app/ Linux ...

    • Increase available memory

      To make Kadeck use all the available memory (min. 8GB recommended), you need to configure the JVM. To increase the memory available to the Java process, use the typical parameters -Xmx for the maximum and -Xms for the initial memory. To pass these ...