Local cluster

Local cluster

This article describes how to set up and connect to the local cluster with KaDeck.

What is the local cluster functionality?

The local cluster, previously known as "embedded cluster", allows you to start up an Apache Kafka broker (and Zookeeper) directly in Kadeck. By "instrumenting" a standard Apache Kafka installation, no flexibility is lost. It is still possible to manually configure all broker settings (Kafka and Zookeeper) and to choose the Apache Kafka version you want.

Setup

Head over to the Apache Kafka project page and download the binary version of Apache Kafka. Un-tar the downloaded folder and move it to a location that is accessible for Kadeck. 

For Windows users: as there are some constraints on path lengths on Windows which can result in strange behavior and weird error messages, we recommend placing the folders content close to the root of your hard drive (e.g. C:/kafka/ so that the path to /bin is C:/kafka/bin).

  1. Next, start Kadeck and click on the "settings" icon of the "Local cluster" card.  Enter the path to your extracted folder's content (e.g. C:/kafka/) in the dialog.
  2. Click on "Start" and wait several seconds. The starting process can take a while, depending on your machine.
  3. If an error occurred, check the log files in your folder (e.g. C:/kafka/zookeeper.out).
Note: Clicking on "Start" does kill any previous Kafka broker and Zookeeper instances first, deletes the data dir and log dir and regularly starts both servers.

FAQ

Why are all topics (or data) gone after restarting my embedded cluster?

Kadeck deletes all information from previous sessions once the embedded cluster is started. This behavior guarantees that you are starting with a clean environment every time and the cluster does not get into an error state.

What happens if I close Kadeck without stopping the cluster?

The cluster is still running in the background. Open Kadeck again and click on "Start" on the embedded cluster card. This will stop any instances (if existent) and start the server again. You can then stop the server regularly by clicking "Stop". 

How can I change the Kafka or Zookeeper settings?

Edit the server.properties and zookeeper.properties inside the /config folder to customize the settings.

 

    • Related Articles

    • Configure a Kadeck cluster setup

      This article describes how to configure multiple Kadeck instances that run as a cluster. Define master nodes First define a list of master nodes. In this example, we will configure 3 Kadeck instances running as a minimum (to handle 2 simultaneous ...

    • Horizontal Scaling

      Architecture of a single node To scale horizontally just add servers behind a load balancer (with sticky sessions active). Don't forget to update the cluster configuration Startup Parameters when starting Kadeck on each server.

    • Installations & Environments

      An installation refers to a Kadeck instance or multiple Kadeck instances that share a database (e.g., PostgreSQL) and run in a cluster configuration (with upstream load balancer). Multiple Apache Kafka clusters One Kadeck installation can manage ...

    • Try out Kadeck Teams on a local machine

      Replace {latest version} with the actual latest released version. Getthe Kadeck Teams docker image via docker from our official repository, using: docker pull xeotek/kadeck:{latest version} Free version Use the following command to run Kadeck Teams ...

    • Startup Parameters / Environment variables

      The following list of startup parameters needs to be supplied through environment variables to the Kadeck Teams instance. Prefix "xeotek_kadeck_" required All values in the table need to be prefixed with 'xeotek_kadeck_', e.g. xeotek_kadeck_secret. ...