Connect to Amazon MSK

Connect to Amazon MSK

This article describes how to connect Kadeck Teams and Kadeck Desktop to Amazon MSK.

Important note: Connecting to Amazon MSK with Kadeck Teams is straight forward if Kadeck Teams is deployed inside the same VPC (e.g., as EC2 or ECS instance). Learn how to deploy Kadeck Teams using Amazon Elastic Cloud Service (ECS).


Kadeck Desktop (and other local clients) are able to connect to Amazon MSK over the internet with the "public access" feature of AWS (documentation, announcement) or by using a third-party proxy. A list of recommended proxies can be found in the last section of this article.

 

Connect to Amazon MSK with Kadeck using IAM

To connect to Amazon MSK with Kadeck using IAM, create a new connection in Kadeck's connection overview page. 

  1. Specify the address of your broker.
  2. Select SASL_SSL as the security protocol in the Security & Authentication section of the Broker Configuration tab.
  3. Enter the SASL JAAS Config string. This looks something like this:
    software.amazon.msk.auth.iam.IAMLoginModule required awsProfileName="myProfile";
  4. Select AWS_MSK_IAM from the SASL mechanism drop-down box.

Amazon MSK Policies

This article from AWS covers how to configure permissions to Apache Kafka actions with IAM.
Your user needs the permission to execute the following actions in Apache Kafka:
  1. ListTopics, DescribeTopics, ListConsumerGroups

Proxies for Amazon MSK

Since November 2021 (AWS Announcement), it is possible to connect from the internet directly to Amazon MSK using the newly introduced "public access" feature:

Public Access on AWS Documentation

Additionaly, proxies can be used instead. This is a list of third-party proxies that allow local clients to connect to Amazon MSK over the internet.

 


    • Related Articles

    • Connect to Kinesis

      In this article, you will learn how to configure Amazon Web Services (AWS) and how to establish a connection from Kadeck to Amazon Kinesis. The article consists of two sections that build on each other: Create a user in Amazon Kinesis Add a ...

    • Deploy Kadeck using Amazon Elastic Container Service (ECS)

      This article describes how to set up a container running Kadeck Teams using Amazon Elastic Container Service (ECS). Please log into your AWS console. A user with permissions to create containers in Amazon ECS is required. 1. Create a task definition ...

    • Using Placeholders in Apache Kafka and Amazon Kinesis connections

      When configuring connections to Apache Kafka or Amazon Kinesis, you can use environment variable placeholders to dynamically insert sensitive or environment-specific values (e.g., credentials, file paths, or hostnames). This feature is supported both ...

    • Kafka Connect Auto Restart

      Introducing Kafka Connect Auto-Restart in Kadeck. To make life easier for infrastructure and operations teams running Kafka Connect connectors and to ensure higher uptime, it is now possible to have Kadeck automatically restart crashed Kafka Connect ...

    • Kafka Connect Rights Management

      To view or manage Kafka Connect connectors or task, the user must have both the KafkaConnectView right and the KafkaConnectManage right. KafkaConnectView and KafkaConnectManage can be assigned for individual connections/environments. Example rights: ...