Connections and Requests

Connections and Requests

General

Connection Timeout (ms)

This applies to kafka and kinesis connections but is also used when Kadeck nodes talk to each other. It's possible that in slow networks and with a lot of data coming from the brokers that 60 secs might not be sufficient. Adjust accordingly.

Data Fetch Timeout (ms) (Currently not in use)

While the above is mostly about connecting, this one is about transferring record data from the broker.

Poll Timeout (ms)

Max time waited by the client to the broker until it gives up trying to poll "Data Poll Max Record (see below)" number of records.

Kinesis 

Shard Recurring Polling Delay (ms)

At which interval does the client poll the broker for data.

Streams Cache (secs) (Currently not in use)

This setting allowed to set a TTL to the cache of streams returned by Kinesis. Until the TTL elapses, loading the list of streams will return the value cached.

    • Related Articles

    • Using Placeholders in Apache Kafka and Amazon Kinesis connections

      When configuring connections to Apache Kafka or Amazon Kinesis, you can use environment variable placeholders to dynamically insert sensitive or environment-specific values (e.g., credentials, file paths, or hostnames). This feature is supported both ...

    • Observability with health endpoints

      Kadeck exposes health endpoints that make it possible to monitor and observe its healthiness. Endpoint Description /metrics An endpoint with metrics split into application, base and vendor. /health An endpoint with the health status for database, ...

    • Public API

      The Public API allows programmatic access to manage your data streams. This guide covers authentication, endpoints, and common use cases. Authentication All API requests require authentication using a service user API key. The key should be included ...

    • Kafka Connect Rights Management

      To view or manage Kafka Connect connectors or task, the user must have both the KafkaConnectView right and the KafkaConnectManage right. KafkaConnectView and KafkaConnectManage can be assigned for individual connections/environments. Example rights: ...

    • Quick Processor Rights Management

      To modify or create a Quick Processor, the user must have both the QuickProcessorModify right and the TopicAccessRead right. QuickProcessorModify can be assigned for individual connections/environments. In combination with the TopicAccessRead right, ...