Consumer Right Management

Consumer Right Management

Troubleshooting consumer groups is an important task in Apache Kafka environments. This includes identifying slow consumers, skipping a so-called "poison pill", a record that always crashes the consumer and prevents it from progressing, as well as cleaning up orphaned consumer groups that are no longer needed.
Kadeck offers a variety of functions and metrics for these tasks, which can be accessed via the Consumers view.

Right management

Access to Consumers can be managed through two approaches: a Topic-related approach and a Consumer-related approach.

Note: All rights are resource rights and can be defined for the respective resource and the respective connection.

The topic-based approach

It is often impractical to assign access rights to specific consumer groups, especially for small teams. Especially if no naming conventions for consumer groups have been defined (by the way, these can be enforced in Kadeck), granting access to specific consumer groups can be laborious and annoying.

Users who have access to a topic often need a view of the names and basic details of a topic's consumer groups.

Therefore, every user with access rights to a topic via TopicAccessRead gets additional read rights to the consuming consumer groups.

The consumer group details are limited to the topics the user has read access to. So the user does not get any information about the existence of further topics of a consumer group for which no TopicAccessRead rights exist.

To give users write or delete access to consumer groups, the ConsumerAccessWrite or ConsumerAccessDelete rights need to be assigned for the respective consumer groups.

These rights are sufficient for most use cases in smaller teams or teams that do not practice naming conventions for consumer groups.

Rights

  • TopicAccessRead
    View topic-related consumer group details; excludes topic information with no access

Unlike the Topic-related approach, the Consumer-related approach requires access rights to be granted at the consumer group level.

Note: the topic-based approach cannot be disabled. This means, users that have read access to a topic will always be able to view topic-related consumer information. However, the consumer details will be restricted to the topics the user has access to. No information about topics that the user has no read access to will be leaked.

ConsumerAccessRead, Write and Delete permissions must then be assigned to specific consumer groups. This can also be done via wildcards or simple pattern matching. ConsumerAccessDelete enables the deletion of consumer groups. ConsumerAccessWrite allows to change the offset of consumer groups.

If the user has access to a consumer group through a ConsumerAccessRead permission, the user will see all topics that the consumer group consumes from. However, the user is not able to view the data of the topics unless the user has read access via TopicAccessRead to the respective topics.

Rights

  • ConsumerAccessRead
    View full details of consumers
  • ConsumerAccessWrite
    Manage offsets of consumers the user has read access to
  • ConsumerAccessDelete
    Delete consumers the user has read access to
  • ConsumerAccessAll
    Shorthand for the above Read, Write, Delete rights

Example:

  1. {
  2.     "action": "ConsumerAccessRead",
  3.     "effect": "Allow",
  4.     "resource": "1:my_consumer_group*"
  5. }

The example above enables users to view all the details of consumer groups of connection with ID 1 that have a consumer group ID beginning with "my_consumer_group". It's important to note that all the topics from which the consumer group is consuming will be displayed. However, the data within these topics will not be visible if the user lacks the "TopicAccessRead" permission.



    • Related Articles

    • ACL Rights Management

      The rights to manage or view ACLs can be configured flexibly: from basic rights that cover all ACLs to resource-specific rights with specifications on the naming scheme. The latter allows teams to independently manage resources within their working ...

    • Advanced: LDAP Module for nested groups

      Note: the basic LDAP module is sufficient in most cases. Use only this module if the basic LDAP module does not return all groups (especially nested groups). This module was created to work around the issue where the memberOf property doesn’t return ...

    • Rights evaluation

      You can assign multiple roles with multiple rights to a user. The following rules apply to evaluate the rights in effect: First Rule: Specific right before less specific right. Second Rule: Allow before deny. Rule 1 example: Right1: Deny      ...

    • Manage rights and roles

      What are roles? Roles are a collection of rights that can be assigned either to groups or directly to users. The rights define the scope of functionality and data to which a user has access. Create roles and rights Go to the Roles screen in the ...

    • Quota Management

      A quota can be applied to limit network and resource utilization for Kafka to only specific users and/or clients. To use user level quotas, authentication via Kerberos or TLS certificates must be enabled in your Apache Kafka setup. Note that ...