Kafka Connect: Not authorized to access topics, using scram-sha-512 authentication and simple authorization

[stitch-marinus]

Hi,

I've been trying to setup a simple Kraft deployment for testing but ran into an authorisation error on KafkaConnect:

2025-04-16 11:32:34,088 ERROR [Worker clientId=connect-local-connect-connect-0.local-connect-connect.kafka.svc:8083, groupId=local-connect-connect] Uncaught exception in herder work thread, exiting:  (org.apache.kafka.connect.runtime.distributed.DistributedHerder) [DistributedHerder-connect-local-connect-connect-0.local-connect-connect.kafka.svc:8083-1]
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [local-connect-offsets]

My configuration is as follows:
Kafka:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  annotations:
    strimzi.io/kraft: enabled
    strimzi.io/node-pools: enabled
  labels:
    name: local
    namespace: kafka
spec:
  entityOperator:
    topicOperator:
      reconciliationIntervalMs: 1000
      watchedNamespace: kafka
    userOperator:
      reconciliationIntervalMs: 1000
      watchedNamespace: kafka
  kafka:
    authorization:
      type: simple
    config: {}
    listeners:
      - name: plain
        port: 9092
        tls: false
        type: internal
      - authentication:
          type: scram-sha-512
        name: tls
        port: 9093
        tls: true
        type: internal
      - name: external
        port: 9094
        tls: false
        type: nodeport
    replicas: 1

User: (this should have super user privileges)

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  labels:
    strimzi.io/cluster: local
  name: kafka-connect
  namespace: kafka
spec:
  authentication:
    type: scram-sha-512

KafkaConnect:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  annotations:
    strimzi.io/use-connector-resources: 'true'
  name: local-connect
  namespace: kafka
spec:
  authentication:
    passwordSecret:
      password: password
      secretName: kafka-connect
    type: scram-sha-512
    username: kafka-connect
  bootstrapServers: local-kafka-bootstrap:9093
  config:
    config.providers: file
    config.providers.file.class: org.apache.kafka.common.config.provider.FileConfigProvider
    config.storage.replication.factor: 3
    config.storage.topic: local-connect-configs
    group.id: local-connect-connect
    offset.storage.replication.factor: 3
    offset.storage.topic: local-connect-offsets
    status.storage.replication.factor: 3
    status.storage.topic: local-connect-status
  image: kafka-connect:local
  replicas: 1
  tls:
    trustedCertificates:
      - pattern: '*.crt'
        secretName: local-cluster-ca-cert

Is there something I'm missing? I can't spot it.

[im-konge]

Hey, from the Kafka and KafkaUser CRs I don't see where the kafka-connect user is configured as super user. You would need to configure it in the Kafka CR, in authorization section.

[im-konge]

 authorization:
      type: simple
      superUsers:
        - kafka-connect

you should have something like this in Kafka spec. And create the KafkaUser kafka-connect with SCRAM-SHA.

[stitch-marinus]

I'll give it a go, but any idea why the specific authorisation would not work? I feel like I'm not seeing something obvious.

Doing that I got some new logs on the kafka cluster

2025-04-16 13:35:41,063 INFO Principal = User:kafka-connect is Denied operation = CREATE from host = 10.42.0.88 on resource = Topic:LITERAL:local-connect-offsets for request = CreateTopics with resourceRefCount = 1 based on rule DefaultDeny (kafka.authorizer.logger) [data-plane-kafka-request-handler-6]
2025-04-16 13:35:41,068 INFO Principal = User:kafka-connect is Denied operation = DESCRIBE_CONFIGS from host = 10.42.0.88 on resource = Topic:LITERAL:local-connect-offsets for request = DescribeConfigs with resourceRefCount = 1 based on rule DefaultDeny (kafka.authorizer.logger) [data-plane-kafka-request-handler-3]

I added acls for the kafka-user.

authentication: 
    type: scram-sha-512
    authorization:
      type: simple
      acls:
        - type: allow
          resource:
            type: topic
            name: "*" 
          operations:
            - READ
            - WRITE 
            - CREATE 
            - DESCRIBE 
            - DESCRIBE_CONFIGS

But the original error is still there in KafkaConnect.

[im-konge]

You would have to check the logs of User Operator (inside Entity Operator Pod).
This is example of KafkaUser with authz - https://github.com/strimzi/strimzi-kafka-operator/blob/main/examples/user/kafka-user.yaml.
I guess that the upper case operations list is maybe wrong + I don't see the type field there. But yeah, more info would have the KafkaUser resource in its .status section or User Operator.

[stitch-marinus]

I deployed the:
https://github.com/strimzi/strimzi-kafka-operator/tree/main/examples/security/scram-sha-512-auth

example and replace the zookeeper cluster with an ephemeral kraft cluster from:

https://github.com/strimzi/strimzi-kafka-operator/blob/main/examples/kafka/kraft/kafka-ephemeral.yaml

Seems to work, fine. Only real difference is the entity operator. Maybe I made a spelling mistake somewhere. Thanks for the help.

[im-konge]

Glad that it works now :)