Skip to content

Commit 43ba942

Browse files
authored
docs(connect): updates the deployment instructions for kafka connect (#11301)
Signed-off-by: prmellor <[email protected]>
1 parent 285ac93 commit 43ba942

File tree

2 files changed

+37
-19
lines changed

2 files changed

+37
-19
lines changed

documentation/modules/configuring/con-config-kafka-connect.adoc

+3-2
Original file line numberDiff line numberDiff line change
@@ -167,8 +167,9 @@ In this example, JSON convertors are specified.
167167
A replication factor of 3 is set for the internal topics used by Kafka Connect (minimum requirement for production environment).
168168
Changing the replication factor after the topics have been created has no effect.
169169
<6> Requests for reservation of supported resources, currently `cpu` and `memory`, and limits to specify the maximum resources that can be consumed.
170-
<7> Authentication for the Kafka Connect cluster, specified as mTLS, token-based OAuth, SASL-based SCRAM-SHA-256/SCRAM-SHA-512, or PLAIN.
171-
By default, Kafka Connect connects to Kafka brokers using a plain text connection.
170+
<7> Authentication for the Kafka Connect cluster, specified as `tls`, `scram-sha-256`, `scram-sha-512`, `plain`, or `oauth`.
171+
By default, Kafka Connect connects to Kafka brokers using a plaintext connection.
172+
For details on configuring authentication, see the link:{BookURLConfiguring}#type-KafkaConnectSpec-schema-reference[`KafkaConnectSpec` schema properties^].
172173
<8> TLS configuration for encrypted connections to the Kafka cluster, with trusted certificates stored in X.509 format within the specified secrets.
173174
<9> Build configuration properties for building a container image with connector plugins automatically.
174175
<10> (Required) Configuration of the container registry where new images are pushed.

documentation/modules/deploying/proc-deploy-kafka-connect.adoc

+34-17
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
[role="_abstract"]
99
This procedure shows how to deploy a Kafka Connect cluster to your Kubernetes cluster using the Cluster Operator.
1010

11-
A Kafka Connect cluster deployment is implemented with a configurable number of nodes (also called _workers_) that distribute the workload of connectors as _tasks_ so that the message flow is highly scalable and reliable.
11+
A Kafka Connect cluster deployment is implemented with a configurable number of nodes (also called _workers_) that distribute the workload of connectors as _tasks_, ensuring a scalable and reliable message flow.
1212

1313
The deployment uses a YAML file to provide the specification to create a `KafkaConnect` resource.
1414

@@ -17,45 +17,62 @@ In this procedure, we use the following example file:
1717

1818
* `examples/connect/kafka-connect.yaml`
1919

20-
IMPORTANT: If deploying Kafka Connect clusters to run in parallel, each instance must use unique names for internal Kafka Connect topics.
21-
To do this, xref:con-config-kafka-connect-multiple-instances-{context}[configure each Kafka Connect instance to replace the defaults].
22-
2320
.Prerequisites
2421

25-
* xref:deploying-cluster-operator-str[The Cluster Operator must be deployed.]
22+
* Cluster Operator is deployed.
23+
* Kafka cluster is running.
24+
+
25+
This procedure assumes that the Kafka cluster was deployed using Strimzi.
2626

2727
.Procedure
2828

29+
30+
. Edit the deployment file to configure connection details (if required).
31+
+
32+
In `examples/connect/kafka-connect.yaml`, add or update the following properties as needed:
33+
+
34+
* `spec.bootstrapServers` to specify the Kafka bootstrap address.
35+
* `spec.authentication` to specify the authentication type as `tls`, `scram-sha-256`, `scram-sha-512`, `plain`, or `oauth`. +
36+
See the link:{BookURLConfiguring}#type-KafkaConnectSpec-schema-reference[`KafkaConnectSpec` schema properties^] for configuration details.
37+
* `spec.tls.trustedCertificates` to configure the TLS certificate. +
38+
Use `[]` (an empty array) to trust the default Java CAs, or specify secrets containing trusted certificates. +
39+
See link:{BookURLConfiguring}#con-common-configuration-trusted-certificates-reference[`trustedCertificates` properties^] for configuration details.
40+
41+
. Configure the deployment for multiple Kafka Connect clusters (if required).
42+
+
43+
If you plan to run more than one Kafka Connect cluster in the same environment, each instance must use unique internal topic names and a unique group ID.
44+
+
45+
Update the `spec.config` properties in `kafka-connect.yaml` to replace the default values.
46+
+
47+
See xref:con-config-kafka-connect-multiple-instances-{context}[Configuring multiple Kafka Connect clusters] for details.
48+
2949
. Deploy Kafka Connect to your Kubernetes cluster.
3050
Use the `examples/connect/kafka-connect.yaml` file to deploy Kafka Connect.
3151
+
32-
[source,shell,subs="attributes+"]
52+
[source,shell]
3353
----
3454
kubectl apply -f examples/connect/kafka-connect.yaml
3555
----
3656

3757
. Check the status of the deployment:
3858
+
39-
[source,shell,subs="+quotes"]
59+
[source,shell]
4060
----
41-
kubectl get pods -n _<my_cluster_operator_namespace>_
61+
kubectl get pods -n <my_cluster_operator_namespace>
4262
----
4363
+
4464
.Output shows the deployment name and readiness
45-
[source,shell,subs="+quotes"]
65+
[source,shell]
4666
----
4767
NAME READY STATUS RESTARTS
4868
my-connect-cluster-connect-<pod_id> 1/1 Running 0
4969
----
5070
+
51-
`my-connect-cluster` is the name of the Kafka Connect cluster.
52-
+
53-
A pod ID identifies each pod created.
54-
+
55-
With the default deployment, you create a single Kafka Connect pod.
56-
+
57-
`READY` shows the number of replicas that are ready/expected.
58-
The deployment is successful when the `STATUS` displays as `Running`.
71+
In this example, `my-connect-cluster` is the name of the Kafka Connect cluster.
72+
A pod ID identifies each created pod.
73+
By default, the deployment creates a single Kafka Connect pod.
74+
`READY` shows the number of ready versus expected replicas.
75+
The deployment is successful when the `STATUS` is `Running`.
5976

6077
[role="_additional-resources"]
6178
.Additional resources

0 commit comments

Comments
 (0)