(It's a) Kind and Strimzi

I have a Kafka cluster running on my laptop in Kind K8s using Strimzi. It just works [certain terms and conditions apply]!

For installing Strimzi etc, refer to a previous blog post of mine.

Then, I deployed kafka-with-dual-role-nodes.yaml with a few minor changes. 

First I changed the cluster name to kafka. Then, I wanted to use ephemeral disks as I didn't care about data loss in a PoC running on my PC:

-        type: persistent-claim

-        size: 100Gi

-        deleteClaim: false

+        type: ephemeral

But the main thing I had to do was create an external nodeport:

+      - name: external

+        port: 9094

+        type: nodeport   # 👈 important

+        tls: false

This meant I could see the service exposing the port to the host:

$ kubectl get svc -n kafka --output=wide

NAME                             TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                        AGE     SELECTOR

kafka-dual-role-0                NodePort    10.96.101.127   <none>        9094:31904/TCP                                 4m43s   statefulset.kubernetes.io/pod-name=kafka-dual-role-0,strimzi.io/cluster=kafka,strimzi.io/kind=Kafka,strimzi.io/name=kafka-kafka,strimzi.io/pool-name=dual-role

kafka-dual-role-1                NodePort    10.96.128.155   <none>        9094:30402/TCP                                 4m43s   statefulset.kubernetes.io/pod-name=kafka-dual-role-1,strimzi.io/cluster=kafka,strimzi.io/kind=Kafka,strimzi.io/name=kafka-kafka,strimzi.io/pool-name=dual-role

kafka-dual-role-2                NodePort    10.96.169.99    <none>        9094:31118/TCP                                 4m43s   statefulset.kubernetes.io/pod-name=kafka-dual-role-2,strimzi.io/cluster=kafka,strimzi.io/kind=Kafka,strimzi.io/name=kafka-kafka,strimzi.io/pool-name=dual-role

It's essentially port forwarding for me.

Note that one does not connect to the CLUSTER-IP. You need to see where these kafka-dual-role-* pods live:

$ kubectl get pods -n kafka --output=wide

NAME                                        READY   STATUS    RESTARTS      AGE   IP            NODE                 NOMINATED NODE   READINESS GATES

kafka-dual-role-0                           1/1     Running   0             22h   10.244.0.38   kind-control-plane   <none>           <none>

kafka-dual-role-1                           1/1     Running   0             22h   10.244.0.39   kind-control-plane   <none>           <none>

kafka-dual-role-2                           1/1     Running   0             22h   10.244.0.40   kind-control-plane   <none>           <none>

Ah, kind-control-plane. Which IP does that have?

$ kubectl get nodes --output=wide

NAME                 STATUS   ROLES           AGE    VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION     CONTAINER-RUNTIME

kind-control-plane   Ready    control-plane   139d   v1.32.3   172.18.0.2    <none>        Debian GNU/Linux 12 (bookworm)   6.8.0-60-generic   containerd://2.0.3

$ ./kafka-topics.sh --bootstrap-server=172.18.0.2:31118 --list

$

(Which is expected as we haven't created any topics yet.)

"NodePort is a Kubernetes Service type designed to make Pods reachable from a port available on the host machine, the worker node.  The first thing to understand is that NodePort Services allow us to access a Pod running on a Kubernetes node, on a port of the node itself. After you expose Pods using the NodePort type Service, you’ll be able to reach the Pods by getting the IP address of the node and the port of the NodePort Service, such as <node_ip_address>:<node port>.  The port can be declared in your YAML declaration or can be randomly assigned by Kubernetes.  Most of the time, the NodePort Service is used as an entry point to your Kubernetes cluster." [The Kubernetes Bible]

So, the port from the svc and the IP address from the nodes.

Aside: one nice thing about Kind is that I can take my laptop to a coffee shop and join a new network and things carry on running despite my IP address changing. I don't think that is currently possible on the reference K8s.

One bad thing about Strimzi is that it barfs with this error when I upgraded the reference K8s implementation to 1.33. The cause was:

Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "emulationMajor" (class io.fabric8.kubernetes.client.VersionInfo), not marked as ignorable (9 known properties: "goVersion", "gitTreeState", "platform", "minor", "gitVersion", "gitCommit", "buildDate", "compiler", "major"])

 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 4, column: 22] (through reference chain: io.fabric8.kubernetes.client.VersionInfo["emulationMajor"])

at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61) ~[com.fasterxml.jackson.core.jackson-databind-2.16.2.jar:2.16.2]

...

at io.fabric8.kubernetes.client.utils.KubernetesSerialization.unmarshal(KubernetesSerialization.java:257) ~[io.fabric8.kubernetes-client-api-6.13.4.jar:?]

This ultimately stops the Strimzi operator. Looks like the Fabric8 library needs updating. The Strimzi version I'm using is:

$ helm pull strimzi/strimzi-kafka-operator --untar

$ helm template test-release ./strimzi-kafka-operator/ | grep "image:"

          image: quay.io/strimzi/operator:0.45.0