Why Are FileStream Connector Versions Overridden in Custom Kafka Connect Images with Strimzi?

I'm experimenting with building custom Kafka Connect images using the Strimzi operator (version 0.45.0), and I'm confused by the connector versions that appear. Initially, I created a Kafka Connect instance without specifying an image using the following configuration:

YAMLCopied
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: my-connect-cluster-connect-build
  namespace: first-kafka-namespace
labels:
  my-connect: my-connect-cluster-connect-build
annotations:
  strimzi.io/use-connector-resources: "true"
spec:
  version: 3.8.0
  replicas: 1
  bootstrapServers: "first-kafka-cluster-kafka-bootstrap:9092"
config:
  group.id: connect-cluster
  offset.storage.topic: connect-cluster-offsets
  config.storage.topic: connect-cluster-configs
  status.storage.topic: connect-cluster-status
  config.storage.replication.factor: 1
  offset.storage.replication.factor: 1
  status.storage.replication.factor: 1

I then checked the available connectors by running curl -s http://localhost:8083/connector-plugins | jq ., which returned only the Mirror connectors with version 3.8.0:

JSONCopied
[
  {
    "class": "org.apache.kafka.connect.mirror.MirrorCheckpointConnector",
    "type": "source",
    "version": "3.8.0"
  },
  {
    "class": "org.apache.kafka.connect.mirror.MirrorHeartbeatConnector",
    "type": "source",
    "version": "3.8.0"
  },
  {
    "class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
    "type": "source",
    "version": "3.8.0"
  }
]

This output made sense, as the Kafka version (3.8.0) matched the connector versions. However, the FileStream connectors, which are part of the default Kafka Connect distribution, were not listed.

Later, I built a custom image and pushed it to GHCR using the following build configuration in a different environment:

YAMLCopied
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: kafka-connect-cluster
  namespace: kafka
annotations:
  strimzi.io/use-connector-resources: "true"
spec:
  version: 3.9.0
  replicas: 1
  bootstrapServers: "kafka-cluster-kafka-bootstrap:9092"
config:
  group.id: connect-cluster
  offset.storage.topic: connect-cluster-offsets
  config.storage.topic: connect-cluster-configs
  status.storage.topic: connect-cluster-status
  config.storage.replication.factor: 1
  offset.storage.replication.factor: 1
  status.storage.replication.factor: 1
build:
  output:
    type: docker
    image: <GHCR Registry>
    pushSecret: ghcr-push-secret
  plugins:
    - name: file-source-connector
      artifacts:
        - type: jar
          url: https://repo1.maven.org/maven2/org/apache/kafka/connect-file/3.6.0/connect-file-3.6.0.jar

The image builds and pushes successfully. To test this image, I deployed a new Kafka Connect instance with this configuration:

YAMLCopied
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: my-connect-cluster-connect-build
  namespace: first-kafka-namespace
labels:
  my-connect: my-connect-cluster-connect-build
annotations:
  strimzi.io/use-connector-resources: "true"
spec:
  image: <image from GHCR>
  replicas: 1
  bootstrapServers: "first-kafka-cluster-kafka-bootstrap:9092"
config:
  group.id: connect-cluster
  offset.storage.topic: connect-cluster-offsets
  config.storage.topic: connect-cluster-configs
  status.storage.topic: connect-cluster-status
  config.storage.replication.factor: 1
  offset.storage.replication.factor: 1
  status.storage.replication.factor: 1
  template:
    pod:
      imagePullSecrets:
        - name: ghcr-pull-secret

After deploying, I expected to see three Mirror connectors with version 3.9.0 and the FileStream connectors with the artifact version 3.6.0. Instead, the output of GET /connector-plugins was:

JSONCopied
[
  {
    "class": "org.apache.kafka.connect.file.FileStreamSinkConnector",
    "type": "sink",
    "version": "3.9.0"
  },
  {
    "class": "org.apache.kafka.connect.file.FileStreamSourceConnector",
    "type": "source",
    "version": "3.9.0"
  },
  {
    "class": "org.apache.kafka.connect.mirror.MirrorCheckpointConnector",
    "type": "source",
    "version": "3.9.0"
  },
  {
    "class": "org.apache.kafka.connect.mirror.MirrorHeartbeatConnector",
    "type": "source",
    "version": "3.9.0"
  },
  {
    "class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
    "type": "source",
    "version": "3.9.0"
  }
]

I don't understand why the FileStream connectors are reported as version 3.9.0 even though I specified a plugin artifact using version 3.6.0 in the build configuration. Could the Kafka Connect instance's version (3.9.0) be overriding the custom FileStream artifact? Also, why did the GET /connector-plugins output differ initially by not showing the FileStream connectors?

I would appreciate any insights into what I might be doing wrong or how to correctly incorporate a specific connector version into a custom image.