k8s kafka怎样部署

在Kubernetes（k8s）上部署Kafka主要涉及以下几个步骤：

1. 准备环境

确保你的Kubernetes集群已经正常运行，并且你已经安装了kubectl和有一个可用的Kubernetes上下文。

2. 创建Kafka命名空间

首先，创建一个命名空间来隔离Kafka资源。

kubectl create namespace kafka

3. 创建Kafka配置文件

创建Kafka的配置文件，包括server.properties和相关的部署文件。以下是一个简单的示例：

`server.properties`

broker.id=0
listeners=PLAINTEXT://:9092
log.dirs=/tmp/kafka-logs
zookeeper.connect=zookeeper:2181

`kafka-deployment.yaml`

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka
  namespace: kafka
spec:
  replicas: 3
  selector:
    matchLabels:
      app: kafka
  template:
    metadata:
      labels:
        app: kafka
    spec:
      containers:
      - name: kafka
        image: bitnami/kafka:2.8.1
        ports:
        - containerPort: 9092
        env:
        - name: KAFKA_BROKER_ID
          value: "0"
        - name: KAFKA_LISTENERS
          value: "PLAINTEXT://:9092"
        - name: KAFKA_LOG_DIRS
          value: "/tmp/kafka-logs"
        - name: KAFKA_ZOO_CONNECT
          value: "zookeeper:2181"
        volumeMounts:
        - name: kafka-storage
          mountPath: /tmp/kafka-logs
      volumes:
      - name: kafka-storage
        persistentVolumeClaim:
          claimName: kafka-pvc
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: kafka-pvc
  namespace: kafka
spec:
  accessModes: [ "ReadWriteOnce" ]
  resources:
    requests:
      storage: 10Gi

4. 创建Zookeeper服务

Kafka依赖于Zookeeper来管理元数据。创建一个Zookeeper服务。

`zookeeper-deployment.yaml`

apiVersion: apps/v1
kind: Deployment
metadata:
  name: zookeeper
  namespace: kafka
spec:
  replicas: 1
  selector:
    matchLabels:
      app: zookeeper
  template:
    metadata:
      labels:
        app: zookeeper
    spec:
      containers:
      - name: zookeeper
        image: bitnami/zookeeper:3.7.0
        ports:
        - containerPort: 2181
        env:
        - name: ZOO_MY_ID
          value: "1"
        - name: ZOO_SERVERS
          value: "server.1:2888:3888"
      volumes:
      - name: zookeeper-storage
        persistentVolumeClaim:
          claimName: zookeeper-pvc
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: zookeeper-pvc
  namespace: kafka
spec:
  accessModes: [ "ReadWriteOnce" ]
  resources:
    requests:
      storage: 10Gi

5. 应用配置文件

使用kubectl应用这些配置文件。

kubectl apply -f kafka-deployment.yaml
kubectl apply -f zookeeper-deployment.yaml

6. 验证部署

验证Kafka和Zookeeper是否正常运行。

kubectl get pods -n kafka
kubectl get services -n kafka

7. 访问Kafka

你可以使用Kafka客户端工具（如kafka-console-producer.sh和kafka-console-consumer.sh）来验证Kafka集群是否正常工作。

启动Kafka生产者

kubectl exec -it <kafka-pod-name> -- /bin/bash
./kafka-console-producer.sh --broker-list <kafka-service-ip>:9092 --topic test

启动Kafka消费者

kubectl exec -it <kafka-pod-name> -- /bin/bash
./kafka-console-consumer.sh --bootstrap-server <kafka-service-ip>:9092 --topic test --from-beginning

通过以上步骤，你应该能够在Kubernetes集群上成功部署Kafka和Zookeeper。