k8s中部署etcd集群
Etcd是一个高可用的、开源的、分布式的Key/value存储系统，提供共享配置、服务的注册和发现、数据TTL失效、数据改变监视、多值、目录监听、分布式原子锁操作等功能。通常，k8s使用etcd进行数据存储，本文将部署etcd集群用做应用的存储。
一、前提
部署好k8s，最好有3个及以上worker节点，部署好StorageClass可以动态创建PVC。本文k8s版本v1.24.8，StorageClass使用nfs-client，使用命名空间gv-public，以部署apisix为例，说明etcd集群部署的过程。
二、yaml文件
以下yaml文件是使用命名空间:gv-public（已创建），sc:nfs-client为例，部署根据实际情况修改。
1）svc.yaml
etcd集群用到两个svc，apisix-etcd-headless和apisix-etcd。
apiVersion: v1
kind: Service
metadata:name: apisix-etcd-headlessnamespace: gv-publiclabels:app.kubernetes.io/instance: apisix-etcdapp.kubernetes.io/name: apisix-etcd
spec:ports:- name: clientport: 2379protocol: TCPtargetPort: 2379- name: peerport: 2380protocol: TCPtargetPort: 2380clusterIP: Noneselector:app.kubernetes.io/instance: apisix-etcdapp.kubernetes.io/name: apisix-etcdpublishNotReadyAddresses: true
---
apiVersion: v1
kind: Service
metadata:name: apisix-etcdnamespace: gv-publiclabels:app.kubernetes.io/instance: apisix-etcdapp.kubernetes.io/name: apisix-etcd
spec:ports:- name: clientport: 2379protocol: TCPtargetPort: 2379- name: peerport: 2380protocol: TCPtargetPort: 2380selector:app.kubernetes.io/instance: apisix-etcdapp.kubernetes.io/name: apisix-etcd2)etcd.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:name: apisix-etcdnamespace: gv-publiclabels:app.kubernetes.io/instance: apisix-etcdapp.kubernetes.io/name: apisix-etcd
spec:podManagementPolicy: Parallelreplicas: 3serviceName: apisix-etcd-headlessselector:matchLabels:app.kubernetes.io/instance: apisix-etcdapp.kubernetes.io/name: apisix-etcdtemplate:metadata:labels:app.kubernetes.io/instance: apisix-etcdapp.kubernetes.io/name: apisix-etcdspec:affinity:podAntiAffinity:preferredDuringSchedulingIgnoredDuringExecution:- podAffinityTerm:labelSelector:matchLabels:app.kubernetes.io/instance: apisix-etcdapp.kubernetes.io/name: apisix-etcdtopologyKey: kubernetes.io/hostnameweight: 1containers:- name: apisix-etcd-appimage: bitnami/etcd:3.4.24imagePullPolicy: IfNotPresentports:- containerPort: 2379name: clientprotocol: TCP- containerPort: 2380name: peerprotocol: TCPenv:- name: BITNAMI_DEBUGvalue: 'false'- name: MY_POD_IPvalueFrom:fieldRef:apiVersion: v1fieldPath: status.podIP- name: MY_POD_NAMEvalueFrom:fieldRef:apiVersion: v1fieldPath: metadata.name- name: MY_STS_NAMEvalue: apisix-etcd- name: ETCDCTL_APIvalue: '3'- name: ETCD_ON_K8Svalue: 'yes'- name: ETCD_START_FROM_SNAPSHOTvalue: 'no'- name: ETCD_DISASTER_RECOVERYvalue: 'no'- name: ETCD_NAMEvalue: $(MY_POD_NAME)- name: ETCD_DATA_DIRvalue: /bitnami/etcd/data- name: ETCD_LOG_LEVELvalue: info- name: ALLOW_NONE_AUTHENTICATIONvalue: 'yes'- name: ETCD_ADVERTISE_CLIENT_URLSvalue: http://$(MY_POD_NAME).apisix-etcd-headless.gv-public.svc.cluster.local:2379- name: ETCD_LISTEN_CLIENT_URLSvalue: http://0.0.0.0:2379- name: ETCD_INITIAL_ADVERTISE_PEER_URLSvalue: http://$(MY_POD_NAME).apisix-etcd-headless.gv-public.svc.cluster.local:2380- name: ETCD_LISTEN_PEER_URLSvalue: http://0.0.0.0:2380- name: ETCD_INITIAL_CLUSTER_TOKENvalue: apisix-etcd-cluster-k8s- name: ETCD_INITIAL_CLUSTER_STATEvalue: new- name: ETCD_INITIAL_CLUSTERvalue: apisix-etcd-0=http://apisix-etcd-0.apisix-etcd-headless.gv-public.svc.cluster.local:2380,apisix-etcd-1=http://apisix-etcd-1.apisix-etcd-headless.gv-public.svc.cluster.local:2380,apisix-etcd-2=http://apisix-etcd-2.apisix-etcd-headless.gv-public.svc.cluster.local:2380- name: ETCD_CLUSTER_DOMAINvalue: apisix-etcd-headless.gv-public.svc.cluster.localvolumeMounts:- name: datamountPath: /bitnami/etcdlifecycle:preStop:exec:command:- /opt/bitnami/scripts/etcd/prestop.shlivenessProbe:exec:command:- /opt/bitnami/scripts/etcd/healthcheck.shinitialDelaySeconds: 60timeoutSeconds: 5periodSeconds: 30successThreshold: 1failureThreshold: 5readinessProbe:exec:command:- /opt/bitnami/scripts/etcd/healthcheck.shinitialDelaySeconds: 60timeoutSeconds: 5periodSeconds: 10successThreshold: 1failureThreshold: 5securityContext:fsGroup: 1001volumeClaimTemplates:- metadata:name: dataspec:accessModes: - ReadWriteOncestorageClassName: nfs-clientresources:requests:storage: 1Gi三、执行yaml
分别执行如下命令，创建etcd集群：
#kubectl create -f svc.yaml
[root@k8s-master apisix]# kubectl get svc -n gv-public | grep etcd
apisix-etcd ClusterIP 10.43.141.181 <none> 2379/TCP,2380/TCP 58m
apisix-etcd-headless ClusterIP None <none> 2379/TCP,2380/TCP 58m#kubectl create -f etcd.yaml
[root@k8s-master apisix]# kubectl get pod -n gv-public | grep etcd
apisix-etcd-0 1/1 Running 0 59m
apisix-etcd-1 1/1 Running 0 59m
apisix-etcd-2 1/1 Running 0 59m进入etcd集群的一个pod容器：
#kubectl exec -it apisix-etcd-0 -n gv-public -- /bin/bash查看集群信息：
I have no name!@apisix-etcd-0:/opt/bitnami/etcd$ etcdctl endpoint status --cluster -w table
+----------------------------------------------------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------------------------------------------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| http://apisix-etcd-2.apisix-etcd-headless.gv-public.svc.cluster.local:2379 | a13daea6a7c5a0ed | 3.4.24 | 49 kB | false | false | 5 | 50 | 50 | |
| http://apisix-etcd-0.apisix-etcd-headless.gv-public.svc.cluster.local:2379 | c3f10b3f6f66b949 | 3.4.24 | 49 kB | false | false | 5 | 50 | 50 | |
| http://apisix-etcd-1.apisix-etcd-headless.gv-public.svc.cluster.local:2379 | c6a5a0cbfafa2786 | 3.4.24 | 49 kB | true | false | 5 | 50 | 50 | |
+----------------------------------------------------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
