本篇内容介绍了“Kubernetes Cluster HA如何配置”的有关知识，在实际案例的操作过程中，不少人都会遇到这样的困境，接下来就让小编带领大家学习一下如何处理这些情况吧！希望大家仔细阅读，能够学有所成！
所有组件可以通过kubelet static pod的方式启动和管理，由kubelet static pod机制保证宿主机上各个组件的高可用, 注意kubelet要添加配置--allow-privileged=true;
管理static pod的kubelet的高可用通过systemd来负责；
当然，你也可以直接通过进程来部署这些组件，systemd来直接管理这些进程；（我们选择的是这种方式，降低复杂度。）
上图中，etcd和Master部署在一起，三个Master节点分别部署了三个etcd，这三个etcd组成一个集群；（当然，如果条件允许，建议将etcd集群和Master节点分开部署。）
每个Master中的apiserver、controller-manager、scheduler都使用hostNetwork, controller-manager和scheduler通过localhost连接到本节点的apiserver，而不会和其他两个Master节点的apiserver连接；
外部的rest-client、kubectl、kubelet、kube-proxy等都通过TLS证书，在LB节点做TLS Termination，LB出来就是http请求发到经过LB策略（RR）到对应的apiserver instance；
apiserver到kubelet server和kube-proxy server的访问也类似，Https到LB这里做TLS Termination，然后http请求出来到对应node的kubelet/kube-proxy server；
apiserver的HA通过经典的haproxy + keepalived来保证，集群对外暴露VIP；
controller-manager和scheduler的HA通过自身提供的leader选举功能（--leader-elect=true），使得3个controller-manager和scheduler都分别只有一个是leader，leader处于正常工作状态，当leader失败，会重新选举新leader来顶替继续工作；
因此，该HA方案中，通过haproxy+keepalived来做apiserver的LB和HA，controller-manager和scheduler通过自身的leader选举来达到HA，etcd通过raft协议保证etcd cluster数据的一致性，达到HA；
keepalived的配置可参考如下：
计算
安全
数据库
网络和加速
企业服务
vrrp_script check_script {
script "/etc/keepalived/check_haproxy.py http://caicloud:caicloud@127.0.0.1/haproxy?stats"
interval 5 # check every 5 seconds
weight 5
fall 2 # require 2 fail for KO
rise 1 # require 1 successes for OK
}
vrrp_instance VI_01 {
state MASTER (BACKUP)
interface eth2
track_interface {
eth2
}
vrrp_garp_master_repeat 5
vrrp_garp_master_refresh 10
virtual_router_id 51
priority 100 (97)
advert_int 1
authentication {
auth_type PASS
auth_pass username
}
virtual_ipaddress {
192.168.205.254 dev eth2 label eth2:vip
}
track_script {
check_script
}
notify "etc/keepalived/notify_state.sh"
}haproxy的配置可参考如下：
global
log 127.0.0.1 local0
maxconn 32768
pidfile /run/haproxy.pid
# turn on stats unix socket
stats socket /run/haproxy.stats
tune.ssl.default-dh-param 2048
default
log global
mode http
option httplog
option dontlognull
retries 3
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
timeout check 50000ms
timeout queue 50000ms
frontend frontend-apisver-http
bind *:8080
option forwardfor
acl local_net src 192.168.205.0/24
http-request allow if local_net
http-request deny
default_backend backend-apiserver-http
frontedn frontend-apiserver-https
# haproxy enable ssl
bind *:443 ssl crt /etc/kubernetes/master-lb.pem
option forwardfor
default_backend backend-apiserver-http
backend backend-apiserver-http
balance roundrobin
option forward-for
server master-1 192.168.205.11:8080 check
server master-2 192.168.205.12:8080 check
server master-3 192.168.205.13:8080 check
listen admin_stats
bind 0.0.0.0:80
log global
mode http
maxconn 10
stats enable
#Hide HAPRoxy version, a necessity for any public-facing site
stats hide-version
stats refresh 30s
stats show-node
stats realm Haproxy\ Statistics
stats auth caicloud:caicloud
stats uri /haproxy?statsLB所在的节点，注意确保ip_vs model已加载、ip_forward和ip_nonlocal_bind已开启；
# make sure ip_vs kernel model is loaded
modprobe ip_vs
modprobe ip_vs_rr
modprobe ip_vs_wrr
# enable ip_forward and ip_nonlocal_bind
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
echo "net.ipv4.ip_nonlocal_bind = 1" >> /etc/sysctl.conf如果你通过pod来部署K8S的组件，可参考官方给出的Yaml：
apiserver
apiVersion: v1
kind: Pod
metadata:
name: kube-apiserver
spec:
hostNetwork: true
containers:
- name: kube-apiserver
image: gcr.io/google_containers/kube-apiserver:9680e782e08a1a1c94c656190011bd02
command:
- /bin/sh
- -c
- /usr/local/bin/kube-apiserver --address=127.0.0.1 --etcd-servers=http://127.0.0.1:4001
--cloud-provider=gce --admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota
--service-cluster-ip-range=10.0.0.0/16 --client-ca-file=/srv/kubernetes/ca.crt
--basic-auth-file=/srv/kubernetes/basic_auth.csv --cluster-name=e2e-test-bburns
--tls-cert-file=/srv/kubernetes/server.cert --tls-private-key-file=/srv/kubernetes/server.key
--secure-port=443 --token-auth-file=/srv/kubernetes/known_tokens.csv --v=2
--allow-privileged=False 1>>/var/log/kube-apiserver.log 2>&1
ports:
- containerPort: 443
hostPort: 443
name: https
- containerPort: 7080
hostPort: 7080
name: http
- containerPort: 8080
hostPort: 8080
name: local
volumeMounts:
- mountPath: /srv/kubernetes
name: srvkube
readOnly: true
- mountPath: /var/log/kube-apiserver.log
name: logfile
- mountPath: /etc/ssl
name: etcssl
readOnly: true
- mountPath: /usr/share/ssl
name: usrsharessl
readOnly: true
- mountPath: /var/ssl
name: varssl
readOnly: true
- mountPath: /usr/ssl
name: usrssl
readOnly: true
- mountPath: /usr/lib/ssl
name: usrlibssl
readOnly: true
- mountPath: /usr/local/openssl
name: usrlocalopenssl
readOnly: true
- mountPath: /etc/openssl
name: etcopenssl
readOnly: true
- mountPath: /etc/pki/tls
name: etcpkitls
readOnly: true
volumes:
- hostPath:
path: /srv/kubernetes
name: srvkube
- hostPath:
path: /var/log/kube-apiserver.log
name: logfile
- hostPath:
path: /etc/ssl
name: etcssl
- hostPath:
path: /usr/share/ssl
name: usrsharessl
- hostPath:
path: /var/ssl
name: varssl
- hostPath:
path: /usr/ssl
name: usrssl
- hostPath:
path: /usr/lib/ssl
name: usrlibssl
- hostPath:
path: /usr/local/openssl
name: usrlocalopenssl
- hostPath:
path: /etc/openssl
name: etcopenssl
- hostPath:
path: /etc/pki/tls
name: etcpkitlsapiVersion: v1
kind: Pod
metadata:
name: kube-controller-manager
spec:
containers:
- command:
- /bin/sh
- -c
- /usr/local/bin/kube-controller-manager --master=127.0.0.1:8080 --cluster-name=e2e-test-bburns
--cluster-cidr=10.245.0.0/16 --allocate-node-cidrs=true --cloud-provider=gce --service-account-private-key-file=/srv/kubernetes/server.key
--v=2 --leader-elect=true 1>>/var/log/kube-controller-manager.log 2>&1
image: gcr.io/google_containers/kube-controller-manager:fda24638d51a48baa13c35337fcd4793
livenessProbe:
httpGet:
path: /healthz
port: 10252
initialDelaySeconds: 15
timeoutSeconds: 1
name: kube-controller-manager
volumeMounts:
- mountPath: /srv/kubernetes
name: srvkube
readOnly: true
- mountPath: /var/log/kube-controller-manager.log
name: logfile
- mountPath: /etc/ssl
name: etcssl
readOnly: true
- mountPath: /usr/share/ssl
name: usrsharessl
readOnly: true
- mountPath: /var/ssl
name: varssl
readOnly: true
- mountPath: /usr/ssl
name: usrssl
readOnly: true
- mountPath: /usr/lib/ssl
name: usrlibssl
readOnly: true
- mountPath: /usr/local/openssl
name: usrlocalopenssl
readOnly: true
- mountPath: /etc/openssl
name: etcopenssl
readOnly: true
- mountPath: /etc/pki/tls
name: etcpkitls
readOnly: true
hostNetwork: true
volumes:
- hostPath:
path: /srv/kubernetes
name: srvkube
- hostPath:
path: /var/log/kube-controller-manager.log
name: logfile
- hostPath:
path: /etc/ssl
name: etcssl
- hostPath:
path: /usr/share/ssl
name: usrsharessl
- hostPath:
path: /var/ssl
name: varssl
- hostPath:
path: /usr/ssl
name: usrssl
- hostPath:
path: /usr/lib/ssl
name: usrlibssl
- hostPath:
path: /usr/local/openssl
name: usrlocalopenssl
- hostPath:
path: /etc/openssl
name: etcopenssl
- hostPath:
path: /etc/pki/tls
name: etcpkitlsapiVersion: v1
kind: Pod
metadata:
name: kube-scheduler
spec:
hostNetwork: true
containers:
- name: kube-scheduler
image: gcr.io/google_containers/kube-scheduler:34d0b8f8b31e27937327961528739bc9
command:
- /bin/sh
- -c
- /usr/local/bin/kube-scheduler --master=127.0.0.1:8080 --v=2 --leader-elect=true 1>>/var/log/kube-scheduler.log
2>&1
livenessProbe:
httpGet:
path: /healthz
port: 10251
initialDelaySeconds: 15
timeoutSeconds: 1
volumeMounts:
- mountPath: /var/log/kube-scheduler.log
name: logfile
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-s8ejd
readOnly: true
volumes:
- hostPath:
path: /var/log/kube-scheduler.log
name: logfileapiVersion: v1
kind: Pod
metadata:
name: etcd-server
spec:
hostNetwork: true
containers:
- image: gcr.io/google_containers/etcd:2.0.9
name: etcd-container
command:
- /usr/local/bin/etcd
- --name
- ${NODE_NAME}
- --initial-advertise-peer-urls
- http://${NODE_IP}:2380
- --listen-peer-urls
- http://${NODE_IP}:2380
- --advertise-client-urls
- http://${NODE_IP}:4001
- --listen-client-urls
- http://127.0.0.1:4001
- --data-dir
- /var/etcd/data
- --discovery
- ${DISCOVERY_TOKEN}
ports:
- containerPort: 2380
hostPort: 2380
name: serverport
- containerPort: 4001
hostPort: 4001
name: clientport
volumeMounts:
- mountPath: /var/etcd
name: varetcd
- mountPath: /etc/ssl
name: etcssl
readOnly: true
- mountPath: /usr/share/ssl
name: usrsharessl
readOnly: true
- mountPath: /var/ssl
name: varssl
readOnly: true
- mountPath: /usr/ssl
name: usrssl
readOnly: true
- mountPath: /usr/lib/ssl
name: usrlibssl
readOnly: true
- mountPath: /usr/local/openssl
name: usrlocalopenssl
readOnly: true
- mountPath: /etc/openssl
name: etcopenssl
readOnly: true
- mountPath: /etc/pki/tls
name: etcpkitls
readOnly: true
volumes:
- hostPath:
path: /var/etcd/data
name: varetcd
- hostPath:
path: /etc/ssl
name: etcssl
- hostPath:
path: /usr/share/ssl
name: usrsharessl
- hostPath:
path: /var/ssl
name: varssl
- hostPath:
path: /usr/ssl
name: usrssl
- hostPath:
path: /usr/lib/ssl
name: usrlibssl
- hostPath:
path: /usr/local/openssl
name: usrlocalopenssl
- hostPath:
path: /etc/openssl
name: etcopenssl
- hostPath:
path: /etc/pki/tls
name: etcpkitls“Kubernetes Cluster HA如何配置”的内容就介绍到这里了，感谢大家的阅读。如果想了解更多行业相关的知识可以关注亿速云网站，小编将为大家输出更多高质量的实用文章！
亿速云「云服务器」，即开即用、新一代英特尔至强铂金CPU、三副本存储NVMe SSD云盘，价格低至29元/月。点击查看>>
免责声明：本站发布的内容（图片、视频和文字）以原创、转载和分享为主，文章观点不代表本网站立场，如果涉及侵权请联系站长邮箱：is@yisu.com进行举报，并提供相关证据，一经查实，将立刻删除涉嫌侵权内容。
原文链接：https://my.oschina.net/jxcdwangtao/blog/1556365
