kubernetes二进制部署（二）多节点部署

这是kubernetes二进制部署的第二篇
如果没有看过前面第一篇的朋友可以看看下面的：
Kubernetes二进制部署（一）单节点部署

实验环境：

负载均衡：
Nginx1:192.168.35.104/24
Nginx2:192.168.35.105/24

Master节点：
master1:192.168.35.100/24
master2:192.168.35.103/24

Node节点：
node1:192.168.35.101/24
node2:192.168.35.102/24

master02部署

1、关闭防火墙

[root@localhost ~]# systemctl stop firewalld.service 
[root@localhost ~]# setenforce 0

2、在master1上操作

（1）复制kubernetes目录到master2

[root@localhost k8s]# scp -r /opt/kubernetes/ root@192.168.35.103:/opt
The authenticity of host '192.168.35.103 (192.168.35.103)' can't be established.
ECDSA key fingerprint is SHA256:VBIpN6lYzzdqZmVfp5cGBOYgDBwYuVkI55sXRAz2C/s.
ECDSA key fingerprint is MD5:fe:66:e7:bd:10:1d:ce:1b:4b:82:9e:e0:99:23:d4:5f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.35.103' (ECDSA) to the list of known hosts.
root@192.168.35.103's password: 
token.csv                                          100%   84    55.0KB/s   00:00    
kube-apiserver                                     100%  934     1.0MB/s   00:00    
kube-scheduler                                     100%   94   122.4KB/s   00:00    
kube-controller-manager                            100%  483   393.2KB/s   00:00    
kube-apiserver                                     100%  184MB  91.9MB/s   00:02    
kubectl                                            100%   55MB  72.2MB/s   00:00    
kube-controller-manager                            100%  155MB  77.3MB/s   00:02    
kube-scheduler                                     100%   55MB  74.8MB/s   00:00    
ca-key.pem                                         100% 1679     1.5MB/s   00:00    
ca.pem                                             100% 1359   984.7KB/s   00:00    
server-key.pem                                     100% 1675     1.0MB/s   00:00    
server.pem                                         100% 1643   338.6KB/s   00:00    

（2）复制master中的三个组件启动脚本kube-apiserver.service、kube-controller-manager.service、kube-scheduler.service。

[root@localhost k8s]# scp /usr/lib/systemd/system/{kube-apiserver,kube-controller-manager,kube-scheduler}.service root@192.168.35.103:/usr/lib/systemd/system/
root@192.168.35.103's password: 
kube-apiserver.service                             100%  282   164.1KB/s   00:00    
kube-controller-manager.service                    100%  317   294.5KB/s   00:00    
kube-scheduler.service                             100%  281   352.0KB/s   00:00    

（3）拷贝master1上已有的etcd证书给master2使用

特别注意：master2一定要有etcd证书，否则apiserver服务无法启动

[root@localhost k8s]# scp -r /opt/etcd/ root@192.168.35.103:/opt/
root@192.168.35.103's password: 
etcd                                               100%  516   196.4KB/s   00:00    
etcd                                               100%   18MB  82.4MB/s   00:00    
etcdctl                                            100%   15MB  55.2MB/s   00:00    
ca-key.pem                                         100% 1679     1.0MB/s   00:00    
ca.pem                                             100% 1265     1.1MB/s   00:00    
server-key.pem                                     100% 1679     2.0MB/s   00:00    
server.pem                                         100% 1338     1.6MB/s   00:00    

3、在master2上操作

（1）修改配置文件kube-apiserver中的IP

[root@localhost ~]# vim /opt/kubernetes/cfg/kube-apiserver 

KUBE_APISERVER_OPTS="--logtostderr=true \
--v=4 \
--etcd-servers=https://192.168.35.100:2379,https://192.168.35.101:2379,https://192.168.35.102:2379 \
--bind-address=192.168.35.103 \
--secure-port=6443 \
--advertise-address=192.168.35.103 \
--allow-privileged=true \
--service-cluster-ip-range=10.0.0.0/24 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--kubelet-https=true \
--enable-bootstrap-token-auth \
--token-auth-file=/opt/kubernetes/cfg/token.csv \
--service-node-port-range=30000-50000 \
--tls-cert-file=/opt/kubernetes/ssl/server.pem  \
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \
--client-ca-file=/opt/kubernetes/ssl/ca.pem \
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
--etcd-cafile=/opt/etcd/ssl/ca.pem \
--etcd-certfile=/opt/etcd/ssl/server.pem \
--etcd-keyfile=/opt/etcd/ssl/server-key.pem"

（2）启动master2中的三个组件服务

[root@localhost ~]# systemctl start kube-apiserver.service 

[root@localhost ~]# systemctl enable kube-apiserver.service 
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-apiserver.service to /usr/lib/systemd/system/kube-apiserver.service.

[root@localhost cfg]# systemctl status kube-apiserver.service
● kube-apiserver.service - Kubernetes API Server
   Loaded: loaded (/usr/lib/systemd/system/kube-apiserver.service; enabled; vendor preset: disabled)
   Active: active (running) since 六 2020-02-08 14:29:42 CST; 22s ago
     Docs: https://github.com/kubernetes/kubernetes
 Main PID: 3287 (kube-apiserver)
   CGroup: /system.slice/kube-apiserver.service
           └─3287 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --et...

2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.583442...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.585482...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.587185...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.588646...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.590160...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.591712...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.593487...
2月 08 14:29:54 localhost.localdomain kube-apiserver[3287]: I0208 14:29:54.598787...
2月 08 14:30:03 localhost.localdomain kube-apiserver[3287]: I0208 14:30:03.331185...
2月 08 14:30:03 localhost.localdomain kube-apiserver[3287]: I0208 14:30:03.333531...
Hint: Some lines were ellipsized, use -l to show in full.

[root@localhost ~]# systemctl start kube-controller-manager.service 

[root@localhost ~]# systemctl enable kube-controller-manager.service 
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-controller-manager.service to /usr/lib/systemd/system/kube-controller-manager.service.

[root@localhost ~]# systemctl start kube-scheduler.service 

[root@localhost ~]# systemctl enable kube-scheduler.service 
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-scheduler.service to /usr/lib/systemd/system/kube-scheduler.service.

（3）增加环境变量（在配置文件末行加入）

[root@localhost ~]# vim /etc/profile

export PATH=$PATH:/opt/kubernetes/bin/                ##添加

[root@localhost ~]# source /etc/profile           ##重新加载

（4）查看群集中的节点

[root@localhost cfg]# kubectl get node
NAME             STATUS   ROLES    AGE   VERSION
192.168.35.101   Ready    <none>   17m   v1.12.3
192.168.35.102   Ready    <none>   10m   v1.12.3