docker中RoutingMesh--Ingress负载均衡是什么，相信很多没有经验的人对此束手无策，为此本文总结了问题出现的原因和解决方法，通过这篇文章希望你能解决这个问题。
我们知道容器之间的通信，例如上图中的10.0.9.3与10.0.9.5通信是通过overlay网络，是通过一个VXLAN tannel来实现的。
但是service和service之间通信是通过VIP实现的。例如client的service与web 的service进行通信，而web有一个scale，因此client访问web是通过访问虚拟IP（VIP）来实现的。那么VIP是怎么映射到具体的10.0.9.5或者10.0.9.6呢？这是通过LVS实现的。
LVS，Linux Virtual Server。可以实现在系统级别的负载均衡。
我们可以在三个节点上任何一个节点访问80端口都可以访问到wordpress，这个实现就是IngressNetWork的作用。任何一台swarm节点上去访问端口服务时，会通过端口服务通过本节点IPVS（IP Virtual Service），并通过LVS给loadbanlance到真正具有service上面，例如上图中我们通过访问Docker Host3
转发到另外两个节点中。
我们的实验环境跟上一节一样，我们将whoami 的scale 变为2
iie4bu@swarm-manager:~$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS h5wlczp85sw5 client replicated 1/1 busybox:1.28.3 9i6wz6cg4koc whoami replicated 3/3 jwilder/whoami:latest *:8000->8000/tcp iie4bu@swarm-manager:~$ docker service scale whoami=2 whoami scaled to 2 overall progress: 2 out of 2 tasks 1/2: running [==================================================>] 2/2: running [==================================================>] verify: Service converged
查看whoami的运行情况：
iie4bu@swarm-manager:~$ docker service ps whoami ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 6hhuf528spdw whoami.1 jwilder/whoami:latest swarm-manager Running Running 17 hours ago 9idgk9jbrlcm whoami.3 jwilder/whoami:latest swarm-worker2 Running Running 16 hours ago
分别运行在swarm-manager和swarm-worker2节点上。
在swarm-manager中访问whoami：
iie4bu@swarm-manager:~$ curl 127.0.0.1:8000 I'm cc9f97cc5056 iie4bu@swarm-manager:~$ curl 127.0.0.1:8000 I'm f47e05019fd9
在swarm-worker1中访问whoami：
iie4bu@swarm-worker1:~$ curl 127.0.0.1:8000 I'm f47e05019fd9 iie4bu@swarm-worker1:~$ curl 127.0.0.1:8000 I'm cc9f97cc5056
为什么对于swarm-worker1来讲本地并没有whoami的service，确能访问8000端口？
通过iptables可以看到本地的转发规则：
iie4bu@swarm-worker1:~$ sudo iptables -nL -t nat [sudo] password for iie4bu: Chain PREROUTING (policy ACCEPT) target prot opt source destination DOCKER-INGRESS all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL Chain INPUT (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination DOCKER-INGRESS all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL Chain POSTROUTING (policy ACCEPT) target prot opt source destination MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match src-type LOCAL MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0 MASQUERADE all -- 172.19.0.0/16 0.0.0.0/0 MASQUERADE all -- 172.18.0.0/16 0.0.0.0/0 MASQUERADE tcp -- 172.17.0.2 172.17.0.2 tcp dpt:443 MASQUERADE tcp -- 172.17.0.2 172.17.0.2 tcp dpt:80 MASQUERADE tcp -- 172.17.0.2 172.17.0.2 tcp dpt:22 Chain DOCKER (2 references) target prot opt source destination RETURN all -- 0.0.0.0/0 0.0.0.0/0 RETURN all -- 0.0.0.0/0 0.0.0.0/0 RETURN all -- 0.0.0.0/0 0.0.0.0/0 DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:443 to:172.17.0.2:443 DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:81 to:172.17.0.2:80 DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:23 to:172.17.0.2:22 Chain DOCKER-INGRESS (2 references) target prot opt source destination DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:8000 to:172.19.0.2:8000 RETURN all -- 0.0.0.0/0 0.0.0.0/0
我们看到DOCKER-INGRESS，它的转发规则是如果我们访问tcp8000duank端口，它会转发到172.19.0.2:8000上，那么这个172.19.0.2:8000是什么呢？
我们先看一下本地的ip:
iie4bu@swarm-worker1:~$ ifconfig br-3f2fc691f5da Link encap:Ethernet HWaddr 02:42:c8:f4:03:ad inet addr:172.18.0.1 Bcast:172.18.255.255 Mask:255.255.0.0 UP BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) docker0 Link encap:Ethernet HWaddr 02:42:43:69:b7:60 inet addr:172.17.0.1 Bcast:172.17.255.255 Mask:255.255.0.0 inet6 addr: fe80::42:43ff:fe69:b760/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:83 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:9208 (9.2 KB) docker_gwbridge Link encap:Ethernet HWaddr 02:42:41:bf:4d:15 inet addr:172.19.0.1 Bcast:172.19.255.255 Mask:255.255.0.0 inet6 addr: fe80::42:41ff:febf:4d15/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:42 errors:0 dropped:0 overruns:0 frame:0 TX packets:142 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:3556 (3.5 KB) TX bytes:13857 (13.8 KB) ...... ......
我们可以看到本地有个网络是docker_gwbridge，它的ip是172.19.0.1，这个地址与172.19.0.2在同一个网段。所以说我们可以猜测172.19.0.2肯定是与docker_gwbridge相连的一个网络。可以通过brctl show 查看：
iie4bu@swarm-worker1:~$ brctl show bridge name bridge id STP enabled interfaces br-3f2fc691f5da 8000.0242c8f403ad no docker0 8000.02424369b760 no veth75a496d docker_gwbridge 8000.024241bf4d15 no veth500f4b4 veth54af5f8
可以看到docker_gwbridge有两个interface，这两个interface哪个是呢？
通过docker network ls：
iie4bu@swarm-worker1:~$ docker network ls NETWORK ID NAME DRIVER SCOPE bdf23298113d bridge bridge local 969e60257ba5 docker_gwbridge bridge local cdcffe1b31cb host host local uz1kgf9j6m48 ingress overlay swarm cojus8blvkdo my-demo overlay swarm 3f2fc691f5da network_default bridge local dba4587ee914 none null local
查看docker_gwbridge的详细信息：
iie4bu@swarm-worker1:~$ docker network inspect docker_gwbridge
[
{
"Name": "docker_gwbridge",
"Id": "969e60257ba50b070374d31ea43a0550d6cd3ae3e68623746642fe8736dee5a4",
"Created": "2019-04-08T09:47:59.343371327+08:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.19.0.0/16",
"Gateway": "172.19.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"5559895ccaea972dad4f2fe52c0ec754d2d7c485dceb35083719768f611552e7": {
"Name": "gateway_bf5031da0049",
"EndpointID": "6ad54f228134798de719549c0f93c804425beb85dd97f024408c4f7fc393fdf9",
"MacAddress": "02:42:ac:13:00:03",
"IPv4Address": "172.19.0.3/16",
"IPv6Address": ""
},
"ingress-sbox": {
"Name": "gateway_ingress-sbox",
"EndpointID": "177757eca7a18630ae91c01b8ac67bada25ce1ea050dad4ac5cc318093062003",
"MacAddress": "02:42:ac:13:00:02",
"IPv4Address": "172.19.0.2/16",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.bridge.enable_icc": "false",
"com.docker.network.bridge.enable_ip_masquerade": "true",
"com.docker.network.bridge.name": "docker_gwbridge"
},
"Labels": {}
}
]可以看到与docker_gwbridge相连的container有两个，分别是gateway_bf5031da0049和gateway_ingress-sbox，而gateway_ingress-sbox的ip正是172.19.0.2。也就是说数据被转发到这个gateway_ingress-sbox network namespace中去了。
进入gateway_ingress-sbox：
iie4bu@swarm-worker1:~$ sudo ls /var/run/docker/netns 1-cojus8blvk 1-uz1kgf9j6m 44e6e70b2177 b1ba5b4dd9f2 bf5031da0049 ingress_sbox iie4bu@swarm-worker1:~$ sudo nsenter --net=//var/run/docker/netns/ingress_sbox -bash: /home/iie4bu/anaconda3/etc/profiel.d/conda.sh: No such file or directory CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. If your shell is Bash or a Bourne variant, enable conda for the current user with $ echo ". /home/iie4bu/anaconda3/etc/profile.d/conda.sh" >> ~/.bashrc or, for all users, enable conda with $ sudo ln -s /home/iie4bu/anaconda3/etc/profile.d/conda.sh /etc/profile.d/conda.sh The options above will permanently enable the 'conda' command, but they do NOT put conda's base (root) environment on PATH. To do so, run $ conda activate in your terminal, or to put the base environment on PATH permanently, run $ echo "conda activate" >> ~/.bashrc Previous to conda 4.4, the recommended way to activate conda was to modify PATH in your ~/.bashrc file. You should manually remove the line that looks like export PATH="/home/iie4bu/anaconda3/bin:$PATH" ^^^ The above line should NO LONGER be in your ~/.bashrc file! ^^^ root@swarm-worker1:~#
这样就进入了ingress_sbox的namespace了。查看ip
root@swarm-worker1:~# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 13: eth0@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default link/ether 02:42:0a:ff:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.255.0.3/16 brd 10.255.255.255 scope global eth0 valid_lft forever preferred_lft forever inet 10.255.0.168/32 brd 10.255.0.168 scope global eth0 valid_lft forever preferred_lft forever 15: eth2@if16: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:ac:13:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 1 inet 172.19.0.2/16 brd 172.19.255.255 scope global eth2 valid_lft forever preferred_lft forever
发现ip是172.19.0.2。
为了掩饰LVS，我们退出ingress_sbox的namespace。在swarm-worker1中安装ipvsadm，这是lvs的一个管理工具。
iie4bu@swarm-worker1:~$ sudo apt-get install ipvsadm
安装成功之后，在进入到ingress_sbox中，然后输入命令iptables -nL -t mangle
root@swarm-worker1:~# iptables -nL -t mangle Chain PREROUTING (policy ACCEPT) target prot opt source destination MARK tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:8000 MARK set 0x100 Chain INPUT (policy ACCEPT) target prot opt source destination MARK all -- 0.0.0.0/0 10.255.0.168 MARK set 0x100 Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain POSTROUTING (policy ACCEPT) target prot opt source destination
MARK一行就表示负载均衡。
输入命令ipvsadm -l
root@swarm-worker1:~# ipvsadm -l IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn FWM 256 rr -> 10.255.0.5:0 Masq 1 0 0 -> 10.255.0.7:0 Masq 1 0 0
10.255.0.5 10.255.0.7就是whoami的service地址。
在swarm-manager上查看whoami：
iie4bu@swarm-manager:~$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES cc9f97cc5056 jwilder/whoami:latest "/app/http" 19 hours ago Up 19 hours 8000/tcp whoami.1.6hhuf528spdw9j9pla7l3tv3t iie4bu@swarm-manager:~$ docker exec cc9 ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet 10.255.0.168/32 brd 10.255.0.168 scope global lo valid_lft forever preferred_lft forever inet 10.0.2.5/32 brd 10.0.2.5 scope global lo valid_lft forever preferred_lft forever 18: eth0@if19: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP link/ether 02:42:0a:ff:00:05 brd ff:ff:ff:ff:ff:ff inet 10.255.0.5/16 brd 10.255.255.255 scope global eth0 valid_lft forever preferred_lft forever 20: eth2@if21: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP link/ether 02:42:ac:12:00:03 brd ff:ff:ff:ff:ff:ff inet 172.18.0.3/16 brd 172.18.255.255 scope global eth2 valid_lft forever preferred_lft forever 23: eth3@if24: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP link/ether 02:42:0a:00:02:07 brd ff:ff:ff:ff:ff:ff inet 10.0.2.7/24 brd 10.0.2.255 scope global eth3 valid_lft forever preferred_lft forever iie4bu@swarm-manager:~$
whoami的IP正是10.255.0.5。
在swarm-worker2上查看：
iie4bu@swarm-worker2:~$ docker container ls CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f47e05019fd9 jwilder/whoami:latest "/app/http" 20 hours ago Up 20 hours 8000/tcp whoami.3.9idgk9jbrlcm3ufvkmbmvv2t8 633ddfc082b9 busybox:1.28.3 "sh -c 'while true; …" 20 hours ago Up 20 hours client.1.3iv3gworpyr5vdo0h9eortlw0 iie4bu@swarm-worker2:~$ docker exec -it f47 ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet 10.255.0.168/32 brd 10.255.0.168 scope global lo valid_lft forever preferred_lft forever inet 10.0.2.5/32 brd 10.0.2.5 scope global lo valid_lft forever preferred_lft forever 24: eth3@if25: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP link/ether 02:42:0a:00:02:0d brd ff:ff:ff:ff:ff:ff inet 10.0.2.13/24 brd 10.0.2.255 scope global eth3 valid_lft forever preferred_lft forever 26: eth2@if27: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP link/ether 02:42:ac:12:00:04 brd ff:ff:ff:ff:ff:ff inet 172.18.0.4/16 brd 172.18.255.255 scope global eth2 valid_lft forever preferred_lft forever 28: eth0@if29: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP link/ether 02:42:0a:ff:00:07 brd ff:ff:ff:ff:ff:ff inet 10.255.0.7/16 brd 10.255.255.255 scope global eth0 valid_lft forever preferred_lft forever
whoami的IP正是10.255.0.7。
因此当我们的数据包进入到ingress_sbox通过lvs做了一个负载均衡，也就是说我们访问8000端口，它会把数据包转发到10.255.0.5和10.255.0.7做一个负载。然后就会进入到swarm节点中了。
看完上述内容，你们掌握docker中RoutingMesh--Ingress负载均衡是什么的方法了吗？如果还想学到更多技能或想了解更多相关内容，欢迎关注亿速云行业资讯频道，感谢各位的阅读！
免责声明：本站发布的内容（图片、视频和文字）以原创、转载和分享为主，文章观点不代表本网站立场，如果涉及侵权请联系站长邮箱：is@yisu.com进行举报，并提供相关证据，一经查实，将立刻删除涉嫌侵权内容。
