Nova与Neutron节点接口架构图
继上一期的Nova节点网络逻辑关系,本期继续上一期话题,主要讲Neutron节点上的组件与接口,继续引用“逻辑图”。
1.通过以上图,从网络节点(Neutron)入手,通过使用相关命令去查询一些信息,操作如下:
[root@controller01 ~]# ip a | grep q
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
6: br-int: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN
11: tapf733605f-e2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
12: tapeb25bcee-1b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
13: tap3f46f81b-4f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
18: tap0a0688b4-f8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
27: tapfb9f4bc2-30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
39: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN
40: eth2.13@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
41: eth2.2@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP
42: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
43: tapf36b911d-fa: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
44: tap4ba24208-80: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UP qlen 1000
从1的命令上可以看到,在这有三个Linux bridge,还有大部分是tap接口,还有一些被隐藏的qr,而其中eth2.2@eth2和eth2.13是使用虚拟网卡技术切片的虚拟网卡,br-ex是出外网络的网桥,继续用命令查看
2.继续查看ovs的接口信息,操作如下:
[root@controller01~]# ovs-vsctl show
2b375334-9ce8-46f2-b5ee-ac60781da7f7
Bridge br-ex
Port "eth2.2"
Interface "eth2.2"
Port "tap4ba24208-80"
Interface "tap4ba24208-80"
Port br-ex
Interface br-ex
type: internal
Bridge br-int
fail_mode: secure
Port "tap0a0688b4-f8"
tag: 1
Interface "tap0a0688b4-f8"
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "tapeb25bcee-1b"
tag: 3
Interface "tapeb25bcee-1b"
Port "tap3f46f81b-4f"
tag: 3
Interface "tap3f46f81b-4f"
Port "tapf733605f-e2"
tag: 4095
Interface "tapf733605f-e2"
Port "tapf36b911d-fa"
tag: 4
Interface "tapf36b911d-fa"
Port "tapfb9f4bc2-30"
tag: 4
Interface "tapfb9f4bc2-30"
Port br-int
Interface br-int
type: internal
Bridge br-tun
fail_mode: secure
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port "vxlan-0af80903"
Interface "vxlan-0af80903"
type: vxlan
options: {df_default="true", in_key=flow, local_ip="10.248.9.2", out_key=flow, remote_ip="10.248.9.3"}
Port "vxlan-0af80905"
Interface "vxlan-0af80905"
type: vxlan
options: {df_default="true", in_key=flow, local_ip="10.248.9.2", out_key=flow, remote_ip="10.248.9.5"}
Port "vxlan-0af80901"
Interface "vxlan-0af80901"
type: vxlan
options: {df_default="true", in_key=flow, local_ip="10.248.9.2", out_key=flow, remote_ip="10.248.9.1"}
Port br-tun
Interface br-tun
type: internal
ovs_version: "2.1.3"
从2上可以看到有很多接口,而此次只针对一个路由器和一个HDCP,还有针对以上三个网桥进行查看。
3.查询以上两个网桥配置信息,操作如下:
[root@controller01 ~]# ovs-vsctl list-ports br-int
patch-tun
tap0a0688b4-f8
tap3f46f81b-4f
tapeb25bcee-1b
tapf36b911d-fa
tapf733605f-e2
tapfb9f4bc2-30
[root@controller01 ~]# ovs-vsctl list-ports br-tun
patch-int
vxlan-0af80901
vxlan-0af80903
vxlan-0af80905
[root@controller01 ~]# ovs-vsctl list-ports br-ex
eth2.2
从3上命令可以看到:
A.br-int有很多接口,其中patch-tun是网桥接口,是和br-tun对接的,而tap接口都是DHCP的接口
B.br-tun有三个Vxlan隧道接口,点对点,这是计算节点和网络节点隧道通讯,而patch-int是网桥接口,是和br-int对接的。
C.br-ex只有一个虚拟的网口eth2.2加入,此网口是通向外部网络IP地址段:124.56.30.0/27,也就是所以虚拟主机与外部网络通讯的基础。
4.根据逻辑图,继续查询qrouter和dhcp,这两个是网络命名空间(network namespace)的元素,有空的同学可以了解一个,在此只举一个例子,让同学们了解它们的接口结构,操作如下:
[root@controller01 ~]# ip netns
qdhcp-98daaa5b-3481-44d5-b697-76c8705a2409 -----DHCP的例子
qdhcp-237be48f-e8c0-4f35-93f6-8193005cdb21
qdhcp-50cabb4c-7b6c-44eb-a26b-d240132798c8
qdhcp-141e2f07-4ef0-4f1e-b0a9-1b7113072d8c
qrouter-4d992933-9f4e-4fe6-9507-a75649ef37db -----软路由的例子
从4以上命令可以查看到上面有四个DHCP和一个软router,接下继续分析这两个网络命名空间的详细配置信息。
5.查询在IP Addr看不到隐藏的端口:qr,qg,ns都可以检查到了,使用以下指令查询:
[root@controller01~]# ip netns exec qdhcp-98daaa5b-3481-44d5-b697-76c8705a2409 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ns-f36b911d-fa: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP qlen 1000 ---在IP Add 可以查看到的tap ID 一样
link/ether fa:16:3e:f4:08:38 brd ff:ff:ff:ff:ff:ff
inet 172.16.0.2/24 brd 172.16.0.255 scope global ns-f36b911d-fa ---这DHCP自有IP
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fef4:838/64 scope link
valid_lft forever preferred_lft forever
[root@controller01 ~]# ip netns exec qrouter-4d992933-9f4e-4fe6-9507-a75649ef37db ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: qr-3f46f81b-4f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP qlen 1000
link/ether fa:16:3e:e6:48:fb brd ff:ff:ff:ff:ff:ff
inet 10.0.0.1/24 brd 10.0.0.255 scope global qr-3f46f81b-4f
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fee6:48fb/64 scope link
valid_lft forever preferred_lft forever
4: qr-fb9f4bc2-30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP qlen 1000 ---到内网的网关
link/ether fa:16:3e:1a:84:f0 brd ff:ff:ff:ff:ff:ff
inet 172.16.0.1/24 brd 172.16.0.255 scope global qr-fb9f4bc2-30
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe1a:84f0/64 scope link
valid_lft forever preferred_lft forever
7: qg-4ba24208-80: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP qlen 1000 ---到外网的网关
link/ether fa:16:3e:d4:19:15 brd ff:ff:ff:ff:ff:ff
inet 124.56.30.119/27 brd 123.58.34.127 scope global qg-4ba24208-80
valid_lft forever preferred_lft forever
inet 124.56.30.118/32 brd 123.58.34.118 scope global qg-4ba24208-80
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fed4:1915/64 scope link
valid_lft forever preferred_lft forever
[root@controller01 ~]# ip netns exec qdhcp-98daaa5b-3481-44d5-b697-76c8705a2409 route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 172.16.0.1 0.0.0.0 UG 0 0 0 ns-f36b911d-fa ---在IP Add 可以查看到的tap ID 一样
172.16.0.0 0.0.0.0 255.255.255.0 U 0 0 0 ns-f36b911d-fa
[root@controller01 ~]# ip netns exec qrouter-4d992933-9f4e-4fe6-9507-a75649ef37db route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 124.56.30.97 0.0.0.0 UG 0 0 0 qg-4ba24208-80 ---到外网的网关
10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-3f46f81b-4f
124.56.30.96 0.0.0.0 255.255.255.224 U 0 0 0 qg-4ba24208-80
172.16.0.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-fb9f4bc2-30 ---到内网的网关
从5加上以前的信息,现在已经完全清楚Nova与Neutron之间逻辑关系,以及数据流向,qg-4ba24208-80 是向外到外网的路由端口,qr-fb9f4bc2-30 是向内网到虚拟云主机的路由端口,这两个端口都是网关端口。 ns-f36b911d-fa实际就是tapf36b911d-fa与br-int连接,所以大家可能假设数据流从instance0开始到外网是怎么流的,尝试自己走一遍,基本就知道Openstack的网络结构。
总结:看完Nova与Neutron的网络逻辑关系后,相信大家对Openstack的网络结构有一个初步了解,对一些问题排查也有一定作用了,当然我个人觉得整个网络结构里面Linux Bridge知识很重要,还有隧道技术,网络命名空间等技术如果大家去了解,将会帮且我们更好了解Neutron,谢谢!
免责声明:本站发布的内容(图片、视频和文字)以原创、转载和分享为主,文章观点不代表本网站立场,如果涉及侵权请联系站长邮箱:is@yisu.com进行举报,并提供相关证据,一经查实,将立刻删除涉嫌侵权内容。