小编给大家分享一下docker中ingress资源和ingress controller的示例分析,相信大部分人都还不怎么了解,因此分享这篇文章给大家参考一下,希望大家阅读完这篇文章后大有收获,下面让我们一起去了解一下吧!
Ingress:就是能利用 Nginx(不常用)、Haproxy(不常用)、Traefik(常用)、Envoy(常用) 啥的负载均衡器暴露集群内服务的工具。
Ingress为您提供七层负载均衡能力,您可以通过 Ingress 配置提供外部可访问的 URL、负载均衡、SSL、基于名称的虚拟主机等。作为集群流量接入层,Ingress 的高可靠性显得尤为重要。
小知识:我们把k8s里面的pod服务发布到集群外部,可以用ingress,也可以用NodePort。
externalLB:外部的负载均衡器
service site:只是用来给pod分组归类的。
[root@master manifests]# kubectl explain ingress
创建名称空间:
[root@master manifests]# kubectl create namespace ingress-nginx namespace/dev created [root@master manifests]# kubectl get ns NAME STATUS AGE default Active 17d ingress-nginx Active 8s kube-public Active 17d kube-system Active 17d
访问 https://github.com/kubernetes/ingress-nginx,进入deploy目录,里面就有我们要用的yaml文件。
各文件的作用: configmap.yaml:提供configmap可以在线更行nginx的配置 default-backend.yaml:提供一个缺省的后台错误页面 404 namespace.yaml:创建一个独立的命名空间 ingress-nginx rbac.yaml:创建对应的role rolebinding 用于rbac tcp-services-configmap.yaml:修改L4负载均衡配置的configmap udp-services-configmap.yaml:修改L4负载均衡配置的configmap with-rbac.yaml:有应用rbac的nginx-ingress-controller组件
访问https://kubernetes.github.io/ingress-nginx/deploy/#generice-deployment,里面是ingress的部署文档
[root@master ~]# mkdir ingress-nginx [root@master ~]# cd ingress-nginx
部署ingress方法一(分步部署):
下载如下配置文件:
[root@master ingress-nginx]# for file in namespace.yaml configmap.yaml rbac.yaml tcp-services-configmap.yaml with-rbac.yaml udp-services-configmap.yaml; do wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/$file; done
[root@master ingress-nginx]# ls configmap.yaml namespace.yaml rbac.yaml tcp-services-configmap.yaml udp-services-configmap.yaml with-rbac.yaml
1、创建名称空间:
[root@master ingress-nginx]# kubectl apply -f namespace.yaml namespace/ingress-nginx configured
2、把剩下的ymal文件全应用
[root@master ingress-nginx]# kubectl apply -f ./ configmap/nginx-configuration created namespace/ingress-nginx configured serviceaccount/nginx-ingress-serviceaccount created clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created role.rbac.authorization.k8s.io/nginx-ingress-role created rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created configmap/tcp-services created configmap/udp-services created deployment.extensions/nginx-ingress-controller created
[root@master ingress-nginx]# kubectl get pods -n ingress-nginx -w NAME READY STATUS RESTARTS AGE default-http-backend-6586bc58b6-qd9fk 0/1 running 0 4m nginx-ingress-controller-6bd7c597cb-zcbbz 0/1 running 0 1m
可以看到ingress-nginx名称空间里面有两个pod都处于running状态
部署ingress方法二(一键部署):
只下载mandatory.yaml文件,因为这个文件里面包含了上面所有yaml文件里面的内容。这是一键部署。
[root@master ingress-nginx]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
[root@master ingress-nginx]#kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
[root@master ~]# kubectl get pods -n ingress-nginx -w -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE default-http-backend-6586bc58b6-qd9fk 1/1 Running 0 11h 10.244.1.95 node1 <none> nginx-ingress-controller-6bd7c597cb-jlqzp 1/1 Running 3 11h 10.244.1.96 node1 <none>
可以看到ingress-nginx名称空间里面有两个pod都处于running状态
安装service-nodeport
上面我们把ingress-nginx部署到了1号node上。接下来我们还需要部署一个service-nodeport服务,才能实现把集群外部流量接入到集群中来。
[root@master ingress]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml
我们为了不让service nodeport自动分配端口,我们自己指定一下nodeport,修改文件中加两个nodePort参数,最终如下:
[root@master ingress]# cat service-nodeport.yaml apiVersion: v1 kind: Service metadata: name: ingress-nginx namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: type: NodePort ports: - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30080 - name: https port: 443 targetPort: 443 protocol: TCP nodePort: 30443 selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx
[root@master ingress]# kubectl apply -f service-nodeport.yaml service/ingress-nginx created
[root@master ingress]# kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default-http-backend ClusterIP 10.110.74.183 <none> 80/TCP 12h ingress-nginx NodePort 10.102.78.188 <none> 80:30080/TCP,443:30443/TCP 2m
上面我看到80对应30080,,43对应30443
我们直接通过node1节点的ip就可以访问到应用:
[root@master ingress]# curl http://172.16.1.101:30080 default backend - 404
定义myapp service
[root@master manifests]# mkdir /root/manifests/ingress
[root@master ~]# kubectl explain service.spec.ports
[root@master ingress]# cat deploy-demo.yaml apiVersion: v1 kind: Service #必须设置为无头service metadata: name: myapp namespace: default spec: selector: app: myapp release: canary ports: - name: http targetPort: 80 #这是容器port port: 80 #这是service port --- apiVersion: apps/v1 kind: Deployment metadata: name: myapp-deploy namespace: default spec: replicas: 2 selector: #标签选择器 matchLabels: #匹配的标签为 app: myapp release: canary template: metadata: labels: app: myapp #和上面的myapp要匹配 release: canary spec: containers: - name: myapp image: ikubernetes/myapp:v1 ports: - name: http containerPort: 80
[root@master ingress]# kubectl apply -f deploy-demo.yaml service/myapp created deployment.apps/myapp-deploy unchanged
[root@master ingress]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17d myapp ClusterIP 10.108.177.62 <none> 80/TCP 1m
[root@master ingress]# kubectl get pods NAME READY STATUS RESTARTS AGE myapp-deploy-69b47bc96d-79fqh 1/1 Running 0 1d myapp-deploy-69b47bc96d-tc54k 1/1 Running 0 1d
把myapp service通过ingress发布出去
下面我们再定义一个清单文件,把myapp应用通过Ingress(相当于nginx的反向代理功能)发布出去:
[root@master ingress]# cat ingress-myapp.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: ingress-myapp namespace: default #要和deployment和要发布的service处于同一个名称空间 annotations: #这个注解说明我们要用到的ingress-controller是nginx,而不是traefic,enjoy kubernetes.io/ingress.class: "nginx" spec: rules: - host: myapp.zhixin.com #表示访问这个域名,就会转发到后端myapp管理的pod上的服务: http: paths: - path: backend: serviceName: myapp servicePort: 80
[root@master ingress]# kubectl apply -f ingress-myapp.yaml ingress.extensions/ingress-myapp created
[root@master ingress]# kubectl get ingress NAME HOSTS ADDRESS PORTS AGE ingress-myapp myapp.zhixin.com 80 8m
[root@master ingress]# kubectl describe ingress
[root@master ingress]# kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGE default-http-backend-6586bc58b6-qd9fk 1/1 Running 0 12h nginx-ingress-controller-6bd7c597cb-jlqzp 1/1 Running 3 12h
进入ingress-controller交互式命令行里面,可以清晰的看到nginx是怎么反向代理我们myapp.zhixin.com的:
[root@master ingress]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-6bd7c597cb-jlqzp -- /bin/sh $ cat nginx.conf ## start server myapp.zhixin.com server { server_name myapp.zhixin.com ; listen 80; set $proxy_upstream_name "-"; location / { set $namespace "default"; set $ingress_name "ingress-myapp"; set $service_name "myapp"; set $service_port "80"; ........
测试,下面我们把myapp.zhixin.com域名解析到node1 ip 172.16.1.101上。
[root@master ingress]# curl myapp.zhixin.com:30080 Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
把tomcat service通过ingress发布出去(新例子)
[root@master ingress]# cat tomcat-demo.yaml apiVersion: v1 kind: Service #必须设置为无头service metadata: name: tomcat namespace: default spec: selector: app: tomcat release: canary ports: - name: http targetPort: 8080 #这是容器port port: 8080 #这是service port - name: ajp targetPort: 8009 port: 8009 --- apiVersion: apps/v1 kind: Deployment metadata: name: tomcat-deploy namespace: default spec: replicas: 2 selector: #标签选择器 matchLabels: #匹配的标签为 app: tomcat release: canary template: metadata: labels: app: tomcat #和上面的myapp要匹配 release: canary spec: containers: - name: tomcat image: tomcat:8.5.34-jre8-alpine #在https://hub.docker.com/r/library/tomcat/tags/上面找 ports: - name: http containerPort: 8080 - name: ajp containerPort: 8009
[root@master ingress]# kubectl apply -f tomcat-demo.yaml service/tomcat created deployment.apps/tomcat-deploy created
[root@master ingress]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17d tomcat ClusterIP 10.109.76.87 <none> 8080/TCP,8009/TCP 1m
[root@master ingress]# kubectl get pods NAME READY STATUS RESTARTS AGE tomcat-deploy-64c4d54df4-68sk8 1/1 Running 0 51s tomcat-deploy-64c4d54df4-7b58g 1/1 Running 0 51s
[root@master ingress]# cat ingress-tomcat.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: ingress-tomcat namespace: default #要和deployment和要发布的service处于同一个名称空间 annotations: #这个注解说明我们要用到的ingress-controller是nginx,而不是traefic,enjoy kubernetes.io/ingress.class: "nginx" spec: rules: - host: tomcat.zhixin.com #表示访问这个域名,就会转发到后端myapp管理的pod上的服务: http: paths: - path: backend: serviceName: tomcat servicePort: 8080
[root@master ingress]# kubectl apply -f ingress-tomcat.yaml ingress.extensions/ingress-myapp configured
[root@master ingress]# kubectl get ingress NAME HOSTS ADDRESS PORTS AGE ingress-tomcat tomcat.zhixin.com 80 11s
[root@master ingress]# kubectl describe ingress ingress-tomcat Name: ingress-tomcat Namespace: default Address: Default backend: default-http-backend:80 (<none>) Rules: Host Path Backends ---- ---- -------- tomcat.zhixin.com tomcat:8080 (<none>) Annotations: kubernetes.io/ingress.class: nginx Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 1m nginx-ingress-controller Ingress default/ingress-tomcat
把tomcat.zhixin.com解析到node1上节点物理ip(我的是172.16.1.101)
测试,可以看到tomcat欢迎界面:
[root@master ingress]# curl tomcat.zhixin.com:30080
使用https访问(新例子)
1、先做个自签的证书(我们这里不演示CA的例子)
[root@master ingress]# openssl genrsa -out tls.key 2048
[root@master ingress]# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/O=DevOps/CN=tomcat.zhixin.com
2、通过secret把证书注入到pod中。
[root@master ingress]# kubectl create secret tls tomcat-infress-secret --cert=tls.crt --key=tls.key secret/tomcat-infress-secret created
[root@master ingress]# kubectl get secret NAME TYPE DATA AGE default-token-5r85r kubernetes.io/service-account-token 3 17d tomcat-ingress-secret kubernetes.io/tls 2 41s
[root@master ingress]# kubectl describe secret tomcat-ingress-secret Name: tomcat-ingress-secret Namespace: default Labels: <none> Annotations: <none> Type: kubernetes.io/tls Data ==== tls.crt: 1245 bytes tls.key: 1679 bytes
3、配置ingress为tls方式
[root@master ingress]# cat ingress-tomcat-tls.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: ingress-tomcat-tls namespace: default #要和deployment和要发布的service处于同一个名称空间 annotations: #这个注解说明我们要用到的ingress-controller是nginx,而不是traefic,enjoy kubernetes.io/ingress.class: "nginx" spec: tls: - hosts: - tomcat.zhixin.com secretName: tomcat-ingress-secret #kubectl get secret命令查到的名字 rules: - host: tomcat.zhixin.com #表示访问这个域名,就会转发到后端myapp管理的pod上的服务: http: paths: - path: backend: serviceName: tomcat servicePort: 8080
[root@master ingress]# kubectl get ingress NAME HOSTS ADDRESS PORTS AGE ingress-tomcat tomcat.zhixin.com 80 2h ingress-tomcat-tls tomcat.zhixin.com 80, 443 3m
[root@master ingress]# kubectl describe ingress ingress-tomcat-tls Name: ingress-tomcat-tls Namespace: default Address: Default backend: default-http-backend:80 (<none>) TLS: tomcat-ingress-secret terminates tomcat.zhixin.com Rules: Host Path Backends ---- ---- -------- tomcat.zhixin.com tomcat:8080 (<none>) Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-tomcat-tls","namespace":"default"},"spec":{"rules":[{"host":"tomcat.zhixin.com","http":{"paths":[{"backend":{"serviceName":"tomcat","servicePort":8080},"path":null}]}}],"tls":[{"hosts":["tomcat.zhixin.com"],"secretName":"tomcat-ingress-secret"}]}} kubernetes.io/ingress.class: nginx Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 4m nginx-ingress-controller Ingress default/ingress-tomcat-tls
4、连如ingress-controller查看nginx.conf的配置
[root@master ingress]# kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGE default-http-backend-6586bc58b6-qd9fk 1/1 Running 0 16h nginx-ingress-controller-6bd7c597cb-jlqzp 1/1 Running 3 16h
[root@master ingress]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-6bd7c597cb-jlqzp -- /bin/sh $ $ cat nginx.conf ## start server tomcat.zhixin.com server { server_name tomcat.zhixin.com ; listen 80; set $proxy_upstream_name "-"; listen 443 ssl http2;
看到有listen 443了。
5、测试https
[root@master ingress]# curl https://tomcat.zhixin.com:30443
前面两种部署方法,是用node ip + 非80端口,访问k8s集群内部的服务。可是,我们实际生产中更希望的是node ip + 80端口的方式,访问k8s集群内的服务。我感觉这个方法最好,下面就就介绍这个方法。
这部分内容参考的是博文http://blog.51cto.com/devingeng/2149377
下载地址
https://github.com/kubernetes/ingress-nginx/archive/nginx-0.11.0.tar.gz
ingress-nginx文件位于deploy目录下,各文件的作用: configmap.yaml:提供configmap可以在线更行nginx的配置 default-backend.yaml:提供一个缺省的后台错误页面 404 namespace.yaml:创建一个独立的命名空间 ingress-nginx rbac.yaml:创建对应的role rolebinding 用于rbac tcp-services-configmap.yaml:修改L4负载均衡配置的configmap udp-services-configmap.yaml:修改L4负载均衡配置的configmap with-rbac.yaml:有应用rbac的nginx-ingress-controller组件
修改with-rbac.yaml
apiVersion: extensions/v1beta1 kind: Daemonset metadata: name: nginx-ingress-controller namespace: ingress-nginx spec: selector: matchLabels: app: ingress-nginx template: metadata: labels: app: ingress-nginx annotations: prometheus.io/port: '10254' prometheus.io/scrape: 'true' spec: serviceAccountName: nginx-ingress-serviceaccount hostNetwork: true containers: - name: nginx-ingress-controller image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.11.0 args: - /nginx-ingress-controller - --default-backend-service=$(POD_NAMESPACE)/default-http-backend - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --annotations-prefix=nginx.ingress.kubernetes.io env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ports: - name: http containerPort: 80 - name: https containerPort: 443 livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 nodeSelector: custom/ingress-controller-ready: "true"
需要修改的地方:
kind: DaemonSet:官方原始文件使用的是deployment,replicate 为 1,这样将会在某一台节点上启动对应的nginx-ingress-controller pod。外部流量访问至该节点,由该节点负载分担至内部的service。测试环境考虑防止单点故障,改为DaemonSet然后删掉replicate ,配合亲和性部署在制定节点上启动nginx-ingress-controller pod,确保有多个节点启动nginx-ingress-controller pod,后续将这些节点加入到外部硬件负载均衡组实现高可用性。
hostNetwork: true:添加该字段,暴露nginx-ingress-controller pod的服务端口(80)
nodeSelector: 增加亲和性部署,有custom/ingress-controller-ready 标签的节点才会部署该DaemonSet
为需要部署nginx-ingress-controller的节点设置lable
kubectl label nodes vmnode2 custom/ingress-controller-ready=true kubectl label nodes vmnode3 custom/ingress-controller-ready=true kubectl label nodes vmnode4 custom/ingress-controller-ready=true
加载yaml文件
kubectl apply -f namespace.yaml kubectl apply -f default-backend.yaml kubectl apply -f configmap.yaml kubectl apply -f tcp-services-configmap.yaml kubectl apply -f udp-services-configmap.yaml kubectl apply -f rbac.yaml kubectl apply -f with-rbac.yaml
查看pod是否正常创建
##下载镜像可能会比较慢,等待一会所有pod都是Running状态,按Ctrl + c 退出
[root@vmnode1 deploy]# kubectl get pods --namespace=ingress-nginx --watch NAME READY STATUS RESTARTS AGEdefault- http-backend-6c59748b9b-hc8q9 1/1 Running 0 6m nginx-ingress-controller-7fmlp 1/1 Running 1 13d nginx-ingress-controller-j95fb 1/1 Running 1 13d nginx-ingress-controller-ld2jw 1/1 Running 1 13d
测试ingress
创建一个tomcat的Service
[root@k8s-master1 test]# cat mytomcat.yaml apiVersion: extensions/v1beta1 kind: Deployment metadata: name: mytomcat spec: replicas: 2 template: metadata: labels: run: mytomcat spec: containers: - name: mytomcat image: tomcat ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: mytomcat labels: run: mytomcat spec: type: NodePort ports: - port: 8080 targetPort: 8080 selector: run: mytomcat
[root@k8s-master1 test]# kubectl apply -f mytomcat.yaml
配置ingress转发文件:
[root@k8s-master1 test]# cat test-ingress.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: test-ingress namespace: default spec: rules: - host: test.zhixin.com http: paths: - path: / backend: serviceName: mytomcat servicePort: 8080
host: 对应的域名
path: url上下文
backend:后向转发 到对应的 serviceName: servicePort:
[root@k8s-master1 test]# kubectl apply -f test-ingress.yaml ingress.extensions/test-ingress created
nginx-ingress-controller运行在node1和nod2两个节点上。如果网络中有dns服务器,在dns中把这两个域名映射到nginx-ingress-controller运行的任意一个节点上,如果没有dns服务器只能修改host文件了。
正规的做法是在node1和node2这两个节点上安装keepalive,生成一个vip。在dns上把域名和vip做映射。
我这里直接在node1节点上操作了:
我这里node1节点的ip是172.16.22.201;node2节点的ip是172.16.22.202
[root@k8s-master1 test]# echo " 172.16.22.201 test.zhixin.com" >> /etc/hosts [root@k8s-master1 test]# echo "172.16.22.202 test.zhixin.com" >> /etc/hosts
然后访问测试:
看到,我们把域名test.zhixin.com绑定到Node节点的ip补上,然后我们直接访问http://test.zhixin.com,就能访问到k8s集群里面的pod服务。
以上是“docker中ingress资源和ingress controller的示例分析”这篇文章的所有内容,感谢各位的阅读!相信大家都有了一定的了解,希望分享的内容对大家有所帮助,如果还想学习更多知识,欢迎关注亿速云行业资讯频道!
免责声明:本站发布的内容(图片、视频和文字)以原创、转载和分享为主,文章观点不代表本网站立场,如果涉及侵权请联系站长邮箱:is@yisu.com进行举报,并提供相关证据,一经查实,将立刻删除涉嫌侵权内容。