本篇是基于k8s-v1.19.0版本
Pod一致性：包含次序（启动、停止次序）、网络一致性。此一致性与Pod相关，与被调度到哪个node节点无关；
1）为什么需要 headless service 无头服务？
在用Deployment时，每一个Pod名称是没有顺序的，是随机字符串，因此是Pod名称是无序的，但是在statefulset中要求必须是有序 ，每一个pod不能被随意取代，pod重建后pod名称还是一样的。而pod IP是变化的，
所以是以Pod名称来识别。pod名称是pod唯一性的标识符，必须持久稳定有效。这时候要用到无头服务，它可以给每个Pod一个唯一的名称 。
2）为什么需要volumeClaimTemplate？
对于有状态的副本集都会用到持久存储，对于分布式系统来讲，它的最大特点是数据是不一样的，所以各个节点不能使用同一存储卷，每个节点有自已的专用存储，但是如果在Deployment中的Pod template里定义的存储卷，
是所有副本集共用一个存储卷，数据是相同的，因为是基于模板来的 ，而statefulset中每个Pod都要自已的专有存储卷，所以statefulset的存储卷就不能再用Pod模板来创建了，于是statefulSet使用volumeClaimTemplate，称为卷申请模板，
它会为每个Pod生成不同的pvc，并绑定pv，从而实现各pod有专用存储。这就是为什么要用volumeClaimTemplate的原因。
本教程假设你的集群被配置为动态的提供 PersistentVolume，动态storeclass参考https://www.cnblogs.com/wuxinchun/p/15266445.html；如果没有这样配置，在开始本教程之前，你需要手动准备存储卷。
如果集群中没有StorageClass的动态供应PVC的机制，也可以提前手动创建多个PV、PVC，手动创建的PVC名称必须符合之后创建的StatefulSet命名规则：(volumeClaimTemplates.name)-(pod_name)
[root@k8s-master statefulset]# pwd /root/k8s_practice/statefulset [root@k8s-master statefulset]# cat nginx-statefulset.yaml apiVersion: v1 kind: Service metadata: name: nginx-headless labels: app: nginx spec: ports: - port: 80 name: web clusterIP: None #注意此处的值,None表示无头服务 selector: app: nginx --- apiVersion: apps/v1 kind: StatefulSet metadata: name: web spec: selector: matchLabels: app: nginx #has to match .spec.template.metadata.labels serviceName: "nginx-headless" replicas: 3 #两个副本 template: metadata: labels: app: nginx spec: containers: - name: nginx image: ikubernetes/myapp:v1 ports: - containerPort: 80 name: web volumeMounts: - name: www mountPath: /usr/share/nginx/html volumeClaimTemplates: - metadata: name: www annotations: volume.beta.kubernetes.io/storage-class: "managed-nfs-storage" #managed-nfs-storage为我们创建的storage-class名称，动态storeclass参考https://www.cnblogs.com/wuxinchun/p/15266445.html spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi [root@k8s-master statefulset]# kubectl apply -f nginx-statefulset.yaml service/nginx-headless created statefulset.apps/web created
1）每个 Pod 都拥有一个基于其顺序索引的稳定的主机名
[root@k8s-master statefulset]# kubectl get pods NAME READY STATUS RESTARTS AGE dns-test 1/1 Running 0 87m nfs-client-provisioner-677fc9c97c-9cj92 1/1 Running 2 7h44m web-0 1/1 Running 0 53s web-1 1/1 Running 0 50s web-2 1/1 Running 0 46s [root@k8s-master statefulset]# kube kubeadm kubectl kubelet [root@k8s-master statefulset]# kubectl exec -it web-0 sh kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. / # hostname web-0 / # exit [root@k8s-master statefulset]# kubectl exec -it web-1 sh kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. / # hostname web-1 / # exit [root@k8s-master statefulset]# kubectl exec -it web-2 sh kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. / # hostname web-2 / # exit
2）使用 kubectl run 运行一个提供 nslookup 命令的容器，该命令来自于 dnsutils 包。通过对 Pod 的主机名执行 nslookup，你可以检查他们在集群内部的 DNS 地址
[root@k8s-master statefulset]# kubectl run -i --tty --image busybox:1.28.4 dns-test1 --restart=Never --rm #创建dns-test测试pod If you don‘t see a command prompt, try pressing enter. / # nslookup web-0 Server: 10.96.0.10 Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local nslookup: can‘t resolve ‘web-0‘ / # nslookup web-1 Server: 10.96.0.10 Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local nslookup: can‘t resolve ‘web-1‘ / # nslookup web-2 Server: 10.96.0.10 Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local nslookup: can‘t resolve ‘web-2‘ / # nslookup nginx-headless Server: 10.96.0.10 Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local Name: nginx-headless #无头service服务名，下面是代理的三个pod地址及其DNS域名 Address 1: 10.244.1.32 web-2.nginx-headless.default.svc.cluster.local Address 2: 10.244.2.29 web-0.nginx-headless.default.svc.cluster.local Address 3: 10.244.1.31 web-1.nginx-headless.default.svc.cluster.local / #
3）删除pod，自动拉起的pod ip地址会变，但是主机名和域名是不变的
[root@k8s-master statefulset]# kubectl delete pod web-0 web-1 web-2 #删除pod pod "web-0" deleted pod "web-1" deleted pod "web-2" deleted [root@k8s-master statefulset]# kubectl get pod #pod自动拉起 NAME READY STATUS RESTARTS AGE nfs-client-provisioner-677fc9c97c-9cj92 1/1 Running 2 7h54m web-0 1/1 Running 0 23s web-1 1/1 Running 0 21s web-2 1/1 Running 0 20s [root@k8s-master statefulset]# kubectl run -i --tty --image busybox:1.28.4 dns-test1 --restart=Never --rm If you don‘t see a command prompt, try pressing enter. / # nslookup nginx-headless Server: 10.96.0.10 Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local Name: nginx-headless #如下查看，pod的IP地址变化了，主机名和dns域名无变化（网络唯一标识） Address 1: 10.244.2.31 web-0.nginx-headless.default.svc.cluster.local Address 2: 10.244.1.33 web-1.nginx-headless.default.svc.cluster.local Address 3: 10.244.2.32 web-2.nginx-headless.default.svc.cluster.local
1）将 Pod 的主机名写入它们的index.html文件并验证 NGINX web 服务器使用该主机名提供服务
[root@k8s-master statefulset]# kubectl exec -it web-0 sh kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. / # echo $(hostname) > /usr/share/nginx/html/index.html / # exit [root@k8s-master statefulset]# kubectl exec -it web-1 sh kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. / # echo $(hostname) > /usr/share/nginx/html/index.html / # exit [root@k8s-master statefulset]# kubectl exec -it web-2 sh kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. / # echo $(hostname) > /usr/share/nginx/html/index.html / # exit
2）删除pod，重新调度后，依然挂载原先PV
[root@k8s-master statefulset]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nfs-client-provisioner-677fc9c97c-9cj92 1/1 Running 2 8h 10.244.1.18 k8s-node1 <none> <none> web-0 1/1 Running 0 6m59s 10.244.2.31 k8s-node2 <none> <none> web-1 1/1 Running 0 6m57s 10.244.1.33 k8s-node1 <none> <none> web-2 1/1 Running 0 6m56s 10.244.2.32 k8s-node2 <none> <none> [root@k8s-master statefulset]# curl 10.244.2.31 web-0 [root@k8s-master statefulset]# curl 10.244.1.33 web-1 [root@k8s-master statefulset]# curl 10.244.2.32 web-2 [root@k8s-master statefulset]# kubectl delete pod web-0 web-1 web-2 pod "web-0" deleted pod "web-1" deleted pod "web-2" deleted [root@k8s-master statefulset]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nfs-client-provisioner-677fc9c97c-9cj92 1/1 Running 2 8h 10.244.1.18 k8s-node1 <none> <none> web-0 1/1 Running 0 14s 10.244.2.33 k8s-node2 <none> <none> web-1 1/1 Running 0 12s 10.244.1.35 k8s-node1 <none> <none> web-2 1/1 Running 0 11s 10.244.2.34 k8s-node2 <none> <none> [root@k8s-master statefulset]# curl 10.244.2.33 web-0 [root@k8s-master statefulset]# curl 10.244.1.35 web-1 [root@k8s-master statefulset]# curl 10.244.2.34 web-2 [root@k8s-master statefulset]#
虽然 web-0 、web-1 和web-2被重新调度了，但它们仍然继续监听各自的主机名，因为和它们的 PersistentVolumeClaim 相关联的 PersistentVolume 被重新挂载到了各自的 volumeMount 上。不管 web-0、web-1、web-3 被调度到了哪个节点上，它们的 PersistentVolumes 将会被挂载到合适的挂载点上
扩容/缩容StatefulSet 指增加或减少它的副本数。这通过更新replicas字段完成。你可以使用kubectl scale 或者kubectl patch来扩容/缩容一个 StatefulSet。
kubectl scale sts web --replicas=4 -n nginx-ss #扩容 kubectl scale sts web --replicas=2 -n nginx-ss #缩容
或者
kubectl patch sts web -p ‘{"spec":{"replicas":4}}‘ -n nginx-ss #扩容 kubectl patch sts web -p ‘{"spec":{"replicas":2}}‘ -n nginx-ss #缩容
1）yaml文件
[root@k8s-master statefulset]# pwd /root/k8s_practice/statefulset [root@k8s-master statefulset]# cat mysql-sts.yaml apiVersion: apps/v1 kind: StatefulSet metadata: name: mysql spec: selector: matchLabels: app: mysql #必须匹配 .spec.template.metadata.labels serviceName: "mysql" #声明它属于哪个Headless Service. replicas: 3 #副本数 template: metadata: labels: app: mysql # 必须配置 .spec.selector.matchLabels spec: terminationGracePeriodSeconds: 10 containers: - name: mysql image: mysql:5.7 ports: - containerPort: 3306 name: mysql env: - name: MYSQL_ROOT_PASSWORD value: "123456" volumeMounts: - name: mysql-pvc mountPath: /var/lib/mysql volumeClaimTemplates: #可看作pvc的模板 - metadata: name: mysql-pvc spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "managed-nfs-storage" #存储类名，改为集群中已存在的 resources: requests: storage: 1Gi
2）statefulset创建mysql
[root@k8s-master statefulset]# kubectl apply -f mysql-sts.yaml statefulset.apps/mysql created [root@k8s-master statefulset]# kubectl get pod NAME READY STATUS RESTARTS AGE mysql-0 1/1 Running 0 2m1s mysql-1 1/1 Running 0 116s mysql-2 1/1 Running 0 110s
3）访问测试
[root@k8s-master statefulset]# kubectl exec -it mysql-0 sh #进入mysql-0 pod kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. # mysql -uroot -p123456 #yaml文件定义root密码为123456 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.35 MySQL Community Server (GPL) Copyright (c) 2000, 2021, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.01 sec) mysql>
4）mysql存储卷
[root@k8s-master volumes]# pwd /nfsdata/volumes [root@k8s-master volumes]# ll total 24 drwxrwxrwx 5 polkitd root 4096 Sep 14 19:35 default-mysql-pvc-mysql-0-pvc-3586c119-3d58-426a-8bbf-a43c6220b27d drwxrwxrwx 5 polkitd root 4096 Sep 14 19:35 default-mysql-pvc-mysql-1-pvc-c2b8838a-ab13-413b-8b80-99fd5d4e5840 drwxrwxrwx 5 polkitd root 4096 Sep 14 19:35 default-mysql-pvc-mysql-2-pvc-614539b7-a3e1-450f-a643-ca980cdb73ff [root@k8s-master volumes]# cd default-mysql-pvc-mysql-0-pvc-3586c119-3d58-426a-8bbf-a43c6220b27d/ [root@k8s-master default-mysql-pvc-mysql-0-pvc-3586c119-3d58-426a-8bbf-a43c6220b27d]# ll total 188476 -rw-r----- 1 polkitd ssh_keys 56 Sep 14 19:35 auto.cnf -rw------- 1 polkitd ssh_keys 1680 Sep 14 19:35 ca-key.pem -rw-r--r-- 1 polkitd ssh_keys 1112 Sep 14 19:35 ca.pem -rw-r--r-- 1 polkitd ssh_keys 1112 Sep 14 19:35 client-cert.pem -rw------- 1 polkitd ssh_keys 1680 Sep 14 19:35 client-key.pem -rw-r----- 1 polkitd ssh_keys 1359 Sep 14 19:35 ib_buffer_pool -rw-r----- 1 polkitd ssh_keys 79691776 Sep 14 19:35 ibdata1 -rw-r----- 1 polkitd ssh_keys 50331648 Sep 14 19:35 ib_logfile0 -rw-r----- 1 polkitd ssh_keys 50331648 Sep 14 19:35 ib_logfile1 -rw-r----- 1 polkitd ssh_keys 12582912 Sep 14 19:35 ibtmp1 drwxr-x--- 2 polkitd ssh_keys 4096 Sep 14 19:35 mysql drwxr-x--- 2 polkitd ssh_keys 4096 Sep 14 19:35 performance_schema -rw------- 1 polkitd ssh_keys 1676 Sep 14 19:35 private_key.pem -rw-r--r-- 1 polkitd ssh_keys 452 Sep 14 19:35 public_key.pem -rw-r--r-- 1 polkitd ssh_keys 1112 Sep 14 19:35 server-cert.pem -rw------- 1 polkitd ssh_keys 1676 Sep 14 19:35 server-key.pem drwxr-x--- 2 polkitd ssh_keys 12288 Sep 14 19:35 sys
注：三个mysql的pod数据是相互独立的
5）创建service使用Nodeport外部访问
[root@k8s-master statefulset]# pwd /root/k8s_practice/statefulset [root@k8s-master statefulset]# cat mysql-svc.yaml apiVersion: v1 kind: Service metadata: name: mysql labels: app: mysql spec: type: NodePort ports: - port: 3306 nodePort: 30001 name: mysql selector: app: mysql [root@k8s-master statefulset]# kubectl delete -f mysql-svc.yaml service "mysql" deleted [root@k8s-master statefulset]# vim mysql-svc.yaml [root@k8s-master statefulset]# kubectl apply -f mysql-svc.yaml service/mysql created [root@k8s-master statefulset]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7d8h mysql NodePort 10.104.90.83 <none> 3306:30001/TCP 3s nginx-headless ClusterIP None <none> 80/TCP 48m [root@k8s-master statefulset]# kubectl get ep NAME ENDPOINTS AGE kubernetes 10.3.104.51:6443 7d8h mysql 10.244.1.36:3306,10.244.1.37:3306,10.244.2.35:3306 6s nginx-headless 10.244.1.35:80,10.244.2.33:80,10.244.2.34:80 48m wxc-nfs-storage <none> 8h [root@k8s-master statefulset]# netstat -antpl | grep 30001 tcp 0 0 0.0.0.0:30001 0.0.0.0:* LISTEN 32270/kube-proxy
注：上述代表通过Nodeport端口访问mysql成功
原文：https://www.cnblogs.com/wuxinchun/p/15269255.html