kubernetes(k8s) 集群 安装配置

张映 发表于 2020-04-28

分类目录： 云计算

标签：, ,

Kubernetes是Google开源的一个容器编排引擎，它支持自动化部署、大规模可伸缩、应用容器化管理。在生产环境中部署一个应用程序时，通常要部署该应用的多个实例以便对应用请求进行负载均衡。

在Kubernetes中，我们可以创建多个容器，每个容器里面运行一个应用实例，然后通过内置的负载均衡策略，实现对这一组应用实例的管理、发现、访问，而这些细节都不需要运维人员去进行复杂的手工配置和处理。

一，服务器介绍

  1. 10.0.40.222 bigserver2 worker  
  2. 10.0.40.193 bigserver3 master  

二，设置hostname

1，修改hostname

  1. # hostname bigserver2  

2，设置hosts文件

  1. # cat <<EOF >>/etc/hosts  
  2. 10.0.40.222 bigserver2  
  3. 10.0.40.193 bigserver3  
  4. EOF  

三，关闭防火墙，selinux，交换分区

查看复制打印?
  1. # systemctl stop firewalld  
  2. # systemctl disable firewalld  
  3.   
  4. # setenforce 0  
  5. # sed -i "s/^SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config  
  6.   
  7. # swapoff -a  
  8. # sed -i 's/.*swap.*/#&/' /etc/fstab  

kubernetes高版本，是需要关闭swap分区，不然启动不起来。本篇文章安装版本是1.18.2

四，配置内核参数，并生效

  1. # cat > /etc/sysctl.d/k8s.conf <<EOF  
  2. net.bridge.bridge-nf-call-ip6tables = 1  
  3. net.bridge.bridge-nf-call-iptables = 1  
  4. net.ipv4.ip_forward = 1  
  5. EOF  
  6.   
  7. # sysctl --system  

五，配置yum源

1，base,epel换阿里源

查看复制打印?
  1. # cd /etc/yum.repos.d/  
  2. # mv CentOS-Base.repo CentOS-Base_bak  
  3. # mv epel.repo epel_bak   
  4.   
  5. # wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo  
  6. # wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo  

2，配置k8s源

查看复制打印?
  1. # cat <<EOF > /etc/yum.repos.d/kubernetes.repo  
  2. [kubernetes]  
  3. name=Kubernetes  
  4. baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/  
  5. enabled=1  
  6. gpgcheck=1  
  7. repo_gpgcheck=1  
  8. gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg  
  9. EOF  

3，设置docker源

查看复制打印?
  1. # wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo  
  2. # yum clean all && yum makecache  

六，安装Kubernetes

1，安装docker

  1. # yum install -y docker-ce  
  2.   
  3. # systemctl enable docker && systemctl start docker  
  4.   
  5. # docker -v  
  6. Docker version 19.03.8, build afacb8b  

2，安装kubelet kubeadm kubectl

  1. # yum install kubelet kubeadm kubectl  
  2. # systemctl enable kubelet  

在这里没有直接安装kubernetes，自带的k8s是1.5版的，太过老旧。通过这种方式安装的是稳定最新版。

在这里要注意，systemctl enable kubelet，不然会面操作时会报warning

[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

Kubelet负责与其他节点集群通信，并进行本节点Pod和容器生命周期的管理。Kubeadm是Kubernetes的自动化部署工具，降低了部署难度，提高效率。Kubectl是Kubernetes集群管理工具。

备注：到此步，以上要在所有节点上配置。

七，master（10.0.40.193）节点，配置kubernetes

1，初始化kubernetes集群

查看复制打印?
  1. [root@bigserver3 ~]# kubeadm init --kubernetes-version=1.18.2 \  
  2. > --apiserver-advertise-address=10.0.40.193 \  
  3. > --image-repository registry.aliyuncs.com/google_containers \  
  4. > --service-cidr=10.1.0.0/16 \  
  5. > --pod-network-cidr=10.244.0.0/16 \  
  6. > --ignore-preflight-errors=Swap  
  7. W0427 18:38:50.464858 20657 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]  
  8. [init] Using Kubernetes version: v1.18.2  
  9. [preflight] Running pre-flight checks  
  10.  [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/  
  11. [preflight] Pulling images required for setting up a Kubernetes cluster  
  12. [preflight] This might take a minute or two, depending on the speed of your internet connection  
  13. [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'  
  14. [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"  
  15. [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"  
  16. [kubelet-start] Starting the kubelet  
  17. [certs] Using certificateDir folder "/etc/kubernetes/pki"  
  18. [certs] Generating "ca" certificate and key  
  19. [certs] Generating "apiserver" certificate and key  
  20. [certs] apiserver serving cert is signed for DNS names [bigserver3 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.1.0.1 10.0.40.193]  
  21. [certs] Generating "apiserver-kubelet-client" certificate and key  
  22. [certs] Generating "front-proxy-ca" certificate and key  
  23. [certs] Generating "front-proxy-client" certificate and key  
  24. [certs] Generating "etcd/ca" certificate and key  
  25. [certs] Generating "etcd/server" certificate and key  
  26. [certs] etcd/server serving cert is signed for DNS names [bigserver3 localhost] and IPs [10.0.40.193 127.0.0.1 ::1]  
  27. [certs] Generating "etcd/peer" certificate and key  
  28. [certs] etcd/peer serving cert is signed for DNS names [bigserver3 localhost] and IPs [10.0.40.193 127.0.0.1 ::1]  
  29. [certs] Generating "etcd/healthcheck-client" certificate and key  
  30. [certs] Generating "apiserver-etcd-client" certificate and key  
  31. [certs] Generating "sa" key and public key  
  32. [kubeconfig] Using kubeconfig folder "/etc/kubernetes"  
  33. [kubeconfig] Writing "admin.conf" kubeconfig file  
  34. [kubeconfig] Writing "kubelet.conf" kubeconfig file  
  35. [kubeconfig] Writing "controller-manager.conf" kubeconfig file  
  36. [kubeconfig] Writing "scheduler.conf" kubeconfig file  
  37. [control-plane] Using manifest folder "/etc/kubernetes/manifests"  
  38. [control-plane] Creating static Pod manifest for "kube-apiserver"  
  39. [control-plane] Creating static Pod manifest for "kube-controller-manager"  
  40. W0427 18:39:40.388099 20657 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"  
  41. [control-plane] Creating static Pod manifest for "kube-scheduler"  
  42. W0427 18:39:40.390080 20657 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"  
  43. [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"  
  44. [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s  
  45. [apiclient] All control plane components are healthy after 34.525978 seconds  
  46. [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace  
  47. [kubelet] Creating a ConfigMap "kubelet-config-1.18" in namespace kube-system with the configuration for the kubelets in the cluster  
  48. [upload-certs] Skipping phase. Please see --upload-certs  
  49. [mark-control-plane] Marking the node bigserver3 as control-plane by adding the label "node-role.kubernetes.io/master=''"  
  50. [mark-control-plane] Marking the node bigserver3 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]  
  51. [bootstrap-token] Using token: 8ylj4t.8kzvk4ro7hgdehwk  
  52. [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles  
  53. [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes  
  54. [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials  
  55. [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token  
  56. [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster  
  57. [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace  
  58. [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key  
  59. [addons] Applied essential addon: CoreDNS  
  60. [addons] Applied essential addon: kube-proxy  
  61.   
  62. Your Kubernetes control-plane has initialized successfully!  
  63.   
  64. To start using your cluster, you need to run the following as a regular user:  
  65.   
  66.  mkdir -p $HOME/.kube   //注意这段要在master节点执行的  
  67.  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config  
  68.  sudo chown $(id -u):$(id -g) $HOME/.kube/config  
  69.   
  70. You should now deploy a pod network to the cluster.  
  71. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:  
  72.   
  73. https://kubernetes.io/docs/concepts/cluster-administration/addons/  
  74.   
  75. Then you can join any number of worker nodes by running the following on each as root:  
  76. //在worker节点执行以下内容  
  77. kubeadm join 10.0.40.193:6443 --token 8ylj4t.8kzvk4ro7hgdehwk \  
  78.  --discovery-token-ca-cert-hash sha256:46f6cf1d84d0eadb4f6e7f05b908e5572025886d9f134db27f92b98e1c3dd3ed  
--image-repository：指定仓库地址
--ignore-preflight-errors：将错误显示为警告的检查列表进行忽略
--pod-network-cidr：指定pod网络的IP地址范围。如果设置，控制平面将为每个节点自动分配CIDRs。
--service-cidr：为service VIPs使用不同的IP地址。(默认“10.96.0.0/12”)

2，配置kubectl

查看复制打印?
  1. # echo $HOME  
  2. /root  
  3. # mkdir -p $HOME/.kube  
  4. # cp -i /etc/kubernetes/admin.conf $HOME/.kube/config  

3，部署flannel网络

查看复制打印?
  1. # wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml  
  2. # kubectl apply -f kube-flannel.yml  
  3. podsecuritypolicy.policy/psp.flannel.unprivileged created  
  4. clusterrole.rbac.authorization.k8s.io/flannel configured  
  5. clusterrolebinding.rbac.authorization.k8s.io/flannel unchanged  
  6. serviceaccount/flannel unchanged  
  7. configmap/kube-flannel-cfg configured  
  8. daemonset.apps/kube-flannel-ds-amd64 created  
  9. daemonset.apps/kube-flannel-ds-arm64 created  
  10. daemonset.apps/kube-flannel-ds-arm created  
  11. daemonset.apps/kube-flannel-ds-ppc64le created  
  12. daemonset.apps/kube-flannel-ds-s390x created  

注意：如果有多个网卡，要指定网卡，防止DNS无法解释

args:
- --ip-masq
- --kube-subnet-mgr
- --iface=eth0    #指定网卡

八，worker节点加入，kubernetes集群

查看复制打印?
  1. [root@bigserver2 yum.repos.d]# kubeadm join 10.0.40.193:6443 --token y4d0ws.w8lxmohfc1o0yq6b \  
  2. > --discovery-token-ca-cert-hash sha256:46f6cf1d84d0eadb4f6e7f05b908e5572025886d9f134db27f92b98e1c3dd3ed --ignore-preflight-errors=all  
  3. W0428 15:07:33.725621 17900 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.  
  4. [preflight] Running pre-flight checks  
  5.  [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/  
  6. [preflight] Reading configuration from the cluster...  
  7. [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'  
  8. [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.18" ConfigMap in the kube-system namespace  
  9. [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"  
  10. [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"  
  11. [kubelet-start] Starting the kubelet  
  12. [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...  
  13.   
  14. This node has joined the cluster:  
  15. * Certificate signing request was sent to apiserver and a response was received.  
  16. * The Kubelet was informed of the new secure connection details.  
  17.   
  18. Run 'kubectl get nodes' on the control-plane to see this node join the cluster.  

如果报以下错误：

[ERROR DirAvailable--etc-kubernetes-manifests]: /etc/kubernetes/manifests is not empty
[ERROR FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists
[ERROR Port-10250]: Port 10250 is in use
[ERROR FileAvailable--etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt already exists

解决办法：加上--ignore-preflight-errors=all

九，在master（10.0.40.193）节点，检查集群状态

1，检查集群状态

查看复制打印?
  1. [root@bigserver3 ~]# kubectl get nodes  
  2. NAME         STATUS   ROLES    AGE     VERSION  
  3. bigserver2   Ready    <none>   4h44m   v1.18.2  
  4. bigserver3   Ready    master   24h     v1.18.2  
  5.   
  6. [root@bigserver3 ~]# kubectl get cs  
  7. NAME                 STATUS    MESSAGE             ERROR  
  8. scheduler            Healthy   ok  
  9. controller-manager   Healthy   ok  
  10. etcd-0               Healthy   {"health":"true"}  

如果报以下错误：

Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")

解决办法：rm -rf $HOME/.kube，删除目录重新copy

2，创建Pod以验证集群

  1. [root@bigserver3 ~]# kubectl get pod,svc  
  2. NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE  
  3. service/kubernetes   ClusterIP   10.1.0.1     <none>       443/TCP   21h  
  4.   
  5. [root@bigserver3 ~]# kubectl create deployment nginx --image=nginx  
  6. deployment.apps/nginx created  
  7.   
  8. [root@bigserver3 ~]# kubectl expose deployment nginx --port=80 --type=NodePort  
  9. service/nginx exposed  
  10.   
  11. [root@bigserver3 ~]# kubectl get pod,svc  
  12. NAME                        READY   STATUS              RESTARTS   AGE  
  13. pod/nginx-f89759699-545w5   0/1     ContainerCreating   0          12s  
  14.   
  15. NAME                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE  
  16. service/kubernetes   ClusterIP   10.1.0.1      <none>       443/TCP        21h  
  17. service/nginx        NodePort    10.1.54.199   <none>       80:30412/TCP   5s  

十，安装dashboard，web端的kubernetes管理工具

1，项目地址

https://github.com/kubernetes/dashboard

2，安装kubernetes-dashboard

查看复制打印?
  1. # wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0/aio/deploy/recommended.yaml  
  2.   
  3. # vim ecommended.yaml  
  4.  ports:  
  5.  - port: 443  
  6.  targetPort: 8443  
  7.  nodePort: 30001 //新增，范围30000-32767  
  8.  type: NodePort //新增  
  9.  selector:  

The Service "kubernetes-dashboard" is invalid: spec.ports[0].nodePort: Invalid value: 28443: provided port is not in the valid range. The range of valid ports is 30000-32767，这个问题就是因为端口设置的不对

  1. [root@bigserver3 ~]# kubectl create -f recommended.yaml  
  2. namespace/kubernetes-dashboard created  
  3. serviceaccount/kubernetes-dashboard created  
  4. service/kubernetes-dashboard created  
  5. secret/kubernetes-dashboard-certs created  
  6. secret/kubernetes-dashboard-csrf created  
  7. secret/kubernetes-dashboard-key-holder created  
  8. configmap/kubernetes-dashboard-settings created  
  9. role.rbac.authorization.k8s.io/kubernetes-dashboard created  
  10. clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created  
  11. rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created  
  12. clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created  
  13. deployment.apps/kubernetes-dashboard created  
  14. service/dashboard-metrics-scraper created  
  15. deployment.apps/dashboard-metrics-scraper created  

3，检查kubernetes-dashboard是否成功

  1. [root@bigserver3 ~]# kubectl get pods -n kube-system -o wide  
  2. NAME                                 READY   STATUS    RESTARTS   AGE     IP            NODE         NOMINATED NODE   READINESS GATES  
  3. coredns-7ff77c879f-bxgtr             1/1     Running   0          22h     10.244.0.2    bigserver3   <none>           <none>  
  4. coredns-7ff77c879f-t4vm9             1/1     Running   0          22h     10.244.0.3    bigserver3   <none>           <none>  
  5. etcd-bigserver3                      1/1     Running   2          22h     10.0.40.193   bigserver3   <none>           <none>  
  6. kube-apiserver-bigserver3            1/1     Running   2          22h     10.0.40.193   bigserver3   <none>           <none>  
  7. kube-controller-manager-bigserver3   1/1     Running   2          22h     10.0.40.193   bigserver3   <none>           <none>  
  8. kube-flannel-ds-amd64-772tm          1/1     Running   0          155m    10.0.40.222   bigserver2   <none>           <none>  
  9. kube-flannel-ds-amd64-d75d5          1/1     Running   0          3h40m   10.0.40.193   bigserver3   <none>           <none>  
  10. kube-proxy-8mpj4                     1/1     Running   0          155m    10.0.40.222   bigserver2   <none>           <none>  
  11. kube-proxy-dmsq4                     1/1     Running   1          22h     10.0.40.193   bigserver3   <none>           <none>  
  12. kube-scheduler-bigserver3            1/1     Running   1          22h     10.0.40.193   bigserver3   <none>           <none>  
  13.   
  14. [root@bigserver3 ~]# kubectl get services -n kube-system  
  15. NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE  
  16. kube-dns   ClusterIP   10.1.0.10    <none>        53/UDP,53/TCP,9153/TCP   22h  
  17.   
  18. [root@bigserver3 ~]# netstat -ntlp|grep 30001  
  19. tcp        0      0 0.0.0.0:30001           0.0.0.0:*               LISTEN      6096/kube-proxy  

4，生成Dashboard的认证令牌

查看复制打印?
  1. [root@bigserver3 ~]# kubectl create serviceaccount  dashboard-admin -n kube-system  
  2. serviceaccount/dashboard-admin created  
  3.   
  4. [root@bigserver3 ~]# kubectl create clusterrolebinding  dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin  
  5. clusterrolebinding.rbac.authorization.k8s.io/dashboard-admin created  
  6.   
  7. [root@bigserver3 ~]# kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk '/dashboard-admin/{print $1}')  
  8. Name:         dashboard-admin-token-whgv9  
  9. Namespace:    kube-system  
  10. Labels:       <none>  
  11. Annotations:  kubernetes.io/service-account.name: dashboard-admin  
  12.               kubernetes.io/service-account.uid: dfbb4498-b69f-41f9-84fa-3480b1a4c437  
  13.   
  14. Type:  kubernetes.io/service-account-token  
  15.   
  16. Data  
  17. ====  
  18. token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IkUtNDZpS3lLbGk5YlhpOFhldTZvcUktOUgxVFk2TkMzN2wwTGlzdlN1aWMifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4td2hndjkiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZGZiYjQ0OTgtYjY5Zi00MWY5LTg0ZmEtMzQ4MGIxYTRjNDM3Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.oguhHisptOdzxVIfJVY6sRUzFgok-NVy2Ob4tvAy3_h9tIwa-5hzshSwXduDTWtgQ8KO5ry5EphAaa9t3BBdy_KqPO2_ysdD-Hrlfa0Fi-8jep1mk34Ol0kjw6EzmAXT6I09-hLj0yjHAM3ub3cnV2Rc-hHcEZKDs3lVCRLiPeFggMhTOQsPXw3mElVuX3PxghwKRw4c2Kw5Vvg5ALRQ6lcrYY2Kex4hNFo6y1ewszyMyDPXIFIDwupjYbuAmEKb2C7j5QHargSBJ88q-iTcwtAAZs1_gOpXYOJVNVfBsbHAK4O8FeKZnvJhgL4aufNRw12_ffw1Ot4VqR8v43pWow  
  19. ca.crt:     1025 bytes  
  20. namespace:  11 bytes  

5，登录Dashboard。https://10.0.40.193:30001，用firefox访问

kubernetes chrome打不开

kubernetes chrome打不开

kubernetes 填上生成的token

kubernetes 填上生成的token

kubernetes dashboard首页

kubernetes dashboard首页

kubernetes 节点页

kubernetes 节点页



转载请注明
作者:海底苍鹰
地址:http://blog.51yip.com/cloud/2399.html

page: 2

<< docker certificate has expired or is not yet valid
kubernetes(k8s) 集群 安装配置 >>

kubernetes join 卡住 token过期

张映 发表于 2020-04-29

分类目录： 云计算

标签：k8s, kubernetes, token

kubeadm join增加worker节点时，卡住

[root@testing yum.repos.d]# kubeadm join 10.0.40.193:6443 --token y4d0ws.w8lxmohfc1o0yq6b --discovery-token-ca-cert-hash sha256:46f6cf1d84d0eadb4f6e7f05b908e5572025886d9f134db27f92b98e1c3dd3ed
W0429 10:21:36.848737 17290 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

然后就不动了，检查了一下原因是token过期了。下面说一下解决办法

1，在master节点查看token

# kubeadm token list   //没token

2，创建token

[root@bigserver3 pki]# kubeadm token create --ttl 0   //永不失效
W0429 11:25:57.187354 14070 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
5mhd9x.73x9n2gy4d28yqzo

[root@bigserver3 pki]# kubeadm token create  //有效期一天
W0429 11:26:15.884356 14216 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
652tg2.t1am119z6tdoosou

[root@bigserver3 pki]# kubeadm token list   //token列表
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
5mhd9x.73x9n2gy4d28yqzo <forever> <never> authentication,signing <none> system:bootstrappers:kubeadm:default-node-token
652tg2.t1am119z6tdoosou 23h 2020-04-30T11:26:15+08:00 authentication,signing <none> system:bootstrappers:kubeadm:default-node-token

注意：TTL，一个显示forever(永久)，一个显示23H

3，查看token

[root@bigserver3 kubernetes]# ll /etc/kubernetes/pki/
总用量 56
-rw-r--r-- 1 root root 1224 4月 27 19:13 apiserver.crt
-rw-r--r-- 1 root root 1090 4月 27 19:13 apiserver-etcd-client.crt
-rw------- 1 root root 1679 4月 27 19:13 apiserver-etcd-client.key
-rw------- 1 root root 1675 4月 27 19:13 apiserver.key
-rw-r--r-- 1 root root 1099 4月 27 19:13 apiserver-kubelet-client.crt
-rw------- 1 root root 1675 4月 27 19:13 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 4月 27 19:13 ca.crt //ca证书
-rw------- 1 root root 1679 4月 27 19:13 ca.key
drwxr-xr-x 2 root root 162 4月 27 19:13 etcd
-rw-r--r-- 1 root root 1038 4月 27 19:13 front-proxy-ca.crt
-rw------- 1 root root 1675 4月 27 19:13 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 4月 27 19:13 front-proxy-client.crt
-rw------- 1 root root 1679 4月 27 19:13 front-proxy-client.key
-rw------- 1 root root 1675 4月 27 19:13 sa.key
-rw------- 1 root root 451 4月 27 19:13 sa.pub

# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
46f6cf1d84d0eadb4f6e7f05b908e5572025886d9f134db27f92b98e1c3dd3ed   //token

4，worker节点执行join

[root@testing kubernetes]# kubeadm join 10.0.40.193:6443 --token 652tg2.t1am119z6tdoosou --discovery-token-ca-cert-hash sha256:46f6cf1d84d0eadb4f6e7f05b908e5572025886d9f134db27f92b98e1c3dd3ed
W0429 11:29:53.078968 19474 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
 [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.18" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

5，master节点查看

[root@bigserver3 ~]# kubectl get nodes
NAME         STATUS   ROLES   AGE   VERSION
bigserver2   Ready   <none>   20h   v1.18.2
bigserver3   Ready   master   40h   v1.18.2
testing      Ready   <none>   54s   v1.18.2

6，登录kubernetes-dashboard查看

kubernetes添加节点

kubernetes添加节点



转载请注明
作者:海底苍鹰
地址:http://blog.51yip.com/cloud/2404.html