Build Kubernetes && Docker virtualization cluster based on CentOS7, 8

最近一位老师在学习虚拟化集群的部署和应用，我闲来无事也跟着搞着玩，过程是真的坎坷。。。

一、环境

I、两台CentOS7虚拟机，一台CentOS8虚拟机
II、Kubernetes1.18.0
III、Docker
IV、Flannel

二、Master

1、修改Master主机的名称

hostnamectl set-hostname Master

2、设置主机名解析

vi /etc/hosts
172.16.255.102   master
172.16.255.103   node1  
172.16.255.104   node2

3、关闭所有防火墙

systemctl disable firewalld  
systemctl stop firewalld  
vi /etc/selinux/config  
将SELINUX=enforcing改成disabled

4、关闭swap

swapoff -a 
sed -i 's/.swap./#&/' /etc/fstab

5、然后重启

reboot

6、重启完后，安装docker、kubectl、kubelet、kubeadm

1.创建docker的源    
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo  

2.创建kubernetes源 
vi /etc/yum.repo

[kubernetes]
name=Kubernetes Repo
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ 
gpgcheck=0 
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg 
enabled=1   

3.安装 
yum install -y docker-ce kubelet kubectl kubeadm

4.启动并设置docker、kubelet开机自启  
systemctl enable docker  
systemctl enable kubelet  
systemctl start docker  
systemctl start kubelet

7、生成并修改默认kubeadm.conf文件

kubeadm config print init-defaults > kubeadm.conf  
vi kubeadm.conf  
将imageRepository修改为registry.aliyuncs.com/google_containers。并把版本改成1.18.0

8、下载镜像

kubeadm config images pull --config kubeadm.conf

9、镜像下载完成后，需要对镜像重新打标签，因为从阿里下载的镜像都是阿里的标签，而kubeadm里面只
认 google的标签，重新打标签后镜像就都带有 k8s.gcr.io 的标识了

docker tag registry.aliyuncs.com/google_containers/kube-proxy:v1.18.0 k8s.gcr.io/kube-proxy:v1.18.0  
docker tag registry.aliyuncs.com/google_containers/kube-apiserver:v1.18.0 k8s.gcr.io/kube-apiserver:v1.18.0  
docker tag registry.aliyuncs.com/google_containers/kube-controller-manager:v1.18.0 k8s.gcr.io/kube-controller-manager:v1.18.0  
docker tag registry.aliyuncs.com/google_containers/kube-scheduler:v1.18.0 k8s.gcr.io/kube-scheduler:v1.18.0  
docker tag registry.aliyuncs.com/google_containers/pause:3.2 k8s.gcr.io/pause:3.2  
docker tag registry.aliyuncs.com/google_containers/coredns:1.6.7 k8s.gcr.io/coredns:1.6.7  
docker tag registry.aliyuncs.com/google_containers/etcd:3.4.3-0 k8s.gcr.io/etcd:3.4.3-0

10、删除无用镜像，重新打标签后，还需要把带有 registry.aliyuncs.com 标识的镜像删除

docker rmi registry.aliyuncs.com/google_containers/kube-proxy:v1.18.0   
docker rmi registry.aliyuncs.com/google_containers/kube-apiserver:v1.18.0   
docker rmi registry.aliyuncs.com/google_containers/kube-controller-manager:v1.18.0   
docker rmi registry.aliyuncs.com/google_containers/kube-scheduler:v1.18.0  
docker rmi registry.aliyuncs.com/google_containers/pause:3.2  
docker rmi registry.aliyuncs.com/google_containers/coredns:1.6.7  
docker rmi registry.aliyuncs.com/google_containers/etcd:3.4.3-0

11、初始化Kubernetes Master，这里我们定义POD的网段为: 192.168.0.0/16，API Server地址为Master节点的IP地址。

kubeadm init --kubernetes-version=v1.18.0 --pod-network-cidr=192.168.0.0/16 --apiserver-advertise-address=172.16.255.102  
然后把这段内容复制下来，后面其他node加入master需要用到  
kubeadm join 172.16.255.102:6443 --token 77l117.gl8jwl5r1o1rba18 \
--discovery-token-ca-cert-hash sha256:c76bf66b8078d5fd75afc17cf e05c470a07f7bf6a0b797

12、执行以下命令配置kubectl，作为普通用户管理集群并在集群上工作。（个人理解，它应该是个环境变量）

mkdir -p $HOME/.kube  
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config  
chown $(id -u):$(id -g) $HOME/.kube/config

13、安装flannel插件

curl https://raw.githubusercontent.com/flannel/kube-flannel.yml   
kubectl apply -f kube-flannel.yml  
如果出现400 bad request的话，就在宿主机上挂全局VPN把这个文件下下来，然后上传到服务器上。

14、安装kubernetes-dashboard
kubernetes-dashboard 官方下载地址.

wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-rc7/aio/deploy/recommended.yaml  

编辑recommended.yaml，由于默认的service类型是ClusterIP，我们是自建的kubernetes，无法自动分配ip给service，所以这里我们需要修改一下dashboard的service类型，指定为NodePort以方便我们访问。  
vi recommended.yaml  


kind: Service  
apiVersion: v1  
metadata:  
  labels:  
    k8s-app: kubernetes-dashboard  
  name: kubernetes-dashboard  
  namespace: kubernetes-dashboard  
spec:  
  type: NodePort #新加此行  
  ports:  
    - port: 443  
      nodePort： 30001 #新加此行  
      targetPort: 8443  
  selector:  
    k8s-app: kubernetes-dashboard  



kubectl apply -f recommended.yaml

15、 之后指定namespace查看pod和service

kubebctl -n kubernetes-dashboard get pod -o wide  
kubectl 0n kubernetes-dashboard get svc

16、通过IP/Domain+port来访问dashboard，chrome、IE访问的话会报错，需要证书，这里我没有创建证书，所以是用搜狗浏览器访问的

17、创建一个dashboard用户来登陆dashboard

1.创建个yaml文件  
touch create-admin.yaml  

2.填入以下内容到create-admin.yaml  
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard 
  
---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

3.创建用户

kubectl apply -f create-admin.yaml

4.获取admin-user的登陆token 

kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep admin-user | awk '{print $1}')

5.将获取到的token填入dashboard里面即可访问dashboard

三、Node1

18、安装docker、kubelet、kubeadm

1.安装docker源

cd /etc/yum.repo  
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo  

2.安装kubernetes源  

vi /etc/yum.repo/kubernetes.repo

[kubernetes]  
name=Kubernetes Repo  
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/   
gpgcheck=0   
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg   
enabled=1   

3.安装docker、kubelet、kubeadm

yum -y install docker -ce kubelet kubeadm

19、执行以下命令配置kubectl，作为普通用户管理集群并在集群上工作。（个人理解，它应该是个环境变量）

mkdir -p $HOME/.kube  
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config  
chown $(id -u):$(id -g) $HOME/.kube/config

20、设置docker、kubelet启动并开机自启

systemctl enable docker  
systemctl enable kubelet  
systemctl start docker  
systemctl start kubelet

21、安装flannel插件

curl https://raw.githubusercontent.com/flannel/kube-flannel.yml   
kubectl apply -f kube-flannel.yml  
如果出现400 bad request的话，就在宿主机上挂全局VPN把这个文件下下来，然后上传到服务器上。

22、加入master

kubeadm join 172.16.255.102:6443 --token 77l117.gl8jwl5r1o1rba18 \
--discovery-token-ca-cert-hash sha256:c76bf66b8078d5fd75afc17cf e05c470a07f7bf6a0b797

四、Node2

操作和node1一样