1. dubbo微服务架构

微服务提供者集群： 暴露服务的服务提供方

微服务消费者集群： 调用远程服务的服务消费方

注册中心： 微服务提供者集群注册， 微服务消费者集群订阅， 微服务消费者集群通过RPC协议调用微服务提供者集群

Monitor: 统计服务的调用次数和调用时间的监控中心

部署的基本流程为 git推送代码到jenkins, jenkins编译代码并打包镜像推送到镜像仓库， 然后通过kubectl或者helm操作k8s的YAML文件部署服务

用户访问的基本流程为用户访问ingress, ingress代理微服务消费者集群， ingress得到响应后， 再将数据返回给用户

2. 部署zookeeper集群

• Zookeeper是Dubbo微服务集群的注册中心
• zookeeper高可用机制和etcd集群一致
• 需要jdk环境

2.1 基础架构

2.2 安装jdk1.8

? jdk下载地址： https://www.oracle.com/java/technologies/javase/javase-jdk8-downloads.html

• 解压文件到指定目录

mkdir -p /usr/java
tar -xf jdk-8u261-linux-x64.tar.gz -C /usr/java/
cd /usr/java/

• 加入环境变量

vim /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_261
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar

source /etc/profile

• 查看java版本

java -version

2.3 安装zookeeper

? zookeeper 下载地址： https://archive.apache.org/dist/zookeeper/

2.3.1 解压，配置

• 解压文件到指定位置

wget https://archive.apache.org/dist/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
tar xf zookeeper-3.4.14.tar.gz -C /opt/

• 创建zookeeper 数据目录 和 日志目录

cd /opt/
mkdir -p /data/zookeeper/data /data/zookeeper/logs

• 编写配置文件

cp /opt/zookeeper-3.4.14/conf/zoo_sample.cfg /opt/zookeeper-3.4.14/conf/zoo.cfg
vim /opt/zookeeper-3.4.14/conf/zoo.cfg

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/data/zookeeper/data
dataLogDir=/data/zookeeper/logs
# the port at which the clients will connect
clientPort=2181
server.1=zk1.od.com:2888:3888
server.2=zk2.od.com:2888:3888

2.3.2 myid

192.168.31.11

echo 1 > /data/zookeeper/data/myid

192.168.31.11

echo 2 > /data/zookeeper/data/myid

2.3.3 做dns解析

vim /var/named/od.com.zone

$ORIGIN od.com.
$TTL 600    ; 10 minutes
@   IN SOA  dns.od.com.  dnsadmin.od.com.  (
                 2020072401   ; serial
                 10800      ; refresh (3 hours)
                 900        ; retry (15 minutes)
                 604800     ; expire (1 week)
                 86400      ; minimun (1 day)
                 )
            NS  dns.od.com.
$TTL 60 ; 1 minute
dns                A      192.168.31.11
harbor             A      192.168.31.200
zk1             A         192.168.31.11
zk2             A         192.168.31.200

systemctl restart named

• 检查dns

2.3.4 依次启动zookeeper

• 启动服务

/opt/zookeeper-3.4.14/bin/zkServer.sh start

• 检查端口

netstat -ntulp|grep 2181

• 检查zookeeper集群的主从状态

/opt/zookeeper-3.4.14/bin/zkServer.sh status

3. 部署jenkins

官网： https://www.jenkins.io/download/

dockerhub： https://hub.docker.com/r/jenkins/jenkins

3.1 准备镜像

docker pull jenkins/jenkins:2.235.3
docker tag 135a0d19f757 harbor.od.com/public/jenkins:2.235.3
docker push harbor.od.com/public/jenkins:2.235.3

3.2 自定义Dockerfile

• 创建目录

mkdir -p /data/dockerfile/jenkins

• 创建密钥对

ssh-keygen -t rsa -b 2048 -C ‘1392263019@qq.com‘ -N "" -f /root/.ssh/id_rsa

• 准备dockerfile文件

cd /data/dockerfile/jenkins
cp /root/.docker/config.json .
cp /root/.ssh/id_rsa .

vim Dockerfile

FROM harbor.od.com/public/jenkins:2.235.3
USER root
RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && echo ‘Asia/Shanghai >/etc/timezone‘
ADD id_rsa /root/.ssh/id_rsa
ADD config.json /root/.docker/config.json
ADD get-docker.sh /get-docker.sh
RUN  cat /etc/os-release
RUN echo "   StrictHostKeyChecking no" >> /etc/ssh/ssh_config && curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun 

3.3 制作自定义镜像

docker build . -t harbor.od.com/infra/jenkins:2.235.3

• 测试下镜像

  需要先把公钥放到gitee上

docker run --rm harbor.od.com/infra/jenkins:2.235.3 ssh -T git@gitee.com

3.4 创建infra仓库

3.5 创建kubernetes 命名空间以及私有仓库infra的secret

kubectl create ns infra
kubectl create secret docker-registry harbor --docker-server=harbor.od.com --docker-username=admin --docker-password=Harbor12345 -n infra

3.6 推送镜像

docker push harbor.od.com/infra/jenkins:2.235.3

3.7 准备nfs共享存储

所有运算节点安装nfs-utils, 并启动服务

yum -y install nfs-utils

• 配置nfs服务

192.168.31.200

echo ‘/data/jenkins 192.168.31.0/24(rw,no_root_squash)‘ >> /etc/exports

• 启动nfs服务

systemctl start rpcbind  &&systemctl enable rpcbind
systemctl start nfs && systemctl enable nfs

• 查看nfs服务

exportfs -v

• 创建目录jenkins_home

mkdir /data/jenkins/jenkins_home/

3.8 准备资源

• dp.yaml

kind: Deployment
apiVersion: apps/v1
metadata:
  name: jenkins
  namespace: infra
  labels:
    name: jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      name: jenkins
  template:
    metadata:
      labels:
        app: jenkins
        name: jenkins
    spec:
      volumes:
      - name: data
        nfs:
          server: hdss200
          path: /data/jenkins/jenkins_home
      - name: docker
        hostPath:
           path: /run/docker.sock
      imagePullSecrets:
      - name: harbor
      containers:
      - name: jenkins
        image: harbor.od.com/infra/jenkins:2.235.3 
        securityContext:
          runAsUser: 0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
          protocol: TCP
        env:
        - name: JAVA_OPTS
          value: -Xmx512m -Xms512m -Dhudson.security.csrf.GlobalCrumbIssuerConfiguration.DISABLE_CSRF_PROTECTION=true
        volumeMounts:
        - name: data
          mountPath: /var/jenkins_home
        - name: docker
          mountPath: /run/docker.sock

• svc.yaml

kind: Service
apiVersion: v1
metadata:
  name: jenkins
  namespace: infra
spec:
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
  selector:
    app: jenkins

• ingress.yaml

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
  name: jenkins
  namespace: infra
  annotations:
   kubernetes.io/ingress.class: intranet4public 
spec:
  rules:
  - host: jenkins.od.com
    http:
      paths:
      - path: /
        backend:
          serviceName: jenkins
          servicePort: 80

3.9 部署资源

kubectl apply -f dp.yaml
kubectl apply -f svc.yaml
kubectl apply -f ingress.yaml

• 查看资源

kubectl get all -n infra
kubectl get ing -n infra

3.10 解析域名

将域名 jenkins.od.com 设定了 kubernetes.io/ingress.class: intranet4public , 指定了ingress为 设定了参数为 intranet4public 的ingress, 所以在本机的C:\Windows\System32\drivers\etc\hosts将域名解析为 192.168.31.41 jenkins.od.com

3.11 浏览器访问

浏览器输入 http://jenkins.od.com/

3.12 初始配置jenkins

• 输入初始密码

cat secrets/initialAdminPassword 

• 将所有的插件都取消勾选

3.13 创建管理者用户

用户名 admin

密码 admin123

3.14 允许匿名用户有读权限

3.15 配置国内镜像源

192.168.31.200

cd /data/jenkins/jenkins_home/updates/
sed -i ‘s/updates.jenkins-ci.org\/download/mirrors.tuna.tsinghua.edu.cn\/jenkins/g‘ default.json
sed -i ‘s/www.google.com/www.baidu.com/g‘ default.json

执行完重启pod

3.16 安装插件 Blue Ocean

Manage Jenkins - Manage Plugins

交付dubbo微服务到k8s

原文：https://www.cnblogs.com/cjwnb/p/13461268.html