Skip to content

Jenkins

测试环境

访问地址：http://47.96.187.116/jenkins/ 用户名：admin 密码：Leyou1105.Jenkins 端口：49000

安装

文档地址：https://www.jenkins.io/zh/doc/book/installing/ 安装方式：docker（jenkins/jenkins:lts-jdk11） 版本：Jenkins 2.361.3 data路径：/app/jenkins

  • jenkins_home: jenkins 的 data 内容（复制自原服务器的 jenkins_home 文件夹，大头 workspace 不用复制，jobs 包含任务配置，任务历史文件夹可以舍弃）
  • start.sh: 启动命令
docker run --name jenkins -u root -d -p 49000:8080  -p 50000:50000  -v /app/jenkins/jenkins_home:/var/jenkins_home  -v /var/run/docker.sock:/var/run/docker.sock --env JENKINS_OPTS="--prefix=/jenkins" jenkins/jenkins:lts-jdk11

注意：~~docker 安装的 jenkins 版本过低，需要页面内升级~~（不能随便升级！只能用原来的死版本）

  • 下载原始版本，复制到容器内的 /usr/share/jenkins，页面重启（/restart）

nginx 配置：https://www.jenkins.io/doc/book/system-administration/reverse-proxy-configuration-with-jenkins/reverse-proxy-configuration-nginx/

  • enable cnpm: 容器内 export PATH=$PATH:/var/jenkins_home/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/NodeJS_16.18/bin

使用

每个项目的打包配置，从 Configure 点击查看： image.png

打包流程

maven 包

image.png

git pull
git checkout dev

/var/jenkins_home/apache-maven-3.8.6/bin/mvn clean deploy -s /var/jenkins_home/apache-maven-3.8.6/conf/settings-ztlydev-shuke.xml -P dist-ztlydev-shuke -U -DskipTests

注意：新增了一份 settings-ztlydev-shuke.xml，同时上传平台为 dist-ztlydev-shuke

打包顺序
  • byteox-parent
  • byteox-base-pojo
  • byteox-public-pojo
  • byteox-uaa-jwt-client
  • byteox-hystrix-config
  • byteox-core

然后是各服务的 api 包：

  • byteox-oss、byteox-bizconfig、byteox-users、byteox-basic、byteox-message、byteox-workflow、byteox-wallet、byteox-house、byteox-promotion、byteox-econtract...

服务打包与启动

image.png以 byteox-app2b 为例：

  1. 执行下述命令：
#!/bin/sh

cd ${WORKSPACE}

if [ ! -d "byteox-app2b" ];then
git clone "git@codeup.aliyun.com:byteox/ka/ztly/backend-ztly/byteox-app2b.git"
else
echo "文件夹已经存在"
fi

cd "byteox-app2b"


git reset --hard
git clean -xdf
git checkout -f master-ym
git pull -f

# Jenkins

#配置参数 e.g. dev、prod
BUILD_ENV=dev
# 客户公司名称 e.g. bjsg
CORPORATE_NAME=ztly
# e.g. bjsgdev, 用于 application-ydyjdev.properties
# 【需更换】
CONFIG_ENV=ztlydev 
# e.g. byteox-app2b
SERVICE_NAME=byteox-app2b


if [ ! -d "byteox-app2b" ];then
    # 不含api目录
    MVN_BUILD_TARGET_DIR=target
    POM_FILE="pom.xml"
else
    MVN_BUILD_TARGET_DIR=byteox-app2b/target
    POM_FILE=byteox-app2b/pom.xml
fi

# 修改pom.xml，替换 <build>...</build>
# 注释：并放入到项目文件中
#```
MAIN_CLASS=$(cat $POM_FILE | grep -P '<mainClass>' | sed 's/\///' | sed 's/<//g;s/>//g;s/mainClass//g' | sed 's/[ \t]*//g')
SEGMENT_BUILD=$(cat ${JENKINS_HOME}/byteox/templates/pom_build.xml | sed "s#{{MAIN_CLASS}}#${MAIN_CLASS}#")

sed -i "/^[ \t]*<build>/,/build>$/s/.*//g" $POM_FILE
sed -i "s#</project>##" $POM_FILE

cat <<EOF >> $POM_FILE
$SEGMENT_BUILD
EOF
#```



MVN_HOME=${JENKINS_HOME}/apache-maven-3.8.6
# 注释：使用特定的 maven settings.xml 【需更换CONFIG_ENV】打包
$MVN_HOME/bin/mvn clean package  -s $MVN_HOME/conf/settings-$CONFIG_ENV.xml -DskipTests


if [ ! -f "$MVN_BUILD_TARGET_DIR/${SERVICE_NAME}-thin.jar" ];then
echo "Maven构建失败 $MVN_BUILD_TARGET_DIR/${SERVICE_NAME}-thin.jar 不存在。"
exit 1
fi


#项目临时目录
# 注释：寓盟服务器：/var/jenkins_home/byteox/_docker_build/dev/byteox-app2b
PROJ_BYTEOX_BUILD_DIR=${JENKINS_HOME}/byteox/_docker_build/${BUILD_ENV}/$SERVICE_NAME
# 注释：寓盟服务器：/var/jenkins_home/byteox/_docker_build/dev/byteox-app2b_docker_build.sh
DOCKER_BYTEOX_BUILD_SH=${JENKINS_HOME}/byteox/_docker_build/${BUILD_ENV}/${SERVICE_NAME}_docker_build.sh
echo `pwd`

mkdir -p $PROJ_BYTEOX_BUILD_DIR
#删除打包目录文件
#rm -rf $PROJ_BYTEOX_BUILD_DIR/*.jar $PROJ_BYTEOX_BUILD_DIR/*.properties $PROJ_BYTEOX_BUILD_DIR/lib $PROJ_BYTEOX_BUILD_DIR/byteox-lib
# 注释：清除上一次的文件及打包产物
rm -rf $PROJ_BYTEOX_BUILD_DIR/*

echo "创建Dockerfile文件 ${PROJ_BYTEOX_BUILD_DIR}/Dockerfile"
cat <<EOF > ${PROJ_BYTEOX_BUILD_DIR}/Dockerfile
from 172.24.137.212:5000/eclipse-temurin:11.0.16.1_1-jre
WORKDIR /app/byteox-app2b
# 设定时区
RUN cp -f /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
COPY lib/ ./lib/
COPY byteox-lib/ ./lib/
COPY byteox-app2b.jar *.properties ./
EOF

# docker构建目录
# 注释：寓盟服务器：将构建目录下的 target/byteox-app2b-thin.jar 移到 /var/jenkins_home/byteox/_docker_build/dev/byteox-app2b/byteox-app2b.jar
mv $MVN_BUILD_TARGET_DIR/${SERVICE_NAME}-thin.jar $PROJ_BYTEOX_BUILD_DIR/${SERVICE_NAME}.jar
mv $MVN_BUILD_TARGET_DIR/lib $PROJ_BYTEOX_BUILD_DIR/
mv $MVN_BUILD_TARGET_DIR/classes/application.properties $PROJ_BYTEOX_BUILD_DIR
# 注释：移动 application-ztlydev.properties
mv $MVN_BUILD_TARGET_DIR/classes/application-$CONFIG_ENV.properties $PROJ_BYTEOX_BUILD_DIR
perl -pi -e "s/spring\.profiles\.active=.*/spring.profiles.active=$CONFIG_ENV/g" $PROJ_BYTEOX_BUILD_DIR/application.properties
# 注释：寓盟服务器：进入 /var/jenkins_home/byteox/_docker_build/dev/byteox-app2b 下
cd $PROJ_BYTEOX_BUILD_DIR

# 分离byteox-*.jar，压缩lib.zip
mkdir byteox-lib
mv lib/byteox-*.jar byteox-lib

ls -l $PROJ_BYTEOX_BUILD_DIR




######## 生成「docker_build.sh」脚本文件准备变量 #######
#源yaml文件存放路径
# 注释：寓盟服务器YAML_BASE_DIR：/var/jenkins_home/byteox/yaml/ztlydev
YAML_BASE_DIR=${JENKINS_HOME}/byteox/yaml/$CONFIG_ENV

########
#目标yaml文件名
TARGET_YAML_NAME=$SERVICE_NAME.yaml
#部署yaml路径 kubectl apply -f [yaml file]
# 注释：寓盟服务器YAML_DIST_DIR：/var/jenkins_home/byteox/_yamldist/ztly/dev
YAML_DIST_DIR=${JENKINS_HOME}/byteox/_yamldist/$CORPORATE_NAME/$BUILD_ENV

#固定参数
IMAGE_VERSION=latest
CURRENT_VERSION=v`date '+%Y%m%d%H%M%S'`


################生成「docker_build.sh」#################################
cat <<EOF > $DOCKER_BYTEOX_BUILD_SH
#!/bin/sh


# 项目 docker build 目录
cd $PROJ_BYTEOX_BUILD_DIR



#构建镜像
docker build -t $SERVICE_NAME .

#推送到仓库
# 【需更换】jenkins配置的全局变量 REGISTRY_HOST
docker tag $SERVICE_NAME:$IMAGE_VERSION $REGISTRY_HOST/$CONFIG_ENV/$SERVICE_NAME:$IMAGE_VERSION
docker tag $SERVICE_NAME:$IMAGE_VERSION $REGISTRY_HOST/$CONFIG_ENV/$SERVICE_NAME:$CURRENT_VERSION
docker push $REGISTRY_HOST/$CONFIG_ENV/$SERVICE_NAME:$IMAGE_VERSION
docker push $REGISTRY_HOST/$CONFIG_ENV/$SERVICE_NAME:$CURRENT_VERSION

#push到私有仓库后删除tag
#docker rmi $REGISTRY_HOST/$BUILD_ENV/$SERVICE_NAME:$IMAGE_VERSION

#替换版本号
# 注释：将寓盟服务器YAML_BASE_DIR：/var/jenkins_home/byteox/yaml/ztlydev/byteox_app2b
# 复制到 /var/jenkins_home/workspace/ztly_dev_byteox-app2b/ 下
cp $YAML_BASE_DIR/$TARGET_YAML_NAME $WORKSPACE
# 注释：进入 /var/jenkins_home/workspace/ztly_dev_byteox/
cd $WORKSPACE
# 注释：将 byteox-app2b.yaml(k8s配置) 里的 latest 替换为当前版本
perl -pi -e "s/latest/$CURRENT_VERSION/g" $TARGET_YAML_NAME

chown elastic:elastic $TARGET_YAML_NAME

# 注释：以下这一大段都没用，因为最终 xx.yaml 会放到 /app/yaml/dist 下
# 复制并发布
# 注释：寓盟服务器YAML_DIST_DIR：/var/jenkins_home/byteox/_yamldist/ztly/dev
mkdir -p $YAML_DIST_DIR
# 注释：将当前 byteox-app2b.yaml 复制到  /var/jenkins_home/byteox/_yamldist/ztly/dev
# 所以最后的 /var/jenkins_home/byteox/_yamldist/ztly/dev/byteox-app2b.yaml 为最终生成配置文件
cp $TARGET_YAML_NAME $YAML_DIST_DIR
# kubectl apply -f $YAML_DIST_DIR/${SERVICE_NAME}.yaml --record=true

EOF
  1. Post-build Actions：远程服务器行为
  2. 执行命令 sh /var/jenkins_home/byteox/_docker_build/dev/byteox-app2b_docker_build.sh推送至 docker 仓库
  3. transfer file: 将 jenkins 工作目录的 byteox-app2b.yaml 复制到 /app/yaml/dist
  4. 通过 k8s 启动： kubectl apply -f /app/yaml/dist/byteox-app2b.yaml
附：k8s yaml 示例
apiVersion: v1
kind: Service
metadata:
  name: byteox-app2b
  namespace: dev-byteox
  labels:
    app: byteox-app2b

spec:
  ports:
  - port: 61003
    targetPort: 61003
    nodePort: 61003
  selector:
    app: byteox-app2b
  type: NodePort

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: byteox-app2b-deployment
  namespace: dev-byteox
  labels:
    app: byteox-app2b
spec:
  replicas: 1
  minReadySeconds: 30
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 2
      maxUnavailable: 0
  selector:
    matchLabels:
      app: byteox-app2b
  template:
    metadata:
      labels:
        app: byteox-app2b
    spec:
      containers:
      - image: elastic/filebeat:7.17.6
        name: filebeat
        args: [
          "-c", "/etc/filebeat/filebeat.yml",
          "-e",
        ]
        volumeMounts:
        - name: byteox-app2b-logs
          mountPath: /log
        - name: byteox-app2b-filebeat-config
          mountPath: /etc/filebeat/
      - name: byteox-app2b
        # 【需更换】
        image: 172.24.137.212:5000/ztlydev/byteox-app2b:latest
        ports:
        - containerPort: 61003
        imagePullPolicy: Always
        lifecycle:
          preStop:
            httpGet:
              port: 61003
              path: isc/shutdown
        livenessProbe:
          initialDelaySeconds: 70
          failureThreshold: 5
          periodSeconds: 15
          timeoutSeconds: 3
          httpGet:
            port: 61003
            path: isc/readiness
        readinessProbe:
          initialDelaySeconds: 10
          periodSeconds: 5
          timeoutSeconds: 2
          failureThreshold: 1
          successThreshold: 1
          httpGet:
            port: 61003
            path: isc/readiness
        command: ["java","-Duser.timezone=Asia/Shanghai","-Xms512m","-Xmx512m","-jar","-Dloader.path=./lib","/app/byteox-app2b/byteox-app2b.jar"]
        volumeMounts:
        - name: byteox-app2b-logs
          mountPath: /app/byteox-app2b/log
      volumes:
      #- name: app2b-log-vol
      #  hostPath:
      #    path: /opt/byteox-volume/byteox-app2b/log
      - name: byteox-app2b-logs
        emptyDir: {}
      - name: byteox-app2b-filebeat-config
        configMap:
          name: byteox-app2b-filebeat-config
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: byteox-app2b-filebeat-config
  namespace: dev-byteox
data:
  filebeat.yml: |
    filebeat.inputs:
    - type: filestream
      id: byteox-app2b-total
      paths:
        - /log/log_total.log
      fields_under_root: true
      fields:
        svc_name: 'byteox-app2b'
      keep_null: true
      parsers:
      - multiline:
          type: pattern
          pattern: '^20'
          negate: true
          match: after
    processors:
      - include_fields:
          fields: ["message", "svc_name"]
      - dissect:
          tokenizer: '%{} %{} %{level}[%{}'
          field: "message"
          target_prefix: ""
          overwrite_keys: true
          ignore_failure: true
          trim_values: right
    setup.ilm:
      rollover_alias: pms-dev
      pattern: "{now/d}-000001"
      overwrite: true
      policy_name: 90-days-default
    output.elasticsearch:
      hosts: ["http://172.24.137.212:9200"]
      username: "elastic"
      password: "Um6bUP5uzHx2VQJJ3Key"
    setup.kibana:
      host: "http://172.24.137.212:5601"
      username: "elastic"
      password: "Um6bUP5uzHx2VQJJ3Key"

与ip地址等相关的配置调整

  • jenkins 配置： CONFIG_ENV=ztlyde 全局替换为 CONFIG_ENV=ztlyde-shuke
  • maven 打包配置：复制 settings-ztlydev.xml 到 settings-ztlydev-shuke.xml 并调整内容 (/app/jenkins/jenkins_home/apache-maven-3.8.6/conf )
  • k8s原始配置：复制 yaml/ztlydev 到 yaml/ztlydev-shuke, 并调整内容 （/app/jenkins/jenkins_home/byteox/yaml/ztlydev-shuke）
  • 代码的 properties 设置： 新增 application-ztlydev-shuke.properties 配置并作对应调整