Gitlab+Jenkins+docker部署实例

TOC

环境架构以及部署步骤(如下图所示)
cicd.001
1.先分别在gitlab和harbor上部署好gitlab代码仓库和harbor镜像仓库,再在harbor上安装git客户端拉取tale代码上传到gitlab代码仓库上
2.在jenkins服务器做好tale服务器的免密登录,方便使用ssh给tale发送部署命令
3.在jenkins服务器上先安装好jdk和maven,其次是jenkins和docker,配置好job编写好部署脚本
4.在tale服务器安装好docker,因为使用docker部署

★准备工作★

一共需要四台服务器
gitlab:192.168.31.101
harbor:192.168.31.102
jenkins:192.168.31.103
tale:192.168.31.104
注意:为了方便四台服务器以以上名字命名
服务器配置
gitlab(192.168.31.101)和jenkins(192.168.31.103)最好给3G运行内存,gitlab(192.168.31.101)能给更高更好,运行gitlab很耗内存
harbor(192.168.31.102)和tale(192.168.31.104)给2G运行内存即可
所需安装包下载地址
gitlab下载地址:https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/gitlab-ce-14.7.0-ce.0.el7.x86_64.rpm
harbor下载地址:https://github.com/goharbor/harbor/releases/download/v2.5.1/harbor-online-installer-v2.5.1.tgz
jenkins下载地址:https://mirrors.tuna.tsinghua.edu.cn/jenkins/war-stable/2.319.2/jenkins.war
maven下载地址:https://dlcdn.apache.org/maven/maven-3/3.8.6/binaries/apache-maven-3.8.6-bin.tar.gz
jdk下载地址:https://www.oracle.com/java/technologies/downloads/#java8

一、安装Gitlab

gitlab(192.168.31.101):
下载安装

yum -y install policycoreutils policycoreutils-python
rpm -ivh gitlab-ce-14.7.0-ce.0.el7.x86_64.rpm

修改gitlab配置文件指定服务器ip和自定义端口

vim /etc/gitlab/gitlab.rb
----------------------------------------
external_url 'http://192.168.31.101'

重置配置文件并启动

gitlab-ctl reconfigure && gitlab-ctl start

访问192.168.31.101进入gitlab
用户为root,初始密码在/etc/gitlab/initial_root_password文件下

二、安装Harbor

安装docker

yum install -y yum-utils
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum -y install docker-ce-20.10.9
systemctl start docker

安装docker-compose
docker的20.10.9需要1.18.0+版本的docker-compose支持,添加可执行权限,查看版本

curl -L https://github.com/docker/compose/releases/download/1.18.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
[root@harbor ~]# docker-compose version
docker-compose version 1.18.0, build 8dd22a9
docker-py version: 2.6.1
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.0.1t  3 May 2016

安装harbor镜像仓库
下载harbor安装包、创建工作目录

wget https://github.com/goharbor/harbor/releases/download/v2.5.1/harbor-online-installer-v2.5.1.tgz
tar zxf harbor-online-installer-v2.5.1.tgz
mv harbor /harbor

配置配置文件,有模板可以复制来用

cp /harbor/harbor.yml.tmpl /harbor/harbor.yml
vim /harbor/harbor.yml
---------------------------------
hostname: 192.168.31.102
http:
  port: 80
#https:
#  port: 443
#  certificate: /your/certificate/path
#  private_key: /your/private/key/path
harbor_admin_password: Harbor12345
database:
  password: root123
  max_idle_conns: 100
  max_open_conns: 900
data_volume: /data
trivy:
  ignore_unfixed: false
  skip_update: false
  offline_scan: false
  insecure: false
jobservice:
  max_job_workers: 10
notification:
  webhook_job_max_retry: 10
chart:
  absolute_url: disabled
log:
  level: info
  local:
    rotate_count: 50
    rotate_size: 200M
    location: /var/log/harbor
_version: 2.5.0
proxy:
  http_proxy:
  https_proxy:
  no_proxy:
  components:
    - core
    - jobservice
    - trivy
upload_purging:
  enabled: true
  age: 168h
  interval: 24h
  dryrun: false

运行准备安装脚本和安装脚本

cd /harbor
./prepare
./install

访问192.168.31.102进入harbor
用户名为admin,密码为配置文件配置的harbor_admin_password项的值

三、安装配置CI/CD流程

1.gitlab创建tale项目

项目详情(如下图所示):
cicd.002

2.harbor创建tale项目

项目详情(如下图所示):
cicd.003

3.创建上传密钥对到gitlab

创建上传harbor(192.168.31.102)和jenkins(192.168.31.103)的密钥对到gitlab服务器上
注意:harbor的密钥对为了拉取tale代码上传到gitlab服务器,jenkins的密钥是为了方便拉取gitlab的代码
harbor(192.168.31.102):

ssh-keygen
cat /root/.ssh/id_rsa.pub
--------------------------------
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7UjzGhqs1oboPGJth9NFtnh5dD4MCdd+/GlpY6tWri1mXGLJDQJ48P8U2v2IWDzNlp9J3lwLdQ8j/g/L64XxXworTUkBUUZk1IOKr9y4Wc/e409gfUrWwVVxIxaW8Eo/hN9ERYKwnq2dPZE3BVLjJL0syRwrUAXG+8CxH6TCMQMMFkFvQu7UNAOQRHzNYv8xDzOBc3sRHhpbrHjqJUqIYlt/NYwxN7N/D4gEhw4iJKokX1mQ/at5QOZlo7s/bF2ir58vftwybHh487Tt8BpJRzMS3gSR7J/1ePi2WEXRwU+9cX1jsYov2/Vj7mmAUpd3Elk4wyXHFHPjAT0JPjceJ root@harbor

cicd.004
jenkins(192.168.31.103):

ssh-keygen
cat /root/.ssh/id_rsa.pub
--------------------------------
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvFpXsMC4g4RV1/q5LCAHOi5SRs51jW51lO3qGF9kuZBGVWCr8ZowtP+K9VkQP7imRgQ50ot42kS/wyMPl8K8nib5fJYwTtJ526XKUlWxSBH9Z6unCxMVzcuCxz5Rrt93AD+sVfoqR0RcQ5ngRKcp4av09IzuVsyEABqb3Mz/OQTnArEWrh7Q++pOEI7Gst3VmEMvCa/+WxD5umkDpkPhZ8yRmb1EjpuUsN6kNPwI6o3g6NIGdZo+8RCBioUHmMEqAi9Udk1dSJiKtdZ7ogYs+s/tZtXOpcKcs2h8qfASDN0y0mxQsuol/Aqxkl5GkU0PnFNWxDcF6Q6Sv6xGw4cq/ root@jenkins

cicd.005

4.上传tale代码

使用harbor(192.168.31.102)的git客户端拉取tale代码上传到gitlab服务上
注意:该步骤只是为了提交代码到gitlab上,可以当作开发写完的代码提交,也可以手动拖拽项目上传到gitlab上
harbor(192.168.31.102):
安装git客户端

yum install curl-devel expat-devel gettext-devel openssl-devel zlib-devel
yum -y install git-core
[root@harbor ~]# git --version
git version 1.8.3.1

下拉tale代码上传到gitlab上

git clone https://github.com/otale/tale.git
cd tale
#1.初始化本地仓库
git init
#2.配置用户名
git config --global user.name "admin"
#3.配置email邮件信息
git config --global user.email "admin@example.com"
#4.添加文件到暂存区
git add .
#5.提交项目到本地仓库
git commit -m "tale code"
#6.与远程仓库关联
git remote add main git@192.168.31.101:root/tale.git
#7.将项目推送到远程仓库
git push -u main master

提交成功后可以在gitlab服务器上看到提交信息(如下图所示)
cicd.006

5.jenkins和tale服务器安装docker

注意:jenkins服务器安装docker是为了制作镜像,tale服务器安装docker是为了运行docker项目
jenkins(192.168.31.103)、tale(192.168.31.104):

yum install -y yum-utils
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum -y install docker-ce-20.10.9
systemctl start docker

配置harbor地址方便http方式访问,要不会报错

echo '{"insecure-registries":["192.168.31.102"]}' > /etc/docker/daemon.json

6.jenkins服务器安装编译环境和jenkins

注意:安装jdk是为了支持maven和jenkins运行,安装maven是为了编译tale代码成jar包
jenkins(192.168.31.103):
安装jdk和maven

tar zxf jdk-8u333-linux-x64.tar.gz
mv jdk1.8.0_333 /usr/local/jdk
tar zxf apache-maven-3.8.6-bin.tar.gz
mv apache-maven-3.8.6 /usr/local/maven

配置环境变量

vim /etc/profile
-------------------------------------
export JAVA_HOME=/usr/local/jdk
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:$CLASSPATH
export JAVA_PATH=${JAVA_HOME}/bin:${JRE_HOME}/bin
export PATH=$PATH:${JAVA_PATH}
export MAVEN_HOME=/usr/local/maven
export PATH=$MAVEN_HOME/bin:$PATH

运行profile文件查看版本

source /etc/profile
[root@jenkins ~]# java -version
java version "1.8.0_333"
Java(TM) SE Runtime Environment (build 1.8.0_333-b02)
Java HotSpot(TM) 64-Bit Server VM (build 25.333-b02, mixed mode)
[root@jenkins ~]# mvn -v
Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: /usr/local/maven
Java version: 1.8.0_333, vendor: Oracle Corporation, runtime: /usr/local/jdk/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-1160.el7.x86_64", arch: "amd64", family: "unix"

下载运行jenkins

mkdir /jenkins
cp jenkins.war /jenkins/jenkins-2.319.2.war
nohup java -jar /jenkins/jenkins-2.319.2.war –-httpPort=8080 >> /jenkins/jenkins.log 2>&1 &

安装git客户端
注意:如果jenkins服务器不安装git客户端,那么配置gitlab信息的话会报错

yum install curl-devel expat-devel gettext-devel openssl-devel zlib-devel
yum -y install git-core

7.配置jenkins到tale服务器的ssh免密

注意:该步骤是为了方便jenkins任务构建时方便通过ssh使得tale服务器执行命令
jenkins(192.168.31.103):
因为之前生成过密钥对了,所以这里直接传递密钥对到192.168.31.104

ssh-copy-id -i /root/.ssh/id_rsa root@192.168.31.104

测试是否能够免密登录192.168.31.104

[root@jenkins ~]# ssh root@192.168.31.104
Last failed login: Tue Jul 26 09:42:07 EDT 2022 from 192.168.31.103 on ssh:notty
There was 1 failed login attempt since the last successful login.
Last login: Tue Jul 26 06:17:38 2022
[root@tale ~]#

8.配置jenkins的部署脚本

注意:该步骤可以说是最重要、最关键的一个步骤
jenkins(192.168.31.103):
第一次配置了ssh密钥需要使用git客户端拉取一下代码,验证密钥

[root@jenkins ~]# git clone git@192.168.31.101:root/tale.git
Cloning into 'tale'...
The authenticity of host '192.168.31.101 (192.168.31.101)' can’t be established.
ECDSA key fingerprint is SHA256:7ViD7QHhinwptvDdKj2M+hw5RU1D0Qf/n5ndJLln2Jo.
ECDSA key fingerprint is MD5:74:f8:44:92:4e:5e:70:49:da:bb:a8:ab:8a:20:02:ec.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.31.101' (ECDSA) to the list of known hosts.
remote: Enumerating objects: 6703, done.
remote: Counting objects: 100% (6703/6703), done.
remote: Compressing objects: 100% (2327/2327), done.
remote: Total 6703 (delta 3516), reused 6700 (delta 3516), pack-reused 0
Receiving objects: 100% (6703/6703), 27.12 MiB | 8.89 MiB/s, done.
Resolving deltas: 100% (3516/3516), done.

tale博客系统安装注意事项:
1.tale博客系统maven打包默认使用prod配置项,避免启动出错
2.maven打包之后会在工作目录出现target/dist目录,里面会有tale.tar.gz和tale.zip两个压缩包,只需将其复制到指定目录解压运行里面的jar包即可
注意:不能只复制解压后里面的jar包,因为里面有jar包启动需要的文件,否则会运行报错
编写jenkins任务前的准备工作
jenkins(192.168.31.103):
创建工作目录和文件

mkdir -p /docker/tale
#tale.txt文件用作更新docker镜像的tag号
echo 0 > /docker/tale/tale.txt

复制下载的jdk安装包到docker的工作目录

cp jdk-8u333-linux-x64.tar.gz /docker/tale

编辑Dockerfile

vim /docker/tale/Dockerfile
---------------------------------
FROM ubuntu:20.04
MAINTAINER www.renjiezhang.vip
RUN mkdir /tale
WORKDIR /tale
ADD jdk-8u333-linux-x64.tar.gz /tale
RUN mv /tale/jdk1.8.0_333 /usr/local/jdk
ADD tale.tar.gz /tale/
EXPOSE 9000
CMD /usr/local/jdk/bin/java -jar /tale/tale-*.jar

tale(192.168.31.104):
创建工作目录编写部署脚本,用作拉取制作好的docker镜像,再运行起来

mkdir /tale
vim /tale/deploy-tale.sh
-------------------------------------
#!/bin/bash
docker login http://192.168.31.102 -u admin -p Harbor12345
docker pull 192.168.31.102/taleproduct/tale:$(cat /tale/tale.txt)
docker run -it -d --name tale -p 9000:9000 192.168.31.101/taleproduct/tale:$(cat /tale/tale.txt)

这里从Harbor仓库拉取镜像运行起来
配置jenkins
jenkins(192.168.31.103):
登录jenkins创建job任务,配置任务模块
配置gitlab仓库
cicd.007
创建jenkins任务
cicd.008
该步骤编写部署流程的脚本,跟写shell脚本一样

cd /root/.jenkins/workspace/tale;mvn clean package -Pprod -Dmaven.test.skip=true -U install
cp -rf /root/.jenkins/workspace/tale/target/dist/tale.tar.gz /docker/tale
cd /docker/tale;docker build -t 192.168.31.102/taleproduct/tale:$(cat tale.txt) --no-cache .
docker login http://192.168.31.102 -u admin -p Harbor12345
cd /docker/tale;docker push 192.168.31.102/taleproduct/tale:$(cat tale.txt)
scp /docker/tale/tale.txt root@192.168.31.104:/tale/
cd /docker/tale;echo $(cat tale.txt)+1|/usr/bin/bc > aaa.txt
cd /docker/tale;mv aaa.txt tale.txt
ssh root@192.168.31.104 'sh /tale/deploy-tale.sh'

测试构建结果

jenkins点击build(如下图所示)
cicd.009
tale(192.168.31.104):
查看docker运行结果

[root@tale ~]# docker ps
CONTAINER ID   IMAGE                               COMMAND                  CREATED          STATUS          PORTS                                       NAMES
3e373b06ccd1   192.168.31.102/taleproduct/tale:0   "/bin/sh -c '/usr/lo…"   36 seconds ago   Up 32 seconds   0.0.0.0:9000->9000/tcp, :::9000->9000/tcp   tale

访问192.168.31.104:9000(如下图所示)
cicd.010