K8S发布系统-基础服务部署

Nw-Base主机(220.140)

MySQL

MySQL部署于172.16.220.140物理主机上,部署方式为源码编译部署;具体部署过程可见相关文档。数据库储存路径为:/data/mysql/。

Redis

Redis也部署于 172.16.220.140 物理主机上,部署方式为源码编译部署;过程如下:

  1. 下载:
    wget http://download.redis.io/releases/redis-5.0.8.tar.gz
    tar zxf redis-5.0.8.tar.gz -C /usr/local/src/
    cd /usr/local/src/redis-5.0.8/
  2. 安装:
    groupadd -g 415 redis
    useradd -g redis -u 415 -s /sbin/nologin -d /data/redis -c "Redis Server" -M redis
    make PREFIX=/data/redis/ CFLAGS="-fPIC"
    make PREFIX=/data/redis/ install
    echo 'export PATH=/data/redis/bin/:$PATH' > /etc/profile.d/redis.sh
    source /etc/profile.d/redis.sh
    redis-cli -v
    mkdir /data/redis/{db,logs}
    chown -R redis:redis /data/redis/
    cp redis.conf /data/redis/
  3. 修改配置,设置密码:
    sed -i 's/^bind 127.0.0.1/bind 0.0.0.0/g' /data/redis/redis.conf
    sed -i 's#^pidfile /var/run/redis_6379.pid#pidfile /data/redis/redis.pid#g' /data/redis/redis.conf
    sed -i 's#^dir ./#dir /data/redis/db#g' /data/redis/redis.conf
    sed -i 's#^logfile ""#logfile "/data/redis/logs/redis.log"#g' /data/redis/redis.conf
    sed -i 's/^# requirepass .*/requirepass Zs20191001/' /data/redis/redis.conf
  4. 设置服务:
    vim /lib/systemd/system/redis.service
    # Redis Service
    [Unit]
    Description=Redis Community Server
    After=network.target
    
    [Install]
    WantedBy=multi-user.target
    
    [Service]
    User=redis
    Group=redis
    LimitNOFILE=65535
    LimitNPROC=65535
    ExecStart=/data/redis/bin/redis-server /data/redis/redis.conf
    Restart=on-failure
    systemctl enable redis
    systemctl start redis
    systemctl status redis

  5. 防火墙设置(如果有):
    firewall-cmd --permanent --add-port=6379/tcp
    firewall-cmd --reload

JAVA环境

tar zxf jdk-8u271-linux-x64.tar.gz -C /usr/local/
echo 'export JAVA_HOME=/usr/local/jdk1.8.0_271' > /etc/profile.d/java.sh
echo 'export PATH=$JAVA_HOME/bin:$PATH' >> /etc/profile.d/java.sh
source /etc/profile.d/java.sh
java -version

Nginx

部署

安装:

yum install -y pcre-devel
tar zxf nginx-1.18.0.tar.gz -C /usr/local/src/
cd /usr/local/src/nginx-1.18.0/
groupadd -g 211 ngx
useradd -g ngx -u 211 -s /sbin/nologin -c "Nginx Server" -M ngx
./configure --prefix=/data/nginx --user=ngx --group=ngx --with-threads --with-http_realip_module --with-http_ssl_module --with-stream --with-stream_ssl_module --with-http_stub_status_module --with-http_gzip_static_module --with-http_slice_module --with-cc-opt="-I/usr/local/openssl/include" --with-ld-opt="-L/usr/local/openssl/lib"
make -j `cat /proc/cpuinfo | grep processor | wc -l` && make install

配置文件:

cd /data/nginx
sed -i '1s/^$/# Nginx Main Configure File./' conf/nginx.conf
sed -i '2s/^#//' conf/nginx.conf
sed -i '2s/nobody/ngx/' conf/nginx.conf
sed -i '35,116d' conf/nginx.conf
sed -i '$ i \ \ \ \ include vhosts/*.conf;' conf/nginx.conf
mkdir conf/vhosts

设置服务:

vim /usr/lib/systemd/system/nginx.service
# systemd service file for Nginx server
[Unit]
Description=The NGINX HTTP and reverse proxy server
After=syslog.target network.target remote-fs.target nss-lookup.target

[Install]
WantedBy=multi-user.target

[Service]
Type=forking
PIDFile=/data/nginx/logs/nginx.pid
ExecStartPre=/data/nginx/sbin/nginx -t
ExecStart=/data/nginx/sbin/nginx
ExecReload=/data/nginx/sbin/nginx -t
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true
systemctl enable nginx
systemctl start nginx
systemctl status nginx

设置转发

设置gitlab转发:

cd /data/nginx/conf/vhosts/
vim gitlab.conf
# Nginx Second Configure File.
upstream nw-gitlab {
        server 172.16.220.139:80;
}
server {
        listen 80;
        server_name nw-gitlab.zongs365.cc;
        access_log logs/gitlab_access.log;
        error_log  logs/gitlab_error.log ;
    location / {
        proxy_pass http://nw-gitlab;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

其它转k8s内服务:

vim k8s.conf
# Nginx Second Configure File.
upstream ingress-http {
        server 172.16.220.141:23456;
        server 172.16.220.142:23456;
}
server {
        listen 80 default_server;
        access_log logs/k8s_access.log;
        error_log  logs/k8s_error.log ;
    location / {
        proxy_pass http://ingress-http;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

使用TCP协议监听443端口:

vim /data/nginx/conf/nginx.conf
...
stream {
    include vhosts/*.stream;
}
...
vim k8s.stream
# Nginx Stream Second Configure File.
upstream ingress-https {
        server 172.16.220.141:23457;
        server 172.16.220.142:23457;
}
server {
        listen 443;
        proxy_pass ingress-https;
}   

使配置生效:

systemctl reload nginx

Jenkins

新建测试用于发布的Jenkins部署于 172.16.220.140 物理主机上,部署方式为WAR包部署,过程如下:

  1. 下载:
    wget https://mirrors.tuna.tsinghua.edu.cn/jenkins/war-stable/2.263.2/jenkins.war
    mkdir -p /data/jenkins/{app,logs,.m2}
    cp -p jenkins.war /data/jenkins/app/
  2. 创建运行用户:
    groupadd -g 498 jenkins
    useradd -g jenkins -u 498 -s /bin/false -d /data/jenkins -c "Continuous Integration Server" -M jenkins
    chown -R jenkins:jenkins /data/jenkins
  3. mvn用户配置文件:
    建立配置文件,设置mvn仓库使用本地nexus私服。
    vim /data/jenkins/.m2/settings.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <settings>
    <profiles>
      <profile>
        <id>nexus</id>
        <properties>
          <maven.compiler.source>1.8</maven.compiler.source>
          <maven.compiler.target>1.8</maven.compiler.target>
          <maven.compiler.compilerVersion>1.8</maven.compiler.compilerVersion>
        </properties>
        <repositories>
          <repository>
            <id>local-nexus</id>
            <url>http://nw-nexus:8081/repository/maven-public/</url>
            <releases>
              <enabled>true</enabled>
            </releases>
            <snapshots>
              <enabled>true</enabled>
            </snapshots>
          </repository>
        </repositories>
        <pluginRepositories>
          <pluginRepository>
            <id>local-nexus</id>
            <url>http://nw-nexus:8081/repository/maven-public/</url>
            <releases>
              <enabled>true</enabled>
            </releases>
            <snapshots>
              <enabled>true</enabled>
            </snapshots>
          </pluginRepository>
        </pluginRepositories>
      </profile>
    </profiles>
    <activeProfiles>
      <activeProfile>nexus</activeProfile>
    </activeProfiles>
    </settings>

    由于我们不直接在全局的settings.xml中插入<repositories>元素,这里添加了一个profile并使用<activeProfile>元素自动将这个profile激活。这里的local-nexus仓库指向了Nexus中默认的“maven-public”仓库组,即所有该仓库组包含的仓库都能供我们使用。此外,通过<releases>和<snapshots>元素激活了Maven对于仓库所有类型构件下载的支持,当然你也可以调节该配置,比如说禁止Maven从Nexus下载snapshot构件。

    并且设置指定了maven构建所使用JAVA编译器 -source 和 -target 的版本为1.8。

    使用该配置,Maven就会从Nexus服务器下载构件了。

  4. 手工方式启动:
    su - jenkins -s /bin/bash -c "cd /data/jenkins/app ; java -server -Xms128m -Xmx1024m -jar jenkins.war -DJENKINS_HOME=/data/jenkins --httpPort=8180 --logfile=/data/jenkins/logs/jenkins.log > /dev/null" &
    tail -f /data/jenkins/logs/jenkins.log

  5. 初始化:
    访问http://172.16.220.140:8180/,将日志中的安装密码复制进去后设置。

    完成之后,会提示创建初始管理员账号:


    Tips:初始化完成之后,用于初始化的密码文件会自动删除。
  6. 设置为服务:
    注意先结束掉手工启动的进程。
    vim /data/jenkins/start.sh
    #!/bin/bash
    cd /data/jenkins/app
    java -server -Xms1024m -Xmx1536m -jar jenkins.war -DJENKINS_HOME=/data/jenkins --httpPort=8180 --logfile=/data/jenkins/logs/jenkins.log > /dev/null &
    chmod +x /data/jenkins/start.sh
    vim /usr/lib/systemd/system/jenkins.service
    # systemd service file for Jenkins
    [Unit]
    Description=Jenkins Automatic
    After=syslog.target network.target remote-fs.target nss-lookup.target
    
    [Install]
    WantedBy=multi-user.target
    
    [Service]
    Type=forking
    User=jenkins
    WorkingDirectory=/data/jenkins
    Environment="JAVA_HOME=/usr/local/jdk1.8.0_271"
    Environment="PATH=/usr/local/jdk1.8.0_271/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/nodejs/bin"
    ExecStart=/data/jenkins/start.sh
    Restart=on-failure
    KillMode=mixed
    systemctl enable jenkins
    systemctl start jenkins
    systemctl status jenkins
  7. 全局工具配置:
    JDK:

    Maven:

发表评论

error: Content is protected !!