gitlab迁移实践

因办公室环境网络调整，近期计划将gitlab从内网机房迁移至公有云。迁移过程做了一下简单的记录，希望对各位同行有所帮助。

一、迁移基本思路
1、采购公有云服务器，自带公网IP、加入onlyyou安全组。
2、安全组开放80端口（所有办公网出口、v隧p道n（公））、9000端口（所有办公网出口、v隧p道n(公)、以及Jks、其他测试服公网IP）。
3、搭建同版本Git服务。
4、发布公告，暂停git服务
5、将完整备份导入新Git。
6、利用iptables映射9000端口至3303（iptables -t nat -A PREROUTING -p tcp --dport 9000 -j REDIRECT --to-ports 3303）。（注：9000为之前frp的远程端口，3303为服务器B的ssh端口）
7、DNS解析（git.bd.com）：删除办公网DNS的解析记录，修改公网DNS解析记录至服务器B公网IP。

二、操作步骤
2.1、备份
1.备份服务器A中的git数据,具体备份操作命令
[root@serverA ~]# gitlab-rake gitlab:backup:create STRATEGY=copy
备份文件在/var/opt/gitlab/backups/下，假设备好的文件为1568659149_2019_03_17_10.6.4_gitlab_backup.tar
注：在备份期间需禁止对gitlab进行任何操作

2.2、新服务器搭建gitlab

    为了备份可用，新服务器上使用跟原服务器相同的版本
    如果旧服务器已是最新版，可以在新服务器直接使用yum安装.
    如果安装最新版本，采用以下方式安装即可
    `[root@serverB ~]#  yum install gitlab-ce`

    本人采用的是指定的相同的版本来进行安装的

    1.因不是最新版本，需要从https://packages.gitlab.com/gitlab/gitlab-ce上下载所需的版本，此处因为安装系统为centos6，所以下载el6版本
    或者使用服务器A中的yum源文件
    `[root@serverB ~]# cat /etc/yum.repos.d/gitlab_gitlab-ce.repo`
    [gitlab_gitlab-ce]
    name=gitlab_gitlab-ce
    baseurl=https://packages.gitlab.com/gitlab/gitlab-ce/el/6/$basearch
    repo_gpgcheck=1
    gpgcheck=0
    enabled=1
    gpgkey=https://packages.gitlab.com/gpg.key
    sslverify=1
    sslcacert=/etc/pki/tls/certs/ca-bundle.crt

    [gitlab_gitlab-ce-source]
    name=gitlab_gitlab-ce-source
    baseurl=https://packages.gitlab.com/gitlab/gitlab-ce/el/6/SRPMS
    repo_gpgcheck=1
    gpgcheck=0
    enabled=1
    gpgkey=https://packages.gitlab.com/gpg.key
    sslverify=1
    sslcacert=/etc/pki/tls/certs/ca-bundle.crt

    2.安装git
    [root@serverB ~]#  `yum -y install git`

    3.安装gitlib
    [root@serverB ~]#  ```
    EXTERNAL_URL="http://gitlab.example.com" yum install -y gitlab-ce-10.6.4-ce.0.el6.x86_64
    ```

    4、修改配置
    安装好后，修改/etc/gitlab/gitlab.rb
    external_url 'http://gitlab.example.com'
    其上所有的http://gitlab.example.com改成自己要使用的gitlab地址

    在/var/opt/gitlab/nginx/conf/gitlab-http.conf 中修改所需端口
    在/var/opt/gitlab/nginx/conf/nginx.conf 中增加
     include /var/opt/gitlab/nginx/conf/custom.conf;

    端口是为了跟原来的环境保持一致，比如原来的是54444，这里也修改成一样的即可
    custom.conf  是自己定义的白名单，比如：
    allow 110.110.110.110;
    deny  all;
    server {
            server_name default;
            listen *:80;
            location / {
                    proxy_pass http://127.0.0.1:54444;
            }
    }

    重新配置升效

    [root@serverB ~]# gitlab-ctl reconfigure

2.3、拷贝备份
将备份从服务器A拷到新服务器B的/var/opt/gitlab/backups/下，此步从服务器A上操作

    [root@serverA ~]# scp /var/opt/gitlab/backups/1568659149_2019_03_17_10.6.4_gitlab_backup.tar root@192.168.100.10:/var/opt/gitlab/backups/
    修改备份文件权限，以免恢复备份时出现权限不足的情况

    [root@serverB ~]# chown git:git 1568659149_2019_03_17_10.6.4_gitlab_backup.tar
    [root@serverB ~]# chmod 777 1568659149_2019_03_17_10.6.4_gitlab_backup.tar

2.4、恢复备份
此步参考官方文档https://docs.gitlab.com/ce/raketasks/backup_restore.html#restore-for-omnibus-installations
1.停止数据相服务

    [root@serverB ~]# gitlab-ctl stop unicorn
    [root@serverB ~]# gitlab-ctl stop sidekiq
    [root@serverB ~]# gitlab-ctl status

    2.恢复备份
    此处命令结尾使用的是上面拷贝过来的文件名，但不是全名，取其_gitlab之前的名称即可
    最开始的时候，会提示输入yes，直接输入即可
    [root@serverB ~]# gitlab-rake gitlab:backup:restore BACKUP=1568659149_2019_03_17_10.6.4
    Unpacking backup ... done
    Before restoring the database, we will remove all existing
    tables to avoid future upgrade problems. Be aware that if you have
    custom tables in the GitLab database these tables and all data will be
    removed.

    ```
    **Do you want to continue (yes/no)? yes**
    ```
    Removing all tables. Press `Ctrl-C` within 5 seconds to abort
    Cleaning the database ... 
    done
    Restoring database ... 
    Restoring PostgreSQL database gitlabhq_production ... SET
    SET
    SET
    SET
    .......

    Put GitLab hooks in repositories dirs [DONE]
    done
    Restoring uploads ... 
    done
    Restoring builds ... 
    done
    Restoring artifacts ... 
    done
    Restoring pages ... 
    done
    Restoring lfs objects ... 
    done
    This will rebuild an authorized_keys file.
    You will lose any data stored in authorized_keys file.

    **Do you want to continue (yes/no)? no  ##这里如果保留旧的权限，输入no
    **Quitting...

    3.重置服务

    [root@serverB ~]# gitlab-ctl restart
    [root@serverB ~]# gitlab-rake gitlab:check SANITIZE=true

三、邮件功能确认
关于gitlab是否可以正常发送邮件，需要做好确认。
旧环境中，serverA中的配置使用的是25端口，但是公有云服务器默认是屏蔽25端口的，发送邮件就会出现如下报错。

最初配置为：
[root@serverB ~]# grep -v "#" /etc/gitlab/gitlab.rb | grep -v "^$"
external_url 'http://git.bd.com'
nginx['listen_port'] = 54444
gitlab_rails['time_zone'] = 'Asia/Shanghai'
gitlab_rails['gitlab_shell_ssh_port'] = 9000
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.exmail.qq.com"
gitlab_rails['smtp_port'] = 25
gitlab_rails['smtp_user_name'] = "git@bd.com"
gitlab_rails['smtp_password'] = "88888888"
gitlab_rails['smtp_domain'] = "exmail.qq.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['gitlab_email_from'] = 'git@bd.com'
user["git_user_email"] = "git@bd.com"
nginx['custom_nginx_config'] = "include /var/opt/gitlab/nginx/conf/custom.conf;"
gitlab_ci['backup_path'] = "/gitlab_backup"
[root@serverB ~]#

测试发送邮件功能
[root@serverB ~]# gitlab-rails console
Loading production environment (Rails 4.2.10)
irb(main):001:0> Notify.test_email("ch@bd.com","title","gitlab").deliver_now

Notify#test_email: processed outbound mail in 180.0ms

Sent mail to ch@bd.com (1885.3ms)
Date: Thu, 19 Sep 2019 19:05:27 +0800
From: GitLab <git@bd.com>
Reply-To: GitLab <noreply@git.bd.com>
To: ch@bd.com
Message-ID: <5d8360f7d0fae_79823fbf26cdb1b033298@serverB.mail>
Subject: title
Mime-Version: 1.0
Content-Type: text/html;
charset=UTF-8
Content-Transfer-Encoding: 7bit
Auto-Submitted: auto-generated
X-Auto-Response-Suppress: All

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html><body><p>gitlab</p></body></html>

Net::OpenTimeout: execution expired
from /opt/gitlab/embedded/lib/ruby/2.3.0/net/smtp.rb:542:in initialize'<br/>from /opt/gitlab/embedded/lib/ruby/2.3.0/net/smtp.rb:542:inopen'
from /opt/gitlab/embedded/lib/ruby/2.3.0/net/smtp.rb:542:in tcp_socket'<br/>from /opt/gitlab/embedded/lib/ruby/2.3.0/net/smtp.rb:552:inblock in do_start'
from /opt/gitlab/embedded/lib/ruby/2.3.0/timeout.rb:101:in timeout'<br/>from /opt/gitlab/embedded/lib/ruby/2.3.0/net/smtp.rb:551:indo_start'
from /opt/gitlab/embedded/lib/ruby/2.3.0/net/smtp.rb:521:in start'<br/>from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mail-2.7.0/lib/mail/network/delivery_methods/smtp.rb:109:instart_smtp_session'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mail-2.7.0/lib/mail/network/delivery_methods/smtp.rb:100:in deliver!'<br/>from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mail-2.7.0/lib/mail/message.rb:2160:indo_delivery'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mail-2.7.0/lib/mail/message.rb:260:in block in deliver'<br/>from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/actionmailer-4.2.10/lib/action_mailer/base.rb:543:inblock in deliver_mail'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/notifications.rb:164:in block in instrument'<br/>from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/notifications/instrumenter.rb:20:ininstrument'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/notifications.rb:164:in instrument'<br/>from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/actionmailer-4.2.10/lib/action_mailer/base.rb:541:indeliver_mail'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/mail-2.7.0/lib/mail/message.rb:260:in deliver'<br/>from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/actionmailer-4.2.10/lib/action_mailer/message_delivery.rb:85:indeliver_now'
from (irb):1
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.10/lib/rails/commands/console.rb:110:in start'<br/>from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.10/lib/rails/commands/console.rb:9:instart'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.10/lib/rails/commands/commands_tasks.rb:68:in console'<br/>from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.10/lib/rails/commands/commands_tasks.rb:39:inrun_command!'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.10/lib/rails/commands.rb:17:in &lt;top (required)&gt;'<br/>from bin/rails:9:inrequire'
from bin/rails:9:in `<main>'

经过排查分析，25端口已经被统一关闭，需要使用安全传输层协议进行发送邮件。

修改配置为
[root@serverB ~]# grep -v "#" /etc/gitlab/gitlab.rb | grep -v "^$"
external_url 'http://git.bd.com'
nginx['listen_port'] = 54444
gitlab_rails['time_zone'] = 'Asia/Shanghai'
gitlab_rails['gitlab_shell_ssh_port'] = 9000
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.exmail.qq.com"
gitlab_rails['smtp_tls'] = true
gitlab_rails['smtp_port'] = 465
gitlab_rails['smtp_user_name'] = "git@bd.com"
gitlab_rails['smtp_password'] = "88888888"
gitlab_rails['smtp_domain'] = "exmail.qq.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['gitlab_email_from'] = 'git@bd.com'
user["git_user_email"] = "git@bd.com"
nginx['custom_nginx_config'] = "include /var/opt/gitlab/nginx/conf/custom.conf;"
gitlab_ci['backup_path'] = "/gitlab_backup"
[root@serverB ~]#

修改后执行gitlab-ctl reconfigure

确认可以发送邮件：
[root@serverB ~]# gitlab-rails console
Loading production environment (Rails 4.2.10)
irb(main):001:0> Notify.test_email("ch@bd.com","title","gitlab").deliver_now

Notify#test_email: processed outbound mail in 180.0ms

Sent mail to ch@bd.com (1885.3ms)
Date: Thu, 19 Sep 2019 19:05:27 +0800
From: GitLab <git@bd.com>
Reply-To: GitLab <noreply@git.bd.com>
To: ch@bd.com
Message-ID: <5d8360f7d0fae_79823fbf26cdb1b033298@serverB.mail>
Subject: title
Mime-Version: 1.0
Content-Type: text/html;
charset=UTF-8
Content-Transfer-Encoding: 7bit
Auto-Submitted: auto-generated
X-Auto-Response-Suppress: All

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html><body><p>gitlab</p></body></html>

=> #<Mail::Message:70089965796760, Multipart: false, Headers: <Date: Thu, 19 Sep 2019 19:05:27 +0800>, <From: GitLab <git@bd.com>>, <Reply-To: GitLab <noreply@git.bd.com>>, <To: ch@bd.com>, <Message-ID: <5d8360f7d0fae_79823fbf26cdb1b033298@serverB.mail>>, <Subject: title>, <Mime-Version: 1.0>, <Content-Type: text/html; charset=UTF-8>, <Content-Transfer-Encoding: 7bit>, <Auto-Submitted: auto-generated>, <X-Auto-Response-Suppress: All>>
irb(main):002:0

若发送不成功请看下是否配置user['git_user_email']，若以上都配置了请查看云服务器安全组有没有开放465端口。

四、验证方法：
办公网环境验证ssh协议命令行操作及Web页面打开情况。 ##通过
办公网v隧p道n环境验证ssh协议命令行操作及Web页面打开情况。 ##通过
公网环境验证ssh协议命令行操作及Web页面禁止访问和打开情况。 ##通过，公网禁止访问
公网v隧p道n环境验证ssh协议命令行操作及Web页面打开情况。 ##通过