从GitLab12.1开始，可以直接使用rake进行备份，如果之前的项目中使用的gitlab-rake进行备份和恢复的方式是否可用这个观点进行了如下实验，通过本文进行简单记录。
备份源环境准备容器化方式，直接使用docker-compose，启动如下服务
[root@host131 gitlab]# cat docker-compose.yml version: '2' services: # Version Control service: Gitlab gitlab: image: gitlab/gitlab-ce:12.10.5-ce.0 ports: - "32001:80" - "30022:22" - "30443:443" volumes: - ./log/:/var/log/gitlab - ./data/:/var/opt/gitlab - ./conf/:/etc/gitlab restart: "no" [root@host131 gitlab]#
修改root用户，并创建如下项目
使用rsync将各数据卷拷贝至备份目标机器，首先同步数据目录
[root@host131 ~]# rsync -r gitlab/data 192.168.163.132:/root/gitlab/data root@192.168.163.132's password: skipping non-regular file "data/gitaly/gitaly.socket" skipping non-regular file "data/gitaly/internal_sockets/internal.sock" skipping non-regular file "data/gitaly/internal_sockets/ruby.0" skipping non-regular file "data/gitaly/internal_sockets/ruby.1" skipping non-regular file "data/gitlab-rails/sockets/gitlab.socket" skipping non-regular file "data/gitlab-workhorse/socket" skipping non-regular file "data/grafana/conf" skipping non-regular file "data/grafana/public" skipping non-regular file "data/nginx/logs" skipping non-regular file "data/postgresql/.s.PGSQL.5432" skipping non-regular file "data/redis/redis.socket" [root@host131 ~]#
注意事前需要在host132备份目标机器创建对应根目录
[root@host132 ~]# pwd /root [root@host132 ~]# ls anaconda-ks.cfg [root@host132 ~]# mkdir gitlab [root@host132 ~]# ls gitlab/ [root@host132 ~]#
不然同步数据的时候会出现类似如下错误提示信息
[root@host131 ~]# rsync -r gitlab/data 192.168.163.132:/root/gitlab/data root@192.168.163.132's password: rsync: mkdir "/root/gitlab/data" failed: No such file or directory (2) rsync error: error in file IO (code 11) at main.c(657) [Receiver=3.1.2] [root@host131 ~]#
- 同步配置数据
[root@host131 ~]# rsync -r gitlab/conf 192.168.163.132:/root/gitlab/conf root@192.168.163.132's password: [root@host131 ~]#
- 同步日志数据
[root@host131 ~]# rsync -r gitlab/log 192.168.163.132:/root/gitlab/log root@192.168.163.132's password: skipping non-regular file "log/gitlab-rails/sidekiq.log" [root@host131 ~]#
PS： 其实也不用这么费劲，直接打个打的tar扔过去就可以了
启动备份目标侧GitLab服务
使用docker-compose启动之后，界面显示如下，看起来和没有数据的方式一样
[root@host131 ~]# docker exec -it gitlab_gitlab_1 sh # gitlab-rake gitlab:backup:create 2020-08-11 23:26:51 +0000 -- Dumping database ... Dumping PostgreSQL database gitlabhq_production ... [DONE] 2020-08-11 23:26:56 +0000 -- done 2020-08-11 23:26:56 +0000 -- Dumping repositories ... * root/testproject (@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b) ... [SKIPPED] [SKIPPED] Wiki 2020-08-11 23:26:57 +0000 -- done 2020-08-11 23:26:57 +0000 -- Dumping uploads ... 2020-08-11 23:26:57 +0000 -- done 2020-08-11 23:26:57 +0000 -- Dumping builds ... 2020-08-11 23:26:57 +0000 -- done 2020-08-11 23:26:57 +0000 -- Dumping artifacts ... 2020-08-11 23:26:57 +0000 -- done 2020-08-11 23:26:57 +0000 -- Dumping pages ... 2020-08-11 23:26:57 +0000 -- done 2020-08-11 23:26:57 +0000 -- Dumping lfs objects ... 2020-08-11 23:26:57 +0000 -- done 2020-08-11 23:26:57 +0000 -- Dumping container registry images ... 2020-08-11 23:26:57 +0000 -- [DISABLED] Creating backup archive: 1597188417_2020_08_11_12.10.5_gitlab_backup.tar ... done Uploading backup archive to remote storage ... skipped Deleting tmp directories ... done done done done done done done done Deleting old backups ... skipping Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data and are not included in this backup. You will need these files to restore a backup. Please back them up manually. Backup task is done. # # cd /var/opt/gitlab/backups # ls 1597188417_2020_08_11_12.10.5_gitlab_backup.tar # du -k *.tar 172 1597188417_2020_08_11_12.10.5_gitlab_backup.tar #
可以看到数据较少，只有170K，而且提示了一个Warning信息，说一些包含敏感数据的文件gitlab.rb和gitlab-secrets.json文件不再备份之中，需要手动备份和恢复。
执行恢复使用restore执行恢复，提示如下错误信息：
[root@host132 backups]# docker exec -it gitlab_gitlab_1 sh # cd /var/opt/gitlab/backups # ls 1597188417_2020_08_11_12.10.5_gitlab_backup.tar # gitlab-rake gitlab:backup:restore BACKUP=1597188417 The backup file 1597188417_gitlab_backup.tar does not exist! #
这是因为后续tar文件名的格式发生了变化，里面加入了GitLab的版本号，所以之前的命令早不到用于恢复的备份文件，修改文件名如下：
# cp 1597188417_2020_08_11_12.10.5_gitlab_backup.tar 1597188417_gitlab_backup.tar #
PS：更为简单的方式是使用如下命令
执行命令：gitlab-rake gitlab:backup:restore BACKUP=1597188417_2020_08_11_12.10.5
在恢复之前需要首先停止服务
# gitlab-ctl stop ok: down: alertmanager: 0s, normally up ok: down: gitaly: 0s, normally up ok: down: gitlab-exporter: 1s, normally up ok: down: gitlab-workhorse: 0s, normally up ok: down: grafana: 1s, normally up ok: down: logrotate: 0s, normally up ok: down: nginx: 1s, normally up ok: down: postgres-exporter: 0s, normally up ok: down: postgresql: 0s, normally up ok: down: prometheus: 1s, normally up ok: down: redis: 0s, normally up ok: down: redis-exporter: 1s, normally up ok: down: sidekiq: 0s, normally up ok: down: sshd: 0s, normally up ok: down: unicorn: 0s, normally up #
导入的时候需要注意权限是否正确否则容易出现如下错误
# gitlab-rake gitlab:backup:restore BACKUP=1597188417 Unpacking backup ... tar: 1597188417_gitlab_backup.tar: Cannot open: Permission denied tar: Error is not recoverable: exiting now unpacking backup failed #
这是因为rsync同步时的文件权限受umask影响的缺省设定，修改即可
# ls -lrt total 344 -rw------- 1 root root 174080 Aug 12 01:23 1597188417_2020_08_11_12.10.5_gitlab_backup.tar -rw------- 1 root root 174080 Aug 12 01:26 1597188417_gitlab_backup.tar # chmod 644 * #
执行备份恢复出现如下错误：
# gitlab-rake gitlab:backup:restore BACKUP=1597188417 Unpacking backup ... done Before restoring the database, we will remove all existing tables to avoid future upgrade problems. Be aware that if you have custom tables in the GitLab database these tables and all data will be removed. Do you want to continue (yes/no)? yes Removing all tables. Press `Ctrl-C` within 5 seconds to abort 2020-08-12 01:28:07 +0000 -- Cleaning the database ... rake aborted! PG::ConnectionBad: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"? /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:27:in `block (3 levels) in ' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:64:in `block (3 levels) in ' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `' Tasks: TOP => gitlab:db:drop_tables (See full trace by running task with --trace) #
提示信息显示无法连接数据库，理论上来说把PostgreSQL启动起来可以继续，这里干脆全部启动
# gitlab-ctl start ok: run: alertmanager: (pid 3009) 1s ok: run: gitaly: (pid 3017) 0s ok: run: gitlab-exporter: (pid 3030) 1s ok: run: gitlab-workhorse: (pid 3033) 0s ok: run: grafana: (pid 3042) 1s ok: run: logrotate: (pid 3050) 0s ok: run: nginx: (pid 3056) 0s ok: run: postgres-exporter: (pid 3062) 1s ok: run: postgresql: (pid 3078) 0s ok: run: prometheus: (pid 3088) 1s ok: run: redis: (pid 3096) 0s ok: run: redis-exporter: (pid 3101) 1s ok: run: sidekiq: (pid 3106) 0s ok: run: sshd: (pid 3112) 1s ok: run: unicorn: (pid 3114) 0s # # gitlab-rake gitlab:backup:restore BACKUP=1597188417 Unpacking backup ... done Before restoring the database, we will remove all existing tables to avoid future upgrade problems. Be aware that if you have custom tables in the GitLab database these tables and all data will be removed. Do you want to continue (yes/no)? yes Removing all tables. Press `Ctrl-C` within 5 seconds to abort 2020-08-12 02:27:25 +0000 -- Cleaning the database ... 2020-08-12 02:27:34 +0000 -- done 2020-08-12 02:27:34 +0000 -- Restoring database ... Restoring PostgreSQL database gitlabhq_production ... SET SET SET SET SET set_config ------------ (1 row) SET SET SET SET ERROR: relation "public.u2f_registrations" does not exist ERROR: relation "public.timelogs" does not exist ...省略 ALTER TABLE [DONE] 2020-08-12 02:28:19 +0000 -- done 2020-08-12 02:28:19 +0000 -- Restoring repositories ... * root/testproject ... [DONE] 2020-08-12 02:28:20 +0000 -- done 2020-08-12 02:28:20 +0000 -- Restoring uploads ... 2020-08-12 02:28:20 +0000 -- done 2020-08-12 02:28:20 +0000 -- Restoring builds ... 2020-08-12 02:28:20 +0000 -- done 2020-08-12 02:28:20 +0000 -- Restoring artifacts ... 2020-08-12 02:28:20 +0000 -- done 2020-08-12 02:28:20 +0000 -- Restoring pages ... 2020-08-12 02:28:20 +0000 -- done 2020-08-12 02:28:20 +0000 -- Restoring lfs objects ... 2020-08-12 02:28:20 +0000 -- done This task will now rebuild the authorized_keys file. You will lose any data stored in the authorized_keys file. Do you want to continue (yes/no)? yes Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data and are not included in this backup. You will need to restore these files manually. Restore task is done. #
整个过程中数据库数据导入最开始提示了relation不不存在的错误和需要手工处理gitlab.rb等文件的Warning，其余均显示正常完成。首先在目标备份机上的登录，可以看到已经不再提示需要进行密码修改了，输入备份侧的用户名和密码信息点击登录按钮
可以看到刚刚在host131的GitLab服务上创建的工程项目信息也已经成功恢复了
