redis 如何在gitlab中使用对象存储，并出现此错误

pkbketx9  于 2023-04-19  发布在  Redis
关注(0)|答案(1)|浏览(271)
==> /var/log/gitlab/gitlab-workhorse/current <==
{"error":"keywatcher: pubsub receive: EOF","level":"error","msg":"","time":"2023-04-07T21:31:17Z"}
{"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2023-04-07T21:31:17Z"}

这是一个错误，我看到在输出时，我试图上传图片到一个仓库，我管理。
当我检查日志文件时，我看到这样的东西：

{"error":"keywatcher: pubsub receive: EOF","level":"error","msg":"","time":"2023-04-07T19:46:33Z"}
{"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2023-04-07T19:46:33Z"}
{"content_type":"text/html; charset=utf-8","correlation_id":"01GXEM4TJHF4R081TXWG1QP8ET","duration_ms":89,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":200,"system":"http","time":"2023-04-07T19:47:26Z","ttfb_ms":88,"uri":"/gitlab/help","user_agent":"curl/8.0.1-DEV","written_bytes":71928}
{"error":"keywatcher: pubsub receive: EOF","level":"error","msg":"","time":"2023-04-07T19:47:34Z"}
{"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2023-04-07T19:47:34Z"}
{"content_type":"text/html; charset=utf-8","correlation_id":"01GXEM6P4P3NFG6KK3JQ9P7ARA","duration_ms":87,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":200,"system":"http","time":"2023-04-07T19:48:27Z","ttfb_ms":87,"uri":"/gitlab/help","user_agent":"curl/8.0.1-DEV","written_bytes":71928}
{"error":"keywatcher: pubsub receive: EOF","level":"error","msg":"","time":"2023-04-07T19:48:35Z"}
{"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2023-04-07T19:48:35Z"}
{"content_type":"text/html; charset=utf-8","correlation_id":"01GXEM8HQQDSA37NH34Y87Z35W","duration_ms":121,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":200,"system":"http","time":"2023-04-07T19:49:28Z","ttfb_ms":121,"uri":"/gitlab/help","user_agent":"curl/8.0.1-DEV","written_bytes":71928}
{"error":"keywatcher: pubsub receive: EOF","level":"error","msg":"","time":"2023-04-07T19:49:36Z"}
{"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2023-04-07T19:49:36Z"}
{"content_type":"text/html; charset=utf-8","correlation_id":"01GXEMAD9BZXVRTQPVD78PW35Q","duration_ms":91,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":200,"system":"http","time":"2023-04-07T19:50:29Z","ttfb_ms":91,"uri":"/gitlab/help","user_agent":"curl/8.0.1-DEV","written_bytes":71928}
{"error":"keywatcher: pubsub receive: EOF","level":"error","msg":"","time":"2023-04-07T19:50:37Z"}
{"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2023-04-07T19:50:37Z"}
{"content_type":"text/html; charset=utf-8","correlation_id":"01GXEMC8VGM92RGMDCMP3VTYP3","duration_ms":88,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":200,"system":"http","time":"2023-04-07T19:51:30Z","ttfb_ms":88,"uri":"/gitlab/help","user_agent":"curl/8.0.1-DEV","written_bytes":71928}
{"error":"keywatcher: pubsub receive: EOF","level":"error","msg":"","time":"2023-04-07T19:51:38Z"}
{"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2023-04-07T19:51:38Z"}
{"content_type":"text/html; charset=utf-8","correlation_id":"01GXEME3NJT1DQHJ1J893DWSG7","duration_ms":87,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":200,"system":"http","time":"2023-04-07T19:52:30Z","ttfb_ms":87,"uri":"/gitlab/help","user_agent":"curl/8.0.1-DEV","written_bytes":71928}
{"error":"keywatcher: pubsub receive: EOF","level":"error","msg":"","time":"2023-04-07T19:52:39Z"}
{"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2023-04-07T19:52:39Z"}
{"content_type":"text/html; charset=utf-8","correlation_id":"01GXEMFZ50G99BNZHP08P46SZG","duration_ms":354,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":200,"system":"http","time":"2023-04-07T19:53:31Z","ttfb_ms":353,"uri":"/gitlab/help","user_agent":"curl/8.0.1-DEV","written_bytes":71928}
{"error":"keywatcher: pubsub receive: EOF","level":"error","msg":"","time":"2023-04-07T19:53:40Z"}
{"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2023-04-07T19:53:40Z"}
{"level":"info","msg":"shutdown initiated","shutdown_timeout_s":0,"signal":"terminated","time":"2023-04-07T19:54:04Z"}
{"level":"info","msg":"keywatcher: shutting down","time":"2023-04-07T19:54:04Z"}
{"error":"context deadline exceeded","level":"fatal","msg":"shutting down","time":"2023-04-07T19:54:04Z"}
{"build_time":"20230330.174908","level":"info","msg":"Starting","time":"2023-04-07T19:54:04Z","version":"v15.8.5-ee"}
{"address":"localhost:9229","level":"info","msg":"Running metrics server","network":"tcp","time":"2023-04-07T19:54:04Z"}
{"level":"info","msg":"keywatcher: starting process loop","time":"2023-04-07T19:54:04Z"}
{"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2023-04-07T19:54:04Z"}
{"correlation_id":"","duration_ms":0,"error":"badgateway: failed to receive response: dial unix /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: connect: no such file or directory","level":"error","method":"GET","msg":"","time":"2023-04-07T19:54:04Z","uri":""}
{"address":"/var/opt/gitlab/gitlab-workhorse/sockets/socket","level":"info","msg":"Running upstream server","network":"unix","time":"2023-04-07T19:54:04Z"}
{"correlation_id":"","duration_ms":0,"error":"badgateway: failed to receive response: dial unix /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: connect: no such file or directory","level":"error","method":"GET","msg":"","time":"2023-04-07T19:54:14Z","uri":""}
{"correlation_id":"","duration_ms":0,"error":"badgateway: failed to receive response: dial unix /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: connect: no such file or directory","level":"error","method":"GET","msg":"","time":"2023-04-07T19:54:24Z","uri":""}
{"correlation_id":"01GXEMHTNAJ16JK0SWENQ5CXFA","duration_ms":0,"error":"badgateway: failed to receive response: dial unix /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: connect: no such file or directory","level":"error","method":"GET","msg":"","time":"2023-04-07T19:54:32Z","uri":"/gitlab/help"}
{"content_type":"text/html; charset=utf-8","correlation_id":"01GXEMHTNAJ16JK0SWENQ5CXFA","duration_ms":0,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":502,"system":"http","time":"2023-04-07T19:54:32Z","ttfb_ms":0,"uri":"/gitlab/help","user_agent":"curl/8.0.1-DEV","written_bytes":3039}
{"correlation_id":"","duration_ms":0,"error":"badgateway: failed to receive response: dial unix /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: connect: no such file or directory","level":"error","method":"GET","msg":"","time":"2023-04-07T19:54:34Z","uri":""}
{"error":"keywatcher: pubsub receive: EOF","level":"error","msg":"","time":"2023-04-07T19:55:05Z"}
{"address":"/var/opt/gitlab/redis/redis.socket","level":"info","msg":"redis: dialing","network":"unix","time":"2023-04-07T19:55:05Z"}
{"content_type":"text/html; charset=utf-8","correlation_id":"01GXEMKP599T40Z0YJ7R3JW6HP","duration_ms":479,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":200,"system":"http","time":"2023-04-07T19:55:33Z","ttfb_ms":479,"uri":"/gitlab/help","user_agent":"curl/8.0.1-DEV","written_bytes":71928}
{"error":"keywatcher: pubsub receive: EOF","level":"error","msg":"","time":"2023-04-07T19:56:06Z"}

这是我最感兴趣的/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: connect: no such file or directory
所以，我检查：

# cd /var/opt/gitlab/gitlab-rails/sockets/
# ls -la
total 8
drwxr-x--- 2 git gitlab-www 4096 Apr  7 21:29 .
drwxr-xr-x 9 git root       4096 Apr  7 21:03 ..
srwxrwxrwx 1 git git           0 Apr  7 21:29 gitlab.socket

关于chat gpt：
“srwxrwxrwx”中的“s”表示可以出现在Unix Domain Socket文件的权限字符串中的Unix文件权限位。“s”表示该文件设置了setuid位，这意味着该文件将以其所有者或组的权限执行，而不是以执行该文件的用户的权限执行。在Unix Domain Socket文件的情况下，setuid位可用于确保只有特定组的所有者或成员可以与套接字通信。
所以...我猜只有git可以使用这个套接字文件？
我完全不知道我能在这里做什么，因为我在容器中运行gitlab-所有这些都在容器中检查。
配置：

gitlab_rails['object_store']['enabled'] = true
gitlab_rails['object_store']['connection'] = {
  'provider' => 'AWS',
  'region' => 'us-east-1',
  'aws_access_key_id' => 'CKKJJJJE88JWM',
  'aws_secret_access_key' => 'IVchgkjhgkjhgfffffffffff6DA+56',
  'enable_signature_v4_streaming' => 'true',
}
gitlab_rails['object_store']['storage_options'] = {
  'server_side_encryption' => 'aws:kms',
  'server_side_encryption_kms_key_id' => 'arn:aws:kms:us-east-1:13777777:key/4602-d6d35'
}
gitlab_rails['object_store']['proxy_download'] = false
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = 'my-bucket/artifacts'
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = 'my-bucket/external_diffs'
gitlab_rails['object_store']['objects']['lfs']['bucket'] = 'my-bucket/lfs'
gitlab_rails['object_store']['objects']['uploads']['bucket'] = 'my-bucket/uploads'
gitlab_rails['object_store']['objects']['packages']['bucket'] = 'my-bucket/packages'
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = 'my-bucket/dependency_proxy'
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = 'my-bucket/terraform_state'
gitlab_rails['object_store']['objects']['ci_secure_files']['bucket'] = 'my-bucket/ci_secure_files'

如何调试这个程序？谢谢帮助。

编辑：

不知道这是否相关，但我看了一下Redis日志：

2023-04-07_17:14:20.48359 511:C 07 Apr 2023 17:14:20.483 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2023-04-07_17:14:20.48367 511:C 07 Apr 2023 17:14:20.483 # Redis version=6.2.11, bits=64, commit=720ea82e, modified=1, pid=511, just started
2023-04-07_17:14:20.48367 511:C 07 Apr 2023 17:14:20.483 # Configuration loaded
2023-04-07_17:14:20.48391 511:M 07 Apr 2023 17:14:20.483 * monotonic clock: POSIX clock_gettime
2023-04-07_17:14:20.48424                 _._
2023-04-07_17:14:20.48425            _.-``__ ''-._
2023-04-07_17:14:20.48425       _.-``    `.  `_.  ''-._           Redis 6.2.11 (720ea82e/1) 64 bit
2023-04-07_17:14:20.48425   .-`` .-```.  ```\/    _.,_ ''-._
2023-04-07_17:14:20.48425  (    '      ,       .-`  | `,    )     Running in standalone mode
2023-04-07_17:14:20.48426  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 0
2023-04-07_17:14:20.48426  |    `-._   `._    /     _.-'    |     PID: 511
2023-04-07_17:14:20.48426   `-._    `-._  `-./  _.-'    _.-'
2023-04-07_17:14:20.48426  |`-._`-._    `-.__.-'    _.-'_.-'|
2023-04-07_17:14:20.48427  |    `-._`-._        _.-'_.-'    |           https://redis.io
2023-04-07_17:14:20.48427   `-._    `-._`-.__.-'_.-'    _.-'
2023-04-07_17:14:20.48427  |`-._`-._    `-.__.-'    _.-'_.-'|
2023-04-07_17:14:20.48427  |    `-._`-._        _.-'_.-'    |
2023-04-07_17:14:20.48427   `-._    `-._`-.__.-'_.-'    _.-'
2023-04-07_17:14:20.48428       `-._    `-.__.-'    _.-'
2023-04-07_17:14:20.48428           `-._        _.-'
2023-04-07_17:14:20.48428               `-.__.-'
2023-04-07_17:14:20.48429
2023-04-07_17:14:20.48429 511:M 07 Apr 2023 17:14:20.484 # Server initialized
2023-04-07_17:14:20.48429 511:M 07 Apr 2023 17:14:20.484 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
2023-04-07_17:14:20.48446 511:M 07 Apr 2023 17:14:20.484 * The server is now ready to accept connections at /var/opt/gitlab/redis/redis.socket
2023-04-07_17:19:21.07981 511:M 07 Apr 2023 17:19:21.079 * 10 changes in 300 seconds. Saving...
2023-04-07_17:19:21.08012 511:M 07 Apr 2023 17:19:21.080 * Background saving started by pid 1388
2023-04-07_17:19:21.08684 1388:C 07 Apr 2023 17:19:21.086 * DB saved on disk
2023-04-07_17:19:21.08709 1388:C 07 Apr 2023 17:19:21.087 * RDB: 1 MB of memory used by copy-on-write
2023-04-07_17:19:21.18093 511:M 07 Apr 2023 17:19:21.180 * Background saving terminated with success
2023-04-07_17:24:22.05724 511:M 07 Apr 2023 17:24:22.057 * 10 changes in 300 seconds. Saving...
2023-04-07_17:24:22.05759 511:M 07 Apr 2023 17:24:22.057 * Background saving started by pid 1493

**编辑2：**尝试使用社区版：

==> /var/log/gitlab/gitlab-workhorse/current <==
{"correlation_id":"01GXH4E8F4WEAB8ZHYR8JDW5E8","error":"blob.GetBlob: copy rpc data: rpc error: code = Canceled desc = context canceled","level":"error","method":"GET","msg":"","time":"2023-04-08T19:10:41Z","uri":"/gitlab/gitlab-instance-49242dc9/gitlab_object_store_test/-/raw/main/musk2.jpg"}
{"content_type":"image/jpeg","correlation_id":"01GXH4E8F4WEAB8ZHYR8JDW5E8","duration_ms":68,"host":"10.235.77.28","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"http://10.235.77.28/gitlab/gitlab-instance-49242dc9/gitlab_object_store_test/-/blob/main/musk2.jpg","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":200,"system":"http","time":"2023-04-08T19:10:41Z","ttfb_ms":63,"uri":"/gitlab/gitlab-instance-49242dc9/gitlab_object_store_test/-/raw/main/musk2.jpg","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36","written_bytes":401518}

"blob.GetBlob: copy rpc data: rpc error: code = Canceled desc = context canceled"
错误消息“blob.GetBlob：复制rpc数据：rpc error：code = Canceled desc = context canceled”通常表示远程过程调用（RPC）由于超时或网络或系统的其他问题而被取消。
在GitLab的上下文中，当GitLab使用的反向代理服务器GitLab Workhorse由于Git服务器或网络连接超时或其他问题而无法完成对Git存储库的请求时，可能会发生此错误。如果Git服务器过载，如果GitLab服务器和Git服务器之间的网络连接存在问题，或者GitLab服务器本身是否存在其他问题。
当我访问http://10.235.77.28/gitlab/gitlab-instance-49242dc9/gitlab_object_store_test/-/blob/main/musk2.jpg时，我可以在我自己托管的gitlab网站上看到图片...
关于chatgpt：
要查看套接字文件的内容，您可以使用socat - UNIX-CONNECT：/path/to/socket这样的命令来建立到套接字的连接并向其读取或写入数据。但是，您看到的内容将取决于通过套接字通信的进程所使用的协议。
当我尝试：
socat - UNIX-CONNECT:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket
我得到：
HTTP/1.1 400 Bad Request
当我尝试：
socat - UNIX-CONNECT:/var/opt/gitlab/redis/redis.socket
它只是坐在那里（所以，我猜它工作吗？）
编辑：
我很确定这就是问题所在

{"severity":"ERROR","time":"2023-04-11T01:25:09.597Z","correlation_id":null,"exception.class":"Redis::CommandError","exception.message":"MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.","exception.backtrace":["lib/gitlab/instrumentation/redis_interceptor.rb:10:in `block in call'","lib/gitlab/instrumentation/redis_interceptor.rb:41:in `instrument_call'","lib/gitlab/instrumentation/redis_interceptor.rb:9:in `call'"],"user.username":null,"tags.program":"sidekiq","tags.locale":"en","tags.feature_category":null,"tags.correlation_id":null,"extra.storage":"queues"}
# df /var/opt/gitlab/redis
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/nvme0n1p1  16275080 12071560   3438732  78% /var/opt/gitlab
redis

1条答案

按热度按时间
kokeuurv

kokeuurv1#

我的测试是错误的。
如果你上传一张图片到Git仓库，它仍然会存储在仓库本身，而不是你配置的任何对象存储桶中。
Git仓库中的文件存储在GitLab服务器上的/var/opt/gitlab/git-data/repositories/目录中，不要与配置中设置的'uploads'对象混淆。
对象存储上传意味着：上传到issues、合并请求和其他GitLab对象的文件（不是Git存储库中的文件）

赞(0）分享  回复(0）举报  2023-04-19
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com