是我的代码还是sidekiq很慢，因为它不应该访问rails模型？

xu3bshqb  于 2021-06-18  发布在  Mysql
关注(0)|答案(2)|浏览(388)

我以前经常把我的很多代码 Package 在 Thread.new {} 然后开始跌跌撞撞地穿过sidekiq，让背景任务变得更容易一些。几周之后，这让人失望，我不知道这是不是我在sidekiq工作人员中添加代码的方式，或者是什么交易。
例如，

[1] pry(#<DomainWorker>)> domain = Domain.find_by(unique_id: @unique_id)
[2] pry(#<DomainWorker>)> urls
=> [{:url=>"http://admin.example.com/"}, {:url=>"http://example.com/"}, {:url=>"http://www.example.com/"}, {:url=>"https://admin.example.com/"}, {:url=>"https://example.com/"}, {:url=>"https://www.example.com/"}]
[3] pry(#<DomainWorker>)> domain.domain_urls.build(urls)
=> [#<DomainUrl:0x0000557c97f2a3c8 id: nil, domain_id: 32, url: "http://admin.example.com/", created_at: nil, updated_at: nil>,
 #<DomainUrl:0x0000557c97cc6e38 id: nil, domain_id: 32, url: "http://example.com/", created_at: nil, updated_at: nil>,
 #<DomainUrl:0x0000557c97cadd70 id: nil, domain_id: 32, url: "http://www.example.com/", created_at: nil, updated_at: nil>,
 #<DomainUrl:0x0000557c97c9c8e0 id: nil, domain_id: 32, url: "https://admin.example.com/", created_at: nil, updated_at: nil>,
 #<DomainUrl:0x0000557c97c7ed18 id: nil, domain_id: 32, url: "https://example.com/", created_at: nil, updated_at: nil>,
 #<DomainUrl:0x0000557c97c43df8 id: nil, domain_id: 32, url: "https://www.example.com/", created_at: nil, updated_at: nil>]

只是简单地运行 domain.domain_urls.build(urls) 命令执行大约需要15秒。如果我敞开心扉 rails c ，此命令立即运行。
这是否违背了从sidekiq工作者那里操纵模型的最佳实践？不知道为什么有时需要15-20秒，下一次需要5秒，等等。这是非常不一致的。我甚至有一个 Postman ，有时需要2秒钟或2分钟来发送电子邮件。不知道是什么交易。
这是我的 config/sidekiq.yml 配置文件：

development:  
  :concurrency: 50
production:  
  :concurrency: 50
:queues:
  - default

是我的代码还是sidekiq不擅长做这些任务？我错过了什么？
编辑
简单地禁用 eager_load 在我的开发环境中似乎取得了一些进展。
在 config/environments/development.rb :
改变 config.eager_load = false 到 config.eager_load = true 它工作了一段时间，但是在一堆任务排队并完成之后，它又开始挂起。现在又卡住了。
如果我跑了 show processlist 在mysql中，我看到了以下内容：

mysql> show full processlist;
+----+------------+-----------+-------------------+---------+------+----------+-----------------------+
| Id | User       | Host      | db                | Command | Time | State    | Info                  |
+----+------------+-----------+-------------------+---------+------+----------+-----------------------+
| 31 | app_dev | localhost | app_portal_dev | Sleep   | 1017 |          | NULL                  |
| 50 | root       | localhost | NULL              | Query   |    0 | starting | show full processlist |
| 55 | app_dev | localhost | app_portal_dev | Sleep   | 1059 |          | NULL                  |
| 57 | app_dev | localhost | app_portal_dev | Sleep   | 1028 |          | NULL                  |
| 58 | app_dev | localhost | app_portal_dev | Sleep   |  420 |          | NULL                  |
| 61 | app_dev | localhost | app_portal_dev | Sleep   | 1078 |          | NULL                  |
| 62 | app_dev | localhost | app_portal_dev | Sleep   |  425 |          | NULL                  |
| 63 | app_dev | localhost | app_portal_dev | Sleep   |  191 |          | NULL                  |
+----+------------+-----------+-------------------+---------+------+----------+-----------------------+
8 rows in set (0.00 sec)

此外，如果我将kill ttin信号发送给sidekiq worker，这就是输出：

2018-11-03T03:20:29.415Z 65977 TID-gsr3safel WARN: Thread TID-gsr4jqd7l processor
2018-11-03T03:20:29.415Z 65977 TID-gsr3safel WARN: /home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `select'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `rescue in _read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:64:in `_read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:56:in `gets'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:363:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:263:in `block in read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:251:in `io'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:262:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `block in call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:232:in `block (2 levels) in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:370:in `ensure_connected'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:222:in `block in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:307:in `logging'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:221:in `process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:210:in `block in call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:281:in `with_socket_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:209:in `call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1172:in `block in _bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `block in synchronize'
/home/myuser/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1169:in `_bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1214:in `brpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `block in retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:95:in `block in redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:65:in `block (2 levels) in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `block in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:92:in `redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:89:in `get_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:99:in `fetch'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:82:in `process_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:71:in `run'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:16:in `watchdog'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:25:in `block in safe_thread'
2018-11-03T03:20:29.416Z 65977 TID-gsr3safel WARN: Thread TID-gsr4jqddx processor
2018-11-03T03:20:29.416Z 65977 TID-gsr3safel WARN: /home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `select'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `rescue in _read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:64:in `_read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:56:in `gets'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:363:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:263:in `block in read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:251:in `io'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:262:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `block in call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:232:in `block (2 levels) in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:370:in `ensure_connected'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:222:in `block in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:307:in `logging'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:221:in `process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:210:in `block in call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:281:in `with_socket_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:209:in `call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1172:in `block in _bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `block in synchronize'
/home/myuser/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1169:in `_bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1214:in `brpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `block in retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:95:in `block in redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:65:in `block (2 levels) in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `block in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:92:in `redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:89:in `get_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:99:in `fetch'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:82:in `process_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:71:in `run'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:16:in `watchdog'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:25:in `block in safe_thread'
2018-11-03T03:20:29.416Z 65977 TID-gsr3safel WARN: Thread TID-gsr4jqdh1 processor
2018-11-03T03:20:29.416Z 65977 TID-gsr3safel WARN: /home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `select'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `rescue in _read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:64:in `_read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:56:in `gets'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:363:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:263:in `block in read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:251:in `io'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:262:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `block in call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:232:in `block (2 levels) in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:370:in `ensure_connected'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:222:in `block in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:307:in `logging'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:221:in `process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:210:in `block in call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:281:in `with_socket_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:209:in `call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1172:in `block in _bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `block in synchronize'
/home/myuser/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1169:in `_bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1214:in `brpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `block in retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:95:in `block in redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:65:in `block (2 levels) in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `block in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:92:in `redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:89:in `get_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:99:in `fetch'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:82:in `process_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:71:in `run'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:16:in `watchdog'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:25:in `block in safe_thread'
2018-11-03T03:20:29.417Z 65977 TID-gsr3safel WARN: Thread TID-gsr4jqcuh processor
2018-11-03T03:20:29.417Z 65977 TID-gsr3safel WARN: /home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `select'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `rescue in _read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:64:in `_read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:56:in `gets'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:363:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:263:in `block in read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:251:in `io'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:262:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `block in call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:232:in `block (2 levels) in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:370:in `ensure_connected'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:222:in `block in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:307:in `logging'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:221:in `process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:210:in `block in call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:281:in `with_socket_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:209:in `call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1172:in `block in _bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `block in synchronize'
/home/myuser/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1169:in `_bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1214:in `brpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `block in retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:95:in `block in redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:65:in `block (2 levels) in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `block in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:92:in `redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:89:in `get_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:99:in `fetch'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:82:in `process_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:71:in `run'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:16:in `watchdog'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:25:in `block in safe_thread'
2018-11-03T03:20:29.417Z 65977 TID-gsr3safel WARN: Thread TID-gsr4jqd4p processor
2018-11-03T03:20:29.417Z 65977 TID-gsr3safel WARN: /home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `select'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `rescue in _read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:64:in `_read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:56:in `gets'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:363:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:263:in `block in read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:251:in `io'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:262:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `block in call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:232:in `block (2 levels) in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:370:in `ensure_connected'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:222:in `block in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:307:in `logging'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:221:in `process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:210:in `block in call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:281:in `with_socket_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:209:in `call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1172:in `block in _bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `block in synchronize'
/home/myuser/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1169:in `_bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1214:in `brpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `block in retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:95:in `block in redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:65:in `block (2 levels) in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `block in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:92:in `redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:89:in `get_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:99:in `fetch'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:82:in `process_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:71:in `run'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:16:in `watchdog'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:25:in `block in safe_thread'
2018-11-03T03:20:29.418Z 65977 TID-gsr3safel WARN: Thread TID-gsr4jqcn5 processor
2018-11-03T03:20:29.418Z 65977 TID-gsr3safel WARN: /home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `select'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:68:in `rescue in _read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:64:in `_read_from_socket'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:56:in `gets'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/connection/ruby.rb:363:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:263:in `block in read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:251:in `io'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:262:in `read'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `block in call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:232:in `block (2 levels) in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:370:in `ensure_connected'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:222:in `block in process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:307:in `logging'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:221:in `process'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:118:in `call'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:210:in `block in call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:281:in `with_socket_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis/client.rb:209:in `call_with_timeout'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1172:in `block in _bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `block in synchronize'
/home/myuser/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:45:in `synchronize'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1169:in `_bpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/redis-4.0.2/lib/redis.rb:1214:in `brpop'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `block in retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:95:in `block in redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:65:in `block (2 levels) in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:64:in `block in with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `handle_interrupt'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/connection_pool-2.2.2/lib/connection_pool.rb:61:in `with'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq.rb:92:in `redis'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/fetch.rb:36:in `retrieve_work'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:89:in `get_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:99:in `fetch'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:82:in `process_one'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/processor.rb:71:in `run'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:16:in `watchdog'
/home/myuser/.rvm/gems/ruby-2.5.1/gems/sidekiq-5.2.2/lib/sidekiq/util.rb:25:in `block in safe_thread'
mysqlruby-on-railsrubysidekiq

2条答案

按热度按时间
twh00eeo

twh00eeo1#

ttin输出或数据库似乎没有问题。sidekiq非常适合使用rails和activerecord。我们运行一个高负载的生产环境，有大约30k个并发线程（分布在几十个工作服务器上），sidekiq的性能非常出色。不要在开发模式中设置不必要的高并发性，尽管如果您正在处理一个并不重要的任务。
尝试在方法的开头调试sidekiq worker本地设置的断点，并找出它变慢的地方。sidekiq本身不太可能做到这一点

赞(0）分享  回复(0）举报  2021-06-18
toiithl6

toiithl62#

rails开发模式自动加载是单线程的。您可以将sidekiq设置为您想要的任何并发性，但您仍将有一个巨大的mutex来单线程处理您的作业，以便它们可以安全地自动加载。打开自动加载或切换到生产模式将禁用自动加载。
并发性：50太高了。不要把它定得高于25。
也有可能你有大量的线程，但是数据库连接的数量却不一样 pool: 25 在 config/database.yml

赞(0）分享  回复(0）举报  2021-06-18
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com