redis Celery未接收到后台运行的任务

gev0vcfq  于 2023-02-03  发布在  Redis
关注(0)|答案(2)|浏览(337)

我在设置celery和我的 flask 应用程序一起工作时遇到了问题。我用一个准系统应用程序测试了配置,发现我的celery worker已经启动,但没有像所有教程中那样执行任何任务。基本上,当您调用. delay时()函数它应该接受你的python函数并把它发送给celery在后台处理,但是事情却挂起了,因为连接无法制作。所以可能是我的配置不正确,或者我下载的某个版本的软件中有一个我不知道的bug。
下面是我的requirements.txt文件的内容:

amqp==5.1.0
anyjson==0.3.3
async-timeout==4.0.2
beautifulsoup4==4.10.0
billiard==3.6.4.0
celery==5.2.3
cffi==1.15.0
click==8.0.4
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
colorama==0.4.4
Deprecated==1.2.13
Flask==2.0.3
Flask-SQLAlchemy==2.5.1
greenlet==1.1.2
itsdangerous==2.1.2
Jinja2==3.1.1
kombu==5.2.4
MarkupSafe==2.1.1
packaging==21.3
prompt-toolkit==3.0.28
pycparser==2.21
pyparsing==3.0.7
pytz==2022.1
redis==4.2.0
six==1.16.0
soupsieve==2.3.1
SQLAlchemy==1.4.32
typing_extensions==4.1.1
vine==5.0.0
wcwidth==0.2.5
Werkzeug==2.0.3
wrapt==1.14.0
yahoofinancials==1.6

这是tasks.py,注意注解掉的行,因为由于某种原因,如果没有指定后端,celery 工作进程就不能正常启动,这也很奇怪。

from celery import Celery
from time import sleep

#app = Celery('tasks', broker='redis://localhost:6379')
app = Celery('tasks', backend='redis://localhost', broker='pyamqp://localhost')

@app.task
def add(x, y):
    return x + y

@app.task
def reverse(myString):
    sleep(5)
    return myString[::-1]

celery 应用程序在虚拟环境中启动正常:

C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\projectFiles>..\Scripts\activate

(testApp) C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\projectFiles>celery -A tasks worker --loglevel=INFO

 -------------- celery@DESKTOP-GHMPTB0 v5.2.3 (dawn-chorus)
--- ***** -----
-- ******* ---- Windows-10-10.0.19043-SP0 2022-03-31 12:07:03
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         tasks:0x24f8cfca1a0
- ** ---------- .> transport:   amqp://guest:**@localhost:5672//
- ** ---------- .> results:     redis://localhost/
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery

[tasks]
  . tasks.add
  . tasks.reverse

[2022-03-31 12:07:03,550: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2022-03-31 12:07:03,565: INFO/MainProcess] mingle: searching for neighbors
[2022-03-31 12:07:04,128: INFO/SpawnPoolWorker-1] child process 240 calling self.run()
[2022-03-31 12:07:04,128: INFO/SpawnPoolWorker-4] child process 13564 calling self.run()
[2022-03-31 12:07:04,128: INFO/SpawnPoolWorker-3] child process 8584 calling self.run()
[2022-03-31 12:07:04,128: INFO/SpawnPoolWorker-2] child process 8344 calling self.run()
[2022-03-31 12:07:04,611: INFO/MainProcess] mingle: all alone
[2022-03-31 12:07:04,642: INFO/MainProcess] celery@DESKTOP-GHMPTB0 ready.

然后发送函数调用到celepie的结果给我一个连接错误,这就是我困惑的部分。

(testApp) C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\projectFiles>python
Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from tasks import *
>>> result = add.delay(2,3)
Traceback (most recent call last):
  File "C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\lib\site-packages\redis\connection.py", line 614, in connect
    sock = self.retry.call_with_retry(
  File "C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\lib\site-packages\redis\retry.py", line 45, in call_with_retry
    return do()
  File "C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\lib\site-packages\redis\connection.py", line 615, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
  File "C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\lib\site-packages\redis\connection.py", line 680, in _connect
    raise err
  File "C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\lib\site-packages\redis\connection.py", line 668, in _connect
    sock.connect(socket_address)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\lib\site-packages\celery\backends\redis.py", line 119, in reconnect_on_error
    yield
  File "C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\lib\site-packages\celery\backends\redis.py", line 169, in _consume_from
    self._pubsub.subscribe(key)
  File "C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\lib\site-packages\redis\client.py", line 1549, in subscribe
    ret_val = self.execute_command("SUBSCRIBE", *new_channels.keys())
  File "C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\lib\site-packages\redis\client.py", line 1390, in execute_command
    self.connection = self.connection_pool.get_connection(
  File "C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\lib\site-packages\redis\connection.py", line 1386, in get_connection
    connection.connect()
  File "C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\lib\site-packages\redis\connection.py", line 620, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 10061 connecting to localhost:6379. No connection could be made because the target machine actively refused it.

为了确认,我正在运行python版本3.10.4,这是一个可接受的celery 版本。

(testApp) C:\Users\Owner\My Drive\Documents\Scripts\virtual_envs\testApp\projectFiles>python --version
Python 3.10.4

有人知道问题出在哪里了吗?如果后台任务不能正常工作,我就无法真正推进我的实际项目。我是celery 的新手,正在尝试解决这个问题,但如果我不能解决这个问题,我愿意更换代理或调度软件。

5vf7fwbs

5vf7fwbs1#

'connection refused',这意味着您试图连接的IP:port没有任何侦听。您应该检查您的Redis示例。
调试提示:你可以删除backend参数并再次测试,如果一切正常,你将确信Redis连接有问题。

gg58donl

gg58donl2#

尝试运行:
celery -任务工作者--日志级别=信息-P单独
或安装eventlet
pip安装事件小程序
结束,尝试运行:
celery -任务工作进程--日志级别=INFO -P事件小程序

相关问题