从当前项目向scrapyd添加新服务

7xzttuei  于 2023-06-06  发布在  其他
关注(0)|答案(1)|浏览(223)

我想创建一个自定义服务scrapyd,从scrapy项目,但我不断得到错误。
我创建了crawler/webservice.py

from scrapyd.webservice import WsResource

class BackInTime(WsResource):
    def render_GET(self, txtrequest):
        return {
            'id': 'something works!'
        }

然后修改了scrapyd.conf,在[services]标签下添加了backintime.json = crawler.webservice.BackInTime行。
但是当我运行scrapyd命令时,我总是得到错误:

...
  File "c:\personal\virtualenvs\crawlie\lib\site-packages\scrapyd\app.py", line 39, in application
    webservice = TCPServer(http_port, server.Site(webcls(config, app)), interface=bind_address)
  File "c:\personal\virtualenvs\crawlie\lib\site-packages\scrapyd\website.py", line 33, in __init__
    servCls = load_object(servClsName)
  File "c:\personal\virtualenvs\crawlie\lib\site-packages\scrapy\utils\misc.py", line 46, in load_object
    mod = import_module(module)
  File "C:\Users\user\Anaconda3\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import

  File "<frozen importlib._bootstrap>", line 971, in _find_and_load

  File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

  File "<frozen importlib._bootstrap>", line 994, in _gcd_import

  File "<frozen importlib._bootstrap>", line 971, in _find_and_load

  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked

builtins.ModuleNotFoundError: No module named 'crawler'

Failed to load application: No module named 'crawler'

尝试遵循以前的一些解决方案,但没有任何成功:

我应该如何添加新服务以便scrapyd可以导入它?

jogvjijk

jogvjijk1#

尝试将应用的导入模块添加到环境变量PYTHONPATH。在我的Dockerfile中,它看起来像这样:

ENV PYTHONPATH "${PYTHONPATH}:${APPDIR}/${APPNAME}"

在您的本地计算机上,它可能看起来像:

export PYTHONPATH="${PYTHONPATH}:/home/app/crawler"

然后运行Scrapyd。

相关问题