在heroku中部署Airflow命令时，由于www.example.com中的typeerror而失败utils.py

0x6upsns 于 2022-11-30 发布在其他

关注(0)|答案(1)|浏览(157)

我在python 3.10上运行的是airflow 2.4.1版，在docker中本地运行airflow时没有问题，但是当我使用他们的容器注册表将其部署到heroku时，airflow webserver命令失败，并显示以下引用通告：

____________       _____________
 ____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:55811
Timeout: 120
Logfiles: - -
Access Logformat: 
=================================================================
[2022-11-10 00:19:39 +0000] [43] [INFO] Starting gunicorn 20.1.0
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/site-packages/gunicorn/__main__.py", line 7, in <module>
    run()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 67, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/app/base.py", line 231, in run
    super().run()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/app/base.py", line 72, in run
    Arbiter(self).run()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/arbiter.py", line 198, in run
    self.start()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/arbiter.py", line 138, in start
    self.cfg.on_starting(self)
  File "/usr/local/lib/python3.10/site-packages/airflow/www/gunicorn_config.py", line 40, in on_starting
    ProvidersManager().connection_form_widgets
  File "/usr/local/lib/python3.10/site-packages/airflow/providers_manager.py", line 908, in connection_form_widgets
    self.initialize_providers_hooks()
  File "/usr/local/lib/python3.10/site-packages/airflow/providers_manager.py", line 300, in wrapped_function
    func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/airflow/providers_manager.py", line 386, in initialize_providers_hooks
    self.initialize_providers_list()
  File "/usr/local/lib/python3.10/site-packages/airflow/providers_manager.py", line 300, in wrapped_function
    func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/airflow/providers_manager.py", line 363, in initialize_providers_list
    self._discover_all_providers_from_packages()
  File "/usr/local/lib/python3.10/site-packages/airflow/providers_manager.py", line 429, in _discover_all_providers_from_packages
    for entry_point, dist in entry_points_with_dist('apache_airflow_provider'):
  File "/usr/local/lib/python3.10/site-packages/airflow/utils/entry_points.py", line 40, in entry_points_with_dist
    key = canonicalize_name(dist.metadata["Name"])
  File "/usr/local/lib/python3.10/site-packages/packaging/utils.py", line 34, in canonicalize_name
    value = _canonicalize_regex.sub("-", name).lower()
TypeError: expected string or bytes-like object
[2022-11-10 00:21:38,935] {webserver_command.py:217} ERROR - No response from gunicorn master within 120 seconds
[2022-11-10 00:21:38,935] {webserver_command.py:218} ERROR - Shutting down webserver

以前有人遇到过这个错误吗？如果需要更多信息，请告诉我。

heroku

来源：https://stackoverflow.com/questions/74382837/airflow-commands-fail-due-to-typeerror-in-utils-py-when-deployed-in-heroku

1条答案

按热度按时间

ig9co6j11#

我刚刚解决了这个问题，在我的情况下，我有包，其中没有METADATA字段的NAME。
首先，您可以检查它的计数None：

import importlib_metadata as metadata
import pandas as pd
s = pd.Series([dist.metadata["Name"] for dist in metadata.distributions()])
s.isnull().sum()

之后，您需要找到导致问题的软件包，然后卸载并安装它。
我使用vscode调试器和justMycode: false解决了这个问题。
我正在调试下一个代码：

for met in metadata.distributions():
    print(met.name)

这里的要点是在下一个路径/PATH/lib/PYTHON_VERSION/site-package/importlib_metadata/__init__.py中添加一个if condition和一个断点，例如在我的例子中：

/home/antonio/anaconda3/Aspect-Based-On-Sentiment-Analysis/lib/python3.7/site-package/importlib_metadata/__init__.py

设置if condition和print('now')中的断点（第598行）：

@property
def name(self):
    """Return the 'Name' metadata for the distribution package."""
    if not self.metadata.json:
        print('now')
    return self.metadata['Name']

您可以在调试窗口中看到类似内容，

字段_normalized_name将指示您需要卸载的软件包名称，因为当名称为None时，其余字段将为None
可能是pip uninstall name_package不起作用。
我建议您在/PATH/lib/PYTHON_VERSION/site-package中搜索软件包，检查NAME不在METADATA文件中或METADATA文件不存在并删除整个软件包，最后，您可以使用pip install name-package并检查NAME是否在METADATA文件中
然后，您检查METADATA中没有NAME的软件包数量，运行：

import importlib_metadata as metadata
import pandas as pd
s = pd.Series([dist.metadata["Name"] for dist in metadata.distributions()])
s.isnull().sum()

你会看到它减少了一。

赞(0）回复(0）举报 2022-11-30

我来回答

在heroku中部署Airflow命令时，由于www.example.com中的typeerror而失败utils.py

1条答案

相关问题

热门标签

最新问答