python 在Bitbucket管道中运行Elastic的问题

嗨，有人能帮我在CI/CD环境中调试Elastic的问题吗？
Elastic在本地运行（用于开发），并在其托管的SaaS平台上运行，用于登台和生产。
我在bitbucket管道中有以下步骤

- step: &unit_tests
    image: python:3.9
    name: Testing
    caches:
      - pip
    script:
      - pip install -r requirements.txt
      - export GC_API_DB_PORT=5432
      - export CS_DEPLOYMENT=development
      - docker run -d -p 9200:9200 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:8.6.2
      - python manage.py search_index --rebuild -f
      - python -m pytest --cov-report term --cov=gc_api --cov=gc_api_auth --cov-fail-under=85
    services:
      - postgres
      - docker

但是，我似乎无法连接到弹性示例（我也尝试过一种配置，其中弹性单独运行）

elasticsearch:
  image: docker.elastic.co/elasticsearch/elasticsearch:8.6.2
  container_name: elasticsearch
  cap_add:
    - IPC_LOCK
  volumes:
    - elasticsearch-data:/usr/share/elasticsearch/data
  ports:
    - 9200:9200
    - 9300:9300
....
    - step: &unit_tests
        image: python:3.9
        name: Testing
        caches:
          - pip
        script:
          - pip install -r requirements.txt
          - export GC_API_DB_PORT=5432
          - export ELASTIC_SEARCH_HOST=$BITBUCKET_DOCKER_HOST_INTERNAL
          - python -m pytest --cov-report term --cov=gc_api --cov=gc_api_auth --cov-fail-under=85
        services:
          - postgres
          - elasticsearch

这是我的错误

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.9/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.9/http/client.py", line 1377, in getresponse
    response.begin()
  File "/usr/local/lib/python3.9/http/client.py", line 320, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.9/http/client.py", line 289, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/connection/http_requests.py", line 166, in perform_request
    response = self.session.send(prepared_request, **send_kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/opt/atlassian/pipelines/agent/build/manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.9/site-packages/django_elasticsearch_dsl/management/commands/search_index.py", line 301, in handle
    for index in self.es_conn.indices.get_alias().values():
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/client/utils.py", line 347, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/client/indices.py", line 642, in get_alias
    return self.transport.perform_request(
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/transport.py", line 417, in perform_request
    self._do_verify_elasticsearch(headers=headers, timeout=timeout)
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/transport.py", line 606, in _do_verify_elasticsearch
    raise error
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/transport.py", line 569, in _do_verify_elasticsearch
    _, info_headers, info_response = conn.perform_request(
  File "/usr/local/lib/python3.9/site-packages/elasticsearch/connection/http_requests.py", line 194, in perform_request
    raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))) caused by: ConnectionError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')))

如果你有任何意见，我将不胜感激

如果你手动启动和分离docker容器，考虑添加一个sleep，以等待容器中的进程实际接受连接

docker run -d -p 9200:9200 docker.elastic.co/elasticsearch/elasticsearch
sleep 10 ?
python -m pytest

如果你选择服务，请注意你正在bitbucket-pipelines. yml中粘贴一个docker-compose服务片段。虽然service部分有一些相似之处，但它是一个完全不同的应用程序，并且该定义无效。检查https://support.atlassian.com/bitbucket-cloud/docs/databases-and-service-containers/，但所有这些ports，valumes和cap_add属性充其量都是忽略。如果Bitbcucket Pipelines只是由于yml无效而失败，可能会更好，因为这是反复出现的混淆。
现在，Bitbucket Pipelines服务将与运行脚本的代理共享网络。我不知道您希望“BITBUCKET_DOCKER_HOST_INTERNAL”变量eval的值是什么，但它不存在，请检查https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/。如果它是用户定义的变量，请注意，使用该前缀是非常规的，并且令人困惑，通常是为具有文档变量的现有应用程序保留的。
无论如何，您应该简单地使用localhost。

definitions:
  services:
    elasticsearch:
      image: docker.elastic.co/elasticsearch/elasticsearch
      variables:
        discovery.type: single-node
pipelines:
  default:
    - step:
      services: [elasticsearch]
      script:
        - pip install -r requirements.txt
        - export ELASTIC_SEARCH_HOST=localhost
        - python -m pytest

pip install指令应该足够慢，以便在测试套件开始运行时elasticsearch服务已经准备就绪，因此这里不需要sleep指令。
还有另一种使用测试容器https://testcontainers-python.readthedocs.io/en/latest/elasticsearch/的方法。AFAIU库已经很小心地阻止执行，直到服务真正就绪。
缺点是

你应该把它作为一个session fixture添加到你的pytest配置中
这增加了显著的设置开销，因为测试容器不能在后续执行中重用（watch https://github.com/testcontainers/testcontainers-python/issues/109）

python 在Bitbucket管道中运行Elastic的问题

1条答案

相关问题

热门标签

最新问答