redis Docker和西芹-错误：PID文件(celerybeat.pid)已存在

ujv3wf0j 于 2022-11-21 发布在 Redis

关注(0)|答案(7)|浏览(189)

应用程序包括：- Django -雷迪斯-celery -多克-波斯格瑞
在将项目合并到Docker之前，一切都运行得很顺利，但是一旦将其移到容器中，就开始出现错误。一开始它运行得很好，但是过了一段时间，我确实收到了以下错误：

celery-beat_1  | ERROR: Pidfile (celerybeat.pid) already exists.

我已经挣扎了一段时间，但现在我真的给予了。我不知道它有什么问题。
停靠文件：

FROM python:3.7

ENV PYTHONUNBUFFERED 1
RUN mkdir -p /opt/services/djangoapp/src

COPY /scripts/startup/entrypoint.sh entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

COPY Pipfile Pipfile.lock /opt/services/djangoapp/src/
WORKDIR /opt/services/djangoapp/src
RUN pip install pipenv && pipenv install --system

COPY . /opt/services/djangoapp/src

RUN find . -type f -name "celerybeat.pid" -exec rm -f {} \;

RUN sed -i "s|django.core.urlresolvers|django.urls |g" /usr/local/lib/python3.7/site-packages/vanilla/views.py
RUN cp /usr/local/lib/python3.7/site-packages/celery/backends/async.py /usr/local/lib/python3.7/site-packages/celery/backends/asynchronous.py
RUN rm /usr/local/lib/python3.7/site-packages/celery/backends/async.py
RUN sed -i "s|async|asynchronous|g" /usr/local/lib/python3.7/site-packages/celery/backends/redis.py
RUN sed -i "s|async|asynchronous|g" /usr/local/lib/python3.7/site-packages/celery/backends/rpc.py

RUN cd app && python manage.py collectstatic --no-input


EXPOSE 8000
CMD ["gunicorn", "-c", "config/gunicorn/conf.py", "--bind", ":8000", "--chdir", "app", "example.wsgi:application", "--reload"]

docker-compose.yml:

version: '3'

services:

  djangoapp:
    build: .
    volumes:
      - .:/opt/services/djangoapp/src
      - static_volume:/opt/services/djangoapp/static  # <-- bind the static volume
      - media_volume:/opt/services/djangoapp/media  # <-- bind the media volume
      - static_local_volume:/opt/services/djangoapp/src/app/static
      - media_local_volume:/opt/services/djangoapp/src/app/media
      - .:/code
    restart: always
    networks:
      - nginx_network
      - database1_network # comment when testing
      # - test_database1_network # uncomment when testing
      - redis_network
    depends_on:
      - database1 # comment when testing
      # - test_database1 # uncomment when testing
      - migration
      - redis

  # base redis server
  redis:
    image: "redis:alpine"
    restart: always
    ports: 
      - "6379:6379"
    networks:
      - redis_network
    volumes:
      - redis_data:/data

  # celery worker
  celery:
    build: .
    command: >
      bash -c "cd app && celery -A example worker --without-gossip --without-mingle --without-heartbeat -Ofair"
    volumes:
      - .:/opt/services/djangoapp/src
      - static_volume:/opt/services/djangoapp/static  # <-- bind the static volume
      - media_volume:/opt/services/djangoapp/media  # <-- bind the media volume    
      - static_local_volume:/opt/services/djangoapp/src/app/static
      - media_local_volume:/opt/services/djangoapp/src/app/media
    networks:
      - redis_network
      - database1_network # comment when testing
      # - test_database1_network # uncomment when testing
    restart: always
    depends_on:
      - database1 # comment when testing
      # - test_database1 # uncomment when testing
      - redis
    links:
      - redis

  celery-beat:
    build: .
    command: >
      bash -c "cd app && celery -A example beat"
    volumes:
      - .:/opt/services/djangoapp/src
      - static_volume:/opt/services/djangoapp/static  # <-- bind the static volume
      - media_volume:/opt/services/djangoapp/media  # <-- bind the media volume
      - static_local_volume:/opt/services/djangoapp/src/app/static
      - media_local_volume:/opt/services/djangoapp/src/app/media
    networks:
      - redis_network
      - database1_network # comment when testing
      # - test_database1_network # uncomment when testing
    restart: always
    depends_on:
      - database1 # comment when testing
      # - test_database1 # uncomment when testing
      - redis
    links:
      - redis

  # migrations needed for proper db functioning
  migration:
    build: .
    command: >
      bash -c "cd app && python3 manage.py makemigrations && python3 manage.py migrate"
    depends_on:
      - database1 # comment when testing
      # - test_database1 # uncomment when testing
    networks:
     - database1_network # comment when testing
     # - test_database1_network # uncomment when testing

  # reverse proxy container (nginx)
  nginx:
    image: nginx:1.13
    ports:
      - 80:80
    volumes:
      - ./config/nginx/conf.d:/etc/nginx/conf.d
      - static_volume:/opt/services/djangoapp/static  # <-- bind the static volume
      - media_volume:/opt/services/djangoapp/media  # <-- bind the media volume
      - static_local_volume:/opt/services/djangoapp/src/app/static
      - media_local_volume:/opt/services/djangoapp/src/app/media 
    restart: always
    depends_on:
      - djangoapp
    networks:
      - nginx_network

  database1: # comment when testing
    image: postgres:10 # comment when testing
    env_file: # comment when testing
      - config/db/database1_env # comment when testing
    networks: # comment when testing
      - database1_network # comment when testing
    volumes: # comment when testing
      - database1_volume:/var/lib/postgresql/data # comment when testing

  # test_database1: # uncomment when testing
    # image: postgres:10 # uncomment when testing
    # env_file: # uncomment when testing
      # - config/db/test_database1_env # uncomment when testing
    # networks: # uncomment when testing
      # - test_database1_network # uncomment when testing
    # volumes: # uncomment when testing
      # - test_database1_volume:/var/lib/postgresql/data # uncomment when testing

networks:
  nginx_network:
    driver: bridge
  database1_network: # comment when testing
    driver: bridge # comment when testing
  # test_database1_network: # uncomment when testing
    # driver: bridge # uncomment when testing
  redis_network:
    driver: bridge
volumes:
  database1_volume: # comment when testing
  # test_database1_volume: # uncomment when testing
  static_volume:  # <-- declare the static volume
  media_volume:  # <-- declare the media volume
  static_local_volume:
  media_local_volume:
  redis_data:

请忽略“test_database1_volume”，因为它仅用于测试目的。

redis

来源：https://stackoverflow.com/questions/53521959/dockercelery-error-pidfile-celerybeat-pid-already-exists

7条答案

按热度按时间

mfpqipee1#

另一个解决方案（取自https://stackoverflow.com/a/17674248/39296）是使用--pidfile=（没有路径）来完全不创建pidfile。

赞(0）回复(0）举报 2022-11-21

polkgigr2#

我相信在你的项目目录 ./ 中有一个 pid 文件，当你运行容器时，它被挂载在。（因此 RUN find . -type f -name "celerybeat.pid" -exec rm -f {} \; 没有任何效果）。
您可以使用 celery --pidfile=/opt/celeryd.pid 指定未装载的路径，使其不在主机上镜像。

赞(0）回复(0）举报 2022-11-21

yhived7q3#

虽然不是专业的丝毫，我发现补充说：
celerybeat.pid
到我的.dockerignore文件是什么修复所说的问题。

赞(0）回复(0）举报 2022-11-21

wtlkbnrh4#

我有这个错误与气流时，我运行它与docker-compose。
如果您不关心气流的当前状态，则可以删除气流容器。
docker rm containerId
然后，再次启动Airflow：
docker-compose up

赞(0）回复(0）举报 2022-11-21

egdjgwm85#

另一种方法是创建一个django命令celery_kill.py

import shlex
import subprocess

from django.core.management.base import BaseCommand

class Command(BaseCommand):
    def handle(self, *args, **options):
        kill_worker_cmd = 'pkill -9 celery'
        subprocess.call(shlex.split(kill_worker_cmd))

docker-compose.yml：我是一个很好的编辑器。

celery:
    build: ./src
    restart: always
    command: celery -A project worker -l info
    volumes:
      - ./src:/var/lib/celery/data/
    depends_on:
      - db
      - redis
      - app

  celery-beat:
    build: ./src
    restart: always
    command: celery -A project beat -l info --pidfile=/tmp/celeryd.pid
    volumes:
      - ./src:/var/lib/beat/data/
    depends_on:
      - db
      - redis
      - app

和Makefile：

run:
    docker-compose up -d --force-recreate
    docker-compose exec app python manage.py celery_kill
    docker-compose restart
    docker-compose exec app python manage.py migrate

赞(0）回复(0）举报 2022-11-21

omqzjyyz6#

此错误的原因是Docker容器在没有正常Celery停止过程的情况下停止。解决方法很简单。在启动之前停止Celery。

**解决方案1.**编写celery启动命令（例如〉docker-entrypoint.sh，...），如下所示

celery multi stopwait w1 -A myproject
&& rm -f /var/run/celery/w1.pid  # remove stale pidfile
&& celery multi start w1 -A myproject-l info --pidfile=/var/run/celery/w1.pid

解决方案2.（不推荐）

始终在“向上扩展”之前运行“向下扩展”。

赞(0）回复(0）举报 2022-11-21

cngwdvgl7#

在 Docker Compose 上，气流设置（ apache-airflow==2.3.4 、 celery==5.2.7 ）的一部分出现相同问题：

ERROR: Pidfile (/airflow/airflow-worker.pid) already exists.
Seems we're already running? (pid: 1)

中的每一个
我试着这样传递 --pidfile （实际上是在 Airflow 伞下的 --pid ）：

airflow celery worker --pid=

格式
然而，这并不起作用，一个 .pid 文件仍在创建中。也许这是由于额外的气流层。
最后，我发现原来的问题与 Docker Compose 重启策略有关（在我的例子中，是 restart: always ）。一旦 worker 失败了一次，随后的重启将找到已经存在的 .pid 文件。这是因为容器在重启时保持状态（参见 this 或 this ）。
一个更持久的解决方案是使用 tmpfs ，并将 .pid 文件指向那里：

# docker-compose.yml

worker:
    image: {...}
    tmpfs:
      - /airflow-worker
    entrypoint: airflow celery worker --pid=/airflow-worker/airflow-worker.pid
    ...
...

格式

赞(0）回复(0）举报 2022-11-21

我来回答

redis Docker和西芹-错误：PID文件(celerybeat.pid)已存在

7条答案

相关问题

热门标签

最新问答