使用docker-compose启动Jupyter notebook时激活Docker容器内的conda env

mspsb9vt  于 2022-11-22  发布在  Docker
关注(0)|答案(2)|浏览(282)

我有下面的Dockerfile

FROM continuumio/miniconda3:4.5.11

# create a new user (defaults to 'al-khawarizmi')
USER root
ARG username=al-khawarizmi
RUN useradd --create-home --home-dir /home/${username} ${username}
ENV HOME /home/${username}

# switch to newly created user to avoid running container as root
USER ${username}
WORKDIR $HOME

# build and activate the specified conda environment from a file (defaults to 'environment.yml')
ARG environment=environment.yml
COPY ${environment} .
RUN conda env create --file ${environment} && \
    echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \ 
    echo "conda activate $(head -1 ${environment} | cut -d' ' -f2)" >> ~/.bashrc

Dockerfile允许用户指定一个conda环境文件作为构建参数。下面是一个典型的environment.yml file

name: nessie-py

channels:
  - conda-forge
  - defaults

dependencies:
  - python=3.6
  - "notebook=5.7.*"
  - "matplotlib=3.0.*"
  - "numpy=1.15.*"
  - "pandas=0.23.*"

用户可以以标准方式运行映像,conda环境将自动激活。运行

$ docker run -it image_name:image_tag

在Docker容器中生成一个bash提示符,并激活conda环境。

(environment_name)$

现在,我想使用docker-compose在容器中启动Jupyter笔记本服务器(使用conda环境文件构建,该文件将Jupyter指定为依赖项)。
当我使用下面的docker-compose.yml

version: "3.7"

services:
  notebook-server:
    build:
      context: ./
    ports:
      - "8888:8888"
    volumes:
      - ./:/home/al-khawarizmi
    command: jupyter notebook --no-browser ip=0.0.0.0

出现以下错误。

$ docker-compose up
Creating network "nessie-py_default" with the default driver
Creating nessie-py_notebook-server_1 ... done
Attaching to nessie-py_notebook-server_1
notebook-server_1  | [FATAL tini (7)] exec jupyter failed: No such file or directory
nessie-py_notebook-server_1 exited with code 127

我怀疑这个错误意味着conda环境没有激活。然后我尝试将tty: truestdin_open: true添加到docker-compose.yml中,认为这应该在运行command之前调用交互式bash提示符。这导致了与上面相同的错误。
我还尝试定义一个start-notebook.sh脚本,在运行笔记本之前显式激活conda环境。

#!/bin/bash
set -e

# activate the environment and start the notebook
conda activate nessie-py
jupyter notebook --no-browser ip=0.0.0.0

导致不同的错误

$ docker-compose up
Creating network "nessie-py_default" with the default driver
Creating nessie-py_notebook-server_1 ... done
Attaching to nessie-py_notebook-server_1
notebook-server_1  | 
notebook-server_1  | CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
notebook-server_1  | If your shell is Bash or a Bourne variant, enable conda for the current user with
notebook-server_1  | 
notebook-server_1  |     $ echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc
notebook-server_1  | 
notebook-server_1  | or, for all users, enable conda with
notebook-server_1  | 
notebook-server_1  |     $ sudo ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh
notebook-server_1  | 
notebook-server_1  | The options above will permanently enable the 'conda' command, but they do NOT
notebook-server_1  | put conda's base (root) environment on PATH.  To do so, run
notebook-server_1  | 
notebook-server_1  |     $ conda activate
notebook-server_1  | 
notebook-server_1  | in your terminal, or to put the base environment on PATH permanently, run
notebook-server_1  | 
notebook-server_1  |     $ echo "conda activate" >> ~/.bashrc
notebook-server_1  | 
notebook-server_1  | Previous to conda 4.4, the recommended way to activate conda was to modify PATH in
notebook-server_1  | your ~/.bashrc file.  You should manually remove the line that looks like
notebook-server_1  | 
notebook-server_1  |     export PATH="/opt/conda/bin:$PATH"
notebook-server_1  | 
notebook-server_1  | ^^^ The above line should NO LONGER be in your ~/.bashrc file! ^^^
notebook-server_1  | 
notebook-server_1  | 
nessie-py_notebook-server_1 exited with code 1

此错误表示bash在运行脚本之前未获取~/.bashrc
我尝试在激活conda环境之前显式获取/opt/conda/etc/profile.d/conda.sh

#!/bin/bash
set -e

# activate the environment and start the notebook
. /opt/conda/etc/profile.d/conda.sh
conda activate nessie-py
jupyter notebook --no-browser ip=0.0.0.0

这会导致不同的错误!

$ docker-compose up
Creating network "nessie-py_default" with the default driver
Creating nessie-py_notebook-server_1 ... done
Attaching to nessie-py_notebook-server_1
notebook-server_1  | Could not find conda environment: nessie-py
notebook-server_1  | You can list all discoverable environments with `conda info --envs`.
notebook-server_1  | 
nessie-py_notebook-server_1 exited with code 1

我可以通过运行

$ docker run -it nessie-py conda info --envs

它说环境确实存在。

$ docker run -it nessie-py_notebook-server conda info --envs
# conda environments:
#
nessie-py                /home/al-khawarizmi/.conda/envs/nessie-py
base                  *  /opt/conda

Here是一个包含docker-compose.yml文件的项目示例,该文件是一个指定conda环境并启动Jupyter笔记本服务器的Dockerfile。
我需要的额外复杂性包括向Dockerfile添加一个非root用户,并创建一个新的conda环境,而不是更新默认的base conda环境。

dgiusagp

dgiusagp1#

所发生的情况是以下情况的后果:
1.在docker-compose.yml中,您在ip=0.0.0.0中有一处排印错误,应为--ip=0.0.0.0
1.将主机的文件夹绑定到容器将覆盖.bashrc
1.您需要在交互模式(-i)下运行bash,以便正确读取.bashrc
例如,在docker-compose.yml中反映的这些点上的更改:

version: "3.7"

    services:
      notebook-server:
        build:
          context: ./
        ports:
          - "8888:8888"
        volumes:
          - ./:/home/al-khawarizmi/hosthome
        command: bash -ic 'jupyter notebook --no-browser --ip=0.0.0.0'
o7jaxewo

o7jaxewo2#

根据这个article,有一个更简单的方法来完成这个任务,如果你把SHELL ["conda", "run", "-n", "<venv>", "/bin/bash", "-c"]放在你的 Dockerfile 中,并在你的 docker-compose.yml 中使用conda run --no-capture-output -n <venv> <your awesome command>
例如,您的 Dockerfile 可能如下所示:

FROM continuumio/anaconda3

WORKDIR /usr/src/app

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

COPY ./environment.yml .
RUN conda env create -f environment.yml
SHELL ["conda", "run", "-n", "venv", "/bin/bash", "-c"]

COPY . .

您的 docker-compose.yml 可能类似于以下内容:

version: '3.3'

services:

  web:
    environment: [.env]
    build: ./project
    ports:
      - 8000:8000
    command: conda run --no-capture-output -n venv uvicorn main:app --host 0.0.0.0 --reload
    volumes:
      - ./project:/usr/src/app

相关问题