我尝试运行一个Flask应用程序,当某个端点被调用时,它会运行一个web scraper。所有内容都使用Docker Compose容器化。
Docker Compose File
services:
selenium:
image: seleniarm/standalone-chromium # using this because I'm on M1
hostname: local
volumes:
- '/dev/shm:/dev/shm'
ports:
- '4444:4444'
api:
build: api
restart: always
hostname: local
environment:
- environment=Local
depends_on:
- selenium
ports:
- '80:80'
command: ["gunicorn", "-w", "3", "-t","300", "-b", "0.0.0.0:80", "app:app"]
字符串
API的Dockerfile
FROM python:3.8
COPY . /api
WORKDIR /api
RUN pip install -r requirements.txt --no-cache-dir
EXPOSE 5000
EXPOSE 4444 # not sure if this is necessary?
型
示例Python函数(这是由Flask路由调用的,我知道它配置正确
from fake_useragent import UserAgent
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import time
def test_webdriver():
logger.info("Beginning to test webdriver")
ua = UserAgent()
user_agent = ua.random
options = webdriver.ChromeOptions()
options.add_argument('--log-level=3')
options.add_argument('--verbose')
options.add_argument('--ignore-ssl-errors=yes')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--remote-debugging-port=9222')
options.add_argument('--disable-dev-shm-usage')
options.add_experimental_option("excludeSwitches", ['enable-logging'])
options.add_argument(f'user-agent={user_agent}')
driver = webdriver.Remote(
command_executor="http://localhost:4444/wd/hub",
options=options
)
driver.get('https://github.com')
time.sleep(5)
driver.quit()
return
型
错误数:
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0xffffaff19490>: Failed to establish a new connection: [Errno 111] Connection refused
型
最终会抛出...
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4444): Max retries exceeded with url: /wd/hub/session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffffaff19490>: Failed to establish a new connection: [Errno 111] Connection refused'))
型
我假设由于某种原因,我的api
容器无法访问我的selenium
容器。我验证了访问http://localhost:4444
会返回Selenium Grid UI,因此我知道容器已启动并正在运行。知道为什么API找不到它吗?
1条答案
按热度按时间ifmq2ha21#
这可能与你的Docker设置有关。你需要确保selenium和API服务可以访问同一个Docker网络。你可以通过在docker-compose文件的根级别添加:
字符串
并且还将该网络添加到这两个服务,
型
对于您的API服务也是如此。