butubb
37ca1b1cd6
- runner: enable_cdp_mode 从硬编码 False 改为系统设置 cdp_enabled。服务器部署下爬虫接管已开启远程调试的 Chrome（默认 9222），复用其 profile 登录态；本机桌面默认仍为关，行为不变 - qrlogin: 新增 CDP 扫码登录。Chrome 在服务器上跑于 Xvfb，show_qrcode 依赖的 PIL 桌面看图程序不存在，二维码无处可显示；改为经 CDP 从页面取出二维码交给 WebUI 渲染。刻意复用 browser.contexts[0]（新建 context 是无痕 profile，扫了也白扫），且绝不调用 browser.close()（会连带关掉操作者自己的 Chrome） - webui: 设置页新增扫码面板，替换原本跳到采集页看终端二维码的入口 - Dockerfile / .dockerignore / docker-compose.yml: 服务器部署。host 网络是必需而非图省事——容器里 127.0.0.1:9222 必须落到宿主机回环 - UPSTREAM.md: 补充 gitcode 镜像，用于 GitHub 大包传输必断时补历史
44 lines
1.8 KiB
Docker
44 lines
1.8 KiB
Docker
# Server deployment image.
|
|
#
|
|
# No browser is bundled on purpose. On this deployment the crawler attaches over
|
|
# CDP to the Chrome already running on the host (see the 接管已有 Chrome setting),
|
|
# so shipping a second copy of Chromium would only add hundreds of megabytes and
|
|
# a login state that nothing uses. The Playwright Python package is still needed
|
|
# -- that is what speaks CDP -- hence PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD.
|
|
FROM python:3.11-slim
|
|
|
|
ENV PYTHONUNBUFFERED=1 \
|
|
PYTHONDONTWRITEBYTECODE=1 \
|
|
PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
|
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 \
|
|
TZ=Asia/Shanghai
|
|
|
|
# asyncmy compiles a Cython extension, so a toolchain has to exist at build time.
|
|
# It is left installed: purging it risks taking libmysqlclient with it, and a
|
|
# slightly larger image is cheaper than a runtime that fails months later.
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends \
|
|
build-essential pkg-config default-libmysqlclient-dev \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
WORKDIR /app
|
|
|
|
# Requirements first: this layer only rebuilds when the pins actually change.
|
|
ARG PIP_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple
|
|
COPY requirements.txt ./
|
|
RUN pip install --no-cache-dir -i "$PIP_INDEX" -r requirements.txt
|
|
|
|
COPY . .
|
|
|
|
# The WebUI bundle lives at api/webui, which is gitignored -- it is built on the
|
|
# workstation (`cd webui && npm run build`) and shipped inside the build context.
|
|
# Without this check a missing bundle is invisible until someone opens the page
|
|
# and gets the bare JSON stub from serve_frontend().
|
|
RUN test -f api/webui/index.html \
|
|
|| (echo "ERROR: api/webui/index.html is missing. Run 'cd webui && npm run build' before building the image." >&2; exit 1)
|
|
|
|
EXPOSE 18051
|
|
|
|
# api.main reads MC_HOST / MC_PORT from the environment; compose supplies both.
|
|
CMD ["python", "-m", "api.main"]
|