feat: CDP 接管开关 + 扫码登录面板 + Docker 部署
- runner: enable_cdp_mode 从硬编码 False 改为系统设置 cdp_enabled。服务器部署下爬虫接管已开启远程调试的 Chrome(默认 9222),复用其 profile 登录态;本机桌面默认仍为关,行为不变 - qrlogin: 新增 CDP 扫码登录。Chrome 在服务器上跑于 Xvfb,show_qrcode 依赖的 PIL 桌面看图程序不存在,二维码无处可显示;改为经 CDP 从页面取出二维码交给 WebUI 渲染。刻意复用 browser.contexts[0](新建 context 是无痕 profile,扫了也白扫),且绝不调用 browser.close()(会连带关掉操作者自己的 Chrome) - webui: 设置页新增扫码面板,替换原本跳到采集页看终端二维码的入口 - Dockerfile / .dockerignore / docker-compose.yml: 服务器部署。host 网络是必需而非图省事——容器里 127.0.0.1:9222 必须落到宿主机回环 - UPSTREAM.md: 补充 gitcode 镜像,用于 GitHub 大包传输必断时补历史
This commit is contained in:
+43
@@ -0,0 +1,43 @@
|
||||
# Server deployment image.
|
||||
#
|
||||
# No browser is bundled on purpose. On this deployment the crawler attaches over
|
||||
# CDP to the Chrome already running on the host (see the 接管已有 Chrome setting),
|
||||
# so shipping a second copy of Chromium would only add hundreds of megabytes and
|
||||
# a login state that nothing uses. The Playwright Python package is still needed
|
||||
# -- that is what speaks CDP -- hence PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD.
|
||||
FROM python:3.11-slim
|
||||
|
||||
ENV PYTHONUNBUFFERED=1 \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
||||
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 \
|
||||
TZ=Asia/Shanghai
|
||||
|
||||
# asyncmy compiles a Cython extension, so a toolchain has to exist at build time.
|
||||
# It is left installed: purging it risks taking libmysqlclient with it, and a
|
||||
# slightly larger image is cheaper than a runtime that fails months later.
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
build-essential pkg-config default-libmysqlclient-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Requirements first: this layer only rebuilds when the pins actually change.
|
||||
ARG PIP_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple
|
||||
COPY requirements.txt ./
|
||||
RUN pip install --no-cache-dir -i "$PIP_INDEX" -r requirements.txt
|
||||
|
||||
COPY . .
|
||||
|
||||
# The WebUI bundle lives at api/webui, which is gitignored -- it is built on the
|
||||
# workstation (`cd webui && npm run build`) and shipped inside the build context.
|
||||
# Without this check a missing bundle is invisible until someone opens the page
|
||||
# and gets the bare JSON stub from serve_frontend().
|
||||
RUN test -f api/webui/index.html \
|
||||
|| (echo "ERROR: api/webui/index.html is missing. Run 'cd webui && npm run build' before building the image." >&2; exit 1)
|
||||
|
||||
EXPOSE 18051
|
||||
|
||||
# api.main reads MC_HOST / MC_PORT from the environment; compose supplies both.
|
||||
CMD ["python", "-m", "api.main"]
|
||||
Reference in New Issue
Block a user