fix: 扫码登录状态可独立查询 + 趋势图改回自适应纵轴
Deploy VitePress site to Pages / build (push) Canceled after 0s
Deploy VitePress site to Pages / Deploy (push) Canceled after 0s

【登录反馈】实测这台浏览器 loggedIn=true,其实早就登录成功了;看不到反馈是判据和设计的问题:

1. 判据不可靠。原先靠 web_session 的值变化判断——对照组显示:一个全新的空 profile 首次访问小红书就会被发一个 web_session,所以「有这个 cookie」什么都证明不了。可信信号是页面自己的 __INITIAL_STATE__.user.loggedIn,但它是 Vue 响应式引用,必须 .value 解包(这就是前面探针读到 [object Object] 和 None 的原因)。
2. 状态绑死在临时会话上。扫码会话是内存状态,进程一重启就没(部署、崩溃都算),面板于是悄悄退回初始态——一次成功扫码看起来像什么都没发生。

改法不是让会话活得久,而是把「登没登录」变成随时可查、与会话无关:
- 新增 GET /api/monitor/login/state,直接问浏览器要答案,带 5 秒缓存;force=true 先重载页面再读,用于状态陈旧
- qrlogin 改为常驻 Playwright 客户端 + 复用同一个标签页,并在重启后认领浏览器里已存在的 xhs 标签页,避免堆孤儿页
- 把「读不到状态」与「未登录」分开——前者显示具体错误,不再悄悄显示成未登录
- 面板顶部常驻显示登录态与昵称,带「重新检测」按钮;扫码成功后自动翻转

【趋势图】上一轮改过头了。dataviz 规范里没有「折线图必须从 0 起」这条——基线相关的条文全是讲柱状图的(柱状图用长度编码数值,不从 0 起比例就是错的;折线图用位置编码,轴只需如实框住数据)。改回自适应,保留上一轮修好的左侧刻度栏让范围始终可见;步长收敛到 1/2/5×10ⁿ,全平序列撑开一档避免除零。
This commit is contained in:
2026-10-07 15:42:58 +08:00
parent e608b51210
commit 6eff6fcc83
8 changed files with 578 additions and 270 deletions
+252 -129
View File
@@ -8,7 +8,7 @@
#
# 声明:本代码仅供学习和研究目的使用。使用者应遵守以下原则:
# 1. 不得用于任何商业用途。
# 2. 使用时应遵守目标平台的使用条款和robots.txt规则。
# 2. 使用时应遵守对应平台的使用条款和robots.txt规则。
# 3. 不得进行大规模爬取或对平台造成运营干扰。
# 4. 应合理控制请求频率,避免给目标平台带来不必要的负担。
# 5. 不得用于任何非法或不当的用途。
@@ -16,32 +16,29 @@
# 详细许可条款请参阅项目根目录下的LICENSE文件。
# 使用本代码即表示您同意遵守上述原则和LICENSE中的所有条款。
"""Show a login QR code to the operator through the WebUI.
"""扫码登录,以及"现在到底登没登录"的查询。
Why this module exists: on a server deployment Chrome runs under Xvfb, so there
is no display to look at, and the helper the crawler normally uses to present the
QR (`show_qrcode` in `tools/crawler_util.py`) calls PIL's ``Image.show()`` -- it
needs a desktop image viewer that such a machine does not have, so the code would
go nowhere and the operator would be stuck. Instead the QR is read straight out
of the page over CDP and handed to the WebUI, which renders it as an ``<img>``.
**为什么需要这个模块**:服务器上 Chrome 跑在 Xvfb 里没有显示器,爬虫原本用
`show_qrcode`(PIL 的 `Image.show()`)弹窗展示二维码,那需要桌面看图程序,服务器上
没有。所以改成经 CDP 把二维码从页面里读出来交给 WebUI。
Three details that are easy to get wrong:
**为什么登录状态要能独立查询**:扫码会话是内存里的临时状态,进程一重启就没了
(部署、崩溃都算)。把"是否已登录"绑在它上面,就会出现"扫完了但界面没反应、
也不知道到底成没成"。所以状态查询是独立的、随时可调用的,二维码只是达成它的手段之一。
* **Reuse the browser's default context.** ``browser.new_context()`` would give
an incognito-like profile, so the scan would land in a cookie jar the crawler
never reads and every run would still look logged out. The real profile -- the
one the crawler attaches to -- is ``browser.contexts[0]``.
* **Never call ``browser.close()``.** On a CDP connection that tears down the
operator's own Chrome and takes every unrelated tab with it. Only the page this
module opened is closed, and the Playwright client is stopped to drop the
socket.
* **A scan is not a login until the cookie changes.** The page can be showing a
QR for an account that is in fact already signed in, so success is decided by
``web_session`` appearing or changing against the value captured at start,
never by anything the page displays.
三个容易搞错的地方:
* **必须复用浏览器默认 context**。`browser.new_context()` 会造出一个无痕式的 profile,
扫了也白扫——爬虫读不到那份 cookie。真正的 profile 在 `browser.contexts[0]`。
* **绝不能调 `browser.close()`**。对 CDP 连接而言那会关掉操作者自己的 Chrome,
连带所有无关标签页。只能关本模块自己开的那一个。
* **不能靠 `web_session` 判断登录**。实测:一个全新的空 profile 首次访问小红书就会
被发一个 `web_session`,所以"有这个 cookie"什么都证明不了。可信信号是页面自己的
`__INITIAL_STATE__.user.loggedIn`。
"""
import asyncio
import json
import os
import time
from typing import Any, Dict, Optional
@@ -52,10 +49,12 @@ from tools import utils
from .platforms import PLATFORM_XHS
# How long a QR stays valid before the session is written off. The platform
# rotates the code well before this; the limit exists so an abandoned attempt
# cannot pin a browser tab open indefinitely.
QR_TTL_SECONDS = 180
# 二维码有效期。平台自己会更早轮换;这个上限只是为了让一次被放弃的尝试不会
# 永久占着一个标签页。
QR_TTL_SECONDS = 300
# 登录状态查询的缓存时长。轮询时不必每次都去问浏览器。
STATE_CACHE_SECONDS = 5
STATUS_IDLE = "idle"
STATUS_WAITING = "waiting"
@@ -63,35 +62,56 @@ STATUS_SUCCESS = "success"
STATUS_EXPIRED = "expired"
STATUS_ERROR = "error"
# Only xhs is wired: it is the only platform whose monitor pipeline works, and
# pretending otherwise would offer the operator a button that cannot succeed.
# 只有小红书接了监控流程,所以扫码也只对它开放。给别的平台显示一个按不动的按钮
# 是在假装功能存在。
LOGIN_URL: Dict[str, str] = {PLATFORM_XHS: "https://www.xiaohongshu.com"}
EXPLORE_URL: Dict[str, str] = {PLATFORM_XHS: "https://www.xiaohongshu.com/explore"}
QR_SELECTOR: Dict[str, str] = {PLATFORM_XHS: "xpath=//img[@class='qrcode-img']"}
LOGIN_BUTTON_SELECTOR: Dict[str, str] = {
PLATFORM_XHS: "xpath=//*[@id='app']/div[1]/div[2]/div[1]/ul/div[1]/button"
}
SESSION_COOKIE: Dict[str, str] = {PLATFORM_XHS: "web_session"}
_IDLE_SNAPSHOT: Dict[str, Any] = {
"status": STATUS_IDLE,
"platform": None,
"image": "",
"message": "",
"elapsed": 0,
"expires_in": 0,
# 页面自己报告的登录态。
#
# `user.loggedIn` 是 Vue 的响应式引用,直接读会得到一个对象(这正是最初探针读到
# "[object Object]" 的原因),必须取 `.value`。返回字符串而不是对象,因为
# `__INITIAL_STATE__` 里有循环引用,`JSON.stringify` 整个结构会抛
# "Converting circular structure to JSON"。
LOGIN_STATE_PROBE = """
() => {
try {
const user = (window.__INITIAL_STATE__ || {}).user;
if (!user) return JSON.stringify({ known: false });
let loggedIn = user.loggedIn;
if (loggedIn && typeof loggedIn === 'object' && 'value' in loggedIn) loggedIn = loggedIn.value;
let info = null;
try { info = user.userInfo || null; } catch (e) { info = null; }
const text = (v) => (v === null || v === undefined ? null : String(v));
return JSON.stringify({
known: true,
loggedIn: Boolean(loggedIn),
nickname: info ? text(info.nickname) : null
});
} catch (e) {
return JSON.stringify({ known: false, error: String(e) });
}
}
"""
_lock = asyncio.Lock()
_current: Optional["QrLoginSession"] = None
# 常驻的 Playwright 客户端和本模块自己的标签页。长期持有是有意的:状态查询要能
# 随时回答,而每次都新建一个标签页会在操作者的浏览器里堆垃圾。
_playwright: Any = None
_page: Any = None
# (时间戳, 结果),避免轮询时反复问浏览器。
_state_cache: Optional[tuple[float, Dict[str, Any]]] = None
def _cdp_url() -> str:
"""Where to reach the browser's DevTools endpoint.
``MC_CDP_URL`` wins so a deployment can point at another host without a code
change; otherwise the port comes from the same config the crawler itself
reads, so the two can never drift apart.
"""
"""浏览器 DevTools 端点。``MC_CDP_URL`` 优先,便于换主机而不用改代码。"""
return os.getenv("MC_CDP_URL") or f"http://127.0.0.1:{config.CDP_DEBUG_PORT}"
@@ -101,47 +121,163 @@ def _login_url(platform: str) -> str:
return LOGIN_URL[platform]
class QrLoginSession:
"""One live QR-login attempt against the CDP browser."""
async def _ensure_context() -> Any:
"""连上浏览器并返回它的默认 context。"""
global _playwright
def __init__(self, platform: str, playwright: Any, page: Any, baseline: str) -> None:
if _playwright is None:
_playwright = await async_playwright().start()
try:
browser = await _playwright.chromium.connect_over_cdp(_cdp_url(), timeout=15000)
except Exception as exc:
await _reset_playwright()
raise RuntimeError(
f"连接浏览器失败({_cdp_url()})。请确认服务器上的 Chrome 以 "
f"--remote-debugging-port 启动。原始错误:{exc}"
) from exc
if not browser.contexts:
raise RuntimeError(
"浏览器没有可用上下文。CDP 已连上,但读不到 profile —— "
"请确认 Chrome 不是以无痕模式启动的。"
)
# contexts[0] 就是真实 profile,用它,不要 new_context()。
return browser.contexts[0]
async def _ensure_page(platform: str = PLATFORM_XHS, reload: bool = False) -> Any:
"""本模块在操作者浏览器里的那一个标签页,复用而不是反复新建。
若已有一个停在目标站点的标签页就认领它——进程重启后页柄会丢,但标签页还在,
认领可以避免在浏览器里留下一堆没人关的孤儿页。
"""
global _page
context = await _ensure_context()
if _page is not None:
try:
if _page.is_closed():
_page = None
except Exception:
_page = None
if _page is None:
for candidate in context.pages:
try:
if "xiaohongshu.com" in candidate.url or "rednote.com" in candidate.url:
_page = candidate
break
except Exception:
continue
if _page is None:
_page = await context.new_page()
try:
url = _page.url
except Exception:
url = ""
if reload or "xiaohongshu.com" not in url and "rednote.com" not in url:
await _page.goto(
EXPLORE_URL.get(platform, EXPLORE_URL[PLATFORM_XHS]),
wait_until="domcontentloaded",
timeout=45000,
)
return _page
async def check_login_state(force: bool = False) -> Dict[str, Any]:
"""问浏览器:现在登录了吗?
``force`` 会先重新加载页面。SPA 的状态会随登录实时更新,所以轮询时不必重载;
但若登录态是在别处失效的,页面上的副本可能是陈旧的,重新检测就该重载。
"""
global _state_cache
now = time.time()
if not force and _state_cache is not None:
cached_at, cached = _state_cache
if now - cached_at < STATE_CACHE_SECONDS:
return cached
try:
page = await _ensure_page(reload=force)
raw = await page.evaluate(LOGIN_STATE_PROBE)
parsed = json.loads(raw) if isinstance(raw, str) else {"known": False}
except Exception as exc:
result = {
"known": False,
"logged_in": False,
"nickname": None,
"error": f"{exc.__class__.__name__}: {exc}",
}
_state_cache = (now, result)
return result
result = {
"known": bool(parsed.get("known")),
"logged_in": bool(parsed.get("loggedIn")),
"nickname": parsed.get("nickname"),
}
_state_cache = (now, result)
return result
async def _reset_playwright() -> None:
global _playwright, _page
_page = None
if _playwright is not None:
try:
await _playwright.stop()
except Exception:
pass
_playwright = None
class QrLoginSession:
"""一次进行中的扫码尝试。"""
def __init__(self, platform: str, page: Any) -> None:
self.platform = platform
self.status = STATUS_WAITING
self.message = "请用小红书 App 扫描二维码"
self.message = "请用手机扫描二维码"
self.image = ""
self.started_at = time.time()
self._playwright = playwright
self.logged_in = False
self.nickname: Optional[str] = None
self._page = page
# The `web_session` value present *before* the scan. An account already
# signed in has a non-empty baseline, which is why success is "changed",
# not merely "present".
self._baseline = baseline
@property
def elapsed(self) -> float:
return time.time() - self.started_at
async def refresh(self) -> None:
"""Poll the browser once for a completed scan."""
"""轮询一次,看扫码是否完成。"""
if self.status != STATUS_WAITING:
return
if self.elapsed > QR_TTL_SECONDS:
self.status = STATUS_EXPIRED
self.message = "二维码已超时,请重新获取"
return
try:
cookies = await self._page.context.cookies()
except Exception:
# The operator may have closed the tab we opened.
self.status = STATUS_ERROR
self.message = "二维码所在页面已被关闭,请重新获取"
return
token = {c["name"]: c["value"] for c in cookies}.get(
SESSION_COOKIE[self.platform], ""
)
if token and token != self._baseline:
state = await check_login_state()
if state.get("logged_in"):
self.logged_in = True
self.nickname = state.get("nickname")
self.status = STATUS_SUCCESS
self.message = "登录成功,登录态已写入浏览器 profile"
who = f"({self.nickname})" if self.nickname else ""
self.message = f"登录成功{who},登录态已写入浏览器 profile"
return
try:
if self._page.is_closed():
self.status = STATUS_ERROR
self.message = "二维码所在页面已被关闭,请重新获取"
except Exception:
pass
def snapshot(self) -> Dict[str, Any]:
return {
@@ -151,27 +287,17 @@ class QrLoginSession:
"message": self.message,
"elapsed": int(self.elapsed),
"expires_in": max(0, int(QR_TTL_SECONDS - self.elapsed)),
"logged_in": self.logged_in,
"nickname": self.nickname,
}
async def close(self) -> None:
"""Drop our page and the Playwright client, leaving Chrome untouched."""
try:
await self._page.close()
except Exception:
pass
try:
await self._playwright.stop()
except Exception:
pass
async def _read_qr(page: Any, platform: str) -> str:
"""Pull the QR image out of the page, opening the login dialog if needed."""
"""把二维码从页面里取出来,必要时先点开登录框。"""
image = await utils.find_login_qrcode(page, selector=QR_SELECTOR[platform])
if image:
return image
# The dialog does not always open on its own. This is the same fallback the
# crawler's own QR flow performs before giving up.
# 登录框不一定自己弹出来。这是爬虫自身扫码流程里同款兜底。
await asyncio.sleep(0.5)
try:
await page.locator(LOGIN_BUTTON_SELECTOR[platform]).click(timeout=5000)
@@ -180,68 +306,45 @@ async def _read_qr(page: Any, platform: str) -> str:
return await utils.find_login_qrcode(page, selector=QR_SELECTOR[platform])
async def _reset_locked() -> None:
async def _discard_current_locked() -> None:
global _current
if _current is not None:
await _current.close()
_current = None
_current = None
async def start(platform: str = PLATFORM_XHS) -> Dict[str, Any]:
"""Open a login page in the CDP browser and return its QR code."""
"""在 CDP 浏览器里打开登录页,取回二维码。"""
global _current
if platform not in LOGIN_URL:
raise ValueError(f"平台 {platform} 尚未接入扫码登录(目前仅支持小红书)")
async with _lock:
await _reset_locked()
await _discard_current_locked()
playwright = await async_playwright().start()
page = await _ensure_page(platform)
try:
browser = await playwright.chromium.connect_over_cdp(_cdp_url(), timeout=15000)
except Exception as exc:
await playwright.stop()
raise RuntimeError(
f"连接浏览器失败({_cdp_url()})。请确认服务器上的 Chrome 以 "
f"--remote-debugging-port 启动。原始错误:{exc}"
) from exc
if not browser.contexts:
await playwright.stop()
raise RuntimeError(
"浏览器没有可用上下文。CDP 已连上,但读不到 profile —— "
"请确认 Chrome 不是以无痕模式启动的。"
await page.goto(
_login_url(platform), wait_until="domcontentloaded", timeout=45000
)
# contexts[0] is the real profile. See the module docstring.
context = browser.contexts[0]
page = await context.new_page()
try:
await page.goto(_login_url(platform), wait_until="domcontentloaded", timeout=30000)
image = await _read_qr(page, platform)
cookies = await context.cookies()
except Exception as exc:
try:
await page.close()
except Exception:
pass
await playwright.stop()
raise RuntimeError(f"打开登录页失败:{exc}") from exc
baseline = {c["name"]: c["value"] for c in cookies}.get(
SESSION_COOKIE[platform], ""
)
session = QrLoginSession(platform, playwright, page, baseline)
session = QrLoginSession(platform, page)
session.image = image
if not image:
if baseline:
session.status = STATUS_SUCCESS
session.message = "浏览器已经是登录状态,无需扫码"
else:
session.status = STATUS_ERROR
session.message = "页面上没找到二维码,请确认站点结构没有变化"
global _current
state = await check_login_state()
if state.get("logged_in"):
# 已经是登录状态时站点不显示二维码——这本身就是成功,不是失败。
session.status = STATUS_SUCCESS
session.logged_in = True
session.nickname = state.get("nickname")
who = f"({session.nickname})" if session.nickname else ""
session.message = f"浏览器已经是登录状态{who},无需扫码"
elif not image:
session.status = STATUS_ERROR
session.message = "页面上没找到二维码,请确认站点结构没有变化"
_current = session
return session.snapshot()
@@ -249,20 +352,40 @@ async def start(platform: str = PLATFORM_XHS) -> Dict[str, Any]:
async def status() -> Dict[str, Any]:
async with _lock:
if _current is None:
return dict(_IDLE_SNAPSHOT)
state = await check_login_state()
return {
"status": STATUS_IDLE,
"platform": None,
"image": "",
"message": "",
"elapsed": 0,
"expires_in": 0,
"logged_in": bool(state.get("logged_in")),
"nickname": state.get("nickname"),
}
await _current.refresh()
return _current.snapshot()
async def cancel() -> Dict[str, Any]:
async with _lock:
await _reset_locked()
snapshot = dict(_IDLE_SNAPSHOT)
snapshot["message"] = "已取消"
return snapshot
await _discard_current_locked()
state = await check_login_state()
return {
"status": STATUS_IDLE,
"platform": None,
"image": "",
"message": "已取消",
"elapsed": 0,
"expires_in": 0,
"logged_in": bool(state.get("logged_in")),
"nickname": state.get("nickname"),
}
async def shutdown() -> None:
"""Release the browser tab at application shutdown."""
"""进程退出时断开连接。刻意不关那个标签页——它是操作者浏览器的一部分。"""
global _current
async with _lock:
await _reset_locked()
_current = None
await _reset_playwright()