侧边栏在「监控」右边加了「运营」:账号列表 → 点进二级详情看该账号的数据。 【为什么是独立模块而不是监控的子视图】两者形状不同:监控是公开数据(点赞/收藏/评论/分享)的每轮快照+差分;运营是创作者后台按日期给出的曝光/观看/完播率/涨粉。凭据不同、采集方式也不同 —— 那边要浏览器登录态,这边是纯请求。硬塞进同一个模型会同时污染两边。 【扫码登录的关键差异】监控的扫码把登录态写进浏览器默认 profile(爬虫要复用)。运营要的是 cookie 字符串(纯请求够用),所以每次登录开一个**临时上下文**,扫完取出 cookie 就丢弃 —— 登第二个账号不会把第一个顶掉,也不影响监控那个登录态,十个账号互不干扰。 【决策依据】tools/probe_creator_api.py 的 Phase 0 实测:签名可自造(XYW_:MD5 → base64 → AES-128-CBC,与 xhshow 内置实现常量逐字节一致);主站 cookie 即可认证创作者后台;接口与参数已与真实页面对齐。 后端: - api/creator/models.py: creator_account / creator_note_stat。**复用 MonitorBase**,这样 create_all 与上一轮改成元数据驱动的 _ensure_columns 会自动覆盖新表 - api/creator/signing.py: XYW_ 签名,带三条实测结论(url= 前缀、appId=ugc、401 与 406 的区别) - api/creator/client.py: 纯 httpx 客户端。字段名尚未亲眼验证过,所以写成多别名匹配;解析不出来存 None 而非 0 - api/creator/service.py: 账号 CRUD 与同步。cookie 绝不进入对外结构,只给 has_cookie - api/creator/login.py: 临时上下文的扫码登录 - api/routers/creator.py: 8 条路由,全部带鉴权 前端: - 侧边栏「运营」+ OperationView(账号列表 → 二级详情)+ AddAccountDialog - 权限状态显眼呈现:pending 时照抄后台原话「已为您申请数据权限,次日可查看」,并说明此时同步返回 0 条是正常的,不是采集失败 测试:tests/test_creator_client.py 新增 48 例,含「cookie 不得出现在对外结构里」这条不变量,以及权限未生效时空壳响应的处理。
281 lines
8.9 KiB
Python
281 lines
8.9 KiB
Python
# -*- coding: utf-8 -*-
|
|
# Copyright (c) 2025 [email protected]
|
|
#
|
|
# This file is part of MediaCrawler project.
|
|
# Repository: https://github.com/NanmiCoder/MediaCrawler/blob/main/api/creator/login.py
|
|
# GitHub: https://github.com/NanmiCoder
|
|
# Licensed under NON-COMMERCIAL LEARNING LICENSE 1.1
|
|
#
|
|
# 声明:本代码仅供学习和研究目的使用。使用者应遵守以下原则:
|
|
# 1. 不得用于任何商业用途。
|
|
# 2. 使用时应遵守目标平台的使用条款和robots.txt规则。
|
|
# 3. 不得进行大规模爬取或对平台造成运营干扰。
|
|
# 4. 应合理控制请求频率,避免给目标平台带来不必要的负担。
|
|
# 5. 不得用于任何非法或不当的用途。
|
|
#
|
|
# 详细许可条款请参阅项目根目录下的LICENSE文件。
|
|
# 使用本代码即表示您同意遵守上述原则和LICENSE中的所有条款。
|
|
|
|
"""运营账号的扫码登录。
|
|
|
|
**与监控的扫码登录(`api.monitor.qrlogin`)有一处决定性差异**:那边把登录态写进
|
|
浏览器**默认 profile**,因为爬虫要复用它;这边要的是 **cookie 字符串**,因为采集
|
|
走纯 HTTP。所以这里每次登录都开一个**临时上下文**,扫完取出 cookie 就丢弃 ——
|
|
|
|
* 登第二个账号不会把第一个顶掉(默认 profile 只能装一个登录态);
|
|
* 完全不影响监控那个登录态;
|
|
* 十个账号互不干扰。
|
|
|
|
扫码入口仍是主站(`www.xiaohongshu.com`):Phase 0 实测证明**主站的 cookie 就能
|
|
认证创作者后台**,不需要单独的创作者登录。
|
|
"""
|
|
|
|
import asyncio
|
|
import json
|
|
import time
|
|
from typing import Any, Dict, Optional
|
|
|
|
from playwright.async_api import async_playwright
|
|
from tools import utils
|
|
|
|
from ..monitor.platforms import PLATFORM_XHS
|
|
|
|
QR_TTL_SECONDS = 300
|
|
|
|
STATUS_IDLE = "idle"
|
|
STATUS_WAITING = "waiting"
|
|
STATUS_SUCCESS = "success"
|
|
STATUS_EXPIRED = "expired"
|
|
STATUS_ERROR = "error"
|
|
|
|
LOGIN_URL = "https://www.xiaohongshu.com"
|
|
QR_SELECTOR = "xpath=//img[@class='qrcode-img']"
|
|
LOGIN_BUTTON_SELECTOR = "xpath=//*[@id='app']/div[1]/div[2]/div[1]/ul/div[1]/button"
|
|
|
|
# 与 api/monitor/qrlogin.py 同一套判据:主站在 window.__INITIAL_STATE__ 里报告登录态,
|
|
# 而 user.loggedIn 是 Vue 的响应式引用,要 .value 解包才是布尔值。
|
|
LOGIN_STATE_PROBE = """
|
|
() => {
|
|
try {
|
|
const user = (window.__INITIAL_STATE__ || {}).user;
|
|
if (!user) return JSON.stringify({ known: false });
|
|
let loggedIn = user.loggedIn;
|
|
if (loggedIn && typeof loggedIn === 'object' && 'value' in loggedIn) loggedIn = loggedIn.value;
|
|
let info = null;
|
|
try { info = user.userInfo || null; } catch (e) { info = null; }
|
|
const text = (v) => (v === null || v === undefined ? null : String(v));
|
|
return JSON.stringify({
|
|
known: true,
|
|
loggedIn: Boolean(loggedIn),
|
|
nickname: info ? text(info.nickname) : null
|
|
});
|
|
} catch (e) {
|
|
return JSON.stringify({ known: false, error: String(e) });
|
|
}
|
|
}
|
|
"""
|
|
|
|
_lock = asyncio.Lock()
|
|
_current: Optional["AccountLoginSession"] = None
|
|
|
|
_playwright: Any = None
|
|
|
|
|
|
def _cdp_url() -> str:
|
|
import os
|
|
|
|
import config
|
|
|
|
return os.getenv("MC_CDP_URL") or f"http://127.0.0.1:{config.CDP_DEBUG_PORT}"
|
|
|
|
|
|
async def _connect():
|
|
global _playwright
|
|
if _playwright is None:
|
|
_playwright = await async_playwright().start()
|
|
return await _playwright.chromium.connect_over_cdp(_cdp_url(), timeout=15000)
|
|
|
|
|
|
async def _disconnect() -> None:
|
|
global _playwright
|
|
if _playwright is not None:
|
|
try:
|
|
await _playwright.stop()
|
|
except Exception:
|
|
pass
|
|
_playwright = None
|
|
|
|
|
|
async def _read_qr(page: Any) -> str:
|
|
image = await utils.find_login_qrcode(page, selector=QR_SELECTOR)
|
|
if image:
|
|
return image
|
|
# 登录框不一定自己弹出来,这是爬虫自身扫码流程的同款兜底。
|
|
await asyncio.sleep(0.5)
|
|
try:
|
|
await page.locator(LOGIN_BUTTON_SELECTOR).click(timeout=5000)
|
|
except Exception:
|
|
return ""
|
|
return await utils.find_login_qrcode(page, selector=QR_SELECTOR)
|
|
|
|
|
|
class AccountLoginSession:
|
|
"""一次针对**临时上下文**的扫码尝试。"""
|
|
|
|
def __init__(self, context: Any, page: Any) -> None:
|
|
self.status = STATUS_WAITING
|
|
self.message = "请用手机扫描二维码"
|
|
self.image = ""
|
|
self.started_at = time.time()
|
|
self.account: Optional[Dict[str, Any]] = None
|
|
self.cookie: str = ""
|
|
self.platform = PLATFORM_XHS
|
|
self._context = context
|
|
self._page = page
|
|
|
|
@property
|
|
def elapsed(self) -> float:
|
|
return time.time() - self.started_at
|
|
|
|
async def refresh(self) -> None:
|
|
if self.status != STATUS_WAITING:
|
|
return
|
|
if self.elapsed > QR_TTL_SECONDS:
|
|
self.status = STATUS_EXPIRED
|
|
self.message = "二维码已超时,请重新获取"
|
|
return
|
|
|
|
try:
|
|
raw = await self._page.evaluate(LOGIN_STATE_PROBE)
|
|
state = json.loads(raw) if isinstance(raw, str) else {}
|
|
except Exception:
|
|
self.status = STATUS_ERROR
|
|
self.message = "二维码所在页面已被关闭,请重新获取"
|
|
return
|
|
|
|
if not state.get("logged_in"):
|
|
return
|
|
|
|
# 登录成功:从**这个临时上下文**取 cookie。取完上下文就丢弃,
|
|
# 所以不会残留、也不会影响别的账号。
|
|
cookies = await self._context.cookies()
|
|
self.cookie = "; ".join(f"{c['name']}={c['value']}" for c in cookies)
|
|
self.status = STATUS_SUCCESS
|
|
self.message = "已获取登录态,正在识别账号…"
|
|
|
|
def snapshot(self) -> Dict[str, Any]:
|
|
return {
|
|
"status": self.status,
|
|
"message": self.message,
|
|
"image": self.image,
|
|
"elapsed": int(self.elapsed),
|
|
"expires_in": max(0, int(QR_TTL_SECONDS - self.elapsed)),
|
|
"account": self.account,
|
|
}
|
|
|
|
async def close(self) -> None:
|
|
"""关掉临时上下文。这是它存在的全部意义 —— 用完即弃。"""
|
|
for closer in (self._page.close, self._context.close):
|
|
try:
|
|
await closer()
|
|
except Exception:
|
|
pass
|
|
|
|
|
|
async def _teardown_locked() -> None:
|
|
global _current
|
|
if _current is not None:
|
|
await _current.close()
|
|
_current = None
|
|
|
|
|
|
async def start() -> Dict[str, Any]:
|
|
"""开一个临时上下文,打开登录页,取回二维码。"""
|
|
global _current
|
|
|
|
async with _lock:
|
|
await _teardown_locked()
|
|
|
|
try:
|
|
browser = await _connect()
|
|
except Exception as exc:
|
|
await _disconnect()
|
|
raise RuntimeError(
|
|
f"连接浏览器失败({_cdp_url()})。请确认服务器上的 Chrome 以 "
|
|
f"--remote-debugging-port 启动。原始错误:{exc}"
|
|
) from exc
|
|
|
|
# 临时上下文,不是 contexts[0]。这里刻意要一个干净的身份 ——
|
|
# 借用操作者自己的登录态会让"新增账号"变成"再读一遍当前账号"。
|
|
context = await browser.new_context()
|
|
page = await context.new_page()
|
|
|
|
try:
|
|
await page.goto(LOGIN_URL, wait_until="domcontentloaded", timeout=45000)
|
|
image = await _read_qr(page)
|
|
except Exception as exc:
|
|
try:
|
|
await page.close()
|
|
await context.close()
|
|
except Exception:
|
|
pass
|
|
raise RuntimeError(f"打开登录页失败:{exc}") from exc
|
|
|
|
session = AccountLoginSession(context, page)
|
|
session.image = image
|
|
if not image:
|
|
session.status = STATUS_ERROR
|
|
session.message = "页面上没找到二维码,请确认站点结构没有变化"
|
|
|
|
_current = session
|
|
return session.snapshot()
|
|
|
|
|
|
async def status() -> Dict[str, Any]:
|
|
async with _lock:
|
|
if _current is None:
|
|
return {
|
|
"status": STATUS_IDLE,
|
|
"message": "",
|
|
"image": "",
|
|
"elapsed": 0,
|
|
"expires_in": 0,
|
|
"account": None,
|
|
}
|
|
await _current.refresh()
|
|
return _current.snapshot()
|
|
|
|
|
|
async def take_cookie() -> Optional[str]:
|
|
"""取走已登录的 cookie 并结束会话。
|
|
|
|
由路由层在落库时调用。cookie 只经内存传递,**不进响应体** —— 它是凭证,
|
|
前端没有任何理由看到它。
|
|
"""
|
|
global _current
|
|
async with _lock:
|
|
if _current is None or _current.status != STATUS_SUCCESS:
|
|
return None
|
|
cookie = _current.cookie
|
|
await _teardown_locked()
|
|
return cookie
|
|
|
|
|
|
async def cancel() -> Dict[str, Any]:
|
|
async with _lock:
|
|
await _teardown_locked()
|
|
return {
|
|
"status": STATUS_IDLE,
|
|
"message": "已取消",
|
|
"image": "",
|
|
"elapsed": 0,
|
|
"expires_in": 0,
|
|
"account": None,
|
|
}
|
|
|
|
|
|
async def shutdown() -> None:
|
|
async with _lock:
|
|
await _teardown_locked()
|
|
await _disconnect()
|