feat(monitor): 抖音 Web 接口客户端 —— 绕开爬虫子进程，直接发 HTTP
移植自 mac-agent-os 的 mediacrawler_adapter：不起子进程、不开页面，用浏览器里那份
登录态直接调抖音 Web 接口。产物键名照抄 store/douyin，所以 ingest 那条链路一个字不用改。
**目前能用的（真环境实测，非推断）**：
profile/other : 200, 7075 字节 —— 博主主页指标（粉丝/获赞/作品数/昵称）
aweme/detail : 200, 45425 字节 —— 单条作品详情（含点赞/评论/收藏/分享）
**目前不能用的：作品列表 `aweme/post`。** 两个互相独立的原因：
1. 这个接口被抖音单独升级成了真校验：不带 x-tt-argus 回 403「Uifid Not Found」，
带上 dummy 值回 200 + **空 body**。也就是说「头在不在」骗得过，「真校验」过不了。
同一套头打 profile/other 和 aweme/detail 都是通的 —— 抖音是挑着接口加保护的，
挑中的恰好是「批量拉作品列表」这个最敏感的动作。
2. 改走页面截获也不行：CDP 浏览器打开博主主页会落到「验证码中间页」（当天大量探测的
代价，过几小时要重测）。
所以现在的边界是：**已知作品的指标刷新能做，自动发现新作品做不了**。
**排查中控住变量后得到的两条事实**（都写进注释了）：
· `Accept` / `Accept-Language` / `Referer` 才是主页接口能返回真数据的原因 —— 只有
UA+client hints+Cookie 时是 200 但仅 121 字节的空壳，补上这三个头变 7074 字节。
（我先前猜的 sec-ch-ua 不是关键。）
· 因此 UA 与 client hints 必须**成套地取自同一个浏览器**，所以 BrowserIdentity 一次
从 CDP 取齐 cookie + UA + hints，而不是各自写死。
「200 + 空 body 必须当场报错」也是刻意写死的：放过去它会在下游变成「这个博主没作品」，
把一次失败伪装成一条正常结果 —— 爬虫那条路正是这么栽的，还被翻译成「账号被封」。
测试 +11：cookie 解析、请求头成套性（含 uifid 缺失/回退）、产物键名与 store 对齐、
以及 _get 的三条失败路径（空 body / 403 带网关原话 / 正常返回）。
This commit is contained in:
@@ -0,0 +1,5 @@
|
||||
"""
|
||||
ops — 多平台操作层
|
||||
|
||||
每个平台独立目录，按操作类型分文件。
|
||||
"""
|
||||
@@ -0,0 +1,420 @@
|
||||
"""
|
||||
ops/_base.py — 平台操作基类 v2.0
|
||||
|
||||
三段式操作模型 v2.0:
|
||||
每个原子操作 = 前置条件检查 + 执行动作 + 后置验证
|
||||
|
||||
1. _capture_state() → PageState 操作前/后页面状态指纹
|
||||
2. _get_pre_conditions(op) 每个操作的前置条件
|
||||
3. _do_execute(op, args) → OpResult 执行动作（子类实现，与原来一样）
|
||||
4. _get_post_conditions(op) 每个操作的后置验证条件
|
||||
5. execute() 统一入口 → 三段式执行，返回完整 OpResult
|
||||
|
||||
改造目标:
|
||||
- 录制时能记录操作前后的状态指纹
|
||||
- Analyzer 通过状态差异推断操作类型 → 不再依赖坐标
|
||||
- Exporter 输出三段式 JSON 蓝图
|
||||
"""
|
||||
import asyncio
|
||||
import time
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass, field, asdict
|
||||
from typing import Optional
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# PageState — 页面状态指纹
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
@dataclass
|
||||
class PageState:
|
||||
"""页面状态指纹 — 操作前后快照（供录制/分析/验证使用）
|
||||
|
||||
url_pattern: 归一化 URL 模式（去掉动态参数，保留路径结构）
|
||||
page_mode: grid / player / player_full / player_modal / search / profile / unknown
|
||||
selectors: {CSS选择器: 是否可见} — 关键元素的可见性状态
|
||||
markers: {标记名: 值} — 自定义状态标记（如 liked=true, like_count=1024）
|
||||
text_snippet: 页面文本片段（用于差异分析）
|
||||
"""
|
||||
url_pattern: str = ""
|
||||
page_mode: str = "unknown"
|
||||
selectors: dict = field(default_factory=dict)
|
||||
markers: dict = field(default_factory=dict)
|
||||
text_snippet: str = ""
|
||||
|
||||
def delta(self, other: 'PageState') -> dict:
|
||||
"""对比两个 PageState，返回差异摘要"""
|
||||
diffs = {}
|
||||
|
||||
if self.page_mode != other.page_mode:
|
||||
diffs["page_mode"] = f"{self.page_mode} → {other.page_mode}"
|
||||
|
||||
# 选择器状态变化
|
||||
sel_diffs = {}
|
||||
all_keys = set(self.selectors) | set(other.selectors)
|
||||
for k in all_keys:
|
||||
v1 = self.selectors.get(k)
|
||||
v2 = other.selectors.get(k)
|
||||
if v1 != v2:
|
||||
sel_diffs[k] = f"{v1} → {v2}"
|
||||
if sel_diffs:
|
||||
diffs["selectors"] = sel_diffs
|
||||
|
||||
# 标记变化
|
||||
mark_diffs = {}
|
||||
all_markers = set(self.markers) | set(other.markers)
|
||||
for k in all_markers:
|
||||
v1 = self.markers.get(k)
|
||||
v2 = other.markers.get(k)
|
||||
if v1 != v2:
|
||||
mark_diffs[k] = f"{v1} → {v2}"
|
||||
if mark_diffs:
|
||||
diffs["markers"] = mark_diffs
|
||||
|
||||
return diffs
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return asdict(self)
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# Condition — 前置/后置条件
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
@dataclass
|
||||
class Condition:
|
||||
"""执行条件（前置条件 / 后置验证）
|
||||
|
||||
条件类型 type:
|
||||
'page_mode' — 页面模式匹配（如 page_mode == 'player'）
|
||||
'selector' — DOM 元素可见性（如 '[data-e2e="digg"]' 可见）
|
||||
'marker' — 状态标记匹配（如 liked == False）
|
||||
'marker_diff' — 状态标记变化（如 like_count 增加了）
|
||||
soft: bool = False — 软条件：前置检查失败时不阻断执行（仅记录），用于底层有兜底的操作
|
||||
"""
|
||||
type: str = "page_mode"
|
||||
target: str = "" # 目标字段
|
||||
expected: any = None # 期望值
|
||||
actual: any = None # 运行时填充的实际值
|
||||
passed: bool = False # 是否通过
|
||||
message: str = "" # 检查结果描述
|
||||
soft: bool = False # 软条件（失败不阻断）
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"type": self.type,
|
||||
"target": self.target,
|
||||
"expected": self.expected,
|
||||
"actual": self.actual,
|
||||
"passed": self.passed,
|
||||
"message": self.message,
|
||||
"soft": self.soft,
|
||||
}
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# OpResult — 操作结果（v2.0 加入三段式字段）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
@dataclass
|
||||
class OpResult:
|
||||
"""单个操作的结构化结果（v2.0 新增 before/after 状态和条件检查）
|
||||
|
||||
原有字段不变，新增:
|
||||
before_state: 操作前页面状态指纹
|
||||
after_state: 操作后页面状态指纹
|
||||
pre_conditions: 前置条件检查结果列表
|
||||
post_conditions: 后置验证检查结果列表
|
||||
"""
|
||||
op: str # 操作名 (如 "like", "goto_home")
|
||||
step_id: int = 0 # 蓝图步骤编号
|
||||
success: bool = True
|
||||
detail: str = "" # 简短描述 (如 "👍", "profile_loaded", "no_btn")
|
||||
elapsed: float = 0.0 # 耗时(秒)
|
||||
error: str = "" # 失败原因
|
||||
|
||||
# v2.0 新增字段
|
||||
before_state: Optional[PageState] = None
|
||||
after_state: Optional[PageState] = None
|
||||
pre_conditions: Optional[list] = None
|
||||
post_conditions: Optional[list] = None
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
d = asdict(self)
|
||||
# 将 Optional 字段中的 None 去掉，保持 JSON 简洁
|
||||
if self.pre_conditions is None:
|
||||
d.pop("pre_conditions", None)
|
||||
if self.post_conditions is None:
|
||||
d.pop("post_conditions", None)
|
||||
if self.before_state is None:
|
||||
d.pop("before_state", None)
|
||||
if self.after_state is None:
|
||||
d.pop("after_state", None)
|
||||
return d
|
||||
|
||||
def __str__(self):
|
||||
icon = "✅" if self.success else "❌"
|
||||
extra = f" | {self.error}" if self.error else ""
|
||||
# 如果有条件检查结果，追加概要
|
||||
if self.pre_conditions:
|
||||
failed = [c for c in self.pre_conditions if not c.passed]
|
||||
if failed:
|
||||
extra += f" | pre_fail: {len(failed)}"
|
||||
if self.post_conditions:
|
||||
failed = [c for c in self.post_conditions if not c.passed]
|
||||
if failed:
|
||||
extra += f" | post_fail: {len(failed)}"
|
||||
return f"{icon} [{self.step_id:2d}] {self.op:18s} → {self.detail[:25]} ({self.elapsed:.1f}s){extra}"
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# PlatformOps — 平台操作基类（v2.0 三段式）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
class PlatformOps(ABC):
|
||||
"""平台操作基类（v2.0 三段式）
|
||||
|
||||
子类需要实现:
|
||||
- supported_ops() → list[str]
|
||||
- _do_execute(op, args, step_id) → OpResult (原有，实现动作)
|
||||
- _get_pre_conditions(op) → list[Condition] (新增，前置条件)
|
||||
- _get_post_conditions(op) → list[Condition] (新增，后置验证)
|
||||
|
||||
基类提供:
|
||||
- execute() → 三段式统一入口（子类不需要覆盖）
|
||||
- _capture_state() → 页面状态指纹采集（子类可重写加强）
|
||||
- _check_conditions() → 条件检查引擎
|
||||
"""
|
||||
|
||||
name: str = "base" # 平台标识 (douyin / xiaohongshu)
|
||||
retry_count: int = 1 # 默认重试次数
|
||||
|
||||
# 关键选择器（子类可覆盖，用于状态采集）
|
||||
STATE_SELECTORS: list = []
|
||||
"""状态采集时检查的关键元素选择器列表"""
|
||||
|
||||
# ── 公开接口 ──────────────────────────────────────────────
|
||||
|
||||
def supported_ops(self) -> list[str]:
|
||||
raise NotImplementedError
|
||||
|
||||
async def execute(self, op: str, args: dict = None,
|
||||
step_id: int = 0) -> OpResult:
|
||||
"""三段式执行入口（v2.0）
|
||||
|
||||
流程:
|
||||
1. 采集操作前状态（before_state）
|
||||
2. 检查前置条件（pre_conditions）
|
||||
3. 执行动作（_do_execute）
|
||||
4. 采集操作后状态（after_state）
|
||||
5. 检查后置条件（post_conditions）
|
||||
6. 返回完整 OpResult
|
||||
"""
|
||||
args = args or {}
|
||||
last_err = ""
|
||||
t0_start = time.time()
|
||||
|
||||
# 1. 采集操作前状态
|
||||
before = await self._capture_state()
|
||||
|
||||
# 2. 检查前置条件
|
||||
pre_conds = self._get_pre_conditions(op)
|
||||
pre_results = await self._check_conditions(pre_conds, before)
|
||||
# soft 条件失败不阻断（底层有兜底的操作放行）
|
||||
pre_all_pass = all(c.passed or c.soft for c in pre_results)
|
||||
pre_detail = f"pre:{sum(1 for c in pre_results if c.passed)}/{len(pre_results)}"
|
||||
|
||||
# 前置条件不满足 → 跳过（不是失败）；但全部为软条件失败时也放行
|
||||
if not pre_all_pass:
|
||||
elapsed = round(time.time() - t0_start, 2)
|
||||
result = OpResult(op, step_id, True, f"skipped({pre_detail})",
|
||||
elapsed, before_state=before,
|
||||
pre_conditions=pre_results)
|
||||
return result
|
||||
|
||||
for attempt in range(self.retry_count + 1):
|
||||
t0 = time.time()
|
||||
try:
|
||||
# 3. 执行动作
|
||||
result = await self._do_execute(op, args, step_id)
|
||||
if result is None:
|
||||
result = OpResult(op, step_id, True, "ok", time.time() - t0)
|
||||
result.elapsed = round(time.time() - t0, 2)
|
||||
|
||||
# 4. 采集操作后状态
|
||||
result.after_state = await self._capture_state()
|
||||
|
||||
# 5. 检查后置条件
|
||||
post_conds = self._get_post_conditions(op)
|
||||
result.post_conditions = await self._check_conditions(
|
||||
post_conds, result.after_state, before)
|
||||
post_all_pass = all(c.passed for c in result.post_conditions)
|
||||
post_detail = f"post:{sum(1 for c in result.post_conditions if c.passed)}/{len(result.post_conditions)}"
|
||||
|
||||
# 补充前置条件和 before_state
|
||||
result.pre_conditions = pre_results
|
||||
result.before_state = before
|
||||
|
||||
# 后置条件不满足且还有重试次数 → 重试
|
||||
if not post_all_pass and attempt < self.retry_count:
|
||||
result.success = False
|
||||
result.detail = f"retry({post_detail})"
|
||||
result.error = f"post_condition_failed: {post_detail}"
|
||||
await asyncio.sleep(1)
|
||||
continue
|
||||
|
||||
# 最终判定：后置条件通过或已用完重试
|
||||
result.success = post_all_pass or (attempt >= self.retry_count and result.success)
|
||||
result.detail = post_detail if post_all_pass else result.detail
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
last_err = f"{type(e).__name__}: {e}"
|
||||
if attempt < self.retry_count:
|
||||
await asyncio.sleep(1)
|
||||
|
||||
elapsed = round(time.time() - t0_start, 2)
|
||||
return OpResult(op, step_id, False, "retry_exhausted", elapsed, last_err,
|
||||
before_state=before, pre_conditions=pre_results)
|
||||
|
||||
# ── 状态采集（基类实现，子类可加强）──
|
||||
|
||||
async def _capture_state(self) -> PageState:
|
||||
"""采集当前页面状态指纹
|
||||
|
||||
子类可以重写此方法以加入平台特定的状态标记。
|
||||
基类实现采集:
|
||||
- URL 模式
|
||||
- 页面模式（通过 _detect_page_state）
|
||||
- 关键选择器的可见性
|
||||
- 页面文本片段
|
||||
"""
|
||||
state = PageState()
|
||||
try:
|
||||
page = getattr(self, 'page', None)
|
||||
if page is None:
|
||||
return state
|
||||
|
||||
# URL 模式
|
||||
url = page.url
|
||||
state.url_pattern = self._normalize_url(url)
|
||||
|
||||
# 页面模式（子类通常有 _detect_page_state 方法）
|
||||
detect = getattr(self, '_detect_page_state', None)
|
||||
if detect:
|
||||
state.page_mode = await detect()
|
||||
elif hasattr(self, '_detect_page_mode'):
|
||||
state.page_mode = await self._detect_page_mode()
|
||||
|
||||
# 关键选择器可见性
|
||||
for sel in self.STATE_SELECTORS:
|
||||
try:
|
||||
el = page.locator(sel)
|
||||
state.selectors[sel] = await el.count() > 0 and await el.first.is_visible()
|
||||
except Exception:
|
||||
state.selectors[sel] = False
|
||||
|
||||
# 页面文本片段
|
||||
try:
|
||||
text = await page.evaluate("(document.body.innerText || '').trim().slice(0, 2000)")
|
||||
state.text_snippet = text
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return state
|
||||
|
||||
def _normalize_url(self, url: str) -> str:
|
||||
"""归一化 URL，去掉动态参数，保留路径结构"""
|
||||
import re
|
||||
# 去掉查询参数
|
||||
url = re.sub(r'\?.*$', '', url)
|
||||
# 去掉 /video/ 后面的 ID
|
||||
url = re.sub(r'/video/[A-Za-z0-9]+', '/video/{id}', url)
|
||||
# 去掉 /user/ 后面的 ID
|
||||
url = re.sub(r'/user/[A-Za-z0-9]+', '/user/{id}', url)
|
||||
return url
|
||||
|
||||
# ── 条件检查引擎 ──────────────────────────────────────────
|
||||
|
||||
def _get_pre_conditions(self, op: str) -> list[Condition]:
|
||||
"""每个操作的前置条件——子类覆盖返回
|
||||
|
||||
默认返回空列表（不检查前置条件，兼容旧版）
|
||||
"""
|
||||
return []
|
||||
|
||||
def _get_post_conditions(self, op: str) -> list[Condition]:
|
||||
"""每个操作的后置验证条件——子类覆盖返回
|
||||
|
||||
默认返回空列表（不检查后置条件，兼容旧版）
|
||||
"""
|
||||
return []
|
||||
|
||||
async def _check_conditions(self, conditions: list[Condition],
|
||||
state: PageState,
|
||||
before_state: PageState = None) -> list[Condition]:
|
||||
"""检查一组条件，填充 actual/passed/message 字段"""
|
||||
for cond in conditions:
|
||||
try:
|
||||
if cond.type == 'page_mode':
|
||||
cond.actual = state.page_mode
|
||||
cond.passed = state.page_mode == cond.expected or (
|
||||
cond.expected == 'player' and state.page_mode in ('player', 'player_full', 'player_modal')
|
||||
)
|
||||
cond.message = f"page_mode={state.page_mode}" if cond.passed \
|
||||
else f"期望={cond.expected}, 实际={state.page_mode}"
|
||||
|
||||
elif cond.type == 'selector':
|
||||
visible = state.selectors.get(cond.target, False)
|
||||
cond.actual = visible
|
||||
cond.passed = (visible == cond.expected)
|
||||
cond.message = f"{cond.target} visible={visible}" if cond.passed \
|
||||
else f"期望可见={cond.expected}, 实际={visible}"
|
||||
|
||||
elif cond.type == 'marker':
|
||||
val = state.markers.get(cond.target)
|
||||
cond.actual = val
|
||||
cond.passed = (val == cond.expected)
|
||||
cond.message = f"{cond.target}={val}" if cond.passed \
|
||||
else f"期望={cond.expected}, 实际={val}"
|
||||
|
||||
elif cond.type == 'marker_diff' and before_state:
|
||||
before_val = before_state.markers.get(cond.target)
|
||||
after_val = state.markers.get(cond.target)
|
||||
cond.actual = (before_val, after_val)
|
||||
# marker_diff: expected 可以是 "increased"/"decreased"/"changed"
|
||||
if cond.expected == "increased":
|
||||
cond.passed = (isinstance(before_val, (int, float))
|
||||
and isinstance(after_val, (int, float))
|
||||
and after_val > before_val)
|
||||
elif cond.expected == "decreased":
|
||||
cond.passed = (isinstance(before_val, (int, float))
|
||||
and isinstance(after_val, (int, float))
|
||||
and after_val < before_val)
|
||||
else:
|
||||
cond.passed = (before_val != after_val)
|
||||
cond.message = f"{cond.target}: {before_val} → {after_val}"
|
||||
|
||||
else:
|
||||
cond.passed = True
|
||||
cond.message = f"unknown_condition({cond.type})"
|
||||
|
||||
except Exception as e:
|
||||
cond.passed = False
|
||||
cond.message = f"检查异常: {e}"
|
||||
|
||||
return conditions
|
||||
|
||||
# ── 子类实现 ──────────────────────────────────────────────
|
||||
|
||||
@abstractmethod
|
||||
async def _do_execute(self, op: str, args: dict, step_id: int) -> Optional[OpResult]:
|
||||
"""子类实现具体的操作分发。返回 OpResult 或 None（None=成功，无额外信息）
|
||||
|
||||
注意：execute() 方法在 _do_execute 之前/之后自动采集状态，
|
||||
所以 _do_execute 只需要做「动作」部分，不需要管条件检查。
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,655 @@
|
||||
"""
|
||||
ops/xhs_ops.py — 小红书平台操作 v1.0
|
||||
从 mc/engine.py (v4.3) 提取，实现 PlatformOps 接口。
|
||||
|
||||
操作清单 (16个):
|
||||
导航: xhs_goto_home, xhs_browse
|
||||
互动: xhs_like, xhs_collect, xhs_comment, xhs_follow
|
||||
内容: xhs_click_note, xhs_scroll_feed, xhs_search
|
||||
主页: xhs_goto_profile, xhs_read_nickname, xhs_read_user_id,
|
||||
xhs_read_following, xhs_read_fans, xhs_read_likes, xhs_read_bio
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from ops._base import PlatformOps, OpResult, Condition
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
HOME = Path.home()
|
||||
PROFILES_JSON = HOME / "workbuddy-agent-os" / "agent-local" / "tools" / "matrix" / "data" / "profiles.json"
|
||||
XHS_HOME_URL = "https://www.xiaohongshu.com/explore"
|
||||
|
||||
|
||||
class XhsOps(PlatformOps):
|
||||
"""小红书平台操作库"""
|
||||
|
||||
name = "xiaohongshu"
|
||||
|
||||
def __init__(self, page):
|
||||
self.page = page
|
||||
self._profile_cache: dict = {} # goto_profile 后缓存
|
||||
self._account_id: str = "" # 由 engine 设置，写 profiles.json 用
|
||||
|
||||
def supported_ops(self) -> list[str]:
|
||||
return [
|
||||
"xhs_goto_home", "xhs_browse", "xhs_scroll_feed", "xhs_click_note",
|
||||
"xhs_click_next_image", "xhs_click_prev_image",
|
||||
"xhs_like", "xhs_collect", "xhs_comment", "xhs_follow", "xhs_search",
|
||||
"xhs_post_comment",
|
||||
"xhs_goto_profile",
|
||||
"xhs_read_nickname", "xhs_read_user_id", "xhs_read_following",
|
||||
"xhs_read_fans", "xhs_read_likes", "xhs_read_bio",
|
||||
"wait_watch", "go_back", "goto_url",
|
||||
]
|
||||
|
||||
def set_account_id(self, account_id: str):
|
||||
"""设置当前账号ID，供写 profiles.json"""
|
||||
self._account_id = account_id
|
||||
|
||||
# ── 三段式操作模型 v2.0 ──────────────────────────────────
|
||||
|
||||
STATE_SELECTORS = [
|
||||
'section.note-item', # feed 中的笔记
|
||||
'span.like-lottie', # 点赞按钮
|
||||
'[class*="collect"]', # 收藏
|
||||
'[class*="follow"]', # 关注
|
||||
'input[placeholder*="搜索"]', # 搜索框
|
||||
'textarea.textarea', # 小红书搜索输入框
|
||||
'p#content-textarea', # 评论输入框（有ID，稳定）
|
||||
'button.btn.submit', # 发评论按钮
|
||||
'img.preview-interactive', # 图片翻页
|
||||
'img.avatar-item', # 头像
|
||||
'a.cover.mask', # 笔记封面链接
|
||||
'div.continue', # 登录继续按钮
|
||||
'input#verificationCodeInput', # 验证码输入框（有ID）
|
||||
'span.login-text', # 登录按钮
|
||||
'div.foot-btn', # 同意并登录
|
||||
'.reds-count', # 登录态指示器
|
||||
'.login-container', # 登录面板
|
||||
'a.bottom-channel', # 底栏导航
|
||||
]
|
||||
|
||||
def _get_pre_conditions(self, op: str) -> list[Condition]:
|
||||
conds = {
|
||||
"xhs_like": [
|
||||
Condition("page_mode", "page_mode", "player", message="需要在笔记详情页"),
|
||||
],
|
||||
"xhs_collect": [
|
||||
Condition("page_mode", "page_mode", "player", message="需要在笔记详情页"),
|
||||
],
|
||||
"xhs_comment": [
|
||||
Condition("page_mode", "page_mode", "player", message="需要在笔记详情页"),
|
||||
],
|
||||
"xhs_follow": [
|
||||
Condition("page_mode", "page_mode", "profile", message="需要在用户主页"),
|
||||
],
|
||||
"xhs_scroll_feed": [
|
||||
Condition("page_mode", "page_mode", "grid", message="需要在发现页"),
|
||||
],
|
||||
"xhs_click_note": [
|
||||
Condition("page_mode", "page_mode", "grid", message="需要在发现页"),
|
||||
],
|
||||
"xhs_search": [
|
||||
Condition("selector", 'input[placeholder*="搜索"]', True,
|
||||
message="需要搜索框可见"),
|
||||
],
|
||||
"xhs_click_next_image": [
|
||||
Condition("page_mode", "page_mode", "player", message="需要在笔记详情页"),
|
||||
],
|
||||
"xhs_click_prev_image": [
|
||||
Condition("page_mode", "page_mode", "player", message="需要在笔记详情页"),
|
||||
],
|
||||
"xhs_post_comment": [
|
||||
Condition("page_mode", "page_mode", "player", message="需要在笔记详情页"),
|
||||
],
|
||||
"xhs_goto_home": [],
|
||||
"xhs_browse": [],
|
||||
"xhs_goto_profile": [],
|
||||
"xhs_read_nickname": [],
|
||||
"xhs_read_user_id": [],
|
||||
"xhs_read_following": [],
|
||||
"xhs_read_fans": [],
|
||||
"xhs_read_likes": [],
|
||||
"xhs_read_bio": [],
|
||||
"wait_watch": [],
|
||||
"go_back": [],
|
||||
}
|
||||
return conds.get(op, [])
|
||||
|
||||
def _get_post_conditions(self, op: str) -> list[Condition]:
|
||||
conds = {
|
||||
"xhs_goto_home": [
|
||||
Condition("page_mode", "page_mode", "grid", message="应回到发现页"),
|
||||
],
|
||||
"xhs_like": [
|
||||
Condition("selector", '[class*="like"]', True,
|
||||
message="点赞按钮应仍在页面"),
|
||||
],
|
||||
}
|
||||
return conds.get(op, [])
|
||||
|
||||
# ── PlatformOps 入口 ────────────────────────────────────
|
||||
|
||||
async def _do_execute(self, op: str, args: dict, step_id: int) -> Optional[OpResult]:
|
||||
method = getattr(self, op, None)
|
||||
if method is None:
|
||||
return OpResult(op, step_id, False, "unsupported", 0, f"未知操作: {op}")
|
||||
return await method(args, step_id)
|
||||
|
||||
# ═══════════════════════════════════════════════════════
|
||||
# 导航类
|
||||
# ═══════════════════════════════════════════════════════
|
||||
|
||||
async def xhs_goto_home(self, args: dict, step_id: int) -> OpResult:
|
||||
await self.page.goto(XHS_HOME_URL, timeout=20000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(4)
|
||||
return OpResult("xhs_goto_home", step_id, True, "home_loaded")
|
||||
|
||||
async def xhs_browse(self, args: dict, step_id: int) -> OpResult:
|
||||
if "explore" not in self.page.url:
|
||||
await self.page.goto(XHS_HOME_URL, timeout=15000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(3)
|
||||
return OpResult("xhs_browse", step_id, True, "browsing")
|
||||
|
||||
async def xhs_scroll_feed(self, args: dict, step_id: int) -> OpResult:
|
||||
await self.page.evaluate("() => window.scrollBy(0, 800)")
|
||||
await asyncio.sleep(2)
|
||||
return OpResult("xhs_scroll_feed", step_id, True, "scrolled")
|
||||
|
||||
async def xhs_click_note(self, args: dict, step_id: int) -> OpResult:
|
||||
# 策略1: a.cover 点击（录制确认最稳定）
|
||||
try:
|
||||
cover = self.page.locator('a.cover.mask, a.cover')
|
||||
if await cover.count() > 0:
|
||||
await cover.first.click()
|
||||
await asyncio.sleep(4)
|
||||
return OpResult("xhs_click_note", step_id, True, "note_opened")
|
||||
except:
|
||||
pass
|
||||
# 策略2: 通用选择器（原有）
|
||||
note = self.page.locator('section.note-item, a[href*="/explore/"], [class*="note-item"]').first
|
||||
if await note.count() > 0:
|
||||
await note.click()
|
||||
await asyncio.sleep(4)
|
||||
return OpResult("xhs_click_note", step_id, True, "note_opened")
|
||||
# 回到首页重试
|
||||
await self.page.goto("https://www.xiaohongshu.com/explore", timeout=15000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(3)
|
||||
note2 = self.page.locator('section.note-item, a[href*="/explore/"], [class*="note-item"]').first
|
||||
if await note2.count() > 0:
|
||||
await note2.click()
|
||||
await asyncio.sleep(4)
|
||||
return OpResult("xhs_click_note", step_id, True, "note_opened_retry")
|
||||
return OpResult("xhs_click_note", step_id, False, "no_note")
|
||||
|
||||
async def xhs_click_next_image(self, args: dict, step_id: int) -> OpResult:
|
||||
"""图片笔记向右翻页 — 点击右侧SVG区域（两次录制确认 (677,268)）"""
|
||||
# 策略1: 坐标点击（录制确认右侧 ~(677,268)）
|
||||
try:
|
||||
await self.page.mouse.click(677, 268)
|
||||
await asyncio.sleep(1.5)
|
||||
return OpResult("xhs_click_next_image", step_id, True, "next")
|
||||
except:
|
||||
pass
|
||||
# 策略2: img.preview-interactive 点击（在预览模式中翻页）
|
||||
try:
|
||||
pi = self.page.locator('img.preview-interactive')
|
||||
if await pi.count() > 0:
|
||||
await pi.first.click()
|
||||
await asyncio.sleep(1.5)
|
||||
return OpResult("xhs_click_next_image", step_id, True, "next")
|
||||
except:
|
||||
pass
|
||||
return OpResult("xhs_click_next_image", step_id, False, "no_next_btn")
|
||||
|
||||
async def xhs_click_prev_image(self, args: dict, step_id: int) -> OpResult:
|
||||
"""图片笔记向左翻页 — 点击左侧SVG区域（录制确认 (28,274)）"""
|
||||
# 策略1: 坐标点击（录制确认左侧 ~(28,274)）
|
||||
try:
|
||||
await self.page.mouse.click(28, 274)
|
||||
await asyncio.sleep(1.5)
|
||||
return OpResult("xhs_click_prev_image", step_id, True, "prev")
|
||||
except:
|
||||
pass
|
||||
# 策略2: 备用坐标（略右移）
|
||||
try:
|
||||
await self.page.mouse.click(70, 274)
|
||||
await asyncio.sleep(1.5)
|
||||
return OpResult("xhs_click_prev_image", step_id, True, "prev")
|
||||
except:
|
||||
pass
|
||||
return OpResult("xhs_click_prev_image", step_id, False, "no_prev_btn")
|
||||
|
||||
# ═══════════════════════════════════════════════════════
|
||||
# 互动类
|
||||
# ═══════════════════════════════════════════════════════
|
||||
|
||||
async def xhs_like(self, args: dict, step_id: int) -> OpResult:
|
||||
"""点赞 — 点 SPAN.like-lottie（force=True 绕过可见性检查）"""
|
||||
try:
|
||||
btn = self.page.locator('span.like-lottie')
|
||||
if await btn.count() > 0:
|
||||
await btn.first.click(force=True)
|
||||
await asyncio.sleep(1)
|
||||
return OpResult("xhs_like", step_id, True, "👍")
|
||||
except:
|
||||
pass
|
||||
# 快速兜底
|
||||
r = await self.page.evaluate("""() => {
|
||||
const all = document.querySelectorAll('span, button, div');
|
||||
for (const el of all) {
|
||||
const c = (el.className || '') + (el.textContent || '');
|
||||
if ((c.includes('like') || c.includes('赞')) && el.offsetParent) {
|
||||
if (c.includes('collect') || c.includes('save')) continue;
|
||||
el.click(); return '👍';
|
||||
}
|
||||
}
|
||||
return '-';
|
||||
}""")
|
||||
await asyncio.sleep(1)
|
||||
return OpResult("xhs_like", step_id, r == "👍", r)
|
||||
|
||||
async def xhs_collect(self, args: dict, step_id: int) -> OpResult:
|
||||
"""收藏 — 底栏第二位（点赞在左，收藏在点赞+45px处）"""
|
||||
# 策略1: 找点赞按钮位置，点击右侧45px（两次录制确认）
|
||||
try:
|
||||
like = self.page.locator('span.like-lottie')
|
||||
if await like.count() > 0:
|
||||
box = await like.first.bounding_box()
|
||||
if box:
|
||||
await self.page.mouse.click(box['x'] + 45, box['y'])
|
||||
await asyncio.sleep(1)
|
||||
return OpResult("xhs_collect", step_id, True, "⭐")
|
||||
except:
|
||||
pass
|
||||
# 策略2: SVG 兜底（录制显示收藏是 svg 标签）
|
||||
try:
|
||||
likes = self.page.locator('span.like-lottie')
|
||||
if await likes.count() > 0:
|
||||
svgs = self.page.locator('span.like-lottie ~ svg, span.like-lottie + svg')
|
||||
if await svgs.count() > 0:
|
||||
await svgs.first.click(force=True)
|
||||
await asyncio.sleep(1)
|
||||
return OpResult("xhs_collect", step_id, True, "⭐svg")
|
||||
except:
|
||||
pass
|
||||
# 策略3: JS查找收藏class
|
||||
r = await self.page.evaluate("""() => {
|
||||
const btns = document.querySelectorAll('[class*="collect"],[class*="save"]');
|
||||
for (const b of btns) {
|
||||
if (b.offsetParent !== null) { b.click(); return '⭐'; }
|
||||
}
|
||||
return '-';
|
||||
}""")
|
||||
await asyncio.sleep(1)
|
||||
return OpResult("xhs_collect", step_id, r == "⭐", r)
|
||||
|
||||
async def xhs_comment(self, args: dict, step_id: int) -> OpResult:
|
||||
await self.page.keyboard.press("x")
|
||||
await asyncio.sleep(2)
|
||||
return OpResult("xhs_comment", step_id, True, "comment_opened")
|
||||
|
||||
async def xhs_follow(self, args: dict, step_id: int) -> OpResult:
|
||||
"""关注 — 点 SPAN.reds-button-new-text（基于录制）"""
|
||||
try:
|
||||
btn = self.page.locator('span.reds-button-new-text')
|
||||
if await btn.count() > 0:
|
||||
await btn.first.click(force=True)
|
||||
await asyncio.sleep(2)
|
||||
return OpResult("xhs_follow", step_id, True, "✅")
|
||||
except:
|
||||
pass
|
||||
# 兜底
|
||||
r = await self.page.evaluate("""() => {
|
||||
const btns = document.querySelectorAll('button, span');
|
||||
for (const b of btns) {
|
||||
const t = (b.textContent || '').trim();
|
||||
if (t.includes('关注') && !t.includes('已关注') && b.offsetParent) { b.click(); return '✅'; }
|
||||
}
|
||||
return '-';
|
||||
}""")
|
||||
await asyncio.sleep(2)
|
||||
return OpResult("xhs_follow", step_id, r == "✅", r)
|
||||
|
||||
async def xhs_post_comment(self, args: dict, step_id: int) -> OpResult:
|
||||
"""小红书发评论 — 优先用ID选择器 p#content-textarea，再 fallback"""
|
||||
text = args.get("text", "")
|
||||
if not text or text == "@corpus":
|
||||
try:
|
||||
from mc.corpus import CorpusManager
|
||||
cm = CorpusManager()
|
||||
picks = cm.get_comments(platform="xiaohongshu", count=5)
|
||||
if picks:
|
||||
text = random.choice(picks)
|
||||
else:
|
||||
text = "好实用呀"
|
||||
except Exception:
|
||||
text = "好实用呀"
|
||||
try:
|
||||
# Step 1: 滑到底部（评论区在底部）
|
||||
await self.page.evaluate("() => window.scrollTo(0, document.body.scrollHeight)")
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# Step 2: 优先用 ID 选择器 p#content-textarea（录制确认稳定）
|
||||
filled = False
|
||||
try:
|
||||
inp = self.page.locator('p#content-textarea')
|
||||
if await inp.count() > 0 and await inp.first.is_visible():
|
||||
await inp.first.click()
|
||||
await asyncio.sleep(0.5)
|
||||
proc = await asyncio.create_subprocess_exec("pbcopy", stdin=asyncio.subprocess.PIPE)
|
||||
await proc.communicate(input=text.encode())
|
||||
await asyncio.sleep(0.5)
|
||||
await self.page.keyboard.press("Meta+V")
|
||||
await asyncio.sleep(1.5)
|
||||
filled = True
|
||||
except:
|
||||
pass
|
||||
|
||||
# Fallback: JS 填字（原有的）
|
||||
if not filled:
|
||||
filled = await self.page.evaluate(f"""() => {{
|
||||
const all = document.querySelectorAll(
|
||||
'textarea, input[type="text"], [contenteditable="true"], ' +
|
||||
'[class*="input"], [class*="editor"], [class*="textarea"]'
|
||||
);
|
||||
for (const el of all) {{
|
||||
if (el.offsetParent !== null) {{
|
||||
el.focus();
|
||||
el.click();
|
||||
const sel = window.getSelection();
|
||||
const range = document.createRange();
|
||||
range.selectNodeContents(el);
|
||||
sel.removeAllRanges();
|
||||
sel.addRange(range);
|
||||
document.execCommand('insertText', false, '{text}');
|
||||
el.dispatchEvent(new Event('input', {{ bubbles: true }}));
|
||||
return true;
|
||||
}}
|
||||
}}
|
||||
return false;
|
||||
}}""")
|
||||
await asyncio.sleep(1)
|
||||
|
||||
# Step 3: 如果JS填字失败，pbcopy 兜底
|
||||
if not filled:
|
||||
proc = await asyncio.create_subprocess_exec("pbcopy", stdin=asyncio.subprocess.PIPE)
|
||||
await proc.communicate(input=text.encode())
|
||||
await asyncio.sleep(0.5)
|
||||
await self.page.keyboard.press("Meta+V")
|
||||
await asyncio.sleep(1.5)
|
||||
|
||||
# Step 4: 找发送按钮（找包含"发送"文本/class含submit的按钮）
|
||||
sent = await self.page.evaluate("""() => {
|
||||
const btns = document.querySelectorAll('button, [class*="submit"], [class*="send"]');
|
||||
for (const b of btns) {
|
||||
const t = (b.textContent || '').trim();
|
||||
if ((t.includes('发送') || t.includes('发布') || b.className.includes('submit'))
|
||||
&& b.offsetParent) {
|
||||
b.click(); return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}""")
|
||||
if not sent:
|
||||
# 没找到发送按钮 → Enter发送
|
||||
await self.page.keyboard.press("Enter")
|
||||
|
||||
await asyncio.sleep(2)
|
||||
return OpResult("xhs_post_comment", step_id, True, f"👍评论({text[:10]})")
|
||||
except Exception as e:
|
||||
return OpResult("xhs_post_comment", step_id, False, "评论失败", error=str(e))
|
||||
|
||||
async def xhs_search(self, args: dict, step_id: int) -> OpResult:
|
||||
kw = args.get("keyword", "热门推荐")
|
||||
if kw == "@random":
|
||||
import random as _r
|
||||
kw = _r.choice(["穿搭推荐","美食日常","旅行攻略","化妆教程","家居好物",
|
||||
"宠物日常","电影推荐","读书分享","健身打卡","摄影技巧"])
|
||||
try:
|
||||
# 小红书搜索栏是 TEXTAREA.textarea（非 input）
|
||||
search_box = self.page.locator('textarea.textarea')
|
||||
if await search_box.count() > 0:
|
||||
await search_box.first.click()
|
||||
await asyncio.sleep(0.5)
|
||||
await search_box.first.fill(kw)
|
||||
await asyncio.sleep(1)
|
||||
await self.page.keyboard.press("Enter")
|
||||
await asyncio.sleep(3)
|
||||
return OpResult("xhs_search", step_id, True, f"searched({kw[:10]})")
|
||||
except:
|
||||
pass
|
||||
# 兜底
|
||||
try:
|
||||
await self.page.evaluate(
|
||||
"(k) => { const i = document.querySelector('input,textarea'); if(i) { i.value=k; i.dispatchEvent(new Event('input')); } }",
|
||||
kw
|
||||
)
|
||||
await asyncio.sleep(1)
|
||||
await self.page.keyboard.press("Enter")
|
||||
await asyncio.sleep(3)
|
||||
return OpResult("xhs_search", step_id, True, f"searched({kw[:10]})")
|
||||
except Exception as e:
|
||||
return OpResult("xhs_search", step_id, False, "search_failed", error=str(e))
|
||||
|
||||
# ═══════════════════════════════════════════════════════
|
||||
# 主页信息采集
|
||||
# ═══════════════════════════════════════════════════════
|
||||
|
||||
async def xhs_goto_profile(self, args: dict, step_id: int) -> OpResult:
|
||||
"""导航到个人主页并提取全部字段到 profile_cache"""
|
||||
# 1. 从页面找 profile URL
|
||||
profile_url = await self.page.evaluate("""() => {
|
||||
const links = document.querySelectorAll('a');
|
||||
for (const a of links) {
|
||||
const href = a.href || '';
|
||||
if (href.includes('/user/profile/')) return href;
|
||||
}
|
||||
return '';
|
||||
}""")
|
||||
target = profile_url or "https://www.xiaohongshu.com/user/profile"
|
||||
await self.page.goto(target, timeout=20000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(5)
|
||||
|
||||
# 2. 统一提取全部字段
|
||||
profile = await self.page.evaluate("""() => {
|
||||
const text = (document.body.innerText || '').trim();
|
||||
const title = (document.title || '').replace(' - 小红书', '').trim();
|
||||
|
||||
const uidM = text.match(/小红书号[：:]\\s*(\\d+)/);
|
||||
const folM = text.match(/(\\d+(?:\\.\\d+)?[万wW]?)\\s*关注/);
|
||||
const fanM = text.match(/(\\d+(?:\\.\\d+)?[万wW]?)\\s*粉丝/);
|
||||
const likM = text.match(/(\\d+(?:\\.\\d+)?[万wW]?)\\s*获赞与收藏/);
|
||||
const posM = text.match(/笔记[・·](\\d+(?:\\.\\d+)?[万wW]?)/);
|
||||
|
||||
let bio = '(无)';
|
||||
if (text.includes('还没有简介') || text.includes('暂无简介')) {
|
||||
bio = '(无)';
|
||||
} else {
|
||||
const bioM = text.match(/IP属地[：:].+?\\n(.+?)(?=\\d+关注|\\d+粉丝)/);
|
||||
if (bioM) bio = bioM[1].trim().slice(0, 50) || '(无)';
|
||||
}
|
||||
|
||||
return { nickname: title, user_id: uidM ? uidM[1] : '?',
|
||||
following: folM ? folM[1] : '?', fans: fanM ? fanM[1] : '?',
|
||||
likes: likM ? likM[1] : '?', posts: posM ? posM[1] : '?',
|
||||
bio: bio };
|
||||
}""")
|
||||
|
||||
self._profile_cache = profile
|
||||
|
||||
# 2.5 检测封禁状态
|
||||
ban_status = await self.check_banned_status()
|
||||
profile["_status"] = ban_status
|
||||
log.info(
|
||||
f" 📊 小红书主页: {profile.get('nickname','?')}"
|
||||
f" ID={profile.get('user_id','?')}"
|
||||
f" 关注={profile.get('following','?')}"
|
||||
f" 粉丝={profile.get('fans','?')}"
|
||||
f" 获赞={profile.get('likes','?')}"
|
||||
f" 笔记={profile.get('posts','?')}"
|
||||
f" 状态={ban_status}"
|
||||
)
|
||||
|
||||
# 3. 保存到 profiles.json
|
||||
self._save_profiles_json()
|
||||
|
||||
return OpResult("xhs_goto_profile", step_id, True, "profile_loaded")
|
||||
|
||||
# ═══════════════════════════════════════════════════════
|
||||
# 读缓存字段
|
||||
# ═══════════════════════════════════════════════════════
|
||||
|
||||
async def xhs_read_nickname(self, args: dict, step_id: int) -> OpResult:
|
||||
v = self._profile_cache.get("nickname", "?")
|
||||
log.info(f" 📝 昵称: {v} (cached)")
|
||||
return OpResult("xhs_read_nickname", step_id, True, f"nickname={v}")
|
||||
|
||||
async def xhs_read_user_id(self, args: dict, step_id: int) -> OpResult:
|
||||
v = self._profile_cache.get("user_id", "?")
|
||||
log.info(f" 🔢 ID: {v} (cached)")
|
||||
return OpResult("xhs_read_user_id", step_id, True, f"user_id={v}")
|
||||
|
||||
async def xhs_read_following(self, args: dict, step_id: int) -> OpResult:
|
||||
v = self._profile_cache.get("following", "?")
|
||||
log.info(f" 👥 关注: {v} (cached)")
|
||||
return OpResult("xhs_read_following", step_id, True, f"following={v}")
|
||||
|
||||
async def xhs_read_fans(self, args: dict, step_id: int) -> OpResult:
|
||||
v = self._profile_cache.get("fans", "?")
|
||||
log.info(f" 👥 粉丝: {v} (cached)")
|
||||
return OpResult("xhs_read_fans", step_id, True, f"fans={v}")
|
||||
|
||||
async def xhs_read_likes(self, args: dict, step_id: int) -> OpResult:
|
||||
v = self._profile_cache.get("likes", "?")
|
||||
log.info(f" 👍 获赞: {v} (cached)")
|
||||
return OpResult("xhs_read_likes", step_id, True, f"likes={v}")
|
||||
|
||||
async def xhs_read_bio(self, args: dict, step_id: int) -> OpResult:
|
||||
v = self._profile_cache.get("bio", "?")
|
||||
log.info(f" 📄 简介: {v} (cached)")
|
||||
return OpResult("xhs_read_bio", step_id, True, f"bio={v}")
|
||||
|
||||
# ═══════════════════════════════════════════════════════
|
||||
# 内部 + 通用操作
|
||||
# ═══════════════════════════════════════════════════════
|
||||
|
||||
async def wait_watch(self, args: dict, step_id: int) -> OpResult:
|
||||
"""等待观看（随机5-15秒）"""
|
||||
import random
|
||||
seconds = args.get("seconds") or random.randint(5, 15)
|
||||
await asyncio.sleep(seconds)
|
||||
return OpResult("wait_watch", step_id, True, f"watched_{seconds}s")
|
||||
|
||||
async def go_back(self, args: dict, step_id: int) -> OpResult:
|
||||
"""返回上一页"""
|
||||
try:
|
||||
await self.page.go_back(wait_until="domcontentloaded")
|
||||
await asyncio.sleep(2)
|
||||
return OpResult("go_back", step_id, True, "back")
|
||||
except Exception as e:
|
||||
return OpResult("go_back", step_id, True, "back_fallback")
|
||||
|
||||
async def goto_url(self, args: dict, step_id: int) -> OpResult:
|
||||
"""跳转到指定URL（定向评论用）"""
|
||||
url = args.get("url", XHS_HOME_URL)
|
||||
try:
|
||||
await self.page.goto(url, timeout=20000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(4)
|
||||
return OpResult("goto_url", step_id, True, f"goto {url[:30]}")
|
||||
except Exception as e:
|
||||
return OpResult("goto_url", step_id, False, "goto_fail", error=str(e))
|
||||
|
||||
async def check_banned_status(self) -> str:
|
||||
"""检测小红书账号是否被封禁
|
||||
Returns: "normal" / "banned" / "unknown"
|
||||
"""
|
||||
try:
|
||||
text = (await self.page.evaluate("document.body.innerText")) or ""
|
||||
# 个人主页：含有"违反"+"社区规范"关键词
|
||||
if "违反" in text and "社区规范" in text:
|
||||
return "banned"
|
||||
# 操作弹窗：div.banned-title 可见
|
||||
try:
|
||||
bt = self.page.locator('div.banned-title')
|
||||
if await bt.count() > 0 and await bt.first.is_visible():
|
||||
return "banned"
|
||||
except:
|
||||
pass
|
||||
except:
|
||||
pass
|
||||
return "normal"
|
||||
|
||||
def _save_profiles_json(self):
|
||||
"""保存主页信息到 profiles.json + homepage_info.json（两个 Dashboard 数据源）"""
|
||||
if not self._account_id:
|
||||
return
|
||||
try:
|
||||
cache = self._profile_cache
|
||||
status = cache.get("_status", "normal")
|
||||
|
||||
# ── 写 profiles.json（供账号管理页面读取）──
|
||||
if PROFILES_JSON.exists():
|
||||
all_p = json.loads(PROFILES_JSON.read_text())
|
||||
else:
|
||||
all_p = {}
|
||||
PROFILES_JSON.parent.mkdir(parents=True, exist_ok=True)
|
||||
all_p[self._account_id] = {
|
||||
"nickname": cache.get("nickname", "?"),
|
||||
"user_id": cache.get("user_id", "?"),
|
||||
"following": cache.get("following", "?"),
|
||||
"fans": cache.get("fans", "?"),
|
||||
"likes": cache.get("likes", "?"),
|
||||
"posts": cache.get("posts", "?"),
|
||||
"bio": cache.get("bio", "?"),
|
||||
"status": status,
|
||||
"platform": "xiaohongshu",
|
||||
"updated": datetime.now().isoformat(),
|
||||
}
|
||||
PROFILES_JSON.write_text(json.dumps(all_p, ensure_ascii=False, indent=2))
|
||||
|
||||
# ── 同步写 homepage_info.json（供信息采集页面读取）──
|
||||
hp_path = PROFILES_JSON.parent / "homepage_info.json"
|
||||
hp_data = {"collected_at": datetime.now().isoformat(), "results": []}
|
||||
if hp_path.exists():
|
||||
try:
|
||||
hp_data = json.loads(hp_path.read_text())
|
||||
except Exception:
|
||||
hp_data = {"collected_at": datetime.now().isoformat(), "results": []}
|
||||
# 查找或创建当前账号的记录
|
||||
entry = None
|
||||
for r in hp_data.get("results", []):
|
||||
rid = r.get("identity_dir", "").replace("identities/", "")
|
||||
if rid == self._account_id or r.get("phone") == cache.get("phone", self._account_id):
|
||||
entry = r
|
||||
break
|
||||
if not entry:
|
||||
hp_data.setdefault("results", []).append({
|
||||
"identity_dir": f"identities/{self._account_id}",
|
||||
"phone": cache.get("phone", self._account_id),
|
||||
"douyin": None, "xiaohongshu": {}
|
||||
})
|
||||
entry = hp_data["results"][-1]
|
||||
# 确保 xiaohongshu 字段存在
|
||||
if entry.get("xiaohongshu") is None:
|
||||
entry["xiaohongshu"] = {}
|
||||
entry["xiaohongshu"].update({
|
||||
"nickname": cache.get("nickname", ""),
|
||||
"fans": cache.get("fans", "0"),
|
||||
"following": cache.get("following", "0"),
|
||||
"likes": cache.get("likes", "0"),
|
||||
"notes": cache.get("posts", "0"),
|
||||
"bio": cache.get("bio", ""),
|
||||
"status": status,
|
||||
"updated": datetime.now().isoformat(),
|
||||
})
|
||||
hp_path.write_text(json.dumps(hp_data, ensure_ascii=False, indent=2))
|
||||
|
||||
except Exception as e:
|
||||
import logging
|
||||
logging.getLogger(__name__).warning(f"_save_profiles_json 失败: {e}")
|
||||
Reference in New Issue
Block a user