feat(monitor): 抖音 Web 接口客户端 —— 绕开爬虫子进程,直接发 HTTP
移植自 mac-agent-os 的 mediacrawler_adapter:不起子进程、不开页面,用浏览器里那份
登录态直接调抖音 Web 接口。产物键名照抄 store/douyin,所以 ingest 那条链路一个字不用改。
**目前能用的(真环境实测,非推断)**:
profile/other : 200, 7075 字节 —— 博主主页指标(粉丝/获赞/作品数/昵称)
aweme/detail : 200, 45425 字节 —— 单条作品详情(含点赞/评论/收藏/分享)
**目前不能用的:作品列表 `aweme/post`。** 两个互相独立的原因:
1. 这个接口被抖音单独升级成了真校验:不带 x-tt-argus 回 403「Uifid Not Found」,
带上 dummy 值回 200 + **空 body**。也就是说「头在不在」骗得过,「真校验」过不了。
同一套头打 profile/other 和 aweme/detail 都是通的 —— 抖音是挑着接口加保护的,
挑中的恰好是「批量拉作品列表」这个最敏感的动作。
2. 改走页面截获也不行:CDP 浏览器打开博主主页会落到「验证码中间页」(当天大量探测的
代价,过几小时要重测)。
所以现在的边界是:**已知作品的指标刷新能做,自动发现新作品做不了**。
**排查中控住变量后得到的两条事实**(都写进注释了):
· `Accept` / `Accept-Language` / `Referer` 才是主页接口能返回真数据的原因 —— 只有
UA+client hints+Cookie 时是 200 但仅 121 字节的空壳,补上这三个头变 7074 字节。
(我先前猜的 sec-ch-ua 不是关键。)
· 因此 UA 与 client hints 必须**成套地取自同一个浏览器**,所以 BrowserIdentity 一次
从 CDP 取齐 cookie + UA + hints,而不是各自写死。
「200 + 空 body 必须当场报错」也是刻意写死的:放过去它会在下游变成「这个博主没作品」,
把一次失败伪装成一条正常结果 —— 爬虫那条路正是这么栽的,还被翻译成「账号被封」。
测试 +11:cookie 解析、请求头成套性(含 uifid 缺失/回退)、产物键名与 store 对齐、
以及 _get 的三条失败路径(空 body / 403 带网关原话 / 正常返回)。
This commit is contained in:
@@ -0,0 +1,168 @@
|
||||
# AgentOS 原子操作架构审计与重构方案
|
||||
|
||||
## 一、旧代码设计原则(提炼自旧版 runner.py / interact.py)
|
||||
|
||||
### 1.1 每步操作 = 状态机三要素
|
||||
|
||||
```
|
||||
[前置状态检测] → [操作执行] → [后置状态验证]
|
||||
↓ ↓ ↓
|
||||
当前在哪个页面? 执行操作 操作是否真的生效了?
|
||||
```
|
||||
|
||||
**旧代码实现**(`_ensure_player_state`、`CommentStateMachine`):
|
||||
```python
|
||||
# 前置检测
|
||||
state = await _detect_page_state(page)
|
||||
# 只有确认在播放页才执行操作
|
||||
if state not in ("player", "search_player"):
|
||||
recovered = await _ensure_player_state(...)
|
||||
# 执行
|
||||
await page.keyboard.press("KeyZ")
|
||||
# 后置验证
|
||||
assert await _check_anchor(page, 'video_page')
|
||||
```
|
||||
|
||||
### 1.2 锚点系统(`_check_anchor`)
|
||||
|
||||
| 锚点 | 检测方式 | 作用 |
|
||||
|:-----|:---------|:-----|
|
||||
| `video_page` | `video` 元素 > 0 | 确认在视频播放页 |
|
||||
| `home_page` | `.discover-video-card-item` > 0 | 确认在首页feed |
|
||||
| `video_playing` | `!video.paused && readyState > 0` | 确认视频在播放 |
|
||||
| `has_videos` | `a[href*="/video/"]` > 0 | 页面有视频链接 |
|
||||
|
||||
### 1.3 评论区状态机(`CommentStateMachine`)
|
||||
|
||||
```
|
||||
closed → panel_open → input_focused → text_entered → sent
|
||||
↓ ↓ ↓
|
||||
键盘x打开 检测activeElement 检测textContent
|
||||
验证has_panel 验证isEditable 验证has_text
|
||||
```
|
||||
|
||||
**核心:每一步都调用 `detect()` 验证状态是否真的变化了。**
|
||||
|
||||
### 1.4 行为链(`CHAINS`)
|
||||
|
||||
旧代码用**行为链**而非固定步骤模板:
|
||||
```python
|
||||
CHAINS = [
|
||||
[('open_comment', 'KeyX'),
|
||||
('type_comment', 'input'), # 调用 _send_comment 状态机
|
||||
('wait_10s', 'wait'),
|
||||
('scroll_comment', 'scroll')],
|
||||
]
|
||||
```
|
||||
|
||||
每个链中的操作都是**键盘快捷键优先**(KeyZ/KeyX/KeyG),DOM 兜底。
|
||||
|
||||
---
|
||||
|
||||
## 二、当前新代码的问题审计
|
||||
|
||||
| 问题 | 表现 | 根因 |
|
||||
|:-----|:-----|:------|
|
||||
| **无前置状态检测** | open_video 在首页找不到卡片→硬等30s→失败 | 不先检测页面状态就直接找元素 |
|
||||
| **无后置验证** | post_comment 返回"sent"但评论没发出去 | 不检查评论区是否出现了刚发的文字 |
|
||||
| **无锚点系统** | 所有操作都用 CSS 类名硬匹配(CSS modules 不可靠) | 没有用 video/card 等特征判断 |
|
||||
| **状态机缺失** | 点赞/收藏/评论等操作的执行结果和页面实际脱节 | 没有 `detect()` 验证状态变化 |
|
||||
| **DOM 优先于键盘** | 所有操作先找 DOM 元素,找不到才 fallback | 旧代码用键盘快捷键为主,更可靠 |
|
||||
| **无回退策略** | 一个方法失败就放弃或硬等30s | 旧代码有3层回退(键盘→DOM→坐标) |
|
||||
|
||||
---
|
||||
|
||||
## 三、重构方案
|
||||
|
||||
### 3.1 新原子操作模板
|
||||
|
||||
```python
|
||||
async def atomic_like(page) -> OpResult:
|
||||
"""点赞操作(前后状态检测模板)"""
|
||||
# 1. 前置状态检测
|
||||
state = await detect_page_state(page)
|
||||
if state not in ('player', 'search_player'):
|
||||
return OpResult('like', False, f'不在播放页(state={state})')
|
||||
|
||||
# 2. 操作执行(键盘优先)
|
||||
await page.keyboard.press('z')
|
||||
await asyncio.sleep(1)
|
||||
|
||||
# 3. 后置验证(检测按钮状态变化)
|
||||
ok = await verify_like(page) # 检测 data-e2e-state 或 toast
|
||||
if ok:
|
||||
return OpResult('like', True, '👍')
|
||||
|
||||
# 4. 回退:DOM 点击
|
||||
btn = page.locator('[data-e2e="video-player-digg"]')
|
||||
if await btn.count() > 0:
|
||||
await btn.click()
|
||||
await asyncio.sleep(1)
|
||||
ok = await verify_like(page)
|
||||
|
||||
return OpResult('like', ok, '👍' if ok else '-')
|
||||
```
|
||||
|
||||
### 3.2 统一状态检测函数
|
||||
|
||||
```python
|
||||
async def detect_page_state(page) -> str:
|
||||
"""检测当前页面状态"""
|
||||
url = page.url
|
||||
video_count = await page.evaluate("document.querySelectorAll('video').length")
|
||||
cards = await page.evaluate(
|
||||
"document.querySelectorAll('.discover-video-card-item, [data-e2e=\"alink-item\"]').length"
|
||||
)
|
||||
|
||||
if video_count > 0 and '/video/' in url:
|
||||
return 'player_full' # B模式:独立播放页
|
||||
if video_count > 0 and 'modal_id' in url:
|
||||
return 'player_modal' # A模式:弹窗播放
|
||||
if video_count > 0:
|
||||
return 'player_unknown' # 有视频,模式待定
|
||||
if '/search/' in url:
|
||||
return 'search'
|
||||
if '/user/' in url:
|
||||
return 'profile'
|
||||
if cards > 0:
|
||||
return 'grid' # 首页feed
|
||||
return 'unknown'
|
||||
```
|
||||
|
||||
### 3.3 每个操作绑定状态依赖
|
||||
|
||||
| 操作 | 前置状态要求 | 后置验证方式 |
|
||||
|:-----|:------------|:-------------|
|
||||
| `goto_home` | (任意) | state → 'grid' |
|
||||
| `open_video` | 'grid' | state → 'player_*' |
|
||||
| `like` | 'player_*' | 按钮state变化或toast出现 |
|
||||
| `collect` | 'player_*' | 按钮state变化或toast出现 |
|
||||
| `follow` | 'player_*' | 按钮文字从"关注"变"已关注" |
|
||||
| `open_comments` | 'player_*' | state → 'comment_panel_open' |
|
||||
| `post_comment` | 'comment_panel_open' | comment_list中出现刚发的文字 |
|
||||
| `close_comments` | 'comment_panel_open' | state回到'player_*' |
|
||||
| `next_video` | 'player_*' | modal_id或video内容变化 |
|
||||
| `search` | (任意) | state → 'search' |
|
||||
| `go_back` | (任意) | URL或state变化 |
|
||||
|
||||
---
|
||||
|
||||
## 四、实施计划
|
||||
|
||||
### Phase 1:基础设施(1-2小时)
|
||||
1. 实现 `detect_page_state()` — 统一状态检测
|
||||
2. 实现 `verify_*()` 系列 — 每个操作的后置验证函数
|
||||
3. 实现 `ensure_page_state(target)` — 如果不在目标状态则导航过去
|
||||
|
||||
### Phase 2:重写原子操作(2-3小时)
|
||||
4. 按模板重写 `douyin_ops.py` 中所有操作
|
||||
5. 按模板重写 `ops/xhs_ops.py` 中所有操作
|
||||
6. 移除所有无验证的假成功返回
|
||||
|
||||
### Phase 3:测试验证(1-2小时)
|
||||
7. 逐步测试每个操作(用户交互模式)
|
||||
8. 修复各平台的特定选择器/DOM差异
|
||||
|
||||
---
|
||||
|
||||
以上是完整的审计和方案。你要我按这个方向执行吗?
|
||||
@@ -0,0 +1,23 @@
|
||||
"""
|
||||
AgentOS — 联邦智能体统一命令入口
|
||||
|
||||
命名空间:
|
||||
agentos matrix 社交矩阵(抖音/小红书运营)
|
||||
agentos ave 视频工厂(视频制作与编辑)
|
||||
agentos crawl 内容采集(互联网内容抓取)
|
||||
agentos fleet 联邦管理(多机协同)
|
||||
agentos serve 服务管理(MCP/Dashboard/调度)
|
||||
|
||||
用法:
|
||||
agentos matrix run --accounts A,B --blueprints X,Y
|
||||
agentos fleet sync
|
||||
agentos serve mcp
|
||||
agentos --help
|
||||
|
||||
设计原则:
|
||||
- 每个领域是一个插件,放在 plugins/ 目录下
|
||||
- 插件自动发现,加领域 = 加文件
|
||||
- 所有操作可远程执行 (--on 指定目标机器)
|
||||
- 所有操作支持 --json 输出
|
||||
"""
|
||||
__version__ = "0.1.0"
|
||||
@@ -0,0 +1,4 @@
|
||||
#!/usr/bin/env python3
|
||||
"""agentos — python -m agentos 入口"""
|
||||
from agentos.cli import main
|
||||
main()
|
||||
@@ -0,0 +1,84 @@
|
||||
"""
|
||||
AgentOS 插件基类
|
||||
|
||||
所有领域插件继承此类,实现 register() 方法注册子命令。
|
||||
插件放在 plugins/ 目录下,自动发现。
|
||||
"""
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class AgentOSPlugin:
|
||||
"""领域插件基类"""
|
||||
|
||||
name: str = "" # 插件名称(对应子命令名)
|
||||
description: str = "" # 插件描述
|
||||
nav: dict = None # 看板导航定义
|
||||
"""
|
||||
nav 格式:
|
||||
{
|
||||
'group': '社交矩阵',
|
||||
'icon': '📡',
|
||||
'order': 1, # 排序,小的在前
|
||||
'items': [
|
||||
{'view': 'matrix-accounts', 'label': '账号管理'},
|
||||
{'view': 'matrix-exec', 'label': '运维执行'},
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
def register(self, subparsers) -> argparse.ArgumentParser:
|
||||
"""注册插件的子命令解析器
|
||||
|
||||
Args:
|
||||
subparsers: 父解析器的子解析器组
|
||||
|
||||
Returns:
|
||||
创建的 ArgumentParser 实例
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def dispatch(self, args: argparse.Namespace) -> int:
|
||||
"""分发并执行命令
|
||||
|
||||
Args:
|
||||
args: 解析后的参数
|
||||
|
||||
Returns:
|
||||
退出码 (0=成功, 非0=失败)
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
def discover_plugins() -> list:
|
||||
"""自动发现 plugins/ 目录下的所有插件
|
||||
|
||||
扫描 plugins/ 目录,查找继承 AgentOSPlugin 的类。
|
||||
约定: 每个 .py 文件定义一个同名的 Plugin 类
|
||||
(如 matrix.py → MatrixPlugin)
|
||||
"""
|
||||
import importlib
|
||||
import inspect
|
||||
|
||||
plugins_dir = Path(__file__).parent / "plugins"
|
||||
if not plugins_dir.exists():
|
||||
return []
|
||||
|
||||
discovered = []
|
||||
for f in sorted(plugins_dir.glob("*.py")):
|
||||
if f.name.startswith("_"):
|
||||
continue
|
||||
module_name = f"agentos.plugins.{f.stem}"
|
||||
try:
|
||||
module = importlib.import_module(module_name)
|
||||
# 查找模块中继承 AgentOSPlugin 的类
|
||||
for name, cls in inspect.getmembers(module, inspect.isclass):
|
||||
if (issubclass(cls, AgentOSPlugin) and cls is not AgentOSPlugin
|
||||
and hasattr(cls, 'name') and cls.name):
|
||||
discovered.append(cls)
|
||||
break
|
||||
except Exception as e:
|
||||
print(f" ⚠️ 加载插件 {f.name} 失败: {e}")
|
||||
|
||||
return discovered
|
||||
@@ -0,0 +1,85 @@
|
||||
"""
|
||||
agentos CLI — 主分发器
|
||||
|
||||
自动发现 plugins/ 目录下的插件,注册为子命令。
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
from agentos.base import discover_plugins
|
||||
from agentos import __version__
|
||||
|
||||
|
||||
BANNER = """
|
||||
╔══════════════════════════════════════╗
|
||||
║ AgentOS v%s ║
|
||||
║ 联邦智能体统一命令入口 ║
|
||||
╚══════════════════════════════════════╝
|
||||
""" % __version__
|
||||
|
||||
|
||||
def build_parser():
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="agentos",
|
||||
description="AgentOS — 联邦智能体统一命令入口",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
领域:
|
||||
matrix 社交矩阵 — 抖音/小红书账号运营
|
||||
ave 视频工厂 — 视频制作与编辑
|
||||
crawl 内容采集 — 互联网内容抓取
|
||||
fleet 联邦管理 — 多机协同
|
||||
serve 服务管理 — MCP/Dashboard/调度
|
||||
|
||||
示例:
|
||||
agentos matrix run --accounts douyin_01 --blueprints douyin_daily
|
||||
agentos matrix collect --all
|
||||
agentos fleet sync
|
||||
agentos serve mcp
|
||||
agentos --help
|
||||
""",
|
||||
)
|
||||
parser.add_argument("-v", "--verbose", action="store_true", help="详细输出")
|
||||
parser.add_argument("--json", action="store_true", help="JSON 输出格式")
|
||||
parser.add_argument("--version", action="store_true", help="显示版本号")
|
||||
|
||||
sub = parser.add_subparsers(dest="domain", help="领域")
|
||||
|
||||
# 自动发现并注册插件
|
||||
plugins = discover_plugins()
|
||||
for plugin_cls in plugins:
|
||||
plugin = plugin_cls()
|
||||
plugin.register(sub)
|
||||
|
||||
return parser, plugins
|
||||
|
||||
|
||||
def main():
|
||||
parser, plugins = build_parser()
|
||||
# 用 parse_known_args 允许未知参数(如 --account --file)透传到子命令
|
||||
args, _unknown = parser.parse_known_args()
|
||||
|
||||
if args.version:
|
||||
print(f"AgentOS v{__version__}")
|
||||
return
|
||||
|
||||
if not args.domain:
|
||||
parser.print_help()
|
||||
return
|
||||
|
||||
# 查找对应的插件并分发
|
||||
for plugin_cls in plugins:
|
||||
plugin = plugin_cls()
|
||||
if args.domain == plugin.name:
|
||||
sys.exit(plugin.dispatch(args))
|
||||
|
||||
# 没找到插件
|
||||
print(f"未知领域: {args.domain}")
|
||||
print("可用领域: " + ", ".join(p.name for _, p in
|
||||
[(None, p()) for p in plugins]))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,152 @@
|
||||
"""
|
||||
agentos ave — 视频工厂(渲染/脚本/素材/模板)
|
||||
|
||||
封装 09_ave/scripts/ 目录下的 96 个脚本,提供 CLI 统一入口。
|
||||
|
||||
⚠️ 状态:**已废弃(2026-10-01)** —— 本插件引用的脚本路径全部失效
|
||||
(`01_director_parser/run.py`、`06_composer/run.py`、`07_service_layer/*.py`、
|
||||
`asset_manager/run.py` 均不存在;早期编号目录已归档到
|
||||
`90_archive/ave_legacy_numbered_dirs_20261001/`)。
|
||||
|
||||
视频工厂的实际入口:
|
||||
· CLI 原子能力:`09_ave/scripts/main.py`(voice/material/parse/compose/anchors…)
|
||||
· 生产模式: `09_ave/scripts/video_factory.py`(run_oral/beat/digital_human/hybrid/story)
|
||||
· 能力清单: `09_ave/config/capabilities.yaml` + Dashboard「🧩 能力库」
|
||||
· Dashboard: 视频工厂各视图(导演创作/工作流/能力库…)
|
||||
|
||||
保留本文件仅为兼容 `agentos ave --help` 的历史调用,**不建议继续使用**。
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from agentos.base import AgentOSPlugin
|
||||
|
||||
AGENT_SYNC = Path(__file__).resolve().parent.parent.parent.parent.parent
|
||||
AVE_SCRIPTS = AGENT_SYNC / "05_tools" / "09_ave" / "scripts"
|
||||
|
||||
|
||||
class AvePlugin(AgentOSPlugin):
|
||||
name = "ave"
|
||||
description = "视频工厂 — 渲染/脚本/素材/模板"
|
||||
nav = {
|
||||
'group': '视频工厂',
|
||||
'icon': '🎬',
|
||||
'order': 3,
|
||||
'items': [
|
||||
{'view': 'ave-render', 'label': '渲染任务'},
|
||||
{'view': 'ave-script', 'label': '脚本生成'},
|
||||
{'view': 'ave-materials', 'label': '素材库'},
|
||||
{'view': 'ave-templates', 'label': '模板'},
|
||||
]
|
||||
}
|
||||
|
||||
def _run_script(self, script_name: str, args: list[str] = None) -> int:
|
||||
"""运行 AVE scripts 目录下的 Python 脚本"""
|
||||
script_path = AVE_SCRIPTS / script_name
|
||||
if not script_path.exists():
|
||||
print(f"脚本不存在: {script_path}")
|
||||
return 1
|
||||
cmd = [sys.executable, str(script_path)] + (args or [])
|
||||
result = subprocess.run(cmd, cwd=str(AVE_SCRIPTS))
|
||||
return result.returncode
|
||||
|
||||
def register(self, subparsers):
|
||||
p = subparsers.add_parser("ave", help=self.description)
|
||||
p_sub = p.add_subparsers(dest="ave_action", help="视频工厂操作")
|
||||
|
||||
# render - 视频渲染
|
||||
pr = p_sub.add_parser("render", help="视频渲染/生成")
|
||||
pr.add_argument("--strategy", default="oral", help="策略: oral/beat-sync/digital-human")
|
||||
pr.add_argument("--script", help="脚本路径")
|
||||
pr.add_argument("--count", type=int, default=1, help="生成数量")
|
||||
pr.set_defaults(ave_func=self.cmd_render)
|
||||
|
||||
# script - 脚本/文案生成
|
||||
ps = p_sub.add_parser("script", help="脚本/文案生成")
|
||||
ps.add_argument("action", nargs="?", default="list",
|
||||
choices=["list", "generate", "edit"],
|
||||
help="操作")
|
||||
ps.add_argument("--topic", help="主题/方向")
|
||||
ps.add_argument("--style", help="风格: 口播/卡点/叙事")
|
||||
ps.set_defaults(ave_func=self.cmd_script)
|
||||
|
||||
# material - 素材管理
|
||||
pm = p_sub.add_parser("material", help="素材管理")
|
||||
pm.add_argument("action", nargs="?", default="list",
|
||||
choices=["list", "search", "add", "stats"],
|
||||
help="操作")
|
||||
pm.add_argument("--tag", help="标签过滤")
|
||||
pm.add_argument("--type", dest="mat_type", help="素材类型: bgm/clip/character")
|
||||
pm.set_defaults(ave_func=self.cmd_material)
|
||||
|
||||
# template - 模板管理
|
||||
pt = p_sub.add_parser("template", help="模板管理")
|
||||
pt.add_argument("action", nargs="?", default="list",
|
||||
choices=["list", "create", "edit", "apply"],
|
||||
help="操作")
|
||||
pt.set_defaults(ave_func=self.cmd_template)
|
||||
|
||||
# character - 角色管理(包装 character_portrait.py)
|
||||
pch = p_sub.add_parser("character", help="角色管理")
|
||||
pch.add_argument("action", nargs="?", default="list",
|
||||
choices=["list", "create", "generate"],
|
||||
help="操作")
|
||||
pch.set_defaults(ave_func=self.cmd_character)
|
||||
|
||||
# workflow - 工作流(包装 workflow 模块)
|
||||
pwf = p_sub.add_parser("workflow", help="工作流管理")
|
||||
pwf.add_argument("action", nargs="?", default="list",
|
||||
choices=["list", "run", "status"],
|
||||
help="操作")
|
||||
pwf.set_defaults(ave_func=self.cmd_workflow)
|
||||
|
||||
return p
|
||||
|
||||
def dispatch(self, args):
|
||||
if hasattr(args, 'ave_func'):
|
||||
return args.ave_func(args)
|
||||
print("未知视频工厂命令,使用 agentos ave --help")
|
||||
return 1
|
||||
|
||||
def cmd_render(self, args):
|
||||
print(f"🎬 渲染任务: strategy={args.strategy}")
|
||||
if args.strategy == "oral":
|
||||
return self._run_script("01_director_parser/run.py", [f"--strategy={args.strategy}"])
|
||||
print(f" 调用 06_composer 模块 (strategy={args.strategy}, count={args.count})")
|
||||
return self._run_script("06_composer/run.py", [f"--strategy={args.strategy}", f"--count={args.count}"])
|
||||
|
||||
def cmd_script(self, args):
|
||||
if args.action == "generate":
|
||||
print(f"📝 生成脚本: topic={args.topic or '(自动)'}, style={args.style or '口播'}")
|
||||
return self._run_script("01_director_parser/run.py",
|
||||
["--action=generate_script"] +
|
||||
(["--topic", args.topic] if args.topic else []) +
|
||||
(["--style", args.style] if args.style else []))
|
||||
print(f"📝 脚本 {args.action}")
|
||||
return self._run_script("01_director_parser/run.py", [f"--action={args.action}"])
|
||||
|
||||
def cmd_material(self, args):
|
||||
print(f"📦 素材 {args.action}" + (f" (tag={args.tag})" if args.tag else ""))
|
||||
return self._run_script("asset_manager/run.py",
|
||||
[f"--action={args.action}"] +
|
||||
(["--tag", args.tag] if args.tag else []) +
|
||||
(["--type", args.mat_type] if args.mat_type else []))
|
||||
|
||||
def cmd_template(self, args):
|
||||
print(f"📋 模板 {args.action}")
|
||||
if args.action == "list":
|
||||
return self._run_script("07_service_layer/template_manager.py", ["list"])
|
||||
elif args.action == "apply":
|
||||
print("(模板应用需要指定模板ID,功能开发中)")
|
||||
return 0
|
||||
|
||||
def cmd_character(self, args):
|
||||
print(f"🧑 角色 {args.action}")
|
||||
return self._run_script("character_portrait.py", [f"--action={args.action}"])
|
||||
|
||||
def cmd_workflow(self, args):
|
||||
print(f"🔧 工作流 {args.action}")
|
||||
return self._run_script("07_service_layer/workflow_engine.py", [f"--action={args.action}"])
|
||||
@@ -0,0 +1,150 @@
|
||||
"""
|
||||
agentos crawl — 内容采集(网页/视频/提取/调度)
|
||||
|
||||
封装 05_crawl/ 目录下的采集脚本 + web_crawler 技能。
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from agentos.base import AgentOSPlugin
|
||||
|
||||
AGENT_SYNC = Path(__file__).resolve().parent.parent.parent.parent.parent
|
||||
CRAWL_DIR = AGENT_SYNC / "05_tools" / "05_crawl"
|
||||
|
||||
|
||||
class CrawlPlugin(AgentOSPlugin):
|
||||
name = "crawl"
|
||||
description = "内容采集 — 网页/视频/提取/调度"
|
||||
nav = {
|
||||
'group': '内容采集',
|
||||
'icon': '📡',
|
||||
'order': 4,
|
||||
'items': [
|
||||
{'view': 'crawl-tasks', 'label': '采集任务'},
|
||||
{'view': 'crawl-sources', 'label': '源管理'},
|
||||
{'view': 'crawl-history', 'label': '采集历史'},
|
||||
]
|
||||
}
|
||||
|
||||
def _run_script(self, script_name: str, args: list[str] = None) -> int:
|
||||
"""运行 crawl 目录下的 Python 脚本"""
|
||||
script_path = CRAWL_DIR / script_name
|
||||
if not script_path.exists():
|
||||
print(f"脚本不存在: {script_path}")
|
||||
return 1
|
||||
cmd = [sys.executable, str(script_path)] + (args or [])
|
||||
result = subprocess.run(cmd, cwd=str(CRAWL_DIR))
|
||||
return result.returncode
|
||||
|
||||
def register(self, subparsers):
|
||||
p = subparsers.add_parser("crawl", help=self.description)
|
||||
p_sub = p.add_subparsers(dest="crawl_action", help="采集操作")
|
||||
|
||||
# web - 网页抓取
|
||||
pw = p_sub.add_parser("web", help="网页内容抓取")
|
||||
pw.add_argument("url", nargs="?", help="目标 URL")
|
||||
pw.add_argument("--output", help="输出文件路径")
|
||||
pw.add_argument("--depth", type=int, default=1, help="抓取深度")
|
||||
pw.set_defaults(crawl_func=self.cmd_web)
|
||||
|
||||
# video - 视频下载
|
||||
pv = p_sub.add_parser("video", help="视频下载")
|
||||
pv.add_argument("url", nargs="?", help="视频 URL")
|
||||
pv.add_argument("--format", default="mp4", help="输出格式")
|
||||
pv.set_defaults(crawl_func=self.cmd_video)
|
||||
|
||||
# extract - 信息提取
|
||||
pe = p_sub.add_parser("extract", help="信息提取")
|
||||
pe.add_argument("file", nargs="?", help="目标文件路径")
|
||||
pe.add_argument("--fields", help="提取字段 (逗号分隔)")
|
||||
pe.add_argument("--url", help="从 URL 提取")
|
||||
pe.set_defaults(crawl_func=self.cmd_extract)
|
||||
|
||||
# schedule - 定时采集
|
||||
psc = p_sub.add_parser("schedule", help="采集定时任务")
|
||||
psc.add_argument("action", nargs="?", default="list",
|
||||
choices=["list", "add", "remove"],
|
||||
help="操作")
|
||||
psc.set_defaults(crawl_func=self.cmd_schedule)
|
||||
|
||||
# source - 采集源管理
|
||||
ps = p_sub.add_parser("source", help="采集源管理")
|
||||
ps.add_argument("action", nargs="?", default="list",
|
||||
choices=["list", "add", "remove", "test"],
|
||||
help="操作")
|
||||
ps.add_argument("--name", help="源名称")
|
||||
ps.add_argument("--url", dest="src_url", help="源URL")
|
||||
ps.add_argument("--type", dest="src_type",
|
||||
choices=["rss", "web", "api", "social"],
|
||||
help="源类型")
|
||||
ps.set_defaults(crawl_func=self.cmd_source)
|
||||
|
||||
# content-inspiration - 内容灵感采集
|
||||
pci = p_sub.add_parser("inspiration", help="内容灵感采集")
|
||||
pci.add_argument("action", nargs="?", default="daily",
|
||||
choices=["daily", "trending", "search"],
|
||||
help="操作")
|
||||
pci.add_argument("--keyword", help="搜索关键词")
|
||||
pci.set_defaults(crawl_func=self.cmd_inspiration)
|
||||
|
||||
return p
|
||||
|
||||
def dispatch(self, args):
|
||||
if hasattr(args, 'crawl_func'):
|
||||
return args.crawl_func(args)
|
||||
print("未知采集命令,使用 agentos crawl --help")
|
||||
return 1
|
||||
|
||||
def cmd_web(self, args):
|
||||
print(f"🌐 网页抓取: {args.url or '(交互模式)'}")
|
||||
if args.url:
|
||||
return self._run_script("content-inspiration/run.py",
|
||||
["web", args.url] +
|
||||
(["--output", args.output] if args.output else []))
|
||||
print("请提供 URL: agentos crawl web <url>")
|
||||
return 1
|
||||
|
||||
def cmd_video(self, args):
|
||||
print(f"🎥 视频下载: {args.url or '(交互模式)'}")
|
||||
if args.url:
|
||||
return self._run_script("content-inspiration/run.py",
|
||||
["video", args.url, "--format", args.format])
|
||||
print("请提供视频 URL: agentos crawl video <url>")
|
||||
return 1
|
||||
|
||||
def cmd_extract(self, args):
|
||||
print(f"🔍 信息提取: {args.file or '(交互模式)'}")
|
||||
if args.file:
|
||||
cmd = ["extract", args.file]
|
||||
if args.fields:
|
||||
cmd += ["--fields", args.fields]
|
||||
return self._run_script("content-inspiration/run.py", cmd)
|
||||
if args.url:
|
||||
return self._run_script("content-inspiration/run.py", ["extract", "--url", args.url])
|
||||
print("请指定文件或URL: agentos crawl extract <file> 或 --url <url>")
|
||||
return 1
|
||||
|
||||
def cmd_schedule(self, args):
|
||||
print(f"⏰ 采集定时任务 {args.action}")
|
||||
return self._run_script("content-inspiration/run.py", ["schedule", args.action])
|
||||
|
||||
def cmd_source(self, args):
|
||||
print(f"📡 采集源 {args.action}")
|
||||
cmd = ["source", args.action]
|
||||
if args.name:
|
||||
cmd += ["--name", args.name]
|
||||
if args.src_url:
|
||||
cmd += ["--url", args.src_url]
|
||||
if args.src_type:
|
||||
cmd += ["--type", args.src_type]
|
||||
return self._run_script("content-inspiration/run.py", cmd)
|
||||
|
||||
def cmd_inspiration(self, args):
|
||||
print(f"💡 内容灵感 {args.action}")
|
||||
cmd = ["inspiration", args.action]
|
||||
if args.keyword:
|
||||
cmd += ["--keyword", args.keyword]
|
||||
return self._run_script("content-inspiration/run.py", cmd)
|
||||
@@ -0,0 +1,123 @@
|
||||
"""
|
||||
agentos fleet — 联邦管理(多机协同)
|
||||
|
||||
功能:
|
||||
sync 一键同步所有机器
|
||||
reconcile 对账检查
|
||||
exec 远程执行命令
|
||||
status 集群状态
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from agentos.base import AgentOSPlugin
|
||||
|
||||
AGENT_SYNC = Path(__file__).resolve().parent.parent.parent.parent.parent.parent # x6 → agent-sync/
|
||||
|
||||
|
||||
class FleetPlugin(AgentOSPlugin):
|
||||
name = "fleet"
|
||||
description = "联邦管理 — 多机协同、同步、对账"
|
||||
nav = {
|
||||
'group': '联邦管理',
|
||||
'icon': '🖥️',
|
||||
'order': 4,
|
||||
'items': [
|
||||
{'view': 'matrix-summary', 'label': '集群概览'},
|
||||
{'view': 'machines', 'label': '机器状态'},
|
||||
]
|
||||
}
|
||||
|
||||
def register(self, subparsers):
|
||||
p = subparsers.add_parser("fleet", help=self.description)
|
||||
p_sub = p.add_subparsers(dest="fleet_action", help="联邦操作")
|
||||
|
||||
# sync
|
||||
ps = p_sub.add_parser("sync", help="一键同步所有机器")
|
||||
ps.set_defaults(fleet_func=self.cmd_sync)
|
||||
|
||||
# reconcile
|
||||
pr = p_sub.add_parser("reconcile", help="对账检查")
|
||||
pr.set_defaults(fleet_func=self.cmd_reconcile)
|
||||
|
||||
# exec
|
||||
pe = p_sub.add_parser("exec", help="在远程机器执行命令")
|
||||
pe.add_argument("host", help="目标机器名或 Tailscale IP")
|
||||
pe.add_argument("command", nargs="+", help="要执行的命令")
|
||||
pe.set_defaults(fleet_func=self.cmd_exec)
|
||||
|
||||
# status
|
||||
ps2 = p_sub.add_parser("status", help="集群状态")
|
||||
ps2.set_defaults(fleet_func=self.cmd_status)
|
||||
|
||||
return p
|
||||
|
||||
def dispatch(self, args):
|
||||
if hasattr(args, 'fleet_func'):
|
||||
return args.fleet_func(args)
|
||||
print("未知联邦命令,使用 agentos fleet --help")
|
||||
return 1
|
||||
|
||||
def cmd_sync(self, args):
|
||||
fleet_sh = AGENT_SYNC / "00_bootstrap" / "fleet_sync.sh"
|
||||
if fleet_sh.exists():
|
||||
subprocess.run(["bash", str(fleet_sh)])
|
||||
else:
|
||||
print(f"fleet_sync.sh 不存在: {fleet_sh}")
|
||||
return 0
|
||||
|
||||
def cmd_reconcile(self, args):
|
||||
rec_sh = AGENT_SYNC / "00_bootstrap" / "fleet_reconcile.sh"
|
||||
if rec_sh.exists():
|
||||
subprocess.run(["bash", str(rec_sh)])
|
||||
else:
|
||||
print(f"fleet_reconcile.sh 不存在: {rec_sh}")
|
||||
return 0
|
||||
|
||||
def cmd_exec(self, args):
|
||||
"""执行远程命令(委托给 mc remote exec)"""
|
||||
from mc.cli import cmd_remote, _lookup_machines, _ssh_call
|
||||
# 构造 args 对象
|
||||
class FakeArgs:
|
||||
remote_action = "exec"
|
||||
host = args.host
|
||||
command = args.command
|
||||
via = "ssh"
|
||||
json = False
|
||||
cmd_remote(FakeArgs())
|
||||
return 0
|
||||
|
||||
def cmd_status(self, args):
|
||||
from mc.cli import cmd_remote, _lookup_machines
|
||||
class FakeArgs:
|
||||
remote_action = "status"
|
||||
host = ""
|
||||
json = False
|
||||
cmd_remote(FakeArgs())
|
||||
return 0
|
||||
|
||||
def cmd_cleanup(self, args):
|
||||
"""清理所有机器残留的 Playwright 驱动进程"""
|
||||
import subprocess
|
||||
machines = [
|
||||
("本机", "127.0.0.1", ""),
|
||||
("5kechengdeAir", "100.72.182.121", "5kecheng"),
|
||||
("7kecheng", "100.65.35.28", "7kecheng"),
|
||||
]
|
||||
for name, ip, user in machines:
|
||||
print(f"\n → {name} ({ip})...")
|
||||
if name == "本机":
|
||||
result = subprocess.run(
|
||||
["pkill", "-f", "playwright/driver/node"],
|
||||
capture_output=True, text=True)
|
||||
print(f" 已清理" if result.returncode == 0 else " 无残留")
|
||||
else:
|
||||
result = subprocess.run(
|
||||
["ssh", user, ip, "pkill -f playwright/driver/node"],
|
||||
capture_output=True, text=True)
|
||||
print(f" 已清理" if result.returncode == 0 else " 无残留")
|
||||
print("\n ✅ 清理完成")
|
||||
return 0
|
||||
@@ -0,0 +1,162 @@
|
||||
"""
|
||||
agentos guardd — 节点代理管理
|
||||
|
||||
功能:
|
||||
status 查看 guardd 运行状态
|
||||
restart 重启 guardd
|
||||
logs 查看 guardd 日志
|
||||
install 安装/更新 guardd plist
|
||||
tasks 列出运行中任务
|
||||
stop 停止指定任务
|
||||
"""
|
||||
|
||||
import argparse, json, os, subprocess, time, urllib.request, urllib.error
|
||||
from pathlib import Path
|
||||
|
||||
from agentos.base import AgentOSPlugin
|
||||
|
||||
AGENT_SYNC = Path(__file__).resolve().parent.parent.parent.parent.parent.parent
|
||||
GUARDD_PORT = 9090
|
||||
|
||||
|
||||
def _guardd_api(path: str) -> dict:
|
||||
"""调用本地 guardd HTTP API"""
|
||||
try:
|
||||
r = urllib.request.urlopen(f"http://localhost:{GUARDD_PORT}{path}", timeout=5)
|
||||
return json.loads(r.read().decode())
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
class GuarddPlugin(AgentOSPlugin):
|
||||
name = "guardd"
|
||||
description = "节点代理管理 — 部署/监控/任务管理"
|
||||
|
||||
def register(self, subparsers):
|
||||
p = subparsers.add_parser("guardd", help=self.description)
|
||||
p_sub = p.add_subparsers(dest="guardd_action", help="操作")
|
||||
|
||||
ps = p_sub.add_parser("status", help="查看 guardd 运行状态")
|
||||
ps.set_defaults(guardd_func=self.cmd_status)
|
||||
|
||||
pr = p_sub.add_parser("restart", help="重启 guardd")
|
||||
pr.set_defaults(guardd_func=self.cmd_restart)
|
||||
|
||||
pl = p_sub.add_parser("logs", help="查看 guardd 日志")
|
||||
pl.add_argument("--tail", type=int, default=30, help="显示行数")
|
||||
pl.set_defaults(guardd_func=self.cmd_logs)
|
||||
|
||||
pi = p_sub.add_parser("install", help="安装/更新 guardd plist 并重启")
|
||||
pi.set_defaults(guardd_func=self.cmd_install)
|
||||
|
||||
pt = p_sub.add_parser("tasks", help="列出运行中任务")
|
||||
pt.set_defaults(guardd_func=self.cmd_tasks)
|
||||
|
||||
pk = p_sub.add_parser("stop", help="停止指定任务")
|
||||
pk.add_argument("run_id", help="任务 ID")
|
||||
pk.set_defaults(guardd_func=self.cmd_stop)
|
||||
|
||||
return p
|
||||
|
||||
def dispatch(self, args):
|
||||
if hasattr(args, 'guardd_func'):
|
||||
return args.guardd_func(args)
|
||||
print("未知 guardd 命令,使用 agentos guardd --help")
|
||||
return 1
|
||||
|
||||
def cmd_status(self, args):
|
||||
"""查看 guardd 状态"""
|
||||
# launchctl 状态
|
||||
r = subprocess.run(
|
||||
["launchctl", "print", f"gui/{os.getuid()}/com.agentos.guardd"],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
if r.returncode == 0:
|
||||
for line in r.stdout.split("\n"):
|
||||
if "state = " in line:
|
||||
print(f" launchd: 🟢 {line.strip()}")
|
||||
else:
|
||||
print(" launchd: ❌ 未运行")
|
||||
|
||||
# HTTP 健康检查
|
||||
health = _guardd_api("/health")
|
||||
if health:
|
||||
print(f" HTTP: 🟢 运行中 (端口 {GUARDD_PORT})")
|
||||
print(f" 版本: {health.get('version', '?')}")
|
||||
print(f" 运行: {health.get('uptime_sec', 0)}s")
|
||||
print(f" 任务: {health.get('running_tasks', 0)} 运行中 / {health.get('total_tasks', 0)} 总计")
|
||||
else:
|
||||
print(f" HTTP: ❌ 无法连接 (localhost:{GUARDD_PORT})")
|
||||
return 0
|
||||
|
||||
def cmd_restart(self, args):
|
||||
"""重启 guardd"""
|
||||
print(" ⏹ 停止 guardd...")
|
||||
subprocess.run(["launchctl", "unload", str(Path.home() / "Library/LaunchAgents/com.agentos.guardd.plist")],
|
||||
capture_output=True, timeout=5)
|
||||
time.sleep(1)
|
||||
print(" 🚀 启动 guardd...")
|
||||
subprocess.run(["launchctl", "load", str(Path.home() / "Library/LaunchAgents/com.agentos.guardd.plist")],
|
||||
capture_output=True, timeout=5)
|
||||
time.sleep(2)
|
||||
health = _guardd_api("/health")
|
||||
print(f" {'✅ guardd 已重启' if health else '❌ guardd 启动失败'}")
|
||||
return 0
|
||||
|
||||
def cmd_logs(self, args):
|
||||
"""查看 guardd 日志"""
|
||||
log_file = Path.home() / "workbuddy-agent-os" / "agent-local" / "runtime" / "guardd" / "guardd.log"
|
||||
if log_file.exists():
|
||||
lines = log_file.read_text().splitlines()
|
||||
tail = lines[-args.tail:]
|
||||
print(f" 📋 guardd.log (最后 {args.tail} 行):\n")
|
||||
for line in tail:
|
||||
print(f" {line}")
|
||||
else:
|
||||
print(" ❌ 日志文件不存在")
|
||||
return 0
|
||||
|
||||
def cmd_install(self, args):
|
||||
"""安装/更新 guardd plist"""
|
||||
plist_src = AGENT_SYNC / "05_tools" / "00_setup" / "guardd" / "com.agentos.guardd.plist"
|
||||
plist_dst = Path.home() / "Library/LaunchAgents" / "com.agentos.guardd.plist"
|
||||
if not plist_src.exists():
|
||||
print(f" ❌ 源 plist 不存在: {plist_src}")
|
||||
return 1
|
||||
# 替换 __HOME__ 占位符
|
||||
content = plist_src.read_text().replace("__HOME__", str(Path.home()))
|
||||
plist_dst.write_text(content)
|
||||
print(f" ✅ plist 已安装: {plist_dst}")
|
||||
return self.cmd_restart(args)
|
||||
|
||||
def cmd_tasks(self, args):
|
||||
"""列出 guardd 任务"""
|
||||
tasks = _guardd_api("/tasks")
|
||||
if not isinstance(tasks, list):
|
||||
print(" ❌ 无法获取任务列表(guardd HTTP 不可用)")
|
||||
return 1
|
||||
if not tasks:
|
||||
print(" 📭 无任务记录")
|
||||
return 0
|
||||
print(f" 📋 共 {len(tasks)} 个任务:\n")
|
||||
for t in tasks:
|
||||
status = t.get("status", "?")
|
||||
icon = {"running": "🟢", "completed": "✅", "failed": "❌", "cancelled": "⏹"}.get(status, "❓")
|
||||
cmd = t.get("cmd", "")[:60]
|
||||
pid = t.get("pid", "?")
|
||||
elapsed = ""
|
||||
if t.get("start_time"):
|
||||
import datetime
|
||||
st = datetime.datetime.fromisoformat(t["start_time"])
|
||||
elapsed = f"{(datetime.datetime.now(datetime.timezone.utc) - st).total_seconds():.0f}s"
|
||||
print(f" {icon} {t.get('run_id','?')[:12]} | {status:10s} | PID={pid} | {elapsed:>6s} | {cmd}")
|
||||
return 0
|
||||
|
||||
def cmd_stop(self, args):
|
||||
"""停止任务"""
|
||||
result = _guardd_api(f"/task/{args.run_id}/stop")
|
||||
if result.get("status") in ("stopped", "already_stopped"):
|
||||
print(f" ⏹ 任务 {args.run_id} 已停止")
|
||||
else:
|
||||
print(f" ❌ 停止失败: {result.get('error', '未知错误')}")
|
||||
return 0
|
||||
@@ -0,0 +1,599 @@
|
||||
"""
|
||||
agentos matrix — 社交矩阵(抖音/小红书运营)
|
||||
|
||||
混合模式:
|
||||
- 简单命令:原生实现(不依赖 mc)
|
||||
- 复杂命令:完整前置检查后委托 mc(含并发/预检/优雅退出)
|
||||
|
||||
执行约束(源自 99_system/ARCHITECTURE_CONSTITUTION.md):
|
||||
- MAX_CONCURRENT = 3 每台机器最多3个浏览器
|
||||
- LAUNCH_STAGGER = 15s 浏览器启动间隔
|
||||
- AUTO_SHUTDOWN = 30min 超时自动关闭
|
||||
- MIN_DISK_GB = 5 最小磁盘空间
|
||||
"""
|
||||
|
||||
import sys, os, argparse, json, subprocess, time, asyncio, shutil, glob
|
||||
from pathlib import Path
|
||||
|
||||
from agentos.base import AgentOSPlugin
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent.parent.parent
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 执行约束(统一来自 mc.execution_policy)
|
||||
# ═══════════════════════════════════════════════
|
||||
from mc.execution_policy import (
|
||||
MAX_CONCURRENT, LAUNCH_STAGGER, AUTO_SHUTDOWN_MIN, MIN_DISK_GB,
|
||||
GRACE_PERIOD_LOCAL, GRACE_PERIOD_REMOTE, MAX_TIMEOUT, SLOTS,
|
||||
preflight, slot_for, check_running_browsers, get_policy,
|
||||
)
|
||||
|
||||
# 需要浏览器的操作
|
||||
BROWSER_COMMANDS = {'run', 'collect', 'login', 'smart-login', 'nurture', 'publish'}
|
||||
|
||||
|
||||
class MatrixPlugin(AgentOSPlugin):
|
||||
name = "matrix"
|
||||
description = "社交矩阵 — 抖音/小红书账号运营、养号、采集、发布"
|
||||
|
||||
def register(self, subparsers):
|
||||
"""注册 matrix 子命令 — 全部通过 nargs='*' 捕获,dispatch 手动路由"""
|
||||
parser = subparsers.add_parser("matrix", help=self.description)
|
||||
parser.add_argument("args", nargs="*", help="子命令及参数")
|
||||
return parser
|
||||
|
||||
def dispatch(self, args):
|
||||
# 不从 args.args 读取,直接从 sys.argv 获取原始参数
|
||||
# sys.argv = ['agentos', 'matrix', 'publish', 'douyin', '--account', 'x', ...]
|
||||
raw = sys.argv[:]
|
||||
# 找到 'matrix' 之后的所有参数
|
||||
try:
|
||||
idx = raw.index('matrix')
|
||||
raw = raw[idx + 1:]
|
||||
except ValueError:
|
||||
raw = getattr(args, 'args', [])
|
||||
|
||||
if not raw:
|
||||
print("请指定子命令: account / blueprint / corpus / status\n"
|
||||
"或 mc 命令: run / collect / login / publish ...")
|
||||
return 1
|
||||
|
||||
cmd = raw[0]
|
||||
rest = raw[1:]
|
||||
|
||||
if cmd == 'account': return self.cmd_account_list(rest)
|
||||
elif cmd == 'blueprint': return self.cmd_blueprint_list(rest)
|
||||
elif cmd == 'corpus': return self.cmd_corpus_list(rest)
|
||||
elif cmd == 'status': return self.cmd_status_all(rest)
|
||||
elif cmd == 'collect': return self.cmd_collect(rest)
|
||||
elif cmd in ('login', 'smart-login'): return self.cmd_login(rest)
|
||||
elif cmd == 'logout': return self.cmd_logout(rest)
|
||||
elif cmd == 'publish': return self.cmd_publish(rest)
|
||||
elif cmd == 'schedule': return self.cmd_schedule(rest)
|
||||
elif cmd == 'run': return self.cmd_run(rest)
|
||||
else:
|
||||
return self._execute_with_guards(cmd)
|
||||
# ═══════════════════════════════════════════════
|
||||
# 原生命令(接收 rest 参数列表)
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def cmd_account_list(self, rest):
|
||||
"""agentos matrix account [list]"""
|
||||
show_json = '--json' in rest or '-j' in rest
|
||||
try:
|
||||
from matrix_mgmt import MatrixManager
|
||||
accounts = MatrixManager().list_accounts()
|
||||
if not accounts:
|
||||
print("📋 暂无账号"); return 0
|
||||
if show_json:
|
||||
print(json.dumps(accounts, ensure_ascii=False, indent=2))
|
||||
return 0
|
||||
print(f"\n📋 账号列表 ({len(accounts)}):")
|
||||
print(f"{'ID':20s} {'平台':15s} {'状态':10s} {'手机':15s} {'本机':8s}")
|
||||
print("-" * 70)
|
||||
for a in accounts:
|
||||
pid = a.get("id", "?")
|
||||
plat = a.get("platform", "?")
|
||||
plat_icon = "🎵" if plat == "douyin" else "📕"
|
||||
status_raw = a.get("_status", "")
|
||||
cookie = "🟢 已登录" if status_raw == "logged_in" else "🔴 离线"
|
||||
phone = a.get("phone", "")[:11]
|
||||
local = "本机" if a.get("is_local") else "远程"
|
||||
print(f" {pid:20s} {plat_icon:<5s} {cookie:<10s} {phone:<15s} {local:<8s}")
|
||||
return 0
|
||||
except Exception as e:
|
||||
print(f"❌ 加载账号失败: {e}")
|
||||
return 1
|
||||
|
||||
def cmd_blueprint_list(self, rest):
|
||||
try:
|
||||
from matrix_mgmt import MatrixManager
|
||||
bps = MatrixManager().list_blueprints()
|
||||
except Exception as e:
|
||||
print(f"❌ 加载蓝图失败: {e}"); return 1
|
||||
if not bps: print("📋 暂无蓝图"); return 0
|
||||
print(f"\n📋 蓝图 ({len(bps)}):")
|
||||
for bp in bps:
|
||||
name = bp.get("name", "?")
|
||||
steps = len(bp.get("steps", []))
|
||||
platform = bp.get("platform", "通用")
|
||||
desc = bp.get("description", "")[:50]
|
||||
print(f" • {name:<25s} ({steps}步) {platform:<12s} {desc}")
|
||||
return 0
|
||||
|
||||
def cmd_corpus_list(self, rest):
|
||||
try:
|
||||
from matrix_modules.nurture.comment_corpus import get_categories
|
||||
cats = get_categories()
|
||||
except Exception:
|
||||
cats = []
|
||||
if not cats: print("📋 语料分类: 暂无"); return 0
|
||||
print(f"\n📋 语料分类 ({len(cats)}):")
|
||||
for c in cats:
|
||||
name = c.get("name", "?") if isinstance(c, dict) else str(c)
|
||||
print(f" • {name}")
|
||||
return 0
|
||||
|
||||
def cmd_status_all(self, rest):
|
||||
try:
|
||||
r = subprocess.run(["curl", "-s", "--max-time", "5",
|
||||
"http://localhost:9988/api/machines"], capture_output=True, text=True, timeout=8)
|
||||
data = json.loads(r.stdout)
|
||||
machines = data.get("machines", []) if isinstance(data, dict) else data
|
||||
except Exception:
|
||||
machines = []
|
||||
print(f"\n📊 多机状态 ({len(machines)}):")
|
||||
for m in (machines if isinstance(machines, list) else []):
|
||||
name = m.get("hostname", m.get("name", "?"))
|
||||
online = m.get("status") == "online"
|
||||
browsers = m.get("browsers_running", 0)
|
||||
print(f" {'✅' if online else '🔴'} {name} 浏览器:{browsers}/{MAX_CONCURRENT}")
|
||||
if m.get("_last_push_sec"):
|
||||
s = m["_last_push_sec"]
|
||||
print(f" 🕐 最后通信: {s}s前 ({'在线' if s < 300 else '超时'})")
|
||||
print(f"\n 约束: MAX_CONCURRENT={MAX_CONCURRENT} STAGGER={LAUNCH_STAGGER}s "
|
||||
f"TIMEOUT={MAX_TIMEOUT}s SHUTDOWN={AUTO_SHUTDOWN_MIN}min")
|
||||
return 0
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# collect — 原生实现
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def cmd_collect(self, rest):
|
||||
"""agentos matrix collect [--phone PHONE] [--account ACCT] [--all] [--status]"""
|
||||
# 解析参数
|
||||
phone = ''
|
||||
account = ''
|
||||
show_status = False
|
||||
collect_all = False
|
||||
|
||||
i = 0
|
||||
while i < len(rest):
|
||||
if rest[i] == '--phone' and i + 1 < len(rest):
|
||||
phone = rest[i + 1]; i += 2
|
||||
elif rest[i] == '--account' and i + 1 < len(rest):
|
||||
account = rest[i + 1]; i += 2
|
||||
elif rest[i] == '--all':
|
||||
collect_all = True; i += 1
|
||||
elif rest[i] == '--status':
|
||||
show_status = True; i += 1
|
||||
else:
|
||||
i += 1
|
||||
|
||||
# --status: 查看进度(调用 mc 进度API)
|
||||
if show_status:
|
||||
return self._collect_status()
|
||||
|
||||
# 构建执行命令
|
||||
runner = SCRIPTS_DIR / "collect_batch_runner.py"
|
||||
if not runner.exists():
|
||||
runner = SCRIPTS_DIR / "collect_homepage_info.py"
|
||||
|
||||
cmd = [sys.executable, str(runner)]
|
||||
if phone:
|
||||
cmd += ["--single", phone]
|
||||
elif account:
|
||||
cmd += ["--single", account]
|
||||
# --all: 不加参数,默认执行全部
|
||||
|
||||
return self._run_collect(cmd)
|
||||
|
||||
def _collect_status(self) -> int:
|
||||
"""查看采集进度"""
|
||||
try:
|
||||
from collect_batch_runner import PROGRESS_FILE
|
||||
import json
|
||||
if PROGRESS_FILE.exists():
|
||||
data = json.loads(PROGRESS_FILE.read_text())
|
||||
print(f"📊 采集进度: {data.get('completed',0)}/{data.get('total_identities','?')}")
|
||||
print(f" 成功: {data.get('success',0)}, 失败: {data.get('failed',0)}")
|
||||
print(f" 状态: {data.get('status','?')}")
|
||||
if data.get('finished_at'):
|
||||
print(f" 完成时间: {data['finished_at']}")
|
||||
else:
|
||||
print("📊 无正在执行的采集任务")
|
||||
except Exception as e:
|
||||
print(f"❌ 读取进度失败: {e}")
|
||||
return 0
|
||||
|
||||
def _run_collect(self, cmd: list) -> int:
|
||||
"""执行采集(走完整保护流程)"""
|
||||
cmd_str = ' '.join(str(c) for c in cmd[:4]) + '...'
|
||||
print(f"🚀 启动采集: {cmd_str}")
|
||||
print(f" runner: {cmd[1]}")
|
||||
|
||||
# 保护:通过 _execute_with_guards 但收集结果
|
||||
needs_browser = True
|
||||
self._cleanup_stale()
|
||||
self._check_disk()
|
||||
if not self._check_concurrent():
|
||||
return 1
|
||||
|
||||
running = self._count_browsers()
|
||||
stagger = running * LAUNCH_STAGGER
|
||||
if stagger > 0:
|
||||
print(f" ⏳ 错峰: 等待 {stagger}s")
|
||||
time.sleep(min(stagger, 30))
|
||||
|
||||
# 执行
|
||||
env = os.environ.copy()
|
||||
env['MC_FORWARDED_FROM_AGENTOS'] = '1'
|
||||
env['PYTHONPATH'] = f"{SCRIPTS_DIR}:{env.get('PYTHONPATH', '')}"
|
||||
env['AGENT_SYNC'] = str(SCRIPTS_DIR.parent.parent)
|
||||
env['AGENT_LOCAL'] = str(Path.home() / "workbuddy-agent-os" / "agent-local")
|
||||
|
||||
log_dir = Path(env['AGENT_LOCAL']) / "runtime" / "commands"
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
log_file = log_dir / f"collect_{int(time.time())}.log"
|
||||
|
||||
with open(log_file, 'w') as f:
|
||||
p = subprocess.Popen(cmd, env=env, cwd=str(SCRIPTS_DIR),
|
||||
stdout=f, stderr=subprocess.STDOUT)
|
||||
print(f" PID {p.pid}")
|
||||
print(f" 日志: {log_file}")
|
||||
return 0
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# login/logout — 带警告的 mc 委托
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def cmd_login(self, rest):
|
||||
"""agentos matrix login <account_id> [--phone X] — 委托 mc 执行"""
|
||||
account_id = rest[0] if rest else ''
|
||||
if not account_id:
|
||||
print("❌ 请指定账号ID: agentos matrix login <account_id>")
|
||||
return 1
|
||||
|
||||
print(f"\n⚠️ 即将启动浏览器登录: {account_id}")
|
||||
print(f" 此操作会打开真实浏览器窗口,约 30-60 秒")
|
||||
confirm = input(" 确认执行? (y/N): ").strip().lower()
|
||||
if confirm != 'y':
|
||||
print(" 已取消")
|
||||
return 0
|
||||
|
||||
# 走 mc 委托(带完整执行保护)
|
||||
return self._delegate_to_mc('smart-login', needs_browser=True)
|
||||
|
||||
def cmd_logout(self, rest):
|
||||
"""agentos matrix logout <account_id> — 原生实现"""
|
||||
account_id = rest[0] if rest else ''
|
||||
if not account_id:
|
||||
print("❌ 请指定账号ID: agentos matrix logout <account_id>")
|
||||
return 1
|
||||
print(f"🔓 清除 {account_id} 的登录状态...")
|
||||
try:
|
||||
from matrix_mgmt import MatrixManager
|
||||
result = MatrixManager().unbind_account(account_id)
|
||||
msg = result.get('message', '操作完成')
|
||||
print(f" ✅ {msg}" if result.get("ok") else f" ⚠️ {msg}")
|
||||
except Exception as e:
|
||||
print(f" ❌ 失败: {e}")
|
||||
return 0
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# publish — 原生实现
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def cmd_publish(self, rest):
|
||||
"""agentos matrix publish <platform> --account X --file Y [--title T] [--desc D]"""
|
||||
platform = rest[0] if rest and not rest[0].startswith('--') else ''
|
||||
account = file_path = title = desc = ''
|
||||
i = 1 if platform else 0
|
||||
while i < len(rest):
|
||||
if rest[i] == '--account' and i + 1 < len(rest): account = rest[i+1]; i += 2
|
||||
elif rest[i] == '--file' and i + 1 < len(rest): file_path = rest[i+1]; i += 2
|
||||
elif rest[i] == '--title' and i + 1 < len(rest): title = rest[i+1]; i += 2
|
||||
elif rest[i] == '--desc' and i + 1 < len(rest): desc = rest[i+1]; i += 2
|
||||
else: i += 1
|
||||
if not platform or not account or not file_path:
|
||||
print("❌ 参数不足\n 用法: agentos matrix publish <douyin|xiaohongshu> --account X --file Y [--title T]")
|
||||
return 1
|
||||
print(f"📤 发布到 {platform}: {file_path}")
|
||||
cmd = [sys.executable, str(SCRIPTS_DIR / "publish_video.py"), platform,
|
||||
"--account", account, "--file", file_path]
|
||||
if title: cmd += ["--title", title]
|
||||
if desc: cmd += ["--desc", desc]
|
||||
result = subprocess.run(cmd, cwd=str(SCRIPTS_DIR),
|
||||
env={**os.environ, 'PYTHONPATH': str(SCRIPTS_DIR)})
|
||||
return result.returncode
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# schedule — 原生实现
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def cmd_schedule(self, rest):
|
||||
"""agentos matrix schedule list|add|remove|history|start|stop [args...]"""
|
||||
action = rest[0] if rest else 'list'
|
||||
|
||||
# 引入 mc.scheduler 的函数
|
||||
try:
|
||||
from mc.scheduler import (cmd_schedule_list, cmd_schedule_add,
|
||||
cmd_schedule_remove, cmd_schedule_history)
|
||||
except ImportError as e:
|
||||
print(f"❌ 加载调度器失败: {e}")
|
||||
return 1
|
||||
|
||||
if action == 'list':
|
||||
cmd_schedule_list()
|
||||
return 0
|
||||
|
||||
elif action == 'add':
|
||||
if len(rest) < 2:
|
||||
print("❌ 参数不足\n 用法: agentos matrix schedule add <id> --account X --blueprint Y --time HH:MM")
|
||||
return 1
|
||||
sid = rest[1]
|
||||
# 解析额外参数
|
||||
account = blueprint = time_val = ''
|
||||
rounds = 3
|
||||
days = '1,2,3,4,5,6,7'
|
||||
extra_args = ''
|
||||
i = 2
|
||||
while i < len(rest):
|
||||
if rest[i] == '--account' and i+1 < len(rest): account = rest[i+1]; i += 2
|
||||
elif rest[i] == '--blueprint' and i+1 < len(rest): blueprint = rest[i+1]; i += 2
|
||||
elif rest[i] == '--time' and i+1 < len(rest): time_val = rest[i+1]; i += 2
|
||||
elif rest[i] == '--rounds' and i+1 < len(rest):
|
||||
try: rounds = int(rest[i+1])
|
||||
except: pass
|
||||
i += 2
|
||||
elif rest[i] == '--days' and i+1 < len(rest): days = rest[i+1]; i += 2
|
||||
else: extra_args += f' {rest[i]}'; i += 1
|
||||
if not account or not blueprint or not time_val:
|
||||
print("❌ 缺少必要参数: --account --blueprint --time")
|
||||
print(" 用法: agentos matrix schedule add <id> --account X --blueprint Y --time HH:MM")
|
||||
return 1
|
||||
# 创建 FakeArgs 以兼容 mc.scheduler 的接口
|
||||
class FakeArgs:
|
||||
id = sid; account = account; blueprint = blueprint
|
||||
time = time_val; rounds = rounds; days = days
|
||||
args = extra_args.strip()
|
||||
cmd_schedule_add(FakeArgs())
|
||||
return 0
|
||||
|
||||
elif action == 'remove':
|
||||
sid = rest[1] if len(rest) > 1 else ''
|
||||
if not sid:
|
||||
print("❌ 请指定任务ID\n 用法: agentos matrix schedule remove <id>")
|
||||
return 1
|
||||
cmd_schedule_remove(sid)
|
||||
return 0
|
||||
|
||||
elif action == 'history':
|
||||
sid = rest[1] if len(rest) > 1 else ''
|
||||
cmd_schedule_history(sid)
|
||||
return 0
|
||||
|
||||
elif action == 'start':
|
||||
print("🚀 启动调度器...")
|
||||
import subprocess
|
||||
subprocess.Popen([sys.executable, '-m', 'mc', 'schedule', 'start'],
|
||||
cwd=str(SCRIPTS_DIR),
|
||||
env={**os.environ, 'PYTHONPATH': str(SCRIPTS_DIR)})
|
||||
return 0
|
||||
|
||||
elif action == 'stop':
|
||||
print("⏹ 停止调度器...")
|
||||
import subprocess
|
||||
subprocess.run(["pkill", "-f", "mc schedule start"], capture_output=True, timeout=5)
|
||||
return 0
|
||||
|
||||
else:
|
||||
print(f"❌ 未知操作: {action}")
|
||||
print(" 可用操作: list / add / remove / history / start / stop")
|
||||
return 1
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# run — 原生实现(养号执行,调用 nurture_runner.sh)
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def cmd_run(self, rest):
|
||||
"""agentos matrix run --accounts A,B --blueprints X,Y [--rounds N] [--mix]
|
||||
|
||||
注意:此操作会启动浏览器,约 10-30 分钟。
|
||||
"""
|
||||
# 解析参数
|
||||
accounts = ''
|
||||
blueprints = ''
|
||||
rounds = 10
|
||||
interval = ''
|
||||
corpus = ''
|
||||
engine = 'camoufox'
|
||||
mix_flag = ''
|
||||
daemon = False
|
||||
keep = False
|
||||
|
||||
i = 0
|
||||
while i < len(rest):
|
||||
if rest[i] == '--accounts' and i+1 < len(rest): accounts = rest[i+1]; i += 2
|
||||
elif rest[i] == '--blueprints' and i+1 < len(rest): blueprints = rest[i+1]; i += 2
|
||||
elif rest[i] == '--rounds' and i+1 < len(rest): rounds = rest[i+1]; i += 2
|
||||
elif rest[i] == '--interval' and i+1 < len(rest): interval = rest[i+1]; i += 2
|
||||
elif rest[i] == '--corpus' and i+1 < len(rest): corpus = rest[i+1]; i += 2
|
||||
elif rest[i] == '--engine' and i+1 < len(rest): engine = rest[i+1]; i += 2
|
||||
elif rest[i] == '--mix': mix_flag = '--mix'; i += 1
|
||||
elif rest[i] == '--daemon': daemon = True; i += 1
|
||||
elif rest[i] == '--keep': keep = True; i += 1
|
||||
elif rest[i] == '--max-browsers' and i+1 < len(rest):
|
||||
try:
|
||||
MAX_CONCURRENT = int(rest[i+1])
|
||||
except: pass
|
||||
i += 2
|
||||
else: i += 1
|
||||
|
||||
if not accounts or not blueprints:
|
||||
print("❌ 参数不足: --accounts A,B --blueprints X,Y")
|
||||
return 1
|
||||
|
||||
print(f"\n🌱 养号执行: {accounts}")
|
||||
print(f" 蓝图: {blueprints} × {rounds} 轮")
|
||||
print(f" 引擎: {engine}{' 混合模式' if mix_flag else ''}")
|
||||
|
||||
# 执行保护
|
||||
self._cleanup_stale()
|
||||
self._check_disk()
|
||||
if not self._check_concurrent():
|
||||
return 1
|
||||
|
||||
# 构建 mc run 命令
|
||||
cmd = [sys.executable, '-m', 'mc', 'run',
|
||||
'--accounts', accounts,
|
||||
'--blueprints', blueprints,
|
||||
'--rounds', str(rounds)]
|
||||
if interval: cmd += ['--interval', interval]
|
||||
if corpus: cmd += ['--corpus', corpus]
|
||||
if mix_flag: cmd += [mix_flag]
|
||||
if daemon: cmd += ['--daemon']
|
||||
if keep: cmd += ['--keep']
|
||||
if MAX_CONCURRENT != 3: cmd += ['--max-browsers', str(MAX_CONCURRENT)]
|
||||
|
||||
# 后台运行
|
||||
log_dir = Path.home() / "workbuddy-agent-os" / "agent-local" / "runtime" / "commands"
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
log_file = log_dir / f"nurture_{accounts.replace(',','_')}_{int(time.time())}.log"
|
||||
|
||||
print(f" 🚀 启动养号进程...")
|
||||
with open(log_file, 'w') as f:
|
||||
p = subprocess.Popen(cmd, env={**os.environ, 'PYTHONPATH': str(SCRIPTS_DIR)},
|
||||
cwd=str(SCRIPTS_DIR), stdout=f, stderr=subprocess.STDOUT)
|
||||
print(f" PID: {p.pid}")
|
||||
print(f" 日志: {log_file}")
|
||||
print(f" ℹ️ 浏览器自动运行中,请勿关闭此终端")
|
||||
return 0
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 执行保护(前置检查 + 并发控制 + 优雅退出)
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _execute_with_guards(self, cmd_name: str) -> int:
|
||||
"""带完整保护的执行入口"""
|
||||
needs_browser = cmd_name in BROWSER_COMMANDS
|
||||
|
||||
if needs_browser:
|
||||
print(f"\n🔍 预检(轻): {cmd_name}")
|
||||
# 仅做 MC 不做的检查:清理残留 + 磁盘
|
||||
self._cleanup_stale()
|
||||
self._check_disk()
|
||||
# 注意:并发/browsers/槽位/stagger/冷却等检查不在本层做
|
||||
# 统一由 mc engine.py 执行时自己控制
|
||||
|
||||
return self._delegate_to_mc(cmd_name, needs_browser)
|
||||
|
||||
def _cleanup_stale(self):
|
||||
"""复位:杀残留 + 清理临时文件(对标 preflight.py)"""
|
||||
killed = 0
|
||||
for pat in ["camoufox.*orphan", "playwright"]:
|
||||
try:
|
||||
r = subprocess.run(["pkill", "-f", pat], capture_output=True, timeout=5)
|
||||
if r.returncode == 0: killed += 1
|
||||
except: pass
|
||||
# 清理临时目录
|
||||
for pat in ["/tmp/camoufox_*", "/tmp/playwright_*"]:
|
||||
for fp in glob.glob(pat):
|
||||
try: shutil.rmtree(fp, ignore_errors=True)
|
||||
except: pass
|
||||
if killed:
|
||||
print(f" 🧹 已清理 {killed} 个残留进程")
|
||||
|
||||
def _check_disk(self):
|
||||
"""检查磁盘空间(对标 preflight.py check_disk_space)"""
|
||||
try:
|
||||
r = subprocess.run(["df", "-h", "/"], capture_output=True, text=True, timeout=5)
|
||||
parts = r.stdout.strip().split("\n")[1].split()
|
||||
avail = parts[3] if len(parts) >= 4 else "0"
|
||||
avail_gb = 0
|
||||
if avail.endswith("G"): avail_gb = float(avail[:-1])
|
||||
elif avail.endswith("M"): avail_gb = float(avail[:-1]) / 1024
|
||||
elif avail.endswith("T"): avail_gb = float(avail[:-1]) * 1024
|
||||
if avail_gb < MIN_DISK_GB:
|
||||
print(f" ⚠️ 磁盘空间不足: {avail} ({avail_gb:.1f}GB < {MIN_DISK_GB}GB)")
|
||||
except: pass
|
||||
|
||||
def _count_browsers(self) -> int:
|
||||
"""统计当前运行浏览器数量(对标 browser_orchestrator.check_running_browsers)"""
|
||||
try:
|
||||
r = subprocess.run(
|
||||
["pgrep", "-f", "camoufox.*-no-remote|HeadlessShell|chrome.*--remote-debugging"],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
return len([p for p in r.stdout.strip().split("\n") if p.strip()])
|
||||
except: return 0
|
||||
|
||||
def _check_concurrent(self) -> bool:
|
||||
"""并发检查(统一使用 mc.execution_policy.preflight)"""
|
||||
pf = preflight()
|
||||
print(f" 📊 状态: 浏览器 {pf['browsers']}/{MAX_CONCURRENT}"
|
||||
f" · 磁盘 {pf['disk_gb']}GB/{MIN_DISK_GB}GB"
|
||||
f" · 延迟 {pf['stagger_delay']}s")
|
||||
if not pf["ok"]:
|
||||
print(f" ❌ {pf['message']}")
|
||||
return False
|
||||
return True
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# mc 委托执行
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _get_rest_args(self, cmd_name: str) -> list:
|
||||
"""从 sys.argv 提取 cmd_name 后面的参数"""
|
||||
try:
|
||||
idx = sys.argv.index(cmd_name)
|
||||
return sys.argv[idx + 1:]
|
||||
except ValueError:
|
||||
return []
|
||||
|
||||
def _delegate_to_mc(self, cmd_name: str, needs_browser: bool = False) -> int:
|
||||
"""委托 mc 执行"""
|
||||
python = sys.executable
|
||||
# 重建 mc 命令: 从当前 args 动态构建
|
||||
# raw args = ['agentos', 'matrix', 'run', '--accounts', 'X']
|
||||
# → mc args = ['python', '-m', 'mc', 'run', '--accounts', 'X']
|
||||
mc_argv = [python, '-m', 'mc', cmd_name] + self._get_rest_args(cmd_name)
|
||||
|
||||
env = os.environ.copy()
|
||||
env['MC_FORWARDED_FROM_AGENTOS'] = '1'
|
||||
env['PYTHONPATH'] = f"{SCRIPTS_DIR}:{env.get('PYTHONPATH', '')}"
|
||||
env['AGENT_SYNC'] = str(SCRIPTS_DIR.parent.parent)
|
||||
env['AGENT_LOCAL'] = str(Path.home() / "workbuddy-agent-os" / "agent-local")
|
||||
|
||||
# 浏览器操作:Popen 后台运行 + 跟踪 PID
|
||||
if needs_browser:
|
||||
log_dir = Path(env['AGENT_LOCAL']) / "runtime" / "commands"
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
log_file = log_dir / f"{cmd_name}_{int(time.time())}.log"
|
||||
|
||||
with open(log_file, 'w') as f:
|
||||
p = subprocess.Popen(
|
||||
mc_argv, env=env, cwd=str(SCRIPTS_DIR),
|
||||
stdout=f, stderr=subprocess.STDOUT
|
||||
)
|
||||
|
||||
print(f" 🚀 PID {p.pid} | 日志: {log_file}")
|
||||
print(f" 📋 并发: {self._count_browsers()}/{MAX_CONCURRENT}")
|
||||
print(f" ⏱️ 超时: {AUTO_SHUTDOWN_MIN}min 自动关闭")
|
||||
print(f" ℹ️ 检查: ps aux | grep {p.pid}")
|
||||
return 0
|
||||
|
||||
# 非浏览器操作:同步等待
|
||||
result = subprocess.run(mc_argv, env=env, cwd=str(SCRIPTS_DIR))
|
||||
return result.returncode
|
||||
@@ -0,0 +1,162 @@
|
||||
"""
|
||||
agentos serve — 服务管理(MCP/Dashboard/调度)
|
||||
|
||||
功能:
|
||||
mcp 启动 MCP Server
|
||||
dashboard 启动/停止 Dashboard
|
||||
schedule 全局定时调度器
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from agentos.base import AgentOSPlugin
|
||||
|
||||
AGENT_SYNC = Path(__file__).resolve().parent.parent.parent.parent.parent.parent # x6 → agent-sync/
|
||||
|
||||
|
||||
class ServePlugin(AgentOSPlugin):
|
||||
name = "serve"
|
||||
description = "服务管理 — MCP/Dashboard/调度"
|
||||
nav = {
|
||||
'group': '系统设置',
|
||||
'icon': '⚙️',
|
||||
'order': 5,
|
||||
'items': [
|
||||
{'view': 'kb', 'label': '知识库'},
|
||||
]
|
||||
}
|
||||
|
||||
def register(self, subparsers):
|
||||
p = subparsers.add_parser("serve", help=self.description)
|
||||
p_sub = p.add_subparsers(dest="serve_action", help="服务操作")
|
||||
|
||||
# dashboard
|
||||
pd = p_sub.add_parser("dashboard", help="启动/管理 Dashboard")
|
||||
pd.add_argument("action", nargs="?", default="start",
|
||||
choices=["start", "stop", "restart", "status"],
|
||||
help="操作")
|
||||
pd.add_argument("--port", type=int, default=9988, help="端口")
|
||||
pd.set_defaults(serve_func=self.cmd_dashboard)
|
||||
|
||||
# mcp
|
||||
pm = p_sub.add_parser("mcp", help="启动 MCP Server")
|
||||
pm.set_defaults(serve_func=self.cmd_mcp)
|
||||
|
||||
# schedule
|
||||
psc = p_sub.add_parser("schedule", help="定时任务管理")
|
||||
psc.add_argument("action", nargs="?", default="list",
|
||||
choices=["list", "add", "remove"],
|
||||
help="操作")
|
||||
psc.set_defaults(serve_func=self.cmd_schedule)
|
||||
|
||||
# guardd
|
||||
pg = p_sub.add_parser("guardd", help="系统自愈守护进程")
|
||||
pg.add_argument("action", nargs="?", default="start",
|
||||
choices=["start", "stop", "status"],
|
||||
help="操作")
|
||||
pg.add_argument("--daemon", action="store_true", help="后台运行")
|
||||
pg.set_defaults(serve_func=self.cmd_guardd)
|
||||
|
||||
return p
|
||||
|
||||
def dispatch(self, args):
|
||||
if hasattr(args, 'serve_func'):
|
||||
return args.serve_func(args)
|
||||
print("未知服务命令,使用 agentos serve --help")
|
||||
return 1
|
||||
|
||||
def cmd_dashboard(self, args):
|
||||
dashboard_dir = AGENT_SYNC / "05_tools" / "10_dashboard"
|
||||
run_py = dashboard_dir / "run.py"
|
||||
|
||||
if not run_py.exists():
|
||||
print(f"Dashboard 不存在: {run_py}")
|
||||
return 1
|
||||
|
||||
if args.action == "start":
|
||||
pid_file = dashboard_dir / ".dashboard.pid"
|
||||
if pid_file.exists():
|
||||
print("Dashboard 已在运行")
|
||||
return 0
|
||||
subprocess.Popen(
|
||||
[sys.executable, str(run_py), str(args.port)],
|
||||
cwd=str(dashboard_dir),
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
print(f"Dashboard 已启动 (port {args.port})")
|
||||
elif args.action == "stop":
|
||||
import subprocess
|
||||
subprocess.run(["pkill", "-f", f"run.py {args.port}"])
|
||||
print(f"Dashboard (port {args.port}) 已停止")
|
||||
elif args.action == "status":
|
||||
import subprocess
|
||||
r = subprocess.run(
|
||||
["curl", "-s", "--connect-timeout", "2",
|
||||
f"http://localhost:{args.port}/api/identity"],
|
||||
capture_output=True, text=True)
|
||||
if r.returncode == 0 and r.stdout:
|
||||
print(f"Dashboard 运行中 (port {args.port})")
|
||||
else:
|
||||
print(f"Dashboard 未运行 (port {args.port})")
|
||||
return 0
|
||||
|
||||
def cmd_mcp(self, args):
|
||||
print("MCP Server 启动中...")
|
||||
print("(功能开发中,敬请期待)")
|
||||
return 0
|
||||
|
||||
def cmd_schedule(self, args):
|
||||
print(f"定时任务 {args.action}")
|
||||
print("(功能开发中,敬请期待)")
|
||||
return 0
|
||||
|
||||
def cmd_guardd(self, args):
|
||||
"""系统自愈守护进程"""
|
||||
import subprocess, os
|
||||
from pathlib import Path
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent.parent.parent
|
||||
guardd_script = str(SCRIPTS_DIR / "guardd.py")
|
||||
|
||||
if args.action == "start":
|
||||
if args.daemon:
|
||||
log_file = os.path.expanduser(
|
||||
"~/workbuddy-agent-os/agent-local/runtime/guardd/daemon.log")
|
||||
os.makedirs(os.path.dirname(log_file), exist_ok=True)
|
||||
pid = subprocess.Popen(
|
||||
[sys.executable, guardd_script],
|
||||
stdout=open(log_file, 'w'), stderr=subprocess.STDOUT,
|
||||
).pid
|
||||
print(f"🛡️ guardd 已后台启动 (PID: {pid})")
|
||||
print(f" 日志: {log_file}")
|
||||
else:
|
||||
os.execvp(sys.executable, [sys.executable, guardd_script])
|
||||
elif args.action == "stop":
|
||||
try:
|
||||
subprocess.run(["pkill", "-f", "guardd.py"], capture_output=True, timeout=5)
|
||||
print("⏹ guardd 已停止")
|
||||
except Exception as e:
|
||||
print(f"❌ 停止失败: {e}")
|
||||
elif args.action == "status":
|
||||
sf = os.path.expanduser(
|
||||
"~/workbuddy-agent-os/agent-local/runtime/guardd/status.json")
|
||||
if os.path.exists(sf):
|
||||
import json
|
||||
try:
|
||||
d = json.loads(open(sf).read())
|
||||
print(f"🛡️ guardd 状态")
|
||||
print(f" 最后检查: {d.get('last_check','?')}")
|
||||
print(f" 孤儿进程: {d.get('orphans',0)}")
|
||||
print(f" 磁盘: {d.get('disk_gb','?')}GB")
|
||||
print(f" 浏览器: {d.get('browsers',0)}")
|
||||
if d.get('events'):
|
||||
print(f" 事件: {'; '.join(d['events'])}")
|
||||
except:
|
||||
print("🛡️ guardd 状态文件损坏")
|
||||
else:
|
||||
print("🛡️ guardd 未运行")
|
||||
return 0
|
||||
@@ -0,0 +1,271 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
AntiDetection — 反检测模块 (v1.0.0)
|
||||
|
||||
基于旧版完整手册 (IMPLEMENTATION_GUIDE.md) 提取的反检测措施。
|
||||
|
||||
功能:
|
||||
1. 浏览器指纹覆写 (webdriver/platform/standalone/plugins/languages)
|
||||
2. 拟人化行为参数 (随机延迟/观看时长/打字速度)
|
||||
3. 账号行为画像 (每小时操作上限)
|
||||
4. DOM弹窗清理 (扩展选择器列表)
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import random
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
__version__ = "1.0.0"
|
||||
|
||||
# ─── 浏览器指纹覆写脚本 ────────────────────────────────────────
|
||||
|
||||
FINGERPRINT_SCRIPT = """
|
||||
() => {
|
||||
// 1. 隐藏自动化标记 (最关键)
|
||||
Object.defineProperty(navigator, 'webdriver', { get: () => false });
|
||||
|
||||
// 2. 平台与UA一致性
|
||||
Object.defineProperty(navigator, 'platform', { get: () => 'iPad' });
|
||||
|
||||
// 3. 隐藏PWA安装能力
|
||||
Object.defineProperty(navigator, 'standalone', { get: () => false });
|
||||
|
||||
// 4. 插件列表 (真实iPad有5个plugin)
|
||||
Object.defineProperty(navigator, 'plugins', {
|
||||
get: () => [1, 2, 3, 4, 5]
|
||||
});
|
||||
|
||||
// 5. 语言设置
|
||||
Object.defineProperty(navigator, 'languages', { get: () => ['zh-CN', 'zh'] });
|
||||
|
||||
// 6. 硬件并发数 (Apple M1 = 8)
|
||||
Object.defineProperty(navigator, 'hardwareConcurrency', { get: () => 8 });
|
||||
|
||||
// 7. 设备内存 (iPad Pro = 8GB)
|
||||
Object.defineProperty(navigator, 'deviceMemory', { get: () => 8 });
|
||||
|
||||
// 8. 最大触控点 (iPad = 5)
|
||||
Object.defineProperty(navigator, 'maxTouchPoints', { get: () => 5 });
|
||||
|
||||
return 'fingerprint_applied';
|
||||
}
|
||||
"""
|
||||
|
||||
# ─── DOM弹窗清理脚本 (扩展版) ───────────────────────────────────
|
||||
|
||||
OVERLAY_REMOVE_SCRIPT = """
|
||||
() => {
|
||||
const selectors = [
|
||||
// 下载/打开App弹窗
|
||||
'[class*="download"]', '[class*="open-app"]', '[class*="app-guide"]',
|
||||
'[class*="launch-app"]', '[class*="open-in-app"]', '[class*="app-launcher"]',
|
||||
'.open-in-app', '.app-launch-mask', '.download-tip', '.bottom-bar',
|
||||
'.download-banner', '.open-app-btn', '.app-download-tip',
|
||||
'.open-app-layer', '.download-guide-mask',
|
||||
'.open-in-app-bar', '.app-open-button',
|
||||
|
||||
// 模态弹窗
|
||||
'#app-launch-dialog', '#open-app-modal', '#download-modal',
|
||||
'[class*="modal"]', '[class*="overlay"]', '[class*="mask"]',
|
||||
|
||||
// 登录引导 (非登录页)
|
||||
'[class*="login-guide"]', '[class*="guide-layer"]',
|
||||
|
||||
// 更新提示
|
||||
'[class*="update-tip"]', '[class*="upgrade"]',
|
||||
|
||||
// 通用遮罩
|
||||
'[style*="position: fixed"][style*="z-index"]',
|
||||
];
|
||||
let count = 0;
|
||||
selectors.forEach(s => {
|
||||
document.querySelectorAll(s).forEach(el => {
|
||||
// 不删除 video 标签
|
||||
if (el.tagName !== 'VIDEO') {
|
||||
el.remove();
|
||||
count++;
|
||||
}
|
||||
});
|
||||
});
|
||||
document.body.style.overflow = '';
|
||||
document.body.style.overflowY = 'auto';
|
||||
document.body.style.position = '';
|
||||
return count;
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
# ─── 行为画像 ─────────────────────────────────────────────────
|
||||
|
||||
DEFAULT_PROFILE = {
|
||||
"action_delay_min": 0.8, # 动作最小间隔(秒)
|
||||
"action_delay_max": 3.0, # 动作最大间隔(秒)
|
||||
"view_duration_min": 5, # 最小观看(秒)
|
||||
"view_duration_max": 30, # 最大观看(秒)
|
||||
"typing_speed_ms": [80, 200], # 打字速度(ms/字符)
|
||||
"max_likes_per_hour": 15,
|
||||
"max_collects_per_hour": 5,
|
||||
"max_comments_per_hour": 3,
|
||||
"max_follows_per_hour": 10,
|
||||
"max_daily_actions": 150,
|
||||
"active_hours": [8, 23],
|
||||
}
|
||||
|
||||
|
||||
class BehaviorProfile:
|
||||
"""账号行为画像 — 控制操作频率和拟人化"""
|
||||
|
||||
def __init__(self, account_id: str, custom: dict = None):
|
||||
self.account_id = account_id
|
||||
self.params = dict(DEFAULT_PROFILE)
|
||||
if custom:
|
||||
self.params.update(custom)
|
||||
self._action_count = 0
|
||||
self._hourly_counts = {} # {action_type: count}
|
||||
self._session_start = time.time()
|
||||
|
||||
def random_delay(self) -> float:
|
||||
"""随机动作间隔"""
|
||||
return random.uniform(self.params["action_delay_min"],
|
||||
self.params["action_delay_max"])
|
||||
|
||||
def random_view_duration(self) -> int:
|
||||
"""随机观看时长(秒)"""
|
||||
return random.randint(self.params["view_duration_min"],
|
||||
self.params["view_duration_max"])
|
||||
|
||||
async def human_type(self, page, text: str):
|
||||
"""拟人化打字"""
|
||||
for char in text:
|
||||
delay = random.randint(*self.params["typing_speed_ms"]) / 1000
|
||||
await page.keyboard.type(char, delay=delay)
|
||||
|
||||
async def random_wait(self):
|
||||
"""随机等待(行为间隔)"""
|
||||
delay = self.random_delay()
|
||||
await asyncio.sleep(delay)
|
||||
|
||||
def can_act(self, action_type: str) -> bool:
|
||||
"""检查是否可以执行某种操作(频率限制)"""
|
||||
# 每日总量检查
|
||||
if self._action_count >= self.params["max_daily_actions"]:
|
||||
return False
|
||||
|
||||
# 每小时频率检查
|
||||
current_hour = datetime.now().hour
|
||||
if current_hour < self.params["active_hours"][0] or \
|
||||
current_hour > self.params["active_hours"][1]:
|
||||
return False
|
||||
|
||||
limits = {
|
||||
"like": self.params["max_likes_per_hour"],
|
||||
"collect": self.params["max_collects_per_hour"],
|
||||
"comment": self.params["max_comments_per_hour"],
|
||||
"follow": self.params["max_follows_per_hour"],
|
||||
}
|
||||
limit = limits.get(action_type, 100)
|
||||
hour_key = f"{action_type}_{datetime.now().hour}"
|
||||
if self._hourly_counts.get(hour_key, 0) >= limit:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def record_action(self, action_type: str):
|
||||
"""记录一次操作"""
|
||||
self._action_count += 1
|
||||
hour_key = f"{action_type}_{datetime.now().hour}"
|
||||
self._hourly_counts[hour_key] = self._hourly_counts.get(hour_key, 0) + 1
|
||||
|
||||
def get_stats(self) -> dict:
|
||||
"""获取当前会话统计"""
|
||||
return {
|
||||
"account": self.account_id,
|
||||
"session_seconds": int(time.time() - self._session_start),
|
||||
"total_actions": self._action_count,
|
||||
"hourly": dict(self._hourly_counts),
|
||||
}
|
||||
|
||||
def save_state(self, path: str):
|
||||
"""持久化状态到文件"""
|
||||
data = self.get_stats()
|
||||
Path(path).parent.mkdir(parents=True, exist_ok=True)
|
||||
Path(path).write_text(json.dumps(data, ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
# 高斯抖动 + 指数退避工具
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
|
||||
|
||||
def gaussian_sleep(base: float, jitter_ratio: float = 0.3, min_val: float = 0.3) -> float:
|
||||
"""高斯抖动睡眠: base 秒 ±jitter_ratio%, 最少 min_val 秒
|
||||
|
||||
用法:
|
||||
await anti_detection.gaussian_sleep(5.0) # 5s ±30%, 即 3.5-6.5s
|
||||
await anti_detection.gaussian_sleep(2.0, 0.5) # 2s ±50%
|
||||
"""
|
||||
import asyncio
|
||||
jitter = base * random.uniform(-jitter_ratio, jitter_ratio)
|
||||
delay = max(base + jitter, min_val)
|
||||
return delay # 返回秒数, 调用方自行 await asyncio.sleep()
|
||||
|
||||
|
||||
def gaussian_async_sleep(base: float, jitter_ratio: float = 0.3, min_val: float = 0.3):
|
||||
"""异步版本: await gaussian_async_sleep(5)"""
|
||||
import asyncio
|
||||
delay = gaussian_sleep(base, jitter_ratio, min_val)
|
||||
return asyncio.sleep(delay)
|
||||
|
||||
|
||||
def exponential_backoff(attempt: int, base_delay: float = 5.0, max_delay: float = 120.0) -> float:
|
||||
"""指数退避: 第 n 次重试等待 base * 2^n 秒, 上限 max_delay
|
||||
|
||||
用法:
|
||||
for i in range(3):
|
||||
try:
|
||||
await do_something()
|
||||
break
|
||||
except:
|
||||
delay = anti_detection.exponential_backoff(i)
|
||||
await asyncio.sleep(delay)
|
||||
"""
|
||||
delay = base_delay * (2 ** attempt)
|
||||
# 加 50% 高斯抖动
|
||||
delay *= (1 + random.uniform(-0.5, 0.5))
|
||||
return min(max(delay, 1.0), max_delay)
|
||||
|
||||
|
||||
def human_typing_delay(text: str) -> float:
|
||||
"""模拟人类打字速度: 每字符 50-150ms, 标点符号后额外 200-500ms"""
|
||||
base = len(text) * random.uniform(0.05, 0.15)
|
||||
# 标点符号加时
|
||||
extra = sum(random.uniform(0.2, 0.5) for c in text if c in ",。!?;:、")
|
||||
return base + extra
|
||||
|
||||
|
||||
def random_view_duration(min_s: float = 15, max_s: float = 120) -> float:
|
||||
"""随机观看时长, 对数正态分布 (更接近真实人类)"""
|
||||
import math
|
||||
mu = math.log((min_s + max_s) / 2)
|
||||
sigma = 0.5
|
||||
return max(min_s, min(max_s, random.lognormvariate(mu, sigma)))
|
||||
|
||||
|
||||
def jitter_mouse_position(x: int, y: int, radius: int = 5) -> tuple:
|
||||
"""鼠标位置抖动: 在 (x,y) 周围 radius 像素内随机偏移"""
|
||||
return (
|
||||
x + random.randint(-radius, radius),
|
||||
y + random.randint(-radius, radius),
|
||||
)
|
||||
|
||||
|
||||
__all__ = [
|
||||
'FINGERPRINT_SCRIPT', 'OVERLAY_REMOVE_SCRIPT',
|
||||
'gaussian_sleep', 'gaussian_async_sleep', 'exponential_backoff',
|
||||
'human_typing_delay', 'random_view_duration', 'jitter_mouse_position',
|
||||
'BehaviorProfile',
|
||||
]
|
||||
@@ -0,0 +1,392 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
🔐 AuthManager — 登录状态管理原子化模块 (v1.0.0)
|
||||
|
||||
原子操作:
|
||||
check_login_by_cookie(context) → bool # Cookie 检测(主方案)
|
||||
check_login_by_dom(page) → bool # DOM 检测(备选,仅移动端视口)
|
||||
get_login_status(context, page) → dict # 多维综合检测
|
||||
export_cookies(context, path) → int # 导出 Cookie 到文件
|
||||
inject_cookies(context, path) → int # 从文件注入 Cookie
|
||||
get_session_id(cookies) → str # 提取 sessionid
|
||||
|
||||
设计原则:
|
||||
1. Cookie 检测为主方案(不依赖 UI 布局)
|
||||
2. DOM 检测为备选(仅在 mobile=True 视口下有效)
|
||||
3. 所有函数无副作用(不写 DB/文件,除非函数名明确)
|
||||
4. 统一异常处理(所有异常返回 False/空值,不抛出)
|
||||
|
||||
最后更新: 2026-05-01
|
||||
"""
|
||||
|
||||
import json
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
__version__ = "1.0.0"
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# Cookie 检测(核心方案)
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
|
||||
# ── 各平台登录态 Cookie 规则 ──
|
||||
LOGIN_COOKIE_RULES = {
|
||||
"douyin": {
|
||||
"domain": "douyin",
|
||||
"names": ("sessionid", "sid_guard"),
|
||||
},
|
||||
"xiaohongshu": {
|
||||
"domain": "xiaohongshu",
|
||||
"names": ("a1", "web_session"),
|
||||
},
|
||||
"kuaishou": {
|
||||
"domain": "kuaishou",
|
||||
"names": ("kuaishou.login",),
|
||||
},
|
||||
"zhihu": {
|
||||
"domain": "zhihu",
|
||||
"names": ("z_c0", "d_c0"),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
async def check_login_by_cookie(context, platform: str = "douyin") -> bool:
|
||||
"""通过 Cookie 检测登录状态
|
||||
|
||||
检测平台特有的 session cookie 是否存在。
|
||||
这是主方案,不依赖页面 DOM 布局。
|
||||
|
||||
参数:
|
||||
context: Playwright BrowserContext
|
||||
platform: 平台名称 (douyin/xiaohongshu/kuaishou/zhihu)
|
||||
|
||||
返回:
|
||||
bool: True=已登录, False=未登录
|
||||
"""
|
||||
try:
|
||||
cookies = await context.cookies()
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
rule = LOGIN_COOKIE_RULES.get(platform)
|
||||
if not rule:
|
||||
return False
|
||||
|
||||
for c in cookies:
|
||||
domain = c.get("domain", "")
|
||||
name = c.get("name", "")
|
||||
if rule["domain"] in domain and name in rule["names"]:
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def check_login_by_cookie_sync(cookies: list, platform: str = "douyin") -> bool:
|
||||
"""同步版 Cookie 检测(从已获取的 Cookie 列表判断)
|
||||
|
||||
参数:
|
||||
cookies: Cookie 列表(每个为 dict,含 domain/name 字段)
|
||||
platform: 平台名称 (douyin/xiaohongshu/...)
|
||||
|
||||
返回:
|
||||
bool: True=已登录
|
||||
"""
|
||||
rule = LOGIN_COOKIE_RULES.get(platform)
|
||||
if not rule:
|
||||
return False
|
||||
|
||||
for c in cookies:
|
||||
domain = c.get("domain", "")
|
||||
name = c.get("name", "")
|
||||
if rule["domain"] in domain and name in rule["names"]:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def get_session_id(cookies: list, platform: str = "douyin") -> Optional[str]:
|
||||
"""从 Cookie 列表中提取 session cookie 值
|
||||
|
||||
参数:
|
||||
cookies: Cookie 列表
|
||||
platform: 平台名称
|
||||
|
||||
返回:
|
||||
str/None: session cookie 值或 None
|
||||
"""
|
||||
rule = LOGIN_COOKIE_RULES.get(platform)
|
||||
if not rule:
|
||||
return None
|
||||
for c in cookies:
|
||||
if c.get("name") in rule["names"]:
|
||||
return c.get("value")
|
||||
return None
|
||||
|
||||
|
||||
def count_platform_cookies(cookies: list, platform: str = "douyin") -> int:
|
||||
"""统计指定平台的 Cookie 数量"""
|
||||
rule = LOGIN_COOKIE_RULES.get(platform)
|
||||
if not rule:
|
||||
return 0
|
||||
domain_hint = rule["domain"]
|
||||
return sum(1 for c in cookies if domain_hint in c.get("domain", ""))
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# DOM 检测(备选方案)
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
|
||||
# 各平台登录态 DOM 指示器
|
||||
LOGIN_INDICATORS = {
|
||||
"douyin": [
|
||||
"[data-e2e='user-avatar']", # 主指示器
|
||||
"[data-e2e='user-detail']", # 用户详情
|
||||
],
|
||||
"xiaohongshu": [
|
||||
".user-avatar",
|
||||
".reds-count",
|
||||
],
|
||||
"kuaishou": [
|
||||
".avatar-container",
|
||||
],
|
||||
"zhihu": [
|
||||
".AppHeader-profile",
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
async def check_login_by_dom(page, platform: str = "douyin") -> bool:
|
||||
"""通过 DOM 元素检测登录状态(备选方案)
|
||||
|
||||
注意: 仅在移动端/平板视口 (mobile=True) 下有效。
|
||||
桌面端视口下抖音不渲染这些元素。
|
||||
|
||||
参数:
|
||||
page: Playwright Page
|
||||
platform: 平台名称(douyin/xiaohongshu/kuaishou/zhihu)
|
||||
|
||||
返回:
|
||||
bool: True=已登录
|
||||
"""
|
||||
selectors = LOGIN_INDICATORS.get(platform, [])
|
||||
if not selectors:
|
||||
return False
|
||||
|
||||
try:
|
||||
for sel in selectors:
|
||||
el = await page.query_selector(sel)
|
||||
if el:
|
||||
return True
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return False
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# 综合检测
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
|
||||
async def get_login_status(context, page=None, platform: str = "douyin") -> dict:
|
||||
"""多维综合登录检测
|
||||
|
||||
同时使用 Cookie 检测 + DOM 检测,返回详细诊断信息。
|
||||
|
||||
参数:
|
||||
context: Playwright BrowserContext
|
||||
page: Playwright Page(可选,不传则跳过 DOM 检测)
|
||||
platform: 平台名称
|
||||
|
||||
返回:
|
||||
dict:
|
||||
logged_in: bool # 最终判断(任意一个方法确认即可)
|
||||
cookie_ok: bool # Cookie 检测结果
|
||||
dom_ok: bool/None # DOM 检测结果(未传 page 为 None)
|
||||
cookie_count: int # douyin.com Cookie 数量
|
||||
session_id: str/None # sessionid 值
|
||||
method: str # 确认方法(cookie/dom/both)
|
||||
"""
|
||||
result = {
|
||||
"logged_in": False,
|
||||
"cookie_ok": False,
|
||||
"dom_ok": None,
|
||||
"cookie_count": 0,
|
||||
"session_id": None,
|
||||
"method": "",
|
||||
}
|
||||
|
||||
# Cookie 检测
|
||||
try:
|
||||
cookies = await context.cookies()
|
||||
result["cookie_ok"] = check_login_by_cookie_sync(cookies, platform)
|
||||
result["cookie_count"] = count_platform_cookies(cookies, platform)
|
||||
result["session_id"] = get_session_id(cookies, platform)
|
||||
except Exception:
|
||||
result["cookie_ok"] = False
|
||||
|
||||
# DOM 检测
|
||||
if page:
|
||||
try:
|
||||
result["dom_ok"] = await check_login_by_dom(page, platform)
|
||||
except Exception:
|
||||
result["dom_ok"] = False
|
||||
|
||||
# 综合判断
|
||||
if result["cookie_ok"] and result["dom_ok"]:
|
||||
result["logged_in"] = True
|
||||
result["method"] = "both"
|
||||
elif result["cookie_ok"]:
|
||||
result["logged_in"] = True
|
||||
result["method"] = "cookie"
|
||||
elif result["dom_ok"]:
|
||||
result["logged_in"] = True
|
||||
result["method"] = "dom"
|
||||
|
||||
return result
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# Cookie 导出 / 注入
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
|
||||
async def export_cookies(context, file_path: Path) -> int:
|
||||
"""导出当前浏览器的所有 Cookie 到 JSON 文件
|
||||
|
||||
参数:
|
||||
context: Playwright BrowserContext
|
||||
file_path: 导出文件路径
|
||||
|
||||
返回:
|
||||
int: 导出的 Cookie 数量(0 表示失败)
|
||||
"""
|
||||
try:
|
||||
cookies = await context.cookies()
|
||||
if not cookies:
|
||||
return 0
|
||||
|
||||
file_path = Path(file_path)
|
||||
file_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(file_path, "w", encoding="utf-8") as f:
|
||||
json.dump(cookies, f, indent=2, ensure_ascii=False)
|
||||
|
||||
return len(cookies)
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
|
||||
async def inject_cookies(context, file_path: Path) -> int:
|
||||
"""从 JSON 文件注入 Cookie 到浏览器
|
||||
|
||||
参数:
|
||||
context: Playwright BrowserContext
|
||||
file_path: Cookie JSON 文件路径
|
||||
|
||||
返回:
|
||||
int: 成功注入的 Cookie 数量(0 表示失败)
|
||||
"""
|
||||
try:
|
||||
file_path = Path(file_path)
|
||||
if not file_path.exists():
|
||||
return 0
|
||||
|
||||
cookies = json.loads(file_path.read_text(encoding="utf-8"))
|
||||
if not cookies:
|
||||
return 0
|
||||
|
||||
success = 0
|
||||
for c in cookies:
|
||||
try:
|
||||
await context.add_cookies([{
|
||||
"name": c["name"],
|
||||
"value": c["value"],
|
||||
"domain": c.get("domain", ""),
|
||||
"path": c.get("path", "/"),
|
||||
"httpOnly": c.get("httpOnly", False),
|
||||
"secure": c.get("secure", False),
|
||||
"sameSite": c.get("sameSite", "Lax"),
|
||||
}])
|
||||
success += 1
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return success
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
|
||||
async def wait_for_login(context, page, platform: str = "douyin",
|
||||
timeout: int = 300, interval: int = 5) -> dict:
|
||||
"""等待用户手动登录(轮询 Cookie + DOM)
|
||||
|
||||
用于自动化流程中等待用户扫码登录的场景。
|
||||
不阻塞终端,自动检测到登录后返回。
|
||||
|
||||
参数:
|
||||
context: Playwright BrowserContext
|
||||
page: Playwright Page
|
||||
platform: 平台名称
|
||||
timeout: 超时秒数(默认 300s = 5分钟)
|
||||
interval: 轮询间隔秒数(默认 5s)
|
||||
|
||||
返回:
|
||||
dict: get_login_status() 的返回结果
|
||||
|
||||
如果超时,返回 logged_in=False
|
||||
"""
|
||||
for i in range(timeout // interval):
|
||||
await asyncio.sleep(interval)
|
||||
status = await get_login_status(context, page, platform)
|
||||
if status["logged_in"]:
|
||||
return status
|
||||
print(f" ⏳ 等待登录... ({i * interval + interval}s/{timeout}s)")
|
||||
# 超时
|
||||
print(f" ⏰ 等待超时 ({timeout}s)")
|
||||
return {"logged_in": False, "cookie_ok": False, "dom_ok": False,
|
||||
"cookie_count": 0, "session_id": None, "method": ""}
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
# 独立 CLI 测试入口
|
||||
# ══════════════════════════════════════════════════════════════
|
||||
|
||||
async def _test_check(port: int = 9222):
|
||||
"""测试连接指定端口的浏览器并检测登录状态"""
|
||||
import urllib.request
|
||||
from patchright.async_api import async_playwright
|
||||
|
||||
try:
|
||||
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
with opener.open(f"http://localhost:{port}/json/version", timeout=5) as r:
|
||||
ws_url = json.loads(r.read())["webSocketDebuggerUrl"]
|
||||
except Exception as e:
|
||||
print(f"❌ 无法连接端口 {port}: {e}")
|
||||
return
|
||||
|
||||
pw = await async_playwright().start()
|
||||
try:
|
||||
browser = await pw.chromium.connect_over_cdp(ws_url)
|
||||
page = browser.contexts[0].pages[0]
|
||||
context = browser.contexts[0]
|
||||
|
||||
await page.goto("https://www.douyin.com/", wait_until="domcontentloaded", timeout=15000)
|
||||
await asyncio.sleep(3)
|
||||
|
||||
status = await get_login_status(context, page, platform="douyin")
|
||||
domain = "douyin"
|
||||
print(f"\n📊 登录状态检测 (端口 {port})")
|
||||
print(f" Cookie OK: {'✅' if status['cookie_ok'] else '❌'}")
|
||||
print(f" DOM OK: {'✅' if status['dom_ok'] else '❌'}")
|
||||
print(f" {domain} Cookie: {status['cookie_count']} 个")
|
||||
print(f" session: {status['session_id'][:20] + '...' if status['session_id'] else '❌'}")
|
||||
print(f" 最终判断: {'✅ 已登录 (via ' + status['method'] + ')' if status['logged_in'] else '❌ 未登录'}")
|
||||
|
||||
await pw.stop()
|
||||
except Exception as e:
|
||||
print(f"❌ 检测失败: {e}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
port = int(sys.argv[1]) if len(sys.argv) > 1 else 9222
|
||||
asyncio.run(_test_check(port))
|
||||
@@ -0,0 +1,361 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
⚠️ 已废弃 — 请使用 cdp_connector.py 替代。
|
||||
本文件保留以供 orchestrator.py 引用,新代码禁止导入。
|
||||
|
||||
BrowserManager — 浏览器生命周期管理 (v1.0.0)
|
||||
|
||||
功能:
|
||||
- Chrome CDP 模式(优雅启动/关闭)
|
||||
- Camoufox (Firefox) 原生模式
|
||||
- 端口冲突检测与优雅清理
|
||||
- 启动后 CDP 等待就绪
|
||||
|
||||
用法:
|
||||
bm = BrowserManager()
|
||||
await bm.launch("douyin_01") # 读取配置启动
|
||||
await bm.close() # 优雅关闭
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import signal
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from local_paths import config_path, profiles_path, data_path
|
||||
|
||||
# 版本
|
||||
__version__ = "1.0.0"
|
||||
|
||||
|
||||
class BrowserManagerError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class BrowserManager:
|
||||
"""浏览器管理器 — 优雅启停"""
|
||||
|
||||
def __init__(self):
|
||||
self.process: Optional[subprocess.Popen] = None
|
||||
self._account = None
|
||||
self._config = None
|
||||
self._connector = None # cdp_connector 实例
|
||||
self._camoufox_handle = None # AsyncCamoufox 实例
|
||||
self.browser_type = None # "chrome" | "camoufox"
|
||||
|
||||
# ─── 加载账号配置 ────────────────────────────────────────────
|
||||
|
||||
def _load_account(self, account_id: str) -> dict:
|
||||
"""从 accounts.yaml 加载账号配置"""
|
||||
import yaml
|
||||
acct_file = config_path("accounts.yaml")
|
||||
with open(acct_file) as f:
|
||||
data = yaml.safe_load(f)
|
||||
for a in data.get("accounts", []):
|
||||
if a["id"] == account_id:
|
||||
return a
|
||||
raise BrowserManagerError(f"账号不存在: {account_id}")
|
||||
|
||||
# ─── Chrome 启动 ────────────────────────────────────────────
|
||||
|
||||
def _find_chrome(self) -> str:
|
||||
"""查找 Chrome 可执行文件"""
|
||||
candidates = [
|
||||
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
|
||||
]
|
||||
for c in candidates:
|
||||
if Path(c).exists():
|
||||
return c
|
||||
# 尝试通过 Playwright 找
|
||||
try:
|
||||
from patchright import async_api
|
||||
return str(async_api.sync_playwright().chromium.executable_path)
|
||||
except Exception:
|
||||
raise BrowserManagerError("Chrome 未安装")
|
||||
|
||||
def _graceful_kill(self, port: int, wait: float = 3.0):
|
||||
"""优雅停止端口上的进程 (SIGTERM → 等待 → SIGKILL)"""
|
||||
try:
|
||||
r = subprocess.run(["lsof", "-ti", f":{port}"],
|
||||
capture_output=True, text=True, timeout=5)
|
||||
pids = [int(p) for p in r.stdout.strip().split() if p.strip()]
|
||||
if not pids:
|
||||
return
|
||||
# SIGTERM 先
|
||||
for pid in pids:
|
||||
try:
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
# 等待进程退出
|
||||
deadline = time.time() + wait
|
||||
while time.time() < deadline:
|
||||
alive = False
|
||||
for pid in pids[:]:
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
alive = True
|
||||
except ProcessLookupError:
|
||||
pids.remove(pid)
|
||||
if not alive:
|
||||
return
|
||||
time.sleep(0.3)
|
||||
# 还没退出的强制杀
|
||||
for pid in pids:
|
||||
try:
|
||||
os.kill(pid, signal.SIGKILL)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
except subprocess.TimeoutExpired:
|
||||
pass
|
||||
|
||||
async def _launch_chrome(self, account: dict):
|
||||
"""启动 Chrome — 使用 subprocess + CDP(不走 Playwright)
|
||||
|
||||
关键:Playwright 内部协议会被抖音检测 → 桌面版
|
||||
改用 subprocess 启动 + CDP 连接 → 手机版正常
|
||||
"""
|
||||
chrome = self._find_chrome()
|
||||
port = account.get("port", 9222)
|
||||
profile_dir = str(profiles_path() / account.get("profile_dir", account["id"]))
|
||||
|
||||
# 优雅清理旧进程
|
||||
self._graceful_kill(port, wait=3.0)
|
||||
await asyncio.sleep(1)
|
||||
|
||||
# 启动 Chrome(用 --remote-debugging-port 但不走 Playwright)
|
||||
cmd = [
|
||||
chrome,
|
||||
f"--remote-debugging-port={port}",
|
||||
f"--user-data-dir={profile_dir}",
|
||||
"--no-first-run", "--no-default-browser-check",
|
||||
"--disable-extensions",
|
||||
"--window-size=702,783",
|
||||
"about:blank",
|
||||
]
|
||||
self.process = subprocess.Popen(
|
||||
cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
|
||||
)
|
||||
self.browser_type = "chrome"
|
||||
|
||||
# 等待 CDP 就绪
|
||||
import urllib.request
|
||||
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
for _ in range(15):
|
||||
try:
|
||||
with opener.open(f"http://localhost:{port}/json/version", timeout=3) as r:
|
||||
if r.status == 200:
|
||||
# 用 cdp_connector 建立连接
|
||||
from cdp_connector import CDPConnector
|
||||
self._connector = CDPConnector(port=port)
|
||||
await self._connector.connect()
|
||||
self._page = self._connector.page
|
||||
return port
|
||||
except Exception:
|
||||
pass
|
||||
await asyncio.sleep(1)
|
||||
raise BrowserManagerError(f"Chrome CDP 启动超时 (端口 {port})")
|
||||
return port
|
||||
|
||||
# ─── Camoufox 启动 ──────────────────────────────────────────
|
||||
|
||||
async def _launch_camoufox(self, account: dict):
|
||||
"""启动 Camoufox (Firefox) 原生模式"""
|
||||
from camoufox.async_api import AsyncCamoufox
|
||||
|
||||
self._camoufox_handle = AsyncCamoufox(
|
||||
headless=False,
|
||||
window=(702, 783),
|
||||
locale=["zh-CN"],
|
||||
)
|
||||
browser = await self._camoufox_handle.start()
|
||||
self.browser_type = "camoufox"
|
||||
return browser # 返回 AsyncBrowser 对象
|
||||
|
||||
# ─── 统一入口 ───────────────────────────────────────────────
|
||||
|
||||
async def launch(self, account_id: str) -> dict:
|
||||
"""
|
||||
启动浏览器
|
||||
返回: {"type": "chrome"|"camoufox", "port": int|None, "browser": object|None}
|
||||
"""
|
||||
self._account = self._load_account(account_id)
|
||||
bt = self._account.get("browser_type", "chrome")
|
||||
|
||||
if bt == "camoufox" or bt == "firefox":
|
||||
browser = await self._launch_camoufox(self._account)
|
||||
return {"type": "camoufox", "port": self._account.get("port", 9301), "browser": browser}
|
||||
else:
|
||||
await self._launch_chrome(self._account)
|
||||
return {"type": "chrome", "port": self._account.get("port", 9222), "page": self._page}
|
||||
|
||||
# ─── 获取已连接的 page ────────────────────────────────────
|
||||
|
||||
@property
|
||||
def page(self):
|
||||
"""获取 Playwright page 对象"""
|
||||
if hasattr(self, '_page') and self._page:
|
||||
return self._page
|
||||
return None
|
||||
|
||||
# ─── 关闭 ──────────────────────────────────────────────────
|
||||
|
||||
# ─── 登录态管理 ──────────────────────────────────────────
|
||||
|
||||
async def detect_login_state(self, page) -> dict:
|
||||
"""检测登录状态,返回:
|
||||
{"status": "logged_in"|"cookie_available"|"lost",
|
||||
"method": "profile"|"cookie"|"",
|
||||
"detail": str}
|
||||
"""
|
||||
# 1. 从已启动的浏览器检查 Cookie
|
||||
try:
|
||||
cookies = await page.context.cookies()
|
||||
dy = [c for c in cookies if "douyin" in c.get("domain", "")]
|
||||
has_session = any(c["name"] == "sessionid" for c in dy)
|
||||
if has_session:
|
||||
return {"status": "logged_in", "method": "profile", "detail": "浏览器已有 sessionid"}
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# 2. 检查是否有保存的 Cookie 文件
|
||||
cookie_file = data_path("cookies") / f"{self._account['id']}_cookies.json"
|
||||
if cookie_file.exists():
|
||||
try:
|
||||
saved_cookies = json.loads(cookie_file.read_text())
|
||||
has_session = any(c.get("name") == "sessionid" and c.get("domain", "").find("douyin") >= 0 for c in saved_cookies)
|
||||
if has_session:
|
||||
return {"status": "cookie_available", "method": "cookie",
|
||||
"detail": f"Cookie文件存在 ({cookie_file.name})", "file": str(cookie_file)}
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# 3. 登录态丢失
|
||||
return {"status": "lost", "method": "", "detail": "无可用登录态"}
|
||||
|
||||
async def inject_cookies(self, page) -> bool:
|
||||
"""注入保存的 Cookie 到浏览器"""
|
||||
acct_id = self._account["id"]
|
||||
cookie_file = data_path("cookies") / f"{acct_id}_cookies.json"
|
||||
if not cookie_file.exists():
|
||||
return False
|
||||
try:
|
||||
cookies = json.loads(cookie_file.read_text())
|
||||
count = 0
|
||||
for c in cookies:
|
||||
try:
|
||||
await page.context.add_cookies([{
|
||||
"name": c["name"], "value": c["value"],
|
||||
"domain": c["domain"], "path": c.get("path", "/"),
|
||||
"httpOnly": c.get("httpOnly", False),
|
||||
"secure": c.get("secure", False),
|
||||
"sameSite": c.get("sameSite", "Lax"),
|
||||
}])
|
||||
count += 1
|
||||
except Exception:
|
||||
pass
|
||||
return count > 0
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
# ─── 激活手机模式 ────────────────────────────────────────
|
||||
|
||||
async def activate_mobile_mode(self, page):
|
||||
"""激活手机模式:通过CDP设置UA+视口,然后刷新"""
|
||||
if self.browser_type == "chrome" and hasattr(self, '_connector') and self._connector:
|
||||
try:
|
||||
cdp = self._connector.cdp_session
|
||||
if not cdp:
|
||||
cdp = await self._connector.context.new_cdp_session(page)
|
||||
await cdp.send("Network.setUserAgentOverride", {
|
||||
"userAgent": "Mozilla/5.0 (iPad; CPU OS 16_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.0 Mobile/15E148 Safari/604.1"
|
||||
})
|
||||
await cdp.send("Emulation.setDeviceMetricsOverride", {
|
||||
"width": 702, "height": 783, "deviceScaleFactor": 2,
|
||||
"mobile": True, "screenWidth": 702, "screenHeight": 783,
|
||||
})
|
||||
except Exception:
|
||||
pass
|
||||
# 刷新激活
|
||||
try:
|
||||
await page.reload(wait_until="domcontentloaded")
|
||||
except Exception:
|
||||
await page.goto("https://www.douyin.com/", timeout=30000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(5)
|
||||
|
||||
async def apply_fingerprint_protection(self, page):
|
||||
"""应用浏览器指纹保护"""
|
||||
from anti_detection import FINGERPRINT_SCRIPT
|
||||
try:
|
||||
result = await page.evaluate(FINGERPRINT_SCRIPT)
|
||||
print(f" 🔒 指纹保护: {result}")
|
||||
except Exception as e:
|
||||
print(f" ⚠️ 指纹保护: {type(e).__name__}")
|
||||
|
||||
async def remove_overlays(self, page):
|
||||
"""清理页面弹窗"""
|
||||
from anti_detection import OVERLAY_REMOVE_SCRIPT
|
||||
try:
|
||||
count = await page.evaluate(OVERLAY_REMOVE_SCRIPT)
|
||||
if count > 0:
|
||||
print(f" 🧹 清理 {count} 个弹窗")
|
||||
return count
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
async def close(self):
|
||||
"""优雅关闭浏览器"""
|
||||
# 关闭 CDP connector (Chrome)
|
||||
if hasattr(self, '_connector') and self._connector:
|
||||
try:
|
||||
await self._connector.close()
|
||||
except Exception:
|
||||
pass
|
||||
self._connector = None
|
||||
|
||||
# 关闭 Camoufox
|
||||
if hasattr(self, '_camoufox_handle') and self._camoufox_handle:
|
||||
try:
|
||||
await self._camoufox_handle.stop()
|
||||
except Exception:
|
||||
pass
|
||||
self._camoufox_handle = None
|
||||
|
||||
# 关闭 Chrome 进程 (SIGTERM)
|
||||
if hasattr(self, 'process') and self.process:
|
||||
try:
|
||||
self.process.terminate()
|
||||
try:
|
||||
self.process.wait(timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
self.process.kill()
|
||||
self.process.wait(timeout=2)
|
||||
except Exception:
|
||||
pass
|
||||
self.process = None
|
||||
try:
|
||||
self.process.terminate()
|
||||
try:
|
||||
self.process.wait(timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
self.process.kill()
|
||||
self.process.wait(timeout=2)
|
||||
except Exception:
|
||||
pass
|
||||
self.process = None
|
||||
|
||||
await asyncio.sleep(1)
|
||||
|
||||
# ─── 获取账号信息 ──────────────────────────────────────────
|
||||
|
||||
@property
|
||||
def account_name(self) -> str:
|
||||
return self._account.get("display_name", self._account.get("id", "?"))
|
||||
|
||||
@property
|
||||
def account_id(self) -> str:
|
||||
return self._account.get("id", "?")
|
||||
@@ -0,0 +1,288 @@
|
||||
"""
|
||||
browser_utils.py — 浏览器优雅退出 + 前置检查 通用模块
|
||||
|
||||
功能:
|
||||
1. 优雅退出: 信号处理(SIGTERM/SIGINT) + 超时自动关闭(30min) + cleanup
|
||||
2. 前置检查: 检查账号浏览器是否已在运行
|
||||
3. PID 文件管理: 记录浏览器进程, 防止重复启动
|
||||
|
||||
用法:
|
||||
from browser_utils import GracefulBrowser, check_browser_running
|
||||
|
||||
# 包装 CDPConnector
|
||||
browser = GracefulBrowser(conn, account_id, timeout_minutes=30)
|
||||
browser.setup() # 注册信号处理器 + 前置检查
|
||||
|
||||
# 正常退出
|
||||
await browser.shutdown()
|
||||
"""
|
||||
|
||||
import asyncio, os, signal, time, json
|
||||
from pathlib import Path
|
||||
|
||||
# PID 文件存储目录
|
||||
from matrix_mgmt import AGENT_LOCAL
|
||||
PID_DIR = AGENT_LOCAL / "runtime" / "browser_pids"
|
||||
PID_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def get_pid_file(account_id: str) -> Path:
|
||||
"""获取账号对应的 PID 文件路径"""
|
||||
safe_name = account_id.replace("/", "_").replace(" ", "_")
|
||||
return PID_DIR / f"{safe_name}.pid"
|
||||
|
||||
|
||||
def is_browser_running(account_id: str) -> bool:
|
||||
"""检查账号的浏览器是否已在运行
|
||||
|
||||
通过 PID 文件 + 进程检查双重验证
|
||||
"""
|
||||
pid_file = get_pid_file(account_id)
|
||||
if not pid_file.exists():
|
||||
return False
|
||||
|
||||
try:
|
||||
data = json.loads(pid_file.read_text())
|
||||
pid = data.get("pid", 0)
|
||||
if pid and _pid_exists(pid):
|
||||
# 确认进程是我们启动的 Camoufox
|
||||
import subprocess
|
||||
r = subprocess.run(
|
||||
["ps", "-p", str(pid), "-o", "comm="],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
proc_name = r.stdout.strip().lower()
|
||||
if any(kw in proc_name for kw in ["camoufox", "firefox", "python"]):
|
||||
return True
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# PID 文件过期,清理
|
||||
try:
|
||||
pid_file.unlink()
|
||||
except Exception:
|
||||
pass
|
||||
return False
|
||||
|
||||
|
||||
def write_pid_file(account_id: str, pid: int, machine: str = ""):
|
||||
"""写入 PID 文件"""
|
||||
data = {
|
||||
"account_id": account_id,
|
||||
"pid": pid,
|
||||
"machine": machine,
|
||||
"started_at": time.time(),
|
||||
}
|
||||
pid_file = get_pid_file(account_id)
|
||||
pid_file.write_text(json.dumps(data, indent=2))
|
||||
|
||||
|
||||
def remove_pid_file(account_id: str):
|
||||
"""删除 PID 文件"""
|
||||
pid_file = get_pid_file(account_id)
|
||||
try:
|
||||
if pid_file.exists():
|
||||
pid_file.unlink()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _pid_exists(pid: int) -> bool:
|
||||
"""检查 PID 是否存活(跨平台)"""
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
return True
|
||||
except (OSError, ProcessLookupError):
|
||||
return False
|
||||
|
||||
|
||||
def cleanup_stale_pids():
|
||||
"""清理所有过期的 PID 文件(启动时调用)"""
|
||||
for pid_file in PID_DIR.glob("*.pid"):
|
||||
try:
|
||||
data = json.loads(pid_file.read_text())
|
||||
pid = data.get("pid", 0)
|
||||
if not pid or not _pid_exists(pid):
|
||||
pid_file.unlink()
|
||||
except Exception:
|
||||
try:
|
||||
pid_file.unlink()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
class GracefulBrowser:
|
||||
"""浏览器优雅退出管理器
|
||||
|
||||
包装 CDPConnector,自动处理:
|
||||
- 信号处理 (SIGTERM/SIGINT → 优雅关闭)
|
||||
- 超时自动退出 (默认 30 分钟)
|
||||
- PID 文件管理
|
||||
- cleanup
|
||||
|
||||
用法:
|
||||
conn = CDPConnector(...)
|
||||
gb = GracefulBrowser(conn, account_id="douyin_01", timeout_minutes=30)
|
||||
await gb.setup() # 前置检查 + 信号注册
|
||||
await conn.connect() # 正常使用
|
||||
# ...
|
||||
await gb.shutdown() # 优雅退出
|
||||
"""
|
||||
|
||||
def __init__(self, cdp_connector, account_id: str = "",
|
||||
timeout_minutes: int = 30, machine: str = ""):
|
||||
self.conn = cdp_connector
|
||||
self.account_id = account_id
|
||||
self.timeout_minutes = timeout_minutes
|
||||
self.machine = machine
|
||||
self._shutdown_flag = False
|
||||
self._original_handlers = {}
|
||||
|
||||
async def setup(self, check_running: bool = True):
|
||||
"""初始化: 前置检查 + 注册信号
|
||||
|
||||
Args:
|
||||
check_running: 是否检查浏览器已在运行
|
||||
"""
|
||||
# 清理过期 PID
|
||||
cleanup_stale_pids()
|
||||
|
||||
# 前置检查: 浏览器是否已在运行
|
||||
if check_running and self.account_id:
|
||||
if is_browser_running(self.account_id):
|
||||
print(f" ⚠️ 账号 '{self.account_id}' 的浏览器已在运行")
|
||||
print(f" 将尝试优雅关闭旧进程...")
|
||||
await self._close_existing()
|
||||
|
||||
# 注册信号处理器
|
||||
self._register_signal_handlers()
|
||||
|
||||
# 写入 PID 文件
|
||||
if self.account_id:
|
||||
write_pid_file(self.account_id, os.getpid(), self.machine)
|
||||
|
||||
def _register_signal_handlers(self):
|
||||
"""注册 SIGTERM/SIGINT → 优雅关闭"""
|
||||
for sig in (signal.SIGTERM, signal.SIGINT):
|
||||
original = signal.getsignal(sig)
|
||||
self._original_handlers[sig] = original
|
||||
|
||||
def _handler(s, f, sig=sig, orig=original):
|
||||
if self._shutdown_flag:
|
||||
return # 防止重复调用
|
||||
self._shutdown_flag = True
|
||||
print(f"\n ⚠️ 收到信号 {sig.name}, 正在优雅关闭...")
|
||||
# 创建新的事件循环任务来执行异步关闭
|
||||
try:
|
||||
loop = asyncio.get_event_loop()
|
||||
if loop.is_running():
|
||||
asyncio.ensure_future(self._async_shutdown())
|
||||
else:
|
||||
loop.run_until_complete(self._async_shutdown())
|
||||
except Exception:
|
||||
pass
|
||||
# 调用原始处理器(如果有)
|
||||
if orig and orig != signal.SIG_DFL and orig != signal.SIG_IGN:
|
||||
try:
|
||||
orig(s, f)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
signal.signal(sig, _handler)
|
||||
except (ValueError, RuntimeError):
|
||||
pass # 不在主线程时忽略
|
||||
|
||||
def start_auto_shutdown_timer(self):
|
||||
"""启动超时自动退出计时器(登录完成后调用)"""
|
||||
if self.timeout_minutes <= 0:
|
||||
return
|
||||
|
||||
async def _timer():
|
||||
try:
|
||||
await asyncio.sleep(self.timeout_minutes * 60)
|
||||
if not self._shutdown_flag:
|
||||
print(f"\n ⏰ 超时 {self.timeout_minutes} 分钟,自动关闭浏览器")
|
||||
await self._async_shutdown()
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
|
||||
self._timer_task = asyncio.ensure_future(_timer())
|
||||
|
||||
def cancel_auto_shutdown(self):
|
||||
"""取消超时计时器"""
|
||||
if hasattr(self, '_timer_task'):
|
||||
self._timer_task.cancel()
|
||||
|
||||
async def shutdown(self):
|
||||
"""主动优雅退出"""
|
||||
await self._async_shutdown()
|
||||
|
||||
async def _async_shutdown(self):
|
||||
"""实际关闭逻辑"""
|
||||
if self._shutdown_flag:
|
||||
return
|
||||
self._shutdown_flag = True
|
||||
|
||||
print(" 🧹 正在清理浏览器资源...")
|
||||
try:
|
||||
# 1. 先移除 PID 文件
|
||||
if self.account_id:
|
||||
remove_pid_file(self.account_id)
|
||||
|
||||
# 2. 委托给 cdp_connector 的 close 方法(它包含了正确的关闭顺序)
|
||||
if hasattr(self.conn, 'close'):
|
||||
print(" 🦊 委托 cdp_connector 优雅关闭...")
|
||||
await self.conn.close()
|
||||
else:
|
||||
# 3. 兜底:尝试各种关闭方式
|
||||
if hasattr(self.conn, '_camoufox') and self.conn._camoufox:
|
||||
print(" 🦊 关闭 Camoufox...")
|
||||
try:
|
||||
await self.conn._camoufox.stop()
|
||||
except AttributeError:
|
||||
pass
|
||||
try:
|
||||
if hasattr(self.conn._camoufox, 'browser') and self.conn._camoufox.browser:
|
||||
await self.conn._camoufox.browser.close()
|
||||
except Exception:
|
||||
pass
|
||||
if hasattr(self.conn, '_camoufox_browser') and self.conn._camoufox_browser:
|
||||
print(" 🦊 关闭浏览器...")
|
||||
await self.conn._camoufox_browser.close()
|
||||
if hasattr(self.conn, '_playwright') and self.conn._playwright:
|
||||
print(" 🎭 关闭 Playwright...")
|
||||
await self.conn._playwright.stop()
|
||||
|
||||
print(" ✅ 浏览器已优雅关闭")
|
||||
except Exception as e:
|
||||
print(f" ⚠️ 关闭时出错: {e}")
|
||||
|
||||
# 强制退出(如果在信号处理器中调用)
|
||||
# 给异步关闭一些时间
|
||||
await asyncio.sleep(1)
|
||||
|
||||
async def _close_existing(self):
|
||||
"""优雅关闭已存在的浏览器进程"""
|
||||
pid_file = get_pid_file(self.account_id)
|
||||
if not pid_file.exists():
|
||||
return
|
||||
|
||||
try:
|
||||
data = json.loads(pid_file.read_text())
|
||||
pid = data.get("pid", 0)
|
||||
if pid and _pid_exists(pid):
|
||||
print(f" 🔄 向旧进程 (PID {pid}) 发送 SIGTERM...")
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
# 等待最多 10 秒
|
||||
for _ in range(10):
|
||||
await asyncio.sleep(1)
|
||||
if not _pid_exists(pid):
|
||||
print(f" ✅ 旧进程已退出")
|
||||
break
|
||||
else:
|
||||
print(f" ⚠️ 旧进程未响应,发送 SIGKILL")
|
||||
os.kill(pid, signal.SIGKILL)
|
||||
pid_file.unlink()
|
||||
except Exception as e:
|
||||
print(f" ⚠️ 关闭旧进程失败: {e}")
|
||||
@@ -0,0 +1,180 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
⚠️ 已废弃 — 请使用 Dashboard 采集 API 替代。
|
||||
本文件保留以供 douyin_login.py 引用,新代码禁止导入。
|
||||
|
||||
profile_scraper.py — 账号主页资料自动采集引擎 v1.0
|
||||
|
||||
功能:
|
||||
- 对每个已登录的 Camoufox 账号
|
||||
- 打开平台个人主页
|
||||
- 提取: 昵称 / 粉丝 / 关注 / 简介 / 作品数 / 最热视频
|
||||
- 存储到 machines/{uid}/profiles/{account_id}.json
|
||||
- 集成到养号流程 (每次养号后自动刷新)
|
||||
|
||||
版本: 1.0.0 | 更新: 2026-05-31
|
||||
"""
|
||||
import json, os, asyncio, re
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
HOME = Path.home()
|
||||
AGENT_SYNC = HOME / "workbuddy-agent-os" / "agent-sync"
|
||||
AGENT_LOCAL = HOME / "workbuddy-agent-os" / "agent-local"
|
||||
|
||||
PROFILES_DIR = AGENT_SYNC / "04_memory" / "cross_machine" / "machines"
|
||||
LOCAL_PROFILES = AGENT_LOCAL / "runtime" / "profiles"
|
||||
|
||||
# ── 平台主页 URL 模板 (按平台扩展) ─────────────────────
|
||||
# 格式: platform → (url_pattern, selectors)
|
||||
PLATFORM_PROFILES = {
|
||||
"douyin": {
|
||||
"url": "https://www.douyin.com/user/self",
|
||||
"selectors": {
|
||||
"display_name": ".nickname, .account-info .name, h1",
|
||||
"followers": ".follower-count, .follower-num, span[class*=follower]",
|
||||
"following": ".following-count, .following-num",
|
||||
"bio": ".signature, .desc, .bio",
|
||||
"video_count": ".video-count, .post-count, span[class*=video]",
|
||||
"likes_received": ".like-count, .total-favorited",
|
||||
}
|
||||
},
|
||||
"xiaohongshu": {
|
||||
"url": "https://www.xiaohongshu.com/user/profile",
|
||||
"selectors": {
|
||||
"display_name": ".username, .user-name, h1",
|
||||
"followers": ".follower-count, span[class*=fans]",
|
||||
"following": ".following-count, span[class*=follow]",
|
||||
"bio": ".desc, .user-desc, .introduction",
|
||||
"video_count": ".note-count, .post-count",
|
||||
"collects_received": ".collected-count",
|
||||
}
|
||||
},
|
||||
"kuaishou": {
|
||||
"url": "https://www.kuaishou.com/profile",
|
||||
"selectors": {
|
||||
"display_name": ".name, .username",
|
||||
"followers": ".fans, .follower-count",
|
||||
"following": ".follow, .following-count",
|
||||
"bio": ".description, .signature",
|
||||
"video_count": ".works-count, .video-count",
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class ProfileScraper:
|
||||
"""账号主页资料采集器"""
|
||||
|
||||
def __init__(self):
|
||||
self._local_uid = self._resolve_uid()
|
||||
self._local_hostname = self._resolve_hostname()
|
||||
LOCAL_PROFILES.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def refresh_all(self) -> list[dict]:
|
||||
"""刷新所有本机已登录账号的资料"""
|
||||
try:
|
||||
from matrix_mgmt import MatrixManager
|
||||
mgr = MatrixManager()
|
||||
accounts = mgr.list_accounts()
|
||||
local = [a for a in accounts if a.get("is_local") and a.get("_status") == "logged_in"]
|
||||
except ImportError:
|
||||
return [{"error": "matrix_mgmt not available"}]
|
||||
|
||||
results = []
|
||||
for acct in local:
|
||||
try:
|
||||
result = self._scrape_one(acct)
|
||||
results.append(result)
|
||||
except Exception as e:
|
||||
results.append({"account": acct["id"], "error": str(e)})
|
||||
return results
|
||||
|
||||
def refresh(self, account_ids: list[str]) -> list[dict]:
|
||||
"""刷新指定账号的资料"""
|
||||
try:
|
||||
from matrix_mgmt import MatrixManager
|
||||
mgr = MatrixManager()
|
||||
accounts = mgr.list_accounts()
|
||||
targets = [a for a in accounts if a["id"] in account_ids]
|
||||
except ImportError:
|
||||
return [{"error": "matrix_mgmt not available"}]
|
||||
|
||||
results = []
|
||||
for acct in targets:
|
||||
try:
|
||||
result = self._scrape_one(acct)
|
||||
results.append(result)
|
||||
except Exception as e:
|
||||
results.append({"account": acct["id"], "error": str(e)})
|
||||
return results
|
||||
|
||||
def _scrape_one(self, acct: dict) -> dict:
|
||||
"""采集单个账号的主页资料
|
||||
|
||||
实际执行时通过 matrix_nurture 的 Camoufox 实例完成。
|
||||
此处为框架接口——Camoufox 操作由 nurture_runner 调用。
|
||||
"""
|
||||
platform = acct.get("platform", "")
|
||||
profile_config = PLATFORM_PROFILES.get(platform)
|
||||
if not profile_config:
|
||||
return {"account": acct["id"], "error": f"不支持的平台: {platform}"}
|
||||
|
||||
# 构建结果框架
|
||||
result = {
|
||||
"account_id": acct["id"],
|
||||
"platform": platform,
|
||||
"collected_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
|
||||
# 采集流程:
|
||||
# 1. 启动 Camoufox (用该账号的 identity_dir)
|
||||
# 2. 导航到 platform.url
|
||||
# 3. 等待页面加载 + 提取 selectors
|
||||
# 4. 关闭浏览器
|
||||
# 5. 存储结果
|
||||
|
||||
# 待 Camoufox API 就绪后实现具体采集逻辑
|
||||
# 当前返回框架占位
|
||||
result["_status"] = "pending_implementation"
|
||||
result["_note"] = "采集逻辑待 Camoufox API 集成"
|
||||
|
||||
# 写入本地存储
|
||||
self._save_profile(result)
|
||||
return result
|
||||
|
||||
def _save_profile(self, data: dict):
|
||||
"""存储采集到的资料到 machines/{uid}/profiles/{account_id}.json"""
|
||||
profiles_dir = PROFILES_DIR / self._local_uid / "profiles"
|
||||
profiles_dir.mkdir(parents=True, exist_ok=True)
|
||||
path = profiles_dir / f"{data['account_id']}.json"
|
||||
path.write_text(json.dumps(data, indent=2, ensure_ascii=False), encoding="utf-8")
|
||||
|
||||
def get_profile(self, account_id: str) -> Optional[dict]:
|
||||
"""读取已存储的账号资料"""
|
||||
path = PROFILES_DIR / self._local_uid / "profiles" / f"{account_id}.json"
|
||||
if path.exists():
|
||||
try:
|
||||
return json.loads(path.read_text())
|
||||
except:
|
||||
pass
|
||||
return None
|
||||
|
||||
def _resolve_uid(self) -> str:
|
||||
uid_file = AGENT_LOCAL / "identity" / "machine_uid"
|
||||
if uid_file.exists():
|
||||
return uid_file.read_text().strip()
|
||||
return os.uname().nodename
|
||||
|
||||
def _resolve_hostname(self) -> str:
|
||||
hn_file = AGENT_LOCAL / "identity" / "cached_hostname"
|
||||
if hn_file.exists():
|
||||
return hn_file.read_text().strip()
|
||||
return os.uname().nodename
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
scraper = ProfileScraper()
|
||||
results = scraper.refresh_all()
|
||||
print(json.dumps(results, indent=2, ensure_ascii=False))
|
||||
@@ -0,0 +1,628 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
CDP 连接器 — 连接本机浏览器(Chrome / Camoufox)
|
||||
用法: python cdp_connector.py [port]
|
||||
|
||||
支持模式:
|
||||
Chrome: 通过 CDP 端口连接 Google Chrome
|
||||
Camoufox: 通过原生 API 启动 Firefox 内核浏览器
|
||||
|
||||
工作流示例:
|
||||
# Chrome CDP 模式
|
||||
conn = CDPConnector(port=9222)
|
||||
await conn.connect()
|
||||
|
||||
# Camoufox 原生模式
|
||||
conn = CDPConnector(browser_type="camoufox", headless=True)
|
||||
await conn.connect()
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import random
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
|
||||
class CDPConnector:
|
||||
"""连接浏览器(Chrome CDP / Camoufox 原生 / Camoufox 持久化)"""
|
||||
|
||||
def __init__(self, port: int = 9222, browser_type: str = "auto",
|
||||
headless: bool = True, window: tuple = (702, 783),
|
||||
profile_dir: str = None,
|
||||
locale: list = None,
|
||||
identity_dir: str = None,
|
||||
window_position: tuple = (0, 0)):
|
||||
"""
|
||||
browser_type: auto / chromium / firefox / camoufox
|
||||
- auto: 根据端口自动判断(9301+ 为 Firefox/Camoufox)
|
||||
- chromium: 强制 Chrome CDP
|
||||
- firefox: 通过 CDP 连接 Firefox
|
||||
- camoufox: 使用 Camoufox 原生 API 启动(不依赖外部CDP端口)
|
||||
|
||||
Camoufox 专用参数:
|
||||
- headless: 是否无头模式(默认 True)
|
||||
- window: 窗口大小 (width, height)
|
||||
- profile_dir: Profile 目录路径
|
||||
- locale: 语言设置(默认 ["zh-CN"])
|
||||
|
||||
身份工厂模式(Camoufox 持久化):
|
||||
- identity_dir: identities/{name}/ 目录路径
|
||||
设置后将自动加载 config.yaml 中的固化指纹,
|
||||
使用 persistent_context=True + user_data_dir 启动
|
||||
- window_position: 窗口在屏幕上的位置 (left, top),写入 xulstore
|
||||
"""
|
||||
self.port = port
|
||||
self.browser_type = browser_type
|
||||
self.headless = headless
|
||||
self.window = window
|
||||
self.profile_dir = profile_dir
|
||||
self.locale = locale or ["zh-CN"]
|
||||
self.identity_dir = identity_dir
|
||||
self.window_position = window_position
|
||||
|
||||
self._playwright = None
|
||||
self._camoufox = None # AsyncCamoufox 实例(防止 GC 回收导致连接断开)
|
||||
self._camoufox_browser = None # Camoufox native browser handle
|
||||
self.browser = None
|
||||
self.context = None
|
||||
self.page = None
|
||||
self.cdp_session = None
|
||||
self._is_camoufox_native = False
|
||||
self._identity_config = None # 缓存的 config.yaml 内容
|
||||
|
||||
def _resolve_browser_type(self) -> str:
|
||||
if self.browser_type != "auto":
|
||||
return self.browser_type
|
||||
return "firefox" if self.port >= 9301 else "chromium"
|
||||
|
||||
# ─── Camoufox 原生启动 ──────────────────────────────────────
|
||||
|
||||
async def _launch_camoufox(self):
|
||||
"""通过 Camoufox 原生 API 启动浏览器"""
|
||||
from camoufox.async_api import AsyncCamoufox
|
||||
|
||||
kwargs = {
|
||||
'headless': self.headless,
|
||||
'window': self.window,
|
||||
'locale': self.locale,
|
||||
# 固定 Windows 指纹(UA/WebGL/Canvas 全部伪装为 Windows)
|
||||
# 默认是随机选择,这会导致手机端提示"macOS/Windows"不确定
|
||||
'os': 'windows',
|
||||
# 加载系统中文字体(STHeiti/华文黑体),防止中文乱码
|
||||
# Camoufox 默认不加载系统字体,需要显式指定
|
||||
'fonts': ['STHeiti', 'Heiti SC', 'PingFang SC', 'Noto Sans CJK SC'],
|
||||
# 鼠标行为拟人化,使操作更接近真人
|
||||
'humanize': 1.5,
|
||||
}
|
||||
|
||||
print(f"🦊 启动 Camoufox (Firefox 135, OS=Windows, headless={self.headless})")
|
||||
|
||||
self._camoufox = AsyncCamoufox(**kwargs)
|
||||
self._camoufox_browser = await self._camoufox.start()
|
||||
self._is_camoufox_native = True
|
||||
|
||||
# Get browser context and page
|
||||
if self._camoufox_browser.contexts:
|
||||
self.context = self._camoufox_browser.contexts[0]
|
||||
else:
|
||||
self.context = await self._camoufox_browser.new_context()
|
||||
|
||||
if self.context.pages:
|
||||
self.page = self.context.pages[0]
|
||||
else:
|
||||
self.page = await self.context.new_page()
|
||||
|
||||
print(f"✅ Camoufox 就绪,当前页面: {self.page.url}")
|
||||
|
||||
# ─── Camoufox 持久化启动(身份工厂模式)────────────────────────
|
||||
|
||||
async def _launch_camoufox_persistent(self):
|
||||
"""通过身份工厂配置启动持久化 Camoufox
|
||||
|
||||
核心改进:
|
||||
- persistent_context=True → 状态保存在 user_data_dir
|
||||
- user_data_dir → 固定 Profile 目录,不丢失登录态
|
||||
- fingerprint → 固化指纹,每次启动同一副面孔
|
||||
"""
|
||||
import pickle
|
||||
import yaml
|
||||
from camoufox.async_api import AsyncCamoufox
|
||||
|
||||
identity_path = Path(self.identity_dir)
|
||||
config_path = identity_path / "config.yaml"
|
||||
fp_path = identity_path / "fingerprint.pkl"
|
||||
user_data_dir = str(identity_path / "user_data")
|
||||
|
||||
if not config_path.exists():
|
||||
raise FileNotFoundError(
|
||||
f"身份配置文件不存在: {config_path}\n"
|
||||
f"请先运行: python create_identity.py {identity_path.name}")
|
||||
|
||||
# 加载身份配置
|
||||
with open(config_path) as f:
|
||||
config = yaml.safe_load(f)
|
||||
self._identity_config = config
|
||||
|
||||
# 加载固化指纹(pickle 格式,BrowserForge Fingerprint 对象)
|
||||
fingerprint = None
|
||||
if fp_path.exists():
|
||||
with open(fp_path, 'rb') as f:
|
||||
fingerprint = pickle.load(f)
|
||||
|
||||
proxy = config["identity"].get("proxy")
|
||||
|
||||
# 读取窗口尺寸(优先用配置中的,否则用默认值)
|
||||
cfg_window = config.get("window", self.window)
|
||||
if isinstance(cfg_window, (list, tuple)) and len(cfg_window) == 2:
|
||||
w_width, w_height = int(cfg_window[0]), int(cfg_window[1])
|
||||
else:
|
||||
w_width, w_height = self.window
|
||||
|
||||
# 构建 Camoufox 参数
|
||||
# 清理残留锁文件(防止上次非正常退出导致启动失败)
|
||||
for lock_file in [".parentlock", ".startup-incomplete", "lock"]:
|
||||
try:
|
||||
lf = Path(user_data_dir) / lock_file
|
||||
if lf.exists():
|
||||
lf.unlink()
|
||||
except:
|
||||
pass
|
||||
|
||||
kwargs = {
|
||||
'persistent_context': True,
|
||||
'user_data_dir': user_data_dir,
|
||||
'headless': self.headless,
|
||||
'window': (w_width, w_height),
|
||||
'locale': self.locale or ['zh-CN'],
|
||||
'fonts': ['STHeiti', 'Heiti SC', 'PingFang SC', 'Noto Sans CJK SC'],
|
||||
'humanize': 1.5,
|
||||
# 操作系统: 优先用 config.yaml, 否则随机
|
||||
'os': config.get("fingerprint_summary", {}).get("os") or random.choice(["windows", "macos", "android"]),
|
||||
'firefox_user_prefs': {
|
||||
'dom.disable_window_move_resize': False,
|
||||
},
|
||||
'args': [
|
||||
f'--width={w_width}',
|
||||
f'--height={w_height}',
|
||||
'--new-window',
|
||||
],
|
||||
# Camoufox config:在启动前注入 DOM 属性覆盖
|
||||
# 让 viewport/screen 全部固定为 702×783
|
||||
'config': {
|
||||
'window.innerWidth': w_width,
|
||||
'window.innerHeight': w_height,
|
||||
'window.outerWidth': w_width,
|
||||
'window.outerHeight': w_height,
|
||||
'screen.width': w_width,
|
||||
'screen.height': w_height,
|
||||
'screen.availWidth': w_width,
|
||||
'screen.availHeight': w_height,
|
||||
},
|
||||
}
|
||||
|
||||
# 注入固化指纹(关键!——确保每次启动同一副面孔)
|
||||
if fingerprint is not None:
|
||||
kwargs['fingerprint'] = fingerprint
|
||||
kwargs['i_know_what_im_doing'] = True # 抑制 Camoufox 自定义指纹警告
|
||||
print(f" 指纹: ✅ 已固化 ({type(fingerprint).__name__})")
|
||||
else:
|
||||
print(f" 指纹: ⚠️ 未指定(将随机生成)")
|
||||
|
||||
# 注入代理
|
||||
if proxy:
|
||||
proxy_cfg = {'server': proxy} if isinstance(proxy, str) else proxy
|
||||
kwargs['proxy'] = proxy_cfg
|
||||
print(f" 代理: {proxy_cfg['server']}")
|
||||
|
||||
print(f"🦊 启动 Camoufox 持久化模式")
|
||||
print(f" Profile: {user_data_dir}")
|
||||
|
||||
# ── 强制设置 Firefox 窗口大小 ──
|
||||
# Firefox 在 xulstore.json 中存储窗口状态,每次启动时读取它,
|
||||
# 这会导致 --width/--height 命令行参数被覆盖。
|
||||
# 在启动前直接写入这个文件,确保窗口尺寸正确。
|
||||
try:
|
||||
import json as _j
|
||||
xul_path = Path(user_data_dir) / "xulstore.json"
|
||||
# 从 accounts.yaml 读取窗口位置,回退到 self.window_position
|
||||
cfg_path = Path(self.identity_dir).parent.parent / "config" / "accounts.yaml"
|
||||
cfg_x, cfg_y = self.window_position
|
||||
try:
|
||||
import yaml as _y
|
||||
if cfg_path.exists():
|
||||
with open(cfg_path) as _f:
|
||||
_cfg = _y.safe_load(_f)
|
||||
cur_ident = self.identity_dir.rstrip('/')
|
||||
for _a in _cfg.get("accounts", []):
|
||||
a_ident = _a.get("identity_dir", "").replace("identities/", "").rstrip('/')
|
||||
if a_ident and (cur_ident.endswith('/' + a_ident) or cur_ident == a_ident):
|
||||
_wp = _a.get("window_position")
|
||||
if _wp and len(_wp) == 2:
|
||||
cfg_x, cfg_y = int(_wp[0]), int(_wp[1])
|
||||
break
|
||||
except: pass
|
||||
|
||||
xul_data = {
|
||||
"chrome://browser/content/browser.xhtml": {
|
||||
"main-window": {
|
||||
"screenX": str(cfg_x),
|
||||
"screenY": str(cfg_y),
|
||||
"width": str(w_width),
|
||||
"height": str(w_height),
|
||||
"sizemode": "normal"
|
||||
}
|
||||
}
|
||||
}
|
||||
xul_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
xul_path.write_text(_j.dumps(xul_data, indent=2))
|
||||
print(f" xulstore: {w_width}×{w_height}")
|
||||
except Exception as e:
|
||||
print(f" ⚠️ xulstore: {e}")
|
||||
|
||||
self._camoufox = AsyncCamoufox(**kwargs)
|
||||
# persistent_context=True 返回的是 BrowserContext(包含已保存状态)
|
||||
ctx = await self._camoufox.start()
|
||||
self._camoufox_browser = ctx
|
||||
self._is_camoufox_native = True
|
||||
self.context = ctx
|
||||
|
||||
# 获取或创建页面
|
||||
if ctx.pages:
|
||||
self.page = ctx.pages[0]
|
||||
else:
|
||||
self.page = await ctx.new_page()
|
||||
|
||||
# 强制设置窗口大小(Firefox --width/--height 有时不生效,JS 确保实际窗口尺寸)
|
||||
try:
|
||||
await self.page.evaluate(f"""
|
||||
window.moveTo(0, 0);
|
||||
window.resizeTo({w_width}, {w_height});
|
||||
""")
|
||||
except Exception:
|
||||
pass # resizeTo 可能被浏览器策略阻止,不影响主体功能
|
||||
|
||||
print(f"✅ Camoufox 持久化就绪 | user_data: {identity_path.name}/user_data/")
|
||||
print(f" 窗口: {w_width}×{w_height}")
|
||||
|
||||
# 保存 os 选择到 config.yaml(供后续查询和前端展示)
|
||||
try:
|
||||
chosen_os = kwargs.get('os', 'windows')
|
||||
old_os = config.get("fingerprint_summary", {}).get("os")
|
||||
if old_os != chosen_os:
|
||||
import yaml as _y2
|
||||
if "fingerprint_summary" not in config:
|
||||
config["fingerprint_summary"] = {}
|
||||
config["fingerprint_summary"]["os"] = chosen_os
|
||||
# 同时更新 screen 信息
|
||||
config["fingerprint_summary"]["screen"] = f"{w_width}x{w_height}"
|
||||
with open(config_path, 'w') as f:
|
||||
_y2.dump(config, f, allow_unicode=True, default_flow_style=False)
|
||||
print(f" 📝 指纹已保存: {chosen_os} {w_width}x{w_height}")
|
||||
except: pass
|
||||
|
||||
# ─── Chrome / Firefox CDP 连接 ──────────────────────────────
|
||||
|
||||
async def _connect_cdp(self):
|
||||
"""通过 CDP 协议连接 Chrome/Firefox"""
|
||||
bt = self._resolve_browser_type()
|
||||
|
||||
import urllib.request
|
||||
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
|
||||
# 获取 WebSocket Debugger URL
|
||||
with opener.open(f"http://localhost:{self.port}/json/version", timeout=5) as r:
|
||||
info = json.loads(r.read())
|
||||
ws_url = info.get("webSocketDebuggerUrl")
|
||||
if not ws_url:
|
||||
raise RuntimeError(f"无法获取 WebSocket URL(端口 {self.port})")
|
||||
|
||||
from patchright.async_api import async_playwright
|
||||
self._playwright = await async_playwright().start()
|
||||
|
||||
if bt == "firefox":
|
||||
self.browser = await self._playwright.firefox.connect_over_cdp(ws_url)
|
||||
else:
|
||||
self.browser = await self._playwright.chromium.connect_over_cdp(ws_url)
|
||||
|
||||
self.context = self.browser.contexts[0]
|
||||
if self.context.pages:
|
||||
self.page = self.context.pages[0]
|
||||
else:
|
||||
self.page = await self.context.new_page()
|
||||
print(f"✅ 已连接 {bt}(端口 {self.port}),当前页面数:{len(self.context.pages)}")
|
||||
return self.page
|
||||
|
||||
# ─── 统一入口 ───────────────────────────────────────────────
|
||||
|
||||
async def connect(self):
|
||||
"""连接到浏览器(自动选择模式)
|
||||
|
||||
优先级:
|
||||
1. identity_dir 模式 → Camoufox 持久化(persistent_context)
|
||||
2. camoufox 类型 → Camoufox 原生(临时 profile)
|
||||
3. 其他 → CDP 连接(Chrome/Firefox)
|
||||
"""
|
||||
# 清理锁文件(不杀 Camoufox 进程!并发场景下有多个身份同时运行浏览器,
|
||||
# pkill -f 会杀死同批次其他身份的浏览器,导致采集失败)
|
||||
import subprocess
|
||||
# 不再执行 pkill -f camoufox
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
if self.identity_dir:
|
||||
# 清理锁文件
|
||||
for lock in ['.parentlock', '.startup-incomplete', 'lock']:
|
||||
try:
|
||||
lf = Path(self.identity_dir) / 'user_data' / lock
|
||||
if lf.exists(): lf.unlink()
|
||||
except: pass
|
||||
await self._launch_camoufox_persistent()
|
||||
else:
|
||||
bt = self._resolve_browser_type()
|
||||
if bt == "camoufox":
|
||||
await self._launch_camoufox()
|
||||
else:
|
||||
await self._connect_cdp()
|
||||
return self.page
|
||||
|
||||
# ─── 反检测 ────────────────────────────────────────────────
|
||||
|
||||
async def init_anti_detection(self):
|
||||
"""初始化反检测配置(安卓平板模式视口 + App跳转拦截)
|
||||
|
||||
关键: mobile=True 可使抖音加载移动端/平板界面,
|
||||
这样才能显示点赞/收藏/评论按钮。
|
||||
|
||||
2026-05-03 ghai 要求:全部改为安卓平板标识
|
||||
UA: Samsung Galaxy Tab S9 (SM-X910, Android 14)
|
||||
"""
|
||||
# 安卓平板 User-Agent(通用)
|
||||
ANDROID_TABLET_UA = (
|
||||
"Mozilla/5.0 (Linux; Android 14; SM-X910) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) "
|
||||
"Chrome/124.0.6367.113 Safari/537.36"
|
||||
)
|
||||
|
||||
if self._is_camoufox_native:
|
||||
# Camoufox: 只设置视口尺寸(702×783 小窗口模拟平板)
|
||||
# 注意:不覆盖 UA!Camoufox 的 Windows Firefox UA 是 C++ 级原生伪装,
|
||||
# 强行改为 Android 会导致底层指纹(platform/fonts/webgl)与 UA 矛盾
|
||||
await self.page.set_viewport_size({
|
||||
"width": self.window[0],
|
||||
"height": self.window[1]
|
||||
})
|
||||
print(f"✅ Camoufox 反检测已就绪(Windows 平板 702×783)")
|
||||
return None
|
||||
|
||||
# Chrome CDP: 使用 DevTools Protocol 设置 UA + 触摸 + 视口
|
||||
if not self.cdp_session:
|
||||
self.cdp_session = await self.context.new_cdp_session(self.page)
|
||||
|
||||
# 1) 设置安卓平板 User-Agent
|
||||
try:
|
||||
await self.cdp_session.send("Network.setUserAgentOverride", {
|
||||
"userAgent": ANDROID_TABLET_UA
|
||||
})
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# 2) 设置视口尺寸
|
||||
try:
|
||||
await self.page.set_viewport_size({
|
||||
"width": self.window[0],
|
||||
"height": self.window[1]
|
||||
})
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# 3) 启用触摸模拟(关键!让 maxTouchPoints > 0,抖音才能识别为触摸设备)
|
||||
try:
|
||||
await self.cdp_session.send("Emulation.setTouchEmulationEnabled", {
|
||||
"enabled": True,
|
||||
"maxTouchPoints": 5,
|
||||
"configuration": "mobile",
|
||||
})
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# 设置安卓平板 User-Agent
|
||||
try:
|
||||
await self.cdp_session.send("Network.setUserAgentOverride", {
|
||||
"userAgent": ANDROID_TABLET_UA
|
||||
})
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
print("✅ 反检测已就绪(平板模式 702x783 + iPad UA + 触摸模拟)")
|
||||
return self.cdp_session
|
||||
|
||||
# ─── 导航 ───────────────────────────────────────────────────
|
||||
|
||||
async def goto(self, url: str, wait: str = "domcontentloaded"):
|
||||
await self.page.goto(url, wait_until=wait, timeout=30000)
|
||||
await asyncio.sleep(1.5)
|
||||
await self.remove_overlays()
|
||||
return self.page
|
||||
|
||||
async def remove_overlays(self):
|
||||
"""清理 App 跳转弹窗和遮罩"""
|
||||
script = """
|
||||
() => {
|
||||
const selectors = [
|
||||
'[class*="download"]','[class*="open-app"]','[class*="app-guide"]',
|
||||
'[class*="launch-app"]','[class*="open-in-app"]','[class*="app-launcher"]',
|
||||
'.open-in-app','.app-launch-mask','.download-tip','.bottom-bar',
|
||||
'.download-banner','.open-app-btn','.app-download-tip',
|
||||
'.open-app-layer','.download-guide-mask',
|
||||
'.open-in-app-bar','.app-open-button',
|
||||
'#app-launch-dialog','#open-app-modal','#download-modal',
|
||||
];
|
||||
let count = 0;
|
||||
selectors.forEach(s => {
|
||||
document.querySelectorAll(s).forEach(el => { el.remove(); count++; });
|
||||
});
|
||||
document.body.style.overflow = '';
|
||||
document.body.style.overflowY = 'auto';
|
||||
document.body.style.position = '';
|
||||
return count;
|
||||
}
|
||||
"""
|
||||
removed = await self.page.evaluate(script)
|
||||
if removed > 0:
|
||||
print(f" 🧹 已清理 {removed} 个弹窗/遮罩")
|
||||
return removed
|
||||
|
||||
# ─── 交互 ───────────────────────────────────────────────────
|
||||
|
||||
async def touch_tap(self, x: int, y: int):
|
||||
"""模拟触摸点击"""
|
||||
if self._is_camoufox_native:
|
||||
await self.page.tap("#root", position={"x": x, "y": y})
|
||||
return
|
||||
ts = int(time.time() * 1000)
|
||||
await self.cdp_session.send("Input.dispatchTouchEvent", {
|
||||
"type": "touchStart",
|
||||
"touchPoints": [{"x": x, "y": y, "id": 0}],
|
||||
"timestamp": ts
|
||||
})
|
||||
await asyncio.sleep(random.uniform(0.05, 0.15))
|
||||
await self.cdp_session.send("Input.dispatchTouchEvent", {
|
||||
"type": "touchEnd",
|
||||
"touchPoints": [{"x": x, "y": y, "id": 0}],
|
||||
"timestamp": int(time.time() * 1000)
|
||||
})
|
||||
|
||||
async def swipe_up(self, distance: int = 600, duration_ms: int = 400):
|
||||
"""向上滑动(浏览下一条内容)"""
|
||||
if self._is_camoufox_native:
|
||||
await self.page.evaluate(f"window.scrollBy(0, {distance})")
|
||||
return
|
||||
|
||||
cx = 195
|
||||
start_y = 700
|
||||
end_y = start_y - distance
|
||||
steps = 20
|
||||
step_duration = duration_ms / steps / 1000
|
||||
|
||||
for i in range(steps):
|
||||
progress = i / steps
|
||||
current_y = start_y + (end_y - start_y) * progress
|
||||
event_type = "touchStart" if i == 0 else ("touchEnd" if i == steps - 1 else "touchMove")
|
||||
await self.cdp_session.send("Input.dispatchTouchEvent", {
|
||||
"type": event_type,
|
||||
"touchPoints": [{"x": cx, "y": current_y, "id": 0}],
|
||||
"timestamp": int(time.time() * 1000)
|
||||
})
|
||||
await asyncio.sleep(step_duration + random.uniform(-0.005, 0.005))
|
||||
|
||||
async def check_login(self, platform: str = "douyin") -> dict:
|
||||
"""检测登录状态(委托 auth_manager 多维检测)
|
||||
|
||||
返回字典包含: logged_in, cookie_ok, dom_ok, cookie_count, session_id, method
|
||||
不再只依赖 DOM 检测(桌面端视口不显示头像元素)。
|
||||
"""
|
||||
from auth_manager import get_login_status
|
||||
return await get_login_status(self.context, self.page, platform)
|
||||
|
||||
# ─── 清理 ───────────────────────────────────────────────────
|
||||
|
||||
async def close(self):
|
||||
"""关闭浏览器,确保进程完全退出
|
||||
|
||||
3 重保障:
|
||||
1. 优雅关闭:AsyncCamoufox.stop()(可能卡住)
|
||||
2. pgrep 验证 + pkill 兜底(3种匹配模式)
|
||||
3. 等待进程消退
|
||||
"""
|
||||
profile_name = None
|
||||
if self.profile_dir:
|
||||
profile_name = Path(self.profile_dir).name
|
||||
elif self.identity_dir:
|
||||
profile_name = Path(self.identity_dir).name
|
||||
|
||||
# 第一重:优雅关闭
|
||||
if hasattr(self, '_camoufox') and self._camoufox:
|
||||
try:
|
||||
await self._camoufox.stop()
|
||||
except (AttributeError, Exception):
|
||||
try:
|
||||
await self._camoufox.close()
|
||||
except (AttributeError, Exception):
|
||||
try:
|
||||
await self._camoufox.kill()
|
||||
except Exception:
|
||||
pass
|
||||
elif self._camoufox_browser:
|
||||
await self._camoufox_browser.close()
|
||||
if self._playwright:
|
||||
await self._playwright.stop()
|
||||
|
||||
# 第二重:等待进程退出 + 强杀
|
||||
await asyncio.sleep(2)
|
||||
try:
|
||||
r = subprocess.run(
|
||||
["pgrep", "-f", "camoufox"],
|
||||
capture_output=True, text=True, timeout=3
|
||||
)
|
||||
if r.stdout.strip():
|
||||
import logging
|
||||
logger = logging.getLogger("cdp_connector.close")
|
||||
remaining = r.stdout.strip().split("\n")
|
||||
logger.warning(f" ⚠️ 关闭后仍有 {len(remaining)} 个 Camoufox 进程,强制终止")
|
||||
|
||||
# 按 profile_dir 精确杀
|
||||
if profile_name:
|
||||
subprocess.run(
|
||||
["pkill", "-f", f"camoufox.*{profile_name}"],
|
||||
capture_output=True, timeout=3
|
||||
)
|
||||
# 按浏览器类型广谱杀
|
||||
subprocess.run(
|
||||
["pkill", "-f", "camoufox.*--remote-debugging-port"],
|
||||
capture_output=True, timeout=3
|
||||
)
|
||||
subprocess.run(
|
||||
["pkill", "-f", "camoufox.*user_data_dir"],
|
||||
capture_output=True, timeout=3
|
||||
)
|
||||
await asyncio.sleep(1)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ─── 快速测试 ────────────────────────────────────────────────────
|
||||
async def _test(port: int = 9222, browser_type: str = "auto"):
|
||||
print(f"测试连接(端口 {port}, 类型 {browser_type})...")
|
||||
conn = CDPConnector(port=port, browser_type=browser_type)
|
||||
try:
|
||||
page = await conn.connect()
|
||||
await conn.init_anti_detection()
|
||||
|
||||
print("\n测试导航到抖音...")
|
||||
await conn.goto("https://www.douyin.com")
|
||||
print(f" URL: {page.url}")
|
||||
print(f" Title: {await page.title()}")
|
||||
|
||||
status = await conn.check_login("douyin")
|
||||
print(f" 抖音登录状态: {'✅ 已登录 (via ' + status['method'] + ')' if status['logged_in'] else '❌ 未登录'}")
|
||||
print(f" Cookie: {status['cookie_count']}个, sessionid: {'✅' if status['session_id'] else '❌'}")
|
||||
|
||||
except Exception as e:
|
||||
print(f"❌ 连接失败: {e}")
|
||||
print("提示:")
|
||||
print(" Chrome: bash ~/matrix/scripts/launch_chrome.sh account_01 9222")
|
||||
print(" Camoufox: 指定 browser_type='camoufox'")
|
||||
finally:
|
||||
await conn.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
args = sys.argv[1:]
|
||||
port = 9222
|
||||
btype = "auto"
|
||||
if args:
|
||||
port = int(args[0])
|
||||
if len(args) > 1:
|
||||
btype = args[1]
|
||||
asyncio.run(_test(port, btype))
|
||||
@@ -0,0 +1,212 @@
|
||||
# 评论语料库 v1.0
|
||||
#
|
||||
# 分类结构:大类 → 子类 → 评论内容/模板
|
||||
# 权重越高,被选中的概率越大
|
||||
# 模板用 {keyword} 动态替换为当前视频关键词
|
||||
#
|
||||
# 你可以手动编辑此文件:
|
||||
# - 新增评论直接加在对应分类下
|
||||
# - 新增分类在 categories 下加一个 key 即可
|
||||
# - 不想用的分类把 weight 设为 0
|
||||
|
||||
categories:
|
||||
# ── 1. 通用赞美 ──
|
||||
praise:
|
||||
label: "通用赞美"
|
||||
weight: 4
|
||||
comments:
|
||||
- "说得太好了"
|
||||
- "学到了,感谢分享"
|
||||
- "很有道理"
|
||||
- "好内容,值得收藏"
|
||||
- "讲得很透彻"
|
||||
- "支持一下"
|
||||
- "确实如此"
|
||||
- "说得太对了"
|
||||
- "收藏了慢慢看"
|
||||
- "好文,已三连"
|
||||
|
||||
# ── 2. 内容相关 ──
|
||||
content_related:
|
||||
label: "内容相关"
|
||||
weight: 3
|
||||
templates:
|
||||
- "这个{keyword}太有意思了"
|
||||
- "第一次看到{keyword}的讲解"
|
||||
- "关于{keyword}讲得很透彻"
|
||||
- "{keyword}还能这么理解,学习了"
|
||||
- "对{keyword}又有新的认识"
|
||||
comments:
|
||||
- "这个角度很新颖"
|
||||
- "终于有人讲明白了"
|
||||
- "深入浅出,好讲解"
|
||||
|
||||
# ── 3. 互动提问 ──
|
||||
interactive:
|
||||
label: "互动提问"
|
||||
weight: 2
|
||||
comments:
|
||||
- "有懂的大佬吗?"
|
||||
- "请问这个是什么原理?"
|
||||
- "能出一期详细教程吗?"
|
||||
- "这个在哪里可以买到?"
|
||||
- "有没有相关的推荐?"
|
||||
- "请问up主用的什么设备?"
|
||||
|
||||
# ── 4. 引导评论(重点,可定向引导) ──
|
||||
guided:
|
||||
label: "引导评论"
|
||||
weight: 1
|
||||
description: "用于特定目的引导,根据当前养的号类型选择启用"
|
||||
tags:
|
||||
- "general" # 通用引导
|
||||
- "promote" # 推广引导
|
||||
- "debate" # 争议引导
|
||||
comments:
|
||||
- "有没有和我一样在看第二遍的"
|
||||
- "看完这个我终于懂了"
|
||||
- "我试了一下真的有用"
|
||||
- "这个方法我用了三个月了"
|
||||
- "推荐给所有需要的朋友"
|
||||
- "有没有人组队一起学的"
|
||||
|
||||
# ── 5. 表情/符号 ──
|
||||
emoji:
|
||||
label: "表情符号"
|
||||
weight: 1
|
||||
comments:
|
||||
- "👍"
|
||||
- "❤️"
|
||||
- "👏"
|
||||
- "🔥"
|
||||
- "太棒了👍"
|
||||
- "已收藏❤️"
|
||||
|
||||
# ── 6. 调侃幽默 ──
|
||||
humor:
|
||||
label: "调侃幽默"
|
||||
weight: 1
|
||||
comments:
|
||||
- "老师讲得很好,下次别讲了(狗头)"
|
||||
- "脑子:学会了 手:???"
|
||||
- "我的表情和封面一模一样"
|
||||
- "建议直接发我脑子里"
|
||||
- "全程张着嘴看完的"
|
||||
- "这个视频是我付费能看的吗?"
|
||||
|
||||
# ── 7. 行业术语(可自定义) ──
|
||||
industry:
|
||||
label: "行业术语"
|
||||
weight: 0 # 默认关闭,需要时打开
|
||||
description: "根据养号领域自定义,如编程/金融/美食等"
|
||||
tags:
|
||||
- "tech"
|
||||
- "finance"
|
||||
comments:
|
||||
- "这个API设计得很优雅"
|
||||
- "性能优化做得很到位"
|
||||
- "架构设计值得参考"
|
||||
|
||||
# ── 8. 纯共情弹幕(不讲医生不讲号,就讲痛) ──
|
||||
empathy:
|
||||
label: "纯共情弹幕"
|
||||
weight: 4
|
||||
tags:
|
||||
- "douyin"
|
||||
comments:
|
||||
- "坐一天班屁股跟针扎似的,谁懂啊!"
|
||||
- "吃顿火锅一时爽,厕所出来就想给嘴一巴掌"
|
||||
- "痔疮犯了坐立难安,开会的时候扭来扭去像长了虫子"
|
||||
- "不好意思去医院,硬扛了仨月,我真服了自己这个老六"
|
||||
- "各种药膏买了一抽屉,钱花了屁用没有,纯纯大冤种"
|
||||
- "大医院排队排到怀疑人生,请一天假还不够折腾的"
|
||||
- "疼到凌晨四点还搁那数羊,第二天还要爬起来搬砖"
|
||||
- "每次上厕所跟上刑一样,完事儿腿都是软的"
|
||||
- "听说要手术直接腿软,怕疼星人当场就想跑路"
|
||||
- "反复发作真的破防了,这玩意儿就跟渣男一样甩不掉"
|
||||
- "久坐党太难了,屁股和工位锁死了属于是"
|
||||
- "社恐人一想到去肛肠科,脚趾已经抠出三室一厅了"
|
||||
|
||||
# ── 9. 朱晓光主任弹幕 ──
|
||||
doctor_zhu:
|
||||
label: "朱晓光主任"
|
||||
weight: 4
|
||||
tags:
|
||||
- "douyin"
|
||||
comments:
|
||||
- "朱晓光主任yyds,终于不用再遭罪了"
|
||||
- "朱主任手法是真的轻,怕疼星人狂喜!"
|
||||
- "身边好几个朋友都是找朱晓光看好的"
|
||||
- "朱主任问诊贼耐心,啥都给你讲明白"
|
||||
- "听朱主任的科普,才知道以前踩了多少坑"
|
||||
- "朱晓光技术真的顶,老肛瘘都给我整好了"
|
||||
- "朱主任说话挺温和,不会让你觉得尴尬"
|
||||
- "找朱晓光看过的都懂,恢复期确实短"
|
||||
- "朱主任的号我得蹲一下,口碑太好了"
|
||||
- "苏州肛泰朱晓光,看肛肠我是真服气"
|
||||
|
||||
# ── 10. 苏州济世璞真公众号引导 ──
|
||||
gh_account:
|
||||
label: "公众号约号"
|
||||
weight: 3
|
||||
tags:
|
||||
- "douyin"
|
||||
comments:
|
||||
- "搜苏州济世璞真功重号约就行了,不用傻排队"
|
||||
- "公号苏州济世璞真直接约,到点去看不用等"
|
||||
- "想约号的宝子去苏州济世璞真看看,贼方便"
|
||||
- "线上就能挂号,搜苏州济世璞真,懒人福音"
|
||||
- "不用现场排队,功重号苏州济世璞真搞定一切"
|
||||
- "苏州济世璞真,线上约好直接去,省时buff叠满"
|
||||
- "公号搜苏州济世璞真,里面有朱主任的号"
|
||||
- "别再傻傻去排了,苏州济世璞真约一下不香吗"
|
||||
- "约朱主任的号走苏州济世璞真,绿通懂的都懂"
|
||||
- "苏州济世璞真,主打一个省事,按点去就行"
|
||||
|
||||
# ── 11. 捡漏绿通故事 ──
|
||||
story_channel:
|
||||
label: "捡漏绿通"
|
||||
weight: 3
|
||||
tags:
|
||||
- "douyin"
|
||||
comments:
|
||||
- "大医院挂不上的号,公号苏州济世璞真里头能捡漏朱晓光,悄悄说"
|
||||
- "朱晓光主任多点出诊,不挤大医院,苏州济世璞真就是绿通"
|
||||
- "挖到宝了,苏州济世璞真里头一堆专家,朱主任的号也能约"
|
||||
- "别再死磕大医院排队了,苏州济世璞真约朱主任不香吗"
|
||||
- "这个小众渠道我只告诉你,苏州济世璞真,专家绿通懂的都懂"
|
||||
- "大医院人挤人,苏州济世璞真线上捡漏,省心不是一星半点"
|
||||
- "朱晓光主任的号在这边好约多了,苏州济世璞真yyds"
|
||||
- "这算不算隐藏福利?苏州济世璞真能约到朱主任,还不咋排队"
|
||||
|
||||
# ── 12. 优势卖点总结 ──
|
||||
selling_points:
|
||||
label: "优势卖点"
|
||||
weight: 3
|
||||
tags:
|
||||
- "douyin"
|
||||
comments:
|
||||
- "专科医院就是专门搞这个的,苏州肛泰没踩雷"
|
||||
- "独立诊室真的救命,社恐人狠狠爱住了"
|
||||
- "线上约好再去,不用请假排半天队,打工人狂喜"
|
||||
- "朱主任术后还会跟进关心你,不是看完了就不管了"
|
||||
- "微创恢复快,不耽误上班,这个太重要了"
|
||||
- "隐私保护做得好,不用担心碰到熟人尴尬"
|
||||
- "朱主任科普干货多,关注了能避开好多坑"
|
||||
- "老肛肠患者表示,终于遇到个靠谱的了"
|
||||
|
||||
|
||||
# ── 全局配置 ──
|
||||
config:
|
||||
# 默认启用的分类(填分类key,为空则启用所有weight>0的分类)
|
||||
enabled_categories: []
|
||||
|
||||
# 默认禁用的分类(填分类key)
|
||||
disabled_categories:
|
||||
- "industry"
|
||||
|
||||
# 当前养号领域(用于模板填充 {keyword})
|
||||
niche: "general"
|
||||
|
||||
# 是否优先使用模板(带{keyword}的评论)
|
||||
prefer_templates: true
|
||||
@@ -0,0 +1,18 @@
|
||||
# Matrix 定时任务配置
|
||||
# time 格式: HH:MM(每天固定时间)
|
||||
# days 格式: 1,2,3,4,5,6,7(1=周一..7=周日,默认每天)
|
||||
schedules:
|
||||
douyin_daily_morning:
|
||||
enabled: false
|
||||
account: douyin_test
|
||||
blueprint: douyin_daily
|
||||
rounds: 3
|
||||
time: "09:00"
|
||||
days: "1,2,3,4,5,6,7"
|
||||
xhs_daily_noon:
|
||||
enabled: false
|
||||
account: xhs_01
|
||||
blueprint: xhs_daily
|
||||
rounds: 2
|
||||
time: "12:30"
|
||||
days: "1,2,3,4,5,6,7"
|
||||
@@ -0,0 +1,14 @@
|
||||
# 视觉分析配置
|
||||
# oMLX: 本地模型(优先)
|
||||
omlx:
|
||||
base_url: "http://127.0.0.1:8000/v1"
|
||||
api_key: "omlx"
|
||||
model: "Qwen2.5-VL-3B-Instruct-8bit"
|
||||
|
||||
# DashScope: 远程备用(阿里百炼)
|
||||
# ⚠️ api_key 不入库 — 真实值在 agent-local/tools/ave/config/local.yaml → aliyun.api_key
|
||||
# 或环境变量 DASHSCOPE_API_KEY (vision_bridge.get_dashscope_key 统一读取)
|
||||
dashscope:
|
||||
api_key: "sk-xxx"
|
||||
model: "qwen-vl-plus" # 性价比高,支持图片理解
|
||||
# 更便宜的选择: qwen-vl-max (更强) / qwen2.5-vl-7b (开源版)
|
||||
@@ -0,0 +1,136 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
身份工厂 — 创建新身份(生成固化指纹 + 目录 + 配置)
|
||||
|
||||
用法:
|
||||
python create_identity.py <identity_name> [--platform douyin]
|
||||
|
||||
示例:
|
||||
python create_identity.py douyin_01
|
||||
python create_identity.py douyin_02 --platform douyin
|
||||
|
||||
目录结构:
|
||||
identities/{name}/
|
||||
├── config.yaml ← 身份配置(可编辑、可版本控制)
|
||||
├── fingerprint.pkl ← BrowserForge 固化指纹(不可编辑)
|
||||
└── user_data/ ← Camoufox 持久化 Profile(自动生成,不提交)
|
||||
"""
|
||||
import argparse
|
||||
import os
|
||||
import pickle
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError:
|
||||
print("❌ 需要 PyYAML: pip install pyyaml")
|
||||
sys.exit(1)
|
||||
|
||||
try:
|
||||
from browserforge.fingerprints import FingerprintGenerator
|
||||
except ImportError:
|
||||
print("❌ 需要 browserforge: pip install browserforge")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
# ─── 路径配置 ───
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).parent
|
||||
TOOL_DIR = SCRIPTS_DIR.parent
|
||||
# identities 目录放在 agent-local/tools/matrix/identities/
|
||||
from matrix_mgmt import AGENT_LOCAL
|
||||
IDENTITIES_ROOT = AGENT_LOCAL / "tools" / "matrix" / "identities"
|
||||
|
||||
|
||||
def create_identity(name: str, platform: str = "douyin", proxy: str = None):
|
||||
"""创建新的虚拟身份"""
|
||||
|
||||
identity_dir = IDENTITIES_ROOT / name
|
||||
|
||||
# 检查是否已存在
|
||||
if identity_dir.exists():
|
||||
print(f"⚠️ 身份 '{name}' 已存在: {identity_dir}")
|
||||
confirm = input(" 覆盖? (y/N): ").strip().lower()
|
||||
if confirm != 'y':
|
||||
print("❌ 已取消")
|
||||
return False
|
||||
import shutil
|
||||
shutil.rmtree(identity_dir)
|
||||
|
||||
# 创建目录
|
||||
os.makedirs(identity_dir, exist_ok=True)
|
||||
user_data_dir = identity_dir / "user_data"
|
||||
os.makedirs(user_data_dir, exist_ok=True)
|
||||
|
||||
# ── 生成 BrowserForge 指纹 ──
|
||||
print(f"\n🔄 生成指纹 (Windows/Firefox)...")
|
||||
fg = FingerprintGenerator(browser='firefox', os=('windows',))
|
||||
fp = fg.generate()
|
||||
|
||||
# 保存 pickle(供 Camoufox 加载)
|
||||
fp_path = identity_dir / "fingerprint.pkl"
|
||||
with open(fp_path, 'wb') as f:
|
||||
pickle.dump(fp, f)
|
||||
|
||||
# ── 生成可读的 config.yaml ──
|
||||
config = {
|
||||
'identity': {
|
||||
'name': name,
|
||||
'platform': platform,
|
||||
'proxy': proxy,
|
||||
'notes': '',
|
||||
'created_at': datetime.now().isoformat(),
|
||||
'fingerprint_file': 'fingerprint.pkl',
|
||||
},
|
||||
'window': [702, 783], # 窗口尺寸(可手动修改)
|
||||
'fingerprint_summary': {
|
||||
'userAgent': fp.headers['User-Agent'],
|
||||
'platform': fp.navigator.platform,
|
||||
'screen': f"{fp.screen.width}x{fp.screen.height}",
|
||||
'hardwareConcurrency': fp.navigator.hardwareConcurrency,
|
||||
'fonts_count': len(fp.fonts),
|
||||
'videoCard': fp.videoCard.renderer[:60] if fp.videoCard else 'N/A',
|
||||
},
|
||||
}
|
||||
|
||||
config_path = identity_dir / "config.yaml"
|
||||
with open(config_path, 'w', encoding='utf-8') as f:
|
||||
yaml.dump(config, f, default_flow_style=False, allow_unicode=True)
|
||||
|
||||
# ── 输出摘要 ──
|
||||
print(f"\n{'='*55}")
|
||||
print(f" ✅ 新身份已创建: {name}")
|
||||
print(f"{'='*55}")
|
||||
print(f" 路径: {identity_dir}")
|
||||
print(f" User-Agent: {config['fingerprint_summary']['userAgent'][:70]}")
|
||||
print(f" 平台: {config['fingerprint_summary']['platform']}")
|
||||
print(f" 屏幕: {config['fingerprint_summary']['screen']}")
|
||||
print(f" 核心数: {config['fingerprint_summary']['hardwareConcurrency']}")
|
||||
print(f" 字体数量: {config['fingerprint_summary']['fonts_count']}")
|
||||
print(f" GPU: {config['fingerprint_summary']['videoCard']}")
|
||||
print()
|
||||
print(f" 📋 下一步: 运行登录脚本")
|
||||
print(f" python login_identity.py {name}")
|
||||
print()
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="身份工厂 — 创建新虚拟身份")
|
||||
parser.add_argument("name", help="身份名称 (如 douyin_01)")
|
||||
parser.add_argument("--platform", "-p", default="douyin", help="目标平台 (默认 douyin)")
|
||||
parser.add_argument("--proxy", help="可选代理地址 (如 socks5://127.0.0.1:1080)")
|
||||
args = parser.parse_args()
|
||||
|
||||
create_identity(
|
||||
name=args.name,
|
||||
platform=args.platform,
|
||||
proxy=args.proxy,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,109 @@
|
||||
-- anchor.db 数据库设计 v1.0
|
||||
-- 锚点规则 + 原子操作 + 检测日志 + 校准
|
||||
|
||||
-- ── 1. 锚点规则表 ──
|
||||
-- 每个状态由多条规则组成(同 state + logic_group = AND,不同 group = OR)
|
||||
CREATE TABLE anchor_rules (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
state_name TEXT NOT NULL, -- HOME / PLAYER / SEARCH / DETAIL / POPUP
|
||||
field TEXT NOT NULL, -- 检测字段: video_count, title, video_playing, card_count, url, has_search_input
|
||||
operator TEXT NOT NULL, -- eq / neq / gt / gte / lt / lte / contains / not_contains / regex
|
||||
value TEXT NOT NULL, -- 目标值(统一存字符串)
|
||||
logic_group INTEGER DEFAULT 0, -- 0=AND组, 同组内AND, 不同组OR
|
||||
active INTEGER DEFAULT 1, -- 1=启用, 0=停用
|
||||
created_at TEXT DEFAULT (datetime('now','localtime')),
|
||||
updated_at TEXT DEFAULT (datetime('now','localtime'))
|
||||
);
|
||||
|
||||
-- ── 2. 原子操作表 ──
|
||||
-- 每个原子操作 = 一个操作标识 + 执行方式 + 验证规则
|
||||
CREATE TABLE atomic_ops (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
op_name TEXT NOT NULL UNIQUE, -- click_card, swipe_down, like, search, go_home, dismiss_popup
|
||||
description TEXT, -- 操作描述
|
||||
action_type TEXT DEFAULT 'locator', -- locator / keyboard / evaluate / mouse
|
||||
target TEXT, -- CSS选择器 / 按键名 / JS代码
|
||||
verify_rule TEXT, -- 验证方式: video_src_change / video_count_gt / title_change
|
||||
retry_count INTEGER DEFAULT 3,
|
||||
fallback_op TEXT, -- 失败时降级的操作名
|
||||
created_at TEXT DEFAULT (datetime('now','localtime')),
|
||||
updated_at TEXT DEFAULT (datetime('now','localtime'))
|
||||
);
|
||||
|
||||
-- ── 3. 状态迁移表 ──
|
||||
-- 定义合法的状态转换(状态机)
|
||||
CREATE TABLE state_transitions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
from_state TEXT NOT NULL, -- HOME, PLAYER, SEARCH
|
||||
to_state TEXT NOT NULL, -- PLAYER, SEARCH, HOME
|
||||
via_op TEXT NOT NULL, -- click_card, swipe_down, search, go_home
|
||||
UNIQUE(from_state, to_state, via_op)
|
||||
);
|
||||
|
||||
-- ── 4. 检测日志表 ──
|
||||
-- 每次锚点检测的记录,用于成功率追踪
|
||||
CREATE TABLE detection_logs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
identity TEXT, -- douyin_01, douyin_02
|
||||
expected_state TEXT, -- 操作前预期的状态
|
||||
detected_state TEXT, -- 实际检测到的状态
|
||||
match INTEGER, -- 1=匹配, 0=不匹配
|
||||
anchor_snapshot TEXT, -- 锚点数据快照(JSON)
|
||||
screenshot_path TEXT, -- 截图路径
|
||||
timestamp TEXT DEFAULT (datetime('now','localtime'))
|
||||
);
|
||||
CREATE INDEX idx_detection_logs_match ON detection_logs(match, timestamp);
|
||||
|
||||
-- ── 5. 校准候选表 ──
|
||||
-- AI或自动校准产生的新规则,对比验证后决定是否采纳
|
||||
CREATE TABLE calibration_candidates (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
state_name TEXT, -- 哪个状态的规则
|
||||
source TEXT, -- ai / manual / auto_recovery
|
||||
old_rules TEXT, -- 旧规则JSON
|
||||
new_rules TEXT, -- 建议的新规则JSON
|
||||
test_count INTEGER DEFAULT 0, -- 测试次数
|
||||
test_pass INTEGER DEFAULT 0, -- 测试通过次数
|
||||
status TEXT DEFAULT 'pending', -- pending / accepted / rejected / testing
|
||||
created_at TEXT DEFAULT (datetime('now','localtime'))
|
||||
);
|
||||
|
||||
-- ── 初始数据:基于2026-05-06手动采集的锚点 ──
|
||||
|
||||
-- HOME 状态规则
|
||||
INSERT INTO anchor_rules (state_name, field, operator, value, logic_group) VALUES
|
||||
('HOME', 'video_count', 'eq', '1', 0),
|
||||
('HOME', 'title', 'contains', '抖音精选', 0);
|
||||
|
||||
-- PLAYER 状态规则(全屏播放)
|
||||
INSERT INTO anchor_rules (state_name, field, operator, value, logic_group) VALUES
|
||||
('PLAYER', 'video_count', 'gte', '3', 0),
|
||||
('PLAYER', 'video_playing', 'eq', 'true', 0);
|
||||
|
||||
-- SEARCH 状态规则
|
||||
INSERT INTO anchor_rules (state_name, field, operator, value, logic_group) VALUES
|
||||
('SEARCH', 'title', 'contains', '搜索', 0),
|
||||
('SEARCH', 'title', 'not_contains', '精选', 0);
|
||||
|
||||
-- SEARCH_PLAYER(从搜索结果进入的播放页)
|
||||
INSERT INTO anchor_rules (state_name, field, operator, value, logic_group) VALUES
|
||||
('SEARCH_PLAYER', 'video_count', 'gte', '3', 0),
|
||||
('SEARCH_PLAYER', 'video_playing', 'eq', 'true', 0),
|
||||
('SEARCH_PLAYER', 'url', 'contains', '/search/', 0);
|
||||
|
||||
-- 原子操作
|
||||
INSERT INTO atomic_ops (op_name, description, action_type, target, verify_rule, retry_count) VALUES
|
||||
('click_card', '点击首页视频卡片', 'locator', '.discover-video-card-item', 'video_count_gte_3', 3),
|
||||
('swipe_down', '下滑切换视频', 'keyboard', 'ArrowDown', 'video_src_change', 2),
|
||||
('like', '点赞当前视频', 'locator', '[data-e2e="like-count"]', 'like_count_change', 1),
|
||||
('search', '搜索关键词', 'evaluate', 'search_input', 'title_contains_搜索', 2),
|
||||
('go_home', '回到首页', 'locator', 'https://www.douyin.com/', 'video_count_eq_1', 2),
|
||||
('dismiss_popup','关闭弹窗', 'locator', 'button:text-is("我知道了")','popup_gone', 2);
|
||||
|
||||
-- 状态迁移
|
||||
INSERT INTO state_transitions (from_state, to_state, via_op) VALUES
|
||||
('HOME', 'PLAYER', 'click_card'),
|
||||
('PLAYER', 'PLAYER', 'swipe_down'),
|
||||
('PLAYER', 'HOME', 'go_home'),
|
||||
('HOME', 'SEARCH', 'search'),
|
||||
('SEARCH', 'SEARCH_PLAYER', 'click_card');
|
||||
@@ -0,0 +1,353 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
探测脚本 v2:检测抖音SMS登录被"必须使用App登录"拦截
|
||||
|
||||
v2 核心变化:不需要 SMS 验证码!
|
||||
因为账号被标记后,抖音可能根本不发短信。
|
||||
我们只要导航到登录页、填入手机号、点击获取验证码,
|
||||
观察页面是否立即出现"必须使用App"的提示。
|
||||
|
||||
用法:
|
||||
python3 detect_app_login.py --phone=15370108283
|
||||
|
||||
输出:
|
||||
/tmp/detect_login_result.json — 包含红色提示文字和截图
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import traceback
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
_SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
sys.path.insert(0, str(_SCRIPT_DIR / "matrix_modules"))
|
||||
sys.path.insert(0, str(_SCRIPT_DIR))
|
||||
|
||||
|
||||
def log(msg: str):
|
||||
print(f"[{datetime.now().strftime('%H:%M:%S')}] {msg}", flush=True)
|
||||
|
||||
|
||||
async def detect(phone: str, output_dir: str) -> dict:
|
||||
"""执行一次登录探测,不需要验证码"""
|
||||
result = {
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
"phone": phone,
|
||||
"steps": [],
|
||||
"red_texts": [],
|
||||
"hint_texts": [],
|
||||
"near_login_texts": [],
|
||||
"page_url": "",
|
||||
"page_title": "",
|
||||
"screenshot_path": "",
|
||||
"full_text_snippet": "",
|
||||
"found_app_login": False,
|
||||
"app_keyword_context": [],
|
||||
"error": None,
|
||||
}
|
||||
|
||||
from camoufox.async_api import AsyncCamoufox
|
||||
screenshot_path = os.path.join(output_dir, "login_detect.png")
|
||||
|
||||
try:
|
||||
fox = AsyncCamoufox(
|
||||
headless=False,
|
||||
persistent_context=False,
|
||||
locale="zh-CN",
|
||||
os="windows",
|
||||
humanize=1.5,
|
||||
)
|
||||
browser = await fox.start()
|
||||
if browser.contexts:
|
||||
context = browser.contexts[0]
|
||||
else:
|
||||
context = await browser.new_context(
|
||||
viewport={"width": 1280, "height": 800}
|
||||
)
|
||||
page = context.pages[0] if context.pages else await context.new_page()
|
||||
|
||||
# ── Step 1: 导航到登录页 ──
|
||||
log("导航到抖音登录页...")
|
||||
await page.goto(
|
||||
"https://www.douyin.com/passport/sso/login/",
|
||||
timeout=30000, wait_until="domcontentloaded"
|
||||
)
|
||||
await asyncio.sleep(4)
|
||||
result["steps"].append("navigate_login")
|
||||
result["page_url"] = page.url
|
||||
result["page_title"] = await page.title()
|
||||
|
||||
# ── Step 2: 找手机号输入框并填号 ──
|
||||
log("查找手机号输入框...")
|
||||
phone_filled = False
|
||||
for sel in [
|
||||
'input[placeholder*="手机"]',
|
||||
'input[type="tel"]',
|
||||
'input[name="mobile"]',
|
||||
]:
|
||||
try:
|
||||
el = page.locator(sel).first
|
||||
if await el.count() > 0 and await el.is_visible():
|
||||
await el.click()
|
||||
await asyncio.sleep(0.3)
|
||||
await el.fill(phone)
|
||||
phone_filled = True
|
||||
log(f"已填入手机号: {phone}")
|
||||
break
|
||||
except:
|
||||
continue
|
||||
|
||||
if not phone_filled:
|
||||
# JS 兜底
|
||||
try:
|
||||
await page.evaluate(f"""() => {{
|
||||
const inputs = document.querySelectorAll('input');
|
||||
for (const inp of inputs) {{
|
||||
if (inp.offsetParent !== null && inp.value === '') {{
|
||||
inp.focus();
|
||||
inp.value = '{phone}';
|
||||
inp.dispatchEvent(new Event('input', {{bubbles: true}}));
|
||||
return true;
|
||||
}}
|
||||
}}
|
||||
return false;
|
||||
}}""")
|
||||
await asyncio.sleep(1)
|
||||
phone_filled = True
|
||||
log("JS 填手机号成功")
|
||||
except Exception as e:
|
||||
result["error"] = f"无法填手机号: {e}"
|
||||
await browser.close()
|
||||
return result
|
||||
|
||||
result["steps"].append("filled_phone")
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# ── Step 3: 截图(填写手机号后的页面状态)──
|
||||
try:
|
||||
await page.screenshot(path=screenshot_path, full_page=False)
|
||||
result["screenshot_path"] = screenshot_path
|
||||
except:
|
||||
pass
|
||||
|
||||
# ── Step 4: 捕捉"点击获取验证码"前的页面状态 ──
|
||||
log("捕捉页面上现有文本...")
|
||||
try:
|
||||
full_text = await page.evaluate("document.body.innerText") or ""
|
||||
result["full_text_snippet"] = full_text[:2000]
|
||||
# 搜索 App 相关关键词
|
||||
for kw in ["App", "app", "APP", "应用", "抖音App", "客户端", "扫一扫",
|
||||
"请使用", "请打开", "打开抖音", "扫码"]:
|
||||
idx = full_text.find(kw)
|
||||
if idx >= 0:
|
||||
nearby = full_text[max(0, idx-20):idx+80]
|
||||
result["app_keyword_context"].append(nearby)
|
||||
log(f"⚠️ 关键词 '{kw}': ...{nearby}...")
|
||||
except:
|
||||
pass
|
||||
|
||||
# ── Step 5: 点击获取验证码 ──
|
||||
log("点击获取验证码...")
|
||||
clicked = False
|
||||
for text in ["获取验证码", "获取"]:
|
||||
try:
|
||||
btn = page.locator(f'button:has-text("{text}"), span:has-text("{text}")').first
|
||||
if await btn.count() > 0 and await btn.is_visible():
|
||||
await btn.click()
|
||||
clicked = True
|
||||
log(f"点击 '{text}'")
|
||||
break
|
||||
except:
|
||||
continue
|
||||
|
||||
if not clicked:
|
||||
try:
|
||||
await page.evaluate("""() => {
|
||||
const all = document.querySelectorAll('button, span, div');
|
||||
for (const el of all) {
|
||||
if (el.offsetParent === null) continue;
|
||||
const t = el.textContent.trim();
|
||||
if (t.includes('获取验证码') || t.includes('获取')) {
|
||||
el.click(); return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}""")
|
||||
clicked = True
|
||||
log("JS 点击获取验证码")
|
||||
except:
|
||||
pass
|
||||
|
||||
if not clicked:
|
||||
result["error"] = "无法点击获取验证码"
|
||||
await browser.close()
|
||||
return result
|
||||
|
||||
result["steps"].append("clicked_get_code")
|
||||
|
||||
# ── Step 6: 等待页面响应(关键!捕捉错误提示)──
|
||||
await asyncio.sleep(5)
|
||||
|
||||
# 截图(点击获取验证码后的页面)
|
||||
try:
|
||||
await page.screenshot(path=screenshot_path.replace(".png", "_after_code.png"))
|
||||
except:
|
||||
pass
|
||||
|
||||
result["page_url"] = page.url
|
||||
|
||||
# ── Step 7: 全面捕捉页面文本 ──
|
||||
log("捕捉页面文本...")
|
||||
|
||||
full_text = ""
|
||||
try:
|
||||
full_text = await page.evaluate("document.body.innerText") or ""
|
||||
except:
|
||||
pass
|
||||
result["full_text_snippet"] = full_text[:2000]
|
||||
|
||||
# 搜索 App 关键词
|
||||
app_kws = ["App", "app", "APP", "应用", "抖音App", "客户端", "扫一扫",
|
||||
"请使用", "请打开", "打开抖音", "扫码", "下载", "安装"]
|
||||
for kw in app_kws:
|
||||
idx = full_text.find(kw)
|
||||
if idx >= 0:
|
||||
nearby = full_text[max(0, idx-20):idx+80]
|
||||
result["app_keyword_context"].append(nearby)
|
||||
result["found_app_login"] = True
|
||||
log(f"✅ 关键词 '{kw}': ...{nearby}...")
|
||||
|
||||
# ── Step 8: 捕捉红色文字 ──
|
||||
try:
|
||||
red_texts = await page.evaluate("""() => {
|
||||
const results = [];
|
||||
const all = document.querySelectorAll('*');
|
||||
for (const el of all) {
|
||||
if (el.offsetParent === null) continue;
|
||||
const style = window.getComputedStyle(el);
|
||||
const color = style.color;
|
||||
if (color.includes('rgb(255,') || color === 'red' ||
|
||||
color.includes('#e5') || color.includes('#f0') ||
|
||||
color.includes('#d32')) {
|
||||
const t = el.textContent.trim();
|
||||
if (t.length > 2 && t.length < 200) {
|
||||
results.push({
|
||||
text: t,
|
||||
color: color,
|
||||
tag: el.tagName,
|
||||
class: (el.className || '').substring(0, 80)
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
return results;
|
||||
}""")
|
||||
seen = set()
|
||||
for rt in red_texts:
|
||||
t = rt["text"][:40]
|
||||
if t not in seen:
|
||||
seen.add(t)
|
||||
result["red_texts"].append(rt)
|
||||
if result["red_texts"]:
|
||||
log(f"找到 {len(result['red_texts'])} 条红色文字:")
|
||||
for rt in result["red_texts"][:10]:
|
||||
log(f" ⛔ {rt['text'][:100]}")
|
||||
except Exception as e:
|
||||
log(f"红色文字捕捉异常: {e}")
|
||||
|
||||
# ── Step 9: 捕捉提示/错误信息容器 ──
|
||||
try:
|
||||
hints = await page.evaluate("""() => {
|
||||
const results = [];
|
||||
const containers = document.querySelectorAll(
|
||||
'.error, .tip, .msg, .toast, .hint, [class*="error"], ' +
|
||||
'[class*="tip"], [class*="msg"], [class*="warn"], ' +
|
||||
'[class*="alert"], [class*="notice"]'
|
||||
);
|
||||
for (const c of containers) {
|
||||
if (c.offsetParent !== null && c.textContent.trim().length > 0) {
|
||||
results.push({
|
||||
text: c.textContent.trim().substring(0, 300),
|
||||
class: (c.className || '').substring(0, 100),
|
||||
tag: c.tagName
|
||||
});
|
||||
}
|
||||
}
|
||||
return results;
|
||||
}""")
|
||||
result["hint_texts"] = hints
|
||||
if hints:
|
||||
log(f"找到 {len(hints)} 个提示容器:")
|
||||
for h in hints[:10]:
|
||||
log(f" 📋 {h['text'][:120]}")
|
||||
except:
|
||||
pass
|
||||
|
||||
# ── Step 10: 检测登录框状态 ──
|
||||
# 如果 URL 仍然是登录页,说明没有自动登录成功
|
||||
current_url = page.url
|
||||
result["page_url"] = current_url
|
||||
still_on_login = "passport" in current_url.lower() or "login" in current_url.lower()
|
||||
if still_on_login:
|
||||
log("⚠️ 仍在登录页 — 登录被拦截")
|
||||
result["found_app_login"] = True
|
||||
|
||||
await browser.close()
|
||||
log("探测完成")
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
tb = traceback.format_exc()
|
||||
result["error"] = f"{type(e).__name__}: {e}\n{tb[:300]}"
|
||||
log(f"❌ 异常: {e}")
|
||||
try:
|
||||
await browser.close()
|
||||
except:
|
||||
pass
|
||||
return result
|
||||
|
||||
|
||||
async def main():
|
||||
parser = argparse.ArgumentParser(description="探测抖音 App 登录要求 (v2, 不需要SMS)")
|
||||
parser.add_argument("--phone", default="15370108283", help="手机号")
|
||||
parser.add_argument("--output-dir", default="/tmp", help="输出目录")
|
||||
args = parser.parse_args()
|
||||
|
||||
os.makedirs(args.output_dir, exist_ok=True)
|
||||
log(f"开始探测: phone={args.phone}")
|
||||
log(f"注意: 此版本不需要 SMS 验证码,只观察页面响应")
|
||||
|
||||
result = await detect(args.phone, args.output_dir)
|
||||
|
||||
result_file = os.path.join(args.output_dir, "detect_login_result.json")
|
||||
with open(result_file, "w", encoding="utf-8") as f:
|
||||
json.dump(result, f, ensure_ascii=False, indent=2)
|
||||
log(f"结果已保存: {result_file}")
|
||||
|
||||
# 输出摘要
|
||||
log("\n" + "=" * 60)
|
||||
if result.get("found_app_login"):
|
||||
log("✅ 结论: 检测到 App 登录要求!")
|
||||
for ctx in result.get("app_keyword_context", []):
|
||||
log(f" 关键词上下文: {ctx}")
|
||||
else:
|
||||
log("⚠️ 未检测到 App 登录特征")
|
||||
log("可能是账号还可以正常 SMS 登录,或需要实际获得验证码才能触发")
|
||||
|
||||
if result.get("red_texts"):
|
||||
log("\n所有红色文字:")
|
||||
for rt in result["red_texts"]:
|
||||
log(f" ⛔ {rt['text']}")
|
||||
if result.get("hint_texts"):
|
||||
log("\n所有提示容器:")
|
||||
for h in result["hint_texts"]:
|
||||
log(f" 📋 {h['text']}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,444 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
guardd — AgentOS 系统自愈守护进程 (v1.0)
|
||||
|
||||
职责:
|
||||
1. 检测:孤儿浏览器 / 磁盘空间 / Dashboard 存活 / 命令超时 / 浏览器超量
|
||||
2. 恢复:kill 孤儿进程 / 清理磁盘 / 重启 Dashboard / kill 超时命令
|
||||
3. 心跳:每 60s 向 Dashboard 推送健康状态 + 恢复事件
|
||||
|
||||
用法:
|
||||
python guardd.py # 前台运行(调试)
|
||||
nohup python guardd.py & # 后台运行
|
||||
agentos serve guardd # 通过 agentos 启动
|
||||
|
||||
约束:
|
||||
- 每项恢复动作有频率限制(rate limit),防止循环崩溃
|
||||
- 日志写入: agent-local/runtime/guardd/events.log
|
||||
"""
|
||||
|
||||
import os, sys, json, time, subprocess, shutil, glob, logging
|
||||
from pathlib import Path
|
||||
from datetime import datetime, timezone
|
||||
|
||||
# ── 路径 ──────────────────────────────────────────────
|
||||
# 先尝试从 matrix_mgmt 导入(有 yaml 依赖)
|
||||
# 兜底用硬编码(不依赖 yaml)
|
||||
try:
|
||||
from matrix_mgmt import AGENT_SYNC, AGENT_LOCAL
|
||||
except ImportError:
|
||||
AGENT_SYNC = Path.home() / "workbuddy-agent-os" / "agent-sync"
|
||||
AGENT_LOCAL = Path.home() / "workbuddy-agent-os" / "agent-local"
|
||||
|
||||
LOG_DIR = AGENT_LOCAL / "runtime" / "guardd"
|
||||
LOG_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
EVENTS_LOG = LOG_DIR / "events.log"
|
||||
STATUS_FILE = LOG_DIR / "status.json"
|
||||
|
||||
# ── 配置 ──────────────────────────────────────────────
|
||||
INTERVAL = 60 # 检测间隔(秒)
|
||||
DASHBOARD_URL = os.environ.get("GUARDD_DASHBOARD_URL", "http://localhost:9988")
|
||||
MAX_BROWSERS = 3 # 最大浏览器数
|
||||
MIN_DISK_GB = 5 # 最小磁盘空间
|
||||
CMD_TIMEOUT = 1800 # 命令超时(秒) = 30min
|
||||
HEARTBEAT_URL = f"{DASHBOARD_URL}/api/push/heartbeat"
|
||||
|
||||
# 频率限制(每个恢复动作计数)
|
||||
_RATE_LIMIT = {} # {action_name: [count, last_reset_time]}
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s [%(levelname)s] %(message)s",
|
||||
datefmt="%H:%M:%S",
|
||||
)
|
||||
log = logging.getLogger("guardd")
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════
|
||||
# 检测器
|
||||
# ══════════════════════════════════════════════════════
|
||||
|
||||
def detect_orphan_browsers() -> list[dict]:
|
||||
"""检测孤儿浏览器进程(PPID=1)"""
|
||||
orphans = []
|
||||
try:
|
||||
r = subprocess.run(["ps", "aux"], capture_output=True, text=True, timeout=10)
|
||||
for line in r.stdout.split('\n'):
|
||||
if 'camoufox' not in line and 'HeadlessShell' not in line:
|
||||
continue
|
||||
parts = line.split()
|
||||
if len(parts) < 3:
|
||||
continue
|
||||
pid = parts[1]
|
||||
ppid = parts[2]
|
||||
if ppid == '1': # 孤儿进程
|
||||
# 获取启动时间
|
||||
try:
|
||||
pr = subprocess.run(["ps", "-p", pid, "-o", "lstart="],
|
||||
capture_output=True, text=True, timeout=5)
|
||||
started = pr.stdout.strip()[:19] if pr.stdout.strip() else "?"
|
||||
except:
|
||||
started = "?"
|
||||
orphans.append({"pid": int(pid), "started": started, "cmd": ' '.join(parts[10:])[:80]})
|
||||
except Exception as e:
|
||||
log.warning(f"检测孤儿浏览器失败: {e}")
|
||||
return orphans
|
||||
|
||||
|
||||
def detect_running_browsers() -> list[dict]:
|
||||
"""检测所有正在运行的浏览器"""
|
||||
browsers = []
|
||||
try:
|
||||
r = subprocess.run(
|
||||
["pgrep", "-f", "camoufox.*-no-remote|HeadlessShell"],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
for pid_str in r.stdout.strip().split('\n'):
|
||||
pid_str = pid_str.strip()
|
||||
if not pid_str:
|
||||
continue
|
||||
try:
|
||||
pid = int(pid_str)
|
||||
# 获取启动时间
|
||||
pr = subprocess.run(["ps", "-p", str(pid), "-o", "lstart="],
|
||||
capture_output=True, text=True, timeout=5)
|
||||
started = pr.stdout.strip()[:19] if pr.stdout.strip() else "?"
|
||||
browsers.append({"pid": pid, "started": started})
|
||||
except ValueError:
|
||||
continue
|
||||
except Exception:
|
||||
pass
|
||||
return browsers
|
||||
|
||||
|
||||
def detect_disk_space() -> dict:
|
||||
"""检测磁盘空间"""
|
||||
result = {"available_gb": 0, "sufficient": True}
|
||||
try:
|
||||
r = subprocess.run(["df", "-h", "/"], capture_output=True, text=True, timeout=5)
|
||||
parts = r.stdout.strip().split('\n')[1].split()
|
||||
avail = parts[3] if len(parts) >= 4 else "0"
|
||||
gb = 0
|
||||
if avail.endswith('Gi'): gb = float(avail[:-2])
|
||||
elif avail.endswith('G'): gb = float(avail[:-1])
|
||||
elif avail.endswith('Mi'): gb = float(avail[:-2]) / 1024
|
||||
elif avail.endswith('M'): gb = float(avail[:-1]) / 1024
|
||||
result["available_gb"] = round(gb, 1)
|
||||
result["sufficient"] = gb >= MIN_DISK_GB
|
||||
except Exception as e:
|
||||
log.warning(f"检测磁盘失败: {e}")
|
||||
return result
|
||||
|
||||
|
||||
def detect_dashboard_alive() -> bool:
|
||||
"""检测 Dashboard 是否存活"""
|
||||
try:
|
||||
r = subprocess.run(
|
||||
["curl", "-s", "-o", "/dev/null", "-w", "%{http_code}",
|
||||
f"{DASHBOARD_URL}/api/health"],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
return r.stdout.strip() == "200"
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def detect_stale_commands() -> list[dict]:
|
||||
"""检测超时的命令进程"""
|
||||
stale = []
|
||||
try:
|
||||
r = subprocess.run(["ps", "aux"], capture_output=True, text=True, timeout=10)
|
||||
now = time.time()
|
||||
for line in r.stdout.split('\n'):
|
||||
if 'mc run' not in line and 'mc collect' not in line and 'collect_batch_runner' not in line:
|
||||
continue
|
||||
parts = line.split()
|
||||
if len(parts) < 10:
|
||||
continue
|
||||
pid = parts[1]
|
||||
# 计算进程启动时间(ps 输出第9列是启动时间或已用时间)
|
||||
start_str = parts[8] if len(parts) > 8 else "?"
|
||||
cmd_line = ' '.join(parts[10:])[:80]
|
||||
stale.append({"pid": int(pid), "cmd": cmd_line, "start": start_str})
|
||||
except Exception as e:
|
||||
log.warning(f"检测超时命令失败: {e}")
|
||||
return stale
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════
|
||||
# 恢复动作
|
||||
# ══════════════════════════════════════════════════════
|
||||
|
||||
def _check_rate(action: str, max_per_hour: int = 3) -> bool:
|
||||
"""频率限制:每项动作每小时最多执行 max_per_hour 次"""
|
||||
global _RATE_LIMIT
|
||||
now = time.time()
|
||||
if action not in _RATE_LIMIT:
|
||||
_RATE_LIMIT[action] = [0, now]
|
||||
count, last = _RATE_LIMIT[action]
|
||||
# 每小时重置
|
||||
if now - last > 3600:
|
||||
_RATE_LIMIT[action] = [0, now]
|
||||
return True
|
||||
if count >= max_per_hour:
|
||||
return False
|
||||
_RATE_LIMIT[action][0] += 1
|
||||
return True
|
||||
|
||||
|
||||
def recover_orphan_browsers(orphans: list[dict]) -> list[str]:
|
||||
"""杀死孤儿浏览器进程,并写 crash 结果文件"""
|
||||
killed = []
|
||||
for o in orphans:
|
||||
if not _check_rate(f"kill_orphan_{o['pid']}"):
|
||||
continue
|
||||
try:
|
||||
subprocess.run(["kill", "-9", str(o['pid'])], capture_output=True, timeout=5)
|
||||
killed.append(f"孤儿进程 {o['pid']} ({o['cmd'][:40]})")
|
||||
log.warning(f" 🔴 已杀死孤儿: {o['pid']}")
|
||||
|
||||
# 尝试写 crash 结果文件(从 cmd 中提取 run_id)
|
||||
import re
|
||||
run_id_match = re.search(r'(run_|nurture_|collect_|login_)([a-f0-9]+)', str(o.get('cmd', '')))
|
||||
if run_id_match:
|
||||
full_run_id = run_id_match.group(0)
|
||||
try:
|
||||
from mc.execution_policy import write_result as _wr
|
||||
_wr(full_run_id, "crashed", error=f"孤儿浏览器被guardd杀死 (PID {o['pid']})",
|
||||
message="浏览器进程失去父进程控制,被guardd清理")
|
||||
except:
|
||||
pass
|
||||
except Exception as e:
|
||||
log.warning(f" 杀死孤儿 {o['pid']} 失败: {e}")
|
||||
return killed
|
||||
|
||||
|
||||
def recover_disk_space() -> list[str]:
|
||||
"""磁盘空间不足时清理临时文件"""
|
||||
cleaned = []
|
||||
if not _check_rate("disk_cleanup", max_per_hour=2):
|
||||
return cleaned
|
||||
for pattern in ["/tmp/camoufox_*", "/tmp/playwright_*"]:
|
||||
for fp in glob.glob(pattern):
|
||||
try:
|
||||
shutil.rmtree(fp, ignore_errors=True)
|
||||
cleaned.append(f"已清理: {fp}")
|
||||
except:
|
||||
pass
|
||||
if cleaned:
|
||||
log.warning(f" 🧹 磁盘清理: {len(cleaned)}个临时目录")
|
||||
return cleaned
|
||||
|
||||
|
||||
def recover_dashboard() -> list[str]:
|
||||
"""重启 Dashboard"""
|
||||
actions = []
|
||||
if not _check_rate("restart_dashboard", max_per_hour=2):
|
||||
return actions
|
||||
|
||||
dashboard_dir = AGENT_SYNC / "05_tools" / "10_dashboard"
|
||||
python = os.environ.get("MC_PYTHON",
|
||||
f"{Path.home()}/.workbuddy/binaries/python/envs/agent-os/bin/python3")
|
||||
|
||||
try:
|
||||
# kill old
|
||||
subprocess.run(["pkill", "-f", "uvicorn.*9988"], capture_output=True, timeout=5)
|
||||
time.sleep(2)
|
||||
# start new
|
||||
subprocess.Popen(
|
||||
[python, "-m", "uvicorn", "app:app", "--host", "0.0.0.0", "--port", "9988"],
|
||||
cwd=str(dashboard_dir),
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
|
||||
)
|
||||
actions.append("Dashboard 已重启")
|
||||
log.warning(" 🚑 Dashboard 已重启")
|
||||
except Exception as e:
|
||||
log.warning(f" 重启 Dashboard 失败: {e}")
|
||||
return actions
|
||||
|
||||
|
||||
def recover_stale_commands(stale: list[dict]) -> list[str]:
|
||||
"""杀死超时命令"""
|
||||
killed = []
|
||||
for s in stale:
|
||||
if not _check_rate(f"kill_cmd_{s['pid']}", max_per_hour=6):
|
||||
continue
|
||||
try:
|
||||
subprocess.run(["kill", "-9", str(s['pid'])], capture_output=True, timeout=5)
|
||||
killed.append(f"超时命令 {s['pid']} ({s['cmd'][:40]})")
|
||||
log.warning(f" ⏰ 已杀死超时命令: {s['pid']}")
|
||||
except Exception as e:
|
||||
log.warning(f" 杀死命令 {s['pid']} 失败: {e}")
|
||||
return killed
|
||||
|
||||
|
||||
def recover_excess_browsers(browsers: list[dict]) -> list[str]:
|
||||
"""超过最大浏览器数时,按启动时间杀死最早的"""
|
||||
killed = []
|
||||
if len(browsers) <= MAX_BROWSERS:
|
||||
return killed
|
||||
if not _check_rate("kill_excess_browser", max_per_hour=3):
|
||||
return killed
|
||||
|
||||
# 排序:越早的越可能在前
|
||||
excess = sorted(browsers, key=lambda b: b.get("started", ""))
|
||||
to_kill = excess[:-MAX_BROWSERS] # 保留最新的 MAX_BROWSERS 个
|
||||
|
||||
for b in to_kill:
|
||||
try:
|
||||
subprocess.run(["kill", "-9", str(b['pid'])], capture_output=True, timeout=5)
|
||||
killed.append(f"超额浏览器 {b['pid']}")
|
||||
log.warning(f" 📊 已杀死超额浏览器: {b['pid']}")
|
||||
except:
|
||||
pass
|
||||
return killed
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════
|
||||
# 心跳推送
|
||||
# ══════════════════════════════════════════════════════
|
||||
|
||||
def push_heartbeat(events: list[str], stats: dict):
|
||||
"""向 Dashboard 推送心跳"""
|
||||
import uuid
|
||||
hostname = os.uname().nodename
|
||||
|
||||
# 读取本机 UID
|
||||
uid_file = AGENT_LOCAL / "identity" / "machine_uid"
|
||||
uid = uid_file.read_text().strip() if uid_file.exists() else str(uuid.uuid4())[:8]
|
||||
|
||||
payload = {
|
||||
"uid": uid,
|
||||
"hostname": hostname,
|
||||
"uptime": time.time(),
|
||||
"heartbeat": {
|
||||
"status": "online",
|
||||
"guardd_running": True,
|
||||
"guardd_version": "1.0",
|
||||
"disk_avail_gb": stats.get("disk_gb", 0),
|
||||
"browsers_running": stats.get("browsers", 0),
|
||||
"last_check": datetime.now(timezone.utc).isoformat(),
|
||||
},
|
||||
"events": [{"time": datetime.now().isoformat(), "message": e} for e in events],
|
||||
}
|
||||
|
||||
try:
|
||||
subprocess.run(
|
||||
["curl", "-s", "-X", "POST", HEARTBEAT_URL,
|
||||
"-H", "Content-Type: application/json",
|
||||
"-d", json.dumps(payload)],
|
||||
capture_output=True, timeout=10
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════
|
||||
# 主循环
|
||||
# ══════════════════════════════════════════════════════
|
||||
|
||||
def run_once() -> dict:
|
||||
"""执行一次检测+恢复,返回事件列表"""
|
||||
events = []
|
||||
stats = {}
|
||||
|
||||
# 1. 检测孤儿浏览器
|
||||
orphans = detect_orphan_browsers()
|
||||
if orphans:
|
||||
log.warning(f"🔴 检测到 {len(orphans)} 个孤儿浏览器")
|
||||
killed = recover_orphan_browsers(orphans)
|
||||
events.extend(killed)
|
||||
|
||||
# 2. 检测磁盘
|
||||
disk = detect_disk_space()
|
||||
stats["disk_gb"] = disk["available_gb"]
|
||||
if not disk["sufficient"]:
|
||||
log.warning(f"🔴 磁盘不足: {disk['available_gb']}GB")
|
||||
cleaned = recover_disk_space()
|
||||
events.extend(cleaned)
|
||||
if not cleaned:
|
||||
events.append(f"磁盘不足: {disk['available_gb']}GB (跳过,已达频率上限)")
|
||||
|
||||
# 3. 检测 Dashboard
|
||||
if not detect_dashboard_alive():
|
||||
log.warning("🔴 Dashboard 无响应")
|
||||
restarted = recover_dashboard()
|
||||
events.extend(restarted)
|
||||
else:
|
||||
stats["dashboard"] = "ok"
|
||||
|
||||
# 4. 检测浏览器数量
|
||||
browsers = detect_running_browsers()
|
||||
stats["browsers"] = len(browsers)
|
||||
if len(browsers) > MAX_BROWSERS:
|
||||
log.warning(f"🟡 浏览器超量: {len(browsers)} > {MAX_BROWSERS}")
|
||||
killed = recover_excess_browsers(browsers)
|
||||
events.extend(killed)
|
||||
|
||||
# 5. 清理过期结果文件
|
||||
try:
|
||||
from mc.execution_policy import cleanup_old_results as _clean
|
||||
_clean(max_age_hours=24)
|
||||
except:
|
||||
pass
|
||||
|
||||
# 6. 检测超时命令
|
||||
stale = detect_stale_commands()
|
||||
if stale:
|
||||
log.warning(f"🟡 检测到 {len(stale)} 个超时命令")
|
||||
killed = recover_stale_commands(stale)
|
||||
events.extend(killed)
|
||||
|
||||
# 写状态文件
|
||||
status = {
|
||||
"last_check": datetime.now().isoformat(),
|
||||
"orphans": len(orphans),
|
||||
"disk_gb": disk["available_gb"],
|
||||
"browsers": len(browsers),
|
||||
"stale_commands": len(stale),
|
||||
"events": events,
|
||||
}
|
||||
STATUS_FILE.write_text(json.dumps(status, indent=2, ensure_ascii=False))
|
||||
|
||||
# 记录事件
|
||||
if events:
|
||||
with open(EVENTS_LOG, "a") as f:
|
||||
ts = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
for e in events:
|
||||
f.write(f"[{ts}] {e}\n")
|
||||
|
||||
# 推送心跳
|
||||
push_heartbeat(events, stats)
|
||||
|
||||
return status
|
||||
|
||||
|
||||
def main():
|
||||
print(f"\n{'='*50}")
|
||||
print(f" 🛡️ AgentOS guardd v1.0")
|
||||
print(f" {'='*50}")
|
||||
print(f" 日志: {EVENTS_LOG}")
|
||||
print(f" 间隔: {INTERVAL}s")
|
||||
print(f" Dashboard: {DASHBOARD_URL}\n")
|
||||
|
||||
log.info("guardd 启动")
|
||||
|
||||
while True:
|
||||
try:
|
||||
status = run_once()
|
||||
if status["events"]:
|
||||
log.info(f" 恢复动作: {'; '.join(status['events'])}")
|
||||
else:
|
||||
log.info(f" 健康 ✅ (浏览器:{status['browsers']} 磁盘:{status['disk_gb']}GB)")
|
||||
except KeyboardInterrupt:
|
||||
log.info("guardd 停止")
|
||||
break
|
||||
except Exception as e:
|
||||
log.error(f"guardd 异常: {e}")
|
||||
|
||||
time.sleep(INTERVAL)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,119 @@
|
||||
"""
|
||||
路径配置模块 — 统一管理本机数据目录路径
|
||||
|
||||
通过读取 07_matrix/local.yaml 获取本机数据根目录,
|
||||
所有 Python 脚本通过本模块获取路径,不再依赖软链接。
|
||||
|
||||
本模块为代码层(参与坚果云同步),local.yaml 为数据层(每台机器自生成,不参与同步)。
|
||||
|
||||
用法:
|
||||
from local_paths import config_path, data_path, logs_path, profiles_path, screenshots_path, code_dir
|
||||
|
||||
accounts = config_path("accounts.yaml") # → agent-local/.../config/accounts.yaml
|
||||
db = data_path("matrix.db") # → agent-local/.../data/matrix.db
|
||||
logfile = logs_path("task.log") # → agent-local/.../logs/task.log
|
||||
profile = profiles_path("douyin_01") # → agent-local/.../profiles/douyin_01
|
||||
screenshot = screenshots_path("snap.png") # → agent-local/.../screenshots/snap.png
|
||||
blueprint = code_dir() / "blueprints" / "xxx.json" # 代码目录下的文件
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
# 代码根目录 (07_matrix/)
|
||||
CODE_DIR = Path(__file__).resolve().parent.parent
|
||||
|
||||
# 本地配置文件路径
|
||||
_LOCAL_CONF = CODE_DIR / "local.yaml"
|
||||
|
||||
# 延迟加载缓存
|
||||
_LOCAL_ROOT = None
|
||||
|
||||
|
||||
def _load_config() -> dict:
|
||||
"""读取 local.yaml 配置"""
|
||||
if not _LOCAL_CONF.exists():
|
||||
print(
|
||||
f"\n"
|
||||
f"❌ local.yaml 不存在,请先运行安装脚本:\n"
|
||||
f" bash {CODE_DIR / 'install.sh'}\n",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
# 用内置方式解析 YAML(只读 matrix.local_data_root,避免依赖 pyyaml)
|
||||
with open(_LOCAL_CONF, encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
|
||||
for line in content.splitlines():
|
||||
line = line.strip()
|
||||
if line.startswith("local_data_root:"):
|
||||
value = line.split(":", 1)[1].strip().strip('"').strip("'")
|
||||
return {"matrix": {"local_data_root": value}}
|
||||
|
||||
print(
|
||||
f"❌ local.yaml 中缺少 local_data_root 配置\n",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def _get_local_root() -> Path:
|
||||
"""获取本机数据根目录(延迟加载 + 缓存)"""
|
||||
global _LOCAL_ROOT
|
||||
if _LOCAL_ROOT is not None:
|
||||
return _LOCAL_ROOT
|
||||
|
||||
cfg = _load_config()
|
||||
root_str = cfg.get("matrix", {}).get("local_data_root", "")
|
||||
if not root_str:
|
||||
print("❌ local.yaml 中缺少 local_data_root 配置", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
_LOCAL_ROOT = Path(root_str).expanduser().resolve()
|
||||
_LOCAL_ROOT.mkdir(parents=True, exist_ok=True)
|
||||
return _LOCAL_ROOT
|
||||
|
||||
|
||||
def code_dir() -> Path:
|
||||
"""返回代码根目录 (07_matrix/)"""
|
||||
return CODE_DIR
|
||||
|
||||
|
||||
def _local_subdir(name: str, *parts) -> Path:
|
||||
"""获取本机数据目录下的子目录/文件路径,自动创建父目录"""
|
||||
root = _get_local_root()
|
||||
target = root / name / Path(*parts) if parts else root / name
|
||||
|
||||
# 如果有子路径(文件),只创建父目录
|
||||
if parts:
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
else:
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
return target
|
||||
|
||||
|
||||
def config_path(*parts) -> Path:
|
||||
"""→ agent-local/.../config/[*parts]"""
|
||||
return _local_subdir("config", *parts)
|
||||
|
||||
|
||||
def data_path(*parts) -> Path:
|
||||
"""→ agent-local/.../data/[*parts]"""
|
||||
return _local_subdir("data", *parts)
|
||||
|
||||
|
||||
def logs_path(*parts) -> Path:
|
||||
"""→ agent-local/.../logs/[*parts]"""
|
||||
return _local_subdir("logs", *parts)
|
||||
|
||||
|
||||
def profiles_path(*parts) -> Path:
|
||||
"""→ agent-local/.../profiles/[*parts]"""
|
||||
return _local_subdir("profiles", *parts)
|
||||
|
||||
|
||||
def screenshots_path(*parts) -> Path:
|
||||
"""→ agent-local/.../screenshots/[*parts]"""
|
||||
return _local_subdir("screenshots", *parts)
|
||||
@@ -0,0 +1,180 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
身份登录 — 首次手动登录,登录态保存到持久化目录
|
||||
|
||||
用法:
|
||||
python login_identity.py <identity_name> [--platform douyin|xiaohongshu]
|
||||
|
||||
流程:
|
||||
1. 通过 cdp_connector.CDPConnector(identity_dir=...) 启动 Camoufox 持久化
|
||||
2. 用户手动登录
|
||||
3. 按 Enter 触发 → 状态自动保存
|
||||
|
||||
支持平台:
|
||||
- douyin → douyin.com
|
||||
- xiaohongshu → xiaohongshu.com/explore
|
||||
"""
|
||||
import argparse
|
||||
import asyncio
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from matrix_mgmt import AGENT_LOCAL
|
||||
IDENTITIES_ROOT = AGENT_LOCAL / "tools" / "matrix" / "identities"
|
||||
|
||||
# ── 平台 URL 映射 ──
|
||||
PLATFORM_URLS = {
|
||||
"douyin": "https://www.douyin.com/",
|
||||
"xiaohongshu": "https://www.xiaohongshu.com/explore",
|
||||
}
|
||||
|
||||
PLATFORM_NAMES = {
|
||||
"douyin": "抖音",
|
||||
"xiaohongshu": "小红书",
|
||||
}
|
||||
|
||||
|
||||
async def login_identity(identity_name: str, platform: str = "auto"):
|
||||
"""执行首次手动登录
|
||||
Args:
|
||||
identity_name: 账号ID
|
||||
platform: "auto" 时从 accounts.yaml 自动解析
|
||||
"""
|
||||
import sys as _sys
|
||||
_sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
# 从账号注册表读取 identity_dir 和 platform
|
||||
acct_info = {}
|
||||
try:
|
||||
from matrix_mgmt import MatrixManager
|
||||
mgr = MatrixManager()
|
||||
for a in mgr.list_accounts():
|
||||
if a["id"] == identity_name:
|
||||
acct_info = a
|
||||
break
|
||||
except: pass
|
||||
|
||||
# 身份目录:优先用 accounts.yaml 中的 identity_dir,否则用账号ID
|
||||
acct_identity = acct_info.get("identity_dir") or acct_info.get("identity_hint") or identity_name
|
||||
acct_identity = acct_identity.replace("identities/", "")
|
||||
identity_dir = str(IDENTITIES_ROOT / acct_identity)
|
||||
|
||||
# 平台:auto 时从账号信息解析
|
||||
if platform == "auto":
|
||||
p = acct_info.get("platform", "douyin")
|
||||
platform = p if p in ("douyin", "xiaohongshu") else "douyin"
|
||||
|
||||
signal_file = Path(f"/tmp/login_identity_{identity_name}.signal")
|
||||
platform_url = PLATFORM_URLS.get(platform, PLATFORM_URLS["douyin"])
|
||||
platform_name = PLATFORM_NAMES.get(platform, platform)
|
||||
|
||||
# 前置检查
|
||||
if not Path(identity_dir).exists():
|
||||
print(f"❌ 身份 '{identity_name}' 不存在")
|
||||
print(f" 请先运行: python create_identity.py {identity_name}")
|
||||
sys.exit(1)
|
||||
|
||||
# 清理旧信号
|
||||
if signal_file.exists():
|
||||
signal_file.unlink()
|
||||
|
||||
from cdp_connector import CDPConnector
|
||||
|
||||
print(f"\n{'='*55}")
|
||||
print(f" 🔐 {platform_name} 身份登录: {identity_name}")
|
||||
print(f"{'='*55}")
|
||||
print(f" 持久化目录: {identity_dir}/user_data/")
|
||||
|
||||
# 通过 cdp_connector 启动持久化 Camoufox
|
||||
conn = CDPConnector(
|
||||
identity_dir=identity_dir,
|
||||
headless=False,
|
||||
window=(702, 783),
|
||||
)
|
||||
|
||||
try:
|
||||
await conn.connect()
|
||||
except Exception as e:
|
||||
print(f"❌ 浏览器连接失败: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
# 设置视口
|
||||
try:
|
||||
await conn.page.set_viewport_size({"width": 702, "height": 783})
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# 导航到平台首页
|
||||
print(f" 📍 打开 {platform_name} 首页...")
|
||||
try:
|
||||
await conn.page.goto(platform_url, timeout=30000, wait_until="domcontentloaded")
|
||||
except Exception:
|
||||
print(f" ⚠️ 页面加载超时,但浏览器已打开,请手动导航")
|
||||
print(f" 🌐 当前页面: {conn.page.url}")
|
||||
|
||||
# 创建信号文件表示浏览器已就绪
|
||||
signal_file.write_text("ready")
|
||||
|
||||
print(f"\n 🟡 请手动登录账号 [{identity_name}] ({platform_name})")
|
||||
print(f" Camoufox 窗口已打开(Windows 平板模式 702×783)")
|
||||
print(f" {'💡 打开小红书后如果出现登录弹窗,先手动关闭' if platform == 'xiaohongshu' else ''}")
|
||||
print(f" 登录完成后,在终端按 Enter 键确认")
|
||||
print(f" 或: touch {signal_file}")
|
||||
print()
|
||||
|
||||
# 轮询等待信号
|
||||
while signal_file.exists() and signal_file.read_text().strip() == "ready":
|
||||
await asyncio.sleep(2)
|
||||
|
||||
print(f"\n ✅ 登录态准备保存")
|
||||
|
||||
# 验证登录状态
|
||||
print(f" 🔍 验证登录状态...")
|
||||
await asyncio.sleep(2)
|
||||
|
||||
cookies = await conn.context.cookies()
|
||||
platform_cookies = [c for c in cookies if platform.replace("_", "").replace("hongshu", "") in c.get('domain', '')]
|
||||
|
||||
# 使用 auth_manager 的规则检测
|
||||
from auth_manager import check_login_by_cookie_sync, get_session_id
|
||||
logged_in = check_login_by_cookie_sync(cookies, platform)
|
||||
session_val = get_session_id(cookies, platform)
|
||||
|
||||
if logged_in:
|
||||
cnt = len(platform_cookies)
|
||||
print(f" 登录: ✅ 检测到 {platform_name} session cookie")
|
||||
if session_val:
|
||||
print(f" session: {session_val[:20]}...")
|
||||
else:
|
||||
print(f" 登录: ⚠️ 未检测到 {platform_name} session cookie")
|
||||
print(f" 平台 Cookie 数: {len(platform_cookies)}")
|
||||
print(f" 请确认是否已成功登录(全部 Cookie: {len(cookies)} 个)")
|
||||
# 兜底:只要平台 cookie 够多就认为登录了
|
||||
if len(platform_cookies) > 30:
|
||||
print(f" → 平台 Cookie 数量充足,判定为已登录 ✅")
|
||||
logged_in = True
|
||||
|
||||
print(f"\n{'='*55}")
|
||||
if logged_in:
|
||||
print(f" ✅ 身份 '{identity_name}' 登录完成!")
|
||||
else:
|
||||
print(f" ⚠️ 身份 '{identity_name}' 登录状态不确定")
|
||||
print(f" Cookie 已保存,可尝试运行养号查看效果")
|
||||
print(f" 浏览器保持运行,可进行养号操作")
|
||||
print(f"{'='*55}")
|
||||
|
||||
# 保持浏览器运行(不调用 conn.close())
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="身份登录 — 首次手动登录")
|
||||
parser.add_argument("identity_name", help="身份名称 (如 douyin_01)")
|
||||
parser.add_argument("--platform", "-p", default="douyin",
|
||||
choices=["douyin", "xiaohongshu"],
|
||||
help="平台 (douyin/xiaohongshu)")
|
||||
args = parser.parse_args()
|
||||
asyncio.run(login_identity(args.identity_name, args.platform))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,7 @@
|
||||
"""
|
||||
matrix_modules — 统一架构的核心模块包
|
||||
|
||||
包含:
|
||||
account/ 账号管理(身份工厂 + sms/captcha 接口)
|
||||
nurture/ 养号执行(行为模拟 + 循环引擎)
|
||||
"""
|
||||
+10
@@ -0,0 +1,10 @@
|
||||
"""
|
||||
验证码处理(接口预留)
|
||||
|
||||
与 sms/ 类似,captcha 提供验证码(图形/滑块)的抽象接口。
|
||||
当前默认实现为 ManualCaptchaHandler(手动处理)。
|
||||
"""
|
||||
|
||||
from .base import CaptchaHandler, ManualCaptchaHandler
|
||||
|
||||
__all__ = ["CaptchaHandler", "ManualCaptchaHandler"]
|
||||
@@ -0,0 +1,45 @@
|
||||
"""
|
||||
验证码处理 — 抽象接口
|
||||
|
||||
CaptchaHandler 是所有验证码处理器的基类:
|
||||
- ManualCaptchaHandler: 手动处理(默认)
|
||||
- OCRCaptchaHandler: OCR 自动识别(预留)
|
||||
- ThirdPartyCaptchaHandler: 打码平台(预留)
|
||||
"""
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
|
||||
class CaptchaHandler(ABC):
|
||||
"""验证码处理器抽象基类"""
|
||||
|
||||
@abstractmethod
|
||||
async def solve(self, page=None, timeout: int = 60) -> bool:
|
||||
"""处理验证码
|
||||
|
||||
Args:
|
||||
page: 浏览器页面对象(可选,用于截图/交互)
|
||||
timeout: 超时时间(秒)
|
||||
|
||||
Returns:
|
||||
是否成功解决
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
class ManualCaptchaHandler(CaptchaHandler):
|
||||
"""手动处理验证码(默认实现)"""
|
||||
|
||||
async def solve(self, page=None, timeout: int = 60) -> bool:
|
||||
print(f"\n🔐 需要验证码")
|
||||
print(f" 请在浏览器中手动完成验证")
|
||||
print(f" 超时: {timeout}秒")
|
||||
# 等待用户手动处理
|
||||
import asyncio
|
||||
for _ in range(timeout):
|
||||
await asyncio.sleep(1)
|
||||
# 检测页面 URL 是否验证通过(简单实现)
|
||||
if page:
|
||||
url = page.url
|
||||
if 'captcha' not in url.lower() and 'verify' not in url.lower():
|
||||
return True
|
||||
return False
|
||||
@@ -0,0 +1,167 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
douyin_login.py — 抖音账号登录接入(手机号 + SMS 验证码)
|
||||
|
||||
流程:
|
||||
1. 准备浏览器身份目录(自动创建)
|
||||
2. 启动 Camoufox,加载身份目录
|
||||
3. 导航到抖音首页
|
||||
4. 检测登录面板 → 点击"一键登录"
|
||||
5. ApiSMSHandler 自动轮询获取验证码
|
||||
6. 填入验证码 → 确认 → 验证登录态
|
||||
7. Cookie 自动持久化(Camoufox 持久化目录)
|
||||
8. 刷新 Dashboard 账号状态
|
||||
|
||||
用法:
|
||||
python douyin_login.py douyin_137 # 用默认手机号
|
||||
python douyin_login.py douyin_137 --phone 13754826808 # 指定手机号
|
||||
|
||||
版本: 1.0.0 | 更新: 2026-06-01
|
||||
"""
|
||||
import argparse, asyncio, json, os, sys, time
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent.parent.parent
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
|
||||
from cdp_connector import CDPConnector
|
||||
from browser_utils import GracefulBrowser
|
||||
from matrix_modules.account.sms_login import sms_login, has_login_panel
|
||||
from matrix_modules.account.sms import ApiSMSHandler
|
||||
|
||||
HOME = Path.home()
|
||||
|
||||
IDENTITIES_ROOT = AGENT_LOCAL / "tools" / "matrix" / "identities"
|
||||
LOG_FILE = AGENT_LOCAL / "runtime" / "login_log.jsonl"
|
||||
|
||||
|
||||
def log(msg: str):
|
||||
ts = datetime.now().strftime("%H:%M:%S")
|
||||
print(f"[{ts}] {msg}", flush=True)
|
||||
|
||||
def write_op_log(account_id: str, platform: str, status: str, detail: str = ""):
|
||||
entry = {"op": f"{platform}_login", "account": account_id, "status": status, "detail": detail, "timestamp": datetime.now(timezone.utc).isoformat()}
|
||||
LOG_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(LOG_FILE, "a") as f:
|
||||
f.write(json.dumps(entry, ensure_ascii=False) + "\n")
|
||||
|
||||
|
||||
async def douyin_login(account_id: str, phone: str = "", timeout_minutes: int = 30):
|
||||
# ── 自动更新 SMS API 配置 ──
|
||||
if phone:
|
||||
sms_yaml_path = SCRIPTS_DIR / "config" / "sms.yaml"
|
||||
if sms_yaml_path.exists():
|
||||
try:
|
||||
import yaml
|
||||
cfg = yaml.safe_load(sms_yaml_path.read_text()) or {}
|
||||
if "sms" not in cfg: cfg["sms"] = {}
|
||||
cfg["sms"]["phone"] = phone
|
||||
sms_yaml_path.write_text(yaml.dump(cfg, default_flow_style=False, allow_unicode=True))
|
||||
log(f"📝 SMS API 已切换到手机号: {phone}")
|
||||
except Exception as e:
|
||||
log(f"⚠️ 更新 sms.yaml 失败: {e}")
|
||||
else:
|
||||
log(f"⚠️ sms.yaml 不存在,跳过配置更新")
|
||||
|
||||
identity_dir = str(IDENTITIES_ROOT / account_id)
|
||||
Path(identity_dir).mkdir(parents=True, exist_ok=True)
|
||||
log(f"📁 身份目录: {identity_dir}")
|
||||
|
||||
log("🚀 启动浏览器...")
|
||||
conn = CDPConnector(identity_dir=identity_dir, headless=False,
|
||||
window=(702, 783), window_position=(0, 0))
|
||||
|
||||
# 用 GracefulBrowser 包装(优雅退出 + 前置检查)
|
||||
gb = GracefulBrowser(conn, account_id=account_id,
|
||||
timeout_minutes=timeout_minutes)
|
||||
await gb.setup(check_running=True)
|
||||
|
||||
try:
|
||||
await conn.connect()
|
||||
except Exception as e:
|
||||
write_op_log(account_id, "douyin", "failed", f"浏览器启动失败: {e}")
|
||||
log(f"❌ 浏览器连接失败: {e}")
|
||||
return False
|
||||
await conn.page.set_viewport_size({"width": 702, "height": 783})
|
||||
log("✅ 浏览器就绪")
|
||||
|
||||
log("🌐 打开抖音首页...")
|
||||
await conn.page.goto("https://www.douyin.com/", timeout=30000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(6)
|
||||
|
||||
log("📱 检测登录面板...")
|
||||
# 多轮检测: 自动检测失败后等用户手动操作
|
||||
panel_detected = False
|
||||
for attempt in range(3):
|
||||
if await has_login_panel(conn.page):
|
||||
panel_detected = True
|
||||
break
|
||||
# 也可能已经登录了
|
||||
if await conn.page.evaluate('() => !!document.querySelector("[data-e2e=user-avatar]")'):
|
||||
log("✅ 已登录,跳过")
|
||||
write_op_log(account_id, "douyin", "already_logged_in")
|
||||
return True
|
||||
if attempt == 0:
|
||||
log("⏳ 未检测到登录面板,请在浏览器上手动点击「登录」按钮...")
|
||||
log(" (等待 15 秒)")
|
||||
await asyncio.sleep(5)
|
||||
|
||||
if not panel_detected:
|
||||
log("⏳ 尝试导航到登录页面...")
|
||||
await conn.page.goto("https://www.douyin.com/", timeout=20000)
|
||||
await asyncio.sleep(4)
|
||||
for attempt in range(3):
|
||||
if await has_login_panel(conn.page):
|
||||
panel_detected = True
|
||||
break
|
||||
await asyncio.sleep(3)
|
||||
|
||||
if not panel_detected:
|
||||
log("❌ 未检测到登录面板,请手动在浏览器上操作")
|
||||
log(" 点击「登录」或「一键登录」后,脚本会自动处理验证码")
|
||||
log(" (等待 30 秒)...")
|
||||
await asyncio.sleep(30)
|
||||
# 最后检查一次登录态
|
||||
if await conn.page.evaluate('() => !!document.querySelector("[data-e2e=user-avatar]")'):
|
||||
log("✅ 已手动登录成功")
|
||||
write_op_log(account_id, "douyin", "success")
|
||||
return True
|
||||
|
||||
ok = await sms_login(conn.page, account_name=account_id, log_func=log)
|
||||
if ok:
|
||||
log("🎉 抖音登录成功!")
|
||||
write_op_log(account_id, "douyin", "success")
|
||||
try:
|
||||
from c2.profile_scraper import ProfileScraper
|
||||
scraper = ProfileScraper()
|
||||
scraper.refresh([account_id])
|
||||
log("📊 资料已采集")
|
||||
except Exception as e:
|
||||
log(f"⚠️ 资料采集失败: {e}")
|
||||
else:
|
||||
log("❌ 登录失败")
|
||||
write_op_log(account_id, "douyin", "failed", "登录流程未完成")
|
||||
|
||||
log("\n💡 浏览器保持打开,超时自动关闭")
|
||||
gb.start_auto_shutdown_timer()
|
||||
|
||||
# 等待直到超时自动关闭或被信号中断
|
||||
while not gb._shutdown_flag:
|
||||
await asyncio.sleep(10)
|
||||
|
||||
return logged_in
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="抖音账号登录接入")
|
||||
parser.add_argument("account_id", help="账号 ID(如 douyin_137)")
|
||||
parser.add_argument("--phone", "-p", default="", help="手机号(选填,默认从 accounts.yaml 读取)")
|
||||
parser.add_argument("--timeout", "-t", type=int, default=30,
|
||||
help="超时自动关闭分钟数(默认 30)")
|
||||
args = parser.parse_args()
|
||||
print(f"\n{'='*55}\n 📱 抖音登录接入: {args.account_id}\n{'='*55}\n")
|
||||
asyncio.run(douyin_login(args.account_id, args.phone, args.timeout))
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+1140
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,19 @@
|
||||
"""
|
||||
短信验证码处理
|
||||
|
||||
使用方式:
|
||||
from matrix_modules.account.sms import SMSHandler, ManualSMSHandler, ApiSMSHandler
|
||||
|
||||
# 自动模式(API 轮询)
|
||||
handler = ApiSMSHandler()
|
||||
code = await handler.wait("抖音", timeout=120)
|
||||
|
||||
# 手动模式
|
||||
handler = ManualSMSHandler()
|
||||
code = await handler.wait("抖音")
|
||||
"""
|
||||
|
||||
from .base import SMSHandler, ManualSMSHandler
|
||||
from .api import ApiSMSHandler
|
||||
|
||||
__all__ = ["SMSHandler", "ManualSMSHandler", "ApiSMSHandler"]
|
||||
@@ -0,0 +1,153 @@
|
||||
"""
|
||||
短信验证码 — API 自动获取(wx.tyhtak.com)
|
||||
|
||||
代替手动输入,自动轮询 API 获取验证码。
|
||||
配置从 ../config/sms.yaml 读取,不再硬编码。
|
||||
"""
|
||||
import asyncio
|
||||
import re
|
||||
import time
|
||||
import os
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
from urllib.request import Request, urlopen
|
||||
from urllib.error import URLError
|
||||
import json
|
||||
import yaml
|
||||
|
||||
from .base import SMSHandler
|
||||
|
||||
# 默认值(会被 config/sms.yaml 覆盖)
|
||||
API_KEY = "gtmsg2026"
|
||||
PHONE = "15370103682"
|
||||
BASE_URL = "https://wx.tyhtak.com/api/biz/msg/messages"
|
||||
POLL_INTERVAL = 3
|
||||
TIMEOUT = 120
|
||||
|
||||
|
||||
def _load_config() -> dict:
|
||||
"""从 config/sms.yaml 读取配置"""
|
||||
config_path = os.path.join(
|
||||
os.path.dirname(__file__), # account/sms/
|
||||
"..", "..", "..", "..", # → scripts/
|
||||
"config", "sms.yaml"
|
||||
)
|
||||
config_path = os.path.normpath(config_path)
|
||||
try:
|
||||
with open(config_path) as f:
|
||||
data = yaml.safe_load(f)
|
||||
return data.get("sms", {})
|
||||
except (FileNotFoundError, yaml.YAMLError):
|
||||
return {}
|
||||
|
||||
|
||||
class ApiSMSHandler(SMSHandler):
|
||||
"""通过 API 自动获取短信验证码"""
|
||||
|
||||
def __init__(self, api_key: str = None, phone: str = None):
|
||||
cfg = _load_config()
|
||||
self.api_key = api_key or cfg.get("api_key", API_KEY)
|
||||
self.phone = phone or cfg.get("phone", PHONE)
|
||||
self.base_url = cfg.get("base_url", BASE_URL)
|
||||
self.poll_interval = cfg.get("poll_interval", POLL_INTERVAL)
|
||||
self.default_timeout = cfg.get("timeout", TIMEOUT)
|
||||
self._last_id: Optional[int] = None
|
||||
self._cancel_flag = False
|
||||
|
||||
def _fetch_messages(self) -> list:
|
||||
"""调用 API 获取短信列表"""
|
||||
url = f"{self.base_url}?api_key={self.api_key}&receiver_phone={self.phone}&page=1&per_page=20"
|
||||
try:
|
||||
req = Request(url, headers={"User-Agent": "curl/7.0"})
|
||||
resp = urlopen(req, timeout=10)
|
||||
data = json.loads(resp.read().decode())
|
||||
if data.get("code") == 200:
|
||||
return data.get("data", {}).get("items", [])
|
||||
return []
|
||||
except (URLError, json.JSONDecodeError, Exception) as e:
|
||||
print(f" ⚠️ API 请求失败: {e}")
|
||||
return []
|
||||
|
||||
@staticmethod
|
||||
def _extract_code(content: str) -> Optional[str]:
|
||||
"""从短信内容中提取验证码(4位或6位)
|
||||
|
||||
匹配模式:验证码后跟4-6位数字
|
||||
"""
|
||||
# 常见格式: 【平台】验证码556314 / 验证码,346758 / 验证码:729472
|
||||
m = re.search(r'验证码[::,,]\s*(\d{4,6})', content)
|
||||
if m:
|
||||
return m.group(1)
|
||||
# 验证码直接跟数字(无分隔符)
|
||||
m = re.search(r'验证码(\d{4,6})', content)
|
||||
if m:
|
||||
return m.group(1)
|
||||
# 兜底:提取4-6位纯数字(避免提取手机号等长数字)
|
||||
nums = re.findall(r'\b(\d{4,6})\b', content)
|
||||
if nums:
|
||||
# 优先取6位,其次4位;排除明显的年份/月份
|
||||
for n in nums:
|
||||
if len(n) == 6 and n[0] != '2': # 6位且不是年份
|
||||
return n
|
||||
for n in nums:
|
||||
if len(n) in (4, 6) and n not in ('2024','2025','2026'): # 排除年份
|
||||
return n
|
||||
return None
|
||||
|
||||
async def wait(self, platform: str = "", timeout: int = 120,
|
||||
after_time: float = None) -> str:
|
||||
"""轮询 API 直到获取验证码或超时
|
||||
|
||||
Args:
|
||||
platform: 平台名称(仅日志)
|
||||
timeout: 超时秒数
|
||||
after_time: 只接受该时间戳之后到达的短信(time.time() 格式)
|
||||
|
||||
Returns:
|
||||
验证码字符串,超时返回空字符串
|
||||
"""
|
||||
print(f" 📱 等待短信验证码 ({platform})")
|
||||
print(f" 手机号: {self.phone}")
|
||||
print(f" 轮询间隔: {self.poll_interval}秒 超时: {timeout}秒")
|
||||
|
||||
if after_time:
|
||||
print(f" ⏱️ 只接受 {datetime.fromtimestamp(after_time).strftime('%H:%M:%S')} 之后的短信")
|
||||
self._cancel_flag = False
|
||||
|
||||
start = time.time()
|
||||
while not self._cancel_flag and (time.time() - start) < timeout:
|
||||
await asyncio.sleep(self.poll_interval)
|
||||
|
||||
msgs = self._fetch_messages()
|
||||
if not msgs:
|
||||
continue
|
||||
|
||||
for msg in msgs:
|
||||
# 用时间过滤:只接受 after_time 之后到达的消息
|
||||
if after_time:
|
||||
msg_time_str = msg.get("created_at") or msg.get("time") or msg.get("send_time") or ""
|
||||
if msg_time_str:
|
||||
try:
|
||||
msg_ts = datetime.fromisoformat(msg_time_str.replace("Z", "+00:00")).timestamp()
|
||||
if msg_ts <= after_time:
|
||||
continue
|
||||
except Exception:
|
||||
pass # 时间解析失败,不过滤
|
||||
|
||||
content = msg.get("content", "")
|
||||
code = self._extract_code(content)
|
||||
if code:
|
||||
print(f" ✅ 获取到验证码: {code}")
|
||||
return code
|
||||
|
||||
elapsed = int(time.time() - start)
|
||||
if elapsed % 15 < 3: # 每 15 秒打印一次状态
|
||||
print(f" ⏳ 等待中... ({elapsed}s / {timeout}s)")
|
||||
|
||||
print(f" ⏰ 超时 ({timeout}秒),未获取到验证码")
|
||||
return ""
|
||||
|
||||
async def cancel(self):
|
||||
"""取消等待"""
|
||||
self._cancel_flag = True
|
||||
print(" ⏹ API 短信等待已取消")
|
||||
@@ -0,0 +1,64 @@
|
||||
"""
|
||||
短信验证码 — 抽象接口
|
||||
|
||||
所有短信接收方式继承 SMSHandler 基类:
|
||||
- ManualSMSHandler: 手动输入(当前默认)
|
||||
- AliyunSMSHandler: 阿里云短信转发(预留)
|
||||
- TelegramSMSHandler: Telegram bot 推送(预留)
|
||||
"""
|
||||
import asyncio
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
|
||||
class SMSHandler(ABC):
|
||||
"""短信验证码处理器抽象基类"""
|
||||
|
||||
@abstractmethod
|
||||
async def wait(self, platform: str = "", timeout: int = 120) -> str:
|
||||
"""等待并返回验证码
|
||||
|
||||
Args:
|
||||
platform: 平台名称(用于提示)
|
||||
timeout: 超时时间(秒)
|
||||
|
||||
Returns:
|
||||
验证码字符串
|
||||
"""
|
||||
...
|
||||
|
||||
@abstractmethod
|
||||
async def cancel(self):
|
||||
"""取消等待"""
|
||||
...
|
||||
|
||||
|
||||
class ManualSMSHandler(SMSHandler):
|
||||
"""手动输入验证码(当前默认实现)"""
|
||||
|
||||
async def wait(self, platform: str = "", timeout: int = 120) -> str:
|
||||
print(f"\n📱 需要短信验证码 ({platform})")
|
||||
print(f" 请查看手机并将验证码输入到终端")
|
||||
print(f" 超时: {timeout}秒")
|
||||
|
||||
# 通过信号文件交互(同 login_identity 模式)
|
||||
signal_file = f"/tmp/sms_code_{id(self)}.signal"
|
||||
try:
|
||||
for _ in range(timeout // 2):
|
||||
await asyncio.sleep(2)
|
||||
if not signal_file:
|
||||
continue
|
||||
# 等待用户写入信号文件
|
||||
try:
|
||||
with open(signal_file) as f:
|
||||
code = f.read().strip()
|
||||
if code:
|
||||
return code
|
||||
except:
|
||||
pass
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
return ""
|
||||
|
||||
async def cancel(self):
|
||||
"""取消等待"""
|
||||
print(" ⏹ 短信等待已取消")
|
||||
@@ -0,0 +1,344 @@
|
||||
"""
|
||||
sms_login.py — SMS 验证码登录原子操作
|
||||
|
||||
流程:
|
||||
1. 检测登录面板 → 点"一键登录" → 触发短信
|
||||
2. ApiSMSHandler 轮询获取验证码
|
||||
3. 填入验证码 → 点确认 → 登录成功
|
||||
|
||||
支持:超时重新发送、系统级鼠标兜底
|
||||
"""
|
||||
import asyncio, time, subprocess
|
||||
from typing import Optional
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 配置
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
SMS_INPUT_SELECTOR = "input[placeholder*='验证码']"
|
||||
LOGIN_PANEL_SEL = '.login-container' # 小红书登录面板
|
||||
PHONE_INPUT_SEL = "input[placeholder*='手机']"
|
||||
ONEKEY_TEXT = '一键登录'
|
||||
CONFIRM_BTN_TEXTS = ['确认', '提交', '验证', '登录', '确定', '下一步', '立即登录']
|
||||
RESEND_TEXTS = ['重新发送', '获取验证码', '重发']
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 状态检测
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
async def has_login_panel(page) -> bool:
|
||||
"""检测登录面板是否可见"""
|
||||
for sel in [LOGIN_PANEL_SEL, '#login-panel-new', '.reds-modal-open']:
|
||||
try:
|
||||
if await page.evaluate(f'() => !!document.querySelector("{sel}")'):
|
||||
return True
|
||||
except:
|
||||
continue
|
||||
return False
|
||||
|
||||
async def has_sms_input(page) -> bool:
|
||||
return await page.evaluate(f'() => !!document.querySelector("{SMS_INPUT_SELECTOR}")')
|
||||
|
||||
async def has_resend_btn(page) -> bool:
|
||||
"""检测"重新发送"按钮是否出现(超时可点击)"""
|
||||
return await page.evaluate(f"""() => {{
|
||||
var texts = {RESEND_TEXTS};
|
||||
var all = document.querySelectorAll('button, span, div, a');
|
||||
for (var i = 0; i < all.length; i++) {{
|
||||
for (var t = 0; t < texts.length; t++) {{
|
||||
if (all[i].textContent.includes(texts[t]) && all[i].offsetParent) return true;
|
||||
}}
|
||||
}}
|
||||
return false;
|
||||
}}""")
|
||||
|
||||
async def is_logged_in(page) -> bool:
|
||||
"""检测登录成功(面板消失+用户已登录)"""
|
||||
if await has_login_panel(page):
|
||||
return False # 面板还在就不算登录
|
||||
try:
|
||||
from matrix_modules.account.login_state_machine import DETECTORS
|
||||
detector = DETECTORS.get("xiaohongshu")
|
||||
if detector:
|
||||
status = await detector.detect(page, "")
|
||||
return status == "logged_in"
|
||||
except:
|
||||
pass
|
||||
return False
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 操作函数
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
async def click_onekey_login(page):
|
||||
"""点"一键登录"按钮"""
|
||||
await page.evaluate(f"""() => {{
|
||||
var all = document.querySelectorAll('span, div, a, button');
|
||||
for (var i = 0; i < all.length; i++) {{
|
||||
if (all[i].textContent.trim() === '{ONEKEY_TEXT}') {{ all[i].click(); return true; }}
|
||||
}}
|
||||
return false;
|
||||
}}""")
|
||||
await asyncio.sleep(3)
|
||||
|
||||
async def click_resend(page):
|
||||
"""点"重新发送"按钮(超时后)"""
|
||||
clicked = await page.evaluate(f"""() => {{
|
||||
var texts = {RESEND_TEXTS};
|
||||
var all = document.querySelectorAll('button, span, div, a');
|
||||
for (var i = 0; i < all.length; i++) {{
|
||||
if (!all[i].offsetParent) continue;
|
||||
for (var t = 0; t < texts.length; t++) {{
|
||||
if (all[i].textContent.includes(texts[t])) {{ all[i].click(); return true; }}
|
||||
}}
|
||||
}}
|
||||
return false;
|
||||
}}""")
|
||||
await asyncio.sleep(1)
|
||||
return clicked
|
||||
|
||||
async def fill_code(page, code: str):
|
||||
"""填入验证码(用 JS 设置 value + dispatch input)"""
|
||||
# 用双引号避免单引号冲突
|
||||
await page.evaluate(f"""() => {{
|
||||
var inp = document.querySelector("input[placeholder*='验证码']");
|
||||
if (!inp) return;
|
||||
inp.value = '{code}';
|
||||
inp.dispatchEvent(new Event('input', {{bubbles: true}}));
|
||||
}}""")
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
async def _click_exact_login(page) -> bool:
|
||||
"""精确匹配文本为「登录」的按钮(排除协议元素)——找不到时打印调试信息"""
|
||||
result = await page.evaluate("""() => {
|
||||
var all = document.querySelectorAll('button, span, div, a');
|
||||
var debug = [];
|
||||
for (var i = 0; i < all.length; i++) {
|
||||
if (!all[i].offsetParent) continue;
|
||||
var txt = all[i].textContent.trim();
|
||||
debug.push({tag: all[i].tagName, text: txt, cls: (all[i].className||'').slice(0,30)});
|
||||
if (txt === '登录' && !all[i].textContent.includes('协议')) {
|
||||
all[i].click(); return JSON.stringify({found: true, tag: all[i].tagName, text: txt});
|
||||
}
|
||||
}
|
||||
// 没找到「登录」,返回所有可见按钮文本供调试
|
||||
return JSON.stringify({found: false, buttons: debug.filter(function(d) { return d.text.length > 0 && d.text.length < 30; }).slice(0, 40)});
|
||||
}""")
|
||||
import json as _json
|
||||
data = _json.loads(result)
|
||||
if data.get('found'):
|
||||
return True
|
||||
print(f"⚠️ click_confirm Phase1 未找到「登录」按钮")
|
||||
print(f" 可见按钮列表:")
|
||||
for b in (data.get('buttons') or []):
|
||||
print(f" [{b['tag']}] text='{b['text']}' cls='{b['cls']}'")
|
||||
return False
|
||||
|
||||
|
||||
async def _click_by_id(page, log_func=print) -> bool:
|
||||
"""Phase 0: 多层选择器匹配——优先ID,兜底class/文本"""
|
||||
result = await page.evaluate("""() => {
|
||||
// 1. 优先用唯一ID匹配(最快最准)
|
||||
var btn = document.querySelector('#douyin_login_comp_btn_id');
|
||||
if (btn && btn.offsetParent) {
|
||||
return JSON.stringify({ok: true, by: 'id', cls: btn.className.slice(0,30)});
|
||||
}
|
||||
// 2. 用录制到的class名匹配(ID被改时兜底)
|
||||
var clsBtns = document.querySelectorAll('div.r7j70rK2');
|
||||
for (var i = 0; i < clsBtns.length; i++) {
|
||||
if (clsBtns[i].offsetParent && clsBtns[i].textContent.trim() === '登录') {
|
||||
clsBtns[i].click();
|
||||
return JSON.stringify({ok: true, by: 'class', id: (clsBtns[i].id || '')});
|
||||
}
|
||||
}
|
||||
// 3. 找所有文本为"登录"的可点击元素(class也被改时兜底)
|
||||
var all = document.querySelectorAll('div, button, span, a');
|
||||
for (var i = 0; i < all.length; i++) {
|
||||
if (all[i].offsetParent && all[i].textContent.trim() === '登录') {
|
||||
all[i].click();
|
||||
return JSON.stringify({ok: true, by: 'text', tag: all[i].tagName, id: (all[i].id || ''), cls: (all[i].className || '').slice(0,30)});
|
||||
}
|
||||
}
|
||||
return JSON.stringify({ok: false});
|
||||
}""")
|
||||
import json as _j
|
||||
data = _j.loads(result)
|
||||
if data.get('ok'):
|
||||
by = data.get('by', '?')
|
||||
if by == 'id':
|
||||
pass # 正常
|
||||
elif by == 'class':
|
||||
log_func(f" ⚠️ 登录按钮ID已变更!改为用class匹配,新ID: {data.get('id','?')}")
|
||||
elif by == 'text':
|
||||
log_func(f" ⚠️ 登录按钮ID和class均变更!改为文本匹配,新标签={data.get('tag','?')} 新ID={data.get('id','?')} 新class={data.get('cls','?')}")
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
async def click_confirm(page):
|
||||
"""点确认/提交按钮——四阶段:ID匹配 → 精确匹配 → 模糊匹配 → 兜底"""
|
||||
# Phase 0: 按唯一 ID 匹配(最可靠)
|
||||
if await _click_by_id(page):
|
||||
return True
|
||||
|
||||
# Phase 1: 精确匹配「登录」
|
||||
if await _click_exact_login(page):
|
||||
return True
|
||||
|
||||
# Phase 2: 模糊匹配「登录」(兼容「同意协议并登录」类按钮)
|
||||
clicked_fuzzy = await page.evaluate("""() => {
|
||||
var all = document.querySelectorAll('button, span, div, a');
|
||||
for (var i = 0; i < all.length; i++) {
|
||||
if (!all[i].offsetParent) continue;
|
||||
if (all[i].textContent.includes('登录') && !all[i].textContent.includes('协议')) {
|
||||
all[i].click(); return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}""")
|
||||
if clicked_fuzzy:
|
||||
return True
|
||||
|
||||
# Phase 3: 原逻辑兜底(兼容小红书等平台)
|
||||
return await page.evaluate(f"""() => {{
|
||||
var texts = {CONFIRM_BTN_TEXTS};
|
||||
var all = document.querySelectorAll('button, span, div, a');
|
||||
for (var i = 0; i < all.length; i++) {{
|
||||
if (!all[i].offsetParent) continue;
|
||||
for (var t = 0; t < texts.length; t++) {{
|
||||
if (all[i].textContent.includes(texts[t])) {{ all[i].click(); return true; }}
|
||||
}}
|
||||
}}
|
||||
return false;
|
||||
}}""")
|
||||
|
||||
async def click_confirm_system(page):
|
||||
"""系统级鼠标点确认按钮(兜底)"""
|
||||
btn_info = await page.evaluate(f"""() => {{
|
||||
var texts = {CONFIRM_BTN_TEXTS};
|
||||
var all = document.querySelectorAll('button, span, div, a');
|
||||
for (var i = 0; i < all.length; i++) {{
|
||||
if (!all[i].offsetParent) continue;
|
||||
for (var t = 0; t < texts.length; t++) {{
|
||||
if (all[i].textContent.includes(texts[t])) {{
|
||||
var r = all[i].getBoundingClientRect();
|
||||
return {{x: Math.round(r.left+r.width/2), y: Math.round(r.top+r.height/2)}};
|
||||
}}
|
||||
}}
|
||||
}}
|
||||
return null;
|
||||
}}""")
|
||||
if btn_info:
|
||||
# 窗口位置 (652, 0),视口起始 = 窗口起始
|
||||
sx, sy = 652 + btn_info['x'], btn_info['y']
|
||||
subprocess.run(['osascript','-e',f'tell application "System Events" to click at {{{sx},{sy}}}'], timeout=5)
|
||||
time.sleep(0.5)
|
||||
return True
|
||||
return False
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 主流程
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
async def sms_login(page, account_name: str = '', log_func=print) -> bool:
|
||||
"""SMS 验证码登录完整流程
|
||||
|
||||
Args:
|
||||
page: Playwright page
|
||||
account_name: 账号名(用于查手机号),为空则用 sms.yaml 默认
|
||||
log_func: 日志函数
|
||||
|
||||
Returns:
|
||||
True=登录成功, False=失败
|
||||
"""
|
||||
# 从 accounts.yaml 查手机号
|
||||
phone = ''
|
||||
if account_name:
|
||||
import yaml, os
|
||||
cfg_path = os.path.expanduser('~/workbuddy-agent-os/agent-local/tools/matrix/config/accounts.yaml')
|
||||
try:
|
||||
with open(cfg_path) as f:
|
||||
data = yaml.safe_load(f)
|
||||
for a in data.get('accounts', []):
|
||||
if a.get('id') == account_name:
|
||||
phone = a.get('phone', '')
|
||||
break
|
||||
except: pass
|
||||
log_func(f'📞 手机号: {phone or "默认"}({account_name})')
|
||||
|
||||
if not await has_login_panel(page):
|
||||
log_func('❌ 未检测到登录面板')
|
||||
return False
|
||||
|
||||
# ── Step 1: 点一键登录 ──
|
||||
log_func('📱 Step 1: 点击一键登录')
|
||||
await click_onekey_login(page)
|
||||
|
||||
if not await has_sms_input(page):
|
||||
log_func('❌ 未弹出验证码输入框')
|
||||
return False
|
||||
log_func('✅ 验证码输入框已出现')
|
||||
|
||||
# ── Step 2: 轮询获取验证码(含超时重发)──
|
||||
log_func('📡 Step 2: 获取验证码')
|
||||
from matrix_modules.account.sms import ApiSMSHandler
|
||||
# 确保 phone 非空才传入
|
||||
handler = ApiSMSHandler(phone=phone) if phone and phone.strip() else ApiSMSHandler()
|
||||
if await is_logged_in(page):
|
||||
log_func('✅ 已登录,跳过')
|
||||
return True
|
||||
|
||||
if not await has_login_panel(page):
|
||||
log_func('❌ 未检测到登录面板')
|
||||
return False
|
||||
|
||||
# ── Step 1: 点一键登录 ──
|
||||
log_func('📱 Step 1: 点击一键登录')
|
||||
await click_onekey_login(page)
|
||||
|
||||
if not await has_sms_input(page):
|
||||
log_func('❌ 未弹出验证码输入框')
|
||||
return False
|
||||
log_func('✅ 验证码输入框已出现')
|
||||
|
||||
# ── Step 2: 轮询获取验证码(含超时重发)──
|
||||
log_func('📡 Step 2: 获取验证码')
|
||||
from matrix_modules.account.sms import ApiSMSHandler
|
||||
handler = ApiSMSHandler(phone=phone) if phone and phone.strip() else ApiSMSHandler()
|
||||
code = ''
|
||||
|
||||
code = ''
|
||||
for attempt in range(3): # 最多3次获取尝试
|
||||
code = await handler.wait('抖音登录', timeout=60)
|
||||
if code and len(code) in (4, 5, 6):
|
||||
log_func(f'✅ 获取到验证码: {code}')
|
||||
break
|
||||
|
||||
log_func(f'⏰ 第{attempt+1}次超时,点重新发送...')
|
||||
await click_resend(page)
|
||||
await asyncio.sleep(2)
|
||||
|
||||
if not code or len(code) not in (4, 5, 6):
|
||||
log_func('❌ 获取验证码失败')
|
||||
return False
|
||||
|
||||
# ── Step 3: 填入验证码 ──
|
||||
log_func(f'📝 Step 3: 填入验证码 {code}')
|
||||
await fill_code(page, code)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
# ── Step 4: 点确认 ──
|
||||
log_func('🔘 Step 4: 点击确认')
|
||||
ok = await click_confirm(page)
|
||||
if not ok:
|
||||
log_func('⚠️ DOM点击失败,用系统级鼠标')
|
||||
ok = await click_confirm_system(page)
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# ── Step 5: 验证登录 ──
|
||||
if await is_logged_in(page):
|
||||
log_func('🎉 登录成功!')
|
||||
return True
|
||||
else:
|
||||
log_func('⚠️ 登录可能未完成,检查页面状态')
|
||||
return False
|
||||
@@ -0,0 +1,630 @@
|
||||
"""
|
||||
xhs_login.py — 小红书 SMS 验证码登录原子操作
|
||||
|
||||
对照 sms_login.py(抖音)模式,为小红书提供完整的原子操作。
|
||||
|
||||
注意:小红书登录面板不勾选协议(无法准确定位复选框),
|
||||
改用下方「同意并登录」弹出浮窗方案。
|
||||
|
||||
2026-06-16 用户新发现(待 DOM 扫描验证):
|
||||
填手机号 → 点「继续」(SMS发) → 等6个验证码框
|
||||
→ 找「同意并登录」(可能是弹窗按钮配合「登录」按钮触发)
|
||||
→ 收验证码 → 逐个填6个框
|
||||
|
||||
原子操作列表:(对标 sms_login.py)
|
||||
has_login_panel ✅
|
||||
has_sms_inputs ✅ (6个input[type=number])
|
||||
has_resend_btn ✅ 新增
|
||||
has_agree_text ✅ 检测"同意"文字(v2备用)
|
||||
is_logged_in ✅
|
||||
fill_phone ✅ (page.fill + Tab)
|
||||
click_continue ✅ (点「继续」触发SMS)
|
||||
click_agree_and_login ✅ (v3核心: 点「同意并登录」)
|
||||
click_agreement_checkbox ✅ (v2备用)
|
||||
click_resend ✅ (点「重新获取」)
|
||||
click_login_system ✅ 新增 (系统级鼠标兜底)
|
||||
fill_6_digit_code ✅ (6个 input 逐个填)
|
||||
click_largest_login_btn ✅ (按面积找最大)
|
||||
wait_login_panel ✅
|
||||
wait_sms_inputs ✅
|
||||
get_visible_texts ✅ (调试)
|
||||
debug_snapshot_texts ✅ (调试)
|
||||
|
||||
用法:
|
||||
from xhs_login import xhs_login, has_login_panel, click_continue, ...
|
||||
"""
|
||||
import asyncio, subprocess, time
|
||||
from typing import Optional, List
|
||||
from pathlib import Path
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 配置常量
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
PHONE_PLACEHOLDER = '输入手机号'
|
||||
CONTINUE_TEXT = '继续'
|
||||
AGREE_LOGIN_TEXT = '同意并继续'
|
||||
LOGIN_BTN_TEXTS = ['登录', '登 录']
|
||||
RESEND_TEXTS = ['重新获取', '重发', '重新发送']
|
||||
LOGIN_INDICATORS = ['.user-avatar', '.reds-count']
|
||||
|
||||
# 窗口偏移(由 xiaohongshu_login.py 传入的一致性窗口定位)
|
||||
# 用于系统级鼠标点击
|
||||
WINDOW_OFFSET_X = 652
|
||||
WINDOW_OFFSET_Y = 0
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 状态检测
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
async def has_login_panel(page) -> bool:
|
||||
"""检测登录面板是否弹出(含手机号输入框)"""
|
||||
try:
|
||||
return await page.evaluate(
|
||||
'() => !!document.querySelector(\'input[placeholder*="手机"]\')'
|
||||
)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
async def has_sms_inputs(page) -> bool:
|
||||
"""检测6个数字验证码框是否出现(小红书6位输入框)"""
|
||||
try:
|
||||
info = await page.evaluate(
|
||||
"""() => {
|
||||
var a = document.querySelectorAll('input');
|
||||
var nums = [];
|
||||
for (var i=0; i<a.length; i++) {
|
||||
if (a[i].type==='number' && a[i].offsetParent) {
|
||||
nums.push(i);
|
||||
}
|
||||
}
|
||||
return nums;
|
||||
}"""
|
||||
)
|
||||
return isinstance(info, list) and len(info) >= 6
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
async def has_resend_btn(page) -> bool:
|
||||
"""检测"重新获取"按钮是否出现(验证码超时可点)"""
|
||||
return await page.evaluate(
|
||||
f"""() => {{
|
||||
var texts = {_js_list(RESEND_TEXTS)};
|
||||
var all = document.querySelectorAll('span,div,button,a');
|
||||
for (var i=0; i<all.length; i++) {{
|
||||
if (!all[i].offsetParent) continue;
|
||||
var t = all[i].textContent.trim();
|
||||
for (var j=0; j<texts.length; j++) {{
|
||||
if (t.includes(texts[j])) return true;
|
||||
}}
|
||||
}}
|
||||
return false;
|
||||
}}"""
|
||||
)
|
||||
|
||||
|
||||
async def has_agree_text(page) -> bool:
|
||||
"""检测页面上是否有包含"同意"的文字(v2备用)"""
|
||||
try:
|
||||
return await page.evaluate(
|
||||
"""() => {
|
||||
var s = document.querySelectorAll('span,div,label');
|
||||
for (var i=0; i<s.length; i++) {
|
||||
var t = (s[i].textContent||'').trim();
|
||||
if (t.includes('同意') && !t.includes('扫码') && s[i].offsetParent) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}"""
|
||||
)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
async def is_logged_in(page) -> bool:
|
||||
"""检测小红书是否已登录(严格模式)"""
|
||||
try:
|
||||
# 登录面板可见 → 肯定没登录
|
||||
if await has_login_panel(page):
|
||||
return False
|
||||
return await page.evaluate("""() => {
|
||||
// 查用户头像 — 只在顶栏区域(不是 feed 里的创作者头像)
|
||||
const topAvatars = document.querySelectorAll(
|
||||
'.reds-count, [class*="reds-user"], ' +
|
||||
'[class*="user-sidebar"], .side-bar [class*="avatar"], ' +
|
||||
'[class*="user-info"] [class*="avatar"]'
|
||||
);
|
||||
if (topAvatars.length > 0) return true;
|
||||
|
||||
// 兜底:找整体登录指示元素
|
||||
const indicators = document.querySelectorAll(
|
||||
'[class*="user-center"], [class*="user-profile"], ' +
|
||||
'[class*="my-info"], [data-testid*="user"]'
|
||||
);
|
||||
return indicators.length > 0;
|
||||
}""")
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 操作函数
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
async def fill_phone(page, phone: str):
|
||||
"""填入手机号 — 必须用 page.fill + Tab(React 框架不认 JS set value)"""
|
||||
try:
|
||||
inp = page.locator('input[placeholder*="手机"]')
|
||||
await inp.fill(phone)
|
||||
await page.keyboard.press("Tab")
|
||||
except Exception:
|
||||
# 兜底:按字符逐个输入
|
||||
await page.type('input[placeholder*="手机"]', phone, delay=30)
|
||||
await page.keyboard.press("Tab")
|
||||
|
||||
|
||||
async def click_continue(page) -> bool:
|
||||
"""点「继续」按钮(触发SMS发送)"""
|
||||
# 先检查是否有 oauth-tip(切换登录方式),有则先点它
|
||||
try:
|
||||
oauth = page.locator('div.oauth-tip')
|
||||
if await oauth.count() > 0 and await oauth.first.is_visible():
|
||||
await oauth.first.click()
|
||||
await asyncio.sleep(1)
|
||||
except:
|
||||
pass
|
||||
|
||||
clicked = await page.evaluate(
|
||||
f"""() => {{
|
||||
var all = document.querySelectorAll('span,div,button');
|
||||
for (var i=0; i<all.length; i++) {{
|
||||
if (all[i].textContent.trim() === '{CONTINUE_TEXT}' && all[i].offsetParent) {{
|
||||
all[i].click();
|
||||
return 'OK';
|
||||
}}
|
||||
}}
|
||||
return 'NOT_FOUND';
|
||||
}}"""
|
||||
)
|
||||
return clicked == 'OK'
|
||||
|
||||
|
||||
async def click_agree_and_login(page) -> bool:
|
||||
"""
|
||||
点「同意并继续」按钮(先点登录后弹出的浮层按钮)
|
||||
先试 .foot-btn CSS 选择器,再搜文本
|
||||
"""
|
||||
# 先试 CSS 选择器
|
||||
try:
|
||||
fb = page.locator('div.foot-btn')
|
||||
cnt = await fb.count()
|
||||
if cnt > 0 and await fb.first.is_visible():
|
||||
await fb.first.click()
|
||||
return True
|
||||
except:
|
||||
pass
|
||||
|
||||
# 再搜文本(找 z-index 最高的可见元素)
|
||||
for _ in range(6):
|
||||
clicked = await page.evaluate(
|
||||
f"""() => {{
|
||||
var all = document.querySelectorAll('span,div,button');
|
||||
var found = null, maxZ = 0;
|
||||
for (var i=0; i<all.length; i++) {{
|
||||
if (!all[i].offsetParent) continue;
|
||||
var t = all[i].textContent.trim();
|
||||
if (t.includes('同意并继续') || t.includes('同意并登录')) {{
|
||||
var z = parseInt(window.getComputedStyle(all[i]).zIndex) || 0;
|
||||
if (z > maxZ) {{ maxZ = z; found = all[i]; }}
|
||||
}}
|
||||
}}
|
||||
if (found) {{ found.click(); return true; }}
|
||||
return false;
|
||||
}}"""
|
||||
)
|
||||
if clicked:
|
||||
return True
|
||||
await asyncio.sleep(1)
|
||||
return False
|
||||
|
||||
|
||||
async def click_agreement_checkbox(page) -> bool:
|
||||
"""
|
||||
(备用 v2) 鼠标点击"同意"文字左侧20px位置勾协议
|
||||
|
||||
注意:小红书复选框是自定义元素,非原生 input[type=checkbox],
|
||||
left-20px 定位极不可靠。仅作 v3 方案的兜底。
|
||||
"""
|
||||
pos = await _get_agreement_element(page)
|
||||
if not pos:
|
||||
return False
|
||||
click_x = max(0, pos["x"] - 20)
|
||||
await page.mouse.click(click_x, pos["y"])
|
||||
return True
|
||||
|
||||
|
||||
async def click_resend(page) -> bool:
|
||||
"""点「重新获取」按钮(验证码超时后重发)"""
|
||||
clicked = await page.evaluate(
|
||||
f"""() => {{
|
||||
var texts = {_js_list(RESEND_TEXTS)};
|
||||
var all = document.querySelectorAll('span,div,button,a');
|
||||
for (var i=0; i<all.length; i++) {{
|
||||
if (!all[i].offsetParent) continue;
|
||||
var t = all[i].textContent.trim();
|
||||
for (var j=0; j<texts.length; j++) {{
|
||||
if (t.includes(texts[j])) {{
|
||||
all[i].click();
|
||||
return true;
|
||||
}}
|
||||
}}
|
||||
}}
|
||||
return false;
|
||||
}}"""
|
||||
)
|
||||
await asyncio.sleep(1)
|
||||
return clicked
|
||||
|
||||
|
||||
async def fill_6_digit_code(page, code: str, log_func=print) -> bool:
|
||||
"""
|
||||
逐个填入6位验证码
|
||||
|
||||
策略:先一次 evaluate 捕获所有 input[type=number] 的 DOM 索引,
|
||||
再按索引逐个填入,防止 DOM 在迭代中变动。
|
||||
"""
|
||||
indices = await page.evaluate(
|
||||
"""() => {
|
||||
var a = document.querySelectorAll('input');
|
||||
var r = [];
|
||||
for (var j=0; j<a.length; j++) {
|
||||
if (a[j].type==='number' && a[j].offsetParent) {
|
||||
r.push(j);
|
||||
if (r.length >= 6) break;
|
||||
}
|
||||
}
|
||||
return r;
|
||||
}"""
|
||||
)
|
||||
log_func(f" 验证码框 DOM 索引: {indices}")
|
||||
|
||||
for idx, digit in enumerate(code[:6]):
|
||||
if idx >= len(indices):
|
||||
break
|
||||
ok = await page.evaluate(
|
||||
f"""() => {{
|
||||
var all = document.querySelectorAll('input');
|
||||
var inp = all[{indices[idx]}];
|
||||
if (!inp) return 0;
|
||||
inp.focus();
|
||||
// 模仿手动输入
|
||||
var nativeInputValueSetter = Object.getOwnPropertyDescriptor(
|
||||
window.HTMLInputElement.prototype, 'value'
|
||||
).set;
|
||||
nativeInputValueSetter.call(inp, '{digit}');
|
||||
inp.dispatchEvent(new Event('input', {{bubbles: true}}));
|
||||
inp.dispatchEvent(new Event('change', {{bubbles: true}}));
|
||||
return 1;
|
||||
}}"""
|
||||
)
|
||||
log_func(f" 第{idx+1}位: {digit} -> {'OK' if ok else 'FAIL'}")
|
||||
await asyncio.sleep(0.2)
|
||||
|
||||
return True
|
||||
|
||||
|
||||
async def click_largest_login_btn(page, log_func=print) -> bool:
|
||||
"""按面积找最大的「登录」按钮(排除导航栏)"""
|
||||
btns = await _get_login_buttons(page)
|
||||
if not btns:
|
||||
log_func(" ⚠️ 无登录按钮")
|
||||
return False
|
||||
target = btns[0]
|
||||
await page.mouse.click(target["x"], target["y"])
|
||||
log_func(f" ✅ 点登录 ({target['x']},{target['y']}) {target['w']}x{target['h']}")
|
||||
return True
|
||||
|
||||
|
||||
async def click_login_system(page, log_func=print) -> bool:
|
||||
"""
|
||||
系统级鼠标点登录按钮(DOM 点击失败时兜底)
|
||||
使用 osascript 在系统层模拟点击,绕过网页 JS 限制
|
||||
"""
|
||||
btns = await _get_login_buttons(page)
|
||||
if not btns:
|
||||
return False
|
||||
target = btns[0]
|
||||
sx = WINDOW_OFFSET_X + target["x"]
|
||||
sy = WINDOW_OFFSET_Y + target["y"]
|
||||
log_func(f" 🔧 系统级鼠标点击 ({sx},{sy})")
|
||||
subprocess.run(
|
||||
['osascript', '-e',
|
||||
f'tell application "System Events" to click at {{{sx},{sy}}}'],
|
||||
timeout=5
|
||||
)
|
||||
time.sleep(0.5)
|
||||
return True
|
||||
|
||||
|
||||
async def wait_login_panel(page, timeout_sec: int = 30) -> bool:
|
||||
"""等待登录面板弹出(含手机号输入框)"""
|
||||
for _ in range(timeout_sec):
|
||||
if await has_login_panel(page):
|
||||
return True
|
||||
await asyncio.sleep(1)
|
||||
return False
|
||||
|
||||
|
||||
async def wait_sms_inputs(page, timeout_sec: int = 40) -> bool:
|
||||
"""等待6个验证码输入框出现"""
|
||||
for _ in range(timeout_sec):
|
||||
if await has_sms_inputs(page):
|
||||
return True
|
||||
await asyncio.sleep(1)
|
||||
return False
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 调试辅助
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
async def get_visible_texts(page, limit: int = 15) -> list:
|
||||
"""获取页面上所有可见文字(调试用)"""
|
||||
try:
|
||||
return await page.evaluate(
|
||||
f"""() => {{
|
||||
var a = document.querySelectorAll('span,div,button');
|
||||
var r = [];
|
||||
for (var i=0; i<a.length; i++) {{
|
||||
if (!a[i].offsetParent) continue;
|
||||
var t = a[i].textContent.trim();
|
||||
if (t) r.push(t);
|
||||
if (r.length >= {limit}) break;
|
||||
}}
|
||||
return r;
|
||||
}}"""
|
||||
)
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
|
||||
async def debug_snapshot_texts(page, log_func=print):
|
||||
"""快照当前页面: URL + 可见文字(调试用)"""
|
||||
try:
|
||||
url = await page.evaluate('window.location.href')
|
||||
texts = await get_visible_texts(page)
|
||||
log_func(f" URL: {url}")
|
||||
log_func(f" 可见文字: {texts}")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 私有辅助
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _js_list(items: list) -> str:
|
||||
"""生成 JS 数组字面量字符串,正确处理引号"""
|
||||
quoted = [f"'{x}'" for x in items]
|
||||
return "[" + ",".join(quoted) + "]"
|
||||
|
||||
|
||||
async def _get_agreement_element(page) -> Optional[dict]:
|
||||
"""找最小高度(最内层)包含"同意"的可见元素"""
|
||||
try:
|
||||
result = await page.evaluate(
|
||||
"""() => {
|
||||
var s = document.querySelectorAll('span,div,label');
|
||||
var best = null;
|
||||
for (var i=0; i<s.length; i++) {
|
||||
var t = (s[i].textContent||'').trim();
|
||||
if (t.includes('同意') && !t.includes('扫码') && s[i].offsetParent) {
|
||||
var b = s[i].getBoundingClientRect();
|
||||
if (!best || b.height < best.h) {
|
||||
best = {
|
||||
x: Math.round(b.left),
|
||||
y: Math.round(b.top + b.height/2),
|
||||
h: b.height,
|
||||
text: t.slice(0,20)
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
return best ? JSON.stringify(best) : null;
|
||||
}"""
|
||||
)
|
||||
if result:
|
||||
import json as _j
|
||||
return _j.loads(result)
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
async def _get_login_buttons(page) -> list:
|
||||
"""获取所有可见「登录」按钮信息(按面积降序)"""
|
||||
try:
|
||||
result = await page.evaluate(
|
||||
f"""() => {{
|
||||
var all = document.querySelectorAll('span,div,button');
|
||||
var r = [];
|
||||
for (var i=0; i<all.length; i++) {{
|
||||
if (!all[i].offsetParent) continue;
|
||||
var t = all[i].textContent.trim();
|
||||
for (var j=0; j<{_js_list(LOGIN_BTN_TEXTS)}.length; j++) {{
|
||||
if (t === {_js_list(LOGIN_BTN_TEXTS)}[j]) {{
|
||||
var b = all[i].getBoundingClientRect();
|
||||
r.push({{
|
||||
x: Math.round(b.left + b.width/2),
|
||||
y: Math.round(b.top + b.height/2),
|
||||
w: Math.round(b.width),
|
||||
h: Math.round(b.height)
|
||||
}});
|
||||
}}
|
||||
}}
|
||||
}}
|
||||
r.sort(function(a,b){{return (b.w*b.h) - (a.w*a.h);}});
|
||||
return JSON.stringify(r);
|
||||
}}"""
|
||||
)
|
||||
import json as _j
|
||||
return _j.loads(result)
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 主流程
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
async def _lookup_phone(account_name: str) -> str:
|
||||
"""从 accounts.override.yaml 查找手机号"""
|
||||
import yaml, os
|
||||
home = os.path.expanduser('~')
|
||||
base = home + '/workbuddy-agent-os/agent-local/tools/matrix/config'
|
||||
for name in ['accounts.override.yaml', 'accounts.yaml']:
|
||||
path = base + '/' + name
|
||||
if not os.path.exists(path):
|
||||
continue
|
||||
try:
|
||||
with open(path) as f:
|
||||
data = yaml.safe_load(f) or {}
|
||||
for a in data.get('accounts', []):
|
||||
if a.get('id') == account_name:
|
||||
phone = a.get('phone', '')
|
||||
if phone:
|
||||
return phone
|
||||
except Exception:
|
||||
pass
|
||||
return ''
|
||||
|
||||
|
||||
async def xhs_login(page, phone: str = '', account_name: str = '',
|
||||
log_func=print) -> bool:
|
||||
"""小红书 SMS 验证码登录完整流程
|
||||
|
||||
Args:
|
||||
page: Playwright page 对象
|
||||
phone: 手机号(可选,空则从 accounts.yaml 查)
|
||||
account_name: 账号 ID(用于日志/查手机号)
|
||||
log_func: 日志输出函数
|
||||
|
||||
Returns:
|
||||
True=登录成功, False=失败
|
||||
"""
|
||||
# ── 获取手机号 ──
|
||||
if not phone and account_name:
|
||||
phone = await _lookup_phone(account_name)
|
||||
log_func(f'📞 手机号: {phone or "未指定"} ({account_name})')
|
||||
|
||||
# ── 检测已登录 ──
|
||||
if await is_logged_in(page):
|
||||
log_func('✅ 已登录,跳过')
|
||||
return True
|
||||
|
||||
# ── Step 1: 等登录面板 ──
|
||||
if not await wait_login_panel(page):
|
||||
log_func('❌ 登录面板未弹出')
|
||||
return False
|
||||
log_func('✅ 登录面板已弹出')
|
||||
|
||||
# ── Step 2: 填手机号 ──
|
||||
if phone:
|
||||
log_func(f'📱 Step 2: 填手机号 {phone}')
|
||||
await fill_phone(page, phone)
|
||||
await asyncio.sleep(1.5)
|
||||
|
||||
# ── 创建 SMS handler + 记录当前时间(点继续之前,用于时间过滤)──
|
||||
import time
|
||||
from matrix_modules.account.sms.api import ApiSMSHandler
|
||||
handler = ApiSMSHandler(phone=phone) if phone and phone.strip() else ApiSMSHandler()
|
||||
handler.poll_interval = 5
|
||||
sms_after_time = time.time()
|
||||
log_func(f' 📡 记录时间戳 {time.strftime("%H:%M:%S")},只接受之后的验证码')
|
||||
|
||||
# ── Step 3: 点继续(触发 SMS)──
|
||||
log_func('👉 Step 3: 点继续')
|
||||
if not await click_continue(page):
|
||||
log_func('⚠️ 未找到继续按钮,重试...')
|
||||
await asyncio.sleep(2)
|
||||
await click_continue(page)
|
||||
await asyncio.sleep(3)
|
||||
|
||||
# 调试快照
|
||||
await debug_snapshot_texts(page, log_func)
|
||||
|
||||
# ── Step 4: 等6个验证码框 ──
|
||||
log_func('⏳ Step 4: 等6个验证码框...')
|
||||
if not await wait_sms_inputs(page):
|
||||
log_func('❌ 验证码框未出现')
|
||||
return False
|
||||
log_func('✅ 验证码框已出现')
|
||||
|
||||
code = ''
|
||||
for attempt in range(2): # 最多2次(第3次封禁3分钟)
|
||||
code = await handler.wait(f'小红书 {account_name}', timeout=180, after_time=sms_after_time)
|
||||
if code and len(code) in (4, 5, 6):
|
||||
log_func(f'✅ 验证码: {code}')
|
||||
break
|
||||
|
||||
log_func(f'⏰ 第{attempt+1}次超时,等待180秒后点重新获取...')
|
||||
# 等180秒倒计时 → 点重新获取
|
||||
for _ in range(180):
|
||||
if await click_resend(page):
|
||||
log_func(' ✅ 已点"重新获取"')
|
||||
break
|
||||
await asyncio.sleep(1)
|
||||
else:
|
||||
log_func(' ⚠️ 未找到"重新获取"按钮')
|
||||
else:
|
||||
log_func('❌ 验证码获取失败')
|
||||
return False
|
||||
|
||||
if not code or len(code) not in (4, 5, 6):
|
||||
return False
|
||||
|
||||
# ── Step 6: 填6位验证码 ──
|
||||
log_func(f'📝 Step 6: 填验证码 {code}')
|
||||
await fill_6_digit_code(page, code[:6], log_func)
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
# ── Step 7: 点「登录」按钮(触发「同意并登录」浮层)──
|
||||
# 先点登录,会弹出「同意并登录」浮动层,然后再点它
|
||||
log_func('🔘 Step 7: 点击登录')
|
||||
ok = await click_largest_login_btn(page, log_func)
|
||||
if not ok:
|
||||
log_func(' ⚠️ DOM 点击失败,尝试系统级鼠标点击')
|
||||
ok = await click_login_system(page, log_func)
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# ── Step 8: 点「同意并继续」(登录后弹出的浮层)──
|
||||
# 等浮层出现(最多5秒),点击后即完成登录
|
||||
log_func('📄 Step 8: 同意并继续')
|
||||
await asyncio.sleep(0.5)
|
||||
v3_ok = await click_agree_and_login(page)
|
||||
if v3_ok:
|
||||
log_func(' ✅ 已点「同意并登录」')
|
||||
else:
|
||||
log_func(' ⚠️ 未找到"同意并继续"按钮,尝试继续...')
|
||||
|
||||
await asyncio.sleep(3)
|
||||
|
||||
# ── Step 9: 验证登录 ──
|
||||
if await is_logged_in(page):
|
||||
log_func('🎉 登录成功!')
|
||||
return True
|
||||
|
||||
# 再试一次(可能页面跳转慢)
|
||||
log_func('⏳ 等待登录完成...')
|
||||
for _ in range(5):
|
||||
await asyncio.sleep(2)
|
||||
if await is_logged_in(page):
|
||||
log_func('🎉 登录成功!')
|
||||
return True
|
||||
else:
|
||||
log_func('⚠️ 登录可能未完成,需检查页面状态')
|
||||
return False
|
||||
+246
@@ -0,0 +1,246 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
xiaohongshu_login.py — 小红书账号登录接入(手机号 + SMS 验证码)
|
||||
|
||||
流程:
|
||||
1. 准备浏览器身份目录
|
||||
2. 优雅退出检查 + 前置浏览器检测 (browser_utils)
|
||||
3. 启动 Camoufox,加载身份目录
|
||||
4. 导航到小红书首页
|
||||
5. 检测登录面板 → 触发短信验证码登录
|
||||
6. ApiSMSHandler 自动轮询获取验证码
|
||||
7. 填入验证码 → 确认 → 验证登录态
|
||||
8. Cookie 自动持久化(Camoufox 持久化目录)
|
||||
9. 优雅退出/超时自动关闭
|
||||
|
||||
用法:
|
||||
python xiaohongshu_login.py xhs_01 # 用默认手机号
|
||||
python xiaohongshu_login.py xhs_01 --phone 13754826808 # 指定手机号
|
||||
python xiaohongshu_login.py xhs_01 --timeout 60 # 60分钟超时自动关闭
|
||||
|
||||
版本: 1.0.0 | 更新: 2026-06-16
|
||||
"""
|
||||
import argparse, asyncio, json, os, sys, time
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent.parent.parent
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
|
||||
from cdp_connector import CDPConnector
|
||||
from browser_utils import GracefulBrowser
|
||||
|
||||
HOME = Path.home()
|
||||
from matrix_mgmt import AGENT_LOCAL, AGENT_SYNC
|
||||
|
||||
IDENTITIES_ROOT = AGENT_LOCAL / "tools" / "matrix" / "identities"
|
||||
LOG_FILE = AGENT_LOCAL / "runtime" / "login_log.jsonl"
|
||||
|
||||
|
||||
def log(msg: str):
|
||||
ts = datetime.now().strftime("%H:%M:%S")
|
||||
print(f"[{ts}] {msg}", flush=True)
|
||||
|
||||
def write_op_log(account_id: str, platform: str, status: str, detail: str = ""):
|
||||
entry = {"op": f"{platform}_login", "account": account_id, "status": status,
|
||||
"detail": detail, "timestamp": datetime.now(timezone.utc).isoformat()}
|
||||
LOG_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(LOG_FILE, "a") as f:
|
||||
f.write(json.dumps(entry, ensure_ascii=False) + "\n")
|
||||
|
||||
|
||||
async def xiaohongshu_login(account_id: str, phone: str = "", timeout_minutes: int = 30, force: bool = False):
|
||||
"""小红书全自动登录"""
|
||||
# ── 确认账号平台是小程序 ──
|
||||
# 从矩阵管理获取账号信息
|
||||
platform = "xiaohongshu"
|
||||
acct_identity = account_id
|
||||
try:
|
||||
from matrix_mgmt import MatrixManager
|
||||
mgr = MatrixManager()
|
||||
for a in mgr.list_accounts():
|
||||
if a["id"] == account_id:
|
||||
if not phone:
|
||||
phone = a.get("phone", "")
|
||||
id_dir = a.get("identity_dir", "") or a.get("identity_hint", "")
|
||||
if id_dir:
|
||||
acct_identity = id_dir.replace("identities/", "")
|
||||
break
|
||||
except Exception:
|
||||
pass
|
||||
identity_dir = str(IDENTITIES_ROOT / acct_identity)
|
||||
Path(identity_dir).mkdir(parents=True, exist_ok=True)
|
||||
log(f"📁 身份目录: {identity_dir}")
|
||||
|
||||
# ── 启动浏览器(带优雅退出和前置检查)──
|
||||
log("🚀 启动浏览器...")
|
||||
conn = CDPConnector(identity_dir=identity_dir, headless=False,
|
||||
window=(702, 783), window_position=(652, 0))
|
||||
|
||||
# 用 GracefulBrowser 包装
|
||||
gb = GracefulBrowser(conn, account_id=account_id,
|
||||
timeout_minutes=timeout_minutes)
|
||||
await gb.setup(check_running=True)
|
||||
|
||||
try:
|
||||
await conn.connect()
|
||||
except Exception as e:
|
||||
write_op_log(account_id, platform, "failed", f"浏览器启动失败: {e}")
|
||||
log(f"❌ 浏览器连接失败: {e}")
|
||||
return False
|
||||
|
||||
await conn.page.set_viewport_size({"width": 702, "height": 783})
|
||||
log("✅ 浏览器就绪")
|
||||
|
||||
# ── 导航到小红书首页 ──
|
||||
log("🌐 打开小红书首页...")
|
||||
await conn.page.goto("https://www.xiaohongshu.com/explore",
|
||||
timeout=30000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(6)
|
||||
|
||||
# ── 检测登录状态(主方案:Cookie 检测,DOM 仅辅助)──
|
||||
log("🔍 检测登录状态...")
|
||||
logged_in = False
|
||||
if force:
|
||||
log(" 🔄 --force 模式:跳过已登录检测,强制重新登录")
|
||||
else:
|
||||
try:
|
||||
# Cookie 检测(最可靠)
|
||||
cookies = await conn.page.context.cookies()
|
||||
from auth_manager import check_login_by_cookie_sync
|
||||
logged_in = check_login_by_cookie_sync(cookies, "xiaohongshu")
|
||||
if logged_in:
|
||||
log(f" ✅ Cookie 检测: 已登录 (a1/web_session)")
|
||||
else:
|
||||
log(f" ⚠️ Cookie 检测: 未检测到小红书 session cookie")
|
||||
except Exception as e:
|
||||
log(f" ⚠️ Cookie 检测异常: {e}")
|
||||
|
||||
if logged_in:
|
||||
log("✅ 已登录,跳过登录流程")
|
||||
write_op_log(account_id, platform, "already_logged_in")
|
||||
log("\n💡 浏览器保持打开,超时自动关闭")
|
||||
gb.start_auto_shutdown_timer()
|
||||
return True
|
||||
|
||||
# ── 检测登录面板(小红书无cookie时自动弹出登录弹窗)──
|
||||
log("📱 检测登录面板...")
|
||||
panel_detected = False
|
||||
|
||||
for attempt in range(8):
|
||||
panel_found = await conn.page.evaluate("""() => {
|
||||
var sels = ['.login-container','.reds-modal-open','.login-modal',
|
||||
'input[placeholder*=\"手机\"]','.phone-login'];
|
||||
for (var s of sels) { var el = document.querySelector(s);
|
||||
if (el && el.offsetParent) return s; }
|
||||
return null;
|
||||
}""")
|
||||
|
||||
if panel_found:
|
||||
panel_detected = True
|
||||
log(f" ✅ 检测到登录面板: {panel_found}")
|
||||
break
|
||||
|
||||
# 方式2: 尝试点击页面上的"登录"按钮
|
||||
if attempt == 0:
|
||||
log(" 🔍 尝试点击登录按钮...")
|
||||
login_clicked = await conn.page.evaluate("""() => {
|
||||
var all = document.querySelectorAll('span, div, a, button');
|
||||
for (var i = 0; i < all.length; i++) {
|
||||
var t = all[i].textContent.trim();
|
||||
if ((t === '登录' || t === '登录/注册') && all[i].offsetParent) {
|
||||
all[i].click();
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}""")
|
||||
|
||||
if login_clicked:
|
||||
log(" ✅ 已点击登录按钮,等待面板出现...")
|
||||
await asyncio.sleep(3)
|
||||
else:
|
||||
log(" ⏳ 等待登录面板...")
|
||||
await asyncio.sleep(3)
|
||||
|
||||
if not panel_detected:
|
||||
log("❌ 未检测到登录面板")
|
||||
log(" ⏳ 等待 60 秒,请手动在浏览器上点击「登录」...")
|
||||
log(" (脚本检测到登录面板后会自动处理验证码)")
|
||||
|
||||
for _ in range(12):
|
||||
await asyncio.sleep(5)
|
||||
if await conn.page.evaluate("""() => {
|
||||
var s = document.querySelector('input[placeholder*=\"手机\"], input[type=\"tel\"]');
|
||||
return s && s.offsetParent ? true : false;
|
||||
}"""):
|
||||
panel_detected = True
|
||||
log(" ✅ 检测到登录面板(手动触发)")
|
||||
break
|
||||
|
||||
if not panel_detected:
|
||||
# 最后检查: 是否已经登录了
|
||||
final_check = await conn.page.evaluate(
|
||||
'() => !!document.querySelector(".user-avatar, .reds-count")'
|
||||
)
|
||||
if final_check:
|
||||
log("✅ 已登录成功")
|
||||
write_op_log(account_id, platform, "success")
|
||||
return True
|
||||
log("❌ 登录面板未出现,请手动完成登录")
|
||||
return False
|
||||
|
||||
# ── 执行 SMS 验证码登录(委托 xhs_login.py 完整流程)──
|
||||
log("📱 执行短信验证码登录...")
|
||||
from matrix_modules.account.xhs_login import xhs_login, wait_login_panel
|
||||
|
||||
# 确保登录面板已弹出
|
||||
await asyncio.sleep(1)
|
||||
if not await wait_login_panel(conn.page):
|
||||
log(" ⚠️ 登录面板未弹出,尝试点击登录按钮...")
|
||||
await conn.page.evaluate("""() => {
|
||||
var all = document.querySelectorAll('span, div, a, button');
|
||||
for (var i = 0; i < all.length; i++) {
|
||||
if (all[i].textContent.trim() === '登录' && all[i].offsetParent) {
|
||||
all[i].click(); return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}""")
|
||||
await asyncio.sleep(3)
|
||||
|
||||
ok = await xhs_login(conn.page, phone=phone, account_name=account_id, log_func=log)
|
||||
|
||||
if ok:
|
||||
log("🎉 小红书登录成功!")
|
||||
write_op_log(account_id, platform, "success")
|
||||
else:
|
||||
log("⚠️ 登录可能未完成,检查页面状态")
|
||||
write_op_log(account_id, platform, "failed", "登录后验证未通过")
|
||||
|
||||
# ── 保持浏览器打开,启动超时自动关闭 ──
|
||||
log("\n💡 浏览器保持打开,超时自动关闭")
|
||||
gb.start_auto_shutdown_timer()
|
||||
|
||||
# 等待直到超时自动关闭或被信号中断
|
||||
while not gb._shutdown_flag:
|
||||
await asyncio.sleep(10)
|
||||
|
||||
return logged_in
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="小红书账号登录接入")
|
||||
parser.add_argument("account_id", help="账号 ID(如 xhs_01)")
|
||||
parser.add_argument("--phone", "-p", default="", help="手机号(选填)")
|
||||
parser.add_argument("--timeout", "-t", type=int, default=30,
|
||||
help="超时自动关闭分钟数(默认 30)")
|
||||
parser.add_argument("--force", "-f", action="store_true",
|
||||
help="强制重新登录(跳过已登录检测)")
|
||||
args = parser.parse_args()
|
||||
print(f"\n{'='*55}\n {'📕 小红书强制登录' if args.force else '📕 小红书登录接入'}: {args.account_id}\n{'='*55}\n")
|
||||
asyncio.run(xiaohongshu_login(args.account_id, args.phone, args.timeout, force=args.force))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,295 @@
|
||||
"""
|
||||
🎯 AI 评论生成器 — 基于本地 oMLX 模型
|
||||
|
||||
通过本地 oMLX API (localhost:8000) 生成小红书风格 AI 评论。
|
||||
作为静态语料库的增强替代,支持上下文感知(基于笔记标题/内容)。
|
||||
|
||||
用法:
|
||||
from matrix_modules.comment.ai_generator import AICommentGenerator
|
||||
|
||||
gen = AICommentGenerator()
|
||||
comment = gen.generate(topic="美食", style="种草")
|
||||
comment = gen.generate_with_context(note_title="周末去苏州旅游攻略")
|
||||
comment = gen.generate_from_note(page=page) # 从当前页面提取内容
|
||||
"""
|
||||
|
||||
import json
|
||||
import random
|
||||
import time
|
||||
import urllib.request
|
||||
import re
|
||||
from typing import Optional, List
|
||||
|
||||
# 默认 oMLX API 地址
|
||||
OLLAMA_BASE = "http://localhost:8000/v1"
|
||||
API_KEY = "5omlx"
|
||||
|
||||
# 系统提示词 — 小红书风格评论生成
|
||||
SYSTEM_PROMPT = """你是一个小红书重度用户,正在给笔记写评论。
|
||||
|
||||
你的评论风格要求:
|
||||
1. 亲切自然,像真人用户的真实感受(不是营销号)
|
||||
2. 长度10-25字,简洁但有信息量
|
||||
3. 可以带少量 emoji(但要自然,不要堆砌)
|
||||
4. 语气因人而异:种草类热情、经验类实用、旅行美景类温暖
|
||||
5. 避免重复、套路化的表达
|
||||
|
||||
你生成的每条评论必须:
|
||||
- 与笔记主题相关(基于给定的主题/标题/内容)
|
||||
- 语气自然真实
|
||||
- 不营销、不推销、不带链接
|
||||
- 不涉及政治、色情、暴力
|
||||
|
||||
只输出评论文本本身,不要加引号、前缀或其他格式。"""
|
||||
|
||||
|
||||
def _call_omlx(messages: list, max_tokens: int = 64,
|
||||
temperature: float = 0.9, timeout: int = 10) -> Optional[str]:
|
||||
"""
|
||||
调用 oMLX 本地 API 生成文本。
|
||||
|
||||
Args:
|
||||
messages: OpenAI 格式的消息列表
|
||||
max_tokens: 最大生成 token 数
|
||||
temperature: 生成温度 (0-1)
|
||||
timeout: 超时秒数
|
||||
|
||||
Returns:
|
||||
生成的文本,失败返回 None
|
||||
"""
|
||||
payload = json.dumps({
|
||||
"model": "Qwen3.5-4B-MLX-4bit", # 可用模型
|
||||
"messages": messages,
|
||||
"max_tokens": max_tokens,
|
||||
"temperature": temperature,
|
||||
"stream": False,
|
||||
"max_completion_tokens": 100,
|
||||
}).encode("utf-8")
|
||||
|
||||
req = urllib.request.Request(
|
||||
f"{OLLAMA_BASE}/chat/completions",
|
||||
data=payload,
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {API_KEY}",
|
||||
},
|
||||
method="POST",
|
||||
)
|
||||
|
||||
try:
|
||||
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
with opener.open(req, timeout=timeout) as resp:
|
||||
result = json.loads(resp.read().decode("utf-8"))
|
||||
text = result["choices"][0]["message"]["content"].strip()
|
||||
# 清理可能的引号
|
||||
text = text.strip("\"'「」『』")
|
||||
return text
|
||||
except Exception as e:
|
||||
print(f" ⚠️ oMLX 调用失败: {e}")
|
||||
return None
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 主类
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
class AICommentGenerator:
|
||||
"""AI 评论生成器 — 基于 oMLX 本地模型"""
|
||||
|
||||
def __init__(self, base_url: str = OLLAMA_BASE, api_key: str = API_KEY,
|
||||
model: str = "Qwen3.5-4B-MLX-4bit"):
|
||||
self.base_url = base_url
|
||||
self.api_key = api_key
|
||||
self.model = model
|
||||
self._used_comments: set = set() # 会话级去重
|
||||
|
||||
def set_url(self, url: str, key: str):
|
||||
"""设置 oMLX 地址和密钥"""
|
||||
self.base_url = url
|
||||
self.api_key = key
|
||||
|
||||
def reset_session(self):
|
||||
"""重置去重记录"""
|
||||
self._used_comments.clear()
|
||||
|
||||
def _is_duplicate(self, text: str) -> bool:
|
||||
"""检查是否与已用评论重复"""
|
||||
text_clean = text.strip().lower()
|
||||
if text_clean in self._used_comments:
|
||||
return True
|
||||
# 相似度检查:如果已有的评论包含当前文本或反之
|
||||
for used in self._used_comments:
|
||||
if text_clean in used or used in text_clean:
|
||||
return True
|
||||
return False
|
||||
|
||||
def generate(self, topic: str = "",
|
||||
style: str = "general",
|
||||
max_tokens: int = 64,
|
||||
temperature: float = 0.9,
|
||||
retries: int = 3) -> str:
|
||||
"""
|
||||
生成一条小红书风格评论。
|
||||
|
||||
Args:
|
||||
topic: 评论主题(空字符串则随机)
|
||||
style: 风格 (general, zhongcao/种草, tips/技巧, travel/旅行, food/美食)
|
||||
max_tokens: 生成长度
|
||||
temperature: 生成温度
|
||||
retries: 失败重试次数
|
||||
|
||||
Returns:
|
||||
评论文本(失败时 fallback 到随机静态语料)
|
||||
"""
|
||||
style_desc = {
|
||||
"general": "日常评论,语气自然亲切",
|
||||
"zhongcao": "种草风,热情推荐,简短有力",
|
||||
"tips": "实用技术型,分享经验感受",
|
||||
"travel": "旅行/风景类,温暖治愈",
|
||||
"food": "美食类,表达馋了/想尝试",
|
||||
}.get(style, "日常评论,语气自然亲切")
|
||||
|
||||
for attempt in range(retries):
|
||||
content = f"生成一条小红书评论。\n"
|
||||
if topic:
|
||||
content += f"主题: {topic}\n"
|
||||
content += f"风格: {style_desc}\n"
|
||||
content += "只输出评论文本,不要加任何前缀或格式。"
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": SYSTEM_PROMPT},
|
||||
{"role": "user", "content": content},
|
||||
]
|
||||
|
||||
text = _call_omlx(messages, max_tokens=max_tokens,
|
||||
temperature=temperature)
|
||||
if text and not self._is_duplicate(text):
|
||||
self._used_comments.add(text.strip().lower())
|
||||
return text
|
||||
|
||||
time.sleep(0.3)
|
||||
|
||||
# 所有重试失败,fallback 到静态语料
|
||||
return self._fallback_comment()
|
||||
|
||||
def generate_with_context(self, note_title: str = "",
|
||||
note_content: str = "",
|
||||
style: str = "general") -> str:
|
||||
"""
|
||||
基于笔记上下文生成相关评论。
|
||||
|
||||
Args:
|
||||
note_title: 笔记标题
|
||||
note_content: 笔记内容摘要
|
||||
style: 评论风格
|
||||
|
||||
Returns:
|
||||
评论文本
|
||||
"""
|
||||
context = "根据以下笔记内容生成一条评论:\n"
|
||||
if note_title:
|
||||
context += f"标题: {note_title}\n"
|
||||
if note_content:
|
||||
context += f"内容: {note_content[:200]}\n"
|
||||
context += "评论要基于笔记内容,像是真实用户的自然感受。"
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": SYSTEM_PROMPT},
|
||||
{"role": "user", "content": context},
|
||||
]
|
||||
|
||||
text = _call_omlx(messages, max_tokens=80, temperature=0.85)
|
||||
if text and not self._is_duplicate(text):
|
||||
self._used_comments.add(text.strip().lower())
|
||||
return text
|
||||
|
||||
return self._fallback_comment(topic=note_title[:20]
|
||||
if note_title else "")
|
||||
|
||||
async def generate_from_note(self, page) -> str:
|
||||
"""
|
||||
从当前页面提取笔记内容并生成评论。
|
||||
|
||||
Args:
|
||||
page: Playwright Page 对象
|
||||
|
||||
Returns:
|
||||
评论文本
|
||||
"""
|
||||
try:
|
||||
note_info = await page.evaluate("""
|
||||
() => {
|
||||
const title = document.querySelector('.title, h1, [class*=note-title]');
|
||||
const content = document.querySelector('.content, .desc, [class*=content]');
|
||||
return {
|
||||
title: title ? title.textContent.trim().substring(0, 100) : '',
|
||||
content: content ? content.textContent.trim().substring(0, 200) : '',
|
||||
};
|
||||
}
|
||||
""")
|
||||
except Exception:
|
||||
note_info = {"title": "", "content": ""}
|
||||
|
||||
return self.generate_with_context(
|
||||
note_title=note_info.get("title", ""),
|
||||
note_content=note_info.get("content", ""),
|
||||
)
|
||||
|
||||
def _fallback_comment(self, topic: str = "") -> str:
|
||||
"""静态语料 fallback"""
|
||||
from matrix_modules.comment.xhs.corpus import get_comment
|
||||
return get_comment(length="medium")
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 便捷函数(与 comment_corpus.py 接口一致)
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
_global_generator: AICommentGenerator = None
|
||||
|
||||
|
||||
def _get_generator() -> AICommentGenerator:
|
||||
"""获取全局生成器实例"""
|
||||
global _global_generator
|
||||
if _global_generator is None:
|
||||
_global_generator = AICommentGenerator()
|
||||
return _global_generator
|
||||
|
||||
|
||||
def ai_comment(topic: str = "", style: str = "general") -> str:
|
||||
"""获取一条 AI 生成的小红书评论"""
|
||||
return _get_generator().generate(topic=topic, style=style)
|
||||
|
||||
|
||||
def reset_ai_session():
|
||||
"""重置 AI 评论去重记录"""
|
||||
_get_generator().reset_session()
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 测试入口
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
if __name__ == "__main__":
|
||||
print("🎯 AI 评论生成器测试\n")
|
||||
|
||||
gen = AICommentGenerator()
|
||||
|
||||
# 测试:不同主题
|
||||
for topic in ["美食探店心得", "周末旅行攻略", "护肤好物分享", "穿搭技巧总结"]:
|
||||
comment = gen.generate(topic=topic)
|
||||
print(f" 📝 [{topic}] → {comment}")
|
||||
|
||||
# 测试:上下文感知
|
||||
print()
|
||||
comment = gen.generate_with_context(
|
||||
note_title="在苏州发现一家宝藏咖啡馆",
|
||||
note_content="周末和朋友去了平江路一家新开的咖啡馆,环境超好,咖啡也很棒!",
|
||||
)
|
||||
print(f" 📝 [上下文] → {comment}")
|
||||
|
||||
# 测试:oMLX 不可用时 fallback
|
||||
print()
|
||||
gen.set_url("http://localhost:9999", "invalid_key")
|
||||
fallback = gen.generate(topic="测试")
|
||||
print(f" 📝 [fallback] → {fallback}")
|
||||
@@ -0,0 +1,6 @@
|
||||
"""
|
||||
小红书评论语料模块
|
||||
"""
|
||||
from .corpus import get_comment, get_comments, random_comment, short_comment, long_comment
|
||||
|
||||
__all__ = ["get_comment", "get_comments", "random_comment", "short_comment", "long_comment"]
|
||||
@@ -0,0 +1,321 @@
|
||||
"""
|
||||
小红书评论语料库
|
||||
风格: 种草/经验分享/生活方式/美食/旅行/美妆/穿搭
|
||||
特点: 语气亲切、emoji 适度、实用性强、避免营销感
|
||||
|
||||
分类标签:
|
||||
- lifestyle: 生活方式/日常
|
||||
- food: 美食/探店
|
||||
- travel: 旅行/出游
|
||||
- beauty: 美妆/护肤
|
||||
- fashion: 穿搭/时尚
|
||||
- tips: 经验/技巧
|
||||
- emotion: 情感共鸣
|
||||
"""
|
||||
|
||||
import random
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 会话级去重 — 同一运行中不重复发送相同评论
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
_used_comments: set = set()
|
||||
|
||||
|
||||
def reset_session():
|
||||
"""清空已用评论记录(新账号/新运行开始时调用)"""
|
||||
_used_comments.clear()
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 按分类组织的语料
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
CORPUS = {
|
||||
"lifestyle": [
|
||||
"这就是我理想的生活状态",
|
||||
"看完立刻去试了,真的有用",
|
||||
"收藏了,周末就安排",
|
||||
"这个思路太棒了,学到了",
|
||||
"已经开始期待了",
|
||||
"这就是我想要的感觉",
|
||||
"看完心情都变好了",
|
||||
"太真实了,完全说到心坎里",
|
||||
"这个分享太及时了",
|
||||
"已经转发给闺蜜了",
|
||||
"这就是生活该有的样子",
|
||||
"看完立刻动手试了一下",
|
||||
"这个角度从来没想过",
|
||||
"太治愈了,反复看了好几遍",
|
||||
"这就是我最近在找的东西",
|
||||
],
|
||||
"food": [
|
||||
"看着就很有食欲",
|
||||
"下次去一定要试试",
|
||||
"这个做法收藏了",
|
||||
"看起来好好吃",
|
||||
"已经流口水了",
|
||||
"这个搭配绝了",
|
||||
"周末就去做",
|
||||
"求具体做法",
|
||||
"这个店在哪里",
|
||||
"看起来就很正宗",
|
||||
"已经加入收藏夹",
|
||||
"这个摆盘太美了",
|
||||
"食材都很新鲜的样子",
|
||||
"这个口味应该很适合我",
|
||||
"已经截图保存了",
|
||||
],
|
||||
"travel": [
|
||||
"这个地方太美了",
|
||||
"已经加入旅行清单",
|
||||
"下次假期就去",
|
||||
"这个机位太绝了",
|
||||
"攻略做得很详细",
|
||||
"风景太治愈了",
|
||||
"已经分享给朋友了",
|
||||
"这个路线规划得很好",
|
||||
"看完立刻想去",
|
||||
"这个地方人不多吧",
|
||||
"住宿看起来不错",
|
||||
"这个角度拍出来太好看了",
|
||||
"已经收藏了,下次参考",
|
||||
"这个景点值得去吗",
|
||||
"交通方便吗",
|
||||
],
|
||||
"beauty": [
|
||||
"这个效果太明显了",
|
||||
"已经种草了",
|
||||
"求链接",
|
||||
"这个颜色很适合我",
|
||||
"肤质看起来变好了",
|
||||
"这个步骤学到了",
|
||||
"已经加入购物车",
|
||||
"这个手法很关键",
|
||||
"效果能维持多久",
|
||||
"敏感肌可以用吗",
|
||||
"这个搭配思路很好",
|
||||
"已经截图保存教程",
|
||||
"这个产品的确不错",
|
||||
"步骤很详细,收藏了",
|
||||
"这个技巧太实用了",
|
||||
],
|
||||
"fashion": [
|
||||
"这套搭配太绝了",
|
||||
"颜色搭配得很好",
|
||||
"这个风格很适合你",
|
||||
"已经截图保存",
|
||||
"求衣服链接",
|
||||
"这个配饰点睛了",
|
||||
"整体比例很好",
|
||||
"这个单品很百搭",
|
||||
"已经加入灵感库",
|
||||
"这个配色可以学",
|
||||
"风格很独特",
|
||||
"这套日常也能穿",
|
||||
"细节处理得很好",
|
||||
"这个搭配思路学到了",
|
||||
"已经转发给姐妹了",
|
||||
],
|
||||
"tips": [
|
||||
"这个技巧太实用了",
|
||||
"已经收藏,下次用",
|
||||
"步骤写得很清楚",
|
||||
"这个坑我也踩过",
|
||||
"感谢分享,省了很多时间",
|
||||
"这个方法效率很高",
|
||||
"已经试过了,确实好用",
|
||||
"这个思路可以举一反三",
|
||||
"细节讲得很到位",
|
||||
"这个工具我也在用",
|
||||
"已经加入知识库",
|
||||
"这个经验太宝贵了",
|
||||
"看完立刻去实践了",
|
||||
"这个总结很到位",
|
||||
"已经分享给同事了",
|
||||
],
|
||||
"emotion": [
|
||||
"太真实了",
|
||||
"完全共鸣",
|
||||
"这就是我想说的",
|
||||
"看完很有感触",
|
||||
"太懂这种感觉了",
|
||||
"说得太对了",
|
||||
"这就是生活",
|
||||
"看完沉默了",
|
||||
"太戳心了",
|
||||
"这个视角很独特",
|
||||
"完全同意",
|
||||
"这就是成长吧",
|
||||
"看完想了很多",
|
||||
"这个分享很有力量",
|
||||
"感谢分享",
|
||||
],
|
||||
}
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 通用短评(不分类,随机使用)
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
GENERAL_SHORT = [
|
||||
"学到了",
|
||||
"收藏了",
|
||||
"确实如此",
|
||||
"很有道理",
|
||||
"太棒了",
|
||||
"感谢分享",
|
||||
"已三连",
|
||||
"好文",
|
||||
"mark",
|
||||
"码住",
|
||||
"实用",
|
||||
"干货",
|
||||
"受教了",
|
||||
"涨知识了",
|
||||
"这个好",
|
||||
"不错",
|
||||
"可以",
|
||||
"666",
|
||||
"绝了",
|
||||
"牛",
|
||||
]
|
||||
|
||||
# ── 三级接力语料(评论→回复→再回复的自然互动链)──
|
||||
# 每个分类含 first/reply/second 三层
|
||||
CHAIN_CORPUS = {
|
||||
"food": [
|
||||
{"first": "看着就很有食欲,周末去试试", "reply": "同意!上次去吃过一次确实不错", "second": "我也去过,他家那个招牌菜绝了"},
|
||||
{"first": "这个做法收藏了,回家就做", "reply": "做了好几次了,家人都说好吃", "second": "是的,跟着做了一次很成功"},
|
||||
],
|
||||
"travel": [
|
||||
{"first": "这个地方太美了,已加入旅行清单", "reply": "去年去过,风景确实好,值得二刷", "second": "求攻略!准备下个月去"},
|
||||
{"first": "这个机位太绝了,拍得真好", "reply": "早上五点去占的位置,值得", "second": "太卷了哈哈哈,不过效果确实好"},
|
||||
],
|
||||
"tech": [
|
||||
{"first": "分析得很透彻,学到了", "reply": "对,第三点我之前完全没想到", "second": "是的,按这个方法试了效果很好"},
|
||||
],
|
||||
"lifestyle": [
|
||||
{"first": "这就是我理想的生活状态", "reply": "同款生活!每天都很充实", "second": "羡慕了,我也要这样过"},
|
||||
],
|
||||
"emotion": [
|
||||
{"first": "太真实了,完全说到心坎里", "reply": "是啊,经历过的人才懂", "second": "抱抱,都会好起来的"},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def get_chain(group: str = "food", position: str = "first") -> str:
|
||||
"""获取三级接力语料。group=分类, position=first/reply/second"""
|
||||
import random
|
||||
pool = CHAIN_CORPUS.get(group, CHAIN_CORPUS["food"])
|
||||
item = random.choice(pool)
|
||||
return item.get(position, item["first"])
|
||||
|
||||
|
||||
def get_chain_group(group: str = "food") -> dict:
|
||||
"""获取一组完整的三级接力语料"""
|
||||
import random
|
||||
pool = CHAIN_CORPUS.get(group, CHAIN_CORPUS["food"])
|
||||
return random.choice(pool)
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 评论生成器
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
|
||||
def get_comment(category: str = None, length: str = "medium") -> str:
|
||||
"""
|
||||
获取一条小红书风格评论(自动去重:同一运行中不会重复返回相同评论)
|
||||
|
||||
Args:
|
||||
category: 分类标签 (lifestyle/food/travel/beauty/fashion/tips/emotion)
|
||||
None 时随机选择分类
|
||||
length: 长度控制
|
||||
- short: 2~8 字短评
|
||||
- medium: 10~25 字常规评论 (默认)
|
||||
- long: 25~50 字长评论
|
||||
|
||||
Returns:
|
||||
评论文本
|
||||
"""
|
||||
global _used_comments
|
||||
|
||||
if length == "short":
|
||||
available = [c for c in GENERAL_SHORT if c not in _used_comments]
|
||||
if not available:
|
||||
available = GENERAL_SHORT # 用完了就复用
|
||||
comment = random.choice(available)
|
||||
_used_comments.add(comment)
|
||||
return comment
|
||||
|
||||
if category and category in CORPUS:
|
||||
pool = [c for c in CORPUS[category] if c not in _used_comments]
|
||||
if not pool:
|
||||
pool = CORPUS[category] # 用完了就复用
|
||||
else:
|
||||
all_items = []
|
||||
for cat_items in CORPUS.values():
|
||||
all_items.extend(cat_items)
|
||||
pool = [c for c in all_items if c not in _used_comments]
|
||||
if not pool:
|
||||
pool = all_items # 用完了就复用
|
||||
|
||||
comment = random.choice(pool)
|
||||
_used_comments.add(comment)
|
||||
|
||||
# 长度控制
|
||||
if length == "long" and len(comment) < 20:
|
||||
# 追加一条短评组合
|
||||
comment += "。" + random.choice(GENERAL_SHORT)
|
||||
elif length == "medium" and len(comment) > 30:
|
||||
# 截断到合适长度
|
||||
comment = comment[:28] + "..."
|
||||
|
||||
return comment
|
||||
|
||||
|
||||
def get_comments(count: int = 3, category: str = None, length: str = "medium") -> list:
|
||||
"""获取多条不重复评论"""
|
||||
results = []
|
||||
pool = []
|
||||
|
||||
if category and category in CORPUS:
|
||||
pool = CORPUS[category].copy()
|
||||
else:
|
||||
for cat_items in CORPUS.values():
|
||||
pool.extend(cat_items)
|
||||
|
||||
# 去重并随机选择
|
||||
unique_pool = list(set(pool))
|
||||
if count > len(unique_pool):
|
||||
count = len(unique_pool)
|
||||
|
||||
selected = random.sample(unique_pool, count)
|
||||
|
||||
for comment in selected:
|
||||
if length == "short":
|
||||
comment = random.choice(GENERAL_SHORT)
|
||||
elif length == "long" and len(comment) < 20:
|
||||
comment += "。" + random.choice(GENERAL_SHORT)
|
||||
results.append(comment)
|
||||
|
||||
return results
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 快捷函数
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
|
||||
def random_comment() -> str:
|
||||
"""随机获取一条中等长度评论(养号默认使用)"""
|
||||
return get_comment(length="medium")
|
||||
|
||||
|
||||
def short_comment() -> str:
|
||||
"""获取短评"""
|
||||
return get_comment(length="short")
|
||||
|
||||
|
||||
def long_comment() -> str:
|
||||
"""获取长评"""
|
||||
return get_comment(length="long")
|
||||
@@ -0,0 +1 @@
|
||||
# nurture 模块
|
||||
@@ -0,0 +1,242 @@
|
||||
"""
|
||||
anchor_db.py — 锚点规则数据库管理
|
||||
|
||||
锚点规则 + 原子操作 + 检测日志 + 校准候选 全部存储在 SQLite 中。
|
||||
可拆解、可更新、可版本管理。
|
||||
"""
|
||||
import json
|
||||
import sqlite3
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
DB_DIR = Path(__file__).parent / "data"
|
||||
DB_PATH = DB_DIR / "anchor.db"
|
||||
SCHEMA_PATH = DB_DIR / "schema.sql"
|
||||
|
||||
|
||||
# ── 数据库初始化 ──
|
||||
|
||||
def get_conn() -> sqlite3.Connection:
|
||||
"""获取数据库连接(每次调用创建新连接,线程安全)"""
|
||||
DB_DIR.mkdir(parents=True, exist_ok=True)
|
||||
conn = sqlite3.connect(str(DB_PATH))
|
||||
conn.row_factory = sqlite3.Row
|
||||
conn.execute("PRAGMA journal_mode=WAL")
|
||||
conn.execute("PRAGMA foreign_keys=ON")
|
||||
return conn
|
||||
|
||||
|
||||
def init_db(force: bool = False):
|
||||
"""初始化/重置数据库"""
|
||||
if force and DB_PATH.exists():
|
||||
DB_PATH.unlink()
|
||||
conn = get_conn()
|
||||
# 检查是否已初始化
|
||||
cursor = conn.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='anchor_rules'")
|
||||
if cursor.fetchone() and not force:
|
||||
conn.close()
|
||||
return
|
||||
|
||||
# 执行 schema
|
||||
schema = SCHEMA_PATH.read_text(encoding='utf-8')
|
||||
conn.executescript(schema)
|
||||
conn.commit()
|
||||
conn.close()
|
||||
print(f"✅ 数据库已初始化: {DB_PATH}")
|
||||
|
||||
|
||||
# ── 锚点规则 CRUD ──
|
||||
|
||||
def get_rules(state_name: str = None) -> list:
|
||||
"""获取锚点规则
|
||||
|
||||
Args:
|
||||
state_name: 指定状态名, None=返回所有
|
||||
Returns:
|
||||
[{state_name, field, operator, value, logic_group}, ...]
|
||||
"""
|
||||
conn = get_conn()
|
||||
if state_name:
|
||||
rows = conn.execute(
|
||||
"SELECT * FROM anchor_rules WHERE state_name=? AND active=1 ORDER BY logic_group, id",
|
||||
(state_name,)
|
||||
).fetchall()
|
||||
else:
|
||||
rows = conn.execute(
|
||||
"SELECT * FROM anchor_rules WHERE active=1 ORDER BY state_name, logic_group, id"
|
||||
).fetchall()
|
||||
conn.close()
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
|
||||
def add_rule(state_name: str, field: str, operator: str, value: str,
|
||||
logic_group: int = 0) -> int:
|
||||
"""添加锚点规则"""
|
||||
conn = get_conn()
|
||||
cursor = conn.execute(
|
||||
"INSERT INTO anchor_rules (state_name, field, operator, value, logic_group) VALUES (?,?,?,?,?)",
|
||||
(state_name, field, operator, value, logic_group)
|
||||
)
|
||||
conn.commit()
|
||||
new_id = cursor.lastrowid
|
||||
conn.close()
|
||||
return new_id
|
||||
|
||||
|
||||
def update_rule(rule_id: int, **kwargs):
|
||||
"""更新规则字段"""
|
||||
allowed = {'field', 'operator', 'value', 'logic_group', 'active'}
|
||||
sets = []
|
||||
params = []
|
||||
for k, v in kwargs.items():
|
||||
if k in allowed:
|
||||
sets.append(f"{k}=?")
|
||||
params.append(v)
|
||||
if not sets:
|
||||
return
|
||||
params.append(rule_id)
|
||||
conn = get_conn()
|
||||
conn.execute(f"UPDATE anchor_rules SET {', '.join(sets)}, updated_at=datetime('now','localtime') WHERE id=?", params)
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
|
||||
def delete_rule(rule_id: int):
|
||||
"""软删除规则"""
|
||||
update_rule(rule_id, active=0)
|
||||
|
||||
|
||||
# ── 原子操作 CRUD ──
|
||||
|
||||
def get_ops(op_name: str = None) -> list:
|
||||
"""获取原子操作"""
|
||||
conn = get_conn()
|
||||
if op_name:
|
||||
rows = conn.execute("SELECT * FROM atomic_ops WHERE op_name=?", (op_name,)).fetchall()
|
||||
else:
|
||||
rows = conn.execute("SELECT * FROM atomic_ops ORDER BY id").fetchall()
|
||||
conn.close()
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
|
||||
def add_op(op_name: str, description: str, action_type: str,
|
||||
target: str, verify_rule: str = None) -> int:
|
||||
"""添加原子操作"""
|
||||
conn = get_conn()
|
||||
cursor = conn.execute(
|
||||
"INSERT INTO atomic_ops (op_name, description, action_type, target, verify_rule) VALUES (?,?,?,?,?)",
|
||||
(op_name, description, action_type, target, verify_rule)
|
||||
)
|
||||
conn.commit()
|
||||
new_id = cursor.lastrowid
|
||||
conn.close()
|
||||
return new_id
|
||||
|
||||
|
||||
# ── 检测日志 ──
|
||||
|
||||
def log_detection(identity: str, expected_state: str, detected_state: str,
|
||||
match: bool, anchor_snapshot: dict = None,
|
||||
screenshot_path: str = None):
|
||||
"""记录一次锚点检测结果"""
|
||||
conn = get_conn()
|
||||
conn.execute(
|
||||
"INSERT INTO detection_logs (identity, expected_state, detected_state, match, anchor_snapshot, screenshot_path) VALUES (?,?,?,?,?,?)",
|
||||
(identity, expected_state, detected_state, int(match),
|
||||
json.dumps(anchor_snapshot, ensure_ascii=False) if anchor_snapshot else None,
|
||||
screenshot_path)
|
||||
)
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_recent_accuracy(identity: str = None, count: int = 50) -> float:
|
||||
"""获取最近 N 次检测的成功率"""
|
||||
conn = get_conn()
|
||||
if identity:
|
||||
rows = conn.execute(
|
||||
"SELECT match FROM detection_logs WHERE identity=? ORDER BY id DESC LIMIT ?",
|
||||
(identity, count)
|
||||
).fetchall()
|
||||
else:
|
||||
rows = conn.execute(
|
||||
"SELECT match FROM detection_logs ORDER BY id DESC LIMIT ?",
|
||||
(count,)
|
||||
).fetchall()
|
||||
conn.close()
|
||||
if not rows:
|
||||
return 1.0
|
||||
return sum(r['match'] for r in rows) / len(rows)
|
||||
|
||||
|
||||
def get_recent_failures(identity: str = None, threshold: int = 5) -> bool:
|
||||
"""检查最近是否连续失败 threshold 次"""
|
||||
conn = get_conn()
|
||||
if identity:
|
||||
rows = conn.execute(
|
||||
"SELECT match FROM detection_logs WHERE identity=? ORDER BY id DESC LIMIT ?",
|
||||
(identity, threshold)
|
||||
).fetchall()
|
||||
else:
|
||||
rows = conn.execute(
|
||||
"SELECT match FROM detection_logs ORDER BY id DESC LIMIT ?",
|
||||
(threshold,)
|
||||
).fetchall()
|
||||
conn.close()
|
||||
if len(rows) < threshold:
|
||||
return False
|
||||
return all(r['match'] == 0 for r in rows)
|
||||
|
||||
|
||||
# ── 校准候选 ──
|
||||
|
||||
def add_candidate(state_name: str, source: str, old_rules: list, new_rules: list) -> int:
|
||||
"""添加校准候选"""
|
||||
conn = get_conn()
|
||||
cursor = conn.execute(
|
||||
"INSERT INTO calibration_candidates (state_name, source, old_rules, new_rules) VALUES (?,?,?,?)",
|
||||
(state_name, source, json.dumps(old_rules, ensure_ascii=False),
|
||||
json.dumps(new_rules, ensure_ascii=False))
|
||||
)
|
||||
conn.commit()
|
||||
new_id = cursor.lastrowid
|
||||
conn.close()
|
||||
return new_id
|
||||
|
||||
|
||||
def update_candidate(candidate_id: int, status: str, test_count: int = None, test_pass: int = None):
|
||||
"""更新校准候选状态"""
|
||||
conn = get_conn()
|
||||
sets = ["status=?"]
|
||||
params = [status]
|
||||
if test_count is not None:
|
||||
sets.append("test_count=?")
|
||||
params.append(test_count)
|
||||
if test_pass is not None:
|
||||
sets.append("test_pass=?")
|
||||
params.append(test_pass)
|
||||
params.append(candidate_id)
|
||||
conn.execute(f"UPDATE calibration_candidates SET {', '.join(sets)} WHERE id=?", params)
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
|
||||
# ── 状态迁移 ──
|
||||
|
||||
def get_valid_next_states(from_state: str) -> list:
|
||||
"""获取从当前状态可以迁移到哪些状态"""
|
||||
conn = get_conn()
|
||||
rows = conn.execute(
|
||||
"SELECT to_state, via_op FROM state_transitions WHERE from_state=?",
|
||||
(from_state,)
|
||||
).fetchall()
|
||||
conn.close()
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# 初始化数据库并导入初始数据
|
||||
init_db(force=True)
|
||||
print(f" 规则数: {len(get_rules())}")
|
||||
print(f" 操作数: {len(get_ops())}")
|
||||
@@ -0,0 +1,222 @@
|
||||
"""
|
||||
anchor_rules.py — 锚点检测引擎
|
||||
|
||||
从 anchor.db 读取规则,执行状态检测。
|
||||
规则可动态更新(通过 anchor_db.py),无需修改代码。
|
||||
"""
|
||||
import re
|
||||
import json
|
||||
from . import anchor_db as db
|
||||
|
||||
|
||||
# ── 运算符映射 ──
|
||||
|
||||
def _apply_op(field_value, operator, target_value):
|
||||
"""对单个字段应用运算符比较(不区分大小写)"""
|
||||
if field_value is None:
|
||||
return False
|
||||
|
||||
# 全部转字符串,不区分大小写
|
||||
fv = str(field_value).lower()
|
||||
tv = str(target_value).lower()
|
||||
|
||||
if operator == 'eq':
|
||||
return fv == tv
|
||||
elif operator == 'neq':
|
||||
return fv != tv
|
||||
elif operator == 'gt':
|
||||
try:
|
||||
return float(fv) > float(tv)
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
elif operator == 'gte':
|
||||
try:
|
||||
return float(fv) >= float(tv)
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
elif operator == 'lt':
|
||||
try:
|
||||
return float(fv) < float(tv)
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
elif operator == 'lte':
|
||||
try:
|
||||
return float(fv) <= float(tv)
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
elif operator == 'contains':
|
||||
return tv in fv
|
||||
elif operator == 'not_contains':
|
||||
return tv not in fv
|
||||
elif operator == 'regex':
|
||||
try:
|
||||
return bool(re.search(tv, fv))
|
||||
except re.error:
|
||||
return False
|
||||
return False
|
||||
|
||||
|
||||
# ── 核心检测逻辑 ──
|
||||
|
||||
def detect_state(anchors: dict, strict: bool = False) -> str:
|
||||
"""检测当前页面状态
|
||||
|
||||
Args:
|
||||
anchors: DOM 锚点数据字典
|
||||
strict: True=必须所有规则满足; False=按置信度选最佳匹配
|
||||
|
||||
Returns:
|
||||
state_name 或 'UNKNOWN'
|
||||
"""
|
||||
rules = db.get_rules()
|
||||
if not rules:
|
||||
return "UNKNOWN"
|
||||
|
||||
# 按 state_name 分组
|
||||
state_rules = {}
|
||||
for r in rules:
|
||||
sn = r['state_name']
|
||||
if sn not in state_rules:
|
||||
state_rules[sn] = {}
|
||||
lg = r['logic_group']
|
||||
if lg not in state_rules[sn]:
|
||||
state_rules[sn][lg] = []
|
||||
state_rules[sn][lg].append(r)
|
||||
|
||||
best_state = "UNKNOWN"
|
||||
best_score = 0
|
||||
|
||||
# 检测优先级(子状态优先于父状态)
|
||||
PRIORITY = ['SEARCH_PLAYER', 'DETAIL', 'PLAYER', 'SEARCH', 'HOME', 'POPUP', 'LOGIN']
|
||||
state_groups = sorted(state_rules.items(),
|
||||
key=lambda x: PRIORITY.index(x[0]) if x[0] in PRIORITY else 99)
|
||||
|
||||
for state_name, groups in state_groups:
|
||||
# 每个 logic_group 内的规则 AND,不同 group 之间 OR
|
||||
group_results = []
|
||||
for group_id, group_rules in groups.items():
|
||||
all_match = all(
|
||||
_apply_op(anchors.get(r['field']), r['operator'], r['value'])
|
||||
for r in group_rules
|
||||
)
|
||||
group_results.append(all_match)
|
||||
|
||||
# OR 连接:任一 group 通过即该状态匹配
|
||||
state_match = any(group_results)
|
||||
|
||||
if strict:
|
||||
if state_match:
|
||||
return state_name
|
||||
else:
|
||||
# 非严格模式:计算匹配得分
|
||||
total_rules = sum(len(g) for g in groups.values())
|
||||
matched_rules = sum(
|
||||
sum(1 for r in g if _apply_op(anchors.get(r['field']), r['operator'], r['value']))
|
||||
for g in groups.values()
|
||||
)
|
||||
score = matched_rules / max(total_rules, 1)
|
||||
if state_match and score > best_score:
|
||||
best_score = score
|
||||
best_state = state_name
|
||||
elif not state_match and score > best_score:
|
||||
# 即使不完全匹配,高分也记录
|
||||
if score > 0.6:
|
||||
best_score = score
|
||||
best_state = f"{state_name}?"
|
||||
|
||||
return best_state
|
||||
|
||||
|
||||
def verify_op(op_name: str, anchors: dict) -> bool:
|
||||
"""验证原子操作是否成功执行"""
|
||||
ops = db.get_ops(op_name)
|
||||
if not ops:
|
||||
return True # 无验证规则默认通过
|
||||
op = ops[0]
|
||||
verify = op.get('verify_rule')
|
||||
if not verify:
|
||||
return True
|
||||
|
||||
# 验证规则映射
|
||||
VERIFY_MAP = {
|
||||
'video_count_gte_3': lambda a: a.get('video_count', 0) >= 3,
|
||||
'video_src_change': lambda a: True, # 需要前后对比
|
||||
'like_count_change': lambda a: True,
|
||||
'title_contains_搜索': lambda a: '搜索' in str(a.get('title', '')),
|
||||
'video_count_eq_1': lambda a: a.get('video_count', 0) == 1,
|
||||
'popup_gone': lambda a: not a.get('has_popup_close', False),
|
||||
}
|
||||
|
||||
check = VERIFY_MAP.get(verify)
|
||||
if check:
|
||||
return check(anchors)
|
||||
return True
|
||||
|
||||
|
||||
def log_and_check(identity: str, expected_state: str, anchors: dict,
|
||||
screenshot_path: str = None) -> dict:
|
||||
"""检测 + 记录 + 返回结果"""
|
||||
detected = detect_state(anchors)
|
||||
match = (detected == expected_state)
|
||||
|
||||
db.log_detection(
|
||||
identity=identity,
|
||||
expected_state=expected_state,
|
||||
detected_state=detected,
|
||||
match=match,
|
||||
anchor_snapshot=anchors,
|
||||
screenshot_path=screenshot_path,
|
||||
)
|
||||
|
||||
return {
|
||||
"expected": expected_state,
|
||||
"detected": detected,
|
||||
"match": match,
|
||||
"accuracy": db.get_recent_accuracy(identity),
|
||||
"needs_calibration": db.get_recent_failures(identity, threshold=5),
|
||||
}
|
||||
|
||||
|
||||
# ── 便捷函数 ──
|
||||
|
||||
def list_rules(state_name: str = None) -> str:
|
||||
"""以人类可读格式列出规则"""
|
||||
rules = db.get_rules(state_name)
|
||||
if not rules:
|
||||
return "暂无锚点规则"
|
||||
|
||||
lines = []
|
||||
current_state = None
|
||||
for r in rules:
|
||||
if r['state_name'] != current_state:
|
||||
current_state = r['state_name']
|
||||
lines.append(f"\n[{current_state}]")
|
||||
lines.append(f" {r['field']} {r['operator']} {r['value']} (group={r['logic_group']})")
|
||||
return '\n'.join(lines)
|
||||
|
||||
|
||||
def list_ops() -> str:
|
||||
"""列出所有原子操作"""
|
||||
ops = db.get_ops()
|
||||
if not ops:
|
||||
return "暂无原子操作"
|
||||
lines = []
|
||||
for op in ops:
|
||||
lines.append(f" {op['op_name']:15s} | {op['action_type']:10s} | {op.get('target', ''):30s} | {op.get('verify_rule', '-')}")
|
||||
return '\n'.join(lines)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# 测试:用采集数据验证规则
|
||||
import json
|
||||
summary_path = db.DB_DIR.parent.parent / "data" / "anchor_summary.json"
|
||||
if summary_path.exists():
|
||||
data = json.loads(summary_path.read_text())
|
||||
print("📊 用采集数据验证锚点规则:")
|
||||
for d in data:
|
||||
step = d['step']
|
||||
anchors = d['anchors']
|
||||
state = detect_state(anchors)
|
||||
print(f" 步{step}: {state:15s} | {d['url'][:40]}")
|
||||
print(f"\n📋 当前规则:\n{list_rules()}")
|
||||
print(f"\n📋 原子操作:\n{list_ops()}")
|
||||
@@ -0,0 +1,187 @@
|
||||
"""
|
||||
行为模拟参数配置 — 拟人化操作行为
|
||||
|
||||
每个身份可以独立配置 behavior 参数,覆盖默认值。
|
||||
默认配置在 DEFAULT_CONFIG 中定义,用户可在 identities/{name}/config.yaml 中覆盖。
|
||||
"""
|
||||
import random
|
||||
import time
|
||||
from typing import Optional
|
||||
|
||||
# ── 默认行为配置 ──────────────────────────────────────────────
|
||||
|
||||
DEFAULT_CONFIG = {
|
||||
# 基础延迟
|
||||
'base_delay': 1.5, # 操作间基础间隔(秒)
|
||||
'delay_variance': 0.8, # 随机波动范围
|
||||
|
||||
# 鼠标行为
|
||||
'mouse': {
|
||||
'speed_range': [0.5, 1.5], # 鼠标移动速度系数
|
||||
'click_delay': [0.08, 0.25], # 点击前停顿(秒)
|
||||
},
|
||||
|
||||
# 滚动行为
|
||||
'scroll': {
|
||||
'mode': 'human', # human / random / fast
|
||||
'pause_chance': 0.3, # 滚动中暂停的概率
|
||||
'pause_duration': [0.5, 2.0], # 暂停时长范围(秒)
|
||||
'distance_range': [200, 700], # 每次滚动距离(px)
|
||||
},
|
||||
|
||||
# 操作顺序
|
||||
'sequence': {
|
||||
'shuffle': True, # 是否打乱操作顺序
|
||||
'repetition_chance': 0.1, # 重复上一个操作的概率
|
||||
'min_gap': 1.5, # 操作间最小间隔(秒)
|
||||
'max_gap': 6.0, # 操作间最大间隔(秒)
|
||||
},
|
||||
|
||||
# 注意力模拟
|
||||
'attention': {
|
||||
'distraction_chance': 0.05, # "分心"概率
|
||||
'distraction_duration': [2, 6], # 分心持续时长(秒)
|
||||
'watch_duration': [4, 12], # 观看视频时长(秒)
|
||||
'linger_after_action': [1, 4], # 操作后的停留(秒)
|
||||
},
|
||||
|
||||
# 轮间休息
|
||||
'round_break': {
|
||||
'min_break': 5, # 轮间最短休息(秒)
|
||||
'max_break': 20, # 轮间最长休息(秒)
|
||||
},
|
||||
|
||||
# ★ 反检测 — 拟人化操作策略(2026-05-04)
|
||||
'anti_detection': {
|
||||
'card_position_range': [0, 4], # 点卡片时选第几个(0=第1个)
|
||||
'search_result_range': [1, 4], # 搜索结果点第几个(0=第1个)
|
||||
'use_keyboard_like': False, # 是否用键盘 L 键点赞(预留)
|
||||
'use_close_button': False, # 是否用X按钮返回(预留)
|
||||
'scroll_method': 'mouse', # 切换方式: mouse/keyboard/js
|
||||
},
|
||||
|
||||
# 异常处理
|
||||
'retry': {
|
||||
'max_retries': 2, # 操作失败最大重试次数
|
||||
'retry_delay': [1, 3], # 重试前等待(秒)
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class BehaviorConfig:
|
||||
"""行为配置 — 控制所有操作的拟人化参数"""
|
||||
|
||||
def __init__(self, config: Optional[dict] = None):
|
||||
self._cfg = DEFAULT_CONFIG.copy()
|
||||
if config:
|
||||
self._deep_merge(self._cfg, config)
|
||||
|
||||
def _deep_merge(self, base: dict, override: dict):
|
||||
"""递归合并配置"""
|
||||
for key, val in override.items():
|
||||
if key in base and isinstance(base[key], dict) and isinstance(val, dict):
|
||||
self._deep_merge(base[key], val)
|
||||
else:
|
||||
base[key] = val
|
||||
|
||||
# ── 延迟方法 ──────────────────────────────────────────
|
||||
|
||||
def action_delay(self) -> float:
|
||||
"""两个操作之间的随机延迟"""
|
||||
base = self._cfg['base_delay']
|
||||
variance = self._cfg['delay_variance']
|
||||
return max(0.3, base + random.uniform(-variance, variance))
|
||||
|
||||
def sequence_gap(self) -> float:
|
||||
"""操作序列之间的间隔"""
|
||||
sg = self._cfg['sequence']
|
||||
return random.uniform(sg['min_gap'], sg['max_gap'])
|
||||
|
||||
def click_delay(self) -> float:
|
||||
"""点击前的微停顿"""
|
||||
cd = self._cfg['mouse']['click_delay']
|
||||
return random.uniform(cd[0], cd[1])
|
||||
|
||||
# ── 滚动方法 ──────────────────────────────────────────
|
||||
|
||||
def scroll_distance(self) -> int:
|
||||
"""每次滚动的距离"""
|
||||
dr = self._cfg['scroll']['distance_range']
|
||||
return random.randint(dr[0], dr[1])
|
||||
|
||||
def should_pause_scroll(self) -> bool:
|
||||
"""滚动中是否应该暂停"""
|
||||
return random.random() < self._cfg['scroll']['pause_chance']
|
||||
|
||||
def scroll_pause(self) -> float:
|
||||
"""滚动暂停时长"""
|
||||
pd = self._cfg['scroll']['pause_duration']
|
||||
return random.uniform(pd[0], pd[1])
|
||||
|
||||
# ── 注意力方法 ────────────────────────────────────────
|
||||
|
||||
def watch_duration(self) -> float:
|
||||
"""观看一个视频的时长"""
|
||||
wd = self._cfg['attention']['watch_duration']
|
||||
return random.uniform(wd[0], wd[1])
|
||||
|
||||
def should_distract(self) -> bool:
|
||||
"""是否应该"分心"(模拟走神)"""
|
||||
return random.random() < self._cfg['attention']['distraction_chance']
|
||||
|
||||
def distraction_duration(self) -> float:
|
||||
"""分心持续时长"""
|
||||
dd = self._cfg['attention']['distraction_duration']
|
||||
return random.uniform(dd[0], dd[1])
|
||||
|
||||
def linger_after_action(self) -> float:
|
||||
"""操作后的停留"""
|
||||
la = self._cfg['attention']['linger_after_action']
|
||||
|
||||
# ── 反检测方法 ────────────────────────────────────────
|
||||
|
||||
def card_position(self) -> int:
|
||||
"""随机选视频卡片位置(避免每次都点第1个)"""
|
||||
cr = self._cfg['anti_detection']['card_position_range']
|
||||
return random.randint(cr[0], cr[1])
|
||||
|
||||
def search_result_position(self) -> int:
|
||||
"""随机选搜索结果位置(避免每次都点第1个)"""
|
||||
sr = self._cfg['anti_detection']['search_result_range']
|
||||
return random.randint(sr[0], sr[1])
|
||||
return random.uniform(la[0], la[1])
|
||||
|
||||
# ── 轮间休息 ──────────────────────────────────────────
|
||||
|
||||
def round_break(self) -> float:
|
||||
"""轮间休息时长"""
|
||||
rb = self._cfg['round_break']
|
||||
return random.uniform(rb['min_break'], rb['max_break'])
|
||||
|
||||
# ── 操作顺序 ──────────────────────────────────────────
|
||||
|
||||
def should_shuffle(self) -> bool:
|
||||
"""是否打乱操作顺序"""
|
||||
return self._cfg['sequence']['shuffle']
|
||||
|
||||
def should_repeat(self) -> bool:
|
||||
"""是否重复上一个操作"""
|
||||
return random.random() < self._cfg['sequence']['repetition_chance']
|
||||
|
||||
# ── 重试 ──────────────────────────────────────────────
|
||||
|
||||
def retry_delay(self) -> float:
|
||||
"""重试前等待"""
|
||||
rd = self._cfg['retry']['retry_delay']
|
||||
return random.uniform(rd[0], rd[1])
|
||||
|
||||
@property
|
||||
def max_retries(self) -> int:
|
||||
return self._cfg['retry']['max_retries']
|
||||
|
||||
# ── 工具方法 ──────────────────────────────────────────
|
||||
|
||||
def sleep(self, seconds: float):
|
||||
"""带拟人化的 sleep(加入微抖动)"""
|
||||
jitter = random.uniform(-0.05, 0.05)
|
||||
time.sleep(max(0, seconds + jitter))
|
||||
@@ -0,0 +1,294 @@
|
||||
"""
|
||||
calibration.py — 校准触发器 + AI 兜底
|
||||
|
||||
功能:
|
||||
1. 定期检查锚点检测成功率
|
||||
2. 触发校准(连续失败/成功率低)
|
||||
3. AI 兜底分析(基于 DOM 数据)
|
||||
4. 生成校准候选规则
|
||||
"""
|
||||
import json
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
from . import anchor_db as db
|
||||
|
||||
# oMLX 配置
|
||||
OLLAMA_BASE = "http://localhost:8000"
|
||||
OLLAMA_API_KEY = "5omlx"
|
||||
LLM_MODEL = "mlx-community--Qwen3.5-4B-MLX-4bit" # 或 gemma-3-4b-it-4bit
|
||||
|
||||
|
||||
# ── 校准检查 ──
|
||||
|
||||
def check_calibration(identity: str) -> dict:
|
||||
"""检查是否需要校准
|
||||
|
||||
Returns:
|
||||
dict: {needs, accuracy, consecutive_failures, level}
|
||||
"""
|
||||
accuracy = db.get_recent_accuracy(identity, count=50)
|
||||
consecutive_fail = db.get_recent_failures(identity, threshold=5)
|
||||
|
||||
# 判断级别
|
||||
if accuracy >= 0.9:
|
||||
level = "normal"
|
||||
elif accuracy >= 0.6:
|
||||
level = "warn"
|
||||
elif accuracy >= 0.3:
|
||||
level = "error"
|
||||
else:
|
||||
level = "alert"
|
||||
|
||||
needs = (accuracy < 0.6) or consecutive_fail
|
||||
|
||||
return {
|
||||
"needs_calibration": needs,
|
||||
"accuracy": round(accuracy, 2),
|
||||
"consecutive_failures": consecutive_fail,
|
||||
"level": level,
|
||||
}
|
||||
|
||||
|
||||
def should_trigger(identity: str) -> bool:
|
||||
"""快速判断是否应触发校准"""
|
||||
acc = db.get_recent_accuracy(identity, count=20)
|
||||
fail = db.get_recent_failures(identity, threshold=5)
|
||||
return acc < 0.6 or fail
|
||||
|
||||
|
||||
# ── AI 兜底 ──
|
||||
|
||||
def _build_ai_prompt(anchors: dict, screenshot_path: str = None) -> str:
|
||||
"""构建发给 LLM 的 prompt"""
|
||||
# 清理敏感数据
|
||||
safe_anchors = {k: v for k, v in anchors.items()
|
||||
if k not in ('body_text', 'video_src_prefix')}
|
||||
|
||||
prompt = f"""你是一个抖音页面状态分析专家。请根据以下页面DOM数据,判断当前页面处于什么状态。
|
||||
|
||||
DOM数据:
|
||||
```json
|
||||
{json.dumps(safe_anchors, ensure_ascii=False, indent=2)}
|
||||
```
|
||||
|
||||
请返回以下JSON格式的分析结果(不要包含其他内容):
|
||||
```json
|
||||
{{
|
||||
"state": "当前页面状态(HOME/PLAYER/SEARCH/SEARCH_PLAYER/DETAIL/POPUP/UNKNOWN)",
|
||||
"confidence": 0.95,
|
||||
"evidence": ["判断依据1", "判断依据2"],
|
||||
"suggested_anchors": [
|
||||
{{"field": "video_count", "operator": "gte", "value": "3", "reason": "为什么这个锚点有效"}}
|
||||
],
|
||||
"next_action": {{
|
||||
"op": "建议执行的操作(swipe_down/click_card/like/search/go_home/dismiss_popup)",
|
||||
"reason": "为什么建议这个操作"
|
||||
}}
|
||||
}}
|
||||
```
|
||||
|
||||
注意:
|
||||
- HOME: 首页/精选页,video_count=1,card_count>0
|
||||
- PLAYER: 视频播放页,video_count>=3,有视频在播放
|
||||
- SEARCH: 搜索结果页,title含"搜索"
|
||||
- SEARCH_PLAYER: 从搜索结果进入的播放页
|
||||
- 如果DOM数据异常(如video_count=0),可能是页面未加载或崩溃"""
|
||||
return prompt
|
||||
|
||||
|
||||
async def ai_analyze(anchors: dict, screenshot_path: str = None) -> dict:
|
||||
"""调用本地LLM分析当前页面状态
|
||||
|
||||
Args:
|
||||
anchors: DOM 锚点数据
|
||||
screenshot_path: 截图路径(暂时不用,模型不支持vision)
|
||||
|
||||
Returns:
|
||||
dict: {state, confidence, evidence, suggested_anchors, next_action}
|
||||
"""
|
||||
prompt = _build_ai_prompt(anchors, screenshot_path)
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=30) as client:
|
||||
resp = await client.post(
|
||||
f"{OLLAMA_BASE}/v1/chat/completions",
|
||||
headers={
|
||||
"Authorization": f"Bearer {OLLAMA_API_KEY}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
json={
|
||||
"model": LLM_MODEL,
|
||||
"messages": [
|
||||
{"role": "system", "content": "你是一个抖音页面状态分析专家,只输出JSON。"},
|
||||
{"role": "user", "content": prompt},
|
||||
],
|
||||
"temperature": 0.1,
|
||||
"max_tokens": 1000,
|
||||
},
|
||||
)
|
||||
result = resp.json()
|
||||
content = result["choices"][0]["message"]["content"]
|
||||
|
||||
# 提取 JSON
|
||||
# 先尝试直接解析
|
||||
try:
|
||||
parsed = json.loads(content)
|
||||
except json.JSONDecodeError:
|
||||
# 尝试从 ```json ... ``` 中提取
|
||||
import re
|
||||
m = re.search(r'```(?:json)?\s*\n?(.*?)\n?```', content, re.DOTALL)
|
||||
if m:
|
||||
parsed = json.loads(m.group(1))
|
||||
else:
|
||||
return {
|
||||
"state": "UNKNOWN",
|
||||
"confidence": 0,
|
||||
"error": f"LLM返回格式异常: {content[:200]}",
|
||||
}
|
||||
|
||||
return parsed
|
||||
|
||||
except Exception as e:
|
||||
return {
|
||||
"state": "UNKNOWN",
|
||||
"confidence": 0,
|
||||
"error": str(e),
|
||||
}
|
||||
|
||||
|
||||
# ── 校准流程 ──
|
||||
|
||||
async def run_calibration(identity: str, anchors: dict,
|
||||
screenshot_path: str = None) -> dict:
|
||||
"""执行完整校准流程
|
||||
|
||||
1. 检查当前成功率
|
||||
2. AI 分析当前页面
|
||||
3. 生成校准候选
|
||||
4. 返回校准结果
|
||||
"""
|
||||
check = check_calibration(identity)
|
||||
print(f"[校准] {identity} 准确率: {check['accuracy']}, 级别: {check['level']}")
|
||||
|
||||
# AI 分析
|
||||
print(f"[校准] AI分析中...")
|
||||
ai_result = await ai_analyze(anchors, screenshot_path)
|
||||
ai_state = ai_result.get("state", "UNKNOWN")
|
||||
ai_confidence = ai_result.get("confidence", 0)
|
||||
print(f"[校准] AI判断: {ai_state} (置信度: {ai_confidence})")
|
||||
|
||||
# 如果 AI 建议了新锚点,生成校准候选
|
||||
suggested = ai_result.get("suggested_anchors", [])
|
||||
if suggested and ai_confidence >= 0.7:
|
||||
# 获取当前规则作为 old_rules
|
||||
old_rules = db.get_rules(ai_state)
|
||||
|
||||
candidate_id = db.add_candidate(
|
||||
state_name=ai_state,
|
||||
source="ai",
|
||||
old_rules=[dict(r) for r in old_rules],
|
||||
new_rules=suggested,
|
||||
)
|
||||
print(f"[校准] 候选规则已生成 (id={candidate_id})")
|
||||
|
||||
return {
|
||||
"calibration_triggered": True,
|
||||
"candidate_id": candidate_id,
|
||||
"ai_state": ai_state,
|
||||
"ai_confidence": ai_confidence,
|
||||
"check": check,
|
||||
"ai_result": ai_result,
|
||||
}
|
||||
|
||||
return {
|
||||
"calibration_triggered": False,
|
||||
"check": check,
|
||||
"ai_result": ai_result,
|
||||
}
|
||||
|
||||
|
||||
# ── 候选规则审批 ──
|
||||
|
||||
async def test_candidate(candidate_id: int, test_data: list) -> dict:
|
||||
"""用测试数据验证候选规则
|
||||
|
||||
Args:
|
||||
candidate_id: 校准候选ID
|
||||
test_data: [{anchors: dict, expected_state: str}, ...]
|
||||
|
||||
Returns:
|
||||
{candidate_id, old_accuracy, new_accuracy, accepted}
|
||||
"""
|
||||
candidate_rows = db.get_conn().execute(
|
||||
"SELECT * FROM calibration_candidates WHERE id=?", (candidate_id,)
|
||||
).fetchall()
|
||||
if not candidate_rows:
|
||||
return {"error": "candidate not found"}
|
||||
candidate = dict(candidate_rows[0])
|
||||
new_rules = json.loads(candidate["new_rules"])
|
||||
old_rules = json.loads(candidate["old_rules"])
|
||||
|
||||
# 不需要真的测试,直接在调用方处理
|
||||
return {
|
||||
"candidate_id": candidate_id,
|
||||
"new_rules": new_rules,
|
||||
"old_rules": old_rules,
|
||||
}
|
||||
|
||||
|
||||
def accept_candidate(candidate_id: int):
|
||||
"""采纳候选规则"""
|
||||
candidate_rows = db.get_conn().execute(
|
||||
"SELECT * FROM calibration_candidates WHERE id=?", (candidate_id,)
|
||||
).fetchall()
|
||||
if not candidate_rows:
|
||||
return
|
||||
candidate = dict(candidate_rows[0])
|
||||
new_rules = json.loads(candidate["new_rules"])
|
||||
state_name = candidate["state_name"]
|
||||
|
||||
# 停用旧规则
|
||||
for r in db.get_rules(state_name):
|
||||
db.update_rule(r["id"], active=0)
|
||||
|
||||
# 添加新规则
|
||||
for rule in new_rules:
|
||||
db.add_rule(
|
||||
state_name=state_name,
|
||||
field=rule["field"],
|
||||
operator=rule["operator"],
|
||||
value=rule["value"],
|
||||
logic_group=rule.get("logic_group", 0),
|
||||
)
|
||||
|
||||
db.update_candidate(candidate_id, "accepted")
|
||||
print(f"[校准] 候选#{candidate_id} 已采纳,{state_name}规则已更新")
|
||||
|
||||
|
||||
def reject_candidate(candidate_id: int):
|
||||
"""拒绝候选规则"""
|
||||
db.update_candidate(candidate_id, "rejected")
|
||||
print(f"[校准] 候选#{candidate_id} 已拒绝")
|
||||
|
||||
|
||||
# ── 快速测试(不依赖外部LLM)──
|
||||
|
||||
async def quick_fallback(anchors: dict) -> dict:
|
||||
"""快速兜底:基于简单规则判断(不调LLM),用于LLM不可用时的降级"""
|
||||
vc = anchors.get("video_count", 0)
|
||||
title = anchors.get("title", "")
|
||||
url = anchors.get("url", "")
|
||||
|
||||
if vc >= 3 and ("搜索" in title or "/search/" in url):
|
||||
return {"state": "SEARCH_PLAYER", "confidence": 0.7, "reason": "video≥3且来自搜索"}
|
||||
if vc >= 3:
|
||||
return {"state": "PLAYER", "confidence": 0.8, "reason": "video≥3"}
|
||||
if "搜索" in title and "精选" not in title:
|
||||
return {"state": "SEARCH", "confidence": 0.8, "reason": "标题含搜索"}
|
||||
if "精选" in title:
|
||||
return {"state": "HOME", "confidence": 0.7, "reason": "标题含精选"}
|
||||
return {"state": "UNKNOWN", "confidence": 0.3, "reason": "无法判断"}
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
"""
|
||||
comment_corpus.py — 评论语料库引擎
|
||||
|
||||
从 comment_corpus.yaml 加载分类评论,按权重随机选取。
|
||||
支持模板填充、分类筛选、手动增加。
|
||||
"""
|
||||
import random
|
||||
import yaml
|
||||
from pathlib import Path
|
||||
|
||||
CORPUS_PATH = Path(__file__).parent.parent.parent / "config" / "comment_corpus.yaml"
|
||||
|
||||
|
||||
def load_corpus() -> dict:
|
||||
"""加载语料库 YAML"""
|
||||
with open(CORPUS_PATH, encoding='utf-8') as f:
|
||||
return yaml.safe_load(f)
|
||||
|
||||
|
||||
def get_comment(keyword: str = "", category: str = None) -> str:
|
||||
"""获取一条随机评论
|
||||
|
||||
Args:
|
||||
keyword: 当前视频关键词,用于模板替换
|
||||
category: 指定分类(None=从所有启用分类中选)
|
||||
|
||||
Returns:
|
||||
评论文本
|
||||
"""
|
||||
corpus = load_corpus()
|
||||
cfg = corpus.get("config", {})
|
||||
enabled = cfg.get("enabled_categories", [])
|
||||
disabled = cfg.get("disabled_categories", [])
|
||||
prefer_templates = cfg.get("prefer_templates", True)
|
||||
|
||||
# 筛选可用分类
|
||||
categories = []
|
||||
for cat_key, cat_data in corpus.get("categories", {}).items():
|
||||
weight = cat_data.get("weight", 0)
|
||||
if weight <= 0:
|
||||
continue
|
||||
if enabled and cat_key not in enabled:
|
||||
continue
|
||||
if cat_key in disabled:
|
||||
continue
|
||||
if category and cat_key != category:
|
||||
continue
|
||||
categories.append((cat_key, cat_data, weight))
|
||||
|
||||
if not categories:
|
||||
return "好内容" # 兜底
|
||||
|
||||
# 按权重选分类
|
||||
keys, datas, weights = zip(*categories)
|
||||
chosen_key = random.choices(keys, weights=weights, k=1)[0]
|
||||
chosen_data = datas[keys.index(chosen_key)]
|
||||
|
||||
# 从选中的分类中选一条评论
|
||||
comments = chosen_data.get("comments", [])
|
||||
templates = chosen_data.get("templates", [])
|
||||
|
||||
# 优先用模板
|
||||
if prefer_templates and templates and random.random() < 0.4:
|
||||
template = random.choice(templates)
|
||||
return template.format(keyword=keyword or "这个")
|
||||
elif comments:
|
||||
return random.choice(comments)
|
||||
elif templates:
|
||||
template = random.choice(templates)
|
||||
return template.format(keyword=keyword or "这个")
|
||||
return "👍"
|
||||
|
||||
|
||||
def add_comment(category: str, comment: str):
|
||||
"""手动增加一条评论到语料库"""
|
||||
with open(CORPUS_PATH, encoding='utf-8') as f:
|
||||
corpus = yaml.safe_load(f)
|
||||
|
||||
cat = corpus.get("categories", {}).get(category)
|
||||
if not cat:
|
||||
print(f"❌ 分类 '{category}' 不存在")
|
||||
return
|
||||
|
||||
if "comments" not in cat:
|
||||
cat["comments"] = []
|
||||
cat["comments"].append(comment)
|
||||
|
||||
with open(CORPUS_PATH, 'w', encoding='utf-8') as f:
|
||||
yaml.dump(corpus, f, allow_unicode=True, indent=2, sort_keys=False)
|
||||
print(f"✅ 已添加评论到 [{category}]: {comment}")
|
||||
|
||||
|
||||
def list_categories() -> str:
|
||||
"""列出所有可用分类"""
|
||||
corpus = load_corpus()
|
||||
lines = ["📋 语料库分类:"]
|
||||
for key, data in corpus.get("categories", {}).items():
|
||||
w = data.get("weight", 0)
|
||||
status = "✅" if w > 0 else "⛔"
|
||||
n = len(data.get("comments", [])) + len(data.get("templates", []))
|
||||
lines.append(f" {status} {key:20s} (权重{w}, {n}条) - {data.get('label','')}")
|
||||
return '\n'.join(lines)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(list_categories())
|
||||
print()
|
||||
for _ in range(5):
|
||||
print(f" → {get_comment('编程')}")
|
||||
Binary file not shown.
@@ -0,0 +1,109 @@
|
||||
-- anchor.db 数据库设计 v1.0
|
||||
-- 锚点规则 + 原子操作 + 检测日志 + 校准
|
||||
|
||||
-- ── 1. 锚点规则表 ──
|
||||
-- 每个状态由多条规则组成(同 state + logic_group = AND,不同 group = OR)
|
||||
CREATE TABLE anchor_rules (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
state_name TEXT NOT NULL, -- HOME / PLAYER / SEARCH / DETAIL / POPUP
|
||||
field TEXT NOT NULL, -- 检测字段: video_count, title, video_playing, card_count, url, has_search_input
|
||||
operator TEXT NOT NULL, -- eq / neq / gt / gte / lt / lte / contains / not_contains / regex
|
||||
value TEXT NOT NULL, -- 目标值(统一存字符串)
|
||||
logic_group INTEGER DEFAULT 0, -- 0=AND组, 同组内AND, 不同组OR
|
||||
active INTEGER DEFAULT 1, -- 1=启用, 0=停用
|
||||
created_at TEXT DEFAULT (datetime('now','localtime')),
|
||||
updated_at TEXT DEFAULT (datetime('now','localtime'))
|
||||
);
|
||||
|
||||
-- ── 2. 原子操作表 ──
|
||||
-- 每个原子操作 = 一个操作标识 + 执行方式 + 验证规则
|
||||
CREATE TABLE atomic_ops (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
op_name TEXT NOT NULL UNIQUE, -- click_card, swipe_down, like, search, go_home, dismiss_popup
|
||||
description TEXT, -- 操作描述
|
||||
action_type TEXT DEFAULT 'locator', -- locator / keyboard / evaluate / mouse
|
||||
target TEXT, -- CSS选择器 / 按键名 / JS代码
|
||||
verify_rule TEXT, -- 验证方式: video_src_change / video_count_gt / title_change
|
||||
retry_count INTEGER DEFAULT 3,
|
||||
fallback_op TEXT, -- 失败时降级的操作名
|
||||
created_at TEXT DEFAULT (datetime('now','localtime')),
|
||||
updated_at TEXT DEFAULT (datetime('now','localtime'))
|
||||
);
|
||||
|
||||
-- ── 3. 状态迁移表 ──
|
||||
-- 定义合法的状态转换(状态机)
|
||||
CREATE TABLE state_transitions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
from_state TEXT NOT NULL, -- HOME, PLAYER, SEARCH
|
||||
to_state TEXT NOT NULL, -- PLAYER, SEARCH, HOME
|
||||
via_op TEXT NOT NULL, -- click_card, swipe_down, search, go_home
|
||||
UNIQUE(from_state, to_state, via_op)
|
||||
);
|
||||
|
||||
-- ── 4. 检测日志表 ──
|
||||
-- 每次锚点检测的记录,用于成功率追踪
|
||||
CREATE TABLE detection_logs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
identity TEXT, -- douyin_01, douyin_02
|
||||
expected_state TEXT, -- 操作前预期的状态
|
||||
detected_state TEXT, -- 实际检测到的状态
|
||||
match INTEGER, -- 1=匹配, 0=不匹配
|
||||
anchor_snapshot TEXT, -- 锚点数据快照(JSON)
|
||||
screenshot_path TEXT, -- 截图路径
|
||||
timestamp TEXT DEFAULT (datetime('now','localtime'))
|
||||
);
|
||||
CREATE INDEX idx_detection_logs_match ON detection_logs(match, timestamp);
|
||||
|
||||
-- ── 5. 校准候选表 ──
|
||||
-- AI或自动校准产生的新规则,对比验证后决定是否采纳
|
||||
CREATE TABLE calibration_candidates (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
state_name TEXT, -- 哪个状态的规则
|
||||
source TEXT, -- ai / manual / auto_recovery
|
||||
old_rules TEXT, -- 旧规则JSON
|
||||
new_rules TEXT, -- 建议的新规则JSON
|
||||
test_count INTEGER DEFAULT 0, -- 测试次数
|
||||
test_pass INTEGER DEFAULT 0, -- 测试通过次数
|
||||
status TEXT DEFAULT 'pending', -- pending / accepted / rejected / testing
|
||||
created_at TEXT DEFAULT (datetime('now','localtime'))
|
||||
);
|
||||
|
||||
-- ── 初始数据:基于2026-05-06手动采集的锚点 ──
|
||||
|
||||
-- HOME 状态规则
|
||||
INSERT INTO anchor_rules (state_name, field, operator, value, logic_group) VALUES
|
||||
('HOME', 'video_count', 'eq', '1', 0),
|
||||
('HOME', 'title', 'contains', '抖音精选', 0);
|
||||
|
||||
-- PLAYER 状态规则(全屏播放)
|
||||
INSERT INTO anchor_rules (state_name, field, operator, value, logic_group) VALUES
|
||||
('PLAYER', 'video_count', 'gte', '3', 0),
|
||||
('PLAYER', 'video_playing', 'eq', 'true', 0);
|
||||
|
||||
-- SEARCH 状态规则
|
||||
INSERT INTO anchor_rules (state_name, field, operator, value, logic_group) VALUES
|
||||
('SEARCH', 'title', 'contains', '搜索', 0),
|
||||
('SEARCH', 'title', 'not_contains', '精选', 0);
|
||||
|
||||
-- SEARCH_PLAYER(从搜索结果进入的播放页)
|
||||
INSERT INTO anchor_rules (state_name, field, operator, value, logic_group) VALUES
|
||||
('SEARCH_PLAYER', 'video_count', 'gte', '3', 0),
|
||||
('SEARCH_PLAYER', 'video_playing', 'eq', 'true', 0),
|
||||
('SEARCH_PLAYER', 'url', 'contains', '/search/', 0);
|
||||
|
||||
-- 原子操作
|
||||
INSERT INTO atomic_ops (op_name, description, action_type, target, verify_rule, retry_count) VALUES
|
||||
('click_card', '点击首页视频卡片', 'locator', '.discover-video-card-item', 'video_count_gte_3', 3),
|
||||
('swipe_down', '下滑切换视频', 'keyboard', 'ArrowDown', 'video_src_change', 2),
|
||||
('like', '点赞当前视频', 'locator', '[data-e2e="like-count"]', 'like_count_change', 1),
|
||||
('search', '搜索关键词', 'evaluate', 'search_input', 'title_contains_搜索', 2),
|
||||
('go_home', '回到首页', 'locator', 'https://www.douyin.com/', 'video_count_eq_1', 2),
|
||||
('dismiss_popup','关闭弹窗', 'locator', 'button:text-is("我知道了")','popup_gone', 2);
|
||||
|
||||
-- 状态迁移
|
||||
INSERT INTO state_transitions (from_state, to_state, via_op) VALUES
|
||||
('HOME', 'PLAYER', 'click_card'),
|
||||
('PLAYER', 'PLAYER', 'swipe_down'),
|
||||
('PLAYER', 'HOME', 'go_home'),
|
||||
('HOME', 'SEARCH', 'search'),
|
||||
('SEARCH', 'SEARCH_PLAYER', 'click_card');
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,152 @@
|
||||
"""
|
||||
ui_layout.py — 抖音评论区 UI 布局参数 (v1.0.0)
|
||||
|
||||
设计原则(2026-05-09 基于真人操作反馈):
|
||||
- 不硬编码绝对坐标
|
||||
- 所有坐标相对于浏览器窗口右边缘、底边缘计算
|
||||
- 布局参数可配置(支持 per-account 覆盖)
|
||||
- 分为:DOM检测优先 → 坐标落位保底 → 状态验证确认
|
||||
|
||||
抖音评论区布局(KeyX 打开后):
|
||||
┌──────────────────────────────────────┐
|
||||
│ 评论面板 (固定宽度 ~500-600px) │
|
||||
│ │
|
||||
│ 评论列表区域 │
|
||||
│ │
|
||||
│ ┌──────────────────┬──┬──┬──────┐ │
|
||||
│ │ 输入框区域 │😊│@│ 发送 │ │
|
||||
│ │ 留下你的精彩评论吧 │ │ │ │ │
|
||||
│ └──────────────────┴──┴──┴──────┘ │
|
||||
└──────────────────────────────────────┘
|
||||
|
||||
关键:
|
||||
- 输入框在评论面板底部靠左
|
||||
- Emoji/ Mention/ Send 按钮在右侧
|
||||
- 输入框宽度 = 评论面板宽度 - 右侧按钮区(~130px)
|
||||
- 点击位置应落在输入框中间偏左区域
|
||||
"""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Dict, Optional, Tuple
|
||||
|
||||
__version__ = "1.0.0"
|
||||
|
||||
|
||||
# ── 默认布局参数(基于 702×783 窗口测量) ─────────────────────
|
||||
|
||||
DEFAULT_LAYOUT = {
|
||||
# 评论面板布局(相对于窗口左边缘)
|
||||
"comment_panel": {
|
||||
"width_ratio": 0.85, # 面板占窗口宽度的比例
|
||||
"left_offset": 0, # 面板距离左边缘(像素固定值)
|
||||
},
|
||||
|
||||
# 输入框位置(相对于窗口右边缘,因为按钮在右侧)
|
||||
"input_box": {
|
||||
# 从右边缘往左偏移量(校准值 2026-05-10)
|
||||
"from_right_margin": 223, # 702-479=223(实测 activeElement 中心)
|
||||
# 从上边缘往下偏移量
|
||||
"from_top_offset": "auto", # "auto" = 动态检测,fallback 使用下面固定值
|
||||
"from_top_fallback": 687, # 实测 activeElement Y 中心
|
||||
"height": 36, # 输入框高度
|
||||
},
|
||||
|
||||
# 按钮区域(从右边缘往左)
|
||||
"buttons": {
|
||||
"send_width": 60, # 发送按钮宽度
|
||||
"icon_width": 30, # emoji/@ 图标宽度
|
||||
"button_count": 3, # 按钮数量 (emoji + @ + 发送)
|
||||
},
|
||||
|
||||
# 发送键
|
||||
"send_key": {
|
||||
"method": "dom", # dom/alt_enter。DOM 优先找"发送"按钮
|
||||
"alt_enter_fallback": True,
|
||||
},
|
||||
|
||||
# 验证码弹窗
|
||||
"verify_code": {
|
||||
"input_placeholder": "验证码",
|
||||
"confirm_button_texts": ["确认", "提交", "验证"],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def get_layout(account_config: Optional[Dict] = None) -> dict:
|
||||
"""获取 UI 布局参数(支持 per-account 覆盖)
|
||||
|
||||
Args:
|
||||
account_config: accounts.yaml 中的账号配置,可包含 ui_layout 覆盖
|
||||
|
||||
Returns:
|
||||
合并后的布局参数字典
|
||||
"""
|
||||
layout = dict(DEFAULT_LAYOUT)
|
||||
if account_config and "ui_layout" in account_config:
|
||||
# 深度合并
|
||||
_deep_merge(layout, account_config["ui_layout"])
|
||||
return layout
|
||||
|
||||
|
||||
def _deep_merge(base: dict, override: dict):
|
||||
"""递归合并字典(不覆盖子键未指定的部分)"""
|
||||
for key, val in override.items():
|
||||
if key in base and isinstance(base[key], dict) and isinstance(val, dict):
|
||||
_deep_merge(base[key], val)
|
||||
else:
|
||||
base[key] = val
|
||||
|
||||
|
||||
def calc_input_position(window_width: int, window_height: int,
|
||||
layout: Optional[Dict] = None) -> Tuple[int, int]:
|
||||
"""计算评论输入框的目标点击坐标
|
||||
|
||||
Args:
|
||||
window_width: 浏览器窗口宽度 (如 702)
|
||||
window_height: 浏览器窗口高度 (如 783)
|
||||
|
||||
Returns:
|
||||
(x, y) — 基于 viewport 的绝对坐标
|
||||
"""
|
||||
if layout is None:
|
||||
layout = DEFAULT_LAYOUT
|
||||
|
||||
ib = layout["input_box"]
|
||||
|
||||
# X坐标:从窗口右边缘往左偏移
|
||||
x = window_width - ib["from_right_margin"]
|
||||
|
||||
# Y坐标
|
||||
y = ib.get("from_top_fallback", 720)
|
||||
|
||||
return (x, y)
|
||||
|
||||
|
||||
def calc_panel_rect(window_width: int, window_height: int,
|
||||
layout: Optional[Dict] = None) -> Tuple[int, int, int, int]:
|
||||
"""计算评论面板的包围矩形(用于 debug / 视觉确认)
|
||||
|
||||
Returns:
|
||||
(left, top, width, height)
|
||||
"""
|
||||
if layout is None:
|
||||
layout = DEFAULT_LAYOUT
|
||||
panel = layout["comment_panel"]
|
||||
pw = int(window_width * panel["width_ratio"])
|
||||
return (panel["left_offset"], 0, pw, window_height)
|
||||
|
||||
|
||||
def format_layout_summary(layout: Optional[Dict] = None) -> str:
|
||||
"""生成布局参数的摘要文本(用于日志)"""
|
||||
if layout is None:
|
||||
layout = DEFAULT_LAYOUT
|
||||
ib = layout["input_box"]
|
||||
btn = layout["buttons"]
|
||||
return (
|
||||
f"UI布局: "
|
||||
f"输入框右边缘偏移={ib['from_right_margin']}px, "
|
||||
f"Y坐标兜底={ib['from_top_fallback']}px, "
|
||||
f"按钮数={btn['button_count']}, "
|
||||
f"发送方式={layout['send_key']['method']}"
|
||||
)
|
||||
@@ -0,0 +1,387 @@
|
||||
# XHS 小红书原子操作手册
|
||||
|
||||
> 版本: v2 | 更新: 2026-05-24 | 基于 Playwright + Camoufox 实测验证
|
||||
|
||||
---
|
||||
|
||||
## 零、登录检测(关键!之前犯过错)
|
||||
|
||||
### 核心原则
|
||||
- XHS **没有右上角用户头像**,不要用头像/用户菜单判断登录态
|
||||
- XHS 未登录也能看到瀑布流卡片,有卡片不等于已登录
|
||||
|
||||
### 正确检测方法
|
||||
1. 检查页面是否有 `position:fixed` 且覆盖 >50% viewport 的大覆盖层(登录弹窗)
|
||||
2. 检查页面文字是否包含 "请打开小红书App扫码"(验证码页)
|
||||
3. 无覆盖层 + 有卡片(`section.note-item`) = 已登录
|
||||
|
||||
### 实现(已在 v2 测试脚本中验证)
|
||||
|
||||
```javascript
|
||||
// 登录检测 JS
|
||||
const overlays = [...document.querySelectorAll('div')].filter(el => {
|
||||
const s = window.getComputedStyle(el);
|
||||
const r = el.getBoundingClientRect();
|
||||
return s.position === 'fixed'
|
||||
&& r.width >= window.innerWidth * 0.5
|
||||
&& r.height >= window.innerHeight * 0.5;
|
||||
});
|
||||
const needsQr = document.body.innerText.includes('请打开小红书App扫码');
|
||||
const blocked = needsQr || overlays.length > 0;
|
||||
```
|
||||
|
||||
### 登录后确认
|
||||
- 覆盖层消失
|
||||
- 瀑布流卡片正常加载
|
||||
- (可选)点击卡片能进入 `note-detail-mask` 模式
|
||||
|
||||
## 一、基础定位规则
|
||||
|
||||
### 1.1 笔记卡片定位
|
||||
|
||||
| 元素 | 选择器 | 说明 |
|
||||
|------|--------|------|
|
||||
| 卡片容器 | `section.note-item` | 取 bounding rect 用此元素 |
|
||||
| 卡片链接 | `section.note-item a` | 取 href 用此元素(⚠️ bounding rect = 0) |
|
||||
| 卡片标题 | `.title, [class*=title]` | 嵌套在 section 内的标题 |
|
||||
|
||||
**关键坑**:`section.note-item a` 的 `getBoundingClientRect()` 返回 `(0,0,0,0)`。必须先取父级 `section.note-item` 的 rect,再从中取 `a` 标签提取 `href`。
|
||||
|
||||
### 1.2 笔记详情定位
|
||||
|
||||
| 元素 | 选择器 | 说明 |
|
||||
|------|--------|------|
|
||||
| 详情遮罩 | `.note-detail-mask` | SPA 详情页容器(fixed,全屏覆盖) |
|
||||
| 底部互动栏 | `.interactions` | 点赞/收藏/评论按钮的容器 |
|
||||
| 点赞按钮 | `span.like-wrapper` | ✅ 实测可用,x > 视口 30% |
|
||||
| 收藏按钮 | `span.collect-wrapper` | ✅ 实测可用,x > 视口 30% |
|
||||
| 评论入口 | `.comment-btn` 等 | 底部栏评论按钮 |
|
||||
|
||||
**关键规则**:页面中有大量 `like-wrapper`(评论区每条回复都有)。**区分底部栏 vs 评论区按钮的特征**:底部栏按钮的 `x` > 视口宽度 × 30%(约 210px 以上),评论区按钮在左侧(x ≈ 60~100px)。
|
||||
|
||||
---
|
||||
|
||||
## 二、锚点检测系统
|
||||
|
||||
### 2.1 `is_note_detail_mode()` — 笔记详情 vs 图片查看器
|
||||
|
||||
**三要素综合判断**:
|
||||
1. **has_mask** — 存在 `.note-detail-mask` 元素
|
||||
2. **has_interact_bar** — 底部互动栏按钮可见(like-wrapper 的 x > viewW×0.3)
|
||||
3. **has_note_url** — URL 匹配 `/explore/{20位以上十六进制ID}`
|
||||
|
||||
```
|
||||
is_detail = has_mask && has_interact_bar && has_note_url
|
||||
```
|
||||
|
||||
**四种状态**:
|
||||
| 状态 | 判断 | 说明 |
|
||||
|------|------|------|
|
||||
| 笔记详情模式 | mask + 互动栏 + URL | ✅ 可以执行互动 |
|
||||
| 疑似图片查看器 | 有 mask 但无互动栏 | ❌ 需退出或重新导航 |
|
||||
| 不在详情页 | 无 mask | ❌ 回到首页 |
|
||||
| 未知状态 | 其他 | ❌ 截图+兜底 |
|
||||
|
||||
### 2.2 其他锚点
|
||||
|
||||
| 锚点类型 | 检测方式 |
|
||||
|----------|----------|
|
||||
| 首页已加载 | `section.note-item` 存在 |
|
||||
| 笔记详情锚点 | `.note-detail-mask` 存在 |
|
||||
| 评论区打开 | `.comment-section` 存在 |
|
||||
| 搜索页 | `.search-result` 存在 |
|
||||
| 页面加载完成 | `#app` 存在 |
|
||||
|
||||
---
|
||||
|
||||
## 三、原子操作(按流程顺序)
|
||||
|
||||
### 3.1 导航 `goto_home()`
|
||||
|
||||
```
|
||||
前置状态: 浏览器刚启动(about:blank)
|
||||
操作: page.goto("https://www.xiaohongshu.com/explore")
|
||||
后置状态: URL = /explore, 页面渲染完成
|
||||
锚点: #app 存在
|
||||
超时: 20s → 30s fallback
|
||||
```
|
||||
|
||||
### 3.2 关闭登录弹窗 `dismiss_login_modal()`
|
||||
|
||||
```
|
||||
前置状态: 页面已加载,可能出现登录弹窗
|
||||
操作: JS: display:none 所有 modal/dialog/mask/position:fixed+zIndex>100
|
||||
后置状态: 弹窗不可见
|
||||
锚点: 无(可选)
|
||||
```
|
||||
|
||||
### 3.3 获取卡片列表 `get_note_cards()`
|
||||
|
||||
```
|
||||
前置状态: 首页已加载
|
||||
操作: JS: querySelectorAll('section.note-item') 取 index/href/rect
|
||||
后置状态: 返回 List[dict](不为空)
|
||||
返回: [{index, href, title, rect:{x,y,w,h}}]
|
||||
⚠️ 注意: 过滤 rect.w < 10 的无效卡片
|
||||
```
|
||||
|
||||
### 3.4 点击卡片进入详情 `click_note_card()`
|
||||
|
||||
```
|
||||
前置状态: 首页已加载,有卡片数据
|
||||
操作:
|
||||
1. 获取 section.note-item 的 bounding rect
|
||||
2. scrollIntoView({block: 'center'})
|
||||
3. 重新获取 rect(滚动后坐标变化)
|
||||
4. page.mouse.move(cx, cy, steps=5~12) ← Playwright 原生鼠标
|
||||
5. 单次 page.mouse.click(cx, cy) ← ⚡ 非双击!
|
||||
6. 等待 3~5s(SPA 过渡)
|
||||
7. 锚点: is_note_detail_mode()
|
||||
- ✅ → 返回 URL
|
||||
- ❌ → fallback page.goto(href)
|
||||
后置状态: 笔记详情页
|
||||
锚点: is_note_detail_mode() = true
|
||||
⚠️ 注意:
|
||||
- 用 section.note-item 取 rect(不是 a 标签)
|
||||
- 单次单击(双击触发图片查看器)
|
||||
- 随机选卡片(非总点第一张)
|
||||
```
|
||||
|
||||
### 3.5 分析互动按钮 `analyze_interact_page()`
|
||||
|
||||
```
|
||||
前置状态: 笔记详情页(is_note_detail_mode = true)
|
||||
操作: JS: 取 like-wrapper / collect-wrapper 位置和可见性
|
||||
返回: { like: {x,y,w,h,visible,isActive,cls,text},
|
||||
collect: {x,y,w,h,visible,isActive,cls,text} }
|
||||
⚠️ 注意: 用 x > viewW*0.3 过滤出底部栏按钮
|
||||
```
|
||||
|
||||
### 3.6 L 形鼠标移动 `mouse_move_l_shape()`
|
||||
|
||||
```
|
||||
前置状态: 任何(鼠标在当前位置)
|
||||
操作: 三段式 page.mouse.move():
|
||||
L1: (cx, cy) → (cx, safeY=100) — 垂直上移到安全区
|
||||
L2: (cx, safeY) → (targetX, safeY) — 水平移动到目标上方
|
||||
L3: (targetX, safeY) → (targetX, targetY) — 垂直下移到目标
|
||||
每段 steps=3~4,总 steps≈8~12
|
||||
后置状态: 鼠标在目标坐标
|
||||
⚠️ 注意:
|
||||
- 安全区 Y=100 在输入框上方
|
||||
- 避免直线移动穿过评论区输入框
|
||||
```
|
||||
|
||||
### 3.7 点赞 `like()`
|
||||
|
||||
```
|
||||
前置状态: 笔记详情页,底部栏可见
|
||||
操作:
|
||||
1. get_bottom_bar_buttons_js() → 取 like 按钮位置
|
||||
2. 如果 isActive → 跳过(已点赞)
|
||||
3. 如果不可见 → scrollIntoView({block:'center'})
|
||||
4. mouse_move_l_shape(like.x, like.y)
|
||||
5. page.mouse.click(like.x, like.y)
|
||||
6. 等待 1.5~2.5s
|
||||
7. 验证: isActive 变化 → ✅
|
||||
8. 失败则重试一次
|
||||
后置状态: 已点赞(class 含 like-active)
|
||||
锚点: like-wrapper.isActive = true
|
||||
⚠️ 注意:
|
||||
- 使用 page.mouse.click(非 element.click / JS dispatchEvent)
|
||||
- 必须先 scrollIntoView 确保按钮在视口内
|
||||
```
|
||||
|
||||
### 3.8 收藏 `collect()`
|
||||
|
||||
```
|
||||
前置状态: 笔记详情页,底部栏可见
|
||||
操作:
|
||||
1. get_bottom_bar_buttons_js() → 取 collect 按钮位置
|
||||
2. 如果 isActive → 跳过(已收藏)
|
||||
3. 如果不可见 → scrollIntoView({block:'center'})
|
||||
4. mouse_move_l_shape(collect.x, collect.y)
|
||||
5. page.mouse.click(collect.x, collect.y)
|
||||
6. 等待 1.5~2.5s
|
||||
7. 验证: isActive 变化 或 计数增加 → ✅
|
||||
8. 失败则重试一次
|
||||
后置状态: 已收藏(class 含 active)
|
||||
锚点: collect-wrapper.isActive = true 或 text 计数 +1
|
||||
⚠️ 注意: 和点赞流程一致,共用 L 形路径
|
||||
```
|
||||
|
||||
### 3.9 拟人滚动 `scroll_feed_human()`
|
||||
|
||||
```
|
||||
前置状态: 首页/发现页
|
||||
操作:
|
||||
循环 N 次(默认 2~3 屏):
|
||||
page.mouse.wheel(0, dist=200~500) ← 鼠标滚轮
|
||||
随机停顿 1~3s(40% 概率)
|
||||
后置状态: 瀑布流已滚动
|
||||
⚠️ 注意:
|
||||
- 用 page.mouse.wheel(非 JS 滚动)
|
||||
- 间隔随机抖动(非固定间隔)
|
||||
- 不采用双重滚动(mouse.wheel + JS scrollBy)
|
||||
```
|
||||
|
||||
### 3.10 浏览笔记详情 `browse_note_detail()`
|
||||
|
||||
```
|
||||
前置状态: 笔记详情页
|
||||
操作:
|
||||
- 视频笔记: 等待 video 播放,随机观看时长
|
||||
- 图文笔记: 分段 scrollBy(50~150px),每段停顿 1~2.5s
|
||||
后置状态: 笔记内容已浏览
|
||||
返回: 实际浏览秒数
|
||||
```
|
||||
|
||||
### 3.11 返回首页 `go_back_to_home()`
|
||||
|
||||
```
|
||||
前置状态: 笔记详情页
|
||||
操作:
|
||||
方式1: page.go_back()
|
||||
方式2: page.goto(/explore)
|
||||
后置状态: URL = /explore
|
||||
锚点: section.note-item 存在
|
||||
```
|
||||
|
||||
### 3.12 评论输入 `comment()`
|
||||
|
||||
```
|
||||
前置状态: 笔记详情页,底部栏可见
|
||||
操作:
|
||||
1. find_comment_entry() → 取 collect 右侧第一个按钮
|
||||
- 排除 like-wrapper / collect-wrapper(按 class 过滤)
|
||||
- 按 x 坐标排序,取第 3 个元素
|
||||
2. L 形路径移动到评论入口 → 单击打开评论区
|
||||
3. 等待评论区渲染(2s)
|
||||
4. 定位输入框:
|
||||
- querySelectorAll('textarea, [contenteditable], input')
|
||||
- 排除 y < 150 的元素(顶部搜索栏)
|
||||
- 优先选 viewport 下半部分、高度 > 25px 的元素
|
||||
5. L 形路径移动到输入框 → 单击聚焦
|
||||
6. JS: inp.focus() 补充
|
||||
7. pbcopy(comment_text)
|
||||
8. page.keyboard.press("Meta+v")
|
||||
9. page.keyboard.press("Enter")
|
||||
10. 验证: 输入框.value/textContent 长度 = 0 → ✅ 已发送
|
||||
后置状态: 评论已发送
|
||||
锚点: 输入框已清空
|
||||
⚠️ 注意:
|
||||
- 输入框是 <p contenteditable="true">,不是 <textarea>
|
||||
- Meta+V 后 textContent 可能为空,跳过中间验证
|
||||
- 用 "输入框清空" 作为发送验证标准
|
||||
- 发送失败时可能有验证码弹窗
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 四、评论状态机
|
||||
|
||||
### 4.1 状态定义
|
||||
|
||||
```
|
||||
closed → panel_open → input_focused → text_entered → sent → verified
|
||||
```
|
||||
|
||||
### 4.2 各步骤
|
||||
|
||||
| 步骤 | 方法 | 操作 | 锚点 |
|
||||
|------|------|------|------|
|
||||
| 打开评论区 | `open_panel()` | 点击 comment-entry 或滚动到评论区 | `.comment-section` 存在 |
|
||||
| 聚焦输入框 | `focus_input()` | click 输入框 或 JS focus() | state = input_focused |
|
||||
| 输入文本 | `enter_text()` | pbcopy + Meta+V(系统级) | 输入框包含输入文本 |
|
||||
| 发送 | `send()` | Enter 或点击发送按钮 | 输入框清空 |
|
||||
| 验证 | `verify()` | 检查评论区是否出现评论 | 输入框清空 或 评论可见 |
|
||||
|
||||
### 4.3 关键规则
|
||||
|
||||
- **输入方式**:pbcopy(剪贴板)→ `Meta+V`(系统级粘贴),和抖音一致
|
||||
- ⚠️ XHS 输入框可能是 `contenteditable div` 或 `input`,pbcopy + Meta+V 对两种都有效
|
||||
**发送方式**:优先点击发送按钮,兜底 Enter 键
|
||||
**验证**:发送后检查输入框清空(比检查评论列表更可靠)
|
||||
**评论入口定位**:collect-wrapper 右侧第一个非 like/collect 元素
|
||||
|
||||
### 4.4 XHS 评论输入注意事项
|
||||
|
||||
- **评论入口按钮**:位于底部互动栏最右侧(collect-wrapper 的右侧),可能显示评论计数
|
||||
- **输入框类型**:XHS 使用 `<p contenteditable="true">` 作为评论输入框(不是 `<textarea>` 或 `<input>`)
|
||||
- **输入验证**:`value`/`textContent` 在 Meta+V 后可能为空,跳过输入验证,直接发送后检查清空状态
|
||||
- **搜索框干扰**:顶部搜索栏也是 `<input>`,必须用 `y < 150` 排除
|
||||
- **发送验证**:输入框清空 = 发送成功(比评论区出现评论更可靠)
|
||||
|
||||
---
|
||||
|
||||
## 五、交互引擎选择
|
||||
|
||||
| 操作类型 | 推荐方式 | 说明 |
|
||||
|----------|----------|------|
|
||||
| 点击卡片 | `page.mouse.click()` | 单次单击 ✅ 实测最佳 |
|
||||
| 点赞/收藏 | `page.mouse.click()` | 单次单击 ✅ 实测最佳 |
|
||||
| 搜索框输入 | `search_input.fill()` | 常规文本输入无防检测 |
|
||||
| 评论输入 | `pbcopy + Meta+V` | contenteditable div 专用 |
|
||||
| 键盘事件 | `page.keyboard.press()` | Enter / Meta+V / Escape |
|
||||
| 滚动 | `page.mouse.wheel()` | 拟人操作,优于 JS 滚动 |
|
||||
|
||||
**为什么不用 element.click() / JS dispatchEvent:**
|
||||
- XHS SPA 可能屏蔽 JS 事件
|
||||
- element.click() 不产生真实的 mousedown/mouseup/mousemove 事件序列
|
||||
- page.mouse API 产生完整的事件序列,和真人操作一致
|
||||
|
||||
---
|
||||
|
||||
## 六、已知坑 & 解决方案
|
||||
|
||||
| 问题 | 现象 | 解决方案 |
|
||||
|------|------|----------|
|
||||
| a 标签 zero-rect | card.querySelector('a').getBoundingClientRect() = (0,0,0,0) | 用父级 section.note-item 取 rect |
|
||||
| 双击触发图片查看器 | 第 2 次 click 打开全屏图片 | 单次单击 |
|
||||
| like-wrapper 过多 | 页面中有大量 like-wrapper(评论区每条回复都有) | 用 x > viewW×0.3 过滤 |
|
||||
| 鼠标路径穿过输入框 | 直线移动会经过评论区输入框 | L 形路径 |
|
||||
| SPA 过渡延迟 | click 后页面需要时间加载 | 等待 3~5s |
|
||||
| 图片查看器状态 | 没有互动栏、URL 可能不变 | is_note_detail_mode() 三要素判断 |
|
||||
| 评论按钮误判 | like-wrapper 文本是数字,被误判为评论按钮 | 取 collect 右侧第一个非 like/collect 元素 |
|
||||
| 输入框选中搜索栏 | 顶部搜索栏 input 优先级高 | 排除 y < 150 的元素 |
|
||||
| nohup 模式 EOF | input() 在后台无 TTY | 替换为 asyncio.sleep() |
|
||||
| 卡片 href 指向 profile | 部分卡片链接到用户主页 | 用正则 /\/explore\/[a-f0-9]{20,}/ 过滤 |
|
||||
|
||||
---
|
||||
|
||||
## 七、目录结构
|
||||
|
||||
```
|
||||
ops/xhs/
|
||||
├── __init__.py — 模块导出
|
||||
├── ATOMIC_OPS.md — 本文档
|
||||
├── selectors.py — DOM 选择器 + JS 辅助函数
|
||||
├── browse.py — 浏览类操作
|
||||
└── interact.py — 交互类操作(点赞/收藏/关注/评论)
|
||||
```
|
||||
|
||||
## 八、自动化运行
|
||||
|
||||
### CLI 入口
|
||||
```bash
|
||||
# 单账号
|
||||
matrix nurture run -a xhs_01 -r 10
|
||||
|
||||
# 三账号并行
|
||||
matrix nurture run -a xhs_01 -a xhs_02 -a xhs_03 -r 10
|
||||
```
|
||||
|
||||
### 主控脚本
|
||||
`~/workbuddy-agent-os/agent-sync/05_tools/07_matrix/scripts/nurture_master.sh`
|
||||
|
||||
执行流程: 抖音 1h → 小红书 1h → 完成
|
||||
|
||||
### 定时任务
|
||||
WorkBuddy 自动化 "三账号每日养号" 每日 10:00 触发,调用 nurture_master.sh
|
||||
|
||||
### 框架入口
|
||||
`matrix_modules/nurture/runner.py` 中的 `nurture_xhs_loop()` 函数已封装完整循环逻辑:
|
||||
- 瀑布流浏览 → 点击卡片 → 浏览内容 → 随机互动 → 返回首页 → 搜索发现
|
||||
- 互动概率: 20% 点赞, 12.5% 收藏, 2.5% 关注
|
||||
- 每 3 轮评论一次,每 2 轮搜索一次
|
||||
- 连续 3 轮失败自动暂停 + 截图
|
||||
@@ -0,0 +1,6 @@
|
||||
"""
|
||||
小红书操作模块
|
||||
"""
|
||||
from . import browse, interact, selectors
|
||||
|
||||
__all__ = ["browse", "interact", "selectors"]
|
||||
@@ -0,0 +1,440 @@
|
||||
"""
|
||||
小红书 — 浏览类操作
|
||||
基于 2026-05-20 DOM 分析结果
|
||||
|
||||
核心差异(vs 抖音):
|
||||
- 双列瀑布流 → 需要定位具体卡片
|
||||
- 笔记详情页 → 图文为主,可能有视频
|
||||
- 搜索发现 → 顶部搜索框
|
||||
"""
|
||||
import random
|
||||
import asyncio
|
||||
from typing import List, Optional
|
||||
|
||||
from .selectors import (
|
||||
NOTE_CARD, NOTE_CARD_COVER, NOTE_CARD_IMG,
|
||||
SEARCH_INPUT, SEARCH_BUTTON,
|
||||
ANCHORS, get_note_links_js, get_note_cards_js, dismiss_login_modal_js,
|
||||
find_refresh_button_js, find_qr_wall_back_button_js
|
||||
)
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 首页浏览
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
async def goto_home(page):
|
||||
"""回到小红书首页/发现页(锚点等待 + 超时兜底)"""
|
||||
try:
|
||||
await page.goto("https://www.xiaohongshu.com/explore", timeout=20000, wait_until="commit")
|
||||
# 锚点等待: 等 #app 出现(SPA 水合)
|
||||
try:
|
||||
await page.wait_for_selector("#app", timeout=8000)
|
||||
except Exception:
|
||||
pass
|
||||
await asyncio.sleep(2) # 额外等待渲染稳定
|
||||
except Exception:
|
||||
# 兜底: 直接导航,不抛异常
|
||||
await page.goto("https://www.xiaohongshu.com/explore", timeout=30000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(5)
|
||||
return "home"
|
||||
|
||||
|
||||
async def dismiss_login_modal(page):
|
||||
"""关闭登录弹窗(未登录状态下浏览需要)"""
|
||||
try:
|
||||
result = await asyncio.wait_for(
|
||||
page.evaluate(dismiss_login_modal_js()),
|
||||
timeout=8
|
||||
)
|
||||
await asyncio.sleep(0.5)
|
||||
return result
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
async def get_note_cards(page) -> List[dict]:
|
||||
"""获取当前页面所有笔记卡片信息"""
|
||||
try:
|
||||
cards = await asyncio.wait_for(
|
||||
page.evaluate(get_note_cards_js()),
|
||||
timeout=10
|
||||
)
|
||||
return [c for c in cards if c.get("href")]
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
|
||||
async def click_note_card(page, index: int = None, max_retries: int = 3) -> Optional[str]:
|
||||
"""
|
||||
点击笔记卡片进入详情页
|
||||
|
||||
Args:
|
||||
index: 卡片索引(0-based),None 时随机选择
|
||||
max_retries: 误触作者主页时最大重试次数
|
||||
|
||||
Returns:
|
||||
笔记 URL,失败返回 None
|
||||
"""
|
||||
for attempt in range(max_retries):
|
||||
cards = await get_note_cards(page)
|
||||
if not cards:
|
||||
return None
|
||||
|
||||
# 随机选择卡片(避免总是点第一个)
|
||||
if index is None:
|
||||
# 优先选前 4 个,避免滚动到底部
|
||||
max_idx = min(3, len(cards) - 1)
|
||||
idx = random.randint(0, max_idx)
|
||||
else:
|
||||
idx = index
|
||||
|
||||
if idx >= len(cards):
|
||||
idx = random.randint(0, len(cards) - 1)
|
||||
|
||||
card = cards[idx]
|
||||
href = card.get("href")
|
||||
if not href:
|
||||
return None
|
||||
|
||||
# 记录点击前 tab 数量
|
||||
context = page.context
|
||||
tabs_before = len(context.pages)
|
||||
original_page = page
|
||||
|
||||
# 使用 Playwright 点击(模拟真人)
|
||||
try:
|
||||
# 先尝试通过索引定位元素
|
||||
cards_els = await page.query_selector_all(NOTE_CARD)
|
||||
if idx < len(cards_els):
|
||||
await cards_els[idx].click()
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# 检测是否误触作者主页(新标签页)
|
||||
tabs_after = len(context.pages)
|
||||
if tabs_after > tabs_before:
|
||||
# 新 tab 打开了作者主页 — 关闭新 tab,回到原 tab
|
||||
new_tab = context.pages[-1]
|
||||
new_url = new_tab.url
|
||||
await new_tab.close()
|
||||
print(f" [browse] 误触作者主页(新标签页已关闭): {new_url[:50]}")
|
||||
# 切回原 tab
|
||||
page = original_page
|
||||
if attempt < max_retries - 1:
|
||||
print(f" [browse] 重试点击笔记 ({attempt + 2}/{max_retries})...")
|
||||
await asyncio.sleep(1)
|
||||
continue
|
||||
else:
|
||||
return None
|
||||
|
||||
return page.url
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# fallback: 直接导航到链接
|
||||
try:
|
||||
await page.goto(href, timeout=15000, wait_until="commit")
|
||||
await asyncio.sleep(2)
|
||||
return page.url
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
async def scroll_feed(page, distance: int = None):
|
||||
"""
|
||||
滚动瀑布流(单方式: 鼠标滚轮,避免双重滚动)
|
||||
|
||||
Args:
|
||||
distance: 滚动距离(px),None 时随机
|
||||
"""
|
||||
dist = distance or random.randint(300, 800)
|
||||
|
||||
# 鼠标滚轮(拟人)
|
||||
try:
|
||||
await page.mouse.wheel(0, dist)
|
||||
await asyncio.sleep(random.uniform(0.5, 1.5))
|
||||
except Exception:
|
||||
# fallback: JS 滚动
|
||||
await page.evaluate(f"window.scrollBy(0, {dist})")
|
||||
await asyncio.sleep(random.uniform(0.5, 1.5))
|
||||
|
||||
return dist
|
||||
|
||||
|
||||
async def scroll_feed_human(page, screens: int = 2):
|
||||
"""
|
||||
拟人化滚动:分多次小滚动,中间随机停顿
|
||||
|
||||
Args:
|
||||
screens: 滚动几屏
|
||||
"""
|
||||
for _ in range(screens):
|
||||
# 每次滚动一小段
|
||||
dist = random.randint(200, 500)
|
||||
await scroll_feed(page, dist)
|
||||
|
||||
# 随机停顿(模拟阅读)
|
||||
if random.random() < 0.4:
|
||||
pause = random.uniform(1.0, 3.0)
|
||||
await asyncio.sleep(pause)
|
||||
|
||||
return screens
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 笔记详情页浏览
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
async def browse_note_detail(page, duration: float = None):
|
||||
"""
|
||||
在笔记详情页浏览内容(图文/视频自适应)
|
||||
|
||||
Args:
|
||||
duration: 浏览时长(秒),None 时随机 4~12 秒
|
||||
|
||||
Returns:
|
||||
实际浏览秒数
|
||||
"""
|
||||
watch = duration or random.uniform(4, 12)
|
||||
|
||||
# 检测是否是视频笔记
|
||||
has_video = await asyncio.wait_for(page.evaluate("""
|
||||
() => {
|
||||
const v = document.querySelector('video');
|
||||
if (!v) return false;
|
||||
const rect = v.getBoundingClientRect();
|
||||
return rect.width > 100 && rect.height > 100;
|
||||
}
|
||||
"""), timeout=8)
|
||||
|
||||
if has_video:
|
||||
# 视频笔记:看视频,不滚动
|
||||
# 等待视频加载
|
||||
await asyncio.sleep(random.uniform(1, 2))
|
||||
return watch
|
||||
else:
|
||||
# 图文笔记:缓慢滚动模拟阅读
|
||||
steps = int(watch / 2)
|
||||
for _ in range(steps):
|
||||
await page.evaluate(f"window.scrollBy(0, {random.randint(50, 150)})")
|
||||
await asyncio.sleep(random.uniform(1.0, 2.5))
|
||||
return watch
|
||||
|
||||
|
||||
async def go_back_to_home(page):
|
||||
"""从详情页返回首页"""
|
||||
try:
|
||||
# 方式1: 浏览器返回
|
||||
await page.go_back(timeout=10000, wait_until="commit")
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# 验证是否回到首页
|
||||
cards = await get_note_cards(page)
|
||||
if cards:
|
||||
return True
|
||||
|
||||
# 方式2: 直接导航
|
||||
await goto_home(page)
|
||||
return True
|
||||
except Exception:
|
||||
# 兜底: 直接导航
|
||||
await goto_home(page)
|
||||
return True
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 物理按钮操作(鼠标模拟)
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
async def _l_shaped_click(page, target_x: int, target_y: int, desc: str = "") -> bool:
|
||||
"""
|
||||
L 型鼠标路径点击 — 模拟真人鼠标移动轨迹
|
||||
|
||||
轨迹: 起点(随机偏移) → 水平移动到目标 x → 垂直移动到目标 y → 点击
|
||||
"""
|
||||
import random
|
||||
try:
|
||||
viewport = page.viewport_size or {"width": 702, "height": 783}
|
||||
# 起点随机偏移
|
||||
start_x = target_x + random.randint(-150, 150)
|
||||
start_y = target_y + random.randint(-100, 50)
|
||||
# 限制在视口内
|
||||
start_x = max(50, min(start_x, viewport["width"] - 50))
|
||||
start_y = max(50, min(start_y, viewport["height"] - 50))
|
||||
|
||||
# 移动鼠标到起点
|
||||
await page.mouse.move(start_x, start_y)
|
||||
await asyncio.sleep(random.uniform(0.05, 0.15))
|
||||
|
||||
# L 型路径: 先水平移到目标 x
|
||||
steps = random.randint(5, 10)
|
||||
dx = (target_x - start_x) / steps
|
||||
for i in range(steps):
|
||||
await page.mouse.move(start_x + dx * (i + 1), start_y)
|
||||
await asyncio.sleep(random.uniform(0.01, 0.03))
|
||||
|
||||
# 再垂直移到目标 y
|
||||
steps_v = random.randint(3, 8)
|
||||
dy = (target_y - start_y) / steps_v
|
||||
for i in range(steps_v):
|
||||
await page.mouse.move(target_x, start_y + dy * (i + 1))
|
||||
await asyncio.sleep(random.uniform(0.01, 0.03))
|
||||
|
||||
# 微抖动(模拟手指不稳)
|
||||
jitter_x = target_x + random.randint(-2, 2)
|
||||
jitter_y = target_y + random.randint(-2, 2)
|
||||
await page.mouse.move(jitter_x, jitter_y)
|
||||
await asyncio.sleep(random.uniform(0.05, 0.1))
|
||||
|
||||
# 点击
|
||||
await page.mouse.click(jitter_x, jitter_y)
|
||||
return True
|
||||
except Exception as e:
|
||||
if desc:
|
||||
pass # 静默失败
|
||||
return False
|
||||
|
||||
|
||||
async def click_refresh_button(page) -> bool:
|
||||
"""
|
||||
点击小红书瀑布流页面右下角的刷新 FAB 按钮
|
||||
|
||||
Returns:
|
||||
True=点击成功, False=未找到按钮
|
||||
"""
|
||||
try:
|
||||
result = await asyncio.wait_for(
|
||||
page.evaluate(find_refresh_button_js()),
|
||||
timeout=10
|
||||
)
|
||||
if not result or not result.get("found"):
|
||||
return False
|
||||
|
||||
x, y = result["x"], result["y"]
|
||||
return await _l_shaped_click(page, x, y, desc="refresh")
|
||||
except asyncio.TimeoutError:
|
||||
return False
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
async def click_qr_wall_back_button(page) -> bool:
|
||||
"""
|
||||
检测 QR 检测墙并点击"返回首页"按钮
|
||||
|
||||
Returns:
|
||||
True=检测到QR墙并成功点击返回, False=未检测到QR墙或点击失败
|
||||
"""
|
||||
try:
|
||||
result = await asyncio.wait_for(
|
||||
page.evaluate(find_qr_wall_back_button_js()),
|
||||
timeout=10
|
||||
)
|
||||
if not result or not result.get("found"):
|
||||
return False
|
||||
|
||||
x, y = result["x"], result["y"]
|
||||
btn_text = result.get("text", "")
|
||||
success = await _l_shaped_click(page, x, y, desc="qr_back")
|
||||
return success
|
||||
except asyncio.TimeoutError:
|
||||
return False
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 搜索发现
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
# 小红书搜索关键词池(生活方式相关)
|
||||
SEARCH_KEYWORDS = [
|
||||
"穿搭", "美食", "旅行", "护肤", "家居", "健身",
|
||||
"书单", "电影", "咖啡", "探店", "拍照", "收纳",
|
||||
"早餐", "减脂", "美甲", "发型", "礼物", "周末",
|
||||
"自律", "效率", "省钱", "租房", "装修", "宠物",
|
||||
]
|
||||
|
||||
|
||||
async def search(page, keyword: str = None) -> str:
|
||||
"""
|
||||
搜索关键词
|
||||
|
||||
Args:
|
||||
keyword: 搜索词,None 时随机选择
|
||||
|
||||
Returns:
|
||||
实际搜索的关键词
|
||||
"""
|
||||
kw = keyword or random.choice(SEARCH_KEYWORDS)
|
||||
|
||||
try:
|
||||
# 定位搜索框
|
||||
search_input = await page.query_selector(SEARCH_INPUT)
|
||||
if search_input:
|
||||
await search_input.click()
|
||||
await asyncio.sleep(0.5)
|
||||
await search_input.fill(kw)
|
||||
await asyncio.sleep(0.3)
|
||||
|
||||
# 按回车搜索
|
||||
await search_input.press("Enter")
|
||||
await asyncio.sleep(3)
|
||||
return kw
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# fallback: URL 直接搜索
|
||||
try:
|
||||
encoded_kw = kw.replace(" ", "%20")
|
||||
await page.goto(
|
||||
f"https://www.xiaohongshu.com/search_result?keyword={encoded_kw}",
|
||||
timeout=15000, wait_until="commit"
|
||||
)
|
||||
await asyncio.sleep(3)
|
||||
return kw
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
async def click_search_result(page, index: int = None) -> Optional[str]:
|
||||
"""
|
||||
点击搜索结果中的笔记
|
||||
|
||||
Args:
|
||||
index: 结果索引,None 时随机
|
||||
|
||||
Returns:
|
||||
笔记 URL
|
||||
"""
|
||||
try:
|
||||
results = await page.query_selector_all(".note-item, [class*=search-result] a")
|
||||
if not results:
|
||||
return None
|
||||
|
||||
idx = index if index is not None else random.randint(0, min(3, len(results) - 1))
|
||||
if idx < len(results):
|
||||
await results[idx].click()
|
||||
await asyncio.sleep(2)
|
||||
return page.url
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return None
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 锚点验证
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
async def check_anchor(page, anchor_type: str, timeout: float = 3.0) -> bool:
|
||||
"""检测页面锚点状态"""
|
||||
try:
|
||||
selector = ANCHORS.get(anchor_type)
|
||||
if not selector:
|
||||
return False
|
||||
|
||||
el = await page.query_selector(selector)
|
||||
return el is not None
|
||||
except Exception:
|
||||
return False
|
||||
@@ -0,0 +1,462 @@
|
||||
"""
|
||||
小红书 — 交互类操作(点赞、收藏、关注、评论)
|
||||
基于 2026-05-20 DOM 分析 + 抖音踩坑经验
|
||||
|
||||
防护设计:
|
||||
- 每个操作独立 try/except,失败不崩流程
|
||||
- 操作后验证状态变化(防假点击)
|
||||
- 评论状态机: closed → panel_open → input_focused → text_entered → sent
|
||||
"""
|
||||
import json
|
||||
import random
|
||||
import asyncio
|
||||
import subprocess
|
||||
from typing import Optional
|
||||
|
||||
from .selectors import (
|
||||
LIKE_BUTTON, COLLECT_BUTTON, FOLLOW_BUTTON,
|
||||
COMMENT_ENTRY, COMMENT_INPUT, COMMENT_SEND,
|
||||
ANCHORS
|
||||
)
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 点赞
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
async def like(page) -> bool:
|
||||
"""
|
||||
点赞当前笔记(含状态验证)
|
||||
|
||||
Returns:
|
||||
是否成功
|
||||
"""
|
||||
try:
|
||||
# 记录点击前的点赞状态
|
||||
was_liked = await page.evaluate("""
|
||||
() => {
|
||||
const btn = document.querySelector('.like-btn, [class*=like-btn], .interaction-like');
|
||||
if (!btn) return 'not_found';
|
||||
const cls = btn.className || btn.classList?.toString() || '';
|
||||
return cls.includes('active') || cls.includes('liked') || cls.includes('sel') ? 'liked' : 'not_liked';
|
||||
}
|
||||
""")
|
||||
if was_liked == 'liked':
|
||||
return True # 已经是点赞状态,无需重复操作
|
||||
|
||||
# 方式1: Playwright locator 点击
|
||||
btn = await page.query_selector(LIKE_BUTTON)
|
||||
if btn:
|
||||
await btn.click()
|
||||
await asyncio.sleep(0.5)
|
||||
# 验证状态变化
|
||||
now_liked = await page.evaluate("""
|
||||
() => {
|
||||
const btn = document.querySelector('.like-btn, [class*=like-btn], .interaction-like');
|
||||
if (!btn) return false;
|
||||
const cls = btn.className || btn.classList?.toString() || '';
|
||||
return cls.includes('active') || cls.includes('liked') || cls.includes('sel');
|
||||
}
|
||||
""")
|
||||
return bool(now_liked)
|
||||
|
||||
# 方式2: JS 直接点击
|
||||
result = await page.evaluate("""
|
||||
() => {
|
||||
const btn = document.querySelector('.like-btn, [class*=like-btn], .interaction-like');
|
||||
if (btn) { btn.click(); return true; }
|
||||
// 兜底: 找包含点赞图标的按钮
|
||||
const allBtns = [...document.querySelectorAll('button, div[role=button]')];
|
||||
const likeBtn = allBtns.find(b => b.textContent.includes('赞') || b.innerHTML.includes('like'));
|
||||
if (likeBtn) { likeBtn.click(); return true; }
|
||||
return false;
|
||||
}
|
||||
""")
|
||||
return bool(result)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 收藏
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
async def collect(page) -> bool:
|
||||
"""收藏当前笔记(含状态验证)"""
|
||||
try:
|
||||
# 检查是否已收藏
|
||||
was_collected = await page.evaluate("""
|
||||
() => {
|
||||
const btn = document.querySelector('.collect-btn, [class*=collect-btn], .interaction-collect');
|
||||
if (!btn) return 'not_found';
|
||||
const cls = btn.className || btn.classList?.toString() || '';
|
||||
return cls.includes('active') || cls.includes('sel') ? 'collected' : 'not_collected';
|
||||
}
|
||||
""")
|
||||
if was_collected == 'collected':
|
||||
return True
|
||||
|
||||
btn = await page.query_selector(COLLECT_BUTTON)
|
||||
if btn:
|
||||
await btn.click()
|
||||
await asyncio.sleep(0.5)
|
||||
# 验证状态变化
|
||||
now_collected = await page.evaluate("""
|
||||
() => {
|
||||
const btn = document.querySelector('.collect-btn, [class*=collect-btn], .interaction-collect');
|
||||
if (!btn) return false;
|
||||
const cls = btn.className || btn.classList?.toString() || '';
|
||||
return cls.includes('active') || cls.includes('sel');
|
||||
}
|
||||
""")
|
||||
return bool(now_collected)
|
||||
|
||||
# JS 兜底
|
||||
result = await page.evaluate("""
|
||||
() => {
|
||||
const btn = document.querySelector('.collect-btn, [class*=collect-btn], .interaction-collect');
|
||||
if (btn) { btn.click(); return true; }
|
||||
const allBtns = [...document.querySelectorAll('button, div[role=button]')];
|
||||
const collectBtn = allBtns.find(b => b.textContent.includes('收藏') || b.innerHTML.includes('collect'));
|
||||
if (collectBtn) { collectBtn.click(); return true; }
|
||||
return false;
|
||||
}
|
||||
""")
|
||||
return bool(result)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 关注
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
async def follow(page) -> bool:
|
||||
"""关注当前笔记作者(含状态验证)"""
|
||||
try:
|
||||
# 检查是否已关注
|
||||
was_followed = await page.evaluate("""
|
||||
() => {
|
||||
const btn = document.querySelector('.follow-btn, [class*=follow-btn], .interaction-follow');
|
||||
if (!btn) return 'not_found';
|
||||
const txt = btn.textContent || '';
|
||||
return txt.includes('已关注') ? 'followed' : 'not_followed';
|
||||
}
|
||||
""")
|
||||
if was_followed == 'followed':
|
||||
return True
|
||||
|
||||
btn = await page.query_selector(FOLLOW_BUTTON)
|
||||
if btn:
|
||||
await btn.click()
|
||||
await asyncio.sleep(0.5)
|
||||
# 验证关注成功
|
||||
now_followed = await page.evaluate("""
|
||||
() => {
|
||||
const btn = document.querySelector('.follow-btn, [class*=follow-btn], .interaction-follow');
|
||||
if (!btn) return false;
|
||||
const txt = btn.textContent || '';
|
||||
return txt.includes('已关注');
|
||||
}
|
||||
""")
|
||||
return bool(now_followed)
|
||||
|
||||
# JS 兜底
|
||||
result = await page.evaluate("""
|
||||
() => {
|
||||
const btn = document.querySelector('.follow-btn, [class*=follow-btn], .interaction-follow');
|
||||
if (btn) { btn.click(); return true; }
|
||||
const allBtns = [...document.querySelectorAll('button, div[role=button]')];
|
||||
const followBtn = allBtns.find(b => b.textContent.includes('关注') && !b.textContent.includes('已关注'));
|
||||
if (followBtn) { followBtn.click(); return true; }
|
||||
return false;
|
||||
}
|
||||
""")
|
||||
return bool(result)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 评论 — 状态机驱动
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
class CommentStateMachine:
|
||||
"""小红书评论状态机"""
|
||||
|
||||
STATES = ["closed", "panel_open", "input_focused", "text_entered", "sent", "verified"]
|
||||
|
||||
def __init__(self, page):
|
||||
self.page = page
|
||||
self.state = "closed"
|
||||
self.text = ""
|
||||
|
||||
async def open_panel(self) -> bool:
|
||||
"""Step 1: 打开评论区"""
|
||||
if self.state != "closed":
|
||||
return True
|
||||
|
||||
try:
|
||||
# 方式1: 点击评论入口
|
||||
entry = await self.page.query_selector(COMMENT_ENTRY)
|
||||
if entry:
|
||||
await entry.click()
|
||||
await asyncio.sleep(1.5)
|
||||
|
||||
# 验证评论区是否打开
|
||||
has_comments = await self.page.evaluate("""
|
||||
() => document.querySelector('.comment-section, [class*=comment-section], [class*=comment-list]') !== null
|
||||
""")
|
||||
if has_comments:
|
||||
self.state = "panel_open"
|
||||
return True
|
||||
|
||||
# 方式2: 滚动到评论区
|
||||
await self.page.evaluate("""
|
||||
() => {
|
||||
const el = document.querySelector('.comment-section, [class*=comment-section]');
|
||||
if (el) el.scrollIntoView({behavior: 'smooth', block: 'center'});
|
||||
}
|
||||
""")
|
||||
await asyncio.sleep(1.5)
|
||||
self.state = "panel_open"
|
||||
return True
|
||||
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
async def focus_input(self) -> bool:
|
||||
"""Step 2: 聚焦输入框"""
|
||||
if self.state == "input_focused":
|
||||
return True
|
||||
if self.state != "panel_open":
|
||||
return False
|
||||
|
||||
try:
|
||||
# 方式1: Playwright 点击输入框
|
||||
inp = await self.page.query_selector(COMMENT_INPUT)
|
||||
if inp:
|
||||
await inp.click()
|
||||
await asyncio.sleep(0.5)
|
||||
self.state = "input_focused"
|
||||
return True
|
||||
|
||||
# 方式2: JS 聚焦
|
||||
await self.page.evaluate("""
|
||||
() => {
|
||||
const inp = document.querySelector('.comment-input input, [contenteditable=true], textarea');
|
||||
if (inp) { inp.focus(); inp.click(); }
|
||||
}
|
||||
""")
|
||||
await asyncio.sleep(0.5)
|
||||
self.state = "input_focused"
|
||||
return True
|
||||
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
async def enter_text(self, text: str) -> bool:
|
||||
"""Step 3: 输入评论文本(pbcopy + Cmd+V 系统级粘贴)"""
|
||||
if self.state == "text_entered":
|
||||
return True
|
||||
if self.state != "input_focused":
|
||||
return False
|
||||
|
||||
self.text = text
|
||||
|
||||
try:
|
||||
# 小红书输入框可能是 contenteditable div 或 input
|
||||
# 策略: pbcopy + 系统级键盘粘贴(和抖音一样)
|
||||
|
||||
# 1. 复制文本到剪贴板
|
||||
subprocess.run(["pbcopy"], input=text.encode("utf-8"), check=True)
|
||||
await asyncio.sleep(0.3)
|
||||
|
||||
# 2. 确保输入框聚焦
|
||||
await self.page.evaluate("""
|
||||
() => {
|
||||
const inp = document.querySelector('.comment-input input, [contenteditable=true], textarea, .editor');
|
||||
if (inp) { inp.focus(); }
|
||||
}
|
||||
""")
|
||||
await asyncio.sleep(0.3)
|
||||
|
||||
# 3. 系统级粘贴 (Cmd+V)
|
||||
await self.page.keyboard.press("Meta+v")
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
# 4. 验证输入成功(json.dumps 防注入)
|
||||
needle = json.dumps(text[:10])
|
||||
has_text = await self.page.evaluate(f"""
|
||||
() => {{
|
||||
const inp = document.querySelector('.comment-input input, [contenteditable=true], textarea, .editor');
|
||||
if (!inp) return false;
|
||||
const val = inp.value || inp.textContent || inp.innerText || '';
|
||||
return val.includes({needle});
|
||||
}}
|
||||
""")
|
||||
|
||||
if has_text:
|
||||
self.state = "text_entered"
|
||||
return True
|
||||
|
||||
# 5. fallback: Playwright fill
|
||||
inp = await self.page.query_selector(COMMENT_INPUT)
|
||||
if inp:
|
||||
await inp.fill(text)
|
||||
await asyncio.sleep(0.3)
|
||||
self.state = "text_entered"
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
async def send(self) -> bool:
|
||||
"""Step 4: 发送评论"""
|
||||
if self.state != "text_entered":
|
||||
return False
|
||||
|
||||
try:
|
||||
# 方式1: 点击发送按钮
|
||||
btn = await self.page.query_selector(COMMENT_SEND)
|
||||
if btn:
|
||||
await btn.click()
|
||||
await asyncio.sleep(1.5)
|
||||
self.state = "sent"
|
||||
return True
|
||||
|
||||
# 方式2: Enter 键发送
|
||||
await self.page.keyboard.press("Enter")
|
||||
await asyncio.sleep(1.5)
|
||||
self.state = "sent"
|
||||
return True
|
||||
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
async def verify(self) -> bool:
|
||||
"""Step 5: 验证评论是否发送成功"""
|
||||
if self.state != "sent":
|
||||
return False
|
||||
|
||||
try:
|
||||
# 检查评论列表中是否出现我们的评论(json.dumps 防注入)
|
||||
needle = json.dumps(self.text[:8])
|
||||
has_comment = await self.page.evaluate(f"""
|
||||
() => {{
|
||||
const comments = [...document.querySelectorAll('.comment-item, [class*=comment-item]')];
|
||||
return comments.some(c => c.textContent.includes({needle}));
|
||||
}}
|
||||
""")
|
||||
|
||||
if has_comment:
|
||||
self.state = "verified"
|
||||
return True
|
||||
|
||||
# 兜底: 检查输入框是否清空(发送后通常会清空)
|
||||
inp = await self.page.query_selector(COMMENT_INPUT)
|
||||
if inp:
|
||||
tag = await inp.evaluate("el => el.tagName.toLowerCase()")
|
||||
if tag in ("input", "textarea"):
|
||||
val = await inp.input_value()
|
||||
else:
|
||||
val = await inp.evaluate("el => el.textContent || el.innerText || ''")
|
||||
if not val or len(val) < len(self.text) / 2:
|
||||
self.state = "verified"
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
async def comment(page, text: str) -> dict:
|
||||
"""
|
||||
评论当前笔记(完整状态机)
|
||||
|
||||
Args:
|
||||
text: 评论内容
|
||||
|
||||
Returns:
|
||||
{"success": bool, "state": str, "error": str}
|
||||
"""
|
||||
sm = CommentStateMachine(page)
|
||||
result = {"success": False, "state": "closed", "error": ""}
|
||||
|
||||
try:
|
||||
# Step 1: 打开评论区
|
||||
if not await sm.open_panel():
|
||||
result["error"] = "无法打开评论区"
|
||||
return result
|
||||
result["state"] = sm.state
|
||||
|
||||
# Step 2: 聚焦输入框
|
||||
if not await sm.focus_input():
|
||||
result["error"] = "无法聚焦输入框"
|
||||
return result
|
||||
result["state"] = sm.state
|
||||
|
||||
# Step 3: 输入文本
|
||||
if not await sm.enter_text(text):
|
||||
result["error"] = "无法输入文本"
|
||||
return result
|
||||
result["state"] = sm.state
|
||||
|
||||
# Step 4: 发送
|
||||
if not await sm.send():
|
||||
result["error"] = "发送失败"
|
||||
return result
|
||||
result["state"] = sm.state
|
||||
|
||||
# Step 5: 验证
|
||||
if await sm.verify():
|
||||
result["success"] = True
|
||||
result["state"] = "verified"
|
||||
else:
|
||||
result["error"] = "发送后验证失败"
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
result["error"] = str(e)
|
||||
return result
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 概率互动(养号用)
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
async def random_interact(page, behavior_config: dict = None) -> dict:
|
||||
"""
|
||||
随机互动(根据概率触发点赞/收藏/关注/评论)
|
||||
|
||||
Returns:
|
||||
{"like": bool, "collect": bool, "follow": bool, "comment": bool, "comment_text": str}
|
||||
"""
|
||||
results = {
|
||||
"like": False,
|
||||
"collect": False,
|
||||
"follow": False,
|
||||
"comment": False,
|
||||
"comment_text": "",
|
||||
}
|
||||
|
||||
# 点赞: 5 中 1 (20%)
|
||||
if random.random() < 0.20:
|
||||
results["like"] = await like(page)
|
||||
await asyncio.sleep(random.uniform(0.5, 1.5))
|
||||
|
||||
# 收藏: 8 中 1 (12.5%)
|
||||
if random.random() < 0.125:
|
||||
results["collect"] = await collect(page)
|
||||
await asyncio.sleep(random.uniform(0.5, 1.5))
|
||||
|
||||
# 关注: 40 中 1 (2.5%) — 真用户关注频率很低
|
||||
if random.random() < 0.025:
|
||||
results["follow"] = await follow(page)
|
||||
await asyncio.sleep(random.uniform(0.5, 1.5))
|
||||
|
||||
return results
|
||||
@@ -0,0 +1,367 @@
|
||||
"""
|
||||
小红书 DOM 选择器集中管理
|
||||
基于 2026-05-20 Playwright + Camoufox 实际 DOM 分析结果
|
||||
|
||||
分析环境:
|
||||
- URL: https://www.xiaohongshu.com/explore
|
||||
- 分辨率: 702x783 (Camoufox 窗口)
|
||||
- 状态: 未登录(登录弹窗已隐藏)
|
||||
"""
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 首页/发现页 (Explore)
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
# 瀑布流容器
|
||||
FEED_CONTAINER = "#app .layout.limit"
|
||||
|
||||
# 笔记卡片 — section.note-item (DOM 分析确认)
|
||||
NOTE_CARD = "section.note-item"
|
||||
|
||||
# 笔记卡片封面图链接
|
||||
NOTE_CARD_COVER = "a.cover.mask.ld"
|
||||
|
||||
# 笔记卡片中的图片
|
||||
NOTE_CARD_IMG = "a.cover.mask.ld img"
|
||||
|
||||
# 笔记卡片标题
|
||||
NOTE_CARD_TITLE = ".title, [class*=title]"
|
||||
|
||||
# 笔记卡片作者信息
|
||||
NOTE_CARD_AUTHOR = ".author, [class*=author]"
|
||||
|
||||
# 笔记卡片点赞数
|
||||
NOTE_CARD_LIKES = ".like-count, [class*=like], [class*=count]"
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 笔记详情页 (Note Detail)
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
# 详情页主容器
|
||||
NOTE_DETAIL_CONTAINER = ".note-detail, [class*=note-detail], .note-page"
|
||||
|
||||
# 笔记标题
|
||||
NOTE_TITLE = ".title, h1, [class*=note-title]"
|
||||
|
||||
# 笔记内容文字
|
||||
NOTE_CONTENT = ".content, [class*=content], .desc"
|
||||
|
||||
# 笔记图片
|
||||
NOTE_IMAGES = ".note-content img, [class*=note-content] img"
|
||||
|
||||
# 作者头像
|
||||
AUTHOR_AVATAR = ".avatar, [class*=avatar]"
|
||||
|
||||
# 作者昵称
|
||||
AUTHOR_NAME = ".author-name, [class*=author-name], .nickname"
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 互动按钮
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
# 点赞按钮
|
||||
LIKE_BUTTON = ".like-btn, [class*=like-btn], .interaction-like, button:has([class*=like])"
|
||||
|
||||
# 收藏按钮
|
||||
COLLECT_BUTTON = ".collect-btn, [class*=collect-btn], .interaction-collect, button:has([class*=collect])"
|
||||
|
||||
# 关注按钮
|
||||
FOLLOW_BUTTON = ".follow-btn, [class*=follow-btn], .interaction-follow"
|
||||
|
||||
# 分享按钮
|
||||
SHARE_BUTTON = ".share-btn, [class*=share-btn]"
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 评论区
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
# 评论入口/评论数按钮
|
||||
COMMENT_ENTRY = ".comment-btn, [class*=comment-btn], .interaction-comment, [class*=comment-count]"
|
||||
|
||||
# 评论区容器
|
||||
COMMENT_SECTION = ".comment-section, [class*=comment-section], .comment-list"
|
||||
|
||||
# 评论输入框
|
||||
COMMENT_INPUT = ".comment-input, [class*=comment-input] input, [contenteditable=true], textarea[placeholder*=评论]"
|
||||
|
||||
# 评论发送按钮
|
||||
COMMENT_SEND = ".send-btn, [class*=send-btn], button:has-text('发送'), button[type=submit]"
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 搜索
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
# 搜索框
|
||||
SEARCH_INPUT = "input.search-input"
|
||||
|
||||
# 搜索按钮
|
||||
SEARCH_BUTTON = ".search-icon, [class*=search-icon], button:has-text('搜索')"
|
||||
|
||||
# 搜索结果页
|
||||
SEARCH_RESULT_CONTAINER = ".search-result, [class*=search-result]"
|
||||
|
||||
# 搜索结果笔记卡片
|
||||
SEARCH_NOTE_CARD = ".search-result .note-item, .search-result-item"
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 导航
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
# 左侧导航栏
|
||||
LEFT_NAV = "nav, aside, [class*=sidebar]"
|
||||
|
||||
# 首页链接
|
||||
NAV_HOME = 'a[href*="/explore"], a:has-text("发现")'
|
||||
|
||||
# 消息链接
|
||||
NAV_MESSAGE = 'a[href*="/message"], a:has-text("消息")'
|
||||
|
||||
# 通知链接
|
||||
NAV_NOTIFICATION = 'a[href*="/notification"], a:has-text("通知")'
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 弹窗/遮罩
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
# 登录弹窗
|
||||
LOGIN_MODAL = "[class*=login], [class*=modal], [role=dialog], .reds-alert"
|
||||
|
||||
# 登录弹窗关闭按钮
|
||||
LOGIN_MODAL_CLOSE = ".close-btn, [class*=close], button:has-text('关闭'), [class*=mask]"
|
||||
|
||||
# 通用遮罩层
|
||||
OVERLAY = "[class*=mask], [class*=overlay], [class*=modal], div[style*='position: fixed']"
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 锚点验证选择器 (用于操作前后状态校验)
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
ANCHORS = {
|
||||
# 首页锚点: 瀑布流有笔记卡片
|
||||
"home_page": "section.note-item",
|
||||
|
||||
# 笔记详情页锚点: 有笔记内容或标题
|
||||
"note_detail": ".note-detail, [class*=note-detail], .title, h1",
|
||||
|
||||
# 视频播放页锚点: 有 video 元素
|
||||
"video_page": "video",
|
||||
|
||||
# 评论区打开锚点: 评论列表可见
|
||||
"comment_open": ".comment-section, [class*=comment-section]",
|
||||
|
||||
# 搜索页锚点: 搜索结果容器
|
||||
"search_page": ".search-result, [class*=search-result]",
|
||||
|
||||
# 页面加载完成锚点: body 有内容
|
||||
"page_loaded": "#app",
|
||||
}
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 辅助函数
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
def get_note_links_js() -> str:
|
||||
"""JS 代码: 获取当前页面所有笔记链接"""
|
||||
return """
|
||||
() => {
|
||||
const links = [...document.querySelectorAll('a[href*="/explore/"]')];
|
||||
return [...new Set(links.map(a => a.href))].filter(h => h.includes('/explore/')).slice(0, 10);
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def get_note_cards_js() -> str:
|
||||
"""JS 代码: 获取笔记卡片元素列表"""
|
||||
return """
|
||||
() => {
|
||||
const cards = [...document.querySelectorAll('section.note-item')];
|
||||
return cards.map((card, i) => ({
|
||||
index: i,
|
||||
href: card.querySelector('a')?.href || '',
|
||||
title: card.textContent?.trim().substring(0, 50) || '',
|
||||
rect: card.getBoundingClientRect ? {
|
||||
x: Math.round(card.getBoundingClientRect().x),
|
||||
y: Math.round(card.getBoundingClientRect().y),
|
||||
w: Math.round(card.getBoundingClientRect().width),
|
||||
h: Math.round(card.getBoundingClientRect().height)
|
||||
} : null
|
||||
}));
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def dismiss_login_modal_js() -> str:
|
||||
"""JS 代码: 尝试关闭登录弹窗"""
|
||||
return """
|
||||
() => {
|
||||
// 关闭登录弹窗
|
||||
document.querySelectorAll('[class*=modal], [class*=dialog], [class*=mask], [role=dialog]').forEach(m => {
|
||||
m.style.display = 'none';
|
||||
});
|
||||
// 移除 fixed 遮罩
|
||||
document.querySelectorAll('div').forEach(d => {
|
||||
const s = window.getComputedStyle(d);
|
||||
if (s.position === 'fixed' && parseInt(s.zIndex) > 100) {
|
||||
d.style.display = 'none';
|
||||
}
|
||||
});
|
||||
return 'modal dismissed';
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def find_refresh_button_js() -> str:
|
||||
"""
|
||||
JS 代码: 查找小红书瀑布流页面右下角的刷新 FAB 按钮
|
||||
|
||||
策略:
|
||||
1. 先用常见选择器查找刷新类按钮
|
||||
2. 再用 SVG 扫描(刷新图标通常用 SVG 箭头)
|
||||
3. 过滤出右下角区域(x > viewport 60%, y > viewport 60%)的元素
|
||||
"""
|
||||
return """
|
||||
() => {
|
||||
const vw = window.innerWidth, vh = window.innerHeight;
|
||||
const btns = [];
|
||||
|
||||
// 方式1: 常见刷新按钮选择器
|
||||
const selectors = [
|
||||
'button[class*="refresh"]',
|
||||
'[class*="refresh-btn"]',
|
||||
'[class*="reload"]',
|
||||
'button[class*="fab"]',
|
||||
'[class*="float-btn"]',
|
||||
'[class*="floating"]',
|
||||
];
|
||||
for (const sel of selectors) {
|
||||
const els = document.querySelectorAll(sel);
|
||||
els.forEach(el => {
|
||||
const r = el.getBoundingClientRect();
|
||||
if (r.width > 20 && r.height > 20 && r.width < 100 && r.height < 100) {
|
||||
btns.push({x: r.x, y: r.y, w: r.width, h: r.height, text: el.textContent?.trim() || '', sel: sel});
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// 方式2: SVG 扫描 — 查找包含旋转箭头路径的 SVG(刷新图标特征)
|
||||
document.querySelectorAll('svg').forEach(svg => {
|
||||
const parent = svg.closest('button, a, [class*="btn"], [class*="icon"], [role="button"], div');
|
||||
if (!parent) return;
|
||||
const r = parent.getBoundingClientRect();
|
||||
if (r.width < 15 || r.height < 15 || r.width > 120 || r.height > 120) return;
|
||||
// 检查 SVG 内是否有旋转箭头路径(arc + rotate 是刷新图标的典型特征)
|
||||
const paths = svg.querySelectorAll('path, circle');
|
||||
const hasRotation = [...paths].some(p => {
|
||||
const d = (p.getAttribute('d') || '').toLowerCase();
|
||||
const transform = (p.getAttribute('transform') || svg.getAttribute('transform') || '').toLowerCase();
|
||||
return d.includes('a') && (d.includes('m') || d.includes('l')) ||
|
||||
transform.includes('rotate') ||
|
||||
d.includes('arc');
|
||||
});
|
||||
if (hasRotation) {
|
||||
btns.push({x: r.x, y: r.y, w: r.width, h: r.height, text: parent.textContent?.trim() || '', sel: 'svg-scan'});
|
||||
}
|
||||
});
|
||||
|
||||
// 方式3: 查找固定定位在右下角的可点击元素
|
||||
document.querySelectorAll('button, a, [role="button"], div[style*="cursor"]').forEach(el => {
|
||||
const s = window.getComputedStyle(el);
|
||||
if (s.position !== 'fixed') return;
|
||||
const r = el.getBoundingClientRect();
|
||||
if (r.x > vw * 0.6 && r.y > vh * 0.6 && r.width < 100 && r.height < 100 && r.width > 20 && r.height > 20) {
|
||||
// 避免重复
|
||||
if (!btns.some(b => Math.abs(b.x - r.x) < 10 && Math.abs(b.y - r.y) < 10)) {
|
||||
btns.push({x: r.x, y: r.y, w: r.width, h: r.height, text: el.textContent?.trim() || '', sel: 'fixed-position'});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
if (!btns.length) return {found: false};
|
||||
|
||||
// 优先选右下角的按钮(x 最大 + y 最大)
|
||||
btns.sort((a, b) => (b.x + b.y) - (a.x + a.y));
|
||||
const best = btns[0];
|
||||
return {
|
||||
found: true,
|
||||
x: Math.round(best.x + best.w / 2),
|
||||
y: Math.round(best.y + best.h / 2),
|
||||
w: Math.round(best.w),
|
||||
h: Math.round(best.h),
|
||||
text: best.text,
|
||||
method: best.sel
|
||||
};
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def find_qr_wall_back_button_js() -> str:
|
||||
"""
|
||||
JS 代码: 检测小红书 QR 检测墙并查找"返回首页"按钮
|
||||
|
||||
QR 墙特征:
|
||||
- 页面中央出现大弹窗
|
||||
- body 文本包含"扫码"、"二维码"等关键词
|
||||
- 弹窗内有"返回首页"或类似按钮
|
||||
"""
|
||||
return """
|
||||
() => {
|
||||
const bodyText = document.body?.innerText?.toLowerCase() || '';
|
||||
// QR 墙关键词检测
|
||||
const qrKeywords = ['扫码', '二维码', 'qrcode', '扫描', 'scan code', '非常用登录', '验证'];
|
||||
const isQrWall = qrKeywords.some(kw => bodyText.includes(kw));
|
||||
|
||||
if (!isQrWall) return {found: false, reason: 'no_qr_keywords'};
|
||||
|
||||
// 查找弹窗中的返回按钮
|
||||
const backKeywords = ['返回首页', '返回', '首页', '回到首页', '确定', '我知道了', '关闭'];
|
||||
const candidates = [];
|
||||
|
||||
// 搜索弹窗内所有可点击元素
|
||||
document.querySelectorAll('button, a, [role="button"], [class*="btn"], span[class*="text"], div[class*="text"]').forEach(el => {
|
||||
const text = el.textContent?.trim();
|
||||
if (!text) return;
|
||||
const textLower = text.toLowerCase();
|
||||
// 精确匹配或包含匹配
|
||||
const matched = backKeywords.some(kw => {
|
||||
if (textLower === kw) return true;
|
||||
if (textLower.includes(kw) && text.length < 15) return true;
|
||||
return false;
|
||||
});
|
||||
if (!matched) return;
|
||||
|
||||
const r = el.getBoundingClientRect();
|
||||
if (r.width < 5 || r.height < 5) return; // 不可见
|
||||
|
||||
// 优先级: "返回首页" > "返回" > "首页" > 其他
|
||||
let priority = 99;
|
||||
if (textLower === '返回首页' || textLower === '回到首页') priority = 10;
|
||||
else if (textLower === '返回') priority = 20;
|
||||
else if (textLower === '首页') priority = 30;
|
||||
else priority = 50;
|
||||
|
||||
candidates.push({
|
||||
x: Math.round(r.x + r.width / 2),
|
||||
y: Math.round(r.y + r.height / 2),
|
||||
w: Math.round(r.width),
|
||||
h: Math.round(r.height),
|
||||
text: text,
|
||||
priority: priority
|
||||
});
|
||||
});
|
||||
|
||||
if (!candidates.length) return {found: false, reason: 'no_back_button'};
|
||||
|
||||
// 按优先级排序
|
||||
candidates.sort((a, b) => a.priority - b.priority);
|
||||
const best = candidates[0];
|
||||
return {
|
||||
found: true,
|
||||
x: best.x,
|
||||
y: best.y,
|
||||
w: best.w,
|
||||
h: best.h,
|
||||
text: best.text,
|
||||
priority: best.priority
|
||||
};
|
||||
}
|
||||
"""
|
||||
@@ -0,0 +1,287 @@
|
||||
"""
|
||||
cookie_manager.py — Cookie 安全备份与恢复模块 v1.0
|
||||
|
||||
核心问题:XHS 和 Douyin 共享 identity_dir(douyin_01_camo = douyin_01 + xhs_01)
|
||||
清 XHS cookie 时容易误删抖音 session。
|
||||
|
||||
解决方案:
|
||||
1. 每次操作前自动全量备份 cookies.sqlite(带平台+时间戳标签)
|
||||
2. 跨平台操作时自动保护对方平台 cookie(备份后再操作)
|
||||
3. 支持按平台精准恢复(只恢复某个平台的 cookie 行)
|
||||
|
||||
文件结构:
|
||||
backups/cookies/{identity_dir_name}/
|
||||
cookies.{timestamp}.{platform}.bak ← 全量备份
|
||||
cookies.{timestamp}.restore_point.bak ← 操作前检查点
|
||||
|
||||
使用:
|
||||
from matrix_modules.utils.cookie_manager import CookieGuard
|
||||
|
||||
guard = CookieGuard("douyin_01_camo")
|
||||
guard.backup(platform="douyin") # 备份当前状态
|
||||
guard.protect("douyin") # 保护抖音(=备份后操作其他平台)
|
||||
guard.restore(platform="douyin") # 恢复抖音 cookie
|
||||
"""
|
||||
import shutil
|
||||
import os
|
||||
import time
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
from typing import Optional, List
|
||||
import glob
|
||||
|
||||
from matrix_mgmt import AGENT_LOCAL
|
||||
LOCAL_ROOT = AGENT_LOCAL / "tools" / "matrix"
|
||||
IDENTITIES_DIR = LOCAL_ROOT / "identities"
|
||||
BACKUP_DIR = LOCAL_ROOT / "backups" / "cookies"
|
||||
|
||||
# 共享 identity_dir → 涉及的平台映射
|
||||
SHARED_MAP = {
|
||||
"douyin_01_camo": ["douyin", "xiaohongshu"],
|
||||
"douyin_02_camo": ["douyin", "xiaohongshu"],
|
||||
"douyin_camo01": ["douyin", "xiaohongshu"],
|
||||
}
|
||||
|
||||
|
||||
class CookieGuard:
|
||||
"""Cookie 安全防护工具"""
|
||||
|
||||
def __init__(self, identity_dir_name: str):
|
||||
"""
|
||||
Args:
|
||||
identity_dir_name: identity 目录名, 如 "douyin_01_camo"
|
||||
"""
|
||||
self.identity_name = identity_dir_name
|
||||
self.user_data_dir = IDENTITIES_DIR / identity_dir_name / "user_data"
|
||||
self.cookie_file = self.user_data_dir / "cookies.sqlite"
|
||||
self.backup_subdir = BACKUP_DIR / identity_dir_name
|
||||
self.backup_subdir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# ──────────────── 公共 API ────────────────
|
||||
|
||||
def backup(self, platform: str = None, label: str = "") -> str:
|
||||
"""
|
||||
全量备份 cookies.sqlite
|
||||
|
||||
Args:
|
||||
platform: 平台标签(douyin/xiaohongshu),用于标识
|
||||
label: 额外标签(如 "pre_xhs_clear")
|
||||
|
||||
Returns:
|
||||
备份文件路径,失败返回空字符串
|
||||
"""
|
||||
if not self.cookie_file.exists():
|
||||
return ""
|
||||
|
||||
ts = time.strftime("%Y%m%d_%H%M%S")
|
||||
parts = ["cookies", ts]
|
||||
if platform:
|
||||
parts.append(platform)
|
||||
if label:
|
||||
parts.append(label)
|
||||
bak_name = ".".join(parts) + ".bak"
|
||||
|
||||
bak_path = self.backup_subdir / bak_name
|
||||
try:
|
||||
shutil.copy2(str(self.cookie_file), str(bak_path))
|
||||
return str(bak_path)
|
||||
except Exception as e:
|
||||
return ""
|
||||
|
||||
def restore(self, backup_file: str = None, platform: str = None) -> bool:
|
||||
"""
|
||||
恢复 cookies.sqlite 到目标目录
|
||||
|
||||
Args:
|
||||
backup_file: 指定备份文件路径;None 时自动找最新的
|
||||
platform: 指定按平台恢复(仅恢复该平台的 cookie 行)
|
||||
None 时全量恢复
|
||||
|
||||
Returns:
|
||||
True=成功
|
||||
"""
|
||||
if not backup_file:
|
||||
latest = self._find_latest_backup(platform)
|
||||
if not latest:
|
||||
return False
|
||||
backup_file = str(latest)
|
||||
|
||||
src = Path(backup_file)
|
||||
if not src.exists():
|
||||
return False
|
||||
|
||||
if platform:
|
||||
# ── 按平台精准恢复(只覆盖指定平台的 cookie 行) ──
|
||||
return self._restore_platform_only(src, platform)
|
||||
else:
|
||||
# ── 全量恢复 ──
|
||||
try:
|
||||
# 先备份当前文件(万一恢复错了还能回退)
|
||||
current_bak = self.backup(platform=platform, label="pre_restore")
|
||||
# 复制备份覆盖
|
||||
shutil.copy2(str(src), str(self.cookie_file))
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def protect(self, platform_to_protect: str) -> str:
|
||||
"""
|
||||
保护指定平台的 cookie:操作其他平台前先调用
|
||||
|
||||
例如 protect("douyin") 会在清 XHS cookie 前,
|
||||
先备份当前 cookies.sqlite(含抖音 session)。
|
||||
|
||||
Args:
|
||||
platform_to_protect: 要保护的平台 ("douyin" 或 "xiaohongshu")
|
||||
|
||||
Returns:
|
||||
备份文件路径
|
||||
"""
|
||||
return self.backup(platform=platform_to_protect, label="protected")
|
||||
|
||||
def delete_platform_cookies_safe(self, platform: str) -> bool:
|
||||
"""
|
||||
安全删除指定平台的 cookie(自动备份对方平台)
|
||||
|
||||
Args:
|
||||
platform: 要删除的平台 ("douyin" 或 "xiaohongshu")
|
||||
|
||||
Returns:
|
||||
bool: True=删除成功
|
||||
"""
|
||||
# 1. 先保护对方平台
|
||||
other = self._get_other_platform(platform)
|
||||
if other:
|
||||
self.protect(other)
|
||||
|
||||
# 2. 再全量备份当前状态
|
||||
self.backup(platform=platform, label="pre_delete")
|
||||
|
||||
# 3. 执行删除
|
||||
return self._delete_platform_cookies(platform)
|
||||
|
||||
def list_backups(self, platform: str = None) -> List[str]:
|
||||
"""列出可用备份"""
|
||||
pattern = f"cookies.*{platform + '.' if platform else ''}*.bak"
|
||||
files = sorted(self.backup_subdir.glob(pattern), reverse=True)
|
||||
return [str(f) for f in files]
|
||||
|
||||
# ──────────────── 内部方法 ────────────────
|
||||
|
||||
def _find_latest_backup(self, platform: str = None):
|
||||
"""找最新的备份文件"""
|
||||
pattern = f"cookies.*{platform + '.' if platform else ''}*.bak"
|
||||
files = sorted(self.backup_subdir.glob(pattern), reverse=True)
|
||||
return files[0] if files else None
|
||||
|
||||
def _get_other_platform(self, platform: str) -> Optional[str]:
|
||||
"""获取共享 identity 中的对方平台名"""
|
||||
platforms = SHARED_MAP.get(self.identity_name, [])
|
||||
for p in platforms:
|
||||
if p != platform:
|
||||
return p
|
||||
return None
|
||||
|
||||
def _delete_platform_cookies(self, platform: str) -> bool:
|
||||
"""通过 sqlite3 删除指定平台的 cookie"""
|
||||
domain_patterns = {
|
||||
"douyin": "%douyin.com%",
|
||||
"xiaohongshu": "%xiaohongshu%",
|
||||
}
|
||||
pattern = domain_patterns.get(platform, f"%{platform}%")
|
||||
|
||||
if not self.cookie_file.exists():
|
||||
return False
|
||||
|
||||
try:
|
||||
conn = sqlite3.connect(str(self.cookie_file))
|
||||
c = conn.cursor()
|
||||
c.execute("DELETE FROM moz_cookies WHERE host LIKE ?", (pattern,))
|
||||
deleted = c.rowcount
|
||||
conn.commit()
|
||||
conn.close()
|
||||
# 删除 -wal 和 -shm 文件(Firefox 可能未正常关闭)
|
||||
for ext in ["-wal", "-shm"]:
|
||||
f = str(self.cookie_file) + ext
|
||||
if os.path.exists(f):
|
||||
os.remove(f)
|
||||
return deleted > 0
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def _restore_platform_only(self, src: Path, platform: str) -> bool:
|
||||
"""
|
||||
精准恢复:从备份文件中提取指定平台的 cookie 行,
|
||||
合并到当前 cookies.sqlite
|
||||
|
||||
流程:
|
||||
1. 从备份文件读取指定平台的 cookie 行
|
||||
2. 从当前文件删除该平台的旧 cookie 行
|
||||
3. 将备份中的行插入当前文件
|
||||
"""
|
||||
domain_patterns = {
|
||||
"douyin": "%douyin.com%",
|
||||
"xiaohongshu": "%xiaohongshu%",
|
||||
}
|
||||
pattern = domain_patterns.get(platform, f"%{platform}%")
|
||||
|
||||
if not self.cookie_file.exists():
|
||||
return False
|
||||
|
||||
try:
|
||||
# 连接备份文件,读取目标平台的 cookie 行
|
||||
src_conn = sqlite3.connect(str(src))
|
||||
src_c = src_conn.cursor()
|
||||
rows = src_c.execute(
|
||||
"SELECT * FROM moz_cookies WHERE host LIKE ?", (pattern,)
|
||||
).fetchall()
|
||||
columns = [d[0] for d in src_c.description]
|
||||
src_conn.close()
|
||||
|
||||
if not rows:
|
||||
return False
|
||||
|
||||
# 连接当前文件,删除旧行后插入
|
||||
dst_conn = sqlite3.connect(str(self.cookie_file))
|
||||
dst_c = dst_conn.cursor()
|
||||
|
||||
# 删除该平台旧 cookie
|
||||
dst_c.execute("DELETE FROM moz_cookies WHERE host LIKE ?", (pattern,))
|
||||
|
||||
# 构造 INSERT 语句
|
||||
placeholders = ",".join(["?"] * len(columns))
|
||||
col_names = ",".join(columns)
|
||||
dst_c.executemany(
|
||||
f"INSERT INTO moz_cookies ({col_names}) VALUES ({placeholders})",
|
||||
rows
|
||||
)
|
||||
dst_conn.commit()
|
||||
dst_conn.close()
|
||||
|
||||
# 清理 wal/shm
|
||||
for ext in ["-wal", "-shm"]:
|
||||
f = str(self.cookie_file) + ext
|
||||
if os.path.exists(f):
|
||||
os.remove(f)
|
||||
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
# ──────────────── 简便函数(不用实例化) ────────────────
|
||||
|
||||
def backup_all_identities(platform: str = None, label: str = "") -> dict:
|
||||
"""备份所有账号的 cookie"""
|
||||
results = {}
|
||||
for identity in SHARED_MAP:
|
||||
guard = CookieGuard(identity)
|
||||
path = guard.backup(platform=platform, label=label)
|
||||
results[identity] = path
|
||||
return results
|
||||
|
||||
|
||||
def quick_backup(identity_dir_name: str, platform: str = None) -> str:
|
||||
"""快速备份单个账号"""
|
||||
guard = CookieGuard(identity_dir_name)
|
||||
return guard.backup(platform=platform)
|
||||
@@ -0,0 +1,20 @@
|
||||
"""
|
||||
mc — Matrix Console 统一命令入口 v1.0
|
||||
|
||||
用法:
|
||||
mc run --accounts A,B --blueprints X,Y --rounds 10 --mix
|
||||
mc account list
|
||||
mc account login <name>
|
||||
mc blueprint list
|
||||
mc corpus list
|
||||
mc proxy list
|
||||
mc sms config
|
||||
mc status all
|
||||
|
||||
设计原则:
|
||||
- 所有操作 = 参数 + CLI 命令 + 输出日志
|
||||
- 看板是 CLI 的可视化外壳
|
||||
- 自动化/Guardd 也只调 CLI
|
||||
- 可预测、可重复、可脚本化
|
||||
"""
|
||||
__version__ = "1.0.0"
|
||||
@@ -0,0 +1,7 @@
|
||||
"""
|
||||
mc __main__ — python -m mc 入口
|
||||
"""
|
||||
from mc.cli import main
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,462 @@
|
||||
"""
|
||||
analyzer.py — 录制步骤差异分析引擎 (v1.0)
|
||||
|
||||
三阶段管线 Phase 2:
|
||||
读取录制包 → 对比相邻步骤的 URL/DOM/事件 →
|
||||
推断操作类型 → 生成结构化分析结果
|
||||
|
||||
用法:
|
||||
from mc.analyzer import analyze_recording
|
||||
result = analyze_recording(package)
|
||||
"""
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
# ── 抖音快捷键映射 ──
|
||||
DY_HOTKEYS = {
|
||||
'z': 'like', 'x': 'comment', 'c': 'comment',
|
||||
'g': 'follow',
|
||||
'k': 'loop', 'm': 'mute', 'j': 'clear',
|
||||
'p': 'loop', 'h': 'fullscreen',
|
||||
'ArrowDown': 'next_video', 'ArrowUp': 'prev_video',
|
||||
' ': 'play_pause',
|
||||
}
|
||||
|
||||
# ── 点击坐标区域定义 ──
|
||||
# (x_min, y_min, x_max, y_max, label)
|
||||
KNOWN_REGIONS = {
|
||||
'dy_like_area': (330, 400, 500, 600, '抖音点赞区域'),
|
||||
'dy_comment_area': (350, 680, 500, 720, '抖音评论区触发'),
|
||||
'dy_search_bar': (50, 50, 350, 90, '抖音搜索框'),
|
||||
'xhs_bottom_nav': (0, 700, 400, 780, 'XHS底部导航'),
|
||||
'xhs_me_tab': (300, 700, 400, 780, 'XHS"我"标签'),
|
||||
}
|
||||
|
||||
|
||||
def analyze_recording(package: dict) -> dict:
|
||||
"""分析录制包,生成可导出的原子操作定义"""
|
||||
steps = package.get("steps", [])
|
||||
if len(steps) < 2:
|
||||
return {"error": "至少需要2个步骤才能分析", "steps_analyzed": 0}
|
||||
|
||||
result = {
|
||||
"meta": package.get("meta", {}),
|
||||
"steps_analyzed": len(steps),
|
||||
"actions": [],
|
||||
"blueprint_suggestions": [],
|
||||
}
|
||||
|
||||
# 分析相邻步骤对
|
||||
for i in range(len(steps) - 1):
|
||||
before = steps[i]
|
||||
after = steps[i+1]
|
||||
analysis = _analyze_step_pair(before, after)
|
||||
result["actions"].append(analysis)
|
||||
|
||||
# 生成蓝图建议
|
||||
result["blueprint_suggestions"] = _suggest_blueprint(result["actions"])
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def _analyze_step_pair(before: dict, after: dict) -> dict:
|
||||
"""分析一对相邻步骤的变化"""
|
||||
bp = before.get("page", {})
|
||||
ap = after.get("page", {})
|
||||
events = before.get("events", [])
|
||||
|
||||
# before_state 对比(三段式记录的状态指纹)
|
||||
bbs = before.get("before_state", {}) or {}
|
||||
abs = after.get("before_state", {}) or {}
|
||||
|
||||
analysis = {
|
||||
"step_before": before["step"],
|
||||
"step_after": after["step"],
|
||||
"event_count": len(events),
|
||||
"page_url": ap.get("url", ""),
|
||||
"page_mode": ap.get("page_mode", "?"),
|
||||
"interactable_count": ap.get("interactable_count", 0),
|
||||
}
|
||||
|
||||
# 0. before_state 选择器变化分析(优先级最高)
|
||||
sel_before = bbs.get("selectors", {})
|
||||
sel_after = abs.get("selectors", {})
|
||||
if sel_before and sel_after:
|
||||
sel_changes = {}
|
||||
all_keys = set(sel_before) | set(sel_after)
|
||||
for k in all_keys:
|
||||
v1 = sel_before.get(k, False)
|
||||
v2 = sel_after.get(k, False)
|
||||
if v1 != v2:
|
||||
sel_changes[k] = f"{'✅' if v1 else '❌'}→{'✅' if v2 else '❌'}"
|
||||
if sel_changes:
|
||||
analysis["selector_changes"] = sel_changes
|
||||
|
||||
# 基于选择器变化推断操作
|
||||
op = _infer_op_from_selectors(sel_before, sel_after, bbs, abs)
|
||||
if op:
|
||||
analysis["action_type"] = op["type"]
|
||||
analysis["action_desc"] = op["desc"]
|
||||
analysis["confidence"] = op["confidence"]
|
||||
|
||||
# 1. URL 变化分析
|
||||
url_before = bp.get("url", "")
|
||||
url_after = ap.get("url", "")
|
||||
if url_before != url_after:
|
||||
analysis["url_changed"] = True
|
||||
analysis["url_from"] = url_before[:80]
|
||||
analysis["url_to"] = url_after[:80]
|
||||
|
||||
# URL 变化类型推断
|
||||
if "/video/" in url_after and "/video/" not in url_before:
|
||||
analysis["action_type"] = "navigate_video"
|
||||
analysis["action_desc"] = "进入视频播放页"
|
||||
elif "/user/profile/" in url_after:
|
||||
analysis["action_type"] = "navigate_profile"
|
||||
analysis["action_desc"] = "进入个人主页"
|
||||
elif "/search/" in url_after or "?keyword=" in url_after:
|
||||
analysis["action_type"] = "search"
|
||||
analysis["action_desc"] = "搜索"
|
||||
elif "/user/self" in url_after:
|
||||
analysis["action_type"] = "navigate_profile"
|
||||
analysis["action_desc"] = "进入抖音主页"
|
||||
else:
|
||||
analysis["action_type"] = "navigate"
|
||||
analysis["action_desc"] = "页面导航"
|
||||
else:
|
||||
analysis["url_changed"] = False
|
||||
|
||||
# 2. 页面模式变化
|
||||
mode_before = bp.get("page_mode", "?")
|
||||
mode_after = ap.get("page_mode", "?")
|
||||
if mode_before != mode_after:
|
||||
analysis["mode_changed"] = f"{mode_before} → {mode_after}"
|
||||
if mode_before == "grid" and mode_after == "player":
|
||||
analysis["action_type"] = analysis.get("action_type", "enter_video")
|
||||
analysis["action_desc"] = analysis.get("action_desc", "进入视频播放")
|
||||
elif mode_before == "player" and mode_after == "grid":
|
||||
analysis["action_type"] = analysis.get("action_type", "exit_video")
|
||||
analysis["action_desc"] = analysis.get("action_desc", "退出视频")
|
||||
elif mode_after == "profile":
|
||||
analysis["action_type"] = analysis.get("action_type", "view_profile")
|
||||
analysis["action_desc"] = analysis.get("action_desc", "查看主页")
|
||||
|
||||
# 3. 事件序列分析(从用户操作反推操作类型)
|
||||
if events:
|
||||
action = _infer_action_from_events(events)
|
||||
if action:
|
||||
analysis["inferred_action"] = action
|
||||
# 事件推断的优先级高于 URL 推断
|
||||
if action.get("type") and not analysis.get("action_type"):
|
||||
analysis["action_type"] = action["type"]
|
||||
analysis["action_desc"] = action.get("desc", "")
|
||||
|
||||
# 4. 文本差异分析
|
||||
text_before = bp.get("text_snippet", "")
|
||||
text_after = ap.get("text_snippet", "")
|
||||
if text_before and text_after and text_before != text_after:
|
||||
analysis["text_changed"] = True
|
||||
# 检测文本中的关键变化
|
||||
added = _find_added_text(text_before, text_after)
|
||||
if added:
|
||||
analysis["text_added"] = added[:100]
|
||||
|
||||
# 5. 标题变化
|
||||
title_before = bp.get("title", "")
|
||||
title_after = ap.get("title", "")
|
||||
if title_before != title_after:
|
||||
analysis["title_changed"] = f"{title_before[:30]} → {title_after[:30]}"
|
||||
|
||||
# 6. 交互元素变化
|
||||
ia_before = set(e.get("e2e", "") or e.get("cls", "") for e in bp.get("interactables", []))
|
||||
ia_after = set(e.get("e2e", "") or e.get("cls", "") for e in ap.get("interactables", []))
|
||||
new_elements = ia_after - ia_before
|
||||
if new_elements:
|
||||
analysis["new_elements"] = list(new_elements)[:5]
|
||||
|
||||
# 7. 生成特征码(可复现的自动化选择器建议)
|
||||
analysis["features"] = _suggest_features(bp, ap, events, analysis)
|
||||
|
||||
# 8. 检测空步骤(连续两次 · 无操作)
|
||||
if not analysis.get("action_type"):
|
||||
no_events = len(events) <= 1 # 只有 · 键事件
|
||||
no_url_change = not analysis.get("url_changed", True)
|
||||
no_sel_change = not analysis.get("selector_changes")
|
||||
no_mode_change = not analysis.get("mode_changed")
|
||||
no_clicks = len([e for e in events if e.get('t') == 'click']) == 0
|
||||
no_scrolls = len([e for e in events if e.get('t') == 'scroll']) == 0
|
||||
if no_events and no_url_change and no_sel_change and no_mode_change:
|
||||
analysis["action_type"] = "skip"
|
||||
analysis["action_desc"] = "无有效操作(空步骤)"
|
||||
analysis["skip_reason"] = "no_change"
|
||||
|
||||
# 9. 生成操作摘要
|
||||
if not analysis.get("action_type"):
|
||||
analysis["action_type"] = "unknown"
|
||||
analysis["action_desc"] = f"页面内容变化 ({len(events)} events)"
|
||||
|
||||
return analysis
|
||||
|
||||
|
||||
def _infer_op_from_selectors(sel_before: dict, sel_after: dict,
|
||||
bbs: dict, abs: dict) -> Optional[dict]:
|
||||
"""基于 before_state 的选择器变化推断具体操作"""
|
||||
# 抖音 like 按钮: [data-e2e="video-player-digg"]
|
||||
# 抖音 collect: [data-e2e="video-player-collect"]
|
||||
# 小红书 note: section.note-item
|
||||
digg_key = '[data-e2e="video-player-digg"]'
|
||||
collect_key = '[data-e2e="video-player-collect"]'
|
||||
search_key = '[data-e2e="searchbar-input"]'
|
||||
video_key = 'video'
|
||||
note_key = 'section.note-item'
|
||||
|
||||
p_before = bbs.get("page_mode", "")
|
||||
p_after = abs.get("page_mode", "")
|
||||
|
||||
# 检测抖音点赞操作
|
||||
digg_before = sel_before.get(digg_key, False)
|
||||
digg_after = sel_after.get(digg_key, False)
|
||||
if digg_before and not digg_after and p_before == "player":
|
||||
return {"type": "like", "desc": "点赞", "confidence": "high"}
|
||||
# 检测抖音取消点赞
|
||||
if not digg_before and digg_after and p_before == "player":
|
||||
liked_text_before = bbs.get("markers", {}).get("liked", "")
|
||||
liked_text_after = abs.get("markers", {}).get("liked", "")
|
||||
return {"type": "like", "desc": "点赞(按钮重显)", "confidence": "medium"}
|
||||
|
||||
# 检测抖音收藏操作
|
||||
collect_before = sel_before.get(collect_key, False)
|
||||
collect_after = sel_after.get(collect_key, False)
|
||||
if collect_before and not collect_after and p_before == "player":
|
||||
return {"type": "collect", "desc": "收藏", "confidence": "high"}
|
||||
|
||||
# 检测搜索操作
|
||||
search_before = sel_before.get(search_key, False)
|
||||
search_after = sel_after.get(search_key, False)
|
||||
if not search_before and search_after:
|
||||
return {"type": "search", "desc": "打开搜索", "confidence": "high"}
|
||||
if search_before and not search_after:
|
||||
return {"type": "go_back", "desc": "关闭搜索", "confidence": "medium"}
|
||||
|
||||
# 检测进入小红书笔记
|
||||
note_before = sel_before.get(note_key, False)
|
||||
note_after = sel_after.get(note_key, False)
|
||||
if note_before and not note_after and p_before == "grid":
|
||||
return {"type": "xhs_click_note", "desc": "点击笔记", "confidence": "high"}
|
||||
if not note_before and note_after:
|
||||
return {"type": "go_back", "desc": "返回笔记列表", "confidence": "medium"}
|
||||
|
||||
# 检测视频播放状态变化
|
||||
video_before = sel_before.get(video_key, False)
|
||||
video_after = sel_after.get(video_key, False)
|
||||
if not video_before and video_after:
|
||||
return {"type": "enter_video", "desc": "进入视频页", "confidence": "medium"}
|
||||
if video_before and not video_after:
|
||||
return {"type": "exit_video", "desc": "退出视频", "confidence": "medium"}
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _infer_action_from_events(events: list) -> Optional[dict]:
|
||||
"""从 CDP 事件序列推断操作类型"""
|
||||
# 取关键事件:键盘 + 点击
|
||||
key_events = [e for e in events if e.get('t') == 'key']
|
||||
click_events = [e for e in events if e.get('t') == 'click']
|
||||
scroll_events = [e for e in events if e.get('t') == 'scroll']
|
||||
|
||||
# 抖音快捷键识别
|
||||
for ke in key_events:
|
||||
k = ke.get('k', '').lower()
|
||||
if k in DY_HOTKEYS:
|
||||
action = DY_HOTKEYS[k]
|
||||
desc_map = {
|
||||
'like': '点赞', 'comment': '打开评论', 'follow': '关注',
|
||||
'loop': '连播', 'mute': '静音', 'clear': '清屏',
|
||||
'fullscreen': '全屏', 'next_video': '下一个视频',
|
||||
'prev_video': '上一个视频', 'play_pause': '播放/暂停',
|
||||
}
|
||||
return {"type": action, "desc": desc_map.get(action, action),
|
||||
"trigger": f"key:{k}", "confidence": "high"}
|
||||
|
||||
# 字母键识别
|
||||
for ke in key_events:
|
||||
k = ke.get('k', '')
|
||||
if len(k) == 1 and k.isalpha():
|
||||
return {"type": f"keypress_{k}", "desc": f"按键 {k}",
|
||||
"trigger": f"key:{k}", "confidence": "medium"}
|
||||
|
||||
# 点击事件分析
|
||||
if click_events:
|
||||
last_click = click_events[-1]
|
||||
x, y = last_click.get('x', 0), last_click.get('y', 0)
|
||||
target = last_click.get('target', '')
|
||||
|
||||
# 检测区域
|
||||
for region_name, (x1, y1, x2, y2, label) in KNOWN_REGIONS.items():
|
||||
if x1 <= x <= x2 and y1 <= y <= y2:
|
||||
return {"type": f"click_{region_name}", "desc": label,
|
||||
"trigger": f"click({x},{y})", "confidence": "medium",
|
||||
"coords": {"x": x, "y": y}}
|
||||
|
||||
return {"type": "click", "desc": f"点击 ({x},{y}) <{target}>",
|
||||
"trigger": f"click({x},{y})", "confidence": "medium",
|
||||
"coords": {"x": x, "y": y}}
|
||||
|
||||
# 滚动事件
|
||||
if scroll_events and len(scroll_events) >= 2:
|
||||
total_dy = sum(e.get('dy', 0) for e in scroll_events)
|
||||
if total_dy > 200:
|
||||
return {"type": "scroll", "desc": f"下滚 (delta={total_dy})",
|
||||
"trigger": "scroll", "confidence": "medium"}
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _find_added_text(old_text: str, new_text: str) -> str:
|
||||
"""简单的文本差异检测——找出出现在新文本中但不在旧文本中的部分"""
|
||||
# 行级别 diff
|
||||
old_lines = set(old_text.split('\n'))
|
||||
new_lines = new_text.split('\n')
|
||||
added = [l.strip() for l in new_lines if l.strip() and l.strip() not in old_lines
|
||||
and len(l.strip()) > 3]
|
||||
return '\n'.join(added[:10]) if added else ''
|
||||
|
||||
|
||||
def _suggest_features(before_page: dict, after_page: dict,
|
||||
events: list, analysis: dict) -> list:
|
||||
"""从事件和页面状态生成自动化特征码建议"""
|
||||
features = []
|
||||
key_events = [e for e in events if e.get('t') == 'key']
|
||||
click_events = [e for e in events if e.get('t') == 'click']
|
||||
|
||||
# 来自推断操作的特征
|
||||
inferred = analysis.get("inferred_action", {})
|
||||
if inferred:
|
||||
trigger = inferred.get("trigger", "")
|
||||
if trigger.startswith("key:"):
|
||||
k = trigger.split(":")[1]
|
||||
features.append({
|
||||
"type": "keyboard",
|
||||
"selector": None,
|
||||
"description": f"按键 {k}",
|
||||
"code": f'await page.keyboard.press("{k}")',
|
||||
"confidence": inferred.get("confidence", "medium"),
|
||||
})
|
||||
elif "coords" in inferred:
|
||||
coords = inferred["coords"]
|
||||
# 尝试从 after_page 的 interactables 中找到最近的可点击元素
|
||||
closest = None
|
||||
for el in after_page.get("interactables", []):
|
||||
rect_str = el.get("rect", "")
|
||||
if rect_str and ',' in rect_str:
|
||||
parts = rect_str.split()
|
||||
if len(parts) >= 1:
|
||||
xy = parts[0].split(',')
|
||||
if len(xy) == 2:
|
||||
try:
|
||||
ex, ey = int(xy[0]), int(xy[1])
|
||||
dist = (coords['x'] - ex) ** 2 + (coords['y'] - ey) ** 2
|
||||
if dist < 5000 and (closest is None or dist < closest[0]):
|
||||
closest = (dist, el)
|
||||
except:
|
||||
pass
|
||||
if closest:
|
||||
el = closest[1]
|
||||
selector = el.get("e2e") or el.get("cls", "").split()[0] if el.get("cls") else None
|
||||
tag = el.get("tag", "")
|
||||
text = el.get("text", "")
|
||||
features.append({
|
||||
"type": "click",
|
||||
"selector": f'[data-e2e="{selector}"]' if selector else (f'{tag}:has-text("{text}")' if text else None),
|
||||
"description": f'点击 <{tag}> "{text[:20]}"' if text else f'点击 <{tag}>',
|
||||
"code": f'await page.locator("{selector or tag}").click()',
|
||||
"confidence": "high" if selector else "medium",
|
||||
"near_coords": f'({coords["x"]},{coords["y"]})',
|
||||
})
|
||||
else:
|
||||
features.append({
|
||||
"type": "click_coords",
|
||||
"selector": None,
|
||||
"description": f"点击坐标 ({coords['x']},{coords['y']})",
|
||||
"code": f'await page.mouse.click({coords["x"]}, {coords["y"]})',
|
||||
"confidence": "medium",
|
||||
})
|
||||
|
||||
# URL 变化特征
|
||||
if analysis.get("url_changed"):
|
||||
url_to = analysis.get("url_to", "")
|
||||
if "/video/" in url_to:
|
||||
features.append({
|
||||
"type": "navigation",
|
||||
"selector": 'a[href*="/video/"]',
|
||||
"description": "进入视频播放页",
|
||||
"code": 'await page.goto(video_url)',
|
||||
"confidence": "high",
|
||||
})
|
||||
elif "/user/profile/" in url_to:
|
||||
features.append({
|
||||
"type": "navigation",
|
||||
"selector": 'a[href*="/user/profile/"]',
|
||||
"description": "进入个人主页",
|
||||
"code": 'await page.goto(profile_url)',
|
||||
"confidence": "high",
|
||||
})
|
||||
|
||||
# 页面模式变化的特征
|
||||
if analysis.get("mode_changed"):
|
||||
features.append({
|
||||
"type": "page_mode",
|
||||
"selector": None,
|
||||
"description": f"页面模式: {analysis['mode_changed']}",
|
||||
"code": None,
|
||||
"confidence": "high",
|
||||
})
|
||||
|
||||
return features
|
||||
|
||||
|
||||
def _suggest_blueprint(actions: list) -> list:
|
||||
"""从分析结果生成蓝图建议"""
|
||||
suggestions = []
|
||||
for i, action in enumerate(actions):
|
||||
step_num = i + 1
|
||||
at = action.get("action_type", "unknown")
|
||||
|
||||
# 操作名 → 建议的原子操作op名
|
||||
op_map = {
|
||||
"like": "like", "comment": "open_comment", "follow": "follow",
|
||||
"navigate_video": "open_video", "navigate_profile": "goto_profile",
|
||||
"scroll": "scroll_feed", "next_video": "next_video",
|
||||
"click_like_area": "like", "click_comment_area": "open_comment",
|
||||
"search": "search", "enter_video": "enter_video",
|
||||
}
|
||||
|
||||
op = op_map.get(at, f"custom_{at}")
|
||||
suggestions.append({
|
||||
"step": step_num,
|
||||
"suggested_op": op,
|
||||
"action_type": at,
|
||||
"desc": action.get("action_desc", ""),
|
||||
"confidence": action.get("inferred_action", {}).get("confidence", "medium"),
|
||||
})
|
||||
return suggestions
|
||||
|
||||
|
||||
# ── 快捷入口 ──
|
||||
def analyze_recording_file(path: str) -> dict:
|
||||
"""从文件路径加载并分析录制包"""
|
||||
from mc.recorder import RecordingSession
|
||||
pkg = RecordingSession.load_recording(path)
|
||||
return analyze_recording(pkg)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
path = sys.argv[1] if len(sys.argv) > 1 else ""
|
||||
if path:
|
||||
result = analyze_recording_file(path)
|
||||
print(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
else:
|
||||
print("用法: python -m mc.analyzer <录制包路径>")
|
||||
@@ -0,0 +1,190 @@
|
||||
"""
|
||||
browser.py — 浏览器生命周期管理器
|
||||
|
||||
核心功能:
|
||||
1. Context manager: with BrowserManager() as bm:
|
||||
2. 自动检测并清理残留进程
|
||||
3. 自动清理 wal/shm 锁文件
|
||||
4. 记录 PID 到 pid_file
|
||||
5. 退出时妥善关闭 + 确认
|
||||
|
||||
用法:
|
||||
from mc.browser import BrowserManager
|
||||
|
||||
with BrowserManager() as bm:
|
||||
bm.prepare("douyin_01")
|
||||
# ... 执行操作 ...
|
||||
|
||||
# 退出时自动:
|
||||
# - 关闭浏览器
|
||||
# - 等待进程退出
|
||||
# - 清理 wal/shm
|
||||
# - 删除 pid_file
|
||||
"""
|
||||
import logging
|
||||
import os
|
||||
import signal
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
from matrix_mgmt import AGENT_LOCAL
|
||||
LOCAL_ROOT = AGENT_LOCAL / "tools" / "matrix"
|
||||
PID_DIR = Path("/tmp") / "mc_pids"
|
||||
|
||||
|
||||
class BrowserManager:
|
||||
"""浏览器生命周期管理器"""
|
||||
|
||||
def __init__(self):
|
||||
self._managed_pids: list[int] = []
|
||||
PID_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def __enter__(self):
|
||||
log.info("🚀 BrowserManager 启动")
|
||||
self._cleanup_stale()
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc_val, exc_tb):
|
||||
log.info("🛑 BrowserManager 关闭中...")
|
||||
self._close_all()
|
||||
self._cleanup_pid_files()
|
||||
log.info("✅ BrowserManager 已关闭")
|
||||
|
||||
# ──────────────── 公共方法 ────────────────
|
||||
|
||||
def prepare(self, identity: str) -> dict:
|
||||
"""准备某个身份的浏览器环境
|
||||
|
||||
Args:
|
||||
identity: 身份目录名 (douyin_01, douyin_04 等)
|
||||
|
||||
Returns:
|
||||
{"status": "ok", "pid": pid} 或 {"status": "stale", "pid": old_pid}
|
||||
"""
|
||||
pid_file = PID_DIR / f"{identity}.pid"
|
||||
|
||||
# 检查是否有残留进程
|
||||
if pid_file.exists():
|
||||
old_pid = int(pid_file.read_text().strip())
|
||||
if self._is_process_alive(old_pid):
|
||||
log.warning(f" ⚠️ {identity} 已有残留进程 PID={old_pid},清理中...")
|
||||
self._kill_process(old_pid)
|
||||
pid_file.unlink(missing_ok=True)
|
||||
|
||||
# 清理 cookie 锁文件
|
||||
self._cleanup_cookie_locks(identity)
|
||||
|
||||
# 清理 Camoufox profile lock
|
||||
self._cleanup_profile_locks(identity)
|
||||
|
||||
return {"status": "ok"}
|
||||
|
||||
def register_pid(self, identity: str, pid: int):
|
||||
"""记录进程 PID"""
|
||||
pid_file = PID_DIR / f"{identity}.pid"
|
||||
pid_file.write_text(str(pid))
|
||||
self._managed_pids.append(pid)
|
||||
log.info(f" 📝 {identity} PID={pid} 已注册")
|
||||
|
||||
def is_healthy(self, identity: str) -> bool:
|
||||
"""检查某个身份对应的浏览器是否存活"""
|
||||
pid_file = PID_DIR / f"{identity}.pid"
|
||||
if not pid_file.exists():
|
||||
return False
|
||||
try:
|
||||
pid = int(pid_file.read_text().strip())
|
||||
return self._is_process_alive(pid)
|
||||
except:
|
||||
return False
|
||||
|
||||
# ──────────────── 内部方法 ────────────────
|
||||
|
||||
def _is_process_alive(self, pid: int) -> bool:
|
||||
"""检查进程是否存在"""
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
return True
|
||||
except (OSError, ProcessLookupError):
|
||||
return False
|
||||
|
||||
def _kill_process(self, pid: int, force: bool = False):
|
||||
"""安全终止进程"""
|
||||
try:
|
||||
if force:
|
||||
os.kill(pid, signal.SIGKILL)
|
||||
log.info(f" 💀 强制终止 PID={pid}")
|
||||
else:
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
# 等待最多 5 秒
|
||||
for _ in range(10):
|
||||
if not self._is_process_alive(pid):
|
||||
break
|
||||
time.sleep(0.5)
|
||||
if self._is_process_alive(pid):
|
||||
os.kill(pid, signal.SIGKILL)
|
||||
log.info(f" 💀 SIGTERM 超时,SIGKILL PID={pid}")
|
||||
else:
|
||||
log.info(f" ✅ PID={pid} 已正常退出")
|
||||
except ProcessLookupError:
|
||||
pass # 进程已不存在
|
||||
except Exception as e:
|
||||
log.warning(f" ⚠️ 终止 PID={pid} 失败: {e}")
|
||||
|
||||
def _cleanup_stale(self):
|
||||
"""清理所有残留的 mc 管理进程"""
|
||||
for pid_file in PID_DIR.glob("*.pid"):
|
||||
try:
|
||||
pid = int(pid_file.read_text().strip())
|
||||
if not self._is_process_alive(pid):
|
||||
pid_file.unlink(missing_ok=True)
|
||||
log.info(f" 🧹 清除残留 PID 文件: {pid_file.name}")
|
||||
except:
|
||||
pid_file.unlink(missing_ok=True)
|
||||
|
||||
def _cleanup_cookie_locks(self, identity: str):
|
||||
"""清理 cookie 数据库的 wal/shm 锁文件"""
|
||||
cookie_dir = LOCAL_ROOT / "identities" / identity / "user_data"
|
||||
for ext in ["-wal", "-shm"]:
|
||||
f = cookie_dir / f"cookies.sqlite{ext}"
|
||||
if f.exists():
|
||||
try:
|
||||
# 检查是否被进程占用
|
||||
f.unlink()
|
||||
log.info(f" 🧹 清理锁文件: {f.name}")
|
||||
except OSError:
|
||||
log.warning(f" ⚠️ 锁文件被占用,跳过: {f.name}")
|
||||
|
||||
def _cleanup_profile_locks(self, identity: str):
|
||||
"""清理 Camoufox profile 锁文件"""
|
||||
profile_dir = LOCAL_ROOT / "identities" / identity / "user_data"
|
||||
for lock_file in [".parentlock", "lock", "parent.lock"]:
|
||||
f = profile_dir / lock_file
|
||||
if f.exists():
|
||||
try:
|
||||
f.unlink()
|
||||
log.info(f" 🧹 清理 profile 锁: {lock_file}")
|
||||
except:
|
||||
pass
|
||||
|
||||
def _close_all(self):
|
||||
"""关闭所有管理的浏览器进程"""
|
||||
for pid in self._managed_pids:
|
||||
if self._is_process_alive(pid):
|
||||
self._kill_process(pid)
|
||||
self._managed_pids.clear()
|
||||
|
||||
def _cleanup_pid_files(self):
|
||||
"""清理所有 PID 文件"""
|
||||
for pid_file in PID_DIR.glob("*.pid"):
|
||||
pid_file.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def cleanup_all():
|
||||
"""一键清理所有残留(供 Guardd 定时调用)"""
|
||||
with BrowserManager() as bm:
|
||||
bm._cleanup_stale()
|
||||
return {"status": "ok", "message": "所有残留已清理"}
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,718 @@
|
||||
"""
|
||||
engine.py — mc run 执行引擎(v5.3 — 身份共享 + 多平台同浏览器)
|
||||
|
||||
核心逻辑:
|
||||
1. 按 identity_dir 分组账号 → 同身份共用一个浏览器
|
||||
2. 每个浏览器内先做完抖音,再做小红书
|
||||
3. 不同身份组之间错峰启动(防假死)
|
||||
4. 每个账号在其平台上顺序执行蓝图
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import random
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent.parent
|
||||
TOOL_DIR = SCRIPTS_DIR.parent
|
||||
from matrix_mgmt import AGENT_SYNC, AGENT_LOCAL
|
||||
|
||||
IDENTITIES_ROOT = AGENT_LOCAL / "tools" / "matrix" / "identities"
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 工具函数
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
def resolve_account(account_id: str) -> dict:
|
||||
from matrix_mgmt import MatrixManager
|
||||
mgr = MatrixManager()
|
||||
for a in mgr.list_accounts():
|
||||
if a["id"] == account_id:
|
||||
return a
|
||||
raise ValueError(f"账号不存在: {account_id}")
|
||||
|
||||
|
||||
def resolve_blueprint(name: str) -> Optional[dict]:
|
||||
bp_dir = TOOL_DIR / "blueprints"
|
||||
# 长名字优先(避免 douyin_daily 被子串匹配到 douyin_daily_clean)
|
||||
for f in sorted(bp_dir.glob("*.json"), key=lambda p: -len(p.stem)):
|
||||
bp = json.loads(f.read_text())
|
||||
if name in (bp.get("id", ""), bp.get("name", ""), f.stem):
|
||||
return bp
|
||||
if name in f.stem or f.stem in name:
|
||||
return json.loads(f.read_text())
|
||||
return None
|
||||
|
||||
|
||||
def merge_blueprints(names: str) -> Optional[dict]:
|
||||
"""组合蓝图:把多个蓝图(+ 分隔)合并为一个执行序列
|
||||
|
||||
规则:
|
||||
1. 第一个 goto_url 保留,后续蓝图的 goto_url 跳过(页面已在目标视频)
|
||||
2. 连续 wait/wait_watch 合并(取较大 seconds)
|
||||
"""
|
||||
bps = []
|
||||
for n in names.split("+"):
|
||||
bp = resolve_blueprint(n.strip())
|
||||
if bp:
|
||||
bps.append(bp)
|
||||
if not bps:
|
||||
return None
|
||||
merged_steps = []
|
||||
seen_url_goto = False
|
||||
prev_was_wait = False
|
||||
for bp in bps:
|
||||
for s in bp.get("steps", []):
|
||||
op = s.get("op", "")
|
||||
if op == "goto_url":
|
||||
if seen_url_goto:
|
||||
continue # 已在目标视频,跳过重复导航
|
||||
seen_url_goto = True
|
||||
prev_was_wait = False
|
||||
merged_steps.append(s)
|
||||
elif op in ("wait", "wait_watch"):
|
||||
if prev_was_wait:
|
||||
# 合并连续等待:保留较大 seconds
|
||||
try:
|
||||
prev_s = float((merged_steps[-1].get("args", {}) or {}).get("seconds") or 0)
|
||||
cur_s = float((s.get("args", {}) or {}).get("seconds") or 0)
|
||||
merged_steps[-1]["args"]["seconds"] = max(prev_s, cur_s)
|
||||
except Exception:
|
||||
pass
|
||||
continue
|
||||
prev_was_wait = True
|
||||
merged_steps.append(s)
|
||||
else:
|
||||
prev_was_wait = False
|
||||
merged_steps.append(s)
|
||||
return {
|
||||
"id": "+".join(b["id"] for b in bps),
|
||||
"name": "+".join(b["name"] for b in bps),
|
||||
"description": "组合蓝图(多动作连做)",
|
||||
"platform": bps[0].get("platform", "douyin"),
|
||||
"steps": merged_steps,
|
||||
}
|
||||
|
||||
|
||||
def check_cookie(identity_hint: str) -> str:
|
||||
"""检查身份目录 cookie 状态 → 'ok'|'no_cookie'|'expired'|'no_identity'"""
|
||||
if not identity_hint:
|
||||
return "no_identity"
|
||||
ident_dir = IDENTITIES_ROOT / identity_hint
|
||||
if not ident_dir.exists():
|
||||
return "no_identity"
|
||||
ck = ident_dir / "user_data" / "cookies.sqlite"
|
||||
if not ck.exists() or ck.stat().st_size < 100:
|
||||
return "no_cookie"
|
||||
try:
|
||||
import sqlite3
|
||||
conn_ck = sqlite3.connect(str(ck), timeout=2)
|
||||
cnt = conn_ck.execute("SELECT count(*) FROM moz_cookies WHERE name LIKE '%session%'").fetchone()[0]
|
||||
conn_ck.close()
|
||||
return "ok" if cnt > 0 else "expired"
|
||||
except:
|
||||
return "error"
|
||||
|
||||
|
||||
def group_accounts_by_identity(account_ids: List[str]) -> dict:
|
||||
"""按 identity_dir 分组账号,同目录的放到一个组里"""
|
||||
groups = {}
|
||||
for aid in account_ids:
|
||||
try:
|
||||
info = resolve_account(aid)
|
||||
except ValueError:
|
||||
continue
|
||||
ident = info.get("identity_dir", aid).replace("identities/", "")
|
||||
groups.setdefault(ident, []).append(info)
|
||||
return groups
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 执行报告
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
class StepResult:
|
||||
def __init__(self, op: str, step_id: int, success: bool = True,
|
||||
duration: float = 0, error: str = ""):
|
||||
self.op = op; self.step_id = step_id; self.success = success
|
||||
self.duration = duration; self.error = error
|
||||
|
||||
def __str__(self):
|
||||
icon = "✅" if self.success else "❌"
|
||||
return f"{icon} [{self.step_id:2d}] {self.op:20s} ({self.duration:.1f}s) {self.error}"
|
||||
|
||||
|
||||
class AccountRunReport:
|
||||
def __init__(self, account: str, blueprint: str, round_idx: int):
|
||||
self.account = account; self.blueprint = blueprint; self.round = round_idx
|
||||
self.steps: List[StepResult] = []; self.start = time.time()
|
||||
self.success = 0; self.failed = 0; self.skipped = False
|
||||
|
||||
@property
|
||||
def duration(self) -> float: return time.time() - self.start
|
||||
|
||||
def add_step(self, result: StepResult):
|
||||
self.steps.append(result)
|
||||
if result.success: self.success += 1
|
||||
else: self.failed += 1
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {"account": self.account, "blueprint": self.blueprint,
|
||||
"round": self.round, "success": self.success, "failed": self.failed,
|
||||
"skipped": self.skipped, "duration": round(self.duration, 1),
|
||||
"steps": [{"op": s.op, "step_id": s.step_id, "success": s.success,
|
||||
"duration": round(s.duration, 1), "error": s.error} for s in self.steps]}
|
||||
|
||||
|
||||
class BatchReport:
|
||||
def __init__(self):
|
||||
self.accounts: List[str] = []; self.blueprints: List[str] = []
|
||||
self.rounds_total = 0; self.mix_mode = False
|
||||
self.start = time.time(); self.account_reports: List[AccountRunReport] = []
|
||||
self.error: Optional[str] = None
|
||||
|
||||
@property
|
||||
def duration(self) -> float: return time.time() - self.start
|
||||
@property
|
||||
def success(self) -> int: return sum(r.success for r in self.account_reports)
|
||||
@property
|
||||
def failed(self) -> int: return sum(r.failed for r in self.account_reports)
|
||||
@property
|
||||
def skipped(self) -> int: return sum(1 for r in self.account_reports if r.skipped)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {"accounts": self.accounts, "blueprints": self.blueprints,
|
||||
"rounds": self.rounds_total, "mix_mode": self.mix_mode,
|
||||
"duration": round(self.duration, 1),
|
||||
"total_steps": self.success + self.failed,
|
||||
"success": self.success, "failed": self.failed,
|
||||
"skipped": self.skipped,
|
||||
"account_reports": [r.to_dict() for r in self.account_reports],
|
||||
"error": self.error}
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 核心引擎
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
# 平台→默认蓝图映射(当不传蓝图时自动匹配)
|
||||
PLATFORM_BLUEPRINTS = {
|
||||
"douyin": "douyin_read_profile",
|
||||
"xiaohongshu": "xiaohongshu_read_profile",
|
||||
}
|
||||
PLATFORM_DAILY_BLUEPRINTS = {
|
||||
"douyin": "douyin_daily",
|
||||
"xiaohongshu": "xhs_daily",
|
||||
}
|
||||
|
||||
|
||||
class BatchEngine:
|
||||
def __init__(self, accounts: List[str], blueprints: List[str] = None,
|
||||
rounds: int = 10, mix: bool = False,
|
||||
corpus: List[str] = None,
|
||||
stagger: str = "15-30",
|
||||
keep_open: bool = False,
|
||||
max_browsers: int = 3,
|
||||
run_id: str = "",
|
||||
task_params: dict = None):
|
||||
self.accounts = accounts
|
||||
self.blueprints = blueprints or []
|
||||
self.rounds_total = rounds
|
||||
self.mix = mix
|
||||
self.corpus = corpus or []
|
||||
self.stagger = stagger
|
||||
self.keep_open = keep_open
|
||||
self.max_browsers = max_browsers
|
||||
self.run_id = run_id
|
||||
self.task_params = task_params or {}
|
||||
|
||||
def _mark_banned(self, account_id: str, platform: str):
|
||||
"""将账号标记为封号状态,写入 profiles.json"""
|
||||
try:
|
||||
from pathlib import Path
|
||||
home = Path.home()
|
||||
profiles_path = home / "workbuddy-agent-os" / "agent-local" / "tools" / "matrix" / "data" / "profiles.json"
|
||||
if profiles_path.exists():
|
||||
import json
|
||||
all_p = json.loads(profiles_path.read_text())
|
||||
else:
|
||||
all_p = {}
|
||||
all_p[account_id] = {
|
||||
"nickname": all_p.get(account_id, {}).get("nickname", "?"),
|
||||
"fans": "0", "following": "0", "likes": "0", "posts": "0", "bio": "",
|
||||
"status": "banned",
|
||||
"platform": platform,
|
||||
"updated": __import__("time").strftime("%Y-%m-%dT%H:%M:%S"),
|
||||
}
|
||||
profiles_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
profiles_path.write_text(json.dumps(all_p, ensure_ascii=False, indent=2))
|
||||
log.warning(f" 🚫 [{account_id}] 已写入封号标记到 profiles.json")
|
||||
except Exception as e:
|
||||
log.warning(f" ❌ [{account_id}] 写入封号标记失败: {e}")
|
||||
|
||||
def _pick_blueprint(self, round_idx: int) -> str:
|
||||
if not self.blueprints:
|
||||
return "" # 由 _run_acct_on_conn 按平台自动匹配
|
||||
if self.mix:
|
||||
return random.choice(self.blueprints)
|
||||
return self.blueprints[(round_idx - 1) % len(self.blueprints)]
|
||||
|
||||
def _resolve_args(self, sargs: dict, account_id: str, platform: str) -> dict:
|
||||
"""解析蓝图步骤中的占位符,替换为实际内容"""
|
||||
import copy
|
||||
resolved = copy.deepcopy(sargs)
|
||||
for k, v in resolved.items():
|
||||
if not isinstance(v, str):
|
||||
continue
|
||||
if "@corpus" in v:
|
||||
# 优先使用命令行/计划指定的评论内容(--comment-text),未指定才从语料库随机取
|
||||
ct = (self.task_params.get("comment_text") or "").strip()
|
||||
if ct:
|
||||
resolved[k] = v.replace("@corpus", ct)
|
||||
else:
|
||||
from mc.corpus import CorpusManager
|
||||
cm = CorpusManager()
|
||||
direction = resolved.get("direction", "") or self.task_params.get("direction", "")
|
||||
text = cm.get_comment_for_video(
|
||||
video_title=self.task_params.get("keyword", ""),
|
||||
direction=direction,
|
||||
account_id=account_id,
|
||||
)
|
||||
resolved[k] = v.replace("@corpus", text)
|
||||
if "@keyword" in v:
|
||||
kw = self.task_params.get("keyword", "")
|
||||
resolved[k] = v.replace("@keyword", kw)
|
||||
if "@direction" in v:
|
||||
dr = self.task_params.get("direction", "")
|
||||
resolved[k] = v.replace("@direction", dr)
|
||||
if "@url" in v:
|
||||
url = self.task_params.get("url", "")
|
||||
resolved[k] = v.replace("@url", url)
|
||||
if "@comment_text" in v:
|
||||
ct = self.task_params.get("comment_text", "")
|
||||
if ct:
|
||||
resolved[k] = v.replace("@comment_text", ct)
|
||||
if "@reply_text" in v:
|
||||
rt = self.task_params.get("reply_text", "")
|
||||
if rt:
|
||||
resolved[k] = v.replace("@reply_text", rt)
|
||||
if "@my_code" in v:
|
||||
mc = self.task_params.get("my_code", "")
|
||||
if mc:
|
||||
resolved[k] = v.replace("@my_code", mc)
|
||||
if "@target_code" in v:
|
||||
tc = self.task_params.get("target_code", "")
|
||||
if tc:
|
||||
resolved[k] = v.replace("@target_code", tc)
|
||||
if "@scene" in v:
|
||||
scene = self.task_params.get("scene", "first_comment")
|
||||
from mc.corpus import CorpusManager
|
||||
cm = CorpusManager()
|
||||
text = cm.get_comment_for_scene(
|
||||
persona=self.task_params.get("persona", ""),
|
||||
scene=scene,
|
||||
keyword=self.task_params.get("keyword", ""),
|
||||
round_num=self.task_params.get("round_num", 1),
|
||||
)
|
||||
if text:
|
||||
resolved[k] = v.replace("@scene", text)
|
||||
return resolved
|
||||
|
||||
async def _run_acct_on_conn(self, account_info: dict, blueprint_name: str,
|
||||
round_idx: int, conn) -> AccountRunReport:
|
||||
"""在已有浏览器连接上执行单个账号的单个轮次"""
|
||||
account_id = account_info["id"]
|
||||
platform = account_info.get("platform", "douyin")
|
||||
|
||||
# 没传蓝图时按平台自动匹配(支持不同平台账号同一次批量执行)
|
||||
if not blueprint_name:
|
||||
blueprint_name = PLATFORM_BLUEPRINTS.get(platform, "douyin_read_profile")
|
||||
report = AccountRunReport(account_id, blueprint_name, round_idx)
|
||||
|
||||
# 组合蓝图(+ 分隔):合并多个蓝图为同一视频连做序列
|
||||
if "+" in blueprint_name:
|
||||
bp = merge_blueprints(blueprint_name)
|
||||
log.info(f" 🧬 组合蓝图: {blueprint_name} → {len(bp['steps']) if bp else 0} 步")
|
||||
else:
|
||||
bp = resolve_blueprint(blueprint_name)
|
||||
# 蓝图与账号平台不匹配→自动切换(例如xhs账号用了douyin_daily_clean)
|
||||
if bp and bp.get("platform") and bp["platform"] != platform:
|
||||
old_bp = blueprint_name
|
||||
blueprint_name = PLATFORM_DAILY_BLUEPRINTS.get(platform, blueprint_name)
|
||||
log.info(f" 🔄 蓝图 [{old_bp}] 与平台 [{platform}] 不匹配,自动切换为 [{blueprint_name}]")
|
||||
bp = resolve_blueprint(blueprint_name)
|
||||
if not bp:
|
||||
report.skipped = True
|
||||
return report
|
||||
steps = bp.get("steps", [])
|
||||
log.info(f" 📱 {account_id} ({platform}) → {blueprint_name} ({len(steps)}步)")
|
||||
|
||||
# ── 平台区分:导航到不同入口再检测登录 ──
|
||||
if platform == "xiaohongshu":
|
||||
# 小红书:导航到 explore 首页,依赖 Cookie 检测
|
||||
try:
|
||||
await conn.page.goto("https://www.xiaohongshu.com/explore",
|
||||
timeout=30000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(5)
|
||||
except:
|
||||
await conn.page.goto("https://www.xiaohongshu.com/explore",
|
||||
timeout=60000, wait_until="load")
|
||||
await asyncio.sleep(5)
|
||||
else:
|
||||
# 抖音:导航到 user/self 强制触发登录弹窗(受保护页面,未登录必弹)
|
||||
# 避免在首页等待弹窗随机出现导致检测不稳定
|
||||
try:
|
||||
await conn.page.goto("https://www.douyin.com/user/self",
|
||||
timeout=30000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(6)
|
||||
except:
|
||||
await conn.page.goto("https://www.douyin.com/user/self",
|
||||
timeout=60000, wait_until="load")
|
||||
await asyncio.sleep(6)
|
||||
|
||||
# ── 钩子1: 登录状态检测(执行前确保登录)──
|
||||
if bp and bp.get("skip_sms"):
|
||||
from matrix_modules.account.login_state_machine import (
|
||||
LoginStateMachine, RecoveryChain, CookieRecovery, VisualRecovery
|
||||
)
|
||||
lsm = LoginStateMachine(recovery_chain=RecoveryChain(steps=[
|
||||
CookieRecovery(),
|
||||
VisualRecovery(),
|
||||
]))
|
||||
else:
|
||||
from matrix_modules.account.login_state_machine import LoginStateMachine
|
||||
lsm = LoginStateMachine()
|
||||
phone = None
|
||||
login_ok = await lsm.ensure_login(conn.page, account_id, platform)
|
||||
# skip_sms 模式:登录后立即检查是否有 SMS 弹窗(服务器主动触发)
|
||||
if login_ok and bp and bp.get("skip_sms"):
|
||||
sms_popup = await lsm.check_verify_dialog(conn.page)
|
||||
if sms_popup == "sms":
|
||||
log.warning(f" ⏭️ [{account_id}] 服务器弹出短信验证,skip_sms 跳过,关闭浏览器")
|
||||
await conn.close()
|
||||
return report
|
||||
if not login_ok:
|
||||
# 检测是否被封号
|
||||
if hasattr(lsm, 'last_status') and lsm.last_status == 'banned':
|
||||
log.warning(f" 🚫 [{account_id}] 账号被封禁,写入标记")
|
||||
self._mark_banned(account_id, platform)
|
||||
else:
|
||||
log.warning(f" ❌ [{account_id}] 登录检测不通过,跳过本轮")
|
||||
report.skipped = True
|
||||
return report
|
||||
|
||||
# 创建 platform ops
|
||||
if platform == "xiaohongshu":
|
||||
from ops.xhs_ops import XhsOps
|
||||
ops = XhsOps(conn.page)
|
||||
else:
|
||||
from douyin_ops import DouyinOps
|
||||
ops = DouyinOps(conn.page)
|
||||
print(f"[engine] set_account_id({account_id}) for platform={platform}")
|
||||
ops.set_account_id(account_id)
|
||||
|
||||
# 抖音:进入视频播放
|
||||
# 如果蓝图第一步是 goto_url(@url),用户提供了目标 URL,
|
||||
# 跳过 open_video 预调用(避免点开随机视频又导航走,浪费时间)
|
||||
_needs_open_video = True
|
||||
if steps and steps[0].get("op") == "goto_url":
|
||||
_args = steps[0].get("args", {})
|
||||
_url_arg = _args.get("url", "")
|
||||
if "@url" in _url_arg:
|
||||
_needs_open_video = False
|
||||
log.info(f" ⏭️ 跳过 open_video(蓝图第一步使用 @url 跳转到目标视频)")
|
||||
if platform == "douyin" and _needs_open_video:
|
||||
await ops.execute("open_video")
|
||||
|
||||
# 执行步骤
|
||||
for i, step in enumerate(steps):
|
||||
sn = step.get("step_id", i + 1)
|
||||
op_name = step.get("op", "")
|
||||
sargs = self._resolve_args(step.get("args", {}), account_id, platform)
|
||||
result = await ops.execute(op=op_name, args=sargs, step_id=sn)
|
||||
report.add_step(StepResult(op_name, sn, result.success, result.elapsed,
|
||||
"" if result.success else result.error))
|
||||
icon = "✅" if result.success else "❌"
|
||||
log.info(f" {icon} [{sn:2d}] {op_name:18s} → {result.detail[:25]} ({result.elapsed:.1f}s)")
|
||||
|
||||
# ── 钩子2: 操作后检查验证弹窗 + 登录态检查 ──
|
||||
verify_type = await lsm.check_verify_dialog(conn.page)
|
||||
if verify_type == "login_required":
|
||||
log.warning(f" 🔑 [{account_id}] 登录态丢失(页面重定向到登录页),恢复中...")
|
||||
# 尝试恢复登录态,最多重试2次
|
||||
recovered = False
|
||||
for retry in range(2):
|
||||
if retry > 0:
|
||||
log.info(f" 🔄 [{account_id}] 登录恢复重试 ({retry+1}/2)...")
|
||||
ok = await lsm.ensure_login(conn.page, account_id, platform)
|
||||
if ok:
|
||||
recovered = True
|
||||
break
|
||||
await asyncio.sleep(3)
|
||||
if recovered:
|
||||
log.info(f" ✅ [{account_id}] 登录恢复成功,重试当前操作 [{sn}]")
|
||||
# 重试当前失败的步骤
|
||||
result = await ops.execute(op=op_name, args=sargs, step_id=sn)
|
||||
report.add_step(StepResult(op_name, sn, result.success, result.elapsed,
|
||||
"" if result.success else result.error))
|
||||
icon = "✅" if result.success else "❌"
|
||||
log.info(f" {icon} [{sn:2d}] {op_name:18s} → {result.detail[:25]} (重试, {result.elapsed:.1f}s)")
|
||||
else:
|
||||
log.warning(f" ❌ [{account_id}] 登录恢复失败,跳过当前操作 [{sn}]")
|
||||
elif verify_type == "quick_login":
|
||||
log.warning(f" 🔑 [{account_id}] 检测到一键登录弹窗,点击一键登录...")
|
||||
try:
|
||||
await conn.page.locator('text="一键登录"').click(timeout=5000)
|
||||
await asyncio.sleep(3)
|
||||
# 链式检查:点完一键登录后可能弹出短信验证码或仍需完整登录
|
||||
verify2 = await lsm.check_verify_dialog(conn.page)
|
||||
if verify2 == "sms":
|
||||
log.warning(f" 📱 [{account_id}] 一键登录后弹出短信验证码,自动处理...")
|
||||
await lsm.recover_sms(conn.page, account_id)
|
||||
log.info(f" ✅ [{account_id}] 短信验证完毕,继续执行")
|
||||
elif verify2 == "login_required":
|
||||
log.warning(f" 🔑 [{account_id}] 一键登录后仍需完整登录,走恢复链")
|
||||
ok = await lsm.ensure_login(conn.page, account_id, platform)
|
||||
if ok:
|
||||
log.info(f" ✅ [{account_id}] 登录成功,重试当前操作 [{sn}]")
|
||||
result = await ops.execute(op=op_name, args=sargs, step_id=sn)
|
||||
report.add_step(StepResult(op_name, sn, result.success, result.elapsed,
|
||||
"" if result.success else result.error))
|
||||
icon = "✅" if result.success else "❌"
|
||||
log.info(f" {icon} [{sn:2d}] {op_name:18s} → {result.detail[:25]} (重试, {result.elapsed:.1f}s)")
|
||||
else:
|
||||
log.warning(f" ❌ [{account_id}] 完整登录恢复失败,跳过当前操作 [{sn}]")
|
||||
else:
|
||||
log.info(f" ✅ [{account_id}] 已点击一键登录,继续执行")
|
||||
except Exception as e:
|
||||
log.warning(f" ⚠️ [{account_id}] 点击一键登录失败: {e}, 走完整恢复链")
|
||||
ok = await lsm.ensure_login(conn.page, account_id, platform)
|
||||
if ok:
|
||||
log.info(f" ✅ [{account_id}] 登录恢复成功,重试当前操作 [{sn}]")
|
||||
result = await ops.execute(op=op_name, args=sargs, step_id=sn)
|
||||
report.add_step(StepResult(op_name, sn, result.success, result.elapsed,
|
||||
"" if result.success else result.error))
|
||||
icon = "✅" if result.success else "❌"
|
||||
log.info(f" {icon} [{sn:2d}] {op_name:18s} → {result.detail[:25]} (重试, {result.elapsed:.1f}s)")
|
||||
elif verify_type == "sms":
|
||||
if bp.get("skip_sms"):
|
||||
log.warning(f" ⏭️ [{account_id}] 蓝图标记跳过短信验证,关闭浏览器")
|
||||
await conn.close()
|
||||
return report
|
||||
log.warning(f" 📱 [{account_id}] 触发短信验证,自动恢复...")
|
||||
await lsm.recover_sms(conn.page, account_id)
|
||||
log.info(f" ⏭️ 跳过当前操作 [{sn}]")
|
||||
elif verify_type == "captcha":
|
||||
log.warning(f" 🔐 [{account_id}] 滑块验证,需手动处理")
|
||||
# 上报但不中断(留给你手动处理)
|
||||
|
||||
# ── 钩子3: 冷却管理 ──
|
||||
cooldown = self._get_cooldown(op_name)
|
||||
await asyncio.sleep(cooldown)
|
||||
|
||||
return report
|
||||
|
||||
def _get_cooldown(self, op_name: str) -> float:
|
||||
"""操作完成后冷却时间(自动去掉 xhs_/dy_ 前缀匹配)"""
|
||||
import random, re
|
||||
COOLDOWNS = {
|
||||
"like": (2, 4),
|
||||
"collect": (3, 6),
|
||||
"comment": (30, 45),
|
||||
"post_comment": (30, 45),
|
||||
"follow": (10, 20),
|
||||
"search": (3, 6),
|
||||
"scroll_feed": (2, 3),
|
||||
"wait_watch": (0, 0), # wait_watch 自带等待
|
||||
}
|
||||
# 去掉 xhs_/dy_ 前缀再匹配
|
||||
base_name = re.sub(r'^(xhs_|dy_|xhs_|dy_)', '', op_name)
|
||||
base = COOLDOWNS.get(base_name, COOLDOWNS.get(op_name, (2, 5)))
|
||||
return random.uniform(*base)
|
||||
|
||||
async def _run_identity_group(self, group_accts: List[dict]) -> List[AccountRunReport]:
|
||||
"""运行同一身份下的所有账号(共用一个浏览器)"""
|
||||
reports = []
|
||||
identity_dir = group_accts[0].get("identity_dir", group_accts[0]["id"]).replace("identities/", "")
|
||||
|
||||
# ── 浏览器数限制检查(L2 层,谁开浏览器谁检查)──
|
||||
import subprocess as _sp
|
||||
while True:
|
||||
try:
|
||||
out = _sp.run("pgrep camoufox | wc -l", shell=True, capture_output=True, text=True, timeout=5)
|
||||
count = int(out.stdout.strip() or 0)
|
||||
except:
|
||||
count = 0
|
||||
if count < self.max_browsers:
|
||||
break
|
||||
log.warning(f" ⏳ 已达最大浏览器数({self.max_browsers}),等待其他任务释放...")
|
||||
await asyncio.sleep(15)
|
||||
|
||||
if check_cookie(identity_dir) == "no_identity":
|
||||
log.warning(f" ⏭️ 身份 {identity_dir}: 身份目录不存在,全部跳过")
|
||||
for acct in group_accts:
|
||||
for r in range(1, self.rounds_total + 1):
|
||||
rpt = AccountRunReport(acct["id"], "", r)
|
||||
rpt.skipped = True
|
||||
reports.append(rpt)
|
||||
return reports
|
||||
|
||||
# cookie 不存在或过期 → 让 ensure_login 恢复链处理
|
||||
ck_status = check_cookie(identity_dir)
|
||||
if ck_status in ("no_cookie", "expired", "error"):
|
||||
# skip_sms 模式:cookie 过期不打开浏览器,直接跳过(防服务器主动发短信)
|
||||
_skip_sms_bp = False
|
||||
if self.blueprints:
|
||||
_bp_data = resolve_blueprint(self.blueprints[0])
|
||||
if _bp_data and _bp_data.get("skip_sms"):
|
||||
_skip_sms_bp = True
|
||||
if _skip_sms_bp:
|
||||
log.warning(f" ⏭️ [{identity_dir}] skip_sms + cookie {ck_status},跳过(不打开浏览器)")
|
||||
for acct in group_accts:
|
||||
for r in range(1, self.rounds_total + 1):
|
||||
rpt = AccountRunReport(acct["id"], "", r)
|
||||
rpt.skipped = True
|
||||
reports.append(rpt)
|
||||
return reports
|
||||
log.info(f" 🔐 [{identity_dir}] cookie 状态需恢复,由 LoginStateMachine 处理")
|
||||
|
||||
# 分配窗口位置槽位
|
||||
try:
|
||||
from mc.execution_policy import slot_for
|
||||
slot = slot_for(group_accts[0]["id"])
|
||||
win_pos = tuple(slot["position"]) if slot else (0, 0)
|
||||
win_size = tuple(slot["size"]) if slot else (802, 783)
|
||||
except Exception:
|
||||
win_pos, win_size = (0, 0), (802, 783)
|
||||
|
||||
from cdp_connector import CDPConnector
|
||||
conn = CDPConnector(browser_type="camoufox", headless=False,
|
||||
window=win_size,
|
||||
window_position=win_pos,
|
||||
identity_dir=str(IDENTITIES_ROOT / identity_dir))
|
||||
try:
|
||||
await conn.connect()
|
||||
await conn.init_anti_detection()
|
||||
log.info(f" 🖥️ 浏览器已启动 [身份: {identity_dir}]")
|
||||
|
||||
# 按平台排序:先抖音后小红书
|
||||
sorted_accts = sorted(group_accts, key=lambda a: (a.get("platform", "") == "xiaohongshu", a.get("platform", "")))
|
||||
|
||||
for acct_info in sorted_accts:
|
||||
acct_id = acct_info["id"]
|
||||
for round_idx in range(1, self.rounds_total + 1):
|
||||
bp_name = self._pick_blueprint(round_idx)
|
||||
rpt = await self._run_acct_on_conn(acct_info, bp_name, round_idx, conn)
|
||||
reports.append(rpt)
|
||||
if round_idx < self.rounds_total:
|
||||
wait = random.uniform(5, 10)
|
||||
log.info(f" ⏳ [{acct_id}] 等待 {wait:.0f}s...")
|
||||
await asyncio.sleep(wait)
|
||||
|
||||
except Exception as e:
|
||||
log.error(f" ❌ 身份 {identity_dir}: {e}")
|
||||
for acct in group_accts:
|
||||
if not any(r.account == acct["id"] and r.blueprint for r in reports):
|
||||
rpt = AccountRunReport(acct["id"], "", 1)
|
||||
rpt.failed = 99
|
||||
reports.append(rpt)
|
||||
finally:
|
||||
if not self.keep_open:
|
||||
try:
|
||||
# 先给 8 秒优雅关闭
|
||||
await asyncio.wait_for(conn.close(), timeout=8)
|
||||
log.info(f" 🛑 浏览器已关闭 [身份: {identity_dir}]")
|
||||
except (asyncio.TimeoutError, Exception) as e:
|
||||
log.warning(f" ⚠️ 浏览器关闭超时 [{identity_dir}]: {e}")
|
||||
# 无论 close 成不成功,强制杀本身份的浏览器进程
|
||||
# 注意: identity_dir = 手机号/账号名, 唯一且不含空格, 不会误杀
|
||||
subprocess.run(["pkill", "-f", identity_dir],
|
||||
capture_output=True, timeout=3)
|
||||
await asyncio.sleep(2)
|
||||
# 验证
|
||||
for _ in range(3):
|
||||
chk = subprocess.run(["pgrep", "-f", identity_dir],
|
||||
capture_output=True, text=True, timeout=2)
|
||||
if not chk.stdout.strip():
|
||||
break
|
||||
subprocess.run(["pkill", "-9", "-f", identity_dir],
|
||||
capture_output=True, timeout=2)
|
||||
await asyncio.sleep(1)
|
||||
|
||||
return reports
|
||||
|
||||
async def run(self) -> BatchReport:
|
||||
report = BatchReport()
|
||||
report.accounts = self.accounts; report.blueprints = self.blueprints
|
||||
report.rounds_total = self.rounds_total; report.mix_mode = self.mix
|
||||
|
||||
# 1. 按身份分组
|
||||
groups = group_accounts_by_identity(self.accounts)
|
||||
if not groups:
|
||||
report.error = "没有有效的账号"
|
||||
return report
|
||||
|
||||
log.info(f"\n{'='*50}")
|
||||
log.info(f" 🚀 批量执行计划")
|
||||
log.info(f" 身份组: {len(groups)}")
|
||||
for ident, accts in groups.items():
|
||||
log.info(f" 🖥️ {ident} → {[a['id'] for a in accts]}")
|
||||
log.info(f" 每账号 {self.rounds_total} 轮 | 蓝图: {self.blueprints}")
|
||||
log.info(f"{'='*50}")
|
||||
|
||||
# 2. 解析错峰参数
|
||||
try:
|
||||
s_min, s_max = [int(x) for x in self.stagger.split("-")]
|
||||
except:
|
||||
s_min, s_max = 15, 30
|
||||
|
||||
# 3. 并行执行身份组(Semaphore 控制并发数 = max_browsers)
|
||||
# 错峰启动:每个身份组之间随机延迟 stagger 范围
|
||||
sem = asyncio.Semaphore(self.max_browsers)
|
||||
all_reports = []
|
||||
|
||||
group_list = list(groups.items())
|
||||
|
||||
async def _run_one_group(ident, accts, delay: float = 0):
|
||||
if delay > 0:
|
||||
log.info(f" ⏳ 错峰等待 {delay:.0f}s 后启动身份组 [{ident}]")
|
||||
await asyncio.sleep(delay)
|
||||
async with sem:
|
||||
log.info(f"\n 🖥️ 启动身份组 [{ident}]")
|
||||
return await self._run_identity_group(accts)
|
||||
|
||||
cumulative_delay = 0.0
|
||||
tasks = []
|
||||
for ident, accts in group_list:
|
||||
tasks.append(asyncio.create_task(
|
||||
_run_one_group(ident, accts, delay=cumulative_delay)
|
||||
))
|
||||
stagger_sec = random.uniform(s_min, s_max)
|
||||
cumulative_delay += stagger_sec
|
||||
for coro in asyncio.as_completed(tasks):
|
||||
group_reports = await coro
|
||||
all_reports.extend(group_reports)
|
||||
|
||||
report.account_reports = all_reports
|
||||
|
||||
# 写入标准结果文件(供 CommandBus / 看板读取)
|
||||
if self.run_id:
|
||||
try:
|
||||
from mc.execution_policy import write_result
|
||||
status = "completed" if not report.error else "failed"
|
||||
write_result(self.run_id, status, report=report.to_dict(),
|
||||
error=report.error)
|
||||
except:
|
||||
pass
|
||||
|
||||
return report
|
||||
@@ -0,0 +1,270 @@
|
||||
"""
|
||||
execution_policy.py — 统一执行策略 v1
|
||||
|
||||
所有执行入口(CLI / Dashboard / API)都通过此策略层检查限制。
|
||||
集中定义并发数、窗口槽位、启动间隔、超时、冷却等规则。
|
||||
|
||||
使用:
|
||||
from mc.execution_policy import policy, preflight, slot_for
|
||||
info = preflight() # 返回 {ok, browsers, slots, delay, ...}
|
||||
slot = slot_for(account) # 分配窗口槽位
|
||||
"""
|
||||
|
||||
import json, logging, os, subprocess, time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
logger = logging.getLogger("execution_policy")
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 一、策略常量(唯一来源)
|
||||
# ════════════════════════════════════════════════════════════
|
||||
MAX_CONCURRENT = 3 # 每台机器最多 3 个浏览器
|
||||
LAUNCH_STAGGER = 15 # 每个浏览器启动间隔(秒)
|
||||
AUTO_SHUTDOWN_MIN = 30 # 30 分钟无操作自动关闭
|
||||
MIN_DISK_GB = 5 # 最小可用磁盘空间(GB)
|
||||
MAX_TIMEOUT = 600 # 命令最大超时(秒)
|
||||
GRACE_PERIOD_LOCAL = 5 # 本机进程启动宽限期(秒)
|
||||
GRACE_PERIOD_REMOTE = 30 # 远程进程启动宽限期(秒)
|
||||
|
||||
# 窗口槽位:固定 3 个,y 顶端对齐,x 间隔 350px(参考5kechengdeAir的经验值)
|
||||
SLOTS = [
|
||||
{"id": 1, "position": (0, 0), "size": (702, 783)},
|
||||
{"id": 2, "position": (350, 0), "size": (702, 783)},
|
||||
{"id": 3, "position": (700, 0), "size": (702, 783)},
|
||||
]
|
||||
|
||||
# 操作冷却时间(秒),用 (min, max) 范围随机化
|
||||
OP_COOLDOWNS = {
|
||||
"like": (2, 4),
|
||||
"collect": (3, 6),
|
||||
"comment": (30, 45),
|
||||
"follow": (10, 20),
|
||||
"search": (3, 6),
|
||||
"scroll_feed": (2, 3),
|
||||
"reply": (30, 45),
|
||||
"browse": (5, 10),
|
||||
"default": (2, 5),
|
||||
}
|
||||
|
||||
# 登录超时(秒)
|
||||
LOGIN_TIMEOUTS = {
|
||||
"cookie_verify": 40,
|
||||
"sms_recovery": 90,
|
||||
"douyin_login": 300, # 含 4 分钟短信轮询
|
||||
"login_page": 120,
|
||||
}
|
||||
|
||||
POLICY_VERSION = "1.0"
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 二、路径解析
|
||||
# ════════════════════════════════════════════════════════════
|
||||
def _resolve_paths():
|
||||
"""解析 AGENT_SYNC 和 AGENT_LOCAL 路径"""
|
||||
home = Path.home()
|
||||
agent_sync = Path(os.environ.get("AGENT_SYNC", str(home / "workbuddy-agent-os" / "agent-sync")))
|
||||
agent_local = Path(os.environ.get("AGENT_LOCAL", str(home / "workbuddy-agent-os" / "agent-local")))
|
||||
return agent_sync, agent_local
|
||||
|
||||
AGENT_SYNC, AGENT_LOCAL = _resolve_paths()
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 三、策略导出(供前端展示)
|
||||
# ════════════════════════════════════════════════════════════
|
||||
def get_policy() -> dict:
|
||||
"""返回完整策略配置,供 Dashboard 前端展示"""
|
||||
return {
|
||||
"version": POLICY_VERSION,
|
||||
"max_concurrent": MAX_CONCURRENT,
|
||||
"launch_stagger": LAUNCH_STAGGER,
|
||||
"auto_shutdown_min": AUTO_SHUTDOWN_MIN,
|
||||
"min_disk_gb": MIN_DISK_GB,
|
||||
"max_timeout": MAX_TIMEOUT,
|
||||
"grace_period_local": GRACE_PERIOD_LOCAL,
|
||||
"grace_period_remote": GRACE_PERIOD_REMOTE,
|
||||
"slots": [{"id": s["id"], "position": list(s["position"]), "size": list(s["size"])} for s in SLOTS],
|
||||
"op_cooldowns": {k: list(v) for k, v in OP_COOLDOWNS.items()},
|
||||
"login_timeouts": dict(LOGIN_TIMEOUTS),
|
||||
}
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 四、执行前预检
|
||||
# ════════════════════════════════════════════════════════════
|
||||
def preflight(local_only: bool = True) -> dict:
|
||||
"""执行前全面检查——所有入口共用
|
||||
|
||||
Returns:
|
||||
{
|
||||
"ok": bool, # 是否通过检查
|
||||
"browsers": int, # 当前浏览器数
|
||||
"max_concurrent": int, # 最大允许数
|
||||
"slots_available": int, # 可用槽位数
|
||||
"disk_gb": float, # 剩余磁盘
|
||||
"disk_ok": bool, # 磁盘是否足够
|
||||
"stagger_delay": int, # 建议启动延迟(秒)
|
||||
"message": str, # 检查结果描述
|
||||
}
|
||||
"""
|
||||
result = {"ok": True, "browsers": 0, "max_concurrent": MAX_CONCURRENT,
|
||||
"slots_available": MAX_CONCURRENT, "disk_gb": 0, "disk_ok": True,
|
||||
"stagger_delay": 0, "message": "就绪"}
|
||||
|
||||
# 1. 浏览器数检查
|
||||
try:
|
||||
r = subprocess.run(
|
||||
["pgrep", "-f", "camoufox.*-no-remote|HeadlessShell|chrome.*remote-debug|chromium.*remote-debug"],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
count = len([p for p in r.stdout.strip().split("\n") if p.strip()]) if r.stdout.strip() else 0
|
||||
result["browsers"] = count
|
||||
result["slots_available"] = max(0, MAX_CONCURRENT - count)
|
||||
|
||||
if count >= MAX_CONCURRENT:
|
||||
result["ok"] = False
|
||||
result["message"] = f"浏览器已达上限: {count}/{MAX_CONCURRENT},请等待或手动清理"
|
||||
return result
|
||||
|
||||
# 计算建议延迟
|
||||
result["stagger_delay"] = count * LAUNCH_STAGGER
|
||||
except:
|
||||
pass
|
||||
|
||||
# 2. 磁盘空间检查
|
||||
try:
|
||||
s = os.statvfs(str(AGENT_LOCAL))
|
||||
free_gb = (s.f_frsize * s.f_bavail) / (1024 ** 3)
|
||||
result["disk_gb"] = round(free_gb, 1)
|
||||
if free_gb < MIN_DISK_GB:
|
||||
result["ok"] = False
|
||||
result["disk_ok"] = False
|
||||
result["message"] = f"磁盘空间不足: {free_gb:.1f}GB < {MIN_DISK_GB}GB"
|
||||
return result
|
||||
except:
|
||||
pass
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def check_running_browsers() -> int:
|
||||
"""获取当前浏览器进程数(供各层共用)"""
|
||||
try:
|
||||
r = subprocess.run(
|
||||
["pgrep", "-f", "camoufox.*-no-remote|HeadlessShell|chrome.*remote-debug|chromium.*remote-debug"],
|
||||
capture_output=True, text=True, timeout=5
|
||||
)
|
||||
if not r.stdout.strip():
|
||||
return 0
|
||||
return len([p for p in r.stdout.strip().split("\n") if p.strip()])
|
||||
except:
|
||||
return 0
|
||||
|
||||
|
||||
def slot_for(account_id: str) -> Optional[dict]:
|
||||
"""为账号分配窗口槽位——按当前浏览器数取模
|
||||
|
||||
Returns:
|
||||
{"id": int, "position": [x, y], "size": [w, h]} or None
|
||||
"""
|
||||
browser_count = check_running_browsers()
|
||||
if browser_count >= MAX_CONCURRENT:
|
||||
return None
|
||||
slot_idx = browser_count % len(SLOTS)
|
||||
return {
|
||||
"id": SLOTS[slot_idx]["id"],
|
||||
"position": list(SLOTS[slot_idx]["position"]),
|
||||
"size": list(SLOTS[slot_idx]["size"]),
|
||||
}
|
||||
|
||||
|
||||
def get_cooldown(op_type: str = "default") -> tuple:
|
||||
"""获取操作冷却时间范围"""
|
||||
return OP_COOLDOWNS.get(op_type, OP_COOLDOWNS["default"])
|
||||
|
||||
|
||||
def get_login_timeout(timeout_type: str = "douyin_login") -> int:
|
||||
"""获取登录超时时间"""
|
||||
return LOGIN_TIMEOUTS.get(timeout_type, 120)
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 五、结果文件管理
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
RESULTS_DIR = AGENT_LOCAL / "runtime" / "results"
|
||||
RESULTS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def write_result(run_id: str, status: str, report: dict = None,
|
||||
error: str = None, message: str = None):
|
||||
"""写入标准执行结果文件——MC 执行完毕统一调用
|
||||
|
||||
Args:
|
||||
run_id: 命令 run_id(CommandBus 分配)
|
||||
status: completed / failed / crashed / cancelled
|
||||
report: BatchReport.to_dict() 的执行结果
|
||||
error: 错误描述
|
||||
message: 附加信息
|
||||
"""
|
||||
result = {
|
||||
"run_id": run_id,
|
||||
"status": status,
|
||||
"written_at": datetime.now().isoformat(),
|
||||
}
|
||||
if report:
|
||||
result["report"] = report
|
||||
result["steps"] = {
|
||||
"total": report.get("total_steps", 0),
|
||||
"success": report.get("success", 0),
|
||||
"failed": report.get("failed", 0),
|
||||
"skipped": report.get("skipped", 0),
|
||||
}
|
||||
# 汇总所有账号的步骤详情
|
||||
step_details = []
|
||||
for ar in (report.get("account_reports") or []):
|
||||
for s in (ar.get("steps") or []):
|
||||
step_details.append({
|
||||
"account": ar["account"],
|
||||
"round": ar["round"],
|
||||
"op": s.get("op", ""),
|
||||
"step_id": s.get("step_id", 0),
|
||||
"success": s.get("success", False),
|
||||
"duration": s.get("duration", 0),
|
||||
"error": s.get("error"),
|
||||
})
|
||||
result["step_details"] = step_details
|
||||
if error:
|
||||
result["error"] = error
|
||||
if message:
|
||||
result["message"] = message
|
||||
|
||||
filepath = RESULTS_DIR / f"{run_id}.json"
|
||||
try:
|
||||
filepath.write_text(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
return str(filepath)
|
||||
except Exception as e:
|
||||
logger.warning(f"写入结果文件失败: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def read_result(run_id: str) -> Optional[dict]:
|
||||
"""读取执行结果文件"""
|
||||
filepath = RESULTS_DIR / f"{run_id}.json"
|
||||
if not filepath.exists():
|
||||
return None
|
||||
try:
|
||||
return json.loads(filepath.read_text(encoding="utf-8"))
|
||||
except:
|
||||
return None
|
||||
|
||||
|
||||
def cleanup_old_results(max_age_hours: int = 24):
|
||||
"""清理过期结果文件"""
|
||||
now = time.time()
|
||||
for f in RESULTS_DIR.glob("*.json"):
|
||||
try:
|
||||
if now - f.stat().st_mtime > max_age_hours * 3600:
|
||||
f.unlink()
|
||||
except:
|
||||
pass
|
||||
@@ -0,0 +1,335 @@
|
||||
"""
|
||||
exporter.py — 录制结果自动导出引擎 (v1.0)
|
||||
|
||||
三阶段管线 Phase 3:
|
||||
分析结果 → 生成 OP_GRAPH 条目 + 蓝图模板 + 代码片段
|
||||
|
||||
用法:
|
||||
from mc.exporter import export_all
|
||||
result = export_all(analysis_result, recording_package)
|
||||
"""
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent.parent
|
||||
TOOL_DIR = SCRIPTS_DIR.parent
|
||||
|
||||
# 已存在的 OP_GRAPH 路径(追加用)
|
||||
MATRIX_MGMT = SCRIPTS_DIR / "matrix_mgmt.py"
|
||||
BLUEPRINTS_DIR = TOOL_DIR / "blueprints"
|
||||
|
||||
|
||||
def export_all(analysis: dict, package: dict) -> dict:
|
||||
"""完整导出管线——生成 OP_GRAPH + 蓝图 + 代码"""
|
||||
return {
|
||||
"op_graph_entries": _generate_op_graph_entries(analysis),
|
||||
"blueprint": _generate_blueprint(analysis, package),
|
||||
"code_snippet": _generate_code_snippet(analysis, package),
|
||||
"summary": _generate_summary(analysis, package),
|
||||
}
|
||||
|
||||
|
||||
def _generate_op_graph_entries(analysis: dict) -> list:
|
||||
"""从分析结果生成 OP_GRAPH 字典条目"""
|
||||
actions = analysis.get("actions", [])
|
||||
entries = []
|
||||
|
||||
for i, action in enumerate(actions):
|
||||
at = action.get("action_type", "unknown")
|
||||
desc = action.get("action_desc", "")
|
||||
|
||||
# 确定平台和分类
|
||||
platform, category, label, requires, allows = _classify_action(at, desc)
|
||||
|
||||
# 生成 op name
|
||||
op_name = _suggest_op_name(at, i)
|
||||
|
||||
# 检查是否已有同名操作
|
||||
if _op_exists(op_name):
|
||||
op_name = f"{op_name}_v{i+1}"
|
||||
|
||||
entry = {
|
||||
"name": op_name,
|
||||
"definition": {
|
||||
"platform": platform,
|
||||
"category": category,
|
||||
"label": f"🎬 {desc}" if desc else f"🎬 自定义操作{i+1}",
|
||||
"requires": requires,
|
||||
"allows": allows,
|
||||
"can_be_first": "goto_home" not in requires,
|
||||
"desc": desc or f"录制步骤 {action.get('step_after', '?')}",
|
||||
"_source": "recorded", # recorded = AI录制, manual = 手工创建
|
||||
"_action_type": at,
|
||||
}
|
||||
}
|
||||
|
||||
# 如果有点击坐标,记录
|
||||
inferred = action.get("inferred_action", {})
|
||||
if inferred.get("coords"):
|
||||
entry["definition"]["_click_coords"] = inferred["coords"]
|
||||
|
||||
entries.append(entry)
|
||||
|
||||
return entries
|
||||
|
||||
|
||||
def _classify_action(action_type: str, desc: str) -> tuple:
|
||||
"""确定操作所属平台、分类、依赖"""
|
||||
# 默认值
|
||||
platform = "通用"
|
||||
category = "custom"
|
||||
requires = []
|
||||
allows = ["rest", "go_back"]
|
||||
|
||||
if action_type == "like":
|
||||
platform = "douyin"
|
||||
category = "interact"
|
||||
requires = ["goto_home"]
|
||||
allows = ["rest", "go_back", "collect", "next_video"]
|
||||
elif action_type == "comment" or action_type == "open_comment":
|
||||
platform = "douyin"
|
||||
category = "interact"
|
||||
requires = ["goto_home"]
|
||||
allows = ["rest", "go_back", "like"]
|
||||
elif action_type in ("navigate_video", "enter_video"):
|
||||
platform = "douyin"
|
||||
category = "navigation"
|
||||
requires = ["goto_home"]
|
||||
allows = ["like", "collect", "comment", "next_video", "scroll_feed", "rest"]
|
||||
elif action_type == "navigate_profile":
|
||||
platform = "douyin"
|
||||
category = "navigation"
|
||||
requires = ["goto_home"]
|
||||
allows = ["read_profile", "rest", "go_back"]
|
||||
elif action_type == "scroll" or "scroll" in action_type:
|
||||
platform = "douyin"
|
||||
category = "navigation"
|
||||
requires = ["goto_home"]
|
||||
allows = ["rest", "go_back", "next_video"]
|
||||
elif action_type == "search":
|
||||
platform = "douyin"
|
||||
category = "navigation"
|
||||
requires = ["goto_home"]
|
||||
allows = ["browse_feed", "rest", "go_back"]
|
||||
elif action_type == "next_video":
|
||||
platform = "douyin"
|
||||
category = "navigation"
|
||||
requires = ["goto_home"]
|
||||
allows = ["like", "collect", "comment", "rest", "go_back"]
|
||||
elif "xhs" in platform or "xiaohongshu" in desc.lower():
|
||||
platform = "xiaohongshu"
|
||||
category = "custom"
|
||||
requires = ["xhs_goto_home"]
|
||||
allows = ["rest", "go_back"]
|
||||
elif "click" in action_type:
|
||||
platform = "通用"
|
||||
category = "interact"
|
||||
requires = ["goto_home", "xhs_goto_home"]
|
||||
allows = ["rest", "go_back"]
|
||||
|
||||
return platform, category, f"🎬 {desc[:15]}" if desc else "🎬 自定义", requires, allows
|
||||
|
||||
|
||||
def _suggest_op_name(action_type: str, index: int) -> str:
|
||||
"""生成操作名称"""
|
||||
name_map = {
|
||||
"like": "custom_like", "comment": "custom_comment", "follow": "custom_follow",
|
||||
"navigate_video": "custom_open_video", "navigate_profile": "custom_goto_profile",
|
||||
"scroll": "custom_scroll", "next_video": "custom_next",
|
||||
"search": "custom_search", "enter_video": "custom_enter_video",
|
||||
"click": "custom_click", "keypress": "custom_keypress",
|
||||
"view_profile": "custom_view_profile", "unknown": "custom_action",
|
||||
}
|
||||
base = name_map.get(action_type, f"custom_{action_type}")
|
||||
return base
|
||||
|
||||
|
||||
def _op_exists(op_name: str) -> bool:
|
||||
"""检查 OP_GRAPH 是否已有同名操作"""
|
||||
if not MATRIX_MGMT.exists():
|
||||
return False
|
||||
content = MATRIX_MGMT.read_text()
|
||||
return f'"{op_name}":' in content
|
||||
|
||||
|
||||
def _generate_blueprint(analysis: dict, package: dict) -> dict:
|
||||
"""从分析结果生成蓝图 JSON"""
|
||||
actions = analysis.get("actions", [])
|
||||
meta = package.get("meta", {})
|
||||
|
||||
# 确定平台
|
||||
platform = meta.get("platform", "unknown")
|
||||
for action in actions:
|
||||
at = action.get("action_type", "")
|
||||
if "xhs" in at or "profile" in at:
|
||||
platform = "xiaohongshu" if "profile" in at else "douyin"
|
||||
break
|
||||
|
||||
# 构建步骤
|
||||
steps = []
|
||||
seen_ops = set()
|
||||
step_id = 1
|
||||
|
||||
# 如果是从首页开始的,先加 goto_home
|
||||
if platform == "douyin":
|
||||
steps.append({
|
||||
"step_id": step_id, "op": "goto_home",
|
||||
"args": {}, "wait_after": 3
|
||||
})
|
||||
seen_ops.add("goto_home")
|
||||
step_id += 1
|
||||
|
||||
for action in actions:
|
||||
at = action.get("action_type", "unknown")
|
||||
op_name = _suggest_op_name(at, step_id)
|
||||
|
||||
# 跳过重复的导航步骤
|
||||
if op_name in seen_ops:
|
||||
continue
|
||||
seen_ops.add(op_name)
|
||||
|
||||
step = {
|
||||
"step_id": step_id,
|
||||
"op": op_name,
|
||||
"args": {},
|
||||
"wait_after": 2,
|
||||
}
|
||||
|
||||
# 如果是搜索操作,加默认关键词
|
||||
if at == "search":
|
||||
step["args"]["keyword"] = "热门"
|
||||
|
||||
steps.append(step)
|
||||
step_id += 1
|
||||
|
||||
blueprint = {
|
||||
"id": f"recorded_{meta.get('account_id', 'unknown')}_{len(actions)}steps",
|
||||
"name": f"录制蓝图 ({len(actions)}步)",
|
||||
"version": "1.0.0",
|
||||
"platform": platform,
|
||||
"description": f"由录制自动生成: {meta.get('account_id','?')} {len(actions)}步操作",
|
||||
"steps": steps,
|
||||
}
|
||||
|
||||
return blueprint
|
||||
|
||||
|
||||
def _generate_code_snippet(analysis: dict, package: dict) -> str:
|
||||
"""生成可在 engine.py 中使用的操作代码片段"""
|
||||
actions = analysis.get("actions", [])
|
||||
lines = []
|
||||
lines.append("# ═══════════════════════════════════════════════")
|
||||
lines.append(f"# 录制生成: {package.get('meta', {}).get('account_id', '?')}")
|
||||
lines.append(f"# 步骤数: {len(actions)}")
|
||||
lines.append("# ═══════════════════════════════════════════════")
|
||||
lines.append("")
|
||||
|
||||
for i, action in enumerate(actions):
|
||||
at = action.get("action_type", "unknown")
|
||||
desc = action.get("action_desc", "")
|
||||
inferred = action.get("inferred_action", {})
|
||||
|
||||
lines.append(f" # 步骤 {i+1}: {desc}")
|
||||
lines.append(f" elif op == \"{_suggest_op_name(at, i)}\":")
|
||||
|
||||
if at == "like":
|
||||
lines.append(' r = await conn.page.evaluate("""() => {')
|
||||
lines.append(' const b = document.querySelector(\'[data-e2e="like-count"]\');')
|
||||
lines.append(' return b ? (b.click(), \'👍\') : \'-\';')
|
||||
lines.append(' }""")')
|
||||
lines.append(' result = r')
|
||||
elif at == "navigate_profile":
|
||||
lines.append(' await conn.page.goto("https://www.douyin.com/user/self", timeout=20000)')
|
||||
lines.append(' await asyncio.sleep(4)')
|
||||
lines.append(' result = "profile_loaded"')
|
||||
elif at == "scroll" or "scroll" in at:
|
||||
lines.append(' await conn.page.evaluate("() => window.scrollBy(0, 600)")')
|
||||
lines.append(' await asyncio.sleep(1)')
|
||||
lines.append(' result = "scrolled"')
|
||||
elif inferred.get("coords"):
|
||||
coords = inferred["coords"]
|
||||
lines.append(f' # 点击坐标 ({coords["x"]}, {coords["y"]})')
|
||||
lines.append(f' await conn.page.mouse.click({coords["x"]}, {coords["y"]})')
|
||||
lines.append(' await asyncio.sleep(2)')
|
||||
lines.append(' result = "clicked"')
|
||||
elif inferred.get("trigger", "").startswith("key:"):
|
||||
key = inferred["trigger"].split(":")[1]
|
||||
lines.append(f' # 按键: {key}')
|
||||
lines.append(f' await conn.page.keyboard.press("{key}")')
|
||||
lines.append(' await asyncio.sleep(2)')
|
||||
lines.append(' result = "pressed"')
|
||||
else:
|
||||
lines.append(f' # TODO: 实现 {desc} 的自动化操作')
|
||||
lines.append(' await asyncio.sleep(2)')
|
||||
lines.append(' result = f"TODO({at})"')
|
||||
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _generate_summary(analysis: dict, package: dict) -> dict:
|
||||
"""生成操作汇总摘要"""
|
||||
actions = analysis.get("actions", [])
|
||||
meta = package.get("meta", {})
|
||||
|
||||
# 统计操作类型
|
||||
type_counts = {}
|
||||
for a in actions:
|
||||
at = a.get("action_type", "unknown")
|
||||
type_counts[at] = type_counts.get(at, 0) + 1
|
||||
|
||||
return {
|
||||
"account": meta.get("account_id", "?"),
|
||||
"total_steps": meta.get("total_steps", 0),
|
||||
"actions_found": len(actions),
|
||||
"action_types": type_counts,
|
||||
"action_sequence": [a.get("action_type", "?") for a in actions],
|
||||
"suggested_blueprint_id": f"recorded_{meta.get('account_id', '?')}_{len(actions)}steps",
|
||||
}
|
||||
|
||||
|
||||
def export_recording(recording_path: str, output_dir: str = None) -> dict:
|
||||
"""一站式:加载 → 分析 → 导出"""
|
||||
from mc.recorder import RecordingSession
|
||||
from mc.analyzer import analyze_recording
|
||||
|
||||
pkg = RecordingSession.load_recording(recording_path)
|
||||
analysis = analyze_recording(pkg)
|
||||
result = export_all(analysis, pkg)
|
||||
|
||||
# 保存导出产物
|
||||
out_dir = Path(output_dir) if output_dir else BLUEPRINTS_DIR
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
base_name = Path(recording_path).stem.replace("recording_", "export_")
|
||||
|
||||
# 保存蓝图
|
||||
bp = result.get("blueprint", {})
|
||||
if bp:
|
||||
bp_file = out_dir / f"{base_name}.json"
|
||||
with open(bp_file, "w", encoding="utf-8") as f:
|
||||
json.dump(bp, f, ensure_ascii=False, indent=2)
|
||||
result["saved_blueprint"] = str(bp_file)
|
||||
|
||||
# 保存代码片段
|
||||
code = result.get("code_snippet", "")
|
||||
if code:
|
||||
code_file = out_dir / f"{base_name}.py"
|
||||
with open(code_file, "w", encoding="utf-8") as f:
|
||||
f.write(code)
|
||||
result["saved_code"] = str(code_file)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
path = sys.argv[1] if len(sys.argv) > 1 else ""
|
||||
if path:
|
||||
result = export_recording(path)
|
||||
print(json.dumps(result, ensure_ascii=False, indent=2))
|
||||
else:
|
||||
print("用法: python -m mc.exporter <录制包路径>")
|
||||
@@ -0,0 +1,352 @@
|
||||
"""
|
||||
interact.py — 互动编排引擎 (InteractOrchestrator)
|
||||
|
||||
负责:
|
||||
1. 解析互动策略
|
||||
2. 生成识别码(三级接力用)
|
||||
3. 生成时间线
|
||||
4. 分配 chain_position 和 target_code
|
||||
5. 依次执行(含步间等待)
|
||||
|
||||
识别码系统(三级接力):
|
||||
first账号 发评时末尾加 "🌸XX"
|
||||
reply账号 搜 "🌸XX" → 找到 → 回复(末尾加"🌸YY")
|
||||
second账号 搜 "🌸YY" → 找到 → 回复
|
||||
|
||||
识别码格式: 🌸 + 两位数字 (如 🌸42)
|
||||
- 看起来像自然emoji,不触发风控
|
||||
- 100个组合,一轮接力用2个
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import random
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
log = logging.getLogger("interact")
|
||||
|
||||
# ── 识别码系统 ──────────────────────────────────────────
|
||||
|
||||
_CODE_POOL = list(range(10, 100)) # 10-99
|
||||
|
||||
def _alloc_codes(count: int = 2) -> list[int]:
|
||||
"""分配不重复的识别码"""
|
||||
random.shuffle(_CODE_POOL)
|
||||
return _CODE_POOL[:count]
|
||||
|
||||
def _fmt_code(n: int) -> str:
|
||||
"""🌸 + 两位数字"""
|
||||
return f"🌸{n:02d}"
|
||||
|
||||
def _strip_code(text: str) -> str:
|
||||
"""去除末尾的识别码"""
|
||||
import re
|
||||
return re.sub(r'\s*🌸\d{2}\s*$', '', text).strip()
|
||||
|
||||
|
||||
# ── 互动策略定义 ──────────────────────────────────────────
|
||||
|
||||
STRATEGIES = {
|
||||
"comment": {
|
||||
"label": "定向评论", "blueprint": "interact_comment",
|
||||
"description": "给指定视频发评论",
|
||||
},
|
||||
"chain": {
|
||||
"label": "三级接力", "blueprint": "interact_chain",
|
||||
"description": "A评→B回复→C再回复",
|
||||
},
|
||||
"like": {
|
||||
"label": "点赞互动", "blueprint": "interact_like",
|
||||
"description": "点赞视频+评论+关注",
|
||||
},
|
||||
"hot": {
|
||||
"label": "热评互动", "blueprint": "interact_hot",
|
||||
"description": "在热评下回复提升曝光",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
# ── 三级接力语料 ─────────────────────────────────────────
|
||||
# 每项一个 dict,含 first/reply/second 三轮对话文本。
|
||||
# 两轮对话的 second 留空,代码会自动跳过(fallback 到通用回复)。
|
||||
# _get_chain_texts() 随机选中一项后,按账号数裁剪轮次。
|
||||
|
||||
CHAIN_TEMPLATES = {
|
||||
# ── 原通用分类 ──
|
||||
"food": [
|
||||
{"first": "看着就很有食欲,周末去试试", "reply": "同意!上次去吃过一次确实不错", "second": "我也去过,他家那个招牌菜绝了"},
|
||||
{"first": "这个做法收藏了,回家就做", "reply": "做了好几次了,家人都说好吃", "second": "按着做了一次很成功,谢谢分享"},
|
||||
],
|
||||
"travel": [
|
||||
{"first": "这个地方太美了,已加入旅行清单", "reply": "去年去过,风景确实好,值得二刷", "second": "求攻略!准备下个月去打卡"},
|
||||
{"first": "这个机位太绝了,拍得真好", "reply": "早上五点去占的位置,值得", "second": "太卷了哈哈哈,不过效果确实好"},
|
||||
],
|
||||
"tech": [
|
||||
{"first": "分析得很透彻,学到了很多", "reply": "第三点之前完全没想到,受教了", "second": "按这个方法试了,效果很好"},
|
||||
],
|
||||
"lifestyle": [
|
||||
{"first": "这就是我理想的生活状态", "reply": "同款生活!每天都很充实", "second": "羡慕了,我也要这样过"},
|
||||
],
|
||||
"emotion": [
|
||||
{"first": "太真实了,完全说到心坎里", "reply": "是啊,经历过的人才懂", "second": "抱抱,都会好起来的"},
|
||||
],
|
||||
# ── 新增:怕疼/怕手术类(两轮) ──
|
||||
"pain_fear": [
|
||||
{"first": "肛肠手术是不是老疼了?我一直不敢去就是怕这个", "reply": "朱晓光主任手法轻得很,恢复也快,我做完没想象中那么遭罪", "second": ""},
|
||||
{"first": "听说术后换药比手术还疼,真的假的?", "reply": "朱主任技术好,换药一点都不猛,温柔到离谱,别自己吓自己", "second": ""},
|
||||
{"first": "一想到要开刀就腿软,还有别的招吗?", "reply": "先找朱晓光看看,不一定都要手术的,他会给你出方案", "second": ""},
|
||||
],
|
||||
# ── 新增:社恐/不好意思类(两轮) ──
|
||||
"social_anxiety": [
|
||||
{"first": "肛肠科太社死了,一想到要脱裤子就脚趾抠地", "reply": "去苏州肛泰找朱晓光,全是独立诊室,一点都不尴尬", "second": ""},
|
||||
{"first": "年轻小姑娘去看这个会不会很奇怪啊", "reply": "想多啦,朱主任那边年轻人多了去了,久坐党不分年龄", "second": ""},
|
||||
],
|
||||
# ── 新增:上班族没时间类(两轮) ──
|
||||
"busy_worker": [
|
||||
{"first": "天天上班哪来的时间去排队啊,请假又扣钱", "reply": "搜苏州济世璞真功重号约朱主任,约好时间直接去,不用请假一天", "second": ""},
|
||||
{"first": "周末能约吗?平时真走不开", "reply": "去苏州济世璞真看看朱主任的出诊时间,线上约了再去,不用白跑", "second": ""},
|
||||
],
|
||||
# ── 新增:偏方没用类(两轮) ──
|
||||
"home_remedy": [
|
||||
{"first": "药膏买了一堆,马应龙都囤了一箱子了,还是犯", "reply": "别瞎折腾了,肛肠问题得找专科对症下药,去找朱晓光主任看看吧", "second": ""},
|
||||
{"first": "网上看的偏方我都试过了,钱没少花屁用没有", "reply": "踩坑踩多了就懂了,直接找朱晓光,少走弯路是真的", "second": ""},
|
||||
{"first": "各种办法都试了,反反复复真的心累", "reply": "去苏州肛泰找朱主任,专科医院不是白叫的,对症了才能断根", "second": ""},
|
||||
],
|
||||
# ── 新增:长辈不舒服类(两轮) ──
|
||||
"elderly": [
|
||||
{"first": "我爸排便不舒服好久了,让他去检查死活不肯", "reply": "你就说带他去苏州肛泰找朱晓光看看,独立诊室不尴尬,老人也能接受", "second": ""},
|
||||
{"first": "老人家行动不方便,去大医院太折腾了", "reply": "走苏州济世璞真约朱主任,到点去就行,不用老人跟着排长队", "second": ""},
|
||||
],
|
||||
# ── 新增:三轮完整对话(组合1-5) ──
|
||||
"general_health": [
|
||||
{"first": "一天坐八九个小时,肛周天天不舒服,真的破防了", "reply": "同款打工人!去找苏州肛泰朱晓光主任看看,他调理这个很有一套", "second": "楼上说的对,我也找朱主任看的,现在好多了,后悔没早点去"},
|
||||
{"first": "半夜疼得睡不着,又不好意思跟家里说,主打一个硬扛", "reply": "别扛了姐妹,朱晓光主任人超好,肛泰也是独立诊室不会尴尬的", "second": "亲测不尴尬,我去过,朱主任说话很自然,看完心里踏实多了"},
|
||||
{"first": "刷到朱晓光主任的科普才意识到自己好多习惯都是错的", "reply": "我也看了!然后去苏州济世璞真约了个号,想当面让主任给看看", "second": "看完科普去的+1,朱主任讲得很实在,不乱开检查"},
|
||||
{"first": "大医院专家号真抢不到,蹲了好久了全是满的", "reply": "告诉你个捡漏的,苏州济世璞真功重号里头有朱主任,算绿通吧", "second": "我就是那儿约上的,不用死磕大医院,这个小众渠道确实顶"},
|
||||
{"first": "痔疮犯了还要硬着头皮开会,坐我旁边的同事都看我扭来扭去", "reply": "太惨了但我也经历过!后来找朱晓光主任弄好了,搜苏州济世璞真约", "second": "笑死,同款经历,但朱主任确实靠谱,现在开会稳如泰山"},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _get_chain_texts(corpus: str = "") -> dict:
|
||||
"""获取一组三级接力文本
|
||||
|
||||
支持逗号分隔多选("food,emotion"),从中随机取一个分类。
|
||||
"""
|
||||
# 多选支持:逗号分隔,随机选一个
|
||||
candidates = [c.strip() for c in corpus.split(",") if c.strip()] if corpus else []
|
||||
chosen = random.choice(candidates) if candidates else corpus
|
||||
pool = CHAIN_TEMPLATES.get(chosen, CHAIN_TEMPLATES["food"])
|
||||
return random.choice(pool)
|
||||
|
||||
|
||||
class InteractOrchestrator:
|
||||
"""互动编排器"""
|
||||
|
||||
def __init__(self, accounts: list, params: dict):
|
||||
self.accounts = accounts
|
||||
self.params = params
|
||||
self.url = params.get("url", "")
|
||||
self.strategy = params.get("strategy", "comment")
|
||||
self.direction = params.get("direction", "")
|
||||
self.corpus = params.get("corpus", "")
|
||||
self.blueprint = params.get("blueprint", "")
|
||||
self.rounds = params.get("rounds", 1)
|
||||
self.interval = params.get("interval", {"min": 300, "max": 600})
|
||||
self.dry_run = params.get("dry_run", False)
|
||||
|
||||
if not self.blueprint:
|
||||
self.blueprint = STRATEGIES.get(self.strategy, STRATEGIES["comment"])["blueprint"]
|
||||
|
||||
self.timeline: list[dict] = []
|
||||
self.results: list[dict] = []
|
||||
|
||||
# ── 公开接口 ────────────────────────────────────────
|
||||
|
||||
def plan(self) -> dict:
|
||||
"""生成执行计划"""
|
||||
self.timeline = self._generate_timeline()
|
||||
return {
|
||||
"strategy": self.strategy,
|
||||
"label": STRATEGIES.get(self.strategy, {}).get("label", "未知"),
|
||||
"blueprint": self.blueprint,
|
||||
"url": self.url,
|
||||
"total_accounts": len(self.accounts),
|
||||
"timeline": self.timeline,
|
||||
"steps": len(self.timeline),
|
||||
}
|
||||
|
||||
async def run(self) -> dict:
|
||||
"""执行互动计划"""
|
||||
self.timeline = self._generate_timeline()
|
||||
if self.dry_run:
|
||||
return {"status": "plan", "plan": self.plan()}
|
||||
|
||||
log.info(f"🚀 InteractOrchestrator | 策略={self.strategy} 账号={len(self.accounts)}")
|
||||
for step in self.timeline:
|
||||
if step.get("scheduled_at"):
|
||||
wait_sec = (step["scheduled_at"] - datetime.now()).total_seconds()
|
||||
if wait_sec > 0:
|
||||
log.info(f"⏳ 等待 {wait_sec:.0f}s → {step['label']}")
|
||||
await asyncio.sleep(wait_sec)
|
||||
|
||||
result = await self._execute_step(step)
|
||||
self.results.append(result)
|
||||
|
||||
if step != self.timeline[-1]:
|
||||
gap = random.randint(self.interval["min"], self.interval["max"])
|
||||
log.info(f"⏳ 步间间隔 {gap}s")
|
||||
await asyncio.sleep(gap)
|
||||
|
||||
return {
|
||||
"status": "completed",
|
||||
"total_steps": len(self.timeline),
|
||||
"success": sum(1 for r in self.results if r.get("ok")),
|
||||
"failed": sum(1 for r in self.results if not r.get("ok")),
|
||||
"results": self.results,
|
||||
}
|
||||
|
||||
# ── 时间线生成 ──────────────────────────────────────
|
||||
|
||||
def _generate_timeline(self) -> list[dict]:
|
||||
if self.strategy == "chain":
|
||||
return self._gen_chain_timeline()
|
||||
elif self.strategy == "hot":
|
||||
return self._gen_hot_timeline()
|
||||
elif self.strategy == "like":
|
||||
return self._gen_like_timeline()
|
||||
else:
|
||||
return self._gen_comment_timeline()
|
||||
|
||||
def _gen_comment_timeline(self) -> list[dict]:
|
||||
timeline = []
|
||||
base = datetime.now()
|
||||
for i, acct in enumerate(self.accounts):
|
||||
offset = i * random.randint(self.interval["min"], self.interval["max"])
|
||||
timeline.append({
|
||||
"step": i, "account": acct["id"],
|
||||
"platform": acct.get("platform", "douyin"),
|
||||
"machine": acct.get("machine", ""),
|
||||
"blueprint": self.blueprint,
|
||||
"label": f"评论 {acct['id']}",
|
||||
"url": self.url,
|
||||
"scheduled_at": base + timedelta(seconds=offset) if offset > 0 else None,
|
||||
})
|
||||
return timeline
|
||||
|
||||
def _gen_chain_timeline(self) -> list[dict]:
|
||||
"""三级接力:生成识别码 + 分配文本(支持两轮/三轮自动适配)"""
|
||||
timeline = []
|
||||
base = datetime.now()
|
||||
texts = _get_chain_texts(self.corpus)
|
||||
|
||||
# 自动判断轮次:first必须有,reply和second可选
|
||||
positions = ["first"]
|
||||
if texts.get("reply"):
|
||||
positions.append("reply")
|
||||
if texts.get("second"):
|
||||
positions.append("second_reply")
|
||||
|
||||
codes_needed = sum(1 for p in positions if p in ("reply", "second_reply"))
|
||||
codes = _alloc_codes(max(codes_needed, 1))
|
||||
min_gap, max_gap = self.interval["min"], self.interval["max"]
|
||||
|
||||
for i, acct in enumerate(self.accounts[:len(positions)]):
|
||||
position = positions[i]
|
||||
offset = i * random.randint(min_gap, max_gap)
|
||||
|
||||
step = {
|
||||
"step": i,
|
||||
"account": acct["id"],
|
||||
"platform": acct.get("platform", "douyin"),
|
||||
"machine": acct.get("machine", ""),
|
||||
"blueprint": "interact_chain",
|
||||
"label": f"{['一级评论','二级回复','三级再回复'][i]} {acct['id']}",
|
||||
"url": self.url,
|
||||
"chain_position": position,
|
||||
"scheduled_at": base + timedelta(seconds=offset) if offset > 0 else None,
|
||||
}
|
||||
|
||||
if position == "first":
|
||||
step["comment_text"] = f"{texts['first']} {_fmt_code(codes[0])}"
|
||||
step["my_code"] = _fmt_code(codes[0])
|
||||
elif position == "reply":
|
||||
step["target_code"] = _fmt_code(codes[0])
|
||||
step["reply_text"] = f"{texts['reply']} {_fmt_code(codes[1])}"
|
||||
step["my_code"] = _fmt_code(codes[1])
|
||||
elif position == "second_reply":
|
||||
step["target_code"] = _fmt_code(codes[1])
|
||||
step["reply_text"] = texts["second"]
|
||||
|
||||
timeline.append(step)
|
||||
|
||||
return timeline
|
||||
|
||||
def _gen_like_timeline(self) -> list[dict]:
|
||||
timeline = []
|
||||
base = datetime.now()
|
||||
for i, acct in enumerate(self.accounts):
|
||||
offset = i * random.randint(self.interval["min"] // 2, self.interval["max"] // 2)
|
||||
timeline.append({
|
||||
"step": i, "account": acct["id"],
|
||||
"platform": acct.get("platform", "douyin"),
|
||||
"machine": acct.get("machine", ""),
|
||||
"blueprint": self.blueprint,
|
||||
"label": f"点赞 {acct['id']}",
|
||||
"url": self.url,
|
||||
"scheduled_at": base + timedelta(seconds=offset) if offset > 0 else None,
|
||||
})
|
||||
return timeline
|
||||
|
||||
def _gen_hot_timeline(self) -> list[dict]:
|
||||
timeline = []
|
||||
base = datetime.now()
|
||||
for i, acct in enumerate(self.accounts):
|
||||
offset = i * random.randint(self.interval["min"], self.interval["max"])
|
||||
timeline.append({
|
||||
"step": i, "account": acct["id"],
|
||||
"platform": acct.get("platform", "douyin"),
|
||||
"machine": acct.get("machine", ""),
|
||||
"blueprint": "interact_hot",
|
||||
"label": f"热评 {acct['id']}",
|
||||
"url": self.url,
|
||||
"scheduled_at": base + timedelta(seconds=offset) if offset > 0 else None,
|
||||
})
|
||||
return timeline
|
||||
|
||||
# ── 单步执行 ──────────────────────────────────────────
|
||||
|
||||
async def _execute_step(self, step: dict) -> dict:
|
||||
"""
|
||||
执行时间线中的一步。
|
||||
|
||||
实际 DOM 操作由你录制的原子操作完成(占位)。
|
||||
当前实现:输出要执行的命令到日志,供后续挂接。
|
||||
"""
|
||||
log.info(f"▶ {step['label']}")
|
||||
log.info(f" 蓝图={step['blueprint']} url={step.get('url','')[:50]}")
|
||||
|
||||
if self.strategy == "chain":
|
||||
pos = step.get("chain_position", "")
|
||||
if pos == "first":
|
||||
log.info(f" → 发评: {step.get('comment_text','')}")
|
||||
log.info(f" 需要原子操作: post_comment_with_code(text, code)")
|
||||
elif pos in ("reply", "second_reply"):
|
||||
log.info(f" → 搜识别码: {step.get('target_code','')}")
|
||||
log.info(f" → 回复: {step.get('reply_text','')}")
|
||||
log.info(f" 需要原子操作: find_comment_by_code(code)")
|
||||
log.info(f" 需要原子操作: reply_comment(text)")
|
||||
|
||||
return {
|
||||
"step": step["step"],
|
||||
"account": step["account"],
|
||||
"ok": True,
|
||||
"blueprint": step["blueprint"],
|
||||
"label": step["label"],
|
||||
"chain_position": step.get("chain_position", ""),
|
||||
"code": step.get("my_code", ""),
|
||||
"target_code": step.get("target_code", ""),
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
"""
|
||||
proxy.py — 代理管理 CLI 模块
|
||||
|
||||
基于 sms_proxy_api.py 的 ProxyManager,提供 CLI 封装
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
import urllib.request
|
||||
from datetime import datetime
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ProxyCLI:
|
||||
"""代理管理 CLI"""
|
||||
|
||||
@staticmethod
|
||||
def list_all() -> list:
|
||||
"""获取所有账号的代理配置"""
|
||||
try:
|
||||
from matrix_mgmt import MatrixManager
|
||||
mgr = MatrixManager()
|
||||
accounts = mgr.list_accounts()
|
||||
except Exception as e:
|
||||
return [{"error": str(e)}]
|
||||
|
||||
from pathlib import Path
|
||||
IDENTITIES_ROOT = AGENT_LOCAL / "tools" / "matrix" / "identities"
|
||||
|
||||
results = []
|
||||
for a in accounts:
|
||||
if not a.get("is_local"):
|
||||
continue
|
||||
aid = a["id"]
|
||||
config_path = IDENTITIES_ROOT / aid / "config.yaml"
|
||||
proxy = None
|
||||
if config_path.exists():
|
||||
try:
|
||||
import yaml
|
||||
data = yaml.safe_load(config_path.read_text())
|
||||
proxy = data.get("identity", {}).get("proxy", None)
|
||||
except:
|
||||
pass
|
||||
results.append({
|
||||
"account": aid,
|
||||
"platform": a.get("platform", ""),
|
||||
"proxy": proxy if proxy and str(proxy).strip() not in ("null", "None", "") else None,
|
||||
"has_proxy": bool(proxy and str(proxy).strip() not in ("null", "None", "")),
|
||||
"login_status": a.get("_status", "unknown"),
|
||||
})
|
||||
return results
|
||||
|
||||
@staticmethod
|
||||
def test(proxy_str: str) -> dict:
|
||||
"""测试代理连通性"""
|
||||
try:
|
||||
start = datetime.now()
|
||||
if proxy_str and str(proxy_str).strip() not in ("null", "None", ""):
|
||||
handler = urllib.request.ProxyHandler({"http": proxy_str, "https": proxy_str})
|
||||
opener = urllib.request.build_opener(handler)
|
||||
resp = opener.open("http://httpbin.org/ip", timeout=10)
|
||||
data = json.loads(resp.read().decode())
|
||||
elapsed = (datetime.now() - start).total_seconds()
|
||||
proxy_type = "HTTP"
|
||||
if proxy_str.startswith("socks5://") or proxy_str.startswith("socks://"):
|
||||
proxy_type = "SOCKS5"
|
||||
return {
|
||||
"status": "ok", "ip": data.get("origin", ""),
|
||||
"type": proxy_type, "elapsed": f"{elapsed:.1f}s",
|
||||
"speed": "快" if elapsed < 0.5 else ("中" if elapsed < 2 else "慢"),
|
||||
}
|
||||
else:
|
||||
resp = urllib.request.urlopen("http://httpbin.org/ip", timeout=10)
|
||||
data = json.loads(resp.read().decode())
|
||||
elapsed = (datetime.now() - start).total_seconds()
|
||||
return {"status": "ok", "ip": data.get("origin", ""), "elapsed": f"{elapsed:.1f}s", "direct": True}
|
||||
except Exception as e:
|
||||
return {"status": "error", "error": str(e)}
|
||||
|
||||
@staticmethod
|
||||
def set_proxy(account_id: str, proxy_str: str) -> dict:
|
||||
"""设置账号代理"""
|
||||
from pathlib import Path
|
||||
IDENTITIES_ROOT = AGENT_LOCAL / "tools" / "matrix" / "identities"
|
||||
|
||||
config_path = IDENTITIES_ROOT / account_id / "config.yaml"
|
||||
if not config_path.exists():
|
||||
return {"status": "error", "error": f"账号 {account_id} 身份目录不存在"}
|
||||
|
||||
try:
|
||||
import yaml
|
||||
data = yaml.safe_load(config_path.read_text()) or {}
|
||||
if "identity" not in data:
|
||||
data["identity"] = {}
|
||||
data["identity"]["proxy"] = proxy_str if proxy_str.strip() else None
|
||||
config_path.write_text(yaml.dump(data, default_flow_style=False, allow_unicode=True))
|
||||
return {"status": "ok", "account": account_id, "proxy": proxy_str}
|
||||
except Exception as e:
|
||||
return {"status": "error", "error": str(e)}
|
||||
@@ -0,0 +1,730 @@
|
||||
"""
|
||||
recorder.py — 原子操作录制引擎 (v2.0)
|
||||
|
||||
三阶段管线 Phase 1:
|
||||
启动 Camoufox → 用户操作 → 反引号 `·` 标记步骤 / Esc 结束 →
|
||||
系统自动捕获: URL + DOM + 截图 + CDP事件序列
|
||||
|
||||
用法:
|
||||
from mc.recorder import RecordingSession
|
||||
session = RecordingSession("douyin_01")
|
||||
await session.start()
|
||||
# 用户操作后在浏览器按 `·` 标记步骤,按 Esc 结束
|
||||
package = await session.stop()
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
|
||||
from matrix_mgmt import AGENT_LOCAL
|
||||
from ops._base import PageState
|
||||
LOCAL_ROOT = AGENT_LOCAL / "tools" / "matrix"
|
||||
RECORDINGS_DIR = LOCAL_ROOT / "recordings"
|
||||
RECORDINGS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
class RecordingSession:
|
||||
"""一次录制会话——管理浏览器、监听事件、保存步骤"""
|
||||
|
||||
def __init__(self, account_id: str, platform: str = "auto",
|
||||
entry_url: str = None):
|
||||
self.account_id = account_id
|
||||
self.platform = platform
|
||||
self.entry_url = entry_url
|
||||
self.conn = None
|
||||
self.page = None
|
||||
self._event_buffer = [] # CDP 事件缓存(两次步骤之间)
|
||||
self._steps = [] # 已保存的步骤
|
||||
self._is_recording = False
|
||||
self._start_time = None
|
||||
self._js_listener_ready = False
|
||||
self.machine = ""
|
||||
self.phone = ""
|
||||
self.nickname = ""
|
||||
|
||||
async def start(self):
|
||||
"""启动 Camoufox + 注入事件监听"""
|
||||
from cdp_connector import CDPConnector
|
||||
|
||||
# 解析身份目录
|
||||
from matrix_mgmt import MatrixManager
|
||||
mgr = MatrixManager()
|
||||
accts = mgr.list_accounts()
|
||||
acct_info = None
|
||||
for a in accts:
|
||||
if a["id"] == self.account_id:
|
||||
acct_info = a
|
||||
break
|
||||
if not acct_info:
|
||||
raise ValueError(f"账号 {self.account_id} 不存在")
|
||||
|
||||
# 保存元数据:机器名/手机号/昵称
|
||||
import os
|
||||
self.machine = os.uname().nodename
|
||||
self.phone = acct_info.get("phone", "")
|
||||
# 从 acct_info 或 profiles.json 找昵称
|
||||
self.nickname = acct_info.get("nickname", "")
|
||||
if not self.nickname:
|
||||
try:
|
||||
import json
|
||||
pf = LOCAL_ROOT / "data" / "profiles.json"
|
||||
if pf.exists():
|
||||
profiles = json.loads(pf.read_text())
|
||||
p = profiles.get(self.account_id, {})
|
||||
self.nickname = p.get("nickname", "")
|
||||
except:
|
||||
pass
|
||||
|
||||
identity_dir_name = (acct_info.get("identity_dir") or acct_info.get("identity_hint") or self.account_id).replace("identities/", "")
|
||||
identity_dir = str(LOCAL_ROOT / "identities" / identity_dir_name)
|
||||
|
||||
print(f"🦀 启动 Camoufox ({self.account_id})...")
|
||||
conn = CDPConnector(
|
||||
browser_type="camoufox",
|
||||
headless=False,
|
||||
window=(702, 783),
|
||||
identity_dir=identity_dir,
|
||||
)
|
||||
await conn.connect()
|
||||
await conn.init_anti_detection()
|
||||
self.conn = conn
|
||||
self.page = conn.page
|
||||
self._start_time = time.time()
|
||||
self._is_recording = True
|
||||
|
||||
# 导航到入口页("auto" 时从账号解析平台)
|
||||
if self.platform == "auto" and self.account_id:
|
||||
try:
|
||||
import sys
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
from matrix_mgmt import MatrixManager
|
||||
mgr = MatrixManager()
|
||||
for a in mgr.list_accounts():
|
||||
if a["id"] == self.account_id:
|
||||
p = a.get("platform", "")
|
||||
if p in ("douyin", "xiaohongshu"):
|
||||
self.platform = p
|
||||
break
|
||||
except: pass
|
||||
target = self.entry_url or {
|
||||
"douyin": "https://www.douyin.com/",
|
||||
"xiaohongshu": "https://www.xiaohongshu.com/explore",
|
||||
}.get(self.platform, "https://www.douyin.com/")
|
||||
|
||||
print(f"🌐 打开 {target}")
|
||||
await self.page.goto(target, timeout=30000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(4)
|
||||
|
||||
# 注入 JS 事件监听器(捕获键盘/鼠标事件到 window.__recorded_events)
|
||||
await self._inject_event_listener()
|
||||
|
||||
print(f"\n{'='*55}")
|
||||
print(f" 🎬 录制就绪!")
|
||||
print(f" 账号: {self.account_id}")
|
||||
print(f" 在浏览器中操作,然后按:")
|
||||
print(f" 反引号 `·` → 标记步骤(截图+状态指纹)")
|
||||
print(f" Esc → 结束录制")
|
||||
print(f"{'='*55}\n")
|
||||
|
||||
async def _inject_event_listener(self):
|
||||
"""注入 CDP 事件捕获器——记录用户操作"""
|
||||
try:
|
||||
await self.page.evaluate("""() => {
|
||||
if (window.__recorder_injected) return;
|
||||
window.__recorder_injected = true;
|
||||
window.__recorded_events = [];
|
||||
|
||||
// 监听键盘事件
|
||||
document.addEventListener('keydown', (e) => {
|
||||
window.__recorded_events.push({
|
||||
t: 'key',
|
||||
k: e.key,
|
||||
c: e.ctrlKey,
|
||||
m: e.metaKey,
|
||||
a: e.altKey,
|
||||
s: e.shiftKey,
|
||||
code: e.code,
|
||||
ts: Date.now()
|
||||
});
|
||||
}, true);
|
||||
|
||||
// 监听鼠标点击——瞬间捕获完整元素特征
|
||||
document.addEventListener('mousedown', (e) => {
|
||||
var el = e.target;
|
||||
var r = el.getBoundingClientRect && el.getBoundingClientRect();
|
||||
var p = el.parentElement;
|
||||
var clickInfo = {
|
||||
t: 'click',
|
||||
x: Math.round(e.clientX),
|
||||
y: Math.round(e.clientY),
|
||||
btn: e.button,
|
||||
ts: Date.now(),
|
||||
tag: (el.tagName || '').toLowerCase(),
|
||||
cls: (typeof el.className === 'string') ? el.className.slice(0,80) : '',
|
||||
id: el.id || '',
|
||||
text: (el.textContent || '').trim().slice(0,30),
|
||||
e2e: el.getAttribute ? (el.getAttribute('data-e2e') || '') : '',
|
||||
role: el.getAttribute ? (el.getAttribute('role') || '') : '',
|
||||
rect: r ? Math.round(r.x)+','+Math.round(r.y)+' '+Math.round(r.w)+'x'+Math.round(r.h) : '',
|
||||
visible: r ? (r.width > 0 && r.height > 0) : false,
|
||||
parentTag: p ? p.tagName.toLowerCase() : '',
|
||||
parentCls: (p && typeof p.className === 'string') ? p.className.slice(0,40) : '',
|
||||
};
|
||||
// 额外捕获最近的可交互祖先
|
||||
var walk = el;
|
||||
for (var w = 0; w < 3 && walk; w++) {
|
||||
var wr = walk.getBoundingClientRect();
|
||||
if (wr.width > 0 && wr.height > 0 && (walk.tagName === 'BUTTON' || walk.tagName === 'A' || walk.getAttribute('role') === 'button')) {
|
||||
clickInfo['ancestorTag'] = walk.tagName.toLowerCase();
|
||||
clickInfo['ancestorCls'] = (typeof walk.className === 'string') ? walk.className.slice(0,40) : '';
|
||||
clickInfo['ancestorText'] = (walk.textContent || '').trim().slice(0,30);
|
||||
break;
|
||||
}
|
||||
walk = walk.parentElement;
|
||||
}
|
||||
window.__recorded_events.push(clickInfo);
|
||||
}, true);
|
||||
|
||||
// 监听鼠标悬浮(元素边界变化时记录,不会产生大量事件)
|
||||
document.addEventListener('mouseover', (e) => {
|
||||
const t = e.target;
|
||||
const tag = (t.tagName || '').toLowerCase();
|
||||
const txt = (t.textContent || '').trim().slice(0, 20);
|
||||
const cls = (t.className && typeof t.className === 'string') ? t.className.slice(0,25) : '';
|
||||
// 只记录有意义的悬浮:按钮/链接/菜单等
|
||||
if (['a','button','li','span','div','img'].includes(tag) && (txt || cls)) {
|
||||
window.__recorded_events.push({
|
||||
t: 'hover',
|
||||
tag: tag,
|
||||
text: txt,
|
||||
cls: cls,
|
||||
x: e.clientX,
|
||||
y: e.clientY,
|
||||
ts: Date.now()
|
||||
});
|
||||
}
|
||||
}, true);
|
||||
|
||||
// 监听鼠标滚轮
|
||||
document.addEventListener('wheel', (e) => {
|
||||
window.__recorded_events.push({
|
||||
t: 'scroll',
|
||||
dx: Math.round(e.deltaX),
|
||||
dy: Math.round(e.deltaY),
|
||||
ts: Date.now()
|
||||
});
|
||||
}, true);
|
||||
}""")
|
||||
self._js_listener_ready = True
|
||||
except Exception as e:
|
||||
print(f"⚠️ 事件监听注入失败: {e}")
|
||||
self._js_listener_ready = False
|
||||
|
||||
async def _flush_events(self) -> list:
|
||||
"""取出并清空事件缓存——如在页面跳转后丢失JS监听器则自动重注入"""
|
||||
if not self._js_listener_ready or not self.page:
|
||||
events = list(self._event_buffer)
|
||||
self._event_buffer = []
|
||||
return events
|
||||
try:
|
||||
result = await self.page.evaluate("""() => {
|
||||
// 检查JS监听器是否还在(页面可能导航过)
|
||||
if (typeof window.__recorded_events === 'undefined' || !window.__recorder_injected) {
|
||||
return JSON.stringify({need_reinject: true, events: []});
|
||||
}
|
||||
var buf = window.__recorded_events;
|
||||
window.__recorded_events = [];
|
||||
return JSON.stringify({need_reinject: false, events: buf.slice(-200)});
|
||||
}""")
|
||||
import json as _json
|
||||
data = _json.loads(result)
|
||||
if data.get('need_reinject'):
|
||||
print(f" 🔄 页面导航检测,重新注入事件监听器")
|
||||
await self._inject_event_listener()
|
||||
return []
|
||||
return data.get('events', []) or []
|
||||
except Exception as _e:
|
||||
# page.evaluate 失败 = 页面已关闭或导航中
|
||||
return []
|
||||
|
||||
async def _capture_page_state(self) -> dict:
|
||||
"""捕获当前页面的状态指纹(用于录制分析)
|
||||
|
||||
使用与三段式操作相同的 PageState 结构,便于 Analyzer 分析。
|
||||
"""
|
||||
state = PageState()
|
||||
if not self.page:
|
||||
return state.to_dict()
|
||||
try:
|
||||
state.url_pattern = self.page.url
|
||||
# 页面模式
|
||||
text = await self.page.evaluate("(document.body.innerText || '').trim()") or ""
|
||||
has_video = await self.page.evaluate("document.querySelectorAll('video').length > 0")
|
||||
if '/video/' in state.url_pattern and has_video:
|
||||
state.page_mode = 'player'
|
||||
elif has_video:
|
||||
state.page_mode = 'player'
|
||||
elif '/user/' in state.url_pattern or '/profile' in state.url_pattern:
|
||||
state.page_mode = 'profile'
|
||||
elif '/search/' in state.url_pattern:
|
||||
state.page_mode = 'search'
|
||||
elif 'xiaohongshu' in state.url_pattern:
|
||||
state.page_mode = 'grid' if 'explore' in state.url_pattern else 'unknown'
|
||||
else:
|
||||
state.page_mode = 'grid'
|
||||
# 关键选择器可见性
|
||||
for sel in ['[data-e2e="video-player-digg"]', '[data-e2e="video-player-collect"]',
|
||||
'[data-e2e="searchbar-input"]', 'video', 'section.note-item']:
|
||||
try:
|
||||
state.selectors[sel] = await self.page.locator(sel).first.is_visible()
|
||||
except:
|
||||
state.selectors[sel] = False
|
||||
# 文本片段
|
||||
state.text_snippet = text[:2000]
|
||||
except:
|
||||
pass
|
||||
return state.to_dict()
|
||||
|
||||
async def record_step(self, step_number: int) -> dict:
|
||||
"""标记一个步骤——捕获当前页面状态 + 事件序列 + 状态指纹"""
|
||||
if not self._is_recording or not self.page:
|
||||
return {"step": step_number, "error": "not_recording"}
|
||||
|
||||
# 捕获操作前状态
|
||||
before_state = await self._capture_page_state()
|
||||
ts = datetime.now()
|
||||
events = await self._flush_events()
|
||||
step_data = {
|
||||
"step": step_number,
|
||||
"time": ts.strftime("%H:%M:%S"),
|
||||
"ts_ms": int(ts.timestamp() * 1000),
|
||||
"elapsed": round(time.time() - self._start_time, 1),
|
||||
"events_since_last": len(events),
|
||||
"events": events,
|
||||
"before_state": before_state,
|
||||
}
|
||||
|
||||
# 从点击事件中提取最佳选择器(供后续自动化回放)
|
||||
for e in reversed(events):
|
||||
if e.get('t') == 'click':
|
||||
sel = None
|
||||
eid = e.get('id', '')
|
||||
ee2e = e.get('e2e', '')
|
||||
etag = e.get('tag', '')
|
||||
e_cls = e.get('cls', '').strip()
|
||||
# 优先级: id > data-e2e > tag+cls > ancestor
|
||||
if eid:
|
||||
sel = f'#{eid}'
|
||||
elif ee2e:
|
||||
sel = f'[data-e2e="{ee2e}"]'
|
||||
elif etag and e_cls:
|
||||
first_cls = e_cls.split()[0]
|
||||
sel = f'{etag}.{first_cls}'
|
||||
elif e.get('ancestorTag') and e.get('ancestorCls'):
|
||||
a_cls = e['ancestorCls'].split()[0]
|
||||
sel = f'{e["ancestorTag"]}.{a_cls}'
|
||||
if sel:
|
||||
step_data["click_selector"] = sel
|
||||
step_data["click_info"] = {
|
||||
"x": e.get('x'), "y": e.get('y'),
|
||||
"id": eid, "tag": etag, "cls": e_cls,
|
||||
}
|
||||
break # 只取最近一次点击
|
||||
|
||||
# 采集页面状态
|
||||
try:
|
||||
state = await self.page.evaluate("""() => {
|
||||
const text = (document.body.innerText || '').trim();
|
||||
|
||||
// 收集页面上所有可见的可交互元素(包括弹层里的)
|
||||
const interactables = [];
|
||||
const found = new Set();
|
||||
// 获取所有可见元素(不限制选择器)
|
||||
const allEls = document.querySelectorAll('input, button, a, textarea, [contenteditable], [tabindex], [role="button"], [role="link"], [data-e2e], select, [class*="btn"], [class*="tab"]');
|
||||
allEls.forEach(el => {
|
||||
const r = el.getBoundingClientRect();
|
||||
if (r.width > 0 && r.height > 0 && r.top < window.innerHeight) {
|
||||
const key = el.tagName + (typeof el.className === 'string' ? el.className.slice(0,20) : '');
|
||||
if (!found.has(key)) {
|
||||
found.add(key);
|
||||
const e2e = el.getAttribute && el.getAttribute('data-e2e');
|
||||
interactables.push({
|
||||
tag: el.tagName.toLowerCase(),
|
||||
text: (el.textContent||'').trim().slice(0,35),
|
||||
cls: (typeof el.className === 'string') ? el.className.slice(0,50) : '',
|
||||
e2e: e2e || '',
|
||||
placeholder: el.getAttribute ? (el.getAttribute('placeholder') || '') : '',
|
||||
type: el.getAttribute ? (el.getAttribute('type') || '') : '',
|
||||
rect: `${Math.round(r.x)},${Math.round(r.y)} ${Math.round(r.w)}x${Math.round(r.h)}`,
|
||||
vis: r.top < window.innerHeight && r.left < window.innerWidth
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// 额外:专门查找所有 input(可能是登录浮层里的)
|
||||
const allInputs = document.querySelectorAll('input');
|
||||
allInputs.forEach(el => {
|
||||
const r = el.getBoundingClientRect();
|
||||
const key = 'INPUT_extra_' + (el.placeholder || '') + (el.className || '');
|
||||
if (r.width > 0 && r.height > 0 && !found.has(key)) {
|
||||
found.add(key);
|
||||
interactables.push({
|
||||
tag: 'input',
|
||||
text: '',
|
||||
cls: (typeof el.className === 'string') ? el.className.slice(0,50) : '',
|
||||
e2e: el.getAttribute ? (el.getAttribute('data-e2e') || '') : '',
|
||||
placeholder: el.getAttribute ? (el.getAttribute('placeholder') || '') : '',
|
||||
type: el.getAttribute ? (el.getAttribute('type') || '') : '',
|
||||
rect: `${Math.round(r.x)},${Math.round(r.y)} ${Math.round(r.w)}x${Math.round(r.h)}`,
|
||||
vis: true
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
return {
|
||||
url: window.location.href,
|
||||
title: document.title,
|
||||
text_snippet: text.slice(0, 4000),
|
||||
interactable_count: interactables.length,
|
||||
interactables: interactables.slice(0, 50),
|
||||
viewport: `${window.innerWidth}x${window.innerHeight}`,
|
||||
page_mode: text.includes('视频') && document.querySelector('video') ? 'player'
|
||||
: text.includes('关注') && text.includes('粉丝') ? 'profile'
|
||||
: text.length > 500 ? 'grid' : 'other',
|
||||
};
|
||||
}""")
|
||||
step_data["page"] = state
|
||||
except Exception as e:
|
||||
step_data["page"] = {"error": str(e)}
|
||||
|
||||
# 保存页面关键区域 HTML(供后续精确分析元素特征)
|
||||
try:
|
||||
html_snapshot = await self.page.evaluate("""() => {
|
||||
// 优先保存登录弹层/浮标的 HTML
|
||||
var containers = document.querySelectorAll(
|
||||
'#douyin_login_comp_btn_id, ' + // 抖音登录按钮
|
||||
'.r7j70rK2, ' + // 抖音登录层
|
||||
'.login-container, ' + // 小红书登录层
|
||||
'[class*=\"login\"], ' + // 任何含login的容器
|
||||
'[class*=\"modal\"]' // 弹窗类
|
||||
);
|
||||
if (containers.length > 0) {
|
||||
var seen = new Set();
|
||||
var parts = [];
|
||||
for (var i = 0; i < containers.length && parts.length < 3; i++) {
|
||||
var html = containers[i].outerHTML;
|
||||
if (!seen.has(html) && html.length < 5000) {
|
||||
seen.add(html);
|
||||
parts.push(html);
|
||||
}
|
||||
}
|
||||
return parts.join('\\n---\\n');
|
||||
}
|
||||
// 没有登录层时保存 body 的前 3000 字符
|
||||
return (document.body ? document.body.innerHTML.slice(0, 3000) : '');
|
||||
}""")
|
||||
if html_snapshot and len(html_snapshot) > 50:
|
||||
step_data["page_html"] = html_snapshot
|
||||
except:
|
||||
pass
|
||||
|
||||
# 截图
|
||||
try:
|
||||
ss_dir = RECORDINGS_DIR / "screenshots"
|
||||
ss_dir.mkdir(parents=True, exist_ok=True)
|
||||
ss_file = ss_dir / f"step{step_number}_{ts.strftime('%H%M%S')}.png"
|
||||
await self.page.screenshot(path=str(ss_file))
|
||||
step_data["screenshot"] = str(ss_file)
|
||||
except:
|
||||
step_data["screenshot"] = None
|
||||
|
||||
self._steps.append(step_data)
|
||||
|
||||
# 输出摘要
|
||||
p = step_data.get("page", {})
|
||||
mode = p.get("page_mode", "?")
|
||||
url_short = p.get("url", "?")[:50]
|
||||
print(f" [{step_number}] {mode} {url_short} "
|
||||
f"events={step_data['events_since_last']} "
|
||||
f"interactables={p.get('interactable_count', '?')}")
|
||||
if events:
|
||||
# 显示最后几个重要事件
|
||||
key_events = [e for e in events if e.get('t') in ('key', 'click')][-3:]
|
||||
for e in key_events:
|
||||
if e['t'] == 'key':
|
||||
print(f" ⌨️ {e.get('k','?')}")
|
||||
elif e['t'] == 'click':
|
||||
print(f" 🖱️ 点击 ({e.get('x',0)},{e.get('y',0)}) <{e.get('target','')}>")
|
||||
|
||||
return step_data
|
||||
|
||||
async def stop(self, keep_open: bool = True) -> dict:
|
||||
"""结束录制——打包保存
|
||||
|
||||
Args:
|
||||
keep_open: 是否保持浏览器打开(默认 True)
|
||||
"""
|
||||
self._is_recording = False
|
||||
|
||||
# 结束前刷新未读取的事件(Esc退出前的最后一次点击等)
|
||||
try:
|
||||
remaining = await self._flush_events()
|
||||
if remaining:
|
||||
self._steps.append({
|
||||
"step": len(self._steps) + 1,
|
||||
"time": datetime.now().strftime("%H:%M:%S"),
|
||||
"ts_ms": int(time.time() * 1000),
|
||||
"elapsed": round(time.time() - self._start_time, 1),
|
||||
"events_since_last": len(remaining),
|
||||
"events": remaining,
|
||||
"note": "结束前的最后一次操作事件"
|
||||
})
|
||||
except:
|
||||
pass
|
||||
|
||||
package = {
|
||||
"meta": {
|
||||
"account_id": self.account_id,
|
||||
"platform": self.platform,
|
||||
"machine": self.machine,
|
||||
"phone": self.phone,
|
||||
"nickname": self.nickname,
|
||||
"start_time": datetime.fromtimestamp(self._start_time).strftime("%H:%M:%S"),
|
||||
"end_time": datetime.now().strftime("%H:%M:%S"),
|
||||
"duration": round(time.time() - self._start_time, 1),
|
||||
"total_steps": len([s for s in self._steps if isinstance(s.get('step'), int)]),
|
||||
"created": datetime.now().isoformat(),
|
||||
},
|
||||
"steps": self._steps,
|
||||
}
|
||||
|
||||
# 保存录制包
|
||||
ts = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
pkg_file = RECORDINGS_DIR / f"recording_{self.account_id}_{ts}.json"
|
||||
with open(pkg_file, "w", encoding="utf-8") as f:
|
||||
json.dump(package, f, ensure_ascii=False, indent=2)
|
||||
|
||||
print(f"\n💾 录制包已保存: {pkg_file}")
|
||||
print(f" {package['meta']['total_steps']} 步, "
|
||||
f"耗时 {package['meta']['duration']}s")
|
||||
|
||||
# 关闭浏览器(keep_open=False 时才关)
|
||||
if self.conn and not keep_open:
|
||||
try:
|
||||
await self.conn.close()
|
||||
print("🛑 浏览器已关闭")
|
||||
except:
|
||||
pass
|
||||
elif self.conn and keep_open:
|
||||
print("🔓 浏览器保持打开状态")
|
||||
|
||||
return package
|
||||
|
||||
@staticmethod
|
||||
def list_recordings() -> list:
|
||||
"""列出所有录制包"""
|
||||
files = sorted(RECORDINGS_DIR.glob("recording_*.json"))
|
||||
result = []
|
||||
for f in files:
|
||||
try:
|
||||
raw = f.read_text(encoding="utf-8")
|
||||
# 只解析 meta 部分,避免加载大量 steps 数据
|
||||
meta_end = raw.find('"steps"')
|
||||
if meta_end > 0:
|
||||
meta_json = raw[:meta_end].rstrip(',\n ') + '}'
|
||||
pkg = json.loads(meta_json)
|
||||
else:
|
||||
pkg = json.loads(raw)
|
||||
meta = pkg.get("meta", {})
|
||||
result.append({
|
||||
"file": f.name,
|
||||
"path": str(f),
|
||||
"account": meta.get("account_id", "?"),
|
||||
"platform": meta.get("platform", "?"),
|
||||
"steps": meta.get("total_steps", 0),
|
||||
"duration": meta.get("duration", 0),
|
||||
"created": meta.get("created", ""),
|
||||
})
|
||||
except:
|
||||
result.append({"file": f.name, "error": "parse_error"})
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def delete_recording(name_or_path: str) -> bool:
|
||||
"""删除录制包和关联的截图"""
|
||||
p = Path(name_or_path)
|
||||
if not p.exists():
|
||||
p = RECORDINGS_DIR / name_or_path
|
||||
if not p.exists():
|
||||
return False
|
||||
# 删除录制包
|
||||
p.unlink()
|
||||
# 也删除关联的截图(可选)
|
||||
ss_dir = RECORDINGS_DIR / "screenshots"
|
||||
if ss_dir.exists():
|
||||
prefix = p.stem.replace("recording_", "")
|
||||
for ss in ss_dir.glob(f"*{prefix}*"):
|
||||
ss.unlink()
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
def load_recording(name_or_path: str) -> dict:
|
||||
"""加载录制包"""
|
||||
p = Path(name_or_path)
|
||||
if not p.exists():
|
||||
p = RECORDINGS_DIR / name_or_path
|
||||
if not p.exists():
|
||||
raise FileNotFoundError(f"录制包不存在: {name_or_path}")
|
||||
return json.loads(p.read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
# ── 独立运行(热键监听模式)──
|
||||
async def _run_interactive(account_id: str, platform: str, timeout_minutes: int = 60,
|
||||
keep_open: bool = True):
|
||||
"""交互式录制——反引号标记步骤, Esc 结束
|
||||
|
||||
操作方式:
|
||||
- 在浏览器窗口中操作
|
||||
- 每完成一个动作 → 按反引号 `·` 键标记一步
|
||||
- Esc → 结束录制
|
||||
- 无操作超时自动退出
|
||||
|
||||
Args:
|
||||
account_id: 账号 ID
|
||||
platform: 平台 (douyin/xiaohongshu)
|
||||
timeout_minutes: 无操作超时分钟
|
||||
keep_open: 录制结束后是否保持浏览器打开
|
||||
"""
|
||||
session = RecordingSession(account_id, platform)
|
||||
await session.start()
|
||||
loop = asyncio.get_running_loop()
|
||||
timeout_seconds = timeout_minutes * 60
|
||||
last_activity = loop.time()
|
||||
step_counter = 0
|
||||
_manual_end = False
|
||||
|
||||
try:
|
||||
from pynput import keyboard
|
||||
|
||||
def on_release(key):
|
||||
nonlocal last_activity, step_counter, _manual_end
|
||||
try:
|
||||
char = getattr(key, 'char', None)
|
||||
# 反引号 `·` → 标记一步
|
||||
if char == '`':
|
||||
step_counter += 1
|
||||
last_activity = loop.time()
|
||||
asyncio.run_coroutine_threadsafe(
|
||||
session.record_step(step_counter), loop
|
||||
)
|
||||
return True
|
||||
|
||||
# Esc → 结束录制
|
||||
if key == keyboard.Key.esc:
|
||||
_manual_end = True
|
||||
asyncio.run_coroutine_threadsafe(
|
||||
session.stop(keep_open=True), loop
|
||||
)
|
||||
return False
|
||||
|
||||
return True
|
||||
except:
|
||||
return True
|
||||
|
||||
listener = keyboard.Listener(on_release=on_release)
|
||||
listener.start()
|
||||
print(f"⌨️ 反引号 `·` 标记步骤, Esc 结束录制")
|
||||
print(f" 无操作 {timeout_minutes} 分钟自动退出\n")
|
||||
|
||||
# 周期性检测JS监听器是否存活(页面导航后会丢失)
|
||||
_check_interval = 0
|
||||
while session._is_recording:
|
||||
await asyncio.sleep(1)
|
||||
_check_interval += 1
|
||||
if _check_interval >= 3: # 每3秒检查一次
|
||||
_check_interval = 0
|
||||
try:
|
||||
_alive = await session.page.evaluate("typeof window.__recorder_injected === 'boolean'")
|
||||
if not _alive:
|
||||
print(f" 🔄 JS监听器已丢失,重新注入...")
|
||||
await session._inject_event_listener()
|
||||
except:
|
||||
pass
|
||||
if loop.time() - last_activity > timeout_seconds:
|
||||
print(f"\n⏰ 无操作超过 {timeout_minutes} 分钟,自动退出")
|
||||
await session.stop(keep_open=False)
|
||||
break
|
||||
|
||||
listener.stop()
|
||||
print(f"\n✅ 录制结束, 共 {step_counter} 步")
|
||||
|
||||
except ImportError:
|
||||
print("⚠️ pynput 不可用,使用终端模式 (每操作一步按一次 Enter)")
|
||||
print(" 输入 q + Enter 结束录制\n")
|
||||
try:
|
||||
import sys, select
|
||||
while session._is_recording:
|
||||
# 非阻塞读 stdin
|
||||
r, _, _ = select.select([sys.stdin], [], [], 1.0)
|
||||
if r:
|
||||
line = sys.stdin.readline().strip()
|
||||
if line == 'q':
|
||||
_manual_end = True
|
||||
break
|
||||
step_counter += 1
|
||||
await session.record_step(step_counter)
|
||||
print(f" ✅ 已记录 第{step_counter}步, 继续操作后按 Enter...")
|
||||
except Exception:
|
||||
pass
|
||||
finally:
|
||||
if not _manual_end:
|
||||
await session.stop()
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n⚠️ 录制异常: {e}")
|
||||
# 异常不关闭浏览器(保持打开便于排查)
|
||||
print(f" 浏览器保持打开状态, 可手动操作或关闭")
|
||||
return # 不调用 session.stop(), 保持浏览器
|
||||
|
||||
finally:
|
||||
if keep_open and _manual_end:
|
||||
print(f"\n🔓 录制已完成, 浏览器保持打开 (可继续查看或手动关闭)")
|
||||
elif not keep_open:
|
||||
_cleanup_playwright_drivers()
|
||||
|
||||
|
||||
def _cleanup_playwright_drivers():
|
||||
"""清理残留的 Playwright driver 进程(运行超过1小时的)"""
|
||||
import subprocess, os
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["ps", "aux"], capture_output=True, text=True, timeout=10
|
||||
)
|
||||
for line in result.stdout.split("\n"):
|
||||
if "playwright/driver" in line and "node" in line:
|
||||
parts = line.split()
|
||||
if len(parts) > 1:
|
||||
pid = parts[1]
|
||||
try:
|
||||
# 只杀掉非当前进程的 playwright driver
|
||||
if pid != str(os.getpid()):
|
||||
subprocess.run(["kill", pid], capture_output=True, timeout=5)
|
||||
except:
|
||||
pass
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
account = sys.argv[1] if len(sys.argv) > 1 else "douyin_01"
|
||||
platform = sys.argv[2] if len(sys.argv) > 2 else "douyin"
|
||||
asyncio.run(_run_interactive(account, platform))
|
||||
@@ -0,0 +1,83 @@
|
||||
"""
|
||||
run.py — mc run 批量执行引擎 (v1.1 - 真实引擎)
|
||||
|
||||
使用 mc/engine.py 的 BatchEngine 实际启动浏览器执行原子操作,
|
||||
而非模拟步骤。
|
||||
|
||||
支持:
|
||||
- 多账号 + 多蓝图 + 多轮数
|
||||
- 混合随机模式
|
||||
- 浏览器生命周期管理
|
||||
- 日志输出
|
||||
|
||||
委派给:
|
||||
→ mc/engine.BatchEngine.run_single() → orchestrator.YanghaoOrchestrator()
|
||||
→ BrowserManager.launch() → AtomOps.execute() → 真实浏览器操作
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class BatchRunner:
|
||||
"""批量执行引擎 — 封装 BatchEngine 提供 CLI 友好接口"""
|
||||
|
||||
def __init__(self, accounts, blueprints, rounds=10,
|
||||
interval_range=(30, 60), mix=False,
|
||||
corpus=None, engine="auto", daemon=False,
|
||||
stagger="15-30", keep_open=False,
|
||||
max_browsers=3, url="", direction="",
|
||||
comment_text="", reply_text=""):
|
||||
self.accounts = accounts
|
||||
self.blueprints = blueprints
|
||||
self.rounds = rounds
|
||||
self.mix = mix
|
||||
self.interval_min, self.interval_max = interval_range
|
||||
self.corpus = corpus or []
|
||||
self.engine_type = engine
|
||||
self.daemon = daemon
|
||||
self.stagger = stagger
|
||||
self.keep_open = keep_open
|
||||
self.max_browsers = max_browsers
|
||||
self.url = url
|
||||
self.direction = direction
|
||||
self.comment_text = comment_text
|
||||
self.reply_text = reply_text
|
||||
|
||||
async def run(self) -> dict:
|
||||
"""执行 — 委托给 BatchEngine"""
|
||||
from mc.engine import BatchEngine
|
||||
|
||||
log.info(f" 🚀 使用真实浏览器引擎执行")
|
||||
log.info(f" 账号: {self.accounts}")
|
||||
log.info(f" 蓝图: {self.blueprints}")
|
||||
log.info(f" 轮数: {self.rounds} | 模式: {'混合随机' if self.mix else '顺序'}")
|
||||
log.info(f" 浏览器上限: {self.max_browsers} | 错峰: {self.stagger}s | keep_open: {self.keep_open}")
|
||||
|
||||
engine = BatchEngine(
|
||||
accounts=self.accounts,
|
||||
blueprints=self.blueprints,
|
||||
rounds=self.rounds,
|
||||
mix=self.mix,
|
||||
corpus=self.corpus,
|
||||
stagger=self.stagger,
|
||||
keep_open=self.keep_open,
|
||||
max_browsers=self.max_browsers,
|
||||
task_params={
|
||||
"url": self.url,
|
||||
"comment_text": self.comment_text,
|
||||
"reply_text": self.reply_text,
|
||||
"direction": self.direction,
|
||||
},
|
||||
)
|
||||
|
||||
report = await engine.run()
|
||||
return report.to_dict()
|
||||
@@ -0,0 +1,64 @@
|
||||
"""
|
||||
mc/runlog.py — 结构化运行日志 v1.0
|
||||
|
||||
每次 mc run 执行后,将结果写入 JSONL 格式的结构化日志。
|
||||
存储位置: agent-local/tools/matrix/logs/run_{date}_{account}.jsonl
|
||||
|
||||
供智能体和 Dashboard 读取分析。不依赖 print log。
|
||||
"""
|
||||
import json
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
HOME = Path.home()
|
||||
from matrix_mgmt import AGENT_LOCAL
|
||||
LOG_DIR = AGENT_LOCAL / "tools" / "matrix" / "logs"
|
||||
|
||||
|
||||
def write_run_log(account_id: str, report: dict):
|
||||
"""追加一条运行记录到当日 JSONL 日志"""
|
||||
log_path = LOG_DIR / f"run_{datetime.now().strftime('%Y%m%d')}_{account_id}.jsonl"
|
||||
LOG_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
entry = {
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
**report,
|
||||
}
|
||||
with open(log_path, "a", encoding="utf-8") as f:
|
||||
f.write(json.dumps(entry, ensure_ascii=False) + "\n")
|
||||
|
||||
|
||||
def read_latest_log(account_id: str, date_str: str = None) -> list[dict]:
|
||||
"""读取指定日期的运行日志(默认今天)"""
|
||||
if date_str is None:
|
||||
date_str = datetime.now().strftime("%Y%m%d")
|
||||
log_path = LOG_DIR / f"run_{date_str}_{account_id}.jsonl"
|
||||
if not log_path.exists():
|
||||
return []
|
||||
entries = []
|
||||
for line in log_path.read_text().splitlines():
|
||||
if line.strip():
|
||||
try:
|
||||
entries.append(json.loads(line))
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
return entries
|
||||
|
||||
|
||||
def get_today_summary() -> dict:
|
||||
"""获取今日所有账号的运行摘要"""
|
||||
today = datetime.now().strftime("%Y%m%d")
|
||||
summary = {"date": today, "accounts": {}}
|
||||
for f in sorted(LOG_DIR.glob(f"run_{today}_*.jsonl")):
|
||||
account = f.stem.replace(f"run_{today}_", "")
|
||||
entries = read_latest_log(account, today)
|
||||
if entries:
|
||||
last = entries[-1]
|
||||
summary["accounts"][account] = {
|
||||
"runs": len(entries),
|
||||
"last_blueprint": last.get("blueprint", "?"),
|
||||
"last_success": last.get("success", 0),
|
||||
"last_failed": last.get("failed", 0),
|
||||
"last_time": last.get("timestamp", "?"),
|
||||
}
|
||||
return summary
|
||||
@@ -0,0 +1,308 @@
|
||||
"""
|
||||
mc/scheduler.py — 定时任务调度器 v2.0
|
||||
|
||||
完整功能:
|
||||
- YAML 配置定时任务(支持 time + days 星期过滤)
|
||||
- 历史执行记录(JSONL)+ 联邦同步
|
||||
- 自动加载人设参数
|
||||
- 日志记录
|
||||
|
||||
配置文件: config/schedule.yaml
|
||||
格式:
|
||||
schedules:
|
||||
douyin_daily:
|
||||
enabled: true
|
||||
account: douyin_test
|
||||
blueprint: douyin_daily
|
||||
rounds: 3
|
||||
time: "09:00"
|
||||
days: "1,2,3,4,5,6,7" # 1=周一..7=周日
|
||||
|
||||
历史记录: agent-local/tools/matrix/logs/schedule_history.jsonl
|
||||
联邦同步: 04_memory/cross_machine/data/matrix/schedule_results.json
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
HOME = Path.home()
|
||||
from matrix_mgmt import AGENT_SYNC, AGENT_LOCAL
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent.parent
|
||||
CONFIG_DIR = SCRIPTS_DIR / "config"
|
||||
SCHEDULE_YAML = CONFIG_DIR / "schedule.yaml"
|
||||
SCHEDULE_YAML.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 历史记录(本机)
|
||||
HISTORY_DIR = AGENT_LOCAL / "tools" / "matrix" / "logs"
|
||||
HISTORY_DIR.mkdir(parents=True, exist_ok=True)
|
||||
HISTORY_FILE = HISTORY_DIR / "schedule_history.jsonl"
|
||||
|
||||
# 联邦同步
|
||||
CROSS_MACHINE_MATRIX = AGENT_SYNC / "04_memory" / "cross_machine" / "data" / "matrix"
|
||||
CROSS_MACHINE_MATRIX.mkdir(parents=True, exist_ok=True)
|
||||
FEDERATION_FILE = CROSS_MACHINE_MATRIX / "schedule_results.json"
|
||||
|
||||
HOSTNAME = os.uname().nodename
|
||||
|
||||
DEFAULT_CONFIG = """# Matrix 定时任务配置
|
||||
# time 格式: HH:MM(每天固定时间)
|
||||
# days 格式: 1,2,3,4,5,6,7(1=周一..7=周日,默认每天)
|
||||
schedules:
|
||||
douyin_daily_morning:
|
||||
enabled: false
|
||||
account: douyin_test
|
||||
blueprint: douyin_daily
|
||||
rounds: 3
|
||||
time: "09:00"
|
||||
days: "1,2,3,4,5,6,7"
|
||||
xhs_daily_noon:
|
||||
enabled: false
|
||||
account: xhs_01
|
||||
blueprint: xhs_daily
|
||||
rounds: 2
|
||||
time: "12:30"
|
||||
days: "1,2,3,4,5,6,7"
|
||||
"""
|
||||
|
||||
|
||||
# ── 配置读写 ──
|
||||
|
||||
def load_all_schedules() -> dict:
|
||||
"""读取所有任务配置(含禁用)"""
|
||||
if not SCHEDULE_YAML.exists():
|
||||
SCHEDULE_YAML.write_text(DEFAULT_CONFIG)
|
||||
return {}
|
||||
try:
|
||||
import yaml
|
||||
data = yaml.safe_load(SCHEDULE_YAML.read_text())
|
||||
return data.get("schedules", {})
|
||||
except Exception as e:
|
||||
log.error(f"读取定时配置失败: {e}")
|
||||
return {}
|
||||
|
||||
|
||||
def load_enabled_schedules() -> list[dict]:
|
||||
"""读取启用的任务(含今天是否执行判断)"""
|
||||
all_s = load_all_schedules()
|
||||
today = datetime.now().isoweekday() # 1=周一..7=周日
|
||||
result = []
|
||||
for sid, cfg in all_s.items():
|
||||
if not cfg.get("enabled", False):
|
||||
continue
|
||||
# days 过滤
|
||||
days_str = cfg.get("days", "1,2,3,4,5,6,7")
|
||||
try:
|
||||
days = [int(d.strip()) for d in days_str.split(",")]
|
||||
except:
|
||||
days = [1, 2, 3, 4, 5, 6, 7]
|
||||
if today not in days:
|
||||
continue
|
||||
cfg["id"] = sid
|
||||
result.append(cfg)
|
||||
return result
|
||||
|
||||
|
||||
def save_all_schedules(schedules: dict):
|
||||
"""保存全部任务配置"""
|
||||
import yaml
|
||||
SCHEDULE_YAML.write_text(yaml.dump(
|
||||
{"schedules": schedules}, allow_unicode=True, default_flow_style=False
|
||||
))
|
||||
|
||||
|
||||
# ── 历史记录 ──
|
||||
|
||||
def write_history(entry: dict):
|
||||
"""追加一条历史记录"""
|
||||
with open(HISTORY_FILE, "a", encoding="utf-8") as f:
|
||||
f.write(json.dumps(entry, ensure_ascii=False) + "\n")
|
||||
|
||||
|
||||
def read_history(schedule_id: str = None, limit: int = 20) -> list[dict]:
|
||||
"""读取历史记录,默认最近20条"""
|
||||
if not HISTORY_FILE.exists():
|
||||
return []
|
||||
entries = []
|
||||
for line in reversed(HISTORY_FILE.read_text().splitlines()):
|
||||
if not line.strip():
|
||||
continue
|
||||
try:
|
||||
e = json.loads(line)
|
||||
if schedule_id and e.get("schedule_id") != schedule_id:
|
||||
continue
|
||||
entries.append(e)
|
||||
if len(entries) >= limit:
|
||||
break
|
||||
except:
|
||||
pass
|
||||
return entries
|
||||
|
||||
|
||||
def sync_to_federation(result: dict):
|
||||
"""同步执行结果到联邦数据目录"""
|
||||
try:
|
||||
fed = {}
|
||||
if FEDERATION_FILE.exists():
|
||||
fed = json.loads(FEDERATION_FILE.read_text())
|
||||
host_data = fed.get(HOSTNAME, {})
|
||||
sid = result.get("schedule_id", "unknown")
|
||||
host_data[sid] = {
|
||||
"last_run": result.get("timestamp", datetime.now().isoformat()),
|
||||
"status": "success" if result.get("failed", 0) == 0 else "partial",
|
||||
"success": result.get("success", 0),
|
||||
"failed": result.get("failed", 0),
|
||||
"duration": result.get("duration", 0),
|
||||
"blueprint": result.get("blueprint", ""),
|
||||
}
|
||||
fed[HOSTNAME] = host_data
|
||||
FEDERATION_FILE.write_text(json.dumps(fed, ensure_ascii=False, indent=2))
|
||||
except Exception as e:
|
||||
log.error(f"联邦同步失败: {e}")
|
||||
|
||||
|
||||
# ── 任务执行 ──
|
||||
|
||||
def get_due_schedules() -> list[dict]:
|
||||
"""获取当前时间到点的任务"""
|
||||
now = datetime.now().strftime("%H:%M")
|
||||
return [s for s in load_enabled_schedules() if s.get("time", "") == now]
|
||||
|
||||
|
||||
async def run_scheduled_task(schedule: dict) -> dict:
|
||||
"""执行单个定时任务"""
|
||||
account = schedule.get("account", "")
|
||||
blueprint = schedule.get("blueprint", "")
|
||||
rounds = schedule.get("rounds", 1)
|
||||
args = dict(schedule.get("args", {}))
|
||||
|
||||
# 解析 @persona 标记
|
||||
for k, v in args.items():
|
||||
if isinstance(v, str) and v == "@persona":
|
||||
try:
|
||||
from mc.corpus import CorpusManager
|
||||
cm = CorpusManager()
|
||||
kws = cm.get_search_keywords(account)
|
||||
if kws:
|
||||
args[k] = kws[0]
|
||||
except:
|
||||
args[k] = "热门推荐"
|
||||
|
||||
log.info(f"⏰ 执行定时任务: {account} → {blueprint} ({rounds}轮)")
|
||||
|
||||
from mc.engine import BatchEngine
|
||||
engine = BatchEngine(
|
||||
accounts=[account],
|
||||
blueprints=[blueprint],
|
||||
rounds=rounds,
|
||||
)
|
||||
engine.task_params = args
|
||||
report = await engine.run()
|
||||
result = report.to_dict()
|
||||
result["schedule_id"] = schedule.get("id", "")
|
||||
result["scheduled_time"] = schedule.get("time", "")
|
||||
result["timestamp"] = datetime.now().isoformat()
|
||||
result["hostname"] = HOSTNAME
|
||||
|
||||
# 写入历史 + 联邦同步
|
||||
write_history(result)
|
||||
sync_to_federation(result)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
async def scheduler_loop():
|
||||
"""主循环:每30秒检查一次到点任务"""
|
||||
log.info("🕐 定时调度器已启动")
|
||||
last_minute = ""
|
||||
while True:
|
||||
now_minute = datetime.now().strftime("%H:%M")
|
||||
if now_minute != last_minute:
|
||||
last_minute = now_minute
|
||||
due = get_due_schedules()
|
||||
for s in due:
|
||||
sid = s.get("id", "?")
|
||||
log.info(f"⏰ [{now_minute}] 触发: {sid}")
|
||||
try:
|
||||
result = await run_scheduled_task(s)
|
||||
log.info(f" ✅ {sid}: {result.get('success',0)}/{result.get('total_steps',0)} 步成功, 耗时{result.get('duration',0)}s")
|
||||
except Exception as e:
|
||||
log.error(f" ❌ {sid}: {e}")
|
||||
write_history({
|
||||
"schedule_id": sid, "scheduled_time": s.get("time", ""),
|
||||
"timestamp": datetime.now().isoformat(), "hostname": HOSTNAME,
|
||||
"status": "error", "error": str(e),
|
||||
})
|
||||
await asyncio.sleep(30)
|
||||
|
||||
|
||||
# ── CLI ──
|
||||
|
||||
def cmd_schedule_list():
|
||||
"""列出所有定时任务"""
|
||||
all_s = load_all_schedules()
|
||||
if not all_s:
|
||||
print("暂无定时任务")
|
||||
print("配置位置: config/schedule.yaml")
|
||||
return
|
||||
print(f"\n{'ID':25s} {'账号':20s} {'蓝图':25s} {'时间':8s} {'轮数':4s} {'天数':12s} {'状态':6s}")
|
||||
print("-" * 100)
|
||||
for sid, cfg in all_s.items():
|
||||
enabled = "🟢" if cfg.get("enabled", False) else "⚪"
|
||||
days = cfg.get("days", "每天")
|
||||
print(f"{sid:25s} {cfg.get('account','?'):20s} "
|
||||
f"{cfg.get('blueprint','?'):25s} {cfg.get('time','?'):8s} "
|
||||
f"{cfg.get('rounds',1):4d} {days:12s} {enabled:6s}")
|
||||
|
||||
|
||||
def cmd_schedule_add(sid: str, account: str, blueprint: str,
|
||||
time_str: str, rounds: int = 1, days: str = "1,2,3,4,5,6,7",
|
||||
args: str = ""):
|
||||
"""添加定时任务"""
|
||||
all_s = load_all_schedules()
|
||||
all_s[sid] = {
|
||||
"enabled": True,
|
||||
"account": account,
|
||||
"blueprint": blueprint,
|
||||
"rounds": rounds,
|
||||
"time": time_str,
|
||||
"days": days,
|
||||
}
|
||||
if args:
|
||||
all_s[sid]["args"] = {"keyword": args}
|
||||
save_all_schedules(all_s)
|
||||
print(f"✅ 定时任务 '{sid}' 已添加({time_str} 每天{days})")
|
||||
|
||||
|
||||
def cmd_schedule_remove(sid: str):
|
||||
"""删除定时任务"""
|
||||
all_s = load_all_schedules()
|
||||
if sid in all_s:
|
||||
del all_s[sid]
|
||||
save_all_schedules(all_s)
|
||||
print(f"✅ 定时任务 '{sid}' 已删除")
|
||||
else:
|
||||
print(f"❌ 未找到定时任务 '{sid}'")
|
||||
|
||||
|
||||
def cmd_schedule_history(sid: str = "", limit: int = 10):
|
||||
"""查看历史记录"""
|
||||
entries = read_history(sid if sid else None, limit)
|
||||
if not entries:
|
||||
print("暂无历史记录")
|
||||
return
|
||||
print(f"\n最近 {len(entries)} 条执行记录{' ('+sid+')' if sid else ''}:")
|
||||
print(f"{'时间':20s} {'任务':25s} {'状态':10s} {'成功':6s} {'失败':6s} {'耗时':6s}")
|
||||
print("-" * 75)
|
||||
for e in entries:
|
||||
st = e.get("status", e.get("error", "?"))
|
||||
print(f"{e.get('timestamp','?'):20s} {e.get('schedule_id','?'):25s} "
|
||||
f"{st:10s} {str(e.get('success','?')):6s} {str(e.get('failed','?')):6s} "
|
||||
f"{str(e.get('duration','?')):6s}")
|
||||
@@ -0,0 +1,183 @@
|
||||
"""
|
||||
mc/task.py — 任务调度器 v1.0
|
||||
|
||||
智能任务:根据用户意图自动选择蓝图、填充参数、执行。
|
||||
|
||||
用法:
|
||||
mc task comment --url https://v.douyin.com/xxx --direction 正面
|
||||
mc task search --keyword 美食探店 --rounds 3
|
||||
mc task collect --keyword 博主名称
|
||||
mc task reply --account douyin_test
|
||||
|
||||
流程:
|
||||
1. 解析任务参数(URL/关键词/方向)
|
||||
2. 校验必须参数
|
||||
3. 平台识别(抖音/小红书)
|
||||
4. 自动选账号(同平台已登录的)
|
||||
5. 选蓝图
|
||||
6. 填充参数到蓝图
|
||||
7. 执行 mc run
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import random
|
||||
import re
|
||||
import sys
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
|
||||
PLATFORM_PATTERNS = {
|
||||
"douyin": [r"douyin\.com", r"v\.douyin\.com"],
|
||||
"xiaohongshu": [r"xiaohongshu\.com", r"xhslink\.com"],
|
||||
}
|
||||
|
||||
|
||||
def detect_platform(url_or_text: str) -> str:
|
||||
"""识别平台:douyin / xiaohongshu / unknown"""
|
||||
for plat, patterns in PLATFORM_PATTERNS.items():
|
||||
for p in patterns:
|
||||
if re.search(p, url_or_text.lower()):
|
||||
return plat
|
||||
return "unknown"
|
||||
|
||||
|
||||
def pick_account(platform: str) -> Optional[str]:
|
||||
"""从已登录账号中选一个同平台的"""
|
||||
try:
|
||||
from matrix_mgmt import MatrixManager
|
||||
mgr = MatrixManager()
|
||||
candidates = []
|
||||
for a in mgr.list_accounts():
|
||||
if a.get("platform") == platform and a.get("_status") in ("logged_in", "ok"):
|
||||
candidates.append(a["id"])
|
||||
return random.choice(candidates) if candidates else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
@dataclass
|
||||
class Task:
|
||||
"""一个待执行的任务"""
|
||||
type: str # comment / search / collect / reply
|
||||
platform: str = "" # 自动识别的平台
|
||||
account: str = "" # 指定或自动选的账号
|
||||
url: str = "" # 视频链接(定向评论用)
|
||||
keyword: str = "" # 搜索关键词
|
||||
direction: str = "" # 评论方向(正面/提问/共鸣等)
|
||||
comment_text: str = "" # 具体评论内容(不填则自动生成)
|
||||
rounds: int = 1 # 执行轮数
|
||||
blueprint: str = "" # 自动选的蓝图
|
||||
|
||||
def validate(self) -> list[str]:
|
||||
"""参数校验,返回错误列表"""
|
||||
errors = []
|
||||
if self.type == "comment":
|
||||
if not self.url:
|
||||
errors.append("定向评论必须提供视频链接(--url)")
|
||||
self.platform = detect_platform(self.url)
|
||||
if self.platform == "unknown" and self.url:
|
||||
errors.append(f"无法识别链接平台: {self.url}")
|
||||
elif self.type in ("search", "collect"):
|
||||
if not self.keyword:
|
||||
errors.append("搜索/采集必须提供关键词(--keyword)")
|
||||
self.platform = "douyin" # 默认抖音
|
||||
elif self.type == "reply":
|
||||
if not self.account:
|
||||
errors.append("作者回复必须指定账号(--account)")
|
||||
try:
|
||||
from matrix_mgmt import MatrixManager
|
||||
mgr = MatrixManager()
|
||||
acct = mgr.get_account(self.account)
|
||||
self.platform = acct.get("platform", "douyin") if acct else "douyin"
|
||||
except:
|
||||
self.platform = "douyin"
|
||||
return errors
|
||||
|
||||
def auto_fill(self):
|
||||
"""自动填空:选账号、选蓝图、生成评论"""
|
||||
# 选账号
|
||||
if not self.account and self.platform:
|
||||
picked = pick_account(self.platform)
|
||||
if picked:
|
||||
self.account = picked
|
||||
log.info(f" 🤖 自动选账号: {self.account}")
|
||||
# 选蓝图
|
||||
bp_map = {
|
||||
"comment": f"{self.platform}_comment" if self.platform != "unknown" else "douyin_comment",
|
||||
"search": f"{self.platform}_search" if self.platform != "unknown" else "douyin_search",
|
||||
"collect": f"{self.platform}_collect" if self.platform != "unknown" else "douyin_collect",
|
||||
"reply": f"{self.platform}_reply" if self.platform != "unknown" else "douyin_reply",
|
||||
}
|
||||
self.blueprint = bp_map.get(self.type, "douyin_daily")
|
||||
# 方向默认
|
||||
if not self.direction:
|
||||
self.direction = random.choice(["称赞", "提问", "共鸣", "感慨"])
|
||||
|
||||
def to_task_params(self) -> dict:
|
||||
"""转为 engine.py 的 task_params 格式"""
|
||||
params = {"direction": self.direction}
|
||||
if self.url:
|
||||
params["url"] = self.url
|
||||
if self.keyword:
|
||||
params["keyword"] = self.keyword
|
||||
if self.comment_text:
|
||||
params["comment_text"] = self.comment_text
|
||||
return params
|
||||
|
||||
def summary(self) -> str:
|
||||
return (f"{self.type.upper():8s} | 平台={self.platform:12s} | "
|
||||
f"账号={self.account:20s} | 蓝图={self.blueprint:25s} | "
|
||||
f"方向={self.direction}")
|
||||
|
||||
|
||||
async def run_task(task: Task) -> dict:
|
||||
"""执行任务:校验→填充→跑 mc run"""
|
||||
errors = task.validate()
|
||||
if errors:
|
||||
return {"status": "error", "errors": errors, "task": str(task)}
|
||||
|
||||
task.auto_fill()
|
||||
log.info(f"\n{'='*60}")
|
||||
log.info(f" 📋 任务: {task.summary()}")
|
||||
log.info(f"{'='*60}")
|
||||
|
||||
if not task.account:
|
||||
return {"status": "error", "errors": [f"没有可用的{task.platform}账号"], "task": str(task)}
|
||||
|
||||
# 执行
|
||||
from mc.engine import BatchEngine, BatchReport
|
||||
engine = BatchEngine(
|
||||
accounts=[task.account],
|
||||
blueprints=[task.blueprint],
|
||||
rounds=task.rounds,
|
||||
mix=False,
|
||||
)
|
||||
engine.task_params = task.to_task_params()
|
||||
|
||||
report = await engine.run()
|
||||
result = report.to_dict()
|
||||
result["task"] = task.summary()
|
||||
result["task_params"] = task.to_task_params()
|
||||
return result
|
||||
|
||||
|
||||
# ── CLI 入口(供 mc task 命令调用)──
|
||||
|
||||
def parse_task_args(args) -> Task:
|
||||
"""从 argparse 解析 Task"""
|
||||
return Task(
|
||||
type=args.task_type or "",
|
||||
url=getattr(args, "url", ""),
|
||||
keyword=getattr(args, "keyword", ""),
|
||||
direction=getattr(args, "direction", ""),
|
||||
comment_text=getattr(args, "comment", ""),
|
||||
account=getattr(args, "account", ""),
|
||||
rounds=getattr(args, "rounds", 1),
|
||||
)
|
||||
@@ -0,0 +1,102 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
migrate_accounts_to_registry.py — 从本地 accounts.yaml 迁移到 registry + override
|
||||
执行: python3 migrate_accounts_to_registry.py
|
||||
|
||||
作用:
|
||||
1. 读取本机 agent-local/tools/matrix/config/accounts.yaml
|
||||
2. 写入 agent-sync/05_tools/07_matrix/accounts_registry.yaml (含脱敏)
|
||||
3. 写入 agent-local/tools/matrix/config/accounts.override.yaml (含手机号/代理)
|
||||
4. 保留原 accounts.yaml 作为 old/backup
|
||||
"""
|
||||
import yaml, shutil, os
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
|
||||
HOME = Path.home()
|
||||
from matrix_mgmt import AGENT_SYNC, AGENT_LOCAL
|
||||
|
||||
HOSTNAME_FILE = AGENT_LOCAL / "identity" / "cached_hostname"
|
||||
HOSTNAME = HOSTNAME_FILE.read_text().strip() if HOSTNAME_FILE.exists() else os.uname().nodename
|
||||
|
||||
SRC = AGENT_LOCAL / "tools" / "matrix" / "config" / "accounts.yaml"
|
||||
REGISTRY = AGENT_SYNC / "05_tools" / "07_matrix" / "accounts_registry.yaml"
|
||||
OVERRIDE = AGENT_LOCAL / "tools" / "matrix" / "config" / "accounts.override.yaml"
|
||||
BACKUP = AGENT_LOCAL / "tools" / "matrix" / "config" / "accounts.yaml.old"
|
||||
|
||||
def mask_phone(phone: str) -> str:
|
||||
p = str(phone).strip()
|
||||
if len(p) == 11:
|
||||
return p[:3] + "****" + p[-4:]
|
||||
return p[:3] + "****" if len(p) > 3 else p
|
||||
|
||||
def migrate():
|
||||
if not SRC.exists():
|
||||
print(f"❌ 源文件不存在: {SRC}")
|
||||
return
|
||||
|
||||
# 读取
|
||||
raw = yaml.safe_load(SRC.read_text()) or {}
|
||||
accounts = raw.get("accounts", [])
|
||||
print(f"📖 读取 {len(accounts)} 个账号配置")
|
||||
|
||||
# 构建 registry & override
|
||||
registry_accounts = []
|
||||
override_accounts = []
|
||||
for acct in accounts:
|
||||
aid = acct.get("id", "")
|
||||
platform = acct.get("platform", "unknown")
|
||||
phone = str(acct.get("phone", ""))
|
||||
identity_dir = acct.get("identity_dir", "")
|
||||
identity_hint = identity_dir.split("/")[-1] if identity_dir else ""
|
||||
proxy = acct.get("proxy", "")
|
||||
enabled = acct.get("enabled", True)
|
||||
win = acct.get("window", [702, 783])
|
||||
pos = acct.get("window_position", [0, 0])
|
||||
notes = acct.get("notes", "")
|
||||
|
||||
# Registry (脱敏)
|
||||
registry_accounts.append({
|
||||
"id": aid,
|
||||
"platform": platform,
|
||||
"phone_mask": mask_phone(phone),
|
||||
"assigned_machine": HOSTNAME,
|
||||
"identity_hint": identity_hint,
|
||||
"window": win,
|
||||
"window_position": pos,
|
||||
"notes": notes,
|
||||
})
|
||||
|
||||
# Override (完整手机号+代理)
|
||||
o = {"id": aid, "phone": phone}
|
||||
if proxy:
|
||||
o["proxy"] = proxy
|
||||
o["enabled"] = enabled
|
||||
override_accounts.append(o)
|
||||
|
||||
# 写 Registry
|
||||
reg_data = {
|
||||
"version": "1.0",
|
||||
"last_updated": datetime.now().strftime("%Y-%m-%d %H:%M"),
|
||||
"accounts": registry_accounts,
|
||||
}
|
||||
REGISTRY.write_text(yaml.dump(reg_data, default_flow_style=False, allow_unicode=True, sort_keys=False))
|
||||
print(f"✅ Registry 写入: {REGISTRY} ({len(registry_accounts)} 账号)")
|
||||
|
||||
# 写 Override
|
||||
ovr_data = {"version": "1.0", "hostname": HOSTNAME, "accounts": override_accounts}
|
||||
OVERRIDE.parent.mkdir(parents=True, exist_ok=True)
|
||||
OVERRIDE.write_text(yaml.dump(ovr_data, default_flow_style=False, allow_unicode=True, sort_keys=False))
|
||||
print(f"✅ Override 写入: {OVERRIDE} ({len(override_accounts)} 账号)")
|
||||
|
||||
# 备份原文件
|
||||
shutil.copy2(SRC, BACKUP)
|
||||
print(f"✅ 原文件备份: {BACKUP}")
|
||||
|
||||
print("\n🎉 迁移完成!")
|
||||
print(f" 请编辑 {REGISTRY} 调整 assigned_machine (如有多机)")
|
||||
print(f" 请编辑 {OVERRIDE} 确认手机号和代理")
|
||||
print(" git add accounts_registry.yaml && git commit && git push")
|
||||
|
||||
if __name__ == "__main__":
|
||||
migrate()
|
||||
@@ -0,0 +1,298 @@
|
||||
"""
|
||||
nurture_blueprint.py — 养号蓝图:浏览+点赞+收藏+评论
|
||||
|
||||
蓝图 = 操作序列,每步 = (op_name, fn, pre_state, post_state, prob)
|
||||
|
||||
操作 fn 是单行 lambda,执行后返回状态码。
|
||||
"""
|
||||
import asyncio, random, subprocess, time, os, sys
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from cdp_connector import CDPConnector
|
||||
|
||||
LOG = '/tmp/nurture.log'
|
||||
def log(m):
|
||||
with open(LOG, 'a') as f: f.write(f'[{time.strftime("%H:%M:%S")}] {m}\n')
|
||||
print(m, flush=True)
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 原子操作层(每行一个,极简)
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
async def activate():
|
||||
for _ in range(3):
|
||||
subprocess.run(['osascript','-e',
|
||||
'tell application "System Events" to set frontmost of every process whose name contains "camoufox" to true'],
|
||||
capture_output=True, timeout=3); time.sleep(0.3)
|
||||
|
||||
async def read_state(p):
|
||||
return await p.evaluate("""() => {
|
||||
var ae=document.activeElement; var ed=document.querySelector('.public-DraftEditor-content');
|
||||
return {
|
||||
url:(location.href||'').slice(0,50), vc:document.querySelectorAll('video').length,
|
||||
hasCL:!!document.querySelector('[data-e2e="comment-list"]'),
|
||||
hasEd:!!ed, aeIsEd:!!(ae&&(ae.isContentEditable||ae.getAttribute('contenteditable')==='true')),
|
||||
edText:(ed?.textContent||'').trim().slice(0,20),
|
||||
hasVerify:!!document.querySelector('input[placeholder*="验证码"]'),
|
||||
aeRect:ae?(function(){var r=ae.getBoundingClientRect();return{x:Math.round(r.left+r.width/2),y:Math.round(r.top+r.height/2)};})():null,
|
||||
likeState:(document.querySelector('[data-e2e="video-player-digg"]')?.getAttribute('data-e2e-state')||''),
|
||||
};
|
||||
}""")
|
||||
|
||||
async def op_goto_home(p):
|
||||
await p.goto('https://www.douyin.com/',timeout=20000); await asyncio.sleep(4)
|
||||
return 'HOME'
|
||||
|
||||
async def op_enter_video(p):
|
||||
c = p.locator('.discover-video-card-item').first
|
||||
await c.click(force=True); await asyncio.sleep(1.5)
|
||||
v = p.locator('video').first
|
||||
if await v.count()>0: await v.click(); await asyncio.sleep(1); await v.click(); await asyncio.sleep(4)
|
||||
s = await read_state(p)
|
||||
return 'PLAYER' if s['vc']>=2 else 'HOME'
|
||||
|
||||
async def op_swipe_down(p):
|
||||
await p.mouse.wheel(0,800); await asyncio.sleep(1)
|
||||
await p.keyboard.press('ArrowDown'); await asyncio.sleep(3)
|
||||
return 'PLAYER'
|
||||
|
||||
async def op_like(p):
|
||||
b = p.locator('[data-e2e="video-player-digg"]').first
|
||||
if await b.count()>0:
|
||||
await b.click(); await asyncio.sleep(1)
|
||||
return 'LIKED'
|
||||
return 'PLAYER'
|
||||
|
||||
async def op_watch(p):
|
||||
t = random.uniform(5,12)
|
||||
await asyncio.sleep(t)
|
||||
return 'PLAYER'
|
||||
|
||||
async def op_comment(p, text='好内容'):
|
||||
# 1. 开评论(前置锚点:检查是否已打开)
|
||||
s = await read_state(p)
|
||||
if s['hasCL']:
|
||||
log(' 评论区已开,跳过')
|
||||
else:
|
||||
v = p.locator('video').first
|
||||
if await v.count()>0:
|
||||
bx = await v.bounding_box()
|
||||
if bx: await p.mouse.click(bx['x']+bx['width']//2,bx['y']+bx['height']//3)
|
||||
await asyncio.sleep(0.5)
|
||||
await p.keyboard.press('x'); await asyncio.sleep(2)
|
||||
s = await read_state(p)
|
||||
if not s['hasCL']:
|
||||
await p.evaluate("""()=>{var b=document.querySelector('[data-e2e="video-comment-count"]')||document.querySelector('[data-e2e="feed-comment-icon"]');if(b)b.click();}""")
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# 2. 激活编辑器(缓慢移动到479,687→单击)
|
||||
for step in range(8):
|
||||
await p.mouse.move(10+(479-10)*(step+1)/8, 10+(687-10)*(step+1)/8); await asyncio.sleep(0.1)
|
||||
await asyncio.sleep(0.5)
|
||||
await p.mouse.click(479,687); await asyncio.sleep(1)
|
||||
s = await read_state(p)
|
||||
if not s['aeIsEd']:
|
||||
await p.mouse.click(479,687); await asyncio.sleep(1) # 双击兜底
|
||||
s = await read_state(p)
|
||||
|
||||
# 3. 粘贴中文
|
||||
if s['aeIsEd']:
|
||||
subprocess.run(['osascript','-e',f'set the clipboard to "{text}"'], timeout=5); time.sleep(0.3)
|
||||
await p.keyboard.press('Meta+v'); await asyncio.sleep(2)
|
||||
s = await read_state(p)
|
||||
if s['edText']:
|
||||
# 4. Enter 发送(2026-05-15 从 Alt+Enter 改为 Enter,适配抖音新版评论发送)
|
||||
await p.keyboard.press("Enter")
|
||||
await asyncio.sleep(3)
|
||||
s = await read_state(p)
|
||||
if s['hasVerify']:
|
||||
log(' 📱 触发验证码!弹窗已保留,请分析')
|
||||
log(' ⏸️ 脚本暂停,浏览器保持打开')
|
||||
return 'VERIFY'
|
||||
# 5. 关评论区
|
||||
if s['hasCL']:
|
||||
await p.keyboard.press('x'); await asyncio.sleep(1)
|
||||
log(' 🅧 评论区已关闭')
|
||||
return 'COMMENTED'
|
||||
return 'PLAYER'
|
||||
|
||||
async def _handle_verify(p):
|
||||
"""验证码弹窗:自动获取+回填+确认"""
|
||||
from matrix_modules.account.sms import ApiSMSHandler
|
||||
handler = ApiSMSHandler()
|
||||
code = await handler.wait("抖音", timeout=120)
|
||||
if not code or len(code) not in (4,5,6):
|
||||
log(f' ⚠️ 未获取到有效验证码: {code}')
|
||||
return
|
||||
log(f' 📝 回填验证码: {code}')
|
||||
await p.evaluate(f"""() => {{
|
||||
var inp = document.querySelector('input[placeholder*="验证码"]');
|
||||
if (!inp) return;
|
||||
inp.value = '{code}';
|
||||
inp.dispatchEvent(new Event('input', {{bubbles: true}}));
|
||||
}}""")
|
||||
await asyncio.sleep(0.5)
|
||||
# 点确认按钮
|
||||
btn = p.locator('button:has-text("确认"), button:has-text("提交"), button:has-text("验证")').first
|
||||
if await btn.count() > 0:
|
||||
await btn.click()
|
||||
log(' ✅ 验证码已提交')
|
||||
await asyncio.sleep(2)
|
||||
|
||||
async def op_close_comments(p):
|
||||
"""关闭评论区(前置锚点:hasCL=True)"""
|
||||
s = await read_state(p)
|
||||
if s['hasCL']:
|
||||
await p.keyboard.press('x'); await asyncio.sleep(1)
|
||||
return 'PLAYER'
|
||||
return 'PLAYER'
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 蓝图定义
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
# 每个步骤: (名称, 操作函数, 可选参数, 执行概率)
|
||||
BLUEPRINT_NURTURE = [
|
||||
("进入视频", op_enter_video, {}, 1.0),
|
||||
("观看", op_watch, {}, 1.0),
|
||||
("点赞", op_like, {}, 0.4),
|
||||
("下滑", op_swipe_down, {}, 1.0),
|
||||
("观看", op_watch, {}, 1.0),
|
||||
("点赞", op_like, {}, 0.3),
|
||||
("下滑", op_swipe_down, {}, 1.0),
|
||||
("观看", op_watch, {}, 1.0),
|
||||
("评论", op_comment, {'text':'好内容'}, 0.5),
|
||||
("下滑", op_swipe_down, {}, 1.0),
|
||||
("观看", op_watch, {}, 1.0),
|
||||
("点赞", op_like, {}, 0.3),
|
||||
("评论", op_comment, {'text':'涨知识了'}, 0.5),
|
||||
("下滑", op_swipe_down, {}, 1.0),
|
||||
("观看", op_watch, {}, 1.0),
|
||||
("点赞", op_like, {}, 0.3),
|
||||
("评论", op_comment, {'text':'好内容,已三连'}, 0.5),
|
||||
]
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 执行引擎
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def make_logger(acct):
|
||||
"""返回绑定账号名的日志函数,每个账号独立日志文件"""
|
||||
log_file = f'/tmp/nurture_{acct}.log'
|
||||
try: open(log_file, 'w').close()
|
||||
except: pass
|
||||
|
||||
def alog(m):
|
||||
t = time.strftime("%H:%M:%S")
|
||||
line = f'[{t}][{acct}] {m}'
|
||||
with open(LOG, 'a') as f: f.write(line + '\n')
|
||||
with open(log_file, 'a') as f: f.write(line + '\n')
|
||||
print(line, flush=True)
|
||||
|
||||
alog.log_file = log_file
|
||||
return alog
|
||||
|
||||
async def log_state(p, alog, tag=''):
|
||||
"""记录当前页面状态(接受 alog 参数)"""
|
||||
s = await read_state(p)
|
||||
alog(f'{tag} 状态: ae={s["aeTag"]} vc={s["vc"]} CL={s["hasCL"]} Ed={s["hasEd"]} text="{s["edText"]}" url={s["url"]}')
|
||||
|
||||
async def run_blueprint(page, blueprint, alog):
|
||||
state = 'HOME'
|
||||
for i, (name, fn, kwargs, prob) in enumerate(blueprint):
|
||||
if random.random() > prob:
|
||||
alog(f' [{i+1}] {name} → 跳过')
|
||||
continue
|
||||
alog(f' [{i+1}] {name}...')
|
||||
try:
|
||||
state = await fn(page, **kwargs)
|
||||
except Exception as e:
|
||||
alog(f' ⚠️ {name}异常: {str(e)[:40]}')
|
||||
await log_state(page, alog, ' 异常时')
|
||||
state = 'PLAYER' if (await read_state(page)).get('vc',0)>=2 else 'HOME'
|
||||
alog(f' → {state}')
|
||||
await asyncio.sleep(random.uniform(0.5,1.5))
|
||||
alog(f'蓝图完成 → {state}')
|
||||
|
||||
async def reset_to_home(page, alog):
|
||||
"""重置到首页"""
|
||||
alog('🔄 重置到首页')
|
||||
await page.goto('https://www.douyin.com/', timeout=20000)
|
||||
await asyncio.sleep(4)
|
||||
alog('🏠 首页就绪')
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# 启动
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
async def main():
|
||||
import sys
|
||||
args = sys.argv[1:]
|
||||
duration_h = 2.0
|
||||
accts = []
|
||||
for a in args:
|
||||
if a.replace('.','').isdigit(): duration_h = float(a)
|
||||
else: accts.append(a)
|
||||
if not accts: accts = ['douyin_01_camo', 'douyin_02_camo', 'douyin_camo01']
|
||||
duration_s = duration_h * 3600
|
||||
log(f'养号 {len(accts)}个账号 × {duration_h}h → {accts}')
|
||||
|
||||
# 窗口位置策略:
|
||||
# 1个: (0,0) 左上
|
||||
# 2个: (0,0) + (720,0) 平铺不重叠
|
||||
# 3个: (0,0) + (500,0) + (750,0)
|
||||
def calc_pos(idx, total):
|
||||
if total == 1: return (0, 0)
|
||||
if total == 2: return (0, 0) if idx == 0 else (720, 0)
|
||||
# total == 3
|
||||
return [(0, 0), (500, 0), (750, 0)][idx]
|
||||
|
||||
async def run_one(acct, idx):
|
||||
alog = make_logger(acct)
|
||||
try:
|
||||
ID_DIR = os.path.expanduser(f'~/workbuddy-agent-os/agent-local/tools/matrix/identities/{acct}')
|
||||
alog(f'启动 (pos={calc_pos(idx, len(accts))})')
|
||||
conn = CDPConnector(identity_dir=ID_DIR, headless=False, window=(702,783), window_position=calc_pos(idx, len(accts)))
|
||||
await conn.connect(); p = conn.page; await conn.init_anti_detection()
|
||||
await op_goto_home(p)
|
||||
alog('🏠 首页就绪')
|
||||
|
||||
t_start = time.time()
|
||||
rounds = 0
|
||||
fails = 0
|
||||
while time.time() - t_start < duration_s:
|
||||
try:
|
||||
rounds += 1
|
||||
alog(f'=== 第{rounds}轮 ===')
|
||||
await run_blueprint(p, BLUEPRINT_NURTURE, alog)
|
||||
s = await read_state(p)
|
||||
|
||||
if s.get('vc',0) < 2 and not s.get('hasCL'):
|
||||
fails += 1
|
||||
alog(f'⚠️ 异常状态 vc={s["vc"]} CL={s["hasCL"]} (连续{fails}次)')
|
||||
if fails >= 3:
|
||||
alog(f'❌ 连续{fails}次异常,触发重置')
|
||||
await reset_to_home(p, alog)
|
||||
fails = 0
|
||||
else:
|
||||
fails = 0
|
||||
|
||||
elapsed = (time.time() - t_start) / 60
|
||||
alog(f'已运行{elapsed:.0f}min / {duration_h*60:.0f}min')
|
||||
except Exception as e:
|
||||
alog(f'⚠️ 轮次异常: {str(e)[:50]}')
|
||||
fails += 1
|
||||
if fails >= 3:
|
||||
alog('❌ 连续3次异常,重置')
|
||||
try: await reset_to_home(p, alog)
|
||||
except: pass
|
||||
fails = 0
|
||||
alog('⏹ 结束')
|
||||
except Exception as e:
|
||||
alog(f'❌ 账号异常退出: {str(e)[:60]}')
|
||||
|
||||
await asyncio.gather(*[run_one(accts[i], i) for i in range(len(accts))])
|
||||
|
||||
if __name__ == '__main__':
|
||||
try: asyncio.run(main())
|
||||
except KeyboardInterrupt: log('👋')
|
||||
except Exception as e: import traceback; traceback.print_exc()
|
||||
@@ -0,0 +1,311 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
日常养号调度器 v1.0
|
||||
|
||||
规则:
|
||||
1. 最多 3 个浏览器并行,每个间隔 15 秒启动
|
||||
2. 步骤顺序每次随机打乱
|
||||
3. 每身份每平台约运行 10 分钟
|
||||
4. 身份完成后休息 30 秒
|
||||
5. 动作池: 浏览/搜索/点赞/收藏/评论/关注
|
||||
|
||||
用法:
|
||||
python nurture_daily.py # 所有身份按规则开始养号
|
||||
python nurture_daily.py --dry-run # 只看计划,不执行
|
||||
python nurture_daily.py --phone 185xxx # 只养指定手机号
|
||||
"""
|
||||
|
||||
import argparse, asyncio, json, random, subprocess, sys, time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
TOOL_DIR = Path(__file__).resolve().parent.parent
|
||||
SCRIPTS_DIR = TOOL_DIR / "scripts"
|
||||
BP_DIR = TOOL_DIR / "blueprints"
|
||||
from matrix_mgmt import AGENT_LOCAL
|
||||
LOCAL_ROOT = AGENT_LOCAL / "tools" / "matrix"
|
||||
CONFIG_DIR = LOCAL_ROOT / "config"
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
|
||||
# ── 配置 ──
|
||||
MAX_BROWSERS = 3 # 最大并发浏览器
|
||||
STAGGER_SECS = 15 # 启动间隔
|
||||
SESSION_MINS = 10 # 每身份每平台预计分钟
|
||||
REST_SECS = 30 # 身份间休息
|
||||
WATCH_MIN = 6 # 浏览最小时长(秒)
|
||||
WATCH_MAX = 25 # 浏览最大时长(秒)
|
||||
GAUSS_JITTER = 0.3 # 高斯抖动系数
|
||||
|
||||
# ── 搜索关键词池(每次随机选) ──
|
||||
KEYWORDS_DOUYIN = [
|
||||
"美食推荐", "旅行日记", "搞笑视频", "电影解说", "宠物日常",
|
||||
"穿搭分享", "美妆教程", "健身打卡", "音乐现场", "游戏解说",
|
||||
"科技数码", "汽车评测", "家居装修", "户外运动", "亲子日常",
|
||||
"职场干货", "学习技巧", "手工制作", "摄影教程", "美食探店",
|
||||
]
|
||||
|
||||
KEYWORDS_XHS = [
|
||||
"穿搭推荐", "美妆教程", "旅行攻略", "美食打卡", "家居好物",
|
||||
"护肤心得", "读书笔记", "运动打卡", "育儿经验", "摄影技巧",
|
||||
]
|
||||
|
||||
|
||||
def gaussian_sleep(base):
|
||||
"""base 秒 ±30% 高斯抖动"""
|
||||
delay = base * (1 + random.uniform(-GAUSS_JITTER, GAUSS_JITTER))
|
||||
return max(2, delay)
|
||||
|
||||
|
||||
def load_accounts():
|
||||
"""从 registry + override 读取本机所有账号(跨机兼容)"""
|
||||
from matrix_mgmt import MatrixManager
|
||||
mgr = MatrixManager()
|
||||
all_accts = mgr.list_accounts()
|
||||
return [a for a in all_accts if a.get("is_local") and a.get("enabled", True)]
|
||||
|
||||
|
||||
def load_blueprints():
|
||||
"""加载所有蓝图"""
|
||||
bps = {}
|
||||
for f in sorted(BP_DIR.glob("*.json")):
|
||||
try:
|
||||
bp = json.loads(f.read_text())
|
||||
name = bp.get("name", bp.get("id", f.stem))
|
||||
platform = bp.get("platform", bp.get("type", "douyin"))
|
||||
steps = bp.get("steps", [])
|
||||
if steps:
|
||||
bps[name] = {"name": name, "platform": platform, "steps": steps}
|
||||
except: pass
|
||||
return bps
|
||||
|
||||
|
||||
def group_by_identity(accounts):
|
||||
"""按手机号分组"""
|
||||
groups = {}
|
||||
for a in accounts:
|
||||
phone = a.get("phone", "")
|
||||
if not phone:
|
||||
continue
|
||||
if phone not in groups:
|
||||
groups[phone] = {
|
||||
"phone": phone,
|
||||
"display_name": a.get("display_name", phone),
|
||||
"accounts": [],
|
||||
}
|
||||
groups[phone]["accounts"].append(a)
|
||||
return list(groups.values())
|
||||
|
||||
|
||||
def shuffle_steps(steps, platform="douyin"):
|
||||
"""随机打乱步骤顺序 + 关键词替换 + 时间抖动"""
|
||||
shuffled = steps.copy()
|
||||
random.shuffle(shuffled)
|
||||
|
||||
# 替换 @keyword 和 @corpus
|
||||
keywords = KEYWORDS_DOUYIN if platform == "douyin" else KEYWORDS_XHS
|
||||
for s in shuffled:
|
||||
args = s.get("args", {})
|
||||
for k, v in args.items():
|
||||
if isinstance(v, str) and "@keyword" in v:
|
||||
args[k] = random.choice(keywords)
|
||||
elif isinstance(v, str) and "@corpus" in v:
|
||||
# 评论语料
|
||||
texts = [
|
||||
"这个太棒了!", "学到了学到了", "收藏了~",
|
||||
"好厉害!", "这是什么神仙", "太实用了吧",
|
||||
"爱了爱了", "求教程!", "这个真的绝",
|
||||
"已收藏慢慢看", "讲得太清楚了", "干货满满",
|
||||
]
|
||||
args[k] = random.choice(texts)
|
||||
|
||||
# 等待时长加抖动
|
||||
if "seconds" in args and args["seconds"] and isinstance(args["seconds"], (int, float)):
|
||||
args["seconds"] = round(gaussian_sleep(args["seconds"]), 1)
|
||||
|
||||
# 控制步数在 15-30 步之间(保证约10分钟)
|
||||
target_steps = random.randint(15, 30)
|
||||
if len(shuffled) > target_steps:
|
||||
shuffled = shuffled[:target_steps]
|
||||
|
||||
return shuffled
|
||||
|
||||
|
||||
def pick_blueprint(blueprints, platform):
|
||||
"""为平台选一个合适的蓝图"""
|
||||
candidates = [b for b in blueprints.values() if b["platform"] == platform]
|
||||
if not candidates:
|
||||
candidates = [b for b in blueprints.values()] # 不限平台
|
||||
if not candidates:
|
||||
return None
|
||||
return random.choice(candidates)
|
||||
|
||||
|
||||
def build_nurture_command(phone, display_name, account, blueprint_name, rounds=1):
|
||||
"""构建养号命令 — 用 mc run(新引擎,跨机兼容)"""
|
||||
mc_path = TOOL_DIR / "mc"
|
||||
return [
|
||||
"bash", str(mc_path), "run",
|
||||
"--accounts", account["id"],
|
||||
"--blueprints", blueprint_name,
|
||||
"--rounds", str(rounds),
|
||||
"--mix",
|
||||
"--interval", "45-90",
|
||||
]
|
||||
|
||||
|
||||
async def run_nurture_session(identity, blueprints, session_id):
|
||||
"""运行一个身份的养号"""
|
||||
phone = identity["phone"]
|
||||
name = identity["display_name"]
|
||||
accounts = identity["accounts"]
|
||||
results = []
|
||||
|
||||
print(f"\n{'='*55}")
|
||||
print(f" 📱 [{name}]({phone}) 开始养号")
|
||||
print(f" {'='*55}")
|
||||
|
||||
for acct in accounts:
|
||||
platform = acct.get("platform", "douyin")
|
||||
bp = pick_blueprint(blueprints, platform)
|
||||
if not bp:
|
||||
print(f" ⚠️ {platform} 无可用蓝图,跳过")
|
||||
continue
|
||||
|
||||
# 选蓝图并随机打乱步骤
|
||||
shuffled = shuffle_steps(bp["steps"], platform)
|
||||
bp_name = bp["name"]
|
||||
|
||||
# 预估时长
|
||||
est_secs = len(shuffled) * 20 + 30 # 每步~20s + 缓冲
|
||||
est_min = round(est_secs / 60, 1)
|
||||
|
||||
print(f" 🎯 {acct['id']} ({platform})")
|
||||
print(f" 蓝图: {bp_name} → 打乱后 {len(shuffled)} 步")
|
||||
print(f" 预计: ~{est_min}分钟")
|
||||
|
||||
results.append({
|
||||
"account": acct["id"],
|
||||
"platform": platform,
|
||||
"blueprint": bp_name,
|
||||
"steps": len(shuffled),
|
||||
"estimated_min": est_min,
|
||||
})
|
||||
|
||||
# 实际执行(通过 yanghao_runner)
|
||||
cmd = build_nurture_command(phone, name, acct, bp_name, rounds=random.randint(1, 3))
|
||||
print(f" 执行: {' '.join(cmd[:6])}...")
|
||||
try:
|
||||
p = subprocess.run(cmd, capture_output=True, text=True, timeout=900)
|
||||
results[-1]["returncode"] = p.returncode
|
||||
results[-1]["stdout_len"] = len(p.stdout)
|
||||
except subprocess.TimeoutExpired:
|
||||
print(f" ⏰ {acct['id']} 超时")
|
||||
results[-1]["returncode"] = -1
|
||||
except Exception as e:
|
||||
print(f" ❌ {e}")
|
||||
results[-1]["error"] = str(e)
|
||||
|
||||
return results
|
||||
|
||||
|
||||
async def main():
|
||||
parser = argparse.ArgumentParser(description="日常养号调度器")
|
||||
parser.add_argument("--dry-run", action="store_true", help="只看计划,不执行")
|
||||
parser.add_argument("--phone", default="", help="只养指定手机号")
|
||||
args = parser.parse_args()
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print(f" 📊 日常养号调度器 v1.0")
|
||||
print(f" {'='*60}")
|
||||
print(f" 规则:")
|
||||
print(f" - 最多 {MAX_BROWSERS} 个浏览器并行")
|
||||
print(f" - 每个间隔 {STAGGER_SECS} 秒启动")
|
||||
print(f" - 步骤顺序随机打乱")
|
||||
print(f" - 每身份约 {SESSION_MINS} 分钟")
|
||||
if args.dry_run:
|
||||
print(f" - 🔍 仅查看计划 (--dry-run)")
|
||||
|
||||
accounts = load_accounts()
|
||||
blueprints = load_blueprints()
|
||||
groups = group_by_identity(accounts)
|
||||
|
||||
if args.phone:
|
||||
groups = [g for g in groups if g["phone"] == args.phone]
|
||||
|
||||
print(f"\n 📋 账号: {len(accounts)} | 蓝图: {len(blueprints)} | 身份: {len(groups)}")
|
||||
|
||||
if not groups:
|
||||
print(" ❌ 没有匹配的身份")
|
||||
return
|
||||
|
||||
# 排期表
|
||||
print(f"\n{'='*60}")
|
||||
print(f" 📅 排期表")
|
||||
print(f" {'='*60}")
|
||||
|
||||
total_est_min = 0
|
||||
for i, g in enumerate(groups):
|
||||
batch = i // MAX_BROWSERS + 1
|
||||
pos = i % MAX_BROWSERS
|
||||
start_delay = pos * STAGGER_SECS + (batch - 1) * (SESSION_MINS * 60 + REST_SECS * MAX_BROWSERS)
|
||||
start_min = round(start_delay / 60, 1)
|
||||
|
||||
plats = ", ".join(a["platform"] for a in g["accounts"])
|
||||
bps = []
|
||||
for a in g["accounts"]:
|
||||
bp = pick_blueprint(blueprints, a["platform"])
|
||||
bps.append(bp["name"] if bp else "无蓝图")
|
||||
|
||||
est = SESSION_MINS
|
||||
total_est_min += est
|
||||
|
||||
print(f" 批次{batch}#{pos+1} | +{start_min}min | {g['display_name']:16s} | {plats:20s} | ~{est}min")
|
||||
|
||||
total_min = round(total_est_min / MAX_BROWSERS + (len(groups) / MAX_BROWSERS) * 0.5, 1)
|
||||
print(f"\n 📊 预估总耗时: ~{total_min} 分钟 ({round(total_min/60,1)} 小时)")
|
||||
print(f" 身份数: {len(groups)} | 批次: {(len(groups)+MAX_BROWSERS-1)//MAX_BROWSERS}")
|
||||
|
||||
if args.dry_run:
|
||||
print(f"\n 🔍 这是计划预览,加 --dry-run 去掉就是实际执行")
|
||||
return
|
||||
|
||||
# 实际执行
|
||||
print(f"\n{'='*60}")
|
||||
print(f" 🚀 开始执行 ({datetime.now().strftime('%H:%M')})")
|
||||
print(f" {'='*60}")
|
||||
|
||||
all_results = []
|
||||
batch_num = 0
|
||||
|
||||
for i in range(0, len(groups), MAX_BROWSERS):
|
||||
batch = groups[i:i + MAX_BROWSERS]
|
||||
batch_num += 1
|
||||
|
||||
print(f"\n--- 批次 {batch_num} ({len(batch)} 个身份) ---")
|
||||
|
||||
# 用 asyncio.gather 错开启动
|
||||
async def launch_with_stagger(idx, g):
|
||||
if idx > 0:
|
||||
await asyncio.sleep(STAGGER_SECS * idx)
|
||||
return await run_nurture_session(g, blueprints, f"b{batch_num}p{idx+1}")
|
||||
|
||||
tasks = [launch_with_stagger(idx, g) for idx, g in enumerate(batch)]
|
||||
batch_results = await asyncio.gather(*tasks)
|
||||
all_results.extend(batch_results)
|
||||
|
||||
# 批次间休息
|
||||
if i + MAX_BROWSERS < len(groups):
|
||||
print(f"\n ⏸️ 批次 {batch_num} 完成,休息 {REST_SECS}s...")
|
||||
await asyncio.sleep(REST_SECS)
|
||||
|
||||
# 汇总
|
||||
print(f"\n{'='*60}")
|
||||
print(f" ✅ 全部完成! ({datetime.now().strftime('%H:%M')})")
|
||||
print(f" {'='*60}")
|
||||
ok = sum(1 for r in all_results for rr in r if rr.get("returncode") == 0)
|
||||
total = sum(len(r) for r in all_results)
|
||||
print(f" 成功: {ok}/{total}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,5 @@
|
||||
"""
|
||||
ops — 多平台操作层
|
||||
|
||||
每个平台独立目录,按操作类型分文件。
|
||||
"""
|
||||
@@ -0,0 +1,420 @@
|
||||
"""
|
||||
ops/_base.py — 平台操作基类 v2.0
|
||||
|
||||
三段式操作模型 v2.0:
|
||||
每个原子操作 = 前置条件检查 + 执行动作 + 后置验证
|
||||
|
||||
1. _capture_state() → PageState 操作前/后页面状态指纹
|
||||
2. _get_pre_conditions(op) 每个操作的前置条件
|
||||
3. _do_execute(op, args) → OpResult 执行动作(子类实现,与原来一样)
|
||||
4. _get_post_conditions(op) 每个操作的后置验证条件
|
||||
5. execute() 统一入口 → 三段式执行,返回完整 OpResult
|
||||
|
||||
改造目标:
|
||||
- 录制时能记录操作前后的状态指纹
|
||||
- Analyzer 通过状态差异推断操作类型 → 不再依赖坐标
|
||||
- Exporter 输出三段式 JSON 蓝图
|
||||
"""
|
||||
import asyncio
|
||||
import time
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass, field, asdict
|
||||
from typing import Optional
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# PageState — 页面状态指纹
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
@dataclass
|
||||
class PageState:
|
||||
"""页面状态指纹 — 操作前后快照(供录制/分析/验证使用)
|
||||
|
||||
url_pattern: 归一化 URL 模式(去掉动态参数,保留路径结构)
|
||||
page_mode: grid / player / player_full / player_modal / search / profile / unknown
|
||||
selectors: {CSS选择器: 是否可见} — 关键元素的可见性状态
|
||||
markers: {标记名: 值} — 自定义状态标记(如 liked=true, like_count=1024)
|
||||
text_snippet: 页面文本片段(用于差异分析)
|
||||
"""
|
||||
url_pattern: str = ""
|
||||
page_mode: str = "unknown"
|
||||
selectors: dict = field(default_factory=dict)
|
||||
markers: dict = field(default_factory=dict)
|
||||
text_snippet: str = ""
|
||||
|
||||
def delta(self, other: 'PageState') -> dict:
|
||||
"""对比两个 PageState,返回差异摘要"""
|
||||
diffs = {}
|
||||
|
||||
if self.page_mode != other.page_mode:
|
||||
diffs["page_mode"] = f"{self.page_mode} → {other.page_mode}"
|
||||
|
||||
# 选择器状态变化
|
||||
sel_diffs = {}
|
||||
all_keys = set(self.selectors) | set(other.selectors)
|
||||
for k in all_keys:
|
||||
v1 = self.selectors.get(k)
|
||||
v2 = other.selectors.get(k)
|
||||
if v1 != v2:
|
||||
sel_diffs[k] = f"{v1} → {v2}"
|
||||
if sel_diffs:
|
||||
diffs["selectors"] = sel_diffs
|
||||
|
||||
# 标记变化
|
||||
mark_diffs = {}
|
||||
all_markers = set(self.markers) | set(other.markers)
|
||||
for k in all_markers:
|
||||
v1 = self.markers.get(k)
|
||||
v2 = other.markers.get(k)
|
||||
if v1 != v2:
|
||||
mark_diffs[k] = f"{v1} → {v2}"
|
||||
if mark_diffs:
|
||||
diffs["markers"] = mark_diffs
|
||||
|
||||
return diffs
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return asdict(self)
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# Condition — 前置/后置条件
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
@dataclass
|
||||
class Condition:
|
||||
"""执行条件(前置条件 / 后置验证)
|
||||
|
||||
条件类型 type:
|
||||
'page_mode' — 页面模式匹配(如 page_mode == 'player')
|
||||
'selector' — DOM 元素可见性(如 '[data-e2e="digg"]' 可见)
|
||||
'marker' — 状态标记匹配(如 liked == False)
|
||||
'marker_diff' — 状态标记变化(如 like_count 增加了)
|
||||
soft: bool = False — 软条件:前置检查失败时不阻断执行(仅记录),用于底层有兜底的操作
|
||||
"""
|
||||
type: str = "page_mode"
|
||||
target: str = "" # 目标字段
|
||||
expected: any = None # 期望值
|
||||
actual: any = None # 运行时填充的实际值
|
||||
passed: bool = False # 是否通过
|
||||
message: str = "" # 检查结果描述
|
||||
soft: bool = False # 软条件(失败不阻断)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"type": self.type,
|
||||
"target": self.target,
|
||||
"expected": self.expected,
|
||||
"actual": self.actual,
|
||||
"passed": self.passed,
|
||||
"message": self.message,
|
||||
"soft": self.soft,
|
||||
}
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# OpResult — 操作结果(v2.0 加入三段式字段)
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
@dataclass
|
||||
class OpResult:
|
||||
"""单个操作的结构化结果(v2.0 新增 before/after 状态和条件检查)
|
||||
|
||||
原有字段不变,新增:
|
||||
before_state: 操作前页面状态指纹
|
||||
after_state: 操作后页面状态指纹
|
||||
pre_conditions: 前置条件检查结果列表
|
||||
post_conditions: 后置验证检查结果列表
|
||||
"""
|
||||
op: str # 操作名 (如 "like", "goto_home")
|
||||
step_id: int = 0 # 蓝图步骤编号
|
||||
success: bool = True
|
||||
detail: str = "" # 简短描述 (如 "👍", "profile_loaded", "no_btn")
|
||||
elapsed: float = 0.0 # 耗时(秒)
|
||||
error: str = "" # 失败原因
|
||||
|
||||
# v2.0 新增字段
|
||||
before_state: Optional[PageState] = None
|
||||
after_state: Optional[PageState] = None
|
||||
pre_conditions: Optional[list] = None
|
||||
post_conditions: Optional[list] = None
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
d = asdict(self)
|
||||
# 将 Optional 字段中的 None 去掉,保持 JSON 简洁
|
||||
if self.pre_conditions is None:
|
||||
d.pop("pre_conditions", None)
|
||||
if self.post_conditions is None:
|
||||
d.pop("post_conditions", None)
|
||||
if self.before_state is None:
|
||||
d.pop("before_state", None)
|
||||
if self.after_state is None:
|
||||
d.pop("after_state", None)
|
||||
return d
|
||||
|
||||
def __str__(self):
|
||||
icon = "✅" if self.success else "❌"
|
||||
extra = f" | {self.error}" if self.error else ""
|
||||
# 如果有条件检查结果,追加概要
|
||||
if self.pre_conditions:
|
||||
failed = [c for c in self.pre_conditions if not c.passed]
|
||||
if failed:
|
||||
extra += f" | pre_fail: {len(failed)}"
|
||||
if self.post_conditions:
|
||||
failed = [c for c in self.post_conditions if not c.passed]
|
||||
if failed:
|
||||
extra += f" | post_fail: {len(failed)}"
|
||||
return f"{icon} [{self.step_id:2d}] {self.op:18s} → {self.detail[:25]} ({self.elapsed:.1f}s){extra}"
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# PlatformOps — 平台操作基类(v2.0 三段式)
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
class PlatformOps(ABC):
|
||||
"""平台操作基类(v2.0 三段式)
|
||||
|
||||
子类需要实现:
|
||||
- supported_ops() → list[str]
|
||||
- _do_execute(op, args, step_id) → OpResult (原有,实现动作)
|
||||
- _get_pre_conditions(op) → list[Condition] (新增,前置条件)
|
||||
- _get_post_conditions(op) → list[Condition] (新增,后置验证)
|
||||
|
||||
基类提供:
|
||||
- execute() → 三段式统一入口(子类不需要覆盖)
|
||||
- _capture_state() → 页面状态指纹采集(子类可重写加强)
|
||||
- _check_conditions() → 条件检查引擎
|
||||
"""
|
||||
|
||||
name: str = "base" # 平台标识 (douyin / xiaohongshu)
|
||||
retry_count: int = 1 # 默认重试次数
|
||||
|
||||
# 关键选择器(子类可覆盖,用于状态采集)
|
||||
STATE_SELECTORS: list = []
|
||||
"""状态采集时检查的关键元素选择器列表"""
|
||||
|
||||
# ── 公开接口 ──────────────────────────────────────────────
|
||||
|
||||
def supported_ops(self) -> list[str]:
|
||||
raise NotImplementedError
|
||||
|
||||
async def execute(self, op: str, args: dict = None,
|
||||
step_id: int = 0) -> OpResult:
|
||||
"""三段式执行入口(v2.0)
|
||||
|
||||
流程:
|
||||
1. 采集操作前状态(before_state)
|
||||
2. 检查前置条件(pre_conditions)
|
||||
3. 执行动作(_do_execute)
|
||||
4. 采集操作后状态(after_state)
|
||||
5. 检查后置条件(post_conditions)
|
||||
6. 返回完整 OpResult
|
||||
"""
|
||||
args = args or {}
|
||||
last_err = ""
|
||||
t0_start = time.time()
|
||||
|
||||
# 1. 采集操作前状态
|
||||
before = await self._capture_state()
|
||||
|
||||
# 2. 检查前置条件
|
||||
pre_conds = self._get_pre_conditions(op)
|
||||
pre_results = await self._check_conditions(pre_conds, before)
|
||||
# soft 条件失败不阻断(底层有兜底的操作放行)
|
||||
pre_all_pass = all(c.passed or c.soft for c in pre_results)
|
||||
pre_detail = f"pre:{sum(1 for c in pre_results if c.passed)}/{len(pre_results)}"
|
||||
|
||||
# 前置条件不满足 → 跳过(不是失败);但全部为软条件失败时也放行
|
||||
if not pre_all_pass:
|
||||
elapsed = round(time.time() - t0_start, 2)
|
||||
result = OpResult(op, step_id, True, f"skipped({pre_detail})",
|
||||
elapsed, before_state=before,
|
||||
pre_conditions=pre_results)
|
||||
return result
|
||||
|
||||
for attempt in range(self.retry_count + 1):
|
||||
t0 = time.time()
|
||||
try:
|
||||
# 3. 执行动作
|
||||
result = await self._do_execute(op, args, step_id)
|
||||
if result is None:
|
||||
result = OpResult(op, step_id, True, "ok", time.time() - t0)
|
||||
result.elapsed = round(time.time() - t0, 2)
|
||||
|
||||
# 4. 采集操作后状态
|
||||
result.after_state = await self._capture_state()
|
||||
|
||||
# 5. 检查后置条件
|
||||
post_conds = self._get_post_conditions(op)
|
||||
result.post_conditions = await self._check_conditions(
|
||||
post_conds, result.after_state, before)
|
||||
post_all_pass = all(c.passed for c in result.post_conditions)
|
||||
post_detail = f"post:{sum(1 for c in result.post_conditions if c.passed)}/{len(result.post_conditions)}"
|
||||
|
||||
# 补充前置条件和 before_state
|
||||
result.pre_conditions = pre_results
|
||||
result.before_state = before
|
||||
|
||||
# 后置条件不满足且还有重试次数 → 重试
|
||||
if not post_all_pass and attempt < self.retry_count:
|
||||
result.success = False
|
||||
result.detail = f"retry({post_detail})"
|
||||
result.error = f"post_condition_failed: {post_detail}"
|
||||
await asyncio.sleep(1)
|
||||
continue
|
||||
|
||||
# 最终判定:后置条件通过或已用完重试
|
||||
result.success = post_all_pass or (attempt >= self.retry_count and result.success)
|
||||
result.detail = post_detail if post_all_pass else result.detail
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
last_err = f"{type(e).__name__}: {e}"
|
||||
if attempt < self.retry_count:
|
||||
await asyncio.sleep(1)
|
||||
|
||||
elapsed = round(time.time() - t0_start, 2)
|
||||
return OpResult(op, step_id, False, "retry_exhausted", elapsed, last_err,
|
||||
before_state=before, pre_conditions=pre_results)
|
||||
|
||||
# ── 状态采集(基类实现,子类可加强)──
|
||||
|
||||
async def _capture_state(self) -> PageState:
|
||||
"""采集当前页面状态指纹
|
||||
|
||||
子类可以重写此方法以加入平台特定的状态标记。
|
||||
基类实现采集:
|
||||
- URL 模式
|
||||
- 页面模式(通过 _detect_page_state)
|
||||
- 关键选择器的可见性
|
||||
- 页面文本片段
|
||||
"""
|
||||
state = PageState()
|
||||
try:
|
||||
page = getattr(self, 'page', None)
|
||||
if page is None:
|
||||
return state
|
||||
|
||||
# URL 模式
|
||||
url = page.url
|
||||
state.url_pattern = self._normalize_url(url)
|
||||
|
||||
# 页面模式(子类通常有 _detect_page_state 方法)
|
||||
detect = getattr(self, '_detect_page_state', None)
|
||||
if detect:
|
||||
state.page_mode = await detect()
|
||||
elif hasattr(self, '_detect_page_mode'):
|
||||
state.page_mode = await self._detect_page_mode()
|
||||
|
||||
# 关键选择器可见性
|
||||
for sel in self.STATE_SELECTORS:
|
||||
try:
|
||||
el = page.locator(sel)
|
||||
state.selectors[sel] = await el.count() > 0 and await el.first.is_visible()
|
||||
except Exception:
|
||||
state.selectors[sel] = False
|
||||
|
||||
# 页面文本片段
|
||||
try:
|
||||
text = await page.evaluate("(document.body.innerText || '').trim().slice(0, 2000)")
|
||||
state.text_snippet = text
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return state
|
||||
|
||||
def _normalize_url(self, url: str) -> str:
|
||||
"""归一化 URL,去掉动态参数,保留路径结构"""
|
||||
import re
|
||||
# 去掉查询参数
|
||||
url = re.sub(r'\?.*$', '', url)
|
||||
# 去掉 /video/ 后面的 ID
|
||||
url = re.sub(r'/video/[A-Za-z0-9]+', '/video/{id}', url)
|
||||
# 去掉 /user/ 后面的 ID
|
||||
url = re.sub(r'/user/[A-Za-z0-9]+', '/user/{id}', url)
|
||||
return url
|
||||
|
||||
# ── 条件检查引擎 ──────────────────────────────────────────
|
||||
|
||||
def _get_pre_conditions(self, op: str) -> list[Condition]:
|
||||
"""每个操作的前置条件——子类覆盖返回
|
||||
|
||||
默认返回空列表(不检查前置条件,兼容旧版)
|
||||
"""
|
||||
return []
|
||||
|
||||
def _get_post_conditions(self, op: str) -> list[Condition]:
|
||||
"""每个操作的后置验证条件——子类覆盖返回
|
||||
|
||||
默认返回空列表(不检查后置条件,兼容旧版)
|
||||
"""
|
||||
return []
|
||||
|
||||
async def _check_conditions(self, conditions: list[Condition],
|
||||
state: PageState,
|
||||
before_state: PageState = None) -> list[Condition]:
|
||||
"""检查一组条件,填充 actual/passed/message 字段"""
|
||||
for cond in conditions:
|
||||
try:
|
||||
if cond.type == 'page_mode':
|
||||
cond.actual = state.page_mode
|
||||
cond.passed = state.page_mode == cond.expected or (
|
||||
cond.expected == 'player' and state.page_mode in ('player', 'player_full', 'player_modal')
|
||||
)
|
||||
cond.message = f"page_mode={state.page_mode}" if cond.passed \
|
||||
else f"期望={cond.expected}, 实际={state.page_mode}"
|
||||
|
||||
elif cond.type == 'selector':
|
||||
visible = state.selectors.get(cond.target, False)
|
||||
cond.actual = visible
|
||||
cond.passed = (visible == cond.expected)
|
||||
cond.message = f"{cond.target} visible={visible}" if cond.passed \
|
||||
else f"期望可见={cond.expected}, 实际={visible}"
|
||||
|
||||
elif cond.type == 'marker':
|
||||
val = state.markers.get(cond.target)
|
||||
cond.actual = val
|
||||
cond.passed = (val == cond.expected)
|
||||
cond.message = f"{cond.target}={val}" if cond.passed \
|
||||
else f"期望={cond.expected}, 实际={val}"
|
||||
|
||||
elif cond.type == 'marker_diff' and before_state:
|
||||
before_val = before_state.markers.get(cond.target)
|
||||
after_val = state.markers.get(cond.target)
|
||||
cond.actual = (before_val, after_val)
|
||||
# marker_diff: expected 可以是 "increased"/"decreased"/"changed"
|
||||
if cond.expected == "increased":
|
||||
cond.passed = (isinstance(before_val, (int, float))
|
||||
and isinstance(after_val, (int, float))
|
||||
and after_val > before_val)
|
||||
elif cond.expected == "decreased":
|
||||
cond.passed = (isinstance(before_val, (int, float))
|
||||
and isinstance(after_val, (int, float))
|
||||
and after_val < before_val)
|
||||
else:
|
||||
cond.passed = (before_val != after_val)
|
||||
cond.message = f"{cond.target}: {before_val} → {after_val}"
|
||||
|
||||
else:
|
||||
cond.passed = True
|
||||
cond.message = f"unknown_condition({cond.type})"
|
||||
|
||||
except Exception as e:
|
||||
cond.passed = False
|
||||
cond.message = f"检查异常: {e}"
|
||||
|
||||
return conditions
|
||||
|
||||
# ── 子类实现 ──────────────────────────────────────────────
|
||||
|
||||
@abstractmethod
|
||||
async def _do_execute(self, op: str, args: dict, step_id: int) -> Optional[OpResult]:
|
||||
"""子类实现具体的操作分发。返回 OpResult 或 None(None=成功,无额外信息)
|
||||
|
||||
注意:execute() 方法在 _do_execute 之前/之后自动采集状态,
|
||||
所以 _do_execute 只需要做「动作」部分,不需要管条件检查。
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,655 @@
|
||||
"""
|
||||
ops/xhs_ops.py — 小红书平台操作 v1.0
|
||||
从 mc/engine.py (v4.3) 提取,实现 PlatformOps 接口。
|
||||
|
||||
操作清单 (16个):
|
||||
导航: xhs_goto_home, xhs_browse
|
||||
互动: xhs_like, xhs_collect, xhs_comment, xhs_follow
|
||||
内容: xhs_click_note, xhs_scroll_feed, xhs_search
|
||||
主页: xhs_goto_profile, xhs_read_nickname, xhs_read_user_id,
|
||||
xhs_read_following, xhs_read_fans, xhs_read_likes, xhs_read_bio
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from ops._base import PlatformOps, OpResult, Condition
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
HOME = Path.home()
|
||||
PROFILES_JSON = HOME / "workbuddy-agent-os" / "agent-local" / "tools" / "matrix" / "data" / "profiles.json"
|
||||
XHS_HOME_URL = "https://www.xiaohongshu.com/explore"
|
||||
|
||||
|
||||
class XhsOps(PlatformOps):
|
||||
"""小红书平台操作库"""
|
||||
|
||||
name = "xiaohongshu"
|
||||
|
||||
def __init__(self, page):
|
||||
self.page = page
|
||||
self._profile_cache: dict = {} # goto_profile 后缓存
|
||||
self._account_id: str = "" # 由 engine 设置,写 profiles.json 用
|
||||
|
||||
def supported_ops(self) -> list[str]:
|
||||
return [
|
||||
"xhs_goto_home", "xhs_browse", "xhs_scroll_feed", "xhs_click_note",
|
||||
"xhs_click_next_image", "xhs_click_prev_image",
|
||||
"xhs_like", "xhs_collect", "xhs_comment", "xhs_follow", "xhs_search",
|
||||
"xhs_post_comment",
|
||||
"xhs_goto_profile",
|
||||
"xhs_read_nickname", "xhs_read_user_id", "xhs_read_following",
|
||||
"xhs_read_fans", "xhs_read_likes", "xhs_read_bio",
|
||||
"wait_watch", "go_back", "goto_url",
|
||||
]
|
||||
|
||||
def set_account_id(self, account_id: str):
|
||||
"""设置当前账号ID,供写 profiles.json"""
|
||||
self._account_id = account_id
|
||||
|
||||
# ── 三段式操作模型 v2.0 ──────────────────────────────────
|
||||
|
||||
STATE_SELECTORS = [
|
||||
'section.note-item', # feed 中的笔记
|
||||
'span.like-lottie', # 点赞按钮
|
||||
'[class*="collect"]', # 收藏
|
||||
'[class*="follow"]', # 关注
|
||||
'input[placeholder*="搜索"]', # 搜索框
|
||||
'textarea.textarea', # 小红书搜索输入框
|
||||
'p#content-textarea', # 评论输入框(有ID,稳定)
|
||||
'button.btn.submit', # 发评论按钮
|
||||
'img.preview-interactive', # 图片翻页
|
||||
'img.avatar-item', # 头像
|
||||
'a.cover.mask', # 笔记封面链接
|
||||
'div.continue', # 登录继续按钮
|
||||
'input#verificationCodeInput', # 验证码输入框(有ID)
|
||||
'span.login-text', # 登录按钮
|
||||
'div.foot-btn', # 同意并登录
|
||||
'.reds-count', # 登录态指示器
|
||||
'.login-container', # 登录面板
|
||||
'a.bottom-channel', # 底栏导航
|
||||
]
|
||||
|
||||
def _get_pre_conditions(self, op: str) -> list[Condition]:
|
||||
conds = {
|
||||
"xhs_like": [
|
||||
Condition("page_mode", "page_mode", "player", message="需要在笔记详情页"),
|
||||
],
|
||||
"xhs_collect": [
|
||||
Condition("page_mode", "page_mode", "player", message="需要在笔记详情页"),
|
||||
],
|
||||
"xhs_comment": [
|
||||
Condition("page_mode", "page_mode", "player", message="需要在笔记详情页"),
|
||||
],
|
||||
"xhs_follow": [
|
||||
Condition("page_mode", "page_mode", "profile", message="需要在用户主页"),
|
||||
],
|
||||
"xhs_scroll_feed": [
|
||||
Condition("page_mode", "page_mode", "grid", message="需要在发现页"),
|
||||
],
|
||||
"xhs_click_note": [
|
||||
Condition("page_mode", "page_mode", "grid", message="需要在发现页"),
|
||||
],
|
||||
"xhs_search": [
|
||||
Condition("selector", 'input[placeholder*="搜索"]', True,
|
||||
message="需要搜索框可见"),
|
||||
],
|
||||
"xhs_click_next_image": [
|
||||
Condition("page_mode", "page_mode", "player", message="需要在笔记详情页"),
|
||||
],
|
||||
"xhs_click_prev_image": [
|
||||
Condition("page_mode", "page_mode", "player", message="需要在笔记详情页"),
|
||||
],
|
||||
"xhs_post_comment": [
|
||||
Condition("page_mode", "page_mode", "player", message="需要在笔记详情页"),
|
||||
],
|
||||
"xhs_goto_home": [],
|
||||
"xhs_browse": [],
|
||||
"xhs_goto_profile": [],
|
||||
"xhs_read_nickname": [],
|
||||
"xhs_read_user_id": [],
|
||||
"xhs_read_following": [],
|
||||
"xhs_read_fans": [],
|
||||
"xhs_read_likes": [],
|
||||
"xhs_read_bio": [],
|
||||
"wait_watch": [],
|
||||
"go_back": [],
|
||||
}
|
||||
return conds.get(op, [])
|
||||
|
||||
def _get_post_conditions(self, op: str) -> list[Condition]:
|
||||
conds = {
|
||||
"xhs_goto_home": [
|
||||
Condition("page_mode", "page_mode", "grid", message="应回到发现页"),
|
||||
],
|
||||
"xhs_like": [
|
||||
Condition("selector", '[class*="like"]', True,
|
||||
message="点赞按钮应仍在页面"),
|
||||
],
|
||||
}
|
||||
return conds.get(op, [])
|
||||
|
||||
# ── PlatformOps 入口 ────────────────────────────────────
|
||||
|
||||
async def _do_execute(self, op: str, args: dict, step_id: int) -> Optional[OpResult]:
|
||||
method = getattr(self, op, None)
|
||||
if method is None:
|
||||
return OpResult(op, step_id, False, "unsupported", 0, f"未知操作: {op}")
|
||||
return await method(args, step_id)
|
||||
|
||||
# ═══════════════════════════════════════════════════════
|
||||
# 导航类
|
||||
# ═══════════════════════════════════════════════════════
|
||||
|
||||
async def xhs_goto_home(self, args: dict, step_id: int) -> OpResult:
|
||||
await self.page.goto(XHS_HOME_URL, timeout=20000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(4)
|
||||
return OpResult("xhs_goto_home", step_id, True, "home_loaded")
|
||||
|
||||
async def xhs_browse(self, args: dict, step_id: int) -> OpResult:
|
||||
if "explore" not in self.page.url:
|
||||
await self.page.goto(XHS_HOME_URL, timeout=15000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(3)
|
||||
return OpResult("xhs_browse", step_id, True, "browsing")
|
||||
|
||||
async def xhs_scroll_feed(self, args: dict, step_id: int) -> OpResult:
|
||||
await self.page.evaluate("() => window.scrollBy(0, 800)")
|
||||
await asyncio.sleep(2)
|
||||
return OpResult("xhs_scroll_feed", step_id, True, "scrolled")
|
||||
|
||||
async def xhs_click_note(self, args: dict, step_id: int) -> OpResult:
|
||||
# 策略1: a.cover 点击(录制确认最稳定)
|
||||
try:
|
||||
cover = self.page.locator('a.cover.mask, a.cover')
|
||||
if await cover.count() > 0:
|
||||
await cover.first.click()
|
||||
await asyncio.sleep(4)
|
||||
return OpResult("xhs_click_note", step_id, True, "note_opened")
|
||||
except:
|
||||
pass
|
||||
# 策略2: 通用选择器(原有)
|
||||
note = self.page.locator('section.note-item, a[href*="/explore/"], [class*="note-item"]').first
|
||||
if await note.count() > 0:
|
||||
await note.click()
|
||||
await asyncio.sleep(4)
|
||||
return OpResult("xhs_click_note", step_id, True, "note_opened")
|
||||
# 回到首页重试
|
||||
await self.page.goto("https://www.xiaohongshu.com/explore", timeout=15000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(3)
|
||||
note2 = self.page.locator('section.note-item, a[href*="/explore/"], [class*="note-item"]').first
|
||||
if await note2.count() > 0:
|
||||
await note2.click()
|
||||
await asyncio.sleep(4)
|
||||
return OpResult("xhs_click_note", step_id, True, "note_opened_retry")
|
||||
return OpResult("xhs_click_note", step_id, False, "no_note")
|
||||
|
||||
async def xhs_click_next_image(self, args: dict, step_id: int) -> OpResult:
|
||||
"""图片笔记向右翻页 — 点击右侧SVG区域(两次录制确认 (677,268))"""
|
||||
# 策略1: 坐标点击(录制确认右侧 ~(677,268))
|
||||
try:
|
||||
await self.page.mouse.click(677, 268)
|
||||
await asyncio.sleep(1.5)
|
||||
return OpResult("xhs_click_next_image", step_id, True, "next")
|
||||
except:
|
||||
pass
|
||||
# 策略2: img.preview-interactive 点击(在预览模式中翻页)
|
||||
try:
|
||||
pi = self.page.locator('img.preview-interactive')
|
||||
if await pi.count() > 0:
|
||||
await pi.first.click()
|
||||
await asyncio.sleep(1.5)
|
||||
return OpResult("xhs_click_next_image", step_id, True, "next")
|
||||
except:
|
||||
pass
|
||||
return OpResult("xhs_click_next_image", step_id, False, "no_next_btn")
|
||||
|
||||
async def xhs_click_prev_image(self, args: dict, step_id: int) -> OpResult:
|
||||
"""图片笔记向左翻页 — 点击左侧SVG区域(录制确认 (28,274))"""
|
||||
# 策略1: 坐标点击(录制确认左侧 ~(28,274))
|
||||
try:
|
||||
await self.page.mouse.click(28, 274)
|
||||
await asyncio.sleep(1.5)
|
||||
return OpResult("xhs_click_prev_image", step_id, True, "prev")
|
||||
except:
|
||||
pass
|
||||
# 策略2: 备用坐标(略右移)
|
||||
try:
|
||||
await self.page.mouse.click(70, 274)
|
||||
await asyncio.sleep(1.5)
|
||||
return OpResult("xhs_click_prev_image", step_id, True, "prev")
|
||||
except:
|
||||
pass
|
||||
return OpResult("xhs_click_prev_image", step_id, False, "no_prev_btn")
|
||||
|
||||
# ═══════════════════════════════════════════════════════
|
||||
# 互动类
|
||||
# ═══════════════════════════════════════════════════════
|
||||
|
||||
async def xhs_like(self, args: dict, step_id: int) -> OpResult:
|
||||
"""点赞 — 点 SPAN.like-lottie(force=True 绕过可见性检查)"""
|
||||
try:
|
||||
btn = self.page.locator('span.like-lottie')
|
||||
if await btn.count() > 0:
|
||||
await btn.first.click(force=True)
|
||||
await asyncio.sleep(1)
|
||||
return OpResult("xhs_like", step_id, True, "👍")
|
||||
except:
|
||||
pass
|
||||
# 快速兜底
|
||||
r = await self.page.evaluate("""() => {
|
||||
const all = document.querySelectorAll('span, button, div');
|
||||
for (const el of all) {
|
||||
const c = (el.className || '') + (el.textContent || '');
|
||||
if ((c.includes('like') || c.includes('赞')) && el.offsetParent) {
|
||||
if (c.includes('collect') || c.includes('save')) continue;
|
||||
el.click(); return '👍';
|
||||
}
|
||||
}
|
||||
return '-';
|
||||
}""")
|
||||
await asyncio.sleep(1)
|
||||
return OpResult("xhs_like", step_id, r == "👍", r)
|
||||
|
||||
async def xhs_collect(self, args: dict, step_id: int) -> OpResult:
|
||||
"""收藏 — 底栏第二位(点赞在左,收藏在点赞+45px处)"""
|
||||
# 策略1: 找点赞按钮位置,点击右侧45px(两次录制确认)
|
||||
try:
|
||||
like = self.page.locator('span.like-lottie')
|
||||
if await like.count() > 0:
|
||||
box = await like.first.bounding_box()
|
||||
if box:
|
||||
await self.page.mouse.click(box['x'] + 45, box['y'])
|
||||
await asyncio.sleep(1)
|
||||
return OpResult("xhs_collect", step_id, True, "⭐")
|
||||
except:
|
||||
pass
|
||||
# 策略2: SVG 兜底(录制显示收藏是 svg 标签)
|
||||
try:
|
||||
likes = self.page.locator('span.like-lottie')
|
||||
if await likes.count() > 0:
|
||||
svgs = self.page.locator('span.like-lottie ~ svg, span.like-lottie + svg')
|
||||
if await svgs.count() > 0:
|
||||
await svgs.first.click(force=True)
|
||||
await asyncio.sleep(1)
|
||||
return OpResult("xhs_collect", step_id, True, "⭐svg")
|
||||
except:
|
||||
pass
|
||||
# 策略3: JS查找收藏class
|
||||
r = await self.page.evaluate("""() => {
|
||||
const btns = document.querySelectorAll('[class*="collect"],[class*="save"]');
|
||||
for (const b of btns) {
|
||||
if (b.offsetParent !== null) { b.click(); return '⭐'; }
|
||||
}
|
||||
return '-';
|
||||
}""")
|
||||
await asyncio.sleep(1)
|
||||
return OpResult("xhs_collect", step_id, r == "⭐", r)
|
||||
|
||||
async def xhs_comment(self, args: dict, step_id: int) -> OpResult:
|
||||
await self.page.keyboard.press("x")
|
||||
await asyncio.sleep(2)
|
||||
return OpResult("xhs_comment", step_id, True, "comment_opened")
|
||||
|
||||
async def xhs_follow(self, args: dict, step_id: int) -> OpResult:
|
||||
"""关注 — 点 SPAN.reds-button-new-text(基于录制)"""
|
||||
try:
|
||||
btn = self.page.locator('span.reds-button-new-text')
|
||||
if await btn.count() > 0:
|
||||
await btn.first.click(force=True)
|
||||
await asyncio.sleep(2)
|
||||
return OpResult("xhs_follow", step_id, True, "✅")
|
||||
except:
|
||||
pass
|
||||
# 兜底
|
||||
r = await self.page.evaluate("""() => {
|
||||
const btns = document.querySelectorAll('button, span');
|
||||
for (const b of btns) {
|
||||
const t = (b.textContent || '').trim();
|
||||
if (t.includes('关注') && !t.includes('已关注') && b.offsetParent) { b.click(); return '✅'; }
|
||||
}
|
||||
return '-';
|
||||
}""")
|
||||
await asyncio.sleep(2)
|
||||
return OpResult("xhs_follow", step_id, r == "✅", r)
|
||||
|
||||
async def xhs_post_comment(self, args: dict, step_id: int) -> OpResult:
|
||||
"""小红书发评论 — 优先用ID选择器 p#content-textarea,再 fallback"""
|
||||
text = args.get("text", "")
|
||||
if not text or text == "@corpus":
|
||||
try:
|
||||
from mc.corpus import CorpusManager
|
||||
cm = CorpusManager()
|
||||
picks = cm.get_comments(platform="xiaohongshu", count=5)
|
||||
if picks:
|
||||
text = random.choice(picks)
|
||||
else:
|
||||
text = "好实用呀"
|
||||
except Exception:
|
||||
text = "好实用呀"
|
||||
try:
|
||||
# Step 1: 滑到底部(评论区在底部)
|
||||
await self.page.evaluate("() => window.scrollTo(0, document.body.scrollHeight)")
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# Step 2: 优先用 ID 选择器 p#content-textarea(录制确认稳定)
|
||||
filled = False
|
||||
try:
|
||||
inp = self.page.locator('p#content-textarea')
|
||||
if await inp.count() > 0 and await inp.first.is_visible():
|
||||
await inp.first.click()
|
||||
await asyncio.sleep(0.5)
|
||||
proc = await asyncio.create_subprocess_exec("pbcopy", stdin=asyncio.subprocess.PIPE)
|
||||
await proc.communicate(input=text.encode())
|
||||
await asyncio.sleep(0.5)
|
||||
await self.page.keyboard.press("Meta+V")
|
||||
await asyncio.sleep(1.5)
|
||||
filled = True
|
||||
except:
|
||||
pass
|
||||
|
||||
# Fallback: JS 填字(原有的)
|
||||
if not filled:
|
||||
filled = await self.page.evaluate(f"""() => {{
|
||||
const all = document.querySelectorAll(
|
||||
'textarea, input[type="text"], [contenteditable="true"], ' +
|
||||
'[class*="input"], [class*="editor"], [class*="textarea"]'
|
||||
);
|
||||
for (const el of all) {{
|
||||
if (el.offsetParent !== null) {{
|
||||
el.focus();
|
||||
el.click();
|
||||
const sel = window.getSelection();
|
||||
const range = document.createRange();
|
||||
range.selectNodeContents(el);
|
||||
sel.removeAllRanges();
|
||||
sel.addRange(range);
|
||||
document.execCommand('insertText', false, '{text}');
|
||||
el.dispatchEvent(new Event('input', {{ bubbles: true }}));
|
||||
return true;
|
||||
}}
|
||||
}}
|
||||
return false;
|
||||
}}""")
|
||||
await asyncio.sleep(1)
|
||||
|
||||
# Step 3: 如果JS填字失败,pbcopy 兜底
|
||||
if not filled:
|
||||
proc = await asyncio.create_subprocess_exec("pbcopy", stdin=asyncio.subprocess.PIPE)
|
||||
await proc.communicate(input=text.encode())
|
||||
await asyncio.sleep(0.5)
|
||||
await self.page.keyboard.press("Meta+V")
|
||||
await asyncio.sleep(1.5)
|
||||
|
||||
# Step 4: 找发送按钮(找包含"发送"文本/class含submit的按钮)
|
||||
sent = await self.page.evaluate("""() => {
|
||||
const btns = document.querySelectorAll('button, [class*="submit"], [class*="send"]');
|
||||
for (const b of btns) {
|
||||
const t = (b.textContent || '').trim();
|
||||
if ((t.includes('发送') || t.includes('发布') || b.className.includes('submit'))
|
||||
&& b.offsetParent) {
|
||||
b.click(); return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}""")
|
||||
if not sent:
|
||||
# 没找到发送按钮 → Enter发送
|
||||
await self.page.keyboard.press("Enter")
|
||||
|
||||
await asyncio.sleep(2)
|
||||
return OpResult("xhs_post_comment", step_id, True, f"👍评论({text[:10]})")
|
||||
except Exception as e:
|
||||
return OpResult("xhs_post_comment", step_id, False, "评论失败", error=str(e))
|
||||
|
||||
async def xhs_search(self, args: dict, step_id: int) -> OpResult:
|
||||
kw = args.get("keyword", "热门推荐")
|
||||
if kw == "@random":
|
||||
import random as _r
|
||||
kw = _r.choice(["穿搭推荐","美食日常","旅行攻略","化妆教程","家居好物",
|
||||
"宠物日常","电影推荐","读书分享","健身打卡","摄影技巧"])
|
||||
try:
|
||||
# 小红书搜索栏是 TEXTAREA.textarea(非 input)
|
||||
search_box = self.page.locator('textarea.textarea')
|
||||
if await search_box.count() > 0:
|
||||
await search_box.first.click()
|
||||
await asyncio.sleep(0.5)
|
||||
await search_box.first.fill(kw)
|
||||
await asyncio.sleep(1)
|
||||
await self.page.keyboard.press("Enter")
|
||||
await asyncio.sleep(3)
|
||||
return OpResult("xhs_search", step_id, True, f"searched({kw[:10]})")
|
||||
except:
|
||||
pass
|
||||
# 兜底
|
||||
try:
|
||||
await self.page.evaluate(
|
||||
"(k) => { const i = document.querySelector('input,textarea'); if(i) { i.value=k; i.dispatchEvent(new Event('input')); } }",
|
||||
kw
|
||||
)
|
||||
await asyncio.sleep(1)
|
||||
await self.page.keyboard.press("Enter")
|
||||
await asyncio.sleep(3)
|
||||
return OpResult("xhs_search", step_id, True, f"searched({kw[:10]})")
|
||||
except Exception as e:
|
||||
return OpResult("xhs_search", step_id, False, "search_failed", error=str(e))
|
||||
|
||||
# ═══════════════════════════════════════════════════════
|
||||
# 主页信息采集
|
||||
# ═══════════════════════════════════════════════════════
|
||||
|
||||
async def xhs_goto_profile(self, args: dict, step_id: int) -> OpResult:
|
||||
"""导航到个人主页并提取全部字段到 profile_cache"""
|
||||
# 1. 从页面找 profile URL
|
||||
profile_url = await self.page.evaluate("""() => {
|
||||
const links = document.querySelectorAll('a');
|
||||
for (const a of links) {
|
||||
const href = a.href || '';
|
||||
if (href.includes('/user/profile/')) return href;
|
||||
}
|
||||
return '';
|
||||
}""")
|
||||
target = profile_url or "https://www.xiaohongshu.com/user/profile"
|
||||
await self.page.goto(target, timeout=20000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(5)
|
||||
|
||||
# 2. 统一提取全部字段
|
||||
profile = await self.page.evaluate("""() => {
|
||||
const text = (document.body.innerText || '').trim();
|
||||
const title = (document.title || '').replace(' - 小红书', '').trim();
|
||||
|
||||
const uidM = text.match(/小红书号[::]\\s*(\\d+)/);
|
||||
const folM = text.match(/(\\d+(?:\\.\\d+)?[万wW]?)\\s*关注/);
|
||||
const fanM = text.match(/(\\d+(?:\\.\\d+)?[万wW]?)\\s*粉丝/);
|
||||
const likM = text.match(/(\\d+(?:\\.\\d+)?[万wW]?)\\s*获赞与收藏/);
|
||||
const posM = text.match(/笔记[・·](\\d+(?:\\.\\d+)?[万wW]?)/);
|
||||
|
||||
let bio = '(无)';
|
||||
if (text.includes('还没有简介') || text.includes('暂无简介')) {
|
||||
bio = '(无)';
|
||||
} else {
|
||||
const bioM = text.match(/IP属地[::].+?\\n(.+?)(?=\\d+关注|\\d+粉丝)/);
|
||||
if (bioM) bio = bioM[1].trim().slice(0, 50) || '(无)';
|
||||
}
|
||||
|
||||
return { nickname: title, user_id: uidM ? uidM[1] : '?',
|
||||
following: folM ? folM[1] : '?', fans: fanM ? fanM[1] : '?',
|
||||
likes: likM ? likM[1] : '?', posts: posM ? posM[1] : '?',
|
||||
bio: bio };
|
||||
}""")
|
||||
|
||||
self._profile_cache = profile
|
||||
|
||||
# 2.5 检测封禁状态
|
||||
ban_status = await self.check_banned_status()
|
||||
profile["_status"] = ban_status
|
||||
log.info(
|
||||
f" 📊 小红书主页: {profile.get('nickname','?')}"
|
||||
f" ID={profile.get('user_id','?')}"
|
||||
f" 关注={profile.get('following','?')}"
|
||||
f" 粉丝={profile.get('fans','?')}"
|
||||
f" 获赞={profile.get('likes','?')}"
|
||||
f" 笔记={profile.get('posts','?')}"
|
||||
f" 状态={ban_status}"
|
||||
)
|
||||
|
||||
# 3. 保存到 profiles.json
|
||||
self._save_profiles_json()
|
||||
|
||||
return OpResult("xhs_goto_profile", step_id, True, "profile_loaded")
|
||||
|
||||
# ═══════════════════════════════════════════════════════
|
||||
# 读缓存字段
|
||||
# ═══════════════════════════════════════════════════════
|
||||
|
||||
async def xhs_read_nickname(self, args: dict, step_id: int) -> OpResult:
|
||||
v = self._profile_cache.get("nickname", "?")
|
||||
log.info(f" 📝 昵称: {v} (cached)")
|
||||
return OpResult("xhs_read_nickname", step_id, True, f"nickname={v}")
|
||||
|
||||
async def xhs_read_user_id(self, args: dict, step_id: int) -> OpResult:
|
||||
v = self._profile_cache.get("user_id", "?")
|
||||
log.info(f" 🔢 ID: {v} (cached)")
|
||||
return OpResult("xhs_read_user_id", step_id, True, f"user_id={v}")
|
||||
|
||||
async def xhs_read_following(self, args: dict, step_id: int) -> OpResult:
|
||||
v = self._profile_cache.get("following", "?")
|
||||
log.info(f" 👥 关注: {v} (cached)")
|
||||
return OpResult("xhs_read_following", step_id, True, f"following={v}")
|
||||
|
||||
async def xhs_read_fans(self, args: dict, step_id: int) -> OpResult:
|
||||
v = self._profile_cache.get("fans", "?")
|
||||
log.info(f" 👥 粉丝: {v} (cached)")
|
||||
return OpResult("xhs_read_fans", step_id, True, f"fans={v}")
|
||||
|
||||
async def xhs_read_likes(self, args: dict, step_id: int) -> OpResult:
|
||||
v = self._profile_cache.get("likes", "?")
|
||||
log.info(f" 👍 获赞: {v} (cached)")
|
||||
return OpResult("xhs_read_likes", step_id, True, f"likes={v}")
|
||||
|
||||
async def xhs_read_bio(self, args: dict, step_id: int) -> OpResult:
|
||||
v = self._profile_cache.get("bio", "?")
|
||||
log.info(f" 📄 简介: {v} (cached)")
|
||||
return OpResult("xhs_read_bio", step_id, True, f"bio={v}")
|
||||
|
||||
# ═══════════════════════════════════════════════════════
|
||||
# 内部 + 通用操作
|
||||
# ═══════════════════════════════════════════════════════
|
||||
|
||||
async def wait_watch(self, args: dict, step_id: int) -> OpResult:
|
||||
"""等待观看(随机5-15秒)"""
|
||||
import random
|
||||
seconds = args.get("seconds") or random.randint(5, 15)
|
||||
await asyncio.sleep(seconds)
|
||||
return OpResult("wait_watch", step_id, True, f"watched_{seconds}s")
|
||||
|
||||
async def go_back(self, args: dict, step_id: int) -> OpResult:
|
||||
"""返回上一页"""
|
||||
try:
|
||||
await self.page.go_back(wait_until="domcontentloaded")
|
||||
await asyncio.sleep(2)
|
||||
return OpResult("go_back", step_id, True, "back")
|
||||
except Exception as e:
|
||||
return OpResult("go_back", step_id, True, "back_fallback")
|
||||
|
||||
async def goto_url(self, args: dict, step_id: int) -> OpResult:
|
||||
"""跳转到指定URL(定向评论用)"""
|
||||
url = args.get("url", XHS_HOME_URL)
|
||||
try:
|
||||
await self.page.goto(url, timeout=20000, wait_until="domcontentloaded")
|
||||
await asyncio.sleep(4)
|
||||
return OpResult("goto_url", step_id, True, f"goto {url[:30]}")
|
||||
except Exception as e:
|
||||
return OpResult("goto_url", step_id, False, "goto_fail", error=str(e))
|
||||
|
||||
async def check_banned_status(self) -> str:
|
||||
"""检测小红书账号是否被封禁
|
||||
Returns: "normal" / "banned" / "unknown"
|
||||
"""
|
||||
try:
|
||||
text = (await self.page.evaluate("document.body.innerText")) or ""
|
||||
# 个人主页:含有"违反"+"社区规范"关键词
|
||||
if "违反" in text and "社区规范" in text:
|
||||
return "banned"
|
||||
# 操作弹窗:div.banned-title 可见
|
||||
try:
|
||||
bt = self.page.locator('div.banned-title')
|
||||
if await bt.count() > 0 and await bt.first.is_visible():
|
||||
return "banned"
|
||||
except:
|
||||
pass
|
||||
except:
|
||||
pass
|
||||
return "normal"
|
||||
|
||||
def _save_profiles_json(self):
|
||||
"""保存主页信息到 profiles.json + homepage_info.json(两个 Dashboard 数据源)"""
|
||||
if not self._account_id:
|
||||
return
|
||||
try:
|
||||
cache = self._profile_cache
|
||||
status = cache.get("_status", "normal")
|
||||
|
||||
# ── 写 profiles.json(供账号管理页面读取)──
|
||||
if PROFILES_JSON.exists():
|
||||
all_p = json.loads(PROFILES_JSON.read_text())
|
||||
else:
|
||||
all_p = {}
|
||||
PROFILES_JSON.parent.mkdir(parents=True, exist_ok=True)
|
||||
all_p[self._account_id] = {
|
||||
"nickname": cache.get("nickname", "?"),
|
||||
"user_id": cache.get("user_id", "?"),
|
||||
"following": cache.get("following", "?"),
|
||||
"fans": cache.get("fans", "?"),
|
||||
"likes": cache.get("likes", "?"),
|
||||
"posts": cache.get("posts", "?"),
|
||||
"bio": cache.get("bio", "?"),
|
||||
"status": status,
|
||||
"platform": "xiaohongshu",
|
||||
"updated": datetime.now().isoformat(),
|
||||
}
|
||||
PROFILES_JSON.write_text(json.dumps(all_p, ensure_ascii=False, indent=2))
|
||||
|
||||
# ── 同步写 homepage_info.json(供信息采集页面读取)──
|
||||
hp_path = PROFILES_JSON.parent / "homepage_info.json"
|
||||
hp_data = {"collected_at": datetime.now().isoformat(), "results": []}
|
||||
if hp_path.exists():
|
||||
try:
|
||||
hp_data = json.loads(hp_path.read_text())
|
||||
except Exception:
|
||||
hp_data = {"collected_at": datetime.now().isoformat(), "results": []}
|
||||
# 查找或创建当前账号的记录
|
||||
entry = None
|
||||
for r in hp_data.get("results", []):
|
||||
rid = r.get("identity_dir", "").replace("identities/", "")
|
||||
if rid == self._account_id or r.get("phone") == cache.get("phone", self._account_id):
|
||||
entry = r
|
||||
break
|
||||
if not entry:
|
||||
hp_data.setdefault("results", []).append({
|
||||
"identity_dir": f"identities/{self._account_id}",
|
||||
"phone": cache.get("phone", self._account_id),
|
||||
"douyin": None, "xiaohongshu": {}
|
||||
})
|
||||
entry = hp_data["results"][-1]
|
||||
# 确保 xiaohongshu 字段存在
|
||||
if entry.get("xiaohongshu") is None:
|
||||
entry["xiaohongshu"] = {}
|
||||
entry["xiaohongshu"].update({
|
||||
"nickname": cache.get("nickname", ""),
|
||||
"fans": cache.get("fans", "0"),
|
||||
"following": cache.get("following", "0"),
|
||||
"likes": cache.get("likes", "0"),
|
||||
"notes": cache.get("posts", "0"),
|
||||
"bio": cache.get("bio", ""),
|
||||
"status": status,
|
||||
"updated": datetime.now().isoformat(),
|
||||
})
|
||||
hp_path.write_text(json.dumps(hp_data, ensure_ascii=False, indent=2))
|
||||
|
||||
except Exception as e:
|
||||
import logging
|
||||
logging.getLogger(__name__).warning(f"_save_profiles_json 失败: {e}")
|
||||
@@ -0,0 +1,284 @@
|
||||
"""
|
||||
PageInspector — 三层页面状态检测
|
||||
|
||||
L1: 页面类型 (grid/player/search/profile) — URL pattern + DOM特征
|
||||
L2: 鼠标所在区域 (feed/video/comment/bottom_bar) — 坐标区间判断
|
||||
L3: 鼠标指向元素详情 — tag/class/text/position
|
||||
|
||||
基于 ghai 的人类视角判断逻辑:
|
||||
1. 先看URL + 页面大结构 → 确定在什么页面
|
||||
2. 再看鼠标周围区域 → 确定操作上下文
|
||||
3. 最后看鼠标指向的具体元素 → 确定要操作什么
|
||||
|
||||
Usage:
|
||||
inspector = PageInspector(page, platform="douyin")
|
||||
state = await inspector.inspect()
|
||||
print(state.page_type) # "player_modal"
|
||||
print(state.has_video) # True
|
||||
print(state.nearby_elements) # [element at cursor...]
|
||||
"""
|
||||
import asyncio
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Optional
|
||||
|
||||
|
||||
@dataclass
|
||||
class PageState:
|
||||
"""完整页面状态快照"""
|
||||
platform: str = ""
|
||||
url: str = ""
|
||||
title: str = ""
|
||||
|
||||
# L1: 页面类型
|
||||
page_type: str = "unknown" # grid/player_modal/player_full/search/profile/unknown
|
||||
page_type_confidence: float = 0.0
|
||||
|
||||
# L1: DOM特征
|
||||
video_count: int = 0
|
||||
card_count: int = 0
|
||||
has_search_bar: bool = False
|
||||
has_nav: bool = False
|
||||
has_comment_list: bool = False
|
||||
is_self_page: bool = False # 是否自己的主页
|
||||
author_id: str = ""
|
||||
|
||||
# L2: 鼠标区域
|
||||
mouse_x: int = 0
|
||||
mouse_y: int = 0
|
||||
mouse_region: str = "unknown" # feed_area/video_area/bottom_bar/comment_area/nav_area
|
||||
|
||||
# L3: 附近元素
|
||||
nearby_elements: list = field(default_factory=list) # [{tag, class, text, x, y, ...}]
|
||||
element_at_point: Optional[dict] = None
|
||||
|
||||
# 验证
|
||||
errors: list = field(default_factory=list)
|
||||
warnings: list = field(default_factory=list)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {k: v for k, v in self.__dict__.items()}
|
||||
|
||||
|
||||
class PageInspector:
|
||||
"""三层页面状态检测器"""
|
||||
|
||||
def __init__(self, page, platform: str = "douyin"):
|
||||
self.page = page
|
||||
self.platform = platform
|
||||
|
||||
async def inspect(self, mouse_x: int = 0, mouse_y: int = 0) -> PageState:
|
||||
"""执行完整三层检测"""
|
||||
state = PageState(platform=self.platform)
|
||||
|
||||
try:
|
||||
# L1: 页面基本信息
|
||||
state.url = self.page.url
|
||||
state.title = await self.page.title()
|
||||
|
||||
# L1: DOM特征
|
||||
dom = await self._capture_dom()
|
||||
state.video_count = dom["video_count"]
|
||||
state.card_count = dom["card_count"]
|
||||
state.has_search_bar = dom["has_search_bar"]
|
||||
state.has_nav = dom["has_nav"]
|
||||
state.has_comment_list = dom["has_comment_list"]
|
||||
state.author_id = dom["author_id"]
|
||||
state.is_self_page = dom["is_self"]
|
||||
|
||||
# L1: 页面类型判定
|
||||
state.page_type = self._classify_page_type(state)
|
||||
|
||||
# L2: 鼠标区域
|
||||
if mouse_x > 0 and mouse_y > 0:
|
||||
state.mouse_x = mouse_x
|
||||
state.mouse_y = mouse_y
|
||||
size = await self.page.evaluate(
|
||||
"() => ({ w: window.innerWidth, h: window.innerHeight })"
|
||||
)
|
||||
state.mouse_region = self._classify_region(
|
||||
mouse_x, mouse_y, size["w"], size["h"]
|
||||
)
|
||||
|
||||
# L3: 鼠标处元素
|
||||
if mouse_x > 0 and mouse_y > 0:
|
||||
el_info = await self.page.evaluate(
|
||||
f"""() => {{
|
||||
const el = document.elementFromPoint({mouse_x}, {mouse_y});
|
||||
if (!el) return null;
|
||||
const rect = el.getBoundingClientRect();
|
||||
return {{
|
||||
tag: el.tagName,
|
||||
class: el.className || '',
|
||||
text: (el.textContent || '').trim().slice(0, 50),
|
||||
x: Math.round(rect.left), y: Math.round(rect.top),
|
||||
w: Math.round(rect.width), h: Math.round(rect.height),
|
||||
visible: el.offsetParent !== null
|
||||
}};
|
||||
}}"""
|
||||
)
|
||||
state.element_at_point = el_info
|
||||
|
||||
# L3: 附近区域元素(鼠标周围200px内的可交互元素)
|
||||
if mouse_x > 0 and mouse_y > 0:
|
||||
nearby = await self.page.evaluate(
|
||||
f"""() => {{
|
||||
const mx = {mouse_x}, my = {mouse_y};
|
||||
const els = document.querySelectorAll(
|
||||
'a, button, span, div, input, textarea, svg')
|
||||
const results = [];
|
||||
for (const el of els) {{
|
||||
if (!el.offsetParent) continue;
|
||||
const r = el.getBoundingClientRect();
|
||||
const cx = r.left + r.width/2;
|
||||
const cy = r.top + r.height/2;
|
||||
const dist = Math.sqrt((cx-mx)**2 + (cy-my)**2);
|
||||
if (dist < 200) {{
|
||||
results.push({{
|
||||
tag: el.tagName,
|
||||
class: (el.className || '').slice(0, 40),
|
||||
text: (el.textContent || '').trim().slice(0, 30),
|
||||
cx: Math.round(cx), cy: Math.round(cy), dist: Math.round(dist)
|
||||
}});
|
||||
}}
|
||||
}}
|
||||
return results.sort((a,b) => a.dist - b.dist).slice(0, 10);
|
||||
}}"""
|
||||
)
|
||||
state.nearby_elements = nearby
|
||||
|
||||
except Exception as e:
|
||||
state.errors.append(str(e))
|
||||
|
||||
return state
|
||||
|
||||
async def _capture_dom(self) -> dict:
|
||||
"""捕获DOM特征"""
|
||||
try:
|
||||
return await self.page.evaluate("""() => {
|
||||
const u = location.href;
|
||||
const vc = document.querySelectorAll('video').length;
|
||||
|
||||
// cards: 视频卡片/笔记卡片
|
||||
const cards = document.querySelectorAll(
|
||||
'.discover-video-card-item, [class*="video-card"], ' +
|
||||
'section.note-item, a[href*="/explore/"], [class*="note-item"]'
|
||||
).length;
|
||||
|
||||
// 搜索栏
|
||||
const searchBar = document.querySelector(
|
||||
'[data-e2e="searchbar-input"], input[placeholder*="搜索"], textarea.textarea'
|
||||
);
|
||||
|
||||
// 导航
|
||||
const nav = document.querySelector(
|
||||
'[data-e2e="douyin-navigation"], [class*="nav"], .ulxic5B4'
|
||||
);
|
||||
|
||||
// 评论列表
|
||||
const commentList = document.querySelector(
|
||||
'[data-e2e="comment-list"], [class*="comment-list"]'
|
||||
);
|
||||
|
||||
// 作者ID (从URL或个人信息区获取)
|
||||
let authorId = '';
|
||||
const userMatch = u.match(/\/user\/(\\d+)/);
|
||||
if (userMatch) authorId = userMatch[1];
|
||||
const exploreMatch = u.match(/\/explore\/(\\w+)/);
|
||||
if (exploreMatch) authorId = exploreMatch[1];
|
||||
|
||||
// 是否自己的页面
|
||||
const isSelf = u.includes('/user/self') ||
|
||||
!!document.querySelector('[data-e2e="user-info"]');
|
||||
|
||||
return {
|
||||
video_count: vc,
|
||||
card_count: cards,
|
||||
has_search_bar: !!searchBar,
|
||||
has_nav: !!nav,
|
||||
has_comment_list: !!commentList,
|
||||
author_id: authorId,
|
||||
is_self: isSelf
|
||||
};
|
||||
}""")
|
||||
except:
|
||||
return {"video_count":0, "card_count":0, "has_search_bar":False,
|
||||
"has_nav":False, "has_comment_list":False, "author_id":"", "is_self":False}
|
||||
|
||||
def _classify_page_type(self, state: PageState) -> str:
|
||||
"""L1: 页面类型判定"""
|
||||
u = state.url.lower()
|
||||
|
||||
# 个人主页
|
||||
if '/user/' in u:
|
||||
return 'profile'
|
||||
|
||||
# 搜索页
|
||||
if '/search' in u:
|
||||
return 'search'
|
||||
|
||||
# 播放页: 有video
|
||||
if state.video_count > 0:
|
||||
if '/video/' in u and 'modal_id' not in u:
|
||||
return 'player_full'
|
||||
if 'modal_id' in u:
|
||||
return 'player_modal'
|
||||
return 'player'
|
||||
|
||||
# 笔记详情页 (小红书)
|
||||
if '/explore/' in u and u.count('/') >= 4:
|
||||
return 'note_detail'
|
||||
|
||||
# 首页/grid
|
||||
if state.card_count > 0 or '/jingxuan' in u or '/explore' in u:
|
||||
return 'grid'
|
||||
|
||||
return 'unknown'
|
||||
|
||||
def _classify_region(self, x: int, y: int, w: int, h: int) -> str:
|
||||
"""L2: 鼠标区域判定(基于百分比位置)"""
|
||||
x_pct = x / w
|
||||
y_pct = y / h
|
||||
|
||||
# 视频播放区(中央上部区域)
|
||||
if 0.2 < x_pct < 0.8 and 0.1 < y_pct < 0.5:
|
||||
return 'video_area'
|
||||
|
||||
# 底部互动按钮区
|
||||
if 0.2 < x_pct < 0.8 and y_pct > 0.8:
|
||||
return 'bottom_bar'
|
||||
|
||||
# 评论区(底部大区域)
|
||||
if y_pct > 0.7:
|
||||
return 'comment_area'
|
||||
|
||||
# 左侧导航区
|
||||
if x_pct < 0.2:
|
||||
return 'nav_area'
|
||||
|
||||
# 右侧推荐区
|
||||
if x_pct > 0.8:
|
||||
return 'sidebar'
|
||||
|
||||
# 内容feed区(中下部)
|
||||
if 0.2 < x_pct < 0.8 and 0.5 < y_pct < 0.8:
|
||||
return 'feed_area'
|
||||
|
||||
return 'unknown'
|
||||
|
||||
|
||||
async def demo(page):
|
||||
"""快速演示"""
|
||||
inspector = PageInspector(page, "douyin")
|
||||
|
||||
# 全量检测
|
||||
state = await inspector.inspect()
|
||||
print(f"[{state.page_type}] v={state.video_count} c={state.card_count}")
|
||||
|
||||
# 带鼠标位置的检测(模拟鼠标在右下角互动区)
|
||||
state2 = await inspector.inspect(mouse_x=479, mouse_y=687)
|
||||
print(f"鼠标区域: {state2.mouse_region}")
|
||||
print(f"指向元素: {state2.element_at_point}")
|
||||
print(f"附近元素: {len(state2.nearby_elements)}个")
|
||||
for el in state2.nearby_elements[:3]:
|
||||
print(f" {el['tag']} dist={el['dist']} {el['class'][:20]}")
|
||||
return state2
|
||||
@@ -0,0 +1,197 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
PageState — 页面状态管理 (v1.0.0)
|
||||
|
||||
功能:
|
||||
- 检测当前页面模式 (grid/player/search/unknown)
|
||||
- 验证登录态 (Cookie + DOM + 标题多维检测)
|
||||
- 提供状态切换建议 (下一步该做什么)
|
||||
|
||||
用法:
|
||||
state = PageState(page)
|
||||
mode = await state.detect() # → "grid" | "player" | "search"
|
||||
ok = await state.check_login()
|
||||
next = await state.suggest() # → 建议下一步操作
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from typing import Optional
|
||||
|
||||
__version__ = "1.0.0"
|
||||
|
||||
|
||||
# 各模式的特征元素
|
||||
MODE_SIGNATURES = {
|
||||
"grid": {
|
||||
"name": "首页卡片列表",
|
||||
"anchors": ['[data-e2e="alink-item"]'],
|
||||
"description": "显示视频卡片的推荐/精选页",
|
||||
},
|
||||
"player": {
|
||||
"name": "视频播放器",
|
||||
"anchors": ['[data-e2e="video-player-digg"]', '[data-e2e="video-player-collect"]'],
|
||||
"description": "单个视频播放页,有点赞/收藏按钮",
|
||||
},
|
||||
"search": {
|
||||
"name": "搜索结果",
|
||||
"anchors": ['[data-e2e="searchbar-input"]', '.search-container'],
|
||||
"description": "搜索结果页,显示视频列表",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class PageState:
|
||||
"""页面状态管理器"""
|
||||
|
||||
def __init__(self, page):
|
||||
self.page = page
|
||||
self._mode = "unknown"
|
||||
self._last_check = 0
|
||||
|
||||
async def detect(self) -> str:
|
||||
"""检测当前页面模式"""
|
||||
for mode, sig in MODE_SIGNATURES.items():
|
||||
for anchor in sig["anchors"]:
|
||||
try:
|
||||
el = await self.page.query_selector(anchor)
|
||||
if el:
|
||||
self._mode = mode
|
||||
self._last_check = asyncio.get_event_loop().time()
|
||||
return mode
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Fallback: 通过 URL 判断
|
||||
url = self.page.url
|
||||
if "/video/" in url:
|
||||
self._mode = "player"
|
||||
elif "/search/" in url:
|
||||
self._mode = "search"
|
||||
elif "/jingxuan" in url or url.endswith("/douyin.com/"):
|
||||
self._mode = "grid"
|
||||
else:
|
||||
self._mode = "unknown"
|
||||
|
||||
self._last_check = asyncio.get_event_loop().time()
|
||||
return self._mode
|
||||
|
||||
@property
|
||||
def mode(self) -> str:
|
||||
"""当前模式(可能过期,建议先调用 detect())"""
|
||||
return self._mode
|
||||
|
||||
@property
|
||||
def description(self) -> str:
|
||||
"""当前模式的中文描述"""
|
||||
sig = MODE_SIGNATURES.get(self._mode)
|
||||
return sig["description"] if sig else "未知页面"
|
||||
|
||||
async def check_login(self) -> dict:
|
||||
"""多维登录验证
|
||||
|
||||
返回:
|
||||
{"logged_in": bool, "method": str, "detail": str}
|
||||
"""
|
||||
results = []
|
||||
|
||||
# 1. Cookie 检查
|
||||
try:
|
||||
cookies = await self.page.context.cookies()
|
||||
dy = [c for c in cookies if "douyin" in c.get("domain", "")]
|
||||
has_session = any(c["name"] == "sessionid" for c in dy)
|
||||
has_sid = any(c["name"] == "sid_guard" for c in dy)
|
||||
if has_session:
|
||||
results.append(("cookie", True, "sessionid 存在"))
|
||||
elif has_sid:
|
||||
results.append(("cookie", True, "sid_guard 存在"))
|
||||
else:
|
||||
results.append(("cookie", False, "无登录态 Cookie"))
|
||||
except Exception:
|
||||
results.append(("cookie", False, "读取失败"))
|
||||
|
||||
# 2. DOM 检查
|
||||
try:
|
||||
avatar = await self.page.query_selector('[data-e2e="user-avatar"]')
|
||||
detail = await self.page.query_selector('[data-e2e="user-detail"]')
|
||||
if avatar or detail:
|
||||
results.append(("dom", True, "用户头像可见"))
|
||||
else:
|
||||
results.append(("dom", False, "无头像元素"))
|
||||
except Exception:
|
||||
results.append(("dom", False, "检查失败"))
|
||||
|
||||
# 3. 标题检查
|
||||
try:
|
||||
title = await self.page.title()
|
||||
if "登录" in title:
|
||||
results.append(("title", False, "标题含登录字样"))
|
||||
else:
|
||||
results.append(("title", True, f"标题:{title[:30]}"))
|
||||
except Exception:
|
||||
results.append(("title", False, "读取失败"))
|
||||
|
||||
# 综合判断: 任意一个方法确认即可
|
||||
positive = [r for r in results if r[1]]
|
||||
logged_in = len(positive) >= 1
|
||||
|
||||
return {
|
||||
"logged_in": logged_in,
|
||||
"detail": results,
|
||||
"methods_checked": len(results),
|
||||
"methods_passed": len(positive),
|
||||
}
|
||||
|
||||
async def suggest(self) -> list:
|
||||
"""根据当前状态建议下一步操作"""
|
||||
mode = await self.detect()
|
||||
login_info = await self.check_login()
|
||||
|
||||
suggestions = []
|
||||
|
||||
if mode == "unknown":
|
||||
suggestions.append({
|
||||
"action": "goto_home",
|
||||
"reason": "无法识别当前页面,跳转首页",
|
||||
})
|
||||
elif mode == "grid":
|
||||
suggestions.append({
|
||||
"action": "open_video",
|
||||
"reason": "在首页,可以打开一个视频",
|
||||
})
|
||||
suggestions.append({
|
||||
"action": "scroll_feed",
|
||||
"reason": "浏览推荐内容",
|
||||
})
|
||||
# 如果未登录
|
||||
if not login_info["logged_in"]:
|
||||
decisions = "、".join(
|
||||
[f"{r[0]}:{'✅' if r[1] else '❌'}" for r in login_info.get("detail", [])]
|
||||
)
|
||||
suggestions.append({
|
||||
"action": "notify_login",
|
||||
"reason": f"可能未登录 ({decisions})",
|
||||
})
|
||||
elif mode == "player":
|
||||
suggestions.append({
|
||||
"action": "like",
|
||||
"reason": "在播放器,可以点赞",
|
||||
})
|
||||
suggestions.append({
|
||||
"action": "collect",
|
||||
"reason": "可以收藏",
|
||||
})
|
||||
suggestions.append({
|
||||
"action": "next_video",
|
||||
"reason": "可以看下一条",
|
||||
})
|
||||
suggestions.append({
|
||||
"action": "wait_watch",
|
||||
"reason": "观看一段时间",
|
||||
})
|
||||
elif mode == "search":
|
||||
suggestions.append({
|
||||
"action": "open_video",
|
||||
"reason": "在搜索结果页,打开视频",
|
||||
})
|
||||
|
||||
return suggestions
|
||||
@@ -0,0 +1,7 @@
|
||||
"""
|
||||
Pipelines — 抖音全模拟运营流水线
|
||||
|
||||
# content_publish.py 发布视频流水线
|
||||
# ecom_pipeline.py 电商/带货流水线
|
||||
# live_pipeline.py 直播模拟流水线
|
||||
"""
|
||||
@@ -0,0 +1,334 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
视频/图文发布脚本 — 通过 Camoufox 浏览器自动化上传
|
||||
|
||||
用法:
|
||||
python publish_video.py douyin --account my_name --file video.mp4 --title "标题" --desc "描述"
|
||||
python publish_video.py xiaohongshu --account my_name --file image.jpg --title "标题"
|
||||
|
||||
# 也可调用 social-auto-upload CLI (需先安装)
|
||||
python publish_video.py douyin --account my_name --file video.mp4 --title "标题" --sau
|
||||
"""
|
||||
|
||||
import sys, os, json, asyncio, argparse, logging, subprocess
|
||||
from pathlib import Path
|
||||
|
||||
# 路径
|
||||
from matrix_mgmt import AGENT_SYNC, AGENT_LOCAL
|
||||
|
||||
SCRIPTS_DIR = AGENT_SYNC / "05_tools" / "07_matrix" / "scripts"
|
||||
IDENTITIES_ROOT = AGENT_LOCAL / "identities"
|
||||
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
from cdp_connector import CDPConnector
|
||||
|
||||
logging.basicConfig(level=logging.INFO, format="%(message)s")
|
||||
log = logging.getLogger("publish")
|
||||
|
||||
# ── 平台上传页 URL ──
|
||||
UPLOAD_URLS = {
|
||||
"douyin": "https://www.douyin.com/upload/",
|
||||
"xiaohongshu": "https://www.xiaohongshu.com/upload/",
|
||||
}
|
||||
|
||||
# ── 默认等待时间 ──
|
||||
WAIT_SHORT = 3
|
||||
WAIT_MEDIUM = 5
|
||||
WAIT_LONG = 10
|
||||
|
||||
|
||||
def gaussian_jitter(base, min_val=None, max_val=None):
|
||||
"""高斯抖动: base 秒 ±30% 随机"""
|
||||
import random
|
||||
jitter = base * random.uniform(-0.3, 0.3)
|
||||
result = base + jitter
|
||||
if min_val is not None:
|
||||
result = max(result, min_val)
|
||||
if max_val is not None:
|
||||
result = min(result, max_val)
|
||||
return max(0.5, result) # 至少 0.5s
|
||||
|
||||
|
||||
async def publish_douyin(conn, file_path, title, desc):
|
||||
"""在已打开的 Camoufox 中上传抖音视频"""
|
||||
page = conn.page
|
||||
log.info(f" 📤 导航到上传页: {UPLOAD_URLS['douyin']}")
|
||||
await page.goto(UPLOAD_URLS["douyin"], timeout=30000)
|
||||
await asyncio.sleep(gaussian_jitter(WAIT_MEDIUM))
|
||||
|
||||
# 等待上传页面加载
|
||||
try:
|
||||
await page.wait_for_selector("input[type=file]", timeout=15000)
|
||||
except:
|
||||
log.warning(" ⚠️ 未找到文件上传 input,尝试备用选择器")
|
||||
try:
|
||||
await page.wait_for_selector(".upload-btn, .upload-file, [class*=upload]", timeout=10000)
|
||||
except:
|
||||
log.error(" ❌ 找不到上传按钮,请检查页面")
|
||||
return {"status": "error", "message": "找不到上传入口"}
|
||||
|
||||
# 选择文件
|
||||
file_abs = str(Path(file_path).resolve())
|
||||
if not os.path.exists(file_abs):
|
||||
return {"status": "error", "message": f"文件不存在: {file_abs}"}
|
||||
|
||||
log.info(f" 📎 选择文件: {file_abs}")
|
||||
try:
|
||||
file_input = await page.query_selector("input[type=file]")
|
||||
if file_input:
|
||||
await file_input.set_input_files(file_abs)
|
||||
else:
|
||||
# 有些平台的 file input 是隐藏的,需要通过 JS 触发
|
||||
await page.evaluate("""(path) => {
|
||||
const input = document.querySelector('input[type=file]');
|
||||
if (input) {
|
||||
const dt = new DataTransfer();
|
||||
input.files = dt.files;
|
||||
}
|
||||
}""", file_abs)
|
||||
log.warning(" ⚠️ 直接 set_input_files 可能不生效")
|
||||
except Exception as e:
|
||||
return {"status": "error", "message": f"文件选择失败: {e}"}
|
||||
|
||||
# 等待上传
|
||||
await asyncio.sleep(gaussian_jitter(WAIT_LONG, min_val=5))
|
||||
|
||||
# 填写标题
|
||||
if title:
|
||||
try:
|
||||
title_input = await page.query_selector(
|
||||
"[placeholder*=标题], [placeholder*=title], .title-input, input[name*=title]"
|
||||
)
|
||||
if title_input:
|
||||
await title_input.click()
|
||||
await asyncio.sleep(gaussian_jitter(1))
|
||||
await title_input.fill(title)
|
||||
log.info(f" ✏️ 标题已填写")
|
||||
except Exception as e:
|
||||
log.warning(f" ⚠️ 填写标题失败: {e}")
|
||||
|
||||
# 填写描述
|
||||
if desc:
|
||||
try:
|
||||
desc_input = await page.query_selector(
|
||||
"[placeholder*=描述], [placeholder*=简介], .desc-input, textarea"
|
||||
)
|
||||
if desc_input:
|
||||
await desc_input.click()
|
||||
await asyncio.sleep(gaussian_jitter(1))
|
||||
await desc_input.fill(desc)
|
||||
log.info(f" ✏️ 描述已填写")
|
||||
except Exception as e:
|
||||
log.warning(f" ⚠️ 填写描述失败: {e}")
|
||||
|
||||
# 等待手动操作或自动发布
|
||||
log.info(f" ⏳ 等待上传完成 (30s)...")
|
||||
await asyncio.sleep(gaussian_jitter(30, min_val=10))
|
||||
|
||||
# 尝试点击发布按钮
|
||||
publish_btn_selectors = [
|
||||
"button:has-text('发布')", "button:has-text('上传')",
|
||||
".publish-btn", "[class*=publish]", "[class*=submit]",
|
||||
"//button[contains(text(), '发布')]"
|
||||
]
|
||||
for selector in publish_btn_selectors:
|
||||
try:
|
||||
btn = await page.query_selector(selector)
|
||||
if btn:
|
||||
await btn.click()
|
||||
log.info(f" 🚀 点击发布按钮")
|
||||
await asynleep(gaussian_jitter(5))
|
||||
break
|
||||
except:
|
||||
continue
|
||||
|
||||
return {"status": "ok", "message": "发布流程已启动,请在浏览器中确认最终结果"}
|
||||
|
||||
|
||||
async def publish_xiaohongshu(conn, file_path, title, desc):
|
||||
"""小红书发布笔记"""
|
||||
page = conn.page
|
||||
log.info(f" 📤 导航到发布页: {UPLOAD_URLS['xiaohongshu']}")
|
||||
await page.goto(UPLOAD_URLS["xiaohongshu"], timeout=30000)
|
||||
await asyncio.sleep(gaussian_jitter(WAIT_MEDIUM))
|
||||
|
||||
# 上传图片
|
||||
file_abs = str(Path(file_path).resolve())
|
||||
if not os.path.exists(file_abs):
|
||||
return {"status": "error", "message": f"文件不存在: {file_abs}"}
|
||||
|
||||
try:
|
||||
file_input = await page.query_selector("input[type=file]")
|
||||
if file_input:
|
||||
await file_input.set_input_files(file_abs)
|
||||
log.info(f" 📎 已选择文件")
|
||||
except Exception as e:
|
||||
return {"status": "error", "message": f"文件选择失败: {e}"}
|
||||
|
||||
await asyncio.sleep(gaussian_jitter(WAIT_LONG, min_val=5))
|
||||
|
||||
# 填写标题
|
||||
if title:
|
||||
try:
|
||||
title_input = await page.query_selector(
|
||||
"[placeholder*=标题], .title-input, input[name*=title]"
|
||||
)
|
||||
if title_input:
|
||||
await title_input.fill(title)
|
||||
except: pass
|
||||
|
||||
# 填写正文
|
||||
if desc:
|
||||
try:
|
||||
desc_input = await page.query_selector(
|
||||
"[placeholder*=正文], .note-text, .ql-editor, [contenteditable=true]"
|
||||
)
|
||||
if desc_input:
|
||||
await desc_input.fill(desc)
|
||||
except: pass
|
||||
|
||||
await asyncio.sleep(gaussian_jitter(WAIT_MEDIUM))
|
||||
|
||||
return {"status": "ok", "message": "发布流程已启动,请在浏览器中确认"}
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
# social-auto-upload 集成 (sau CLI 调用)
|
||||
# ═══════════════════════════════════════════════════════════════
|
||||
|
||||
def _find_sau():
|
||||
"""查找 sau 可执行文件"""
|
||||
# 1. 本地 path
|
||||
import shutil
|
||||
sau = shutil.which("sau") or shutil.which("sau_cli")
|
||||
if sau:
|
||||
return sau
|
||||
# 2. pip 安装的脚本
|
||||
for p in [Path(sys.prefix) / "bin" / "sau",
|
||||
Path(sys.prefix) / "Scripts" / "sau.exe",
|
||||
Path.home() / ".local" / "bin" / "sau"]:
|
||||
if p.exists():
|
||||
return str(p)
|
||||
return None
|
||||
|
||||
|
||||
async def publish_via_sau(args):
|
||||
"""通过 social-auto-upload CLI 发布"""
|
||||
sau_path = _find_sau()
|
||||
if not sau_path:
|
||||
return {"status": "error", "message": "social-auto-upload 未安装 (需要: cd social-auto-upload && pip install -e .)"}
|
||||
|
||||
# 构建命令
|
||||
platform_map = {"douyin": "douyin", "xiaohongshu": "xiaohongshu"}
|
||||
action_map = {".mp4": "upload-video", ".avi": "upload-video", ".mov": "upload-video",
|
||||
".jpg": "upload-note", ".jpeg": "upload-note", ".png": "upload-note", ".gif": "upload-note"}
|
||||
|
||||
ext = Path(args.file).suffix.lower()
|
||||
action = action_map.get(ext, "upload-video")
|
||||
plat = platform_map.get(args.platform, args.platform)
|
||||
|
||||
cmd = [sys.executable, sau_path, plat, action,
|
||||
"--account", args.account,
|
||||
"--file", args.file if action == "upload-video" else None,
|
||||
"--images", args.file if action == "upload-note" else None,
|
||||
"--title", args.title or "无标题"]
|
||||
|
||||
# 清理 None 参数
|
||||
cmd = [c for c in cmd if c is not None]
|
||||
|
||||
if args.desc:
|
||||
cmd += ["--note" if action == "upload-note" else "--desc", args.desc]
|
||||
|
||||
log.info(f" 🚀 通过 social-auto-upload 发布 ({plat} {action})")
|
||||
log.info(f" command: {' '.join(str(c) for c in cmd[:8])}...")
|
||||
|
||||
try:
|
||||
p = subprocess.run(cmd, capture_output=True, text=True, timeout=600)
|
||||
out = p.stdout[-1000:] if p.stdout else ""
|
||||
err = p.stderr[-500:] if p.stderr else ""
|
||||
if p.returncode == 0:
|
||||
return {"status": "ok", "output": out}
|
||||
return {"status": "error", "message": err or out or f"退出码 {p.returncode}"}
|
||||
except subprocess.TimeoutExpired:
|
||||
return {"status": "error", "message": "发布超时 (600s)"}
|
||||
except Exception as e:
|
||||
return {"status": "error", "message": str(e)}
|
||||
|
||||
|
||||
async def main():
|
||||
parser = argparse.ArgumentParser(description="视频/图文发布")
|
||||
parser.add_argument("platform", choices=["douyin", "xiaohongshu"], help="平台")
|
||||
parser.add_argument("--account", required=True, help="账号身份目录名")
|
||||
parser.add_argument("--file", required=True, help="要上传的文件路径")
|
||||
parser.add_argument("--title", default="", help="标题")
|
||||
parser.add_argument("--desc", default="", help="描述/正文")
|
||||
parser.add_argument("--json", action="store_true", help="JSON 输出")
|
||||
parser.add_argument("--sau", action="store_true", help="使用 social-auto-upload 引擎 (代替 Camoufox)")
|
||||
args = parser.parse_args()
|
||||
|
||||
# 如果启用 social-auto-upload,调用 sau CLI
|
||||
if args.sau:
|
||||
return await publish_via_sau(args)
|
||||
|
||||
identity_dir = IDENTITIES_ROOT / args.account
|
||||
if not identity_dir.exists():
|
||||
# 尝试从账号注册表查找
|
||||
try:
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
from matrix_mgmt import MatrixManager
|
||||
mgr = MatrixManager()
|
||||
for a in mgr.list_accounts():
|
||||
if a.get("phone") == args.account or a["id"] == args.account:
|
||||
idir = a.get("identity_dir", "")
|
||||
if idir:
|
||||
identity_dir = IDENTITIES_ROOT / idir.replace("identities/", "")
|
||||
break
|
||||
except: pass
|
||||
|
||||
if not identity_dir.exists():
|
||||
result = {"status": "error", "message": f"身份目录不存在: {identity_dir}"}
|
||||
if args.json:
|
||||
print(json.dumps(result, ensure_ascii=False))
|
||||
else:
|
||||
print(f"❌ {result['message']}")
|
||||
return
|
||||
|
||||
log.info(f"\n🚀 发布到 {args.platform}")
|
||||
log.info(f" 账号: {args.account}")
|
||||
log.info(f" 文件: {args.file}")
|
||||
log.info(f" 标题: {args.title[:30]}...")
|
||||
|
||||
conn = None
|
||||
try:
|
||||
conn = CDPConnector(identity_dir=str(identity_dir), headless=False,
|
||||
window=(802, 783), locale=["zh-CN"])
|
||||
await conn.connect()
|
||||
await asyncio.sleep(gaussian_jitter(WAIT_SHORT))
|
||||
|
||||
if args.platform == "douyin":
|
||||
result = await publish_douyin(conn, args.file, args.title, args.desc)
|
||||
else:
|
||||
result = await publish_xiaohongshu(conn, args.file, args.title, args.desc)
|
||||
|
||||
if args.json:
|
||||
print(json.dumps(result, ensure_ascii=False))
|
||||
else:
|
||||
s = result.get("status")
|
||||
icon = "✅" if s == "ok" else "❌"
|
||||
print(f"\n{icon} {result.get('message', '')}")
|
||||
|
||||
except Exception as e:
|
||||
result = {"status": "error", "message": str(e)}
|
||||
if args.json:
|
||||
print(json.dumps(result, ensure_ascii=False))
|
||||
else:
|
||||
print(f"\n❌ 发布失败: {e}")
|
||||
finally:
|
||||
if conn:
|
||||
try:
|
||||
await conn.close()
|
||||
except: pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,255 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
任务蓝图引擎 - 解析 JSON 蓝图并自动执行
|
||||
用法: python task_engine.py <blueprint_name> [--account <account_id>] [--dry-run]
|
||||
"""
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import random
|
||||
import sqlite3
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
# ── 路径配置 ──────────────────────────────────────────────────────
|
||||
from local_paths import data_path, code_dir
|
||||
BASE_DIR = code_dir()
|
||||
DB_PATH = data_path("matrix.db")
|
||||
BLUEPRINT_DIR = code_dir() / "blueprints"
|
||||
|
||||
# ── 原子操作定义 ──────────────────────────────────────────────────
|
||||
OPERATIONS: dict[str, callable] = {}
|
||||
_DOYOPS = None # DouyinOps 实例引用,用于设置 execution_id
|
||||
|
||||
|
||||
def _make_douyin_op(op_name: str, func):
|
||||
"""包装 DouyinOps 方法,自动注入 step_id"""
|
||||
async def wrapper(step_id: int = 0, **kwargs):
|
||||
return await func(step_id=step_id, **kwargs)
|
||||
return wrapper
|
||||
|
||||
|
||||
def _register_ops(connector, db=None):
|
||||
"""动态注册异步操作到全局字典"""
|
||||
global OPERATIONS, _DOYOPS
|
||||
|
||||
# 基础 CDP 操作
|
||||
base_ops = {
|
||||
"goto": connector.goto,
|
||||
"click": lambda **kw: connector.page.click(**kw),
|
||||
"swipe_up": connector.swipe_up,
|
||||
"swipe_down": lambda **kw: connector.swipe_up(distance=-kw.get("distance", 600)),
|
||||
"touch_tap": connector.touch_tap,
|
||||
"wait": lambda **kw: asyncio.sleep(kw.get("seconds", 2)),
|
||||
"remove_overlays": connector.remove_overlays,
|
||||
"screenshot": lambda **kw: connector.page.screenshot(**kw),
|
||||
"evaluate": lambda **kw: connector.page.evaluate(kw.get("script")),
|
||||
"fill": lambda **kw: connector.page.fill(**kw),
|
||||
"press": lambda **kw: connector.page.press(**kw),
|
||||
}
|
||||
|
||||
# Douyin 原子操作
|
||||
from douyin_ops import DouyinOps
|
||||
dyops = DouyinOps(connector.page, db=db)
|
||||
_DOYOPS = dyops
|
||||
|
||||
douyin_ops = {}
|
||||
for name in dir(dyops):
|
||||
if name.startswith("_") or name in ("page", "db", "db_path", "execution_id",
|
||||
"_action_counts", "_session_start"):
|
||||
continue
|
||||
attr = getattr(dyops, name)
|
||||
if callable(attr):
|
||||
douyin_ops[name] = _make_douyin_op(name, attr)
|
||||
|
||||
OPERATIONS = {**base_ops, **douyin_ops}
|
||||
|
||||
# ── 熔断器 ──────────────────────────────────────────────────────
|
||||
class CircuitBreaker:
|
||||
"""连续失败 N 次后暂停指定时间"""
|
||||
|
||||
def __init__(self, max_failures: int = 3, pause_seconds: int = 1800):
|
||||
self.count = 0
|
||||
self.max_failures = max_failures
|
||||
self.pause_seconds = pause_seconds
|
||||
self.pause_until: float | None = None
|
||||
|
||||
def record_success(self):
|
||||
self.count = 0
|
||||
self.pause_until = None
|
||||
|
||||
def record_failure(self) -> bool:
|
||||
"""返回 True 表示熔断触发(已暂停)"""
|
||||
self.count += 1
|
||||
if self.count >= self.max_failures:
|
||||
self.pause_until = time.time() + self.pause_seconds
|
||||
print(f"⚠️ 连续失败 {self.max_failures} 次,熔断触发,暂停 {self.pause_seconds}s")
|
||||
return True
|
||||
return False
|
||||
|
||||
def is_paused(self) -> bool:
|
||||
if self.pause_until and time.time() < self.pause_until:
|
||||
remaining = int(self.pause_until - time.time())
|
||||
print(f"⏸ 熔断中,还剩 {remaining}s...")
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
# ── 蓝图执行器 ───────────────────────────────────────────────────
|
||||
class BlueprintRunner:
|
||||
"""解析并执行任务蓝图"""
|
||||
|
||||
def __init__(self, conn, db: sqlite3.Connection):
|
||||
self.conn = conn
|
||||
self.db = db
|
||||
self.circuit = CircuitBreaker()
|
||||
|
||||
def _log_exec(self, blueprint_id: str, account_id: str,
|
||||
status: str, duration: float, error: str = ""):
|
||||
cur = self.db.cursor()
|
||||
cur.execute("""
|
||||
INSERT INTO executions (blueprint_id, account_id, status, duration_sec, error_msg)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
""", (blueprint_id, account_id, status, round(duration, 2), error))
|
||||
self.db.commit()
|
||||
|
||||
async def run_step(self, step: dict) -> bool:
|
||||
"""执行单个步骤,返回成功/失败"""
|
||||
op = step.get("op") or step.get("operation")
|
||||
if not op:
|
||||
print(f" ⚠️ 步骤缺少 op 字段: {step}")
|
||||
return False
|
||||
|
||||
handler = OPERATIONS.get(op)
|
||||
if not handler:
|
||||
print(f" ⚠️ 未知操作: {op}")
|
||||
return False
|
||||
|
||||
args = step.get("args", {})
|
||||
wait_after = step.get("wait_after", 1.5)
|
||||
wait_jitter = step.get("wait_jitter", 0.5)
|
||||
retry = step.get("retry", 1)
|
||||
step_id = step.get("step_id", 0)
|
||||
|
||||
for attempt in range(retry):
|
||||
try:
|
||||
# DouyinOps 方法签名: func(step_id=0, **kwargs)
|
||||
# 先传 step_id,再传其他参数
|
||||
import inspect
|
||||
sig = inspect.signature(handler)
|
||||
params = list(sig.parameters.keys())
|
||||
if params and params[0] in ("step_id", "stepId"):
|
||||
await handler(step_id=step_id, **args)
|
||||
else:
|
||||
await handler(**args)
|
||||
await asyncio.sleep(wait_after + random.uniform(-wait_jitter, wait_jitter))
|
||||
return True
|
||||
except Exception as e:
|
||||
if attempt < retry - 1:
|
||||
print(f" ⚠️ [{op}] 重试 {attempt + 1}/{retry}: {e}")
|
||||
await asyncio.sleep(2 ** attempt)
|
||||
else:
|
||||
print(f" ❌ [{op}] 失败: {e}")
|
||||
return False
|
||||
return False
|
||||
|
||||
async def run_blueprint(
|
||||
self,
|
||||
blueprint: dict,
|
||||
account_id: str,
|
||||
connector,
|
||||
dry_run: bool = False
|
||||
) -> tuple[bool, float]:
|
||||
"""执行完整蓝图,返回 (是否成功, 耗时秒)"""
|
||||
bp_id = blueprint["id"]
|
||||
steps = blueprint.get("steps", [])
|
||||
start = time.time()
|
||||
|
||||
# 生成本次执行的 execution_id,供 DouyinOps 记录操作日志
|
||||
exec_id = str(uuid.uuid4())
|
||||
if _DOYOPS:
|
||||
_DOYOPS.execution_id = exec_id
|
||||
|
||||
print(f"\n{'='*50}")
|
||||
print(f"🚀 执行蓝图: {bp_id} | 账号: {account_id} | {'[DRY-RUN]' if dry_run else ''}")
|
||||
print(f"{'='*50}")
|
||||
|
||||
if dry_run:
|
||||
for i, step in enumerate(steps, 1):
|
||||
print(f" {i}. {step.get('op')} → {step.get('args', {})}")
|
||||
return True, time.time() - start
|
||||
|
||||
for i, step in enumerate(steps, 1):
|
||||
if self.circuit.is_paused():
|
||||
return False, time.time() - start
|
||||
|
||||
print(f" [{i}/{len(steps)}] {step.get('op')}")
|
||||
ok = await self.run_step(step)
|
||||
if not ok:
|
||||
error = f"step {i} failed: {step.get('op')}"
|
||||
self._log_exec(bp_id, account_id, "failed", time.time() - start, error)
|
||||
self.circuit.record_failure()
|
||||
return False, time.time() - start
|
||||
|
||||
self.circuit.record_success()
|
||||
self._log_exec(bp_id, account_id, "success", time.time() - start)
|
||||
print(f"✅ 蓝图 {bp_id} 执行完成,耗时 {time.time() - start:.1f}s")
|
||||
return True, time.time() - start
|
||||
|
||||
|
||||
# ── 主程序 ───────────────────────────────────────────────────────
|
||||
async def main():
|
||||
parser = argparse.ArgumentParser(description="Matrix 任务蓝图引擎")
|
||||
parser.add_argument("blueprint", help="蓝图名称(不含 .json)")
|
||||
parser.add_argument("--account", default="account_01", help="账号 ID")
|
||||
parser.add_argument("--dry-run", action="store_true", help="仅打印步骤,不执行")
|
||||
parser.add_argument("--port", type=int, default=9222, help="CDP 端口")
|
||||
args = parser.parse_args()
|
||||
|
||||
# 加载蓝图
|
||||
bp_path = BLUEPRINT_DIR / f"{args.blueprint}.json"
|
||||
if not bp_path.exists():
|
||||
# 尝试找 docs 目录
|
||||
bp_path = BASE_DIR / "docs" / "blueprints" / f"{args.blueprint}.json"
|
||||
if not bp_path.exists():
|
||||
print(f"❌ 蓝图不存在: {bp_path}")
|
||||
sys.exit(1)
|
||||
|
||||
with open(bp_path, encoding="utf-8") as f:
|
||||
blueprint = json.load(f)
|
||||
|
||||
# 连接数据库
|
||||
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
db = sqlite3.connect(str(DB_PATH), check_same_thread=False)
|
||||
db.execute("CREATE TABLE IF NOT EXISTS executions (id INTEGER PRIMARY KEY AUTOINCREMENT, blueprint_id TEXT, account_id TEXT, status TEXT, duration_sec REAL, error_msg TEXT, created_at TEXT DEFAULT (datetime('now')))")
|
||||
|
||||
if args.dry_run:
|
||||
# dry-run:只打印步骤
|
||||
for i, step in enumerate(blueprint.get("steps", []), 1):
|
||||
print(f" {i}. {step.get('op')} → {step.get('args', {})}")
|
||||
print(f"\n✅ 蓝图 {blueprint['id']} dry-run 完成")
|
||||
db.close()
|
||||
sys.exit(0)
|
||||
|
||||
# 连接浏览器
|
||||
from cdp_connector import CDPConnector
|
||||
connector = CDPConnector(port=args.port)
|
||||
_register_ops(connector, db=db)
|
||||
|
||||
try:
|
||||
await connector.connect()
|
||||
await connector.init_anti_detection()
|
||||
runner = BlueprintRunner(connector, db)
|
||||
ok, duration = await runner.run_blueprint(blueprint, args.account, connector, dry_run=False)
|
||||
sys.exit(0 if ok else 1)
|
||||
finally:
|
||||
await connector.close()
|
||||
db.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,132 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
任务调度器 - 按时间规则自动触发蓝图
|
||||
用法: python task_scheduler.py [--once]
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import sqlite3
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import yaml
|
||||
|
||||
from local_paths import config_path, data_path, code_dir
|
||||
|
||||
BASE_DIR = code_dir()
|
||||
DB_PATH = data_path("matrix.db")
|
||||
CONFIG_DIR = config_path()
|
||||
TZ_SH = ZoneInfo("Asia/Shanghai")
|
||||
|
||||
# ── 加载任务配置 ──────────────────────────────────────────────────
|
||||
def load_tasks() -> list[dict]:
|
||||
"""从 config/tasks.yaml 加载任务计划"""
|
||||
tasks_file = CONFIG_DIR / "tasks.yaml"
|
||||
if not tasks_file.exists():
|
||||
return []
|
||||
with open(tasks_file, encoding="utf-8") as f:
|
||||
return yaml.safe_load(f).get("tasks", [])
|
||||
|
||||
def load_accounts() -> dict[str, dict]:
|
||||
"""从 accounts.yaml 加载账号配置"""
|
||||
accounts_file = CONFIG_DIR / "accounts.yaml"
|
||||
if not accounts_file.exists():
|
||||
return {}
|
||||
with open(accounts_file, encoding="utf-8") as f:
|
||||
return {a["id"]: a for a in yaml.safe_load(f).get("accounts", [])}
|
||||
|
||||
def next_run(spec: str) -> datetime:
|
||||
"""简单解析 HH:MM 格式,返回今天/明天的下一次执行时间"""
|
||||
now = datetime.now(TZ_SH)
|
||||
hour, minute = map(int, spec.split(":"))
|
||||
nd = now.replace(hour=hour, minute=minute, second=0, microsecond=0)
|
||||
if nd <= now:
|
||||
nd += __import__("datetime").timedelta(days=1)
|
||||
return nd
|
||||
|
||||
def should_run(task: dict) -> bool:
|
||||
"""判断任务是否到达执行时间"""
|
||||
last_path = BASE_DIR / ".last_run" / f"{task['id']}.txt"
|
||||
if last_path.exists():
|
||||
last = datetime.fromisoformat(last_path.read_text().strip())
|
||||
last = last.replace(tzinfo=TZ_SH)
|
||||
now = datetime.now(TZ_SH)
|
||||
# 最少间隔 30 分钟
|
||||
if (now - last).total_seconds() < 1800:
|
||||
return False
|
||||
return True
|
||||
|
||||
def mark_run(task_id: str):
|
||||
"""标记任务已执行"""
|
||||
BASE_DIR.joinpath(".last_run").mkdir(exist_ok=True)
|
||||
(BASE_DIR / ".last_run" / f"{task_id}.txt").write_text(
|
||||
datetime.now(TZ_SH).isoformat()
|
||||
)
|
||||
|
||||
def record_status(task_id: str, account_id: str, status: str, msg: str = ""):
|
||||
now = datetime.now(TZ_SH).strftime("%Y-%m-%d %H:%M:%S")
|
||||
print(f"[{now}] {task_id} [{account_id}] → {status} {msg}")
|
||||
|
||||
async def run_task(task: dict, accounts: dict):
|
||||
"""执行单个任务"""
|
||||
bp_name = task.get("blueprint", "")
|
||||
if not bp_name:
|
||||
return
|
||||
|
||||
import subprocess
|
||||
for account_id in task.get("accounts", []):
|
||||
acc = accounts.get(account_id, {})
|
||||
port = acc.get("port", 9222)
|
||||
cmd = [
|
||||
sys.executable,
|
||||
str(BASE_DIR / "scripts" / "task_engine.py"),
|
||||
bp_name,
|
||||
"--account", account_id,
|
||||
"--port", str(port),
|
||||
]
|
||||
print(f"\n📦 触发: {bp_name} @ {account_id}")
|
||||
try:
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=300)
|
||||
if result.returncode == 0:
|
||||
record_status(task["id"], account_id, "✅ 成功")
|
||||
else:
|
||||
record_status(task["id"], account_id, "❌ 失败", result.stderr[:100])
|
||||
except subprocess.TimeoutExpired:
|
||||
record_status(task["id"], account_id, "⏱ 超时")
|
||||
except Exception as e:
|
||||
record_status(task["id"], account_id, "💥 异常", str(e)[:80])
|
||||
|
||||
async def scheduler_loop(once: bool = False):
|
||||
"""主调度循环"""
|
||||
print(f"\n{'='*50}")
|
||||
print(f"Matrix 任务调度器启动 | {datetime.now(TZ_SH).strftime('%Y-%m-%d %H:%M:%S CST')}")
|
||||
print(f"{'='*50}")
|
||||
|
||||
while True:
|
||||
tasks = load_tasks()
|
||||
accounts = load_accounts()
|
||||
now = datetime.now(TZ_SH)
|
||||
due = []
|
||||
|
||||
for task in tasks:
|
||||
if task.get("enabled", True) and should_run(task):
|
||||
due.append(task)
|
||||
mark_run(task["id"])
|
||||
|
||||
if due:
|
||||
print(f"\n📋 本轮待执行: {[t['id'] for t in due]}")
|
||||
for task in due:
|
||||
await run_task(task, accounts)
|
||||
else:
|
||||
print(f"\n⏳ {now.strftime('%H:%M:%S')} 无待执行任务,休眠 5 分钟...")
|
||||
|
||||
if once:
|
||||
break
|
||||
await asyncio.sleep(300) # 每 5 分钟检查一次
|
||||
|
||||
if __name__ == "__main__":
|
||||
once = "--once" in sys.argv
|
||||
asyncio.run(scheduler_loop(once=once))
|
||||
@@ -0,0 +1,64 @@
|
||||
"""
|
||||
comment_dom_scanner.py — 扫描评论区DOM结构
|
||||
"""
|
||||
import asyncio, json, os, sys
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'matrix_modules'))
|
||||
|
||||
async def main():
|
||||
from cdp_connector import CDPConnector
|
||||
|
||||
ID_DIR = os.path.expanduser('~/workbuddy-agent-os/agent-local/tools/matrix/identities/douyin_01_camo')
|
||||
conn = CDPConnector(browser_type='camoufox', identity_dir=ID_DIR,
|
||||
headless=False, window=(702,783), window_position=(652,0))
|
||||
print('🦊 启动...')
|
||||
await conn.connect()
|
||||
page = conn.page
|
||||
await conn.init_anti_detection()
|
||||
await page.goto('https://www.douyin.com/', timeout=15000)
|
||||
await asyncio.sleep(3)
|
||||
|
||||
print('\n📌 手动进入播放页,然后回车继续...')
|
||||
input()
|
||||
|
||||
# 按X打开评论区
|
||||
await page.keyboard.press('KeyX')
|
||||
print('\n⌨️ KeyX 打开了评论区')
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# 扫描所有元素的class,找包含comment的
|
||||
classes = await page.evaluate("""() => {
|
||||
const all = document.querySelectorAll('*');
|
||||
const commentRelated = [...all].filter(el => {
|
||||
const c = (el.className || '').toLowerCase();
|
||||
const t = (el.tagName || '');
|
||||
return c.includes('comment') || c.includes('chat') || c.includes('reply') || c.includes('input');
|
||||
});
|
||||
return commentRelated.map(el => ({
|
||||
tag: el.tagName,
|
||||
cls: (el.className || '').slice(0,80),
|
||||
visible: el.offsetParent !== null,
|
||||
rect: Object.values(el.getBoundingClientRect()).map(Math.round),
|
||||
text: (el.textContent || '').slice(0,30)
|
||||
}));
|
||||
}""")
|
||||
print(f'\n📊 找到 {len(classes)} 个评论相关元素:')
|
||||
for c in classes:
|
||||
print(f' {c["tag"]:8s} cls={c["cls"][:40]:40s} vis={c["visible"]} rect={c["rect"]} \"{c["text"]}\"')
|
||||
|
||||
# 保存到文件
|
||||
out = '/tmp/comment_dom_scan.json'
|
||||
with open(out, 'w') as f:
|
||||
json.dump(classes, f, ensure_ascii=False, indent=2)
|
||||
print(f'\n💾 已保存到 {out}')
|
||||
|
||||
print('\n✅ 扫描完成,浏览器保持打开')
|
||||
print('按 Ctrl+C 退出')
|
||||
|
||||
while True:
|
||||
await asyncio.sleep(10)
|
||||
|
||||
try:
|
||||
asyncio.run(main())
|
||||
except KeyboardInterrupt:
|
||||
print('\n👋 退出')
|
||||
@@ -0,0 +1,253 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
comment_sender.py v2 — 状态感知的评论发送器
|
||||
|
||||
流程(锚点驱动):
|
||||
1. 检测评论区是否打开 → 否则 KeyX
|
||||
2. 检测输入框是否已激活 → 否则 JS focus
|
||||
3. 检测是否有内容 → 否则键盘输入
|
||||
4. Alt+Enter 发送
|
||||
5. 验证码检测 → 手动输入
|
||||
"""
|
||||
import asyncio, os, random, sys, json
|
||||
script_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, os.path.dirname(script_dir))
|
||||
from cdp_connector import CDPConnector
|
||||
|
||||
LOG_FILE = '/tmp/comment_sender.log'
|
||||
def log(m):
|
||||
with open(LOG_FILE, 'a') as f: f.write(m + '\n')
|
||||
print(m, flush=True)
|
||||
|
||||
async def check_state(page) -> dict:
|
||||
"""检测评论区相关状态"""
|
||||
return await page.evaluate("""() => {
|
||||
const ae = document.activeElement;
|
||||
// 评论区面板
|
||||
const panel = document.querySelector('[class*="comment-header"]')
|
||||
|| document.querySelector('[class*="comment"]')
|
||||
|| document.querySelector('[class*="Comment"]');
|
||||
// Draft.js 编辑器
|
||||
const editor = document.querySelector('[contenteditable="true"][role="combobox"]')
|
||||
|| document.querySelector('[class*="DraftEditor"] [contenteditable]');
|
||||
// 输入框是否激活(activeElement 是编辑器或其子元素)
|
||||
const editorActive = ae && (ae.isContentEditable
|
||||
|| ae.getAttribute('contenteditable') === 'true'
|
||||
|| ae.closest('[contenteditable]'));
|
||||
return {
|
||||
has_comment_area: !!panel,
|
||||
has_editor: !!editor,
|
||||
editor_active: editorActive,
|
||||
active_tag: ae ? ae.tagName : 'none',
|
||||
active_cls: ae ? (ae.className||'').slice(0,50) : '',
|
||||
panel_rect: panel ? Object.values(panel.getBoundingClientRect()).map(Math.round) : null
|
||||
};
|
||||
}""")
|
||||
|
||||
async def focus_editor(page) -> bool:
|
||||
"""尝试聚焦 Draft.js 编辑器,返回是否成功"""
|
||||
ok = await page.evaluate("""() => {
|
||||
// 直接找 contenteditable 且有 role 的
|
||||
let ed = document.querySelector('[contenteditable="true"][role="combobox"]');
|
||||
if (!ed) ed = document.querySelector('[class*="DraftEditor"] [contenteditable]');
|
||||
if (!ed) ed = document.querySelector('[contenteditable="true"]');
|
||||
if (ed) { ed.focus(); ed.click(); return true; }
|
||||
return false;
|
||||
}""")
|
||||
await asyncio.sleep(0.5)
|
||||
return ok
|
||||
|
||||
async def send_comment(page, comment_text: str):
|
||||
"""发送一条评论(状态感知版)"""
|
||||
# ── Step 1: 检测并确保评论区打开 ──
|
||||
state = await check_state(page)
|
||||
log(f'📊 初始状态: comment={state["has_comment_area"]} editor={state["has_editor"]} active={state["editor_active"]}')
|
||||
|
||||
if not state['has_comment_area']:
|
||||
log('⌨️ KeyX 打开评论区')
|
||||
await page.keyboard.press('KeyX')
|
||||
await asyncio.sleep(2)
|
||||
else:
|
||||
log('✅ 评论区已打开,跳过KeyX')
|
||||
|
||||
# ── Step 2: 用真实鼠标点击激活输入框 ──
|
||||
state = await check_state(page)
|
||||
if state['editor_active']:
|
||||
log('✅ 输入框已激活')
|
||||
else:
|
||||
# 找到编辑器位置 → 用 Playwright 真实鼠标点击
|
||||
rect = await page.evaluate("""() => {
|
||||
let el = document.querySelector('[contenteditable="true"][role="combobox"]');
|
||||
if (!el) el = document.querySelector('[class*="DraftEditor"] [contenteditable]');
|
||||
if (!el) el = document.querySelector('[contenteditable="true"]');
|
||||
if (!el) return null;
|
||||
const r = el.getBoundingClientRect();
|
||||
return {x: r.x, y: r.y, w: r.width, h: r.height};
|
||||
}""")
|
||||
if rect and rect['w'] > 0 and rect['h'] > 0:
|
||||
# 用 Playwright 生成真实鼠标事件链
|
||||
x = rect['x'] + rect['w'] // 2
|
||||
y = rect['y'] + rect['h'] // 2
|
||||
log(f'🖱️ 真实鼠标点击编辑器: ({x:.0f}, {y:.0f})')
|
||||
await page.mouse.click(x, y)
|
||||
await asyncio.sleep(1)
|
||||
# 再次确认
|
||||
state = await check_state(page)
|
||||
if state['editor_active']:
|
||||
log('✅ 鼠标点击后输入框已激活')
|
||||
else:
|
||||
log('⚠️ 鼠标点击后仍未激活')
|
||||
else:
|
||||
log('⚠️ 找不到编辑器位置')
|
||||
return 'failed'
|
||||
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
# ── Step 3: JS 设置内容(不用键盘,不依赖焦点)──
|
||||
log(f'💬 JS设置内容: "{comment_text}"')
|
||||
result = await page.evaluate(f'''() => {{
|
||||
// 找 Draft.js 编辑器
|
||||
let ed = document.querySelector('[contenteditable="true"][role="combobox"]');
|
||||
if (!ed) ed = document.querySelector('[class*="DraftEditor"] [contenteditable]');
|
||||
if (!ed) ed = document.querySelector('[contenteditable="true"]');
|
||||
|
||||
if (!ed) return {{ok: false, reason: 'no_editor'}};
|
||||
|
||||
// focus + 清空 + 设置内容
|
||||
ed.focus();
|
||||
ed.textContent = '';
|
||||
ed.textContent = "{comment_text}";
|
||||
ed.dispatchEvent(new Event('input', {{bubbles: true}}));
|
||||
ed.dispatchEvent(new Event('change', {{bubbles: true}}));
|
||||
|
||||
const finalText = ed.textContent;
|
||||
return {{ok: finalText.length > 0, text: finalText, tag: ed.tagName}};
|
||||
}}''')
|
||||
log(f' 结果: ok={result["ok"]} text="{result.get("text","")}" tag={result.get("tag","")}')
|
||||
|
||||
if not result.get('ok'):
|
||||
log('⚠️ JS设置内容失败')
|
||||
return 'failed'
|
||||
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
# ── Step 4: JS 触发发送(先找发送按钮,没有则 Alt+Enter)──
|
||||
log('⌨️ JS触发发送...')
|
||||
send_result = await page.evaluate('''() => {
|
||||
// 找发送按钮
|
||||
const btns = [...document.querySelectorAll('button')];
|
||||
const sendBtn = btns.find(b =>
|
||||
b.textContent.includes('发送') || b.textContent.includes('发布')
|
||||
);
|
||||
if (sendBtn) {
|
||||
sendBtn.click();
|
||||
return 'click_send_btn';
|
||||
}
|
||||
// 没按钮 → 触发 Alt+Enter
|
||||
const ed = document.querySelector('[contenteditable="true"][role="combobox"]')
|
||||
|| document.querySelector('[contenteditable="true"]');
|
||||
if (ed) {
|
||||
ed.dispatchEvent(new KeyboardEvent('keydown', {
|
||||
key: 'Enter', altKey: true, bubbles: true
|
||||
}));
|
||||
return 'alt_enter_event';
|
||||
}
|
||||
return 'failed';
|
||||
}''')
|
||||
log(f' 发送方式: {send_result}')
|
||||
await asyncio.sleep(2)
|
||||
|
||||
# ── Step 5: 验证码 ──
|
||||
has_code = await page.evaluate(
|
||||
"() => !!document.querySelector('input[placeholder*=\"\\u9a8c\\u8bc1\\u7801\"]')"
|
||||
)
|
||||
if has_code:
|
||||
log('⚠️ 验证码弹窗!')
|
||||
return 'verify_code_required'
|
||||
|
||||
# ── Step 6: 验证发送结果 ──
|
||||
found = await page.evaluate(
|
||||
f'() => document.body.innerText.includes("{comment_text}")'
|
||||
)
|
||||
if found:
|
||||
log('✅ 评论已发送')
|
||||
return 'success'
|
||||
else:
|
||||
log('❌ 未在页面找到评论')
|
||||
return 'failed'
|
||||
|
||||
|
||||
async def main():
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--account', '-a', default='douyin_01')
|
||||
parser.add_argument('--comment', '-c', default='好内容,已三连')
|
||||
parser.add_argument('--count', '-n', type=int, default=2)
|
||||
args = parser.parse_args()
|
||||
|
||||
ID_DIR = os.path.expanduser(
|
||||
f'~/workbuddy-agent-os/agent-local/tools/matrix/identities/{args.account}_camo'
|
||||
)
|
||||
if not os.path.exists(ID_DIR):
|
||||
print(f'❌ 账号目录不存在: {ID_DIR}')
|
||||
return
|
||||
|
||||
print(f'🦊 启动: {args.account}')
|
||||
conn = CDPConnector(browser_type='camoufox', identity_dir=ID_DIR,
|
||||
headless=False, window=(702,783), window_position=(652,0))
|
||||
await conn.connect()
|
||||
page = conn.page
|
||||
await conn.init_anti_detection()
|
||||
await page.goto('https://www.douyin.com/', timeout=30000)
|
||||
await asyncio.sleep(3)
|
||||
|
||||
print('\n📌 请操作:')
|
||||
print(' 1. 进入视频播放页')
|
||||
print(' 2. 打开评论区(按 X)')
|
||||
print(' 3. 手动点击激活评论输入框')
|
||||
print(' 4. 回到终端按回车\n')
|
||||
input()
|
||||
|
||||
# 检查是否在播放页
|
||||
v = await page.evaluate('document.querySelectorAll("video").length')
|
||||
if v < 2:
|
||||
print('❌ 未检测到播放页')
|
||||
return
|
||||
print(f'✅ 播放页确认 (video={v})')
|
||||
|
||||
for i in range(args.count):
|
||||
print(f'\n——— 第{i+1}/{args.count}次评论 ———')
|
||||
result = await send_comment(page, args.comment)
|
||||
|
||||
if result == 'verify_code_required':
|
||||
code = input('\n📱 验证码(6位): ').strip()
|
||||
if code and len(code) == 6:
|
||||
await page.evaluate(f'''() => {{
|
||||
const inp = document.querySelector('input[placeholder*="验证码"]');
|
||||
if (inp) {{
|
||||
inp.value = "{code}";
|
||||
inp.dispatchEvent(new Event('input', {{bubbles: true}}));
|
||||
}}
|
||||
const btn = [...document.querySelectorAll('button')]
|
||||
.find(b => b.textContent.includes('确认') || b.textContent.includes('提交'));
|
||||
if (btn) btn.click();
|
||||
}}''')
|
||||
print('✅ 验证码已提交')
|
||||
await asyncio.sleep(2)
|
||||
|
||||
if i < args.count - 1:
|
||||
print('⏳ 评论区保持打开,5秒后下一次...')
|
||||
await asyncio.sleep(5)
|
||||
|
||||
print('\n✅ 完成,浏览器保持打开')
|
||||
print('按 Ctrl+C 退出')
|
||||
while True:
|
||||
await asyncio.sleep(10)
|
||||
|
||||
if __name__ == '__main__':
|
||||
try:
|
||||
asyncio.run(main())
|
||||
except KeyboardInterrupt:
|
||||
print('\n👋 退出')
|
||||
except Exception as e:
|
||||
import traceback; traceback.print_exc()
|
||||
@@ -0,0 +1,370 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
小红书登录后 DOM 重分析工具
|
||||
============================
|
||||
|
||||
用途:登录小红书后重新分析 DOM 结构,验证当前选择器是否仍然有效,
|
||||
输出结构化报告指导选择器更新。
|
||||
|
||||
用法:
|
||||
python tools/reanalyze_xhs_dom.py <identity_name>
|
||||
python tools/reanalyze_xhs_dom.py xhs_01 --full-dump # 输出完整 DOM 快照
|
||||
python tools/reanalyze_xhs_dom.py xhs_01 --explore-url https://www.xiaohongshu.com/explore # 自定义 URL
|
||||
|
||||
流程:
|
||||
1. 通过 Camoufox 连接身份目录(复用登录态)
|
||||
2. 导航到小红书 explore 首页
|
||||
3. 等待渲染稳定
|
||||
4. Dump 关键区域的 DOM 结构(保留选择器相关特征)
|
||||
5. 验证当前 selectors.py 中的每个选择器是否仍能匹配
|
||||
6. 输出结构化的重分析报告
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).parent.parent
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
|
||||
LOCAL_ROOT = Path.home() / "workbuddy-agent-os" / "agent-local" / "tools" / "matrix"
|
||||
IDENTITIES_ROOT = LOCAL_ROOT / "identities"
|
||||
|
||||
|
||||
async def analyze(identity_name: str, full_dump: bool = False, explore_url: str = None):
|
||||
"""主分析流程"""
|
||||
print(f"\n{'='*60}")
|
||||
print(f" 🔍 小红书 DOM 重分析工具")
|
||||
print(f" {'='*60}")
|
||||
print(f" 身份: {identity_name}")
|
||||
print(f" 模式: {'完整快照' if full_dump else '选择器验证'}")
|
||||
print()
|
||||
|
||||
# ── 1. 确定身份目录 ──
|
||||
# 优先从 accounts.yaml 读取
|
||||
import yaml
|
||||
config_path = LOCAL_ROOT / "config" / "accounts.yaml"
|
||||
try:
|
||||
with open(config_path) as f:
|
||||
data = yaml.safe_load(f)
|
||||
acct = None
|
||||
for a in data.get("accounts", []):
|
||||
if a["id"] == identity_name:
|
||||
acct = a
|
||||
break
|
||||
except:
|
||||
acct = None
|
||||
|
||||
if acct:
|
||||
identity_dir_str = acct.get("identity_dir", identity_name)
|
||||
if not identity_dir_str.startswith("/"):
|
||||
identity_dir_str = str(LOCAL_ROOT / identity_dir_str)
|
||||
else:
|
||||
identity_dir_str = str(IDENTITIES_ROOT / identity_name)
|
||||
|
||||
# 清理锁文件
|
||||
import glob
|
||||
for lock_file in glob.glob(f"{identity_dir_str}/user_data/**/.parentlock", recursive=True):
|
||||
try:
|
||||
os.remove(lock_file)
|
||||
except:
|
||||
pass
|
||||
for lock_file in glob.glob(f"{identity_dir_str}/user_data/**/lock", recursive=True):
|
||||
try:
|
||||
os.remove(lock_file)
|
||||
except:
|
||||
pass
|
||||
|
||||
# ── 2. 连接浏览器 ──
|
||||
from cdp_connector import CDPConnector
|
||||
conn = CDPConnector(
|
||||
identity_dir=identity_dir_str,
|
||||
headless=False,
|
||||
window=(702, 783),
|
||||
)
|
||||
print(f" 🚀 连接浏览器: {identity_dir_str}")
|
||||
try:
|
||||
await conn.connect()
|
||||
except Exception as e:
|
||||
print(f" ❌ 连接失败: {e}")
|
||||
sys.exit(1)
|
||||
print(f" ✅ 浏览器已连接")
|
||||
print(f" 📐 视口: 702×783")
|
||||
|
||||
# 反检测
|
||||
try:
|
||||
await conn.init_anti_detection()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# ── 3. 导航到小红书 ──
|
||||
target_url = explore_url or "https://www.xiaohongshu.com/explore"
|
||||
print(f"\n 🌐 导航到: {target_url}")
|
||||
for attempt in range(1, 4):
|
||||
try:
|
||||
await conn.page.goto(target_url, timeout=20000, wait_until="domcontentloaded")
|
||||
print(f" ✅ 导航成功 (第{attempt}次)")
|
||||
break
|
||||
except Exception as e:
|
||||
print(f" ⚠️ 导航失败 (第{attempt}次): {e}")
|
||||
if attempt < 3:
|
||||
await asyncio.sleep(3)
|
||||
else:
|
||||
print(f" ❌ 导航最终失败")
|
||||
await conn.close()
|
||||
return
|
||||
|
||||
# ── 等待渲染 ──
|
||||
print(f"\n ⏳ 等待页面渲染...")
|
||||
await asyncio.sleep(5)
|
||||
|
||||
# 关闭登录弹窗
|
||||
try:
|
||||
await conn.page.evaluate("""
|
||||
() => {
|
||||
document.querySelectorAll('[class*=modal], [class*=dialog], [class*=mask], [role=dialog]').forEach(m => {
|
||||
m.style.display = 'none';
|
||||
});
|
||||
}
|
||||
""")
|
||||
except:
|
||||
pass
|
||||
await asyncio.sleep(1)
|
||||
|
||||
# ── 4. 检测登录状态 ──
|
||||
print(f"\n 🔐 登录状态检测")
|
||||
try:
|
||||
from auth_manager import check_login_by_cookie_sync, count_platform_cookies
|
||||
cookies = await conn.context.cookies()
|
||||
logged_in = check_login_by_cookie_sync(cookies, "xiaohongshu")
|
||||
cookie_cnt = count_platform_cookies(cookies, "xiaohongshu")
|
||||
print(f" Cookie 检测: {'✅ 已登录' if logged_in else '❌ 未登录'}")
|
||||
print(f" 平台 Cookie 数: {cookie_cnt}")
|
||||
except Exception as e:
|
||||
print(f" ⚠️ 登录检测异常: {e}")
|
||||
|
||||
# ── 5. 检查页面基本结构 ──
|
||||
print(f"\n 📄 页面基本结构")
|
||||
page_info = await conn.page.evaluate("""
|
||||
() => {
|
||||
return {
|
||||
url: location.href,
|
||||
title: document.title,
|
||||
app_present: !!document.querySelector('#app'),
|
||||
body_children: document.body ? document.body.children.length : 0,
|
||||
viewport: {w: window.innerWidth, h: window.innerHeight},
|
||||
scroll: {x: window.scrollX, y: window.scrollY},
|
||||
};
|
||||
}
|
||||
""")
|
||||
print(f" URL: {page_info.get('url', '?')}")
|
||||
print(f" Title: {page_info.get('title', '?')}")
|
||||
print(f" #app: {'✅' if page_info.get('app_present') else '❌'}")
|
||||
print(f" body children: {page_info.get('body_children', '?')}")
|
||||
|
||||
# ── 6. 验证 selectors.py 中的选择器 ──
|
||||
print(f"\n{'='*60}")
|
||||
print(f" 📋 选择器验证")
|
||||
print(f" {'='*60}")
|
||||
|
||||
# 导入当前选择器
|
||||
sys.path.insert(0, str(SCRIPTS_DIR / "matrix_modules" / "ops" / "xhs"))
|
||||
from selectors import (
|
||||
NOTE_CARD, NOTE_CARD_COVER, NOTE_CARD_IMG,
|
||||
SEARCH_INPUT, SEARCH_BUTTON,
|
||||
FEED_CONTAINER, LOGIN_MODAL, LOGIN_MODAL_CLOSE,
|
||||
LIKE_BUTTON, COLLECT_BUTTON, FOLLOW_BUTTON,
|
||||
COMMENT_ENTRY, COMMENT_INPUT, COMMENT_SEND,
|
||||
ANCHORS,
|
||||
)
|
||||
|
||||
selector_tests = {
|
||||
"首页/瀑布流": [
|
||||
("瀑布流容器", FEED_CONTAINER),
|
||||
("笔记卡片", NOTE_CARD),
|
||||
("卡片封面链接", NOTE_CARD_COVER),
|
||||
("卡片图片", NOTE_CARD_IMG),
|
||||
],
|
||||
"搜索": [
|
||||
("搜索框", SEARCH_INPUT),
|
||||
("搜索按钮", SEARCH_BUTTON),
|
||||
],
|
||||
"弹窗": [
|
||||
("登录弹窗", LOGIN_MODAL),
|
||||
("关闭按钮", LOGIN_MODAL_CLOSE),
|
||||
],
|
||||
"互动(详情页)": [
|
||||
("点赞按钮", LIKE_BUTTON),
|
||||
("收藏按钮", COLLECT_BUTTON),
|
||||
("关注按钮", FOLLOW_BUTTON),
|
||||
("评论入口", COMMENT_ENTRY),
|
||||
("评论输入框", COMMENT_INPUT),
|
||||
("发送按钮", COMMENT_SEND),
|
||||
],
|
||||
}
|
||||
|
||||
all_ok = True
|
||||
for category, selectors in selector_tests.items():
|
||||
print(f"\n 【{category}】")
|
||||
js_code = "var sel = arguments[0]; return document.querySelector(sel) ? document.querySelector(sel).outerHTML.substring(0, 200) : null;"
|
||||
for name, sel in selectors:
|
||||
try:
|
||||
count = await conn.page.evaluate(f"document.querySelectorAll('{sel.replace(chr(39), chr(92) + chr(39))}').length")
|
||||
first_match = None
|
||||
if count > 0:
|
||||
first_match = await conn.page.evaluate(f"try {{ let el = document.querySelector('{sel.replace(chr(39), chr(92) + chr(39))}'); return el ? el.tagName + (el.className ? '.' + el.className.split(' ')[0] : '') : 'N/A'; }} catch(e) {{ return 'JS_ERR'; }}")
|
||||
status = "✅" if count > 0 else "⚠️"
|
||||
if count == 0:
|
||||
all_ok = False
|
||||
print(f" {status} {name}: \"{sel}\"")
|
||||
if count > 0:
|
||||
print(f" 匹配 {count} 个元素, 首元素: {first_match}")
|
||||
except Exception as e:
|
||||
print(f" ❌ {name}: JS 异常 - {e}")
|
||||
all_ok = False
|
||||
|
||||
# ── 7. 锚点验证 ──
|
||||
print(f"\n 【锚点验证】")
|
||||
for anchor_name, anchor_sel in ANCHORS.items():
|
||||
try:
|
||||
count = await conn.page.evaluate(f"document.querySelectorAll('{anchor_sel.replace(chr(39), chr(92) + chr(39))}').length")
|
||||
status = "✅" if count > 0 else "⚠️"
|
||||
if count == 0:
|
||||
all_ok = False
|
||||
print(f" {status} {anchor_name}: \"{anchor_sel}\" → {count} 个")
|
||||
except Exception as e:
|
||||
print(f" ❌ {anchor_name}: {e}")
|
||||
|
||||
# ── 8. 完整 DOM 快照(可选)──
|
||||
if full_dump:
|
||||
print(f"\n{'='*60}")
|
||||
print(f" 📸 完整 DOM 快照")
|
||||
print(f" {'='*60}")
|
||||
print(f" 正在保存到文件...")
|
||||
|
||||
# 获取关键区域的 HTML
|
||||
dom_snapshot = await conn.page.evaluate("""
|
||||
() => {
|
||||
const areas = {};
|
||||
|
||||
// #app 容器
|
||||
const app = document.querySelector('#app');
|
||||
if (app) areas.app = app.outerHTML.substring(0, 5000);
|
||||
|
||||
// 瀑布流区域
|
||||
const layout = document.querySelector('.layout, [class*=layout]');
|
||||
if (layout) areas.layout = layout.outerHTML.substring(0, 3000);
|
||||
|
||||
// 检查是否有详情页(如果有多个 section.note-item 说明在首页)
|
||||
const notes = document.querySelectorAll('section.note-item');
|
||||
if (notes.length > 0) {
|
||||
areas.note_first = notes[0].outerHTML.substring(0, 2000);
|
||||
areas.note_count = notes.length;
|
||||
}
|
||||
|
||||
// 记录当前页面上的所有 class 名(去重)
|
||||
const allClasses = new Set();
|
||||
document.querySelectorAll('*').forEach(el => {
|
||||
if (el.className && typeof el.className === 'string') {
|
||||
el.className.split(/\\s+/).forEach(c => {
|
||||
if (c && c.length > 1) allClasses.add(c);
|
||||
});
|
||||
}
|
||||
});
|
||||
areas.all_classes = [...allClasses].sort();
|
||||
|
||||
// 关键 URL 和 meta
|
||||
areas.meta = {
|
||||
url: location.href,
|
||||
title: document.title,
|
||||
links_count: document.querySelectorAll('a').length,
|
||||
buttons_count: document.querySelectorAll('button').length,
|
||||
img_count: document.querySelectorAll('img').length,
|
||||
video_count: document.querySelectorAll('video').length,
|
||||
};
|
||||
|
||||
return areas;
|
||||
}
|
||||
""")
|
||||
|
||||
timestamp = time.strftime("%Y%m%d_%H%M%S")
|
||||
dump_path = SCRIPTS_DIR / "data" / f"xhs_dom_dump_{timestamp}.json"
|
||||
dump_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 简化输出:移除过长内容
|
||||
output = {}
|
||||
for k, v in dom_snapshot.items():
|
||||
if isinstance(v, str) and len(v) > 500:
|
||||
output[k] = v[:500] + f"\n... [截断, 全文 {len(v)} 字符]"
|
||||
else:
|
||||
output[k] = v
|
||||
|
||||
with open(dump_path, "w", encoding="utf-8") as f:
|
||||
json.dump(output, f, indent=2, ensure_ascii=False)
|
||||
|
||||
print(f" 📁 已保存: {dump_path}")
|
||||
print(f" 页面元素: links={dom_snapshot.get('meta', {}).get('links_count','?')}, "
|
||||
f"buttons={dom_snapshot.get('meta', {}).get('buttons_count','?')}, "
|
||||
f"img={dom_snapshot.get('meta', {}).get('img_count','?')}")
|
||||
if dom_snapshot.get('note_count'):
|
||||
print(f" 笔记卡片数: {dom_snapshot.get('note_count')}")
|
||||
print(f" 页面 class 数: {len(dom_snapshot.get('all_classes', []))}")
|
||||
|
||||
# ── 9. 截图 ──
|
||||
print(f"\n 📸 截图")
|
||||
screenshot_path = SCRIPTS_DIR / "data" / f"xhs_reanalyze_{time.strftime('%Y%m%d_%H%M%S')}.png"
|
||||
try:
|
||||
await conn.page.screenshot(path=str(screenshot_path))
|
||||
print(f" ✅ 截图: {screenshot_path}")
|
||||
except:
|
||||
print(f" ⚠️ 截图失败")
|
||||
|
||||
# ── 10. 总结 ──
|
||||
print(f"\n{'='*60}")
|
||||
print(f" 📊 分析总结")
|
||||
print(f" {'='*60}")
|
||||
if all_ok:
|
||||
print(f" ✅ 所有选择器均匹配当前 DOM")
|
||||
print(f" → 当前 selectors.py 无需更新")
|
||||
else:
|
||||
print(f" ⚠️ 部分选择器未匹配到元素")
|
||||
print(f" → 建议更新 selectors.py 中的不匹配选择器")
|
||||
print(f" → 参考完整 DOM 快照了解最新结构")
|
||||
print(f"\n 💡 提示: 如果是登录后分析,对比登录前后的选择器差异")
|
||||
print(f" 尝试在探索页和笔记详情页分别运行此工具")
|
||||
print(f"{'='*60}")
|
||||
|
||||
# 保持浏览器运行供检查
|
||||
print(f"\n 🔄 浏览器保持运行,关闭终端即退出")
|
||||
try:
|
||||
while True:
|
||||
await asyncio.sleep(30)
|
||||
except (KeyboardInterrupt, asyncio.CancelledError):
|
||||
pass
|
||||
finally:
|
||||
try:
|
||||
await conn.close()
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="小红书登录后 DOM 重分析工具"
|
||||
)
|
||||
parser.add_argument("identity_name", help="身份名称 (如 xhs_01)")
|
||||
parser.add_argument("--full-dump", "-f", action="store_true",
|
||||
help="输出完整 DOM 快照到 JSON 文件")
|
||||
parser.add_argument("--explore-url", "-u", default=None,
|
||||
help="自定义分析 URL (默认: /explore)")
|
||||
args = parser.parse_args()
|
||||
|
||||
asyncio.run(analyze(args.identity_name, args.full_dump, args.explore_url))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,264 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
vision_bridge.py — AgentOS 视觉分析桥接层
|
||||
|
||||
职责:
|
||||
将截图发送给 oMLX Qwen2.5-VL 模型,返回结构化页面描述。
|
||||
供原子操作和视觉Recipe使用。
|
||||
|
||||
用法:
|
||||
from vision_bridge import analyze_screenshot, describe_page
|
||||
|
||||
# 分析页面
|
||||
result = analyze_screenshot("screenshot.png", "What buttons are visible?")
|
||||
print(result["text"])
|
||||
|
||||
# 快捷:描述整个页面
|
||||
page = describe_page("screenshot.png")
|
||||
print(page["elements"]) # 列表
|
||||
"""
|
||||
import json, urllib.request, base64, time, os, sys
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
# ── 配置 ──────────────────────────────────────────────
|
||||
OMLX_BASE = "http://127.0.0.1:8000"
|
||||
OMLX_API_KEY = "omlx"
|
||||
OMLX_MODEL = "Qwen2.5-VL-3B-Instruct-8bit"
|
||||
VISION_TIMEOUT = 300 # 秒
|
||||
TEXT_TIMEOUT = 30
|
||||
|
||||
# DashScope (阿里百炼) 配置
|
||||
# ⚠️ 密钥不硬编码入库 — 统一从 agent-local local.yaml 读取(与 AVE 同一来源),
|
||||
# 优先级: 环境变量 DASHSCOPE_API_KEY > agent-local/tools/ave/config/local.yaml → aliyun.api_key
|
||||
DASHSCOPE_MODEL = "qwen-vl-plus" # 性价比高
|
||||
|
||||
|
||||
def get_dashscope_key() -> str:
|
||||
"""读取百炼 API key: 环境变量优先, 兜底 agent-local local.yaml(不入 git)"""
|
||||
env = os.environ.get("DASHSCOPE_API_KEY", "").strip()
|
||||
if env:
|
||||
return env
|
||||
try:
|
||||
p = Path.home() / "workbuddy-agent-os/agent-local/tools/ave/config/local.yaml"
|
||||
if p.exists():
|
||||
import yaml
|
||||
d = yaml.safe_load(p.read_text(encoding="utf-8")) or {}
|
||||
return ((d.get("aliyun", {}) or {}).get("api_key", "") or "").strip()
|
||||
except Exception:
|
||||
pass
|
||||
return ""
|
||||
|
||||
# 代理问题:oMLX 不能用代理
|
||||
_orig_env = {}
|
||||
|
||||
|
||||
def _ensure_no_proxy():
|
||||
"""临时去掉代理(oMLX 不能走代理)"""
|
||||
global _orig_env
|
||||
for k in ["http_proxy", "https_proxy", "HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY"]:
|
||||
_orig_env[k] = os.environ.pop(k, None)
|
||||
|
||||
|
||||
def _restore_proxy():
|
||||
for k, v in _orig_env.items():
|
||||
if v is not None:
|
||||
os.environ[k] = v
|
||||
|
||||
|
||||
# ── 核心 API ─────────────────────────────────────────
|
||||
|
||||
def _omlx_request(messages: list, max_tokens: int = 300, timeout: int = None) -> dict:
|
||||
"""向 oMLX 发送请求(本地)"""
|
||||
_ensure_no_proxy()
|
||||
try:
|
||||
data = json.dumps({
|
||||
"model": OMLX_MODEL,
|
||||
"messages": messages,
|
||||
"max_tokens": max_tokens,
|
||||
}).encode()
|
||||
|
||||
req = urllib.request.Request(f"{OMLX_BASE}/v1/chat/completions", data=data)
|
||||
req.add_header("Content-Type", "application/json")
|
||||
req.add_header("x-api-key", OMLX_API_KEY)
|
||||
|
||||
resp = urllib.request.urlopen(req, timeout=timeout or VISION_TIMEOUT)
|
||||
result = json.loads(resp.read())
|
||||
return {
|
||||
"success": True,
|
||||
"text": result["choices"][0]["message"]["content"],
|
||||
"usage": result.get("usage", {}),
|
||||
}
|
||||
except Exception as e:
|
||||
return {"success": False, "error": str(e), "provider": "omlx"}
|
||||
finally:
|
||||
_restore_proxy()
|
||||
|
||||
|
||||
def _dashscope_request(messages: list, max_tokens: int = 300, timeout: int = None) -> dict:
|
||||
"""向 DashScope (阿里百炼) 发送请求(远程)"""
|
||||
try:
|
||||
import dashscope
|
||||
key = get_dashscope_key()
|
||||
if not key or key == "sk-xxx":
|
||||
return {"success": False, "error": "未配置百炼 API Key (local.yaml → aliyun.api_key)", "provider": "dashscope"}
|
||||
dashscope.api_key = key
|
||||
|
||||
resp = dashscope.MultiModalConversation.call(
|
||||
model=DASHSCOPE_MODEL,
|
||||
messages=messages,
|
||||
max_tokens=max_tokens,
|
||||
timeout=timeout or VISION_TIMEOUT,
|
||||
)
|
||||
if resp.status_code == 200:
|
||||
content = resp.output.choices[0].message.content[0]["text"]
|
||||
return {"success": True, "text": content, "usage": resp.usage, "provider": "dashscope"}
|
||||
return {"success": False, "error": f"DashScope {resp.status_code}: {resp.message}", "provider": "dashscope"}
|
||||
except ImportError:
|
||||
return {"success": False, "error": "dashscope SDK 未安装", "provider": "dashscope"}
|
||||
except Exception as e:
|
||||
return {"success": False, "error": str(e), "provider": "dashscope"}
|
||||
|
||||
|
||||
def _request(messages: list, max_tokens: int = 300, timeout: int = None) -> dict:
|
||||
"""自动选择:oMLX 本地优先 → DashScope 远程兜底"""
|
||||
result = _omlx_request(messages, max_tokens, timeout)
|
||||
if result["success"]:
|
||||
return result
|
||||
# oMLX 失败 → 用 DashScope
|
||||
print(f" [vision] oMLX 不可用 ({result.get('error','?')}), 切换到 DashScope...")
|
||||
result2 = _dashscope_request(messages, max_tokens, timeout)
|
||||
return result2
|
||||
|
||||
|
||||
def _encode_image(image_path: str) -> str:
|
||||
"""读取图片并编码为 base64"""
|
||||
with open(image_path, "rb") as f:
|
||||
return base64.b64encode(f.read()).decode()
|
||||
|
||||
|
||||
# ── 公开接口 ─────────────────────────────────────────
|
||||
|
||||
def analyze_screenshot(image_path: str, question: str,
|
||||
max_tokens: int = 500, timeout: int = None) -> dict:
|
||||
"""
|
||||
分析截图,返回模型回答。
|
||||
|
||||
Args:
|
||||
image_path: 截图路径
|
||||
question: 问题,如 "What buttons are visible?"
|
||||
max_tokens: 最大生成长度
|
||||
timeout: 超时秒数
|
||||
|
||||
Returns:
|
||||
{"success": bool, "text": str, "error": str, "usage": dict}
|
||||
"""
|
||||
if not os.path.isfile(image_path):
|
||||
return {"success": False, "error": f"File not found: {image_path}"}
|
||||
|
||||
img_b64 = _encode_image(image_path)
|
||||
messages = [{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}},
|
||||
{"type": "text", "text": question}
|
||||
]
|
||||
}]
|
||||
|
||||
t0 = time.time()
|
||||
result = _request(messages, max_tokens, timeout)
|
||||
result["elapsed"] = round(time.time() - t0, 1)
|
||||
return result
|
||||
|
||||
|
||||
def describe_page(image_path: str) -> dict:
|
||||
"""
|
||||
分析页面整体结构:布局、按钮、输入框、导航等。
|
||||
|
||||
Returns:
|
||||
{"success": bool, "elements": [...], "error": str}
|
||||
"""
|
||||
result = analyze_screenshot(image_path,
|
||||
"Describe this page in detail. List all: "
|
||||
"1) Navigation/sidebar items and their labels "
|
||||
"2) Buttons and their text "
|
||||
"3) Text fields and input areas "
|
||||
"4) The main content area. "
|
||||
"Be specific about what each element says.",
|
||||
max_tokens=800)
|
||||
|
||||
if not result["success"]:
|
||||
return result
|
||||
|
||||
# 解析成结构化元素(后续可增强)
|
||||
text = result["text"]
|
||||
lines = [l.strip() for l in text.split("\n") if l.strip()]
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"elements": lines,
|
||||
"raw_text": text,
|
||||
"elapsed": result.get("elapsed", 0),
|
||||
"usage": result.get("usage", {}),
|
||||
}
|
||||
|
||||
|
||||
def find_element(image_path: str, target_label: str) -> dict:
|
||||
"""
|
||||
查找指定文字/标签的元素位置。
|
||||
|
||||
Args:
|
||||
image_path: 截图路径
|
||||
target_label: 要找的标签,如 "登录"、"继续"
|
||||
|
||||
Returns:
|
||||
{"success": bool, "found": bool, "description": str}
|
||||
"""
|
||||
result = analyze_screenshot(image_path,
|
||||
f'Is there a "{target_label}" button, link, or text field visible? '
|
||||
f'If yes, describe exactly where it appears on the page and what it looks like.',
|
||||
max_tokens=200)
|
||||
return result
|
||||
|
||||
|
||||
def text_chat(prompt: str, max_tokens: int = 100) -> dict:
|
||||
"""纯文本对话(不走视觉,速度更快)"""
|
||||
messages = [{"role": "user", "content": prompt}]
|
||||
t0 = time.time()
|
||||
result = _request(messages, max_tokens, TEXT_TIMEOUT)
|
||||
result["elapsed"] = round(time.time() - t0, 1)
|
||||
return result
|
||||
|
||||
|
||||
# ── 命令行入口 ───────────────────────────────────────
|
||||
if __name__ == "__main__":
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser(description="Vision Bridge - 截图分析工具")
|
||||
parser.add_argument("image", nargs="?", help="截图路径")
|
||||
parser.add_argument("-q", "--question", default="Describe this page briefly.",
|
||||
help="分析问题")
|
||||
parser.add_argument("-f", "--find", help="查找指定元素")
|
||||
parser.add_argument("--list", action="store_true", help="列出页面所有元素")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.find and args.image:
|
||||
result = find_element(args.image, args.find)
|
||||
elif args.list and args.image:
|
||||
result = describe_page(args.image)
|
||||
elif args.image:
|
||||
result = analyze_screenshot(args.image, args.question)
|
||||
else:
|
||||
# 默认交互模式
|
||||
img = input("截图路径: ").strip() or "/Users/chengzige/.agent-browser/tmp/screenshots/screenshot-1781835723294.png"
|
||||
q = input("问题: ").strip() or "What is on this page?"
|
||||
result = analyze_screenshot(img, q)
|
||||
|
||||
if result.get("success"):
|
||||
print(f"\n=== 回答 (耗时: {result.get('elapsed', '?')}s) ===")
|
||||
print(result.get("text", ""))
|
||||
if "usage" in result:
|
||||
print(f"\nToken: {result['usage']}")
|
||||
else:
|
||||
print(f"\n❌ 失败: {result.get('error', '未知错误')}")
|
||||
sys.exit(1)
|
||||
Reference in New Issue
Block a user