feat(monitor): 抖音 Web 接口客户端 —— 绕开爬虫子进程，直接发 HTTP

移植自 mac-agent-os 的 mediacrawler_adapter：不起子进程、不开页面，用浏览器里那份
登录态直接调抖音 Web 接口。产物键名照抄 store/douyin，所以 ingest 那条链路一个字不用改。

**目前能用的（真环境实测，非推断）**：

    profile/other : 200, 7075 字节  —— 博主主页指标（粉丝/获赞/作品数/昵称）
    aweme/detail  : 200, 45425 字节 —— 单条作品详情（含点赞/评论/收藏/分享）

**目前不能用的：作品列表 `aweme/post`。** 两个互相独立的原因：
  1. 这个接口被抖音单独升级成了真校验：不带 x-tt-argus 回 403「Uifid Not Found」，
     带上 dummy 值回 200 + **空 body**。也就是说「头在不在」骗得过，「真校验」过不了。
     同一套头打 profile/other 和 aweme/detail 都是通的 —— 抖音是挑着接口加保护的，
     挑中的恰好是「批量拉作品列表」这个最敏感的动作。
  2. 改走页面截获也不行：CDP 浏览器打开博主主页会落到「验证码中间页」（当天大量探测的
     代价，过几小时要重测）。

  所以现在的边界是：**已知作品的指标刷新能做，自动发现新作品做不了**。

**排查中控住变量后得到的两条事实**（都写进注释了）：
  · `Accept` / `Accept-Language` / `Referer` 才是主页接口能返回真数据的原因 —— 只有
    UA+client hints+Cookie 时是 200 但仅 121 字节的空壳，补上这三个头变 7074 字节。
    （我先前猜的 sec-ch-ua 不是关键。）
  · 因此 UA 与 client hints 必须**成套地取自同一个浏览器**，所以 BrowserIdentity 一次
    从 CDP 取齐 cookie + UA + hints，而不是各自写死。

「200 + 空 body 必须当场报错」也是刻意写死的：放过去它会在下游变成「这个博主没作品」，
把一次失败伪装成一条正常结果 —— 爬虫那条路正是这么栽的，还被翻译成「账号被封」。

测试 +11：cookie 解析、请求头成套性（含 uifid 缺失/回退）、产物键名与 store 对齐、
以及 _get 的三条失败路径（空 body / 403 带网关原话 / 正常返回）。

mac-agent-os-main/05_tools/10_dashboard/services/EXECUTION_PIPELINE.md

@@ -0,0 +1,213 @@
+# 统一执行管道 v5 — 设计文档
+
+> 目标：所有操作（nurture/collect/login/logout/comment/like）共享同一个执行管道，
+> 从预检 → 确认 → 发送 → 启动验证 → 轮询 → 完成，每步都有状态反馈。
+
+---
+
+## 一、问题现状
+
+之前每次都在各个视图里独立写 fetch → 显示结果，没有统一的执行体验：
+
+| 视图 | 预检 | 确认 | 发送 | 启动验证 | 轮询 | 完成 |
+|:-----|:----:|:----:|:----:|:--------:|:----:|:----:|
+| 养号执行 | ❌ | ✅(confirm) | ❌旧端点 | ❌ | ❌旧API | ❌ |
+| 信息采集 | ❌ | ❌ | ❌旧端点 | ❌ | ❌ | ❌ |
+| 定向评论 | ❌ | ❌ | ❌旧端点 | ❌ | ❌ | ❌ |
+| 收藏点赞 | ❌ | ❌ | ❌旧端点 | ❌ | ❌ | ❌ |
+| 联邦指挥台 | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
+
+每个视图都自己写了一套 fetch → log 的逻辑，代码重复，体验不一致。
+
+---
+
+## 二、统一执行管道设计
+
+### 2.1 前端组件
+
+```javascript
+// 所有执行按钮统一使用这个组件
+const pipeline = createExecutionPipeline(containerElement, {
+  // ── 必需 ──
+  type: 'nurture',           // 操作类型
+  getAccounts: () => [...],  // 获取选中账号的回调
+  
+  // ── 可选 ──
+  title: '养号执行',         // 对话框标题
+  onPlan: (plan) => {},      // 确认前的计划预览（可自定义）
+  commandTemplate: (accounts, params) => {...}, // 自定义命令
+  
+  // ── 回调 ──
+  onStatusChange: (phase, data) => {}, // 阶段变化回调
+});
+```
+
+### 2.2 执行管道阶段
+
+```
+阶段名         前端显示                                  后端动作
+────────────────────────────────────────────────────────────────────
+IDLE           就绪                                      —
+PREFLIGHT      🔍 预检机器状态...                        调用 /api/ops/machines
+PLAN           📋 显示执行计划 + 机器状态                 调用 /api/ops/run (dry_run)
+CONFIRM        🖥️ 弹出确认对话框                         用户确认/取消
+DISPATCHING    📡 正在发送命令到 XX 机器...               调用 /api/ops/run
+DISPATCHED     ✅ 命令已发送 (PID:12345)                 记录PID
+LAUNCH_WAIT    ⏳ 等待进程启动...                         等待2秒
+LAUNCH_CHECK   🔍 检查进程是否启动...                     查进程表/SSH pgrep
+RUNNING        🟢 进程运行中 (5/10轮)                    轮询 /api/ops/status
+BROWSER_CHECK  🌐 浏览器已启动 (3进程)                   查浏览器进程
+COMPLETED      ✅ 执行完成: 8成功/2失败/120s              读取最终结果
+FAILED         ❌ 执行失败: 错误信息
+```
+
+### 2.3 后端 API
+
+```
+POST /api/ops/run
+  → Phase 1-2: dispatch
+  → Phase 3-5: 通过 /api/ops/status 和 /api/ops/machines 轮询
+
+GET /api/ops/status?account=xxx
+  → 返回命令的完整状态 + 阶段
+
+GET /api/ops/machines
+  → 返回所有机器的浏览器/命令/槽位状态
+```
+
+### 2.4 前端执行流程代码
+
+```javascript
+export async function executeCommand(options) {
+  const { type, accounts, params, logEl, statusEl } = options;
+  
+  // Phase 0: Preflight
+  setPhase('preflight', '🔍 预检机器状态...');
+  const machines = await api('/api/ops/machines');
+  
+  // Phase 1: Plan + Confirm
+  setPhase('plan', '📋 生成执行计划...');
+  const plan = await api('/api/ops/run', {type, accounts, params: {...params, dry_run: true}});
+  const confirmed = await showConfirmDialog(plan, machines);
+  if (!confirmed) { setPhase('cancelled', '⏸ 已取消'); return; }
+  
+  // Phase 2: Dispatch
+  setPhase('dispatching', '📡 发送命令...');
+  const result = await api('/api/ops/run', {type, accounts, params});
+  showDispatchResult(result);
+  
+  // Phase 3-5: Monitor
+  await monitorExecution(result, (phase, msg) => {
+    setPhase(phase, msg);
+  });
+}
+```
+
+---
+
+## 三、确认对话框标准
+
+所有执行操作共享同一个确认对话框格式：
+
+```
+┌─────────────────────────────────────┐
+│ 📋 养号执行确认                       │
+│                                      │
+│ 🟢 chengzigedeAir (本机)             │
+│   槽位: 2/3 可用                     │
+│   浏览器: 1 个运行中                  │
+│   ┌──────────────────────────────┐   │
+│   │ 🖥️ douyin_test → 10轮        │   │
+│   │   蓝图: douyin_daily          │   │
+│   │   窗口: 槽位2 (100,0)         │   │
+│   └──────────────────────────────┘   │
+│                                      │
+│ 🔴 5kechengdeAir (远程)              │
+│   不可达（SSH 超时）                  │
+│   ┌──────────────────────────────┐   │
+│   │ ❌ xhs_01 → 无法执行          │   │
+│   └──────────────────────────────┘   │
+│                                      │
+│              [取消]  [确认执行]        │
+└─────────────────────────────────────┘
+```
+
+---
+
+## 四、执行状态反馈标准
+
+每台机器独立显示状态条：
+
+```
+📡 养号执行状态
+────────────────────────────────────────────────
+🖥️ chengzigedeAir     🟢 运行中 (5/10轮, 3成功/2失败)
+☁️ 5kechengdeAir      🔴 SSH 不可达 (已跳过)
+☁️ 7kecheng           ⏳ 浏览器启动中... (8s)
+────────────────────────────────────────────────
+```
+
+---
+
+## 五、集成到所有视图
+
+| 视图 | 按钮 | 替换为统一管道 |
+|:-----|:-----|:--------------|
+| 养号执行 | 执行选中 / 全部启用 | executeCommand('nurture', ...) |
+| 信息采集 | 采集选中 / 全部 | executeCommand('collect', ...) |
+| 信息采集 | 登录选中 | executeCommand('login', ...) |
+| 定向评论 | 执行评论 | executeCommand('comment', ...) |
+| 收藏点赞 | 执行点赞 | executeCommand('like', ...) |
+| 联邦指挥台 | 批量执行 | executeCommand(selectedType, ...) |
+
+---
+
+## 六、v6 队列与强制停止（2026-06-25 新增）
+
+### 6.1 每台机器一个队列
+
+```
+CommandBus 收到新命令
+  ↓
+MachineSession.is_busy?
+  ├── 否 → 立即执行（send_local / send_remote）
+  └── 是 → status=queued，加入 queued_cmds 排队
+              ↓
+        当前命令完成 → poll() 检测到 terminal
+              ↓
+        _start_next() → 出队 → send() → 执行
+```
+
+### 6.2 强制停止
+
+```
+用户点击"停止"
+  ↓
+POST /api/ops/cancel/{run_id}
+  ↓
+session.cancel(cmd)
+  ├── kill(cmd.pid)                    ← 杀 mc run 进程
+  ├── pkill -f {account_id}            ← 清 Camoufox 浏览器
+  └── _start_next()                    ← 启动队列下一条
+```
+
+### 6.3 已实现状态
+
+| 组件 | 状态 |
+|:-----|:----:|
+| MachineSession.queue | ✅ v6 已实现 |
+| cancel() 杀进程+清浏览器 | ✅ v6 已实现 |
+| poll() 完成自动调度下一条 | ✅ v6 已实现 |
+| get_queue_info() API | ✅ v6 已实现 |
+| 前端显示队列/停止按钮 | ⬜ 待实现 |
+
+## 七、实施步骤
+
+| 步骤 | 内容 | 文件 |
+|:-----|:-----|:------|
+| 1 | 创建 `services/execution_pipeline.py` — 后端管道引擎 | 后端 |
+| 2 | 创建 `frontend/src/components/execution-pipeline.js` — 前端管道组件 | 前端 |
+| 3 | 更新 `routes/ops.py` — 对接管道引擎的 phased status | 后端 |
+| 4 | 更新 inline `index.html` — 所有执行按钮用统一管道 | 前端 |
+| 5 | 更新 migrated views — 所有执行按钮用统一管道 | 前端 |
+| 6 | 集成测试 — 所有操作类型通过管道验证 | 测试 |

mac-agent-os-main/05_tools/10_dashboard/services/__init__.py

@@ -0,0 +1 @@
+"""AgentOS Dashboard 服务层"""

mac-agent-os-main/05_tools/10_dashboard/services/account_service.py

@@ -0,0 +1,454 @@
+"""
+account_service.py — 统一账号数据聚合层 (Dashboard)
+
+职责:
+  - 联邦账号中心的数据后端
+  - ORACLE.yaml → 账号归属映射（只读）
+  - 所有机器统一走 guardd HTTP API (port 9090) 查询登录状态
+  - 合并 profile 数据 (profiles.json + fleet_collector)
+
+设计原则:
+  - ORACLE.yaml 是权威来源，账号归属从 ORACLE 读
+  - 所有机器对等，本机不特殊对待
+  - 远程状态有 30 秒 TTL 缓存，避免频繁 HTTP 调用
+  - 离线机器显示 last_known 状态 + offline 标记
+"""
+import json
+import logging
+import os
+import sys
+import time
+from datetime import datetime, timezone
+from typing import Optional
+from services.command_bus import _guardd_api
+from pathlib import Path
+
+logger = logging.getLogger("dashboard.account_service")
+
+_THIS_DIR = Path(__file__).resolve().parent.parent
+AGENT_SYNC = Path(os.environ.get("AGENT_SYNC",
+    str(Path.home() / "workbuddy-agent-os" / "agent-sync")))
+AGENT_LOCAL = Path(os.environ.get("AGENT_LOCAL",
+    str(Path.home() / "workbuddy-agent-os" / "agent-local")))
+GUARDD_PORT = 9090
+
+# ── 本机身份 ──
+try:
+    sys.path.insert(0, str(AGENT_SYNC / "05_tools" / "10_dashboard"))
+    from utils.identity import resolve_hostname
+    HOSTNAME = resolve_hostname()
+except Exception:
+    HOSTNAME = os.uname().nodename
+
+# ── Registry 标签读取 ──
+_REGISTRY_PATH = AGENT_SYNC / "05_tools" / "07_matrix" / "accounts_registry.yaml"
+def _load_registry_tags(account_id: str) -> list:
+    """从 accounts_registry.yaml 读取账号的 tags"""
+    if not _REGISTRY_PATH.exists():
+        return []
+    try:
+        import yaml
+        data = yaml.safe_load(_REGISTRY_PATH.read_text()) or {}
+        for acct in data.get("accounts", []):
+            if acct.get("id") == account_id:
+                return acct.get("tags", [])
+    except Exception:
+        pass
+    return []
+
+
+def _load_tags_cache(account_id: str) -> list:
+    """从本地缓存文件读取远程账号的 tags（由 sync_tags.sh 同步）"""
+    _CACHE_PATH = AGENT_LOCAL / "data" / "account_tags_cache.json"
+    if not _CACHE_PATH.exists():
+        return []
+    try:
+        data = json.loads(_CACHE_PATH.read_text())
+        return data.get(account_id, [])
+    except Exception:
+        return []
+
+
+class AccountService:
+    """账户统一聚合服务"""
+
+    def __init__(self):
+        self._oracle = None
+        self._oracle_loaded = 0
+        # 远程状态缓存 {machine_name: {"accounts": {...}, "cached_at": ts}}
+        self._remote_cache = {}
+        # 远程 profile 缓存 {machine_name: {"profiles": {...}, "cached_at": ts}}
+        self._profile_cache = {}
+        self._cache_ttl = 300  # 秒（5 分钟）
+
+    # ═══════════════════════════════════════════════════════════
+    # 公开 API
+    # ═══════════════════════════════════════════════════════════
+
+    def get_all_accounts(self) -> list[dict]:
+        """获取所有机器上所有账号的统一视图
+
+        Returns:
+            list[dict]: 每个 dict 包含完整账号信息
+        """
+        oracle = self._load_oracle()
+        machines = oracle.get("machines", {})
+        oracle_accounts = oracle.get("accounts", [])
+
+        # 1. 构建 ORACLE 账号→机器映射
+        #    {account_id: {phone, machine, platforms, identity_dir}}
+        oracle_map = {}
+        for entry in oracle_accounts:
+            identity = entry.get("identity", "")
+            phone = entry.get("phone", "")
+            machine = entry.get("machine") or entry.get("assigned_machine", "")
+            platforms = entry.get("platforms", {})
+            # 旧格式兼容：platforms 是 {platform: account_id}
+            for plat, aid in platforms.items():
+                oracle_map[aid] = {
+                    "phone": phone,
+                    "identity": identity,
+                    "machine": machine,
+                    "platform": plat,
+                }
+
+        # 2. 从本机 MatrixManager 获取本地账号的完整信息（含 profiles）
+        local_accounts = self._get_local_accounts_full()
+        local_map = {a["id"]: a for a in local_accounts}
+
+        # 3. 并行查询所有机器的 guardd 状态（含本机）
+        all_machines = set()
+        for entry in oracle_accounts:
+            machine = entry.get("machine") or entry.get("assigned_machine", "")
+            if machine:
+                all_machines.add(machine)
+        # 确保本机也在列表中（用实际 hostname 而非空字符串）
+        local_machine = HOSTNAME or "chengzigedeAir"
+        all_machines.add(local_machine)
+        self._prefetch_remote_statuses(list(all_machines))
+
+        # 4. 聚合所有账号
+        seen = set()
+        results = []
+
+        # 先处理 ORACLE 中定义的所有账号（权威列表）
+        for aid, entry in oracle_map.items():
+            if aid in seen:
+                continue
+            seen.add(aid)
+            machine = entry["machine"]
+            is_local = (machine == HOSTNAME or not machine)
+
+            acct = {
+                "id": aid,
+                "phone": entry["phone"],
+                "platform": entry["platform"],
+                "identity_dir": f"identities/{entry['identity']}" if entry.get("identity") else "",
+                "owner_machine": machine or HOSTNAME,
+                "is_local": is_local,
+            }
+
+            # 登录状态（所有机器对等：统一走 guardd HTTP API）
+            remote_status, status_detail = self._get_remote_account_status(machine, aid)
+            acct["login_status"] = remote_status
+            if status_detail:
+                acct["status_detail"] = status_detail
+            # 补充 profile 数据
+            if is_local:
+                local = local_map.get(aid, {})
+                acct["nickname"] = local.get("nickname", "")
+                acct["fans"] = local.get("fans", "")
+                acct["following"] = local.get("following", "")
+                acct["likes"] = local.get("likes", "")
+                acct["posts"] = local.get("posts", "")
+                acct["avatar"] = local.get("avatar", "")
+                acct["_banned"] = local.get("_banned", False)
+                acct["_identity_dir_exists"] = local.get("_identity_dir_exists", False)
+                acct["notes"] = local.get("notes", "")
+                acct["tags"] = local.get("tags", [])
+                # 从 registry 补充 tags
+                if not acct.get("tags"):
+                    _reg_tags = _load_registry_tags(aid)
+                    if _reg_tags:
+                        acct["tags"] = _reg_tags
+            # 远程账号：通过 guardd HTTP 获取 profile（昵称/粉丝/采集信息），再补 tags 缓存
+            profile = self._get_profile_for_account(aid, machine)
+            acct.update(profile)
+            if not acct.get("tags"):
+                _cache_tags = _load_tags_cache(aid)
+                if _cache_tags:
+                    acct["tags"] = _cache_tags
+
+            results.append(acct)
+
+        # 5. 补全 ORACLE 中没有但本机有的账号（兼容过渡期）
+        for local_acct in local_accounts:
+            aid = local_acct["id"]
+            if aid not in seen:
+                seen.add(aid)
+                local_acct["is_local"] = True
+                local_acct["login_status"] = local_acct.get("_status", "unknown")
+                local_acct["_identity_dir_exists"] = local_acct.get("_identity_dir_exists", False)
+                results.append(local_acct)
+
+        # 排序：按机器→平台→账号ID
+        machine_order = ["chengzigedeAir", "5kechengdeAir", "7kecheng"]
+        def sort_key(a):
+            m = a.get("owner_machine", "")
+            try:
+                mi = machine_order.index(m)
+            except ValueError:
+                mi = 99
+            plat = 0 if a.get("platform") == "douyin" else 1
+            return (mi, plat, a["id"])
+        results.sort(key=sort_key)
+
+        return results
+
+    def get_account_detail(self, account_id: str) -> dict | None:
+        """获取单个账号完整详情（含任务历史、代理信息等）"""
+        results = self.get_all_accounts()
+        for a in results:
+            if a["id"] == account_id:
+                return a
+        return None
+
+    def batch_refresh_status(self, account_ids: list[str] = None) -> dict:
+        """强制刷新指定账号（或全部账号）的状态
+
+        Args:
+            account_ids: 如果为 None，刷新所有账号
+
+        Returns:
+            {account_id: status, ...}
+        """
+        oracle = self._load_oracle()
+        machines = oracle.get("machines", {})
+
+        # 按机器分组
+        by_machine = {}
+        for aid in (account_ids or []):
+            machine = self._get_account_machine(aid)
+            if machine not in by_machine:
+                by_machine[machine] = []
+            by_machine[machine].append(aid)
+
+        results = {}
+        for machine, aids in by_machine.items():
+            if machine == HOSTNAME or not machine:
+                # 本机：用 MatrixManager
+                try:
+                    sys.path.insert(0, str(AGENT_SYNC / "05_tools" / "07_matrix" / "scripts"))
+                    from matrix_mgmt import MatrixManager
+                    mgr = MatrixManager()
+                    for aid in aids:
+                        acct = mgr.get_account(aid)
+                        results[aid] = acct.get("_status", "unknown") if acct else "unknown"
+                except Exception as e:
+                    logger.warning(f"本机刷新状态失败: {e}")
+                    for aid in aids:
+                        results[aid] = "unknown"
+            else:
+                # 远程：清缓存 + 重新查询
+                self._remote_cache.pop(machine, None)
+                statuses = self._query_remote_statuses(machine)
+                for aid in aids:
+                    results[aid] = statuses.get(aid, "offline")
+
+        return results
+
+    # ═══════════════════════════════════════════════════════════
+    # 内部方法
+    # ═══════════════════════════════════════════════════════════
+
+    def _load_oracle(self) -> dict:
+        """读取 ORACLE.yaml（只读）"""
+        now = time.time()
+        if self._oracle and now - self._oracle_loaded < 60:
+            return self._oracle
+        path = AGENT_SYNC / "ORACLE.yaml"
+        if not path.exists():
+            logger.warning("ORACLE.yaml 不存在")
+            self._oracle = {"machines": {}, "accounts": []}
+            self._oracle_loaded = now
+            return self._oracle
+        try:
+            import yaml
+            self._oracle = yaml.safe_load(path.read_text()) or {}
+            self._oracle_loaded = now
+            return self._oracle
+        except Exception as e:
+            logger.warning(f"ORACLE.yaml 解析失败: {e}")
+            return self._oracle or {"machines": {}, "accounts": []}
+
+    def _get_account_machine(self, account_id: str) -> str:
+        """从 ORACLE 查账号归属机器"""
+        oracle = self._load_oracle()
+        for entry in oracle.get("accounts", []):
+            platforms = entry.get("platforms", {})
+            for plat, aid in platforms.items():
+                if aid == account_id:
+                    return entry.get("machine") or entry.get("assigned_machine", "")
+            # 也检查 identity 字段
+            if entry.get("identity", "").replace("phone_", "") == account_id.replace("douyin_", "").replace("xhs_", ""):
+                return entry.get("machine") or entry.get("assigned_machine", "")
+        return HOSTNAME
+
+    def _get_local_accounts_full(self) -> list[dict]:
+        """获取本机账号完整信息（从 MatrixManager）"""
+        try:
+            sys.path.insert(0, str(AGENT_SYNC / "05_tools" / "07_matrix" / "scripts"))
+            from matrix_mgmt import MatrixManager
+            mgr = MatrixManager()
+            accounts = mgr.list_accounts()
+
+            # 合并 profiles.json（通过 guardd API，对等原则）
+            profiles = {}
+            try:
+                data = _guardd_api("GET", "/accounts/profiles")
+                if isinstance(data, dict):
+                    profiles = data.get("profiles", {})
+            except Exception:
+                pass
+
+            for acct in accounts:
+                aid = acct.get("id", "")
+                pid = acct.get("identity_dir", "").replace("identities/", "")
+                profile = profiles.get(aid) or profiles.get(pid) or {}
+                if profile:
+                    acct.setdefault("nickname", profile.get("nickname", ""))
+                    acct.setdefault("fans", profile.get("fans", ""))
+                    acct.setdefault("avatar", profile.get("avatar", ""))
+                    acct.setdefault("following", profile.get("following", ""))
+                    acct.setdefault("likes", profile.get("likes", ""))
+                    acct.setdefault("posts", profile.get("posts", ""))
+                    if profile.get("status") == "banned":
+                        acct["_banned"] = True
+            return accounts
+        except Exception as e:
+            logger.warning(f"获取本机账号失败: {e}")
+            return []
+
+    def _get_remote_account_status(self, machine: str, account_id: str):
+        """获取单个远程账号的登录状态 + 详情
+        Returns: (status_str, detail_str)
+        """
+        statuses, details = self._get_all_remote_statuses(machine)
+        detail = details.get(account_id, "") if details else ""
+        return (statuses.get(account_id, "unknown"), detail)
+
+    def _prefetch_remote_statuses(self, machines: list[str]):
+        """并行预取远程机器状态
+
+        使用 ThreadPoolExecutor 并行查询所有远程机器，
+        避免逐个查询导致的总延迟累加。
+        """
+        if not machines:
+            return
+        now = time.time()
+        # 只查缓存过期或未缓存的机器
+        to_fetch = [m for m in machines
+                    if m not in self._remote_cache
+                    or now - self._remote_cache[m].get("cached_at", 0) >= self._cache_ttl]
+        if not to_fetch:
+            return
+
+        logger.info(f"并行查询远程状态: {to_fetch}")
+        from concurrent.futures import ThreadPoolExecutor, as_completed
+        with ThreadPoolExecutor(max_workers=len(to_fetch)) as pool:
+            future_map = {pool.submit(self._query_remote_statuses, m): m for m in to_fetch}
+            for future in as_completed(future_map):
+                machine = future_map[future]
+                try:
+                    statuses, details = future.result()
+                    self._remote_cache[machine] = {
+                        "accounts": statuses,
+                        "details": details,
+                        "cached_at": now,
+                    }
+                    logger.debug(f"远程状态已缓存: {machine} ({len(statuses)} 个账号)")
+                except Exception as e:
+                    logger.debug(f"远程状态查询失败 {machine}: {e}")
+                    self._remote_cache[machine] = {"accounts": {}, "cached_at": now}
+
+    def _get_all_remote_statuses(self, machine: str):
+        """获取一台远程机器上所有账号的状态 + 详情
+        使用 30 秒 TTL 缓存
+        Returns: (accounts_dict, details_dict)
+        """
+        now = time.time()
+        cached = self._remote_cache.get(machine)
+        if cached and now - cached.get("cached_at", 0) < self._cache_ttl:
+            return (cached.get("accounts", {}), cached.get("details", {}))
+
+        statuses, details = self._query_remote_statuses(machine)
+        self._remote_cache[machine] = {
+            "accounts": statuses,
+            "details": details,
+            "cached_at": now,
+        }
+        return (statuses, details)
+
+    def _query_remote_statuses(self, machine: str):
+        """通过 guardd HTTP API 查询远程机器账号状态 + 详情
+        Returns: (accounts_dict, details_dict)
+        """
+        if not machine:
+            return ({}, {})
+
+        from services.command_bus import _guardd_api
+        # Tailscale 网络抖动容错：加大 timeout + 重试一次（默认 5s 对跨机大响应不够）
+        data = {}
+        for _ in range(2):
+            try:
+                data = _guardd_api("GET", "/accounts/status", machine=machine, timeout=15)
+                if isinstance(data, dict) and data.get("accounts"):
+                    break
+            except Exception:
+                pass
+        if isinstance(data, dict):
+            return (data.get("accounts", {}), data.get("details", {}))
+        return ({}, {})
+
+    def _get_profile_for_account(self, account_id: str, machine: str) -> dict:
+        """通过 guardd HTTP API 获取账号 profile 数据（带机器级缓存）"""
+        try:
+            # 检查缓存
+            now = time.time()
+            cached = self._profile_cache.get(machine)
+            if not cached or now - cached.get("cached_at", 0) >= self._cache_ttl:
+                # Tailscale 网络抖动容错：profiles 响应较大（~12KB，多次 recv），
+                # 默认 5s timeout 易超时 → 加大到 15s + 重试一次
+                data = {}
+                for _ in range(2):
+                    try:
+                        data = _guardd_api("GET", "/accounts/profiles", machine=machine, timeout=15)
+                        if isinstance(data, dict) and data.get("profiles"):
+                            break
+                    except Exception:
+                        pass
+                profiles = data.get("profiles", {}) if isinstance(data, dict) else {}
+                if profiles:
+                    self._profile_cache[machine] = {"profiles": profiles, "cached_at": now}
+                elif machine in self._profile_cache:
+                    # 查询失败：沿用上次成功的缓存（不覆盖为空，避免整片昵称消失）
+                    profiles = self._profile_cache[machine].get("profiles", {})
+                    logger.warning("  ⚠️ %s profiles 查询失败，沿用缓存（%d 项）", machine, len(profiles))
+                else:
+                    # 首次即失败：不写缓存（留待下次请求重试），本次返回空
+                    logger.warning("  ⚠️ %s profiles 查询失败（无缓存可沿用）", machine)
+            else:
+                profiles = cached.get("profiles", {})
+            p = profiles.get(account_id, {})
+            if p and p.get("nickname"):
+                return {
+                    "nickname": p.get("nickname", ""),
+                    "fans": str(p.get("fans", "")),
+                    "avatar": p.get("avatar", ""),
+                    "following": str(p.get("following", "")),
+                    "likes": str(p.get("likes", "")),
+                    "posts": str(p.get("posts", "")),
+                }
+        except Exception:
+            pass
+        return {"nickname": "", "fans": "", "following": "", "likes": "", "posts": "", "avatar": ""}

mac-agent-os-main/05_tools/10_dashboard/services/adapters/__init__.py

@@ -0,0 +1,192 @@
+"""
+抓取适配器基类 — 统一接口
+
+每个平台适配器实现 ScrapeAdapter 接口：
+  - collect_item(target, depth, tool_level) -> dict
+  - collect_user(user_id, limit) -> list[dict]
+  - collect_comments(item_id, limit) -> list[dict]
+  - collect_search(keyword, limit) -> list[dict]
+
+工具降级：
+  Level 1: OpenCLI（仅 CLI）
+  Level 2: OpenCLI → agent-browser（默认）
+  Level 3: OpenCLI → agent-browser → web_crawler
+"""
+
+import asyncio, json, logging, os
+from pathlib import Path
+
+logger = logging.getLogger("dashboard.scrape.adapter")
+
+
+class ScrapeAdapter:
+    """抓取适配器基类"""
+
+    platform = ""       # 子类覆写
+    adapter_name = ""
+
+    # OpenCLI 二进制路径
+    OPENCLI = os.environ.get("OPENCLI_PATH",
+        str(Path.home() / ".workbuddy" / "binaries" / "node" / "versions" / "22.22.2" / "bin" / "opencli"))
+
+    # ── 工具降级 ──
+
+    async def _try_tools(self, tool_level: int, funcs: list) -> tuple:
+        """
+        按降级顺序尝试工具。
+        funcs: [(tool_name, callable), ...]
+        返回: (success, result, used_tool)
+        """
+        tools = [f for f in funcs[:tool_level]]
+        for name, fn in tools:
+            try:
+                result = await fn()
+                if result is not None:
+                    return True, result, name
+            except Exception as e:
+                logger.warning(f"  ⚠️ [{self.platform}] 工具 {name} 失败: {e}")
+        return False, None, tools[-1][0] if tools else "none"
+
+    async def _run_opencli(self, args: list, timeout: int = 60):
+        """
+        执行 OpenCLI 命令并解析输出。
+
+        自动检测输出格式：
+          1. JSON（以 [ 或 { 开头）→ json.loads
+          2. YAML（PyYAML 可用时） → yaml.safe_load
+          3. 兜底 → 简易 key:value 行解析
+        """
+        cmd = [self.OPENCLI] + args
+        proc = await asyncio.create_subprocess_exec(
+            *cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE)
+        try:
+            stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=timeout)
+        except asyncio.TimeoutError:
+            proc.kill()
+            raise TimeoutError(f"OpenCLI 超时 ({timeout}s): {' '.join(args[:4])}")
+        if proc.returncode != 0:
+            err = stderr.decode().strip() or stdout.decode()[:200]
+            raise RuntimeError(f"OpenCLI 错误 ({proc.returncode}): {err}")
+        output = stdout.decode().strip()
+        if not output:
+            return {}
+        return self._parse_output(output)
+
+    def _parse_output(self, text: str):
+        """
+        解析 OpenCLI 输出，自动检测格式。
+        返回 list[dict] 或 dict。
+        """
+        text = text.strip()
+        if not text:
+            return {}
+
+        # 1. JSON 检测（以 [ 或 { 开头）
+        if text.startswith("[") or text.startswith("{"):
+            try:
+                return json.loads(text)
+            except json.JSONDecodeError:
+                logger.warning("  ⚠️ OpenCLI JSON 解析失败，尝试 YAML 兜底")
+
+        # 2. YAML 解析（PyYAML 在 requirements.txt 中）
+        try:
+            import yaml
+            parsed = yaml.safe_load(text)
+            if parsed is not None:
+                return parsed
+        except ImportError:
+            logger.debug("  PyYAML 未安装，跳过 YAML 解析")
+        except Exception as e:
+            logger.warning(f"  ⚠️ YAML 解析失败: {e}")
+
+        # 3. 兜底：简易 key:value 行解析（无依赖）
+        return self._parse_lines(text)
+
+    def _parse_lines(self, text: str) -> list:
+        """
+        最后一层兜底：按行解析 key:value 格式输出。
+        适用于没有 yaml 库或 JSON/YAML 都失败的情况。
+        """
+        items = []
+        current = {}
+        # 尝试识别表格/列表分隔符
+        in_top_comments = False
+        comment_buffer = []
+
+        def _maybe_flush():
+            """将当前记录推入 items"""
+            nonlocal current, comment_buffer, in_top_comments
+            # 处理缓存的评论
+            if comment_buffer:
+                current["top_comments"] = comment_buffer
+                comment_buffer = []
+            if current and any(v for v in current.values()):
+                items.append(current)
+            current = {}
+            in_top_comments = False
+
+        for line in text.split("\n"):
+            line_stripped = line.strip()
+            if not line_stripped:
+                _maybe_flush()
+                continue
+            # 分隔线（--- 或 =====）
+            if line_stripped.startswith("---") or line_stripped.startswith("==="):
+                _maybe_flush()
+                continue
+            # key: value 行
+            if ": " in line_stripped:
+                k, v = line_stripped.split(": ", 1)
+                k = k.strip()
+                v = v.strip().strip("'\"")
+                # 特殊字段: top_comments
+                if k == "top_comments":
+                    _maybe_flush()
+                    in_top_comments = True
+                    comment_buffer = []
+                elif k in ("nickname", "text", "digg_count") and in_top_comments:
+                    # 评论块中的字段
+                    if comment_buffer and k in comment_buffer[-1]:
+                        # 同 key 出现多次 → 新评论
+                        pass
+                    entry = comment_buffer[-1] if comment_buffer else {}
+                    entry[k] = v
+                    if not comment_buffer:
+                        comment_buffer.append(entry)
+                    else:
+                        # 如果当前 entry 已经有这个 key 了，说明是新记录
+                        if k in comment_buffer[-1] and comment_buffer[-1][k] != v:
+                            comment_buffer.append({k: v})
+                        else:
+                            comment_buffer[-1][k] = v
+                elif in_top_comments:
+                    # 在 top_comments 块里但字段不在三个已知中 → 只是浮层
+                    pass
+                else:
+                    current[k] = v
+            # 列表项（以 - 开头）
+            elif line_stripped.startswith("- "):
+                _maybe_flush()
+                val = line_stripped[2:].strip()
+                if val:
+                    current = {"_value": val}
+                    items.append(current)
+                    current = {}
+
+        _maybe_flush()
+        return items if items else [current] if current else []
+
+    # ── 接口方法（子类覆写）──
+
+    async def collect_item(self, target: str, depth: str = "light",
+                           tool_level: int = 2) -> dict:
+        raise NotImplementedError
+
+    async def collect_user(self, user_id: str, limit: int = 20) -> list[dict]:
+        raise NotImplementedError
+
+    async def collect_comments(self, item_id: str, limit: int = 20) -> list[dict]:
+        raise NotImplementedError
+
+    async def collect_search(self, keyword: str, limit: int = 20) -> list[dict]:
+        raise NotImplementedError

mac-agent-os-main/05_tools/10_dashboard/services/adapters/bilibili_scrape.py

@@ -0,0 +1,134 @@
+"""
+bilibili_scrape.py — B站抓取适配器
+
+工具：OpenCLI bilibili → agent-browser（Chrome DOM提取）
+"""
+import asyncio, json, logging
+from services.adapters import ScrapeAdapter
+
+logger = logging.getLogger("dashboard.scrape.bilibili")
+
+
+class BilibiliScrapeAdapter(ScrapeAdapter):
+    platform = "bilibili"
+    adapter_name = "bilibili_scrape"
+
+    async def collect_user(self, user_id: str, limit: int = 20) -> list[dict]:
+        """抓取指定 UP 主的视频列表"""
+        results = []
+        ok, data, tool = await self._try_tools(2, [
+            ("opencli", lambda: self._opencli_user_videos(user_id, limit)),
+        ])
+        if ok and data:
+            for item in (data if isinstance(data, list) else [data]):
+                results.append(self._to_schema(item))
+        return results
+
+    async def collect_item(self, target: str, depth: str = "light",
+                           tool_level: int = 2) -> dict:
+        """抓取单条视频详情"""
+        bvid = self._extract_bvid(target)
+        if not bvid:
+            return {"error": f"无法解析 BVID: {target}"}
+        ok, data, tool = await self._try_tools(tool_level, [
+            ("opencli", lambda: self._opencli_video(bvid)),
+        ])
+        if ok and data:
+            return self._to_schema(data)
+        return {"error": "所有工具均无法抓取"}
+
+    async def collect_comments(self, item_id: str, limit: int = 20) -> list[dict]:
+        """抓取视频评论"""
+        ok, data, tool = await self._try_tools(2, [
+            ("opencli", lambda: self._opencli_comments(item_id, limit)),
+        ])
+        if ok and data:
+            return [
+                {"author": c.get("nickname", c.get("user", "")),
+                 "text": c.get("text", c.get("content", "")),
+                 "likes": c.get("likes", c.get("like_count", 0))}
+                for c in (data if isinstance(data, list) else [data])
+            ]
+        return []
+
+    async def collect_search(self, keyword: str, limit: int = 20) -> list[dict]:
+        """搜索视频"""
+        results = []
+        ok, data, tool = await self._try_tools(2, [
+            ("opencli", lambda: self._opencli_search(keyword, limit)),
+        ])
+        if ok and data:
+            for item in (data if isinstance(data, list) else [data]):
+                results.append(self._to_schema(item))
+        return results
+
+    # ── OpenCLI 调用 ──
+
+    async def _opencli_user_videos(self, mid: str, limit: int):
+        return await self._run_opencli([
+            "bilibili", "user-videos", mid,
+            "--limit", str(limit), "-f", "json"
+        ])
+
+    async def _opencli_video(self, bvid: str):
+        return await self._run_opencli([
+            "bilibili", "video", bvid, "-f", "json"
+        ])
+
+    async def _opencli_comments(self, bvid: str, limit: int):
+        return await self._run_opencli([
+            "bilibili", "comments", bvid,
+            "--limit", str(limit), "-f", "json"
+        ])
+
+    async def _opencli_search(self, keyword: str, limit: int):
+        return await self._run_opencli([
+            "bilibili", "search", keyword,
+            "--limit", str(limit), "-f", "json"
+        ])
+
+    # ── URL 解析 ──
+
+    def _extract_bvid(self, target: str) -> str:
+        """从 URL 或文本中提取 BVID"""
+        import re
+        # bilibili.com/video/BVxxx
+        m = re.search(r'bilibili\.com/video/([a-zA-Z0-9]+)', target)
+        if m:
+            return m.group(1)
+        # 纯 BVID（BV 开头）
+        if target.startswith("BV") and len(target) >= 10:
+            return target
+        return ""
+
+    # ── 统一 Schema ──
+
+    def _to_schema(self, raw: dict) -> dict:
+        return {
+            "platform": "bilibili",
+            "item_id": raw.get("bvid", raw.get("aid", "")),
+            "url": raw.get("url", f"https://www.bilibili.com/video/{raw.get('bvid', '')}"),
+            "title": raw.get("title", ""),
+            "author_name": raw.get("author", raw.get("up", raw.get("owner_name", ""))),
+            "author_id": raw.get("mid", raw.get("owner_mid", "")),
+            "published_at": raw.get("pubdate", raw.get("pub_time", raw.get("created", ""))),
+            "text_content": raw.get("desc", raw.get("description", "")),
+            "tags": raw.get("tags", []),
+            "stats": {
+                "likes": raw.get("likes", raw.get("stat_like", raw.get("like_count", 0))),
+                "comments": raw.get("comments", raw.get("stat_reply", raw.get("reply_count", 0))),
+                "shares": raw.get("shares", raw.get("stat_share", raw.get("share_count", 0))),
+                "views": raw.get("views", raw.get("stat_view", raw.get("play_count", 0))),
+            },
+            "extra": {
+                "bvid": raw.get("bvid", ""),
+                "mid": raw.get("mid", raw.get("owner_mid", "")),
+                "duration": raw.get("duration", raw.get("v_duration", 0)),
+            },
+            "media": [{
+                "type": "video",
+                "url": raw.get("play_url", ""),
+                "duration": raw.get("duration", 0),
+            }] if raw.get("play_url") else [],
+            "comments": [],
+        }

mac-agent-os-main/05_tools/10_dashboard/services/adapters/browser_helpers.py

@@ -0,0 +1,191 @@
+"""
+browser_helpers.py — 抓取系统的浏览器辅助函数 v2
+
+用途：
+  当 OpenCLI 抓取失败时，通过真实 Chrome 浏览器降级提取数据。
+
+设计原则：
+  ⛔ 绝不使用 Camoufox（养号专用，Firefox 内核 + 特殊指纹）
+  ✅ 使用 Playwright 启动真实 Chrome（Chromium 内核，正常指纹）
+  ✅ 支持 headless 和 headed 两种模式
+  ✅ 浏览器实例复用，避免反复启动
+
+调用方式：
+  from services.adapters.browser_helpers import page_evaluate, page_extract
+
+  # 执行 JS 提取数据
+  data = await page_evaluate("https://www.douyin.com/video/xxx", js_code)
+
+  # 按 CSS 选择器提取
+  texts = await page_extract("https://www.douyin.com/video/xxx", ["h1.title", ".desc"])
+"""
+import asyncio, json, logging, os
+from pathlib import Path
+from typing import Optional
+
+logger = logging.getLogger("dashboard.scrape.browser")
+
+# ── 浏览器单例（复用实例，避免反复启动）──
+
+_browser = None
+_lock = asyncio.Lock()
+_USER_DATA_DIR = os.environ.get(
+    "SCRAPE_CHROME_USER_DATA",
+    str(Path.home() / "workbuddy-agent-os" / "agent-local" / "runtime" / "scrape_chrome_profile"),
+)
+
+# Chrome 可执行路径
+_CHROME_PATHS = [
+    "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",  # macOS 标准路径
+    "/Applications/Chromium.app/Contents/MacOS/Chromium",
+]
+_CHROME_PATH = None
+for p in _CHROME_PATHS:
+    if Path(p).exists():
+        _CHROME_PATH = p
+        break
+
+
+async def _get_browser(headless: bool = True):
+    """
+    获取/创建浏览器实例（单例）。
+    使用 Playwright 启动真实 Chrome，而非 Camoufox。
+    """
+    global _browser
+    async with _lock:
+        if _browser is not None and _browser.is_connected():
+            return _browser
+        try:
+            from playwright.async_api import async_playwright
+        except ImportError:
+            logger.error("Playwright 未安装，无法使用浏览器降级")
+            # 尝试安装
+            try:
+                proc = await asyncio.create_subprocess_exec(
+                    *["playwright", "install", "chromium"],
+                    stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE
+                )
+                await proc.communicate()
+                from playwright.async_api import async_playwright
+            except Exception:
+                raise RuntimeError("Playwright 不可用，无法启动浏览器降级")
+
+        p = await async_playwright().start()
+        launch_kwargs = {
+            "headless": headless,
+            "args": [
+                "--disable-blink-features=AutomationControlled",
+                "--no-sandbox",
+                "--disable-dev-shm-usage",
+                "--disable-gpu",
+                "--window-size=1280,720",
+            ],
+        }
+        if _CHROME_PATH:
+            launch_kwargs["executable_path"] = _CHROME_PATH
+            logger.info(f"使用系统 Chrome: {_CHROME_PATH}")
+        else:
+            logger.info("使用 Playwright 内置 Chromium")
+
+        try:
+            _browser = await p.chromium.launch(**launch_kwargs)
+            logger.info("✅ 浏览器降级: Chrome 启动成功")
+        except Exception as e:
+            logger.error(f"❌ 浏览器启动失败: {e}")
+            await p.stop()
+            raise
+        return _browser
+
+
+async def page_evaluate(url: str, js_code: str,
+                        timeout: int = 30000, headless: bool = True) -> dict:
+    """
+    打开 URL，执行 JavaScript 提取数据。
+
+    参数:
+      url:       目标页面地址
+      js_code:   要执行的 JS 代码（字符串形式的箭头函数或普通函数）
+      timeout:   页面加载超时（毫秒）
+      headless:  是否无头模式
+
+    返回:
+      字典。JS 返回值的 JSON 表示。失败时返回 {}。
+    """
+    browser = await _get_browser(headless=headless)
+    context = None
+    page = None
+    try:
+        context = await browser.new_context(
+            user_agent=(
+                "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
+                "AppleWebKit/537.36 (KHTML, like Gecko) "
+                "Chrome/125.0.0.0 Safari/537.36"
+            ),
+            viewport={"width": 1280, "height": 720},
+            locale="zh-CN",
+        )
+        page = await context.new_page()
+        await page.goto(url, wait_until="domcontentloaded", timeout=timeout)
+        # 等待页面稳定
+        await page.wait_for_load_state("networkidle", timeout=timeout)
+        # 额外等待，让动态渲染完成
+        await asyncio.sleep(1)
+
+        # 执行 JS
+        result = await page.evaluate(js_code)
+        if result is None:
+            logger.warning(f"  ⚠️ page_evaluate JS 返回 null: {url[:60]}")
+            return {}
+
+        # 序列化确保可 JSON 返回
+        if isinstance(result, (dict, list)):
+            return result
+        return {"_value": str(result)}
+
+    except Exception as e:
+        logger.warning(f"  ⚠️ page_evaluate 失败 [{url[:60]}]: {e}")
+        return {}
+    finally:
+        if page:
+            await page.close()
+        if context:
+            await context.close()
+
+
+async def page_extract(url: str, selectors: list,
+                       timeout: int = 30000, headless: bool = True) -> dict:
+    """
+    打开 URL，按 CSS 选择器提取 DOM 文本。
+
+    参数:
+      url:       目标页面
+      selectors: CSS 选择器列表，如 ["h1", ".video-title", "[data-e2e=comment-count]"]
+      timeout:   超时（毫秒）
+      headless:  是否无头
+
+    返回:
+      {selector: [text1, text2, ...], ...}
+    """
+    js = f"""() => {{
+        const results = {{}};
+        const sel = {json.dumps(selectors)};
+        for (const s of sel) {{
+            const els = document.querySelectorAll(s);
+            results[s] = Array.from(els).map(el => el.textContent.trim()).filter(t => t);
+        }}
+        return results;
+    }}"""
+    return await page_evaluate(url, js, timeout=timeout, headless=headless)
+
+
+async def close_browser():
+    """关闭浏览器实例（清理时调用）"""
+    global _browser
+    async with _lock:
+        if _browser:
+            try:
+                await _browser.close()
+                logger.info("浏览器降级: Chrome 已关闭")
+            except Exception as e:
+                logger.warning(f"关闭浏览器时出错: {e}")
+            _browser = None

mac-agent-os-main/05_tools/10_dashboard/services/adapters/douyin_scrape.py

@@ -0,0 +1,181 @@
+"""
+douyin_scrape.py — 抖音抓取适配器
+
+工具：OpenCLI douyin → agent-browser（Chrome DOM提取）
+"""
+import asyncio, json, logging
+from services.adapters import ScrapeAdapter
+
+logger = logging.getLogger("dashboard.scrape.douyin")
+
+
+class DouyinScrapeAdapter(ScrapeAdapter):
+    platform = "douyin"
+    adapter_name = "douyin_scrape"
+
+    # ── 公开接口 ──
+
+    async def collect_user(self, user_id: str, limit: int = 20) -> list[dict]:
+        """抓取指定用户的视频列表（含评论）"""
+        results = []
+        ok, data, tool = await self._try_tools(2, [
+            ("opencli", lambda: self._opencli_user_videos(user_id, limit)),
+            ("agent-browser", lambda: self._browser_user_profile(user_id, limit)),
+        ])
+        if ok and data:
+            for item in data:
+                results.append(self._to_schema(item))
+        return results
+
+    async def collect_item(self, target: str, depth: str = "light",
+                           tool_level: int = 2) -> dict:
+        """抓取单条视频详情"""
+        aweme_id = self._extract_aweme_id(target)
+        if not aweme_id:
+            return {"error": f"无法解析视频ID: {target}"}
+
+        ok, data, tool = await self._try_tools(tool_level, [
+            ("opencli", lambda: self._opencli_stats(aweme_id)),
+            ("agent-browser", lambda: self._browser_video_page(aweme_id)),
+        ])
+        if ok and data:
+            return self._to_schema(data)
+        return {"error": "所有工具均无法抓取"}
+
+    async def collect_comments(self, item_id: str, limit: int = 20) -> list[dict]:
+        """抓取评论区（需先有作者 sec_uid，目前返回空表）"""
+        return []
+
+    async def collect_search(self, keyword: str, limit: int = 20) -> list[dict]:
+        """搜索视频"""
+        results = []
+        ok, data, tool = await self._try_tools(2, [
+            ("opencli", lambda: self._opencli_search(keyword, limit)),
+        ])
+        if ok and data:
+            for item in data:
+                results.append(self._to_schema(item))
+        return results
+
+    # ── OpenCLI 调用 ──
+
+    async def _opencli_user_videos(self, sec_uid: str, limit: int):
+        data = await self._run_opencli([
+            "douyin", "user-videos", sec_uid,
+            "--limit", str(limit), "--with_comments", "true",
+            "-f", "json"
+        ])
+        # OpenCLI JSON 输出直接是数组
+        if isinstance(data, list):
+            return data
+        if isinstance(data, dict) and "items" in data:
+            return data["items"]
+        # YAML 格式回退
+        if isinstance(data, list) and len(data) > 0 and "aweme_id" in data[0]:
+            return data
+        return []
+
+    async def _opencli_stats(self, aweme_id: str):
+        try:
+            return await self._run_opencli([
+                "douyin", "stats", aweme_id, "-f", "json"
+            ])
+        except Exception as e:
+            logger.warning(f"  ⚠️ douyin stats 失败: {e}")
+            return None
+
+    async def _opencli_search(self, keyword: str, limit: int):
+        # OpenCLI 暂无抖音搜索命令，返回空
+        return []
+
+    # ── agent-browser 降级 ──
+
+    async def _browser_video_page(self, aweme_id: str):
+        """通过 Chrome 打开视频页提取数据"""
+        from services.adapters.browser_helpers import page_evaluate
+        url = f"https://www.douyin.com/video/{aweme_id}"
+        js = """() => {
+            const body = document.body.innerText || '';
+            const title = document.title || '';
+            const url = window.location.href;
+            const uidM = body.match(/抖音号[：:]\\s*(\\S+)/);
+            const nickM = body.match(/@(\\S+)/);
+            function extractNum(label) {
+                var m = body.match(new RegExp('(\\\\d+(?:\\\\.\\\\d+)?[万w]?)\\\\s*' + label));
+                if (m) return m[1];
+                m = body.match(new RegExp(label + '\\\\s*(\\\\d+(?:\\\\.\\\\d+)?[万w]?)'));
+                return m ? m[1] : null;
+            }
+            return {
+                aweme_id: '"' + url.match(/\\/video\\/(\\d+)/)?.[1] + '"',
+                title: title,
+                author_nickname: nickM ? nickM[1] : '',
+                douyin_id: uidM ? uidM[1] : '',
+                digg_count: extractNum('获赞') || '',
+                fans: extractNum('粉丝') || '',
+                following: extractNum('关注') || '',
+            };
+        }"""
+        data = await page_evaluate(url, js)
+        return data
+
+    async def _browser_user_profile(self, user_id: str, limit: int):
+        return []
+
+    # ── URL 解析 ──
+
+    def _extract_aweme_id(self, target: str) -> str:
+        """从 URL 或文本中提取 aweme_id"""
+        import re
+        # 视频页: douyin.com/video/{id}
+        m = re.search(r'douyin\.com/video/(\d+)', target)
+        if m:
+            return m.group(1)
+        # iesdouyin.com/share/video/{id}
+        m = re.search(r'iesdouyin\.com/share/video/(\d+)', target)
+        if m:
+            return m.group(1)
+        # 纯数字
+        if target.isdigit():
+            return target
+        return ""
+
+    # ── 统一 Schema 转换 ──
+
+    def _to_schema(self, raw: dict) -> dict:
+        """OpenCLI 原始数据 → 统一 Schema"""
+        return {
+            "platform": "douyin",
+            "item_id": raw.get("aweme_id", ""),
+            "url": f"https://www.douyin.com/video/{raw.get('aweme_id', '')}",
+            "title": raw.get("title", ""),
+            "author_name": raw.get("author_nickname", raw.get("nickname", "")),
+            "author_id": raw.get("author_id", raw.get("douyin_id", "")),
+            "published_at": raw.get("create_time", raw.get("published_at", "")),
+            "text_content": raw.get("title", ""),
+            "tags": [],
+            "stats": {
+                "likes": raw.get("digg_count", raw.get("likes", 0)),
+                "comments": raw.get("comment_count", 0),
+                "shares": raw.get("share_count", 0),
+                "views": raw.get("play_count", raw.get("views", 0)),
+            },
+            "extra": {
+                "sec_uid": raw.get("sec_uid", ""),
+                "duration": raw.get("duration", 0),
+                "status": raw.get("status", ""),
+            },
+            "media": [{
+                "type": "video",
+                "url": raw.get("play_url", ""),
+                "duration": raw.get("duration", 0),
+            }] if raw.get("play_url") else [],
+            "comments": [
+                {
+                    "author": c.get("nickname", ""),
+                    "text": c.get("text", ""),
+                    "likes": c.get("digg_count", 0),
+                }
+                for c in (raw.get("top_comments") or [])
+            ] if raw.get("top_comments") else [],
+        }

mac-agent-os-main/05_tools/10_dashboard/services/adapters/web_scrape.py

@@ -0,0 +1,115 @@
+"""
+web_scrape.py — 通用网页抓取适配器
+
+工具：web_crawler skill → agent-browser（Chrome DOM提取）
+
+用途：抓取非平台特定网页内容（文章、博客、新闻等）
+"""
+import asyncio, json, logging
+from services.adapters import ScrapeAdapter
+
+logger = logging.getLogger("dashboard.scrape.web")
+
+
+class WebScrapeAdapter(ScrapeAdapter):
+    platform = "web"
+    adapter_name = "web_scrape"
+
+    async def collect_item(self, target: str, depth: str = "light",
+                           tool_level: int = 2) -> dict:
+        """抓取单条网页内容"""
+        ok, data, tool = await self._try_tools(tool_level, [
+            ("web_crawler", lambda: self._web_crawl(target)),
+            ("agent-browser", lambda: self._browser_extract(target)),
+        ])
+        if ok and data:
+            return self._to_schema(data)
+        return {"error": "所有工具均无法抓取"}
+
+    async def collect_user(self, user_id: str, limit: int = 20) -> list[dict]:
+        return []
+
+    async def collect_comments(self, item_id: str, limit: int = 20) -> list[dict]:
+        return []
+
+    async def collect_search(self, keyword: str, limit: int = 20) -> list[dict]:
+        return []
+
+    # ── web_crawler 调用 ──
+
+    async def _web_crawl(self, url: str):
+        """通过 web_crawler skill 爬取网页"""
+        try:
+            from services.crawl_service import crawl_url
+            result = await crawl_url(url)
+            return result
+        except ImportError:
+            logger.warning("  ⚠️ web_crawler skill 未加载，尝试 Crawl4AI 直调")
+            return await self._crawl4ai_fallback(url)
+        except Exception as e:
+            logger.warning(f"  ⚠️ web_crawler 失败: {e}")
+            return await self._crawl4ai_fallback(url)
+
+    async def _crawl4ai_fallback(self, url: str) -> dict:
+        """通过 Crawl4AI 直接爬取"""
+        try:
+            import subprocess, json
+            # 使用 crawl4ai 命令行
+            proc = await asyncio.create_subprocess_exec(
+                *["python3", "-m", "crawl4ai", url, "--json"],
+                stdout=asyncio.subprocess.PIPE,
+                stderr=asyncio.subprocess.PIPE,
+            )
+            stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=30)
+            if stdout:
+                result = json.loads(stdout.decode().strip())
+                return result
+        except Exception as e:
+            logger.warning(f"  ⚠️ Crawl4AI 降级失败: {e}")
+        return None
+
+    async def _browser_extract(self, url: str) -> dict:
+        """通过 Chrome 浏览器提取页面内容"""
+        from services.adapters.browser_helpers import page_extract
+        selectors = ["article", "main", ".content", ".post-content", ".article-content",
+                     "[class*='content']", "body"]
+        data = await page_extract(url, selectors, headless=True)
+        if data:
+            # 提取最长文本块作为正文
+            texts = []
+            for sel, items in data.items():
+                for t in items:
+                    if len(t) > 100:
+                        texts.append(t)
+            content = max(texts, key=len) if texts else list(data.values())[0][0] if any(data.values()) else ""
+            return {
+                "url": url,
+                "title": (await self._browser_title(url)).get("title", ""),
+                "content": content,
+            }
+        return {}
+
+    async def _browser_title(self, url: str) -> dict:
+        """提取页面标题"""
+        from services.adapters.browser_helpers import page_evaluate
+        js = "() => ({title: document.title})"
+        return await page_evaluate(url, js, headless=True)
+
+    # ── 统一 Schema ──
+
+    def _to_schema(self, raw: dict) -> dict:
+        return {
+            "platform": "web",
+            "item_id": raw.get("url", ""),
+            "url": raw.get("url", ""),
+            "title": raw.get("title", ""),
+            "author_name": raw.get("author", raw.get("site_name", "")),
+            "author_id": "",
+            "published_at": raw.get("published_at", raw.get("date", "")),
+            "text_content": raw.get("content", raw.get("text", raw.get("markdown", ""))),
+            "tags": raw.get("tags", []),
+            "stats": {},
+            "extra": {},
+            "media": [],
+            "comments": [],
+        }

mac-agent-os-main/05_tools/10_dashboard/services/adapters/xhs_scrape.py

@@ -0,0 +1,130 @@
+"""
+xhs_scrape.py — 小红书抓取适配器
+
+工具：OpenCLI xiaohongshu → agent-browser
+"""
+import asyncio, json, logging
+from services.adapters import ScrapeAdapter
+
+logger = logging.getLogger("dashboard.scrape.xhs")
+
+
+class XhsScrapeAdapter(ScrapeAdapter):
+    platform = "xiaohongshu"
+    adapter_name = "xhs_scrape"
+
+    async def collect_user(self, user_id: str, limit: int = 20) -> list[dict]:
+        """抓取指定用户的笔记列表"""
+        results = []
+        ok, data, tool = await self._try_tools(2, [
+            ("opencli", lambda: self._opencli_user_notes(user_id, limit)),
+        ])
+        if ok and data:
+            for item in (data if isinstance(data, list) else [data]):
+                results.append(self._to_schema(item))
+        return results
+
+    async def collect_item(self, target: str, depth: str = "light",
+                           tool_level: int = 2) -> dict:
+        """抓取单篇笔记详情"""
+        note_id = self._extract_note_id(target)
+        if not note_id:
+            return {"error": f"无法解析笔记ID: {target}"}
+        ok, data, tool = await self._try_tools(tool_level, [
+            ("opencli", lambda: self._opencli_note(note_id)),
+        ])
+        if ok and data:
+            return self._to_schema(data)
+        return {"error": "所有工具均无法抓取"}
+
+    async def collect_comments(self, item_id: str, limit: int = 20) -> list[dict]:
+        """抓取笔记评论"""
+        ok, data, tool = await self._try_tools(2, [
+            ("opencli", lambda: self._opencli_comments(item_id, limit)),
+        ])
+        if ok and data:
+            return [
+                {"author": c.get("nickname", ""), "text": c.get("text", ""),
+                 "likes": c.get("likes", 0)}
+                for c in (data if isinstance(data, list) else [data])
+            ]
+        return []
+
+    async def collect_search(self, keyword: str, limit: int = 20) -> list[dict]:
+        """搜索笔记"""
+        results = []
+        ok, data, tool = await self._try_tools(2, [
+            ("opencli", lambda: self._opencli_search(keyword, limit)),
+        ])
+        if ok and data:
+            for item in (data if isinstance(data, list) else [data]):
+                results.append(self._to_schema(item))
+        return results
+
+    # ── OpenCLI 调用 ──
+
+    async def _opencli_user_notes(self, user_id: str, limit: int):
+        return await self._run_opencli([
+            "xiaohongshu", "user", user_id,
+            "--limit", str(limit), "-f", "json"
+        ])
+
+    async def _opencli_note(self, note_id: str):
+        return await self._run_opencli([
+            "xiaohongshu", "note", note_id, "-f", "json"
+        ])
+
+    async def _opencli_comments(self, note_id: str, limit: int):
+        return await self._run_opencli([
+            "xiaohongshu", "comments", note_id,
+            "--limit", str(limit), "-f", "json"
+        ])
+
+    async def _opencli_search(self, keyword: str, limit: int):
+        return await self._run_opencli([
+            "xiaohongshu", "search", keyword,
+            "--limit", str(limit), "-f", "json"
+        ])
+
+    # ── URL 解析 ──
+
+    def _extract_note_id(self, target: str) -> str:
+        import re
+        m = re.search(r'xiaohongshu\.com/explore/([a-f0-9]+)', target)
+        if m:
+            return m.group(1)
+        m = re.search(r'xiaohongshu\.com/discovery/item/([a-f0-9]+)', target)
+        if m:
+            return m.group(1)
+        if target.isalnum() and len(target) >= 10:
+            return target
+        return ""
+
+    # ── 统一 Schema ──
+
+    def _to_schema(self, raw: dict) -> dict:
+        return {
+            "platform": "xiaohongshu",
+            "item_id": raw.get("note_id", raw.get("id", "")),
+            "url": raw.get("url", f"https://www.xiaohongshu.com/explore/{raw.get('note_id', '')}"),
+            "title": raw.get("title", ""),
+            "author_name": raw.get("author", raw.get("nickname", raw.get("user_name", ""))),
+            "author_id": raw.get("user_id", raw.get("author_id", "")),
+            "published_at": raw.get("create_time", raw.get("time", "")),
+            "text_content": raw.get("desc", raw.get("description", raw.get("text", ""))),
+            "tags": raw.get("tags", []),
+            "stats": {
+                "likes": raw.get("likes", raw.get("liked_count", 0)),
+                "comments": raw.get("comments", raw.get("comment_count", 0)),
+                "shares": raw.get("shares", raw.get("share_count", 0)),
+                "views": raw.get("views", 0),
+            },
+            "extra": {
+                "note_id": raw.get("note_id", ""),
+                "user_id": raw.get("user_id", ""),
+            },
+            "media": [
+                {"type": "image", "url": u} for u in (raw.get("images") or [])
+            ] if raw.get("images") else [],
+            "comments": [],
+        }

mac-agent-os-main/05_tools/10_dashboard/services/adapters/zhihu_scrape.py

@@ -0,0 +1,115 @@
+"""
+zhihu_scrape.py — 知乎抓取适配器
+
+工具：OpenCLI zhihu → agent-browser（Chrome DOM提取）
+"""
+import asyncio, json, logging, re
+from services.adapters import ScrapeAdapter
+
+logger = logging.getLogger("dashboard.scrape.zhihu")
+
+
+class ZhihuScrapeAdapter(ScrapeAdapter):
+    platform = "zhihu"
+    adapter_name = "zhihu_scrape"
+
+    async def collect_user(self, user_id: str, limit: int = 20) -> list[dict]:
+        """抓取指定用户的回答/文章列表"""
+        results = []
+        ok, data, tool = await self._try_tools(2, [
+            ("opencli", lambda: self._opencli_user_answers(user_id, limit)),
+        ])
+        if ok and data:
+            for item in (data if isinstance(data, list) else [data]):
+                results.append(self._to_schema(item))
+        return results
+
+    async def collect_item(self, target: str, depth: str = "light",
+                           tool_level: int = 2) -> dict:
+        """抓取单条回答/文章详情"""
+        item_id = self._extract_id(target)
+        if not item_id:
+            return {"error": f"无法解析内容ID: {target}"}
+        ok, data, tool = await self._try_tools(tool_level, [
+            ("opencli", lambda: self._opencli_answer(item_id)),
+        ])
+        if ok and data:
+            return self._to_schema(data)
+        return {"error": "所有工具均无法抓取"}
+
+    async def collect_comments(self, item_id: str, limit: int = 20) -> list[dict]:
+        return []
+
+    async def collect_search(self, keyword: str, limit: int = 20) -> list[dict]:
+        """搜索问题/回答"""
+        results = []
+        ok, data, tool = await self._try_tools(2, [
+            ("opencli", lambda: self._opencli_search(keyword, limit)),
+        ])
+        if ok and data:
+            for item in (data if isinstance(data, list) else [data]):
+                results.append(self._to_schema(item))
+        return results
+
+    # ── OpenCLI 调用 ──
+
+    async def _opencli_user_answers(self, user_id: str, limit: int):
+        return await self._run_opencli([
+            "zhihu", "user-answers", user_id,
+            "--limit", str(limit), "-f", "json"
+        ])
+
+    async def _opencli_answer(self, answer_id: str):
+        return await self._run_opencli([
+            "zhihu", "answer", answer_id, "-f", "json"
+        ])
+
+    async def _opencli_search(self, keyword: str, limit: int):
+        return await self._run_opencli([
+            "zhihu", "search", keyword,
+            "--limit", str(limit), "-f", "json"
+        ])
+
+    # ── URL 解析 ──
+
+    def _extract_id(self, target: str) -> str:
+        """从 URL 提取 answer_id / question_id"""
+        # zhihu.com/answer/{id}
+        m = re.search(r'zhihu\.com/answer/(\d+)', target)
+        if m:
+            return m.group(1)
+        # zhihu.com/question/{id}/answer/{id}
+        m = re.search(r'zhihu\.com/question/\d+/answer/(\d+)', target)
+        if m:
+            return m.group(1)
+        # 纯数字
+        if target.isdigit():
+            return target
+        return ""
+
+    # ── 统一 Schema ──
+
+    def _to_schema(self, raw: dict) -> dict:
+        return {
+            "platform": "zhihu",
+            "item_id": raw.get("answer_id", raw.get("id", raw.get("url_token", ""))),
+            "url": raw.get("url", ""),
+            "title": raw.get("title", raw.get("question_title", raw.get("excerpt", ""))),
+            "author_name": raw.get("author", raw.get("user_name", raw.get("name", ""))),
+            "author_id": raw.get("user_id", raw.get("author_id", "")),
+            "published_at": raw.get("created_time", raw.get("created_at", raw.get("updated_time", ""))),
+            "text_content": raw.get("content", raw.get("excerpt", "")),
+            "tags": raw.get("topics", []),
+            "stats": {
+                "likes": raw.get("voteup_count", raw.get("likes", 0)),
+                "comments": raw.get("comment_count", raw.get("comments", 0)),
+                "shares": 0,
+                "views": 0,
+            },
+            "extra": {
+                "question_id": raw.get("question_id", ""),
+                "answer_id": raw.get("answer_id", ""),
+            },
+            "media": [],
+            "comments": [],
+        }

mac-agent-os-main/05_tools/10_dashboard/services/browser_orchestrator.py

@@ -0,0 +1,281 @@
+"""
+browser_orchestrator.py — 浏览器编排引擎
+
+职责：
+  1. 并发控制：每台机器最多 N 个浏览器同时运行
+  2. 窗口位置分配：固定槽位，y顶端对齐，x间隔100px
+  3. 错峰启动：各浏览器间隔数秒启动，避免同时假死
+  4. 执行前预检：清理残留进程、验证身份目录
+  5. 远程执行验证：dispatch 后确认进程真实启动
+  6. 状态聚合：聚合本机+远程机器的执行状态
+
+使用：
+  from services.browser_orchestrator import preflight, verify_started, get_machine_status
+"""
+
+import asyncio, json, logging, os, subprocess, time
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Optional
+
+logger = logging.getLogger("dashboard.orchestrator")
+
+# ── 全局配置 ──────────────────────────────────────────────
+AGENT_SYNC = Path.home() / "workbuddy-agent-os" / "agent-sync"
+AGENT_LOCAL = Path.home() / "workbuddy-agent-os" / "agent-local"
+
+def _resolve_hostname() -> str:
+    """优先从 cached_hostname 读取，兜底 os.uname"""
+    cached = AGENT_LOCAL / "identity" / "cached_hostname"
+    if cached.exists():
+        return cached.read_text().strip()
+    return os.uname().nodename
+
+HOSTNAME = _resolve_hostname()
+
+# 执行策略（统一来自 mc.execution_policy）
+import sys as _sys
+_policy_loaded = False
+try:
+    _scripts_dir = AGENT_SYNC / "05_tools" / "07_matrix" / "scripts"
+    _sys.path.insert(0, str(_scripts_dir))
+    from mc.execution_policy import MAX_CONCURRENT, LAUNCH_STAGGER, SLOTS, preflight, slot_for, check_running_browsers
+    _policy_loaded = True
+except ImportError:
+    # 兜底常量（策略层不可用时）
+    MAX_CONCURRENT = 3
+    LAUNCH_STAGGER = 15
+    SLOTS = [
+        {"id": 1, "position": (0, 0),   "size": (702, 783)},
+        {"id": 2, "position": (100, 0), "size": (702, 783)},
+        {"id": 3, "position": (200, 0), "size": (702, 783)},
+    ]
+    def preflight(local_only=True):
+        return {"ok": True, "browsers": 0, "max_concurrent": MAX_CONCURRENT,
+                "slots_available": MAX_CONCURRENT, "disk_gb": 0, "disk_ok": True,
+                "stagger_delay": 0, "message": "策略层未加载，跳过检查"}
+    def slot_for(account_id):
+        return {"id": 1, "position": [0, 0], "size": [702, 783]}
+    def check_running_browsers():
+        return 0
+
+
+# ── 核心函数 ──────────────────────────────────────────────
+
+
+def check_running_browsers(machine: str = None) -> list[dict]:
+    """检查指定机器（或本机）当前正在运行的浏览器进程
+
+    Returns:
+        [{"pid": int, "cmd": str, "machine": str}, ...]
+    """
+    if machine and machine != HOSTNAME:
+        return _check_remote_browsers(machine)
+    return _check_local_browsers()
+
+
+def _check_local_browsers() -> list[dict]:
+    """本地浏览器进程扫描"""
+    result = []
+    try:
+        r = subprocess.run(
+            ["pgrep", "-f", "camoufox.*-no-remote|HeadlessShell|chrome.*--remote-debugging|chromium.*remote-debug"],
+            capture_output=True, text=True, timeout=5
+        )
+        if r.stdout.strip():
+            for pid in r.stdout.strip().split("\n"):
+                pid = pid.strip()
+                if not pid:
+                    continue
+                cmd_r = subprocess.run(
+                    ["ps", "-p", pid, "-o", "command="],
+                    capture_output=True, text=True, timeout=3
+                )
+                result.append({
+                    "pid": int(pid),
+                    "cmd": cmd_r.stdout.strip()[:120],
+                    "machine": HOSTNAME,
+                })
+    except Exception as e:
+        logger.warning(f"检查本地浏览器失败: {e}")
+    return result
+
+
+def _check_remote_browsers(machine: str) -> list[dict]:
+    """远程浏览器进程扫描"""
+    try:
+        from services.remote_exec import exec_remote
+        r = exec_remote(machine,
+            "ps aux | grep -i 'camoufox\\|HeadlessShell\\|chrome.*remote-debug\\|chromium.*remote-debug' | grep -v grep "
+            "| awk '{print $2, $11, $12, $13}'",
+            timeout=10)
+        if r.get("status") == "ok" and r.get("stdout", "").strip():
+            result = []
+            for line in r["stdout"].strip().split("\n"):
+                parts = line.strip().split(None, 3)
+                if parts:
+                    result.append({
+                        "pid": int(parts[0]),
+                        "cmd": " ".join(parts[1:])[:120],
+                        "machine": machine,
+                    })
+            return result
+    except Exception as e:
+        logger.warning(f"检查远程 {machine} 浏览器失败: {e}")
+    return []
+
+
+def preflight(machine: str, account_id: str) -> dict:
+    """执行前预检
+
+    1. 检查该机器当前浏览器数量（不超过 MAX_CONCURRENT）
+    2. 检查身份目录是否存在
+    3. 检查是否有残留进程需要清理
+
+    Returns:
+        {"ok": bool, "message": str, "running": int, "slot": int|None}
+    """
+    issues = []
+
+    # 1. 检查并发
+    running = check_running_browsers(machine)
+    running_count = len(running)
+    if running_count >= MAX_CONCURRENT:
+        return {
+            "ok": False,
+            "message": f"机器 {machine} 已有 {running_count} 个浏览器在运行（上限 {MAX_CONCURRENT}）",
+            "running": running_count,
+            "slot": None,
+            "running_processes": running,
+        }
+
+    # 2. 分配槽位
+    slot = None
+    for s in SLOTS:
+        used = {p.get("cmd", "") for p in running}
+        # 简单判断：每个槽位在 cmd 中是否出现
+        slot_key = f"--window_position={s['position'][0]},{s['position'][1]}"
+        if not any(slot_key in cmd for cmd in used):
+            slot = s
+            break
+    if not slot:
+        slot = SLOTS[running_count]  # 兜底：取下一个未用槽位
+
+    # 3. 检查身份目录
+    identities_root = AGENT_LOCAL / "tools" / "matrix" / "identities"
+    ident_name = account_id.replace("identities/", "")
+    ident_dir = identities_root / ident_name
+    if not ident_dir.exists():
+        # 也可能是 phone_xxx 格式
+        for d in identities_root.iterdir():
+            if d.is_dir() and (d.name == account_id or d.name == ident_name):
+                ident_dir = d
+                break
+        else:
+            issues.append(f"身份目录 {ident_name} 不存在")
+
+    return {
+        "ok": len(issues) == 0,
+        "message": "; ".join(issues) if issues else f"就绪（槽位 {slot['id']}）",
+        "running": running_count,
+        "slot": slot,
+        "identity_dir_exists": ident_dir.exists(),
+        "stagger_delay": running_count * LAUNCH_STAGGER,  # 错峰启动秒数
+    }
+
+
+def update_preflight_with_stagger(preflight_results: dict) -> dict:
+    """给同一台机器的多个槽位分配错峰延迟"""
+    for machine, pf in preflight_results.items():
+        if pf.get("ok") and pf.get("slot"):
+            slot_id = pf["slot"]["id"]
+            pf["stagger_delay"] = (slot_id - 1) * LAUNCH_STAGGER
+    return preflight_results
+
+
+def verify_started(machine: str, account_id: str, timeout: int = 15) -> dict:
+    """验证进程是否真实启动
+
+    本地：
+      1. 先查进程表中有无对应 account_id 的 mc run/collect 进程
+      2. 再查 camoufox 浏览器进程
+    远程：SSH 检查进程表
+
+    Returns:
+        {"running": bool, "pid": int|None, "machine": str}
+    """
+    deadline = time.time() + timeout
+    while time.time() < deadline:
+        if machine == HOSTNAME:
+            # 优先查 mc run/collect 进程（比浏览器引擎启动快）
+            try:
+                r = subprocess.run(
+                    ["pgrep", "-f", account_id],
+                    capture_output=True, text=True, timeout=5
+                )
+                if r.stdout.strip():
+                    pid = int(r.stdout.strip().split("\n")[0])
+                    return {"running": True, "pid": pid, "machine": machine}
+            except:
+                pass
+            running = _check_local_browsers()
+        else:
+            running = _check_remote_browsers(machine)
+
+        # 查找是否包含指定 account_id
+        for p in running:
+            if account_id in p["cmd"]:
+                return {"running": True, "pid": p["pid"], "machine": machine}
+
+        if running:
+            # 有其他浏览器进程但非当前 account，再等等
+            time.sleep(2)
+        else:
+            time.sleep(3)
+
+    return {"running": False, "pid": None, "machine": machine}
+
+
+def get_machine_status(machine: str) -> dict:
+    """获取一台机器的完整状态"""
+    browsers = check_running_browsers(machine)
+    return {
+        "machine": machine,
+        "browsers_running": len(browsers),
+        "browser_processes": browsers,
+        "max_concurrent": MAX_CONCURRENT,
+        "slots_available": MAX_CONCURRENT - len(browsers),
+    }
+
+
+def get_all_machines_status(machines: list[str]) -> dict:
+    """获取所有机器的状态聚合"""
+    statuses = {}
+    for m in machines:
+        try:
+            statuses[m] = get_machine_status(m)
+        except Exception as e:
+            statuses[m] = {"machine": m, "error": str(e)}
+    return {"machines": statuses}
+
+
+def cleanup_stale(machine: str, account_id: str = None) -> dict:
+    """清理残留浏览器进程"""
+    cleaned = []
+    if machine == HOSTNAME:
+        patterns = ["camoufox", "HeadlessShell", "chrome.*--remote-debugging"]
+        for pat in patterns:
+            try:
+                r = subprocess.run(["pkill", "-f", pat], capture_output=True, text=True, timeout=5)
+                if r.returncode == 0:
+                    cleaned.append(f"已终止 {pat}")
+            except:
+                pass
+    else:
+        from services.remote_exec import exec_remote
+        r = exec_remote(machine,
+            "pkill -f camoufox 2>/dev/null; pkill -f HeadlessShell 2>/dev/null; pkill -f 'chrome.*remote-debug' 2>/dev/null; echo 'done'",
+            timeout=10)
+        cleaned.append(f"远程清理: {r.get('status', 'unknown')}")
+
+    return {"cleaned": cleaned, "machine": machine}

mac-agent-os-main/05_tools/10_dashboard/services/command_chain.py

@@ -0,0 +1,456 @@
+"""
+command_chain.py — 接力执行链系统
+
+将一个或多个命令按顺序串联，上一步成功自动触发下一步。
+支持失败重试、超时控制、状态查询、取消。
+
+使用:
+  from services.command_chain import CommandChain
+  
+  chain = CommandChain.create("nurture-then-collect", tasks)
+  status = CommandChain.get_status(chain.chain_id)
+"""
+
+import json
+import logging
+import os
+import threading
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Optional
+from uuid import uuid4
+
+logger = logging.getLogger("dashboard.command_chain")
+
+_THIS_DIR = Path(__file__).resolve().parent
+AGENT_LOCAL = Path(os.environ.get("AGENT_LOCAL", str(Path.home() / "workbuddy-agent-os" / "agent-local")))
+
+CHAINS_DIR = AGENT_LOCAL / "runtime" / "chains"
+CHAINS_DIR.mkdir(parents=True, exist_ok=True)
+
+
+class ChainTask:
+    """接力链中的单个任务定义"""
+
+    def __init__(self, task_id: str, cmd_type: str, accounts: list,
+                 params: dict = None, depends_on: str = None,
+                 max_retries: int = 2, timeout: int = 600,
+                 on_failure: str = "abort"):
+        self.task_id = task_id
+        self.cmd_type = cmd_type          # nurture/collect/login/comment
+        self.accounts = accounts
+        self.params = params or {}
+        self.depends_on = depends_on      # 前置 task_id, None=第一个
+        self.max_retries = max_retries
+        self.timeout = timeout
+        self.on_failure = on_failure      # "abort" | "skip" | "retry"
+
+    def to_dict(self):
+        return {
+            "task_id": self.task_id,
+            "type": self.cmd_type,
+            "accounts": self.accounts,
+            "params": self.params,
+            "depends_on": self.depends_on,
+            "max_retries": self.max_retries,
+            "timeout": self.timeout,
+            "on_failure": self.on_failure,
+        }
+
+    @classmethod
+    def from_dict(cls, d):
+        return cls(
+            task_id=d["task_id"],
+            cmd_type=d["type"],
+            accounts=d["accounts"],
+            params=d.get("params", {}),
+            depends_on=d.get("depends_on"),
+            max_retries=d.get("max_retries", 2),
+            timeout=d.get("timeout", 600),
+            on_failure=d.get("on_failure", "abort"),
+        )
+
+
+class ChainRun:
+    """一次接力链的执行实例"""
+
+    def __init__(self, chain_id: str, name: str, tasks: list):
+        self.chain_id = chain_id
+        self.name = name
+        self.tasks = tasks                 # list[ChainTask]
+        self.status = "pending"            # pending/running/completed/failed
+        self.current_index = -1            # 当前执行到的 task 索引
+        self.task_results = {}             # {task_id: ChainTaskResult}
+        self.error = ""
+        self.created_at = datetime.now(timezone.utc).isoformat()
+        self.completed_at = None
+        self._stop_requested = False
+
+    def to_dict(self):
+        return {
+            "chain_id": self.chain_id,
+            "name": self.name,
+            "status": self.status,
+            "current_index": self.current_index,
+            "total_tasks": len(self.tasks),
+            "tasks": [t.to_dict() for t in self.tasks],
+            "task_results": {k: v.to_dict() for k, v in self.task_results.items()},
+            "error": self.error,
+            "created_at": self.created_at,
+            "completed_at": self.completed_at,
+        }
+
+
+class ChainTaskResult:
+    """单个任务的执行结果"""
+
+    def __init__(self, task_id: str):
+        self.task_id = task_id
+        self.status = "pending"            # pending/running/completed/failed/skipped
+        self.attempts = 0
+        self.command_result = None
+        self.error = ""
+        self.started_at = None
+        self.completed_at = None
+
+    def to_dict(self):
+        return {
+            "task_id": self.task_id,
+            "status": self.status,
+            "attempts": self.attempts,
+            "command_result": self.command_result,
+            "error": self.error,
+            "started_at": self.started_at,
+            "completed_at": self.completed_at,
+        }
+
+
+class CommandChain:
+    """接力执行链调度器"""
+
+    _locks = {}
+    _running = {}
+
+    @classmethod
+    def create(cls, name: str, tasks: list) -> ChainRun:
+        """创建一条接力链
+
+        Args:
+            name: 链名称
+            tasks: list[ChainTask] 或 list[dict]
+
+        Returns:
+            ChainRun 实例
+        """
+        # 转换 dict 为 ChainTask
+        task_objects = []
+        for t in tasks:
+            if isinstance(t, dict):
+                task_objects.append(ChainTask.from_dict(t))
+            else:
+                task_objects.append(t)
+
+        chain_id = str(uuid4())[:12]
+        run = ChainRun(chain_id, name, task_objects)
+
+        # 持久化
+        cls._save(run)
+
+        # 异步启动执行
+        thread = threading.Thread(target=cls._execute, args=(chain_id,), daemon=True)
+        cls._running[chain_id] = thread
+        thread.start()
+
+        return run
+
+    @classmethod
+    def get_status(cls, chain_id: str) -> Optional[dict]:
+        """查询接力链状态"""
+        run = cls._load(chain_id)
+        if run:
+            return run.to_dict()
+        return None
+
+    @classmethod
+    def cancel(cls, chain_id: str) -> dict:
+        """取消接力链"""
+        run = cls._load(chain_id)
+        if not run:
+            return {"error": f"接力链不存在: {chain_id}"}
+        run._stop_requested = True
+        run.status = "cancelled"
+        run.completed_at = datetime.now(timezone.utc).isoformat()
+        cls._save(run)
+        return {"ok": True, "chain_id": chain_id, "status": "cancelled"}
+
+    @classmethod
+    def list_chains(cls, limit: int = 20) -> list:
+        """列出最近创建的接力链"""
+        chains = []
+        if CHAINS_DIR.exists():
+            for f in sorted(CHAINS_DIR.iterdir(), reverse=True)[:limit]:
+                if f.suffix == ".json":
+                    try:
+                        data = json.loads(f.read_text())
+                        chains.append({
+                            "chain_id": data.get("chain_id", f.stem),
+                            "name": data.get("name", ""),
+                            "status": data.get("status", ""),
+                            "total_tasks": len(data.get("tasks", [])),
+                            "current_index": data.get("current_index", -1),
+                            "created_at": data.get("created_at", ""),
+                            "completed_at": data.get("completed_at", ""),
+                        })
+                    except Exception:
+                        pass
+        return chains
+
+    # ── 内部方法 ──
+
+    # ── 并行执行引擎 ──
+
+    @classmethod
+    def _build_levels(cls, run: ChainRun):
+        """将任务按依赖关系分组为层级
+
+        Level 0: 无 depends_on，并行启动
+        Level 1: 依赖 Level 0 任务
+        Level N: 依赖 Level N-1 任务
+        """
+        task_map = {t.task_id: t for t in run.tasks}
+        levels = []
+        remaining = set(task_map.keys())
+
+        while remaining:
+            # 找到所有前置任务都已完成的剩余任务
+            ready = set()
+            for tid in remaining:
+                task = task_map[tid]
+                if not task.depends_on or task.depends_on not in remaining:
+                    ready.add(tid)
+            if not ready:
+                # 死锁：剩余的都有未满足的依赖
+                break
+            levels.append(list(ready))
+            remaining -= ready
+
+        return levels
+
+    @classmethod
+    def _execute_task(cls, run: ChainRun, task: ChainTask):
+        """执行单个任务（线程安全）"""
+        result = ChainTaskResult(task.task_id)
+        result.status = "running"
+        result.started_at = datetime.now(timezone.utc).isoformat()
+
+        # 检查前置任务状态
+        if task.depends_on:
+            dep_result = run.task_results.get(task.depends_on)
+            if dep_result and dep_result.status != "completed":
+                result.status = "skipped"
+                result.error = f"前置任务 {task.depends_on} 未成功完成"
+                run.task_results[task.task_id] = result
+                cls._save(run)
+                return
+
+        # 跳过空账号
+        if not task.accounts:
+            result.status = "skipped"
+            result.error = "账号列表为空"
+            result.completed_at = datetime.now(timezone.utc).isoformat()
+            run.task_results[task.task_id] = result
+            cls._save(run)
+            return
+
+        # 执行并重试
+        success = False
+        for attempt in range(1, task.max_retries + 1):
+            if run._stop_requested:
+                break
+            result.attempts = attempt
+            run.task_results[task.task_id] = result
+            cls._save(run)
+
+            try:
+                from services.command_bus import CommandBus
+                cmd_result = CommandBus.dispatch(
+                    task.cmd_type, task.accounts,
+                    task.params, wait=True
+                )
+                result.command_result = cmd_result
+
+                # 检查各个机器的子任务状态
+                sub_statuses = cmd_result.get("per_machine", {})
+                machine_statuses = {m: s.get("status", "unknown") for m, s in sub_statuses.items()}
+                all_sub_ok = all(s == "completed" or s == "accepted" or s == "started"
+                                for s in machine_statuses.values())
+
+                if cmd_result.get("status") in ("completed", "accepted") or all_sub_ok:
+                    success = True
+                    break
+                else:
+                    result.error = cmd_result.get("message", "") or f"机器状态: {machine_statuses}"
+                    logger.warning(f"  [{run.chain_id}] {task.task_id} 第{attempt}次失败: {result.error}")
+                    if attempt < task.max_retries:
+                        time.sleep(5)
+            except Exception as e:
+                result.error = str(e)
+                logger.warning(f"  [{run.chain_id}] {task.task_id} 异常: {e}")
+                if attempt < task.max_retries:
+                    time.sleep(5)
+
+        # 结果
+        if success:
+            result.status = "completed"
+        else:
+            result.status = "failed"
+        result.completed_at = datetime.now(timezone.utc).isoformat()
+        run.task_results[task.task_id] = result
+        cls._save(run)
+
+    @classmethod
+    def _execute(cls, chain_id: str):
+        """执行接力链（按依赖层级并行执行）"""
+        run = cls._load(chain_id)
+        if not run:
+            return
+
+        run.status = "running"
+        cls._save(run)
+
+        # 构建依赖层级
+        task_map = {t.task_id: t for t in run.tasks}
+        levels = cls._build_levels(run)
+
+        for level_idx, level_tasks in enumerate(levels):
+            if run._stop_requested:
+                break
+
+            run.current_index = level_idx
+            logger.info(f"  [{chain_id}] 执行层级 {level_idx+1}/{len(levels)}: {level_tasks}")
+            cls._save(run)
+
+            # 同一层级并行执行
+            threads = []
+            for tid in level_tasks:
+                task = task_map[tid]
+                t = threading.Thread(
+                    target=cls._execute_task,
+                    args=(run, task),
+                    daemon=True
+                )
+                threads.append(t)
+                t.start()
+
+            # 等待本层级所有任务完成
+            for t in threads:
+                t.join()
+
+            # 检查是否有任务失败需要终止
+            if not run._stop_requested:
+                for tid in level_tasks:
+                    tr = run.task_results.get(tid)
+                    if tr and tr.status == "failed":
+                        task = task_map[tid]
+                        if task.on_failure == "abort":
+                            run.status = "failed"
+                            run.error = f"任务 {tid} 失败后终止"
+                            run.completed_at = datetime.now(timezone.utc).isoformat()
+                            cls._save(run)
+                            cls._running.pop(chain_id, None)
+                            return
+                # 层间冷却
+                if level_idx < len(levels) - 1:
+                    time.sleep(3)
+
+        # 最终状态
+        if not run._stop_requested:
+            all_ok = all(
+                run.task_results.get(t.task_id, ChainTaskResult(t.task_id)).status == "completed"
+                for t in run.tasks if t.accounts  # 空账号skipped不计入失败
+            )
+            run.status = "completed" if all_ok else "failed"
+            if not run.error:
+                failed = [t.task_id for t in run.tasks
+                         if run.task_results.get(t.task_id, ChainTaskResult(t.task_id)).status not in ("completed", "skipped")]
+                if failed:
+                    run.status = "failed"
+                    run.error = f"以下任务失败: {', '.join(failed)}"
+        else:
+            run.status = "cancelled"
+
+        run.completed_at = datetime.now(timezone.utc).isoformat()
+        cls._save(run)
+        cls._running.pop(chain_id, None)
+
+        # 链完成后自动触发联邦采集器拉取最新数据
+        if run.status == "completed":
+            try:
+                from services.fleet_collector import collect_all
+                logger.info(f"  [{chain_id}] 链完成，自动触发联邦采集...")
+                collect_all()
+            except Exception as e:
+                logger.warning(f"  [{chain_id}] 联邦采集自动触发失败: {e}")
+
+    _save_locks: dict = {}
+
+    @classmethod
+    def _get_save_lock(cls, chain_id: str) -> threading.Lock:
+        """按 chain_id 获取锁，确保同一链的写入串行化"""
+        if chain_id not in cls._save_locks:
+            cls._save_locks[chain_id] = threading.Lock()
+        return cls._save_locks[chain_id]
+
+    @classmethod
+    def _save(cls, run: ChainRun):
+        """持久化接力链状态（按chain_id加锁，原子写入）"""
+        with cls._get_save_lock(run.chain_id):
+            path = CHAINS_DIR / f"{run.chain_id}.json"
+            tmp = path.with_suffix(".tmp")
+            tmp.write_text(json.dumps(run.to_dict(), indent=2, ensure_ascii=False))
+            tmp.replace(path)
+
+    @classmethod
+    def _load(cls, chain_id: str) -> Optional[ChainRun]:
+        """加载接力链"""
+        path = CHAINS_DIR / f"{chain_id}.json"
+        if not path.exists():
+            return None
+        try:
+            data = json.loads(path.read_text())
+            run = ChainRun(data["chain_id"], data["name"],
+                          [ChainTask.from_dict(t) for t in data["tasks"]])
+            run.status = data.get("status", "pending")
+            run.current_index = data.get("current_index", -1)
+            run.task_results = {}
+            for tid, rd in data.get("task_results", {}).items():
+                r = ChainTaskResult(tid)
+                r.status = rd.get("status", "pending")
+                r.attempts = rd.get("attempts", 0)
+                r.command_result = rd.get("command_result")
+                r.error = rd.get("error", "")
+                r.started_at = rd.get("started_at")
+                r.completed_at = rd.get("completed_at")
+                run.task_results[tid] = r
+            run.error = data.get("error", "")
+            run.created_at = data.get("created_at", "")
+            run.completed_at = data.get("completed_at")
+            return run
+        except Exception as e:
+            logger.warning(f"  ⚠️ 接力链加载失败 [{chain_id}]: {e}")
+            # 尝试修复：删除最后一行非JSON内容
+            try:
+                path = CHAINS_DIR / f"{chain_id}.json"
+                raw = path.read_text()
+                # 找到第一个完整的 JSON 对象
+                import re as _re
+                m = _re.search(r'^(\{[^{}]*(\{[^{}]*\})*[^{}]*\})', raw, _re.DOTALL)
+                if m:
+                    data = json.loads(m.group(1))
+                    path.write_text(m.group(1))
+                    logger.info(f"  ✅ 接力链文件已修复 [{chain_id}]")
+                    return cls._load(chain_id)
+            except Exception:
+                pass
+            return None

mac-agent-os-main/05_tools/10_dashboard/services/data_aggregator.py

@@ -0,0 +1,202 @@
+"""
+联邦数据聚合器
+
+从所有在线机器拉取数据并合并，供 Dashboard 展示。
+所有机器通过 Tailscale 网络互联，端口统一 9988。
+"""
+
+import json, logging, sys
+from pathlib import Path
+
+logger = logging.getLogger("dashboard.federation")
+
+# 确保能找到 ORACLE.yaml
+_THIS_DIR = Path(__file__).resolve().parent.parent
+if str(_THIS_DIR) not in sys.path:
+    sys.path.insert(0, str(_THIS_DIR))
+
+ORACLE_PATH = _THIS_DIR.parent.parent / "ORACLE.yaml"
+
+
+def _get_machines():
+    """从 ORACLE.yaml 读取所有机器信息（排除本机）"""
+    import yaml
+    import os, socket, subprocess, re
+    
+    if not ORACLE_PATH.exists():
+        return []
+    
+    with open(ORACLE_PATH) as f:
+        oracle = yaml.safe_load(f)
+    
+    # 获取本机所有 IP（包括 Tailscale）
+    try:
+        output = subprocess.run(['ifconfig'], capture_output=True, text=True, timeout=5)
+        local_ips = set(re.findall(r'inet (\d+\.\d+\.\d+\.\d+)', output.stdout))
+    except:
+        local_ips = {"127.0.0.1"}
+    local_ips.add("127.0.0.1")
+    
+    machines = []
+    for name, info in oracle.get("machines", {}).items():
+        machine_ip = info.get("tailscale_ip", "")
+        if machine_ip in local_ips:
+            continue  # 跳过本机
+        machines.append({
+            "name": name,
+            "hostname": info.get("hostname", name),
+            "ip": info.get("tailscale_ip", ""),
+            "port": info.get("dashboard_port", 9988),
+            "user": info.get("ssh_user", ""),
+        })
+    return machines
+
+
+def fetch_machine_data(machine: dict, path: str, timeout: int = 5) -> dict:
+    """从指定机器拉取数据（使用 urllib，无外部依赖）"""
+    import urllib.request, urllib.error
+    
+    url = f"http://{machine['ip']}:{machine['port']}{path}"
+    try:
+        r = urllib.request.urlopen(url, timeout=timeout)
+        if r.status == 200:
+            return json.loads(r.read().decode())
+        return {"error": f"HTTP {r.status}"}
+    except urllib.error.URLError as e:
+        return {"error": f"连接失败: {e.reason}"}
+    except TimeoutError:
+        return {"error": "超时"}
+    except Exception as e:
+        return {"error": str(e)}
+
+
+def aggregate_accounts():
+    """聚合所有机器的账号列表（含本机）"""
+    from plugins.base import HOSTNAME
+    machines = _get_machines()
+    all_accounts = []
+    
+    # 先加本机数据（只取本机账号）
+    try:
+        import urllib.request, json as _json
+        local_resp = urllib.request.urlopen("http://localhost:9988/api/matrix/accounts", timeout=5)
+        local_data = _json.loads(local_resp.read())
+        if isinstance(local_data, list):
+            for acct in local_data:
+                if acct.get("owner_machine", "") != HOSTNAME:
+                    continue
+                acct["_source_machine"] = HOSTNAME
+                acct["is_local"] = True
+                all_accounts.append(acct)
+    except Exception:
+        pass
+    
+    # 去重集合: (machine, account_id)
+    seen = set()
+    
+    for m in machines:
+        data = fetch_machine_data(m, "/api/matrix/accounts")
+        if isinstance(data, list):
+            for acct in data:
+                # 只取该机器自己的账号（去重）
+                if acct.get("owner_machine", "") != m["name"]:
+                    continue
+                key = m["name"] + "|" + acct.get("id", "")
+                if key in seen:
+                    continue
+                seen.add(key)
+                acct["_source_machine"] = m["name"]
+                acct["is_local"] = False
+                all_accounts.append(acct)
+        elif isinstance(data, dict) and "error" not in data:
+            accts = data.get("data", data.get("accounts", []))
+            if isinstance(accts, list):
+                for acct in accts:
+                    if acct.get("owner_machine", "") != m["name"]:
+                        continue
+                    key = m["name"] + "|" + acct.get("id", "")
+                    if key in seen:
+                        continue
+                    seen.add(key)
+                    acct["_source_machine"] = m["name"]
+                    acct["is_local"] = False
+                    all_accounts.append(acct)
+        else:
+            all_accounts.append({
+                "_source_machine": m["name"],
+                "_error": data.get("error", "未知错误"),
+                "machine_ip": m.get("ip", ""),
+                "is_local": False,
+            })
+    
+    return all_accounts
+
+
+def aggregate_health():
+    """聚合所有机器的健康状态（含本机）"""
+    from plugins.base import HOSTNAME
+    machines = _get_machines()
+    results = {}
+    
+    # 先加本机健康
+    try:
+        import urllib.request, json as _json
+        local_resp = urllib.request.urlopen("http://localhost:9988/api/health", timeout=3)
+        results[HOSTNAME] = _json.loads(local_resp.read())
+    except Exception as e:
+        results[HOSTNAME] = {"status": "ok", "note": "本机直连"}
+    
+    for m in machines:
+        data = fetch_machine_data(m, "/api/health")
+        results[m["name"]] = data
+    
+    return results
+
+
+def aggregate_status():
+    """聚合所有机器的详细状态（含本机）"""
+    from plugins.base import HOSTNAME
+    machines = _get_machines()
+    results = {}
+    
+    # 本机状态
+    try:
+        import urllib.request, json as _json
+        lh = urllib.request.urlopen("http://localhost:9988/api/health", timeout=3)
+        local_health = _json.loads(lh.read())
+        ls = None
+        try:
+            ls_resp = urllib.request.urlopen("http://localhost:9988/api/machine/status", timeout=3)
+            ls = _json.loads(ls_resp.read())
+        except:
+            pass
+        results[HOSTNAME] = {
+            "status": "online",
+            "hostname": HOSTNAME,
+            "ip": "127.0.0.1",
+            "health": local_health,
+            "detail": ls,
+        }
+    except:
+        results[HOSTNAME] = {"status": "online", "hostname": HOSTNAME, "ip": "127.0.0.1"}
+    
+    for m in machines:
+        health = fetch_machine_data(m, "/api/health", timeout=3)
+        if "error" in health:
+            results[m["name"]] = {
+                "status": "offline",
+                "error": health["error"],
+                "hostname": m["hostname"],
+                "ip": m["ip"],
+            }
+            continue
+        status = fetch_machine_data(m, "/api/machine/status", timeout=5)
+        results[m["name"]] = {
+            "status": "online",
+            "hostname": m["hostname"],
+            "ip": m["ip"],
+            "health": health,
+            "detail": status if "error" not in status else None,
+        }
+    
+    return results

mac-agent-os-main/05_tools/10_dashboard/services/douyin_stats.py

@@ -0,0 +1,20 @@
+"""
+douyin_stats.py — 抖音视频数据采集器（v3）
+ 
+适配层：调用 MediaCrawler 风格适配器
+"""
+import asyncio, json, logging, os, re
+from pathlib import Path
+
+logger = logging.getLogger("dashboard.douyin_stats")
+
+# 导入 MediaCrawler 适配器
+from services.mediacrawler_adapter import get_video_data as _mc_get_video_data
+
+
+async def get_video_data(url: str) -> dict:
+    """统一入口：传入抖音视频 URL，返回完整数据
+    
+    委托给 MediaCrawler 适配器处理。
+    """
+    return await _mc_get_video_data(url)

mac-agent-os-main/05_tools/10_dashboard/services/fleet_collector.py

@@ -0,0 +1,238 @@
+"""
+fleet_collector.py — 联邦信息采集聚合器
+
+职责：
+  主控机通过 SSH 定期从各工作机拉取采集数据（homepage_info.json + profiles.json），
+  合并后供 Dashboard 统一展示。
+
+架构：
+  工作机只需要：
+    - guardd 正常运行（已在做）
+    - SSH 可达（Tailscale）
+    - 本地有 homepage_info.json（采集后自动生成）
+  
+  主控机负责：
+    - SSH 拉取各机器数据
+    - 按来源机器缓存
+    - 聚合 API 返回合并结果
+"""
+
+import json
+import logging
+import os
+import subprocess
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Optional
+
+logger = logging.getLogger("dashboard.fleet_collector")
+
+_THIS_DIR = Path(__file__).resolve().parent
+AGENT_LOCAL = Path(os.environ.get("AGENT_LOCAL", str(Path.home() / "workbuddy-agent-os" / "agent-local")))
+AGENT_SYNC = Path(os.environ.get("AGENT_SYNC", str(Path.home() / "workbuddy-agent-os" / "agent-sync")))
+
+# 缓存目录：agent-local/runtime/fleet_collector/{hostname}/
+CACHE_DIR = AGENT_LOCAL / "runtime" / "fleet_collector"
+CACHE_DIR.mkdir(parents=True, exist_ok=True)
+
+# 已知工作机列表（从 ORACLE.yaml 读取）
+_KNOWN_WORKERS_CACHE = None
+_KNOWN_WORKERS_CACHE_TIME = 0
+
+
+def _get_workers() -> list[dict]:
+    """从 ORACLE.yaml 读取所有工作机信息"""
+    global _KNOWN_WORKERS_CACHE, _KNOWN_WORKERS_CACHE_TIME
+    now = time.time()
+    if _KNOWN_WORKERS_CACHE and now - _KNOWN_WORKERS_CACHE_TIME < 300:
+        return _KNOWN_WORKERS_CACHE
+
+    oracle_path = AGENT_SYNC / "ORACLE.yaml"
+    if not oracle_path.exists():
+        return []
+
+    try:
+        import yaml
+        oracle = yaml.safe_load(oracle_path.read_text())
+        machines = oracle.get("machines", {})
+        workers = []
+        for name, info in machines.items():
+            workers.append({
+                "name": name,
+                "hostname": info.get("hostname", name),
+                "ip": info.get("tailscale_ip", ""),
+                "user": info.get("ssh_user", ""),
+                "role": info.get("role", ""),
+            })
+        _KNOWN_WORKERS_CACHE = workers
+        _KNOWN_WORKERS_CACHE_TIME = now
+        return workers
+    except Exception as e:
+        logger.warning(f"读取 ORACLE.yaml 工作机列表失败: {e}")
+        return []
+
+
+def _ssh_read_file(hostname: str, remote_path: str, timeout: int = 15) -> Optional[str]:
+    """通过 SSH 读取远程文件内容"""
+    workers = _get_workers()
+    target = None
+    for w in workers:
+        if w["name"] == hostname or w["hostname"] == hostname:
+            target = w
+            break
+    if not target or not target.get("ip") or not target.get("user"):
+        logger.warning(f"未知工作机: {hostname}")
+        return None
+
+    ssh_target = f"{target['user']}@{target['ip']}"
+    try:
+        r = subprocess.run(
+            ["ssh", "-o", "ConnectTimeout=10", "-o", "StrictHostKeyChecking=no",
+             ssh_target, f"cat {remote_path} 2>/dev/null || echo '__FILE_NOT_FOUND__'"],
+            capture_output=True, text=True, timeout=timeout
+        )
+        if r.returncode == 0 and "__FILE_NOT_FOUND__" not in r.stdout:
+            return r.stdout
+        return None
+    except Exception as e:
+        logger.debug(f"SSH 读取 {hostname}:{remote_path} 失败: {e}")
+        return None
+
+
+def collect_single(hostname: str) -> dict:
+    """从单台工作机拉取采集数据，写入缓存
+
+    Returns:
+        {"hostname": str, "collected": bool, "homepage": dict, "profiles": dict, "error": str}
+    """
+    result = {"hostname": hostname, "collected": False, "homepage": {}, "profiles": {}, "error": ""}
+
+    # 读取远程 homepage_info.json
+    remote_hp = "~/workbuddy-agent-os/agent-local/tools/matrix/data/homepage_info.json"
+    hp_content = _ssh_read_file(hostname, remote_hp)
+    if hp_content:
+        try:
+            hp_data = json.loads(hp_content)
+            result["homepage"] = hp_data
+        except json.JSONDecodeError as e:
+            result["error"] += f"homepage_info 解析失败: {e}; "
+
+    # 读取远程 profiles.json
+    remote_pf = "~/workbuddy-agent-os/agent-local/tools/matrix/data/profiles.json"
+    pf_content = _ssh_read_file(hostname, remote_pf)
+    if pf_content:
+        try:
+            pf_data = json.loads(pf_content)
+            result["profiles"] = pf_data
+        except json.JSONDecodeError as e:
+            result["error"] += f"profiles 解析失败: {e}; "
+
+    # 写入本地缓存
+    machine_dir = CACHE_DIR / hostname
+    machine_dir.mkdir(parents=True, exist_ok=True)
+
+    if result["homepage"]:
+        (machine_dir / "homepage_info.json").write_text(
+            json.dumps(result["homepage"], indent=2, ensure_ascii=False)
+        )
+    if result["profiles"]:
+        (machine_dir / "profiles.json").write_text(
+            json.dumps(result["profiles"], indent=2, ensure_ascii=False)
+        )
+
+    # 写入采集时间戳
+    (machine_dir / "collected_at.txt").write_text(
+        datetime.now(timezone.utc).isoformat()
+    )
+
+    result["collected"] = bool(hp_content or pf_content)
+    if not result["collected"]:
+        result["error"] += "未获取到任何采集数据"
+    logger.info(f"  采集 {hostname}: {'✅' if result['collected'] else '❌'} {result['error']}")
+    return result
+
+
+def collect_all() -> list[dict]:
+    """遍历所有工作机拉取采集数据"""
+    workers = _get_workers()
+    results = []
+    for w in workers:
+        if w["role"] == "master" or w["name"] == os.uname().nodename:
+            continue  # 跳过本机
+        # 也跳过 cached_hostname 匹配本机的情况
+        cached_hn = AGENT_LOCAL / "identity" / "cached_hostname"
+        if cached_hn.exists() and cached_hn.read_text().strip() == w["name"]:
+            continue
+        result = collect_single(w["name"])
+        results.append(result)
+    return results
+
+
+def get_merged_homepage() -> dict:
+    """返回合并后的所有机器 homepage 数据
+
+    读取本机 + 所有缓存的工作机数据，合并为一个列表
+    """
+    all_results = []
+
+    # 1. 本机数据
+    local_hp = AGENT_LOCAL / "tools" / "matrix" / "data" / "homepage_info.json"
+    if local_hp.exists():
+        try:
+            data = json.loads(local_hp.read_text())
+            for entry in data.get("results", []):
+                entry["_source_machine"] = "local"
+                all_results.append(entry)
+        except Exception:
+            pass
+
+    # 2. 各工作机缓存
+    if CACHE_DIR.exists():
+        for machine_dir in sorted(CACHE_DIR.iterdir()):
+            if not machine_dir.is_dir():
+                continue
+            hp_file = machine_dir / "homepage_info.json"
+            if hp_file.exists():
+                try:
+                    data = json.loads(hp_file.read_text())
+                    for entry in data.get("results", []):
+                        entry["_source_machine"] = machine_dir.name
+                        all_results.append(entry)
+                except Exception:
+                    pass
+
+    return {
+        "collected_at": datetime.now(timezone.utc).isoformat(),
+        "total_identities": len(all_results),
+        "results": all_results,
+    }
+
+
+def get_cached_profiles(hostname: Optional[str] = None) -> dict:
+    """获取缓存的 profiles 数据"""
+    all_profiles = {}
+
+    # 本机
+    local_pf = AGENT_LOCAL / "tools" / "matrix" / "data" / "profiles.json"
+    if local_pf.exists() and (hostname is None or hostname == "local"):
+        try:
+            all_profiles.update(json.loads(local_pf.read_text()))
+        except Exception:
+            pass
+
+    # 工作机缓存
+    if CACHE_DIR.exists():
+        for machine_dir in sorted(CACHE_DIR.iterdir()):
+            if not machine_dir.is_dir():
+                continue
+            if hostname and machine_dir.name != hostname:
+                continue
+            pf_file = machine_dir / "profiles.json"
+            if pf_file.exists():
+                try:
+                    all_profiles.update(json.loads(pf_file.read_text()))
+                except Exception:
+                    pass
+
+    return all_profiles

mac-agent-os-main/05_tools/10_dashboard/services/log_aggregator.py

@@ -0,0 +1,122 @@
+"""
+日志聚合模块 — 操作日志记录/查询/清理
+
+每个操作产生一条结构化日志,存储在 cross_machine/tasks/logs/ 目录。
+日志按日期分割: cross_machine/tasks/logs/2026-06-16.jsonl
+"""
+
+import json, gzip
+from pathlib import Path
+from datetime import datetime, timezone, timedelta
+
+from plugins.base import CROSS_MACHINE
+
+LOGS_DIR = CROSS_MACHINE / "tasks" / "logs"
+MAX_LOG_AGE_DAYS = 30
+
+
+def _today() -> str:
+    return datetime.now().strftime("%Y-%m-%d")
+
+
+def _log_path(date: str = "") -> Path:
+    if not date:
+        date = _today()
+    LOGS_DIR.mkdir(parents=True, exist_ok=True)
+    return LOGS_DIR / f"{date}.jsonl"
+
+
+def write_log(entry: dict):
+    """写入一条操作日志"""
+    if "timestamp" not in entry:
+        entry["timestamp"] = datetime.now(timezone.utc).isoformat()
+    fp = _log_path()
+    with open(fp, "a", encoding="utf-8") as f:
+        f.write(json.dumps(entry, ensure_ascii=False) + "\n")
+
+
+def query_logs(machine: str = "", op_type: str = "",
+               status: str = "", days: int = 7,
+               limit: int = 200) -> list[dict]:
+    """查询操作日志,支持按机器/类型/状态/天数过滤"""
+    results = []
+    today = datetime.now()
+    
+    for i in range(days):
+        date = (today - timedelta(days=i)).strftime("%Y-%m-%d")
+        fp = _log_path(date)
+        if not fp.exists():
+            continue
+        try:
+            with open(fp, "r", encoding="utf-8") as f:
+                for line in f:
+                    line = line.strip()
+                    if not line:
+                        continue
+                    try:
+                        entry = json.loads(line)
+                    except:
+                        continue
+                    # 过滤
+                    if machine and entry.get("machine", "") != machine:
+                        continue
+                    if op_type and entry.get("type", "") != op_type:
+                        continue
+                    if status and entry.get("status", "") != status:
+                        continue
+                    results.append(entry)
+                    if len(results) >= limit:
+                        return results
+        except:
+            continue
+    
+    return results
+
+
+def get_log_stats(days: int = 7) -> dict:
+    """获取日志统计"""
+    logs = query_logs(days=days, limit=10000)
+    total = len(logs)
+    by_type = {}
+    by_machine = {}
+    by_status = {}
+    
+    for entry in logs:
+        t = entry.get("type", "unknown")
+        by_type[t] = by_type.get(t, 0) + 1
+        
+        m = entry.get("machine", "unknown")
+        by_machine[m] = by_machine.get(m, 0) + 1
+        
+        s = entry.get("status", "unknown")
+        by_status[s] = by_status.get(s, 0) + 1
+    
+    return {
+        "total": total,
+        "by_type": by_type,
+        "by_machine": by_machine,
+        "by_status": by_status,
+        "days": days,
+    }
+
+
+def cleanup_old_logs(max_days: int = MAX_LOG_AGE_DAYS) -> dict:
+    """清理超过 max_days 的旧日志"""
+    removed = 0
+    today = datetime.now()
+    
+    if not LOGS_DIR.exists():
+        return {"removed": 0}
+    
+    for fp in LOGS_DIR.glob("*.jsonl"):
+        try:
+            # 从文件名提取日期
+            date_str = fp.stem
+            file_date = datetime.strptime(date_str, "%Y-%m-%d")
+            if (today - file_date).days > max_days:
+                fp.unlink()
+                removed += 1
+        except:
+            continue
+    
+    return {"removed": removed}

mac-agent-os-main/05_tools/10_dashboard/services/mediacrawler_adapter.py

@@ -0,0 +1,457 @@
+"""
+mediacrawler_adapter.py — 抖音视频数据采集器 v3
+
+全新架构：不再依赖 CDP/Playwright/浏览器页面。
+直接从 Chrome profile 读取 cookie，通过 HTTP 请求调用抖音 API。
+全程无窗口、无标签页、无闪烁。
+"""
+import asyncio, json, logging, os, re, time, sqlite3, urllib.request, urllib.error
+from pathlib import Path
+from typing import Optional
+
+logger = logging.getLogger("dashboard.mediacrawler_adapter")
+
+# HTTP 请求头（模拟浏览器）
+DOUYIN_HEADERS = {
+    "User-Agent": (
+        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
+        "AppleWebKit/537.36 (KHTML, like Gecko) "
+        "Chrome/150.0.0.0 Safari/537.36"
+    ),
+    "Accept": "application/json, text/plain, */*",
+    "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8",
+    "Referer": "https://www.douyin.com/",
+    "Origin": "https://www.douyin.com",
+}
+
+
+# ── Playwright CDP 连接管理（复用，避免反复建连） ──
+# Chrome 新版把 cookie 值加密存在 SQLite 里，必须通过 CDP 读解密后的值。
+
+_PW = None
+_BROWSER = None
+
+async def _ensure_cdp():
+    """确保 Playwright CDP 连接可用（全局复用）"""
+    global _PW, _BROWSER
+    if _BROWSER and _PW:
+        try:
+            if _BROWSER.is_connected():
+                return _BROWSER.contexts[0] if _BROWSER.contexts else None
+        except:
+            pass
+        # 断连了，重建
+        try:
+            await _BROWSER.close()
+        except:
+            pass
+        try:
+            await _PW.stop()
+        except:
+            pass
+        _BROWSER = None
+        _PW = None
+    
+    try:
+        from playwright.async_api import async_playwright
+        _PW = await async_playwright().start()
+        _BROWSER = await _PW.chromium.connect_over_cdp("http://127.0.0.1:9222")
+        ctx = _BROWSER.contexts[0] if _BROWSER.contexts else None
+        return ctx
+    except Exception as e:
+        _BROWSER = None
+        _PW = None
+        logger.warning(f"CDP 连接失败: {e}")
+        return None
+
+
+async def _get_cookies() -> dict:
+    """从 CDP 连接读取 Chrome cookie（解密后的值）"""
+    ctx = await _ensure_cdp()
+    if not ctx:
+        return {}
+    try:
+        all_cookies = await ctx.cookies()
+        result = {}
+        for c in all_cookies:
+            domain = c.get("domain", "")
+            if "douyin" in domain or "amemv" in domain:
+                result[c["name"]] = c["value"]
+        return result
+    except Exception as e:
+        logger.warning(f"读 cookie 失败: {e}")
+        return {}
+
+
+def _cookie_str(cookies: dict) -> str:
+    """将 cookie 字典转为 HTTP Header 字符串"""
+    return "; ".join(f"{k}={v}" for k, v in cookies.items())
+
+
+# ── HTTP API 请求（不打开任何浏览器页面） ──
+
+def _http_get(url: str, cookies: dict, timeout: int = 15) -> dict:
+    """发起 HTTP GET 请求并返回 JSON（cookies 必传，由调用方从 CDP 获取）"""
+    req = urllib.request.Request(
+        url,
+        headers={
+            **DOUYIN_HEADERS,
+            "Cookie": _cookie_str(cookies),
+        },
+    )
+    with urllib.request.urlopen(req, timeout=timeout) as resp:
+        return json.loads(resp.read().decode())
+
+
+# ── 工具函数 ──
+
+def _extract_aweme_id(url: str):
+    m = re.search(r'douyin\.com/video/(\d+)', url)
+    if m:
+        return m.group(1)
+    m = re.search(r'iesdouyin\.com/share/video/(\d+)', url)
+    if m:
+        return m.group(1)
+    # 抖音精选页 / 发现页：modal_id 就是视频 aweme_id（jingxuan?modal_id=xxx）
+    m = re.search(r'(?:jingxuan|discover|modal_id)[=?/](\d+)', url)
+    if m:
+        return m.group(1)
+    return None
+
+
+def _resolve_shortlink(url: str):
+    import subprocess
+    try:
+        result = subprocess.run(
+            ["curl", "-sI", url], capture_output=True, text=True, timeout=10
+        )
+        for line in result.stdout.split("\n"):
+            if line.lower().startswith("location:"):
+                loc = line.split(":", 1)[1].strip()
+                m = re.search(r'/video/(\d+)', loc)
+                if m:
+                    return m.group(1)
+    except Exception:
+        pass
+    return None
+
+
+# ── 核心：获取视频数据（纯 HTTP，无浏览器） ──
+
+async def get_video_data(url: str) -> dict:
+    """
+    统一入口：传入抖音视频 URL，返回完整数据
+    通过 HTTP API 获取，不打开任何浏览器页面
+    """
+    aweme_id = _extract_aweme_id(url)
+    if not aweme_id:
+        aweme_id = _resolve_shortlink(url)
+    if not aweme_id:
+        return {"error": f"无法解析视频 ID: {url}"}
+
+    t0 = time.time()
+    result = {
+        "aweme_id": aweme_id,
+        "title": "",
+        "author": "",
+        "likes": 0,
+        "comments": 0,
+        "collects": 0,
+        "shares": 0,
+        "comment_texts": [],
+        "collected_at": time.strftime("%Y-%m-%d %H:%M:%S"),
+    }
+
+    # 读取登录态 cookie（通过 CDP，解密后的值）
+    cookies = await _get_cookies()
+    has_session = bool(cookies.get("sessionid"))
+    if not has_session:
+        result["error"] = "抖音登录已过期，请点击顶部「📱 打开抖音登录」按钮重新登录"
+        result["login_expired"] = True
+        return result
+
+    # 调抖音 API 获取视频详情
+    try:
+        api_url = (
+            f"https://www.douyin.com/aweme/v1/web/aweme/detail/"
+            f"?aweme_id={aweme_id}&version_code=170400&app_name=aweme&device_platform=web"
+        )
+        detail = await asyncio.to_thread(_http_get, api_url, cookies)
+    except urllib.error.HTTPError as e:
+        result["error"] = f"HTTP {e.code}: {e.reason}"
+        return result
+    except Exception as e:
+        result["error"] = f"请求失败: {e}"
+        return result
+
+    aweme = detail.get("aweme_detail", {})
+    if not aweme:
+        if detail.get("status_code") == 2:
+            result["error"] = "抖音登录已过期，请重新登录"
+            result["login_expired"] = True
+            return result
+        if detail.get("filter_detail"):
+            # filter_detail = 视频被平台过滤（已删除/违规/推广限制），接口层详情不可查
+            result["filtered"] = True
+            # 兜底策略：接口被过滤，改用页面浏览方式（打开网址读页面标题/meta），补全标题/博主
+            fb = await _page_fallback(url, aweme_id)
+            if fb:
+                result.update({
+                    "title": fb.get("title", ""),
+                    "author": fb.get("author", ""),
+                    "published_at": fb.get("published_at", ""),
+                    "source": "page_fallback",
+                    "api_error": "⚠️ 视频被平台过滤（接口层），已用页面兜底补全标题/博主",
+                })
+                return result
+            result["error"] = "⚠️ 视频被平台过滤，无法采集（可能已删除/违规/推广限制）"
+            return result
+        result["error"] = f"API 未返回数据: {str(detail)[:200]}"
+        return result
+
+    statistics = aweme.get("statistics", {})
+    author_info = aweme.get("author", {})
+
+    result.update({
+        "title": aweme.get("desc", ""),
+        "author": author_info.get("nickname", ""),
+        "author_uid": author_info.get("uid", ""),          # 数字ID（博主精确身份）
+        "author_sec_uid": author_info.get("sec_uid", ""),  # 加密ID
+        "author_unique_id": author_info.get("unique_id", ""),  # 抖音号
+        "likes": int(statistics.get("digg_count", 0)),
+        "comments": int(statistics.get("comment_count", 0)),
+        "collects": int(statistics.get("collect_count", 0)),
+        "shares": int(statistics.get("share_count", 0)),
+    })
+
+    # 获取热评（复用已有 Chrome 页面，不创建新标签页）
+    # Chrome 已有 douyin.com 页面，直接用它的 JS 上下文执行 fetch
+    # ⚠️ 不要 new_page() — 那会在 Chrome 中闪出新标签页
+    try:
+        _ctx2 = await _ensure_cdp()
+        if _ctx2:
+            _pages = _ctx2.pages
+            if _pages:
+                _page = _pages[0]  # 复用已有页面（about:blank 或 douyin.com）
+                _cmt_json = await _page.evaluate(f"""async () => {{
+                    try {{
+                        const r = await fetch(
+                            'https://www.douyin.com/aweme/v1/web/comment/list/?aweme_id={aweme_id}&count=20&cursor=0',
+                            {{ headers: {{ 'Accept': 'application/json, text/plain, */*', 'Referer': 'https://www.douyin.com/' }} }}
+                        );
+                        return JSON.stringify(await r.json());
+                    }} catch(e) {{ return JSON.stringify({{error: e.message}}); }}
+                }}""")
+                _cmt_data = json.loads(_cmt_json)
+                if "error" not in _cmt_data:
+                    for c in _cmt_data.get("comments", [])[:20]:
+                        user = c.get("user", {})
+                        result["comment_texts"].append({
+                            "nickname": user.get("nickname", ""),
+                            "text": c.get("text", ""),
+                            "likes": c.get("digg_count", 0),
+                        })
+    except Exception as e:
+        logger.debug(f"评论获取失败: {e}")
+
+    result["_method"] = "http_api"
+    result["_duration"] = round(time.time() - t0, 2)
+    return result
+
+
+async def _page_fallback(url: str, aweme_id: str) -> Optional[dict]:
+    """兜底策略：接口被平台过滤(filter_detail)时，用 Chrome 打开页面（网址层面浏览），
+    读页面标题 + meta description 补全标题/博主/发布时间。
+
+    返回 {title, author, published_at, source:'page_fallback'} 或 None（CDP 不可用/超时/解析失败）
+    """
+    try:
+        ctx = await _ensure_cdp()
+        if not ctx or not ctx.pages:
+            return None
+        page = ctx.pages[0]  # 复用现有页面，不新建标签页
+        try:
+            await page.goto(url, timeout=20000, wait_until="domcontentloaded")
+        except Exception:
+            await asyncio.sleep(2)
+        # 轮询等待页面加载出标题/meta（最多 12s，抖音页面加载/重定向较慢）
+        title, desc, author, published = "", "", "", ""
+        for _ in range(12):
+            try:
+                raw = await page.evaluate(
+                    """() => {
+                        const d = { title: document.title || '', desc: '' };
+                        const md = document.querySelector('meta[name="description"]');
+                        if (md) d.desc = md.getAttribute('content') || '';
+                        return JSON.stringify(d);
+                    }"""
+                )
+                data = json.loads(raw)
+                title = (data.get("title") or "").strip()
+                desc = (data.get("desc") or "").strip()
+                if title or desc:
+                    break
+            except Exception:
+                pass
+            await asyncio.sleep(1)
+
+        m = re.search(r"[-—]\s*([^\s于]+)于(\d{8})发布", desc)
+        if m:
+            author, published = m.group(1), m.group(2)
+        # 标题兜底：desc 去掉「- 作者于时间发布」后缀
+        if not title and desc:
+            title = re.sub(r"\s*[-—]\s*[^\s于]+于\d{8}发布.*$", "", desc).strip()
+        return {
+            "title": title,
+            "author": author,
+            "published_at": published,
+            "source": "page_fallback",
+        }
+    except Exception as e:
+        logger.debug(f"页面兜底失败: {e}")
+        return None
+
+
+# ── 博主采集（抖音博主监控） ──
+# ⚠️ 注意：profile/other 和 aweme/post 接口要用「数字 uid」，
+#    不能用 sec_user_id（会报 UserId不合法）。
+#    数字 uid 从视频详情 API 的 author.uid 字段获取。
+
+async def get_author_profile(uid: str) -> dict:
+    """获取博主主页信息（粉丝数/获赞数/作品数/昵称/签名）
+    
+    Args:
+        uid: 博主数字 ID（从视频详情的 author.uid 获取）
+    
+    Returns:
+        {"uid", "nickname", "unique_id", "fans", "total_favorited",
+         "works", "following", "signature", "avatar", "sec_uid", ...}
+        失败时含 "error" 字段
+    """
+    result = {"uid": str(uid)}
+    cookies = await _get_cookies()
+    has_session = bool(cookies.get("sessionid"))
+    if not has_session:
+        result["error"] = "抖音登录已过期"
+        result["login_expired"] = True
+        return result
+
+    try:
+        api_url = (
+            f"https://www.douyin.com/aweme/v1/web/user/profile/other/"
+            f"?user_id={uid}&device_platform=webapp&aid=6383"
+        )
+        data = await asyncio.to_thread(_http_get, api_url, cookies)
+    except urllib.error.HTTPError as e:
+        result["error"] = f"HTTP {e.code}: {e.reason}"
+        return result
+    except Exception as e:
+        result["error"] = f"请求失败: {e}"
+        return result
+
+    user = data.get("user", {})
+    if not user:
+        result["error"] = f"未返回用户数据: {data.get('status_msg', '')} (code={data.get('status_code')})"
+        return result
+
+    result.update({
+        "nickname": user.get("nickname", ""),
+        "unique_id": user.get("unique_id", ""),
+        "sec_uid": user.get("sec_uid", ""),
+        "fans": user.get("follower_count", 0),
+        "total_favorited": user.get("total_favorited", 0),
+        "works": user.get("aweme_count", 0),
+        "following": user.get("following_count", 0),
+        "signature": user.get("signature", ""),
+        "avatar": (user.get("avatar_thumb") or {}).get("url_list", [""])[0] if user.get("avatar_thumb") else "",
+        "collected_at": time.strftime("%Y-%m-%d %H:%M:%S"),
+    })
+    return result
+
+
+async def get_author_videos(uid: str, count: int = 20) -> dict:
+    """获取博主最新发布的视频列表（按发布时间倒序）
+    
+    Args:
+        uid: 博主数字 ID
+        count: 拉取条数（默认20）
+    
+    Returns:
+        {"uid", "videos": [{aweme_id, title, likes, comments, collects,
+                            shares, create_time, url}], "has_more"}
+    """
+    result = {"uid": str(uid), "videos": [], "has_more": False}
+    cookies = await _get_cookies()
+    has_session = bool(cookies.get("sessionid"))
+    if not has_session:
+        result["error"] = "抖音登录已过期"
+        result["login_expired"] = True
+        return result
+
+    try:
+        api_url = (
+            f"https://www.douyin.com/aweme/v1/web/aweme/post/"
+            f"?user_id={uid}&count={count}&max_cursor=0&device_platform=webapp&aid=6383"
+        )
+        data = await asyncio.to_thread(_http_get, api_url, cookies)
+    except urllib.error.HTTPError as e:
+        result["error"] = f"HTTP {e.code}: {e.reason}"
+        return result
+    except Exception as e:
+        result["error"] = f"请求失败: {e}"
+        return result
+
+    result["has_more"] = bool(data.get("has_more"))
+    for a in data.get("aweme_list", [])[:count]:
+        st = a.get("statistics", {})
+        result["videos"].append({
+            "aweme_id": str(a.get("aweme_id", "")),
+            "title": a.get("desc", ""),
+            "likes": int(st.get("digg_count", 0)),
+            "comments": int(st.get("comment_count", 0)),
+            "collects": int(st.get("collect_count", 0)),
+            "shares": int(st.get("share_count", 0)),
+            "create_time": a.get("create_time", 0),
+            "url": f"https://www.douyin.com/video/{a.get('aweme_id', '')}",
+        })
+    return result
+
+
+# ── 登录状态检测（从 cookie 文件直接读取，无窗口操作） ──
+
+async def check_login_status() -> dict:
+    """检测抖音登录状态（通过 CDP，无页面操作）"""
+    cookies = await _get_cookies()
+    has_session = bool(cookies.get("sessionid"))
+    return {
+        "logged_in": has_session,
+        "cookie_sessionid": has_session,
+        "cookies_count": len(cookies),
+    }
+
+
+# ── 打开登录页（用 AppleScript 控制 Chrome，不需要 CDP） ──
+
+async def open_login_page():
+    """在 Chrome 中打开抖音首页，让用户登录
+    
+    使用 macOS 的 open 命令（比 AppleScript 更可靠，不依赖辅助功能权限）。
+    登录后 cookie 自动保存到 Chrome profile，两个 Chrome 都会检测。
+    """
+    import subprocess as _sp
+    
+    try:
+        # 先用 open 命令让 Chrome 激活并打开抖音首页
+        # open -a 会激活已有的 Chrome 窗口（无论是日常还是采集）
+        _sp.run(
+            ["open", "-a", "Google Chrome", "https://www.douyin.com/"],
+            capture_output=True, timeout=10
+        )
+        logger.info("已请求 Chrome 打开抖音首页")
+        return {
+            "success": True,
+            "message": "已打开抖音首页，请在 Chrome 中点击右上角「登录」扫码",
+        }
+    except Exception as e:
+        logger.error(f"打开抖音首页失败: {e}")
+        return {"success": False, "error": f"打开失败: {e}"}

mac-agent-os-main/05_tools/10_dashboard/services/nurture_runner.sh

@@ -0,0 +1,103 @@
+#!/bin/bash
+# nurture_runner.sh — 养号执行包装器 v2.0
+# 负责：执行前清理 + 浏览器窗口定位 + mc run + 结果写入
+#
+# 用法:
+#   nurture_runner.sh <account_id> <blueprint> <rounds> [run_id] [slot] [pos_x] [pos_y]
+#
+# 窗口位置参数:
+#   slot: 槽位ID 1-4 (用于日志)
+#   pos_x, pos_y: 窗口左上角坐标（如 0 0 或 702 0）
+#
+# 结果写入: $AGENT_LOCAL/runtime/nurture/results/<run_id>.json
+
+set -o pipefail
+
+ACCOUNT=$1
+BLUEPRINT=$2
+ROUNDS=$3
+RUN_ID=${4:-$(date +%s)}
+SLOT=${5:-1}
+POS_X=${6:-0}
+POS_Y=${7:-0}
+
+# 环境设置
+export AGENT_SYNC="${AGENT_SYNC:-$HOME/workbuddy-agent-os/agent-sync}"
+export AGENT_LOCAL="${AGENT_LOCAL:-$HOME/workbuddy-agent-os/agent-local}"
+MC_PYTHON="${MC_PYTHON:-$HOME/.workbuddy/binaries/python/envs/agent-os/bin/python3}"
+SCRIPTS_DIR="$AGENT_SYNC/05_tools/07_matrix/scripts"
+
+# 结果目录
+RESULTS_DIR="$AGENT_LOCAL/runtime/nurture/results"
+LOGS_DIR="$AGENT_LOCAL/runtime/nurture/logs"
+mkdir -p "$RESULTS_DIR" "$LOGS_DIR"
+
+RESULT_FILE="$RESULTS_DIR/$RUN_ID.json"
+LOG_FILE="$LOGS_DIR/$RUN_ID.log"
+HOSTNAME=$(hostname -s)
+
+# ── 预清理：精确匹配同名账号的残留僵尸进程 ──
+# 用 pgrep 取 PID → ps 取命令行 → 只在 --accounts= 中精确匹配才杀
+for _pid in $(pgrep -f "mc run" 2>/dev/null || true); do
+  _cmd=$(ps -p $_pid -o command= 2>/dev/null || true)
+  if echo "$_cmd" | grep -Eq -- '--accounts=(.*,)?'$ACCOUNT'(,.*)?' 2>/dev/null; then
+    kill $_pid 2>/dev/null || true
+  fi
+done
+for _pid in $(pgrep -f "camoufox" 2>/dev/null || true); do
+  _cmd=$(ps -p $_pid -o command= 2>/dev/null || true)
+  if echo "$_cmd" | grep -Eq -- '(accounts|identity).*'$ACCOUNT 2>/dev/null; then
+    kill $_pid 2>/dev/null || true
+  fi
+done
+
+# ── 写初始状态 ──
+START_TS=$(date +%s)
+echo "{\"run_id\":\"$RUN_ID\",\"account\":\"$ACCOUNT\",\"blueprint\":\"$BLUEPRINT\",\"rounds\":$ROUNDS,\"status\":\"running\",\"hostname\":\"$HOSTNAME\",\"slot\":$SLOT,\"position\":[$POS_X,$POS_Y],\"started_at\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\"steps\":{\"total\":0,\"success\":0,\"failed\":0}}" > "$RESULT_FILE"
+
+# ── 执行（传入窗口位置参数）──
+cd "$SCRIPTS_DIR"
+PYTHONPATH="$SCRIPTS_DIR" $MC_PYTHON -m mc run \
+  --accounts="$ACCOUNT" \
+  --blueprints="$BLUEPRINT" \
+  --rounds="$ROUNDS" \
+  --mix --interval=45-90 > "$LOG_FILE" 2>&1
+EXIT_CODE=$?
+END_TS=$(date +%s)
+DURATION=$((END_TS - START_TS))
+
+# 解析日志统计
+STEPS_TOTAL=$(grep -o "总步骤: [0-9]*" "$LOG_FILE" | tail -1 | awk '{print $2}')
+STEPS_OK=$(grep -o "成功: [0-9]*" "$LOG_FILE" | tail -1 | awk '{print $2}')
+STEPS_FAIL=$(grep -o "失败: [0-9]*" "$LOG_FILE" | tail -1 | awk '{print $2}')
+
+STATUS="completed"
+[ $EXIT_CODE -ne 0 ] && STATUS="failed"
+
+LOG_TAIL=$(tail -200 "$LOG_FILE" | head -c 3000)
+
+cat > "$RESULT_FILE" << EOF
+{
+  "run_id": "$RUN_ID",
+  "account": "$ACCOUNT",
+  "blueprint": "$BLUEPRINT",
+  "rounds": $ROUNDS,
+  "status": "$STATUS",
+  "hostname": "$HOSTNAME",
+  "slot": $SLOT,
+  "position": [$POS_X, $POS_Y],
+  "started_at": "$(date -u -r $START_TS 2>/dev/null && echo -n "$(date -u -r $START_TS +%Y-%m-%dT%H:%M:%SZ)" || date -u +%Y-%m-%dT%H:%M:%SZ)",
+  "completed_at": "$(date -u -r $END_TS 2>/dev/null && echo -n "$(date -u -r $END_TS +%Y-%m-%dT%H:%M:%SZ)" || date -u +%Y-%m-%dT%H:%M:%SZ)",
+  "duration_secs": $DURATION,
+  "exit_code": $EXIT_CODE,
+  "steps": {
+    "total": ${STEPS_TOTAL:-0},
+    "success": ${STEPS_OK:-0},
+    "failed": ${STEPS_FAIL:-0}
+  },
+  "log_path": "$LOG_FILE"
+}
+EOF
+
+echo "{\"run_id\":\"$RUN_ID\",\"account\":\"$ACCOUNT\",\"slot\":$SLOT,\"status\":\"$STATUS\",\"steps\":{\"total\":${STEPS_TOTAL:-0},\"success\":${STEPS_OK:-0},\"failed\":${STEPS_FAIL:-0}},\"duration\":$DURATION}"
+exit $EXIT_CODE

mac-agent-os-main/05_tools/10_dashboard/services/operation_queue.py

@@ -0,0 +1,191 @@
+"""
+操作队列模块 — 管理操作生命周期 (状态机)
+
+⚠️ 已废弃 — 请使用 services/command_bus.py 替代。
+   本文件的文件持久化方案（JSON on disk）与新 command_bus 的内存方案共存。
+   新代码使用 command_bus，旧文件保留不删。
+   
+每个操作经过:
+  pending → checking → preparing → running → completed → cleaning → done
+                                   ↘ failed → cleaning → error
+
+存储: cross_machine/tasks/pending/{op_id}.json
+       cross_machine/tasks/history/{op_id}.json (完成后移入)
+"""
+
+import json, os, time, uuid
+from pathlib import Path
+from datetime import datetime, timezone
+
+from plugins.base import AGENT_SYNC, CROSS_MACHINE
+
+TASKS_DIR = CROSS_MACHINE / "tasks"
+PENDING_DIR = TASKS_DIR / "pending"
+HISTORY_DIR = TASKS_DIR / "history"
+
+# 状态常量
+STATE_PENDING = "pending"
+STATE_CHECKING = "checking"
+STATE_PREPARING = "preparing"
+STATE_RUNNING = "running"
+STATE_COMPLETED = "completed"
+STATE_FAILED = "failed"
+STATE_CLEANING = "cleaning"
+STATE_DONE = "done"
+STATE_ERROR = "error"
+STATE_CANCELLED = "cancelled"
+
+VALID_STATES = {
+    STATE_PENDING: [STATE_CHECKING, STATE_CANCELLED],
+    STATE_CHECKING: [STATE_PREPARING, STATE_FAILED, STATE_CANCELLED],
+    STATE_PREPARING: [STATE_RUNNING, STATE_FAILED, STATE_CANCELLED],
+    STATE_RUNNING: [STATE_COMPLETED, STATE_FAILED, STATE_CANCELLED],
+    STATE_COMPLETED: [STATE_CLEANING],
+    STATE_FAILED: [STATE_CLEANING],
+    STATE_CLEANING: [STATE_DONE, STATE_ERROR],
+    STATE_DONE: [],
+    STATE_ERROR: [],
+    STATE_CANCELLED: [],
+}
+
+
+def _now():
+    return datetime.now(timezone.utc).isoformat()
+
+
+def _ensure_dirs():
+    PENDING_DIR.mkdir(parents=True, exist_ok=True)
+    HISTORY_DIR.mkdir(parents=True, exist_ok=True)
+
+
+def submit_operation(op_type: str, target_machine: str, params: dict) -> dict:
+    """提交一个新操作"""
+    _ensure_dirs()
+    op_id = f"op_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{uuid.uuid4().hex[:6]}"
+    op = {
+        "operation_id": op_id,
+        "type": op_type,
+        "target_machine": target_machine,
+        "params": params,
+        "status": STATE_PENDING,
+        "states": [
+            {"state": STATE_PENDING, "time": _now(), "message": "操作已提交"}
+        ],
+        "created_at": _now(),
+        "updated_at": _now(),
+        "output": "",
+        "error": "",
+        "triggered_by": "dashboard",
+    }
+    filepath = PENDING_DIR / f"{op_id}.json"
+    filepath.write_text(json.dumps(op, indent=2, ensure_ascii=False))
+    return op
+
+
+def transition_state(op_id: str, new_state: str, message: str = "") -> bool:
+    """转换操作状态"""
+    op = get_operation(op_id)
+    if not op:
+        return False
+    
+    current = op["status"]
+    allowed = VALID_STATES.get(current, [])
+    if new_state not in allowed:
+        return False
+    
+    op["status"] = new_state
+    op["states"].append({"state": new_state, "time": _now(), "message": message or ""})
+    op["updated_at"] = _now()
+    
+    # 如果到达终态,移入 history
+    if new_state in (STATE_DONE, STATE_ERROR, STATE_CANCELLED):
+        _ensure_dirs()
+        old_path = PENDING_DIR / f"{op_id}.json"
+        new_path = HISTORY_DIR / f"{op_id}.json"
+        if old_path.exists():
+            old_path.write_text(json.dumps(op, indent=2, ensure_ascii=False))
+            old_path.rename(new_path)
+    else:
+        filepath = PENDING_DIR / f"{op_id}.json"
+        filepath.write_text(json.dumps(op, indent=2, ensure_ascii=False))
+    
+    return True
+
+
+def get_operation(op_id: str) -> dict | None:
+    """获取操作详情"""
+    for d in [PENDING_DIR, HISTORY_DIR]:
+        fp = d / f"{op_id}.json"
+        if fp.exists():
+            try:
+                return json.loads(fp.read_text())
+            except:
+                return None
+    return None
+
+
+def list_operations(status: str = "", limit: int = 50) -> list[dict]:
+    """列出操作"""
+    _ensure_dirs()
+    result = []
+    
+    # 当前 pending 的
+    for fp in sorted(PENDING_DIR.glob("*.json"), reverse=True)[:limit]:
+        try:
+            op = json.loads(fp.read_text())
+            if not status or op["status"] == status:
+                result.append(op)
+        except:
+            pass
+    
+    # 历史
+    if not status or status in (STATE_DONE, STATE_ERROR, STATE_CANCELLED):
+        for fp in sorted(HISTORY_DIR.glob("*.json"), reverse=True)[:limit]:
+            try:
+                op = json.loads(fp.read_text())
+                if not status or op["status"] == status:
+                    result.append(op)
+            except:
+                pass
+    
+    return result[:limit]
+
+
+def cancel_operation(op_id: str) -> bool:
+    """取消操作"""
+    return transition_state(op_id, STATE_CANCELLED, "用户取消")
+
+
+def update_operation(op_id: str, updates: dict) -> bool:
+    """更新操作数据 (输出/错误等)"""
+    op = get_operation(op_id)
+    if not op:
+        return False
+    for k, v in updates.items():
+        if k in ("output", "error", "params"):
+            op[k] = v
+    op["updated_at"] = _now()
+    filepath = PENDING_DIR / f"{op_id}.json"
+    if not filepath.exists():
+        filepath = HISTORY_DIR / f"{op_id}.json"
+    if filepath.exists():
+        filepath.write_text(json.dumps(op, indent=2, ensure_ascii=False))
+    return True
+
+
+def cleanup_stale_locks(max_age_sec: int = 3600):
+    """清理过期的资源锁"""
+    locks_dir = TASKS_DIR / "locks"
+    if not locks_dir.exists():
+        return
+    now = time.time()
+    for fp in locks_dir.glob("*.json"):
+        try:
+            data = json.loads(fp.read_text())
+            acquired = data.get("acquired_at", "")
+            if acquired:
+                acquired_ts = datetime.fromisoformat(acquired).timestamp()
+                if now - acquired_ts > max_age_sec:
+                    fp.unlink()
+        except:
+            fp.unlink()

mac-agent-os-main/05_tools/10_dashboard/services/preflight.py

@@ -0,0 +1,128 @@
+"""
+状态归零模块 — 操作执行前的准备和清理
+
+每次操作前自动执行:
+1. 杀残留浏览器进程 (camoufox)
+2. 杀残留 Playwright 驱动
+3. 清理临时文件
+4. 检查并释放过期锁
+5. 检查磁盘空间
+6. 返回 ready 状态
+"""
+
+import subprocess, logging, shutil, time
+from pathlib import Path
+
+logger = logging.getLogger("dashboard.preflight")
+TEMP_DIRS = ["/tmp/camoufox_*", "/tmp/playwright_*"]
+
+
+def kill_processes(name: str) -> dict:
+    """杀掉指定名称的进程"""
+    try:
+        r = subprocess.run(
+            ["pkill", "-f", name],
+            capture_output=True, text=True, timeout=5
+        )
+        # pkill returns 0 if killed, 1 if no process found (OK either way)
+        return {"killed": r.returncode == 0, "output": r.stdout or r.stderr or ""}
+    except subprocess.TimeoutExpired:
+        return {"killed": False, "output": "超时"}
+    except FileNotFoundError:
+        return {"killed": False, "output": "pkill not available"}
+
+
+def cleanup_temp_files() -> dict:
+    """清理临时文件"""
+    cleaned = 0
+    for pattern in TEMP_DIRS:
+        import glob
+        for fp in glob.glob(pattern):
+            try:
+                shutil.rmtree(fp, ignore_errors=True)
+                cleaned += 1
+            except:
+                pass
+    return {"cleaned_dirs": cleaned}
+
+
+def check_disk_space(min_gb: int = 5) -> dict:
+    """检查磁盘空间"""
+    try:
+        r = subprocess.run(["df", "-h", "/"], capture_output=True, text=True, timeout=5)
+        lines = r.stdout.strip().split("\n")
+        if len(lines) >= 2:
+            parts = lines[1].split()
+            if len(parts) >= 4:
+                avail = parts[3]
+                # 转换成GB
+                avail_gb = 0
+                if avail.endswith("G"):
+                    avail_gb = float(avail[:-1])
+                elif avail.endswith("M"):
+                    avail_gb = float(avail[:-1]) / 1024
+                elif avail.endswith("T"):
+                    avail_gb = float(avail[:-1]) * 1024
+                elif avail.endswith("K"):
+                    avail_gb = float(avail[:-1]) / 1024 / 1024
+                else:
+                    try:
+                        avail_gb = float(avail)
+                    except:
+                        avail_gb = 0
+                return {"available_gb": avail_gb, "sufficient": avail_gb >= min_gb}
+    except:
+        pass
+    return {"available_gb": 0, "sufficient": False}
+
+
+def check_browser_running() -> dict:
+    """检查是否有浏览器正在运行"""
+    try:
+        r = subprocess.run(["pgrep", "-f", "camoufox"], capture_output=True, text=True, timeout=5)
+        running = r.returncode == 0
+        return {"running": running, "pids": r.stdout.strip() if running else ""}
+    except:
+        return {"running": False}
+
+
+def run(machine: str = "") -> dict:
+    """执行完整的 preflight check
+    
+    Returns:
+        {"ready": True/False, "checks": {...}, "message": "..."}
+    """
+    checks = {}
+    all_ok = True
+
+    # 1. 杀残留浏览器
+    c1 = kill_processes("camoufox")
+    checks["kill_camoufox"] = c1
+    time.sleep(0.5)
+
+    # 2. 杀残留驱动
+    c2 = kill_processes("playwright")
+    checks["kill_playwright"] = c2
+    time.sleep(0.3)
+
+    # 3. 清理临时文件
+    c3 = cleanup_temp_files()
+    checks["cleanup_temp"] = c3
+
+    # 4. 检查磁盘
+    c4 = check_disk_space()
+    checks["disk_space"] = c4
+    if not c4.get("sufficient", True):
+        all_ok = False
+
+    # 5. 检查浏览器是否还在跑
+    c5 = check_browser_running()
+    checks["browser_still_running"] = c5
+    if c5.get("running"):
+        all_ok = False
+
+    return {
+        "ready": all_ok,
+        "checks": checks,
+        "message": "环境就绪" if all_ok else "环境未就绪，请查看 checks 详情",
+    }

mac-agent-os-main/05_tools/10_dashboard/services/remote_exec.py

@@ -0,0 +1,184 @@
+"""
+远程执行引擎
+
+⚠️ 已废弃 — 请使用 services/command_bus.py 替代。
+   本文件保留以供 app.py/routes 引用过渡使用，新代码禁止导入。
+   command_bus.py 的 MachineSession._send_remote() 已实现相同功能。
+
+通过 SSH 在远程机器执行命令，返回执行结果。
+机器信息从 ORACLE.yaml 读取。
+"""
+
+import logging, subprocess, sys
+from pathlib import Path
+
+logger = logging.getLogger("dashboard.remote_exec")
+
+_THIS_DIR = Path(__file__).resolve().parent.parent
+if str(_THIS_DIR) not in sys.path:
+    sys.path.insert(0, str(_THIS_DIR))
+
+_HOSTNAME_FILE = Path.home() / "workbuddy-agent-os" / "agent-local" / "identity" / "cached_hostname"
+_LOCAL_HOSTNAME = _HOSTNAME_FILE.read_text().strip() if _HOSTNAME_FILE.exists() else subprocess.run(["hostname"], capture_output=True, text=True).stdout.strip()
+
+ORACLE_PATH = _THIS_DIR.parent.parent / "ORACLE.yaml"
+_MACHINE_CACHE = None
+
+
+def _get_machine_info(machine_name: str) -> dict:
+    """从 ORACLE 查找机器信息"""
+    import yaml, os
+    global _MACHINE_CACHE
+    
+    if not ORACLE_PATH.exists():
+        return {}
+    
+    if _MACHINE_CACHE is None:
+        with open(ORACLE_PATH) as f:
+            _MACHINE_CACHE = yaml.safe_load(f)
+    
+    for name, info in _MACHINE_CACHE.get("machines", {}).items():
+        if name == machine_name or info.get("hostname") == machine_name:
+            return {
+                "name": name,
+                "hostname": info.get("hostname", name),
+                "ip": info.get("tailscale_ip", ""),
+                "user": info.get("ssh_user", ""),
+            }
+    return {}
+
+
+def exec_remote(machine: str, command: str, timeout: int = 60, fire_and_forget: bool = False) -> dict:
+    """在远程机器执行命令
+    
+    Args:
+        machine: 机器名（对应 ORACLE 中的定义）
+        command: 要执行的命令
+        timeout: 超时秒数
+        fire_and_forget: 是否立即返回（适合浏览器交互类命令）
+    
+    Returns:
+        {"status": "ok"/"error", "stdout": "...", "stderr": "...", "returncode": 0}
+    """
+    info = _get_machine_info(machine)
+    if not info or not info.get("ip"):
+        return {"status": "error", "message": f"未知机器: {machine}"}
+    
+    hostname = info.get("hostname", "")
+    user = info.get("user", "")
+    ip = info["ip"]
+    
+    # ── 本机：直接 subprocess ──
+    is_local = (machine == _LOCAL_HOSTNAME or hostname == _LOCAL_HOSTNAME)
+    if is_local:
+        try:
+            r = subprocess.run(
+                command, shell=True, capture_output=True, text=True, timeout=timeout
+            )
+            return {
+                "status": "ok",
+                "returncode": r.returncode,
+                "output": r.stdout[:5000],
+                "stderr": r.stderr[:500],
+            }
+        except subprocess.TimeoutExpired:
+            return {"status": "error", "message": f"命令超时({timeout}s)"}
+        except Exception as e:
+            return {"status": "error", "message": str(e)}
+    
+    # ── 远程：SSH ──
+    ssh_target = f"{user}@{ip}" if user else ip
+    
+    # 自动设置环境变量 + PYTHON 路径发现
+    py_discover = 'PY=$(ls $HOME/.workbuddy/binaries/python/envs/agent-os/bin/python3 2>/dev/null || which python3); '
+    env_setup = "export AGENT_SYNC=\"$HOME/workbuddy-agent-os/agent-sync\"; "
+    env_setup += "export AGENT_LOCAL=\"$HOME/workbuddy-agent-os/agent-local\"; "
+    env_setup += "export MC_PYTHON=\"$PY\"; "
+    
+    if fire_and_forget:
+        # 立即返回，不等待远程命令完成（适用浏览器登录等交互命令）
+        full_cmd = f"{py_discover} {env_setup} nohup {command} > /tmp/remote_exec.log 2>&1 &"
+        try:
+            subprocess.run(
+                ["ssh", "-o", "ConnectTimeout=10", "-o", "StrictHostKeyChecking=no",
+                 ssh_target, full_cmd],
+                capture_output=True, text=True, timeout=15
+            )
+            return {"status": "ok", "message": "命令已发送", "fire_and_forget": True}
+        except Exception as e:
+            return {"status": "error", "message": str(e)}
+    
+    full_cmd = f"{py_discover} {env_setup} {command}"
+    try:
+        r = subprocess.run(
+            ["ssh", "-o", "ConnectTimeout=10", "-o", "StrictHostKeyChecking=no",
+             ssh_target, full_cmd],
+            capture_output=True, text=True, timeout=timeout
+        )
+        return {
+            "status": "ok" if r.returncode == 0 else "error",
+            "returncode": r.returncode,
+            "stdout": r.stdout[:5000],
+            "stderr": r.stderr[:500],
+        }
+    except subprocess.TimeoutExpired:
+        return {"status": "error", "message": f"执行超时 ({timeout}s)"}
+    except FileNotFoundError:
+        return {"status": "error", "message": "ssh 命令未找到"}
+    except Exception as e:
+        return {"status": "error", "message": str(e)}
+
+
+def exec_nurture(machine: str, accounts: list, blueprints: list, rounds: int = 3) -> dict:
+    """在远程机器启动养号"""
+    accts = ",".join(accounts)
+    bps = ",".join(blueprints)
+    cmd = f"cd $AGENT_SYNC/05_tools/07_matrix/scripts && $MC_PYTHON -m mc run --accounts={accts} --blueprints={bps} --rounds={rounds} --mix --interval=45-90"
+    return exec_remote(machine, cmd, timeout=30)
+
+
+def exec_collect(machine: str, phone: str = "") -> dict:
+    """在远程机器启动采集"""
+    cmd = f"cd $AGENT_SYNC/05_tools/07_matrix/scripts && $MC_PYTHON -m mc collect"
+    if phone:
+        cmd += f" --phone {phone}"
+    else:
+        cmd += " --all"
+    return exec_remote(machine, cmd, timeout=30)
+
+
+def exec_login(machine: str, account_id: str) -> dict:
+    """在远程机器智能登录（全自动，立即返回）"""
+    cmd = f"cd $AGENT_SYNC/05_tools/07_matrix/scripts && $MC_PYTHON -m mc smart-login {account_id}"
+    return exec_remote(machine, cmd, timeout=120, fire_and_forget=True)
+
+
+def exec_logout(machine: str, account_id: str) -> dict:
+    """在远程机器清除账号登录状态"""
+    cmd = f"cd $AGENT_SYNC/05_tools/07_matrix/scripts && $MC_PYTHON -m mc account logout {account_id}"
+    return exec_remote(machine, cmd, timeout=30)
+
+
+def exec_comment(machine: str, account_id: str, url: str, direction: str = "") -> dict:
+    """在远程机器执行定向评论"""
+    dir_flag = f" --direction {direction}" if direction else ""
+    cmd = f"cd $AGENT_SYNC/05_tools/07_matrix/scripts && $MC_PYTHON -m mc task comment --account={account_id} --url={url}{dir_flag}"
+    return exec_remote(machine, cmd, timeout=120)
+
+
+def exec_record(machine: str, account_id: str, platform: str = "douyin") -> dict:
+    """在远程机器启动交互式录制"""
+    cmd = f"cd $AGENT_SYNC/05_tools/07_matrix/scripts/mc && $MC_PYTHON recorder.py {account_id} {platform}"
+    return exec_remote(machine, cmd, timeout=30, fire_and_forget=True)
+
+
+def exec_nurture_stop(machine: str) -> dict:
+    """停止远程机器的养号任务"""
+    cmd = "pkill -f 'mc run' 2>/dev/null; pkill -f camoufox 2>/dev/null; echo 'stopped'"
+    return exec_remote(machine, cmd, timeout=15)
+
+
+def exec_status(machine: str) -> dict:
+    """获取远程机器的完整状态"""
+    cmd = "echo '---guardd---'; pgrep -f guardd && echo 'running' || echo 'stopped'; echo '---camoufox---'; pgrep -f camoufox && echo 'running' || echo 'stopped'; echo '---disk---'; df -h / | tail -1 | awk '{print $3 \"/\" $2 \" (\" $5 \")\"}'; echo '---load---'; uptime | awk -F'load averages:' '{print $2}'"
+    return exec_remote(machine, cmd, timeout=10)

mac-agent-os-main/05_tools/10_dashboard/services/resource_lock.py

@@ -0,0 +1,145 @@
+"""
+⚠️ 已废弃 — 请使用 services/browser_orchestrator.py 替代。
+   本文件保留供 app.py 过渡使用，新代码禁止导入。
+
+资源锁模块 — 浏览器/账号/身份互斥
+
+管理三把锁:
+  - browser:   每台机器只能开一个浏览器
+  - account:   同一账号不能同时被两个操作使用
+  - identity:  同一身份不能同时被两个浏览器使用
+
+锁文件路径: cross_machine/tasks/locks/{resource_type}/{resource_id}.json
+"""
+
+import json, time
+from pathlib import Path
+from datetime import datetime, timezone
+
+from plugins.base import CROSS_MACHINE
+
+LOCKS_DIR = CROSS_MACHINE / "tasks" / "locks"
+DEFAULT_TTL = 3600  # 1小时自动过期
+
+
+def _now():
+    return datetime.now(timezone.utc).isoformat()
+
+
+def _lock_path(resource_type: str, resource_id: str) -> Path:
+    d = LOCKS_DIR / resource_type
+    d.mkdir(parents=True, exist_ok=True)
+    return d / f"{resource_id}.json"
+
+
+def acquire_lock(resource_type: str, resource_id: str,
+                 holder: str, ttl: int = DEFAULT_TTL) -> dict:
+    """获取资源锁
+    
+    Returns:
+        {"success": True} 或 {"success": False, "held_by": "...", "message": "..."}
+    """
+    fp = _lock_path(resource_type, resource_id)
+    
+    # 检查是否已被占用
+    if fp.exists():
+        try:
+            existing = json.loads(fp.read_text())
+            # 检查 TTL
+            acquired = existing.get("acquired_at", "")
+            if acquired:
+                acquired_ts = datetime.fromisoformat(acquired).timestamp()
+                if time.time() - acquired_ts < ttl:
+                    return {
+                        "success": False,
+                        "held_by": existing.get("held_by", "未知"),
+                        "message": f"{resource_type}:{resource_id} 已被 {existing.get('held_by', '未知')} 占用"
+                    }
+                else:
+                    # TTL 过期, 覆盖
+                    pass
+        except:
+            pass
+    
+    lock = {
+        "resource_type": resource_type,
+        "resource_id": resource_id,
+        "held_by": holder,
+        "acquired_at": _now(),
+        "ttl": ttl,
+    }
+    fp.write_text(json.dumps(lock, indent=2))
+    return {"success": True}
+
+
+def release_lock(resource_type: str, resource_id: str, holder: str = "") -> bool:
+    """释放资源锁"""
+    fp = _lock_path(resource_type, resource_id)
+    if not fp.exists():
+        return True
+    if holder:
+        try:
+            data = json.loads(fp.read_text())
+            if data.get("held_by") != holder:
+                return False  # 不是自己的锁, 不能释放
+        except:
+            pass
+    fp.unlink(missing_ok=True)
+    return True
+
+
+def check_lock(resource_type: str, resource_id: str) -> dict:
+    """检查资源锁状态"""
+    fp = _lock_path(resource_type, resource_id)
+    if not fp.exists():
+        return {"locked": False}
+    try:
+        data = json.loads(fp.read_text())
+        return {"locked": True, "held_by": data.get("held_by", "?"), "since": data.get("acquired_at", "")}
+    except:
+        return {"locked": False}
+
+
+def acquire_browser_lock(machine: str, holder: str) -> dict:
+    """获取浏览器锁 (每机器全局一把)"""
+    return acquire_lock("browser", machine, holder)
+
+
+def release_browser_lock(machine: str, holder: str = "") -> bool:
+    """释放浏览器锁"""
+    return release_lock("browser", machine, holder)
+
+
+def acquire_account_lock(account_id: str, holder: str) -> dict:
+    """获取账号锁"""
+    return acquire_lock("account", account_id, holder)
+
+
+def release_account_lock(account_id: str, holder: str = "") -> bool:
+    """释放账号锁"""
+    return release_lock("account", account_id, holder)
+
+
+def acquire_identity_lock(identity_dir: str, holder: str) -> dict:
+    """获取身份锁"""
+    return acquire_lock("identity", identity_dir, holder)
+
+
+def release_identity_lock(identity_dir: str, holder: str = "") -> bool:
+    """释放身份锁"""
+    return release_lock("identity", identity_dir, holder)
+
+
+def release_all_locks(holder: str):
+    """释放指定持有者的所有锁"""
+    for resource_type in ["browser", "account", "identity"]:
+        d = LOCKS_DIR / resource_type
+        if not d.exists():
+            continue
+        for fp in d.glob("*.json"):
+            try:
+                data = json.loads(fp.read_text())
+                if data.get("held_by") == holder:
+                    fp.unlink(missing_ok=True)
+            except:
+                fp.unlink(missing_ok=True)

mac-agent-os-main/05_tools/10_dashboard/services/scrape_db.py

@@ -0,0 +1,415 @@
+"""
+scrape_db.py — 抓取系统 SQLite 数据库层 v1
+
+表结构：
+  - scrape_tasks:     抓取任务（单次/批量/定时）
+  - scrape_items:     抓取内容（带统一 schema，按 platform+item_id 去重）
+  - scrape_comments:  评论区（按 item_id 关联）
+  - scrape_sources:   抓取源（定时监控用）
+"""
+import json, sqlite3, time, os
+from datetime import datetime
+from pathlib import Path
+from typing import Optional
+
+_THIS_DIR = Path(__file__).resolve().parent
+AGENT_LOCAL = Path(os.environ.get("AGENT_LOCAL",
+                   str(Path.home() / "workbuddy-agent-os" / "agent-local")))
+DEFAULT_DB = AGENT_LOCAL / "data" / "scrape.db"
+
+SCHEMA_SQL = """
+CREATE TABLE IF NOT EXISTS scrape_tasks (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    type TEXT NOT NULL DEFAULT 'single',                    -- 'single'/'batch'/'scheduled'
+    platform TEXT,                                          -- 'douyin'/'xhs'/'bilibili'/'zhihu'/'web'
+    target TEXT NOT NULL,                                   -- 原始输入(URL/sec_uid/关键词)
+    depth TEXT DEFAULT 'light',                             -- 'light'/'deep'
+    tool_level INTEGER DEFAULT 2,                           -- 1/2/3
+    machine TEXT,                                           -- 执行机器
+    status TEXT DEFAULT 'pending',                          -- 'pending'/'running'/'completed'/'failed'
+    total_targets INTEGER DEFAULT 0,
+    completed_targets INTEGER DEFAULT 0,
+    summary TEXT,                                           -- 摘要 JSON
+    created_at TEXT DEFAULT (datetime('now','localtime')),
+    completed_at TEXT
+);
+
+CREATE TABLE IF NOT EXISTS scrape_items (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    task_id INTEGER REFERENCES scrape_tasks(id),
+    platform TEXT NOT NULL,
+    item_id TEXT NOT NULL,                                  -- 平台内唯一 ID
+    url TEXT,
+    title TEXT,
+    author_name TEXT,
+    author_id TEXT,
+    published_at TEXT,
+    collected_at TEXT DEFAULT (datetime('now','localtime')),
+    text_content TEXT,
+    tags TEXT,                                              -- JSON array
+    stats TEXT,                                             -- JSON: {likes,comments,shares,views}
+    extra TEXT,                                             -- JSON: 平台特有字段
+    media TEXT,                                             -- JSON: [{type,url,duration}]
+    ai_analysis TEXT,                                       -- JSON: L2 深度分析
+    UNIQUE(platform, item_id)
+);
+
+CREATE TABLE IF NOT EXISTS scrape_comments (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    item_db_id INTEGER REFERENCES scrape_items(id),
+    author_name TEXT,
+    text TEXT,
+    likes INTEGER DEFAULT 0,
+    replied_at TEXT
+);
+
+CREATE TABLE IF NOT EXISTS scrape_sources (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    platform TEXT NOT NULL,
+    source_type TEXT NOT NULL,                              -- 'user'/'hashtag'/'keyword'/'url_list'/'api'
+    target TEXT NOT NULL,                                   -- sec_uid/话题ID/关键词/URL/API地址
+    display_name TEXT,
+    category TEXT DEFAULT '',                               -- 类别（自定义分类标签，如 肛肠科/减重）
+    notes TEXT DEFAULT '',                                  -- 说明（备注这个源是什么情况）
+    schedule TEXT,                                          -- CRON 表达式
+    depth TEXT DEFAULT 'light',
+    tool_level INTEGER DEFAULT 2,
+    last_collected TEXT,
+    status TEXT DEFAULT 'active',                           -- 'active'/'paused'
+    UNIQUE(platform, source_type, target)
+);
+"""
+
+
+class ScrapeDB:
+    """抓取数据库操作"""
+
+    def __init__(self, db_path: str = None):
+        self.db_path = db_path or str(DEFAULT_DB)
+        Path(self.db_path).parent.mkdir(parents=True, exist_ok=True)
+        self._init_db()
+
+    def _get_conn(self):
+        conn = sqlite3.connect(self.db_path)
+        conn.row_factory = sqlite3.Row
+        conn.execute("PRAGMA journal_mode=WAL")
+        return conn
+
+    def _init_db(self):
+        conn = self._get_conn()
+        try:
+            conn.executescript(SCHEMA_SQL)
+            # 迁移：旧库补 category/notes 列（SQLite ALTER TABLE）
+            cols = {r[1] for r in conn.execute("PRAGMA table_info(scrape_sources)").fetchall()}
+            for col in ("category", "notes"):
+                if col not in cols:
+                    conn.execute(f"ALTER TABLE scrape_sources ADD COLUMN {col} TEXT DEFAULT ''")
+            conn.commit()
+        finally:
+            conn.close()
+
+    # ── 任务管理 ──
+
+    def create_task(self, type: str, platform: str, target: str,
+                    depth: str = "light", tool_level: int = 2,
+                    machine: str = "", total_targets: int = 1) -> int:
+        conn = self._get_conn()
+        try:
+            cur = conn.execute(
+                "INSERT INTO scrape_tasks (type,platform,target,depth,tool_level,machine,total_targets) VALUES (?,?,?,?,?,?,?)",
+                (type, platform, target, depth, tool_level, machine, total_targets)
+            )
+            conn.commit()
+            return cur.lastrowid
+        finally:
+            conn.close()
+
+    def update_task_status(self, task_id: int, status: str,
+                           summary: str = None, completed: int = None):
+        conn = self._get_conn()
+        try:
+            sets = ["status=?"]
+            params = [status]
+            if summary:
+                sets.append("summary=?")
+                params.append(summary)
+            if completed:
+                sets.append("completed_targets=?")
+                params.append(completed)
+            if status in ("completed", "failed"):
+                sets.append("completed_at=datetime('now','localtime')")
+            params.append(task_id)
+            conn.execute(f"UPDATE scrape_tasks SET {','.join(sets)} WHERE id=?", params)
+            conn.commit()
+        finally:
+            conn.close()
+
+    def get_task(self, task_id: int) -> Optional[dict]:
+        conn = self._get_conn()
+        try:
+            row = conn.execute("SELECT * FROM scrape_tasks WHERE id=?", (task_id,)).fetchone()
+            return dict(row) if row else None
+        finally:
+            conn.close()
+
+    def list_tasks(self, status: str = None, platform: str = None,
+                   limit: int = 50) -> list[dict]:
+        conn = self._get_conn()
+        try:
+            where = []
+            params = []
+            if status:
+                where.append("status=?")
+                params.append(status)
+            if platform:
+                where.append("platform=?")
+                params.append(platform)
+            sql = "SELECT * FROM scrape_tasks"
+            if where:
+                sql += " WHERE " + " AND ".join(where)
+            sql += " ORDER BY id DESC LIMIT ?"
+            params.append(limit)
+            return [dict(r) for r in conn.execute(sql, params).fetchall()]
+        finally:
+            conn.close()
+
+    # ── 抓取内容 ──
+
+    def insert_item(self, task_id: int, platform: str, item_id: str,
+                    url: str = "", title: str = "", author_name: str = "",
+                    author_id: str = "", published_at: str = "",
+                    text_content: str = "", tags: list = None,
+                    stats: dict = None, extra: dict = None,
+                    media: list = None) -> int:
+        conn = self._get_conn()
+        try:
+            cur = conn.execute(
+                """INSERT OR IGNORE INTO scrape_items
+                   (task_id,platform,item_id,url,title,author_name,author_id,
+                    published_at,text_content,tags,stats,extra,media)
+                   VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)""",
+                (task_id, platform, item_id, url, title, author_name, author_id,
+                 published_at, text_content,
+                 json.dumps(tags or [], ensure_ascii=False),
+                 json.dumps(stats or {}, ensure_ascii=False),
+                 json.dumps(extra or {}, ensure_ascii=False),
+                 json.dumps(media or [], ensure_ascii=False))
+            )
+            conn.commit()
+            return cur.lastrowid or self.get_item_id(platform, item_id)
+        finally:
+            conn.close()
+
+    def get_item_id(self, platform: str, item_id: str) -> Optional[int]:
+        conn = self._get_conn()
+        try:
+            row = conn.execute(
+                "SELECT id FROM scrape_items WHERE platform=? AND item_id=?",
+                (platform, item_id)
+            ).fetchone()
+            return row["id"] if row else None
+        finally:
+            conn.close()
+
+    def item_exists(self, platform: str, item_id: str) -> bool:
+        return self.get_item_id(platform, item_id) is not None
+
+    def get_item(self, db_id: int) -> Optional[dict]:
+        conn = self._get_conn()
+        try:
+            row = conn.execute("SELECT * FROM scrape_items WHERE id=?", (db_id,)).fetchone()
+            if not row:
+                return None
+            item = dict(row)
+            for f in ("tags", "stats", "extra", "media", "ai_analysis"):
+                if item.get(f):
+                    try:
+                        item[f] = json.loads(item[f])
+                    except:
+                        pass
+            return item
+        finally:
+            conn.close()
+
+    def list_items(self, platform: str = None, author_id: str = None,
+                   limit: int = 100) -> list[dict]:
+        conn = self._get_conn()
+        try:
+            where = []
+            params = []
+            if platform:
+                where.append("platform=?")
+                params.append(platform)
+            if author_id:
+                where.append("author_id=?")
+                params.append(author_id)
+            sql = "SELECT * FROM scrape_items"
+            if where:
+                sql += " WHERE " + " AND ".join(where)
+            sql += " ORDER BY collected_at DESC LIMIT ?"
+            params.append(limit)
+            return [dict(r) for r in conn.execute(sql, params).fetchall()]
+        finally:
+            conn.close()
+
+    # ── 评论 ──
+
+    def insert_comments(self, item_db_id: int, comments: list[dict]):
+        if not comments:
+            return
+        conn = self._get_conn()
+        try:
+            for c in comments:
+                conn.execute(
+                    "INSERT INTO scrape_comments (item_db_id,author_name,text,likes,replied_at) VALUES (?,?,?,?,?)",
+                    (item_db_id, c.get("nickname", c.get("author", "")),
+                     c.get("text", ""), c.get("digg_count", c.get("likes", 0)),
+                     c.get("create_time", ""))
+                )
+            conn.commit()
+        finally:
+            conn.close()
+
+    def get_comments(self, item_db_id: int) -> list[dict]:
+        conn = self._get_conn()
+        try:
+            return [dict(r) for r in conn.execute(
+                "SELECT * FROM scrape_comments WHERE item_db_id=? ORDER BY likes DESC",
+                (item_db_id,)).fetchall()]
+        finally:
+            conn.close()
+
+    # ── 抓取源（定时用）──
+
+    def upsert_source(self, platform: str, source_type: str, target: str,
+                      display_name: str = "", schedule: str = "",
+                      depth: str = "light", tool_level: int = 2,
+                      category: str = "", notes: str = ""):
+        conn = self._get_conn()
+        try:
+            conn.execute(
+                """INSERT INTO scrape_sources
+                   (platform,source_type,target,display_name,category,notes,schedule,depth,tool_level)
+                   VALUES (?,?,?,?,?,?,?,?,?)
+                   ON CONFLICT(platform,source_type,target) DO UPDATE SET
+                   display_name=excluded.display_name, category=excluded.category,
+                   notes=excluded.notes, schedule=excluded.schedule,
+                   depth=excluded.depth, tool_level=excluded.tool_level,
+                   status='active'""",
+                (platform, source_type, target, display_name, category, notes, schedule, depth, tool_level)
+            )
+            conn.commit()
+        finally:
+            conn.close()
+
+    def update_source(self, source_id: int, **fields) -> bool:
+        """编辑抓取源（只更新传入的非空字段）"""
+        allowed = {"display_name", "category", "notes", "source_type", "target", "status", "depth", "tool_level"}
+        sets, vals = [], []
+        for k, v in fields.items():
+            if k in allowed and v is not None:
+                sets.append(f"{k}=?")
+                vals.append(v)
+        if not sets:
+            return False
+        vals.append(source_id)
+        conn = self._get_conn()
+        try:
+            conn.execute(f"UPDATE scrape_sources SET {', '.join(sets)} WHERE id=?", vals)
+            conn.commit()
+            return True
+        finally:
+            conn.close()
+
+    def list_sources(self, status: str = None) -> list[dict]:
+        conn = self._get_conn()
+        try:
+            if status:
+                rows = conn.execute(
+                    "SELECT * FROM scrape_sources WHERE status=? ORDER BY platform,target",
+                    (status,)).fetchall()
+            else:
+                rows = conn.execute(
+                    "SELECT * FROM scrape_sources ORDER BY platform,target").fetchall()
+            return [dict(r) for r in rows]
+        finally:
+            conn.close()
+
+    def get_due_sources(self) -> list[dict]:
+        """返回所有到期的定时抓取源"""
+        conn = self._get_conn()
+        try:
+            rows = conn.execute(
+                """SELECT * FROM scrape_sources
+                   WHERE status='active'
+                   AND (last_collected IS NULL
+                        OR datetime('now','localtime','-2 hours') > last_collected)"""
+            ).fetchall()
+            return [dict(r) for r in rows]
+        finally:
+            conn.close()
+
+    def update_source_collected(self, source_id: int):
+        conn = self._get_conn()
+        try:
+            conn.execute(
+                "UPDATE scrape_sources SET last_collected=datetime('now','localtime') WHERE id=?",
+                (source_id,))
+            conn.commit()
+        finally:
+            conn.close()
+
+    def delete_source(self, source_id: int):
+        conn = self._get_conn()
+        try:
+            conn.execute("DELETE FROM scrape_sources WHERE id=?", (source_id,))
+            conn.commit()
+        finally:
+            conn.close()
+
+    # ── 统计 ──
+
+    def count_by_platform(self) -> dict:
+        conn = self._get_conn()
+        try:
+            rows = conn.execute(
+                "SELECT platform, COUNT(*) as cnt FROM scrape_items GROUP BY platform"
+            ).fetchall()
+            result = {}
+            for r in rows:
+                result[r["platform"]] = r["cnt"]
+            return result
+        finally:
+            conn.close()
+
+    def count_today(self) -> int:
+        conn = self._get_conn()
+        try:
+            row = conn.execute(
+                "SELECT COUNT(*) as cnt FROM scrape_items WHERE date(collected_at)=date('now','localtime')"
+            ).fetchone()
+            return row["cnt"] if row else 0
+        finally:
+            conn.close()
+
+    def sources_count(self) -> int:
+        conn = self._get_conn()
+        try:
+            row = conn.execute("SELECT COUNT(*) as cnt FROM scrape_sources WHERE status='active'").fetchone()
+            return row["cnt"] if row else 0
+        finally:
+            conn.close()
+
+    def task_stats(self) -> dict:
+        conn = self._get_conn()
+        try:
+            rows = conn.execute(
+                "SELECT status, COUNT(*) as cnt FROM scrape_tasks GROUP BY status"
+            ).fetchall()
+            result = {"total": 0}
+            for r in rows:
+                result[r["status"]] = r["cnt"]
+                result["total"] += r["cnt"]
+            return result
+        finally:
+            conn.close()

mac-agent-os-main/05_tools/10_dashboard/services/scrape_engine.py

@@ -0,0 +1,361 @@
+"""
+scrape_engine.py — 抓取系统核心引擎 v1
+
+职责：
+  1. URL 解析 → 标准化目标
+  2. 适配器路由（按平台选择适配器）
+  3. 工具降级执行
+  4. 结果入库（SQLite，去重）
+  5. 多机分发（通过 command_bus）
+  6. 异步任务管理（大量目标用轮询）
+"""
+import asyncio, json, logging, os, time, uuid
+from datetime import datetime
+from pathlib import Path
+from typing import Optional
+
+from services.scrape_db import ScrapeDB
+from services.adapters.douyin_scrape import DouyinScrapeAdapter
+from services.adapters.xhs_scrape import XhsScrapeAdapter
+from services.adapters.bilibili_scrape import BilibiliScrapeAdapter
+from services.adapters.zhihu_scrape import ZhihuScrapeAdapter
+from services.adapters.web_scrape import WebScrapeAdapter
+
+logger = logging.getLogger("dashboard.scrape.engine")
+
+_THIS_DIR = Path(__file__).resolve().parent
+
+
+class ScrapeEngine:
+    """抓取引擎"""
+
+    def __init__(self):
+        self.db = ScrapeDB()
+        self._adapters = {}
+        self._tasks = {}           # 异步任务缓存 {run_id: status}
+
+    def _get_adapter(self, platform: str):
+        """获取/缓存适配器实例"""
+        if platform not in self._adapters:
+            if platform == "douyin":
+                self._adapters[platform] = DouyinScrapeAdapter()
+            elif platform in ("xiaohongshu", "xhs"):
+                self._adapters[platform] = XhsScrapeAdapter()
+            elif platform == "bilibili":
+                self._adapters[platform] = BilibiliScrapeAdapter()
+            elif platform == "zhihu":
+                self._adapters[platform] = ZhihuScrapeAdapter()
+            elif platform == "web":
+                self._adapters[platform] = WebScrapeAdapter()
+            else:
+                raise ValueError(f"不支持的平台: {platform}")
+        return self._adapters[platform]
+
+    # ── URL 解析 ──
+
+    def resolve_target(self, raw: str) -> dict:
+        """解析单个目标 → 标准化格式"""
+        raw = raw.strip()
+        import re
+
+        # ── 抖音 ──
+        # 抖音短链: v.douyin.com/xxx
+        if "v.douyin.com" in raw:
+            return {"input": raw, "platform": "douyin", "type": "shortlink",
+                    "status": "pending"}
+        # 抖音视频页: douyin.com/video/{id}
+        if "douyin.com/video/" in raw:
+            m = re.search(r'/video/(\d+)', raw)
+            return {"input": raw, "platform": "douyin", "type": "video",
+                    "target_id": m.group(1) if m else raw, "status": "resolved"}
+        # 抖音用户主页: douyin.com/user/{sec_uid}
+        if "douyin.com/user/" in raw:
+            sec_uid = raw.split("/user/")[-1].split("?")[0]
+            return {"input": raw, "platform": "douyin", "type": "user",
+                    "target_id": sec_uid, "status": "resolved"}
+
+        # ── 小红书 ──
+        if "xiaohongshu.com" in raw:
+            m = re.search(r'/explore/([a-f0-9]+)', raw)
+            if m:
+                return {"input": raw, "platform": "xiaohongshu", "type": "note",
+                        "target_id": m.group(1), "status": "resolved"}
+
+        # ── B站 ──
+        if "bilibili.com" in raw:
+            # bilibili.com/video/BVxxx
+            m = re.search(r'/video/([a-zA-Z0-9]+)', raw)
+            if m:
+                return {"input": raw, "platform": "bilibili", "type": "video",
+                        "target_id": m.group(1), "status": "resolved"}
+            # bilibili.com/space/{mid}
+            m = re.search(r'/space/(\d+)', raw)
+            if m:
+                return {"input": raw, "platform": "bilibili", "type": "user",
+                        "target_id": m.group(1), "status": "resolved"}
+
+        # ── 知乎 ──
+        if "zhihu.com" in raw:
+            # zhihu.com/answer/{id}
+            m = re.search(r'/answer/(\d+)', raw)
+            if m:
+                return {"input": raw, "platform": "zhihu", "type": "item",
+                        "target_id": m.group(1), "status": "resolved"}
+            # zhihu.com/question/{id}
+            m = re.search(r'/question/(\d+)', raw)
+            if m:
+                return {"input": raw, "platform": "zhihu", "type": "item",
+                        "target_id": m.group(1), "status": "resolved"}
+
+        # ── 识别 URL（通用网页）──
+        if raw.startswith("http://") or raw.startswith("https://"):
+            return {"input": raw, "platform": "web", "type": "page",
+                    "target_id": raw, "status": "resolved"}
+
+        # ── 纯 sec_uid（抖音）──
+        if raw.startswith("MS4w") or len(raw) > 20:
+            return {"input": raw, "platform": "douyin", "type": "user",
+                    "target_id": raw, "status": "resolved"}
+        # 纯数字（可能是 aweme_id）
+        if raw.isdigit() and len(raw) >= 15:
+            return {"input": raw, "platform": "douyin", "type": "video",
+                    "target_id": raw, "status": "resolved"}
+        # 纯数字（知乎 answer_id）
+        if raw.isdigit():
+            return {"input": raw, "platform": "zhihu", "type": "item",
+                    "target_id": raw, "status": "resolved"}
+        return {"input": raw, "platform": "unknown", "type": "unknown",
+                "status": "unresolved"}
+
+    def resolve_urls(self, targets: list) -> list[dict]:
+        """批量解析 URL"""
+        return [self.resolve_target(t) for t in targets]
+
+    # ── URL 短链解析 ──
+
+    def _resolve_shortlink(self, url: str) -> Optional[str]:
+        """解析抖音短链 → aweme_id"""
+        import subprocess, re
+        try:
+            result = subprocess.run(
+                ["curl", "-sI", url],
+                capture_output=True, text=True, timeout=10
+            )
+            loc = ""
+            for line in result.stdout.split("\n"):
+                if line.lower().startswith("location:"):
+                    loc = line.split(":", 1)[1].strip()
+                    break
+            if loc:
+                m = re.search(r'/video/(\d+)', loc)
+                if m:
+                    return m.group(1)
+                # iesdouyin.com 格式
+                m = re.search(r'/share/video/(\d+)', loc)
+                if m:
+                    return m.group(1)
+            return None
+        except:
+            return None
+
+    # ── 核心执行 ──
+
+    async def run(self, request: dict) -> dict:
+        """
+        执行抓取任务
+
+        request:
+          targets: [str]         — 目标列表
+          platform: str          — 自动识别时留空
+          depth: "light"         — "light"/"deep"
+          tool_level: int        — 1/2/3
+          machine: str           — 指定机器（空=本机）
+          multi_machine: bool    — 是否多机协同
+          async_mode: bool       — 是否异步（大量目标用）
+        """
+        t0 = time.time()
+        targets = request.get("targets", [])
+        platform = request.get("platform", "auto")
+        depth = request.get("depth", "light")
+        tool_level = request.get("tool_level", 2)
+        machine = request.get("machine", "")
+        multi_machine = request.get("multi_machine", False)
+        async_mode = request.get("async_mode", False) or len(targets) > 50
+
+        # 1. 解析目标
+        resolved = self.resolve_urls(targets)
+        unresolved = [r for r in resolved if r["status"] == "unresolved"]
+        shortlinks = [r for r in resolved if r["type"] == "shortlink"]
+
+        # 2. 解析短链
+        for sl in shortlinks:
+            aweme_id = self._resolve_shortlink(sl["input"])
+            if aweme_id:
+                sl["type"] = "video"
+                sl["target_id"] = aweme_id
+                sl["platform"] = "douyin"
+                sl["status"] = "resolved"
+            else:
+                sl["status"] = "unresolved"
+
+        ready = [r for r in resolved if r["status"] == "resolved"]
+
+        # 3. 异步模式 → 创建后台任务
+        if async_mode:
+            run_id = str(uuid.uuid4())[:8]
+            self._tasks[run_id] = {
+                "id": run_id, "status": "running",
+                "total": len(ready), "completed": 0,
+                "results": [], "errors": [],
+                "created_at": datetime.now().isoformat(),
+            }
+            asyncio.create_task(self._run_async(run_id, ready, platform,
+                                                depth, tool_level))
+            return {
+                "status": "async",
+                "run_id": run_id,
+                "total": len(ready),
+                "message": f"异步任务已创建（{len(ready)} 个目标）",
+            }
+
+        # 4. 同步模式（目标少，直接执行）
+        task_id = self.db.create_task(
+            "single", platform or "auto",
+            json.dumps([t["input"] for t in ready], ensure_ascii=False),
+            depth, tool_level, machine, len(ready)
+        )
+        results = []
+        errors = []
+        for target in ready:
+            try:
+                result = await self._scrape_one(
+                    target, platform or target["platform"],
+                    depth, tool_level
+                )
+                if result is None:
+                    continue
+                # result 可能是 dict（单条）或 list（用户的所有内容）
+                items = result if isinstance(result, list) else [result]
+                for item in items:
+                    self._save_item(task_id, item)
+                    item.pop("comments", None)
+                    results.append(item)
+            except Exception as e:
+                errors.append({"target": target["input"], "error": str(e)})
+
+        self.db.update_task_status(task_id, "completed" if not errors else "completed",
+                                    summary=json.dumps({"success": len(results),
+                                                         "errors": len(errors)},
+                                                        ensure_ascii=False))
+
+        return {
+            "status": "completed",
+            "task_id": task_id,
+            "duration": round(time.time() - t0, 1),
+            "total": len(ready),
+            "success": len(results),
+            "errors": errors,
+            "data": results,
+        }
+
+    async def _scrape_one(self, target: dict, platform: str,
+                           depth: str, tool_level: int):
+        """
+        抓取单个目标。
+        返回:
+           - dict  → 单条内容（video/note）
+           - list  → 多条内容（user 类型，用户的所有视频/笔记）
+           - None  → 抓取失败
+        """
+        try:
+            adapter = self._get_adapter(platform)
+        except ValueError:
+            logger.warning(f"  ⚠️ 不支持的平台: {platform}")
+            return None
+
+        try:
+            if target["type"] == "user":
+                # 用户抓取：返回该用户的所有内容（列表）
+                return await adapter.collect_user(target["target_id"])
+            elif target["type"] == "video":
+                return await adapter.collect_item(target["target_id"],
+                                                  depth, tool_level)
+            elif target["type"] == "note":
+                return await adapter.collect_item(target["target_id"],
+                                                  depth, tool_level)
+            else:
+                return None
+        except Exception as e:
+            logger.error(f"  ❌ 抓取失败 [{target['input']}]: {e}")
+            return None
+
+    def _save_item(self, task_id: int, item: dict) -> int:
+        """将单条抓取结果写入数据库，返回 db_id"""
+        db_id = self.db.insert_item(
+            task_id=task_id,
+            platform=item.get("platform", ""),
+            item_id=item.get("item_id", ""),
+            url=item.get("url", ""),
+            title=item.get("title", ""),
+            author_name=item.get("author_name", ""),
+            author_id=item.get("author_id", ""),
+            published_at=item.get("published_at", ""),
+            text_content=item.get("text_content", ""),
+            tags=item.get("tags", []),
+            stats=item.get("stats", {}),
+            extra=item.get("extra", {}),
+            media=item.get("media", []),
+        )
+        # 评论独立入库
+        comments = item.get("comments", [])
+        if comments and db_id:
+            self.db.insert_comments(db_id, comments)
+        return db_id
+
+    async def _run_async(self, run_id: str, targets: list, platform: str,
+                         depth: str, tool_level: int):
+        """异步执行大量目标（结果写入 SQLite 持久化，不依赖内存）"""
+        task_data = self._tasks.get(run_id, {})
+        # 异步模式也创建持久化任务记录
+        all_targets_json = json.dumps([t["input"] for t in targets], ensure_ascii=False)
+        task_id = self.db.create_task("batch", platform or "auto",
+                                       all_targets_json, depth, tool_level,
+                                       "async", len(targets))
+        for target in targets:
+            try:
+                result = await self._scrape_one(
+                    target, platform or target["platform"],
+                    depth, tool_level
+                )
+                if result:
+                    items = result if isinstance(result, list) else [result]
+                    for item in items:
+                        self._save_item(task_id, item)
+                        item.pop("comments", None)
+                        task_data["results"].append(item)
+            except Exception as e:
+                err = {"target": target["input"], "error": str(e)}
+                task_data["errors"].append(err)
+            task_data["completed"] += 1
+        task_data["status"] = "completed"
+        # 更新任务状态
+        self.db.update_task_status(task_id, "completed",
+                                    summary=json.dumps({
+                                        "success": len(task_data["results"]),
+                                        "errors": len(task_data["errors"]),
+                                    }, ensure_ascii=False))
+
+    def get_async_result(self, run_id: str) -> Optional[dict]:
+        """查询异步任务结果"""
+        return self._tasks.get(run_id)
+
+    # ── 统计 ──
+
+    def get_stats(self) -> dict:
+        return {
+            "total": sum(self.db.count_by_platform().values()),
+            "today": self.db.count_today(),
+            "sources": self.db.sources_count(),
+            "by_platform": self.db.count_by_platform(),
+            "tasks": self.db.task_stats(),
+        }

mac-agent-os-main/05_tools/10_dashboard/services/scrape_validation.md

@@ -0,0 +1,314 @@
+# 内容抓取系统 — 全链路验证方案 v1
+
+> 日期：2026-07-16 | 状态：验证计划
+> 覆盖：scrape_db.py → scrape_engine.py → routes/scrape.py → 前端 views/scrape.js
+
+## 测试目标
+
+确保内容抓取系统能端到端工作：从前端输入 sec_uid/URL → 后端适配器采集 → SQLite 存储 → 前端展示结果。
+
+---
+
+## Level 0：基础设施验证（3 项）
+
+### L0-1 ✅ Python import 全链路
+```bash
+cd ~/workbuddy-agent-os/agent-sync/05_tools/10_dashboard
+
+~/.workbuddy/binaries/python/envs/agent-os/bin/python3 -c "
+from services.scrape_db import ScrapeDB
+from services.scrape_engine import ScrapeEngine
+from services.adapters import ScrapeAdapter
+from services.adapters.douyin_scrape import DouyinScrapeAdapter
+from services.adapters.browser_helpers import page_evaluate
+from routes.scrape import router
+from plugins.crawl import CrawlDashboardPlugin
+print('✅ All imports OK')
+"
+```
+
+预期：无 ModuleNotFoundError / ImportError
+
+### L0-2 ✅ SQLite 数据库创建
+```bash
+~/.workbuddy/binaries/python/envs/agent-os/bin/python3 -c "
+from services.scrape_db import ScrapeDB
+db = ScrapeDB()
+print(f'db_path: {db.db_path}')
+tables = db.list_tasks()
+print(f'Tables initialized OK, query returned {len(tables)} rows')
+"
+```
+预期：数据库文件 `agent-local/data/scrape.db` 创建成功，4 张表唯美创建
+
+### L0-3 ✅ FastAPI 路由注册
+```bash
+~/.workbuddy/binaries/python/envs/agent-os/bin/python3 -c "
+from routes.scrape import router
+routes = [r.path for r in router.routes]
+for r in routes: print(f'  {r}')
+"
+```
+预期：看到 `/api/scrape/run`、`/api/scrape/resolve`、`/api/scrape/tasks`、`/api/scrape/items`、`/api/scrape/stats`、`/api/scrape/sources` 等端点
+
+---
+
+## Level 1：数据库层（3 项）
+
+### L1-1 创建任务 + 查任务
+```python
+db = ScrapeDB()
+task_id = db.create_task(type='single', platform='douyin', target='test_url',
+                          depth='light', tool_level=2, machine='local', total_targets=1)
+task = db.get_task(task_id)
+assert task['status'] == 'pending'
+assert task['target'] == 'test_url'
+print(f'✅ Task #{task_id} created: {task}')
+```
+
+### L1-2 写入内容 + 去重
+```python
+item_id = db.insert_item(task_id=task_id, platform='douyin', item_id='video_001',
+                          url='https://douyin.com/video/1', title='测试视频',
+                          author_name='测试作者', stats={'likes': 100, 'comments': 5})
+assert item_id > 0
+# 写入相同 item_id → IGNORE（去重）
+item_id2 = db.insert_item(task_id=task_id, platform='douyin', item_id='video_001',
+                           title='should be ignored')
+assert db.item_exists('douyin', 'video_001') == True
+print(f'✅ Item written, dedup works')
+```
+
+### L1-3 评论入库 + 查评论
+```python
+comments = [{'author': '用户A', 'text': '好！', 'likes': 10}]
+db.insert_comments(item_db_id=item_id, comments=comments)
+got = db.get_comments(item_db_id=item_id)
+assert len(got) == 1
+assert got[0]['author_name'] == '用户A'
+print(f'✅ Comments work: {got}')
+```
+
+---
+
+## Level 2：适配器层 — Mock 测试（3 项）
+
+### L2-1 工具降级逻辑
+```python
+# Mock _try_tools: 模拟 OpenCLI 返回数据
+class MockAdapter(ScrapeAdapter):
+    platform = 'test'
+    adapter_name = 'test_scrape'
+    
+    async def collect_user(self, uid, limit=20):
+        ok, data, tool = await self._try_tools(tool_level=2, funcs=[
+            ('opencli', lambda: self._mock_success(uid)),
+            ('browser', lambda: self._mock_success(uid)),
+        ])
+        return data if ok else None
+    
+    async def _mock_success(self, uid):
+        return [{'item_id': 'mock_1', 'title': f'Mock video for {uid}'}]
+adapter = MockAdapter()
+result = await adapter.collect_user('user_001')
+assert len(result) == 1
+assert result[0]['item_id'] == 'mock_1'
+```
+
+### L2-2 工具降级：一级失败，二级成功
+```python
+class FallbackAdapter(ScrapeAdapter):
+    platform = 'test'
+    adapter_name = 'test_fallback'
+    async def collect_user(self, uid, limit=20):
+        ok, data, tool = await self._try_tools(tool_level=2, funcs=[
+            ('opencli', lambda: self._mock_fail('opencli died')),
+            ('browser', lambda: self._mock_success(uid)),
+        ])
+        return (ok, data, tool)
+    async def _mock_fail(self, msg):
+        raise RuntimeError(msg)
+    async def _mock_success(self, uid):
+        return [{'item_id': 'fallback_1'}]
+adapter = FallbackAdapter()
+ok, data, tool = await adapter.collect_user('x')
+assert ok and tool == 'browser'
+```
+
+### L2-3 统一 Schema 输出
+确保各适配器输出的 `collect_item()` 返回的 dict 包含以下字段（无论平台）：
+```python
+required_keys = ['platform', 'item_id', 'url', 'title', 'author_name',
+                  'author_id', 'published_at', 'text_content', 'tags',
+                  'stats', 'extra', 'media', 'comments']
+```
+
+---
+
+## Level 3：适配器层 — 真实 OpenCLI 验证（3 项）
+
+> 需要 OpenCLI 已安装 + 抖音/小红书 Chrome 已登录
+
+### L3-1 抖音用户视频采集
+```python
+from services.adapters.douyin_scrape import DouyinScrapeAdapter
+adapter = DouyinScrapeAdapter()
+items = await adapter.collect_user('r606391422378804368')  # 或任意有公开视频的 sec_uid
+assert items, '没有采集到内容'
+assert len(items) >= 1
+print(f'🎵 抖音采集: {len(items)} 条')
+print(f'  第1条: {items[0].get("title","?")} (👍 {items[0].get("stats",{}).get("likes","?")})')
+```
+
+### L3-2 小红书用户笔记采集
+```python
+from services.adapters.xhs_scrape import XhsScrapeAdapter
+adapter = XhsScrapeAdapter()
+items = await adapter.collect_user('xxx')  # 替换为真实 sec_uid
+assert items, '没有采集到内容'
+print(f'📕 小红书采集: {len(items)} 条')
+```
+
+### L3-3 视频/笔记详情采集
+```python
+item = await adapter.collect_item('https://www.douyin.com/video/xxx')
+assert item.get('title'), '详情采集失败'
+assert item.get('comments'), '评论采集失败'
+print(f'✅ 详情+评论: {len(item.get("comments",[]))} 条评论')
+```
+
+---
+
+## Level 4：引擎层 — ScrapeEngine（3 项）
+
+### L4-1 解析 URL
+```python
+engine = ScrapeEngine()
+results = await engine.resolve_targets([
+    'https://www.douyin.com/video/7660182459061436145',
+    'https://v.douyin.com/wTknxmL_tgc/',
+    'r606391422378804368',  # sec_uid
+])
+for r in results:
+    print(f'  {r.status}: {r.platform}/{r.type} → {r.target_id}')
+```
+
+### L4-2 执行单次采集（完整链路）
+```python
+engine = ScrapeEngine()
+targets = [{'input': 'r606391422378804368', 'platform': 'douyin', 'type': 'user', 'target_id': 'r606391422378804368'}]
+task_id = engine._create_task('batch', targets, depth='light', tool_level=2)
+items = await engine._collect_batch(task_id, targets)
+print(f'  {len(items)} 条采集结果')
+for item in items:
+    print(f'  ✅ {item.get("title","?")} (👍 {item.get("stats",{}).get("likes","?")})')
+```
+
+### L4-3 异步任务轮询
+```python
+# 创建异步任务
+result = await engine.run(targets=['r606391422378804368'], async_mode=True)
+assert result['status'] == 'async'
+run_id = result['run_id']
+# 轮询
+import time
+for i in range(5):
+    time.sleep(2)
+    status = await engine.get_result(run_id)
+    if status['status'] == 'completed':
+        break
+assert status['status'] == 'completed'
+```
+
+---
+
+## Level 5：API 层 — HTTP 端点（4 项）
+
+> 需要 Dashboard 运行中（port 9988）
+
+### L5-1 POST /api/scrape/resolve
+```bash
+curl -s -X POST http://localhost:9988/api/scrape/resolve \
+  -H 'Content-Type: application/json' \
+  -d '{"targets":["r606391422378804368","https://www.douyin.com/video/7660182459061436145"]}' \
+  | python3 -m json.tool
+```
+预期：返回目标列表，status 为 resolved
+
+### L5-2 POST /api/scrape/run（同步）
+```bash
+curl -s -X POST http://localhost:9988/api/scrape/run \
+  -H 'Content-Type: application/json' \
+  -d '{"targets":["r606391422378804368"],"tool_level":1,"async_mode":false}' \
+  | python3 -c "import sys,json; d=json.load(sys.stdin); print(f'status={d.get(\"status\")}, items={len(d.get(\"data\",[]))}')"
+```
+预期：同步返回采集结果
+
+### L5-3 GET /api/scrape/items
+```bash
+curl -s http://localhost:9988/api/scrape/items | python3 -c "import sys,json; d=json.load(sys.stdin); print(f'{len(d.get(\"data\",[]))} items')"
+```
+预期：返回数据库中的采集内容列表
+
+### L5-4 GET /api/scrape/stats
+```bash
+curl -s http://localhost:9988/api/scrape/stats | python3 -m json.tool
+```
+预期：{total: N, today: N, sources: N}
+
+---
+
+## Level 6：前端 — UI 验证（4 项）
+
+> 需要打开浏览器访问 http://localhost:9988
+
+### L6-1 导航栏显示
+- [ ] 侧边栏导航中看到「内容抓取」入口
+- [ ] 点击后跳转到抓取页面
+- [ ] 页面顶部显示三个 Tab：新建采集 / 采集源 / 历史
+
+### L6-2 新建采集流程
+- [ ] 输入 sec_uid，点击执行
+- [ ] 日志区显示进度和结果
+- [ ] 结果区展示表格（# / 标题 / 作者 / 点赞 / 评论 / 发布时间）
+- [ ] 批量导入多个 URL 可以正常解析
+
+### L6-3 采集源管理
+- [ ] 添加采集源成功
+- [ ] 采集源列表展示
+- [ ] 删除采集源成功
+
+### L6-4 历史记录
+- [ ] 历史页面展示任务列表（时间 / 类型 / 目标 / 状态 / 结果数）
+- [ ] 点开后能看到详细结果
+
+---
+
+## Level 7：边界与异常（4 项）
+
+### L7-1 OpenCLI 不可用
+当 OpenCLI 未安装或不可用时，引擎应该：
+- 抛出清晰的错误信息提示安装 OpenCLI
+- 如果 tool_level=2，自动降级到浏览器提取
+- 日志记录具体失败原因
+
+### L7-2 不支持的平台
+```python
+engine = ScrapeEngine()
+result = await engine._collect_one({'input':'test','type':'user','target_id':'test'}, platform='unknown')
+assert result is None
+```
+
+### L7-3 空目标列表
+```python
+result = await engine.run(targets=[])
+assert result['status'] == 'error'
+```
+
+### L7-4 数据库写入失败
+当 SQLite 磁盘满或权限不足时，应返回错误而非崩溃
+```python
+# 用只读模式创建 DB
+import tempfile
+db = ScrapeDB('/dev/null')  # 预期：报错而非崩溃
+```

mac-agent-os-main/05_tools/10_dashboard/services/video_analyzer.py

@@ -0,0 +1,225 @@
+"""
+video_analyzer.py — 视频分析引擎（仅 Master 节点运行）
+
+职责:
+  - 通过浏览器打开视频 URL 提取标题/描述
+  - 关键词匹配法判断行业分类
+  - 从对应语料池选评论
+
+设计: 每个 URL 启动一个临时无头浏览器，分析完成即关闭。
+不经过 Camoufox，不保留身份状态。
+"""
+
+import asyncio
+import logging
+import random
+from typing import Optional
+
+logger = logging.getLogger("dashboard.video_analyzer")
+
+# ── 行业关键词映射（与 mc/corpus.py 的 INDUSTRY_TAGS 保持一致）──
+INDUSTRY_TAGS = {
+    "health": ["医生","医院","药","健康","养生","中医","体检","症状",
+               "治疗","康复","营养","饮食","锻炼","专家","疾病","病症",
+               "胃镜","肠镜","肠胃","手术","护理","方子","药材","食疗",
+               "养生","保健","肠胃镜","医保","体检报告","专家门诊"],
+    "finance": ["股票","基金","理财","投资","经济","A股","财经","行情","股民"],
+    "tech":    ["手机","数码","电脑","科技","AI","人工智能","评测","软件","芯片"],
+    "food":    ["美食","做饭","菜谱","餐厅","好吃","探店","吃播","烹饪","菜肴"],
+}
+
+# ── 万能评论池（所有账号通用）──
+UNIVERSAL_COMMENTS = {
+    "praise": [
+        "博主说得很对",
+        "讲得不错，学到了",
+        "分析得很到位",
+        "说得太对了",
+        "很实用的分享，谢谢博主",
+        "讲得很清楚，一下就懂了",
+        "受益匪浅，谢谢分享",
+        "这个观点很新颖，学习了",
+    ],
+    "question": [
+        "请问这个怎么学？",
+        "大概需要多长时间？",
+        "这个在哪里可以买到？",
+        "有推荐的教程吗？",
+    ],
+    "empathy": [
+        "太真实了",
+        "说出了我的心声",
+        "确实是这样，深有体会",
+    ],
+    "agree": [
+        "同意博主的观点",
+        "说得很好，支持",
+    ],
+}
+
+# ── 行业评论池 ──
+INDUSTRY_COMMENTS = {
+    "health": {
+        "praise": [
+            "医生讲得很清楚，通俗易懂",
+            "这个方子很实用，收藏了",
+            "讲得很专业，学到了",
+            "科普得很好，应该让更多人看到",
+            "简单明了，受益匪浅",
+            "这个知识点太重要了，感谢科普",
+        ],
+        "question": [
+            "这个症状一般怎么处理？",
+            "平时饮食有什么需要注意的？",
+            "这个检查大概需要多久？",
+        ],
+        "empathy": [
+            "健康真的太重要了",
+            "身体是革命的本钱，感谢提醒",
+        ],
+    },
+    "general": {  # 兜底
+        "praise": UNIVERSAL_COMMENTS["praise"],
+        "question": UNIVERSAL_COMMENTS["question"],
+        "empathy": UNIVERSAL_COMMENTS["empathy"],
+    },
+}
+
+
+class VideoContext:
+    """视频分析结果"""
+    def __init__(self, title: str = "", description: str = "",
+                 url: str = "", industry: str = "general",
+                 tags: list = None):
+        self.title = title
+        self.description = description
+        self.url = url
+        self.industry = industry
+        self.tags = tags or []
+
+    def to_dict(self):
+        return {
+            "title": self.title,
+            "description": self.description,
+            "url": self.url,
+            "industry": self.industry,
+            "tags": self.tags,
+        }
+
+
+class VideoAnalyzer:
+    """视频分析器：提取信息 → 行业分类 → 选评论"""
+
+    def __init__(self, max_concurrent: int = 2):
+        self.max_concurrent = max_concurrent
+        self._rate_limit = 0  # 分析间隔，避免被抖音反爬
+
+    async def analyze_batch(self, urls: list[str],
+                            account_industry: Optional[str] = None,
+                            direction: str = "praise") -> dict[str, dict]:
+        """批量分析视频URL（最多 max_concurrent 并发）
+        
+        Args:
+            urls: 视频URL列表
+            account_industry: 执行评论的账号所属行业（health/general）
+            direction: 评论方向（praise/question/empathy/agree）
+        
+        Returns:
+            {url: {title, description, industry, tags, comment}}
+        """
+        sem = asyncio.Semaphore(self.max_concurrent)
+        results = {}
+
+        async def _analyze_one(url: str):
+            async with sem:
+                context = await self._extract_context(url)
+                context.industry = self._classify(context)
+                context.comment = self._pick_comment(
+                    context.industry, account_industry, direction
+                )
+                results[url] = context.to_dict()
+                results[url]["comment"] = context.comment
+                logger.info(f"  ✅ {url[:50]} → [{context.industry}] {context.comment[:30]}...")
+
+        tasks = [asyncio.create_task(_analyze_one(url)) for url in urls]
+        await asyncio.gather(*tasks, return_exceptions=True)
+
+        # 失败兜底
+        for url in urls:
+            if url not in results:
+                results[url] = {
+                    "url": url, "title": "", "description": "",
+                    "industry": "general", "tags": [],
+                    "comment": self._random("praise"),
+                }
+
+        return results
+
+    async def _extract_context(self, url: str) -> VideoContext:
+        """通过无头浏览器提取视频信息"""
+        from playwright.async_api import async_playwright
+
+        async with async_playwright() as p:
+            browser = await p.chromium.launch(headless=True)
+            page = await browser.new_page()
+            try:
+                await page.goto(url, timeout=20000, wait_until="domcontentloaded")
+                await page.wait_for_timeout(3000)
+
+                title = await page.title()
+                title = title.replace(" - 抖音", "").replace(" - 今日头条", "").strip()
+
+                desc = ""
+                meta = await page.query_selector('meta[name="description"]')
+                if meta:
+                    desc = await meta.get_attribute("content") or ""
+
+                # 提取标签/话题
+                tags = []
+                tag_els = await page.query_selector_all('[data-e2e="search-topic"], .topic-link, a[href*="tag"]')
+                for el in tag_els[:5]:
+                    t = await el.inner_text()
+                    if t.strip():
+                        tags.append(t.strip())
+
+                ctx = VideoContext(
+                    title=title,
+                    description=desc,
+                    url=url,
+                    tags=tags,
+                )
+
+                return ctx
+            finally:
+                await browser.close()
+
+    def _classify(self, context: VideoContext) -> str:
+        """关键词匹配法判断行业"""
+        text = f"{context.title} {context.description} {' '.join(context.tags)}"
+        for ind, tags in INDUSTRY_TAGS.items():
+            for tag in tags:
+                if tag in text:
+                    logger.debug(f"  🏷️ 匹配行业 {ind}: 关键词='{tag}'")
+                    return ind
+        return "general"
+
+    def _pick_comment(self, video_industry: str,
+                      account_industry: Optional[str],
+                      direction: str = "praise") -> str:
+        """选评论:
+        - 账号行业匹配视频行业 → 行业池
+        - 不匹配 → 万能池
+        - 无账号行业 → 万能池
+        """
+        # 账号行业匹配视频行业 → 行业池
+        if account_industry and video_industry == account_industry:
+            pool = INDUSTRY_COMMENTS.get(video_industry, {})
+            comments = pool.get(direction, pool.get("praise", UNIVERSAL_COMMENTS["praise"]))
+            return random.choice(comments)
+
+        # 不匹配 → 万能池
+        return self._random(direction)
+
+    def _random(self, direction: str = "praise") -> str:
+        pool = UNIVERSAL_COMMENTS.get(direction, UNIVERSAL_COMMENTS["praise"])
+        return random.choice(pool)