feat(monitor): 抖音 Web 接口客户端 —— 绕开爬虫子进程，直接发 HTTP

移植自 mac-agent-os 的 mediacrawler_adapter：不起子进程、不开页面，用浏览器里那份登录态直接调抖音 Web 接口。产物键名照抄 store/douyin，所以 ingest 那条链路一个字不用改。 **目前能用的（真环境实测，非推断）**： profile/other : 200, 7075 字节 —— 博主主页指标（粉丝/获赞/作品数/昵称） aweme/detail : 200, 45425 字节 —— 单条作品详情（含点赞/评论/收藏/分享） **目前不能用的：作品列表 `aweme/post`。** 两个互相独立的原因： 1. 这个接口被抖音单独升级成了真校验：不带 x-tt-argus 回 403「Uifid Not Found」，带上 dummy 值回 200 + **空 body**。也就是说「头在不在」骗得过，「真校验」过不了。同一套头打 profile/other 和 aweme/detail 都是通的 —— 抖音是挑着接口加保护的，挑中的恰好是「批量拉作品列表」这个最敏感的动作。 2. 改走页面截获也不行：CDP 浏览器打开博主主页会落到「验证码中间页」（当天大量探测的代价，过几小时要重测）。所以现在的边界是：**已知作品的指标刷新能做，自动发现新作品做不了**。 **排查中控住变量后得到的两条事实**（都写进注释了）： · `Accept` / `Accept-Language` / `Referer` 才是主页接口能返回真数据的原因 —— 只有 UA+client hints+Cookie 时是 200 但仅 121 字节的空壳，补上这三个头变 7074 字节。（我先前猜的 sec-ch-ua 不是关键。） · 因此 UA 与 client hints 必须**成套地取自同一个浏览器**，所以 BrowserIdentity 一次从 CDP 取齐 cookie + UA + hints，而不是各自写死。「200 + 空 body 必须当场报错」也是刻意写死的：放过去它会在下游变成「这个博主没作品」，把一次失败伪装成一条正常结果 —— 爬虫那条路正是这么栽的，还被翻译成「账号被封」。测试 +11：cookie 解析、请求头成套性（含 uifid 缺失/回退）、产物键名与 store 对齐、以及 _get 的三条失败路径（空 body / 403 带网关原话 / 正常返回）。
2026-10-10 17:12:25 +08:00
parent 21ff01b894
commit 2112c1a870
1601 changed files with 367425 additions and 0 deletions
@@ -0,0 +1,210 @@
+# AgentOS Agent Initialization Guide
+
+> **Version**: 1.0.0 | **Updated**: 2026-06-17  
+> **Purpose**: Teach AI agents how to understand and operate this system correctly.  
+> **Scope**: 3-machine federated system (chengzigedeAir / 5kechengdeAir / 7kecheng)
+
+---
+
+## 1. Identity Card
+
+| Field | Value |
+|:------|:------|
+| System | AgentOS — Federated AI Exoskeleton |
+| Machines | chengzigedeAir (master), 5kechengdeAir (worker), 7kecheng (worker) |
+| Connection | Tailscale mesh network (100.x.x.x) |
+| Dashboard | Port 9988 on each machine, launchd auto-start |
+| Git Repo | `~/workbuddy-agent-os/agent-sync/` (synced to all 3 machines) |
+
+---
+
+## 2. Python Environment (MOST IMPORTANT)
+
+**Never use system Python.** The correct path is:
+
+```
+$HOME/.workbuddy/binaries/python/envs/agent-os/bin/python3
+```
+
+Or use the dynamic variable (for scripts):
+
+```bash
+$MC_PYTHON
+```
+
+### Key packages installed:
+- camoufox 0.4.11
+- playwright 1.58.0
+- uvicorn 0.46.0
+- orjson 3.11.9
+
+### Common mistakes:
+- ❌ `python3 -m mc run` → ⚠️ No module named mc (needs PYTHONPATH)
+- ✅ `cd $AGENT_SYNC/05_tools/07_matrix/scripts && $MC_PYTHON -m mc run ...`
+- ❌ Running `pip install camoufox` → Already installed in venv
+- ❌ Using `matrix nurture run` → Legacy tool, use `mc run` instead
+
+---
+
+## 3. Command Architecture
+
+```
+User → Dashboard (port 9988)
+  → /api/matrix/nurture/start         ← 养号执行入口
+    → routes/matrix.py                  ← 自动选蓝图 + 路由到机器
+      → 本机: subprocess mc run         ← 模式1
+      → 远程: remote_exec exec_nurture  ← 模式2 (SSH)
+        → mc run                        ← 真正的执行引擎
+          → mc/engine.py                ← 身份分组 + 浏览器启动
+            → cdp_connector.py          ← Camoufox 启动
+              → matrix_modules/nurture/* ← 原子操作
+```
+
+### Key commands for agents:
+
+```bash
+# 正确的养号执行（新路径，推荐）
+cd $AGENT_SYNC/05_tools/07_matrix/scripts
+$MC_PYTHON -m mc run --accounts=douyin_01 --blueprints=douyin_daily --rounds=10
+
+# 查看账号列表
+$MC_PYTHON -m mc account list
+
+# 智能登录
+$MC_PYTHON -m mc smart-login douyin_01
+
+# 远程执行（通过 Dashboard API）
+curl -X POST http://localhost:9988/api/federation/nurture \
+  -H "Content-Type: application/json" \
+  -d '{"machine":"7kecheng","accounts":["douyin_137"],"blueprints":["douyin_daily"],"rounds":10}'
+```
+
+---
+
+## 4. Identity Directory Resolution
+
+**Directory naming**: Identity dirs are named by phone number, NOT account ID:
+```
+identities/
+├── phone_18550099569/        ← 身份目录（手机号命名）
+│   ├── config.yaml
+│   └── user_data/            ← 浏览器持久化数据
+├── phone_15358497926/
+│   ├── config.yaml
+│   └── user_data/
+```
+
+**Account-to-identity mapping**: `accounts.yaml` links account IDs to identity dirs:
+```yaml
+- id: douyin_136               # 账号 ID
+  identity_dir: phone_18550099569   # 身份目录（映射关系）
+  phone: '18550099569'
+  platform: douyin
+  owner_machine: 7kecheng        # 所属机器
+```
+
+**How `mc run` resolves it** (`mc/engine.py:269`):
+```python
+# Step 1: Get identity_dir from account info
+identity_dir = acct.get("identity_dir", acct["id"])  # → "phone_18550099569"
+
+# Step 2: Strip "identities/" prefix if present
+identity_dir = identity_dir.replace("identities/", "")  # → "phone_18550099569"
+
+# Step 3: Resolve full path
+full_path = IDENTITIES_ROOT / identity_dir  # → ~/.../identities/phone_18550099569/
+```
+
+**❗ NEVER look for `identities/{account_id}/`.** The directories are named by phone number.
+
+---
+
+## 5. Machine Routing & Blueprint Auto-Selection
+
+### Machine routing:
+Each account has an `owner_machine` field. The API auto-routes:
+- Local (owner_machine == this machine) → `subprocess.Popen`
+- Remote (owner_machine != this machine) → SSH via `remote_exec.py`
+
+### Blueprint auto-selection:
+| Account Platform | Default Blueprint |
+|:-----------------|:------------------|
+| `douyin` | `douyin_daily` |
+| `xiaohongshu` | `xhs_daily` |
+
+---
+
+## 6. Dashboard API
+
+The Dashboard runs on each machine (port 9988, launchd auto-start on boot).
+
+### Key API endpoints:
+
+| Endpoint | Method | Description |
+|:---------|:-------|:------------|
+| `/api/identity` | GET | Returns this machine's identity |
+| `/api/machines` | GET | Returns all machines' status |
+| `/api/matrix/accounts` | GET | List all accounts (federated) |
+| `/api/matrix/nurture/start` | POST | Execute nurture (see usage below) |
+| `/api/federation/nurture` | POST | Remote nurture execution |
+| `/api/health` | GET | Health check |
+
+### `/api/matrix/nurture/start` request format:
+
+```json
+{
+  "accounts": ["douyin_01", "xhs_01"],
+  "rounds": 10,
+  "blueprint": "douyin_daily",        // optional, auto-detect if empty
+  "machine": "7kecheng",              // optional, auto-route if empty
+  "dry_run": true                     // preview only, don't execute
+}
+```
+
+---
+
+## 7. Common Pitfalls for AI Agents
+
+| Mistake | Why | Fix |
+|:--------|:----|:----|
+| Using `python3` | Wrong Python (no packages) | Use `$MC_PYTHON` or absolute venv path |
+| Running `matrix nurture run` | Legacy CLI, doesn't handle identity mapping | Use `mc run` instead |
+| Looking for `identities/{account_id}/` | Dirs named by phone number | Check `accounts.yaml` → `identity_dir` field |
+| Using `--dry-run` flag | `mc run` doesn't support it | Use API's `dry_run: true` field instead |
+| Installing packages with pip | Already in agent-os venv | Use `$HOME/.workbuddy/binaries/python/envs/agent-os/bin/pip3` if needed |
+| Running `mc run` from wrong directory | Module not found | Must `cd` to `$AGENT_SYNC/05_tools/07_matrix/scripts` first |
+| Using CDP port 9222 | Legacy Chrome, now uses Camoufox | Default: camoufox persistence mode |
+
+---
+
+## 8. Quick Health Check
+
+```bash
+# Check Python venv
+$HOME/.workbuddy/binaries/python/envs/agent-os/bin/python3 -c "from camoufox import AsyncCamoufox; print('OK')"
+
+# Check mc CLI
+PYTHONPATH=$AGENT_SYNC/05_tools/07_matrix/scripts $HOME/.workbuddy/binaries/python/envs/agent-os/bin/python3 -m mc --help
+
+# Check Dashboard
+curl -s --connect-timeout 3 http://localhost:9988/api/identity
+
+# List accounts
+PYTHONPATH=$AGENT_SYNC/05_tools/07_matrix/scripts $HOME/.workbuddy/binaries/python/envs/agent-os/bin/python3 -m mc account list
+
+# Test nurture preview
+curl -s -X POST http://localhost:9988/api/matrix/nurture/start \
+  -H "Content-Type: application/json" \
+  -d '{"accounts":["douyin_test"],"rounds":1,"dry_run":true}'
+```
+
+---
+
+## 9. Environment Variables
+
+| Variable | Typical Value | Purpose |
+|:---------|:--------------|:--------|
+| `$AGENT_SYNC` | `~/workbuddy-agent-os/agent-sync` | Synced git repo root |
+| `$AGENT_LOCAL` | `~/workbuddy-agent-os/agent-local` | Local (gitignored) data |
+| `$MC_PYTHON` | `~/.../agent-os/bin/python3` | Dynamic Python discovery |
+| `$HOME` | `/Users/chengzige` (or `5kecheng`/`7kecheng`) | Machine-specific home |