butubb
2112c1a870
移植自 mac-agent-os 的 mediacrawler_adapter：不起子进程、不开页面，用浏览器里那份
登录态直接调抖音 Web 接口。产物键名照抄 store/douyin，所以 ingest 那条链路一个字不用改。
**目前能用的（真环境实测，非推断）**：
profile/other : 200, 7075 字节 —— 博主主页指标（粉丝/获赞/作品数/昵称）
aweme/detail : 200, 45425 字节 —— 单条作品详情（含点赞/评论/收藏/分享）
**目前不能用的：作品列表 `aweme/post`。** 两个互相独立的原因：
1. 这个接口被抖音单独升级成了真校验：不带 x-tt-argus 回 403「Uifid Not Found」，
带上 dummy 值回 200 + **空 body**。也就是说「头在不在」骗得过，「真校验」过不了。
同一套头打 profile/other 和 aweme/detail 都是通的 —— 抖音是挑着接口加保护的，
挑中的恰好是「批量拉作品列表」这个最敏感的动作。
2. 改走页面截获也不行：CDP 浏览器打开博主主页会落到「验证码中间页」（当天大量探测的
代价，过几小时要重测）。
所以现在的边界是：**已知作品的指标刷新能做，自动发现新作品做不了**。
**排查中控住变量后得到的两条事实**（都写进注释了）：
· `Accept` / `Accept-Language` / `Referer` 才是主页接口能返回真数据的原因 —— 只有
UA+client hints+Cookie 时是 200 但仅 121 字节的空壳，补上这三个头变 7074 字节。
（我先前猜的 sec-ch-ua 不是关键。）
· 因此 UA 与 client hints 必须**成套地取自同一个浏览器**，所以 BrowserIdentity 一次
从 CDP 取齐 cookie + UA + hints，而不是各自写死。
「200 + 空 body 必须当场报错」也是刻意写死的：放过去它会在下游变成「这个博主没作品」，
把一次失败伪装成一条正常结果 —— 爬虫那条路正是这么栽的，还被翻译成「账号被封」。
测试 +11：cookie 解析、请求头成套性（含 uifid 缺失/回退）、产物键名与 store 对齐、
以及 _get 的三条失败路径（空 body / 403 带网关原话 / 正常返回）。
289 lines
12 KiB
Python
289 lines
12 KiB
Python
"""
|
|
plugins/guardd.py — 联邦状态监控插件 (v3.1)
|
|
WPRA v2.0: 优先 machines/*/heartbeat.json, 降级 status/live/
|
|
版本: 3.1.0 | 更新: 2026-05-31
|
|
"""
|
|
import json, os
|
|
from datetime import datetime, timezone
|
|
from pathlib import Path
|
|
|
|
from plugins.base import DashboardPlugin, CROSS_MACHINE, MACHINE_UID, HOSTNAME, AGENT_LOCAL
|
|
from plugins._registry import get_machine_list
|
|
|
|
# ── WPRA v2.0 路径 ────────────────────────────────
|
|
MACHINES_DIR = CROSS_MACHINE / "machines"
|
|
|
|
# ── 人工维护的 IP→hostname 映射（旧版兼容）───────────
|
|
_MACHINE_ALIASES: dict[str, str] = {
|
|
"192.168.31.95": "7kecheng",
|
|
"7kechengdeAir": "7kecheng",
|
|
"7kechengdeMacBook-Air.local": "7kecheng",
|
|
"Redmi-12C": "chengzigedeAir",
|
|
"192.168.31.96": "chengzigedeAir",
|
|
}
|
|
|
|
|
|
class GuarddPlugin(DashboardPlugin):
|
|
name = "guardd"
|
|
label = "联邦机器"
|
|
icon = "🖥"
|
|
version = "3.1.0"
|
|
description = "跨机器联邦状态监控：WPRA v2.0 心跳聚合"
|
|
order = 10
|
|
|
|
# ═══════════════════════════════════════════════════════
|
|
# WPRA v2.0: machines/*/heartbeat.json (主数据源)
|
|
# ═══════════════════════════════════════════════════════
|
|
|
|
def _read_wpra_heartbeats(self):
|
|
"""主数据源: machines/*/heartbeat.json
|
|
|
|
WPRA v2.0: 每台机器只写自己的 machines/{uid}/heartbeat.json
|
|
统一字段名映射到旧格式，保持 _build_machine_detail 兼容。
|
|
"""
|
|
data = {}
|
|
if not MACHINES_DIR.exists():
|
|
return data
|
|
for md in sorted(MACHINES_DIR.iterdir()):
|
|
if not md.is_dir():
|
|
continue
|
|
uid = md.name
|
|
hb_file = md / "heartbeat.json"
|
|
if not hb_file.exists():
|
|
continue
|
|
try:
|
|
hb = json.loads(hb_file.read_text())
|
|
|
|
# 统一字段名映射 (新→旧格式兼容)
|
|
machine_name = hb.get("machine_name", uid[:8])
|
|
updated_at = hb.get("updated_at", "")
|
|
mapped = {
|
|
"_hostname": machine_name,
|
|
"_uid": uid,
|
|
"hostname": machine_name,
|
|
"_received_at": updated_at,
|
|
"status": hb.get("status", "offline"),
|
|
"guardd_version": hb.get("guardd_version", ""),
|
|
"current_task": hb.get("current_task", None),
|
|
# 从 WPRA 扁平字段转为旧版嵌套格式
|
|
"cpu": {"load_1m": hb.get("cpu_load", 0)},
|
|
"disk": {
|
|
"available_gb": hb.get("disk_avail_gb", 0),
|
|
"total_gb": hb.get("disk_total_gb", 0),
|
|
"used_gb": hb.get("disk_used_gb", 0),
|
|
},
|
|
"memory": {"percent": hb.get("memory_pct", 0)},
|
|
"os": "",
|
|
"_live": bool(updated_at),
|
|
"_last_push_sec": 0,
|
|
"_source": "wpra_heartbeat",
|
|
}
|
|
|
|
# 计算 _last_push_sec
|
|
if updated_at:
|
|
try:
|
|
delta = (datetime.now(timezone.utc) -
|
|
datetime.fromisoformat(updated_at.replace("Z", "+00:00"))).total_seconds()
|
|
mapped["_last_push_sec"] = round(delta)
|
|
mapped["_live"] = delta < 120
|
|
except:
|
|
pass
|
|
|
|
data[uid] = mapped
|
|
except Exception:
|
|
continue
|
|
return data
|
|
|
|
# ═══════════════════════════════════════════════════════
|
|
# 旧版: status/live/{uid}.json (降级)
|
|
# ═══════════════════════════════════════════════════════
|
|
|
|
def _read_live_data(self):
|
|
"""实时数据优先: HTTP Push 写入的 status/live/, 补充 WPRA machines/*/
|
|
|
|
双轨制 (v4.2.0):
|
|
- 远程机器通过 HTTP Push 实时推送心跳 → status/live/{uid}.json（主数据源）
|
|
- 本地 guardd 写入 → machines/{uid}/heartbeat.json（降级补充）
|
|
优先读取实时推送数据，WPRA 文件仅作补充（适用于未推送的机器）。
|
|
"""
|
|
# 主数据源: HTTP Push 实时推送 → status/live/{uid}.json
|
|
live_dir = CROSS_MACHINE / "status" / "live"
|
|
data = {}
|
|
if live_dir.exists():
|
|
for f in live_dir.iterdir():
|
|
if f.suffix != ".json" or f.name.startswith("_"):
|
|
continue
|
|
try:
|
|
uid = f.name.replace(".json", "")
|
|
data[uid] = json.loads(f.read_text())
|
|
except:
|
|
pass
|
|
|
|
# 补充: WPRA machines/*/heartbeat.json（如 live 中没有该机器的数据）
|
|
wpra = self._read_wpra_heartbeats()
|
|
for uid, hb in wpra.items():
|
|
if uid not in data:
|
|
data[uid] = hb
|
|
|
|
return data
|
|
|
|
def _read_registry(self):
|
|
"""补充数据源: cross_machine/registry/{name}.json"""
|
|
reg_dir = CROSS_MACHINE / "registry"
|
|
machines = []
|
|
if reg_dir.exists():
|
|
for f in sorted(reg_dir.iterdir()):
|
|
if f.suffix == ".json":
|
|
try:
|
|
d = json.loads(f.read_text())
|
|
machines.append({
|
|
"hostname": d.get("hostname", f.stem),
|
|
"uid": d.get("uid", ""),
|
|
"role": d.get("role", ""),
|
|
"source": "registry",
|
|
})
|
|
except:
|
|
pass
|
|
return machines
|
|
|
|
def _read_plugin_data(self):
|
|
"""补充数据源: 本机 guardd 状态 (local) + 旧 cross_machine/data/ (兼容)"""
|
|
# 新: 本机 local 目录
|
|
local_guardd = AGENT_LOCAL / "runtime" / "guardd" / "data"
|
|
machines = {}
|
|
if local_guardd.exists():
|
|
for plugin_dir in local_guardd.iterdir():
|
|
if not plugin_dir.is_dir():
|
|
continue
|
|
for f in plugin_dir.iterdir():
|
|
if f.suffix == ".json":
|
|
try:
|
|
d = json.loads(f.read_text())
|
|
uid = d.get("machine_uid", "")
|
|
hn = d.get("hostname", "")
|
|
if uid and hn:
|
|
if uid not in machines:
|
|
machines[uid] = {"hostname": hn, "uid": uid, "source": "local_guardd"}
|
|
except: pass
|
|
# 旧: cross_machine/data/ (兼容旧数据)
|
|
data_dir = CROSS_MACHINE / "data"
|
|
if data_dir.exists():
|
|
for plugin_dir in data_dir.iterdir():
|
|
if not plugin_dir.is_dir():
|
|
continue
|
|
for f in plugin_dir.iterdir():
|
|
if f.suffix == ".json":
|
|
try:
|
|
d = json.loads(f.read_text())
|
|
uid = d.get("machine_uid", "")
|
|
hn = d.get("hostname", "")
|
|
if uid and hn and uid not in machines:
|
|
machines[uid] = {"hostname": hn, "uid": uid, "source": "legacy_cross_machine"}
|
|
except: pass
|
|
return list(machines.values())
|
|
|
|
# ═══════════════════════════════════════════════════════
|
|
# 概览 / 详情
|
|
# ═══════════════════════════════════════════════════════
|
|
|
|
def summary(self, machines: list[str]) -> dict:
|
|
live = self._read_live_data()
|
|
now = datetime.now(timezone.utc)
|
|
|
|
by_machine = {}
|
|
for uid, l in live.items():
|
|
hn = l.get("_hostname", uid[:8])
|
|
received = l.get("_received_at", "")
|
|
online = False
|
|
realtime = False
|
|
if received:
|
|
try:
|
|
delta = (now - datetime.fromisoformat(
|
|
received.replace("Z", "+00:00"))).total_seconds()
|
|
online = delta < 300
|
|
realtime = delta < 120
|
|
except:
|
|
pass
|
|
by_machine[hn] = {
|
|
"在线": online,
|
|
"实时推送": realtime,
|
|
"guardd版本": l.get("guardd_version", ""),
|
|
}
|
|
|
|
return {
|
|
"总机器": len(by_machine),
|
|
"在线": sum(1 for m in by_machine.values() if m["在线"]),
|
|
"实时": sum(1 for m in by_machine.values() if m["实时推送"]),
|
|
"各机器": by_machine,
|
|
}
|
|
|
|
def detail(self, machine: str = "") -> dict:
|
|
live = self._read_live_data()
|
|
now = datetime.now(timezone.utc)
|
|
|
|
if machine:
|
|
for uid, l in live.items():
|
|
if l.get("_hostname") == machine:
|
|
return self._build_machine_detail(machine, uid, l, now)
|
|
return {"_note": "未找到该机器数据"}
|
|
|
|
result = {}
|
|
seen = set()
|
|
for uid, l in sorted(live.items(), key=lambda x: x[1].get("_hostname", "")):
|
|
hn = l.get("_hostname", uid[:8])
|
|
if hn in seen:
|
|
continue
|
|
seen.add(hn)
|
|
result[hn] = self._build_machine_detail(hn, uid, l, now)
|
|
return result
|
|
|
|
def _build_machine_detail(self, hostname, uid, live, now):
|
|
received = live.get("_received_at", "")
|
|
entry = {
|
|
"hostname": hostname,
|
|
"_uid": uid[:8]+"..." if uid else "",
|
|
"status": "offline",
|
|
"last_seen": received,
|
|
"os": live.get("os", ""),
|
|
"cpu_load": 0,
|
|
"disk_used_gb": 0,
|
|
"disk_total_gb": 0,
|
|
"disk_avail_gb": 0,
|
|
"guardd_version": live.get("guardd_version", ""),
|
|
"current_task": live.get("current_task", ""),
|
|
"minutes_ago": 999,
|
|
"_live": False,
|
|
"_last_push_sec": 0,
|
|
}
|
|
if received:
|
|
try:
|
|
delta = (now - datetime.fromisoformat(
|
|
received.replace("Z", "+00:00"))).total_seconds()
|
|
entry["_live"] = delta < 120
|
|
entry["_last_push_sec"] = round(delta)
|
|
entry["minutes_ago"] = round(delta / 60)
|
|
entry["status"] = "online" if delta < 300 else ("recent" if delta < 3600 else "offline")
|
|
except:
|
|
pass
|
|
cpu = live.get("cpu", {})
|
|
entry["cpu_load"] = cpu.get("load_1m", 0)
|
|
disk = live.get("disk", {})
|
|
entry["disk_used_gb"] = disk.get("used_gb", 0)
|
|
entry["disk_total_gb"] = disk.get("total_gb", 0)
|
|
entry["disk_avail_gb"] = disk.get("available_gb", 0)
|
|
return entry
|
|
|
|
def actions(self) -> list[dict]:
|
|
machines = get_machine_list()
|
|
return [
|
|
{"name": f"唤醒 {hn}", "method": "POST", "endpoint": f"/api/wakeup/{hn}"}
|
|
for hn in machines
|
|
]
|
|
|
|
def get_productions(self, limit=50, offset=0, strategy=None, status=None):
|
|
result = self.detail()
|
|
return list(result.values())
|
|
|
|
def get_summary(self) -> dict:
|
|
return self.summary(get_machine_list())
|