feat(monitor): 抖音 Web 接口客户端 —— 绕开爬虫子进程，直接发 HTTP

移植自 mac-agent-os 的 mediacrawler_adapter：不起子进程、不开页面，用浏览器里那份
登录态直接调抖音 Web 接口。产物键名照抄 store/douyin，所以 ingest 那条链路一个字不用改。

**目前能用的（真环境实测，非推断）**：

    profile/other : 200, 7075 字节  —— 博主主页指标（粉丝/获赞/作品数/昵称）
    aweme/detail  : 200, 45425 字节 —— 单条作品详情（含点赞/评论/收藏/分享）

**目前不能用的：作品列表 `aweme/post`。** 两个互相独立的原因：
  1. 这个接口被抖音单独升级成了真校验：不带 x-tt-argus 回 403「Uifid Not Found」，
     带上 dummy 值回 200 + **空 body**。也就是说「头在不在」骗得过，「真校验」过不了。
     同一套头打 profile/other 和 aweme/detail 都是通的 —— 抖音是挑着接口加保护的，
     挑中的恰好是「批量拉作品列表」这个最敏感的动作。
  2. 改走页面截获也不行：CDP 浏览器打开博主主页会落到「验证码中间页」（当天大量探测的
     代价，过几小时要重测）。

  所以现在的边界是：**已知作品的指标刷新能做，自动发现新作品做不了**。

**排查中控住变量后得到的两条事实**（都写进注释了）：
  · `Accept` / `Accept-Language` / `Referer` 才是主页接口能返回真数据的原因 —— 只有
    UA+client hints+Cookie 时是 200 但仅 121 字节的空壳，补上这三个头变 7074 字节。
    （我先前猜的 sec-ch-ua 不是关键。）
  · 因此 UA 与 client hints 必须**成套地取自同一个浏览器**，所以 BrowserIdentity 一次
    从 CDP 取齐 cookie + UA + hints，而不是各自写死。

「200 + 空 body 必须当场报错」也是刻意写死的：放过去它会在下游变成「这个博主没作品」，
把一次失败伪装成一条正常结果 —— 爬虫那条路正是这么栽的，还被翻译成「账号被封」。

测试 +11：cookie 解析、请求头成套性（含 uifid 缺失/回退）、产物键名与 store 对齐、
以及 _get 的三条失败路径（空 body / 403 带网关原话 / 正常返回）。

mac-agent-os-main/02_skills/kb_manager/SKILL.md

@@ -0,0 +1,119 @@
+---
+name: kb_manager
+version: 1.1.0
+description: 知识库管理技能——新增知识入库、清洗、标签分类、属性变更移动、搜索、备份
+triggers:
+  - 知识库管理
+  - 入库
+  - 保存知识
+  - 知识入库
+  - 知识分类
+  - 查知识库
+  - 知识检索
+  - 知识备份
+  - 知识属性变更
+  - 知识重分类
+  - ingest
+  - kb
+---
+
+# KB Manager 技能
+
+## 概述
+
+管理 AgentOS 的 Obsidian 知识库（03_knowledge/），负责知识入库、清洗、分类、属性变更移动、搜索和备份。
+
+## 核心能力
+
+### 1. 知识入库（kb_ingest）
+- 输入：网页 URL 或纯文本
+- 流程：
+  1. 抓取/接收原始内容
+  2. 清洗（去广告、去导航、提取正文）
+  3. 写入 `03_knowledge/00_inbox/` 作为待提纯内容
+  4. 由 `collect_to_inbox` 每日 2:30 提取各目录内容转入收件箱，再由 `inbox_refine` 技能每日凌晨 3:00 自动提纯归档
+  5. 或手动说"提纯"立即执行
+
+### 2. 知识重分类（reclassify）
+- 当知识的 nature 属性变更时（如 opinion → fact）：
+  1. 更新 Frontmatter 中 nature 和 status
+  2. 将文件从旧目录移动到新目录
+  3. 自动更新 Obsidian 内链
+  4. 记录到 04_memory/logs/kb_reclassify.log
+
+### 3. 知识检索（kb_query）
+- 检索流程：
+  1. 先在 L2 摘要层检索（SQLite 索引）
+  2. 命中后按需加载知识文件指定行号片段
+  3. 不加载整个文件，节省 token
+
+### 4. 知识备份（kb_backup）
+- 全量备份 03_knowledge/ 目录到指定位置
+- 支持增量备份（仅备份变更文件）
+
+## 入库规范
+
+### Frontmatter 必填字段
+
+```yaml
+---
+id: KB-YYYYMMDD-NNN
+title: "知识标题"
+type: concept|method|fact|reference|resource|opinion
+status: draft|review|published|archived
+nature: fact|opinion|method|regulation|reference|data|quote|axiom
+domain: [领域1, 领域2]
+confidence: 0.0-1.0
+source: "原始来源 URL"
+date_created: YYYY-MM-DD
+date_modified: YYYY-MM-DD
+version: 1
+---
+```
+
+### 可选字段
+
+```yaml
+subdomain: [子领域]
+tags: [标签列表]
+author: 作者
+source_type: personal_exp|literature|webpage|conversation|code|observation
+related: [KB-ID 列表]
+previous_version: KB-ID
+superseded_by: KB-ID
+aliases: [别称]
+keywords: [关键词]
+summary: 一句话摘要
+```
+
+## 预定义领域列表
+
+计算机科学, 人工智能, 金融, 法律, 医学, 物理, 数学, 心理学, 哲学, 历史, 工程, 设计, 商业, 个人管理, 个人洞见, 其他
+
+> 当"其他"类别超过 5% 时，memory_manager 技能会提示新增领域
+
+## 文件结构
+
+```
+kb_manager/
+├── SKILL.md
+├── kb_ingest.py           # 知识入库脚本
+├── kb_query.py            # 知识检索脚本
+├── kb_backup.py           # 知识备份脚本
+├── kb_reclassify.py       # 知识重分类脚本
+└── version.json
+```
+
+## 依赖
+
+| 包 | 用途 | 安装状态 |
+|---|---|---|
+| trafilatura | 网页内容提取 | ✅ 已安装 |
+| chromadb | 向量存储（备选语义检索） | ✅ 已安装 |
+| sqlite-utils | 数据库操作 | ✅ 已安装 |
+
+## 自动分类标准
+
+- nature 字段：fact / opinion / method / regulation / reference / data / quote / axiom
+- domain 字段：从预定义领域列表选择 1-3 个
+- confidence 评分：官方文档 0.8+、个人博客 0.4-0.6、推测 < 0.4

mac-agent-os-main/02_skills/kb_manager/SKILL_CARD.yaml

@@ -0,0 +1,56 @@
+skill:
+  name: "知识库管理器"
+  id: "kb_manager"
+  version: "1.1.0"
+  category: "core-system"
+  status: "active"
+  created: "2026-04-25"
+  updated: "2026-04-29"
+
+description:
+  short: "知识入库、分类归档、frontmatter生成、双轨向量检索"
+  use_cases:
+    - "将inbox各分区内容分类归档到知识库"
+    - "从URL/文件/文本生成标准知识卡片"
+    - "双轨向量检索（本地+协同）"
+  tags: ["知识库", "入库", "分类", "检索"]
+
+ownership:
+  master_editor: "my-macbook_macOS14.4_16GB_master"
+  readonly_on_other_devices: true
+
+environment:
+  os: "macos"
+  python_path: "~/.workbuddy/binaries/python/envs/agent-os/bin/python3"
+  pip_dependencies:
+    - "trafilatura>=1.6.0"
+    - "chromadb>=1.0.0"
+
+setup:
+  install_script: ""
+  post_check: "python3 -c 'import trafilatura; print(\"OK\")'"
+
+usage:
+  entry_point: "kb_ingest.py"
+  scheduled: ""
+  manual_trigger: "知识入库 --url <url>"
+
+scripts:
+  - kb_ingest.py     # 知识入库主程序
+  - kb_search.py     # 双轨向量检索
+
+inbox_paths:
+  knowledge_clipping: "03_knowledge/00_stream/inbox/knowledge/clipping/"
+  knowledge_feed:     "03_knowledge/00_stream/inbox/knowledge/feed/"
+  memory:             "03_knowledge/00_stream/inbox/memory/"
+  tools:              "03_knowledge/00_stream/inbox/tools/"
+  personal:           "03_knowledge/00_stream/inbox/personal/"
+  media:              "03_knowledge/00_stream/inbox/media/"
+
+changelog:
+  - version: "1.1.0"
+    date: "2026-04-29"
+    changes: "适配新inbox五分区+clipping/feed子分区，新增kb_search.py双轨检索"
+  - version: "1.0.0"
+    date: "2026-04-25"
+    changes: "初始版本"

mac-agent-os-main/02_skills/kb_manager/kb_ingest.py

@@ -0,0 +1,287 @@
+#!/usr/bin/env python3
+"""
+AgentOS 知识入库脚本
+用途：将网页内容或纯文本清洗、分类、生成知识卡片并写入 Obsidian 知识库
+使用：python3 kb_ingest.py --root ~/workbuddy-agent-os/agent-sync --url https://example.com
+     python3 kb_ingest.py --root ~/workbuddy-agent-os/agent-sync --file input.md
+     python3 kb_ingest.py --root ~/workbuddy-agent-os/agent-sync --text "原始文本内容"
+"""
+
+import argparse
+import json
+import os
+import re
+from datetime import datetime
+from pathlib import Path
+
+
+# 预定义领域列表
+DOMAINS = [
+    "计算机科学", "人工智能", "金融", "法律", "医学", "物理", "数学",
+    "心理学", "哲学", "历史", "工程", "设计", "商业", "个人管理",
+    "个人洞见", "其他"
+]
+
+# 知识属性类型
+NATURE_TYPES = ["fact", "opinion", "method", "regulation", "reference", "data", "quote", "axiom"]
+
+# nature → 目录映射
+NATURE_DIR_MAP = {
+    "concept": "10_concepts",
+    "method": "20_methods",
+    "fact": "30_facts",
+    "reference": "40_references/docs",
+    "resource": "50_resources",
+    "opinion": "60_opinions",
+    "data": "30_facts",
+    "regulation": "30_facts",
+    "quote": "60_opinions",
+    "axiom": "10_concepts",
+}
+
+# 领域英文 → 中文映射
+DOMAIN_DIR_MAP = {
+    "计算机科学": "cs",
+    "人工智能": "ai",
+    "金融": "finance",
+    "法律": "law",
+    "医学": "medicine",
+    "物理": "physics",
+    "数学": "math",
+    "心理学": "psychology",
+    "哲学": "philosophy",
+    "历史": "history",
+    "工程": "engineering",
+    "设计": "design",
+    "商业": "business",
+    "个人管理": "personal-management",
+    "个人洞见": "personal-insight",
+    "其他": "other",
+}
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description="AgentOS 知识入库")
+    parser.add_argument("--root", required=True, help="agent-os 根目录路径")
+    parser.add_argument("--url", default=None, help="网页 URL")
+    parser.add_argument("--file", default=None, help="输入文件路径")
+    parser.add_argument("--text", default=None, help="直接输入文本")
+    parser.add_argument("--title", default=None, help="知识标题")
+    parser.add_argument("--domain", default=None, help="领域（逗号分隔）")
+    parser.add_argument("--nature", default=None, help="知识属性")
+    parser.add_argument("--tags", default=None, help="标签（逗号分隔）")
+    parser.add_argument("--inbox", action="store_true", help="仅放入收件箱，不分类")
+    return parser.parse_args()
+
+
+def fetch_url_content(url: str) -> str:
+    """抓取网页内容"""
+    try:
+        import trafilatura
+        downloaded = trafilatura.fetch_url(url)
+        if downloaded:
+            content = trafilatura.extract(downloaded)
+            return content or ""
+    except ImportError:
+        pass
+
+    # fallback: 用 urllib
+    import urllib.request
+    req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
+    with urllib.request.urlopen(req, timeout=30) as resp:
+        html = resp.read().decode("utf-8", errors="ignore")
+
+    # 简单提取正文（去除 HTML 标签）
+    text = re.sub(r'<[^>]+>', ' ', html)
+    text = re.sub(r'\s+', ' ', text).strip()
+    return text[:10000]  # 限制长度
+
+
+def classify_content(content: str, title: str = "") -> dict:
+    """
+    分类知识内容（简化版，实际应调用 LLM）
+    返回: {domain, nature, confidence, tags, summary}
+    """
+    # 简单规则分类
+    domain = "其他"
+    nature = "fact"
+    confidence = 0.5
+    tags = []
+    summary = content[:60]
+
+    # 关键词匹配领域
+    domain_keywords = {
+        "计算机科学": ["代码", "编程", "程序", "数据库", "服务器", "API", "Python", "JavaScript", "算法"],
+        "人工智能": ["AI", "模型", "训练", "深度学习", "LLM", "GPT", "embedding", "向量"],
+        "金融": ["股票", "基金", "投资", "利率", "市场", "财报", "估值"],
+        "个人洞见": ["我觉得", "我认为", "感悟", "心得", "人生", "经历"],
+    }
+
+    for d, keywords in domain_keywords.items():
+        for kw in keywords:
+            if kw.lower() in content.lower():
+                domain = d
+                break
+        if domain != "其他":
+            break
+
+    # 关键词匹配 nature
+    if any(kw in content for kw in ["应该", "步骤", "方法", "如何"]):
+        nature = "method"
+    elif any(kw in content for kw in ["我认为", "我觉得", "可能", "也许"]):
+        nature = "opinion"
+    elif any(kw in content for kw in ["规定", "法规", "制度", "条例"]):
+        nature = "regulation"
+
+    return {
+        "domain": domain,
+        "nature": nature,
+        "confidence": confidence,
+        "tags": tags,
+        "summary": summary
+    }
+
+
+def generate_kb_id(date: str, index: int = 1) -> str:
+    """生成知识 ID"""
+    return f"KB-{date.replace('-', '')}-{index:03d}"
+
+
+def generate_frontmatter(kb_id: str, title: str, classification: dict, source: str = "") -> str:
+    """生成 Frontmatter"""
+    domain = classification.get("domain", "其他")
+    if isinstance(domain, str):
+        domain = [domain]
+
+    fm = f"""---
+id: {kb_id}
+title: "{title}"
+type: {classification.get('nature', 'concept')}
+status: draft
+nature: {classification.get('nature', 'fact')}
+domain: {json.dumps(domain, ensure_ascii=False)}
+confidence: {classification.get('confidence', 0.5)}
+source: "{source}"
+date_created: {datetime.now().strftime('%Y-%m-%d')}
+date_modified: {datetime.now().strftime('%Y-%m-%d')}
+version: 1
+summary: "{classification.get('summary', '')[:60]}"
+---"""
+    return fm
+
+
+def determine_target_dir(root: str, nature: str, domain: str) -> Path:
+    """确定知识卡片的目标目录"""
+    base_dir = NATURE_DIR_MAP.get(nature, "10_concepts")
+    domain_dir = DOMAIN_DIR_MAP.get(domain, "other")
+
+    # 概念类知识按领域分子目录
+    if base_dir == "10_concepts":
+        return Path(root) / "03_knowledge" / base_dir / domain_dir
+    else:
+        return Path(root) / "03_knowledge" / base_dir
+
+
+def write_knowledge_card(root: str, target_dir: Path, filename: str,
+                         frontmatter: str, content: str):
+    """写入知识卡片文件"""
+    target_dir.mkdir(parents=True, exist_ok=True)
+    file_path = target_dir / filename
+
+    full_content = frontmatter + "\n\n" + content
+    file_path.write_text(full_content, encoding="utf-8")
+    return file_path
+
+
+def log_ingest(root: str, kb_id: str, decision: str, reason: str, confidence: float):
+    """记录入库日志"""
+    log_dir = Path(root) / "04_memory" / "logs"
+    log_dir.mkdir(parents=True, exist_ok=True)
+    log_file = log_dir / "kb_ingest.log"
+
+    timestamp = datetime.now().isoformat()
+    with open(log_file, "a", encoding="utf-8") as f:
+        f.write(f"{timestamp} | {kb_id} | {decision} | {reason} | {confidence}\n")
+
+
+def main():
+    args = parse_args()
+    root = os.path.expanduser(args.root)
+
+    # 获取内容
+    content = ""
+    source = ""
+
+    if args.url:
+        print(f"[INFO] 抓取网页: {args.url}")
+        content = fetch_url_content(args.url)
+        source = args.url
+    elif args.file:
+        file_path = Path(args.file)
+        if not file_path.exists():
+            print(f"[ERROR] 文件不存在: {args.file}")
+            return
+        content = file_path.read_text(encoding="utf-8")
+        source = str(file_path)
+    elif args.text:
+        content = args.text
+        source = "direct_input"
+    else:
+        print("[ERROR] 请指定 --url, --file 或 --text")
+        return
+
+    if not content.strip():
+        print("[ERROR] 内容为空")
+        return
+
+    title = args.title or "未命名知识"
+
+    # 分类
+    if args.inbox:
+        # 仅放入收件箱
+        target_dir = Path(root) / "03_knowledge" / "00_inbox"
+        classification = {
+            "domain": args.domain or "其他",
+            "nature": args.nature or "fact",
+            "confidence": 0.5,
+            "tags": [],
+            "summary": content[:60]
+        }
+    else:
+        # 自动分类
+        classification = classify_content(content, title)
+        # 命令行参数覆盖
+        if args.domain:
+            classification["domain"] = args.domain
+        if args.nature:
+            classification["nature"] = args.nature
+
+    # 生成知识卡片
+    today = datetime.now().strftime("%Y-%m-%d")
+    kb_id = generate_kb_id(today)
+    frontmatter = generate_frontmatter(kb_id, title, classification, source)
+
+    if args.inbox:
+        target_dir = Path(root) / "03_knowledge" / "00_inbox"
+    else:
+        target_dir = determine_target_dir(root, classification["nature"], classification["domain"])
+
+    # 生成文件名（英文，用时间戳确保唯一）
+    safe_title = re.sub(r'[^\w\u4e00-\u9fff-]', '_', title)[:50]
+    filename = f"{datetime.now().strftime('%Y%m%d_%H%M%S')}_{safe_title}.md"
+
+    # 写入
+    file_path = write_knowledge_card(root, target_dir, filename, frontmatter, content)
+    print(f"[OK] 知识卡片已创建: {file_path}")
+
+    # 记录日志
+    log_ingest(root, kb_id, "created", "自动分类入库", classification["confidence"])
+
+    print(f"[INFO] KB-ID: {kb_id}")
+    print(f"[INFO] 领域: {classification['domain']}")
+    print(f"[INFO] 属性: {classification['nature']}")
+    print(f"[INFO] 置信度: {classification['confidence']}")
+
+
+if __name__ == "__main__":
+    main()

mac-agent-os-main/02_skills/kb_manager/kb_search.py

@@ -0,0 +1,293 @@
+#!/usr/bin/env python3
+"""
+kb_search.py — 双轨知识库检索
+
+先查本地向量库（agent-local/vector_db/local/），
+再查协同向量库（agent-local/vector_db/global/），
+结果合并展示，优先显示本地记忆。
+
+向量库由 vector_db_rebuild.py 构建，基于 ChromaDB。
+
+使用：
+  python3 kb_search.py --query "记忆固化机制"
+  python3 kb_search.py --query "Python装饰器" --top 10
+  python3 kb_search.py --query "知识库" --track global   # 只查协同库
+  python3 kb_search.py --query "今天的会议" --track local  # 只查本地库
+  python3 kb_search.py --query "向量" --keyword           # 纯关键词搜索（无需向量库）
+"""
+
+import argparse
+import json
+import os
+import re
+import sys
+from datetime import datetime
+from pathlib import Path
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description="AgentOS 双轨知识库检索")
+    parser.add_argument("--query", required=True, help="检索查询语句")
+    parser.add_argument("--local", default="~/workbuddy-agent-os/agent-local", help="~/workbuddy-agent-os/agent-local")
+    parser.add_argument("--root", default="~/workbuddy-agent-os/agent-sync", help="agent-os 根目录")
+    parser.add_argument("--top", type=int, default=5, help="返回结果数（默认5）")
+    parser.add_argument("--track", choices=["local", "global", "both"], default="both",
+                        help="检索范围: local(本地) | global(协同) | both(默认)")
+    parser.add_argument("--keyword", action="store_true",
+                        help="纯关键词搜索（不使用向量库，适合没有重建向量库的环境）")
+    parser.add_argument("--domain", default=None, help="过滤领域（如 ai / finance）")
+    return parser.parse_args()
+
+
+# ─── 向量库检索 ───────────────────────────────────────────────────
+
+def get_embedding_model():
+    """获取嵌入模型（优先 SentenceTransformers，fallback 到关键词）"""
+    try:
+        from sentence_transformers import SentenceTransformer
+        model = SentenceTransformer("all-MiniLM-L6-v2")
+        return model
+    except ImportError:
+        pass
+    try:
+        from chromadb.utils import embedding_functions
+        return embedding_functions.DefaultEmbeddingFunction()
+    except Exception:
+        pass
+    return None
+
+
+def search_chroma(db_path: Path, query: str, top_n: int, domain_filter: str = None) -> list:
+    """在指定 ChromaDB 中搜索"""
+    try:
+        import chromadb
+    except ImportError:
+        print("[WARN] chromadb 未安装，跳过向量检索")
+        return []
+
+    if not db_path.exists():
+        return []
+
+    try:
+        client = chromadb.PersistentClient(path=str(db_path))
+        collections = client.list_collections()
+        if not collections:
+            return []
+
+        results = []
+        for col in collections:
+            collection = client.get_collection(col.name)
+            where = {"domain": domain_filter} if domain_filter else None
+            try:
+                res = collection.query(
+                    query_texts=[query],
+                    n_results=min(top_n, collection.count()),
+                    where=where,
+                    include=["documents", "metadatas", "distances"]
+                )
+            except Exception:
+                res = collection.query(
+                    query_texts=[query],
+                    n_results=min(top_n, collection.count()),
+                    include=["documents", "metadatas", "distances"]
+                )
+
+            docs = res.get("documents", [[]])[0]
+            metas = res.get("metadatas", [[]])[0]
+            dists = res.get("distances", [[]])[0]
+
+            for doc, meta, dist in zip(docs, metas, dists):
+                results.append({
+                    "content": doc[:300],
+                    "meta": meta,
+                    "score": round(1 - dist, 4),  # 转换为相似度分数
+                    "track": "local" if "local" in str(db_path) else "global"
+                })
+
+        return sorted(results, key=lambda x: x["score"], reverse=True)[:top_n]
+    except Exception as e:
+        print(f"[WARN] 向量检索出错: {e}")
+        return []
+
+
+# ─── 关键词检索（fallback） ───────────────────────────────────────
+
+def search_keyword_index(index_path: Path, query: str, top_n: int) -> list:
+    """从 keyword_index.json 关键词索引中搜索"""
+    if not index_path.exists():
+        return []
+    try:
+        with open(index_path, encoding="utf-8") as f:
+            data = json.load(f)
+        entries = data.get("entries", [])
+    except Exception:
+        return []
+
+    query_terms = set(re.findall(r'[\u4e00-\u9fff]{2,}|[a-zA-Z]{3,}', query.lower()))
+    scored = []
+    for entry in entries:
+        kws = {k.lower() for k in entry.get("keywords", [])}
+        summary = entry.get("summary", "").lower()
+        # 计算匹配分数
+        kw_hits = len(query_terms & kws)
+        content_hits = sum(1 for t in query_terms if t in summary)
+        score = kw_hits * 2 + content_hits
+        if score > 0:
+            scored.append({
+                "content": entry.get("summary", "")[:300],
+                "meta": {
+                    "id": entry.get("fact_id", ""),
+                    "domain": entry.get("domain", ""),
+                    "nature": entry.get("nature", ""),
+                    "date": entry.get("date", "")
+                },
+                "score": score,
+                "track": "local_index"
+            })
+
+    return sorted(scored, key=lambda x: x["score"], reverse=True)[:top_n]
+
+
+def search_md_files(search_dir: Path, query: str, top_n: int, domain_filter: str = None) -> list:
+    """直接在 MD 文件中做关键词全文搜索"""
+    if not search_dir.exists():
+        return []
+
+    query_terms = re.findall(r'[\u4e00-\u9fff]{2,}|[a-zA-Z]{3,}', query.lower())
+    if not query_terms:
+        return []
+
+    scored = []
+    for md_file in search_dir.rglob("*.md"):
+        try:
+            content = md_file.read_text(encoding="utf-8", errors="ignore")
+            content_lower = content.lower()
+            hits = sum(content_lower.count(term) for term in query_terms)
+            if hits == 0:
+                continue
+
+            # 解析 frontmatter 获取元数据
+            meta = {}
+            if content.startswith("---"):
+                end = content.find("---", 3)
+                if end != -1:
+                    for line in content[3:end].split("\n"):
+                        if ":" in line:
+                            k, _, v = line.partition(":")
+                            meta[k.strip()] = v.strip().strip('"')
+
+            if domain_filter and meta.get("domain") and domain_filter not in meta.get("domain", ""):
+                continue
+
+            # 提取匹配片段
+            snippet = ""
+            for term in query_terms:
+                idx = content_lower.find(term)
+                if idx >= 0:
+                    start = max(0, idx - 60)
+                    end_idx = min(len(content), idx + 200)
+                    snippet = "..." + content[start:end_idx].replace("\n", " ") + "..."
+                    break
+
+            scored.append({
+                "content": snippet or content[:300],
+                "meta": {
+                    "id": meta.get("id", ""),
+                    "title": meta.get("title", md_file.stem),
+                    "domain": meta.get("domain", ""),
+                    "nature": meta.get("nature", ""),
+                    "file": str(md_file),
+                },
+                "score": hits,
+                "track": "global_md"
+            })
+        except Exception:
+            continue
+
+    return sorted(scored, key=lambda x: x["score"], reverse=True)[:top_n]
+
+
+# ─── 结果展示 ─────────────────────────────────────────────────────
+
+def format_result(idx: int, result: dict) -> str:
+    meta = result.get("meta", {})
+    track = result.get("track", "?")
+    score = result.get("score", 0)
+    title = meta.get("title", meta.get("id", "未知"))
+    domain = meta.get("domain", "?")
+    nature = meta.get("nature", "?")
+    content = result.get("content", "")[:200].replace("\n", " ")
+
+    track_icon = {"local": "🔒", "global": "🌐", "local_index": "📇", "global_md": "📄"}.get(track, "❓")
+
+    return (f"\n{idx}. {track_icon}[{track}] {title}\n"
+            f"   领域:{domain} 类型:{nature} 相关度:{score}\n"
+            f"   {content}")
+
+
+def main():
+    args = parse_args()
+    local_root = Path(args.local).expanduser()
+    root = Path(args.root).expanduser()
+    query = args.query
+    top_n = args.top
+
+    print(f"\n🔍 检索: '{query}' (范围:{args.track}, top:{top_n})")
+    print("=" * 60)
+
+    all_results = []
+
+    if args.keyword:
+        # 纯关键词模式
+        if args.track in ("local", "both"):
+            idx_path = local_root / "vector_db" / "local" / "keyword_index.json"
+            results = search_keyword_index(idx_path, query, top_n)
+            all_results.extend(results)
+
+        if args.track in ("global", "both"):
+            kb_dir = root / "03_knowledge"
+            results = search_md_files(kb_dir, query, top_n, args.domain)
+            all_results.extend(results)
+
+    else:
+        # 向量检索模式
+        if args.track in ("local", "both"):
+            local_db = local_root / "vector_db" / "local" / "chroma"
+            results = search_chroma(local_db, query, top_n, args.domain)
+            if not results:
+                # fallback 到关键词索引
+                idx_path = local_root / "vector_db" / "local" / "keyword_index.json"
+                results = search_keyword_index(idx_path, query, top_n)
+            all_results.extend(results)
+
+        if args.track in ("global", "both"):
+            global_db = local_root / "vector_db" / "global" / "chroma"
+            results = search_chroma(global_db, query, top_n, args.domain)
+            if not results:
+                # fallback 到 MD 全文搜索
+                results = search_md_files(root / "03_knowledge", query, top_n, args.domain)
+            all_results.extend(results)
+
+    # 去重 + 合并排序（local优先，同分数local排前）
+    seen = set()
+    deduped = []
+    for r in sorted(all_results, key=lambda x: (-x["score"], 0 if "local" in x["track"] else 1)):
+        key = r["meta"].get("id") or r["meta"].get("file") or r["content"][:50]
+        if key not in seen:
+            seen.add(key)
+            deduped.append(r)
+
+    if not deduped:
+        print("\n未找到相关结果。")
+        print("提示: 运行 vector_db_rebuild.py 重建向量库后检索效果更好")
+        return
+
+    for i, result in enumerate(deduped[:top_n], 1):
+        print(format_result(i, result))
+
+    print(f"\n共找到 {len(deduped)} 条结果（显示前{min(top_n, len(deduped))}条）")
+    print("🔒=本地记忆  🌐=协同知识库  📇=关键词索引  📄=MD全文")
+
+
+if __name__ == "__main__":
+    main()

mac-agent-os-main/02_skills/kb_manager/vector_db_rebuild.py

@@ -0,0 +1,292 @@
+#!/usr/bin/env python3
+"""
+vector_db_rebuild.py — 双轨向量库重建工具
+
+重建两套向量库：
+  - local：agent-os-local/ 下的私有内容（记忆、本机笔记）
+  - global：agent-sync/03_knowledge/ 下的协同知识库
+
+向量库存储在 agent-os-local/vector_db/ 下（永不同步）：
+  agent-os-local/vector_db/local/chroma/    ← 私有内容
+  agent-os-local/vector_db/global/chroma/   ← 协同知识（每机自行重建）
+
+使用：
+  python3 vector_db_rebuild.py --local ~/workbuddy-agent-os/agent-local --root ~/workbuddy-agent-os/agent-sync
+  python3 vector_db_rebuild.py --local ~/workbuddy-agent-os/agent-local --root ~/workbuddy-agent-os/agent-sync --track local
+  python3 vector_db_rebuild.py --local ~/workbuddy-agent-os/agent-local --root ~/workbuddy-agent-os/agent-sync --track global
+  python3 vector_db_rebuild.py --local ~/workbuddy-agent-os/agent-local --root ~/workbuddy-agent-os/agent-sync --incremental
+"""
+
+import argparse
+import json
+import os
+import re
+import sys
+from datetime import datetime
+from pathlib import Path
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description="AgentOS 双轨向量库重建")
+    parser.add_argument("--root", default="~/workbuddy-agent-os/agent-sync", help="agent-os 根目录")
+    parser.add_argument("--local", default="~/workbuddy-agent-os/agent-local", help="~/workbuddy-agent-os/agent-local")
+    parser.add_argument("--track", choices=["local", "global", "both"], default="both",
+                        help="重建哪套向量库 (默认 both)")
+    parser.add_argument("--incremental", action="store_true",
+                        help="增量模式：只添加新文件，不清空已有向量库")
+    parser.add_argument("--dry-run", action="store_true",
+                        help="预览模式：只列出待处理文件，不写入向量库")
+    return parser.parse_args()
+
+
+# ─── 文件扫描 ─────────────────────────────────────────────────────
+
+def scan_md_files(dirs: list[Path], exclude_patterns: list = None) -> list[dict]:
+    """扫描 MD 文件，返回文档列表"""
+    exclude_patterns = exclude_patterns or []
+    docs = []
+    for base_dir in dirs:
+        if not base_dir.exists():
+            continue
+        for md_file in sorted(base_dir.rglob("*.md")):
+            # 跳过隐藏文件和排除目录
+            if any(p in str(md_file) for p in exclude_patterns):
+                continue
+            if md_file.name.startswith("."):
+                continue
+            try:
+                content = md_file.read_text(encoding="utf-8", errors="ignore")
+                if len(content.strip()) < 50:
+                    continue
+
+                meta = extract_frontmatter(content)
+                body = strip_frontmatter(content)
+
+                docs.append({
+                    "id": meta.get("id") or f"FILE-{md_file.stat().st_mtime_ns}",
+                    "file": str(md_file),
+                    "title": meta.get("title", md_file.stem),
+                    "content": body[:2000],  # 限制长度
+                    "metadata": {
+                        "id": meta.get("id", ""),
+                        "title": meta.get("title", md_file.stem),
+                        "nature": meta.get("nature", ""),
+                        "domain": meta.get("domain", ""),
+                        "confidence": str(meta.get("confidence", "")),
+                        "source_type": meta.get("source_type", ""),
+                        "date_created": meta.get("date_created", ""),
+                        "file_path": str(md_file),
+                    }
+                })
+            except Exception as e:
+                print(f"  [SKIP] {md_file.name}: {e}")
+    return docs
+
+
+def extract_frontmatter(content: str) -> dict:
+    if not content.startswith("---"):
+        return {}
+    end = content.find("---", 3)
+    if end == -1:
+        return {}
+    meta = {}
+    for line in content[3:end].split("\n"):
+        if ":" in line:
+            k, _, v = line.partition(":")
+            meta[k.strip()] = v.strip().strip('"').strip("'")
+    return meta
+
+
+def strip_frontmatter(content: str) -> str:
+    if not content.startswith("---"):
+        return content
+    end = content.find("---", 3)
+    return content[end + 3:].strip() if end != -1 else content
+
+
+# ─── 向量库操作 ───────────────────────────────────────────────────
+
+def get_chroma_client(db_path: Path):
+    """获取 ChromaDB 客户端"""
+    try:
+        import chromadb
+        db_path.mkdir(parents=True, exist_ok=True)
+        return chromadb.PersistentClient(path=str(db_path))
+    except ImportError:
+        print("[ERROR] chromadb 未安装。请运行: pip install chromadb")
+        return None
+    except Exception as e:
+        print(f"[ERROR] 初始化 ChromaDB 失败: {e}")
+        return None
+
+
+def upsert_documents(client, collection_name: str, docs: list, incremental: bool = False):
+    """向 ChromaDB 写入文档"""
+    collection = client.get_or_create_collection(
+        name=collection_name,
+        metadata={"hnsw:space": "cosine"}
+    )
+
+    if not incremental:
+        # 全量重建：清空后写入
+        existing = collection.get(include=[])
+        if existing["ids"]:
+            collection.delete(ids=existing["ids"])
+
+    # 增量模式：获取已有 ID
+    existing_ids = set()
+    if incremental:
+        existing = collection.get(include=[])
+        existing_ids = set(existing["ids"])
+
+    new_docs = [d for d in docs if d["id"] not in existing_ids]
+    if not new_docs:
+        print(f"  [INFO] {collection_name}: 无新文档（共{len(docs)}个，全部已存在）")
+        return 0
+
+    # 批量写入（ChromaDB 建议每批 ≤ 5000）
+    batch_size = 500
+    total_added = 0
+    for i in range(0, len(new_docs), batch_size):
+        batch = new_docs[i:i + batch_size]
+        collection.add(
+            ids=[d["id"] for d in batch],
+            documents=[d["content"] for d in batch],
+            metadatas=[d["metadata"] for d in batch]
+        )
+        total_added += len(batch)
+
+    return total_added
+
+
+# ─── 关键词索引（轻量fallback） ──────────────────────────────────
+
+def build_keyword_index(docs: list, index_path: Path):
+    """构建关键词索引（JSON格式，无需向量库也可检索）"""
+    entries = []
+    for doc in docs:
+        meta = doc["metadata"]
+        content = doc["content"]
+        keywords = list(set(re.findall(r'[\u4e00-\u9fff]{2,}|[a-zA-Z]{4,}', content)))[:20]
+        entries.append({
+            "fact_id": doc["id"],
+            "title": meta.get("title", ""),
+            "keywords": keywords,
+            "domain": meta.get("domain", ""),
+            "nature": meta.get("nature", ""),
+            "summary": content[:150].replace("\n", " "),
+            "date": meta.get("date_created", ""),
+            "source": meta.get("source_type", ""),
+            "file": meta.get("file_path", "")
+        })
+
+    index_data = {
+        "version": "2.0.0",
+        "last_updated": datetime.now().isoformat(),
+        "total": len(entries),
+        "entries": entries
+    }
+    index_path.parent.mkdir(parents=True, exist_ok=True)
+    with open(index_path, "w", encoding="utf-8") as f:
+        json.dump(index_data, f, ensure_ascii=False, indent=2)
+    print(f"  [OK] 关键词索引: {len(entries)} 条 → {index_path}")
+
+
+# ─── 主流程 ──────────────────────────────────────────────────────
+
+def rebuild_local(local_root: Path, incremental: bool, dry_run: bool):
+    """重建本地向量库（私有内容）"""
+    print("\n── 重建 local 向量库 ──")
+    local_db_path = local_root / "vector_db" / "local" / "chroma"
+    keyword_index_path = local_root / "vector_db" / "local" / "keyword_index.json"
+
+    # 扫描本机私有内容
+    scan_dirs = [
+        local_root / "memory" / "raw",
+        local_root / "memory" / "long_term",
+        local_root / "materials" / "refined_for_inbox",
+    ]
+    docs = scan_md_files(scan_dirs)
+    print(f"  扫描到 {len(docs)} 个文档")
+
+    if dry_run:
+        for d in docs[:5]:
+            print(f"  [预览] {d['title']} ({d['metadata'].get('nature', '?')})")
+        return
+
+    # 构建关键词索引（始终构建，不依赖 chromadb）
+    build_keyword_index(docs, keyword_index_path)
+
+    # 构建向量索引（可选，需 chromadb）
+    client = get_chroma_client(local_db_path)
+    if client:
+        added = upsert_documents(client, "local_memory", docs, incremental)
+        print(f"  [OK] ChromaDB local: 新增 {added} 条向量")
+    else:
+        print("  [INFO] 跳过 ChromaDB，仅使用关键词索引")
+
+
+def rebuild_global(root: Path, local_root: Path, incremental: bool, dry_run: bool):
+    """重建协同向量库（知识库内容）"""
+    print("\n── 重建 global 向量库 ──")
+    global_db_path = local_root / "vector_db" / "global" / "chroma"
+    keyword_index_path = local_root / "vector_db" / "global" / "keyword_index.json"
+
+    # 扫描协同知识库（排除 inbox 等临时目录）
+    kb_root = root / "03_knowledge"
+    scan_dirs = [
+        kb_root / "10_concepts",
+        kb_root / "20_methods",
+        kb_root / "30_facts",
+        kb_root / "40_references",
+        kb_root / "50_resources",
+        kb_root / "60_opinions",
+        kb_root / "01_daily",
+    ]
+    exclude = ["00_inbox", "00_stream", "90_archive", "99_system", ".obsidian"]
+    docs = scan_md_files(scan_dirs, exclude_patterns=exclude)
+    print(f"  扫描到 {len(docs)} 个文档")
+
+    if dry_run:
+        for d in docs[:5]:
+            print(f"  [预览] {d['title']} ({d['metadata'].get('domain', '?')})")
+        return
+
+    build_keyword_index(docs, keyword_index_path)
+
+    client = get_chroma_client(global_db_path)
+    if client:
+        added = upsert_documents(client, "global_knowledge", docs, incremental)
+        print(f"  [OK] ChromaDB global: 新增 {added} 条向量")
+    else:
+        print("  [INFO] 跳过 ChromaDB，仅使用关键词索引")
+
+
+def main():
+    args = parse_args()
+    root = Path(args.root).expanduser()
+    local_root = Path(args.local).expanduser()
+
+    print("=" * 60)
+    print("AgentOS 双轨向量库重建")
+    print(f"  agent-os:       {root}")
+    print(f"  agent-os-local: {local_root}")
+    print(f"  重建范围: {args.track}")
+    if args.incremental:
+        print("  [增量模式]")
+    if args.dry_run:
+        print("  [DRY-RUN 模式]")
+    print("=" * 60)
+
+    if args.track in ("local", "both"):
+        rebuild_local(local_root, args.incremental, args.dry_run)
+
+    if args.track in ("global", "both"):
+        rebuild_global(root, local_root, args.incremental, args.dry_run)
+
+    print("\n[完成] 向量库重建完毕")
+    print("  下次检索时将自动使用新的向量库（kb_search.py）")
+
+
+if __name__ == "__main__":
+    main()

mac-agent-os-main/02_skills/kb_manager/version.json

@@ -0,0 +1,7 @@
+{
+  "name": "kb_manager",
+  "version": "1.1.0",
+  "description": "知识库管理技能——入库、分类、检索、备份、重分类",
+  "dependencies": ["sqlite3"],
+  "mcp_tools": []
+}