feat(monitor): 抖音 Web 接口客户端 —— 绕开爬虫子进程,直接发 HTTP
Deploy VitePress site to Pages / build (push) Canceled after 0s
Deploy VitePress site to Pages / Deploy (push) Canceled after 0s

移植自 mac-agent-os 的 mediacrawler_adapter:不起子进程、不开页面,用浏览器里那份
登录态直接调抖音 Web 接口。产物键名照抄 store/douyin,所以 ingest 那条链路一个字不用改。

**目前能用的(真环境实测,非推断)**:

    profile/other : 200, 7075 字节  —— 博主主页指标(粉丝/获赞/作品数/昵称)
    aweme/detail  : 200, 45425 字节 —— 单条作品详情(含点赞/评论/收藏/分享)

**目前不能用的:作品列表 `aweme/post`。** 两个互相独立的原因:
  1. 这个接口被抖音单独升级成了真校验:不带 x-tt-argus 回 403「Uifid Not Found」,
     带上 dummy 值回 200 + **空 body**。也就是说「头在不在」骗得过,「真校验」过不了。
     同一套头打 profile/other 和 aweme/detail 都是通的 —— 抖音是挑着接口加保护的,
     挑中的恰好是「批量拉作品列表」这个最敏感的动作。
  2. 改走页面截获也不行:CDP 浏览器打开博主主页会落到「验证码中间页」(当天大量探测的
     代价,过几小时要重测)。

  所以现在的边界是:**已知作品的指标刷新能做,自动发现新作品做不了**。

**排查中控住变量后得到的两条事实**(都写进注释了):
  · `Accept` / `Accept-Language` / `Referer` 才是主页接口能返回真数据的原因 —— 只有
    UA+client hints+Cookie 时是 200 但仅 121 字节的空壳,补上这三个头变 7074 字节。
    (我先前猜的 sec-ch-ua 不是关键。)
  · 因此 UA 与 client hints 必须**成套地取自同一个浏览器**,所以 BrowserIdentity 一次
    从 CDP 取齐 cookie + UA + hints,而不是各自写死。

「200 + 空 body 必须当场报错」也是刻意写死的:放过去它会在下游变成「这个博主没作品」,
把一次失败伪装成一条正常结果 —— 爬虫那条路正是这么栽的,还被翻译成「账号被封」。

测试 +11:cookie 解析、请求头成套性(含 uifid 缺失/回退)、产物键名与 store 对齐、
以及 _get 的三条失败路径(空 body / 403 带网关原话 / 正常返回)。
This commit is contained in:
2026-10-10 17:12:25 +08:00
parent 21ff01b894
commit 2112c1a870
1601 changed files with 367425 additions and 0 deletions
@@ -0,0 +1,418 @@
"""
schemas — 视频工厂统一分镜 Schema(标准接口)
================================================
**所有模块的"普通话"**:任何导演系统输出它、分镜编辑器编辑它、执行器消费它。
换了导演系统 / 能力供应商 / 模型,下游全不用改 —— 这是解耦的中枢。
来源:DCS 说明书 v2.0 第四节(导演创作系统统一分镜 Schema)
用法:
from schemas import Storyboard, Shot, GlobalAnchors
sb = Storyboard(project_id="vid_001", global_anchors=GlobalAnchors(...), shots=[...])
sb.to_dict() # → JSON 可传输
Storyboard.from_dict(data) # ← 从 JSON 还原
sb.save("path.json") / Storyboard.load("path.json")
"""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any, Literal, Optional
from pydantic import BaseModel, Field
# ═══════════════════════════════════════════════════════════
# 全局锚点(跨镜头一致性)
# ═══════════════════════════════════════════════════════════
class CastMember(BaseModel):
"""演员表成员:**戏内代号 + 绑定的角色库资产**
设计(用户拍板):
· `slot` 戏内代号(主角 / 配角 / 人物一…)—— 分镜里用它指代"谁出场"
· `asset_id` / `name` 指向角色库资产 —— 实际生成时取参考图(保人脸一致)
· `note` 补充要求(年龄/外貌)—— 资产未绑定或不够贴合时,给导演的提示
两者兼备:代号管"戏里是谁",资产管"长相是谁"。
"""
slot: str = "" # 戏内代号
asset_id: str = "" # 角色库资产 id(可为空 = 暂未绑定)
name: str = "" # 资产名(展示用)
note: str = "" # 补充要求(年龄/外貌/气质)
voice_id: str = "" # 配音音色(CosyVoice voice 参数,如 longanyang)
# 空 = 按代号从路人音色池稳定分配(同一角色全片同音色)
class GlobalAnchors(BaseModel):
"""全局锚点:整个片子的统一设定(角色/场地/产品/光线/镜头语言)
★ **资产注入机制**(即"提前喂给导演"):
`*_id` 指向资产库;`*_context` 是该资产的**详情文本**(名称 + 描述 + 特征字段),
由接口在生成分镜**之前**从资产库查出并填入
→ 导演系统据此写出**具体的**画面描述,而不是只拿到一个 ID 靠猜。
★ **演员表(cast)**:短片常有多个人物。`cast` 声明全片的角色(代号 + 资产),
分镜用 `Shot.cast_slots` 标注"这一镜谁出场"。逐镜头指定优先于全局。
"""
character_id: str = "" # ← 保留(单角色场景 / 向后兼容)
character_3d_id: Optional[str] = None
location_id: str = "" # 主场地(短片≈一场戏,全片默认)
product_id: Optional[str] = None
wardrobe: str = "" # 服装(可写"第1套/第2套"或具体描述)
lighting_state: str = "" # 光线状态
camera_lens: str = "" # 镜头语言
shooting_style: str = "documentary_realism" # 拍摄风格(伪纪录片)
realism_boost: str = "mild" # 反 AI 感强度: off/mild/strong
director_style: str = "" # 导演风格 key(如 wong_kar_wai / nolan),空=不用
consistency_seed: int = 42 # 一致性种子(跨镜头固定)
# ── ★ 演员表(多角色;用户 2026-10-03 要求)──
cast: list[CastMember] = Field(default_factory=list)
# ── 资产详情(由接口从资产库注入,供导演提示词直接引用)──
character_context: str = "" # 角色详情(名称 · 年龄 · 外貌 · 服装)
location_context: str = "" # 场地详情(名称 · 空间类型 · 光线 · 材质)
product_context: str = "" # 产品详情
props_context: str = "" # 道具详情
cast_context: str = "" # ★ 演员表详情(逐人:代号 · 资产名 · 外貌特征)
def effective_cast(self) -> list["CastMember"]:
"""向后兼容:cast 为空但设了 character_id → 视为单个「主角」"""
if self.cast:
return list(self.cast)
if self.character_id:
return [CastMember(slot="主角", asset_id=self.character_id, name="(未命名)")]
return []
# ═══════════════════════════════════════════════════════════
# 镜头契约(景别/角度/运动/光学)
# ═══════════════════════════════════════════════════════════
class ShotContract(BaseModel):
"""镜头契约:专业摄影语言(v5.2 导演技能的核心产出)"""
shot_size: str = "" # 景别(特写/近景/中景/全景/远景)
angle: str = "" # 机位角度(平视/俯视/仰视/过肩)
camera_move: str = "" # 运动(固定/推/拉/摇/移/跟/环绕)
lens_feel: str = "" # 光学质感(广角/标准/长焦/浅景深)
depth_of_field: str = "" # 景深
frame_rate_feel: str = "" # 帧率感(常规/升格/降格)
# ═══════════════════════════════════════════════════════════
# 特效 / 表情 / 音效
# ═══════════════════════════════════════════════════════════
class VFXConfig(BaseModel):
"""特效配置(子弹时间/环绕运镜/慢动作/焦点锁定)"""
vfx_id: Optional[str] = None
vfx_type: Optional[Literal["bullet_time", "orbit", "slow_motion",
"focus_lock", "custom"]] = None
provider: Optional[str] = None
params: Optional[dict] = None
class ExpressionDetail(BaseModel):
"""微表情控制(面部细节)"""
eyes: Optional[str] = None
mouth: Optional[str] = None
brow: Optional[str] = None
class SFX(BaseModel):
"""音效点(某秒插入某音效)"""
id: str = "" # 素材 id(执行时匹配到素材后回填;导演只给 desc)
at_second: float = 0.0
volume: float = 0.4
desc: str = "" # 音效描述(导演产出,执行时据此选素材)
class AudioConfig(BaseModel):
"""音频配置(人声/BGM/音效)"""
voice_volume: float = 1.0
bgm_id: Optional[str] = None
bgm_volume: float = 0.15
bgm_fade_in: float = 1.0
bgm_fade_out: float = 1.0
sfx: list[SFX] = Field(default_factory=list)
audio_cue: Optional[str] = None # 声音提示(环境音等)
denoise: bool = False
class Subtitle(BaseModel):
"""字幕"""
enabled: bool = True
text: str = ""
style: str = "手机系统默认"
position: str = "bottom_center"
# ═══════════════════════════════════════════════════════════
# 转场 / 速度 / 调色 / 后期
# ═══════════════════════════════════════════════════════════
class Transition(BaseModel):
"""转场"""
type: Literal["cut", "dissolve", "fade", "wipe"] = "cut"
duration: float = 0.0
class Speed(BaseModel):
"""变速"""
factor: float = 1.0
ramp: Optional[str] = None
class ColorConfig(BaseModel):
"""调色(去 AI 化链的一环)"""
lut: Optional[str] = None
saturation_offset: float = -0.12
temperature_shift: float = 0.03
class PostProcess(BaseModel):
"""后期处理(去 AI 化:模糊 + 颗粒)"""
grain_intensity: int = 12
blur_sigma: float = 0.4
frame_interpolation: bool = False
class OutputConfig(BaseModel):
"""输出规格"""
resolution: str = "1080x1920"
fps: int = 30
bitrate: str = "8M"
# ═══════════════════════════════════════════════════════════
# 镜头 / 分镜表
# ═══════════════════════════════════════════════════════════
SHOT_TYPES = Literal["talking_head", "action", "background",
"comfyui", "vfx", "3d_character"]
CONTINUITY_MODES = Literal["fresh_location",
"previous_frame_continuation",
"location_reappearance"]
class Shot(BaseModel):
"""单个镜头(统一分镜 Schema 的核心单元)"""
shot_id: str
scene_id: str = ""
duration: float = 5.0 # 时长(秒)
type: SHOT_TYPES = "action" # 镜头类型 → 决定调哪个能力
# 生成能力(**手动覆盖**自动判定)
# "" = 自动(按 type + 是否绑角色判定)
# text_to_video / image_to_video / vfx / comfyui = 手动指定
generation_capability: str = ""
# 资产引用(按 id 去资产库取)
character_id: str = "" # 单角色(向后兼容;多角色看 cast_slots)
character_3d_id: Optional[str] = None
location_id: str = "" # 为空则用全局主场地(短片常为同一场景)
product_id: Optional[str] = None
product_focus: bool = False
product_placement: Optional[str] = None
props: list[str] = Field(default_factory=list)
# ★ 出场人物(戏内代号,对应 GlobalAnchors.cast[].slot)
# 逐镜头指定 **优先于** 全局演员表:空 = 未指定(导演/AI 自行判断)
cast_slots: list[str] = Field(default_factory=list)
# 连续性
continuity_mode: CONTINUITY_MODES = "fresh_location"
# ★ 技术需求声明(创作层 → 技术优选层的输入)
# 导演只声明"需要什么技术条件",不指定用什么工具/模型;
# 由技术优选层(capability_matcher)据此匹配能力并产出缺口报告。
# 结构示例:
# {
# "continuity": {"character": "must|prefer|none", "wardrobe": ..., "location": ...,
# "expression": ..., "props": ...},
# "chain_from_prev": false, # 是否要求承接上一镜尾帧
# "motion_level": "static|small|medium|large",
# "camera_difficulty": "fixed|simple|complex",
# "needs": ["特殊需求自由文本"]
# }
tech_requirements: Optional[dict] = None
# 摄影契约
shot_contract: ShotContract = Field(default_factory=ShotContract)
# 内容
vfx: Optional[VFXConfig] = None
visual_description: str = "" # 画面描述(提示词主体)
text: Optional[str] = None # 台词/旁白
sentiment: Optional[str] = None # 情绪
expression_detail: Optional[ExpressionDetail] = None
action: Optional[str] = None # 动作描述
# 音频/字幕/转场/后期
audio: Optional[AudioConfig] = None
subtitle: Optional[Subtitle] = None
transition_in: Transition = Field(default_factory=Transition)
transition_out: Transition = Field(default_factory=Transition)
speed: Speed = Field(default_factory=Speed)
color: ColorConfig = Field(default_factory=ColorConfig)
post_process: PostProcess = Field(default_factory=PostProcess)
output: OutputConfig = Field(default_factory=OutputConfig)
# 运行时状态(不进 Schema 契约,供 UI 展示执行进度)
status: Optional[str] = None # pending/generating/done/failed
artifact: Optional[str] = None # 产物路径
error: Optional[str] = None
class Storyboard(BaseModel):
"""分镜表(完整片子)—— 导演系统 / 编辑器 / 执行器的共同契约"""
project_id: str
schema_version: str = "2.0"
title: str = ""
story: str = "" # 原始故事/文案
content_type: str = "short_drama" # 内容类型
director_id: str = "legacy_scene_planner" # 用了哪个导演系统
global_anchors: GlobalAnchors = Field(default_factory=GlobalAnchors)
shots: list[Shot] = Field(default_factory=list)
# 运行时状态
status: Optional[str] = None
output_path: Optional[str] = None
created_at: Optional[str] = None
# 交付状态(三级诚实标注,见 produce-ai-video 验收方法论)
# 合格成片 —— 全部镜头成功 + 已完整播放审查
# 候选成片 —— 可播放,但有镜头失败或未经完整播放审查
# 未完成/待验证 —— 未生成 / 拼接失败
delivery_status: Optional[str] = None
delivery_note: Optional[str] = None
# ── 便捷方法 ──────────────────────────────────────
def total_duration(self) -> float:
"""总时长(秒)"""
return round(sum(s.duration for s in self.shots), 2)
def shot_by_id(self, shot_id: str) -> Optional[Shot]:
for s in self.shots:
if s.shot_id == shot_id:
return s
return None
def reindex(self) -> None:
"""重排镜头序号(删/插后调用)"""
for i, s in enumerate(self.shots, 1):
s.shot_id = f"shot_{i:02d}"
def to_dict(self) -> dict:
"""→ JSON 可传输结构"""
return json.loads(self.model_dump_json())
@classmethod
def from_dict(cls, data: dict) -> "Storyboard":
"""← 从 JSON 还原(容错:忽略多余字段)"""
return cls.model_validate(data or {})
def save(self, path: str | Path) -> Path:
p = Path(path)
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(json.dumps(self.to_dict(), ensure_ascii=False, indent=2), encoding="utf-8")
return p
@classmethod
def load(cls, path: str | Path) -> "Storyboard":
return cls.from_dict(json.loads(Path(path).read_text(encoding="utf-8")))
def summary(self) -> dict:
"""给 UI 用的概览"""
by_type: dict[str, int] = {}
for s in self.shots:
by_type[s.type] = by_type.get(s.type, 0) + 1
return {
"project_id": self.project_id,
"title": self.title,
"director_id": self.director_id,
"shot_count": len(self.shots),
"total_duration": self.total_duration(),
"by_type": by_type,
"status": self.status,
"output_path": self.output_path,
}
# ═══════════════════════════════════════════════════════════
# 辅助:分镜 ↔ 现有 scene_planner 场景格式互转
# ═══════════════════════════════════════════════════════════
def scenes_to_storyboard(scenes: list[dict], project_id: str, story: str = "",
anchors: GlobalAnchors | None = None,
director_id: str = "legacy_scene_planner") -> Storyboard:
"""把 story_director/scene_planner 的场景列表 → 统一分镜 Schema
scene_planner 的场景字段(兼容多种命名):scene_id/index/description/prompt/
duration/text/sentiment/character/action
"""
shots: list[Shot] = []
for i, sc in enumerate(scenes or [], 1):
desc = (sc.get("description") or sc.get("prompt") or sc.get("visual") or "").strip()
shots.append(Shot(
shot_id=f"shot_{i:02d}",
scene_id=str(sc.get("scene_id") or sc.get("index") or i),
duration=float(sc.get("duration") or 5),
type=sc.get("type") or "action",
character_id=str(sc.get("character") or (anchors.character_id if anchors else "")),
location_id=str(sc.get("location") or (anchors.location_id if anchors else "")),
visual_description=desc,
text=sc.get("text") or sc.get("dialogue") or sc.get("narration"),
sentiment=sc.get("sentiment") or sc.get("emotion"),
action=sc.get("action"),
))
return Storyboard(
project_id=project_id,
title=story[:40] if story else "",
story=story,
director_id=director_id,
global_anchors=anchors or GlobalAnchors(),
shots=shots,
)
def storyboard_to_scenes(sb: Storyboard) -> list[dict]:
"""统一分镜 Schema → scene_planner 格式(供现有 batch_generator 消费)"""
return [
{
"scene_id": s.scene_id or s.shot_id,
"index": i,
"description": s.visual_description,
"prompt": s.visual_description,
"duration": s.duration,
"type": s.type,
"text": s.text,
"sentiment": s.sentiment,
"action": s.action,
"character": s.character_id,
"location": s.location_id,
}
for i, s in enumerate(sb.shots, 1)
]