feat(monitor): 抖音 Web 接口客户端 —— 绕开爬虫子进程，直接发 HTTP
移植自 mac-agent-os 的 mediacrawler_adapter：不起子进程、不开页面，用浏览器里那份
登录态直接调抖音 Web 接口。产物键名照抄 store/douyin，所以 ingest 那条链路一个字不用改。
**目前能用的（真环境实测，非推断）**：
profile/other : 200, 7075 字节 —— 博主主页指标（粉丝/获赞/作品数/昵称）
aweme/detail : 200, 45425 字节 —— 单条作品详情（含点赞/评论/收藏/分享）
**目前不能用的：作品列表 `aweme/post`。** 两个互相独立的原因：
1. 这个接口被抖音单独升级成了真校验：不带 x-tt-argus 回 403「Uifid Not Found」，
带上 dummy 值回 200 + **空 body**。也就是说「头在不在」骗得过，「真校验」过不了。
同一套头打 profile/other 和 aweme/detail 都是通的 —— 抖音是挑着接口加保护的，
挑中的恰好是「批量拉作品列表」这个最敏感的动作。
2. 改走页面截获也不行：CDP 浏览器打开博主主页会落到「验证码中间页」（当天大量探测的
代价，过几小时要重测）。
所以现在的边界是：**已知作品的指标刷新能做，自动发现新作品做不了**。
**排查中控住变量后得到的两条事实**（都写进注释了）：
· `Accept` / `Accept-Language` / `Referer` 才是主页接口能返回真数据的原因 —— 只有
UA+client hints+Cookie 时是 200 但仅 121 字节的空壳，补上这三个头变 7074 字节。
（我先前猜的 sec-ch-ua 不是关键。）
· 因此 UA 与 client hints 必须**成套地取自同一个浏览器**，所以 BrowserIdentity 一次
从 CDP 取齐 cookie + UA + hints，而不是各自写死。
「200 + 空 body 必须当场报错」也是刻意写死的：放过去它会在下游变成「这个博主没作品」，
把一次失败伪装成一条正常结果 —— 爬虫那条路正是这么栽的，还被翻译成「账号被封」。
测试 +11：cookie 解析、请求头成套性（含 uifid 缺失/回退）、产物键名与 store 对齐、
以及 _get 的三条失败路径（空 body / 403 带网关原话 / 正常返回）。
This commit is contained in:
@@ -0,0 +1,376 @@
|
||||
"""
|
||||
AVE speed_ramp — 变速卡点引擎
|
||||
|
||||
功能:
|
||||
1. 对视频/音频施加匀速变速
|
||||
2. 缓变速度曲线: 将视频分段, 每段不同速度, 组合成渐变效果
|
||||
3. 卡点策略集成: 根据能量/节拍密度生成动态速度曲线
|
||||
|
||||
核心: FFmpeg setpts(视频) + atempo(音频) 滤波器链
|
||||
|
||||
用法:
|
||||
from composer.speed_ramp import speed_ramp, speed_curve
|
||||
|
||||
# 匀速 0.8x
|
||||
speed_ramp("input.mp4", "slow.mp4", speed=0.8)
|
||||
|
||||
# 缓变 0.7→1.0→1.3 (6段渐变)
|
||||
speed_ramp("input.mp4", "ramp.mp4", curve=[0.7, 1.0, 1.3], segments=6)
|
||||
|
||||
# 节拍驱动的变速 (高能段加速, 低能段减速)
|
||||
speed_by_energy("input.mp4", beat_info={"segments": [...]}, output="energy_ramp.mp4")
|
||||
"""
|
||||
import sys, os; sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from lib.logger import get_logger
|
||||
from lib.ffmpeg import FFMPEG # noqa: E402 (FFmpeg 绝对路径)
|
||||
|
||||
logger = get_logger("speed_ramp")
|
||||
|
||||
# ── 常量 ──────────────────────────────────────────────────
|
||||
|
||||
# atempo 支持的范围: 0.5~2.0 (单次)
|
||||
# 需要更极端的变速时, 级联多个 atempo
|
||||
ATEMPO_MIN = 0.5
|
||||
ATEMPO_MAX = 2.0
|
||||
|
||||
|
||||
# ── 公共 API ──────────────────────────────────────────────
|
||||
|
||||
def speed_ramp(
|
||||
input_path: str,
|
||||
output_path: str = "speed_ramp.mp4",
|
||||
speed: float = 1.0,
|
||||
curve: list[float] | None = None,
|
||||
segments: int = 6,
|
||||
keep_audio: bool = True,
|
||||
) -> str:
|
||||
"""
|
||||
变速处理
|
||||
|
||||
参数:
|
||||
input_path: 输入视频路径
|
||||
output_path: 输出视频路径
|
||||
speed: 匀速倍率 (curve 为空时使用)
|
||||
curve: 缓变速度曲线 [起点倍率, ..., 终点倍率] (优先于 speed)
|
||||
例: [0.7, 1.0, 1.3] 表示从 0.7x 渐变到 1.3x
|
||||
segments: 缓变分段数 (越多越平滑, 默认 6)
|
||||
keep_audio: 是否保留音频 (默认 True)
|
||||
|
||||
返回: output_path
|
||||
"""
|
||||
if not os.path.exists(input_path):
|
||||
raise FileNotFoundError(f"输入文件不存在: {input_path}")
|
||||
|
||||
duration = _get_duration(input_path)
|
||||
|
||||
if curve and len(curve) >= 2:
|
||||
return _apply_curve(input_path, output_path, curve, segments, duration, keep_audio)
|
||||
else:
|
||||
return _apply_constant(input_path, output_path, speed, duration, keep_audio)
|
||||
|
||||
|
||||
def speed_by_energy(
|
||||
input_path: str,
|
||||
output_path: str = "energy_ramp.mp4",
|
||||
beat_segments: list[dict] | None = None,
|
||||
base_speed: float = 1.0,
|
||||
high_speed: float = 1.5,
|
||||
low_speed: float = 0.7,
|
||||
keep_audio: bool = True,
|
||||
) -> str:
|
||||
"""
|
||||
按节拍能量生成变速曲线
|
||||
|
||||
beat_segments 格式: [{"start": 0.0, "end": 2.0, "beats": 4, "energy": 0.8}, ...]
|
||||
高能量(>0.6) → 加速, 低能量(<0.3) → 减速, 中间 → 基础速度
|
||||
|
||||
返回: output_path
|
||||
"""
|
||||
if not os.path.exists(input_path):
|
||||
raise FileNotFoundError(f"输入文件不存在: {input_path}")
|
||||
|
||||
if not beat_segments:
|
||||
logger.warning("无节拍段落, 回落匀速")
|
||||
return _apply_constant(input_path, output_path, base_speed, _get_duration(input_path), keep_audio)
|
||||
|
||||
# 按能量生成每段速度
|
||||
temp_dir = tempfile.mkdtemp(prefix="ave_spd_")
|
||||
seg_files = []
|
||||
total_duration = _get_duration(input_path)
|
||||
|
||||
for idx, seg in enumerate(beat_segments):
|
||||
start = seg["start"]
|
||||
end = seg["end"]
|
||||
dur = end - start
|
||||
if dur < 0.3:
|
||||
continue
|
||||
|
||||
# 根据能量决定速度 (线性插值)
|
||||
energy = seg.get("energy", 0.5)
|
||||
if energy > 0.6:
|
||||
spd = base_speed + (high_speed - base_speed) * min((energy - 0.6) / 0.4, 1.0)
|
||||
elif energy < 0.3:
|
||||
spd = low_speed + (base_speed - low_speed) * (energy / 0.3)
|
||||
else:
|
||||
spd = base_speed
|
||||
|
||||
spd = max(ATEMPO_MIN, min(ATEMPO_MAX, spd))
|
||||
seg_out = os.path.join(temp_dir, f"seg_{idx:03d}.mp4")
|
||||
_apply_constant(input_path, seg_out, spd, dur, keep_audio, ss=start, t=dur)
|
||||
seg_files.append(seg_out)
|
||||
|
||||
if not seg_files:
|
||||
shutil.rmtree(temp_dir, ignore_errors=True)
|
||||
return _apply_constant(input_path, output_path, base_speed, total_duration, keep_audio)
|
||||
|
||||
# 拼接
|
||||
_concat_segments(seg_files, output_path, keep_audio)
|
||||
import shutil
|
||||
shutil.rmtree(temp_dir, ignore_errors=True)
|
||||
|
||||
logger.info(f"✅ 能量变速完成: {output_path} ({len(seg_files)} 段)")
|
||||
return output_path
|
||||
|
||||
|
||||
def make_energy_curve(
|
||||
beat_info: dict,
|
||||
base_speed: float = 1.0,
|
||||
high_speed: float = 1.5,
|
||||
low_speed: float = 0.7,
|
||||
) -> list[tuple[float, float, float]]:
|
||||
"""
|
||||
从节拍检测结果生成能量驱动速度曲线
|
||||
|
||||
返回: [(start_sec, end_sec, speed), ...]
|
||||
"""
|
||||
segments = beat_info.get("segments", [])
|
||||
if not segments:
|
||||
return [(0, beat_info.get("total_duration", 30), base_speed)]
|
||||
|
||||
# 计算每段的归一化能量 (基于拍点密度)
|
||||
max_beats = max(s.get("beats", 0) for s in segments) or 1
|
||||
curves = []
|
||||
for s in segments:
|
||||
dur = s["end"] - s["start"]
|
||||
if dur <= 0:
|
||||
continue
|
||||
# 拍点密度 → 能量
|
||||
density = s.get("beats", 0) / max(max_beats, 1)
|
||||
energy = max(0.0, min(1.0, density))
|
||||
|
||||
if energy > 0.6:
|
||||
spd = base_speed + (high_speed - base_speed) * min((energy - 0.6) / 0.4, 1.0)
|
||||
elif energy < 0.3:
|
||||
spd = low_speed + (base_speed - low_speed) * (energy / 0.3)
|
||||
else:
|
||||
spd = base_speed
|
||||
spd = max(ATEMPO_MIN, min(ATEMPO_MAX, spd))
|
||||
curves.append((s["start"], s["end"], spd))
|
||||
|
||||
return curves
|
||||
|
||||
|
||||
# ── 内部实现 ──────────────────────────────────────────────
|
||||
|
||||
def _apply_constant(
|
||||
input_path: str,
|
||||
output_path: str,
|
||||
speed: float,
|
||||
duration: float | None = None,
|
||||
keep_audio: bool = True,
|
||||
ss: float = 0.0,
|
||||
t: float | None = None,
|
||||
) -> str:
|
||||
"""匀速变速"""
|
||||
speed = max(ATEMPO_MIN, min(ATEMPO_MAX, speed))
|
||||
vf = f"setpts={1/speed}*PTS"
|
||||
|
||||
cmd = [FFMPEG, "-y"]
|
||||
|
||||
# 如果指定了裁剪区间
|
||||
if ss > 0:
|
||||
cmd += ["-ss", str(ss)]
|
||||
cmd += ["-i", input_path]
|
||||
if t is not None:
|
||||
cmd += ["-t", str(t)]
|
||||
|
||||
if keep_audio:
|
||||
# atempo 不支持超出 [0.5, 2.0], 级联处理
|
||||
af = _build_atempo_filter(speed)
|
||||
# ⚠️ af 自带输入/输出标签（`[0:a]atempo=…[a]`），不能再拼一个 [a]
|
||||
# （否则 filter 图非法 → 音频编码 0 字节，输出只有画面）
|
||||
cmd += ["-filter_complex", f"{vf}[v];{af}"]
|
||||
cmd += ["-map", "[v]", "-map", "[a]"]
|
||||
else:
|
||||
cmd += ["-filter_complex", f"{vf}[v]", "-map", "[v]", "-an"]
|
||||
|
||||
cmd += ["-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-c:a", "aac", "-b:a", "192k", "-pix_fmt", "yuv420p", output_path]
|
||||
|
||||
try:
|
||||
subprocess.run(cmd, capture_output=True, timeout=300, check=True)
|
||||
logger.debug(f" 匀速 {speed:.2f}x → {output_path}")
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.warning(f" 变速失败: {e.stderr[:200]}")
|
||||
# 回落: 不变速
|
||||
subprocess.run([FFMPEG, "-y", "-i", input_path,
|
||||
"-c", "copy", output_path], capture_output=True, timeout=60, check=True)
|
||||
|
||||
return output_path
|
||||
|
||||
|
||||
def _apply_curve(
|
||||
input_path: str,
|
||||
output_path: str,
|
||||
curve: list[float],
|
||||
num_segments: int,
|
||||
total_duration: float,
|
||||
keep_audio: bool,
|
||||
) -> str:
|
||||
"""缓变速度曲线: 将输入切成 N 段, 每段按曲线插值变速, 再拼接"""
|
||||
# 生成每段的速度
|
||||
speeds = _interpolate_curve(curve, num_segments)
|
||||
seg_duration = total_duration / num_segments
|
||||
|
||||
temp_dir = tempfile.mkdtemp(prefix="ave_curve_")
|
||||
seg_files = []
|
||||
|
||||
for idx, spd in enumerate(speeds):
|
||||
seg_start = idx * seg_duration
|
||||
seg_out = os.path.join(temp_dir, f"seg_{idx:03d}.mp4")
|
||||
_apply_constant(input_path, seg_out, spd, seg_duration,
|
||||
keep_audio, ss=seg_start, t=seg_duration)
|
||||
seg_files.append(seg_out)
|
||||
|
||||
# 拼接
|
||||
_concat_segments(seg_files, output_path, keep_audio)
|
||||
import shutil
|
||||
shutil.rmtree(temp_dir, ignore_errors=True)
|
||||
|
||||
logger.info(f"✅ 曲线变速完成: {output_path} ({len(speeds)} 段, "
|
||||
f"速度 {speeds[0]:.2f}→{speeds[-1]:.2f})")
|
||||
return output_path
|
||||
|
||||
|
||||
def _interpolate_curve(curve: list[float], num_segments: int) -> list[float]:
|
||||
"""在曲线控制点之间线性插值"""
|
||||
if len(curve) >= num_segments:
|
||||
return curve[:num_segments]
|
||||
|
||||
# 在 curve 点之间均匀插值到 num_segments 个点
|
||||
result = []
|
||||
n_controls = len(curve) - 1
|
||||
for i in range(num_segments):
|
||||
# 映射到 [0, n_controls] 区间
|
||||
pos = i / (num_segments - 1) * n_controls if num_segments > 1 else 0
|
||||
idx_low = int(pos)
|
||||
idx_high = min(idx_low + 1, n_controls)
|
||||
frac = pos - idx_low
|
||||
# 线性插值
|
||||
val = curve[idx_low] * (1 - frac) + curve[idx_high] * frac
|
||||
result.append(max(ATEMPO_MIN, min(ATEMPO_MAX, val)))
|
||||
return result
|
||||
|
||||
|
||||
def _build_atempo_filter(speed: float) -> str:
|
||||
"""构建 atempo 滤波器链 (支持 [0.5, 2.0], 超范围级联)
|
||||
|
||||
atempo 单次只能 [0.5, 2.0].
|
||||
例如 speed=3.0 → atempo=2.0, atempo=1.5
|
||||
例如 speed=0.4 → atempo=0.5, atempo=0.8
|
||||
"""
|
||||
if ATEMPO_MIN <= speed <= ATEMPO_MAX:
|
||||
return f"[0:a]atempo={speed}[a]"
|
||||
|
||||
# 级联: 分解为多个在范围内的 atempo
|
||||
factors = []
|
||||
remaining = speed
|
||||
if speed > ATEMPO_MAX:
|
||||
while remaining > ATEMPO_MAX:
|
||||
factors.append(ATEMPO_MAX)
|
||||
remaining /= ATEMPO_MAX
|
||||
if remaining >= ATEMPO_MIN:
|
||||
factors.append(remaining)
|
||||
elif speed < ATEMPO_MIN:
|
||||
while remaining < ATEMPO_MIN:
|
||||
factors.append(ATEMPO_MIN)
|
||||
remaining /= ATEMPO_MIN
|
||||
if remaining <= ATEMPO_MAX:
|
||||
factors.append(remaining)
|
||||
|
||||
if not factors:
|
||||
factors = [speed]
|
||||
|
||||
# 构建级联链: [0:a]atempo=f0[a0];[a0]atempo=f1[a1];...
|
||||
chain = []
|
||||
prev = "0:a"
|
||||
for i, f in enumerate(factors):
|
||||
label = f"a{i}"
|
||||
chain.append(f"[{prev}]atempo={f:.4f}[{label}]")
|
||||
prev = label
|
||||
return ";".join(chain)
|
||||
|
||||
|
||||
def _concat_segments(seg_files: list[str], output_path: str, keep_audio: bool):
|
||||
"""拼接变速后的视频段"""
|
||||
if len(seg_files) == 1:
|
||||
subprocess.run([FFMPEG, "-y", "-i", seg_files[0],
|
||||
"-c", "copy", output_path], capture_output=True, timeout=60, check=True)
|
||||
return
|
||||
|
||||
# 用 concat demuxer
|
||||
concat_file = tempfile.NamedTemporaryFile(suffix=".txt", delete=False, mode="w").name
|
||||
with open(concat_file, "w") as f:
|
||||
for sf in seg_files:
|
||||
f.write(f"file '{os.path.abspath(sf)}'\n")
|
||||
|
||||
cmd = [FFMPEG, "-y", "-f", "concat", "-safe", "0", "-i", concat_file]
|
||||
if keep_audio:
|
||||
cmd += ["-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-c:a", "aac", "-b:a", "192k", "-pix_fmt", "yuv420p"]
|
||||
else:
|
||||
cmd += ["-c:v", "libx264", "-preset", "fast", "-crf", "23", "-an"]
|
||||
cmd += [output_path]
|
||||
|
||||
try:
|
||||
subprocess.run(cmd, capture_output=True, timeout=300, check=True)
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.warning(f" 拼接失败, 回落 concat: {e.stderr[:200]}")
|
||||
# 回落: filter concat
|
||||
concat_filter = "".join(f"[{i}:v]" for i in range(len(seg_files)))
|
||||
cmd2 = [FFMPEG, "-y"]
|
||||
for sf in seg_files:
|
||||
cmd2 += ["-i", sf]
|
||||
cmd2 += ["-filter_complex",
|
||||
f"{concat_filter}concat=n={len(seg_files)}:v=1{'a=1' if keep_audio else '':1}[v]"
|
||||
+ (f";{concat_filter.replace(':v',':a')}concat=n={len(seg_files)}:v=0:a=1[a]"
|
||||
if keep_audio else ""),
|
||||
"-map", "[v]"]
|
||||
if keep_audio:
|
||||
cmd2 += ["-map", "[a]"]
|
||||
cmd2 += ["-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-c:a", "aac", "-b:a", "192k", "-pix_fmt", "yuv420p", output_path]
|
||||
subprocess.run(cmd2, capture_output=True, timeout=300, check=True)
|
||||
|
||||
os.unlink(concat_file)
|
||||
|
||||
|
||||
def _get_duration(path: str) -> float:
|
||||
"""获取媒体文件时长（秒）
|
||||
|
||||
用 lib.ffmpeg.probe_duration（ffmpeg -i 解析）——
|
||||
本机 ffprobe 实为 ffmpeg 的符号链接，不支持 -show_entries（会静默失败）
|
||||
"""
|
||||
try:
|
||||
from lib.ffmpeg import probe_duration
|
||||
d = probe_duration(path)
|
||||
return d if d > 0 else 30.0
|
||||
except Exception:
|
||||
return 30.0
|
||||
Reference in New Issue
Block a user