feat(monitor): 抖音 Web 接口客户端 —— 绕开爬虫子进程，直接发 HTTP
移植自 mac-agent-os 的 mediacrawler_adapter：不起子进程、不开页面，用浏览器里那份
登录态直接调抖音 Web 接口。产物键名照抄 store/douyin，所以 ingest 那条链路一个字不用改。
**目前能用的（真环境实测，非推断）**：
profile/other : 200, 7075 字节 —— 博主主页指标（粉丝/获赞/作品数/昵称）
aweme/detail : 200, 45425 字节 —— 单条作品详情（含点赞/评论/收藏/分享）
**目前不能用的：作品列表 `aweme/post`。** 两个互相独立的原因：
1. 这个接口被抖音单独升级成了真校验：不带 x-tt-argus 回 403「Uifid Not Found」，
带上 dummy 值回 200 + **空 body**。也就是说「头在不在」骗得过，「真校验」过不了。
同一套头打 profile/other 和 aweme/detail 都是通的 —— 抖音是挑着接口加保护的，
挑中的恰好是「批量拉作品列表」这个最敏感的动作。
2. 改走页面截获也不行：CDP 浏览器打开博主主页会落到「验证码中间页」（当天大量探测的
代价，过几小时要重测）。
所以现在的边界是：**已知作品的指标刷新能做，自动发现新作品做不了**。
**排查中控住变量后得到的两条事实**（都写进注释了）：
· `Accept` / `Accept-Language` / `Referer` 才是主页接口能返回真数据的原因 —— 只有
UA+client hints+Cookie 时是 200 但仅 121 字节的空壳，补上这三个头变 7074 字节。
（我先前猜的 sec-ch-ua 不是关键。）
· 因此 UA 与 client hints 必须**成套地取自同一个浏览器**，所以 BrowserIdentity 一次
从 CDP 取齐 cookie + UA + hints，而不是各自写死。
「200 + 空 body 必须当场报错」也是刻意写死的：放过去它会在下游变成「这个博主没作品」，
把一次失败伪装成一条正常结果 —— 爬虫那条路正是这么栽的，还被翻译成「账号被封」。
测试 +11：cookie 解析、请求头成套性（含 uifid 缺失/回退）、产物键名与 store 对齐、
以及 _get 的三条失败路径（空 body / 403 带网关原话 / 正常返回）。
This commit is contained in:
@@ -0,0 +1,480 @@
|
||||
# AVE 视频工厂 — 全链路架构规划 v1.0
|
||||
|
||||
> 日期: 2026-05-18 | 版本: 1.0
|
||||
> 定位: 梳理所有生产路径 + 底层能力原子化 + 自审可行性
|
||||
|
||||
---
|
||||
|
||||
## 目录
|
||||
|
||||
1. [核心洞察](#1-核心洞察)
|
||||
2. [AVE 六条生产路径总览](#2-ave-六条生产路径总览)
|
||||
3. [底层能力层（原子化）](#3-底层能力层原子化)
|
||||
4. [各生产路径的完整链路](#4-各生产路径的完整链路)
|
||||
5. [新增：模仿视频线（Action Transfer）](#5-新增模仿视频线action-transfer)
|
||||
6. [交叉能力矩阵](#6-交叉能力矩阵)
|
||||
7. [优先级与落地路线图](#7-优先级与落地路线图)
|
||||
8. [自审与可行性分析](#8-自审与可行性分析)
|
||||
9. [具体落地建议](#9-具体落地建议)
|
||||
|
||||
---
|
||||
|
||||
## 1. 核心洞察
|
||||
|
||||
### 1.1 用户提到的三条线本质是同一逻辑
|
||||
|
||||
| 用户说的 | 本质 | 核心操作 |
|
||||
|:---------|:-----|:---------|
|
||||
| 卡点视频 | 音频节拍 → 视觉同步切换 | BPM分析 + 拍点对齐 + 素材切换 |
|
||||
| 模仿视频 | 参考视频 → 提取节奏/动作 → 复用结构 | 参考视频分析 + 动作迁移 + 结构复用 |
|
||||
| 主人公更换 | 已有视频 → 替换角色 → 保持动作/场景 | 角色识别 + 角色替换 + 动作保持 |
|
||||
|
||||
**三者共享同一条底层流水线**: `参考源分析 → 结构提取 → 角色/素材替换 → 重新合成`
|
||||
|
||||
### 1.2 AVE 当前状态
|
||||
|
||||
已实现 5 条独立生产路径，但存在三个结构性问题：
|
||||
|
||||
| 问题 | 后果 |
|
||||
|:----|:-----|
|
||||
| 各策略独立实现，底层能力重复 | 口播的素材搜索和卡点的素材搜索各写一遍 |
|
||||
| 角色系统只在"故事"策略中使用 | 口播、卡点、数字人都需要角色但没接入 |
|
||||
| AudioLine 刚刚建成但未接入主流程 | 音频处理能力独立于视频工厂 |
|
||||
|
||||
### 1.3 解决思路
|
||||
|
||||
**分层架构**: 底层能力层(原子操作) → 策略编排层(生产路径) → 接入层(CLI/Dashboard)
|
||||
|
||||
---
|
||||
|
||||
## 2. AVE 六条生产路径总览
|
||||
|
||||
```
|
||||
AVE 视频工厂
|
||||
│
|
||||
├─ 1. 口播线 文案→TTS→素材→BGM→字幕→合成
|
||||
├─ 2. 卡点线 BGM→节拍检测→素材→拍点切换→xfade
|
||||
├─ 3. 数字人线 照片/视频→OmniHuman对口型/DreamActor动作模仿
|
||||
├─ 4. 口播+卡点线 人声锚点+BGM变速→帧锁定拼接 (hybrid)
|
||||
├─ 5. 故事/漫剧线 剧本→场景分解→Kling批量→角色拼接
|
||||
├─ 6. 模仿视频线 ★ 参考视频→结构分析→动作迁移/角色替换→合成
|
||||
│
|
||||
└─ 7. 音频制造线 (独立)
|
||||
├─ 原创 AudioLine Layer3 创作模块
|
||||
├─ 二创 分层改编 (节奏/歌词/旋律/混音)
|
||||
├─ 卡点改编 节拍重塑 + BPM对齐
|
||||
└─ → 接入上述 1-6 条线输出成品
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. 底层能力层（原子化）
|
||||
|
||||
> 所有生产路径共享的原子能力。每个能力可独立测试、独立优化、被多种策略调用。
|
||||
|
||||
### 3.1 音频原子能力 ← AudioLine
|
||||
|
||||
| 原子能力 | 输入 | 输出 | 被哪些路径使用 |
|
||||
|:---------|:-----|:-----|:-------------|
|
||||
| A1. 音轨分离 | 音频文件 | {vocals, drums, bass, other} 分轨 | 2-卡点, 4-hybrid, 6-模仿, 7-音频 |
|
||||
| A2. BPM/节拍检测 | 音频文件 | BPM + 节拍时间戳 + 重拍 | 2-卡点, 4-hybrid, 6-模仿 |
|
||||
| A3. 歌曲结构解析 | 音频文件 | [{label, start, end}] 段落 | 6-模仿, 7-二创 |
|
||||
| A4. 情绪分析 | 音频文件 | mood + energy + danceability | 1-口播(BGM选曲), 5-故事(风格匹配) |
|
||||
| A5. 节奏生成 | BPM+风格参数 | 鼓点/节奏音频 | 2-卡点, 7-原创 |
|
||||
| A6. 歌词创作 | 原歌词 + 主题 | 改写歌词 | 7-二创 |
|
||||
| A7. 旋律改编 | 音频 + 风格 | 改编旋律 | 7-二创 |
|
||||
| A8. 混音母带 | 多轨音频 | 混合主轨 + 响度归一 | 1-口播, 2-卡点, 7-原创/二创 |
|
||||
| A9. 分轨时间对齐 | 分轨 + 目标BPM | 对齐后分轨 | 2-卡点, 6-模仿 |
|
||||
| A10. 歌词-节拍对齐 | 歌词 + 节拍 | 时间轴 + LRC/SRT | 1-口播, 2-卡点 |
|
||||
| A11. 多格式导出 | 音频 + 格式 | WAV/MP3/FLAC/AAC | 全部 |
|
||||
|
||||
### 3.2 视觉原子能力
|
||||
|
||||
| 原子能力 | 输入 | 输出 | 被哪些路径使用 |
|
||||
|:---------|:-----|:-----|:-------------|
|
||||
| V1. 素材搜索 | 关键词 + 数量 | 视频素材列表(URL/本地) | 1-口播, 2-卡点, 4-hybrid |
|
||||
| V2. Ken Burns 效果 | 静态图片 | 缓慢缩放/平移视频 | 1-口播, 5-故事 |
|
||||
| V3. 节拍视觉对齐 | 素材 + 节拍 | 按拍点切割/过渡的视频段 | 2-卡点, 4-hybrid, 6-模仿 |
|
||||
| V4. 变速卡点 | 素材 + 节拍能量 | 高潮加速/低谷减速 | 4-hybrid |
|
||||
| V5. 视频拼接 | 视频片段列表 | 完整视频 (xfade/crossfade) | 全部 |
|
||||
| V6. 字幕叠加 | 时间轴 + 文本 | ASS字幕文件 + 视频叠加 | 1-口播, 2-卡点, 5-故事 |
|
||||
| V7. 定妆照生成 | 角色描述 | 角色多角度参考图 (Grid Method) | 5-故事, 6-模仿 ★ |
|
||||
| V8. 图生视频 | 图片 + Prompt | 短视频片段 (Kling) | 5-故事, 6-模仿 ★ |
|
||||
| V9. 首尾帧过渡 | 首帧 + 尾帧 + Prompt | 平滑过渡视频 | 5-故事 |
|
||||
| V10. 语音驱动视频 | 图片 + 音频 | 对口型视频 (LipSync) | 1-口播, 3-数字人 |
|
||||
|
||||
### 3.3 动作迁移原子能力 ★ 新增
|
||||
|
||||
> 源自 Doubao 对话的 Animate 模型思路，也可使用现有 DreamActor
|
||||
|
||||
| 原子能力 | 输入 | 输出 | 被哪些路径使用 |
|
||||
|:---------|:-----|:-----|:-------------|
|
||||
| M1. 参考视频结构分析 | 视频文件 | 分段结构 + 镜头节奏 + 动作时间轴 | 6-模仿 |
|
||||
| M2. 角色识别/分离 | 视频 + 角色标识 | 角色掩码/区域 | 6-模仿 |
|
||||
| M3. 动作迁移 (2D/3D) | 参考视频 + 目标角色 | 目标角色复刻动作的视频 | 3-数字人(DreamActor), 6-模仿 |
|
||||
| M4. 动作骨架提取 | 视频 | 2D/3D 骨骼关键帧序列 | 6-模仿 |
|
||||
| M5. 背景替换/保留 | 视频 + 掩码 | 去除/保留背景后的视频 | 6-模仿 |
|
||||
| M6. 视频结构模板化 | 视频 → 分镜模板 | 可复用的 {时间轴 + 切换点 + 动作标记} | 6-模仿 |
|
||||
|
||||
### 3.4 角色原子能力（跨线复用）
|
||||
|
||||
| 原子能力 | 输入 | 输出 | 被哪些路径使用 |
|
||||
|:---------|:-----|:-----|:-------------|
|
||||
| C1. 角色注册 | 角色描述 → registry | 角色 ID + 属性 + 参考图 | 全部 |
|
||||
| C2. 角色锁定 | 剧本 + 角色 ID | 注入角色描述块 | 5-故事, 6-模仿 |
|
||||
| C3. 角色一致性保持 | 角色参考图 + 场景 | 固定 seed + 定妆照 → 多场景生成 | 5-故事, 6-模仿 |
|
||||
| C4. 角色替换 | 原视频 + 新角色 | 新角色 + 保留原动作/表情 | 6-模仿 |
|
||||
|
||||
### 3.5 通用基础能力
|
||||
|
||||
| 原子能力 | 说明 |
|
||||
|:---------|:-----|
|
||||
| G1. 配置加载 | config.yaml + local.yaml 合并 |
|
||||
| G2. 成本追踪 | 每步操作计费 + API 调用计数 |
|
||||
| G3. 进度报告 | 状态机 + Dashboard 埋点 |
|
||||
| G4. 缓存管理 | 素材/模型/results 缓存 |
|
||||
| G5. 错误容错 | 模块失败不阻塞整条链路 |
|
||||
|
||||
---
|
||||
|
||||
## 4. 各生产路径的完整链路
|
||||
|
||||
> 用 `[A1]` 标记引用底层原子能力；`[★]` 标记待完善/新增
|
||||
|
||||
### 4.1 口播线 (Oral)
|
||||
|
||||
```
|
||||
当前: 文案 → [TTS] → [素材搜索] → [BGM] → [字幕] → [合成]
|
||||
↓
|
||||
优化后: 文案 → [TTS] → [素材搜索+V2 Ken Burns] → [AudioLine A4情绪分析选BGM]
|
||||
→ [混音A8 人声+BGM] → [A10歌词对齐→字幕] → [V10 LipSync] → [合成]
|
||||
```
|
||||
|
||||
**待完善**:
|
||||
- [ ] 接入 AudioLine A4 情绪分析 → 自动匹配 BGM 风格
|
||||
- [ ] 接入 AudioLine A8 混音 → 专业级人声+BGM 电平平衡
|
||||
- [ ] V2 Ken Burns 效果 (已有 Pexels 素材多是静态风格需要)
|
||||
|
||||
### 4.2 卡点线 (Beat)
|
||||
|
||||
```
|
||||
当前: BGM → [节拍检测] → [素材搜索] → [拍点切换] → [xfade]
|
||||
↓
|
||||
优化后: BGM → [AudioLine A2节拍检测] → [A5节奏生成(替换/增强鼓点)]
|
||||
→ [V3节拍视觉对齐] → [V4变速卡点]
|
||||
→ [A9分轨对齐(对齐到目标BPM)] → [合成]
|
||||
```
|
||||
|
||||
**待完善**:
|
||||
- [ ] 接入 AudioLine A5 节奏生成 → 在原BGM上叠加鼓点增强
|
||||
- [ ] 接入 AudioLine A9 分轨对齐 → BPM 变更时各轨统一伸缩
|
||||
- [ ] V3 节拍视觉对齐优化 → 当前只是简单切，需要更精确的拍点类型映射
|
||||
|
||||
### 4.3 数字人线 (Digital Human)
|
||||
|
||||
```
|
||||
当前: 照片/视频 → [OmniHuman 对口型] / [DreamActor 动作模仿]
|
||||
↓
|
||||
优化后: 角色[C1注册] → 生成[C7定妆照] → [TTS] → [V10 LipSync]
|
||||
→ 可选 [M3动作迁移] → [合成+背景]
|
||||
```
|
||||
|
||||
**待完善**:
|
||||
- [ ] 接入 C1 角色注册 → 数字人可被角色库管理
|
||||
- [ ] 接入 Asset Manager → 素材复用
|
||||
- [ ] DreamActor 轮询等待优化 → 当前 60 次循环硬编码
|
||||
|
||||
### 4.4 口播+卡点线 (Hybrid)
|
||||
|
||||
```
|
||||
当前: 人声 + BGM → [变速卡点] → [帧锁定拼接]
|
||||
↓
|
||||
优化后: 人声 → [A1音轨分离] → 人声轨道锚点分析
|
||||
+ BGM → [A2节拍检测] → BGM能量曲线
|
||||
→ [V4变速卡点(人声锚点+BGM能量双驱动)]
|
||||
→ [A8混音(人声+BGM电平平衡)]
|
||||
→ [V3节拍对齐素材] → [合成]
|
||||
```
|
||||
|
||||
**待完善**:
|
||||
- [ ] 双驱动变速逻辑完善 (人声锚点优先级 > BGM 能量)
|
||||
- [ ] 接入 AudioLine A8 混音
|
||||
- [ ] 当前 `compose_hybrid` 需要和 beat_sync 统一底层
|
||||
|
||||
### 4.5 故事/漫剧线 (Story)
|
||||
|
||||
```
|
||||
当前: 剧本 → [场景分解] → [桥接] → [Kling批量生成] → [角色拼接]
|
||||
↓
|
||||
优化后: 剧本 → [C2角色锁定] → [C7定妆照生成]
|
||||
→ [场景分解(Multi-agent)] → [C3角色一致性保持]
|
||||
→ [V8图生视频 + V9首尾帧过渡]
|
||||
→ [AudioLine A6歌词创作 + A10对齐] # 如有旁白/主题曲
|
||||
→ [V6字幕叠加] → [合成]
|
||||
```
|
||||
|
||||
**待完善**:
|
||||
- [ ] 角色锁定已实现 (character_locker.py) 但需要和 AudioLine 歌词输出配合
|
||||
- [ ] 定妆照 → 多场景的角色一致性已部分实现，需扩展到模仿视频线
|
||||
|
||||
### 5. 新增：模仿视频线 (Action Transfer)
|
||||
|
||||
> 这是本方案的核心新增。整合了"卡点模仿" + "主人公更换"两个场景。
|
||||
|
||||
```
|
||||
[参考视频] → Layer1 分析层 → Layer2 动作迁移 → Layer3 替换合成
|
||||
```
|
||||
|
||||
#### 5.1 完整链路
|
||||
|
||||
```
|
||||
输入: 参考视频 + (可选: 目标BGM / 目标角色 / 目标风格)
|
||||
|
||||
Step 1 — 参考视频分析 (M1 + A2 + A3)
|
||||
├─ 视频结构分解: 镜头边界检测 → 分段标记
|
||||
├─ 音频分析: [A2 BPM检测] + [A3结构解析] + [A4情绪分析]
|
||||
├─ 视觉分析: [M4骨架提取] → 动作关键帧序列
|
||||
└─ 节奏模板: 镜头切换时间轴 + 动作标记 + 节拍映射
|
||||
|
||||
Step 2 — 素材/角色准备
|
||||
├─ 使用原视频素材 → 只替换角色 [M5背景保留 + C4角色替换]
|
||||
├─ 使用新素材 → [V1素材搜索] + [V3节拍对齐]
|
||||
└─ 使用原音频 → [A9分轨对齐到目标BPM]
|
||||
|
||||
Step 3 — 动作迁移 (M3)
|
||||
├─ 路径A: DreamActor (火山引擎) → 高质量真人动作迁移
|
||||
├─ 路径B: Animate模型 (ComfyUI+阿里) → 2D卡通动作迁移
|
||||
└─ 路径C: Pose2Pose (轻量开源) → 简单动作映射
|
||||
|
||||
Step 4 — 合成输出
|
||||
├─ 角色一致性保持 [C3]
|
||||
├─ 替换音频: [A8混音] 新BGM/语音 + 原视频动作节奏
|
||||
├─ [V5视频拼接] + [V6字幕叠加]
|
||||
└─ [G3进度报告 + G2成本追踪]
|
||||
```
|
||||
|
||||
#### 5.2 三个子场景映射
|
||||
|
||||
| 用户场景 | Step1 输入 | Step2 替换 | Step3 迁移方式 | Step4 输出 |
|
||||
|:---------|:-----------|:-----------|:---------------|:-----------|
|
||||
| **卡点模仿** | 参考视频 | 换BGM + 换素材 | 保留原动作节奏 | 新BGM+新素材的卡点视频 |
|
||||
| **主人公更换** | 有角色视频 | 换角色 + 保留动作 | DreamActor/Animate | 同场景新角色 |
|
||||
| **风格重制** | 参考视频 | 换BGM + 换角色 + 换风格 | 全链路 | 全新风格视频 |
|
||||
|
||||
#### 5.3 与现有能力的复用关系
|
||||
|
||||
```
|
||||
模仿视频线 ← 已有 → 复用成熟度
|
||||
├─ A2 BPM检测 → AudioLine ✅ 已实现
|
||||
├─ A3结构解析 → AudioLine ✅ 已实现
|
||||
├─ A9分轨对齐 → AudioLine ✅ 已实现
|
||||
├─ A8混音 → AudioLine ✅ 已实现
|
||||
├─ V1素材搜索 → pexels/search ✅ 已实现
|
||||
├─ V3节拍视觉对齐 → beat_sync.py ✅ 已实现
|
||||
├─ V5视频拼接 → ffmpeg.py ✅ 已实现
|
||||
├─ V6字幕叠加 → ffmpeg.py ✅ 已实现
|
||||
├─ C3角色一致性 → character_registry ✅ 部分实现
|
||||
├─ M3动作迁移 → DreamActor ✅ 已有(DreamActor)
|
||||
│ → Animate模型 ⚠️ 需接入(ComfyUI)
|
||||
├─ M1参考视频分析 → ❌ 需新建
|
||||
├─ M4骨架提取 → ❌ 需新建 (或集成 OpenPose)
|
||||
├─ M5背景替换 → ❌ 需新建
|
||||
└─ C4角色替换 → ❌ 需新建 (组合 M3 + M5 + C3)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. 交叉能力矩阵
|
||||
|
||||
> 每个 ⬜ = 可用，⬛ = 核心依赖，↔ = 可选接入
|
||||
|
||||
| 原子能力 | 口播 | 卡点 | 数字人 | Hybrid | 故事 | 模仿 | 音频制造 |
|
||||
|:---------|:----:|:----:|:------:|:------:|:----:|:----:|:--------:|
|
||||
| **音频原子 (A1-A11)** | | | | | | | |
|
||||
| A1 音轨分离 | ↔ | ⬛ | | ⬛ | | ⬛ | ⬛ |
|
||||
| A2 BPM检测 | | ⬛ | | ⬛ | | ⬛ | ⬛ |
|
||||
| A3 结构解析 | | ↔ | | | | ⬛ | ⬛ |
|
||||
| A4 情绪分析 | ⬛ | ↔ | | ↔ | ⬛ | ⬛ | ⬛ |
|
||||
| A5 节奏生成 | | ⬛ | | ↔ | | ↔ | ⬛ |
|
||||
| A6 歌词创作 | ↔ | | | | ⬛ | | ⬛ |
|
||||
| A7 旋律改编 | | | | | | | ⬛ |
|
||||
| A8 混音母带 | ⬛ | ⬛ | ↔ | ⬛ | | ↔ | ⬛ |
|
||||
| A9 分轨对齐 | | ⬛ | | ↔ | | ⬛ | ⬛ |
|
||||
| A10 歌词对齐 | ⬛ | ↔ | | | ⬛ | | ↔ |
|
||||
| A11 多格式导出 | ↔ | ↔ | ↔ | ↔ | ↔ | ↔ | ⬛ |
|
||||
| **视觉原子 (V1-V10)** | | | | | | | |
|
||||
| V1 素材搜索 | ⬛ | ⬛ | | ⬛ | ↔ | ⬛ | |
|
||||
| V2 Ken Burns | ⬛ | | | | ⬛ | | |
|
||||
| V3 节拍对齐 | | ⬛ | | ⬛ | | ⬛ | |
|
||||
| V4 变速卡点 | | ⬛ | | ⬛ | | ↔ | |
|
||||
| V5 视频拼接 | ⬛ | ⬛ | ⬛ | ⬛ | ⬛ | ⬛ | |
|
||||
| V6 字幕叠加 | ⬛ | ↔ | | ↔ | ⬛ | ↔ | |
|
||||
| V7 定妆照 | | | ↔ | | ⬛ | ⬛ | |
|
||||
| V8 图生视频 | | | | | ⬛ | ↔ | |
|
||||
| V9 首尾帧 | | | | | ⬛ | | |
|
||||
| V10 LipSync | ⬛ | | ⬛ | ↔ | ⬛ | ↔ | |
|
||||
| **动作原子 (M1-M6) ★** | | | | | | | |
|
||||
| M1 视频结构分析 | | | | | | ⬛ | |
|
||||
| M2 角色识别 | | | ↔ | | | ⬛ | |
|
||||
| M3 动作迁移 | | | ⬛ | | | ⬛ | |
|
||||
| M4 骨架提取 | | | | | | ⬛ | |
|
||||
| M5 背景替换 | | | | | | ⬛ | |
|
||||
| M6 模板化 | | | | | | ⬛ | |
|
||||
| **角色原子 (C1-C4)** | | | | | | | |
|
||||
| C1 角色注册 | ↔ | | ⬛ | | ⬛ | ⬛ | |
|
||||
| C2 角色锁定 | | | | | ⬛ | ⬛ | |
|
||||
| C3 一致性保持 | | | | | ⬛ | ⬛ | |
|
||||
| C4 角色替换 | | | | | | ⬛ | |
|
||||
|
||||
---
|
||||
|
||||
## 7. 优先级与落地路线图
|
||||
|
||||
### Phase 1: 接入现有能力 (1-2天) ← 立即开始
|
||||
|
||||
| # | 任务 | 工作量 | 价值 |
|
||||
|:-:|:-----|:------:|:----:|
|
||||
| 1 | AudioLine 接入 AVE 主流程 (video_factory.py 调 AudioLine.run) | 2h | ⭐⭐⭐ |
|
||||
| 2 | 口播线接入 AudioLine A4 情绪分析 → 自动选BGM | 1h | ⭐⭐ |
|
||||
| 3 | 卡点线接入 AudioLine A9 分轨对齐 → 支持BPM变化 | 1h | ⭐⭐⭐ |
|
||||
| 4 | 口播线接入 AudioLine A8 混音替代简单叠加 | 1h | ⭐⭐ |
|
||||
| 5 | 角色注册接入数字人线 | 1h | ⭐ |
|
||||
|
||||
### Phase 2: 模仿视频线 MVP (3-5天)
|
||||
|
||||
| # | 任务 | 工作量 | 依赖 |
|
||||
|:-:|:-----|:------:|:----:|
|
||||
| 6 | M1 参考视频结构分析 (ffmpeg 场景检测 + A2节拍) | 4h | A2, V5 |
|
||||
| 7 | M4 骨架提取集成 (OpenPose 轻量版) | 6h | - |
|
||||
| 8 | M3 动作迁移路径C (Pose2Pose 轻量) | 4h | M4 |
|
||||
| 9 | 模仿视频线编排器 (复用 orchestrator 模式) | 4h | M1, M3, A2 |
|
||||
| 10 | CLI 接入: `video-factory --strategy 模仿` | 2h | #9 |
|
||||
|
||||
### Phase 3: 角色替换 + 高级动作迁移 (5-7天)
|
||||
|
||||
| # | 任务 | 工作量 | 依赖 |
|
||||
|:-:|:-----|:------:|:----:|
|
||||
| 11 | M5 背景替换 (RMBG/rembg) | 3h | - |
|
||||
| 12 | C4 角色替换组合 (M3+M5+C3) | 4h | M3, M5 |
|
||||
| 13 | M6 视频结构模板化 (保存/加载模板) | 3h | M1 |
|
||||
| 14 | M3 路径A DreamActor 优化 (异步轮询改为 Webhook) | 2h | M3 |
|
||||
| 15 | M3 路径B Animate 模型 ComfyUI 接入 | 8h | - |
|
||||
|
||||
### Phase 4: 优化 + 自动化 (持续)
|
||||
|
||||
| # | 任务 | 说明 |
|
||||
|:-:|:-----|:------|
|
||||
| 16 | 原子操作 CLI 化 → 每个原子能力可独立调用 | |
|
||||
| 17 | Dashboard 接入模仿线 + 动作线数据 | |
|
||||
| 18 | 素材缓存 + 结果缓存 (避免重复 API 调用) | |
|
||||
| 19 | 批量处理模式 (一次性处理多个参考视频) | |
|
||||
|
||||
---
|
||||
|
||||
## 8. 自审与可行性分析
|
||||
|
||||
### 8.1 架构合理性审查
|
||||
|
||||
| 审查维度 | 评分 | 说明 |
|
||||
|:---------|:----:|:-----|
|
||||
| 模块内聚性 | ⭐⭐⭐⭐ | 每条生产路径独立，底层能力原子化 |
|
||||
| 耦合度控制 | ⭐⭐⭐⭐ | 上层通过 orchestrator 调用下层，不跨层调用 |
|
||||
| 可扩展性 | ⭐⭐⭐⭐⭐ | 新增生产路径 = 新编排器 + 复用底层原子 |
|
||||
| 测试可行性 | ⭐⭐⭐ | 原子能力可单独测试，但端到端链路依赖API |
|
||||
| 容错能力 | ⭐⭐⭐ | 模块级容错已实现，但动作迁移线无回退方案 |
|
||||
|
||||
### 8.2 具体落地问题
|
||||
|
||||
#### 问题1: DreamActor 视频轮询耗时太长
|
||||
- **现状**: 60×10s = 10 分钟轮询，阻塞整条链路
|
||||
- **建议**: 改为异步任务，提交流程后立即返回 task_id，后台 Dashboard 显示进度
|
||||
|
||||
#### 问题2: Animate 模型依赖 ComfyUI
|
||||
- **现状**: ComfyUI 非本地部署，需额外启动
|
||||
- **建议**: Phase 2 先用 DreamActor 跑通 MVP，ComfyUI 接入作为 Phase 3 优化项
|
||||
|
||||
#### 问题3: 骨架提取 (M4) 精度
|
||||
- **现状**: OpenPose 需要 GPU，轻量版(MediaPipe Pose)精度有限
|
||||
- **建议**:
|
||||
- 路径A: 用 MediaPipe (免费，精准度70%)
|
||||
- 路径B: 用阿里 Animate 内置骨骼提取 (精准度90%，需 ComfyUI)
|
||||
- 路径C: 跳过骨架提取，直接用原始视频做帧匹配 (最简单)
|
||||
|
||||
#### 问题4: AudioLine 依赖 oMLX + Suno + Mubert
|
||||
- **现状**: oMLX 需手动启动，Suno/Mubert 需 API key
|
||||
- **建议**:
|
||||
- 无 API key 时模块自动回退到本地方案 (已实现)
|
||||
- oMLX 启动自动化加入 launchd 或 Start Server 脚本
|
||||
|
||||
#### 问题5: 角色替换的"合理性"问题
|
||||
- **现状**: 替换角色后，原视频中的人脸/手部动作可能与新角色不匹配
|
||||
- **建议**:
|
||||
- 2D 卡通角色替换比真人替换更容易获得合理结果
|
||||
- 真人替换建议用 DreamActor（原生支持面部替换）
|
||||
- 首次测试建议用卡通角色 + 卡点场景，不要一步上真人
|
||||
|
||||
### 8.3 是否有更优解？
|
||||
|
||||
| 现有方案 | 更优解 | 评估 |
|
||||
|:---------|:-------|:-----|
|
||||
| 每条线独立编排器 | **统一编排器 + 插件模式** | 更优但重构成本高，建议 Phase 4 |
|
||||
| DreamActor 轮询 | **Webhook 回调** | 更优，但需服务端支持，暂不可行 |
|
||||
| OpenPose 本地骨架 | **阿里 Animate 内置骨骼** | 更优，但需接入 ComfyUI |
|
||||
| 手动 API Key 配置 | **环境变量 + .env 文件** | 更优，建议 Phase 1 顺便完成 |
|
||||
| stand-alone AudioLine | **AudioLine as AVE plugin** | 已在设计范围内 |
|
||||
|
||||
### 8.4 风险矩阵
|
||||
|
||||
| 风险 | 概率 | 影响 | 缓解措施 |
|
||||
|:----|:----:|:----:|:---------|
|
||||
| API 限频/涨价 | 中 | 高 | 本地回退 + 成本预警 |
|
||||
| 动作迁移质量不达标 | 中 | 高 | Phase 2 MVP 先用简单场景验证 |
|
||||
| ComfyUI 部署复杂 | 高 | 中 | 先用 DreamActor，ComfyUI 后接 |
|
||||
| 角色替换效果假 | 中 | 中 | 用卡通角色测试，真人暂缓 |
|
||||
| AudioLine 依赖未启动 | 低 | 中 | oMLX 自动启动 + 模块自检告警 |
|
||||
|
||||
---
|
||||
|
||||
## 9. 具体落地建议
|
||||
|
||||
### 9.1 立即执行 (今天/明天)
|
||||
|
||||
```bash
|
||||
# Step 1: AudioLine 接入 AVE 主流程
|
||||
# 在 video_factory.py 中新增:
|
||||
# from audio_line import AudioLine
|
||||
# al = AudioLine()
|
||||
#
|
||||
# # 口播线: 用 AudioLine 情绪分析选BGM
|
||||
# emotion = al.emotion_analyzer.analyze(voice_path)
|
||||
# bgm_style = emotion_to_bgm(emotion.mood)
|
||||
#
|
||||
# # 卡点线: 用 AudioLine BPM分析 + 分轨对齐
|
||||
# beat = al.beat_detector.detect(bgm_path)
|
||||
# aligned_stems = al.stem_aligner.align_to_bpm(...)
|
||||
```
|
||||
|
||||
### 9.2 本周内
|
||||
|
||||
```
|
||||
1. 模仿视频线 MVP:
|
||||
- 输入: 参考视频
|
||||
- 输出: 提取的结构模板 + 骨架序列
|
||||
- 不生成最终视频，只验证分析层
|
||||
|
||||
2. 角色系统统一:
|
||||
- 数字人注册到 CharacterRegistry
|
||||
- 口播脚本支持引用角色
|
||||
```
|
||||
|
||||
### 9.3 需要注意的坑
|
||||
|
||||
1. **不要一步到位**：先跑通 MVP 再迭代优化
|
||||
2. **每个原子能力先单测再集成**：特别是 M1-M6 动作迁移线
|
||||
3. **AudioLine 已经写好，但 demo/ 里还没跑的端到端测试**：先跑通一个卡点场景的端到端
|
||||
4. **模仿视频线的骨架提取不要依赖 GPU**：用 MediaPipe 或完全跳过骨架，直接用帧匹配
|
||||
|
||||
---
|
||||
|
||||
> 本方案已包含自审环节 (Section 8)，所有建议均有明确的落地路径。
|
||||
> 下一步建议: 确认方案方向 → Phase 1 将 AudioLine 接入 AVE → Phase 2 开始模仿视频线 MVP
|
||||
@@ -0,0 +1,69 @@
|
||||
"""
|
||||
批量生成 8 张漫剧分镜图 — WorkBuddy 云服务 (混元生图 3.0)
|
||||
"""
|
||||
import sys, json, os, subprocess, time
|
||||
|
||||
OUT_DIR = "/Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/generated/PROD-20260517-0003"
|
||||
os.makedirs(OUT_DIR, exist_ok=True)
|
||||
|
||||
CLOUD_SCRIPT = "/Applications/WorkBuddy.app/Contents/Resources/app.asar.unpacked/resources/builtin-skills/buddy-multimodal-generation/scripts/buddy-cloud.py"
|
||||
PYTHON = "/Users/7kecheng/.workbuddy/binaries/python/envs/agent-os/bin/python"
|
||||
|
||||
# 从 visual_prompts JSON 加载
|
||||
with open("/Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/PROD-20260517-0003_visual_prompts.json") as f:
|
||||
prompts = json.load(f)
|
||||
|
||||
# 读取 token
|
||||
token = open("/dev/tty", "r") if False else "" # placeholder — will pass via pipe
|
||||
|
||||
print(f"将生成 {len(prompts)} 张分镜图")
|
||||
print()
|
||||
|
||||
results = []
|
||||
for i, vp in enumerate(prompts):
|
||||
seg_id = vp["id"]
|
||||
scene = vp["scene"]
|
||||
prompt = vp["visual_prompt"]
|
||||
|
||||
# 清理 prompt — 去掉架构标记，转为纯文本描述
|
||||
clean_prompt = (
|
||||
f"日系清新漫画风格。{scene}。"
|
||||
f"主角阿远，黑短发戴细框眼镜，"
|
||||
f"情绪：{vp['emotion']}。"
|
||||
f"镜头：{vp.get('reference_image', '中景')}"
|
||||
)
|
||||
# 使用更精确的 prompt
|
||||
final_prompt = (
|
||||
f"日系清新漫画风格，{scene}。"
|
||||
f"一个25岁黑短发戴细框金属眼镜的偏瘦男生阿远，"
|
||||
f"表情{vp['emotion']}。"
|
||||
f"柔和色调，线条干净，温暖治愈氛围。"
|
||||
)
|
||||
|
||||
out_path = f"{OUT_DIR}/seg_{seg_id:02d}_{scene}.png"
|
||||
print(f"[{i+1}/{len(prompts)}] #{seg_id} {scene} — 生成中...", end=" ", flush=True)
|
||||
|
||||
start = time.time()
|
||||
# 使用 --no-poll 先提交，后续再检查状态
|
||||
# 先试试直接同步生成
|
||||
result = subprocess.run(
|
||||
[PYTHON, CLOUD_SCRIPT, "image", final_prompt,
|
||||
"--resolution", "1248:832",
|
||||
"--revise", "1",
|
||||
"--token-stdin"],
|
||||
input=token,
|
||||
capture_output=True, text=True, timeout=300
|
||||
)
|
||||
elapsed = time.time() - start
|
||||
|
||||
if result.returncode == 0:
|
||||
# 从输出中提取图片 URL
|
||||
output = result.stdout.strip()
|
||||
print(f"✅ ({elapsed:.0f}s)")
|
||||
# 保存输出信息
|
||||
results.append({"id": seg_id, "scene": scene, "output": output, "path": out_path})
|
||||
else:
|
||||
print(f"❌ 失败: {result.stderr.strip()[:100]}")
|
||||
results.append({"id": seg_id, "scene": scene, "error": result.stderr.strip()})
|
||||
|
||||
print(f"\n完成: {len([r for r in results if 'output' in r])}/{len(prompts)} 张")
|
||||
@@ -0,0 +1,155 @@
|
||||
"""
|
||||
|
||||
AVE 04_anchor_extractor — 双轨锚点提取
|
||||
|
||||
职责:
|
||||
1. 人声锚点: 检测人声活跃段(word-level timestamps)
|
||||
2. BGM 锚点: 检测节奏/情绪变化点(section boundaries)
|
||||
3. 输出 anchors.json: 供 composer 做画面切换时间参考
|
||||
|
||||
依赖:
|
||||
- librosa (本地 CPU 即可)
|
||||
- numpy
|
||||
"""
|
||||
import sys, os; sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
|
||||
from lib.logger import get_logger
|
||||
|
||||
logger = get_logger("anchor_extractor")
|
||||
|
||||
|
||||
def extract_anchors(
|
||||
voice_path: str,
|
||||
bgm_path: str | None = None,
|
||||
output_path: str = "anchors.json",
|
||||
) -> dict[str, Any]:
|
||||
"""
|
||||
双轨锚点提取: 人声 + BGM → anchors.json
|
||||
|
||||
- voice_path: 人声 WAV 路径
|
||||
- bgm_path: BGM 路径 (可选)
|
||||
- 返回: anchors dict
|
||||
|
||||
anchors 格式:
|
||||
{
|
||||
"voice_anchors": [
|
||||
{"time": 1.2, "type": "word_start", "word": "今天"},
|
||||
{"time": 3.5, "type": "word_end", "word": "今天"},
|
||||
{"time": 5.0, "type": "silence", "duration": 0.3},
|
||||
],
|
||||
"bgm_anchors": [
|
||||
{"time": 0.0, "section": "intro"},
|
||||
{"time": 8.0, "section": "main"},
|
||||
{"time": 50.0, "section": "climax"},
|
||||
],
|
||||
"metadata": {
|
||||
"voice_duration_sec": 75.0,
|
||||
"bgm_duration_sec": 75.0,
|
||||
"sample_rate": 24000
|
||||
}
|
||||
}
|
||||
"""
|
||||
import librosa
|
||||
|
||||
anchors: dict[str, Any] = {
|
||||
"voice_anchors": [],
|
||||
"bgm_anchors": [],
|
||||
"metadata": {},
|
||||
}
|
||||
|
||||
# ── 1. 人声锚点 ──
|
||||
if not os.path.exists(voice_path):
|
||||
raise FileNotFoundError(f"人声文件不存在: {voice_path}")
|
||||
|
||||
y, sr = librosa.load(voice_path, sr=24000, mono=True)
|
||||
duration = librosa.get_duration(y=y, sr=sr)
|
||||
anchors["metadata"]["voice_duration_sec"] = duration
|
||||
anchors["metadata"]["sample_rate"] = sr
|
||||
|
||||
# 语音活动检测: 能量阈值法
|
||||
frame_length = int(sr * 0.025) # 25ms 帧
|
||||
hop_length = int(sr * 0.010) # 10ms 步进
|
||||
energy = librosa.feature.rms(y=y, frame_length=frame_length, hop_length=hop_length)[0]
|
||||
threshold = np.mean(energy) * 0.3
|
||||
|
||||
# 检测活跃段
|
||||
is_active = energy > threshold
|
||||
voice_anchors = []
|
||||
current_start = None
|
||||
|
||||
for i, active in enumerate(is_active):
|
||||
time_sec = i * hop_length / sr
|
||||
if active and current_start is None:
|
||||
current_start = time_sec
|
||||
voice_anchors.append({"time": round(time_sec, 2), "type": "voice_start"})
|
||||
elif not active and current_start is not None:
|
||||
voice_anchors.append({"time": round(time_sec, 2), "type": "voice_end"})
|
||||
current_start = None
|
||||
|
||||
if current_start is not None:
|
||||
voice_anchors.append({"time": round(duration, 2), "type": "voice_end"})
|
||||
|
||||
anchors["voice_anchors"] = voice_anchors
|
||||
logger.info(f"人声锚点: {len(voice_anchors)} 个")
|
||||
|
||||
# ── 2. BGM 锚点 (节奏变化检测) ──
|
||||
if bgm_path and os.path.exists(bgm_path):
|
||||
y_bgm, sr_bgm = librosa.load(bgm_path, sr=22050, mono=True)
|
||||
bgm_duration = librosa.get_duration(y=y_bgm, sr=sr_bgm)
|
||||
anchors["metadata"]["bgm_duration_sec"] = bgm_duration
|
||||
|
||||
# 节奏检测 (tempo)
|
||||
tempo, beats = librosa.beat.beat_track(y=y_bgm, sr=sr_bgm)
|
||||
beat_times = librosa.frames_to_time(beats, sr=sr_bgm)
|
||||
|
||||
bgm_anchors = [{"time": round(t, 2), "type": "beat"} for t in beat_times]
|
||||
anchors["bgm_anchors"] = bgm_anchors
|
||||
anchors["metadata"]["bgm_tempo"] = round(float(tempo), 1)
|
||||
logger.info(f"BGM 节奏: {float(tempo):.1f} BPM, {len(bgm_anchors)} 个拍点")
|
||||
else:
|
||||
logger.info("无 BGM 文件，跳过 BGM 锚点")
|
||||
|
||||
# ── 写入文件 ──
|
||||
with open(output_path, "w", encoding="utf-8") as f:
|
||||
json.dump(anchors, f, ensure_ascii=False, indent=2)
|
||||
logger.info(f"锚点已写入: {output_path}")
|
||||
|
||||
return anchors
|
||||
|
||||
|
||||
def get_silence_periods(
|
||||
voice_path: str,
|
||||
min_silence_sec: float = 0.5,
|
||||
) -> list[dict]:
|
||||
"""
|
||||
检测人声中的静音段落 (可用于画面过渡时机)
|
||||
|
||||
返回: [{"start": 1.0, "end": 1.5, "duration": 0.5}, ...]
|
||||
"""
|
||||
import librosa
|
||||
|
||||
y, sr = librosa.load(voice_path, sr=24000, mono=True)
|
||||
intervals = librosa.effects.split(
|
||||
y, top_db=30, frame_length=int(sr * 0.025), hop_length=int(sr * 0.010)
|
||||
)
|
||||
|
||||
# intervals 是语音段，间隙就是静音段
|
||||
silence_periods = []
|
||||
prev_end = 0
|
||||
for start, end in intervals:
|
||||
gap = start / sr - prev_end
|
||||
if gap >= min_silence_sec:
|
||||
silence_periods.append({
|
||||
"start": round(prev_end, 2),
|
||||
"end": round(start / sr, 2),
|
||||
"duration": round(gap, 2),
|
||||
})
|
||||
prev_end = end / sr
|
||||
|
||||
return silence_periods
|
||||
@@ -0,0 +1,14 @@
|
||||
"""
|
||||
AVE asset_manager — 素材资产管理包
|
||||
|
||||
提供素材索引、缓存管理、标签搜索三大能力。
|
||||
|
||||
模块:
|
||||
index.py — AssetIndex: 扫描目录、提取元数据、入库 SQLite
|
||||
cache.py — CacheManager: 增量扫描、缓存清理、孤立文件回收
|
||||
tags.py — AssetSearch: 自动标签、模糊搜索、CLI 查询
|
||||
|
||||
依赖:
|
||||
- lib/dashboard.py (assets 表 schema 已存在)
|
||||
- ffprobe (FFmpeg, 用于提取视频元数据)
|
||||
"""
|
||||
@@ -0,0 +1,367 @@
|
||||
"""
|
||||
asset_manager/cache — 缓存管理与自动入库
|
||||
|
||||
职责:
|
||||
1. 增量扫描: 跟踪目录 mtime, 只扫描新文件
|
||||
2. 自动入库: 发现新文件 → 调用 AssetIndex 入库
|
||||
3. 缓存清理: 按策略清理孤立/过期文件
|
||||
4. 生产者钩子: Pexels/Kling 下载完成后自动通知
|
||||
|
||||
用法:
|
||||
from asset_manager.cache import CacheManager
|
||||
cm = CacheManager()
|
||||
cm.scan_new() # 扫描所有目录的新文件
|
||||
cm.cleanup(days=30) # 清理 30 天前无关联的素材
|
||||
cm.register_scan("/tmp/") # 注册额外扫描目录
|
||||
|
||||
设计原则:
|
||||
- 无持续进程/守护: 每次 CLI 调用时触发扫描
|
||||
- mtime 记录在 SQLite meta_json 中, 跨会话持久
|
||||
- 重要素材 (有 production_id 关联) 不会被清理
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sqlite3
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Callable, Optional, Union
|
||||
|
||||
from lib.logger import get_logger
|
||||
from asset_manager.index import AssetIndex
|
||||
|
||||
logger = get_logger("asset_cache")
|
||||
|
||||
# ── 全局路径 (复用 index.py) ─────────────────────────────
|
||||
_LOCAL_ROOT = Path(os.environ.get(
|
||||
"AVE_LOCAL_DIR",
|
||||
str(Path.home() / "workbuddy-agent-os/agent-local"),
|
||||
))
|
||||
_CACHE_DIR = _LOCAL_ROOT / "tools" / "ave" / "cache"
|
||||
_DB_PATH = _LOCAL_ROOT / "tools" / "ave" / "data" / "ave.db"
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 核心类
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
class CacheManager:
|
||||
"""缓存管理器
|
||||
|
||||
属性:
|
||||
index: AssetIndex 实例 (共享 DB)
|
||||
scan_records: {dir_path: last_mtime} 记录上次扫描时间
|
||||
"""
|
||||
|
||||
def __init__(self, cache_dir: Union[str, Path] = _CACHE_DIR,
|
||||
db_path: Union[str, Path] = _DB_PATH):
|
||||
self.cache_dir = Path(cache_dir)
|
||||
self.db_path = Path(db_path)
|
||||
self.index = AssetIndex(cache_dir=cache_dir, db_path=db_path)
|
||||
# 额外扫描目录注册表: {路径: 回调函数}
|
||||
self._extra_scans: list[tuple[str, Optional[Callable]]] = []
|
||||
|
||||
# ── 注册额外扫描目录 ─────────────────────────────────
|
||||
|
||||
def register_scan(self, directory: str,
|
||||
callback: Optional[Callable] = None):
|
||||
"""注册额外目录到扫描列表 (如外部导入的素材目录)"""
|
||||
self._extra_scans.append((directory, callback))
|
||||
logger.info(f"注册扫描目录: {directory}")
|
||||
|
||||
# ── 增量扫描 ─────────────────────────────────────────
|
||||
|
||||
def _load_scan_record(self, subdir: str) -> float:
|
||||
"""从 meta_json 读取上次扫描记录的 mtime"""
|
||||
conn = sqlite3.connect(str(self.db_path))
|
||||
try:
|
||||
# 用特殊 hash 值 '_scan_record' 标记扫描记录
|
||||
row = conn.execute(
|
||||
"SELECT meta_json FROM assets WHERE hash = ? AND source = ?",
|
||||
(f"_scan_{subdir}", "__cache_mgr__"),
|
||||
).fetchone()
|
||||
if row:
|
||||
rec = json.loads(row[0])
|
||||
return float(rec.get("last_mtime", 0))
|
||||
except Exception:
|
||||
pass
|
||||
finally:
|
||||
conn.close()
|
||||
return 0.0
|
||||
|
||||
def _save_scan_record(self, subdir: str, mtime: float):
|
||||
"""持久化扫描记录到 DB"""
|
||||
conn = sqlite3.connect(str(self.db_path))
|
||||
try:
|
||||
# 先删除旧记录
|
||||
conn.execute(
|
||||
"DELETE FROM assets WHERE hash = ? AND source = ?",
|
||||
(f"_scan_{subdir}", "__cache_mgr__"),
|
||||
)
|
||||
conn.execute(
|
||||
"""INSERT INTO assets (asset_type, name, file_path, file_size, hash, source, meta_json)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)""",
|
||||
("__meta__", f"scan_record_{subdir}", "",
|
||||
0, f"_scan_{subdir}", "__cache_mgr__",
|
||||
json.dumps({"last_mtime": mtime, "updated_at": datetime.now().isoformat()})),
|
||||
)
|
||||
conn.commit()
|
||||
except Exception as e:
|
||||
logger.debug(f"保存扫描记录失败: {e}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def scan_new(self, force_rescan: bool = False) -> dict:
|
||||
"""
|
||||
增量扫描: 只扫描 mtime 超过上次记录的文件。
|
||||
|
||||
参数:
|
||||
force_rescan: 强制全量重新扫描
|
||||
|
||||
返回:
|
||||
{"total_new": int, "per_dir": {subdir: new_count}}
|
||||
"""
|
||||
from asset_manager.index import WATCHED_SUBDIRS, EXT_TYPE_MAP
|
||||
|
||||
results: dict = {"total_new": 0, "per_dir": {}}
|
||||
valid_exts = set(EXT_TYPE_MAP.keys())
|
||||
|
||||
for subdir in WATCHED_SUBDIRS:
|
||||
target = self.cache_dir / subdir
|
||||
if not target.is_dir():
|
||||
continue
|
||||
|
||||
last_mtime = 0.0 if force_rescan else self._load_scan_record(subdir)
|
||||
new_count = 0
|
||||
|
||||
for fpath in sorted(target.rglob("*")):
|
||||
if not fpath.is_file():
|
||||
continue
|
||||
if fpath.suffix.lower() not in valid_exts:
|
||||
continue
|
||||
|
||||
file_mtime = fpath.stat().st_mtime
|
||||
if file_mtime > last_mtime:
|
||||
ret = self.index.index_file(str(fpath))
|
||||
if ret > 0:
|
||||
new_count += 1
|
||||
|
||||
# 更新扫描记录 (用当前最大 mtime)
|
||||
current_max = 0.0
|
||||
for fpath in target.rglob("*"):
|
||||
if fpath.is_file():
|
||||
current_max = max(current_max, fpath.stat().st_mtime)
|
||||
self._save_scan_record(subdir, current_max)
|
||||
|
||||
if new_count > 0:
|
||||
logger.info(f" [{subdir}] 新增 {new_count} 个素材")
|
||||
results["per_dir"][subdir] = new_count
|
||||
results["total_new"] += new_count
|
||||
|
||||
# 额外扫描目录
|
||||
for extra_dir, callback in self._extra_scans:
|
||||
extra_path = Path(extra_dir)
|
||||
if not extra_path.is_dir():
|
||||
continue
|
||||
for fpath in extra_path.rglob("*"):
|
||||
if not fpath.is_file():
|
||||
continue
|
||||
if fpath.suffix.lower() not in valid_exts:
|
||||
continue
|
||||
ret = self.index.index_file(str(fpath))
|
||||
if ret > 0:
|
||||
results["total_new"] += 1
|
||||
if callback:
|
||||
try:
|
||||
callback(str(fpath))
|
||||
except Exception as e:
|
||||
logger.warning(f"回调失败 {fpath}: {e}")
|
||||
|
||||
logger.info(f"增量扫描完成, 共新增 {results['total_new']} 个素材")
|
||||
return results
|
||||
|
||||
# ── 全量扫描 ─────────────────────────────────────────
|
||||
|
||||
def scan_all(self, production_id: int = 0) -> dict:
|
||||
"""全量扫描并重置所有扫描记录"""
|
||||
results = self.index.scan_all(production_id)
|
||||
|
||||
# 重置扫描记录为最新 mtime
|
||||
from asset_manager.index import WATCHED_SUBDIRS
|
||||
for subdir in WATCHED_SUBDIRS:
|
||||
target = self.cache_dir / subdir
|
||||
if target.is_dir():
|
||||
current_max = max(
|
||||
(f.stat().st_mtime for f in target.rglob("*") if f.is_file()),
|
||||
default=0.0,
|
||||
)
|
||||
self._save_scan_record(subdir, current_max)
|
||||
|
||||
return results
|
||||
|
||||
# ── 缓存清理 ─────────────────────────────────────────
|
||||
|
||||
def cleanup(self, days: int = 30, dry_run: bool = False) -> dict:
|
||||
"""
|
||||
清理过期缓存。
|
||||
|
||||
清理规则:
|
||||
- 无 production_id 关联
|
||||
- 文件创建时间 > days 天前
|
||||
- 保留有 production 引用的素材
|
||||
|
||||
参数:
|
||||
days: 保留天数
|
||||
dry_run: 仅预览不删除
|
||||
|
||||
返回:
|
||||
{"deleted": int, "freed_bytes": int, "files": [路径]}
|
||||
"""
|
||||
conn = sqlite3.connect(str(self.db_path))
|
||||
cutoff = (datetime.now() - timedelta(days=days)).isoformat()
|
||||
result: dict = {"deleted": 0, "freed_bytes": 0, "files": []}
|
||||
|
||||
try:
|
||||
# 查找无关联的旧资产
|
||||
rows = conn.execute(
|
||||
"""SELECT id, file_path, file_size FROM assets
|
||||
WHERE production_id IS NULL
|
||||
AND created_at < ?
|
||||
AND asset_type != '__meta__'
|
||||
ORDER BY created_at ASC""",
|
||||
(cutoff,),
|
||||
).fetchall()
|
||||
|
||||
for row in rows:
|
||||
aid, fpath, fsize = row["id"], row["file_path"], row["file_size"] or 0
|
||||
|
||||
if dry_run:
|
||||
result["deleted"] += 1
|
||||
result["freed_bytes"] += fsize
|
||||
result["files"].append(f"{fpath} ({_human_size(fsize)})")
|
||||
continue
|
||||
|
||||
# 删除磁盘文件
|
||||
if fpath and os.path.isfile(fpath):
|
||||
try:
|
||||
os.remove(fpath)
|
||||
logger.debug(f" 删除文件: {fpath}")
|
||||
except OSError as e:
|
||||
logger.warning(f" 删除失败: {fpath}: {e}")
|
||||
|
||||
# 删除 DB 记录
|
||||
conn.execute("DELETE FROM asset_tags WHERE asset_id = ?", (aid,))
|
||||
conn.execute("DELETE FROM production_assets WHERE asset_id = ?", (aid,))
|
||||
conn.execute("DELETE FROM assets WHERE id = ?", (aid,))
|
||||
|
||||
result["deleted"] += 1
|
||||
result["freed_bytes"] += fsize
|
||||
result["files"].append(fpath)
|
||||
|
||||
conn.commit()
|
||||
except Exception as e:
|
||||
logger.error(f"cleanup 失败: {e}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
action = "预览" if dry_run else "清理"
|
||||
logger.info(f"缓存{action}: {result['deleted']} 个文件, "
|
||||
f"释放 {_human_size(result['freed_bytes'])}")
|
||||
return result
|
||||
|
||||
# ── 目录统计 ─────────────────────────────────────────
|
||||
|
||||
def disk_usage(self) -> dict:
|
||||
"""统计每个缓存目录的磁盘占用"""
|
||||
usage: dict = {}
|
||||
for subdir in os.listdir(self.cache_dir):
|
||||
target = self.cache_dir / subdir
|
||||
if not target.is_dir():
|
||||
continue
|
||||
total = 0
|
||||
count = 0
|
||||
for fpath in target.rglob("*"):
|
||||
if fpath.is_file():
|
||||
total += fpath.stat().st_size
|
||||
count += 1
|
||||
usage[subdir] = {
|
||||
"count": count,
|
||||
"size_bytes": total,
|
||||
"size_human": _human_size(total),
|
||||
}
|
||||
return usage
|
||||
|
||||
def print_disk_usage(self):
|
||||
"""打印磁盘占用表"""
|
||||
usage = self.disk_usage()
|
||||
print(f"\n{'='*50}")
|
||||
print(f" 缓存磁盘占用")
|
||||
print(f"{'='*50}")
|
||||
print(f" {'目录':20s} {'文件数':>8s} {'大小':>12s}")
|
||||
print(f" {'-'*40}")
|
||||
total_size = 0
|
||||
for subdir, info in sorted(usage.items(), key=lambda x: -x[1]["size_bytes"]):
|
||||
print(f" {subdir:20s} {info['count']:>8d} {info['size_human']:>12s}")
|
||||
total_size += info["size_bytes"]
|
||||
print(f" {'-'*40}")
|
||||
print(f" {'总计':20s} {'':>8s} {_human_size(total_size):>12s}")
|
||||
print(f"{'='*50}\n")
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 生产者自动入库钩子
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
def auto_index_on_download(file_path: str, production_id: int = 0) -> int:
|
||||
"""
|
||||
生产者在下载/生成素材后, 调用此函数自动入库。
|
||||
|
||||
用法:
|
||||
from asset_manager.cache import auto_index_on_download
|
||||
|
||||
path = pexels_search.download(url)
|
||||
auto_index_on_download(path, production_id=pid)
|
||||
"""
|
||||
idx = AssetIndex()
|
||||
return idx.index_file(file_path, production_id=production_id)
|
||||
|
||||
|
||||
def index_output_video(output_path: str, production_id: int) -> int:
|
||||
"""
|
||||
最终成片入库 (特殊标记 source=output)。
|
||||
"""
|
||||
if not os.path.isfile(output_path):
|
||||
return -1
|
||||
idx = AssetIndex()
|
||||
# 强制标记 source 为 output
|
||||
conn = sqlite3.connect(str(idx.db_path))
|
||||
try:
|
||||
# 先通过 index_file 入库
|
||||
aid = idx.index_file(output_path, production_id=production_id)
|
||||
if aid > 0:
|
||||
# 覆盖 source
|
||||
conn.execute(
|
||||
"UPDATE assets SET source = 'production_output' WHERE id = ?",
|
||||
(aid,),
|
||||
)
|
||||
conn.commit()
|
||||
return aid
|
||||
return aid
|
||||
except Exception as e:
|
||||
logger.error(f"index_output 失败: {e}")
|
||||
return -1
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ── 辅助函数 ─────────────────────────────────────────────
|
||||
|
||||
def _human_size(bytes_: int) -> str:
|
||||
if bytes_ < 1024:
|
||||
return f"{bytes_} B"
|
||||
elif bytes_ < 1024**2:
|
||||
return f"{bytes_/1024:.1f} KB"
|
||||
elif bytes_ < 1024**3:
|
||||
return f"{bytes_/(1024**2):.1f} MB"
|
||||
return f"{bytes_/(1024**3):.2f} GB"
|
||||
@@ -0,0 +1,469 @@
|
||||
"""
|
||||
asset_manager/index — 素材索引引擎
|
||||
|
||||
职责:
|
||||
1. 扫描所有已知素材目录，发现新文件
|
||||
2. 通过 ffprobe 提取视频/音频元数据 (时长/分辨率/编码/FPS)
|
||||
3. 计算文件 hash (首 64KB + 文件大小, 快速去重)
|
||||
4. 写入 SQLite assets 表, 避免重复入库
|
||||
|
||||
用法:
|
||||
from asset_manager.index import AssetIndex
|
||||
|
||||
idx = AssetIndex()
|
||||
stats = idx.scan_all() # 全量扫描所有目录
|
||||
stats = idx.scan_dir("materials") # 扫描单个子目录
|
||||
print(idx.summarize()) # 打印统计
|
||||
|
||||
AssetIndex 发现策略:
|
||||
- 只扫描已知目录 (materials/kling/wan2_2/bgm/bgm_library/lipsync/character_sheet)
|
||||
- 按 hash 去重: 同一文件即使出现在不同目录也只入库一次
|
||||
- 增量模式: 只扫描 mtime > 上次扫描时间的文件
|
||||
"""
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional, Union
|
||||
|
||||
from lib.logger import get_logger
|
||||
|
||||
logger = get_logger("asset_index")
|
||||
|
||||
# ── 全局路径 ───────────────────────────────────────────────
|
||||
_LOCAL_ROOT = Path(os.environ.get(
|
||||
"AVE_LOCAL_DIR",
|
||||
str(Path.home() / "workbuddy-agent-os/agent-local"),
|
||||
))
|
||||
_CACHE_DIR = _LOCAL_ROOT / "tools" / "ave" / "cache"
|
||||
_DB_PATH = _LOCAL_ROOT / "tools" / "ave" / "data" / "ave.db"
|
||||
|
||||
# ── 已知素材目录 (相对于 cache) ──────────────────────────
|
||||
WATCHED_SUBDIRS: dict[str, str] = {
|
||||
"materials": "pexels 下载素材",
|
||||
"kling": "Kling 生成视频",
|
||||
"wan2_2": "Wan2.2 数字人视频",
|
||||
"bgm": "BGM 缓存",
|
||||
"bgm_library": "BGM 素材库",
|
||||
"lipsync": "唇形同步产出",
|
||||
"character_sheet": "定妆照/角色图",
|
||||
"outputs": "最终成片",
|
||||
"beat_sync": "节拍分析缓存",
|
||||
}
|
||||
|
||||
# ── 已知扩展名 → asset_type ─────────────────────────────
|
||||
EXT_TYPE_MAP = {
|
||||
".mp4": "video",
|
||||
".mov": "video",
|
||||
".webm": "video",
|
||||
".avi": "video",
|
||||
".mkv": "video",
|
||||
".wav": "audio",
|
||||
".mp3": "audio",
|
||||
".aac": "audio",
|
||||
".m4a": "audio",
|
||||
".flac": "audio",
|
||||
".ogg": "audio",
|
||||
".png": "image",
|
||||
".jpg": "image",
|
||||
".jpeg": "image",
|
||||
".webp": "image",
|
||||
".gif": "image",
|
||||
}
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 核心类
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
class AssetIndex:
|
||||
"""素材索引引擎"""
|
||||
|
||||
def __init__(self, cache_dir: Union[str, Path] = _CACHE_DIR,
|
||||
db_path: Union[str, Path] = _DB_PATH):
|
||||
self.cache_dir = Path(cache_dir)
|
||||
self.db_path = Path(db_path)
|
||||
self._init_db()
|
||||
|
||||
# ── DB 初始化 ──────────────────────────────────────────
|
||||
|
||||
def _init_db(self):
|
||||
"""确保 assets 相关表存在（复用 dashboard.py 的 schema）"""
|
||||
try:
|
||||
conn = sqlite3.connect(str(self.db_path))
|
||||
conn.executescript("""
|
||||
CREATE TABLE IF NOT EXISTS assets (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
asset_type TEXT NOT NULL,
|
||||
name TEXT,
|
||||
file_path TEXT,
|
||||
file_size INTEGER,
|
||||
hash TEXT,
|
||||
source TEXT,
|
||||
meta_json TEXT,
|
||||
production_id INTEGER,
|
||||
created_at TEXT DEFAULT (datetime('now','localtime'))
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS asset_tags (
|
||||
asset_id INTEGER NOT NULL REFERENCES assets(id),
|
||||
tag TEXT NOT NULL,
|
||||
PRIMARY KEY (asset_id, tag)
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS production_assets (
|
||||
production_id INTEGER NOT NULL REFERENCES productions(id),
|
||||
asset_id INTEGER NOT NULL REFERENCES assets(id),
|
||||
role TEXT,
|
||||
PRIMARY KEY (production_id, asset_id)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_assets_hash ON assets(hash);
|
||||
CREATE INDEX IF NOT EXISTS idx_assets_type ON assets(asset_type);
|
||||
CREATE INDEX IF NOT EXISTS idx_assets_path ON assets(file_path);
|
||||
CREATE INDEX IF NOT EXISTS idx_asset_tags_tag ON asset_tags(tag);
|
||||
""")
|
||||
conn.commit()
|
||||
conn.close()
|
||||
except Exception as e:
|
||||
logger.error(f"DB 初始化失败: {e}")
|
||||
|
||||
# ── 文件级操作 ────────────────────────────────────────
|
||||
|
||||
def _compute_hash(self, file_path: str) -> str:
|
||||
"""快速 hash: 读首 64KB + 文件大小 (比全量 MD5 快 100x)"""
|
||||
try:
|
||||
sz = os.path.getsize(file_path)
|
||||
with open(file_path, "rb") as f:
|
||||
head = f.read(65536) # 64KB
|
||||
h = hashlib.md5(head).hexdigest()[:16]
|
||||
return f"{h}_{sz}"
|
||||
except OSError:
|
||||
return ""
|
||||
|
||||
def _extract_media_info(self, file_path: str) -> dict:
|
||||
"""通过 ffprobe 提取媒体元数据"""
|
||||
meta: dict = {}
|
||||
ext = Path(file_path).suffix.lower()
|
||||
|
||||
# 非视频/音频文件跳过 ffprobe
|
||||
if ext not in (".mp4", ".mov", ".webm", ".avi", ".mkv",
|
||||
".wav", ".mp3", ".aac", ".m4a", ".flac", ".ogg"):
|
||||
return meta
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["ffprobe", "-v", "quiet", "-print_format", "json",
|
||||
"-show_format", "-show_streams", file_path],
|
||||
capture_output=True, text=True, timeout=30,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return meta
|
||||
|
||||
data = json.loads(result.stdout)
|
||||
|
||||
# 格式信息
|
||||
fmt = data.get("format", {})
|
||||
meta["format"] = fmt.get("format_name", "")
|
||||
meta["bitrate"] = int(fmt.get("bit_rate", 0))
|
||||
meta["duration_sec"] = round(float(fmt.get("duration", 0)), 2)
|
||||
meta["size_bytes"] = int(fmt.get("size", 0))
|
||||
|
||||
# 视频流
|
||||
for stream in data.get("streams", []):
|
||||
if stream.get("codec_type") == "video":
|
||||
meta["video_codec"] = stream.get("codec_name", "")
|
||||
meta["width"] = stream.get("width", 0)
|
||||
meta["height"] = stream.get("height", 0)
|
||||
meta["fps"] = _parse_fps(stream.get("r_frame_rate", ""))
|
||||
meta["pixel_format"] = stream.get("pix_fmt", "")
|
||||
break
|
||||
|
||||
# 音频流
|
||||
for stream in data.get("streams", []):
|
||||
if stream.get("codec_type") == "audio":
|
||||
meta["audio_codec"] = stream.get("codec_name", "")
|
||||
meta["sample_rate"] = int(stream.get("sample_rate", 0))
|
||||
meta["channels"] = stream.get("channels", 0)
|
||||
break
|
||||
|
||||
except (subprocess.TimeoutExpired, json.JSONDecodeError,
|
||||
FileNotFoundError) as e:
|
||||
logger.debug(f"ffprobe 失败 {file_path}: {e}")
|
||||
|
||||
return meta
|
||||
|
||||
def _source_from_path(self, file_path: str) -> str:
|
||||
"""从文件路径推断来源"""
|
||||
p = Path(file_path)
|
||||
try:
|
||||
rel = p.relative_to(self.cache_dir)
|
||||
parts = rel.parts
|
||||
if len(parts) >= 1:
|
||||
return parts[0] # 顶级子目录名 = source
|
||||
except ValueError:
|
||||
pass
|
||||
return "unknown"
|
||||
|
||||
def _infer_tags(self, file_path: str, meta: dict, source: str) -> list[str]:
|
||||
"""基于文件属性自动生成标签"""
|
||||
tags: list[str] = [source]
|
||||
ext = Path(file_path).suffix.lower()
|
||||
|
||||
# 文件类型标签
|
||||
asset_type = EXT_TYPE_MAP.get(ext, "other")
|
||||
tags.append(asset_type)
|
||||
|
||||
# 尺寸标签 (视频)
|
||||
w = meta.get("width", 0)
|
||||
h = meta.get("height", 0)
|
||||
if w and h:
|
||||
if w >= 3840 or h >= 2160:
|
||||
tags.append("4k")
|
||||
elif w >= 1920 or h >= 1080:
|
||||
tags.append("1080p")
|
||||
elif w >= 1280 or h >= 720:
|
||||
tags.append("720p")
|
||||
else:
|
||||
tags.append("sd")
|
||||
|
||||
# 时长标签
|
||||
dur = meta.get("duration_sec", 0)
|
||||
if dur > 0:
|
||||
if dur < 2:
|
||||
tags.append("ultra_short")
|
||||
elif dur < 5:
|
||||
tags.append("short")
|
||||
elif dur < 15:
|
||||
tags.append("medium")
|
||||
elif dur < 60:
|
||||
tags.append("long")
|
||||
else:
|
||||
tags.append("extra_long")
|
||||
|
||||
# 编码标签
|
||||
vcodec = meta.get("video_codec", "")
|
||||
if vcodec:
|
||||
tags.append(f"codec_{vcodec}")
|
||||
|
||||
return tags
|
||||
|
||||
# ── 单文件入库 ────────────────────────────────────────
|
||||
|
||||
def index_file(self, file_path: str, production_id: int = 0,
|
||||
force: bool = False) -> int:
|
||||
"""
|
||||
索引单个文件并入库。
|
||||
|
||||
参数:
|
||||
file_path: 文件绝对路径
|
||||
production_id: 关联的 production ID (0 = 不关联)
|
||||
force: 强制重新索引 (即使 hash 已存在)
|
||||
|
||||
返回:
|
||||
asset_id (失败返回 -1, 已存在且未 force 返回 -2)
|
||||
"""
|
||||
if not os.path.isfile(file_path):
|
||||
logger.warning(f"文件不存在: {file_path}")
|
||||
return -1
|
||||
|
||||
ext = Path(file_path).suffix.lower()
|
||||
asset_type = EXT_TYPE_MAP.get(ext, "other")
|
||||
name = Path(file_path).name
|
||||
file_size = os.path.getsize(file_path)
|
||||
file_hash = self._compute_hash(file_path)
|
||||
source = self._source_from_path(file_path)
|
||||
meta = self._extract_media_info(file_path)
|
||||
tags = self._infer_tags(file_path, meta, source)
|
||||
|
||||
if not file_hash:
|
||||
return -1
|
||||
|
||||
conn = sqlite3.connect(str(self.db_path))
|
||||
try:
|
||||
# 去重检查
|
||||
if not force and file_hash:
|
||||
existing = conn.execute(
|
||||
"SELECT id FROM assets WHERE hash = ? LIMIT 1",
|
||||
(file_hash,),
|
||||
).fetchone()
|
||||
if existing:
|
||||
conn.close()
|
||||
return -2 # 已存在
|
||||
|
||||
# 插入资产
|
||||
cur = conn.execute(
|
||||
"""INSERT INTO assets
|
||||
(asset_type, name, file_path, file_size, hash, source, meta_json, production_id)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)""",
|
||||
(asset_type, name, file_path, file_size, file_hash, source,
|
||||
json.dumps(meta, ensure_ascii=False),
|
||||
production_id if production_id > 0 else None),
|
||||
)
|
||||
aid = cur.lastrowid
|
||||
|
||||
# 标签
|
||||
for tag in set(tags): # set 去重
|
||||
conn.execute(
|
||||
"INSERT OR IGNORE INTO asset_tags (asset_id, tag) VALUES (?, ?)",
|
||||
(aid, tag),
|
||||
)
|
||||
|
||||
# 关联 production
|
||||
if production_id > 0:
|
||||
conn.execute(
|
||||
"""INSERT OR IGNORE INTO production_assets
|
||||
(production_id, asset_id, role) VALUES (?, ?, ?)""",
|
||||
(production_id, aid, "output" if source in ("materials", "kling", "wan2_2")
|
||||
else "cache"),
|
||||
)
|
||||
|
||||
conn.commit()
|
||||
logger.info(f" 入库: {name} (type={asset_type}, hash={file_hash[:12]}...)")
|
||||
return aid
|
||||
except Exception as e:
|
||||
logger.error(f"index_file 失败: {file_path}: {e}")
|
||||
return -1
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
# ── 目录扫描 ──────────────────────────────────────────
|
||||
|
||||
def scan_dir(self, subdir: str, production_id: int = 0,
|
||||
progress: bool = True) -> dict:
|
||||
"""
|
||||
扫描一个子目录, 入库所有新文件。
|
||||
|
||||
参数:
|
||||
subdir: WATCHED_SUBDIRS 中的 key (如 "materials")
|
||||
production_id: 可选关联 production
|
||||
progress: 打印进度
|
||||
|
||||
返回:
|
||||
{"dir": str, "total": int, "new": int, "skipped": int, "errors": int}
|
||||
"""
|
||||
target_dir = self.cache_dir / subdir
|
||||
stat = {"dir": subdir, "total": 0, "new": 0, "skipped": 0, "errors": 0}
|
||||
|
||||
if not target_dir.is_dir():
|
||||
logger.warning(f"目录不存在: {target_dir}")
|
||||
return stat
|
||||
|
||||
files = sorted(target_dir.rglob("*"))
|
||||
valid_exts = set(EXT_TYPE_MAP.keys())
|
||||
|
||||
for i, fpath in enumerate(files):
|
||||
if not fpath.is_file():
|
||||
continue
|
||||
if fpath.suffix.lower() not in valid_exts:
|
||||
continue
|
||||
|
||||
stat["total"] += 1
|
||||
ret = self.index_file(str(fpath), production_id)
|
||||
if ret == -1:
|
||||
stat["errors"] += 1
|
||||
elif ret == -2:
|
||||
stat["skipped"] += 1
|
||||
else:
|
||||
stat["new"] += 1
|
||||
|
||||
if progress and i % 20 == 0 and i > 0:
|
||||
logger.info(f" [{subdir}] {i} 文件已处理...")
|
||||
|
||||
logger.info(f" [{subdir}] 共 {stat['total']} 文件, "
|
||||
f"新增 {stat['new']}, 跳过 {stat['skipped']}, 错误 {stat['errors']}")
|
||||
return stat
|
||||
|
||||
def scan_all(self, production_id: int = 0, progress: bool = True) -> dict:
|
||||
"""
|
||||
扫描所有已知素材目录。
|
||||
|
||||
返回:
|
||||
{"total_new": int, "dirs": {subdir: stat_dict}}
|
||||
"""
|
||||
results: dict = {"total_new": 0, "dirs": {}}
|
||||
for subdir in WATCHED_SUBDIRS:
|
||||
stat = self.scan_dir(subdir, production_id, progress)
|
||||
results["dirs"][subdir] = stat
|
||||
results["total_new"] += stat["new"]
|
||||
|
||||
logger.info(f"全量扫描完成, 新增 {results['total_new']} 个素材")
|
||||
return results
|
||||
|
||||
# ── 统计查询 ──────────────────────────────────────────
|
||||
|
||||
def summarize(self) -> dict:
|
||||
"""汇总当前素材库统计信息"""
|
||||
conn = sqlite3.connect(str(self.db_path))
|
||||
try:
|
||||
total = conn.execute("SELECT COUNT(*) FROM assets").fetchone()[0]
|
||||
by_type = conn.execute(
|
||||
"SELECT asset_type, COUNT(*) as cnt FROM assets GROUP BY asset_type"
|
||||
).fetchall()
|
||||
by_source = conn.execute(
|
||||
"SELECT source, COUNT(*) as cnt FROM assets GROUP BY source"
|
||||
).fetchall()
|
||||
total_size = conn.execute(
|
||||
"SELECT COALESCE(SUM(file_size), 0) FROM assets"
|
||||
).fetchone()[0]
|
||||
orphan = conn.execute(
|
||||
"SELECT COUNT(*) FROM assets WHERE production_id IS NULL"
|
||||
).fetchone()[0]
|
||||
|
||||
return {
|
||||
"total_assets": total,
|
||||
"by_type": dict(by_type),
|
||||
"by_source": dict(by_source),
|
||||
"total_size_bytes": total_size,
|
||||
"total_size_mb": round(total_size / (1024 * 1024), 1),
|
||||
"orphan_count": orphan,
|
||||
"watched_dirs": list(WATCHED_SUBDIRS.keys()),
|
||||
}
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def print_summary(self):
|
||||
"""打印人类可读的统计"""
|
||||
s = self.summarize()
|
||||
print(f"\n{'='*50}")
|
||||
print(f" 素材库统计")
|
||||
print(f"{'='*50}")
|
||||
print(f" 素材总数: {s['total_assets']}")
|
||||
print(f" 总大小: {s['total_size_mb']} MB")
|
||||
print(f" 无关联素材: {s['orphan_count']}")
|
||||
print(f"\n ─ 按类型 ─")
|
||||
for t, c in sorted(s["by_type"].items(), key=lambda x: -x[1]):
|
||||
print(f" {t:12s} {c}")
|
||||
print(f"\n ─ 按来源 ─")
|
||||
for src, c in sorted(s["by_source"].items(), key=lambda x: -x[1]):
|
||||
print(f" {src:16s} {c}")
|
||||
print(f"\n ─ 监控目录 ─")
|
||||
for d in s["watched_dirs"]:
|
||||
print(f" {d}")
|
||||
print(f"{'='*50}\n")
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 辅助函数
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
def _parse_fps(r_frame_rate: str) -> float:
|
||||
"""解析 ffprobe 的 r_frame_rate (如 '30000/1001' → 29.97)"""
|
||||
try:
|
||||
if "/" in r_frame_rate:
|
||||
num, den = r_frame_rate.split("/")
|
||||
return round(float(num) / float(den), 2)
|
||||
return float(r_frame_rate)
|
||||
except (ValueError, ZeroDivisionError):
|
||||
return 0.0
|
||||
|
||||
|
||||
def _human_size(bytes_: int) -> str:
|
||||
if bytes_ < 1024:
|
||||
return f"{bytes_} B"
|
||||
elif bytes_ < 1024**2:
|
||||
return f"{bytes_/1024:.1f} KB"
|
||||
elif bytes_ < 1024**3:
|
||||
return f"{bytes_/(1024**2):.1f} MB"
|
||||
return f"{bytes_/(1024**3):.2f} GB"
|
||||
@@ -0,0 +1,317 @@
|
||||
"""
|
||||
asset_manager/tags — 标签系统 + 素材搜索
|
||||
|
||||
职责:
|
||||
1. 自动标签: 根据文件属性和路径生成标签 (复用 index.py 的 _infer_tags)
|
||||
2. 标签管理: 查看所有标签、标签频次、批量标记
|
||||
3. 搜素引擎: 按名称/路径/标签/类型/来源 多维度过滤
|
||||
4. 搜索结果格式化输出 (表格/JSON)
|
||||
|
||||
用法:
|
||||
from asset_manager.tags import AssetSearch
|
||||
srch = AssetSearch()
|
||||
|
||||
# 搜索
|
||||
results = srch.search("人物", tags=["video"])
|
||||
results = srch.search_by_tag("pexels", limit=20)
|
||||
|
||||
# 标签管理
|
||||
srch.tag_stats() # 所有标签频次
|
||||
srch.tag_asset(42, ["人物", "户外"]) # 给资产打标签
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional, Union
|
||||
|
||||
from lib.logger import get_logger
|
||||
|
||||
logger = get_logger("asset_tags")
|
||||
|
||||
# ── 全局路径 ─────────────────────────────────────────────
|
||||
_LOCAL_ROOT = Path(os.environ.get(
|
||||
"AVE_LOCAL_DIR",
|
||||
str(Path.home() / "workbuddy-agent-os/agent-local"),
|
||||
))
|
||||
_DB_PATH = _LOCAL_ROOT / "tools" / "ave" / "data" / "ave.db"
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 搜索类
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
class AssetSearch:
|
||||
"""素材搜索引擎 (基于 SQLite LIKE + 标签过滤)"""
|
||||
|
||||
def __init__(self, db_path: Union[str, Path] = _DB_PATH):
|
||||
self.db_path = Path(db_path)
|
||||
|
||||
def _connect(self) -> sqlite3.Connection:
|
||||
conn = sqlite3.connect(str(self.db_path))
|
||||
conn.row_factory = sqlite3.Row
|
||||
return conn
|
||||
|
||||
# ── 搜索 ─────────────────────────────────────────────
|
||||
|
||||
def search(
|
||||
self,
|
||||
keyword: str = "",
|
||||
*,
|
||||
asset_type: Optional[str] = None,
|
||||
source: Optional[str] = None,
|
||||
tags: Optional[list[str]] = None,
|
||||
min_size: int = 0,
|
||||
max_size: int = 0,
|
||||
min_duration: float = 0.0,
|
||||
max_duration: float = 0.0,
|
||||
production_id: Optional[int] = None,
|
||||
limit: int = 50,
|
||||
offset: int = 0,
|
||||
order_by: str = "id DESC",
|
||||
) -> list[dict]:
|
||||
"""
|
||||
多维度素材搜索。
|
||||
|
||||
参数:
|
||||
keyword: 关键字 (模糊匹配 name/file_path)
|
||||
asset_type: 文件类型 (video/audio/image/other)
|
||||
source: 来源 (materials/kling/wan2_2/bgm/...)
|
||||
tags: 标签列表 (AND 逻辑, 需同时命中)
|
||||
min_size: 最小文件大小 (bytes)
|
||||
max_size: 最大文件大小 (0 = 不限制)
|
||||
min_duration: 最小时长 (秒)
|
||||
max_duration: 最大时长 (0 = 不限制)
|
||||
production_id:关联 production ID
|
||||
limit/offset: 分页
|
||||
order_by: 排序 (SQL 片段, 如 "file_size DESC")
|
||||
|
||||
返回:
|
||||
[{
|
||||
"id", "asset_type", "name", "file_path", "file_size",
|
||||
"hash", "source", "meta", "tags", "created_at",
|
||||
"production_id"
|
||||
}, ...]
|
||||
"""
|
||||
conn = self._connect()
|
||||
try:
|
||||
where = []
|
||||
params: list = []
|
||||
|
||||
# 关键字
|
||||
if keyword:
|
||||
like = f"%{keyword}%"
|
||||
where.append("(a.name LIKE ? OR a.file_path LIKE ?)")
|
||||
params.extend([like, like])
|
||||
|
||||
# 类型
|
||||
if asset_type:
|
||||
where.append("a.asset_type = ?")
|
||||
params.append(asset_type)
|
||||
|
||||
# 来源
|
||||
if source:
|
||||
where.append("a.source = ?")
|
||||
params.append(source)
|
||||
|
||||
# 标签 (AND: 需要所有指定标签)
|
||||
if tags:
|
||||
tag_conditions = []
|
||||
for tag in tags:
|
||||
tag_conditions.append(
|
||||
"EXISTS (SELECT 1 FROM asset_tags WHERE asset_id = a.id AND tag = ?)"
|
||||
)
|
||||
params.append(tag.lower())
|
||||
where.append("(" + " AND ".join(tag_conditions) + ")")
|
||||
|
||||
# 文件大小
|
||||
if min_size > 0:
|
||||
where.append("a.file_size >= ?")
|
||||
params.append(min_size)
|
||||
if max_size > 0:
|
||||
where.append("a.file_size <= ?")
|
||||
params.append(max_size)
|
||||
|
||||
# 时长 (meta_json 中的 duration_sec)
|
||||
if min_duration > 0:
|
||||
where.append(
|
||||
"json_extract(a.meta_json, '$.duration_sec') >= ?"
|
||||
)
|
||||
params.append(min_duration)
|
||||
if max_duration > 0:
|
||||
where.append(
|
||||
"json_extract(a.meta_json, '$.duration_sec') <= ?"
|
||||
)
|
||||
params.append(max_duration)
|
||||
|
||||
# production
|
||||
if production_id is not None:
|
||||
where.append("a.production_id = ?")
|
||||
params.append(production_id)
|
||||
|
||||
# 排除内部记录
|
||||
where.append("a.asset_type != '__meta__'")
|
||||
|
||||
where_clause = " AND ".join(where) if where else "1=1"
|
||||
|
||||
# 安全校验 order_by (防止 SQL 注入)
|
||||
allowed_orders = {
|
||||
"id", "id DESC", "id ASC",
|
||||
"file_size", "file_size DESC", "file_size ASC",
|
||||
"created_at", "created_at DESC", "created_at ASC",
|
||||
"name", "name DESC", "name ASC",
|
||||
}
|
||||
safe_order = order_by if order_by in allowed_orders else "id DESC"
|
||||
|
||||
rows = conn.execute(
|
||||
f"""SELECT a.*, GROUP_CONCAT(at.tag, ',') as tag_list
|
||||
FROM assets a
|
||||
LEFT JOIN asset_tags at ON at.asset_id = a.id
|
||||
WHERE {where_clause}
|
||||
GROUP BY a.id
|
||||
ORDER BY {safe_order}
|
||||
LIMIT ? OFFSET ?""",
|
||||
params + [limit, offset],
|
||||
).fetchall()
|
||||
|
||||
results = []
|
||||
for r in rows:
|
||||
d = dict(r)
|
||||
# 解析 meta_json
|
||||
try:
|
||||
d["meta"] = json.loads(d.pop("meta_json", "{}") or "{}")
|
||||
except json.JSONDecodeError:
|
||||
d["meta"] = {}
|
||||
# 标签列表
|
||||
d["tags"] = d.pop("tag_list", "").split(",") if d.get("tag_list") else []
|
||||
d["file_size_human"] = _human_size(d.get("file_size", 0))
|
||||
results.append(d)
|
||||
|
||||
return results
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"search 失败: {e}")
|
||||
return []
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
# ── 标签搜索 ─────────────────────────────────────────
|
||||
|
||||
def search_by_tag(self, tag: str, limit: int = 50) -> list[dict]:
|
||||
"""按单个标签搜索"""
|
||||
return self.search(tags=[tag], limit=limit)
|
||||
|
||||
def search_by_type(self, asset_type: str, limit: int = 50) -> list[dict]:
|
||||
"""按类型搜索"""
|
||||
return self.search(asset_type=asset_type, limit=limit)
|
||||
|
||||
# ── 标签管理 ─────────────────────────────────────────
|
||||
|
||||
def tag_stats(self) -> list[dict]:
|
||||
"""返回所有标签及出现频次, 按频次降序"""
|
||||
conn = self._connect()
|
||||
try:
|
||||
rows = conn.execute(
|
||||
"""SELECT tag, COUNT(*) as count
|
||||
FROM asset_tags
|
||||
GROUP BY tag
|
||||
ORDER BY count DESC"""
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def tag_asset(self, asset_id: int, tags: list[str]) -> bool:
|
||||
"""给指定资产打标签"""
|
||||
conn = self._connect()
|
||||
try:
|
||||
for tag in tags:
|
||||
tag_clean = tag.strip().lower()
|
||||
if tag_clean:
|
||||
conn.execute(
|
||||
"INSERT OR IGNORE INTO asset_tags (asset_id, tag) VALUES (?, ?)",
|
||||
(asset_id, tag_clean),
|
||||
)
|
||||
conn.commit()
|
||||
logger.info(f"资产 {asset_id} 标记: {tags}")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"标记失败 asset {asset_id}: {e}")
|
||||
return False
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def untag_asset(self, asset_id: int, tags: list[str]) -> bool:
|
||||
"""移除标签"""
|
||||
conn = self._connect()
|
||||
try:
|
||||
for tag in tags:
|
||||
conn.execute(
|
||||
"DELETE FROM asset_tags WHERE asset_id = ? AND tag = ?",
|
||||
(asset_id, tag.strip().lower()),
|
||||
)
|
||||
conn.commit()
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"移除标签失败: {e}")
|
||||
return False
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def print_results(self, results: list[dict], show_meta: bool = False):
|
||||
"""打印搜索结果表格"""
|
||||
if not results:
|
||||
print(" 未找到匹配的素材")
|
||||
return
|
||||
|
||||
print(f"\n {'ID':>4s} {'类型':8s} {'来源':12s} {'大小':10s} {'名称'}")
|
||||
print(f" {'-'*80}")
|
||||
for r in results:
|
||||
print(f" {r['id']:>4d} {r['asset_type']:8s} {r['source'] or '?':12s} "
|
||||
f"{r.get('file_size_human', ''):>10s} {r['name'] or '?'}")
|
||||
if show_meta and r.get("meta"):
|
||||
meta = r["meta"]
|
||||
parts = []
|
||||
if meta.get("duration_sec"):
|
||||
parts.append(f"dur={meta['duration_sec']}s")
|
||||
if meta.get("width"):
|
||||
parts.append(f"{meta['width']}x{meta['height']}")
|
||||
if meta.get("fps"):
|
||||
parts.append(f"{meta['fps']}fps")
|
||||
if parts:
|
||||
print(f" {'':>4s} {'':8s} {'':12s} {'':10s} [{', '.join(parts)}]")
|
||||
if r.get("tags"):
|
||||
print(f" {'':>4s} {'':8s} {'':12s} {'':10s} #{', #'.join(r['tags'][:5])}")
|
||||
print(f" {'-'*80}")
|
||||
print(f" 共 {len(results)} 条结果\n")
|
||||
|
||||
def print_tag_cloud(self):
|
||||
"""打印标签云"""
|
||||
stats = self.tag_stats()
|
||||
if not stats:
|
||||
print(" 暂无标签")
|
||||
return
|
||||
max_count = max(s["count"] for s in stats) if stats else 1
|
||||
|
||||
print(f"\n {'标签':24s} {'频次':>6s} 分布")
|
||||
print(f" {'-'*60}")
|
||||
for s in stats:
|
||||
bar_len = int(s["count"] / max_count * 20)
|
||||
bar = "█" * bar_len + "░" * (20 - bar_len)
|
||||
print(f" {s['tag']:24s} {s['count']:>6d} {bar}")
|
||||
print(f" 共 {len(stats)} 个标签\n")
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 辅助函数
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
def _human_size(bytes_: int) -> str:
|
||||
if bytes_ < 1024:
|
||||
return f"{bytes_} B"
|
||||
elif bytes_ < 1024**2:
|
||||
return f"{bytes_/1024:.1f} KB"
|
||||
elif bytes_ < 1024**3:
|
||||
return f"{bytes_/(1024**2):.1f} MB"
|
||||
return f"{bytes_/(1024**3):.2f} GB"
|
||||
@@ -0,0 +1,421 @@
|
||||
"""
|
||||
assets_manager — 视频工厂资源库（数据层）
|
||||
==========================================
|
||||
|
||||
按《DCS 说明说明书 v2.0》第五节「资产库规范」实现。
|
||||
|
||||
目录结构（默认 `agent-local/tools/ave/assets/`）：
|
||||
characters/{char_id}/
|
||||
meta.json 元数据（名字/类型/用途/标签/一致性种子…）
|
||||
ref_front.png 正面参考
|
||||
ref_side.png 侧面
|
||||
ref_back.png 背面
|
||||
ref_full.png 全身
|
||||
expressions/ 表情图（happy/sad/angry…）
|
||||
wardrobe/{w_id}.png 服装变体
|
||||
voice_sample.mp3 音色样本
|
||||
locations/{loc_id}/ AI 生成场景
|
||||
real_locations/{loc_id}/ 真实场地
|
||||
products/{prod_id}/ 产品（含 must_keep）
|
||||
props/{prop_id}/ 道具
|
||||
voices/{voice_id}/ 音色
|
||||
bgm/ sfx/ vfx_presets/
|
||||
|
||||
设计要点：
|
||||
· 每个资产 = 一个目录（图/音/元数据分离，便于直接喂给 API）
|
||||
· meta.json 是唯一真相源（列表/筛选都读它）
|
||||
· 支持旧数据迁移（character_registry/registry.yaml → characters/）
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
|
||||
log = logging.getLogger("assets_manager")
|
||||
|
||||
# 资产类型 → 目录名
|
||||
ASSET_DIRS = {
|
||||
"characters": "角色",
|
||||
"locations": "场景",
|
||||
"real_locations": "真实场地",
|
||||
"products": "产品",
|
||||
"props": "道具",
|
||||
"wardrobe": "服装",
|
||||
"voices": "音色",
|
||||
"bgm": "BGM",
|
||||
"sfx": "音效",
|
||||
"ambience": "环境音",
|
||||
"vfx_presets": "特效预设",
|
||||
"materials": "素材",
|
||||
}
|
||||
|
||||
# 角色：类型 / 用途枚举
|
||||
CHAR_TYPES = {"real": "真人", "anime": "动漫", "3d": "3D"}
|
||||
CHAR_ROLES = {"lead": "主角", "support": "配角", "extra": "群演"}
|
||||
|
||||
|
||||
def assets_root() -> Path:
|
||||
"""资源库根目录（可用 AVE_ASSETS_DIR 覆盖）"""
|
||||
p = os.environ.get("AVE_ASSETS_DIR")
|
||||
if p:
|
||||
return Path(p)
|
||||
base = Path(os.environ.get("AGENT_LOCAL")
|
||||
or (Path.home() / "workbuddy-agent-os" / "agent-local"))
|
||||
return base / "tools" / "ave" / "assets"
|
||||
|
||||
|
||||
# 🔒 锁定后受保护的字段（关键一致性字段：改动会破坏角色一致性 / 版本可追溯）
|
||||
LOCK_PROTECTED_FIELDS = (
|
||||
"reference_images", "generation_history", "active_version",
|
||||
"consistency_seed", "voice", "locked",
|
||||
)
|
||||
|
||||
|
||||
def _slug(name: str) -> str:
|
||||
"""中文/特殊字符 → 安全 ID（保留字母数字与中文，其余转下划线）"""
|
||||
s = re.sub(r"[^\w\u4e00-\u9fff]+", "_", (name or "").strip())
|
||||
return s.strip("_") or f"item_{int(time.time())}"
|
||||
|
||||
|
||||
class AssetStore:
|
||||
"""通用资产仓库（一个子目录 = 一个资产）"""
|
||||
|
||||
def __init__(self, kind: str, root: Optional[Path] = None):
|
||||
if kind not in ASSET_DIRS:
|
||||
raise ValueError(f"未知资产类型: {kind}（可选: {list(ASSET_DIRS)}）")
|
||||
self.kind = kind
|
||||
self.dir = (root or assets_root()) / kind
|
||||
self.dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# ── 读 ─────────────────────────────────────────────
|
||||
|
||||
def _meta_path(self, aid: str) -> Path:
|
||||
return self.dir / aid / "meta.json"
|
||||
|
||||
def kind_dir(self, aid: str) -> Path:
|
||||
"""某资产的目录（`assets/<kind>/<aid>/`）
|
||||
|
||||
供外部写入文件用：参考图（`ref_front.png`）/ 音色（`voice_sample.mp3`）等。
|
||||
"""
|
||||
d = self.dir / aid
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
def get(self, aid: str) -> Optional[dict]:
|
||||
p = self._meta_path(aid)
|
||||
if not p.exists():
|
||||
return None
|
||||
try:
|
||||
d = json.loads(p.read_text(encoding="utf-8"))
|
||||
d.setdefault("id", aid)
|
||||
d["files"] = self._list_files(aid)
|
||||
return d
|
||||
except Exception as e:
|
||||
log.warning("读取 %s 失败: %s", p, e)
|
||||
return None
|
||||
|
||||
def _list_files(self, aid: str) -> dict:
|
||||
"""列出资产目录下的文件（供前端展示）"""
|
||||
d = self.dir / aid
|
||||
out: dict[str, list[str]] = {}
|
||||
if not d.exists():
|
||||
return out
|
||||
for f in sorted(d.rglob("*")):
|
||||
if f.is_file() and f.name != "meta.json":
|
||||
rel = str(f.relative_to(d))
|
||||
out.setdefault(f.parent.name if f.parent != d else "root", []).append(rel)
|
||||
return out
|
||||
|
||||
def list(self, q: str = "", filters: Optional[dict] = None,
|
||||
sort: str = "updated_desc", limit: int = 500) -> list[dict]:
|
||||
"""列表（带搜索/筛选/排序）
|
||||
|
||||
q: 关键词（匹配 name / tags / description）
|
||||
filters: 字段等值筛选（如 {"type": "real", "role": "lead"}）
|
||||
sort: updated_desc | updated_asc | name | usage
|
||||
"""
|
||||
filters = {k: v for k, v in (filters or {}).items() if v not in (None, "", [])}
|
||||
out: list[dict] = []
|
||||
if not self.dir.exists():
|
||||
return out
|
||||
for sub in sorted(self.dir.iterdir()):
|
||||
if not sub.is_dir():
|
||||
continue
|
||||
m = self.get(sub.name)
|
||||
if not m:
|
||||
continue
|
||||
if filters and any(str(m.get(k, "")) != str(v) for k, v in filters.items()):
|
||||
continue
|
||||
if q:
|
||||
hay = " ".join([
|
||||
str(m.get("name", "")), str(m.get("description", "")),
|
||||
" ".join(m.get("tags") or []), str(m.get("project", "")),
|
||||
]).lower()
|
||||
if q.lower() not in hay:
|
||||
continue
|
||||
out.append(m)
|
||||
if sort == "name":
|
||||
out.sort(key=lambda x: str(x.get("name", "")))
|
||||
elif sort == "usage":
|
||||
out.sort(key=lambda x: -int(x.get("usage_count") or 0))
|
||||
elif sort == "updated_asc":
|
||||
out.sort(key=lambda x: str(x.get("updated_at", "")))
|
||||
else: # updated_desc（默认）
|
||||
out.sort(key=lambda x: str(x.get("updated_at", "")), reverse=True)
|
||||
return out[:limit]
|
||||
|
||||
# ── 写 ─────────────────────────────────────────────
|
||||
|
||||
def create(self, data: dict, aid: Optional[str] = None) -> dict:
|
||||
# 优先用显式 id（调用方可指定稳定标识），否则由 name 生成
|
||||
# ⚠️ 原实现写成 name 优先 → 传入的 id 被忽略（曾导致上传时按 id 找不到资产）
|
||||
aid = aid or _slug(data.get("id") or data.get("name") or "")
|
||||
d = self.dir / aid
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
now = time.strftime("%Y-%m-%d %H:%M:%S")
|
||||
meta = {**data}
|
||||
meta["id"] = aid
|
||||
meta.setdefault("created_at", now)
|
||||
meta["updated_at"] = now
|
||||
meta.setdefault("usage_count", 0)
|
||||
(d / "meta.json").write_text(
|
||||
json.dumps(meta, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
log.info(" ➕ 新建 %s: %s", ASSET_DIRS[self.kind], aid)
|
||||
return self.get(aid) or meta
|
||||
|
||||
def update(self, aid: str, patch: dict) -> Optional[dict]:
|
||||
cur = self.get(aid)
|
||||
if not cur:
|
||||
return None
|
||||
# 🔒 锁定保护：已锁定的资产不允许修改「关键字段」（保证角色一致性 / 版本可追溯）
|
||||
if cur.get("locked"):
|
||||
blocked = [k for k in patch if k in LOCK_PROTECTED_FIELDS]
|
||||
if blocked:
|
||||
raise PermissionError(
|
||||
f"资产已锁定（{cur.get('locked_at') or ''}），不能修改：{'、'.join(blocked)}。"
|
||||
"如需修改请先解锁。")
|
||||
cur.pop("files", None)
|
||||
cur.update({k: v for k, v in patch.items() if k not in ("id", "created_at")})
|
||||
cur["updated_at"] = time.strftime("%Y-%m-%d %H:%M:%S")
|
||||
(self.dir / aid / "meta.json").write_text(
|
||||
json.dumps(cur, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
log.info(" ✏️ 更新 %s: %s", ASSET_DIRS[self.kind], aid)
|
||||
return self.get(aid)
|
||||
|
||||
# ── 🔒 锁定 / 版本记录（角色一致性保障）────────────────
|
||||
|
||||
def lock(self, aid: str, reason: str = "", by: str = "user") -> Optional[dict]:
|
||||
"""锁定资产 → 冻结关键字段（reference_images / 生成历史 / seed / 音色）
|
||||
|
||||
何时锁定：
|
||||
· 用户手动点「🔒 锁定」
|
||||
· 首次被流程引用（`bump_usage` 自动锁定）
|
||||
锁定后如需修改，必须先 `unlock()`（会记入 `unlock_history`，便于追溯）。
|
||||
"""
|
||||
cur = self.get(aid)
|
||||
if not cur:
|
||||
return None
|
||||
cur.pop("files", None)
|
||||
if cur.get("locked"):
|
||||
return self.get(aid)
|
||||
cur["locked"] = True
|
||||
cur["locked_at"] = time.strftime("%Y-%m-%d %H:%M:%S")
|
||||
cur["locked_reason"] = reason or "手动锁定"
|
||||
cur["locked_by"] = by
|
||||
(self.dir / aid / "meta.json").write_text(
|
||||
json.dumps(cur, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
log.info(" 🔒 锁定 %s: %s（%s）", ASSET_DIRS[self.kind], aid, cur["locked_reason"])
|
||||
return self.get(aid)
|
||||
|
||||
def unlock(self, aid: str, reason: str = "") -> Optional[dict]:
|
||||
"""解锁（改动会记入解锁历史，供版本追溯）"""
|
||||
cur = self.get(aid)
|
||||
if not cur or not cur.get("locked"):
|
||||
return self.get(aid) if cur else None
|
||||
cur.pop("files", None)
|
||||
hist = list(cur.get("unlock_history") or [])
|
||||
hist.append({
|
||||
"at": time.strftime("%Y-%m-%d %H:%M:%S"),
|
||||
"was_locked_at": cur.get("locked_at"),
|
||||
"reason": reason or "手动解锁",
|
||||
})
|
||||
cur["unlock_history"] = hist
|
||||
cur["locked"] = False
|
||||
cur.pop("locked_at", None)
|
||||
cur.pop("locked_reason", None)
|
||||
(self.dir / aid / "meta.json").write_text(
|
||||
json.dumps(cur, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
log.info(" 🔓 解锁 %s: %s", ASSET_DIRS[self.kind], aid)
|
||||
return self.get(aid)
|
||||
|
||||
def add_generation(self, aid: str, record: dict) -> Optional[dict]:
|
||||
"""追加一次**生成记录**（版本历史）—— 记录 prompt / seed / 模型 / 引擎
|
||||
|
||||
record 建议字段：
|
||||
prompt, negative_prompt, seed, engine, model, slot, file, params, at
|
||||
→ 自动编号 version，并更新 active_version + last_generation
|
||||
"""
|
||||
cur = self.get(aid)
|
||||
if not cur:
|
||||
return None
|
||||
cur.pop("files", None)
|
||||
hist = list(cur.get("generation_history") or [])
|
||||
rec = {**record}
|
||||
rec.setdefault("at", time.strftime("%Y-%m-%d %H:%M:%S"))
|
||||
rec["version"] = len(hist) + 1
|
||||
hist.append(rec)
|
||||
cur["generation_history"] = hist
|
||||
cur["active_version"] = rec["version"]
|
||||
cur["last_generation"] = rec
|
||||
# 若 record 带 slot + file → 同步写入参考图（**锁定状态下也允许新增/更新该槽位**，
|
||||
# 因为这是"追加新版本产物"，不是篡改已有设定）
|
||||
if rec.get("slot") and rec.get("file"):
|
||||
refs = dict(cur.get("reference_images") or {})
|
||||
refs[rec["slot"]] = rec["file"]
|
||||
cur["reference_images"] = refs
|
||||
# 记录 seed（供后续生成保持一致）
|
||||
if rec.get("seed") is not None and not cur.get("consistency_seed"):
|
||||
cur["consistency_seed"] = rec["seed"]
|
||||
cur["updated_at"] = time.strftime("%Y-%m-%d %H:%M:%S")
|
||||
(self.dir / aid / "meta.json").write_text(
|
||||
json.dumps(cur, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
log.info(" 📌 %s 生成记录 v%d（%s / %s / seed=%s）",
|
||||
aid, rec["version"], rec.get("engine"), rec.get("model"), rec.get("seed"))
|
||||
return self.get(aid)
|
||||
|
||||
def delete(self, aid: str, hard: bool = False) -> bool:
|
||||
d = self.dir / aid
|
||||
if not d.exists():
|
||||
return False
|
||||
if hard:
|
||||
shutil.rmtree(d)
|
||||
else: # 软删：移到 _trash/
|
||||
t = self.dir / "_trash"
|
||||
t.mkdir(exist_ok=True)
|
||||
shutil.move(str(d), str(t / f"{aid}_{int(time.time())}"))
|
||||
log.info(" 🗑 删除 %s: %s", ASSET_DIRS[self.kind], aid)
|
||||
return True
|
||||
|
||||
def bump_usage(self, aid: str) -> None:
|
||||
"""使用次数 +1（流程引用资产时调用）"""
|
||||
m = self.get(aid)
|
||||
if not m:
|
||||
return
|
||||
m.pop("files", None)
|
||||
m["usage_count"] = int(m.get("usage_count") or 0) + 1
|
||||
# 🆕 首次被使用 → 自动锁定（用户要求："角色开始使用就要锁定"，冻结关键字段保证一致性）
|
||||
if not m.get("locked"):
|
||||
m["locked"] = True
|
||||
m["locked_at"] = time.strftime("%Y-%m-%d %H:%M:%S")
|
||||
m["locked_reason"] = f"已被流程引用（第 {m['usage_count']} 次，自动锁定以保证角色一致性）"
|
||||
m["locked_by"] = "auto"
|
||||
(self.dir / aid / "meta.json").write_text(
|
||||
json.dumps(m, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
def save_file(self, aid: str, filename: str, content: bytes) -> Path:
|
||||
"""保存资产文件（参考图/音色等）"""
|
||||
d = self.dir / aid
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
sub = d / filename
|
||||
sub.parent.mkdir(parents=True, exist_ok=True)
|
||||
sub.write_bytes(content)
|
||||
return sub
|
||||
|
||||
|
||||
class CharacterStore(AssetStore):
|
||||
"""角色库（AssetStore 的特化：带类型/用途枚举、使用统计）"""
|
||||
|
||||
def __init__(self, root: Optional[Path] = None):
|
||||
super().__init__("characters", root)
|
||||
|
||||
def stats(self) -> dict:
|
||||
"""统计（给 UI 显示）"""
|
||||
items = self.list(limit=10000)
|
||||
by_type: dict[str, int] = {}
|
||||
by_role: dict[str, int] = {}
|
||||
with_voice = 0
|
||||
for m in items:
|
||||
by_type[str(m.get("type") or "real")] = by_type.get(str(m.get("type") or "real"), 0) + 1
|
||||
by_role[str(m.get("role") or "lead")] = by_role.get(str(m.get("role") or "lead"), 0) + 1
|
||||
if (m.get("voice") or {}).get("sample"):
|
||||
with_voice += 1
|
||||
return {"total": len(items), "by_type": by_type, "by_role": by_role,
|
||||
"with_voice": with_voice}
|
||||
|
||||
|
||||
def migrate_from_registry(registry_path: Optional[str] = None,
|
||||
root: Optional[Path] = None) -> dict:
|
||||
"""旧角色库（character_registry/registry.yaml）→ 新目录结构
|
||||
|
||||
保留原字段，映射到新 schema；已存在的角色不覆盖（除非 force）。
|
||||
"""
|
||||
import yaml
|
||||
scripts = Path(__file__).resolve().parent.parent # 09_ave/scripts/
|
||||
p = Path(registry_path) if registry_path else (scripts / "character_registry" / "registry.yaml")
|
||||
if not p.exists():
|
||||
return {"ok": False, "reason": f"旧角色库不存在: {p}"}
|
||||
|
||||
raw = yaml.safe_load(p.read_text(encoding="utf-8")) or {}
|
||||
chars = raw.get("characters") or {}
|
||||
store = CharacterStore(root)
|
||||
created, skipped = 0, 0
|
||||
|
||||
reg_dir = p.parent
|
||||
for name, c in chars.items():
|
||||
aid = _slug(name)
|
||||
if (store.dir / aid / "meta.json").exists():
|
||||
skipped += 1
|
||||
continue
|
||||
# 旧 reference_images → 新 reference_images
|
||||
ri = c.get("reference_images") or {}
|
||||
refs: dict[str, str] = {}
|
||||
for k in ("front", "side", "back", "full", "portrait", "full_body"):
|
||||
if ri.get(k):
|
||||
refs[k] = str(ri[k])
|
||||
body = ri.get("body") or {}
|
||||
for k, v in (body.items() if isinstance(body, dict) else []):
|
||||
if v:
|
||||
refs[f"body_{k}"] = str(v)
|
||||
store.create({
|
||||
"name": name,
|
||||
"type": "anime", # 旧库多为漫剧风，默认 anime（可改）
|
||||
"role": "lead",
|
||||
"project": "",
|
||||
"tags": list(c.get("preferred_bgm_tags") or []),
|
||||
"description": c.get("description") or "",
|
||||
"appearance": list(c.get("appearance") or []),
|
||||
"personality": c.get("personality") or "",
|
||||
"art_style": c.get("art_style") or "",
|
||||
"voice_style": c.get("voice_style") or "",
|
||||
"bgm_style": c.get("default_bgm_style") or "",
|
||||
"prompt_template": c.get("prompt_template") or {},
|
||||
"reference_images": refs,
|
||||
"consistency_seed": 42,
|
||||
"voice": {},
|
||||
"wardrobe": [],
|
||||
"lora": None,
|
||||
"_migrated_from": str(p),
|
||||
}, aid=aid)
|
||||
created += 1
|
||||
# 同时把旧图拷进新目录（如果存在）
|
||||
try:
|
||||
for k, rel in refs.items():
|
||||
src = (reg_dir / rel) if not Path(rel).is_absolute() else Path(rel)
|
||||
if src.exists() and src.is_file():
|
||||
store.save_file(aid, f"ref_{k}{src.suffix}", src.read_bytes())
|
||||
except Exception as e:
|
||||
log.debug("拷贝参考图失败 %s: %s", name, e)
|
||||
|
||||
log.info(" 迁移完成: 新建 %d / 跳过 %d（共 %d）", created, skipped, len(chars))
|
||||
return {"ok": True, "created": created, "skipped": skipped, "total": len(chars)}
|
||||
|
||||
|
||||
# 便捷入口
|
||||
def get_store(kind: str = "characters") -> AssetStore:
|
||||
return CharacterStore() if kind == "characters" else AssetStore(kind)
|
||||
@@ -0,0 +1,198 @@
|
||||
"""
|
||||
AudioLine — 音频创作管线
|
||||
|
||||
独立于 AVE 视频工厂的音频处理线。
|
||||
支持卡点改编、换词创作、风格迁移、Remix 等。
|
||||
|
||||
快速开始:
|
||||
from audio_line import AudioLine
|
||||
|
||||
# 1. 初始化管线
|
||||
al = AudioLine(config_path="config.yaml")
|
||||
|
||||
# 2. 从文件创建任务
|
||||
task = al.from_file("input.mp3", operation="cardio", target_bpm=128)
|
||||
|
||||
# 3. 执行全管线
|
||||
result = al.run(task)
|
||||
|
||||
# 4. 查看结果
|
||||
print(result.summary)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .orchestrator import AudioLineOrchestrator, AudioLineResult
|
||||
from .layer1_input.collector import AudioTask, AudioSource, AudioCollector
|
||||
from .layer2_analysis.beat_detector import BeatResult
|
||||
from .layer2_analysis.structure_parser import StructureResult
|
||||
from .layer2_analysis.emotion_analyzer import EmotionResult
|
||||
from .layer3_creation.module_a_rhythm import RhythmResult
|
||||
from .layer3_creation.module_b_lyrics import LyricsResult
|
||||
from .layer3_creation.module_c_melody import MelodyResult
|
||||
from .layer3_creation.module_d_mix import MixResult
|
||||
from .layer4_alignment.lyric_aligner import LyricTimeline
|
||||
from .layer5_output.exporter import ExportResult
|
||||
|
||||
|
||||
class AudioLine:
|
||||
"""
|
||||
AudioLine — 音频创作管线外部 API
|
||||
|
||||
封装 AudioLineOrchestrator，提供简洁的接口。
|
||||
支持从文件/URL/指令创建任务并执行全管线。
|
||||
"""
|
||||
|
||||
def __init__(self, config_path: str | Path | None = None):
|
||||
"""
|
||||
初始化管线
|
||||
|
||||
参数:
|
||||
config_path: 配置文件路径 (YAML)
|
||||
"""
|
||||
self._orchestrator = AudioLineOrchestrator(config_path)
|
||||
self._collector = self._orchestrator.collector
|
||||
|
||||
# ---- 任务创建 ----
|
||||
|
||||
def from_file(
|
||||
self,
|
||||
path: str | Path,
|
||||
operation: str = "auto",
|
||||
target_bpm: int = 0,
|
||||
target_key: str = "",
|
||||
style: str = "",
|
||||
topic: str = "",
|
||||
duration: float = 0.0,
|
||||
dry_run: bool = False,
|
||||
modules: set[str] | None = None,
|
||||
**kwargs,
|
||||
) -> AudioTask:
|
||||
"""
|
||||
从本地文件创建音频处理任务
|
||||
|
||||
参数:
|
||||
path: 音频文件路径
|
||||
operation: 操作类型 (auto/remix/rewrite/speed_up/cardio)
|
||||
target_bpm: 目标 BPM (0=自动)
|
||||
target_key: 目标调性
|
||||
style: 目标风格
|
||||
topic: 歌词主题 (换词时使用)
|
||||
duration: 目标时长 (秒)
|
||||
dry_run: 仅分析不创作
|
||||
modules: 模块子集
|
||||
|
||||
返回:
|
||||
AudioTask
|
||||
"""
|
||||
return self._collector.from_file(path, {
|
||||
"operation": operation,
|
||||
"target_bpm": target_bpm,
|
||||
"target_key": target_key,
|
||||
"style": style,
|
||||
"topic": topic,
|
||||
"duration": duration,
|
||||
"dry_run": dry_run,
|
||||
"modules": modules or {"separation", "analysis", "rhythm", "lyrics", "mix", "alignment", "export"},
|
||||
**kwargs,
|
||||
})
|
||||
|
||||
def from_url(
|
||||
self,
|
||||
url: str,
|
||||
**kwargs,
|
||||
) -> AudioTask:
|
||||
"""
|
||||
从 URL 下载并创建任务
|
||||
|
||||
参数同 from_file()
|
||||
"""
|
||||
return self._collector.from_url(url, kwargs)
|
||||
|
||||
def from_command(self, text: str) -> AudioTask:
|
||||
"""
|
||||
从自然语言指令创建任务
|
||||
|
||||
参数:
|
||||
text: 如 "把这首歌改成128BPM的卡点版"
|
||||
"""
|
||||
return self._collector.from_text_command(text)
|
||||
|
||||
# ---- 管线执行 ----
|
||||
|
||||
def run(self, task: AudioTask) -> AudioLineResult:
|
||||
"""
|
||||
执行音频处理管线
|
||||
|
||||
参数:
|
||||
task: AudioTask 任务对象
|
||||
|
||||
返回:
|
||||
AudioLineResult
|
||||
"""
|
||||
return self._orchestrator.run(task)
|
||||
|
||||
def run_file(
|
||||
self,
|
||||
path: str | Path,
|
||||
**kwargs,
|
||||
) -> AudioLineResult:
|
||||
"""
|
||||
快速执行: 从文件到结果一步完成
|
||||
|
||||
参数:
|
||||
path: 音频文件路径
|
||||
**kwargs: 传给 from_file() 的额外参数
|
||||
|
||||
返回:
|
||||
AudioLineResult
|
||||
"""
|
||||
task = self.from_file(path, **kwargs)
|
||||
return self._orchestrator.run(task)
|
||||
|
||||
# ---- 单模块执行 ----
|
||||
|
||||
@property
|
||||
def config(self) -> dict:
|
||||
"""获取当前配置"""
|
||||
return self._orchestrator.config
|
||||
|
||||
@property
|
||||
def stem_separator(self):
|
||||
"""直接访问音轨分离器"""
|
||||
return self._orchestrator.stem_sep
|
||||
|
||||
@property
|
||||
def beat_detector(self):
|
||||
"""直接访问节拍检测器"""
|
||||
return self._orchestrator.beat_det
|
||||
|
||||
@property
|
||||
def structure_parser(self):
|
||||
"""直接访问结构解析器"""
|
||||
return self._orchestrator.structure_parser
|
||||
|
||||
@property
|
||||
def emotion_analyzer(self):
|
||||
"""直接访问情绪分析器"""
|
||||
return self._orchestrator.emotion_analyzer
|
||||
|
||||
|
||||
__all__ = [
|
||||
"AudioLine",
|
||||
"AudioLineOrchestrator",
|
||||
"AudioLineResult",
|
||||
"AudioTask",
|
||||
"AudioSource",
|
||||
"BeatResult",
|
||||
"StructureResult",
|
||||
"EmotionResult",
|
||||
"RhythmResult",
|
||||
"LyricsResult",
|
||||
"MelodyResult",
|
||||
"MixResult",
|
||||
"LyricTimeline",
|
||||
"ExportResult",
|
||||
]
|
||||
@@ -0,0 +1,85 @@
|
||||
# audio_line —— 音频创作管线配置
|
||||
# 复制到 agent-local/tools/ave/config/local.yaml 并填入真实 API 密钥
|
||||
# 优先级: local.yaml > config.yaml > 默认值
|
||||
|
||||
# === 输入/输出路径 ===
|
||||
paths:
|
||||
work_dir: "./work/audio_line" # 临时工作目录
|
||||
output_dir: "./output/audio_line" # 成品输出目录
|
||||
model_cache: "./cache/models" # 模型缓存
|
||||
|
||||
# === 音轨分离 ===
|
||||
separation:
|
||||
model: "htdemucs_ft" # Demucs 模型: htdemucs | htdemucs_ft | htdemucs_6s
|
||||
device: "cpu" # cpu | mps (M1/M2/M3 GPU)
|
||||
shifts: 2 # 推理增强次数 (≥1), >1 提高质量但慢
|
||||
overlap: 0.25 # 分帧重叠率
|
||||
segments: "default" # 分帧长度 "default" | int
|
||||
|
||||
# === BPM/节拍检测 ===
|
||||
beat:
|
||||
min_bpm: 60 # 最小 BPM
|
||||
max_bpm: 200 # 最大 BPM
|
||||
onset_backend: "librosa" # 节拍 onset 后端
|
||||
downbeat_model: "madmom" # 重拍检测模型 (madmom | librosa)
|
||||
|
||||
# === 节奏重塑 (Mubert API) ===
|
||||
mubert:
|
||||
api_key: "" # Mubert API Key
|
||||
base_url: "https://api.mubert.com/v2"
|
||||
default_duration: 30 # 默认生成时长(秒)
|
||||
default_genre: "electronic" # 默认风格
|
||||
default_bpm: 128 # 默认 BPM
|
||||
max_retries: 3
|
||||
timeout: 60
|
||||
|
||||
# === 歌词创作 (oMLX 本地 LLM) ===
|
||||
lyrics:
|
||||
llm_url: "http://localhost:8000/v1" # oMLX 地址
|
||||
llm_model: "Qwen3-8B" # 可用模型
|
||||
max_tokens: 1024
|
||||
temperature: 0.8
|
||||
# 提示词模板
|
||||
rewrite_prompt: >
|
||||
你是一位专业歌词改编者。请根据以下要求对原曲歌词进行二次创作：
|
||||
主题: {topic}
|
||||
风格: {style}
|
||||
保留原曲的段落结构(Intro/Verse/Chorus/Bridge/Outro)和押韵模式。
|
||||
输出格式: 每段以[Intro][Verse][Chorus]标记开头。
|
||||
lyric_analysis_prompt: >
|
||||
分析以下歌词的段落结构、押韵模式和情感走向。
|
||||
输出 JSON 格式。
|
||||
|
||||
# === 旋律编曲 (Suno API) ===
|
||||
suno:
|
||||
api_key: "" # Suno API Key
|
||||
base_url: "https://api.suno.ai/v1"
|
||||
default_duration: 30 # 默认生成时长
|
||||
model: "suno-v4" # Suno 模型版本
|
||||
max_retries: 2
|
||||
timeout: 120
|
||||
|
||||
# === 混音母带 ===
|
||||
mixing:
|
||||
target_loudness: -14.0 # LUFS 目标响度 (Spotify: -14, YouTube: -14, Apple: -16)
|
||||
headroom: 1.0 # 峰值头上空间 (dB)
|
||||
sample_rate: 44100 # 输出采样率
|
||||
bit_depth: 16 # 输出位深
|
||||
eq_preset: "default" # EQ 预设
|
||||
compressor_threshold: -20 # 压缩器阈值 (dB)
|
||||
compressor_ratio: 3.0 # 压缩比
|
||||
compressor_attack: 5 # 启动时间 (ms)
|
||||
compressor_release: 100 # 释放时间 (ms)
|
||||
|
||||
# === 对齐 ===
|
||||
alignment:
|
||||
time_tolerance: 0.1 # 时间对准容差 (秒)
|
||||
crossfade: 0.05 # 交叉淡入淡出时长 (秒)
|
||||
stretch_method: "phase_vocoder" # 时间伸缩方法: phase_vocoder | wsola | rubberband
|
||||
|
||||
# === 输出 ===
|
||||
export:
|
||||
formats: ["wav", "mp3"] # 默认导出格式
|
||||
mp3_bitrate: "320k" # MP3 比特率
|
||||
include_stems: true # 是否同时导出分轨
|
||||
stem_labels: ["vocals", "drums", "bass", "other", "full_mix"]
|
||||
@@ -0,0 +1,52 @@
|
||||
"""
|
||||
layer1_input/_parser.py — 自然语言指令解析
|
||||
|
||||
将用户的口语化指令（如"把这首歌改快一点、加卡点"）解析为 AudioTask。
|
||||
当前用规则匹配，后续可接入 LLM 增强。
|
||||
"""
|
||||
from .collector import AudioTask, AudioSource
|
||||
|
||||
|
||||
def parse_command_text(text: str) -> AudioTask:
|
||||
"""解析自然语言指令 → AudioTask (不含音频源)"""
|
||||
text = text.strip().lower()
|
||||
|
||||
modules = {"separation", "analysis", "rhythm", "lyrics", "mix"}
|
||||
operation = "auto"
|
||||
target_bpm = 0
|
||||
|
||||
# — 检测操作类型 —
|
||||
if any(kw in text for kw in ["卡点", "变速", "改快", "改慢"]):
|
||||
operation = "cardio"
|
||||
elif any(kw in text for kw in ["换词", "改词", "填词", "翻唱"]):
|
||||
operation = "rewrite"
|
||||
elif any(kw in text for kw in ["remix", "改编", "混音"]):
|
||||
operation = "remix"
|
||||
elif any(kw in text for kw in ["加速", "慢速", "变速度"]):
|
||||
operation = "speed_up"
|
||||
|
||||
# — 检测 BPM —
|
||||
import re
|
||||
bpm_match = re.search(r"(\d+)\s*bpm", text)
|
||||
if bpm_match:
|
||||
target_bpm = int(bpm_match.group(1))
|
||||
|
||||
# — 模块选择 —
|
||||
if "只分析" in text or "仅分析" in text:
|
||||
modules = {"separation", "analysis"}
|
||||
|
||||
# — 主题/风格提取 (简单规则) —
|
||||
style = ""
|
||||
for kw, st in [("可爱", "可爱"), ("伤感", "伤感"), ("燃", "燃系"), ("DJ", "电音"),
|
||||
("古风", "古风"), ("摇滚", "摇滚"), ("爵士", "爵士")]:
|
||||
if kw in text:
|
||||
style = st
|
||||
break
|
||||
|
||||
return AudioTask(
|
||||
source=AudioSource(path="", source_type="command"),
|
||||
operation=operation,
|
||||
target_bpm=target_bpm,
|
||||
style=style,
|
||||
modules=modules,
|
||||
)
|
||||
@@ -0,0 +1,133 @@
|
||||
"""
|
||||
layer1_input/collector.py — 输入收集模块
|
||||
|
||||
职责：接收各种来源的输入，标准化为统一的 AudioTask 数据类。
|
||||
支持：本地文件 / URL下载 / 文本指令（如"把这首歌改成卡点版"）
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
import urllib.request
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 数据类
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class AudioSource:
|
||||
"""标准化音频源描述"""
|
||||
path: str # 本地路径
|
||||
original_filename: str = "" # 原始文件名
|
||||
source_type: Literal["file", "url", "command"] = "file"
|
||||
duration: float = 0.0 # 时长 (秒)，预读取后填充
|
||||
sample_rate: int = 0 # 采样率，预读取后填充
|
||||
|
||||
|
||||
@dataclass
|
||||
class AudioTask:
|
||||
"""完整的音频处理任务"""
|
||||
# ——— 源信息 ———
|
||||
source: AudioSource
|
||||
|
||||
# ——— 创作参数 ———
|
||||
operation: Literal["remix", "rewrite", "speed_up", "cardio", "auto"] = "auto"
|
||||
target_bpm: int = 0 # 0 = 自动
|
||||
target_key: str = "" # 空 = 保持原调
|
||||
style: str = "" # 目标风格描述
|
||||
topic: str = "" # 歌词主题 (换词时使用)
|
||||
duration: float = 0.0 # 目标时长 (0 = 与原曲相同)
|
||||
|
||||
# ——— 控制标记 ———
|
||||
modules: set[str] = field(default_factory=lambda: {
|
||||
"separation", "analysis", "rhythm", "lyrics", "mix"
|
||||
}) # 要执行的模块集合
|
||||
dry_run: bool = False # 仅分析，不执行创作模块
|
||||
|
||||
# ——— 元信息 ———
|
||||
task_id: str = ""
|
||||
created_at: str = ""
|
||||
metadata: dict = field(default_factory=dict)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 收集器
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class AudioCollector:
|
||||
"""输入收集器 — 接受文件/URL/命令 → AudioTask"""
|
||||
|
||||
SUPPORTED_FORMATS = {".mp3", ".wav", ".flac", ".m4a", ".aac", ".ogg", ".wma"}
|
||||
|
||||
def __init__(self, work_dir: str | Path = "./work/audio_line"):
|
||||
self.work_dir = Path(work_dir)
|
||||
self.work_dir.mkdir(parents=True, exist_ok=True)
|
||||
self._upload_counter = 0
|
||||
|
||||
# ---- 公开 API ----
|
||||
|
||||
def from_file(self, path: str | Path, task_kwargs: dict | None = None) -> AudioTask:
|
||||
"""从本地文件创建任务"""
|
||||
path = Path(path)
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"音频文件不存在: {path}")
|
||||
if path.suffix.lower() not in self.SUPPORTED_FORMATS:
|
||||
raise ValueError(f"不支持的格式 '{path.suffix}', 支持: {self.SUPPORTED_FORMATS}")
|
||||
|
||||
source = AudioSource(
|
||||
path=str(path.resolve()),
|
||||
original_filename=path.name,
|
||||
source_type="file",
|
||||
)
|
||||
return self._build_task(source, task_kwargs)
|
||||
|
||||
def from_url(self, url: str, task_kwargs: dict | None = None) -> AudioTask:
|
||||
"""从 URL 下载并创建任务"""
|
||||
self._upload_counter += 1
|
||||
ext = self._guess_ext_from_url(url)
|
||||
dest = self.work_dir / f"download_{self._upload_counter}{ext}"
|
||||
|
||||
print(f" [collector] 下载: {url}")
|
||||
urllib.request.urlretrieve(url, dest)
|
||||
print(f" [collector] 已保存: {dest}")
|
||||
|
||||
source = AudioSource(
|
||||
path=str(dest.resolve()),
|
||||
original_filename=dest.name,
|
||||
source_type="url",
|
||||
)
|
||||
return self._build_task(source, task_kwargs)
|
||||
|
||||
def from_text_command(self, text: str) -> AudioTask:
|
||||
"""从自然语言指令创建任务（不含音频源，需后续绑定）"""
|
||||
# 简单规则解析；生产环境可接入 LLM
|
||||
from ._parser import parse_command_text
|
||||
parsed = parse_command_text(text)
|
||||
return parsed
|
||||
|
||||
# ---- 内部 ----
|
||||
|
||||
def _build_task(self, source: AudioSource, kwargs: dict | None = None) -> AudioTask:
|
||||
kwargs = kwargs or {}
|
||||
return AudioTask(
|
||||
source=source,
|
||||
operation=kwargs.pop("operation", "auto"),
|
||||
target_bpm=kwargs.pop("target_bpm", 0),
|
||||
target_key=kwargs.pop("target_key", ""),
|
||||
style=kwargs.pop("style", ""),
|
||||
topic=kwargs.pop("topic", ""),
|
||||
duration=kwargs.pop("duration", 0.0),
|
||||
modules=kwargs.pop("modules", {"separation", "analysis", "rhythm", "lyrics", "mix"}),
|
||||
dry_run=kwargs.pop("dry_run", False),
|
||||
metadata=kwargs,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _guess_ext_from_url(url: str) -> str:
|
||||
path = urllib.parse.urlparse(url).path
|
||||
ext = Path(path).suffix.lower()
|
||||
return ext if ext in AudioCollector.SUPPORTED_FORMATS else ".mp3"
|
||||
+180
@@ -0,0 +1,180 @@
|
||||
"""
|
||||
layer2_analysis/beat_detector.py — 节拍/BPM 检测器
|
||||
|
||||
输入：原始音频路径
|
||||
输出：BPM、节拍时间戳、重拍位置、节拍置信度
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import librosa
|
||||
|
||||
|
||||
@dataclass
|
||||
class BeatResult:
|
||||
"""节拍分析结果"""
|
||||
bpm: float # 检测到的 BPM
|
||||
beats: list[float] = field(default_factory=list) # 节拍时间戳 (秒)
|
||||
downbeats: list[float] = field(default_factory=list) # 重拍时间戳 (秒)
|
||||
onset_frames: list[int] = field(default_factory=list) # 起音帧
|
||||
onset_times: list[float] = field(default_factory=list) # 起音时间 (秒)
|
||||
beat_frames: list[int] = field(default_factory=list) # 节拍帧索引
|
||||
confidence: float = 0.0 # BPM 置信度 (0~1)
|
||||
tempo_stability: float = 0.0 # 节奏稳定性 (0~1)
|
||||
time_signature: str = "4/4" # 拍号推断
|
||||
segment_bpms: list[dict] = field(default_factory=list) # [{start, end, bpm}, ...]
|
||||
|
||||
@property
|
||||
def beat_count(self) -> int:
|
||||
return len(self.beats)
|
||||
|
||||
@property
|
||||
def duration_sec(self) -> float:
|
||||
return self.beats[-1] if self.beats else 0.0
|
||||
|
||||
|
||||
class BeatDetector:
|
||||
"""Librosa 节拍检测器"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
min_bpm: float = 60,
|
||||
max_bpm: float = 200,
|
||||
onset_backend: str = "librosa",
|
||||
):
|
||||
self.min_bpm = min_bpm
|
||||
self.max_bpm = max_bpm
|
||||
self.onset_backend = onset_backend
|
||||
|
||||
def detect(self, audio_path: str | Path) -> BeatResult:
|
||||
"""
|
||||
分析音频的 BPM / 节拍 / 重拍
|
||||
|
||||
参数:
|
||||
audio_path: 音频文件路径
|
||||
|
||||
返回:
|
||||
BeatResult 包含完整节拍分析
|
||||
"""
|
||||
audio_path = Path(audio_path)
|
||||
print(f" [beat_det] 分析: {audio_path.name}")
|
||||
|
||||
# 加载音频 (librosa 自动重采样到 22050)
|
||||
y, sr = librosa.load(str(audio_path), sr=None, mono=True)
|
||||
|
||||
# — 1. BPM 检测 —
|
||||
tempo, beat_frames = librosa.beat.beat_track(
|
||||
y=y, sr=sr,
|
||||
min_bpm=self.min_bpm,
|
||||
max_bpm=self.max_bpm,
|
||||
units="frames",
|
||||
)
|
||||
bpm = float(tempo)
|
||||
|
||||
# — 2. 节拍时间戳 —
|
||||
beat_times = librosa.frames_to_time(beat_frames, sr=sr).tolist()
|
||||
|
||||
# — 3. 起音检测 —
|
||||
onset_frames = librosa.onset.onset_detect(y=y, sr=sr, units="frames")
|
||||
onset_times = librosa.frames_to_time(onset_frames, sr=sr).tolist()
|
||||
|
||||
# — 4. 重拍检测 (使用 librosa 的 beat tracker 中的第一拍检测) —
|
||||
# librosa 不直接提供 downbeat, 我们通过 beat 强度和周期性估算
|
||||
# 方法：取每小节第一拍
|
||||
beat_strength = self._estimate_beat_strength(y, sr, beat_frames)
|
||||
downbeats = []
|
||||
if len(beat_times) > 4:
|
||||
# 取最强节拍作为小节的起始
|
||||
# 近似: 假设 4/4 拍，每 4 拍一个重拍
|
||||
period = self._estimate_bar_period(beat_times)
|
||||
downbeats = beat_times[::period] if period > 0 else []
|
||||
|
||||
# — 5. 置信度 —
|
||||
confidence = self._calc_confidence(y, sr, bpm, onset_frames)
|
||||
|
||||
# — 6. 节奏稳定性 —
|
||||
stability = self._calc_stability(beat_times)
|
||||
|
||||
# — 7. 分段 BPM —
|
||||
seg_bpms = self._segment_bpm(y, sr, beat_times)
|
||||
|
||||
print(f" [beat_det] ✓ BPM={bpm:.1f} 节拍={len(beat_times)} 重拍={len(downbeats)} "
|
||||
f"置信度={confidence:.2f}")
|
||||
|
||||
return BeatResult(
|
||||
bpm=bpm,
|
||||
beats=beat_times,
|
||||
downbeats=downbeats,
|
||||
onset_frames=onset_frames.tolist(),
|
||||
onset_times=onset_times,
|
||||
beat_frames=beat_frames.tolist(),
|
||||
confidence=confidence,
|
||||
tempo_stability=stability,
|
||||
segment_bpms=seg_bpms,
|
||||
)
|
||||
|
||||
# ---- 内部方法 ----
|
||||
|
||||
def _estimate_beat_strength(self, y, sr, beat_frames) -> list[float]:
|
||||
"""估算每个节拍的强度 (用频谱通量)"""
|
||||
onset_env = librosa.onset.onset_strength(y=y, sr=sr)
|
||||
strengths = []
|
||||
for f in beat_frames:
|
||||
idx = min(f, len(onset_env) - 1)
|
||||
strengths.append(float(onset_env[idx]))
|
||||
return strengths
|
||||
|
||||
def _estimate_bar_period(self, beat_times: list[float]) -> int:
|
||||
"""估算每小节拍数"""
|
||||
if len(beat_times) < 8:
|
||||
return 4
|
||||
# 最差选择 4 (4/4 拍最常见)
|
||||
return 4
|
||||
|
||||
def _calc_confidence(self, y, sr, bpm, onset_frames) -> float:
|
||||
"""BPM 置信度评估"""
|
||||
# 更规则 → 更高置信度
|
||||
if len(onset_frames) < 5:
|
||||
return 0.3
|
||||
# 简单启发式: 起音密度接近 BPM/60
|
||||
duration = len(y) / sr
|
||||
onset_density = len(onset_frames) / duration if duration > 0 else 0
|
||||
expected_density = bpm / 60
|
||||
ratio = min(onset_density / expected_density, 1.0) if expected_density > 0 else 0.5
|
||||
return min(0.4 + 0.6 * ratio, 1.0)
|
||||
|
||||
def _calc_stability(self, beat_times: list[float]) -> float:
|
||||
"""节奏稳定性: 节拍间隔的变异系数"""
|
||||
if len(beat_times) < 4:
|
||||
return 0.5
|
||||
intervals = [beat_times[i+1] - beat_times[i] for i in range(len(beat_times)-1)]
|
||||
mean_interval = sum(intervals) / len(intervals)
|
||||
if mean_interval == 0:
|
||||
return 0.0
|
||||
variance = sum((i - mean_interval) ** 2 for i in intervals) / len(intervals)
|
||||
cv = (variance ** 0.5) / mean_interval
|
||||
return max(0.0, min(1.0, 1.0 - cv))
|
||||
|
||||
def _segment_bpm(self, y, sr, beat_times: list[float],
|
||||
segment_len: float = 10.0) -> list[dict]:
|
||||
"""按时间窗口分段分析 BPM"""
|
||||
if len(beat_times) < 4:
|
||||
return []
|
||||
total_duration = len(y) / sr
|
||||
segments = []
|
||||
t = 0.0
|
||||
while t < total_duration:
|
||||
end = min(t + segment_len, total_duration)
|
||||
seg_beats = [b for b in beat_times if t <= b < end]
|
||||
if len(seg_beats) >= 4:
|
||||
intervals = [seg_beats[i+1] - seg_beats[i] for i in range(len(seg_beats)-1)]
|
||||
mean_int = sum(intervals) / len(intervals)
|
||||
seg_bpm = 60.0 / mean_int if mean_int > 0 else 0
|
||||
else:
|
||||
seg_bpm = 0
|
||||
segments.append({"start": round(t, 2), "end": round(end, 2), "bpm": round(seg_bpm, 1)})
|
||||
t = end
|
||||
return segments
|
||||
+166
@@ -0,0 +1,166 @@
|
||||
"""
|
||||
layer2_analysis/emotion_analyzer.py — 情绪/风格分析
|
||||
|
||||
输入：Stem 分离后的 vocals + 原曲
|
||||
输出：情绪标签、风格描述、BPM 范围推荐
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import librosa
|
||||
import numpy as np
|
||||
|
||||
|
||||
@dataclass
|
||||
class EmotionResult:
|
||||
"""情绪分析结果"""
|
||||
mood: str = "neutral" # 主要情绪 (happy/sad/energetic/calm/neutral)
|
||||
energy: float = 0.5 # 能量值 0~1
|
||||
valence: float = 0.5 # 效价 (积极度) 0~1
|
||||
danceability: float = 0.5 # 舞曲性 0~1
|
||||
acousticness: float = 0.5 # 原声性 0~1
|
||||
tempo_category: str = "medium" # 速度分类: slow/medium/fast
|
||||
description: str = "" # 自然语言风格描述
|
||||
dominant_frequencies: dict = field(default_factory=dict) # 频率分布
|
||||
spectral_features: dict = field(default_factory=dict) # 频谱特征
|
||||
|
||||
|
||||
class EmotionAnalyzer:
|
||||
"""
|
||||
音频情绪分析器
|
||||
|
||||
基于频谱特征 + 节奏特征进行音乐情绪分类。
|
||||
纯 librosa 实现，不依赖外部模型。
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.mood_map: dict[str, tuple[float, float, float]] = {
|
||||
# (energy_low, valence_low, dance_low) → label
|
||||
"energetic": (0.6, 0.5, 0.6),
|
||||
"happy": (0.4, 0.7, 0.5),
|
||||
"calm": (0.2, 0.4, 0.2),
|
||||
"sad": (0.2, 0.2, 0.2),
|
||||
"aggressive": (0.8, 0.3, 0.4),
|
||||
"neutral": (0.5, 0.5, 0.5),
|
||||
}
|
||||
|
||||
def analyze(self, audio_path: str | Path, bpm: float = 0.0) -> EmotionResult:
|
||||
"""
|
||||
分析音频情绪
|
||||
|
||||
参数:
|
||||
audio_path: 音频路径 (可以是原始音频或 vocals stem)
|
||||
bpm: 可选的 BPM 值 (已检测到的)
|
||||
|
||||
返回:
|
||||
EmotionResult
|
||||
"""
|
||||
audio_path = Path(audio_path)
|
||||
print(f" [emotion] 分析: {audio_path.name}")
|
||||
|
||||
y, sr = librosa.load(str(audio_path), sr=None, mono=True)
|
||||
|
||||
# — 1. 频谱特征 —
|
||||
spectral_centroids = librosa.feature.spectral_centroid(y=y, sr=sr)[0]
|
||||
spectral_rolloff = librosa.feature.spectral_rolloff(y=y, sr=sr)[0]
|
||||
spectral_bandwidth = librosa.feature.spectral_bandwidth(y=y, sr=sr)[0]
|
||||
zero_crossing_rate = librosa.feature.zero_crossing_rate(y)[0]
|
||||
rms = librosa.feature.rms(y=y)[0]
|
||||
|
||||
# — 2. 计算特征指标 —
|
||||
avg_centroid = float(np.mean(spectral_centroids))
|
||||
avg_rolloff = float(np.mean(spectral_rolloff))
|
||||
avg_bandwidth = float(np.mean(spectral_bandwidth))
|
||||
avg_zcr = float(np.mean(zero_crossing_rate))
|
||||
avg_rms = float(np.mean(rms))
|
||||
|
||||
# — 3. 情绪分类 —
|
||||
# energy: RMS + ZCR + centroid
|
||||
energy = min(1.0, (avg_rms * 5 + avg_zcr * 2 + avg_centroid / 5000) / 3)
|
||||
energy = max(0.0, min(1.0, energy))
|
||||
|
||||
# valence (积极度): ZCR 高 → 积极, centroid 高 → 明亮
|
||||
valence = min(1.0, (avg_zcr * 3 + avg_centroid / 3000) / 2)
|
||||
valence = max(0.0, min(1.0, valence))
|
||||
|
||||
# danceability: 节奏清晰度 (BPM 在 100-130 之间最可舞)
|
||||
if bpm > 0:
|
||||
dance = 1.0 - abs(bpm - 120) / 80
|
||||
else:
|
||||
dance = 0.5 - abs(energy - 0.5) * 0.3
|
||||
danceability = max(0.0, min(1.0, dance))
|
||||
|
||||
# acousticness: 原声性 (low bandwidth + low centroid)
|
||||
acousticness = max(0.0, 1.0 - (avg_bandwidth / 3000 + avg_centroid / 8000) / 2)
|
||||
|
||||
# — 4. 情绪标签 —
|
||||
mood = self._classify_mood(energy, valence, danceability)
|
||||
|
||||
# — 5. 速度分类 —
|
||||
if bpm > 0:
|
||||
tempo_category = "fast" if bpm >= 120 else ("slow" if bpm <= 80 else "medium")
|
||||
else:
|
||||
tempo_category = "medium"
|
||||
|
||||
# — 6. 自然语言描述 —
|
||||
description = self._describe(mood, energy, tempo_category)
|
||||
|
||||
# — 7. 频谱分布 —
|
||||
dominant_freqs = self._dominant_frequencies(spectral_centroids)
|
||||
spec_features = {
|
||||
"avg_centroid_hz": round(avg_centroid, 1),
|
||||
"avg_bandwidth_hz": round(avg_bandwidth, 1),
|
||||
"avg_zcr": round(float(avg_zcr), 4),
|
||||
"avg_rms": round(float(avg_rms), 4),
|
||||
}
|
||||
|
||||
return EmotionResult(
|
||||
mood=mood,
|
||||
energy=round(energy, 3),
|
||||
valence=round(valence, 3),
|
||||
danceability=round(danceability, 3),
|
||||
acousticness=round(acousticness, 3),
|
||||
tempo_category=tempo_category,
|
||||
description=description,
|
||||
dominant_frequencies=dominant_freqs,
|
||||
spectral_features=spec_features,
|
||||
)
|
||||
|
||||
# ---- 内部 ----
|
||||
|
||||
def _classify_mood(self, energy: float, valence: float, dance: float) -> str:
|
||||
"""基于能量/效价/舞曲性分类情绪"""
|
||||
scores = {}
|
||||
for label, (e_th, v_th, d_th) in self.mood_map.items():
|
||||
score = (
|
||||
(1.0 - abs(energy - e_th)) * 0.4 +
|
||||
(1.0 - abs(valence - v_th)) * 0.4 +
|
||||
(1.0 - abs(dance - d_th)) * 0.2
|
||||
)
|
||||
scores[label] = score
|
||||
return max(scores, key=scores.get)
|
||||
|
||||
def _describe(self, mood: str, energy: float, tempo: str) -> str:
|
||||
"""生成自然语言风格描述"""
|
||||
descs = {
|
||||
"energetic": f"充满活力的{tempo}速曲风，能量充沛",
|
||||
"happy": f"明快愉悦的{tempo}速曲风，情绪积极",
|
||||
"calm": f"舒缓平静的{tempo}速曲风，适合放松",
|
||||
"sad": f"忧伤低沉的{tempo}速曲风，情绪内敛",
|
||||
"aggressive": f"强劲有力的{tempo}速曲风，表现力强",
|
||||
"neutral": f"中性均衡的{tempo}速曲风",
|
||||
}
|
||||
return descs.get(mood, f"{mood}风格的{tempo}速曲风")
|
||||
|
||||
def _dominant_frequencies(self, centroids: np.ndarray) -> dict:
|
||||
"""分析频谱能量分布"""
|
||||
mean_c = float(np.mean(centroids))
|
||||
std_c = float(np.std(centroids))
|
||||
return {
|
||||
"mean_hz": round(mean_c, 1),
|
||||
"std_hz": round(std_c, 1),
|
||||
"range": f"{max(0, round(mean_c - std_c, 1))}-{round(mean_c + std_c, 1)} Hz",
|
||||
}
|
||||
+98
@@ -0,0 +1,98 @@
|
||||
"""
|
||||
layer2_analysis/stem_separator.py — Demucs 音轨分离
|
||||
|
||||
输入：原始音频文件路径
|
||||
输出：分离后的分轨字典 {vocals, drums, bass, other, ...}
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
|
||||
class StemSeparator:
|
||||
"""Demucs 音轨分离器"""
|
||||
|
||||
STEM_NAMES = ["vocals", "drums", "bass", "other"]
|
||||
MODEL_OPTIONS = ["htdemucs", "htdemucs_ft", "htdemucs_6s"]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
model: str = "htdemucs_ft",
|
||||
device: str = "cpu",
|
||||
shifts: int = 2,
|
||||
overlap: float = 0.25,
|
||||
segments: str = "default",
|
||||
out_dir: str | Path | None = None,
|
||||
):
|
||||
if model not in self.MODEL_OPTIONS:
|
||||
raise ValueError(f"不支持的模型 '{model}', 可选: {self.MODEL_OPTIONS}")
|
||||
self.model = model
|
||||
self.device = device
|
||||
self.shifts = shifts
|
||||
self.overlap = overlap
|
||||
self.segments = segments
|
||||
self.out_dir = Path(out_dir) if out_dir else Path(tempfile.mkdtemp(prefix="demucs_"))
|
||||
|
||||
def separate(self, audio_path: str | Path) -> dict[str, str]:
|
||||
"""
|
||||
执行音轨分离
|
||||
|
||||
参数:
|
||||
audio_path: 输入音频路径
|
||||
|
||||
返回:
|
||||
{stem_name: output_path, ...}
|
||||
"""
|
||||
audio_path = Path(audio_path)
|
||||
if not audio_path.exists():
|
||||
raise FileNotFoundError(f"音频文件不存在: {audio_path}")
|
||||
|
||||
print(f" [stem_sep] 模型={self.model} 设备={self.device}")
|
||||
print(f" [stem_sep] 输入: {audio_path}")
|
||||
|
||||
# 构建 Demucs CLI 命令
|
||||
output_dir = self.out_dir / "separated" / self.model
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
cmd = (
|
||||
f"python3 -m demucs.separate"
|
||||
f" --name {self.model}"
|
||||
f" --device {self.device}"
|
||||
f" --shifts {self.shifts}"
|
||||
f" --overlap {self.overlap}"
|
||||
f" -o {self.out_dir.resolve()}"
|
||||
f" \"{audio_path.resolve()}\""
|
||||
)
|
||||
|
||||
os.system(cmd)
|
||||
|
||||
# Demucs 输出: {out_dir}/{model}/{input_stem}/
|
||||
input_stem = audio_path.stem
|
||||
track_dir = output_dir / input_stem
|
||||
|
||||
# 收集分轨路径
|
||||
stems = {}
|
||||
for name in self.STEM_NAMES:
|
||||
stem_path = track_dir / f"{name}.wav"
|
||||
if stem_path.exists():
|
||||
stems[name] = str(stem_path.resolve())
|
||||
else:
|
||||
# 尝试 .mp3
|
||||
stem_path_mp3 = track_dir / f"{name}.mp3"
|
||||
if stem_path_mp3.exists():
|
||||
stems[name] = str(stem_path_mp3.resolve())
|
||||
|
||||
print(f" [stem_sep] ✓ 分离完成: {len(stems)} 轨")
|
||||
for name, path in stems.items():
|
||||
size_mb = Path(path).stat().st_size / 1_048_576
|
||||
print(f" {name}: {Path(path).name} ({size_mb:.1f} MB)")
|
||||
|
||||
return stems
|
||||
|
||||
@classmethod
|
||||
def available_models(cls) -> list[str]:
|
||||
"""列出可用模型"""
|
||||
return cls.MODEL_OPTIONS
|
||||
+207
@@ -0,0 +1,207 @@
|
||||
"""
|
||||
layer2_analysis/structure_parser.py — 歌曲结构解析
|
||||
|
||||
输入：原始音频 + 节拍分析结果
|
||||
输出：段落标记 {intro, verse, chorus, bridge, outro, ...}
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import librosa
|
||||
import numpy as np
|
||||
|
||||
|
||||
@dataclass
|
||||
class StructureResult:
|
||||
"""歌曲结构解析结果"""
|
||||
segments: list[dict] = field(default_factory=list) # [{label, start, end, confidence}]
|
||||
section_count: int = 0
|
||||
has_chorus: bool = False
|
||||
has_bridge: bool = False
|
||||
structure_summary: str = "" # 如 "Intro→Verse→Chorus→Verse→Chorus→Bridge→Outro"
|
||||
|
||||
@property
|
||||
def section_labels(self) -> list[str]:
|
||||
return [s["label"] for s in self.segments]
|
||||
|
||||
|
||||
class StructureParser:
|
||||
"""
|
||||
歌曲结构解析器
|
||||
|
||||
基于频谱特征变化 + MFCC 聚类进行段落分割。
|
||||
不依赖外部模型，纯 librosa 实现。
|
||||
"""
|
||||
|
||||
SECTION_TAGS = {
|
||||
0: "Intro", 1: "Verse", 2: "Chorus",
|
||||
3: "Bridge", 4: "Outro", 5: "Instrumental",
|
||||
}
|
||||
|
||||
def __init__(self, n_segments: int = 8, min_section_sec: float = 4.0):
|
||||
self.n_segments = n_segments # 期望段落段数
|
||||
self.min_section_sec = min_section_sec # 最短段落时长
|
||||
|
||||
def parse(self, audio_path: str | Path, beat_times: list[float] | None = None) -> StructureResult:
|
||||
"""
|
||||
解析歌曲结构
|
||||
|
||||
参数:
|
||||
audio_path: 音频路径
|
||||
beat_times: 可选，节拍时间戳用于辅助对齐
|
||||
|
||||
返回:
|
||||
StructureResult
|
||||
"""
|
||||
audio_path = Path(audio_path)
|
||||
print(f" [structure] 分析: {audio_path.name}")
|
||||
|
||||
y, sr = librosa.load(str(audio_path), sr=None, mono=True)
|
||||
duration = len(y) / sr
|
||||
|
||||
# — 1. 提取 MFCC 序列 —
|
||||
mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=13, hop_length=512)
|
||||
mfcc_delta = librosa.feature.delta(mfcc)
|
||||
features = np.vstack([mfcc, mfcc_delta]) # (26, n_frames)
|
||||
|
||||
# — 2. 频谱特征 —
|
||||
spectral_centroid = librosa.feature.spectral_centroid(y=y, sr=sr, hop_length=512)
|
||||
spectral_contrast = librosa.feature.spectral_contrast(y=y, sr=sr, hop_length=512)
|
||||
|
||||
# 对齐到相同帧数
|
||||
min_frames = min(features.shape[1], spectral_centroid.shape[1], spectral_contrast.shape[1])
|
||||
features = features[:, :min_frames]
|
||||
spec_feats = np.vstack([
|
||||
spectral_centroid[:, :min_frames],
|
||||
spectral_contrast[:, :min_frames],
|
||||
])
|
||||
all_features = np.vstack([features, spec_feats]) # (~33, n_frames)
|
||||
|
||||
# — 3. 计算自相似矩阵 + 结构边界 —
|
||||
n_fft = all_features.shape[1]
|
||||
# 用滑动窗口计算相似度变化
|
||||
window = max(int(n_fft * 0.1), 4)
|
||||
novelty = librosa.onset.onset_strength(
|
||||
y=y, sr=sr, hop_length=512
|
||||
)
|
||||
|
||||
# — 4. 频谱变化点检测 —
|
||||
# 用 MFCC 的 novelty 曲线找变化点
|
||||
mfcc_novelty = self._compute_novelty(mfcc)
|
||||
change_points = self._find_peaks(mfcc_novelty, n_fft, self.n_segments)
|
||||
|
||||
# — 5. 节拍对齐 —
|
||||
change_times = librosa.frames_to_time(change_points, sr=sr, hop_length=512)
|
||||
if beat_times:
|
||||
change_times = self._snap_to_beats(change_times, beat_times)
|
||||
|
||||
# — 6. 段落标记 —
|
||||
segments = self._label_segments(change_times, duration, all_features)
|
||||
|
||||
# — 7. 结构分析 —
|
||||
summary = "→".join(s["label"] for s in segments)
|
||||
|
||||
print(f" [structure] ✓ 结构: {summary}")
|
||||
|
||||
return StructureResult(
|
||||
segments=segments,
|
||||
section_count=len(segments),
|
||||
has_chorus=any(s["label"] == "Chorus" for s in segments),
|
||||
has_bridge=any(s["label"] == "Bridge" for s in segments),
|
||||
structure_summary=summary,
|
||||
)
|
||||
|
||||
# ---- 内部 ----
|
||||
|
||||
def _compute_novelty(self, features: np.ndarray, kernel_size: int = 3) -> np.ndarray:
|
||||
"""计算 MFCC 的新颖性曲线"""
|
||||
# 局部相似度矩阵的对角线核卷积
|
||||
sim = librosa.segment.recurrence_matrix(features, mode="affinity")
|
||||
kernel = np.zeros(kernel_size * 2 + 1)
|
||||
kernel[:kernel_size] = -1
|
||||
kernel[kernel_size] = 0
|
||||
kernel[kernel_size + 1:] = 1
|
||||
from scipy.ndimage import convolve1d
|
||||
novelty = convolve1d(sim.mean(axis=0), kernel, mode="constant")
|
||||
return novelty
|
||||
|
||||
def _find_peaks(self, novelty: np.ndarray, max_frames: int, n_peaks: int) -> np.ndarray:
|
||||
"""找 novelty 曲线峰值作为段落边界"""
|
||||
from scipy.signal import find_peaks
|
||||
if len(novelty) == 0:
|
||||
return np.array([0, max_frames - 1])
|
||||
peaks, properties = find_peaks(novelty, distance=max_frames // max(n_peaks, 2))
|
||||
if len(peaks) < 2:
|
||||
return np.array([0, max_frames - 1])
|
||||
|
||||
# 取最强的 n_peaks 个边界
|
||||
strengths = properties.get("peak_heights", np.ones(len(peaks)))
|
||||
top_idx = np.argsort(strengths)[-n_peaks:]
|
||||
top_peaks = np.sort(peaks[top_idx])
|
||||
# 补首尾
|
||||
return np.unique(np.concatenate([[0], top_peaks, [max_frames - 1]]))
|
||||
|
||||
def _snap_to_beats(self, times: list[float], beats: list[float]) -> list[float]:
|
||||
"""将段落边界对齐到最近的节拍"""
|
||||
snapped = []
|
||||
for t in times:
|
||||
nearest = min(beats, key=lambda b: abs(b - t)) if beats else t
|
||||
snapped.append(nearest)
|
||||
return snapped
|
||||
|
||||
def _label_segments(self, boundaries: list[float], duration: float,
|
||||
features: np.ndarray) -> list[dict]:
|
||||
"""为每个段落打标签"""
|
||||
segments = []
|
||||
for i in range(len(boundaries) - 1):
|
||||
start = round(boundaries[i], 2)
|
||||
end = round(boundaries[i + 1], 2)
|
||||
if end - start < self.min_section_sec:
|
||||
continue
|
||||
|
||||
# 用 MFCC 特征中心 + 能量判断标签
|
||||
energy = self._segment_energy(features, boundaries, i)
|
||||
label = self._infer_label(i, len(boundaries), energy)
|
||||
|
||||
segments.append({
|
||||
"label": label,
|
||||
"start": start,
|
||||
"end": end,
|
||||
"confidence": 0.7, # 固定置信度，后续可改进
|
||||
})
|
||||
|
||||
if not segments:
|
||||
segments.append({"label": "Full", "start": 0.0, "end": round(duration, 2),
|
||||
"confidence": 1.0})
|
||||
|
||||
return segments
|
||||
|
||||
def _segment_energy(self, features: np.ndarray, boundaries: list[float],
|
||||
idx: int) -> float:
|
||||
"""估算段落能量"""
|
||||
# 简化为基于索引位置的启发式
|
||||
total = len(boundaries) - 1
|
||||
# Chorus 通常在中间偏后，能量最高
|
||||
center = total / 2
|
||||
distance_from_center = abs(idx - center)
|
||||
energy = 1.0 - (distance_from_center / max(center, 1)) * 0.5
|
||||
return max(0.3, energy)
|
||||
|
||||
def _infer_label(self, idx: int, total: int, energy: float) -> str:
|
||||
"""基于位置启发式推断段落标签"""
|
||||
if total <= 2:
|
||||
return "Full"
|
||||
if idx == 0:
|
||||
return "Intro"
|
||||
if idx == total - 2:
|
||||
return "Bridge" if total > 4 else "Outro"
|
||||
if idx == total - 1:
|
||||
return "Outro"
|
||||
# Chorus 在中间偏后位置
|
||||
mid = total / 2
|
||||
if abs(idx - mid) < 1.5 and energy > 0.6:
|
||||
return "Chorus"
|
||||
return "Verse"
|
||||
@@ -0,0 +1,12 @@
|
||||
"""AudioLine 创作层 — 节奏/歌词/旋律/混音"""
|
||||
from .module_a_rhythm import RhythmModule, RhythmResult
|
||||
from .module_b_lyrics import LyricsModule, LyricsResult
|
||||
from .module_c_melody import MelodyModule, MelodyResult
|
||||
from .module_d_mix import MixModule, MixResult
|
||||
|
||||
__all__ = [
|
||||
"RhythmModule", "RhythmResult",
|
||||
"LyricsModule", "LyricsResult",
|
||||
"MelodyModule", "MelodyResult",
|
||||
"MixModule", "MixResult",
|
||||
]
|
||||
+227
@@ -0,0 +1,227 @@
|
||||
"""
|
||||
layer3_creation/module_a_rhythm.py — 节奏重塑模块
|
||||
|
||||
功能：基于 Mubert/Mubert-like API 生成节奏/鼓点轨道
|
||||
卡点改编的核心：原曲鼓点替换 → BPM 匹配 → 节奏增强
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from lib.http import urlopen as _noproxy_urlopen # 禁代理+重试
|
||||
|
||||
|
||||
@dataclass
|
||||
class RhythmResult:
|
||||
"""节奏生成结果"""
|
||||
drum_track_path: str = "" # 生成的鼓点音频路径
|
||||
bpm: float = 0.0 # 实际 BPM
|
||||
genre: str = "" # 使用的风格
|
||||
duration: float = 0.0 # 时长 (秒)
|
||||
api_used: str = "" # 使用的 API
|
||||
cost: float = 0.0 # API 调用成本
|
||||
|
||||
|
||||
class RhythmModule:
|
||||
"""
|
||||
节奏重塑模块
|
||||
|
||||
支持两种模式:
|
||||
1. Mubert API — 在线生成高质量鼓点 (需要 API key)
|
||||
2. 本地替代 — 用 librosa 节拍合成简单鼓点 (免费)
|
||||
"""
|
||||
|
||||
def __init__(self, config: dict | None = None):
|
||||
self.config = config or {}
|
||||
self.api_key = self.config.get("mubert", {}).get("api_key", "")
|
||||
self.base_url = self.config.get("mubert", {}).get(
|
||||
"base_url", "https://api.mubert.com/v2"
|
||||
)
|
||||
self.default_genre = self.config.get("mubert", {}).get(
|
||||
"default_genre", "electronic"
|
||||
)
|
||||
self.default_duration = self.config.get("mubert", {}).get(
|
||||
"default_duration", 30
|
||||
)
|
||||
self.timeout = self.config.get("mubert", {}).get("timeout", 60)
|
||||
|
||||
def generate(
|
||||
self,
|
||||
bpm: float,
|
||||
genre: str = "",
|
||||
duration: float = 0.0,
|
||||
output_path: str | Path | None = None,
|
||||
) -> RhythmResult:
|
||||
"""
|
||||
生成节奏/鼓点轨道
|
||||
|
||||
参数:
|
||||
bpm: 目标 BPM
|
||||
genre: 风格 (electronic/house/techno/hiphop/pop/rock)
|
||||
duration: 目标时长 (秒), 0=默认
|
||||
output_path: 输出路径
|
||||
|
||||
返回:
|
||||
RhythmResult
|
||||
"""
|
||||
genre = genre or self.default_genre
|
||||
duration = duration if duration > 0 else self.default_duration
|
||||
output_path = Path(output_path or f"./output/rhythm_{int(bpm)}_{genre}.wav")
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
print(f" [rhythm] 目标: BPM={bpm} 风格={genre} 时长={duration}s")
|
||||
|
||||
if self.api_key:
|
||||
result = self._via_mubert(bpm, genre, duration, output_path)
|
||||
else:
|
||||
print(f" [rhythm] 无 Mubert API key, 使用本地合成替代")
|
||||
result = self._local_fallback(bpm, duration, output_path)
|
||||
|
||||
if output_path.exists():
|
||||
result.drum_track_path = str(output_path.resolve())
|
||||
size_mb = output_path.stat().st_size / 1_048_576
|
||||
print(f" [rhythm] ✓ 输出: {output_path.name} ({size_mb:.1f} MB)")
|
||||
else:
|
||||
print(f" [rhythm] ⚠ 输出文件不存在")
|
||||
|
||||
return result
|
||||
|
||||
# ---- Mubert API ----
|
||||
|
||||
def _via_mubert(
|
||||
self, bpm: float, genre: str, duration: float, output_path: Path
|
||||
) -> RhythmResult:
|
||||
"""通过 Mubert API 生成"""
|
||||
result = RhythmResult(bpm=bpm, genre=genre, duration=duration, api_used="mubert")
|
||||
|
||||
try:
|
||||
# Mubert API: POST /v2/track/render
|
||||
payload = json.dumps({
|
||||
"pat": self.api_key,
|
||||
"bpm": int(bpm),
|
||||
"genre": genre,
|
||||
"duration": int(duration),
|
||||
"format": "wav",
|
||||
}).encode("utf-8")
|
||||
|
||||
req = urllib.request.Request(
|
||||
f"{self.base_url}/track/render",
|
||||
data=payload,
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
|
||||
with _noproxy_urlopen(req, timeout=self.timeout) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
# 下载生成的音频
|
||||
if "data" in data and "url" in data["data"]:
|
||||
audio_url = data["data"]["url"]
|
||||
urllib.request.urlretrieve(audio_url, output_path)
|
||||
result.cost = data.get("data", {}).get("cost", 0.0)
|
||||
|
||||
except (urllib.error.HTTPError, urllib.error.URLError, json.JSONDecodeError) as e:
|
||||
print(f" [rhythm] Mubert API 请求失败: {e}")
|
||||
print(f" [rhythm] 回退到本地合成")
|
||||
return self._local_fallback(bpm, duration, output_path)
|
||||
|
||||
return result
|
||||
|
||||
# ---- 本地替代方案 ----
|
||||
|
||||
def _local_fallback(
|
||||
self, bpm: float, duration: float, output_path: Path
|
||||
) -> RhythmResult:
|
||||
"""
|
||||
本地合成简单鼓点节奏
|
||||
|
||||
用 librosa 的 clicks 功能生成节拍点击声，
|
||||
配合简单的频谱叠加模拟 kick + snare + hi-hat。
|
||||
"""
|
||||
import librosa
|
||||
import numpy as np
|
||||
import soundfile as sf
|
||||
|
||||
sr = 22050
|
||||
total_samples = int(duration * sr)
|
||||
beat_interval = int(60.0 / bpm * sr) # 每拍采样数
|
||||
half_beat = beat_interval // 2
|
||||
|
||||
# — 构建鼓点模式 (4/4 拍: kick on 1,3, snare on 2,4, hi-hat 8th notes) —
|
||||
audio = np.zeros(total_samples, dtype=np.float32)
|
||||
|
||||
for beat_idx in range(int(duration * bpm / 60) + 1):
|
||||
pos = beat_idx * beat_interval
|
||||
if pos >= total_samples:
|
||||
break
|
||||
|
||||
beat_in_bar = beat_idx % 4
|
||||
|
||||
# Kick: beats 0, 2 (1st and 3rd beat)
|
||||
if beat_in_bar in (0, 2):
|
||||
kick = self._make_kick(sr, int(beat_interval * 0.4))
|
||||
end = min(pos + len(kick), total_samples)
|
||||
audio[pos:end] += kick[:end - pos] * 1.0
|
||||
|
||||
# Snare: beats 1, 3 (2nd and 4th beat)
|
||||
if beat_in_bar in (1, 3):
|
||||
snare = self._make_snare(sr, int(beat_interval * 0.3))
|
||||
end = min(pos + len(snare), total_samples)
|
||||
audio[pos:end] += snare[:end - pos] * 0.8
|
||||
|
||||
# Hi-hat: every half beat
|
||||
for off in (0, half_beat):
|
||||
hh_pos = pos + off
|
||||
if hh_pos < total_samples:
|
||||
hh = self._make_hihat(sr, int(beat_interval * 0.15))
|
||||
end = min(hh_pos + len(hh), total_samples)
|
||||
audio[hh_pos:end] += hh[:end - hh_pos] * 0.3
|
||||
|
||||
# 归一化
|
||||
peak = np.max(np.abs(audio))
|
||||
if peak > 0:
|
||||
audio = audio / peak * 0.9
|
||||
|
||||
sf.write(str(output_path), audio, sr)
|
||||
return RhythmResult(
|
||||
bpm=bpm,
|
||||
genre="local_fallback",
|
||||
duration=duration,
|
||||
drum_track_path=str(output_path.resolve()),
|
||||
api_used="local",
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _make_kick(sr: int, length: int) -> np.ndarray:
|
||||
"""合成 kick 鼓声"""
|
||||
import numpy as np
|
||||
t = np.arange(length) / sr
|
||||
# 频率指数衰减 150→40Hz
|
||||
freq = 150 * np.exp(-t * 25)
|
||||
phase = 2 * np.pi * np.cumsum(freq) / sr
|
||||
return np.sin(phase) * np.exp(-t * 30)
|
||||
|
||||
@staticmethod
|
||||
def _make_snare(sr: int, length: int) -> np.ndarray:
|
||||
"""合成 snare 鼓声"""
|
||||
import numpy as np
|
||||
t = np.arange(length) / sr
|
||||
tone = np.sin(2 * np.pi * 200 * t)
|
||||
noise = np.random.randn(length) * 0.5
|
||||
return (tone + noise) * np.exp(-t * 20)
|
||||
|
||||
@staticmethod
|
||||
def _make_hihat(sr: int, length: int) -> np.ndarray:
|
||||
"""合成 hi-hat 声"""
|
||||
import numpy as np
|
||||
t = np.arange(length) / sr
|
||||
noise = np.random.randn(length)
|
||||
# 高通滤波
|
||||
filtered = noise * np.exp(-t * 60)
|
||||
return filtered * 0.5
|
||||
+174
@@ -0,0 +1,174 @@
|
||||
"""
|
||||
layer3_creation/module_b_lyrics.py — 歌词创作模块
|
||||
|
||||
功能：基于 oMLX 本地 LLM 进行歌词创作
|
||||
核心场景：换词 (rewrite) — 保持原曲结构，替换内容
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from lib.http import urlopen as _noproxy_urlopen # 禁代理+重试
|
||||
|
||||
|
||||
@dataclass
|
||||
class LyricsResult:
|
||||
"""歌词创作结果"""
|
||||
original_lyrics: str = "" # 原歌词
|
||||
rewritten_lyrics: str = "" # 改写后歌词
|
||||
structure: list[dict] = field(default_factory=list) # [{section, lines}]
|
||||
analysis: dict = field(default_factory=dict) # 段落分析 JSON
|
||||
model_used: str = "" # 使用的模型
|
||||
token_count: int = 0 # 消耗 token 数
|
||||
|
||||
|
||||
class LyricsModule:
|
||||
"""
|
||||
歌词创作模块
|
||||
|
||||
使用 oMLX 本地 LLM 进行:
|
||||
- 歌词分析 (结构/押韵/情感)
|
||||
- 歌词改写 (换词创作)
|
||||
- 风格迁移
|
||||
"""
|
||||
|
||||
DEFAULT_PROMPTS = {
|
||||
"rewrite": (
|
||||
"你是一位专业歌词改编者。请根据以下要求对原曲歌词进行二次创作：\n"
|
||||
"主题: {topic}\n风格: {style}\n"
|
||||
"保留原曲的段落结构(Intro/Verse/Chorus/Bridge/Outro)和押韵模式。\n"
|
||||
"输出格式: 每段以[Intro][Verse][Chorus]标记开头。"
|
||||
),
|
||||
"analyze": (
|
||||
"分析以下歌词的段落结构、押韵模式和情感走向。\n"
|
||||
"输出 JSON 格式，包含: sections, rhyme_scheme, emotion_flow。"
|
||||
),
|
||||
}
|
||||
|
||||
def __init__(self, config: dict | None = None):
|
||||
self.config = config or {}
|
||||
self.llm_url = self.config.get("lyrics", {}).get(
|
||||
"llm_url", "http://localhost:8000/v1"
|
||||
)
|
||||
self.llm_model = self.config.get("lyrics", {}).get(
|
||||
"llm_model", "Qwen3-8B"
|
||||
)
|
||||
self.max_tokens = self.config.get("lyrics", {}).get("max_tokens", 1024)
|
||||
self.temperature = self.config.get("lyrics", {}).get("temperature", 0.8)
|
||||
self.rewrite_template = self.config.get(
|
||||
"lyrics", {}
|
||||
).get("rewrite_prompt", self.DEFAULT_PROMPTS["rewrite"])
|
||||
|
||||
def rewrite(
|
||||
self,
|
||||
original_lyrics: str,
|
||||
topic: str = "",
|
||||
style: str = "",
|
||||
output_path: str | Path | None = None,
|
||||
) -> LyricsResult:
|
||||
"""
|
||||
歌词改写
|
||||
|
||||
参数:
|
||||
original_lyrics: 原歌词文本
|
||||
topic: 新主题
|
||||
style: 目标风格
|
||||
output_path: 输出路径 (可选)
|
||||
|
||||
返回:
|
||||
LyricsResult
|
||||
"""
|
||||
print(f" [lyrics] 改写创作: topic='{topic}' style='{style}'")
|
||||
|
||||
prompt = self.rewrite_template.format(topic=topic or "相同主题", style=style or "原风格")
|
||||
messages = [
|
||||
{"role": "system", "content": "你是专业歌词作家。只输出歌词内容，不要额外解释。"},
|
||||
{"role": "user", "content": f"{prompt}\n\n原歌词:\n{original_lyrics}"},
|
||||
]
|
||||
rewritten = self._call_llm(messages)
|
||||
|
||||
result = LyricsResult(
|
||||
original_lyrics=original_lyrics,
|
||||
rewritten_lyrics=rewritten,
|
||||
model_used=self.llm_model,
|
||||
)
|
||||
|
||||
if output_path:
|
||||
Path(output_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(output_path, "w", encoding="utf-8") as f:
|
||||
f.write(rewritten)
|
||||
print(f" [lyrics] ✓ 已保存: {output_path}")
|
||||
|
||||
print(f" [lyrics] ✓ 改写完成 ({len(rewritten)} 字符)")
|
||||
return result
|
||||
|
||||
def analyze(self, lyrics: str) -> LyricsResult:
|
||||
"""
|
||||
分析歌词结构
|
||||
|
||||
返回 JSON 格式的段落分析
|
||||
"""
|
||||
print(f" [lyrics] 分析结构")
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": "你是一个歌词分析专家。只输出 JSON。"},
|
||||
{"role": "user", "content": f"{self.DEFAULT_PROMPTS['analyze']}\n\n歌词:\n{lyrics}"},
|
||||
]
|
||||
|
||||
response = self._call_llm(messages)
|
||||
|
||||
analysis = {}
|
||||
try:
|
||||
# 尝试解析 JSON
|
||||
start = response.find("{")
|
||||
end = response.rfind("}") + 1
|
||||
if start >= 0 and end > start:
|
||||
analysis = json.loads(response[start:end])
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
analysis = {"raw": response}
|
||||
|
||||
return LyricsResult(
|
||||
original_lyrics=lyrics,
|
||||
analysis=analysis,
|
||||
model_used=self.llm_model,
|
||||
)
|
||||
|
||||
# ---- LLM 调用 ----
|
||||
|
||||
def _call_llm(self, messages: list[dict]) -> str:
|
||||
"""调用 oMLX API"""
|
||||
payload = json.dumps({
|
||||
"model": self.llm_model,
|
||||
"messages": messages,
|
||||
"max_tokens": self.max_tokens,
|
||||
"temperature": self.temperature,
|
||||
}).encode("utf-8")
|
||||
|
||||
try:
|
||||
req = urllib.request.Request(
|
||||
f"{self.llm_url}/chat/completions",
|
||||
data=payload,
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
with _noproxy_urlopen(req, timeout=30) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
content = data["choices"][0]["message"]["content"]
|
||||
self._last_tokens = data.get("usage", {}).get("total_tokens", 0)
|
||||
return content
|
||||
|
||||
except (urllib.error.HTTPError, urllib.error.URLError,
|
||||
json.JSONDecodeError, KeyError) as e:
|
||||
print(f" [lyrics] oMLX API 调用失败: {e}")
|
||||
print(f" [lyrics] 请确保 oMLX Server 已启动 (localhost:8000)")
|
||||
return ""
|
||||
|
||||
@property
|
||||
def last_tokens(self) -> int:
|
||||
"""上次调用的 token 数"""
|
||||
return getattr(self, "_last_tokens", 0)
|
||||
+135
@@ -0,0 +1,135 @@
|
||||
"""
|
||||
layer3_creation/module_c_melody.py — 旋律编曲模块
|
||||
|
||||
功能：基于 Suno API 进行旋律改编/Remix
|
||||
核心场景：曲风迁移、旋律重编、Remix
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from lib.http import urlopen as _noproxy_urlopen # 禁代理+重试
|
||||
|
||||
|
||||
@dataclass
|
||||
class MelodyResult:
|
||||
"""旋律编曲结果"""
|
||||
output_path: str = ""
|
||||
style: str = ""
|
||||
duration: float = 0.0
|
||||
api_used: str = ""
|
||||
cost: float = 0.0
|
||||
task_id: str = ""
|
||||
|
||||
|
||||
class MelodyModule:
|
||||
"""
|
||||
旋律编曲模块
|
||||
|
||||
使用 Suno API 进行旋律创作/Remix。
|
||||
注意: 需要有效的 Suno API key 才能使用在线服务。
|
||||
无 key 时回退到本地音频拼接 (仅保留结构，不生成新旋律)。
|
||||
"""
|
||||
|
||||
def __init__(self, config: dict | None = None):
|
||||
self.config = config or {}
|
||||
self.api_key = self.config.get("suno", {}).get("api_key", "")
|
||||
self.base_url = self.config.get("suno", {}).get(
|
||||
"base_url", "https://api.suno.ai/v1"
|
||||
)
|
||||
self.model = self.config.get("suno", {}).get("model", "suno-v4")
|
||||
self.timeout = self.config.get("suno", {}).get("timeout", 120)
|
||||
self.max_retries = self.config.get("suno", {}).get("max_retries", 2)
|
||||
|
||||
def remix(
|
||||
self,
|
||||
audio_path: str | Path,
|
||||
target_style: str = "",
|
||||
duration: float = 0.0,
|
||||
output_path: str | Path | None = None,
|
||||
) -> MelodyResult:
|
||||
"""
|
||||
旋律改编
|
||||
|
||||
参数:
|
||||
audio_path: 原曲音频路径
|
||||
target_style: 目标风格 (如 "jazz", "electronic", "orchestral")
|
||||
duration: 目标时长
|
||||
output_path: 输出路径
|
||||
|
||||
返回:
|
||||
MelodyResult
|
||||
"""
|
||||
audio_path = Path(audio_path)
|
||||
output_path = Path(
|
||||
output_path or f"./output/melody_{audio_path.stem}_{target_style or 'remix'}.wav"
|
||||
)
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
print(f" [melody] 编曲: style='{target_style}' duration={duration}s")
|
||||
|
||||
if self.api_key:
|
||||
return self._via_suno(audio_path, target_style, duration, output_path)
|
||||
else:
|
||||
print(f" [melody] 无 Suno API key, 跳过旋律编曲")
|
||||
print(f" [melody] 提示: module_c_melody 需要 Suno API key 才能工作")
|
||||
return MelodyResult(
|
||||
output_path=str(output_path) if output_path.exists() else "",
|
||||
style=target_style,
|
||||
api_used="none",
|
||||
)
|
||||
|
||||
def _via_suno(
|
||||
self, audio_path: Path, style: str, duration: float, output_path: Path
|
||||
) -> MelodyResult:
|
||||
"""通过 Suno API 进行 Remix"""
|
||||
for attempt in range(self.max_retries):
|
||||
try:
|
||||
# Suno API: POST /v1/generate/remix
|
||||
payload = json.dumps({
|
||||
"model": self.model,
|
||||
"audio_url": str(audio_path.resolve()),
|
||||
"style": style or "pop",
|
||||
"duration": int(duration) if duration > 0 else 30,
|
||||
"format": "wav",
|
||||
}).encode("utf-8")
|
||||
|
||||
req = urllib.request.Request(
|
||||
f"{self.base_url}/generate/remix",
|
||||
data=payload,
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {self.api_key}",
|
||||
},
|
||||
method="POST",
|
||||
)
|
||||
|
||||
with _noproxy_urlopen(req, timeout=self.timeout) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
# 下载生成的音频
|
||||
audio_url = data.get("data", {}).get("url", "")
|
||||
if audio_url:
|
||||
urllib.request.urlretrieve(audio_url, output_path)
|
||||
|
||||
return MelodyResult(
|
||||
output_path=str(output_path.resolve()) if output_path.exists() else "",
|
||||
style=style,
|
||||
duration=duration,
|
||||
api_used="suno",
|
||||
task_id=data.get("data", {}).get("id", ""),
|
||||
)
|
||||
|
||||
except (urllib.error.HTTPError, urllib.error.URLError,
|
||||
json.JSONDecodeError) as e:
|
||||
print(f" [melody] Suno API 失败 (尝试 {attempt+1}/{self.max_retries}): {e}")
|
||||
if attempt < self.max_retries - 1:
|
||||
time.sleep(2)
|
||||
|
||||
return MelodyResult(style=style, api_used="suno_failed")
|
||||
@@ -0,0 +1,291 @@
|
||||
"""
|
||||
layer3_creation/module_d_mix.py — 混音母带模块
|
||||
|
||||
功能：将多轨音频(vocals/drums/bass/other)混音为完整立体声
|
||||
支持 EQ / 压缩 / 限制器 / 混响 / 响度归一化
|
||||
|
||||
基于 Pedalboard (Spotify 开源音频处理库)。
|
||||
回退方案：无 Pedalboard 时使用 pydub + numpy 基本叠加。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
@dataclass
|
||||
class MixResult:
|
||||
"""混音母带结果"""
|
||||
master_path: str = "" # 混音后主轨路径
|
||||
stem_paths: dict[str, str] = field(default_factory=dict) # 处理后分轨路径
|
||||
loudness: float = 0.0 # 最终 LUFS 响度
|
||||
peak_dB: float = 0.0 # 峰值 (dB)
|
||||
headroom: float = 0.0 # 头上空间 (dB)
|
||||
duration: float = 0.0 # 时长 (秒)
|
||||
engine: str = "" # 使用的引擎 (pedalboard | pydub | numpy)
|
||||
|
||||
|
||||
class MixModule:
|
||||
"""
|
||||
混音母带模块
|
||||
|
||||
处理流程：
|
||||
1. 加载各分轨音频
|
||||
2. 电平平衡 (Volume/gain staging)
|
||||
3. EQ 均衡 (可选预设)
|
||||
4. 压缩 (可选)
|
||||
5. 混响 (可选)
|
||||
6. 立体声叠加
|
||||
7. 峰值限制 → 响度归一化
|
||||
8. 导出
|
||||
"""
|
||||
|
||||
def __init__(self, config: dict | None = None):
|
||||
self.config = config or {}
|
||||
mix_cfg = config.get("mixing", {}) if config else {}
|
||||
|
||||
self.target_loudness = float(mix_cfg.get("target_loudness", -14.0))
|
||||
self.headroom_target = float(mix_cfg.get("headroom", 1.0))
|
||||
self.sample_rate = int(mix_cfg.get("sample_rate", 44100))
|
||||
self.bit_depth = int(mix_cfg.get("bit_depth", 16))
|
||||
|
||||
self.eq_preset = mix_cfg.get("eq_preset", "default")
|
||||
self.compressor_threshold = float(mix_cfg.get("compressor_threshold", -20))
|
||||
self.compressor_ratio = float(mix_cfg.get("compressor_ratio", 3.0))
|
||||
self.compressor_attack = float(mix_cfg.get("compressor_attack", 5))
|
||||
self.compressor_release = float(mix_cfg.get("compressor_release", 100))
|
||||
|
||||
self._pedalboard = None
|
||||
self._has_pedalboard = False
|
||||
self._init_pedalboard()
|
||||
|
||||
def _init_pedalboard(self):
|
||||
"""尝试加载 Pedalboard"""
|
||||
try:
|
||||
import pedalboard
|
||||
self._pedalboard = pedalboard
|
||||
self._has_pedalboard = True
|
||||
except ImportError:
|
||||
self._has_pedalboard = False
|
||||
|
||||
def mix(
|
||||
self,
|
||||
stems: dict[str, str],
|
||||
output_path: str | Path = "./output/mix/master.wav",
|
||||
stem_levels: dict[str, float] | None = None,
|
||||
apply_eq: bool = True,
|
||||
apply_compressor: bool = True,
|
||||
apply_reverb: bool = False,
|
||||
include_stems: bool = True,
|
||||
) -> MixResult:
|
||||
"""
|
||||
混音主入口
|
||||
|
||||
参数:
|
||||
stems: {stem_name: audio_path, ...}
|
||||
output_path: 混音输出路径
|
||||
stem_levels: 分轨电平增益 {name: dB}，默认 ±0
|
||||
apply_eq: 是否应用 EQ
|
||||
apply_compressor: 是否应用压缩
|
||||
apply_reverb: 是否应用混响
|
||||
include_stems: 是否同时导出处理后的分轨
|
||||
|
||||
返回:
|
||||
MixResult
|
||||
"""
|
||||
output_path = Path(output_path)
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
print(f" [mix] 混音开始: {len(stems)} 轨")
|
||||
|
||||
stem_levels = stem_levels or {}
|
||||
processed_stems: dict[str, Path] = {}
|
||||
stem_audio: dict[str, np.ndarray] = {}
|
||||
ref_sr = self.sample_rate
|
||||
|
||||
# — 1. 加载 + 处理各分轨 —
|
||||
for name, audio_path in stems.items():
|
||||
if not Path(audio_path).exists():
|
||||
print(f" [mix] ⚠ 分轨不存在: {audio_path}, 跳过")
|
||||
continue
|
||||
|
||||
y, sr = self._load_audio(audio_path, ref_sr)
|
||||
gain_db = stem_levels.get(name, 0.0)
|
||||
|
||||
# 应用增益
|
||||
if gain_db != 0:
|
||||
y = y * (10 ** (gain_db / 20))
|
||||
|
||||
# 应用处理 (EQ/压缩)
|
||||
if self._has_pedalboard:
|
||||
y = self._apply_processing(
|
||||
y, sr, name, apply_eq, apply_compressor, apply_reverb
|
||||
)
|
||||
|
||||
stem_audio[name] = y
|
||||
|
||||
# 导出处理后分轨
|
||||
if include_stems:
|
||||
stem_out = output_path.parent / f"processed_{name}.wav"
|
||||
self._save_audio(stem_out, y, sr)
|
||||
processed_stems[name] = str(stem_out.resolve())
|
||||
|
||||
print(f" [mix] {name}: {gain_db:+.1f}dB {len(y)/sr:.1f}s")
|
||||
|
||||
if not stem_audio:
|
||||
raise ValueError("没有可用的分轨进行混音")
|
||||
|
||||
# — 2. 立体声叠加 —
|
||||
ref_sr = self._get_common_sr(stem_audio)
|
||||
master = self._sum_stems(stem_audio)
|
||||
|
||||
# — 3. 峰值限制 + 响度归一化 —
|
||||
master, peak_db, loudness = self._master_bus(master, ref_sr)
|
||||
|
||||
# — 4. 导出 —
|
||||
self._save_audio(output_path, master, ref_sr)
|
||||
|
||||
headroom = abs(peak_db) if peak_db < 0 else 0
|
||||
|
||||
result = MixResult(
|
||||
master_path=str(output_path.resolve()),
|
||||
stem_paths={k: str(v) for k, v in processed_stems.items()},
|
||||
loudness=round(loudness, 2),
|
||||
peak_dB=round(peak_db, 2),
|
||||
headroom=round(headroom, 2),
|
||||
duration=round(len(master) / ref_sr, 2),
|
||||
engine="pedalboard" if self._has_pedalboard else "numpy",
|
||||
)
|
||||
|
||||
print(f" [mix] ✓ 混音完成: {output_path.name}")
|
||||
print(f" [mix] 响度={loudness:.1f} LUFS 峰值={peak_db:.1f} dBFS "
|
||||
f"头上空间={headroom:.1f} dB")
|
||||
return result
|
||||
|
||||
# ---- 内部处理 ----
|
||||
|
||||
def _load_audio(self, path: str | Path, target_sr: int) -> tuple[np.ndarray, int]:
|
||||
"""加载音频文件"""
|
||||
import librosa
|
||||
y, sr = librosa.load(str(path), sr=target_sr, mono=False)
|
||||
# 确保为 2D: (channels, samples)
|
||||
if y.ndim == 1:
|
||||
y = np.stack([y, y]) # mono → stereo
|
||||
return y, sr
|
||||
|
||||
def _get_common_sr(self, stem_audio: dict) -> int:
|
||||
"""取所有分轨的共用采样率"""
|
||||
return self.sample_rate
|
||||
|
||||
def _apply_processing(
|
||||
self,
|
||||
y: np.ndarray,
|
||||
sr: int,
|
||||
stem_name: str,
|
||||
apply_eq: bool,
|
||||
apply_compressor: bool,
|
||||
apply_reverb: bool,
|
||||
) -> np.ndarray:
|
||||
"""Pedalboard 处理链"""
|
||||
import pedalboard as pb
|
||||
|
||||
chain = []
|
||||
|
||||
# EQ
|
||||
if apply_eq:
|
||||
chain.append(
|
||||
pb.Filterband(
|
||||
cutoff_frequency=80.0,
|
||||
filter_type="highpass",
|
||||
)
|
||||
)
|
||||
if stem_name == "vocals":
|
||||
# 人声: 提升中频 (presence)
|
||||
chain.append(pb.Equalizer(num_eq_bands=3))
|
||||
|
||||
# 压缩
|
||||
if apply_compressor and stem_name in ("vocals", "bass"):
|
||||
chain.append(
|
||||
pb.Compressor(
|
||||
threshold_db=self.compressor_threshold,
|
||||
ratio=self.compressor_ratio,
|
||||
attack_ms=self.compressor_attack,
|
||||
release_ms=self.compressor_release,
|
||||
)
|
||||
)
|
||||
|
||||
# 混响 (仅应用到 vocal/master)
|
||||
if apply_reverb and stem_name == "vocals":
|
||||
chain.append(pb.Reverb(room_size=0.3, damping=0.5))
|
||||
|
||||
if not chain:
|
||||
return y
|
||||
|
||||
board = pb.Pedalboard(chain, sample_rate=sr)
|
||||
return board.process(y)
|
||||
|
||||
def _sum_stems(self, stem_audio: dict[str, np.ndarray]) -> np.ndarray:
|
||||
"""叠加多轨音频"""
|
||||
# 找出最长轨道
|
||||
max_len = max(y.shape[1] for y in stem_audio.values())
|
||||
|
||||
master = np.zeros((2, max_len), dtype=np.float64)
|
||||
|
||||
for name, y in stem_audio.items():
|
||||
y_len = y.shape[1]
|
||||
if y_len < max_len:
|
||||
pad = np.zeros((2, max_len - y_len))
|
||||
master[:, :y_len] += y[:, :] * 0.8
|
||||
master[:, y_len:] += pad
|
||||
else:
|
||||
master += y * 0.8
|
||||
|
||||
# 防止削波前先归一化
|
||||
peak = np.max(np.abs(master))
|
||||
if peak > 0.95:
|
||||
master = master / peak * 0.95
|
||||
|
||||
return master.astype(np.float32)
|
||||
|
||||
def _master_bus(
|
||||
self, audio: np.ndarray, sr: int
|
||||
) -> tuple[np.ndarray, float, float]:
|
||||
"""母带总线: 限制器 + 响度归一化"""
|
||||
|
||||
if self._has_pedalboard:
|
||||
import pedalboard as pb
|
||||
|
||||
# 限制器
|
||||
limiter = pb.Limiter(
|
||||
threshold_db=-self.headroom_target,
|
||||
release_ms=100,
|
||||
)
|
||||
board = pb.Pedalboard([limiter], sample_rate=sr)
|
||||
audio = board.process(audio)
|
||||
|
||||
# 测量峰值
|
||||
peak_db = float(20 * np.log10(np.max(np.abs(audio)) + 1e-10))
|
||||
|
||||
# 响度归一化 (简单 RMS 归一化，完整 LUFS 需要 pyloudnorm)
|
||||
rms = np.sqrt(np.mean(audio ** 2))
|
||||
if rms > 0:
|
||||
target_rms = 10 ** (self.target_loudness / 20)
|
||||
gain = target_rms / rms
|
||||
audio = audio * min(gain, 2.0) # 限制最大增益 6dB
|
||||
|
||||
loudness = self.target_loudness # 近似
|
||||
|
||||
return audio, peak_db, loudness
|
||||
|
||||
def _save_audio(self, path: Path, audio: np.ndarray, sr: int):
|
||||
"""保存音频"""
|
||||
import soundfile as sf
|
||||
# soundfile 期望 (samples, channels)
|
||||
if audio.ndim == 2 and audio.shape[0] <= 8:
|
||||
audio = audio.T
|
||||
sf.write(str(path), audio, sr, subtype="PCM_16")
|
||||
@@ -0,0 +1,8 @@
|
||||
"""AudioLine 对齐层 — 歌词节拍对齐 / 分轨时间对齐"""
|
||||
from .lyric_aligner import LyricAligner, LyricTimeline
|
||||
from .stem_aligner import StemAligner, AlignResult
|
||||
|
||||
__all__ = [
|
||||
"LyricAligner", "LyricTimeline",
|
||||
"StemAligner", "AlignResult",
|
||||
]
|
||||
+309
@@ -0,0 +1,309 @@
|
||||
"""
|
||||
layer4_alignment/lyric_aligner.py — 歌词-节拍对齐模块
|
||||
|
||||
功能：将歌词文本按段落结构对齐到节拍时间戳。
|
||||
输出格式：[{start, end, text, section}] 时间轴，支持转 LRC/SRT。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
@dataclass
|
||||
class LyricTimeline:
|
||||
"""歌词时间轴"""
|
||||
entries: list[dict] = field(default_factory=list) # [{start, end, text, section}]
|
||||
total_duration: float = 0.0
|
||||
sections: list[dict] = field(default_factory=list) # [{label, start, end}]
|
||||
|
||||
@property
|
||||
def lyric_count(self) -> int:
|
||||
return len(self.entries)
|
||||
|
||||
def to_lrc(self) -> str:
|
||||
"""转为 LRC 歌词格式"""
|
||||
lines = ["[ti:AudioLine LRC]", "[offset:0]"]
|
||||
for entry in self.entries:
|
||||
mm, ss = divmod(entry["start"], 60)
|
||||
cs = int((entry["start"] - int(entry["start"])) * 100)
|
||||
lines.append(f"[{int(mm):02d}:{int(ss):02d}.{cs:02d}]{entry['text']}")
|
||||
return "\n".join(lines)
|
||||
|
||||
def to_srt(self) -> str:
|
||||
"""转为 SRT 字幕格式"""
|
||||
blocks = []
|
||||
for i, entry in enumerate(self.entries, 1):
|
||||
start_srt = self._sec_to_srt(entry["start"])
|
||||
end_srt = self._sec_to_srt(entry["end"])
|
||||
blocks.append(f"{i}\n{start_srt} --> {end_srt}\n{entry['text']}\n")
|
||||
return "\n".join(blocks)
|
||||
|
||||
@staticmethod
|
||||
def _sec_to_srt(sec: float) -> str:
|
||||
h, m = divmod(sec, 3600)
|
||||
m, s = divmod(m, 60)
|
||||
cs = int((s - int(s)) * 1000)
|
||||
return f"{int(h):02d}:{int(m):02d}:{int(s):02d},{cs:03d}"
|
||||
|
||||
|
||||
class LyricAligner:
|
||||
"""
|
||||
歌词-节拍对齐器
|
||||
|
||||
将结构化的歌词段落匹配到分析出的节拍/段落边界。
|
||||
支持两种模式：
|
||||
1. 自动对齐 — 根据段落标签映射
|
||||
2. 手动指定 — 提供精确的时间偏移
|
||||
"""
|
||||
|
||||
def __init__(self, tolerance: float = 0.1):
|
||||
self.tolerance = tolerance # 时间容差 (秒)
|
||||
|
||||
def align(
|
||||
self,
|
||||
lyrics: str,
|
||||
beats: list[float],
|
||||
sections: list[dict] | None = None,
|
||||
total_duration: float = 0.0,
|
||||
) -> LyricTimeline:
|
||||
"""
|
||||
将对齐歌词到节拍
|
||||
|
||||
参数:
|
||||
lyrics: 歌词文本 (含 [Intro][Verse] 等段落标记)
|
||||
beats: 节拍时间戳列表 (秒)
|
||||
sections: 段落结构 [{label, start, end}] (可选)
|
||||
total_duration: 总时长 (秒)
|
||||
|
||||
返回:
|
||||
LyricTimeline
|
||||
"""
|
||||
# — 1. 解析歌词段落 —
|
||||
parsed_sections = self._parse_lyric_sections(lyrics)
|
||||
|
||||
if sections:
|
||||
# — 2a. 使用已分析的段落边界 —
|
||||
timeline = self._align_with_sections(
|
||||
parsed_sections, beats, sections, total_duration
|
||||
)
|
||||
else:
|
||||
# — 2b. 仅用节拍均匀分配 —
|
||||
timeline = self._align_by_beats(parsed_sections, beats, total_duration)
|
||||
|
||||
return timeline
|
||||
|
||||
def align_lines(
|
||||
self,
|
||||
lines: list[str],
|
||||
section: str,
|
||||
section_start: float,
|
||||
section_end: float,
|
||||
beats_in_section: list[float],
|
||||
) -> list[dict]:
|
||||
"""
|
||||
对齐单个段落内的歌词行到节拍
|
||||
|
||||
参数:
|
||||
lines: 歌词行列表
|
||||
section: 段落标签
|
||||
section_start: 段落开始时间
|
||||
section_end: 段落结束时间
|
||||
beats_in_section: 段落内的节拍
|
||||
|
||||
返回:
|
||||
[{start, end, text, section}]
|
||||
"""
|
||||
if not lines:
|
||||
return []
|
||||
|
||||
if not beats_in_section:
|
||||
# 无节拍: 均匀分配
|
||||
return self._distribute_evenly(lines, section, section_start, section_end)
|
||||
|
||||
entries = []
|
||||
beats_per_line = max(len(beats_in_section) // len(lines), 1)
|
||||
|
||||
for i, line in enumerate(lines):
|
||||
start_idx = i * beats_per_line
|
||||
end_idx = min((i + 1) * beats_per_line, len(beats_in_section))
|
||||
|
||||
start = beats_in_section[start_idx] if start_idx < len(beats_in_section) else section_start
|
||||
end = beats_in_section[end_idx - 1] if end_idx <= len(beats_in_section) else section_end
|
||||
|
||||
# 对最后一行: 用段落结束而不是最后一个节拍
|
||||
if i == len(lines) - 1:
|
||||
end = section_end
|
||||
|
||||
entries.append({
|
||||
"start": round(start, 3),
|
||||
"end": round(end, 3),
|
||||
"text": line.strip(),
|
||||
"section": section,
|
||||
})
|
||||
|
||||
return entries
|
||||
|
||||
# ---- 内部 ----
|
||||
|
||||
def _parse_lyric_sections(self, lyrics: str) -> list[dict[str, Any]]:
|
||||
"""解析歌词中的段落标记"""
|
||||
sections = []
|
||||
# 匹配 [Intro], [Verse 1], [Chorus] 等标记
|
||||
pattern = r"\[(\w+(?:\s*\d*)?)\](.*?)(?=\[\w|$)"
|
||||
matches = re.findall(pattern, lyrics, re.DOTALL)
|
||||
|
||||
if not matches:
|
||||
# 无标记: 整段视为一个段落
|
||||
lines = [l for l in lyrics.strip().split("\n") if l.strip()]
|
||||
sections.append({"label": "Full", "lines": lines})
|
||||
else:
|
||||
for label, content in matches:
|
||||
lines = [l.strip() for l in content.strip().split("\n") if l.strip()]
|
||||
if lines:
|
||||
sections.append({"label": label.strip(), "lines": lines})
|
||||
|
||||
return sections
|
||||
|
||||
def _align_with_sections(
|
||||
self,
|
||||
parsed: list[dict],
|
||||
beats: list[float],
|
||||
sections: list[dict],
|
||||
total_duration: float,
|
||||
) -> LyricTimeline:
|
||||
"""基于段落结构对齐"""
|
||||
all_entries = []
|
||||
|
||||
for ps in parsed:
|
||||
label = ps["label"]
|
||||
# 找对应的段落
|
||||
section = next(
|
||||
(s for s in sections if label.lower() in s.get("label", "").lower()),
|
||||
None,
|
||||
)
|
||||
|
||||
if section:
|
||||
start = section["start"]
|
||||
end = section["end"]
|
||||
beats_in_sec = [b for b in beats if start <= b <= end]
|
||||
else:
|
||||
# 找不到: 用标签顺序估算位置
|
||||
idx = parsed.index(ps)
|
||||
total = len(parsed)
|
||||
start = (idx / total) * total_duration
|
||||
end = ((idx + 1) / total) * total_duration
|
||||
beats_in_sec = [b for b in beats if start <= b <= end]
|
||||
|
||||
entries = self.align_lines(
|
||||
ps["lines"], label, start, end, beats_in_sec
|
||||
)
|
||||
all_entries.extend(entries)
|
||||
|
||||
return LyricTimeline(
|
||||
entries=all_entries,
|
||||
total_duration=total_duration or (all_entries[-1]["end"] if all_entries else 0),
|
||||
sections=sections,
|
||||
)
|
||||
|
||||
def _align_by_beats(
|
||||
self,
|
||||
parsed: list[dict],
|
||||
beats: list[float],
|
||||
total_duration: float,
|
||||
) -> LyricTimeline:
|
||||
"""仅基于节拍均匀分配"""
|
||||
if not beats:
|
||||
return self._fallback_linear(parsed, total_duration)
|
||||
|
||||
all_entries = []
|
||||
total_lines = sum(len(ps["lines"]) for ps in parsed)
|
||||
if total_lines == 0:
|
||||
return LyricTimeline()
|
||||
|
||||
beats_per_line = max(len(beats) // total_lines, 1)
|
||||
line_idx = 0
|
||||
|
||||
for ps in parsed:
|
||||
for line in ps["lines"]:
|
||||
start_idx = line_idx * beats_per_line
|
||||
end_idx = min((line_idx + 1) * beats_per_line, len(beats))
|
||||
|
||||
start = beats[start_idx] if start_idx < len(beats) else 0
|
||||
end = beats[end_idx - 1] if end_idx <= len(beats) else (total_duration or end + 2)
|
||||
|
||||
all_entries.append({
|
||||
"start": round(start, 3),
|
||||
"end": round(end, 3),
|
||||
"text": line.strip(),
|
||||
"section": ps["label"],
|
||||
})
|
||||
line_idx += 1
|
||||
|
||||
return LyricTimeline(
|
||||
entries=all_entries,
|
||||
total_duration=total_duration or (all_entries[-1]["end"] if all_entries else 0),
|
||||
)
|
||||
|
||||
def _distribute_evenly(
|
||||
self,
|
||||
lines: list[str],
|
||||
section: str,
|
||||
start: float,
|
||||
end: float,
|
||||
) -> list[dict]:
|
||||
"""均匀分配歌词行"""
|
||||
if not lines:
|
||||
return []
|
||||
if len(lines) == 1:
|
||||
return [{"start": round(start, 3), "end": round(end, 3),
|
||||
"text": lines[0].strip(), "section": section}]
|
||||
|
||||
duration = end - start
|
||||
line_duration = duration / len(lines)
|
||||
entries = []
|
||||
|
||||
for i, line in enumerate(lines):
|
||||
line_start = start + i * line_duration
|
||||
line_end = line_start + line_duration
|
||||
entries.append({
|
||||
"start": round(line_start, 3),
|
||||
"end": round(line_end, 3),
|
||||
"text": line.strip(),
|
||||
"section": section,
|
||||
})
|
||||
|
||||
return entries
|
||||
|
||||
def _fallback_linear(
|
||||
self, parsed: list[dict], total_duration: float
|
||||
) -> LyricTimeline:
|
||||
"""无节拍时的回退方案"""
|
||||
if not total_duration:
|
||||
return LyricTimeline()
|
||||
|
||||
all_lines = []
|
||||
for ps in parsed:
|
||||
for line in ps["lines"]:
|
||||
all_lines.append({"text": line.strip(), "section": ps["label"]})
|
||||
|
||||
if not all_lines:
|
||||
return LyricTimeline()
|
||||
|
||||
per_line = total_duration / len(all_lines)
|
||||
entries = [
|
||||
{
|
||||
"start": round(i * per_line, 3),
|
||||
"end": round((i + 1) * per_line, 3),
|
||||
"text": e["text"],
|
||||
"section": e["section"],
|
||||
}
|
||||
for i, e in enumerate(all_lines)
|
||||
]
|
||||
|
||||
return LyricTimeline(
|
||||
entries=entries,
|
||||
total_duration=total_duration,
|
||||
)
|
||||
+392
@@ -0,0 +1,392 @@
|
||||
"""
|
||||
layer4_alignment/stem_aligner.py — 分轨时间对齐模块
|
||||
|
||||
功能：将各分轨时间伸缩到目标 BPM / 目标时长
|
||||
核心场景：卡点改编时原曲 BPM -> 目标 BPM 的时间映射
|
||||
|
||||
支持方法:
|
||||
1. librosa.effects.time_stretch (相位声码器)
|
||||
2. 节拍网格对齐 (beat grid snapping)
|
||||
3. 交叉淡入淡出连接 (拼接时)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
@dataclass
|
||||
class AlignResult:
|
||||
"""对齐结果"""
|
||||
aligned_paths: dict[str, str] = field(default_factory=dict) # {stem: path}
|
||||
original_duration: float = 0.0
|
||||
target_duration: float = 0.0
|
||||
stretch_ratios: dict[str, float] = field(default_factory=dict) # {stem: ratio}
|
||||
method_used: str = ""
|
||||
|
||||
@property
|
||||
def is_aligned(self) -> bool:
|
||||
return len(self.aligned_paths) > 0
|
||||
|
||||
|
||||
class StemAligner:
|
||||
"""
|
||||
分轨时间对齐器
|
||||
|
||||
将分轨从原始节奏伸缩到目标节奏。
|
||||
支持独立对齐每轨或统一对齐。
|
||||
"""
|
||||
|
||||
STRETCH_METHODS: list[str] = ["phase_vocoder", "librosa", "wsola"]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
method: str = "phase_vocoder",
|
||||
crossfade: float = 0.05,
|
||||
time_tolerance: float = 0.1,
|
||||
):
|
||||
if method not in self.STRETCH_METHODS:
|
||||
raise ValueError(f"不支持的时间伸缩方法 '{method}', 可选: {self.STRETCH_METHODS}")
|
||||
self.method = method
|
||||
self.crossfade = crossfade
|
||||
self.time_tolerance = time_tolerance
|
||||
|
||||
def align_to_bpm(
|
||||
self,
|
||||
stems: dict[str, str],
|
||||
source_bpm: float,
|
||||
target_bpm: float,
|
||||
output_dir: str | Path = "./output/aligned/",
|
||||
align_separately: bool = False,
|
||||
) -> AlignResult:
|
||||
"""
|
||||
按 BPM 比率对齐分轨
|
||||
|
||||
参数:
|
||||
stems: {stem_name: audio_path}
|
||||
source_bpm: 原曲 BPM
|
||||
target_bpm: 目标 BPM
|
||||
output_dir: 输出目录
|
||||
align_separately: True=每轨独立比例, False=统一比例
|
||||
|
||||
返回:
|
||||
AlignResult
|
||||
"""
|
||||
if target_bpm <= 0 or source_bpm <= 0:
|
||||
raise ValueError("BPM 必须 > 0")
|
||||
|
||||
output_dir = Path(output_dir)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
ratio = target_bpm / source_bpm
|
||||
print(f" [align] BPM 对齐: {source_bpm} → {target_bpm} (比率: {ratio:.4f})")
|
||||
|
||||
return self._align(
|
||||
stems, ratio, output_dir, align_separately
|
||||
)
|
||||
|
||||
def align_to_duration(
|
||||
self,
|
||||
stems: dict[str, str],
|
||||
source_duration: float,
|
||||
target_duration: float,
|
||||
output_dir: str | Path = "./output/aligned/",
|
||||
align_separately: bool = False,
|
||||
) -> AlignResult:
|
||||
"""
|
||||
按时长比例对齐分轨
|
||||
|
||||
参数:
|
||||
stems: {stem_name: audio_path}
|
||||
source_duration: 原曲时长 (秒)
|
||||
target_duration: 目标时长 (秒)
|
||||
output_dir: 输出目录
|
||||
align_separately: True=每轨独立
|
||||
|
||||
返回:
|
||||
AlignResult
|
||||
"""
|
||||
if target_duration <= 0 or source_duration <= 0:
|
||||
raise ValueError("时长必须 > 0")
|
||||
|
||||
output_dir = Path(output_dir)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
ratio = target_duration / source_duration
|
||||
print(f" [align] 时长对齐: {source_duration:.1f}s → {target_duration:.1f}s"
|
||||
f" (比率: {ratio:.4f})")
|
||||
|
||||
return self._align(
|
||||
stems, ratio, output_dir, align_separately
|
||||
)
|
||||
|
||||
def align_with_beats(
|
||||
self,
|
||||
stems: dict[str, str],
|
||||
source_beats: list[float],
|
||||
target_beats: list[float],
|
||||
output_dir: str | Path = "./output/aligned/",
|
||||
) -> AlignResult:
|
||||
"""
|
||||
按节拍网格对齐 (精确卡点)
|
||||
|
||||
将源节拍映射到目标节拍位置，逐段伸缩。
|
||||
|
||||
参数:
|
||||
stems: {stem_name: audio_path}
|
||||
source_beats: 原节奏拍列表 (秒)
|
||||
target_beats: 目标节拍列表 (秒)
|
||||
output_dir: 输出目录
|
||||
|
||||
返回:
|
||||
AlignResult
|
||||
"""
|
||||
if not source_beats or not target_beats:
|
||||
raise ValueError("节拍列表不能为空")
|
||||
|
||||
output_dir = Path(output_dir)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
print(f" [align] 节拍网格对齐: {len(source_beats)} → {len(target_beats)} 拍")
|
||||
|
||||
aligned = {}
|
||||
stretch_ratios = {}
|
||||
|
||||
for stem_name, audio_path in stems.items():
|
||||
if not Path(audio_path).exists():
|
||||
print(f" [align] ⚠ 分轨不存在: {audio_path}, 跳过")
|
||||
continue
|
||||
|
||||
out_path = output_dir / f"aligned_{stem_name}.wav"
|
||||
result_path = self._beat_grid_stretch(
|
||||
str(audio_path), source_beats, target_beats, str(out_path)
|
||||
)
|
||||
if result_path:
|
||||
aligned[stem_name] = result_path
|
||||
stretch_ratios[stem_name] = len(target_beats) / len(source_beats)
|
||||
|
||||
return AlignResult(
|
||||
aligned_paths=aligned,
|
||||
original_duration=source_beats[-1] if source_beats else 0,
|
||||
target_duration=target_beats[-1] if target_beats else 0,
|
||||
stretch_ratios=stretch_ratios,
|
||||
method_used="beat_grid",
|
||||
)
|
||||
|
||||
# ---- 内部 ----
|
||||
|
||||
def _align(
|
||||
self,
|
||||
stems: dict[str, str],
|
||||
ratio: float,
|
||||
output_dir: Path,
|
||||
separate: bool,
|
||||
) -> AlignResult:
|
||||
"""通用对齐"""
|
||||
aligned = {}
|
||||
stretch_ratios = {}
|
||||
|
||||
# — 加载第一轨获取原时长 —
|
||||
first_path = next(iter(stems.values()))
|
||||
y_ref, sr_ref = self._load_audio(first_path)
|
||||
orig_duration = len(y_ref) / sr_ref
|
||||
|
||||
for stem_name, audio_path in stems.items():
|
||||
if not Path(audio_path).exists():
|
||||
print(f" [align] ⚠ 分轨不存在: {audio_path}, 跳过")
|
||||
continue
|
||||
|
||||
out_path = output_dir / f"aligned_{stem_name}.wav"
|
||||
|
||||
if separate:
|
||||
# 独立对齐：每轨先测实际时长再计算比例
|
||||
y, sr = self._load_audio(audio_path)
|
||||
actual_duration = len(y) / sr
|
||||
target_dur = actual_duration * ratio
|
||||
actual_ratio = target_dur / actual_duration
|
||||
else:
|
||||
actual_ratio = ratio
|
||||
|
||||
result_path = self._time_stretch(
|
||||
str(audio_path), actual_ratio, str(out_path)
|
||||
)
|
||||
if result_path:
|
||||
aligned[stem_name] = result_path
|
||||
stretch_ratios[stem_name] = actual_ratio
|
||||
|
||||
return AlignResult(
|
||||
aligned_paths=aligned,
|
||||
original_duration=orig_duration,
|
||||
target_duration=orig_duration * ratio,
|
||||
stretch_ratios=stretch_ratios,
|
||||
method_used=self.method,
|
||||
)
|
||||
|
||||
def _time_stretch(self, input_path: str, ratio: float, output_path: str) -> str | None:
|
||||
"""
|
||||
时间伸缩
|
||||
|
||||
参数:
|
||||
input_path: 输入音频路径
|
||||
ratio: 伸缩比 (>1 变快/缩短, <1 变慢/拉长)
|
||||
output_path: 输出路径
|
||||
|
||||
返回:
|
||||
输出路径，失败返回 None
|
||||
"""
|
||||
if abs(ratio - 1.0) < 0.001:
|
||||
# 无需伸缩: 直接复制
|
||||
import shutil
|
||||
shutil.copy2(input_path, output_path)
|
||||
return output_path
|
||||
|
||||
print(f" [align] 伸缩: ratio={ratio:.4f}")
|
||||
|
||||
try:
|
||||
y, sr = self._load_audio(input_path)
|
||||
|
||||
if self.method == "librosa":
|
||||
import librosa
|
||||
y_stretched = librosa.effects.time_stretch(y, rate=ratio)
|
||||
elif self.method == "phase_vocoder":
|
||||
# 用相位声码器
|
||||
y_stretched = self._phase_vocoder_stretch(y, ratio)
|
||||
else: # wsola
|
||||
y_stretched = self._wsola_stretch(y, ratio)
|
||||
|
||||
self._save_audio(output_path, y_stretched, sr)
|
||||
return output_path
|
||||
|
||||
except Exception as e:
|
||||
print(f" [align] ⚠ 伸缩失败: {e}")
|
||||
# 回退: 复制原文件
|
||||
import shutil
|
||||
shutil.copy2(input_path, output_path)
|
||||
return output_path
|
||||
|
||||
def _beat_grid_stretch(
|
||||
self,
|
||||
audio_path: str,
|
||||
source_beats: list[float],
|
||||
target_beats: list[float],
|
||||
output_path: str,
|
||||
) -> str | None:
|
||||
"""
|
||||
逐段节拍网格伸缩
|
||||
|
||||
将源节拍分段，每段分别伸缩到目标节拍间隔。
|
||||
"""
|
||||
try:
|
||||
y, sr = self._load_audio(audio_path)
|
||||
|
||||
# 限制处理范围
|
||||
max_beats = min(len(source_beats), len(target_beats))
|
||||
if max_beats < 2:
|
||||
import shutil
|
||||
shutil.copy2(audio_path, output_path)
|
||||
return output_path
|
||||
|
||||
segments = []
|
||||
for i in range(max_beats - 1):
|
||||
s_start = int(source_beats[i] * sr)
|
||||
s_end = int(source_beats[i + 1] * sr)
|
||||
t_len = target_beats[i + 1] - target_beats[i]
|
||||
s_len = source_beats[i + 1] - source_beats[i]
|
||||
|
||||
if s_len <= 0 or t_len <= 0:
|
||||
continue
|
||||
|
||||
segment = y[..., s_start:s_end] if y.ndim == 1 else y[:, s_start:s_end]
|
||||
if segment.size == 0:
|
||||
continue
|
||||
|
||||
ratio = s_len / t_len # 注意: librosa 的 stretch 反向
|
||||
stretched = librosa.effects.time_stretch(
|
||||
np.asfortranarray(segment), rate=1/ratio
|
||||
)
|
||||
|
||||
segments.append(stretched)
|
||||
|
||||
if not segments:
|
||||
import shutil
|
||||
shutil.copy2(audio_path, output_path)
|
||||
return output_path
|
||||
|
||||
# 拼接 (带交叉淡入淡出)
|
||||
result = self._crossfade_join(segments, sr)
|
||||
self._save_audio(output_path, result, sr)
|
||||
return output_path
|
||||
|
||||
except Exception as e:
|
||||
print(f" [align] ⚠ 节拍网格对齐失败: {e}")
|
||||
import shutil
|
||||
shutil.copy2(audio_path, output_path)
|
||||
return output_path
|
||||
|
||||
def _phase_vocoder_stretch(self, y: np.ndarray, ratio: float) -> np.ndarray:
|
||||
"""相位声码器实现的时间伸缩"""
|
||||
from librosa.core import phase_vocoder
|
||||
from librosa import stft, istft
|
||||
|
||||
D = stft(y)
|
||||
D_stretched = phase_vocoder(D, rate=ratio)
|
||||
y_stretched = istft(D_stretched)
|
||||
return y_stretched
|
||||
|
||||
def _wsola_stretch(self, y: np.ndarray, ratio: float) -> np.ndarray:
|
||||
"""WSOLA 时间伸缩 (librosa 回退)"""
|
||||
import librosa
|
||||
return librosa.effects.time_stretch(y, rate=ratio)
|
||||
|
||||
def _crossfade_join(
|
||||
self, segments: list[np.ndarray], sr: int
|
||||
) -> np.ndarray:
|
||||
"""带交叉淡入淡出的音频拼接"""
|
||||
if len(segments) == 1:
|
||||
return segments[0]
|
||||
|
||||
fade_len = int(self.crossfade * sr)
|
||||
# 确保所有段 mono (1D)
|
||||
segments_1d = []
|
||||
for seg in segments:
|
||||
if seg.ndim > 1:
|
||||
seg = np.mean(seg, axis=0)
|
||||
segments_1d.append(seg)
|
||||
|
||||
result = segments_1d[0]
|
||||
for seg in segments_1d[1:]:
|
||||
if len(result) < fade_len or len(seg) < fade_len:
|
||||
result = np.concatenate([result, seg])
|
||||
continue
|
||||
|
||||
fade_out = np.linspace(1, 0, fade_len)
|
||||
fade_in = np.linspace(0, 1, fade_len)
|
||||
|
||||
cross_start = len(result) - fade_len
|
||||
result[cross_start:] *= fade_out
|
||||
seg[:fade_len] *= fade_in
|
||||
|
||||
result = np.concatenate([
|
||||
result[:cross_start],
|
||||
result[cross_start:] + seg[:fade_len],
|
||||
seg[fade_len:],
|
||||
])
|
||||
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def _load_audio(path: str) -> tuple[np.ndarray, int]:
|
||||
"""加载音频"""
|
||||
import librosa
|
||||
return librosa.load(path, sr=None, mono=True)
|
||||
|
||||
@staticmethod
|
||||
def _save_audio(path: str, y: np.ndarray, sr: int):
|
||||
"""保存音频"""
|
||||
import soundfile as sf
|
||||
sf.write(path, y, sr)
|
||||
@@ -0,0 +1,8 @@
|
||||
"""AudioLine 输出层 — 导出 / 歌词格式化"""
|
||||
from .exporter import Exporter, ExportResult
|
||||
from .lyric_formatter import LyricFormatter, FormattedLyrics
|
||||
|
||||
__all__ = [
|
||||
"Exporter", "ExportResult",
|
||||
"LyricFormatter", "FormattedLyrics",
|
||||
]
|
||||
@@ -0,0 +1,282 @@
|
||||
"""
|
||||
layer5_output/exporter.py — 多格式输出导出模块
|
||||
|
||||
功能：将混音结果导出为多种格式
|
||||
支持：WAV (PCM) / MP3 / FLAC / AAC
|
||||
可选：整轨导出 / 分轨导出 / 带歌词时间轴导出
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
@dataclass
|
||||
class ExportResult:
|
||||
"""导出结果"""
|
||||
exported_files: list[dict] = field(default_factory=list) # [{format, path, size_kb}]
|
||||
formats_completed: list[str] = field(default_factory=list)
|
||||
total_size_kb: float = 0.0
|
||||
duration: float = 0.0
|
||||
has_lyrics: bool = False
|
||||
|
||||
@property
|
||||
def summary(self) -> str:
|
||||
return f"{len(self.exported_files)} 文件, {self.total_size_kb:.0f} KB, " \
|
||||
f"{self.duration:.1f}s, 格式: {', '.join(self.formats_completed)}"
|
||||
|
||||
|
||||
class Exporter:
|
||||
"""
|
||||
多格式导出器
|
||||
|
||||
支持格式:
|
||||
- WAV: 无损 PCM 16/24-bit
|
||||
- MP3: CBR/VBR, 支持 128k/192k/320k
|
||||
- FLAC: 无损压缩
|
||||
- AAC: 高效有损 (M4A)
|
||||
"""
|
||||
|
||||
SUPPORTED_FORMATS = {"wav", "mp3", "flac", "aac"}
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
config: dict | None = None,
|
||||
):
|
||||
self.config = config or {}
|
||||
export_cfg = config.get("export", {}) if config else {}
|
||||
|
||||
self.default_formats = export_cfg.get("formats", ["wav", "mp3"])
|
||||
self.mp3_bitrate = export_cfg.get("mp3_bitrate", "320k")
|
||||
self.include_stems = export_cfg.get("include_stems", True)
|
||||
self.stem_labels = export_cfg.get("stem_labels",
|
||||
["vocals", "drums", "bass", "other", "full_mix"])
|
||||
|
||||
self.sample_rate = int(config.get("mixing", {}).get("sample_rate", 44100)) if config else 44100
|
||||
self.bit_depth = int(config.get("mixing", {}).get("bit_depth", 16)) if config else 16
|
||||
|
||||
def export_master(
|
||||
self,
|
||||
master_path: str | Path,
|
||||
output_dir: str | Path = "./output/export/",
|
||||
formats: list[str] | None = None,
|
||||
metadata: dict | None = None,
|
||||
) -> ExportResult:
|
||||
"""
|
||||
导出主混音
|
||||
|
||||
参数:
|
||||
master_path: 主混音 WAV 路径
|
||||
output_dir: 输出目录
|
||||
formats: 目标格式列表 (默认 config 中的设置)
|
||||
metadata: 元数据 {title, artist, album, ...}
|
||||
|
||||
返回:
|
||||
ExportResult
|
||||
"""
|
||||
master_path = Path(master_path)
|
||||
if not master_path.exists():
|
||||
raise FileNotFoundError(f"主混音文件不存在: {master_path}")
|
||||
|
||||
output_dir = Path(output_dir)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
formats = formats or self.default_formats
|
||||
stem_name = master_path.stem
|
||||
|
||||
print(f" [export] 导出主轨: {master_path.name}")
|
||||
print(f" [export] 目标格式: {formats}")
|
||||
|
||||
exported = []
|
||||
for fmt in formats:
|
||||
fmt = fmt.lower().lstrip(".")
|
||||
if fmt not in self.SUPPORTED_FORMATS:
|
||||
print(f" [export] ⚠ 不支持格式 '{fmt}', 跳过")
|
||||
continue
|
||||
|
||||
out_path = output_dir / f"{stem_name}.{fmt}"
|
||||
success = self._convert(master_path, out_path, fmt, metadata)
|
||||
if success:
|
||||
size_kb = out_path.stat().st_size / 1024
|
||||
exported.append({
|
||||
"format": fmt,
|
||||
"path": str(out_path.resolve()),
|
||||
"size_kb": round(size_kb, 1),
|
||||
})
|
||||
print(f" [export] {fmt}: {out_path.name} ({size_kb:.0f} KB)")
|
||||
|
||||
if not exported:
|
||||
raise RuntimeError("所有格式导出均失败")
|
||||
|
||||
# 获取时长
|
||||
duration = self._get_duration(master_path)
|
||||
|
||||
return ExportResult(
|
||||
exported_files=exported,
|
||||
formats_completed=[e["format"] for e in exported],
|
||||
total_size_kb=sum(e["size_kb"] for e in exported),
|
||||
duration=duration,
|
||||
)
|
||||
|
||||
def export_stems(
|
||||
self,
|
||||
stems: dict[str, str],
|
||||
output_dir: str | Path = "./output/export/",
|
||||
formats: list[str] | None = None,
|
||||
) -> list[ExportResult]:
|
||||
"""
|
||||
导出所有分轨
|
||||
|
||||
参数:
|
||||
stems: {stem_name: audio_path, ...}
|
||||
output_dir: 输出目录
|
||||
formats: 目标格式
|
||||
|
||||
返回:
|
||||
每轨的 ExportResult 列表
|
||||
"""
|
||||
results = []
|
||||
for name, path in stems.items():
|
||||
result = self.export_master(
|
||||
Path(path), output_dir / name, formats
|
||||
)
|
||||
results.append(result)
|
||||
return results
|
||||
|
||||
def export_with_lyrics(
|
||||
self,
|
||||
master_path: str | Path,
|
||||
lyric_timeline: Any, # LyricTimeline
|
||||
output_dir: str | Path = "./output/export/",
|
||||
formats: list[str] | None = None,
|
||||
) -> ExportResult:
|
||||
"""
|
||||
导出带歌词的主混音
|
||||
|
||||
同时导出音频 + LRC/SRT 歌词文件
|
||||
"""
|
||||
output_dir = Path(output_dir)
|
||||
result = self.export_master(master_path, output_dir, formats)
|
||||
|
||||
# 导出 LRC
|
||||
lrc_path = output_dir / f"{Path(master_path).stem}.lrc"
|
||||
lrc_content = lyric_timeline.to_lrc()
|
||||
lrc_path.write_text(lrc_content, encoding="utf-8")
|
||||
result.exported_files.append({
|
||||
"format": "lrc",
|
||||
"path": str(lrc_path.resolve()),
|
||||
"size_kb": round(lrc_path.stat().st_size / 1024, 1),
|
||||
})
|
||||
|
||||
# 导出 SRT
|
||||
srt_path = output_dir / f"{Path(master_path).stem}.srt"
|
||||
srt_content = lyric_timeline.to_srt()
|
||||
srt_path.write_text(srt_content, encoding="utf-8")
|
||||
result.exported_files.append({
|
||||
"format": "srt",
|
||||
"path": str(srt_path.resolve()),
|
||||
"size_kb": round(srt_path.stat().st_size / 1024, 1),
|
||||
})
|
||||
|
||||
result.has_lyrics = True
|
||||
result.formats_completed.extend(["lrc", "srt"])
|
||||
result.total_size_kb = sum(e["size_kb"] for e in result.exported_files)
|
||||
|
||||
print(f" [export] 歌词: LRC + SRT 已导出")
|
||||
return result
|
||||
|
||||
# ---- 内部 ----
|
||||
|
||||
def _convert(
|
||||
self,
|
||||
src: Path,
|
||||
dst: Path,
|
||||
fmt: str,
|
||||
metadata: dict | None = None,
|
||||
) -> bool:
|
||||
"""格式转换"""
|
||||
try:
|
||||
if fmt == "wav":
|
||||
return self._to_wav(src, dst)
|
||||
elif fmt == "mp3":
|
||||
return self._to_mp3(src, dst, metadata)
|
||||
elif fmt == "flac":
|
||||
return self._to_flac(src, dst)
|
||||
elif fmt == "aac":
|
||||
return self._to_aac(src, dst, metadata)
|
||||
return False
|
||||
except Exception as e:
|
||||
print(f" [export] {fmt} 转换失败: {e}")
|
||||
return False
|
||||
|
||||
def _to_wav(self, src: Path, dst: Path) -> bool:
|
||||
"""复制/重编码为 WAV (PCM 16-bit)"""
|
||||
import soundfile as sf
|
||||
y, sr = sf.read(str(src))
|
||||
sf.write(str(dst), y, sr, subtype="PCM_16")
|
||||
return dst.exists()
|
||||
|
||||
def _to_mp3(self, src: Path, dst: Path, metadata: dict | None = None) -> bool:
|
||||
"""转 MP3 (通过 ffmpeg)"""
|
||||
cmd = [
|
||||
"ffmpeg", "-y", "-i", str(src),
|
||||
"-codec:a", "libmp3lame",
|
||||
"-b:a", self.mp3_bitrate,
|
||||
"-ar", str(self.sample_rate),
|
||||
]
|
||||
if metadata:
|
||||
if "title" in metadata:
|
||||
cmd += ["-metadata", f"title={metadata['title']}"]
|
||||
if "artist" in metadata:
|
||||
cmd += ["-metadata", f"artist={metadata['artist']}"]
|
||||
cmd.append(str(dst))
|
||||
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=120)
|
||||
return result.returncode == 0 and dst.exists()
|
||||
|
||||
def _to_flac(self, src: Path, dst: Path) -> bool:
|
||||
"""转 FLAC (通过 ffmpeg)"""
|
||||
cmd = [
|
||||
"ffmpeg", "-y", "-i", str(src),
|
||||
"-codec:a", "flac",
|
||||
"-ar", str(self.sample_rate),
|
||||
str(dst),
|
||||
]
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=120)
|
||||
return result.returncode == 0 and dst.exists()
|
||||
|
||||
def _to_aac(self, src: Path, dst: Path, metadata: dict | None = None) -> bool:
|
||||
"""转 AAC/M4A (通过 ffmpeg)"""
|
||||
cmd = [
|
||||
"ffmpeg", "-y", "-i", str(src),
|
||||
"-codec:a", "aac",
|
||||
"-b:a", "256k",
|
||||
"-ar", str(self.sample_rate),
|
||||
]
|
||||
if metadata:
|
||||
if "title" in metadata:
|
||||
cmd += ["-metadata", f"title={metadata['title']}"]
|
||||
if "artist" in metadata:
|
||||
cmd += ["-metadata", f"artist={metadata['artist']}"]
|
||||
cmd.append(str(dst))
|
||||
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=120)
|
||||
return result.returncode == 0 and dst.exists()
|
||||
|
||||
@staticmethod
|
||||
def _get_duration(path: Path) -> float:
|
||||
"""获取音频时长（用 lib.ffmpeg —— 本机 ffprobe 实为 ffmpeg 符号链接）"""
|
||||
try:
|
||||
import sys as _s
|
||||
from pathlib import Path as _P
|
||||
_scripts = _P(__file__).resolve().parents[3]
|
||||
if str(_scripts) not in _s.path:
|
||||
_s.path.insert(0, str(_scripts))
|
||||
from lib.ffmpeg import probe_duration
|
||||
return round(probe_duration(str(path)), 2)
|
||||
except Exception:
|
||||
return 0.0
|
||||
+226
@@ -0,0 +1,226 @@
|
||||
"""
|
||||
layer5_output/lyric_formatter.py — 歌词格式化模块
|
||||
|
||||
功能：将歌词时间轴输出为多种字幕/歌词格式
|
||||
支持格式：
|
||||
- LRC (LyRiCs) — 标准歌词格式
|
||||
- SRT (SubRip) — 通用字幕格式
|
||||
- ASS (Advanced SubStation Alpha) — 高级字幕 (带样式)
|
||||
- TTML (Timed Text Markup Language) — WebVTT 替代
|
||||
- plain text — 纯文本 (无时间戳)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
@dataclass
|
||||
class FormattedLyrics:
|
||||
"""格式化歌词结果"""
|
||||
lrc: str = ""
|
||||
srt: str = ""
|
||||
ass: str = ""
|
||||
ttml: str = ""
|
||||
plain: str = ""
|
||||
|
||||
def save_all(self, output_dir: str | Path, basename: str = "lyrics") -> dict[str, str]:
|
||||
"""保存所有格式到文件"""
|
||||
output_dir = Path(output_dir)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
saved = {}
|
||||
fmt_map = {
|
||||
"lrc": self.lrc,
|
||||
"srt": self.srt,
|
||||
"ass": self.ass,
|
||||
"ttml": self.ttml,
|
||||
"txt": self.plain,
|
||||
}
|
||||
|
||||
for ext, content in fmt_map.items():
|
||||
if content:
|
||||
path = output_dir / f"{basename}.{ext}"
|
||||
path.write_text(content, encoding="utf-8")
|
||||
saved[ext] = str(path.resolve())
|
||||
|
||||
return saved
|
||||
|
||||
|
||||
class LyricFormatter:
|
||||
"""
|
||||
歌词格式化器
|
||||
|
||||
将时间轴条目(entries)转换为各种字幕格式。
|
||||
支持自定义 ASS 样式。
|
||||
"""
|
||||
|
||||
# ASS 默认样式
|
||||
DEFAULT_ASS_STYLE = (
|
||||
"[V4+ Styles]\n"
|
||||
"Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, "
|
||||
"OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, "
|
||||
"ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, "
|
||||
"Alignment, MarginL, MarginR, MarginV, Encoding\n"
|
||||
"Style: Default, Arial, 24, &H00FFFFFF, &H000000FF, "
|
||||
"&H00000000, &H00000000, 0, 0, 0, 0, "
|
||||
"100, 100, 0, 0, 1, 1, 1, 2, 20, 20, 40, 1\n"
|
||||
)
|
||||
|
||||
def format_all(
|
||||
self,
|
||||
entries: list[dict],
|
||||
total_duration: float = 0.0,
|
||||
title: str = "AudioLine",
|
||||
ass_style: str | None = None,
|
||||
) -> FormattedLyrics:
|
||||
"""
|
||||
生成所有格式的歌词
|
||||
|
||||
参数:
|
||||
entries: [{start, end, text, section}] 时间轴条目
|
||||
total_duration: 总时长
|
||||
title: 标题
|
||||
ass_style: ASS 样式定义 (可选)
|
||||
|
||||
返回:
|
||||
FormattedLyrics
|
||||
"""
|
||||
return FormattedLyrics(
|
||||
lrc=self.to_lrc(entries, title),
|
||||
srt=self.to_srt(entries),
|
||||
ass=self.to_ass(entries, ass_style or self.DEFAULT_ASS_STYLE),
|
||||
ttml=self.to_ttml(entries, total_duration, title),
|
||||
plain=self.to_plain(entries),
|
||||
)
|
||||
|
||||
# ---- LRC ----
|
||||
|
||||
def to_lrc(self, entries: list[dict], title: str = "AudioLine") -> str:
|
||||
"""转为 LRC 格式"""
|
||||
lines = [
|
||||
"[ti:{}]".format(title),
|
||||
"[offset:0]",
|
||||
]
|
||||
|
||||
last_section = ""
|
||||
for entry in entries:
|
||||
section = entry.get("section", "")
|
||||
if section and section != last_section:
|
||||
lines.append("")
|
||||
lines.append("[{: >8}]".format(section))
|
||||
last_section = section
|
||||
|
||||
start = entry["start"]
|
||||
mm, ss = divmod(start, 60)
|
||||
cs = int((start - int(start)) * 100)
|
||||
lines.append("[{:02d}:{:02d}.{:02d}]{:s}".format(
|
||||
int(mm), int(ss), cs, entry["text"]
|
||||
))
|
||||
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
# ---- SRT ----
|
||||
|
||||
def to_srt(self, entries: list[dict]) -> str:
|
||||
"""转为 SRT 格式"""
|
||||
blocks = []
|
||||
for i, entry in enumerate(entries, 1):
|
||||
start_srt = self._sec_to_srt(entry["start"])
|
||||
end_srt = self._sec_to_srt(entry["end"])
|
||||
|
||||
section = entry.get("section", "")
|
||||
text = entry["text"]
|
||||
if section:
|
||||
text = f"{text}\n({section})"
|
||||
|
||||
blocks.append("{:d}\n{:s} --> {:s}\n{:s}\n".format(
|
||||
i, start_srt, end_srt, text
|
||||
))
|
||||
|
||||
return "\n".join(blocks)
|
||||
|
||||
# ---- ASS ----
|
||||
|
||||
def to_ass(self, entries: list[dict], style_block: str = "") -> str:
|
||||
"""转为 ASS 格式"""
|
||||
style_block = style_block or self.DEFAULT_ASS_STYLE
|
||||
|
||||
header = (
|
||||
"[Script Info]\n"
|
||||
"ScriptType: v4.00+\n"
|
||||
"Title: AudioLine Lyrics\n"
|
||||
"WrapStyle: 2\n"
|
||||
"ScaledBorderAndShadow: yes\n"
|
||||
)
|
||||
|
||||
body = ""
|
||||
for entry in entries:
|
||||
start_ass = self._sec_to_ass(entry["start"])
|
||||
end_ass = self._sec_to_ass(entry["end"])
|
||||
text = entry["text"].replace("\n", "\\N")
|
||||
|
||||
section = entry.get("section", "")
|
||||
if section:
|
||||
text = f"{{\\i1}}{section}{{\\i0}} - {text}"
|
||||
|
||||
body += "Dialogue: 0,{start},{end},Default,,0,0,0,,{text}\n".format(
|
||||
start=start_ass, end=end_ass, text=text
|
||||
)
|
||||
|
||||
return header + style_block + "\n[Events]\nFormat: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text\n" + body
|
||||
|
||||
# ---- TTML ----
|
||||
|
||||
def to_ttml(self, entries: list[dict], total_duration: float, title: str = "") -> str:
|
||||
"""转为 TTML (类似 WebVTT) 格式"""
|
||||
lines = ["WEBVTT", f"Title: {title}"]
|
||||
if total_duration:
|
||||
lines.append(f"Duration: {self._sec_to_srt(total_duration)}")
|
||||
lines.append("")
|
||||
|
||||
for entry in entries:
|
||||
start_vtt = self._sec_to_srt(entry["start"]).replace(",", ".")
|
||||
end_vtt = self._sec_to_srt(entry["end"]).replace(",", ".")
|
||||
lines.append("{:s} --> {:s}".format(start_vtt, end_vtt))
|
||||
lines.append(entry["text"])
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
# ---- Plain ----
|
||||
|
||||
def to_plain(self, entries: list[dict]) -> str:
|
||||
"""转为纯文本"""
|
||||
last_section = ""
|
||||
lines = []
|
||||
|
||||
for entry in entries:
|
||||
section = entry.get("section", "")
|
||||
if section and section != last_section:
|
||||
lines.append("")
|
||||
lines.append("[{}]".format(section))
|
||||
last_section = section
|
||||
lines.append(entry["text"])
|
||||
|
||||
return "\n".join(lines).strip()
|
||||
|
||||
# ---- 辅助 ----
|
||||
|
||||
@staticmethod
|
||||
def _sec_to_srt(sec: float) -> str:
|
||||
"""秒 → SRT 时间格式 (HH:MM:SS,mmm)"""
|
||||
h, rem = divmod(sec, 3600)
|
||||
m, s = divmod(rem, 60)
|
||||
ms = int((s - int(s)) * 1000)
|
||||
return "{:02d}:{:02d}:{:02d},{:03d}".format(int(h), int(m), int(s), ms)
|
||||
|
||||
@staticmethod
|
||||
def _sec_to_ass(sec: float) -> str:
|
||||
"""秒 → ASS 时间格式 (H:MM:SS.cc)"""
|
||||
h, rem = divmod(sec, 3600)
|
||||
m, s = divmod(rem, 60)
|
||||
cs = int((s - int(s)) * 100)
|
||||
return "{:d}:{:02d}:{:02d}.{:02d}".format(int(h), int(m), int(s), cs)
|
||||
@@ -0,0 +1,434 @@
|
||||
"""
|
||||
orchestrator.py — 音频创作管线编排控制器
|
||||
|
||||
职责：串联所有 6 层模块，按顺序执行 AudioTask。
|
||||
支持模块选择、dry-run、状态报告、错误恢复。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable
|
||||
|
||||
import yaml
|
||||
|
||||
# Layer 1 — 输入
|
||||
from layer1_input.collector import AudioTask, AudioSource, AudioCollector
|
||||
|
||||
# Layer 2 — 分析
|
||||
from layer2_analysis.stem_separator import StemSeparator
|
||||
from layer2_analysis.beat_detector import BeatDetector, BeatResult
|
||||
from layer2_analysis.structure_parser import StructureParser, StructureResult
|
||||
from layer2_analysis.emotion_analyzer import EmotionAnalyzer, EmotionResult
|
||||
|
||||
# Layer 3 — 创作
|
||||
from layer3_creation.module_a_rhythm import RhythmModule, RhythmResult
|
||||
from layer3_creation.module_b_lyrics import LyricsModule, LyricsResult
|
||||
from layer3_creation.module_c_melody import MelodyModule, MelodyResult
|
||||
from layer3_creation.module_d_mix import MixModule, MixResult
|
||||
|
||||
# Layer 4 — 对齐
|
||||
from layer4_alignment.lyric_aligner import LyricAligner, LyricTimeline
|
||||
from layer4_alignment.stem_aligner import StemAligner, AlignResult
|
||||
|
||||
# Layer 5 — 输出
|
||||
from layer5_output.exporter import Exporter, ExportResult
|
||||
from layer5_output.lyric_formatter import LyricFormatter, FormattedLyrics
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 结果数据类
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class AudioLineResult:
|
||||
"""完整管线输出"""
|
||||
task: AudioTask = field(default_factory=lambda: AudioTask(
|
||||
source=AudioSource(path="", source_type="command")
|
||||
))
|
||||
|
||||
# Layer 2 结果
|
||||
stems: dict[str, str] = field(default_factory=dict)
|
||||
beat: BeatResult | None = None
|
||||
structure: StructureResult | None = None
|
||||
emotion: EmotionResult | None = None
|
||||
|
||||
# Layer 3 结果
|
||||
rhythm: RhythmResult | None = None
|
||||
lyrics: LyricsResult | None = None
|
||||
melody: MelodyResult | None = None
|
||||
mix: MixResult | None = None
|
||||
|
||||
# Layer 4 结果
|
||||
lyric_timeline: LyricTimeline | None = None
|
||||
stem_alignment: AlignResult | None = None
|
||||
|
||||
# Layer 5 结果
|
||||
export: ExportResult | None = None
|
||||
formatted_lyrics: FormattedLyrics | None = None
|
||||
|
||||
# 状态
|
||||
modules_run: list[str] = field(default_factory=list)
|
||||
errors: list[dict] = field(default_factory=list) # [{module, error}]
|
||||
timing: dict[str, float] = field(default_factory=dict) # {module: elapsed_sec}
|
||||
success: bool = False
|
||||
|
||||
@property
|
||||
def summary(self) -> str:
|
||||
"""生成执行摘要"""
|
||||
lines = [
|
||||
f"任务: {self.task.operation} | 源: {self.task.source.original_filename or '(指令)'}",
|
||||
f"模块: {', '.join(self.modules_run) or '(无)'}",
|
||||
]
|
||||
|
||||
if self.beat:
|
||||
lines.append(f"BPM: {self.beat.bpm:.1f} | 情绪: {self.emotion.mood if self.emotion else 'N/A'}")
|
||||
if self.mix:
|
||||
lines.append(f"混音: {self.mix.master_path}")
|
||||
if self.export:
|
||||
lines.append(f"导出: {self.export.summary}")
|
||||
|
||||
if self.errors:
|
||||
lines.append(f"错误: {len(self.errors)} 个")
|
||||
for e in self.errors[:3]:
|
||||
lines.append(f" ⚠ {e['module']}: {e['error']}")
|
||||
|
||||
lines.append(f"成功: {'✅' if self.success else '❌'}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 编排器
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class AudioLineOrchestrator:
|
||||
"""
|
||||
音频创作管线编排器
|
||||
|
||||
按 AudioTask.modules 配置依次执行各层。
|
||||
跳过失败的模块（不阻塞整条管线）。
|
||||
"""
|
||||
|
||||
def __init__(self, config_path: str | Path | None = None):
|
||||
self.config = self._load_config(config_path)
|
||||
self.work_dir = Path(self.config.get("paths", {}).get("work_dir", "./work/audio_line"))
|
||||
self.output_dir = Path(self.config.get("paths", {}).get("output_dir", "./output/audio_line"))
|
||||
self.work_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# — 初始化各层模块 —
|
||||
self.collector = AudioCollector(str(self.work_dir))
|
||||
|
||||
sep_cfg = self.config.get("separation", {})
|
||||
self.stem_sep = StemSeparator(
|
||||
model=sep_cfg.get("model", "htdemucs_ft"),
|
||||
device=sep_cfg.get("device", "cpu"),
|
||||
shifts=sep_cfg.get("shifts", 2),
|
||||
overlap=sep_cfg.get("overlap", 0.25),
|
||||
out_dir=str(self.work_dir / "separated"),
|
||||
)
|
||||
|
||||
beat_cfg = self.config.get("beat", {})
|
||||
self.beat_det = BeatDetector(
|
||||
min_bpm=beat_cfg.get("min_bpm", 60),
|
||||
max_bpm=beat_cfg.get("max_bpm", 200),
|
||||
)
|
||||
|
||||
self.structure_parser = StructureParser()
|
||||
self.emotion_analyzer = EmotionAnalyzer()
|
||||
self.rhythm_module = RhythmModule(self.config)
|
||||
self.lyrics_module = LyricsModule(self.config)
|
||||
self.melody_module = MelodyModule(self.config)
|
||||
self.mix_module = MixModule(self.config)
|
||||
|
||||
align_cfg = self.config.get("alignment", {})
|
||||
self.lyric_aligner = LyricAligner(
|
||||
tolerance=align_cfg.get("time_tolerance", 0.1),
|
||||
)
|
||||
self.stem_aligner = StemAligner(
|
||||
method=align_cfg.get("stretch_method", "phase_vocoder"),
|
||||
crossfade=align_cfg.get("crossfade", 0.05),
|
||||
)
|
||||
|
||||
self.exporter = Exporter(self.config)
|
||||
self.lyric_formatter = LyricFormatter()
|
||||
|
||||
def run(self, task: AudioTask) -> AudioLineResult:
|
||||
"""
|
||||
执行管线
|
||||
|
||||
参数:
|
||||
task: AudioTask — 由 AudioCollector 创建的标准化任务
|
||||
|
||||
返回:
|
||||
AudioLineResult
|
||||
"""
|
||||
result = AudioLineResult(task=task)
|
||||
print(f"\n{'='*60}")
|
||||
print(f"AudioLine 管线启动")
|
||||
print(f" operation: {task.operation}")
|
||||
print(f" source: {task.source.path or '(指令模式)'}")
|
||||
print(f" modules: {', '.join(sorted(task.modules))}")
|
||||
print(f"{'='*60}\n")
|
||||
|
||||
modules_to_run = self._plan_modules(task)
|
||||
|
||||
for module_name, callable_fn, ctx_key in modules_to_run:
|
||||
if module_name not in task.modules:
|
||||
continue
|
||||
|
||||
start = time.time()
|
||||
print(f"\n--- [{module_name}] ---")
|
||||
|
||||
try:
|
||||
callable_fn(result)
|
||||
elapsed = time.time() - start
|
||||
result.modules_run.append(module_name)
|
||||
result.timing[module_name] = round(elapsed, 2)
|
||||
print(f" [{module_name}] ✅ 完成 ({elapsed:.1f}s)")
|
||||
except Exception as e:
|
||||
elapsed = time.time() - start
|
||||
result.errors.append({"module": module_name, "error": str(e)})
|
||||
result.timing[module_name] = round(elapsed, 2)
|
||||
print(f" [{module_name}] ❌ 失败: {e}")
|
||||
print(f" [{module_name}] 跳过，继续下一步")
|
||||
|
||||
result.success = len(result.errors) == 0
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print(f"管线结束 | 模块: {len(result.modules_run)}/{len(modules_to_run)}"
|
||||
f" | 错误: {len(result.errors)} | 耗时: {sum(result.timing.values()):.1f}s")
|
||||
if result.success:
|
||||
print(f"状态: ✅ 成功")
|
||||
else:
|
||||
print(f"状态: ⚠ 部分成功 ({len(result.errors)} 个错误)")
|
||||
print(f"{'='*60}\n")
|
||||
|
||||
return result
|
||||
|
||||
# ---- 模块路由 ----
|
||||
|
||||
def _plan_modules(self, task: AudioTask) -> list[tuple[str, Callable, str]]:
|
||||
"""
|
||||
规划模块执行顺序
|
||||
|
||||
返回: [(module_name, callable_fn, context_key)]
|
||||
"""
|
||||
return [
|
||||
# Layer 2 — 分析
|
||||
("separation", self._run_stem_separation, "stems"),
|
||||
("analysis", self._run_analysis, "analysis"),
|
||||
|
||||
# Layer 3 — 创作
|
||||
("rhythm", self._run_rhythm, "rhythm"),
|
||||
("lyrics", self._run_lyrics, "lyrics"),
|
||||
("melody", self._run_melody, "melody"),
|
||||
("mix", self._run_mix, "mix"),
|
||||
|
||||
# Layer 4 — 对齐
|
||||
("alignment", self._run_alignment, "alignment"),
|
||||
|
||||
# Layer 5 — 输出
|
||||
("export", self._run_export, "export"),
|
||||
]
|
||||
|
||||
# ---- Layer 2 ----
|
||||
|
||||
def _run_stem_separation(self, result: AudioLineResult):
|
||||
"""音轨分离"""
|
||||
task = result.task
|
||||
if not task.source.path:
|
||||
print(" [orchestrator] ⚠ 无音频源，跳过音轨分离")
|
||||
return
|
||||
|
||||
result.stems = self.stem_sep.separate(task.source.path)
|
||||
|
||||
def _run_analysis(self, result: AudioLineResult):
|
||||
"""执行所有分析"""
|
||||
# 用原始音频或 vocals stem 分析
|
||||
audio_for_analysis = result.stems.get("vocals", "")
|
||||
if not audio_for_analysis or not Path(audio_for_analysis).exists():
|
||||
audio_for_analysis = result.task.source.path
|
||||
|
||||
if not audio_for_analysis:
|
||||
print(" [orchestrator] ⚠ 无音频可用于分析")
|
||||
return
|
||||
|
||||
# — BPM/节拍 —
|
||||
result.beat = self.beat_det.detect(audio_for_analysis)
|
||||
|
||||
# — 结构解析 —
|
||||
result.structure = self.structure_parser.parse(
|
||||
audio_for_analysis, result.beat.beats
|
||||
)
|
||||
|
||||
# — 情绪分析 —
|
||||
result.emotion = self.emotion_analyzer.analyze(
|
||||
audio_for_analysis, result.beat.bpm
|
||||
)
|
||||
|
||||
# ---- Layer 3 ----
|
||||
|
||||
def _run_rhythm(self, result: AudioLineResult):
|
||||
"""节奏重塑"""
|
||||
task = result.task
|
||||
bpm = result.beat.bpm if result.beat else 120.0
|
||||
target_bpm = task.target_bpm or bpm
|
||||
|
||||
output_dir = self.output_dir / "rhythm"
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
result.rhythm = self.rhythm_module.generate(
|
||||
bpm=target_bpm,
|
||||
genre=task.style,
|
||||
duration=task.duration or 30,
|
||||
output_path=str(output_dir / "drum_track.wav"),
|
||||
)
|
||||
|
||||
def _run_lyrics(self, result: AudioLineResult):
|
||||
"""歌词创作"""
|
||||
# 如果有原始歌词，从原曲分析获取
|
||||
original_lyrics = "" # 默认空，需要时用户提供
|
||||
|
||||
output_dir = self.output_dir / "lyrics"
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 仅当有主题时执行改写
|
||||
if result.task.topic:
|
||||
result.lyrics = self.lyrics_module.rewrite(
|
||||
original_lyrics=original_lyrics,
|
||||
topic=result.task.topic,
|
||||
style=result.task.style,
|
||||
output_path=str(output_dir / "rewritten_lyrics.txt"),
|
||||
)
|
||||
else:
|
||||
print(" [orchestrator] ⚠ 无歌词主题，跳过歌词改写")
|
||||
|
||||
def _run_melody(self, result: AudioLineResult):
|
||||
"""旋律编曲"""
|
||||
audio_path = result.stems.get("vocals", "") or result.task.source.path
|
||||
if not audio_path:
|
||||
print(" [orchestrator] ⚠ 无音频源，跳过旋律编曲")
|
||||
return
|
||||
|
||||
output_dir = self.output_dir / "melody"
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
result.melody = self.melody_module.remix(
|
||||
audio_path=audio_path,
|
||||
target_style=result.task.style,
|
||||
duration=result.task.duration or 0,
|
||||
output_path=str(output_dir / "remix.wav"),
|
||||
)
|
||||
|
||||
def _run_mix(self, result: AudioLineResult):
|
||||
"""混音母带"""
|
||||
if not result.stems:
|
||||
print(" [orchestrator] ⚠ 无分轨可供混音，跳过")
|
||||
return
|
||||
|
||||
output_dir = self.output_dir / "mix"
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 如果有生成的鼓点轨道，加入混音
|
||||
stems_to_mix = dict(result.stems)
|
||||
if result.rhythm and result.rhythm.drum_track_path:
|
||||
stems_to_mix["drum_track"] = result.rhythm.drum_track_path
|
||||
|
||||
result.mix = self.mix_module.mix(
|
||||
stems=stems_to_mix,
|
||||
output_path=str(output_dir / "master.wav"),
|
||||
include_stems=True,
|
||||
)
|
||||
|
||||
# ---- Layer 4 ----
|
||||
|
||||
def _run_alignment(self, result: AudioLineResult):
|
||||
"""对齐处理"""
|
||||
# — 歌词对齐 —
|
||||
if result.lyrics and result.beat:
|
||||
lyrics_text = result.lyrics.rewritten_lyrics or result.lyrics.original_lyrics
|
||||
sections = result.structure.segments if result.structure else None
|
||||
|
||||
result.lyric_timeline = self.lyric_aligner.align(
|
||||
lyrics=lyrics_text,
|
||||
beats=result.beat.beats,
|
||||
sections=sections,
|
||||
total_duration=result.beat.duration_sec,
|
||||
)
|
||||
|
||||
# — 分轨对齐 —
|
||||
if result.stems and result.beat:
|
||||
# 如果有目标 BPM，进行 BPM 对齐
|
||||
task = result.task
|
||||
target_bpm = task.target_bpm
|
||||
if target_bpm and target_bpm != result.beat.bpm:
|
||||
output_dir = self.output_dir / "aligned"
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
result.stem_alignment = self.stem_aligner.align_to_bpm(
|
||||
stems=result.stems,
|
||||
source_bpm=result.beat.bpm,
|
||||
target_bpm=target_bpm,
|
||||
output_dir=str(output_dir),
|
||||
)
|
||||
|
||||
# ---- Layer 5 ----
|
||||
|
||||
def _run_export(self, result: AudioLineResult):
|
||||
"""导出"""
|
||||
output_dir = self.output_dir / "export"
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# — 导出混音 —
|
||||
master_path = result.mix.master_path if result.mix else ""
|
||||
if master_path and Path(master_path).exists():
|
||||
result.export = self.exporter.export_master(
|
||||
master_path=master_path,
|
||||
output_dir=str(output_dir),
|
||||
)
|
||||
|
||||
# — 导出带歌词版本 —
|
||||
if result.lyric_timeline:
|
||||
result.export = self.exporter.export_with_lyrics(
|
||||
master_path=master_path,
|
||||
lyric_timeline=result.lyric_timeline,
|
||||
output_dir=str(output_dir),
|
||||
)
|
||||
|
||||
# — 歌词格式化 —
|
||||
if result.lyric_timeline and result.lyric_timeline.entries:
|
||||
result.formatted_lyrics = self.lyric_formatter.format_all(
|
||||
entries=result.lyric_timeline.entries,
|
||||
total_duration=result.lyric_timeline.total_duration,
|
||||
title=f"AudioLine_{result.task.operation}",
|
||||
)
|
||||
result.formatted_lyrics.save_all(
|
||||
str(output_dir / "lyrics"),
|
||||
basename="lyrics",
|
||||
)
|
||||
|
||||
# ---- 配置加载 ----
|
||||
|
||||
@staticmethod
|
||||
def _load_config(config_path: str | Path | None) -> dict:
|
||||
"""加载 YAML 配置"""
|
||||
if config_path and Path(config_path).exists():
|
||||
with open(config_path, "r", encoding="utf-8") as f:
|
||||
return yaml.safe_load(f) or {}
|
||||
|
||||
# 尝试默认路径
|
||||
for path in [
|
||||
"./config.yaml",
|
||||
"./config/local.yaml",
|
||||
]:
|
||||
p = Path(path)
|
||||
if p.exists():
|
||||
with open(p, "r", encoding="utf-8") as f:
|
||||
return yaml.safe_load(f) or {}
|
||||
|
||||
print("[orchestrator] ⚠ 未找到配置文件，使用默认值")
|
||||
return {}
|
||||
@@ -0,0 +1,10 @@
|
||||
"""
|
||||
AVE 07_bgm_generator — 背景音乐生成 (三阶路由)
|
||||
|
||||
用法:
|
||||
from bgm_generator.suno import generate_bgm, get_available_moods
|
||||
generate_bgm(mood="calm", duration=30, output="/tmp/bgm.wav")
|
||||
"""
|
||||
from .suno import generate_bgm, get_available_moods
|
||||
|
||||
__all__ = ["generate_bgm", "get_available_moods"]
|
||||
@@ -0,0 +1,203 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
AVE BGM 音乐库助手 v1.0
|
||||
下载 Pixabay Music 免费可商用 BGM 到本地库
|
||||
|
||||
用法:
|
||||
python bgm_download.py search --mood calm
|
||||
python bgm_download.py list
|
||||
python bgm_download.py scan # 扫描已下载文件更新 library.json
|
||||
python bgm_download.py import --file /path/to/music.mp3 --mood happy
|
||||
|
||||
注意:
|
||||
Pixabay Music 没有公开 API，本脚本提供两种方式添加音乐:
|
||||
1. 打开 https://pixabay.com/music/ 手动下载 → 放到对应情绪目录
|
||||
2. 运行 scan 自动扫描目录更新 library.json
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, os.path.join(SCRIPT_DIR, ".."))
|
||||
|
||||
from lib.logger import get_logger
|
||||
|
||||
logger = get_logger("bgm_lib")
|
||||
|
||||
# ── 常量 ──
|
||||
LIBRARY_DIR = Path.home() / "workbuddy-agent-os" / "agent-local" / "tools" / "ave" / "cache" / "bgm_library"
|
||||
LIBRARY_CONFIG = LIBRARY_DIR / "library.json"
|
||||
|
||||
MOODS = ["calm", "soothing", "happy", "excited", "sad", "mystery",
|
||||
"professional", "funny", "inspiring", "normal"]
|
||||
|
||||
|
||||
def main():
|
||||
setup()
|
||||
parser = argparse.ArgumentParser(description="BGM 音乐库助手")
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
# list
|
||||
sub.add_parser("list", help="列出库中所有音乐")
|
||||
|
||||
# scan
|
||||
sub.add_parser("scan", help="扫描文件系统更新 library.json")
|
||||
|
||||
# search (open browser)
|
||||
p_search = sub.add_parser("search", help="打开浏览器搜索指定情绪的 BGM")
|
||||
p_search.add_argument("--mood", default="calm", choices=MOODS, help="情绪")
|
||||
|
||||
# import
|
||||
p_import = sub.add_parser("import", help="手动导入一个音乐文件")
|
||||
p_import.add_argument("--file", required=True, help="音乐文件路径")
|
||||
p_import.add_argument("--mood", required=True, choices=MOODS, help="情绪分类")
|
||||
p_import.add_argument("--title", default="", help="曲目标题")
|
||||
p_import.add_argument("--source", default="manual", help="来源")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.command == "list":
|
||||
cmd_list()
|
||||
elif args.command == "scan":
|
||||
cmd_scan()
|
||||
elif args.command == "search":
|
||||
cmd_search(args.mood)
|
||||
elif args.command == "import":
|
||||
cmd_import(args.file, args.mood, args.title, args.source)
|
||||
else:
|
||||
parser.print_help()
|
||||
|
||||
|
||||
def setup():
|
||||
"""确保目录结构存在"""
|
||||
LIBRARY_DIR.mkdir(parents=True, exist_ok=True)
|
||||
for m in MOODS:
|
||||
(LIBRARY_DIR / m).mkdir(parents=True, exist_ok=True)
|
||||
if not LIBRARY_CONFIG.exists():
|
||||
logger.info("初始化 library.json")
|
||||
cfg = {"库": {m: {"曲目": [], "说明": ""} for m in MOODS}}
|
||||
cfg["基础路径"] = str(LIBRARY_DIR)
|
||||
cfg["版本"] = "v1.0"
|
||||
with open(LIBRARY_CONFIG, "w", encoding="utf-8") as f:
|
||||
json.dump(cfg, f, ensure_ascii=False, indent=2)
|
||||
|
||||
|
||||
def cmd_list():
|
||||
"""列出所有已导入的音乐"""
|
||||
cfg = load_config()
|
||||
total = 0
|
||||
for mood in MOODS:
|
||||
tracks = cfg.get("库", {}).get(mood, {}).get("曲目", [])
|
||||
valid = [t for t in tracks if t.get("文件名")]
|
||||
if valid:
|
||||
print(f"\n [{mood}] ({len(valid)} 首)")
|
||||
for t in valid:
|
||||
fname = t.get("文件名", "?")
|
||||
title = t.get("标题", fname)
|
||||
duration = t.get("时长", 0)
|
||||
print(f" {title:30s} {duration:4.1f}s {fname}")
|
||||
total += len(valid)
|
||||
print(f"\n 共 {total} 首 BGM")
|
||||
|
||||
|
||||
def cmd_scan():
|
||||
"""扫描目录更新 library.json"""
|
||||
cfg = load_config()
|
||||
new_count = 0
|
||||
for mood in MOODS:
|
||||
mood_dir = LIBRARY_DIR / mood
|
||||
if not mood_dir.exists():
|
||||
continue
|
||||
existing = {t.get("文件名") for t in cfg["库"][mood]["曲目"]}
|
||||
for f in sorted(mood_dir.iterdir()):
|
||||
if f.suffix.lower() in (".mp3", ".wav", ".m4a", ".ogg", ".flac"):
|
||||
if f.name not in existing:
|
||||
duration = get_duration(str(f))
|
||||
cfg["库"][mood]["曲目"].append({
|
||||
"文件名": f.name,
|
||||
"标题": f.stem,
|
||||
"时长": duration,
|
||||
"源": "scan_import",
|
||||
})
|
||||
new_count += 1
|
||||
print(f" ✅ 新增 [{mood}] {f.name} ({duration:.1f}s)")
|
||||
|
||||
save_config(cfg)
|
||||
print(f"\n扫描完成，新增 {new_count} 首")
|
||||
|
||||
|
||||
def cmd_search(mood: str):
|
||||
"""打开浏览器搜索 Pixabay Music"""
|
||||
urls = {
|
||||
"calm": "https://pixabay.com/music/search/calm/",
|
||||
"soothing": "https://pixabay.com/music/search/relaxing/",
|
||||
"happy": "https://pixabay.com/music/search/upbeat/",
|
||||
"excited": "https://pixabay.com/music/search/energetic/",
|
||||
"sad": "https://pixabay.com/music/search/sad/",
|
||||
"mystery": "https://pixabay.com/music/search/cinematic/",
|
||||
"professional": "https://pixabay.com/music/search/corporate/",
|
||||
"funny": "https://pixabay.com/music/search/playful/",
|
||||
"inspiring": "https://pixabay.com/music/search/inspiring/",
|
||||
"normal": "https://pixabay.com/music/search/background/",
|
||||
}
|
||||
url = urls.get(mood, urls["normal"])
|
||||
print(f" 打开浏览器: {url}")
|
||||
print(f" 操作: 试听 → 免费下载 → 放入 bgm_library/{mood}/ 目录")
|
||||
print(f" 然后运行: python bgm_download.py scan 自动入库")
|
||||
subprocess.run(["open", url])
|
||||
|
||||
|
||||
def cmd_import(file_path: str, mood: str, title: str, source: str):
|
||||
"""导入一个音乐文件"""
|
||||
if not os.path.exists(file_path):
|
||||
logger.error(f"文件不存在: {file_path}")
|
||||
return
|
||||
|
||||
dest = LIBRARY_DIR / mood / os.path.basename(file_path)
|
||||
if os.path.exists(dest):
|
||||
logger.warning(f"文件已存在: {dest}")
|
||||
return
|
||||
|
||||
import shutil
|
||||
shutil.copy2(file_path, dest)
|
||||
duration = get_duration(str(dest))
|
||||
|
||||
cfg = load_config()
|
||||
cfg["库"][mood]["曲目"].append({
|
||||
"文件名": dest.name,
|
||||
"标题": title or dest.stem,
|
||||
"时长": duration,
|
||||
"源": source,
|
||||
})
|
||||
save_config(cfg)
|
||||
print(f"✅ 导入完成: {dest.name} ({duration:.1f}s) → [{mood}]")
|
||||
|
||||
|
||||
def get_duration(file_path: str) -> float:
|
||||
"""获取音频时长（用 lib.ffmpeg —— 本机 ffprobe 实为 ffmpeg 符号链接）"""
|
||||
try:
|
||||
from lib.ffmpeg import probe_duration
|
||||
d = probe_duration(file_path)
|
||||
return round(d, 1) if d > 0 else 30.0
|
||||
except Exception:
|
||||
return 30.0
|
||||
|
||||
|
||||
def load_config() -> dict:
|
||||
if LIBRARY_CONFIG.exists():
|
||||
with open(LIBRARY_CONFIG, encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
return {"库": {}}
|
||||
|
||||
|
||||
def save_config(cfg: dict):
|
||||
with open(LIBRARY_CONFIG, "w", encoding="utf-8") as f:
|
||||
json.dump(cfg, f, ensure_ascii=False, indent=2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,164 @@
|
||||
"""
|
||||
|
||||
AVE 07_bgm_generator — Tier 3 FFmpeg 和弦垫音
|
||||
|
||||
FFmpeg 和弦氛围音生成，作为离线保障 (v2 移植)
|
||||
|
||||
用法:
|
||||
generate_chord_pad("funny", 60, "/tmp/bgm.wav")
|
||||
"""
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from lib.logger import get_logger
|
||||
|
||||
logger = get_logger("bgm_chord")
|
||||
|
||||
CACHE_DIR = Path(os.environ.get("AVE_CACHE_DIR",
|
||||
str(Path.home() / "workbuddy-agent-os/agent-local/tools/ave/cache/bgm")))
|
||||
CACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# ── 和弦进行预设 ──
|
||||
CHORD_PRESETS = {
|
||||
"calm": {
|
||||
"chords": [[220, 277, 330], [196, 247, 294], [261, 329, 392], [220, 277, 330]],
|
||||
"bpm": 60, "lowpass": 800, "reverb_decay": 0.6, "vol_factor": 0.08,
|
||||
},
|
||||
"soothing": {
|
||||
"chords": [[196, 233, 294], [220, 261, 330], [175, 220, 277], [196, 233, 294]],
|
||||
"bpm": 55, "lowpass": 700, "reverb_decay": 0.7, "vol_factor": 0.07,
|
||||
},
|
||||
"happy": {
|
||||
"chords": [[262, 330, 392], [330, 415, 523], [392, 494, 587], [440, 554, 659]],
|
||||
"bpm": 90, "lowpass": 1200, "reverb_decay": 0.3, "vol_factor": 0.06,
|
||||
},
|
||||
"excited": {
|
||||
"chords": [[294, 370, 440], [349, 440, 554], [392, 493, 587], [440, 554, 659]],
|
||||
"bpm": 110, "lowpass": 1500, "reverb_decay": 0.2, "vol_factor": 0.05,
|
||||
},
|
||||
"sad": {
|
||||
"chords": [[165, 196, 247], [147, 175, 220], [131, 165, 196], [147, 175, 220]],
|
||||
"bpm": 50, "lowpass": 600, "reverb_decay": 0.8, "vol_factor": 0.09,
|
||||
},
|
||||
"mystery": {
|
||||
"chords": [[165, 208, 262], [147, 185, 233], [131, 175, 220], [139, 185, 233]],
|
||||
"bpm": 50, "lowpass": 500, "reverb_decay": 0.9, "vol_factor": 0.10,
|
||||
},
|
||||
"angry": {
|
||||
"chords": [[131, 165, 196], [165, 208, 262], [147, 185, 233], [175, 220, 277]],
|
||||
"bpm": 130, "lowpass": 1000, "reverb_decay": 0.4, "vol_factor": 0.06,
|
||||
},
|
||||
"professional": {
|
||||
"chords": [[262, 330, 392], [392, 494, 587], [262, 330, 392], [330, 415, 523]],
|
||||
"bpm": 70, "lowpass": 1000, "reverb_decay": 0.4, "vol_factor": 0.06,
|
||||
},
|
||||
"normal": {
|
||||
"chords": [[220, 277, 330], [261, 329, 392], [196, 247, 294], [261, 329, 392]],
|
||||
"bpm": 70, "lowpass": 900, "reverb_decay": 0.5, "vol_factor": 0.07,
|
||||
},
|
||||
"funny": {
|
||||
"chords": [[523, 659, 784], [659, 784, 988], [784, 988, 1175], [523, 659, 784]],
|
||||
"bpm": 100, "lowpass": 1400, "reverb_decay": 0.25, "vol_factor": 0.04,
|
||||
},
|
||||
"inspiring": {
|
||||
"chords": [[262, 330, 392, 523], [294, 370, 440, 587], [330, 415, 523, 659], [392, 494, 587, 784]],
|
||||
"bpm": 80, "lowpass": 1200, "reverb_decay": 0.5, "vol_factor": 0.05,
|
||||
},
|
||||
# ── BGM_LIBRARY 扩展 ──
|
||||
"epic": {
|
||||
"chords": [[262, 330, 392], [330, 415, 523], [392, 494, 587], [440, 554, 659]],
|
||||
"bpm": 120, "lowpass": 1500, "reverb_decay": 0.3, "vol_factor": 0.06,
|
||||
},
|
||||
"tech": {
|
||||
"chords": [[294, 370, 440], [349, 440, 554], [392, 493, 587], [440, 554, 659]],
|
||||
"bpm": 120, "lowpass": 1800, "reverb_decay": 0.2, "vol_factor": 0.05,
|
||||
},
|
||||
"warm": {
|
||||
"chords": [[220, 277, 330], [196, 247, 294], [261, 329, 392], [220, 277, 330]],
|
||||
"bpm": 80, "lowpass": 700, "reverb_decay": 0.6, "vol_factor": 0.07,
|
||||
},
|
||||
"cinematic": {
|
||||
"chords": [[175, 220, 277], [196, 247, 294], [220, 277, 330], [261, 329, 392]],
|
||||
"bpm": 80, "lowpass": 900, "reverb_decay": 0.7, "vol_factor": 0.08,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def generate_chord_pad(mood: str, duration: float, output: str) -> str:
|
||||
"""生成和弦垫音 (Tier 3)"""
|
||||
preset = CHORD_PRESETS.get(mood, CHORD_PRESETS["normal"])
|
||||
chord_freqs_list = preset["chords"]
|
||||
bpm = preset["bpm"]
|
||||
lowpass_freq = preset["lowpass"]
|
||||
reverb_decay = preset["reverb_decay"]
|
||||
vol_factor = preset["vol_factor"]
|
||||
|
||||
num_chords = len(chord_freqs_list)
|
||||
chord_duration = duration / num_chords
|
||||
transition = min(2.0, chord_duration * 0.3)
|
||||
|
||||
logger.info(f" Tier 3: {num_chords} 和弦 × {chord_duration:.1f}s, "
|
||||
f"低通={lowpass_freq}Hz, 混响={reverb_decay}")
|
||||
|
||||
# 生成每个和弦
|
||||
chord_wavs = []
|
||||
for idx, freqs in enumerate(chord_freqs_list):
|
||||
part_file = str(CACHE_DIR / f"_chord_{mood}_{idx}.wav")
|
||||
if os.path.exists(part_file):
|
||||
try: os.remove(part_file)
|
||||
except: pass
|
||||
|
||||
expressions = []
|
||||
for freq in freqs:
|
||||
for detune in [0.995, 1.000, 1.005]:
|
||||
expressions.append(f"sin(2*PI*{freq * detune}*t)")
|
||||
expr = f"({'+'.join(expressions)})/{len(expressions)}"
|
||||
|
||||
subprocess.run([
|
||||
"ffmpeg", "-y", "-f", "lavfi",
|
||||
"-i", f"aevalsrc={expr}:s=44100:c=mono",
|
||||
"-t", str(chord_duration),
|
||||
"-af", f"afade=t=in:ss=0:d={transition},"
|
||||
f"afade=t=out:st={chord_duration - transition}:d={transition},"
|
||||
f"volume={vol_factor}",
|
||||
"-acodec", "pcm_s16le", part_file,
|
||||
], capture_output=True, timeout=120, check=True)
|
||||
chord_wavs.append(part_file)
|
||||
|
||||
# 拼接
|
||||
concat_file = tempfile.NamedTemporaryFile(suffix=".txt", delete=False, mode="w").name
|
||||
with open(concat_file, "w", encoding="utf-8") as f:
|
||||
for w in chord_wavs:
|
||||
f.write(f"file '{w}'\n")
|
||||
|
||||
# 全局效果
|
||||
if reverb_decay < 0.5:
|
||||
delays = "50|80|110"
|
||||
gains = f"{reverb_decay*0.6}|{reverb_decay*0.4}|{reverb_decay*0.2}"
|
||||
else:
|
||||
delays = "60|100|150|200"
|
||||
gains = f"{reverb_decay*0.6}|{reverb_decay*0.4}|{reverb_decay*0.2}|{reverb_decay*0.1}"
|
||||
|
||||
try:
|
||||
subprocess.run([
|
||||
"ffmpeg", "-y", "-f", "concat", "-safe", "0", "-i", concat_file,
|
||||
"-af", f"lowpass=f={lowpass_freq},aecho=0.8:0.7:{delays}:{gains}",
|
||||
"-acodec", "pcm_s16le", output,
|
||||
], capture_output=True, timeout=120, check=True)
|
||||
except subprocess.CalledProcessError:
|
||||
logger.warning("全局效果失败，回落无效果拼接")
|
||||
subprocess.run([
|
||||
"ffmpeg", "-y", "-f", "concat", "-safe", "0", "-i", concat_file,
|
||||
"-acodec", "pcm_s16le", output,
|
||||
], capture_output=True, check=True)
|
||||
|
||||
# 清理
|
||||
for w in chord_wavs:
|
||||
try: os.remove(w)
|
||||
except: pass
|
||||
try: os.remove(concat_file)
|
||||
except: pass
|
||||
|
||||
return output
|
||||
@@ -0,0 +1,273 @@
|
||||
"""
|
||||
|
||||
AVE 07_bgm_generator — 背景音乐生成模块 v3.0
|
||||
|
||||
版本: v3.0 | 更新: 2026-05-06
|
||||
|
||||
三阶路由策略:
|
||||
Tier 1: 本地 BGM 音乐库 (真实音乐，最高质量)
|
||||
Tier 2: mlx-audiocraft AI 生成 (本地 MLX，按需定制)
|
||||
Tier 3: FFmpeg 和弦垫音 (离线保障)
|
||||
|
||||
Tier 1 操作流程:
|
||||
1. 从 library.json 读取 mood→曲目映射
|
||||
2. 如果有匹配曲目 → 选择一首，用 FFmpeg 循环/截取到目标时长
|
||||
3. 应用淡入淡出过渡
|
||||
4. 混音时音量设为背景级 (0.15-0.25)
|
||||
|
||||
用法:
|
||||
generate_bgm(mood="funny", duration=60, output="/tmp/bgm.wav")
|
||||
generate_bgm(mood="happy", duration=30, output="/tmp/bgm.wav")
|
||||
"""
|
||||
import sys, os; sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from lib.logger import get_logger
|
||||
|
||||
logger = get_logger("bgm")
|
||||
|
||||
# ── 路径 ──
|
||||
LIBRARY_DIR = Path.home() / "workbuddy-agent-os" / "agent-local" / "tools" / "ave" / "cache" / "bgm_library"
|
||||
LIBRARY_CONFIG = LIBRARY_DIR / "library.json"
|
||||
CACHE_DIR = Path(os.environ.get("AVE_CACHE_DIR",
|
||||
str(Path.home() / "workbuddy-agent-os/agent-local/tools/ave/cache/bgm")))
|
||||
CACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def generate_bgm(
|
||||
mood: str = "normal",
|
||||
duration: float = 30.0,
|
||||
output: str = "bgm.wav",
|
||||
pixabay_key: str = "",
|
||||
style: str = "",
|
||||
use_mlx: bool = False,
|
||||
) -> str:
|
||||
"""
|
||||
三阶背景音乐生成 v3.0
|
||||
|
||||
参数:
|
||||
mood: 情绪类型
|
||||
duration: 目标时长(秒)
|
||||
output: 输出路径
|
||||
pixabay_key: (保留，暂未用)
|
||||
style: 风格别名
|
||||
use_mlx: 是否尝试 mlx-audiocraft (需提前安装)
|
||||
|
||||
返回:
|
||||
生成的 BGM 文件路径
|
||||
"""
|
||||
if style and not mood:
|
||||
mood = _style_to_mood(style)
|
||||
|
||||
effective_mood = mood or "normal"
|
||||
logger.info(f"BGM v3: mood={effective_mood}, duration={duration}s, output={output}")
|
||||
|
||||
# ── Tier 1: 音乐库 ──
|
||||
result = _try_library(effective_mood, duration, output)
|
||||
if result:
|
||||
logger.info(f"✅ Tier 1 (音乐库) → {output}")
|
||||
return result
|
||||
|
||||
# ── Tier 2: mlx-audiocraft (按需) ──
|
||||
if use_mlx:
|
||||
result = _try_mlx_audiocraft(effective_mood, duration, output)
|
||||
if result:
|
||||
logger.info(f"✅ Tier 2 (mlx-audiocraft) → {output}")
|
||||
return result
|
||||
logger.warning("mlx-audiocraft 回落 → Tier 3")
|
||||
|
||||
# ── Tier 3: FFmpeg 和弦垫音 ──
|
||||
result = _chord_pad(effective_mood, duration, output)
|
||||
logger.info(f"✅ Tier 3 (和弦垫音) → {output}")
|
||||
return result
|
||||
|
||||
|
||||
def _try_library(mood: str, duration: float, output: str) -> str | None:
|
||||
"""Tier 1: 从本地音乐库载入 BGM 并匹配时长"""
|
||||
if not LIBRARY_CONFIG.exists():
|
||||
logger.info(" library.json 不存在，跳过 Tier 1")
|
||||
return None
|
||||
|
||||
with open(LIBRARY_CONFIG, encoding="utf-8") as f:
|
||||
lib = json.load(f)
|
||||
|
||||
tracks = lib.get("库", {}).get(mood, {}).get("曲目", [])
|
||||
valid = [t for t in tracks if t.get("文件名")]
|
||||
if not valid:
|
||||
logger.info(f" [{mood}] 库中无曲目")
|
||||
return None
|
||||
|
||||
# 选择曲目: 优先选最接近目标时长的，没有完美匹配就选最短的
|
||||
chosen = min(valid, key=lambda t: abs((t.get("时长") or 30) - duration))
|
||||
track_file = LIBRARY_DIR / mood / chosen["文件名"]
|
||||
|
||||
if not track_file.exists():
|
||||
logger.warning(f" 曲目文件不存在: {track_file}")
|
||||
return None
|
||||
|
||||
track_duration = chosen.get("时长", 30)
|
||||
title = chosen.get("标题", chosen["文件名"])
|
||||
logger.info(f" [{mood}] 选用: {title} ({track_duration:.1f}s) → 适配 {duration:.0f}s")
|
||||
|
||||
os.makedirs(os.path.dirname(output) or ".", exist_ok=True)
|
||||
|
||||
# FFmpeg 循环/截取
|
||||
if track_duration >= duration:
|
||||
# 截取前段
|
||||
cmd = [
|
||||
"ffmpeg", "-y",
|
||||
"-i", str(track_file),
|
||||
"-t", str(duration),
|
||||
"-af", "afade=t=in:d=1,afade=t=out:st={}:d=2".format(max(0, duration - 2)),
|
||||
"-acodec", "pcm_s16le",
|
||||
output,
|
||||
]
|
||||
else:
|
||||
# 循环补齐
|
||||
loops = int(duration / track_duration) + 1
|
||||
cmd = [
|
||||
"ffmpeg", "-y",
|
||||
"-stream_loop", str(loops),
|
||||
"-i", str(track_file),
|
||||
"-t", str(duration),
|
||||
"-af", "afade=t=in:d=1,afade=t=out:st={}:d=2".format(max(0, duration - 2)),
|
||||
"-acodec", "pcm_s16le",
|
||||
output,
|
||||
]
|
||||
|
||||
try:
|
||||
subprocess.run(cmd, capture_output=True, check=True, timeout=30)
|
||||
return output
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.warning(f" FFmpeg 处理失败: {e.stderr[:200]}")
|
||||
return None
|
||||
|
||||
|
||||
def _try_mlx_audiocraft(mood: str, duration: float, output: str) -> str | None:
|
||||
"""Tier 2: 使用 mlx-audiocraft 生成音乐 (需 pip install mlx-audiocraft)
|
||||
|
||||
注意事项:
|
||||
- 国内用户需设置 HF_ENDPOINT=https://hf-mirror.com 下载模型
|
||||
- 首次运行会下载 ~300MB 模型 (musicgen-small)
|
||||
- 模型缓存后二次加载约 15s
|
||||
- M1 上 10s 音乐生成约 46s
|
||||
"""
|
||||
try:
|
||||
import mlx_audiocraft
|
||||
except ImportError:
|
||||
logger.info(" mlx-audiocraft 未安装，跳过 Tier 2")
|
||||
return None
|
||||
|
||||
# mood→prompt 映射 (与 BGM_LIBRARY.md 同步)
|
||||
prompts = {
|
||||
"calm": "soft ambient piano, gentle pad, nature sounds, 60 BPM, no percussion, no vocals",
|
||||
"soothing": "warm ambient drone, slow strings, peaceful atmosphere, 50 BPM, no vocals",
|
||||
"happy": "bright ukulele, cheerful melody, acoustic guitar, 90 BPM, no vocals",
|
||||
"excited": "upbeat electronic, driving beat, synth pads, 120 BPM, no vocals",
|
||||
"sad": "melancholic piano, slow strings, emotional, 55 BPM, no vocals",
|
||||
"mystery": "dark ambient, deep synth pad, suspenseful, 60 BPM, no vocals",
|
||||
"angry": "intense orchestral, heavy drums, dramatic, 110 BPM, no vocals",
|
||||
"professional": "corporate soft jazz, clean piano, polished, 75 BPM, no vocals",
|
||||
"funny": "playful xylophone, quirky light music, cartoon style, 100 BPM, no vocals",
|
||||
"inspiring": "uplifting cinematic, building strings, triumphant, 85 BPM, no vocals",
|
||||
"normal": "soft background music, gentle, ambient, 70 BPM, no vocals",
|
||||
# ── BGM_LIBRARY.md 扩展 ──
|
||||
"epic": "epic orchestral, brass and strings, powerful drums, triumphant, building tension, 120 BPM, no vocals",
|
||||
"tech": "electronic futuristic, synth pads, pulsing beat, modern, innovative, 120 BPM, no vocals",
|
||||
"warm": "warm acoustic guitar, soft piano, gentle, cozy atmosphere, 80 BPM, no vocals",
|
||||
"cinematic": "cinematic ambient, orchestral swells, emotional strings, majestic, 80 BPM, no vocals",
|
||||
"documentary": "documentary style, soft piano, ambient strings, professional, serious, 90 BPM, no vocals",
|
||||
}
|
||||
prompt = prompts.get(mood, prompts["normal"])
|
||||
|
||||
logger.info(f" mlx-audiocraft: {prompt}")
|
||||
try:
|
||||
from mlx_audiocraft import MusicGen
|
||||
import os as _os
|
||||
|
||||
# 设置国内镜像 (如果没设且连不上 huggingface)
|
||||
if "HF_ENDPOINT" not in _os.environ:
|
||||
_os.environ["HF_ENDPOINT"] = "https://hf-mirror.com"
|
||||
|
||||
# 加载模型 (会缓存，第二次更快)
|
||||
gen_duration = min(duration, 15) # 音乐生成最长 15s
|
||||
model = MusicGen.get_pretrained("facebook/musicgen-small")
|
||||
|
||||
# 生成
|
||||
import mlx.core as mx
|
||||
wav = model.generate(
|
||||
descriptions=[prompt],
|
||||
progress=False,
|
||||
)
|
||||
# wav 是 mlx.core.array, shape=(1, samples), float32, 32000Hz
|
||||
audio = wav[0] # shape: (samples,)
|
||||
sample_rate = model.sample_rate # 32000
|
||||
|
||||
# 转为 numpy
|
||||
import numpy as np
|
||||
audio_np = np.array(audio, dtype=np.float32)
|
||||
|
||||
# 保存为 raw float -> FFmpeg 转 WAV (避免 soundfile/scipy 依赖问题)
|
||||
raw_path = str(CACHE_DIR / f"_mlx_raw_{mood}.f32")
|
||||
audio_np.tofile(raw_path)
|
||||
|
||||
# 用 FFmpeg 转 44.1kHz WAV 并调整时长
|
||||
if duration <= gen_duration:
|
||||
# 截取
|
||||
subprocess.run([
|
||||
"ffmpeg", "-y",
|
||||
"-f", "f32le", "-ar", str(sample_rate), "-ac", "1",
|
||||
"-i", raw_path,
|
||||
"-t", str(duration),
|
||||
"-ar", "44100",
|
||||
"-af", "afade=t=in:d=1,afade=t=out:st={}:d=2".format(max(0, duration - 2)),
|
||||
"-acodec", "pcm_s16le", output,
|
||||
], capture_output=True, check=True, timeout=30)
|
||||
else:
|
||||
# 循环
|
||||
loops = int(duration / gen_duration) + 1
|
||||
subprocess.run([
|
||||
"ffmpeg", "-y",
|
||||
"-stream_loop", str(loops),
|
||||
"-f", "f32le", "-ar", str(sample_rate), "-ac", "1",
|
||||
"-i", raw_path,
|
||||
"-t", str(duration),
|
||||
"-ar", "44100",
|
||||
"-af", "afade=t=in:d=1,afade=t=out:st={}:d=2".format(max(0, duration - 2)),
|
||||
"-acodec", "pcm_s16le", output,
|
||||
], capture_output=True, check=True, timeout=30)
|
||||
|
||||
# 清理
|
||||
try: _os.remove(raw_path)
|
||||
except: pass
|
||||
|
||||
return output
|
||||
except Exception as e:
|
||||
logger.warning(f" mlx-audiocraft 失败: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def _chord_pad(mood: str, duration: float, output: str) -> str:
|
||||
"""Tier 3: FFmpeg 和弦垫音 (v2 保留)"""
|
||||
from .chord_pad import generate_chord_pad
|
||||
return generate_chord_pad(mood, duration, output)
|
||||
|
||||
|
||||
def _style_to_mood(style: str) -> str:
|
||||
s = style.lower()
|
||||
if "upbeat" in s or "playful" in s: return "happy"
|
||||
if "cinematic" in s or "dramatic" in s: return "mystery"
|
||||
if "jazz" in s or "corporate" in s: return "professional"
|
||||
if "nature" in s or "ambient" in s: return "calm"
|
||||
if "funny" in s or "cartoon" in s: return "funny"
|
||||
if "inspiring" in s or "motivational" in s: return "inspiring"
|
||||
return "normal"
|
||||
|
||||
|
||||
def get_available_moods() -> list[str]:
|
||||
from .chord_pad import CHORD_PRESETS
|
||||
return list(CHORD_PRESETS.keys())
|
||||
@@ -0,0 +1,232 @@
|
||||
"""
|
||||
capabilities — 视频工厂能力注册表加载器
|
||||
==========================================
|
||||
|
||||
「能力真相源」的读写接口。全项目通过这里查询能力/供应商/参数默认值。
|
||||
|
||||
用法：
|
||||
from capabilities import CapabilityRegistry
|
||||
reg = CapabilityRegistry()
|
||||
caps = reg.all() # 全部能力
|
||||
cap = reg.get("tts_generate") # 单个能力
|
||||
reg.set_default("tts_generate", provider="aliyun") # 改默认供应商（写回 yaml）
|
||||
reg.set_param_default("tts_generate", "speed", 1.2) # 改参数默认值
|
||||
|
||||
配置状态：provider 的 api_key_ref 指向「🔑 API 配置」管理的字段，
|
||||
本模块只**检查是否已配置**，不读取密钥内容。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
log = logging.getLogger("capabilities")
|
||||
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent.parent # 09_ave/scripts/
|
||||
AVE_DIR = SCRIPTS_DIR.parent # 09_ave/
|
||||
REGISTRY_PATH = AVE_DIR / "config" / "capabilities.yaml"
|
||||
|
||||
# agent-local 配置中心（密钥状态查询用）
|
||||
_AGENT_LOCAL = Path(os.environ.get("AGENT_LOCAL") or (AVE_DIR.parents[2] / "agent-local"))
|
||||
_LOCAL_CFG = _AGENT_LOCAL / "tools" / "ave" / "config" / "local.yaml"
|
||||
|
||||
|
||||
class CapabilityRegistry:
|
||||
"""能力注册表（yaml 读写 + 查询 + 密钥状态）"""
|
||||
|
||||
def __init__(self, path: Path | None = None):
|
||||
self.path = Path(path) if path else REGISTRY_PATH
|
||||
self._data: dict = {}
|
||||
self.load()
|
||||
|
||||
# ── 读写 ──────────────────────────────────────────
|
||||
|
||||
def load(self) -> dict:
|
||||
if not self.path.exists():
|
||||
log.warning(" ⚠️ 能力注册表不存在: %s", self.path)
|
||||
self._data = {"version": "0", "categories": {}, "capabilities": []}
|
||||
return self._data
|
||||
try:
|
||||
self._data = yaml.safe_load(self.path.read_text(encoding="utf-8")) or {}
|
||||
except Exception as exc:
|
||||
log.error(" ❌ 能力注册表解析失败: %s", exc)
|
||||
self._data = {"version": "0", "categories": {}, "capabilities": []}
|
||||
return self._data
|
||||
|
||||
def save(self) -> bool:
|
||||
"""写回 yaml（保留可读格式）"""
|
||||
try:
|
||||
self._data["updated"] = _today()
|
||||
with self.path.open("w", encoding="utf-8") as f:
|
||||
yaml.safe_dump(
|
||||
self._data, f, allow_unicode=True, sort_keys=False,
|
||||
default_flow_style=False, width=120,
|
||||
)
|
||||
log.info(" 💾 能力注册表已保存: %s", self.path.name)
|
||||
return True
|
||||
except Exception as exc:
|
||||
log.error(" ❌ 能力注册表保存失败: %s", exc)
|
||||
return False
|
||||
|
||||
# ── 查询 ──────────────────────────────────────────
|
||||
|
||||
def all(self) -> list[dict]:
|
||||
return list(self._data.get("capabilities") or [])
|
||||
|
||||
def categories(self) -> dict:
|
||||
return dict(self._data.get("categories") or {})
|
||||
|
||||
def get(self, cap_id: str) -> dict | None:
|
||||
for c in self.all():
|
||||
if c.get("id") == cap_id:
|
||||
return c
|
||||
return None
|
||||
|
||||
def providers_of(self, cap_id: str) -> list[dict]:
|
||||
cap = self.get(cap_id)
|
||||
return list(cap.get("providers") or []) if cap else []
|
||||
|
||||
def by_category(self) -> dict[str, list[dict]]:
|
||||
out: dict[str, list[dict]] = {}
|
||||
for c in self.all():
|
||||
out.setdefault(c.get("category", "other"), []).append(c)
|
||||
return out
|
||||
|
||||
# ── 修改（S1 的编辑能力）──────────────────────────
|
||||
|
||||
def set_default(self, cap_id: str, provider: str | None = None,
|
||||
model: str | None = None) -> bool:
|
||||
"""改默认供应商 / 模型"""
|
||||
cap = self.get(cap_id)
|
||||
if not cap:
|
||||
return False
|
||||
defaults = cap.setdefault("defaults", {})
|
||||
if provider:
|
||||
valid = [p.get("id") for p in (cap.get("providers") or [])]
|
||||
if provider not in valid:
|
||||
log.warning(" ⚠️ 供应商不在列表中: %s（可用: %s）", provider, valid)
|
||||
return False
|
||||
defaults["provider"] = provider
|
||||
# 同步 providers 的 default 标记
|
||||
for p in cap.get("providers") or []:
|
||||
p["default"] = (p.get("id") == provider)
|
||||
if model:
|
||||
defaults["model"] = model
|
||||
return self.save()
|
||||
|
||||
def set_param_default(self, cap_id: str, param: str, value: Any) -> bool:
|
||||
"""改参数默认值（option 级默认 + defaults 同步）"""
|
||||
cap = self.get(cap_id)
|
||||
if not cap:
|
||||
return False
|
||||
schema = cap.get("params_schema") or {}
|
||||
if param not in schema:
|
||||
log.warning(" ⚠️ 参数不存在: %s.%s", cap_id, param)
|
||||
return False
|
||||
spec = schema[param]
|
||||
# 范围校验
|
||||
if spec.get("type") == "number":
|
||||
lo, hi = spec.get("min"), spec.get("max")
|
||||
try:
|
||||
value = float(value)
|
||||
if lo is not None and value < lo:
|
||||
value = lo
|
||||
if hi is not None and value > hi:
|
||||
value = hi
|
||||
if float(value).is_integer():
|
||||
value = int(value)
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
elif spec.get("type") == "boolean":
|
||||
value = bool(value)
|
||||
elif spec.get("type") == "enum":
|
||||
if value not in (spec.get("options") or []):
|
||||
log.warning(" ⚠️ 枚举值非法: %s", value)
|
||||
return False
|
||||
schema[param]["default"] = value
|
||||
cap.setdefault("defaults", {})[param] = value
|
||||
return self.save()
|
||||
|
||||
# ── 密钥配置状态（不读密钥内容）────────────────────
|
||||
|
||||
def key_configured(self, api_key_ref: str) -> bool | None:
|
||||
"""检查 api_key_ref 指向的密钥是否已配置。None = 本地能力（无需密钥）"""
|
||||
if not api_key_ref:
|
||||
return None
|
||||
if not _LOCAL_CFG.exists():
|
||||
return False
|
||||
try:
|
||||
cfg = yaml.safe_load(_LOCAL_CFG.read_text(encoding="utf-8")) or {}
|
||||
except Exception:
|
||||
return False
|
||||
node: Any = cfg
|
||||
for part in api_key_ref.split("."):
|
||||
if isinstance(node, dict) and part in node:
|
||||
node = node[part]
|
||||
else:
|
||||
return False
|
||||
return bool(str(node or "").strip())
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""供前端消费的完整结构（含密钥配置状态）"""
|
||||
cats = self.categories()
|
||||
caps = []
|
||||
for c in self.all():
|
||||
item = dict(c)
|
||||
# 供应商密钥状态（兼容 api_key_ref / key_ref 两种字段名）
|
||||
for p in item.get("providers") or []:
|
||||
p["key_configured"] = self.key_configured(
|
||||
p.get("api_key_ref") or p.get("key_ref") or "")
|
||||
caps.append(item)
|
||||
return {
|
||||
"version": self._data.get("version", ""),
|
||||
"updated": self._data.get("updated", ""),
|
||||
"categories": [
|
||||
{"id": k, **v} for k, v in sorted(
|
||||
cats.items(), key=lambda kv: kv[1].get("order", 99))
|
||||
],
|
||||
"capabilities": caps,
|
||||
"total": len(caps),
|
||||
"by_status": _count_by(caps, "status"),
|
||||
"by_category": _count_by(caps, "category"),
|
||||
}
|
||||
|
||||
# ── 修复/维护 ─────────────────────────────────────
|
||||
|
||||
def remove_legacy_dirs(self) -> list[str]:
|
||||
"""列出可归档的编号空壳目录（01-07）—— 不自动删除，仅报告"""
|
||||
dead = []
|
||||
for d in sorted(SCRIPTS_DIR.iterdir()):
|
||||
if d.is_dir() and d.name[:2].isdigit() and d.name[2:3] == "_":
|
||||
init = d / "__init__.py"
|
||||
if init.exists() and init.stat().st_size == 0:
|
||||
dead.append(d.name)
|
||||
return dead
|
||||
|
||||
|
||||
def _count_by(items: list[dict], field: str) -> dict:
|
||||
out: dict[str, int] = {}
|
||||
for it in items:
|
||||
k = str(it.get(field) or "unknown")
|
||||
out[k] = out.get(k, 0) + 1
|
||||
return out
|
||||
|
||||
|
||||
def _today() -> str:
|
||||
from datetime import date
|
||||
return date.today().isoformat()
|
||||
|
||||
|
||||
# 便捷单例
|
||||
_registry: CapabilityRegistry | None = None
|
||||
|
||||
|
||||
def get_registry(reload: bool = False) -> CapabilityRegistry:
|
||||
global _registry
|
||||
if _registry is None or reload:
|
||||
_registry = CapabilityRegistry()
|
||||
return _registry
|
||||
@@ -0,0 +1,350 @@
|
||||
"""百炼（DashScope）视频生成 —— 首尾帧 KF2V / 多图参考 R2V
|
||||
|
||||
为什么单独一个模块：可灵走 `kling_video.py`（channel=kling），
|
||||
百炼走这里（channel=dashscope）—— 两家接口完全不同，不能混。
|
||||
|
||||
价格对比（`config/video_models.yaml`）：
|
||||
wan2.2-kf2v-flash ¥0.1/秒（480P） ← 首尾帧专用，最省
|
||||
wan2.6-r2v-flash ¥0.3/秒（720P） ← 多图参考（角色+场景+首帧）
|
||||
kling-v3 ¥0.6/秒（720P） ← 现行默认
|
||||
→ 同样 4 镜 5 秒：可灵 ¥12，kf2v ¥2。
|
||||
|
||||
调用范式（与 `material_producer/wan2_2/wan2_2.py` 一致，四步）：
|
||||
① 上传图片 → oss:// URL（每张都要）
|
||||
② POST video-synthesis（X-DashScope-Async: enable）
|
||||
③ GET /api/v1/tasks/{task_id} 轮询
|
||||
④ 下载 video_url
|
||||
|
||||
字段（来自千问AI平台 OpenAPI spec，见 docs/pipeline_research/03）：
|
||||
KF2V: input.first_frame_url + input.last_frame_url + input.prompt
|
||||
R2V : input.media[] = [{type: first_frame|reference_image|reference_video, url}]
|
||||
+ input.prompt
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import yaml
|
||||
|
||||
log = logging.getLogger("ave.dashscope_video")
|
||||
|
||||
# ── 端点 ────────────────────────────────────────────────────
|
||||
# ⚠️ 两个模型走**不同端点**（据官方 OpenAPI 规格，见 docs/pipeline_research/03）：
|
||||
# KF2V（首尾帧） → /services/aigc/image2video/video-synthesis
|
||||
# R2V （多图参考）→ /services/aigc/video-generation/video-synthesis
|
||||
BASE = "https://dashscope.aliyuncs.com"
|
||||
UPLOAD_API = f"{BASE}/api/v1/uploads"
|
||||
SUBMIT_API_KF2V = f"{BASE}/api/v1/services/aigc/image2video/video-synthesis"
|
||||
SUBMIT_API_R2V = f"{BASE}/api/v1/services/aigc/video-generation/video-synthesis"
|
||||
SUBMIT_API = SUBMIT_API_R2V # 兼容旧引用
|
||||
TASK_API = f"{BASE}/api/v1/tasks/{{task_id}}"
|
||||
|
||||
# ⚠️ 所有 httpx 调用都要 trust_env=False —— macOS 系统代理会让
|
||||
# 国内 API 时通时不通（本项目踩过多次，见 docs/参数速查.md）
|
||||
|
||||
_LOCAL_YAML = Path.home() / "workbuddy-agent-os" / "agent-local" / "tools" / "ave" / "config" / "local.yaml"
|
||||
|
||||
|
||||
def api_key() -> str:
|
||||
"""读百炼 api_key（agent-local，不进 git）"""
|
||||
env = os.environ.get("DASHSCOPE_API_KEY")
|
||||
if env:
|
||||
return env
|
||||
try:
|
||||
d = yaml.safe_load(_LOCAL_YAML.read_text(encoding="utf-8")) or {}
|
||||
return str((d.get("aliyun") or {}).get("api_key") or "")
|
||||
except Exception as e:
|
||||
log.warning("读 local.yaml 失败: %s", e)
|
||||
return ""
|
||||
|
||||
|
||||
def available() -> bool:
|
||||
return bool(api_key())
|
||||
|
||||
|
||||
# ── ① 上传 ──────────────────────────────────────────────────
|
||||
def _mime(p: str) -> str:
|
||||
return {"png": "image/png", "jpg": "image/jpeg", "jpeg": "image/jpeg",
|
||||
"webp": "image/webp", "bmp": "image/bmp",
|
||||
"mp4": "video/mp4",
|
||||
"mp3": "audio/mpeg", "wav": "audio/wav", "m4a": "audio/mp4"}.get(
|
||||
Path(p).suffix.lower().lstrip("."), "application/octet-stream")
|
||||
|
||||
|
||||
def upload_file(path: str, model: str, key: str = "") -> str:
|
||||
"""上传到百炼临时存储 → oss:// URL
|
||||
|
||||
已上传过（同名 + 同模型）会复用，避免重复上传。
|
||||
"""
|
||||
key = key or api_key()
|
||||
p = Path(path)
|
||||
if not p.exists():
|
||||
raise FileNotFoundError(f"文件不存在: {path}")
|
||||
|
||||
r = httpx.get(UPLOAD_API, params={"action": "getPolicy", "model": model},
|
||||
headers={"Authorization": f"Bearer {key}"}, timeout=20, trust_env=False)
|
||||
r.raise_for_status()
|
||||
d = r.json()["data"]
|
||||
|
||||
oss_key = f"{d['upload_dir']}/{p.name}"
|
||||
with open(p, "rb") as f:
|
||||
files = {
|
||||
"OSSAccessKeyId": (None, d["oss_access_key_id"]),
|
||||
"policy": (None, d["policy"]),
|
||||
"Signature": (None, d["signature"]),
|
||||
"key": (None, oss_key),
|
||||
"x-oss-object-acl": (None, d["x_oss_object_acl"]),
|
||||
"x-oss-forbid-overwrite": (None, d["x_oss_forbid_overwrite"]),
|
||||
"success_action_status": (None, "200"),
|
||||
"file": (p.name, f, _mime(str(p))),
|
||||
}
|
||||
up = httpx.post(d["upload_host"], files=files, timeout=180, trust_env=False)
|
||||
up.raise_for_status()
|
||||
return f"oss://{oss_key}"
|
||||
|
||||
|
||||
def _as_url(v: str, model: str, key: str) -> str:
|
||||
"""本地路径 → oss:// ；已是 http(s)/oss:// /data: 则原样返回"""
|
||||
if not v:
|
||||
return ""
|
||||
s = str(v)
|
||||
if s.startswith(("http://", "https://", "oss://", "data:")):
|
||||
return s
|
||||
return upload_file(s, model, key)
|
||||
|
||||
|
||||
# ── ② 提交 ──────────────────────────────────────────────────
|
||||
def _submit(payload: dict, key: str, endpoint: str = "") -> str:
|
||||
headers = {
|
||||
"Authorization": f"Bearer {key}",
|
||||
"X-DashScope-Async": "enable",
|
||||
"X-DashScope-OssResourceResolve": "enable",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
r = httpx.post(endpoint or SUBMIT_API_R2V, headers=headers, json=payload,
|
||||
timeout=40, trust_env=False)
|
||||
if r.status_code >= 400:
|
||||
raise RuntimeError(f"百炼提交失败 HTTP {r.status_code}: {r.text[:300]}")
|
||||
out = (r.json() or {}).get("output") or {}
|
||||
tid = out.get("task_id")
|
||||
if not tid:
|
||||
raise RuntimeError(f"百炼未返回 task_id: {json.dumps(r.json(), ensure_ascii=False)[:300]}")
|
||||
return tid
|
||||
|
||||
|
||||
def submit_kf2v(first_frame: str, last_frame: str, prompt: str = "", *,
|
||||
model: str = "wan2.2-kf2v-flash", resolution: str = "480P",
|
||||
negative_prompt: str = "", prompt_extend: bool = True,
|
||||
seed: int | None = None, template: str = "",
|
||||
key: str = "") -> str:
|
||||
"""首尾帧生视频 —— 给「首帧 + 尾帧」，模型在两帧之间插值
|
||||
|
||||
⚠️ 官方规格（docs/pipeline_research/03 §二）：
|
||||
· 端点 = /services/aigc/image2video/video-synthesis（**不是** video-generation）
|
||||
· duration **固定 5 秒，不可修改** → 不传该参数
|
||||
· 输出宽高比跟随 first_frame_url
|
||||
· 用了 template（特效模板）时只需传 first_frame_url
|
||||
"""
|
||||
key = key or api_key()
|
||||
if not key:
|
||||
raise RuntimeError("未配置百炼 key（agent-local/tools/ave/config/local.yaml → aliyun.api_key）")
|
||||
if not last_frame:
|
||||
raise ValueError("KF2V 必须同时给首帧和尾帧")
|
||||
inp = {"first_frame_url": _as_url(first_frame, model, key)}
|
||||
if not template:
|
||||
inp["last_frame_url"] = _as_url(last_frame, model, key)
|
||||
if prompt:
|
||||
inp["prompt"] = prompt
|
||||
if negative_prompt:
|
||||
inp["negative_prompt"] = negative_prompt[:500]
|
||||
params: dict = {"resolution": resolution, "prompt_extend": bool(prompt_extend)}
|
||||
if template:
|
||||
params["template"] = template
|
||||
if seed is not None:
|
||||
params["seed"] = int(seed)
|
||||
log.info("KF2V 提交：%s · %s · 固定5s · %s", model, resolution,
|
||||
"模板" if template else "首尾帧")
|
||||
return _submit({"model": model, "input": inp, "parameters": params}, key, SUBMIT_API_KF2V)
|
||||
|
||||
|
||||
def submit_r2v(media: list[dict], prompt: str = "", *,
|
||||
model: str = "wan2.6-r2v-flash", resolution: str = "720P",
|
||||
duration: int = 5, key: str = "") -> str:
|
||||
"""多图参考生视频
|
||||
|
||||
media: [{"type": "first_frame"|"reference_image"|"reference_video", "url": 本地路径或URL}]
|
||||
最多 5 个（config 里的 limits.max_reference_images）
|
||||
"""
|
||||
key = key or api_key()
|
||||
if not key:
|
||||
raise RuntimeError("未配置百炼 key")
|
||||
if not media:
|
||||
raise ValueError("R2V 至少要给 1 个 media 元素")
|
||||
if not (prompt or "").strip():
|
||||
# 官方规格：prompt 为 ★必填（「期望视频内容」）—— 不传会直接失败
|
||||
raise ValueError("R2V 的 prompt 是必填项（描述期望的视频内容），不能为空")
|
||||
items = []
|
||||
for m in media[:5]:
|
||||
u = _as_url(m.get("url") or "", model, key)
|
||||
if not u:
|
||||
continue
|
||||
it = {"type": m.get("type") or "reference_image", "url": u}
|
||||
if m.get("reference_voice"):
|
||||
it["reference_voice"] = m["reference_voice"]
|
||||
items.append(it)
|
||||
if not items:
|
||||
raise ValueError("media 里没有可用的 url")
|
||||
payload = {
|
||||
"model": model,
|
||||
"input": {"media": items},
|
||||
"parameters": {"resolution": resolution, "duration": duration},
|
||||
}
|
||||
if prompt:
|
||||
payload["input"]["prompt"] = prompt
|
||||
log.info("R2V 提交：%s · %s · %d 个参考", model, resolution, len(items))
|
||||
return _submit(payload, key)
|
||||
|
||||
|
||||
# ── ③ 轮询 ──────────────────────────────────────────────────
|
||||
def poll(task_id: str, *, interval: int = 10, timeout: int = 900,
|
||||
key: str = "", progress_cb=None) -> str:
|
||||
"""轮询直到完成 → 返回视频 URL
|
||||
|
||||
兼容两种结果结构：
|
||||
output.video_url （KF2V / R2V 新结构）
|
||||
output.results.video_url （wan2.2-s2v 旧结构）
|
||||
"""
|
||||
key = key or api_key()
|
||||
headers = {"Authorization": f"Bearer {key}", "X-DashScope-OssResourceResolve": "enable"}
|
||||
start = time.time()
|
||||
while time.time() - start < timeout:
|
||||
r = httpx.get(TASK_API.format(task_id=task_id), headers=headers,
|
||||
timeout=40, trust_env=False)
|
||||
r.raise_for_status()
|
||||
out = (r.json() or {}).get("output") or {}
|
||||
st = out.get("task_status")
|
||||
el = int(time.time() - start)
|
||||
|
||||
if st == "SUCCEEDED":
|
||||
url = (out.get("video_url")
|
||||
or (out.get("results") or {}).get("video_url")
|
||||
or ((out.get("results") or {}).get("videos") or [{}])[0].get("url"))
|
||||
if not url:
|
||||
raise RuntimeError(f"任务成功但没有视频 URL：{json.dumps(out, ensure_ascii=False)[:300]}")
|
||||
if progress_cb:
|
||||
progress_cb(f"✅ 生成完成（{el}s）")
|
||||
return url
|
||||
if st in ("FAILED", "CANCELED", "UNKNOWN"):
|
||||
code = out.get("code") or "?"
|
||||
msg = out.get("message") or "?"
|
||||
raise RuntimeError(f"百炼任务失败 [{code}]: {msg}")
|
||||
|
||||
if progress_cb:
|
||||
progress_cb(f"⏳ 百炼生成中… {el}s（{st}）")
|
||||
time.sleep(interval)
|
||||
raise TimeoutError(f"百炼任务超时（>{timeout}s）")
|
||||
|
||||
|
||||
# ── ④ 下载 ──────────────────────────────────────────────────
|
||||
def download(url: str, dest: str) -> str:
|
||||
Path(dest).parent.mkdir(parents=True, exist_ok=True)
|
||||
with httpx.stream("GET", url, timeout=300, trust_env=False,
|
||||
follow_redirects=True) as r:
|
||||
r.raise_for_status()
|
||||
with open(dest, "wb") as f:
|
||||
for chunk in r.iter_bytes(1 << 16):
|
||||
f.write(chunk)
|
||||
return dest
|
||||
|
||||
|
||||
# ── 一步到位 ────────────────────────────────────────────────
|
||||
def generate_kf2v(first_frame: str, last_frame: str, dest: str, prompt: str = "",
|
||||
*, model: str = "wan2.2-kf2v-flash", resolution: str = "480P",
|
||||
progress_cb=None) -> str:
|
||||
"""首尾帧一步到位（提交 → 轮询 → 下载）
|
||||
|
||||
⚠️ duration 固定 5 秒（官方规格），所以这里不提供 duration 参数。
|
||||
"""
|
||||
tid = submit_kf2v(first_frame, last_frame, prompt, model=model, resolution=resolution)
|
||||
if progress_cb:
|
||||
progress_cb(f"📤 已提交（task {tid}）")
|
||||
url = poll(tid, progress_cb=progress_cb)
|
||||
download(url, dest)
|
||||
return dest
|
||||
|
||||
|
||||
def generate_r2v(media: list[dict], dest: str, prompt: str = "",
|
||||
*, model: str = "wan2.6-r2v-flash", resolution: str = "720P",
|
||||
duration: int = 5, progress_cb=None) -> str:
|
||||
tid = submit_r2v(media, prompt, model=model, resolution=resolution, duration=duration)
|
||||
if progress_cb:
|
||||
progress_cb(f"📤 已提交（task {tid}）")
|
||||
url = poll(tid, progress_cb=progress_cb)
|
||||
download(url, dest)
|
||||
return dest
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════
|
||||
# LivePortrait（灵动人像）—— 数字人播报，¥0.02/秒
|
||||
#
|
||||
# 人物肖像图 + 语音音频 → 口型同步的播报视频
|
||||
# ★ 输入正好是我们的现有资产：角色定妆照 + TTS 配音
|
||||
# ⚠️ 只适合「口播/播报」类镜头（talking_head），有动作的镜头不适用
|
||||
# ══════════════════════════════════════════════════════════
|
||||
LIVEPORTRAIT_MODEL = "liveportrait"
|
||||
|
||||
|
||||
def submit_liveportrait(image: str, audio: str, *, template_id: str = "calm",
|
||||
eye_move_freq: float | None = None,
|
||||
model: str = LIVEPORTRAIT_MODEL, key: str = "") -> str:
|
||||
"""数字人播报任务提交
|
||||
|
||||
image : 人物肖像图（本地路径或 URL）—— 建议用角色定妆照
|
||||
audio : 驱动音频（本地路径或 URL）—— TTS 配音；1 秒 ~ 3 分钟
|
||||
template_id : normal（中等幅度）| calm（小幅度·推荐播报）| active（大幅度·演唱）
|
||||
"""
|
||||
key = key or api_key()
|
||||
if not key:
|
||||
raise RuntimeError("未配置百炼 key")
|
||||
if not (image and audio):
|
||||
raise ValueError("liveportrait 必须同时给 image 和 audio")
|
||||
inp = {
|
||||
"image_url": _as_url(image, model, key),
|
||||
"audio_url": _as_url(audio, model, key),
|
||||
}
|
||||
params: dict = {"template_id": template_id if template_id in
|
||||
("normal", "calm", "active") else "calm"}
|
||||
if eye_move_freq is not None:
|
||||
params["eye_move_freq"] = float(eye_move_freq)
|
||||
log.info("LivePortrait 提交：模板=%s", params["template_id"])
|
||||
# 与 KF2V 同端点（官方文档：image2video/video-synthesis）
|
||||
return _submit({"model": model, "input": inp, "parameters": params},
|
||||
key, SUBMIT_API_KF2V)
|
||||
|
||||
|
||||
def generate_liveportrait(image: str, audio: str, dest: str, *,
|
||||
template_id: str = "calm", progress_cb=None) -> str:
|
||||
"""数字人播报一步到位（提交 → 轮询 → 下载）
|
||||
|
||||
⚠️ 输出时长由**音频时长**决定（不是固定 5 秒）
|
||||
"""
|
||||
tid = submit_liveportrait(image, audio, template_id=template_id)
|
||||
if progress_cb:
|
||||
progress_cb(f"📤 已提交（task {tid}）")
|
||||
url = poll(tid, progress_cb=progress_cb)
|
||||
download(url, dest)
|
||||
return dest
|
||||
|
||||
|
||||
def estimate_cost(seconds: float, model: str = "wan2.2-kf2v-flash") -> float:
|
||||
if model == LIVEPORTRAIT_MODEL:
|
||||
return round(seconds * 0.02, 3)
|
||||
rate = 0.1 if "kf2v" in model else 0.3
|
||||
return round(seconds * rate, 2)
|
||||
@@ -0,0 +1,604 @@
|
||||
"""
|
||||
capabilities.kling_video — 可灵（Kling）视频生成能力实现
|
||||
==========================================================
|
||||
|
||||
支持：文生视频（text2video）/ 图生视频（image2video）+ 任务轮询 + 下载
|
||||
|
||||
鉴权：api_key（Bearer）优先 → access_key + secret_key（JWT HS256）回退
|
||||
配置来源：agent-local/tools/ave/config/local.yaml → kling 节（或环境变量）
|
||||
|
||||
用途：视频工厂 S2-6「逐镜头生成」的默认引擎实现。
|
||||
余额不足（code 1102）等错误会带明确提示，不会静默失败。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import ssl
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
|
||||
log = logging.getLogger("capabilities.kling")
|
||||
|
||||
DEFAULT_BASE = "https://api-beijing.klingai.com"
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# ★ 网络统一入口：**禁用系统代理** + 自动重试
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 踩坑记录（2026-10-04 定位）：
|
||||
# macOS 开了系统代理（127.0.0.1:7892）时，urllib 会**自动走代理**，
|
||||
# 而代理对可灵 API 时通时不通 → `SSL: UNEXPECTED_EOF_WHILE_READING`
|
||||
# → 表现为"生成随机失败"（用户遇到的角色生成失败即此因）。
|
||||
# 直连实测：3/3 次 HTTP 200、0.2~0.3 秒。
|
||||
#
|
||||
# 解决：所有 API 调用走**禁代理 opener** + 失败自动重试 3 次（指数退避）。
|
||||
_NO_PROXY_OPENER = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
|
||||
|
||||
def _urlopen(req, timeout: int = 60):
|
||||
"""API 调用统一入口：禁代理 + 重试 3 次（应对网络抖动）"""
|
||||
last_exc = None
|
||||
for attempt in range(3):
|
||||
try:
|
||||
return _NO_PROXY_OPENER.open(req, timeout=timeout)
|
||||
except Exception as e: # URLError / SSLError / OSError / timeout
|
||||
last_exc = e
|
||||
if attempt < 2:
|
||||
time.sleep(1.5 * (attempt + 1))
|
||||
raise last_exc
|
||||
|
||||
|
||||
# ── 配置 ───────────────────────────────────────────────
|
||||
|
||||
def _load_config() -> dict:
|
||||
"""读 kling 配置（local.yaml 优先，环境变量覆盖）"""
|
||||
cfg: dict = {}
|
||||
try:
|
||||
import yaml
|
||||
_agent_local = Path(os.environ.get("AGENT_LOCAL") or (Path.home() / "workbuddy-agent-os" / "agent-local"))
|
||||
p = _agent_local / "tools" / "ave" / "config" / "local.yaml"
|
||||
if p.exists():
|
||||
raw = yaml.safe_load(p.read_text(encoding="utf-8")) or {}
|
||||
cfg.update(raw.get("kling") or {})
|
||||
except Exception as e:
|
||||
log.debug("读取 kling 配置失败: %s", e)
|
||||
if os.environ.get("KLING_API_KEY"):
|
||||
cfg["api_key"] = os.environ["KLING_API_KEY"]
|
||||
cfg.setdefault("base_url", DEFAULT_BASE)
|
||||
return cfg
|
||||
|
||||
|
||||
class KlingError(RuntimeError):
|
||||
"""可灵调用错误（含 code / message，便于按余额不足等场景提示）"""
|
||||
|
||||
def __init__(self, message: str, code: Any = None):
|
||||
super().__init__(message)
|
||||
self.code = code
|
||||
|
||||
|
||||
# ── 防人物漂移负向词（研究来源：madebysaira/CharacterConsistency 实测词表）
|
||||
# 作者反馈：加上这些后多镜一致性问题显著减少（重生成率 60% → 15%）
|
||||
# 用法：图生视频时随 prompt 一起传；最强的漂移词放最前
|
||||
DRIFT_NEGATIVE = (
|
||||
"face change, different person, another woman, another man, "
|
||||
"clothing shift, outfit change, hair style change, hair length change, "
|
||||
"body proportion change, extra limbs, deformed face, inconsistent lighting, "
|
||||
"skin tone shift, eye color change, blurry face, morphing, identity shift, "
|
||||
"reference image ignored, low face similarity"
|
||||
)
|
||||
|
||||
# 图生视频时的参考图权重（研究：Kling 推荐 0.75-0.85；0.5 会导致身份不稳）
|
||||
DEFAULT_IMAGE_CFG_SCALE = 0.8
|
||||
|
||||
|
||||
class KlingVideoClient:
|
||||
"""可灵视频生成客户端（文生视频 / 图生视频）"""
|
||||
|
||||
def __init__(self, config: Optional[dict] = None):
|
||||
self.cfg = config or _load_config()
|
||||
self.base = (self.cfg.get("base_url") or DEFAULT_BASE).rstrip("/")
|
||||
self.api_key = (self.cfg.get("api_key") or "").strip()
|
||||
self.access_key = (self.cfg.get("access_key") or "").strip()
|
||||
self.secret_key = (self.cfg.get("secret_key") or "").strip()
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
return bool(self.api_key or (self.access_key and self.secret_key))
|
||||
|
||||
def _headers(self) -> dict:
|
||||
if self.api_key:
|
||||
return {"Authorization": f"Bearer {self.api_key}",
|
||||
"Content-Type": "application/json"}
|
||||
# AK/SK → JWT HS256（官方方式）
|
||||
import hashlib
|
||||
import hmac
|
||||
header = base64.urlsafe_b64encode(json.dumps(
|
||||
{"alg": "HS256", "typ": "JWT"}).encode()).rstrip(b"=")
|
||||
now = int(time.time())
|
||||
payload = base64.urlsafe_b64encode(json.dumps(
|
||||
{"iss": self.access_key, "exp": now + 1800, "nbf": now - 5}).encode()).rstrip(b"=")
|
||||
signing = header + b"." + payload
|
||||
sig = base64.urlsafe_b64encode(
|
||||
hmac.new(self.secret_key.encode(), signing, hashlib.sha256).digest()).rstrip(b"=")
|
||||
return {"Authorization": f"Bearer {(signing + b'.' + sig).decode()}",
|
||||
"Content-Type": "application/json"}
|
||||
|
||||
def _post(self, path: str, payload: dict, _retries: int = 3) -> dict:
|
||||
"""POST 请求
|
||||
|
||||
内置两类处理：
|
||||
· **并发超限（code 1303）→ 退避重试**（官方建议：初始延迟 ≥1s）
|
||||
账号并发：视频 5 / 图片 9（图片按参数 n 计），超限即触发
|
||||
· 余额不足（1102）/ 鉴权失败（1101）→ 直接抛明确错误，不重试
|
||||
"""
|
||||
req = urllib.request.Request(self.base + path,
|
||||
data=json.dumps(payload).encode("utf-8"),
|
||||
headers=self._headers())
|
||||
for attempt in range(max(1, _retries)):
|
||||
try:
|
||||
with _urlopen(req, timeout=60) as r:
|
||||
return json.loads(r.read().decode("utf-8"))
|
||||
except urllib.error.HTTPError as e:
|
||||
body = e.read().decode("utf-8", errors="ignore")
|
||||
try:
|
||||
d = json.loads(body)
|
||||
code, msg = d.get("code"), d.get("message", "")
|
||||
except Exception:
|
||||
code, msg = e.code, body[:200]
|
||||
|
||||
# ★ 并发超限：官方明确建议退避重试（非参数错误）
|
||||
if code == 1303 or "parallel task over" in str(msg).lower():
|
||||
if attempt < _retries - 1:
|
||||
delay = 2.0 * (attempt + 1)
|
||||
log.warning(" ⏳ 可灵并发超限（1303），%.0fs 后重试（%d/%d）",
|
||||
delay, attempt + 1, _retries - 1)
|
||||
time.sleep(delay)
|
||||
continue
|
||||
raise KlingError(
|
||||
f"可灵并发超限（code 1303）—— 已重试 {_retries} 次仍失败。"
|
||||
f"账号并发上限：视频 5 / 图片 9（图片按参数 n 计）；"
|
||||
f"请稍后再试或降低并发生成数", code)
|
||||
|
||||
if code == 1102 or "balance" in str(msg).lower():
|
||||
raise KlingError("可灵账户余额不足（code 1102）—— 请到可灵开放平台充值后重试", code)
|
||||
if code == 1101 or "auth" in str(msg).lower():
|
||||
raise KlingError(f"可灵鉴权失败（code {code}）—— 检查 API Key 是否正确/过期", code)
|
||||
raise KlingError(f"可灵接口错误 HTTP {e.code} code={code}: {msg}", code)
|
||||
except KlingError:
|
||||
raise
|
||||
except Exception as e:
|
||||
raise KlingError(f"可灵请求失败: {e}")
|
||||
|
||||
def _get(self, path: str) -> dict:
|
||||
req = urllib.request.Request(self.base + path, headers=self._headers())
|
||||
try:
|
||||
with _urlopen(req, timeout=60) as r:
|
||||
return json.loads(r.read().decode("utf-8"))
|
||||
except urllib.error.HTTPError as e:
|
||||
body = e.read().decode("utf-8", errors="ignore")
|
||||
raise KlingError(f"可灵查询失败 HTTP {e.code}: {body[:150]}")
|
||||
except Exception as e:
|
||||
raise KlingError(f"可灵查询失败: {e}")
|
||||
|
||||
# ── 任务提交 ───────────────────────────────────────
|
||||
|
||||
def submit_text2video(self, prompt: str, model: str = "kling-v3",
|
||||
duration: str = "5", aspect_ratio: str = "9:16",
|
||||
negative_prompt: str = "", cfg_scale: float = 0.5,
|
||||
camera_control: Optional[dict] = None,
|
||||
sound: str = "off") -> str:
|
||||
"""文生视频 → task_id（camera_control：运镜控制，见 build_camera_control）
|
||||
|
||||
sound：**原生音频开关**（"on" / "off"）。实测（2026-10-04）：
|
||||
· off（默认）→ 输出只有视频流
|
||||
· on → 输出含音频流（aac 44.1k stereo），
|
||||
提示词里写对话即生成对应语音 + 环境音
|
||||
"""
|
||||
payload = {"model_name": model, "prompt": prompt, "duration": duration,
|
||||
"aspect_ratio": aspect_ratio, "cfg_scale": cfg_scale}
|
||||
if negative_prompt:
|
||||
payload["negative_prompt"] = negative_prompt
|
||||
if camera_control:
|
||||
payload["camera_control"] = camera_control
|
||||
if sound == "on":
|
||||
payload["sound"] = "on"
|
||||
d = self._post("/v1/videos/text2video", payload)
|
||||
return self._extract_task_id(d)
|
||||
|
||||
def submit_image2video(self, image_path: str, prompt: str = "", model: str = "kling-v3",
|
||||
duration: str = "5", tail_image_path: str = "",
|
||||
cfg_scale: float = DEFAULT_IMAGE_CFG_SCALE,
|
||||
camera_control: Optional[dict] = None,
|
||||
negative_prompt: str = DRIFT_NEGATIVE,
|
||||
sound: str = "off") -> str:
|
||||
"""图生视频 → task_id（image：本地文件路径 → base64 或 URL）
|
||||
|
||||
⚠️ 研究结论（2026-10-03）：
|
||||
· **cfg_scale（参考图权重）0.75-0.85** —— 原来用 0.5 会导致人物身份不稳、
|
||||
跨镜换脸。默认已提到 0.8。
|
||||
· **必须传防漂移负向词** —— 否则模型容易改变脸/服装/发型。
|
||||
"""
|
||||
payload: dict = {"model_name": model, "duration": duration,
|
||||
"cfg_scale": cfg_scale, "image": self._to_image_value(image_path)}
|
||||
if prompt:
|
||||
payload["prompt"] = prompt
|
||||
if negative_prompt:
|
||||
payload["negative_prompt"] = negative_prompt
|
||||
if tail_image_path:
|
||||
payload["image_tail"] = self._to_image_value(tail_image_path)
|
||||
if camera_control:
|
||||
payload["camera_control"] = camera_control
|
||||
if sound == "on":
|
||||
payload["sound"] = "on" # ★ 原生音频（见 submit_text2video 注释）
|
||||
d = self._post("/v1/videos/image2video", payload)
|
||||
return self._extract_task_id(d)
|
||||
|
||||
@staticmethod
|
||||
def _to_image_value(image: str) -> str:
|
||||
"""本地路径 → base64（不带前缀）；http(s) 或已是 base64 则原样返回"""
|
||||
if image.startswith(("http://", "https://")):
|
||||
return image
|
||||
p = Path(image)
|
||||
if p.exists():
|
||||
return base64.b64encode(p.read_bytes()).decode("utf-8")
|
||||
return image
|
||||
|
||||
@staticmethod
|
||||
def _extract_task_id(d: dict) -> str:
|
||||
if d.get("code") != 0:
|
||||
raise KlingError(f"可灵提交失败 code={d.get('code')}: {d.get('message')}", d.get("code"))
|
||||
tid = (d.get("data") or {}).get("task_id")
|
||||
if not tid:
|
||||
raise KlingError(f"可灵未返回 task_id: {str(d)[:200]}")
|
||||
return tid
|
||||
|
||||
# ── 轮询 + 下载 ────────────────────────────────────
|
||||
|
||||
def wait_for_video(self, task_id: str, kind: str = "text2video",
|
||||
timeout: int = 600, interval: int = 10,
|
||||
progress_cb=None) -> str:
|
||||
"""轮询任务直至完成 → 返回视频 URL"""
|
||||
path = f"/v1/videos/{kind}/{task_id}"
|
||||
deadline = time.time() + timeout
|
||||
while time.time() < deadline:
|
||||
d = self._get(path)
|
||||
data = d.get("data") or {}
|
||||
status = data.get("task_status")
|
||||
if status == "succeed":
|
||||
videos = (data.get("task_result") or {}).get("videos") or []
|
||||
if not videos:
|
||||
raise KlingError("可灵任务成功但无视频输出")
|
||||
return videos[0].get("url", "")
|
||||
if status == "failed":
|
||||
raise KlingError(f"可灵生成失败: {data.get('task_status_msg') or '未知原因'}")
|
||||
if progress_cb:
|
||||
progress_cb(status or "processing")
|
||||
time.sleep(interval)
|
||||
raise KlingError(f"可灵任务超时（{timeout}s）未完成")
|
||||
|
||||
def download(self, url: str, dest: str | Path) -> Path:
|
||||
"""下载视频到本地
|
||||
|
||||
可灵返回的是**带签名的完整 URL**（含 cacheKey 参数），通常**无需额外鉴权**；
|
||||
若首次返回 401/403，再带鉴权头重试一次（兼容不同资源域名）。
|
||||
"""
|
||||
p = Path(dest)
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
attempts: list[dict] = [{}]
|
||||
try:
|
||||
headers = self._headers()
|
||||
headers.pop("Content-Type", None)
|
||||
attempts.append(headers)
|
||||
except Exception:
|
||||
pass
|
||||
last_err: Exception | None = None
|
||||
for hdrs in attempts:
|
||||
try:
|
||||
req = urllib.request.Request(url, headers=hdrs)
|
||||
with _urlopen(req, timeout=300) as r, open(p, "wb") as f:
|
||||
f.write(r.read())
|
||||
return p
|
||||
except urllib.error.HTTPError as e:
|
||||
last_err = e
|
||||
if e.code not in (401, 403):
|
||||
break
|
||||
except Exception as e:
|
||||
last_err = e
|
||||
break
|
||||
code = getattr(last_err, "code", "?")
|
||||
raise KlingError(
|
||||
f"下载生成视频失败（HTTP {code}）—— 请确认任务返回的 URL 完整"
|
||||
"（含 ?cacheKey=... 签名参数；可灵资源有效期 30 天）"
|
||||
)
|
||||
|
||||
# ── 一站式 ─────────────────────────────────────────
|
||||
|
||||
# ════════════════════════════════════════════════════════
|
||||
# 账号与费用（官方接口，**免费**）
|
||||
# ════════════════════════════════════════════════════════
|
||||
|
||||
def get_account_costs(self, days: int = 30) -> dict:
|
||||
"""查询账号**资源包余量**（官方免费接口，QPS ≤ 1）
|
||||
|
||||
⚠️ 官方说明：余量统计有 **12 小时延迟**，非实时。
|
||||
⚠️ 只返回资源包（含平台赠送的试用包）；**现金余额需在可灵控制台查看**。
|
||||
|
||||
实测字段（2026-10-04）：resource_pack_name / total_quantity /
|
||||
remaining_quantity / resource_pack_type / status / invalid_time
|
||||
"""
|
||||
import time as _t
|
||||
now_ms = int(_t.time() * 1000)
|
||||
start_ms = now_ms - int(days) * 24 * 3600 * 1000
|
||||
try:
|
||||
j = self._get(f"/account/costs?start_time={start_ms}&end_time={now_ms}")
|
||||
except Exception as e:
|
||||
return {"packs": [], "summary": {}, "error": str(e)[:200]}
|
||||
if j.get("code") != 0:
|
||||
return {"packs": [], "summary": {}, "error": str(j.get("message", ""))[:200]}
|
||||
|
||||
packs = []
|
||||
for p in (j.get("data") or {}).get("resource_pack_subscribe_infos") or []:
|
||||
total = float(p.get("total_quantity") or 0)
|
||||
left = float(p.get("remaining_quantity") or 0)
|
||||
packs.append({
|
||||
"name": p.get("resource_pack_name", ""),
|
||||
"type": p.get("resource_pack_type", ""),
|
||||
"total": total,
|
||||
"remaining": left,
|
||||
"used": round(total - left, 2),
|
||||
"used_pct": round((total - left) / total * 100, 1) if total else 0,
|
||||
"status": p.get("status", ""),
|
||||
"expire_at": p.get("invalid_time"),
|
||||
})
|
||||
summary: dict = {}
|
||||
for p in packs:
|
||||
nm = p["name"]
|
||||
key = "video" if "视频" in nm else ("image" if ("图像" in nm or "图片" in nm) else "other")
|
||||
s = summary.setdefault(key, {"total": 0.0, "remaining": 0.0, "used": 0.0, "packs": 0})
|
||||
s["total"] += p["total"]
|
||||
s["remaining"] += p["remaining"]
|
||||
s["used"] += p["used"]
|
||||
s["packs"] += 1
|
||||
return {"packs": packs, "summary": summary, "days": days,
|
||||
"note": "官方余量有 12 小时延迟；现金余额需在可灵控制台查看"}
|
||||
|
||||
# ════════════════════════════════════════════════════════
|
||||
# 视频生音效（给已有视频自动配音效 / BGM）—— ¥0.25/次
|
||||
# ════════════════════════════════════════════════════════
|
||||
|
||||
def submit_video_to_audio(self, video_id: str = "", video_url: str = "",
|
||||
sound_effect_prompt: str = "", bgm_prompt: str = "",
|
||||
asmr_mode: bool = False) -> str:
|
||||
"""给视频生成音效/BGM（POST /v1/audio/video-to-audio）
|
||||
|
||||
· 成本 ¥0.25/次
|
||||
· video_id 与 video_url **二选一**（不能都填）
|
||||
· video_id 仅支持 30 天内的可灵生成视频，时长 3~20 秒
|
||||
· prompt ≤ 200 字符；asmr_mode=True 增强细节音效
|
||||
|
||||
用途：我们的成片默认**完全静音**（"AI 感"的重要来源），
|
||||
用此接口为镜头补真实环境音 → 显著提升真实感。
|
||||
"""
|
||||
if bool(video_id) == bool(video_url):
|
||||
raise ValueError("video_id 与 video_url 必须二选一")
|
||||
payload: dict = {}
|
||||
if video_id:
|
||||
payload["video_id"] = video_id
|
||||
else:
|
||||
payload["video_url"] = video_url
|
||||
if sound_effect_prompt:
|
||||
payload["sound_effect_prompt"] = sound_effect_prompt[:200]
|
||||
if bgm_prompt:
|
||||
payload["bgm_prompt"] = bgm_prompt[:200]
|
||||
if asmr_mode:
|
||||
payload["asmr_mode"] = True
|
||||
d = self._post("/v1/audio/video-to-audio", payload)
|
||||
return self._extract_task_id(d)
|
||||
|
||||
# ════════════════════════════════════════════════════════
|
||||
# 多图参考生图（主体 + 场景 + 风格 **分离**）—— ¥0.4/张
|
||||
# ════════════════════════════════════════════════════════
|
||||
|
||||
def submit_multi_image2image(self, subject_images: list, prompt: str = "",
|
||||
scene_image: str = "", style_image: str = "",
|
||||
aspect_ratio: str = "9:16", n: int = 1,
|
||||
model: str = "kling-v2-1") -> str:
|
||||
"""多图参考生图（POST /v1/images/multi-image2image）
|
||||
|
||||
· 成本 ¥0.4/张
|
||||
· subject_images: **1~4 张主体参考图**（URL 或 base64 字符串，不加前缀）
|
||||
我们的「5 视角套装」正适合喂进来（正面 + 45° + 侧面）
|
||||
· scene_image: **场景参考图** ← 解决「画面没有环境」的正解
|
||||
· style_image: 风格参考图
|
||||
· prompt ≤ 2500 字符
|
||||
· aspect_ratio: 16:9 / 9:16 / 1:1 / 4:3 / 3:4 / 3:2 / 2:3 / 21:9
|
||||
|
||||
⚠️ **实测约束（2026-10-04 探测）**：
|
||||
① **只支持 model=kling-v2-1**（kling-v3 / kling-image-o1 报 model is not supported）
|
||||
② **主体 + 场景 + 风格三组图「合计」至少 2 张**
|
||||
solo 1 张会报错；最小组合 = 1 主体 + 1 场景，或 2 张主体（正面+侧面）
|
||||
⚠️ 并发按 n 计（账号图片并发 9）→ 建议 n=1 串行调用，避免 1303
|
||||
|
||||
用途（回答「参考图该用空白背景还是场景」）：
|
||||
官方接口把主体与场景**分开** —— 角色图只管人物，场景图只管环境，
|
||||
两者不互相污染。这印证了业界结论。
|
||||
"""
|
||||
subs = [s for s in (subject_images or []) if s][:4]
|
||||
if not subs:
|
||||
raise ValueError("subject_images 至少需要 1 张主体参考图")
|
||||
if len(subs) + (1 if scene_image else 0) + (1 if style_image else 0) < 2:
|
||||
raise ValueError("三组图（主体/场景/风格）**合计至少 2 张**："
|
||||
"请补一张场景图，或提供 2 张以上主体图（如正面+侧面）")
|
||||
payload: dict = {
|
||||
"model_name": model,
|
||||
"subject_image_list": [{"subject_image": s} for s in subs],
|
||||
"aspect_ratio": aspect_ratio,
|
||||
"n": int(n),
|
||||
}
|
||||
if prompt:
|
||||
payload["prompt"] = prompt[:2500]
|
||||
if scene_image:
|
||||
payload["scene_image"] = scene_image
|
||||
if style_image:
|
||||
payload["style_image"] = style_image
|
||||
d = self._post("/v1/images/multi-image2image", payload)
|
||||
return self._extract_task_id(d)
|
||||
|
||||
# ── 通用：把可灵 task_id 记到 sidecar 文件（供「视频生音效」用）──
|
||||
@staticmethod
|
||||
def save_task_id(dest, task_id: str) -> None:
|
||||
"""把 task_id 写到 `{dest}.taskid`（video-to-audio 只认可灵 video_id）"""
|
||||
try:
|
||||
p = Path(dest)
|
||||
p.with_suffix(p.suffix + ".taskid").write_text(str(task_id), encoding="utf-8")
|
||||
except Exception as e:
|
||||
log.debug("记录 task_id 失败: %s", e)
|
||||
|
||||
@staticmethod
|
||||
def load_task_id(video_path) -> str:
|
||||
"""读取镜头对应的可灵 task_id（无则返回空串）"""
|
||||
try:
|
||||
p = Path(video_path)
|
||||
f = p.with_suffix(p.suffix + ".taskid")
|
||||
return f.read_text(encoding="utf-8").strip() if f.exists() else ""
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
def wait_for_audio(self, task_id: str, timeout: int = 600) -> str:
|
||||
"""等待「视频生音效」任务完成 → 返回音频 URL（通常 30~90 秒）"""
|
||||
import time as _t
|
||||
deadline = _t.time() + timeout
|
||||
last = ""
|
||||
while _t.time() < deadline:
|
||||
try:
|
||||
d = self._get(f"/v1/audio/video-to-audio/{task_id}")
|
||||
except Exception as e:
|
||||
log.debug("音效任务查询失败（重试）: %s", str(e)[:80])
|
||||
_t.sleep(8)
|
||||
continue
|
||||
data = d.get("data") or {}
|
||||
st = data.get("task_status", "")
|
||||
if st == "succeed":
|
||||
r = data.get("task_result") or {}
|
||||
urls = []
|
||||
for key in ("sound_effects", "bgm", "audios"):
|
||||
v = r.get(key)
|
||||
if isinstance(v, list):
|
||||
urls += [x.get("url") for x in v if isinstance(x, dict) and x.get("url")]
|
||||
elif isinstance(v, dict) and v.get("url"):
|
||||
urls.append(v["url"])
|
||||
if not urls:
|
||||
for key in ("video_url", "url"):
|
||||
if r.get(key):
|
||||
urls.append(r[key])
|
||||
if urls:
|
||||
return urls[0]
|
||||
raise KlingError(f"音效任务成功但无音频 URL: {str(r)[:200]}")
|
||||
if st in ("failed", "canceled"):
|
||||
raise KlingError(f"音效任务 {st}: {str(data.get('task_status_msg') or '')[:200]}")
|
||||
if st != last:
|
||||
last = st
|
||||
log.info(" 🎧 音效任务 %s: %s", task_id[:8], st)
|
||||
_t.sleep(8)
|
||||
raise KlingError(f"音效任务超时（>{timeout}s）")
|
||||
|
||||
def generate_sfx_for_video(self, video_id: str, dest: str | Path,
|
||||
sound_effect_prompt: str = "", bgm_prompt: str = "",
|
||||
asmr_mode: bool = False, timeout: int = 600) -> Path:
|
||||
"""给指定镜头生成音效并下载（**¥0.25/次**）
|
||||
|
||||
用途：我们的成片默认**完全静音**（"AI 感"的主要来源之一）。
|
||||
给镜头补真实环境音/BGM → 混进最终音轨后真实感显著提升。
|
||||
|
||||
⚠️ video_id 必须是**可灵平台的任务 ID**（生成镜头时记下的 task_id）
|
||||
"""
|
||||
tid = self.submit_video_to_audio(video_id=video_id,
|
||||
sound_effect_prompt=sound_effect_prompt,
|
||||
bgm_prompt=bgm_prompt, asmr_mode=asmr_mode)
|
||||
log.info(" 🎧 音效任务已提交: %s（¥0.25）", tid[:12])
|
||||
url = self.wait_for_audio(tid, timeout=timeout)
|
||||
self.download(url, dest)
|
||||
log.info(" ✅ 音效已下载: %s", Path(dest).name)
|
||||
return Path(dest)
|
||||
|
||||
def generate_video(self, prompt: str, dest: str | Path,
|
||||
image_path: str = "", duration: str = "5",
|
||||
aspect_ratio: str = "9:16", model: str = "kling-v3",
|
||||
timeout: int = 600, progress_cb=None,
|
||||
vfx_type: str = "", vfx_params: Optional[dict] = None,
|
||||
tail_image_path: str = "", sound: str = "off") -> Path:
|
||||
"""生成一个镜头片段（有参考图走图生视频，否则文生视频）
|
||||
|
||||
vfx_type：特效类型（orbit / bullet_time / slow_motion / focus_lock …）
|
||||
→ 自动转成可灵 `camera_control` 运镜参数
|
||||
tail_image_path：尾帧图 —— **首尾帧衔接**用（可灵 image_tail）：
|
||||
本镜接上一镜时，把上一镜的尾帧传进来，生成结果会从该尾帧收束
|
||||
sound：**原生音频**（"on"/"off"，实测可用）。开着它时提示词里写对话，
|
||||
会一次生成「带语音的视频」→ 音画天然同步，无需后期 TTS / 对口型。
|
||||
"""
|
||||
cam = self.build_camera_control(vfx_type, vfx_params)
|
||||
if cam:
|
||||
log.info(" 🎥 运镜控制: %s", cam)
|
||||
if sound == "on":
|
||||
log.info(" 🔊 原生有声模式（sound=on）—— 提示词里的对话将被念出来")
|
||||
if image_path and Path(image_path).exists():
|
||||
tid = self.submit_image2video(image_path, prompt, model=model, duration=duration,
|
||||
camera_control=cam, tail_image_path=tail_image_path,
|
||||
sound=sound)
|
||||
kind = "image2video"
|
||||
log.info(" 🎬 可灵图生视频任务: %s", tid)
|
||||
else:
|
||||
tid = self.submit_text2video(prompt, model=model, duration=duration,
|
||||
aspect_ratio=aspect_ratio, camera_control=cam,
|
||||
sound=sound)
|
||||
kind = "text2video"
|
||||
log.info(" 🎬 可灵文生视频任务: %s", tid)
|
||||
# ★ 记录 task_id（供后续「视频生音效」—— video-to-audio 只认可灵 video_id）
|
||||
self.save_task_id(dest, tid)
|
||||
url = self.wait_for_video(tid, kind=kind, timeout=timeout, progress_cb=progress_cb)
|
||||
return self.download(url, dest)
|
||||
|
||||
# ── 特效 → 运镜控制映射 ──────────────────────────
|
||||
|
||||
@staticmethod
|
||||
def build_camera_control(vfx_type: str,
|
||||
vfx_params: Optional[dict] = None) -> Optional[dict]:
|
||||
"""把「特效类型」映射成可灵 `camera_control` 运镜参数
|
||||
|
||||
可灵规范：
|
||||
type = simple（配合 config 六选一）| down_back | forward_up
|
||||
| right_turn_forward | left_turn_forward
|
||||
config（type=simple 时）：horizontal / vertical / pan / tilt / roll / zoom
|
||||
取值 [-10, 10]，**只能填一个且不能为 0**
|
||||
|
||||
映射：
|
||||
orbit 环绕运镜 → horizontal 6
|
||||
bullet_time 子弹时间 → roll 5（配合提示词升格 + 后处理变速）
|
||||
slow_motion 慢动作 → vertical 3（缓慢推进）
|
||||
focus_lock 焦点锁定 → zoom 4（变焦）
|
||||
down_back / forward_up / right_turn_forward / left_turn_forward（官方整段运镜预设）
|
||||
"""
|
||||
vt = (vfx_type or "").strip().lower()
|
||||
if not vt:
|
||||
return None
|
||||
presets: dict = {
|
||||
"orbit": {"type": "simple", "config": {"horizontal": 6}},
|
||||
"bullet_time": {"type": "simple", "config": {"roll": 5}},
|
||||
"slow_motion": {"type": "simple", "config": {"vertical": 3}},
|
||||
"focus_lock": {"type": "simple", "config": {"zoom": 4}},
|
||||
"down_back": {"type": "down_back", "config": {}},
|
||||
"forward_up": {"type": "forward_up", "config": {}},
|
||||
"right_turn_forward": {"type": "right_turn_forward", "config": {}},
|
||||
"left_turn_forward": {"type": "left_turn_forward", "config": {}},
|
||||
}
|
||||
if vt in presets:
|
||||
return presets[vt]
|
||||
if vfx_params: # 自定义运镜（直接给 config）
|
||||
return {"type": "simple", "config": vfx_params}
|
||||
return None
|
||||
@@ -0,0 +1,231 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
capability_doctor.py — 能力体检（一条命令验证「清单说的 ready 是不是真能跑」）
|
||||
|
||||
背景：capabilities.yaml 是能力唯一真相源，但清单可能与实际脱节
|
||||
（实测曾发现：frame_chain 代码早已实现却报 missing；
|
||||
lipsync 实际用百炼却登记成 fal）。
|
||||
本工具做静态体检，把「名不副实」的能力揪出来。
|
||||
|
||||
用法：
|
||||
python capability_doctor.py # 全部能力体检
|
||||
python capability_doctor.py --capability lipsync
|
||||
python capability_doctor.py --json # 机器可读（供 Dashboard）
|
||||
python capability_doctor.py --only-issues # 只看有问题的
|
||||
|
||||
检查项（全部零成本，不调用任何付费接口）：
|
||||
1. impl 文件是否存在（清单 impl 字段指向的代码）
|
||||
2. provider 的 api_key 是否已配置（api_key_ref → config.yaml + local.yaml）
|
||||
3. 状态一致性：status=ready 但没有任何可用 provider → ✗ 名不副实
|
||||
4. 结构完整性：是否声明了 providers / status
|
||||
|
||||
退出码：0 = 无问题；1 = 存在 ⚠️ / ✗
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# ── 定位 ──
|
||||
_SCRIPTS = Path(__file__).resolve().parent
|
||||
_AVE = _SCRIPTS.parent # 05_tools/09_ave
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError:
|
||||
print("需要 pyyaml：pip install pyyaml")
|
||||
sys.exit(2)
|
||||
|
||||
CAP_FILE = _AVE / "config" / "capabilities.yaml"
|
||||
PLACEHOLDERS = {"", "sk-xxx", "xxx", "your_key", "your_openai_api_key", "omlx", "changeme", "todo"}
|
||||
|
||||
|
||||
def load_caps() -> list:
|
||||
d = yaml.safe_load(CAP_FILE.read_text(encoding="utf-8")) or {}
|
||||
return d.get("capabilities") or []
|
||||
|
||||
|
||||
def load_config_merged() -> dict:
|
||||
"""合并 config.yaml（模板）与 local.yaml（本机真实密钥，优先级更高）"""
|
||||
merged: dict = {}
|
||||
|
||||
def _merge(src: dict):
|
||||
for k, v in (src or {}).items():
|
||||
if isinstance(v, dict) and isinstance(merged.get(k), dict):
|
||||
merged[k].update(v)
|
||||
else:
|
||||
merged[k] = v
|
||||
|
||||
tmpl = _AVE / "config.yaml"
|
||||
if tmpl.exists():
|
||||
try:
|
||||
_merge(yaml.safe_load(tmpl.read_text(encoding="utf-8")) or {})
|
||||
except Exception:
|
||||
pass
|
||||
local = Path.home() / "workbuddy-agent-os/agent-local/tools/ave/config/local.yaml"
|
||||
if local.exists():
|
||||
try:
|
||||
_merge(yaml.safe_load(local.read_text(encoding="utf-8")) or {})
|
||||
except Exception:
|
||||
pass
|
||||
return merged
|
||||
|
||||
|
||||
def _get_by_path(cfg: dict, dotted: str):
|
||||
cur = cfg
|
||||
for part in (dotted or "").split("."):
|
||||
if not isinstance(cur, dict) or part not in cur:
|
||||
return None
|
||||
cur = cur[part]
|
||||
return cur
|
||||
|
||||
|
||||
def _key_ok(val) -> bool:
|
||||
if val is None:
|
||||
return False
|
||||
s = str(val).strip()
|
||||
return bool(s) and s.lower() not in PLACEHOLDERS and not s.startswith("${")
|
||||
|
||||
|
||||
def _impl_targets(impl: str) -> list:
|
||||
"""从 impl 描述提取可核对的路径（文件或目录都可）
|
||||
|
||||
· 'composer/lipsync.py（lipsync_videoretalk）' → ['composer/lipsync.py']
|
||||
· 'music_selector/' → ['music_selector']
|
||||
· 'assets_manager/ + character_generator/' → ['assets_manager', 'character_generator']
|
||||
"""
|
||||
if not impl:
|
||||
return []
|
||||
files = re.findall(r"([\w./-]+\.py)", impl)
|
||||
if files:
|
||||
return files
|
||||
dirs = re.findall(r"([\w][\w./-]*)/", impl)
|
||||
seen, out = set(), []
|
||||
for d in dirs:
|
||||
d = d.rstrip('/')
|
||||
if d and d not in seen:
|
||||
seen.add(d)
|
||||
out.append(d)
|
||||
return out or [impl.strip()]
|
||||
|
||||
|
||||
def check_capability(cap: dict, cfg: dict) -> dict:
|
||||
"""单个能力的静态体检结果"""
|
||||
cid = cap.get("id", "?")
|
||||
status = (cap.get("status") or "").strip() or "(未声明)"
|
||||
providers = cap.get("providers") or []
|
||||
issues: list[str] = []
|
||||
warns: list[str] = []
|
||||
|
||||
# ① impl 实现（文件或目录）
|
||||
impl = cap.get("impl") or ""
|
||||
targets = _impl_targets(impl)
|
||||
impl_ok = None
|
||||
if targets:
|
||||
found = []
|
||||
missing = []
|
||||
for rel in targets:
|
||||
cands = [_SCRIPTS / rel, _SCRIPTS / "capabilities" / rel, _AVE / rel]
|
||||
(found if any(c.exists() for c in cands) else missing).append(rel)
|
||||
impl_ok = bool(found)
|
||||
# planned / missing 能力允许尚无实现；ready/partial 缺实现才是问题
|
||||
if missing and status in ("ready", "partial"):
|
||||
warns.append(f"impl 未找到: {', '.join(missing[:3])}")
|
||||
elif status in ("ready", "partial"):
|
||||
warns.append("未声明 impl（无法核对实现）")
|
||||
|
||||
# ② provider 的 key 配置情况
|
||||
prov_ok: list[str] = []
|
||||
prov_missing: list[str] = []
|
||||
for p in providers:
|
||||
ref = p.get("api_key_ref") or ""
|
||||
pid = p.get("id") or "?"
|
||||
if not ref:
|
||||
prov_ok.append(f"{pid}(无需key)") # 本地实现（ffmpeg 等）
|
||||
continue
|
||||
val = _get_by_path(cfg, ref)
|
||||
if _key_ok(val):
|
||||
prov_ok.append(pid)
|
||||
else:
|
||||
prov_missing.append(f"{pid}({ref})")
|
||||
|
||||
# ③ 状态一致性
|
||||
if status == "ready":
|
||||
if providers and not prov_ok:
|
||||
issues.append(f"status=ready 但没有任何可用 provider（缺: {', '.join(prov_missing)}）")
|
||||
if impl_ok is False:
|
||||
issues.append("status=ready 但 impl 文件不存在")
|
||||
elif status == "partial":
|
||||
if providers and not prov_ok:
|
||||
warns.append(f"partial 且全部 provider 不可用（缺: {', '.join(prov_missing)}）")
|
||||
elif status in ("planned", "missing", ""):
|
||||
pass
|
||||
|
||||
# ④ 结构完整性
|
||||
if not providers:
|
||||
warns.append("未声明 providers")
|
||||
|
||||
verdict = "✗" if issues else ("⚠️" if warns else "✓")
|
||||
return {
|
||||
"id": cid,
|
||||
"name": cap.get("name", ""),
|
||||
"status": status,
|
||||
"impl": impl,
|
||||
"impl_ok": impl_ok,
|
||||
"providers_ok": prov_ok,
|
||||
"providers_missing": prov_missing,
|
||||
"issues": issues,
|
||||
"warns": warns,
|
||||
"verdict": verdict,
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description="能力体检（静态，零成本）")
|
||||
ap.add_argument("--capability", help="只查某个能力 id")
|
||||
ap.add_argument("--json", action="store_true", help="输出 JSON")
|
||||
ap.add_argument("--only-issues", action="store_true", help="只显示 ⚠️/✗")
|
||||
args = ap.parse_args()
|
||||
|
||||
caps = load_caps()
|
||||
cfg = load_config_merged()
|
||||
if args.capability:
|
||||
caps = [c for c in caps if c.get("id") == args.capability]
|
||||
if not caps:
|
||||
print(f"未找到能力: {args.capability}")
|
||||
return 2
|
||||
|
||||
results = [check_capability(c, cfg) for c in caps]
|
||||
if args.only_issues:
|
||||
results = [r for r in results if r["verdict"] != "✓"]
|
||||
|
||||
if args.json:
|
||||
print(json.dumps({"total": len(results), "results": results}, ensure_ascii=False, indent=1))
|
||||
else:
|
||||
bad = [r for r in results if r["verdict"] == "✗"]
|
||||
warn = [r for r in results if r["verdict"] == "⚠️"]
|
||||
good = [r for r in results if r["verdict"] == "✓"]
|
||||
print(f"\n能力体检（共 {len(results)} 项） ✓ {len(good)} · ⚠️ {len(warn)} · ✗ {len(bad)}\n")
|
||||
for r in results:
|
||||
if args.only_issues and r["verdict"] == "✓":
|
||||
continue
|
||||
line = f" {r['verdict']} {r['id']:26s} [{r['status']:8s}]"
|
||||
if r["providers_ok"]:
|
||||
line += f" ✓{','.join(r['providers_ok'])[:36]}"
|
||||
if r["providers_missing"]:
|
||||
line += f" ✗{','.join(r['providers_missing'])[:36]}"
|
||||
print(line)
|
||||
for m in r["issues"]:
|
||||
print(f" ✗ {m}")
|
||||
for m in r["warns"]:
|
||||
print(f" · {m}")
|
||||
print()
|
||||
|
||||
return 1 if any(r["verdict"] == "✗" for r in results) else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,382 @@
|
||||
"""
|
||||
capability_matcher — 技术优选层（创作意图 → 技术方案 + 缺口报告）
|
||||
|
||||
职责：
|
||||
1. 读分镜（Shot 列表，含 tech_requirements 技术需求声明）
|
||||
2. 对照能力目录（capabilities.yaml）与可行性规则库（feasibility_rules.yaml）
|
||||
3. 产出每个镜头的「技术方案」（用什么能力/引擎/参数）与「缺口清单」
|
||||
4. 计算能力目录指纹（版本+内容 hash），供「能力更新后自动升级方案」使用
|
||||
|
||||
设计原则：
|
||||
· 导演只管创作表达，不指定工具；本层负责把需求翻译成技术方案
|
||||
· 缺口是显性产物：无能力满足的需求 → 缺口清单 → 驱动能力建设
|
||||
· 能力目录更新 → 指纹变化 → 自动标记「方案可升级 / 缺口已解决」
|
||||
|
||||
用法：
|
||||
from capability_matcher import match_storyboard, catalog_fingerprint, check_upgradable
|
||||
plan = match_storyboard(storyboard_dict)
|
||||
fp = catalog_fingerprint()
|
||||
up = check_upgradable(old_plan)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
from lib.logger import get_logger # noqa: E402
|
||||
|
||||
log = get_logger("capability_matcher")
|
||||
|
||||
_AVE_DIR = _SCRIPTS.parent
|
||||
_CAPABILITIES = _AVE_DIR / "config" / "capabilities.yaml"
|
||||
_RULES = _AVE_DIR / "config" / "feasibility_rules.yaml"
|
||||
|
||||
# 生成能力 → 能力目录 id 映射（分镜 generation_capability / type → capability id）
|
||||
_CAP_MAP = {
|
||||
"text_to_video": "text_to_video",
|
||||
"image_to_video": "image_to_video",
|
||||
"vfx": "vfx_apply",
|
||||
"comfyui": "comfyui_workflow",
|
||||
"3d_character": "character_3d",
|
||||
}
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 目录加载与指纹
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
def load_capabilities() -> dict:
|
||||
try:
|
||||
return yaml.safe_load(_CAPABILITIES.read_text(encoding="utf-8")) or {}
|
||||
except Exception as e:
|
||||
log.warning("能力目录读取失败: %s", e)
|
||||
return {}
|
||||
|
||||
|
||||
def load_rules() -> dict:
|
||||
try:
|
||||
return yaml.safe_load(_RULES.read_text(encoding="utf-8")) or {}
|
||||
except Exception as e:
|
||||
log.warning("规则库读取失败: %s", e)
|
||||
return {}
|
||||
|
||||
|
||||
def catalog_fingerprint() -> dict:
|
||||
"""能力目录 + 规则库的内容指纹（版本号 + hash，两者任一变化都能检测）"""
|
||||
def _h(p: Path) -> str:
|
||||
try:
|
||||
return hashlib.sha256(p.read_bytes()).hexdigest()[:16]
|
||||
except Exception:
|
||||
return ""
|
||||
caps = load_capabilities()
|
||||
rules = load_rules()
|
||||
return {
|
||||
"caps_version": caps.get("version", ""),
|
||||
"caps_updated": caps.get("updated", ""),
|
||||
"caps_hash": _h(_CAPABILITIES),
|
||||
"rules_version": rules.get("version", ""),
|
||||
"rules_hash": _h(_RULES),
|
||||
}
|
||||
|
||||
|
||||
def _cap_index() -> dict:
|
||||
"""能力 id → 能力定义"""
|
||||
caps = load_capabilities().get("capabilities", []) or []
|
||||
return {c.get("id"): c for c in caps if c.get("id")}
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 规则引擎
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
def _rule_hit(rule: dict, shot: dict) -> bool:
|
||||
"""判断单条规则是否命中该镜头"""
|
||||
tr = shot.get("tech_requirements") or {}
|
||||
cont = (tr.get("continuity") or {})
|
||||
tg = rule.get("trigger") or {}
|
||||
t = tg.get("type", "")
|
||||
|
||||
if t == "shot_duration_over":
|
||||
try:
|
||||
return float(shot.get("duration") or 0) > float(tg.get("value") or 0)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
if t == "duration_and_motion":
|
||||
try:
|
||||
over = float(shot.get("duration") or 0) > float(tg.get("duration_over") or 0)
|
||||
except Exception:
|
||||
over = False
|
||||
return over and (tr.get("motion_level") in (tg.get("motion_in") or []))
|
||||
|
||||
if t == "text_fit":
|
||||
text = shot.get("text") or ""
|
||||
if not text:
|
||||
return False
|
||||
try:
|
||||
need = len(str(text).strip()) / float(tg.get("chars_per_sec") or 4.5)
|
||||
return need > float(shot.get("duration") or 0) + 0.3 # 容差
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
if t == "tech_flag":
|
||||
return tr.get(tg.get("field", "")) == tg.get("value")
|
||||
|
||||
if t == "tech_continuity":
|
||||
return cont.get(tg.get("field", "")) in (tg.get("values") or [])
|
||||
|
||||
if t == "shot_or_need":
|
||||
if shot.get("type") == tg.get("shot_type"):
|
||||
return True
|
||||
kws = tg.get("need_keywords") or []
|
||||
blob = " ".join([str(shot.get("visual_description") or ""), str(shot.get("action") or "")]
|
||||
+ [str(n) for n in (tr.get("needs") or [])])
|
||||
return any(k in blob for k in kws)
|
||||
|
||||
if t == "need_keywords":
|
||||
blob = " ".join([str(shot.get("visual_description") or ""), str(shot.get("action") or "")]
|
||||
+ [str(n) for n in (tr.get("needs") or [])])
|
||||
return any(k in blob for k in (tg.get("keywords") or []))
|
||||
|
||||
if t == "manual_check":
|
||||
return False # 人工核对项，不在自动匹配中命中（由前端提示）
|
||||
|
||||
return False
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 基础能力判定（镜头 → 用哪个生成能力）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
def base_capability(shot: dict) -> str:
|
||||
"""镜头的生成能力：显式指定 > 按类型推导
|
||||
|
||||
talking_head → image_to_video（绑角色）或 text_to_video
|
||||
vfx → vfx_apply；comfyui → comfyui_workflow；3d_character → character_3d
|
||||
其他 → image_to_video（有角色锚点）或 text_to_video
|
||||
"""
|
||||
explicit = (shot.get("generation_capability") or "").strip()
|
||||
if explicit:
|
||||
return _CAP_MAP.get(explicit, explicit)
|
||||
t = shot.get("type") or "action"
|
||||
if t in ("vfx",):
|
||||
return "vfx_apply"
|
||||
if t == "comfyui":
|
||||
return "comfyui_workflow"
|
||||
if t == "3d_character":
|
||||
return "character_3d"
|
||||
# 绑了角色（含多角色演员表 cast_slots）或口播 → 图生视频
|
||||
_slots = [s for s in (shot.get("cast_slots") or []) if s]
|
||||
if (shot.get("character_id") or "").strip() or _slots or t == "talking_head":
|
||||
return "image_to_video"
|
||||
return "text_to_video"
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 主入口：匹配
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
def match_storyboard(storyboard: dict | list, project_id: str = "") -> dict:
|
||||
"""分镜 → 技术方案 + 缺口报告
|
||||
|
||||
参数 storyboard: Storyboard.to_dict() 或 Shot 列表
|
||||
返回：
|
||||
{
|
||||
project_id, generated_at, catalog: {...指纹},
|
||||
shots: [ {shot_id, capability_id, capability_name, provider, maturity,
|
||||
plan_status, issues:[{rule_id,name,level,problem,action}], duration_slot } ],
|
||||
gaps: [ {need_key, name, status, problem, action, shot_ids:[...]} ],
|
||||
summary: {total, ok, downgraded, blocked, gap_count}
|
||||
}
|
||||
"""
|
||||
shots = storyboard.get("shots", []) if isinstance(storyboard, dict) else (storyboard or [])
|
||||
if isinstance(storyboard, dict) and not project_id:
|
||||
project_id = storyboard.get("project_id", "")
|
||||
|
||||
caps = _cap_index()
|
||||
rules = (load_rules().get("rules") or [])
|
||||
needs_map = load_rules().get("capability_needs") or {}
|
||||
|
||||
shot_plans: list[dict] = []
|
||||
gap_acc: dict[str, dict] = {}
|
||||
|
||||
for s in shots:
|
||||
sid = s.get("shot_id", "")
|
||||
cap_id = base_capability(s)
|
||||
cap = caps.get(cap_id) or {}
|
||||
maturity = cap.get("status", "unknown")
|
||||
providers = cap.get("providers") or []
|
||||
provider = ""
|
||||
if providers and isinstance(providers[0], dict):
|
||||
provider = providers[0].get("name", "")
|
||||
|
||||
# 跑规则
|
||||
issues = []
|
||||
for r in rules:
|
||||
if r.get("level") == "info":
|
||||
continue # info 不进问题列表（可另作提示）
|
||||
if _rule_hit(r, s):
|
||||
issues.append({
|
||||
"rule_id": r.get("id"), "name": r.get("name"),
|
||||
"level": r.get("level"), "nature": r.get("nature", ""),
|
||||
"problem": r.get("problem", ""),
|
||||
"action": r.get("action", ""),
|
||||
})
|
||||
nk = r.get("need")
|
||||
if nk:
|
||||
g = gap_acc.setdefault(nk, {
|
||||
"need_key": nk,
|
||||
"name": (needs_map.get(nk) or {}).get("name", nk),
|
||||
"status": (needs_map.get(nk) or {}).get("status", "unknown"),
|
||||
"note": (needs_map.get(nk) or {}).get("note", ""),
|
||||
"problem": r.get("problem", ""),
|
||||
"action": r.get("action", ""),
|
||||
"shot_ids": [],
|
||||
})
|
||||
if sid and sid not in g["shot_ids"]:
|
||||
g["shot_ids"].append(sid)
|
||||
|
||||
# 方案状态
|
||||
if any(i["level"] == "blocking" for i in issues):
|
||||
status = "blocked"
|
||||
elif issues:
|
||||
status = "downgraded"
|
||||
elif maturity in ("planned", "missing"):
|
||||
status = "blocked"
|
||||
else:
|
||||
status = "ok"
|
||||
|
||||
# 时长档位映射（生成能力档位：5s / 10s）
|
||||
try:
|
||||
dur = float(s.get("duration") or 0)
|
||||
except Exception:
|
||||
dur = 0.0
|
||||
slot = 10 if dur > 5 else 5
|
||||
if dur > 10:
|
||||
slot = None # 超限，需切条/接续
|
||||
|
||||
shot_plans.append({
|
||||
"shot_id": sid,
|
||||
"capability_id": cap_id,
|
||||
"capability_name": cap.get("name", cap_id),
|
||||
"provider": provider,
|
||||
"maturity": maturity,
|
||||
"duration_slot": slot,
|
||||
"plan_status": status,
|
||||
"issues": issues,
|
||||
"manual_override": bool(s.get("tech_plan_manual")),
|
||||
})
|
||||
|
||||
gaps = sorted(gap_acc.values(), key=lambda g: g["need_key"])
|
||||
summary = {
|
||||
"total": len(shot_plans),
|
||||
"ok": sum(1 for p in shot_plans if p["plan_status"] == "ok"),
|
||||
"downgraded": sum(1 for p in shot_plans if p["plan_status"] == "downgraded"),
|
||||
"blocked": sum(1 for p in shot_plans if p["plan_status"] == "blocked"),
|
||||
"gap_count": len(gaps),
|
||||
}
|
||||
return {
|
||||
"project_id": project_id,
|
||||
"generated_at": _now(),
|
||||
"catalog": catalog_fingerprint(),
|
||||
"shots": shot_plans,
|
||||
"gaps": gaps,
|
||||
"summary": summary,
|
||||
}
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 能力更新 → 自动升级检测
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
def check_upgradable(old_plan: dict) -> dict:
|
||||
"""对比旧技术方案的能力目录指纹与当前指纹
|
||||
|
||||
返回：
|
||||
{
|
||||
outdated: bool,
|
||||
reason: str,
|
||||
resolved_gaps: [ {need_key, name, was_status, now_status} ], # 缺口已解决
|
||||
affected_shots: [shot_id], # 需重算的镜头
|
||||
summary: {resolved, still_open}
|
||||
}
|
||||
"""
|
||||
if not old_plan:
|
||||
return {"outdated": False, "reason": "无历史方案", "resolved_gaps": [],
|
||||
"affected_shots": [], "summary": {"resolved": 0, "still_open": 0}}
|
||||
|
||||
old_fp = old_plan.get("catalog") or {}
|
||||
now_fp = catalog_fingerprint()
|
||||
changed = (old_fp.get("caps_hash") != now_fp.get("caps_hash")
|
||||
or old_fp.get("rules_hash") != now_fp.get("rules_hash"))
|
||||
|
||||
needs_map = load_rules().get("capability_needs") or {}
|
||||
resolved, still_open, affected = [], [], []
|
||||
|
||||
for g in (old_plan.get("gaps") or []):
|
||||
nk = g.get("need_key")
|
||||
was = g.get("status")
|
||||
now = (needs_map.get(nk) or {}).get("status", "missing")
|
||||
if was in ("missing", "planned") and now in ("ready", "partial"):
|
||||
resolved.append({"need_key": nk, "name": g.get("name", nk),
|
||||
"was_status": was, "now_status": now})
|
||||
affected.extend(g.get("shot_ids") or [])
|
||||
else:
|
||||
still_open.append(nk)
|
||||
|
||||
if not changed:
|
||||
reason = "能力目录未变化"
|
||||
elif resolved:
|
||||
reason = f"能力目录已更新：{len(resolved)} 项缺口已解决，可重新生成方案"
|
||||
else:
|
||||
reason = "能力目录已更新（无缺口变化）"
|
||||
|
||||
return {
|
||||
"outdated": changed,
|
||||
"reason": reason,
|
||||
"resolved_gaps": resolved,
|
||||
"affected_shots": sorted(set(affected)),
|
||||
"summary": {"resolved": len(resolved), "still_open": len(still_open)},
|
||||
}
|
||||
|
||||
|
||||
# ── CLI 自测 ──
|
||||
if __name__ == "__main__":
|
||||
import json
|
||||
fp = catalog_fingerprint()
|
||||
print("目录指纹:", json.dumps(fp, ensure_ascii=False))
|
||||
demo = {
|
||||
"project_id": "demo",
|
||||
"shots": [
|
||||
{"shot_id": "shot_01", "duration": 4, "type": "talking_head",
|
||||
"text": "你好，这是测试台词。", "character_id": "c1",
|
||||
"tech_requirements": {"continuity": {"character": "must", "expression": "prefer"},
|
||||
"chain_from_prev": False, "motion_level": "small",
|
||||
"camera_difficulty": "fixed", "needs": []}},
|
||||
{"shot_id": "shot_02", "duration": 8, "type": "action",
|
||||
"tech_requirements": {"continuity": {"location": "must"}, "chain_from_prev": True,
|
||||
"motion_level": "large", "camera_difficulty": "complex",
|
||||
"needs": ["子弹时间"]}},
|
||||
{"shot_id": "shot_03", "duration": 12, "type": "action", "text": "",
|
||||
"tech_requirements": {"needs": ["手部特写"]}},
|
||||
],
|
||||
}
|
||||
plan = match_storyboard(demo)
|
||||
print("\n方案摘要:", json.dumps(plan["summary"], ensure_ascii=False))
|
||||
for p in plan["shots"]:
|
||||
print(f" {p['shot_id']}: {p['capability_name']} slot={p['duration_slot']} status={p['plan_status']} issues={[i['rule_id'] for i in p['issues']]}")
|
||||
print("\n缺口:")
|
||||
for g in plan["gaps"]:
|
||||
print(f" [{g['status']}] {g['name']} ← {g['shot_ids']}")
|
||||
@@ -0,0 +1,397 @@
|
||||
"""
|
||||
角色适配器 (Character Adapter) — 漫剧视频工厂
|
||||
|
||||
功能:
|
||||
- 接收脚本 + 角色注册中心的角色信息
|
||||
- 遍历脚本的每个 segment，注入角色描述块
|
||||
- 生成适配后的分镜列表（含角色参考信息）
|
||||
- 输出可用于 Gate 2 展示的数据
|
||||
|
||||
用法:
|
||||
from character_adapter import CharacterAdapter
|
||||
from character_registry import CharacterRegistry
|
||||
|
||||
registry = CharacterRegistry()
|
||||
adapter = CharacterAdapter(registry)
|
||||
adapted = adapter.adapt(script)
|
||||
"""
|
||||
|
||||
import copy
|
||||
import os
|
||||
from typing import Optional
|
||||
|
||||
|
||||
# ── 表情映射 ──
|
||||
|
||||
EMOTION_MAP = {
|
||||
"中性": "neutral",
|
||||
"微笑": "smile",
|
||||
"开心": "happy",
|
||||
"愤怒": "angry",
|
||||
"悲伤": "sad",
|
||||
"害怕": "fear",
|
||||
"惊讶": "surprise",
|
||||
"厌恶": "disgust",
|
||||
"平静": "neutral",
|
||||
"喜悦": "happy",
|
||||
"激动": "excited",
|
||||
"忧郁": "sad",
|
||||
"冷漠": "neutral",
|
||||
}
|
||||
|
||||
|
||||
def _map_emotion(chinese_emotion: str) -> str:
|
||||
"""将中文情绪映射为表情键名"""
|
||||
return EMOTION_MAP.get(chinese_emotion, "neutral")
|
||||
|
||||
|
||||
class CharacterAdapter:
|
||||
"""角色适配器 — 将脚本中的主角替换为注册角色的描述"""
|
||||
|
||||
def __init__(self, registry=None):
|
||||
"""
|
||||
参数:
|
||||
registry: CharacterRegistry 实例，None 则延迟初始化
|
||||
"""
|
||||
self._registry = registry
|
||||
|
||||
@property
|
||||
def registry(self):
|
||||
if self._registry is None:
|
||||
from character_registry import CharacterRegistry
|
||||
self._registry = CharacterRegistry()
|
||||
return self._registry
|
||||
|
||||
# ── 核心方法 ──
|
||||
|
||||
def adapt(self, script: dict) -> dict:
|
||||
"""
|
||||
将脚本中的每个分镜适配为注册角色版本
|
||||
|
||||
参数:
|
||||
script: 标准格式脚本字典
|
||||
|
||||
返回:
|
||||
适配后的脚本（含 character_block 字段）
|
||||
"""
|
||||
char = self.registry.get_active_character()
|
||||
adapted = copy.deepcopy(script)
|
||||
|
||||
meta = adapted.get("meta", {})
|
||||
meta["adapted_character"] = char.name
|
||||
meta["adapt_time"] = __import__("time").strftime("%Y-%m-%d %H:%M:%S")
|
||||
|
||||
for seg in adapted.get("segments", []):
|
||||
self._adapt_segment(seg, char)
|
||||
|
||||
return adapted
|
||||
|
||||
def adapt_to_storyboard(self, script: dict) -> list[dict]:
|
||||
"""
|
||||
将脚本转换为适配后的分镜列表（展平，无 meta）
|
||||
|
||||
返回:
|
||||
每个分镜包含角色描述块信息，供 Gate 2 展示和视觉生成使用
|
||||
"""
|
||||
char = self.registry.get_active_character()
|
||||
result = []
|
||||
|
||||
for seg in script.get("segments", []):
|
||||
adapted_seg = copy.deepcopy(seg)
|
||||
self._adapt_segment(adapted_seg, char)
|
||||
result.append(adapted_seg)
|
||||
|
||||
return result
|
||||
|
||||
def _adapt_segment(self, seg: dict, char) -> dict:
|
||||
"""
|
||||
适配单个分镜 — 注入角色描述块
|
||||
|
||||
对每个分镜:
|
||||
1. 生成角色描述块 (character_block)
|
||||
2. 在 scene/description 中确保角色名正确
|
||||
3. 注入角色参考信息
|
||||
"""
|
||||
# 生成角色描述块
|
||||
seg["character_block"] = char.build_prompt_block(
|
||||
style=seg.get("visual_style", "manhua"),
|
||||
expression=seg.get("emotion", "中性"),
|
||||
scene=seg.get("scene", ""),
|
||||
camera_angle=seg.get("camera", "中景"),
|
||||
)
|
||||
|
||||
# 注入角色参考信息
|
||||
seg["character_ref"] = char.to_storyboard_block()
|
||||
|
||||
# 注入参考图路径（用于 Kling character_refs）
|
||||
ref_images = char.reference_images
|
||||
if ref_images.get("portrait"):
|
||||
seg["reference_image"] = ref_images["portrait"]
|
||||
elif ref_images.get("grid"):
|
||||
seg["reference_image"] = ref_images["grid"]
|
||||
|
||||
# 注入表情参考图（根据 segment 的情绪）
|
||||
emotion = seg.get("emotion", "中性")
|
||||
expressions = ref_images.get("expressions", {})
|
||||
emotion_key = _map_emotion(emotion)
|
||||
if emotion_key in expressions and expressions[emotion_key]:
|
||||
seg["emotion_reference"] = expressions[emotion_key]
|
||||
|
||||
# 确保场景描述中出现角色名（如果还没出现）
|
||||
if char.name not in seg.get("description", ""):
|
||||
seg["description"] = f"{char.name}{seg.get('description', '')}"
|
||||
|
||||
return seg
|
||||
|
||||
# ── 批量适配 ──
|
||||
|
||||
def adapt_batch(self, scripts: list[dict]) -> list[dict]:
|
||||
"""批量适配多个脚本"""
|
||||
return [self.adapt(s) for s in scripts]
|
||||
|
||||
# ── 预览 ──
|
||||
|
||||
def preview_adaptation(self, script: dict) -> dict:
|
||||
"""
|
||||
预览适配效果（不修改原始脚本）
|
||||
|
||||
返回:
|
||||
{
|
||||
"character": "小漫",
|
||||
"segments_before": [...], # 适配前的 scene 列表
|
||||
"segments_after": [...] # 适配后的 scene+character_block
|
||||
}
|
||||
"""
|
||||
char = self.registry.get_active_character()
|
||||
before = []
|
||||
after = []
|
||||
|
||||
for seg in script.get("segments", []):
|
||||
before.append({
|
||||
"id": seg.get("id"),
|
||||
"scene": seg.get("scene"),
|
||||
"description": seg.get("description"),
|
||||
})
|
||||
|
||||
adapted = copy.deepcopy(seg)
|
||||
self._adapt_segment(adapted, char)
|
||||
after.append({
|
||||
"id": adapted.get("id"),
|
||||
"scene": adapted.get("scene"),
|
||||
"description": adapted.get("description"),
|
||||
"character_block": adapted.get("character_block", ""),
|
||||
})
|
||||
|
||||
return {
|
||||
"character": char.name,
|
||||
"total_segments": len(script.get("segments", [])),
|
||||
"before": before,
|
||||
"after": after,
|
||||
}
|
||||
|
||||
|
||||
# ── Kling character_refs 构建 ──
|
||||
|
||||
def build_kling_character_refs(self, name: str = "") -> list[dict]:
|
||||
"""
|
||||
构建 Kling API 支持的 character_refs 格式
|
||||
|
||||
Kling 图生视频支持传入 character_refs 数组来保持角色一致性:
|
||||
[{"image_url": "path/to/portrait.png", "image_type": "人物全身照"}]
|
||||
|
||||
返回:
|
||||
[{"image_url": "...", "image_type": "人物全身照"}, ...]
|
||||
"""
|
||||
char = self.registry.get_character(name) if name else self.registry.get_active_character()
|
||||
ref_images = char.reference_images
|
||||
refs = []
|
||||
|
||||
# 优先级: portrait > grid > cells
|
||||
portrait = ref_images.get("portrait", "")
|
||||
grid = ref_images.get("grid", "")
|
||||
|
||||
# 如果 portrait 可用 → 直接使用
|
||||
if portrait:
|
||||
refs.append({
|
||||
"image_url": _ensure_file_url(portrait),
|
||||
"image_type": "人物全身照",
|
||||
})
|
||||
# 添加表情参考
|
||||
expressions = ref_images.get("expressions", {})
|
||||
for emotion, img_path in expressions.items():
|
||||
if img_path:
|
||||
refs.append({
|
||||
"image_url": _ensure_file_url(img_path),
|
||||
"image_type": f"表情_{emotion}",
|
||||
})
|
||||
elif grid:
|
||||
refs.append({
|
||||
"image_url": _ensure_file_url(grid),
|
||||
"image_type": "人物全身照",
|
||||
})
|
||||
|
||||
return refs
|
||||
|
||||
def inject_character_refs(self, segments: list[dict], char_name: str = "") -> list[dict]:
|
||||
"""
|
||||
将 character_refs 注入到每个 segment 中
|
||||
|
||||
参数:
|
||||
segments: 分镜列表
|
||||
char_name: 角色名（可选，默认当前活跃角色）
|
||||
|
||||
返回:
|
||||
注入 character_refs 后的分镜列表
|
||||
"""
|
||||
char = self.registry.get_character(char_name) if char_name else self.registry.get_active_character()
|
||||
refs = self.build_kling_character_refs(char_name or char.name)
|
||||
|
||||
result = []
|
||||
for seg in segments:
|
||||
adapted = copy.deepcopy(seg)
|
||||
# 注入 character_refs（每个分镜独立一份，方便逐帧控制）
|
||||
adapted["character_refs"] = refs
|
||||
result.append(adapted)
|
||||
|
||||
return result
|
||||
|
||||
# ── Visual Storyboard 构建（Gate 1.5 用） ──
|
||||
|
||||
def build_visual_prompts(self, storyboard: list[dict]) -> list[dict]:
|
||||
"""
|
||||
从适配后的分镜生成视觉提示词（Gate 1.5 审核内容）
|
||||
|
||||
每个分镜包含完整视觉 prompt:
|
||||
- character_block: 角色描述块
|
||||
- scene: 场景描述
|
||||
- visual_prompt: 完整视觉 prompt = 角色块 + 场景 + 镜头
|
||||
- reference_image: 参考图路径
|
||||
- emotion: 情绪
|
||||
- duration_sec: 时长
|
||||
|
||||
返回:
|
||||
视觉提示词列表
|
||||
"""
|
||||
char = self.registry.get_active_character()
|
||||
char_block = char.build_prompt_block(style="manhua")
|
||||
|
||||
result = []
|
||||
for seg in storyboard:
|
||||
# 构建完整 visual prompt
|
||||
scene_desc = seg.get("scene", "")
|
||||
camera = seg.get("camera", "中景")
|
||||
emotion = seg.get("emotion", "中性")
|
||||
duration = seg.get("duration_sec", 5)
|
||||
|
||||
visual_prompt = (
|
||||
f"{char_block}\n"
|
||||
f"[场景] {scene_desc}\n"
|
||||
f"[镜头] {camera}\n"
|
||||
f"[表情] {emotion}"
|
||||
)
|
||||
|
||||
entry = {
|
||||
"id": seg.get("id", 0),
|
||||
"scene": scene_desc,
|
||||
"visual_prompt": visual_prompt,
|
||||
"reference_image": seg.get("reference_image",
|
||||
char.reference_images.get("portrait", "")),
|
||||
"emotion": emotion,
|
||||
"duration_sec": duration,
|
||||
}
|
||||
|
||||
# 如果有关键词/台词
|
||||
if seg.get("narration") or seg.get("dialogue"):
|
||||
entry["narration"] = seg.get("narration", "")
|
||||
entry["dialogue"] = seg.get("dialogue", "")
|
||||
|
||||
result.append(entry)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
# ── 工具函数 ──
|
||||
|
||||
def _ensure_file_url(path: str) -> str:
|
||||
"""确保路径以 file:// 开头（Kling API 需要 URL）"""
|
||||
if not path:
|
||||
return ""
|
||||
if path.startswith("http://") or path.startswith("https://") or path.startswith("file://"):
|
||||
return path
|
||||
return f"file://{os.path.abspath(path)}"
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# CLI
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
def cli():
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser(description="角色适配器")
|
||||
parser.add_argument("action", nargs="?", default="preview",
|
||||
choices=["adapt", "preview", "batch", "refs", "visual-prompts"])
|
||||
parser.add_argument("--script", default="", help="脚本 YAML 文件路径")
|
||||
parser.add_argument("--character", default="", help="指定角色（可选）")
|
||||
parser.add_argument("--output", default="", help="输出路径")
|
||||
|
||||
args = parser.parse_args()
|
||||
from character_registry import CharacterRegistry
|
||||
from script_schemas import load_script
|
||||
|
||||
registry = CharacterRegistry()
|
||||
adapter = CharacterAdapter(registry)
|
||||
|
||||
# 如果有指定角色，切换过去
|
||||
if args.character:
|
||||
registry.switch_to(args.character)
|
||||
|
||||
print(f"角色适配器: 当前角色 = {registry.get_active_name()}")
|
||||
print()
|
||||
|
||||
if args.action in ("adapt", "preview"):
|
||||
if not args.script:
|
||||
print("❌ 请指定 --script")
|
||||
return
|
||||
|
||||
script = load_script(args.script)
|
||||
|
||||
if args.action == "preview":
|
||||
preview = adapter.preview_adaptation(script)
|
||||
print(f"预览适配 ({preview['total_segments']} 个分镜):")
|
||||
print(f"角色: {preview['character']}")
|
||||
for b, a in zip(preview["before"], preview["after"]):
|
||||
print(f"\n #{b['id']} {b['scene']}")
|
||||
print(f" 适配前: {b['description'][:50]}...")
|
||||
print(f" 适配后: {a['description'][:50]}...")
|
||||
print(f" 角色块: {a['character_block'][:80]}...")
|
||||
else:
|
||||
adapted = adapter.adapt(script)
|
||||
if args.output:
|
||||
import yaml
|
||||
with open(args.output, "w", encoding="utf-8") as f:
|
||||
yaml.dump({"script": adapted}, f, allow_unicode=True,
|
||||
default_flow_style=False, sort_keys=False)
|
||||
print(f"✅ 已保存适配后脚本: {args.output}")
|
||||
else:
|
||||
print(json.dumps(adapted, indent=2, ensure_ascii=False))
|
||||
|
||||
|
||||
elif args.action == "refs":
|
||||
refs = adapter.build_kling_character_refs(args.character)
|
||||
print(f"Kling character_refs for {args.character or registry.get_active_name()}:")
|
||||
print(json.dumps(refs, indent=2, ensure_ascii=False))
|
||||
|
||||
elif args.action == "visual-prompts":
|
||||
if not args.script:
|
||||
print("❌ 请指定 --script")
|
||||
return
|
||||
script = load_script(args.script)
|
||||
storyboard = adapter.adapt_to_storyboard(script)
|
||||
prompts = adapter.build_visual_prompts(storyboard)
|
||||
print(f"Visual prompts (Gate 1.5):")
|
||||
print(json.dumps(prompts, indent=2, ensure_ascii=False))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import json
|
||||
cli()
|
||||
@@ -0,0 +1,11 @@
|
||||
"""
|
||||
角色生成器包 — Character Generator Package
|
||||
|
||||
提供从粗方向到完整角色变体素材的全流程：
|
||||
- DirectionExpander 方向扩展（粗方向→详细描述）
|
||||
- AttributeExtractor 属性提取（自然语言→结构化属性）
|
||||
- PromptAssembler 提示词组装（结构化属性→Kling/即梦提示词）
|
||||
- VariantGenerator 变体生成（Kling API 批量生成多角度/表情）
|
||||
- AssetRegistrar 资产注册（注册到 registry.yaml + 知识库）
|
||||
- CharacterGenerationPipeline 全流程编排器
|
||||
"""
|
||||
@@ -0,0 +1,304 @@
|
||||
"""
|
||||
素材注册器 (AssetRegistrar) — 将生成的角色变体素材注册到 registry.yaml + 知识库
|
||||
|
||||
注册内容:
|
||||
- 更新 registry.yaml 中的 reference_images (face_anchor, body各角度, expressions)
|
||||
- 更新 outfit_presets（如果有自定义服装变体）
|
||||
- 生成标签元数据供 asset_manager 索引
|
||||
- 回写到 prompts/character_sheet_prompts.md（知识库）
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import yaml
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
|
||||
path = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, path)
|
||||
|
||||
from lib.logger import get_logger
|
||||
|
||||
logger = get_logger('asset_registrar')
|
||||
|
||||
# ── 路径 ──
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent
|
||||
REGISTRY_PATH = SCRIPTS_DIR.parent / 'character_registry' / 'registry.yaml'
|
||||
REGISTRY_BACKUP_DIR = SCRIPTS_DIR.parent / 'character_registry' / 'backups'
|
||||
PROMPTS_DIR = SCRIPTS_DIR / 'prompts'
|
||||
CHARACTER_SHEET_PROMPTS = PROMPTS_DIR / 'character_sheet_prompts.md'
|
||||
|
||||
# ── 变体标签映射 ──
|
||||
VARIANT_TAGS = {
|
||||
'front_face': ('面部特写', '基准', '正面'),
|
||||
'multi_angle': {
|
||||
'default': ('全身', '正面', '体型'),
|
||||
'front': ('全身', '正面', '体型'),
|
||||
'side': ('全身', '侧面', '体型'),
|
||||
'right_three_quarter': ('全身', '右侧45°', '体型'),
|
||||
'right_side': ('全身', '纯右侧', '体型'),
|
||||
'back': ('全身', '背面', '体型'),
|
||||
},
|
||||
'expression': {
|
||||
'default': ('表情', '中性'),
|
||||
'neutral': ('表情', '中性'),
|
||||
'smile': ('表情', '微笑'),
|
||||
'focused': ('表情', '专注'),
|
||||
'laugh': ('表情', '大笑'),
|
||||
'serious': ('表情', '严肃'),
|
||||
},
|
||||
'scene': {
|
||||
'morning_run': ('场景', '晨跑', '运动'),
|
||||
'gym_workout': ('场景', '健身房', '力量训练'),
|
||||
'street_basketball': ('场景', '篮球', '街头'),
|
||||
'office_work': ('场景', '办公室', '工作'),
|
||||
'outdoor_fitness': ('场景', '户外', '健身'),
|
||||
},
|
||||
'sport_action': {
|
||||
'running': ('运动', '奔跑', '动态'),
|
||||
'jumping': ('运动', '跳跃', '动态'),
|
||||
'defensive': ('运动', '防守', '姿态'),
|
||||
'stretching': ('运动', '拉伸', '热身'),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class AssetRegistrar:
|
||||
"""将生成的角色素材注册到 registry.yaml + 知识库。"""
|
||||
|
||||
def __init__(self):
|
||||
self._registry_data = self._load_registry()
|
||||
self._registry_path = REGISTRY_PATH
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# 底层 IO
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def _load_registry(self) -> dict:
|
||||
"""从 registry.yaml 加载数据。"""
|
||||
if not REGISTRY_PATH.exists():
|
||||
return {'active_character': None, 'characters': {}}
|
||||
with open(REGISTRY_PATH, 'r', encoding='utf-8') as f:
|
||||
return yaml.safe_load(f) or {'active_character': None, 'characters': {}}
|
||||
|
||||
def _save_registry(self):
|
||||
"""保存 registry.yaml（先备份再写入）
|
||||
|
||||
⚠️ registry.yaml 已**废弃**（账号/角色归属以 ORACLE.yaml 为准，见 CONSTITUTION 规则 3），
|
||||
默认**不再写入**以免污染仓库；如需临时启用，设置环境变量 `AVE_WRITE_REGISTRY=1`。
|
||||
"""
|
||||
import os
|
||||
if os.environ.get("AVE_WRITE_REGISTRY") != "1":
|
||||
print(" ⚠️ registry.yaml 已废弃，跳过写入（如需启用设 AVE_WRITE_REGISTRY=1）")
|
||||
return
|
||||
REGISTRY_BACKUP_DIR.mkdir(parents=True, exist_ok=True)
|
||||
if REGISTRY_PATH.exists():
|
||||
timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
|
||||
backup_path = REGISTRY_BACKUP_DIR / f'registry_{timestamp}.yaml'
|
||||
with open(backup_path, 'w', encoding='utf-8') as f:
|
||||
f.write(REGISTRY_PATH.read_text(encoding='utf-8'))
|
||||
|
||||
with open(REGISTRY_PATH, 'w', encoding='utf-8') as f:
|
||||
yaml.dump(
|
||||
self._registry_data,
|
||||
f,
|
||||
allow_unicode=True,
|
||||
default_flow_style=False,
|
||||
sort_keys=False,
|
||||
)
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# 注册角色属性
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def register_character_properties(self, name: str, attrs: dict) -> bool:
|
||||
"""注册/更新角色的属性信息到 registry.yaml。
|
||||
|
||||
Args:
|
||||
name: 角色名
|
||||
attrs: 结构化属性字典
|
||||
|
||||
Returns:
|
||||
是否成功
|
||||
"""
|
||||
chars = self._registry_data['characters']
|
||||
if name not in chars:
|
||||
chars[name] = {}
|
||||
char = chars[name]
|
||||
|
||||
# 写入永久属性
|
||||
for key in ('name', 'description', 'base', 'personality', 'voice_style', 'art_style'):
|
||||
if key in attrs and attrs[key]:
|
||||
char[key] = attrs[key]
|
||||
|
||||
# 写入分层特征
|
||||
for category in ('appearance', 'face', 'body', 'clothing', 'accessories'):
|
||||
if category in attrs and attrs[category]:
|
||||
char[category] = attrs[category]
|
||||
|
||||
self._save_registry()
|
||||
logger.info(f' 已注册角色属性: {name}')
|
||||
return True
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# 注册参考图
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def register_reference_images(
|
||||
self,
|
||||
name: str,
|
||||
ref_type: str,
|
||||
filepath: str,
|
||||
option: str = '',
|
||||
) -> bool:
|
||||
"""注册单张参考图到 registry。
|
||||
|
||||
Args:
|
||||
name: 角色名
|
||||
ref_type: 类型 (face_anchor/baseline/body/expression/real_photo)
|
||||
filepath: 图片路径
|
||||
option: 选项（如 body 的 front/side, expression 的 smile/neutral）
|
||||
|
||||
Returns:
|
||||
是否成功
|
||||
"""
|
||||
chars = self._registry_data['characters']
|
||||
if name not in chars:
|
||||
chars[name] = {}
|
||||
char = chars[name]
|
||||
|
||||
if 'reference_images' not in char:
|
||||
char['reference_images'] = {}
|
||||
|
||||
refs = char['reference_images']
|
||||
if ref_type not in refs:
|
||||
refs[ref_type] = []
|
||||
|
||||
entry = {'path': filepath}
|
||||
if option:
|
||||
entry['option'] = option
|
||||
|
||||
refs[ref_type].append(entry)
|
||||
self._save_registry()
|
||||
return True
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# 批量注册定妆照
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def register_portrait_set(
|
||||
self,
|
||||
name: str,
|
||||
attrs: dict,
|
||||
baseline: Optional[str] = None,
|
||||
angles: Optional[dict] = None,
|
||||
expressions: Optional[dict] = None,
|
||||
) -> bool:
|
||||
"""批量注册一套完整的定妆照。
|
||||
|
||||
Args:
|
||||
name: 角色名
|
||||
attrs: 结构化属性
|
||||
baseline: 基准照路径
|
||||
angles: {angle_name: filepath, ...}
|
||||
expressions: {expr_name: filepath, ...}
|
||||
|
||||
Returns:
|
||||
是否成功
|
||||
"""
|
||||
if baseline:
|
||||
self.register_reference_images(name, 'face_anchor', baseline)
|
||||
|
||||
if angles:
|
||||
for angle, path in angles.items():
|
||||
self.register_reference_images(name, 'body', path, option=angle)
|
||||
|
||||
if expressions:
|
||||
for expr, path in expressions.items():
|
||||
self.register_reference_images(name, 'expression', path, option=expr)
|
||||
|
||||
return True
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# 更新知识库
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def update_knowledge_base(self, name: str, attrs: dict) -> bool:
|
||||
"""将角色描述更新到 character_sheet_prompts.md。
|
||||
|
||||
Args:
|
||||
name: 角色名
|
||||
attrs: 结构化属性
|
||||
|
||||
Returns:
|
||||
是否成功
|
||||
"""
|
||||
base = attrs.get('base', '')
|
||||
face = attrs.get('face', {})
|
||||
body = attrs.get('body', {})
|
||||
|
||||
# 构建角色描述块
|
||||
char_block_parts = [f'### {name}']
|
||||
char_block_parts.append(f'- **描述**: {attrs.get("description", "")}')
|
||||
char_block_parts.append(
|
||||
f'- **面部**: 脸型={face.get("shape", "?")}, '
|
||||
f'眼睛={face.get("eyes", "?")}, '
|
||||
f'发型={face.get("hair", "?")}, '
|
||||
f'肤色={face.get("skin", "?")}'
|
||||
)
|
||||
char_block_parts.append(
|
||||
f'- **体型**: 体态={body.get("build", "?")}, '
|
||||
f'身高={body.get("height", "?")}'
|
||||
)
|
||||
char_block_parts.append(
|
||||
f'- **性格**: {attrs.get("personality", "?")}'
|
||||
)
|
||||
char_block_parts.append(
|
||||
f'- **语音**: {attrs.get("voice_style", "?")}'
|
||||
)
|
||||
|
||||
char_block = '\n'.join(char_block_parts)
|
||||
new_section = f'\n\n{char_block} # 自动注册）\n\n'
|
||||
|
||||
# 写入 prompts/character_sheet_prompts.md
|
||||
PROMPTS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
if CHARACTER_SHEET_PROMPTS.exists():
|
||||
existing = CHARACTER_SHEET_PROMPTS.read_text(encoding='utf-8')
|
||||
if f'### {name}' not in existing:
|
||||
with open(CHARACTER_SHEET_PROMPTS, 'a', encoding='utf-8') as f:
|
||||
f.write(new_section)
|
||||
else:
|
||||
with open(CHARACTER_SHEET_PROMPTS, 'w', encoding='utf-8') as f:
|
||||
f.write(f'# 角色定妆照提示词库\n{new_section}')
|
||||
|
||||
logger.info(f' 已更新知识库: {name}')
|
||||
return True
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# 标签工具
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def get_variant_tags(self, variant_type: str, option: str = '') -> tuple:
|
||||
"""获取变体类型的标签元数据。
|
||||
|
||||
Args:
|
||||
variant_type: 变体类型
|
||||
option: 选项名
|
||||
|
||||
Returns:
|
||||
标签元组
|
||||
"""
|
||||
tags = VARIANT_TAGS.get(variant_type, {})
|
||||
if isinstance(tags, tuple):
|
||||
return tags
|
||||
if isinstance(tags, dict):
|
||||
return tags.get(option, tags.get('default', ()))
|
||||
return ()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
registrar = AssetRegistrar()
|
||||
print(f'素材注册器就绪, registry: {REGISTRY_PATH}')
|
||||
print(f'知识库: {CHARACTER_SHEET_PROMPTS}')
|
||||
@@ -0,0 +1,276 @@
|
||||
"""
|
||||
属性提取器 (AttributeExtractor) — 从自然语言角色描述中提取结构化属性
|
||||
|
||||
输出结构:
|
||||
{
|
||||
"name": "角色名",
|
||||
"description": "完整自然语言描述",
|
||||
"base": "核心特征（不含服装的永久特征）",
|
||||
"appearance": ["特征1", "特征2", ...], # 外观特征列表
|
||||
"face": {"shape": "国字脸", "eyes": "单眼皮", ...},
|
||||
"body": {"build": "健壮", "height": "175cm", ...},
|
||||
"clothing": {"top": "深灰夹克", "inner": "白色内衬", ...},
|
||||
"accessories": ["蓝牙耳机"],
|
||||
"personality": "沉稳内敛",
|
||||
"voice_style": "成熟稳重男声",
|
||||
"art_style": "写实摄影风格",
|
||||
}
|
||||
"""
|
||||
|
||||
import json
|
||||
import re
|
||||
from typing import Optional
|
||||
|
||||
# ── 语音风格枚举 ──
|
||||
VOICE_STYLES = (
|
||||
'活泼少女音',
|
||||
'温柔青年音',
|
||||
'成熟稳重男声',
|
||||
'清澈少年音',
|
||||
'低沉磁性',
|
||||
'甜美少女音',
|
||||
'知性女声',
|
||||
'治愈系',
|
||||
'默认',
|
||||
)
|
||||
|
||||
# ── 画风枚举 ──
|
||||
ART_STYLES = (
|
||||
'写实摄影风格',
|
||||
'日系漫画风格',
|
||||
'电影级摄影风格',
|
||||
'超写实风格',
|
||||
'赛博朋克风格',
|
||||
'水墨风格',
|
||||
)
|
||||
|
||||
|
||||
class AttributeExtractor:
|
||||
"""从自然语言描述中提取结构化角色属性。"""
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# 主入口
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def extract(self, description: str, name: str = '') -> dict:
|
||||
"""从自然语言描述中提取结构化属性。
|
||||
|
||||
Args:
|
||||
description: 自然语言角色描述
|
||||
name: 角色名称
|
||||
|
||||
Returns:
|
||||
结构化属性字典
|
||||
"""
|
||||
if not description:
|
||||
return self._default_attributes(name)
|
||||
|
||||
attrs = self._default_attributes(name)
|
||||
attrs['description'] = description
|
||||
|
||||
attrs['base'] = self._extract_base(description)
|
||||
attrs['appearance'] = self._extract_appearance_list(description)
|
||||
attrs['face'] = self._extract_face(description)
|
||||
attrs['body'] = self._extract_body(description)
|
||||
attrs['clothing'] = self._extract_clothing(description)
|
||||
attrs['accessories'] = self._extract_accessories(description)
|
||||
attrs['personality'] = self._extract_personality(description)
|
||||
attrs['voice_style'] = self._extract_voice(description)
|
||||
attrs['art_style'] = self._extract_art_style(description)
|
||||
|
||||
return attrs
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# 默认值
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def _default_attributes(self, name: str = '') -> dict:
|
||||
"""返回默认空属性模板。"""
|
||||
return {
|
||||
'name': name or '未命名',
|
||||
'description': '',
|
||||
'base': '',
|
||||
'appearance': [],
|
||||
'face': {k: '' for k in ('shape', 'eyes', 'nose', 'lips', 'hair', 'skin', 'facial_hair', 'glasses', 'distinctive')},
|
||||
'body': {k: '' for k in ('build', 'height', 'posture', 'age_group')},
|
||||
'clothing': {k: '' for k in ('top', 'inner', 'bottom', 'shoes', 'style')},
|
||||
'accessories': [],
|
||||
'personality': '默认',
|
||||
'voice_style': '写实摄影风格',
|
||||
}
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# 各类别提取
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def _extract_base(self, desc: str) -> str:
|
||||
"""提取核心特征（去服装、去场景的永久性特征）。"""
|
||||
base = re.sub(r'身穿[^，。]*', '', desc)
|
||||
base = re.sub(r'穿着[^，。]*', '', base)
|
||||
base = re.sub(r'（[^）]*）', '', base)
|
||||
base = base.replace('，', '').replace('。', '').replace(' ', '')
|
||||
return base.strip()
|
||||
|
||||
def _extract_appearance_list(self, desc: str) -> list:
|
||||
"""提取外观特征列表。
|
||||
|
||||
按分类匹配模式，返回 ["脸型:国字脸", "眼部:单眼皮", ...]。
|
||||
"""
|
||||
patterns = [
|
||||
('(国字脸|圆脸|方脸|瓜子脸|长脸)', '脸型'),
|
||||
('(单眼皮|双眼皮|内双|大眼睛|小眼睛|细长眼)', '眼部'),
|
||||
('(黑色短发|黑短发|棕色短发|寸头|齐刘海|中长发|三七分|背头|卷发)', '发型'),
|
||||
('(健壮|偏瘦|壮硕|匀称|标准|苗条|丰满|粗犷)', '体型'),
|
||||
('(深灰[色]?[^，。]*夹克|浅灰[色]?[^，。]*卫衣|运动[^，。]*|连帽[^，。]*)', '服装'),
|
||||
]
|
||||
features = []
|
||||
for pattern, category in patterns:
|
||||
m = re.search(pattern, desc)
|
||||
if m:
|
||||
matched = m.group(1) if m.lastindex and m.lastindex >= 1 else m.group(0)
|
||||
feature = f'{category}:{matched}'
|
||||
if len(feature) <= 8:
|
||||
features.append(feature)
|
||||
return features
|
||||
|
||||
def _extract_face(self, desc: str) -> dict:
|
||||
"""提取面部详细特征。"""
|
||||
face = {k: '' for k in ('shape', 'eyes', 'nose', 'lips', 'hair', 'skin', 'facial_hair', 'glasses', 'distinctive')}
|
||||
|
||||
m = re.search(r'(国字脸|圆脸|方脸|瓜子脸|长脸|菱形脸)', desc)
|
||||
if m:
|
||||
face['shape'] = m.group(1)
|
||||
|
||||
m = re.search(r'(单眼皮|双眼皮|内双|大眼睛|小眼睛|细长眼|眼尾[^，。]*)', desc)
|
||||
if m:
|
||||
face['eyes'] = m.group(1)
|
||||
|
||||
m = re.search(r'(鼻梁[^，。]*?|鼻翼[^，。]*?|[^，。]*鼻[^，。]*)', desc)
|
||||
if m:
|
||||
face['nose'] = m.group(1)
|
||||
|
||||
m = re.search(r'(嘴唇[^，。]*?|嘴角[^，。]*?)', desc)
|
||||
if m:
|
||||
face['lips'] = m.group(1)
|
||||
|
||||
m = re.search(r'(黑[色]?[^，。]*发|棕[色]?[^，。]*发|寸头|平头|光头|卷发|直发|[^，。]*短发[^，。]*)', desc)
|
||||
if m:
|
||||
face['hair'] = m.group(1)
|
||||
|
||||
m = re.search(r'(小麦色|白皙|健康[^，。]*|古铜色|偏[^，。]*[肤皮])', desc)
|
||||
if m:
|
||||
face['skin'] = m.group(1)
|
||||
|
||||
m = re.search(r'(胡须|胡子|络腮胡|山羊胡|胡茬)', desc)
|
||||
if m:
|
||||
face['facial_hair'] = m.group(1)
|
||||
|
||||
m = re.search(r'([^，。]*眼镜[^，。]*)', desc)
|
||||
if m:
|
||||
face['glasses'] = m.group(1)
|
||||
|
||||
m = re.search(r'(耳[机塞][^，。]*|耳机|特殊特征[^，。]*)', desc)
|
||||
if m:
|
||||
face['distinctive'] = m.group(1)
|
||||
|
||||
return face
|
||||
|
||||
def _extract_body(self, desc: str) -> dict:
|
||||
"""提取体型特征。"""
|
||||
body = {k: '' for k in ('build', 'height', 'posture', 'age_group')}
|
||||
|
||||
m = re.search(r'(健壮|偏瘦|壮硕|匀称|标准|苗条|丰满|粗犷|肌肉[^，。]*)', desc)
|
||||
if m:
|
||||
body['build'] = m.group(1)
|
||||
|
||||
m = re.search(r'(\d+cm|\d+m\d+)', desc)
|
||||
if m:
|
||||
body['height'] = m.group(1)
|
||||
|
||||
m = re.search(r'(\d+岁)', desc)
|
||||
if m:
|
||||
body['age_group'] = m.group(1)
|
||||
|
||||
m = re.search(r'(挺拔|微驼|笔直|自然)', desc)
|
||||
if m:
|
||||
body['posture'] = m.group(1)
|
||||
|
||||
return body
|
||||
|
||||
def _extract_clothing(self, desc: str) -> dict:
|
||||
"""提取服装特征。"""
|
||||
clothing = {k: '' for k in ('top', 'inner', 'bottom', 'shoes', 'style')}
|
||||
|
||||
m = re.search(r'([^，。]*?[夹克卫衣衬衫T恤外套][^，。]*)', desc)
|
||||
if m:
|
||||
clothing['top'] = m.group(1)
|
||||
|
||||
m = re.search(r'([^，。]*?[内衬打底背心][^，。]*)', desc)
|
||||
if m:
|
||||
clothing['inner'] = m.group(1)
|
||||
|
||||
m = re.search(r'([^，。]*?[裤子短裤牛仔裤][^，。]*)', desc)
|
||||
if m:
|
||||
clothing['bottom'] = m.group(1)
|
||||
|
||||
m = re.search(r'(运动[^，。]*|商务[^，。]*|休闲[^，。]*|正式[^，。]*)', desc)
|
||||
if m:
|
||||
clothing['style'] = m.group(1)
|
||||
|
||||
return clothing
|
||||
|
||||
def _extract_accessories(self, desc: str) -> list:
|
||||
"""提取配饰列表。"""
|
||||
accessories = []
|
||||
for m in re.finditer(r'(耳机|眼镜|手表|手环|项链|帽子|围巾|腰带|护腕|背包)', desc):
|
||||
accessories.append(m.group(1))
|
||||
return accessories
|
||||
|
||||
def _extract_personality(self, desc: str) -> str:
|
||||
"""提取性格特征。"""
|
||||
m = re.search(r'(性格[^，。]*|气质[^，。]*|性[^，。]*？)', desc)
|
||||
if m:
|
||||
text = m.group(1)
|
||||
keywords = ('沉稳', '开朗', '温和', '内敛', '专注', '活泼', '坚韧', '倔强')
|
||||
found = [k for k in keywords if k in text]
|
||||
if found:
|
||||
return '、'.join(found)
|
||||
return '默认'
|
||||
|
||||
def _extract_voice(self, desc: str) -> str:
|
||||
"""提取语音风格。"""
|
||||
m = re.search(r'([^，。]*[音声][^，。]*)', desc)
|
||||
if m:
|
||||
voice_text = m.group(1)
|
||||
# 关键词匹配
|
||||
if '音' in voice_text or '声' in voice_text:
|
||||
if '中年' in voice_text or '沉稳' in voice_text:
|
||||
return '成熟稳重男声'
|
||||
elif '少女' in voice_text or '活泼' in voice_text:
|
||||
return '活泼少女音'
|
||||
elif '青年' in voice_text or '温和' in voice_text:
|
||||
return '温柔青年音'
|
||||
return '默认'
|
||||
|
||||
def _extract_art_style(self, desc: str) -> str:
|
||||
"""提取画风。"""
|
||||
m = re.search(r'([^，。]*?[风格][^，。]*)', desc)
|
||||
if m:
|
||||
style_text = m.group(1)
|
||||
if '风' in style_text or '格' in style_text:
|
||||
return '写实摄影风格'
|
||||
return '写实摄影风格'
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
extractor = AttributeExtractor()
|
||||
|
||||
test_desc = (
|
||||
'一位约42岁的中国男性，体型健壮肩宽，黑色短发利落，'
|
||||
'国字脸单眼皮眼尾微垂，鼻梁挺直，神态沉稳内敛，'
|
||||
'肤色健康自然。身穿深灰色拉链运动夹克配白色速干内衬，'
|
||||
'右耳戴着蓝色无线运动耳机。'
|
||||
)
|
||||
|
||||
result = extractor.extract(test_desc, 'ghai_sports')
|
||||
print(json.dumps(result, indent=2, ensure_ascii=False))
|
||||
@@ -0,0 +1,132 @@
|
||||
"""
|
||||
方向扩展器 (DirectionExpander) — 将用户给的粗方向补充为完整自然语言描述
|
||||
|
||||
流程:
|
||||
用户输入 → 调用本地 oMLX LLM (Qwen3.5) → 返回完整角色自然语言描述
|
||||
|
||||
扩展内容:
|
||||
- 面部特征（五官、发型、脸型、肤色）
|
||||
- 体型（身高、肩宽、体态）
|
||||
- 服装（上装、下装、鞋子）
|
||||
- 标志性配件/装饰
|
||||
- 性格/气质
|
||||
- 声音/语态
|
||||
- 画风/风格倾向
|
||||
|
||||
如果用户描述已经足够详细，则直接使用，不做过度补充。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
path = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, path)
|
||||
|
||||
from lib.config import load_config
|
||||
from lib.logger import get_logger
|
||||
|
||||
logger = get_logger('direction_expander')
|
||||
|
||||
environ = os.environ.get
|
||||
OMLX_BASE = environ('OMLX_BASE', 'http://localhost:8000/v1')
|
||||
OMLX_MODEL = environ('OMLX_MODEL', 'Qwen3.5-4B-MLX-4bit')
|
||||
|
||||
SYSTEM_PROMPT = (
|
||||
"你是一个专业的角色设计师助理。用户会给你一个角色的\"粗方向\"（一两句话的粗略描述），\n"
|
||||
"你需要将它扩展为一段丰富、详细、可执行的自然语言角色描述。\n\n"
|
||||
"扩展规则：\n"
|
||||
"1. 保留用户原始描述中的所有信息，不得修改或扭曲\n"
|
||||
"2. 补充合理细节：面部特征（五官形态、脸型、肤质）、发型发色、体型、身高体态、服装风格、标志性配件\n"
|
||||
"3. 补充性格气质和声音特征\n"
|
||||
"4. 如果用户给出了具体的细节关键词（如\"国字脸\"\"单眼皮\"\"蓝牙耳机\"），保留并丰富它们\n"
|
||||
"5. 如果用户描述已经足够详细（超过80字），则基本照原样输出，仅微调润色\n"
|
||||
"6. 描述要写实，不要说\"动漫风格\"——写实角色以真人摄影为参照\n"
|
||||
"7. 输出**只有**角色描述本身，不要带任何前缀、标签或解释\n\n"
|
||||
"示例输入：\n"
|
||||
"\"一个运动型中年男性，喜欢户外跑步\"\n\n"
|
||||
"示例输出：\n"
|
||||
"\"一位约42岁的中国男性，体型健壮肩宽，黑色短发利落，国字脸单眼皮眼尾微垂，"
|
||||
"鼻梁挺直，神态沉稳内敛，肤色健康自然偏小麦色。"
|
||||
"身穿深灰色拉链运动夹克配白色速干内衬。"
|
||||
"整体气质坚毅沉稳，给人可靠踏实的感觉。声音偏向成熟稳重的中低音。\""
|
||||
)
|
||||
|
||||
|
||||
class DirectionExpander:
|
||||
"""将粗方向扩展为详细自然语言描述"""
|
||||
|
||||
def __init__(self):
|
||||
self._base_url = OMLX_BASE
|
||||
self._model = OMLX_MODEL
|
||||
|
||||
def expand(self, direction: str) -> str:
|
||||
"""将粗方向扩展为详细自然语言描述。
|
||||
|
||||
Args:
|
||||
direction: 用户输入的粗方向，如"运动型中年男性"
|
||||
|
||||
Returns:
|
||||
扩展后的完整角色描述字符串
|
||||
|
||||
扩展策略:
|
||||
- 空输入 → 返回空字符串
|
||||
- 超过80字 → 已足够详细，直接使用（log info）
|
||||
- 80字以内 → 调用本地 LLM 扩展（异常时回退到原始方向）
|
||||
"""
|
||||
if not direction or not direction.strip():
|
||||
return ''
|
||||
if len(direction) > 80:
|
||||
logger.info('方向描述已足够详细（>80字），直接使用')
|
||||
return direction.strip()
|
||||
try:
|
||||
return self._call_llm(direction)
|
||||
except Exception as e:
|
||||
logger.warning(f'LLM 扩展失败 ({e})，回退到直接使用原始方向')
|
||||
return direction.strip()
|
||||
|
||||
def _call_llm(self, direction: str) -> str:
|
||||
"""调用本地 oMLX LLM 扩展方向描述。
|
||||
|
||||
Args:
|
||||
direction: 原始方向描述
|
||||
|
||||
Returns:
|
||||
LLM 返回的角色详细描述，去除首尾空白
|
||||
|
||||
Raises:
|
||||
网络/API/解析异常 — 由调用者处理
|
||||
"""
|
||||
import httpx
|
||||
|
||||
payload = {
|
||||
'model': self._model,
|
||||
'messages': [
|
||||
{'role': 'system', 'content': SYSTEM_PROMPT},
|
||||
{'role': 'user', 'content': direction},
|
||||
],
|
||||
'temperature': 0.7,
|
||||
'max_tokens': 500,
|
||||
}
|
||||
|
||||
resp = httpx.post(
|
||||
f'{self._base_url}/chat/completions',
|
||||
json=payload,
|
||||
timeout=30,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
content = data.get('choices', [{}])[0].get('message', {}).get('content', '')
|
||||
return content.strip()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
expander = DirectionExpander()
|
||||
test_inputs = [
|
||||
'一个运动型中年男性',
|
||||
'一位28岁的城市程序员，黑短发，戴细框金属眼镜',
|
||||
]
|
||||
for inp in test_inputs:
|
||||
print(f'\n输入: {inp}')
|
||||
print(f'输出: {expander.expand(inp)}')
|
||||
@@ -0,0 +1,417 @@
|
||||
"""
|
||||
角色生成完整流水线 (CharacterGenerationPipeline)
|
||||
|
||||
完整流程:
|
||||
|
||||
用户方向 (粗方向)
|
||||
↓
|
||||
[DirectionExpander] — 调用本地 LLM 补充为完整自然语言描述
|
||||
↓
|
||||
详细角色描述
|
||||
↓
|
||||
[AttributeExtractor] — 提取结构化的面部/体型/服装/配件/性格
|
||||
↓
|
||||
结构化属性
|
||||
↓
|
||||
[PromptAssembler] — 按不同类型组装生成提示词
|
||||
↓
|
||||
生成提示词
|
||||
↓
|
||||
[VariantGenerator] — 调用 Kling API 生成各类变体
|
||||
↓
|
||||
变体图像
|
||||
↓
|
||||
[AssetRegistrar] — 写入 registry.yaml + 知识库
|
||||
↓
|
||||
完成
|
||||
|
||||
用法:
|
||||
from character_generator.pipeline import CharacterGenerationPipeline
|
||||
pipeline = CharacterGenerationPipeline()
|
||||
|
||||
# 完整流程：从粗方向开始
|
||||
result = pipeline.run_full("一个运动型中年男性", character_name="new_char")
|
||||
|
||||
# 仅扩展描述（不生成图像）
|
||||
desc = pipeline.expand_direction("一个运动型中年男性")
|
||||
|
||||
# 从已有描述开始（生成+注册）
|
||||
result = pipeline.run_from_description("详细描述...", character_name="new_char")
|
||||
|
||||
# 仅生成定妆照网格
|
||||
result = pipeline.generate_portrait("角色名", force=True)
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
path = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, path)
|
||||
|
||||
from character_generator.direction_expander import DirectionExpander
|
||||
from character_generator.attribute_extractor import AttributeExtractor
|
||||
from character_generator.prompt_assembler import PromptAssembler
|
||||
from character_generator.variant_generator import VariantGenerator
|
||||
from character_generator.asset_registrar import AssetRegistrar
|
||||
from lib.logger import get_logger
|
||||
|
||||
logger = get_logger('character_pipeline')
|
||||
|
||||
|
||||
class CharacterGenerationPipeline:
|
||||
"""角色生成完整流水线
|
||||
|
||||
整合五个子模块，提供从粗方向到角色变体素材的一站式接口。
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self._expander = DirectionExpander()
|
||||
self._extractor = AttributeExtractor()
|
||||
self._assembler = PromptAssembler()
|
||||
self._generator = VariantGenerator()
|
||||
self._registrar = AssetRegistrar()
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# 主流程
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
def run_full(
|
||||
self,
|
||||
direction: str,
|
||||
character_name: str = '',
|
||||
lang: str = 'zh',
|
||||
generate_variants: bool = True,
|
||||
seed: int = 0,
|
||||
) -> dict:
|
||||
"""完整流水线：从粗方向到注册完成。
|
||||
|
||||
Args:
|
||||
direction: 用户输入的粗方向，如"一个运动型中年男性"
|
||||
character_name: 角色名（为空则自动生成）
|
||||
lang: 语言
|
||||
generate_variants: 是否生成变体图像
|
||||
seed: 固定种子（0=随机）
|
||||
|
||||
Returns:
|
||||
{
|
||||
"name": str,
|
||||
"description": str,
|
||||
"attributes": dict,
|
||||
"baseline": dict or None,
|
||||
"angles": dict or None,
|
||||
"expressions": dict or None,
|
||||
"grid": dict or None,
|
||||
"registered": True,
|
||||
}
|
||||
"""
|
||||
step_results = {}
|
||||
|
||||
# [1/5] 扩展方向
|
||||
logger.info(f'[1/5] 扩展方向: {direction[:50]}...')
|
||||
description = self._expander.expand(direction)
|
||||
step_results['description'] = description
|
||||
|
||||
# 自动生成角色名
|
||||
if not character_name:
|
||||
character_name = self._generate_name(direction, description)
|
||||
|
||||
# [2/5] 提取属性
|
||||
logger.info(f'[2/5] 提取属性: {character_name}')
|
||||
attrs = self._extractor.extract(description, name=character_name)
|
||||
step_results['attributes'] = attrs
|
||||
|
||||
# [3/5] 注册角色属性
|
||||
logger.info('[3/5] 注册角色属性')
|
||||
self._registrar.register_character_properties(character_name, attrs)
|
||||
|
||||
# [4/5] 生成变体
|
||||
if generate_variants:
|
||||
generated = self._generate_all(attrs, lang, seed)
|
||||
step_results['generated'] = generated
|
||||
else:
|
||||
step_results['generated'] = {}
|
||||
|
||||
# 注册定妆照到 registry
|
||||
generated = step_results.get('generated', {})
|
||||
baseline = generated.get('baseline', {}).get('path', '')
|
||||
angles = {
|
||||
k: v['path']
|
||||
for k, v in generated.get('angles', {}).items()
|
||||
if v.get('path')
|
||||
}
|
||||
expressions = {
|
||||
k: v['path']
|
||||
for k, v in generated.get('expressions', {}).items()
|
||||
if v.get('path')
|
||||
}
|
||||
|
||||
if baseline or angles or expressions:
|
||||
self._registrar.register_portrait_set(
|
||||
character_name,
|
||||
attrs,
|
||||
baseline=baseline or None,
|
||||
angles=angles or None,
|
||||
expressions=expressions or None,
|
||||
)
|
||||
|
||||
# [5/5] 更新知识库
|
||||
logger.info('[4/5] 注册素材到知识库')
|
||||
self._registrar.update_knowledge_base(character_name, attrs)
|
||||
|
||||
return {
|
||||
'name': character_name,
|
||||
'description': description,
|
||||
'attributes': attrs,
|
||||
'baseline': generated.get('baseline', {}),
|
||||
'angles': generated.get('angles', {}),
|
||||
'expressions': generated.get('expressions', {}),
|
||||
'grid': generated.get('grid', {}),
|
||||
'registered': True,
|
||||
}
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# 从已有描述开始（跳过方向扩展）
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
def run_from_description(
|
||||
self,
|
||||
description: str,
|
||||
character_name: str = '',
|
||||
lang: str = 'zh',
|
||||
seed: int = 0,
|
||||
) -> dict:
|
||||
"""从完整描述直接开始，绕过方向扩展步骤。
|
||||
|
||||
Args:
|
||||
description: 完整的角色自然语言描述
|
||||
character_name: 角色名（为空则自动生成）
|
||||
lang: 语言
|
||||
seed: 固定种子
|
||||
|
||||
Returns:
|
||||
同 run_full()
|
||||
"""
|
||||
if not character_name:
|
||||
character_name = self._generate_name(description, description)
|
||||
|
||||
attrs = self._extractor.extract(description, name=character_name)
|
||||
self._registrar.register_character_properties(character_name, attrs)
|
||||
|
||||
generated = self._generate_all(attrs, lang, seed)
|
||||
|
||||
# 注册定妆照到 registry
|
||||
baseline = generated.get('baseline', {}).get('path', '')
|
||||
angles = {
|
||||
k: v.get('path', '')
|
||||
for k, v in generated.get('angles', {}).items()
|
||||
if v.get('path')
|
||||
}
|
||||
expressions = {
|
||||
k: v.get('path', '')
|
||||
for k, v in generated.get('expressions', {}).items()
|
||||
if v.get('path')
|
||||
}
|
||||
|
||||
self._registrar.register_portrait_set(
|
||||
character_name,
|
||||
attrs,
|
||||
baseline=baseline or None,
|
||||
angles=angles or None,
|
||||
expressions=expressions or None,
|
||||
)
|
||||
|
||||
self._registrar.update_knowledge_base(character_name, attrs)
|
||||
|
||||
return {
|
||||
'name': character_name,
|
||||
'description': description,
|
||||
'attributes': attrs,
|
||||
'baseline': generated.get('baseline', {}),
|
||||
'angles': generated.get('angles', {}),
|
||||
'expressions': generated.get('expressions', {}),
|
||||
'grid': generated.get('grid', {}),
|
||||
'registered': True,
|
||||
}
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# 仅生成定妆照
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
def generate_portrait(self, character_name: str, force: bool = False, seed: int = 0) -> dict:
|
||||
"""仅基于已注册的角色信息生成定妆照网格。
|
||||
|
||||
需要角色已在 registry 中注册（有 attrs 属性）。
|
||||
|
||||
Args:
|
||||
character_name: 角色名
|
||||
force: 强制重新生成
|
||||
seed: 固定种子
|
||||
|
||||
Returns:
|
||||
变体生成结果 dict
|
||||
"""
|
||||
from character_registry import CharacterRegistry
|
||||
|
||||
registry = CharacterRegistry()
|
||||
char = registry.get_character(character_name)
|
||||
attrs = char.to_dict()
|
||||
|
||||
generated = self._generate_all(attrs, 'zh', seed, force)
|
||||
|
||||
# 注册定妆照
|
||||
baseline = generated.get('baseline', {}).get('path', '')
|
||||
if baseline:
|
||||
self._registrar.register_reference_images(
|
||||
character_name, 'face_anchor', baseline
|
||||
)
|
||||
|
||||
for angle, path in generated.get('angles', {}).items():
|
||||
self._registrar.register_reference_images(
|
||||
character_name, 'body', path, option=angle
|
||||
)
|
||||
|
||||
for expr, path in generated.get('expressions', {}).items():
|
||||
self._registrar.register_reference_images(
|
||||
character_name, 'expression', path, option=expr
|
||||
)
|
||||
|
||||
return generated
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# 单步操作
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
def expand_direction(self, direction: str) -> str:
|
||||
"""仅扩展方向描述（不提取属性、不生成图像）。"""
|
||||
return self._expander.expand(direction)
|
||||
|
||||
def extract_attributes(self, description: str, name: str = '') -> dict:
|
||||
"""仅提取结构化属性（不生成图像、不注册）。"""
|
||||
return self._extractor.extract(description, name=name)
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# 内部：批量生成变体
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
def _generate_all(
|
||||
self,
|
||||
attrs: dict,
|
||||
lang: str = 'zh',
|
||||
seed: int = 0,
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""调用 VariantGenerator 生成基准照 + 多角度 + 表情 + 网格。
|
||||
|
||||
每个步骤独立 try/except，失败时记录 warning 并继续。
|
||||
保证不会因为单步失败而中断整个流程。
|
||||
|
||||
Args:
|
||||
attrs: 结构化属性 dict
|
||||
lang: 语言
|
||||
seed: 固定种子
|
||||
force: 强制重新生成
|
||||
|
||||
Returns:
|
||||
{
|
||||
"baseline": dict or {"error": str},
|
||||
"angles": dict or {"error": str},
|
||||
"expressions": dict or {"error": str},
|
||||
"grid": dict or {"error": str},
|
||||
}
|
||||
"""
|
||||
logger.info(f'[5/5] 生成变体 (seed={seed if seed else "随机"})')
|
||||
result = {}
|
||||
|
||||
# 1/4: 正面基准肖像
|
||||
try:
|
||||
logger.info(' 1/4: 正面基准肖像...')
|
||||
result['baseline'] = self._generator.generate_baseline_portrait(
|
||||
attrs, lang, force, seed
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f' ⚠ 基准照生成失败: {e}')
|
||||
result['baseline'] = {'error': str(e)}
|
||||
|
||||
# 2/4: 多角度全身
|
||||
try:
|
||||
logger.info(' 2/4: 多角度全身...')
|
||||
result['angles'] = self._generator.generate_all_angles(
|
||||
attrs, lang, force, seed
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f' ⚠ 多角度生成失败: {e}')
|
||||
result['angles'] = {'error': str(e)}
|
||||
|
||||
# 3/4: 表情变体
|
||||
try:
|
||||
logger.info(' 3/4: 表情变体...')
|
||||
result['expressions'] = self._generator.generate_all_expressions(
|
||||
attrs, lang, force, seed
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f' ⚠ 表情变体生成失败: {e}')
|
||||
result['expressions'] = {'error': str(e)}
|
||||
|
||||
# 4/4: 网格定妆照
|
||||
try:
|
||||
mode = attrs.get('sheet_mode', 'standard')
|
||||
logger.info(' 4/4: 网格定妆照...')
|
||||
result['grid'] = self._generator.generate_grid_sheet(
|
||||
attrs, mode, lang, force, seed
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f' ⚠ 网格定妆照生成失败: {e}')
|
||||
result['grid'] = {'error': str(e)}
|
||||
|
||||
return result
|
||||
|
||||
# ──────────────────────────────────────────────
|
||||
# 内部：从方向/描述生成角色名
|
||||
# ──────────────────────────────────────────────
|
||||
|
||||
def _generate_name(self, direction: str, description: str) -> str:
|
||||
"""从方向/描述中提取角色名。
|
||||
|
||||
策略：
|
||||
1. 在 direction 中查找 "XX岁的中/老/青年男性/女性" 模式
|
||||
2. 若未命中，查找 "[角色名]男/女" 模式
|
||||
3. 若仍未命中，取 direction 前 3 字
|
||||
4. 追加当前时间戳（%H%M）避免重名
|
||||
"""
|
||||
import re
|
||||
|
||||
m = re.search(r'(\d+岁)?[的]?([\u4e00-\u9fff]{2,4}(?:男性|女性|男生|女生|少年|少女|中年|老人))', direction)
|
||||
if not m:
|
||||
m = re.search(r'([\u4e00-\u9fff]{2,6})[男女人]', direction)
|
||||
|
||||
if m:
|
||||
base = m.group(1)
|
||||
base = base.replace('的', '').replace('男性', '').replace('女性', '')
|
||||
name = base if len(base) <= 3 else base[:-3]
|
||||
else:
|
||||
name = direction[:3]
|
||||
|
||||
from datetime import datetime
|
||||
timestamp = datetime.now().strftime('%H%M')
|
||||
return f'{name}_{timestamp}'
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
pipeline = CharacterGenerationPipeline()
|
||||
|
||||
print('\n=== 测试1: 方向扩展 ===')
|
||||
direction = '一个运动型中年男性，喜欢户外跑步和健身'
|
||||
desc = pipeline.expand_direction(direction)
|
||||
print(f'方向: {direction}')
|
||||
print(f'扩展: {desc}')
|
||||
|
||||
print('\n=== 测试2: 属性提取 ===')
|
||||
attrs = pipeline.extract_attributes(desc, name='test_char')
|
||||
print(f'外观: {attrs.get("appearance", "N/A")}')
|
||||
print(f'面部: {attrs.get("face", "N/A")}')
|
||||
print(f'服装: {attrs.get("clothing", "N/A")}')
|
||||
print(f'配件: {attrs.get("accessories", "N/A")}')
|
||||
print(f'性格: {attrs.get("personality", "N/A")}')
|
||||
@@ -0,0 +1,370 @@
|
||||
"""
|
||||
character_generator/portrait_set.py — 短剧角色参考图**套装**（5 视角 · 半身）
|
||||
|
||||
## 为什么需要
|
||||
短剧角色锁定需要「一套 5 张同人多视角」参考图。两种做法对比：
|
||||
|
||||
| | 一次生成（网格图） | **分开生成（本模块）** |
|
||||
|---|---|---|
|
||||
| 分辨率 | 每格仅整图的 1/6（≈416px，五官糊） | **每张全分辨率** ✓ |
|
||||
| 一致性 | 格间互相干扰（串表情/服装） | **以主定妆为参考图 → 五官锁定** ✓ |
|
||||
| 可控性 | 一格崩了整张重来 | 单张可重生成 ✓ |
|
||||
| 成本 | 1 张 | 5 张（≈¥1） |
|
||||
|
||||
→ **短剧级角色卡必须分开生成**。
|
||||
|
||||
## 核心流程（关键技巧）
|
||||
```
|
||||
① 生成【正面半身主定妆】
|
||||
② 用户确认满意
|
||||
③ 以主定妆图作为**参考图**（I2I），逐张生成其余 4 个角度
|
||||
④ 5 张全部入库为角色参考图
|
||||
```
|
||||
「把主定妆当参考图」是五官不崩的关键 —— 纯文生图跨角度时模型会重新"想象"脸。
|
||||
|
||||
## 5 个视角（源自短剧角色卡通用规范）
|
||||
1. front_half 正面半身 · 主定妆（直视镜头）← 基准
|
||||
2. side_45 45° 侧半身（脸微转向镜头）
|
||||
3. profile 纯侧面轮廓（不看镜头）
|
||||
4. face_closeup 面部特写（低头再抬眼看向镜头）
|
||||
5. shoulder_look 回头斜侧半身（背部 45°，头回转）
|
||||
"""
|
||||
from pathlib import Path
|
||||
|
||||
# ── 视角定义（半身构图；短剧主要拍上半身，比全身更实用）──
|
||||
# ── 画风层（写实 —— 源自 shuohao 实测：不写 photorealistic，会往 CG 带）──
|
||||
LOOK_REALISM = (
|
||||
"A real photograph, not a render. Full-frame camera, 85mm lens at f/5.6, "
|
||||
"true-to-life white balance, unretouched, no beauty filter, slight film grain. "
|
||||
"Soft directional key light with gentle fill; seamless cool-grey backdrop "
|
||||
"with nothing else in frame."
|
||||
)
|
||||
|
||||
# ── 短一致性清单（派生图用；**不要**把锚点的长描述整段重复进来）──
|
||||
def _detect_pronoun(character_desc: str) -> tuple:
|
||||
"""按角色描述自动判断英文人称（he/his · she/her · they/their）
|
||||
|
||||
⚠️ bug 教训（2026-10-04）：构图指令原先把人称写死成 she/her，
|
||||
用户生成「男医生」时 desc 正确、但英文指令里的 she 把模型带成了女性。
|
||||
"""
|
||||
d = str(character_desc or "")
|
||||
male = any(k in d for k in ("男", "先生", "父亲", "爸爸", "哥哥", "弟弟", "叔叔", "大爷",
|
||||
"male", " man", "boy", "gentleman"))
|
||||
female = any(k in d for k in ("女", "小姐", "女士", "母亲", "妈妈", "姐姐", "妹妹", "阿姨",
|
||||
"female", "woman", "girl", "lady"))
|
||||
if male and not female:
|
||||
return "he", "his", "him"
|
||||
if female and not male:
|
||||
return "she", "her", "her"
|
||||
return "they", "their", "them"
|
||||
|
||||
|
||||
_CONSISTENCY = ("Keep {his} exactly the same person as in the reference image: "
|
||||
"same face, same hairstyle and hair color, same top garment.")
|
||||
|
||||
# ── 通用负面词（5 张共用；含防侧身 / 防多视图 / 压 AI 感）──
|
||||
PORTRAIT_NEGATIVE_ZH = (
|
||||
# 防误侧身（实测：不写几何又不禁这些，正面会画成微侧身 20~30°）
|
||||
"three-quarter view, turned body, contrapposto, "
|
||||
# 防多视图（character sheet / turnaround 会让模型在一张图里画多个视图）
|
||||
"character sheet, multiple views, turnaround, model sheet, "
|
||||
# 压 AI 感（写实）
|
||||
"CGI, plastic skin, waxy skin, doll face, over-symmetry, HDR, "
|
||||
"oversaturated, flat lighting, photorealistic, "
|
||||
# 通用质量
|
||||
"扭曲五官，变形，多手指，畸形，模糊，水印，文字，杂乱背景，多余物体，"
|
||||
"夸张滤镜，崩坏，丑化，面部变色，肢体崩坏"
|
||||
)
|
||||
|
||||
# ── 视角定义 ──
|
||||
# ★ 关键规范（shuohao-skills 实测）：
|
||||
# 1. **锚点写完整角色描述；派生图只写「构图指令 + 短清单 + 画风层」**
|
||||
# 2. **改图模型只听第一句** —— `shot` 必须是明确的构图改动指令
|
||||
# 3. 正面必须写**几何**（肩正对 / 鼻子在中线 / 两耳可见），否则会侧身
|
||||
# 4. 一致性清单**按视图取层**（大头照只写脸/头发/上装 —— 写下装会把镜头拉远）
|
||||
PORTRAIT_VIEWS: dict = {
|
||||
"front_half": {
|
||||
"label": "正面半身（主定妆）",
|
||||
"order": 1,
|
||||
"is_anchor": True,
|
||||
"shot": ("Half-body front view: {he} faces the camera squarely, shoulders square to the lens, "
|
||||
"nose on the centerline of the face, both ears equally visible, "
|
||||
"looking straight into the lens with a neutral expression."),
|
||||
"consistency": "", # 锚点自带完整描述，无需清单
|
||||
"default_bg": "seamless cool-grey studio backdrop",
|
||||
"default_light": "soft even studio lighting",
|
||||
"default_expr": "neutral expression",
|
||||
},
|
||||
"side_45": {
|
||||
"label": "45° 侧半身",
|
||||
"order": 2,
|
||||
"is_anchor": False,
|
||||
"shot": ("Turn {his} body 45 degrees toward the image-left: shoulders now angled away, "
|
||||
"the far shoulder partly hidden behind the near one, about three-quarters of "
|
||||
"{his} face still visible, eyes toward the lens."),
|
||||
"consistency": _CONSISTENCY,
|
||||
"default_bg": "seamless cool-grey studio backdrop",
|
||||
"default_light": "soft even studio lighting",
|
||||
"default_expr": "calm expression",
|
||||
},
|
||||
"profile": {
|
||||
"label": "纯侧面轮廓",
|
||||
"order": 3,
|
||||
"is_anchor": False,
|
||||
"shot": ("Full side profile: turn {his} body 90 degrees so {he} faces the right edge of the "
|
||||
"image, only the right half of {his} face is visible, {he} does NOT look at the camera."),
|
||||
"consistency": _CONSISTENCY,
|
||||
"default_bg": "seamless cool-grey studio backdrop",
|
||||
"default_light": "soft even studio lighting, gentle rim on the profile edge",
|
||||
"default_expr": "quiet, eyes lowered slightly",
|
||||
},
|
||||
"face_closeup": {
|
||||
"label": "面部特写",
|
||||
"order": 4,
|
||||
"is_anchor": False,
|
||||
"shot": ("Zoom in to an extreme close-up head-and-shoulders portrait, passport-photo "
|
||||
"framing: top of head near the top edge, chin at the vertical center, bottom "
|
||||
"edge cutting at the collarbone. {He} tilts {his} head slightly down, then lifts "
|
||||
"{his} eyes into the lens."),
|
||||
# ★ 大头照只写脸/头发/上装（写下装会把镜头拉远 —— 实测）
|
||||
"consistency": _CONSISTENCY,
|
||||
"default_bg": "seamless cool-grey studio backdrop",
|
||||
"default_light": "soft even studio lighting",
|
||||
"default_expr": "focused gaze",
|
||||
},
|
||||
"shoulder_look": {
|
||||
"label": "回头斜侧半身",
|
||||
"order": 5,
|
||||
"is_anchor": False,
|
||||
"shot": ("Rotate the camera 180 degrees around {his} so {his} back faces the lens, then {he} "
|
||||
"turns {his} head back over {his} shoulder to look at the camera: visible are {his} "
|
||||
"back, the back of {his} hair, the back of {his} garment, plus {his} turned face."),
|
||||
"consistency": _CONSISTENCY,
|
||||
"default_bg": "seamless cool-grey studio backdrop",
|
||||
"default_light": "soft even studio lighting",
|
||||
"default_expr": "calm expression",
|
||||
},
|
||||
# ── 全身（2026-10-04 补：全景 / 远景 / 走位镜头需要全身参考）──
|
||||
"full_body_front": {
|
||||
"label": "全身正面",
|
||||
"order": 6,
|
||||
"is_anchor": False,
|
||||
"shot": ("Zoom out to a full-body front view: {his} entire figure from the top of {his} head to "
|
||||
"the soles of {his} shoes is visible, standing naturally with arms relaxed at {his} "
|
||||
"sides, feet visible, facing the lens squarely."),
|
||||
"consistency": _CONSISTENCY,
|
||||
"default_bg": "seamless cool-grey studio backdrop, no cast shadow on the floor",
|
||||
"default_light": "soft even studio lighting, no harsh split",
|
||||
"default_expr": "relaxed neutral stance",
|
||||
},
|
||||
"full_body_3q": {
|
||||
"label": "全身 3/4 侧",
|
||||
"order": 7,
|
||||
"is_anchor": False,
|
||||
"shot": ("Zoom out to a full-body three-quarter view: {his} whole figure from head to shoes is "
|
||||
"visible, body turned about 30-45 degrees toward the image-right, head still turned "
|
||||
"back toward the lens side, feet visible."),
|
||||
"consistency": _CONSISTENCY,
|
||||
"default_bg": "seamless cool-grey studio backdrop, no cast shadow on the floor",
|
||||
"default_light": "soft front-side lighting showing the body's form",
|
||||
"default_expr": "relaxed stance, eyes toward the lens",
|
||||
},
|
||||
}
|
||||
|
||||
# ── 视角顺序（生成用）──
|
||||
VIEW_ORDER = sorted(PORTRAIT_VIEWS, key=lambda k: PORTRAIT_VIEWS[k]["order"])
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# ★ 按镜头类型选参考图 —— 让套装每一张都真正发挥作用
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 背景（2026-10-04 用户指出）：套装生成 5~7 张，但代码只取了 front 一张，
|
||||
# 其余全闲置。正确做法是按镜头景别/角度选**最贴近**的那张：
|
||||
# · 侧面镜头喂 profile → 侧脸更准
|
||||
# · 全景/远景喂全身图 → 有全身比例可依
|
||||
# · 特写喂 face_closeup → 五官细节最准
|
||||
# 这样在"可灵只能吃一张图"的限制下，把一致性做到最好。
|
||||
#
|
||||
# 用关键词匹配（导演输出的描述很长，如「局部特写（腿部至脚趾）」）。
|
||||
|
||||
_SIZE_FULL = ("全景", "远景", "全身", "wide", "full", "long", "establish")
|
||||
_SIZE_CLOSE = ("特写", "大特写", "close", "extreme", "macro")
|
||||
_ANGLE_SIDE = ("侧", "45", "三分", "profile", "side", "three-quarter")
|
||||
_ANGLE_BACK = ("背", "回头", "身后", "behind", "over-the-shoulder", "从后")
|
||||
|
||||
|
||||
def pick_reference_for_shot(shot_size: str = "", angle: str = "", refs: dict = None) -> str:
|
||||
"""按镜头景别/角度选最合适的**参考图槽位名**
|
||||
|
||||
refs: 角色资产的 reference_images（如 {"front": "ref_front_half.png", ...}）
|
||||
返回：槽位名（如 "side_45"）；无可用时返回 ""（调用方回退文生视频）
|
||||
|
||||
优先级：全身 > 特写 > 侧面 > 背面 > 半身正面（默认）
|
||||
"""
|
||||
refs = refs or {}
|
||||
if not refs:
|
||||
return ""
|
||||
size = str(shot_size or "")
|
||||
ang = str(angle or "")
|
||||
|
||||
def _pick(*keys) -> str:
|
||||
for k in keys:
|
||||
if refs.get(k):
|
||||
return k
|
||||
return ""
|
||||
|
||||
# ① 全景 / 远景 / 全身 → 全身图（没有则退半身）
|
||||
if any(k in size for k in _SIZE_FULL):
|
||||
p = _pick("full_body_front", "full_body_3q", "full", "front")
|
||||
if p:
|
||||
return p
|
||||
# ② 特写 → 面部特写
|
||||
if any(k in size for k in _SIZE_CLOSE):
|
||||
p = _pick("face_closeup", "front")
|
||||
if p:
|
||||
return p
|
||||
# ③ 侧面 / 45° → 侧视图
|
||||
if any(k in ang for k in _ANGLE_SIDE):
|
||||
p = _pick("side_45", "profile", "front")
|
||||
if p:
|
||||
return p
|
||||
# ④ 背面 / 回头 → 回头斜侧
|
||||
if any(k in ang for k in _ANGLE_BACK):
|
||||
p = _pick("shoulder_look", "side_45", "front")
|
||||
if p:
|
||||
return p
|
||||
# ⑤ 默认：半身正面（主定妆）
|
||||
return _pick("front", "front_half") or next(iter(refs.values()), "")
|
||||
|
||||
|
||||
def build_view_prompt(view: str, character_desc: str, bg: str = "", light: str = "",
|
||||
expression: str = "", extra: str = "") -> str:
|
||||
"""拼一条视角提示词（**锚点与派生图结构不同**）
|
||||
|
||||
锚点（front_half）：
|
||||
{构图几何} + {完整角色描述} + {画风层}
|
||||
派生图（其余 4 张）：
|
||||
{构图指令} + {短一致性清单} + {画风层}
|
||||
—— 只写「要改什么」，不重复角色长描述（shuohao 实测：重复会稀释改动）
|
||||
|
||||
规范依据：shuohao-skills/character-refs 的 prompting.md（实测定稿）
|
||||
"""
|
||||
v = PORTRAIT_VIEWS.get(view) or PORTRAIT_VIEWS["front_half"]
|
||||
# ★ 人称按角色描述动态注入（修复"男医生被生成成女性"）
|
||||
he, his, him = _detect_pronoun(character_desc)
|
||||
subst = {"he": he, "He": he.capitalize(), "his": his, "His": his.capitalize(), "him": him}
|
||||
|
||||
def _fill(s: str) -> str:
|
||||
for k, v2 in subst.items():
|
||||
s = s.replace("{" + k + "}", v2)
|
||||
return s
|
||||
|
||||
look = (f"Backdrop: {bg.strip()}。" if bg else "") + LOOK_REALISM
|
||||
|
||||
if v["is_anchor"]:
|
||||
# 锚点：构图几何 + 完整角色描述 + 画风
|
||||
desc = (character_desc or "").strip().strip("。，, ")
|
||||
parts = [_fill(v["shot"]), desc, look]
|
||||
else:
|
||||
# 派生图：构图指令 + 短清单 + 画风（不重复长描述）
|
||||
parts = [_fill(v["shot"]), _fill(v["consistency"]), look]
|
||||
if expression:
|
||||
parts.append(f"Expression: {expression.strip().strip('。，, ')}。")
|
||||
if extra:
|
||||
parts.append(extra.strip().strip("。，, ") + "。")
|
||||
return " ".join(p for p in parts if p)
|
||||
|
||||
|
||||
def build_full_set(character_desc: str, bg: str = "", light: str = "",
|
||||
expressions: dict = None) -> dict:
|
||||
"""生成完整 5 视角提示词（不调用生成，纯拼接）
|
||||
|
||||
返回 {view_id: {"label", "order", "is_anchor", "prompt"}}
|
||||
"""
|
||||
expressions = expressions or {}
|
||||
out = {}
|
||||
for vid in VIEW_ORDER:
|
||||
v = PORTRAIT_VIEWS[vid]
|
||||
out[vid] = {
|
||||
"label": v["label"],
|
||||
"order": v["order"],
|
||||
"is_anchor": v["is_anchor"],
|
||||
"prompt": build_view_prompt(vid, character_desc, bg=bg, light=light,
|
||||
expression=expressions.get(vid, "")),
|
||||
}
|
||||
return out
|
||||
|
||||
|
||||
def set_summary() -> str:
|
||||
"""人类可读的套装说明（供前端展示）"""
|
||||
lines = ["短剧角色参考图套装（5 视角 · 半身）", ""]
|
||||
for vid in VIEW_ORDER:
|
||||
v = PORTRAIT_VIEWS[vid]
|
||||
mark = "★基准" if v["is_anchor"] else " "
|
||||
lines.append(f" {v['order']}. {mark} {v['label']:16s} {v['composition']}")
|
||||
lines += [
|
||||
"",
|
||||
"流程：先生成「正面半身主定妆」→ 确认满意 → 以它为参考图逐张生成其余 4 张",
|
||||
"（把主定妆当参考图是五官不崩的关键）",
|
||||
"",
|
||||
f"通用负面词：{PORTRAIT_NEGATIVE_ZH}",
|
||||
]
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# ── 预设角色包（config/portrait_presets.yaml）──
|
||||
|
||||
def _presets_path() -> Path:
|
||||
return Path(__file__).resolve().parent.parent.parent / "config" / "portrait_presets.yaml"
|
||||
|
||||
|
||||
def load_presets() -> list:
|
||||
"""读取预设角色包（6 套短剧人设）；文件缺失/损坏时返回空列表（不抛错）"""
|
||||
p = _presets_path()
|
||||
if not p.exists():
|
||||
return []
|
||||
try:
|
||||
import yaml
|
||||
d = yaml.safe_load(p.read_text(encoding="utf-8")) or {}
|
||||
return d.get("presets") or []
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
|
||||
def get_preset(preset_id: str) -> dict:
|
||||
"""按 id 取预设（未知返回空 dict）"""
|
||||
for x in load_presets():
|
||||
if x.get("id") == preset_id:
|
||||
return x
|
||||
return {}
|
||||
|
||||
|
||||
def preset_to_set(preset_id: str) -> dict:
|
||||
"""预设 → 完整 5 视角提示词（含该人设的背景/光/神态）"""
|
||||
ps = get_preset(preset_id)
|
||||
if not ps:
|
||||
return {}
|
||||
return build_full_set(ps.get("desc", ""), bg=ps.get("bg", ""),
|
||||
light=ps.get("light", ""),
|
||||
expressions=ps.get("expressions") or {})
|
||||
|
||||
|
||||
# ── CLI ──
|
||||
if __name__ == "__main__":
|
||||
import argparse
|
||||
ap = argparse.ArgumentParser(description="短剧角色参考图套装（提示词预览）")
|
||||
ap.add_argument("--desc", default="24 岁年轻中国女生，黑茶色中长发，面色苍白，浅蓝条纹病号服")
|
||||
ap.add_argument("--bg", default="")
|
||||
ap.add_argument("--light", default="")
|
||||
ap.add_argument("--json", action="store_true")
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.json:
|
||||
import json
|
||||
print(json.dumps(build_full_set(args.desc, args.bg, args.light), ensure_ascii=False, indent=1))
|
||||
else:
|
||||
print(set_summary())
|
||||
print("\n" + "=" * 72)
|
||||
for vid, item in build_full_set(args.desc, args.bg, args.light).items():
|
||||
print(f"\n【{item['order']}. {item['label']}】{'★ 主定妆' if item['is_anchor'] else ''}")
|
||||
print(f" {item['prompt']}")
|
||||
@@ -0,0 +1,866 @@
|
||||
"""
|
||||
提示词组装器 V3 (PromptAssembler) — 模块化10层角色描述系统
|
||||
|
||||
架构: 10层模块，从粗到细逐层构建角色
|
||||
L0 身份层 — 姓名/性别/年龄/种族/世界观
|
||||
L1 面部轮廓 — 脸型/头型/下颌线/颧骨
|
||||
L2 五官细节 — 眼(形/色/神)/眉/鼻/嘴/耳
|
||||
L3 头发层 — 发型/长度/颜色/质地/发际线
|
||||
L4 皮肤妆容 — 肤色/肤质/底妆/特殊标记/质感
|
||||
L5 体型姿态 — 身高/体型/肩宽/站姿/手部
|
||||
L6 服装配饰 — 上装/下装/外套/鞋/配饰
|
||||
L7 神态表情 — 微表情/眼神/情绪基调/气质
|
||||
L8 光线镜头 — 光型/色温/镜头/景深/氛围
|
||||
L9 背景场景 — 环境/背景色/季节/天气
|
||||
|
||||
每个层级包含:
|
||||
- build_{layer}(data) → 自然语言描述片段
|
||||
- vocab_{layer} → 该层级的完整词汇/选项
|
||||
- default_{layer} → 默认值
|
||||
|
||||
用法:
|
||||
from prompt_assembler_v3 import CharacterPromptSystem, CHAR_LAYERS
|
||||
system = CharacterPromptSystem()
|
||||
|
||||
# 逐层构建
|
||||
system.set_layer("L2_face_eyes", {"shape": "丹凤眼", "color": "深褐", "spirit": "锐利"})
|
||||
|
||||
# 生成最终提示词
|
||||
prompt = system.build_prompt(variant="front_face")
|
||||
|
||||
# 获取当前完整描述
|
||||
desc = system.get_layers_descriptions()
|
||||
|
||||
# 获取某层的可用选项
|
||||
options = system.get_layer_options("L2_face_eyes")
|
||||
"""
|
||||
|
||||
from typing import Optional, Any
|
||||
import json
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 第零层: L0 — 核心身份
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
L0_IDENTITY = {
|
||||
"label": "核心身份",
|
||||
"icon": "🆔",
|
||||
"order": 0,
|
||||
"fields": [
|
||||
{"key": "name", "label": "角色名", "type": "text", "placeholder": "如: 阿远"},
|
||||
{"key": "gender", "label": "性别", "type": "select", "options": ["男性", "女性"]},
|
||||
{"key": "age", "label": "年龄", "type": "select", "options": ["少年(12-18)", "青年(18-30)", "壮年(30-45)", "中年(45-60)", "老年(60+)"]},
|
||||
{"key": "ethnicity", "label": "种族/地域", "type": "select", "options": ["中国", "东亚", "东南亚", "南亚", "中东", "欧美", "非洲", "拉美"]},
|
||||
{"key": "worldview", "label": "世界观/时代", "type": "select", "options": ["现代", "古风", "科幻", "奇幻", "民国", "未来", "赛博朋克"]},
|
||||
],
|
||||
"build": lambda d: f"{d.get('name','')}，一位{d.get('age','青年').split('(')[0]}{d.get('gender','')}，{d.get('ethnicity','')}人，{d.get('worldview','现代')}背景",
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 第一层: L1 — 面部轮廓
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
L1_FACE_SHAPE = {
|
||||
"label": "面部轮廓",
|
||||
"icon": "🗿",
|
||||
"order": 1,
|
||||
"fields": [
|
||||
{"key": "shape", "label": "脸型", "type": "select",
|
||||
"options": ["鹅蛋脸", "瓜子脸", "圆脸", "方脸", "国字脸", "长脸", "菱形脸", "心形脸"],
|
||||
"description": "国字脸/方脸 → 稳重正气；鹅蛋脸 → 温婉；瓜子脸 → 精致秀气"},
|
||||
{"key": "jaw", "label": "下颌线", "type": "select",
|
||||
"options": ["柔和模糊", "清晰分明", "棱角突出", "圆润温和"]},
|
||||
{"key": "cheekbone", "label": "颧骨", "type": "select",
|
||||
"options": ["适中自然", "高突出", "饱满圆润", "低平"]},
|
||||
{"key": "forehead", "label": "额头", "type": "select",
|
||||
"options": ["饱满宽阔", "适中", "窄小", "高额头"]},
|
||||
],
|
||||
"build": lambda d: (
|
||||
f"{d.get('shape','')}"
|
||||
f"{'，' + d.get('jaw','') if d.get('jaw') else ''}"
|
||||
f"{'，' + d.get('cheekbone','') + '颧骨' if d.get('cheekbone') else ''}"
|
||||
f"{'，' + d.get('forehead','') + '额头' if d.get('forehead') else ''}"
|
||||
),
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 第二层: L2 — 五官细节
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
L2_FACIAL_FEATURES = {
|
||||
"label": "五官细节",
|
||||
"icon": "👁️",
|
||||
"order": 2,
|
||||
"sub_layers": {
|
||||
"eyes": {
|
||||
"label": "眼睛",
|
||||
"icon": "👁️",
|
||||
"fields": [
|
||||
{"key": "shape", "label": "眼型", "type": "select",
|
||||
"options": ["丹凤眼", "杏眼", "桃花眼", "圆眼", "细长眼", "下垂眼", "上挑眼", "单眼皮", "双眼皮", "内双"],
|
||||
"description": "丹凤眼 → 古典东方美；杏眼 → 圆润可爱；桃花眼 → 含情脉脉"},
|
||||
{"key": "color", "label": "瞳孔颜色", "type": "select",
|
||||
"options": ["深褐", "浅褐", "黑色", "琥珀色", "灰色", "蓝色", "绿色"]},
|
||||
{"key": "size", "label": "大小", "type": "select",
|
||||
"options": ["大而有神", "适中", "细长", "小而有神"]},
|
||||
{"key": "spirit", "label": "眼神", "type": "text",
|
||||
"placeholder": "如: 深邃锐利、清澈温柔、沉稳内敛", "multi": True},
|
||||
{"key": "detail", "label": "细节补充", "type": "text",
|
||||
"placeholder": "如: 眼尾微垂、卧蚕明显、睫毛浓密"},
|
||||
],
|
||||
"build": lambda d: (
|
||||
f"{d.get('size','')}{d.get('shape','')}"
|
||||
f"{'（' + d.get('color','') + '色瞳孔' + '）' if d.get('color') else ''}"
|
||||
f"{'，' + d.get('spirit','') if d.get('spirit') else ''}"
|
||||
f"{'，' + d.get('detail','') if d.get('detail') else ''}"
|
||||
),
|
||||
},
|
||||
"eyebrows": {
|
||||
"label": "眉毛",
|
||||
"icon": "🧐",
|
||||
"fields": [
|
||||
{"key": "shape", "label": "眉型", "type": "select",
|
||||
"options": ["剑眉", "柳叶眉", "一字眉", "上挑眉", "弯眉", "粗眉", "细长眉"]},
|
||||
{"key": "state", "label": "状态", "type": "select",
|
||||
"options": ["舒展自然", "微微蹙起", "高挑", "平缓"]},
|
||||
{"key": "color", "label": "眉色", "type": "select",
|
||||
"options": ["浓黑", "深棕", "浅棕", "自然黑"]},
|
||||
],
|
||||
"build": lambda d: (
|
||||
f"{d.get('shape','')}"
|
||||
f"{'，' + d.get('color','') if d.get('color') else ''}"
|
||||
f"{'，' + d.get('state','') if d.get('state') else ''}"
|
||||
),
|
||||
},
|
||||
"nose": {
|
||||
"label": "鼻子",
|
||||
"icon": "👃",
|
||||
"fields": [
|
||||
{"key": "bridge", "label": "鼻梁", "type": "select",
|
||||
"options": ["高挺", "挺直窄小", "中等挺拔", "低平柔和"]},
|
||||
{"key": "tip", "label": "鼻头", "type": "select",
|
||||
"options": ["小巧圆润", "圆润饱满", "尖翘", "略宽"]},
|
||||
{"key": "wing", "label": "鼻翼", "type": "select",
|
||||
"options": ["适中", "略宽", "窄小"]},
|
||||
],
|
||||
"build": lambda d: (
|
||||
f"鼻梁{d.get('bridge','')}"
|
||||
f"{'，鼻头' + d.get('tip','') if d.get('tip') else ''}"
|
||||
f"{'，鼻翼' + d.get('wing','') if d.get('wing') else ''}"
|
||||
),
|
||||
},
|
||||
"lips": {
|
||||
"label": "嘴唇",
|
||||
"icon": "👄",
|
||||
"fields": [
|
||||
{"key": "shape", "label": "唇型", "type": "select",
|
||||
"options": ["薄唇", "厚唇", "M形唇", "饱满", "樱桃小嘴", "微笑唇"]},
|
||||
{"key": "color", "label": "唇色", "type": "select",
|
||||
"options": ["自然淡粉", "健康红润", "偏苍白", "暗紫红", "裸色"]},
|
||||
{"key": "state", "label": "状态", "type": "select",
|
||||
"options": ["自然闭合", "微张", "紧抿", "嘴角微扬", "含笑意"]},
|
||||
],
|
||||
"build": lambda d: (
|
||||
f"{d.get('shape','')}"
|
||||
f"{'（' + d.get('color','') + '）' if d.get('color') else ''}"
|
||||
f"{'，' + d.get('state','') if d.get('state') else ''}"
|
||||
),
|
||||
},
|
||||
},
|
||||
"build": lambda d: "", # 由子层分别构建
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 第三层: L3 — 头发
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
L3_HAIR = {
|
||||
"label": "头发",
|
||||
"icon": "💇",
|
||||
"order": 3,
|
||||
"fields": [
|
||||
{"key": "style", "label": "发型", "type": "select",
|
||||
"options": ["短发", "寸头", "中长发", "长发", "背头", "三七分", "齐刘海", "斜刘海",
|
||||
"大背头", "碎发", "丸子头", "高马尾", "低马尾", "披肩发", "盘发"]},
|
||||
{"key": "color", "label": "发色", "type": "select",
|
||||
"options": ["黑色", "深棕", "浅棕", "亚麻色", "灰色", "银白", "渐变挑染"]},
|
||||
{"key": "texture", "label": "发质", "type": "select",
|
||||
"options": ["柔顺直发", "自然微卷", "波浪卷", "小卷", "粗硬", "细软", "蓬松"]},
|
||||
{"key": "length", "label": "长度", "type": "select",
|
||||
"options": ["板寸(3mm)", "短(3-6cm)", "及耳", "及肩", "及胸", "及腰", "过腰"]},
|
||||
{"key": "detail", "label": "细节补充", "type": "text",
|
||||
"placeholder": "如: 两侧收短顶部略长、发际线略高"},
|
||||
],
|
||||
"build": lambda d: (
|
||||
f"{d.get('texture','')}{d.get('style','')}"
|
||||
f"{'（' + d.get('color','') + '）' if d.get('color') else ''}"
|
||||
f"{'，' + d.get('length','') if d.get('length') else ''}"
|
||||
f"{'，' + d.get('detail','') if d.get('detail') else ''}"
|
||||
),
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 第四层: L4 — 皮肤与妆容
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
L4_SKIN_MAKEUP = {
|
||||
"label": "皮肤与妆容",
|
||||
"icon": "✨",
|
||||
"order": 4,
|
||||
"fields": [
|
||||
{"key": "skin_tone", "label": "肤色", "type": "select",
|
||||
"options": ["冷白皮", "暖白皮", "自然偏白", "健康小麦", "蜜色", "古铜色", "深色"],
|
||||
"description": "冷白→清冷；暖白→温润；小麦→阳光健康"},
|
||||
{"key": "skin_texture", "label": "肤质", "type": "select",
|
||||
"options": ["细腻光滑", "自然肌理", "毛孔可见", "粗糙", "哑光", "水润光泽"],
|
||||
"description": "人像特写建议 '细腻光滑'+'自然肌理' 组合使用"},
|
||||
{"key": "complexion", "label": "气色", "type": "select",
|
||||
"options": ["红润健康", "略显苍白", "疲惫暗沉", "容光焕发", "自然均匀"]},
|
||||
{"key": "makeup_base", "label": "底妆", "type": "select",
|
||||
"options": ["素颜", "淡妆", "精致妆容", "烟熏妆", "裸妆", "浓妆"]},
|
||||
{"key": "makeup_lip", "label": "唇妆", "type": "select",
|
||||
"options": ["自然唇色", "淡裸色", "豆沙色", "正红色", "暗紫色", "水光唇"]},
|
||||
{"key": "makeup_eye", "label": "眼妆", "type": "select",
|
||||
"options": ["无眼妆", "自然眼线", "淡眼影", "烟熏", "猫眼眼线"]},
|
||||
{"key": "blemish", "label": "特殊标记", "type": "text",
|
||||
"placeholder": "如: 右眉上细疤、左颊美人痣、法令纹明显、笑纹"},
|
||||
],
|
||||
"build": lambda d: (
|
||||
f"{d.get('skin_tone','')}"
|
||||
f"{'，' + d.get('skin_texture','') if d.get('skin_texture') else ''}"
|
||||
f"{'，' + d.get('complexion','') if d.get('complexion') else ''}"
|
||||
f"{'，' + d.get('makeup_base','') if d.get('makeup_base') else ''}"
|
||||
f"{'（' + d.get('makeup_eye','') + '，' + d.get('makeup_lip','') + '）' if d.get('makeup_eye') or d.get('makeup_lip') else ''}"
|
||||
f"{'，' + d.get('blemish','') if d.get('blemish') else ''}"
|
||||
),
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 第五层: L5 — 体型与姿态
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
L5_BODY = {
|
||||
"label": "体型与姿态",
|
||||
"icon": "🏋️",
|
||||
"order": 5,
|
||||
"fields": [
|
||||
{"key": "height", "label": "身高", "type": "select",
|
||||
"options": ["矮小(155-165cm)", "中等(165-175cm)", "中高(175-185cm)", "高大(185cm+)"]},
|
||||
{"key": "build", "label": "体型", "type": "select",
|
||||
"options": ["纤细瘦弱", "偏瘦", "标准匀称", "健壮", "壮硕", "丰满", "魁梧"]},
|
||||
{"key": "shoulder", "label": "肩宽", "type": "select",
|
||||
"options": ["窄肩", "标准", "宽肩", "厚肩"]},
|
||||
{"key": "posture", "label": "气质姿态", "type": "select",
|
||||
"options": ["挺拔", "微驼", "自然放松", "端庄", "慵懒", "紧绷"]},
|
||||
{"key": "hands", "label": "手部", "type": "text",
|
||||
"placeholder": "如: 手指修长、关节分明、指甲修剪整齐"},
|
||||
],
|
||||
"build": lambda d: (
|
||||
f"身高{d.get('height','中等').split('(')[0]}"
|
||||
f"体型{d.get('build','')}"
|
||||
f"{'，' + d.get('shoulder','') if d.get('shoulder') else ''}"
|
||||
f"{'，姿态' + d.get('posture','') if d.get('posture') else ''}"
|
||||
f"{'，' + d.get('hands','') if d.get('hands') else ''}"
|
||||
),
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 第六层: L6 — 服装与配饰
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
L6_CLOTHING = {
|
||||
"label": "服装与配饰",
|
||||
"icon": "👔",
|
||||
"order": 6,
|
||||
"fields": [
|
||||
{"key": "top", "label": "上装", "type": "text",
|
||||
"placeholder": "如: 深灰色拉链运动夹克"},
|
||||
{"key": "inner", "label": "内搭", "type": "text",
|
||||
"placeholder": "如: 白色圆领速干内衬"},
|
||||
{"key": "bottom", "label": "下装", "type": "text",
|
||||
"placeholder": "如: 深蓝色牛仔裤、黑色运动短裤"},
|
||||
{"key": "outerwear", "label": "外套", "type": "text",
|
||||
"placeholder": "如: 深藏蓝亚麻外套"},
|
||||
{"key": "shoes", "label": "鞋子", "type": "text",
|
||||
"placeholder": "如: 白色跑步鞋"},
|
||||
{"key": "accessories", "label": "配饰", "type": "text", "multi": True,
|
||||
"placeholder": "如: 蓝色无线运动耳机(右耳)、细框金属眼镜、简约电子手表、运动护腕"},
|
||||
{"key": "style_tag", "label": "穿搭风格", "type": "select",
|
||||
"options": ["运动休闲", "商务正装", "商务休闲", "街头潮流", "日系清新", "简约素雅", "军事工装", "复古"]},
|
||||
],
|
||||
"build": lambda d: (
|
||||
f"身穿{d.get('top','')}"
|
||||
f"{'，内搭' + d.get('inner','') if d.get('inner') else ''}"
|
||||
f"{'，下穿' + d.get('bottom','') if d.get('bottom') else ''}"
|
||||
f"{'，外披' + d.get('outerwear','') if d.get('outerwear') else ''}"
|
||||
f"{'，脚蹬' + d.get('shoes','') if d.get('shoes') else ''}"
|
||||
f"{'，佩戴' + d.get('accessories','') if d.get('accessories') else ''}"
|
||||
f"{'，' + d.get('style_tag','') + '风格' if d.get('style_tag') else ''}"
|
||||
),
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 第七层: L7 — 神态与表情
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
L7_EXPRESSION = {
|
||||
"label": "神态与表情",
|
||||
"icon": "😊",
|
||||
"order": 7,
|
||||
"fields": [
|
||||
{"key": "base_mood", "label": "情绪基调", "type": "select",
|
||||
"options": ["平静从容", "温和友善", "冷峻严肃", "开朗活泼", "忧郁内敛", "自信昂扬", "恬静淡然"]},
|
||||
{"key": "expression", "label": "表面表情", "type": "select",
|
||||
"options": ["中性无表情", "淡淡微笑", "开怀大笑", "微笑", "皱眉沉思", "瞪大眼惊讶", "绷着脸"]},
|
||||
{"key": "eye_spirit", "label": "眼神细节", "type": "text",
|
||||
"placeholder": "如: 目光坚定直视前方、眼神游离若有所思、眼中带笑意、目光如炬"},
|
||||
{"key": "micro_expr", "label": "微表情", "type": "text", "multi": True,
|
||||
"placeholder": "如: 嘴角微微上翘、眉头轻蹙、眼角细纹、鼻翼轻动"},
|
||||
{"key": "aura", "label": "气质气场", "type": "text",
|
||||
"placeholder": "如: 沉稳内敛不怒自威、亲和温暖如沐春风、疏离清冷生人勿近"},
|
||||
],
|
||||
"build": lambda d: (
|
||||
f"神色{d.get('base_mood','')}"
|
||||
f"{'，表情' + d.get('expression','') if d.get('expression') else ''}"
|
||||
f"{'，' + d.get('eye_spirit','') if d.get('eye_spirit') else ''}"
|
||||
f"{'，' + d.get('micro_expr','') if d.get('micro_expr') else ''}"
|
||||
f"{'，整体气质' + d.get('aura','') if d.get('aura') else ''}"
|
||||
),
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 第八层: L8 — 光线与镜头
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
L8_LIGHTING_CAMERA = {
|
||||
"label": "光线与镜头",
|
||||
"icon": "📸",
|
||||
"order": 8,
|
||||
"fields": [
|
||||
{"key": "light_type", "label": "光型", "type": "select",
|
||||
"options": ["柔光箱均匀布光", "美人碟柔光", "伦勃朗光", "分割光", "侧光/塑形光",
|
||||
"逆光/轮廓光", "阴天漫射光", "顶光/戏剧性"],
|
||||
"description": "美人碟光 → 人像经典；伦勃朗光 → 立体感；分割光 → 戏剧性"},
|
||||
{"key": "color_temp", "label": "色温", "type": "select",
|
||||
"options": ["暖色调(金色)", "冷色调(青蓝)", "中性白平衡", "混合色温", "自然日光"]},
|
||||
{"key": "lens", "label": "镜头焦距", "type": "select",
|
||||
"options": ["广角(24mm)", "标准(50mm)", "中焦(85mm)", "长焦(135mm)", "微距"]},
|
||||
{"key": "depth", "label": "景深", "type": "select",
|
||||
"options": ["浅景深/背景虚化", "中等景深", "大景深/全清晰"]},
|
||||
{"key": "light_detail", "label": "光线细节", "type": "text",
|
||||
"placeholder": "如: 光线从右侧45°打来，在左脸形成柔和阴影"},
|
||||
],
|
||||
"build": lambda d: (
|
||||
f"{d.get('light_type','柔光箱均匀布光')}"
|
||||
f"{'，' + d.get('color_temp','') if d.get('color_temp') else ''}"
|
||||
f"{'，' + d.get('lens','') + '镜头' if d.get('lens') else ''}"
|
||||
f"{'，' + d.get('depth','') if d.get('depth') else ''}"
|
||||
f"{'，' + d.get('light_detail','') if d.get('light_detail') else ''}"
|
||||
),
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 第九层: L9 — 背景与场景
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
L9_BACKGROUND = {
|
||||
"label": "背景与场景",
|
||||
"icon": "🌄",
|
||||
"order": 9,
|
||||
"fields": [
|
||||
{"key": "bg_type", "label": "背景类型", "type": "select",
|
||||
"options": ["纯色背景", "影棚布景", "自然户外", "城市街景", "室内空间", "特定场景"]},
|
||||
{"key": "bg_color", "label": "背景色/氛围", "type": "select",
|
||||
"options": ["纯白/干净", "纯灰/专业", "纯黑/神秘", "暖色/温馨", "冷色/冷静", "渐变色"]},
|
||||
{"key": "environment", "label": "具体环境", "type": "text",
|
||||
"placeholder": "如: 清晨城市街道、江南古巷青砖墙、健身房器械区"},
|
||||
{"key": "season", "label": "季节/天气", "type": "select",
|
||||
"options": ["晴天", "阴天", "雨后", "雪天", "黄昏", "深夜", "清晨", "午后"]},
|
||||
],
|
||||
"build": lambda d: (
|
||||
f"{d.get('environment','')}"
|
||||
f"{'，' + d.get('bg_color','') if d.get('bg_color') else ''}"
|
||||
f"{'，' + d.get('season','') if d.get('season') else ''}"
|
||||
),
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 层级注册表
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
CHAR_LAYERS = {
|
||||
"L0_identity": L0_IDENTITY,
|
||||
"L1_face_shape": L1_FACE_SHAPE,
|
||||
"L2_facial_features": L2_FACIAL_FEATURES,
|
||||
"L3_hair": L3_HAIR,
|
||||
"L4_skin_makeup": L4_SKIN_MAKEUP,
|
||||
"L5_body": L5_BODY,
|
||||
"L6_clothing": L6_CLOTHING,
|
||||
"L7_expression": L7_EXPRESSION,
|
||||
"L8_lighting_camera": L8_LIGHTING_CAMERA,
|
||||
"L9_background": L9_BACKGROUND,
|
||||
}
|
||||
|
||||
# 各子层（L2的各部位）
|
||||
SUB_LAYERS = {
|
||||
"L2_face_eyes": L2_FACIAL_FEATURES["sub_layers"]["eyes"],
|
||||
"L2_face_eyebrows": L2_FACIAL_FEATURES["sub_layers"]["eyebrows"],
|
||||
"L2_face_nose": L2_FACIAL_FEATURES["sub_layers"]["nose"],
|
||||
"L2_face_lips": L2_FACIAL_FEATURES["sub_layers"]["lips"],
|
||||
}
|
||||
|
||||
# 所有可配置项合并
|
||||
ALL_LAYERS = {**CHAR_LAYERS, **SUB_LAYERS}
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 变体提示词模板
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
# ── 正面面部特写（定妆照基准） ──
|
||||
FRONT_FACE_TEMPLATE_ZH = (
|
||||
"纯色无缝冷灰摄影棚背景，背景上除了人物什么都没有（无墙、无家具、无道具、无窗、无植物、无风景、无地面阴影）。"
|
||||
"正面胸像定妆照，面向相机，中性自然表情。" +
|
||||
"{L0}。{L1}。{eyes}。{eyebrows}。{nose}。{lips}。{L3}。{L4}。{L5}。{L6}。{L7}。{L8}。"
|
||||
"摄影棚均匀柔光，真实皮肤质感，中性素净不加调色。比例9:16。"
|
||||
)
|
||||
|
||||
FRONT_FACE_TEMPLATE_EN = (
|
||||
"{L0}. {L1}. {eyes}. {eyebrows}. {nose}. {lips}. {L3}. {L4}. {L5}. {L6}. {L7}. {L8}. {L9}. "
|
||||
"Professional portrait photography, front-facing camera, neutral expression. "
|
||||
"Commercial photography quality, 8k resolution, photorealistic style, high detail. --ar 9:16"
|
||||
)
|
||||
|
||||
# ── 全身照 ──
|
||||
FULL_BODY_TEMPLATE_ZH = (
|
||||
"纯色无缝冷灰摄影棚背景，背景上除了人物什么都没有（无墙、无家具、无道具、无窗、无植物、无风景、无地面阴影）。"
|
||||
"全身站立定妆照，头顶到脚底完整入镜，人物居中。" +
|
||||
"{L0}。{L1}。{eyes}。{L3}。{L4}。{L5}。{L6}。{L7}。"
|
||||
"摄影棚均匀柔光，真实摄影质感，中性素净。比例9:16。"
|
||||
)
|
||||
|
||||
# ── 网格定妆照 ──
|
||||
# ── 全身各视角模板（★ 修复：原来 multi_angle 全部错用 front_face 模板）──
|
||||
# 判据：每个视角必须有自己的构图描述，否则模型只会画同一种正面。
|
||||
_BODY_BG = ("纯色无缝冷灰摄影棚背景，背景上除了人物什么都没有"
|
||||
"（无墙、无家具、无道具、无窗、无植物、无风景、无地面阴影）。")
|
||||
_BODY_STYLE = ("Fujifilm X100V 35mm 镜头直出，unfiltered，"
|
||||
"真实皮肤质感、毛孔可见、不磨皮不塑料感，摄影棚均匀柔光。")
|
||||
|
||||
BODY_ANGLE_TEMPLATES = {
|
||||
"front": "{desc}。全身正面站立定妆照，从头顶到脚底完整入镜（**必须拍到脚**），"
|
||||
"双臂自然垂于身侧，面向相机。" + _BODY_BG + _BODY_STYLE + "比例9:16。",
|
||||
"side": "{desc}。全身左侧面定妆照，从头顶到脚底完整入镜（**必须拍到脚**），"
|
||||
"正左侧 profile 视角，面向画面右侧。" + _BODY_BG + _BODY_STYLE + "比例9:16。",
|
||||
"right_three_quarter": "{desc}。全身右前 3/4 侧定妆照，从头顶到脚底完整入镜"
|
||||
"（**必须拍到脚**），身体朝右前方约 45 度。" + _BODY_BG + _BODY_STYLE + "比例9:16。",
|
||||
"right_side": "{desc}。全身右侧面定妆照，从头顶到脚底完整入镜（**必须拍到脚**），"
|
||||
"正右侧 profile 视角，面向画面左侧。" + _BODY_BG + _BODY_STYLE + "比例9:16。",
|
||||
"back": "{desc}。全身背面定妆照，从正后方拍摄，从头顶到脚踝完整入镜"
|
||||
"（**必须拍到脚**），看不见脸。" + _BODY_BG + _BODY_STYLE + "比例9:16。",
|
||||
}
|
||||
|
||||
GRID_STANDARD_ZH = (
|
||||
"角色定妆参考表，三格并排（左：胸像正面；中：全身正面；右：全身背面），"
|
||||
"三格为同一人，五官发型服装体型完全一致。" + "纯色无缝冷灰摄影棚背景，背景上除了人物什么都没有（无墙、无家具、无道具、无窗、无植物、无风景、无地面阴影）。"
|
||||
"摄影棚均匀柔光，真实皮肤质感，中性素净不加调色。展示："
|
||||
"{L0}。{L1}。{eyes}。{L3}。{L6}。{L7}。比例16:9。"
|
||||
)
|
||||
|
||||
GRID_SPORT_ZH = (
|
||||
"一张写实风格的角色参考定妆照，展示{L0}。{L1}。{eyes}。{L3}。{L5}。{L6}。{L7}。"
|
||||
"画面为4列×2行共8格的网格布局，格间有细灰线分隔。"
|
||||
"上一行：全身正面中性站姿、全身侧面站姿、面部正面中性表情特写、面部四分之三左侧表情特写。"
|
||||
"下一行：面部纯左侧轮廓特写、面部四分之三右侧表情特写、面部专注坚定眼神特写、面部发力表情特写。"
|
||||
"8格人物为完全相同的同一人（五官/发型/体型高度一致）。"
|
||||
"纯白无缝背景，无任何背景元素（无家具/道具/窗/户外风景），人物边界清晰可抠图。"
|
||||
"真实摄影质感，保留皮肤纹理与毛孔，自然柔和光线，非插画非动漫。比例3:2。"
|
||||
"负向：场景背景、环境元素、道具、窗户、风景、杂乱背景、文字、水印。"
|
||||
)
|
||||
|
||||
# ── 场景变体 ──
|
||||
SCENE_TEMPLATE_ZH = (
|
||||
"{L0}。{L1}。{eyes}。{L3}。{L4}。{L5}。{L6}。{L7}。"
|
||||
"{scene_desc}。{L8}。{L9}。"
|
||||
"电影级摄影品质，写实风格。比例9:16"
|
||||
)
|
||||
|
||||
# ── 表情变体 ──
|
||||
EXPRESSION_TEMPLATE_ZH = (
|
||||
"面部特写肖像，{L0}。{L1}。{eyes}。{eyebrows}。{nose}。{lips}。{L3}。{L4}。{L6}。"
|
||||
"{expr_desc}。{L8}。"
|
||||
"专业影室肖像，纯灰色背景。比例9:16"
|
||||
)
|
||||
|
||||
# ── 负面提示词（通用） ──
|
||||
NEGATIVE_PROMPT = (
|
||||
"different face, inconsistent features, changed eye color, altered hair, "
|
||||
"morphed facial structure, deformed face, asymmetric face, "
|
||||
"extra limbs, bad anatomy, distorted, blurry, low quality, "
|
||||
"cartoon, anime, illustration, painting, 3D render"
|
||||
)
|
||||
|
||||
NEGATIVE_PROMPT_ZH = (
|
||||
"面部不一致，五官改变，眼睛颜色变化，发型变化，"
|
||||
"面部结构扭曲，面部不对称，多余肢体，解剖错误，"
|
||||
"模糊，低质量，卡通，动漫，插画，3D渲染"
|
||||
)
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# CharacterPromptSystem — 角色提示词系统
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
class CharacterPromptSystem:
|
||||
"""模块化10层角色提示词系统"""
|
||||
|
||||
def __init__(self):
|
||||
# 每层的数据存储
|
||||
self._layers = {}
|
||||
for layer_id in CHAR_LAYERS:
|
||||
self._layers[layer_id] = {}
|
||||
for sub_id in SUB_LAYERS:
|
||||
self._layers[sub_id] = {}
|
||||
self._scene_override = ""
|
||||
self._expression_override = ""
|
||||
|
||||
# ── 设置某层数据 ──
|
||||
|
||||
def set_layer(self, layer_id: str, data: dict):
|
||||
"""设置某层的数据"""
|
||||
if layer_id in self._layers:
|
||||
self._layers[layer_id].update(data)
|
||||
else:
|
||||
self._layers[layer_id] = data
|
||||
|
||||
def get_layer(self, layer_id: str) -> dict:
|
||||
"""获取某层的数据"""
|
||||
return self._layers.get(layer_id, {})
|
||||
|
||||
def set_scene(self, scene_desc: str):
|
||||
"""设置场景描述（用于场景变体）"""
|
||||
self._scene_override = scene_desc
|
||||
|
||||
def set_expression(self, expr_desc: str):
|
||||
"""设置表情描述（用于表情变体）"""
|
||||
self._expression_override = expr_desc
|
||||
|
||||
# ── 从自然语言描述批量设置 ──
|
||||
|
||||
def from_natural_description(self, desc: str):
|
||||
"""从自然语言描述中尝试提取各层数据（简易版）"""
|
||||
# 这个功能由 attribute_extractor 完成更准确
|
||||
# 这里只做兜底
|
||||
self._layers["L0_identity"]["_raw"] = desc
|
||||
|
||||
# ── 构建提示词 ──
|
||||
|
||||
def build_prompt(self, variant: str = "front_face", lang: str = "zh",
|
||||
mode: str = "standard", **kwargs) -> str:
|
||||
"""
|
||||
构建完整的生成提示词
|
||||
|
||||
参数:
|
||||
variant: 变体类型 (front_face|full_body|grid|scene|expression)
|
||||
lang: 语言
|
||||
mode: 网格模式 (standard|sport)
|
||||
**kwargs: 额外参数
|
||||
|
||||
返回:
|
||||
完整提示词字符串
|
||||
"""
|
||||
# 先构建各层描述
|
||||
layers_desc = self._build_layers_descriptions()
|
||||
|
||||
if variant == "front_face":
|
||||
return self._build_front_face(layers_desc, lang)
|
||||
elif variant == "full_body":
|
||||
return self._build_full_body(layers_desc, lang)
|
||||
elif variant == "multi_angle":
|
||||
angle = kwargs.get("angle") or kwargs.get("expression_name") or "front"
|
||||
return self._build_multi_angle(layers_desc, lang, angle)
|
||||
elif variant == "grid":
|
||||
return self._build_grid(layers_desc, lang, mode)
|
||||
elif variant == "scene":
|
||||
scene = kwargs.get("scene_desc", self._scene_override)
|
||||
return self._build_scene(layers_desc, lang, scene)
|
||||
elif variant == "expression":
|
||||
expr = kwargs.get("expr_desc", self._expression_override)
|
||||
return self._build_expression(layers_desc, lang, expr)
|
||||
else:
|
||||
return self._build_front_face(layers_desc, lang)
|
||||
|
||||
def _build_layers_descriptions(self) -> dict:
|
||||
"""构建所有层的描述文本"""
|
||||
result = {}
|
||||
|
||||
# L0 身份
|
||||
result["L0"] = CHAR_LAYERS["L0_identity"]["build"](self._layers.get("L0_identity", {}))
|
||||
|
||||
# L1 面部轮廓
|
||||
result["L1"] = CHAR_LAYERS["L1_face_shape"]["build"](self._layers.get("L1_face_shape", {}))
|
||||
|
||||
# L2 五官子层
|
||||
for sub_id, sub_def in SUB_LAYERS.items():
|
||||
result[sub_id.replace("L2_face_", "")] = sub_def["build"](self._layers.get(sub_id, {}))
|
||||
|
||||
# L3-L9
|
||||
for layer_id in ["L3_hair", "L4_skin_makeup", "L5_body",
|
||||
"L6_clothing", "L7_expression", "L8_lighting_camera", "L9_background"]:
|
||||
key = layer_id.split("_")[0] # L3, L4, etc
|
||||
result[key] = CHAR_LAYERS[layer_id]["build"](self._layers.get(layer_id, {}))
|
||||
|
||||
return result
|
||||
|
||||
def _build_front_face(self, d: dict, lang: str) -> str:
|
||||
"""构建正面面部特写提示词"""
|
||||
template = FRONT_FACE_TEMPLATE_ZH if lang == "zh" else FRONT_FACE_TEMPLATE_EN
|
||||
prompt = template.format(
|
||||
L0=d.get("L0", ""), L1=d.get("L1", ""),
|
||||
eyes=d.get("eyes", ""), eyebrows=d.get("eyebrows", ""),
|
||||
nose=d.get("nose", ""), lips=d.get("lips", ""),
|
||||
L3=d.get("L3", ""), L4=d.get("L4", ""),
|
||||
L5=d.get("L5", ""), L6=d.get("L6", ""),
|
||||
L7=d.get("L7", ""), L8=d.get("L8", ""),
|
||||
L9=d.get("L9", ""),
|
||||
)
|
||||
# 清理空片段留下的多余标点
|
||||
return self._clean_prompt(prompt)
|
||||
|
||||
def _build_full_body(self, d: dict, lang: str) -> str:
|
||||
"""构建全身照提示词"""
|
||||
prompt = FULL_BODY_TEMPLATE_ZH.format(
|
||||
L0=d.get("L0", ""), L1=d.get("L1", ""),
|
||||
eyes=d.get("eyes", ""), L3=d.get("L3", ""),
|
||||
L4=d.get("L4", ""), L5=d.get("L5", ""),
|
||||
L6=d.get("L6", ""), L7=d.get("L7", ""),
|
||||
) if lang == "zh" else ""
|
||||
return self._clean_prompt(prompt)
|
||||
|
||||
def _build_multi_angle(self, d: dict, lang: str, angle: str) -> str:
|
||||
"""构建**单视角全身**提示词（front / side / right_three_quarter / right_side / back）
|
||||
|
||||
★ 修复说明：原来 variant_map 把 multi_angle 映射成 front_face，
|
||||
导致 5 个角度全部生成"正面特写" —— 用户看到的"同一张脸重复"就是由此而来。
|
||||
现在每个角度有自己的构图描述（含"必须拍到脚"的全身边界）。
|
||||
"""
|
||||
tpl = BODY_ANGLE_TEMPLATES.get(angle) or BODY_ANGLE_TEMPLATES["front"]
|
||||
# 人物描述（各层拼接）
|
||||
desc = self._join_desc(d)
|
||||
prompt = tpl.format(desc=desc)
|
||||
return self._clean_prompt(prompt)
|
||||
|
||||
@staticmethod
|
||||
def _join_desc(d: dict) -> str:
|
||||
"""把各层描述拼成一句人物说明（供全身模板的 {desc} 使用）"""
|
||||
bits = [d.get(k, "") for k in
|
||||
("L0", "L1", "eyes", "eyebrows", "nose", "lips",
|
||||
"L3", "L4", "L5", "L6")]
|
||||
seen, out = set(), []
|
||||
for b in bits:
|
||||
b = (b or "").strip().strip("。，, ")
|
||||
if b and b not in seen:
|
||||
seen.add(b); out.append(b)
|
||||
return "，".join(out)
|
||||
|
||||
def _build_grid(self, d: dict, lang: str, mode: str) -> str:
|
||||
"""构建网格定妆照提示词"""
|
||||
template = GRID_SPORT_ZH if mode == "sport" else GRID_STANDARD_ZH
|
||||
prompt = template.format(
|
||||
L0=d.get("L0", ""), L1=d.get("L1", ""),
|
||||
eyes=d.get("eyes", ""), L3=d.get("L3", ""),
|
||||
L5=d.get("L5", ""), L6=d.get("L6", ""),
|
||||
L7=d.get("L7", ""),
|
||||
)
|
||||
return self._clean_prompt(prompt)
|
||||
|
||||
def _build_scene(self, d: dict, lang: str, scene: str) -> str:
|
||||
"""构建场景变体提示词"""
|
||||
scene_desc = scene or "自然场景"
|
||||
prompt = SCENE_TEMPLATE_ZH.format(
|
||||
L0=d.get("L0", ""), L1=d.get("L1", ""),
|
||||
eyes=d.get("eyes", ""), L3=d.get("L3", ""),
|
||||
L4=d.get("L4", ""), L5=d.get("L5", ""),
|
||||
L6=d.get("L6", ""), L7=d.get("L7", ""),
|
||||
scene_desc=scene_desc,
|
||||
L8=d.get("L8", ""), L9=d.get("L9", ""),
|
||||
)
|
||||
return self._clean_prompt(prompt)
|
||||
|
||||
def _build_expression(self, d: dict, lang: str, expr: str) -> str:
|
||||
"""构建表情变体提示词"""
|
||||
expr_desc = expr or "中性表情"
|
||||
prompt = EXPRESSION_TEMPLATE_ZH.format(
|
||||
L0=d.get("L0", ""), L1=d.get("L1", ""),
|
||||
eyes=d.get("eyes", ""), eyebrows=d.get("eyebrows", ""),
|
||||
nose=d.get("nose", ""), lips=d.get("lips", ""),
|
||||
L3=d.get("L3", ""), L4=d.get("L4", ""),
|
||||
L6=d.get("L6", ""), expr_desc=expr_desc,
|
||||
L8=d.get("L8", ""),
|
||||
)
|
||||
return self._clean_prompt(prompt)
|
||||
|
||||
def _clean_prompt(self, prompt: str) -> str:
|
||||
"""清理提示词中的多余标点和连续空格"""
|
||||
import re
|
||||
# 移除连续的逗号/句号
|
||||
prompt = re.sub(r'[，,]{2,}', '，', prompt)
|
||||
prompt = re.sub(r'[。.]{2,}', '。', prompt)
|
||||
# 移除空括号
|
||||
prompt = re.sub(r'（[^）]*?）', '', prompt)
|
||||
prompt = re.sub(r'\([^)]*?\)', '', prompt)
|
||||
# 移除开头的Punctuation
|
||||
prompt = re.sub(r'^[，。、\s]+', '', prompt)
|
||||
return prompt.strip()
|
||||
|
||||
# ── 获取当前完整自然语言描述 ──
|
||||
|
||||
def get_full_description(self) -> str:
|
||||
"""生成完整的自然语言角色描述（用于知识库回写）"""
|
||||
d = self._build_layers_descriptions()
|
||||
|
||||
parts = []
|
||||
for key in ["L0", "L1", "eyes", "eyebrows", "nose", "lips",
|
||||
"L3", "L4", "L5", "L6", "L7"]:
|
||||
val = d.get(key, "")
|
||||
if val:
|
||||
parts.append(val)
|
||||
|
||||
desc = "，".join(parts)
|
||||
return self._clean_prompt(desc)
|
||||
|
||||
def get_layer_options(self, layer_id: str) -> dict:
|
||||
"""获取某层的所有可选项"""
|
||||
layer = ALL_LAYERS.get(layer_id)
|
||||
if not layer:
|
||||
return {}
|
||||
return {
|
||||
"label": layer.get("label", ""),
|
||||
"icon": layer.get("icon", ""),
|
||||
"fields": layer.get("fields", []),
|
||||
}
|
||||
|
||||
def get_all_layers_config(self) -> dict:
|
||||
"""获取所有层的配置（供前端渲染）"""
|
||||
return {
|
||||
layer_id: self.get_layer_options(layer_id)
|
||||
for layer_id in ALL_LAYERS
|
||||
}
|
||||
|
||||
def get_variant_types(self) -> dict:
|
||||
"""获取支持的变体类型"""
|
||||
return {
|
||||
"front_face": "正面面部特写",
|
||||
"full_body": "全身照",
|
||||
"grid": "网格定妆照",
|
||||
"scene": "场景变体",
|
||||
"expression": "表情变体",
|
||||
}
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 兼容旧接口（包装器）
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
class PromptAssembler:
|
||||
"""兼容旧版 PromptAssembler 接口"""
|
||||
|
||||
def __init__(self):
|
||||
self._system = CharacterPromptSystem()
|
||||
|
||||
def assemble(self, attrs: dict, variant_type: str,
|
||||
lang: str = "zh", **kwargs) -> str:
|
||||
"""兼容旧版 assemble 方法"""
|
||||
# 从旧式 attrs 填充系统
|
||||
if attrs.get("name") or attrs.get("description"):
|
||||
self._system.set_layer("L0_identity", {
|
||||
"name": attrs.get("name", ""),
|
||||
"gender": "男性" if "男" in attrs.get("description", "") else "女性",
|
||||
})
|
||||
|
||||
face = attrs.get("face", {})
|
||||
if face:
|
||||
self._system.set_layer("L2_face_eyes", {
|
||||
"shape": face.get("eyes", ""),
|
||||
"spirit": "",
|
||||
})
|
||||
self._system.set_layer("L2_face_nose", {
|
||||
"bridge": face.get("nose", ""),
|
||||
})
|
||||
self._system.set_layer("L2_face_lips", {
|
||||
"shape": face.get("lips", ""),
|
||||
})
|
||||
if face.get("hair"):
|
||||
self._system.set_layer("L3_hair", {"style": face["hair"]})
|
||||
if face.get("skin"):
|
||||
self._system.set_layer("L4_skin_makeup", {"skin_tone": face["skin"]})
|
||||
|
||||
clothing = attrs.get("clothing", {})
|
||||
if clothing:
|
||||
self._system.set_layer("L6_clothing", {
|
||||
"top": clothing.get("top", ""),
|
||||
"inner": clothing.get("inner", ""),
|
||||
})
|
||||
|
||||
personality = attrs.get("personality", "")
|
||||
if personality:
|
||||
self._system.set_layer("L7_expression", {"base_mood": personality})
|
||||
|
||||
# 映射变体类型
|
||||
variant_map = {
|
||||
"front_face": "front_face",
|
||||
"full_body": "full_body",
|
||||
"multi_angle": "multi_angle", # ★ 修复：原来错映射成 front_face（5 个角度全画正面）
|
||||
"expression": "expression",
|
||||
"scene": "scene",
|
||||
"grid": "grid",
|
||||
"sport_action": "scene",
|
||||
}
|
||||
mapped_variant = variant_map.get(variant_type, "front_face")
|
||||
mode = kwargs.get("mode", "standard")
|
||||
|
||||
expr = kwargs.get("expression_name", "")
|
||||
scene = kwargs.get("scene_name", "")
|
||||
if expr:
|
||||
self._system.set_expression(expr)
|
||||
if scene:
|
||||
self._system.set_scene(scene)
|
||||
|
||||
return self._system.build_prompt(mapped_variant, lang, mode=mode)
|
||||
|
||||
def get_available_variants(self) -> dict:
|
||||
"""兼容旧接口"""
|
||||
return self._system.get_variant_types()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# 测试
|
||||
system = CharacterPromptSystem()
|
||||
|
||||
# 逐层设置
|
||||
system.set_layer("L0_identity", {"name": "阿远", "gender": "男性", "age": "青年(18-30)", "ethnicity": "中国"})
|
||||
system.set_layer("L1_face_shape", {"shape": "国字脸", "jaw": "清晰分明"})
|
||||
system.set_layer("L2_face_eyes", {"shape": "单眼皮", "color": "深褐", "spirit": "锐利深邃"})
|
||||
system.set_layer("L2_face_eyebrows", {"shape": "剑眉", "state": "舒展自然"})
|
||||
system.set_layer("L2_face_nose", {"bridge": "高挺", "tip": "圆润饱满"})
|
||||
system.set_layer("L2_face_lips", {"shape": "微笑唇", "state": "嘴角微扬"})
|
||||
system.set_layer("L3_hair", {"style": "短发", "color": "黑色", "detail": "两侧收短顶部略长"})
|
||||
system.set_layer("L4_skin_makeup", {"skin_tone": "健康小麦", "skin_texture": "自然肌理"})
|
||||
system.set_layer("L5_body", {"build": "健壮", "shoulder": "宽肩", "posture": "挺拔"})
|
||||
system.set_layer("L6_clothing", {"top": "深灰色拉链运动夹克", "inner": "白色速干内衬", "accessories": "蓝色无线运动耳机(右耳)"})
|
||||
system.set_layer("L7_expression", {"base_mood": "沉稳内敛", "eye_spirit": "目光坚定直视前方"})
|
||||
system.set_layer("L8_lighting_camera", {"light_type": "柔光箱均匀布光", "color_temp": "暖色调(金色)"})
|
||||
system.set_layer("L9_background", {"bg_type": "纯色背景", "bg_color": "纯灰/专业"})
|
||||
|
||||
print("=== 正面面部特写 ===")
|
||||
print(system.build_prompt("front_face"))
|
||||
print("\n=== 网格定妆照 ===")
|
||||
print(system.build_prompt("grid", mode="standard"))
|
||||
print("\n=== 完整描述 ===")
|
||||
print(system.get_full_description())
|
||||
|
||||
print("\n=== 前端配置数据 ===")
|
||||
config = system.get_all_layers_config()
|
||||
for layer_id, cfg in config.items():
|
||||
print(f" {layer_id}: {cfg['icon']} {cfg['label']} ({len(cfg.get('fields',[]))} 字段)")
|
||||
+89
@@ -0,0 +1,89 @@
|
||||
# 角色定妆照提示词库
|
||||
|
||||
|
||||
### 夺
|
||||
- **描述**: 奇才
|
||||
- **面部**: 脸型=, 眼睛=, 发型=, 肤色=
|
||||
- **体型**: 体态=, 身高=
|
||||
- **性格**: 默认
|
||||
- **语音**: 默认 # 自动注册）
|
||||
|
||||
|
||||
|
||||
### 测试
|
||||
- **描述**: 测试角色描述
|
||||
- **面部**: 脸型=, 眼睛=, 发型=, 肤色=
|
||||
- **体型**: 体态=, 身高=
|
||||
- **性格**: 默认
|
||||
- **语音**: 默认 # 自动注册）
|
||||
|
||||
|
||||
|
||||
### 测试角色
|
||||
- **描述**: 测试方向描述
|
||||
- **面部**: 脸型=, 眼睛=, 发型=, 肤色=
|
||||
- **体型**: 体态=, 身高=
|
||||
- **性格**: 默认
|
||||
- **语音**: 默认 # 自动注册）
|
||||
|
||||
|
||||
|
||||
### 测试角色2
|
||||
- **描述**: 测试方向描述
|
||||
- **面部**: 脸型=, 眼睛=, 发型=, 肤色=
|
||||
- **体型**: 体态=, 身高=
|
||||
- **性格**: 默认
|
||||
- **语音**: 默认 # 自动注册）
|
||||
|
||||
|
||||
|
||||
### 中国
|
||||
- **描述**: 【身份】女性、青年、中国、张三
|
||||
【面部轮廓】适中自然、适中
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
【体型姿态】标准、挺拔
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景
|
||||
- **面部**: 脸型=, 眼睛=, 发型=【身份】女性、青年、中国、张三
|
||||
【面部轮廓】适中自然、适中
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
【体型姿态】标准、挺拔
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景, 肤色=健康小麦、自然肌理、红润健康、素颜
|
||||
【体型姿态】标准、挺拔
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景
|
||||
- **体型**: 体态=标准, 身高=
|
||||
- **性格**: 默认
|
||||
- **语音**: 默认 # 自动注册）
|
||||
|
||||
|
||||
|
||||
### 台湾
|
||||
- **描述**: 【身份】青年、中国、少女
|
||||
【面部轮廓】适中自然、适中
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
【体型姿态】标准、挺拔
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景
|
||||
- **面部**: 脸型=, 眼睛=, 发型=【身份】青年、中国、少女
|
||||
【面部轮廓】适中自然、适中
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
【体型姿态】标准、挺拔
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景, 肤色=健康小麦、自然肌理、红润健康、素颜
|
||||
【体型姿态】标准、挺拔
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景
|
||||
- **体型**: 体态=标准, 身高=
|
||||
- **性格**: 默认
|
||||
- **语音**: 默认 # 自动注册）
|
||||
|
||||
|
||||
|
||||
### test
|
||||
- **描述**: 【身份】男性
|
||||
【面部】标准
|
||||
- **面部**: 脸型=, 眼睛=, 发型=, 肤色=
|
||||
- **体型**: 体态=标准, 身高=
|
||||
- **性格**: 默认
|
||||
- **语音**: 默认 # 自动注册）
|
||||
|
||||
@@ -0,0 +1,361 @@
|
||||
"""
|
||||
character_generator/quality_check.py — 角色参考图「生成后质检」
|
||||
|
||||
## 为什么需要
|
||||
实测发现：角色参考图若带场景背景，图生视频时模型会把背景一起"绑定"到角色上
|
||||
→ 生成的画面环境错乱、跨镜不一致。**参考图必须是干净背景的定妆照**。
|
||||
|
||||
实测（2026-10-03，用户角色库）：
|
||||
害羞美女患者 边缘标准差 47.1 ❌ 有场景
|
||||
麻醉医生_帅气男 边缘标准差 52.3 ❌ 有场景
|
||||
找回的2张生成图 58.4 / 51.2 ❌ 有场景
|
||||
中国 16.8 较干净
|
||||
|
||||
## 判据
|
||||
取图片**四条边缘**的像素，统计颜色标准差：
|
||||
· 纯色背景 → 边缘高度一致 → 标准差小（< 12）
|
||||
· 带场景 → 边缘颜色杂乱 → 标准差大（> 25）
|
||||
|
||||
无需 AI 模型，纯本地计算（PIL），零成本。
|
||||
|
||||
## 用法
|
||||
from character_generator.quality_check import check_reference_quality
|
||||
r = check_reference_quality("/path/to/ref.png")
|
||||
# {"ok": bool, "score": 0-100, "edge_std": float, "issues": [...], "suggestion": str}
|
||||
"""
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from PIL import Image, ImageStat
|
||||
except ImportError: # pragma: no cover
|
||||
Image = None
|
||||
|
||||
# 阈值（实测校准）
|
||||
EDGE_CLEAN = 12.0 # 标准差 < 12 → 干净（纯色背景）
|
||||
EDGE_ACCEPT = 25.0 # < 25 → 可接受；≥ 25 → 判定有场景
|
||||
MIN_SIDE = 512 # 短边最小像素（定妆照清晰度下限）
|
||||
|
||||
|
||||
def check_reference_quality(path: str) -> dict:
|
||||
"""检查角色参考图是否合格（背景是否干净 + 分辨率）
|
||||
|
||||
返回：
|
||||
ok bool 是否合格
|
||||
score int 0-100（越高越干净）
|
||||
edge_std float 边缘颜色标准差（越小越干净）
|
||||
width/height int
|
||||
issues list[str]
|
||||
suggestion str 不合格时的修改建议
|
||||
"""
|
||||
p = Path(path)
|
||||
if not p.exists():
|
||||
return {"ok": False, "score": 0, "edge_std": -1, "issues": [f"文件不存在: {path}"],
|
||||
"suggestion": "重新生成或重新上传参考图"}
|
||||
|
||||
if Image is None:
|
||||
return {"ok": True, "score": 60, "edge_std": -1,
|
||||
"issues": ["未安装 Pillow，跳过背景检测（pip install pillow）"],
|
||||
"suggestion": ""}
|
||||
|
||||
try:
|
||||
im = Image.open(p).convert("RGB")
|
||||
except Exception as e:
|
||||
return {"ok": False, "score": 0, "edge_std": -1,
|
||||
"issues": [f"图片无法解码: {e}"], "suggestion": "重新上传标准 jpg/png"}
|
||||
|
||||
w, h = im.size
|
||||
# ⚠️ 用**四角**采样（而非四条边）：
|
||||
# 半身像/人像的人物常顶到上下边缘，用整边采样必然采到人物 → 误判"背景脏"。
|
||||
# 四角是纯背景概率最高的区域（实测：竖屏半身像整边 29.2 → 四角 6~12）
|
||||
band = max(6, min(w, h) // 18)
|
||||
pad = max(2, band // 6)
|
||||
corners = [
|
||||
(pad, pad, pad + band, pad + band), # 左上
|
||||
(w - pad - band, pad, w - pad, pad + band), # 右上
|
||||
(pad, h - pad - band, pad + band, h - pad), # 左下
|
||||
(w - pad - band, h - pad - band, w - pad, h - pad), # 右下
|
||||
]
|
||||
stds = []
|
||||
for box in corners:
|
||||
try:
|
||||
st = ImageStat.Stat(im.crop(box))
|
||||
stds.append(sum(st.stddev) / 3.0)
|
||||
except Exception:
|
||||
continue
|
||||
edge_std = round(sum(stds) / len(stds), 1) if stds else -1.0
|
||||
|
||||
# 四角平均色的通道离散度（辅助判断"是否纯色背景"）
|
||||
try:
|
||||
mean_of_edges = [0.0, 0.0, 0.0]
|
||||
for box in corners:
|
||||
m = ImageStat.Stat(im.crop(box)).mean
|
||||
for i in range(3):
|
||||
mean_of_edges[i] += m[i] / len(corners)
|
||||
chan_spread = round(max(mean_of_edges) - min(mean_of_edges), 1)
|
||||
except Exception:
|
||||
chan_spread = 0.0
|
||||
|
||||
issues: list[str] = []
|
||||
if min(w, h) < MIN_SIDE:
|
||||
issues.append(f"分辨率偏低（{w}×{h}，短边建议 ≥ {MIN_SIDE}px）")
|
||||
if edge_std >= EDGE_ACCEPT:
|
||||
issues.append(f"背景不干净（边缘颜色标准差 {edge_std}，≥{EDGE_ACCEPT} 判定含场景元素）")
|
||||
elif edge_std >= EDGE_CLEAN:
|
||||
issues.append(f"背景不够纯净（边缘标准差 {edge_std}，建议 < {EDGE_CLEAN}）")
|
||||
|
||||
# 打分：边缘越干净分越高
|
||||
if edge_std < 0:
|
||||
score = 60
|
||||
else:
|
||||
score = int(max(0, min(100, 100 - (edge_std / EDGE_ACCEPT) * 100))) if edge_std < EDGE_ACCEPT \
|
||||
else int(max(0, 40 - (edge_std - EDGE_ACCEPT)))
|
||||
|
||||
ok = not any(i.startswith("背景不干净") for i in issues) and min(w, h) >= MIN_SIDE
|
||||
|
||||
suggestion = ""
|
||||
if not ok:
|
||||
if edge_std >= EDGE_ACCEPT:
|
||||
suggestion = (
|
||||
"参考图带场景背景 → 图生视频时模型会把背景绑定到角色上。"
|
||||
"建议：用「标准定妆照」模式重新生成（纯白无缝背景 + 多视角网格），"
|
||||
"或在生成器的「🌄 背景与场景」里把环境/季节清空、背景类型选「纯色背景」。"
|
||||
)
|
||||
else:
|
||||
suggestion = "建议提高分辨率或改用纯色背景的定妆照。"
|
||||
|
||||
return {
|
||||
"ok": ok,
|
||||
"score": score,
|
||||
"edge_std": edge_std,
|
||||
"chan_spread": chan_spread,
|
||||
"width": w,
|
||||
"height": h,
|
||||
"issues": issues,
|
||||
"suggestion": suggestion,
|
||||
}
|
||||
|
||||
|
||||
def is_clean_reference(path: str) -> bool:
|
||||
"""便捷判定：参考图是否干净可用"""
|
||||
return bool(check_reference_quality(path).get("ok"))
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# ★ 参考图「白底清理」（后处理）—— 消除场景背景
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 外网参考（muse-character-sheet / style-consistency-ai）：
|
||||
# "每个姿势都要跑一遍白底清理（ComfyUI-RMBG）" ——
|
||||
# 后处理抠图换底，比指望模型自己画干净背景**可靠得多**。
|
||||
#
|
||||
# 实现：rembg（本地 U2Net）+ 纯色底合成，零 API 成本。
|
||||
# 模型（~176MB）首次自动下载；不可用时优雅降级（不阻塞流程）。
|
||||
|
||||
_BG_COLORS = {
|
||||
"cool_grey": (132, 138, 145), # 研究结论：冷灰优于纯白（白底会削平高光）
|
||||
"white": (255, 255, 255),
|
||||
"light_grey": (225, 227, 230),
|
||||
}
|
||||
|
||||
_rembg_available = None
|
||||
_rembg_error = ""
|
||||
|
||||
|
||||
def _check_rembg() -> tuple:
|
||||
"""检查 rembg 是否可用（惰性 + 结果缓存）"""
|
||||
global _rembg_available, _rembg_error
|
||||
if _rembg_available is not None:
|
||||
return _rembg_available, _rembg_error
|
||||
try:
|
||||
from rembg import remove # noqa: F401
|
||||
_rembg_available = True
|
||||
except Exception as e:
|
||||
_rembg_available = False
|
||||
_rembg_error = f"rembg 不可用: {e}（pip install rembg onnxruntime）"
|
||||
return _rembg_available, _rembg_error
|
||||
|
||||
|
||||
# 模型文件的最小字节数（判断下载是否完整）
|
||||
_MODEL_REQ = {"u2net": 50_000_000, "isnet-general-use": 40_000_000, "u2netp": 3_000_000}
|
||||
|
||||
|
||||
def pick_rembg_model() -> str:
|
||||
"""选一个**已就绪**的抠图模型（大模型优先，回退小模型）
|
||||
|
||||
· u2net ≈176MB（效果好）
|
||||
· u2netp ≈4.7MB（快，效果略差）—— 网络受限时先下小的即可用起来
|
||||
返回 "" 表示没有可用模型（调用方跳过清理）。
|
||||
"""
|
||||
home = Path.home() / ".u2net"
|
||||
for name in ("u2net", "isnet-general-use", "u2netp"):
|
||||
f = home / f"{name}.onnx"
|
||||
try:
|
||||
if f.exists() and f.stat().st_size >= _MODEL_REQ.get(name, 1_000_000):
|
||||
return name
|
||||
except Exception:
|
||||
continue
|
||||
return ""
|
||||
|
||||
|
||||
def clean_reference_background(src_path: str, dst_path: str = "",
|
||||
bg: str = "cool_grey", keep_alpha: bool = False) -> dict:
|
||||
"""把参考图抠出来合成到纯色背景（消除场景背景）
|
||||
|
||||
这是**后处理**方案 —— 比指望模型生成干净背景更可靠。
|
||||
⚠️ 首次调用会下载 U2Net 模型（~176MB），耗时较长；之后走缓存。
|
||||
|
||||
参数：
|
||||
bg cool_grey（推荐）| white | light_grey
|
||||
keep_alpha 保留透明通道（不合成背景，输出 PNG）
|
||||
返回：
|
||||
{ok, output, before, after, improved, skipped, reason}
|
||||
· skipped=True 表示环境不具备（**不阻塞流程**，调用方直接用原图即可）
|
||||
"""
|
||||
ok_rb, err = _check_rembg()
|
||||
if not ok_rb:
|
||||
return {"ok": False, "skipped": True, "reason": err, "output": src_path}
|
||||
|
||||
model_name = pick_rembg_model()
|
||||
if not model_name:
|
||||
return {"ok": False, "skipped": True,
|
||||
"reason": "抠图模型未就绪（需 u2net.onnx 或 u2netp.onnx 放入 ~/.u2net/）",
|
||||
"output": src_path, "model": ""}
|
||||
|
||||
src = Path(src_path)
|
||||
if not src.exists():
|
||||
return {"ok": False, "skipped": True, "reason": "源文件不存在", "output": src_path}
|
||||
|
||||
dst = Path(dst_path) if dst_path else src.with_name(f"{src.stem}_clean.png")
|
||||
before = check_reference_quality(str(src))
|
||||
try:
|
||||
from rembg import remove, new_session
|
||||
from PIL import Image
|
||||
im = Image.open(src).convert("RGB")
|
||||
_session = new_session(model_name) # ★ 用选中的模型（u2net / u2netp）
|
||||
cut = remove(im, session=_session) # → RGBA
|
||||
if keep_alpha:
|
||||
cut.save(dst)
|
||||
else:
|
||||
color = _BG_COLORS.get(bg) or _BG_COLORS["cool_grey"]
|
||||
canvas = Image.new("RGB", cut.size, color)
|
||||
canvas.paste(cut, (0, 0), cut if cut.mode in ("RGBA", "LA") else None)
|
||||
canvas.save(dst)
|
||||
after = check_reference_quality(str(dst))
|
||||
improved = (before.get("edge_std", 999) - after.get("edge_std", 999))
|
||||
return {
|
||||
"ok": True, "output": str(dst),
|
||||
"before": {"edge_std": before.get("edge_std"), "score": before.get("score"),
|
||||
"ok": before.get("ok")},
|
||||
"after": {"edge_std": after.get("edge_std"), "score": after.get("score"),
|
||||
"ok": after.get("ok")},
|
||||
"improved": round(improved, 1),
|
||||
}
|
||||
except Exception as e:
|
||||
return {"ok": False, "skipped": False, "reason": f"抠图失败: {str(e)[:200]}",
|
||||
"output": src_path, "before": {"edge_std": before.get("edge_std")}}
|
||||
|
||||
|
||||
# ── CLI：批量体检角色库 ──
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
from assets_manager import AssetStore
|
||||
|
||||
st = AssetStore("characters")
|
||||
items = st.list()
|
||||
print(f"=== 角色参考图质检（{len(items)} 个角色）===\n")
|
||||
bad = 0
|
||||
for it in items:
|
||||
cid = it.get("id")
|
||||
refs = it.get("reference_images") or {}
|
||||
if not refs:
|
||||
print(f" ⚠️ {cid:16s} 无参考图")
|
||||
bad += 1
|
||||
continue
|
||||
for slot, fname in refs.items():
|
||||
p = Path(st.kind_dir(cid)) / Path(str(fname)).name
|
||||
r = check_reference_quality(str(p))
|
||||
mark = "✅" if r["ok"] else "❌"
|
||||
print(f" {mark} {cid:16s} [{slot:8s}] 分 {r['score']:3d} | 边缘标准差 {r['edge_std']:5.1f} | {r['width']}x{r['height']}")
|
||||
for i in r["issues"]:
|
||||
print(f" · {i}")
|
||||
if not r["ok"]:
|
||||
bad += 1
|
||||
print(f"\n不合格 {bad} 张 —— 建议重新生成（定妆照模式）")
|
||||
|
||||
# ── 网格定妆照专用质检（★ 按格检查）────────────────────────────
|
||||
# 为什么单写一个：整图边缘检测对**多格图无效** ——
|
||||
# 格子越多，图片边缘越必然包含相邻格的内容，导致误判为"脏背景"。
|
||||
# 实测教训（2026-10-03）：三格版 14.2 ✅ / 四格版 50.1 ❌，
|
||||
# 两者提示词质量相近，差异主要来自"格数影响了边缘采样"。
|
||||
|
||||
def check_sheet_quality(path: str, grid: tuple = None) -> dict:
|
||||
"""网格定妆照质检：**逐格检查背景**（每格取四角），而非整图边缘
|
||||
|
||||
⚠️ **只适用于真正的网格图**（多格并排）。判据与方法必须配套：
|
||||
· 单张人像图 → 用 check_reference_quality()（整图边缘）
|
||||
· 网格图 → 用本函数（按格检查）
|
||||
两者不可混用来比较不同格式的方案（实测教训，2026-10-03）。
|
||||
|
||||
grid: (cols, rows) —— **强烈建议显式传入**；不传会按宽高比猜（可能猜错，
|
||||
把单图当成网格切开，导致误判为"干净"）
|
||||
返回 {ok, score, grid, median_cell_std, worst_cell_std, per_cell, issues}
|
||||
"""
|
||||
p = Path(path)
|
||||
if not p.exists() or Image is None:
|
||||
return {"ok": False, "score": 0, "median_cell_std": -1, "worst_cell_std": -1,
|
||||
"per_cell": [], "issues": ["文件不存在或缺少 Pillow"]}
|
||||
|
||||
try:
|
||||
im = Image.open(p).convert("RGB")
|
||||
except Exception as e:
|
||||
return {"ok": False, "score": 0, "median_cell_std": -1, "worst_cell_std": -1,
|
||||
"per_cell": [], "issues": [f"无法解码: {e}"]}
|
||||
|
||||
w, h = im.size
|
||||
if grid is None:
|
||||
ar = w / max(1, h)
|
||||
if ar >= 1.7:
|
||||
grid = (4, 1)
|
||||
elif ar >= 1.4:
|
||||
grid = (3, 1)
|
||||
elif ar >= 1.15:
|
||||
grid = (2, 1)
|
||||
else:
|
||||
grid = (2, 2)
|
||||
cols, rows = grid
|
||||
cw, ch = w // cols, h // rows
|
||||
band = max(3, min(cw, ch) // 40)
|
||||
|
||||
per_cell = []
|
||||
for r in range(rows):
|
||||
for c in range(cols):
|
||||
x0, y0 = c * cw, r * ch
|
||||
pad = max(2, band // 2)
|
||||
corners = [
|
||||
(x0 + pad, y0 + pad, x0 + pad + band, y0 + pad + band),
|
||||
(x0 + cw - pad - band, y0 + pad, x0 + cw - pad, y0 + pad + band),
|
||||
(x0 + pad, y0 + ch - pad - band, x0 + pad + band, y0 + ch - pad),
|
||||
(x0 + cw - pad - band, y0 + ch - pad - band, x0 + cw - pad, y0 + ch - pad),
|
||||
]
|
||||
stds = []
|
||||
for box in corners:
|
||||
try:
|
||||
st = ImageStat.Stat(im.crop(box))
|
||||
stds.append(sum(st.stddev) / 3.0)
|
||||
except Exception:
|
||||
continue
|
||||
cell_std = round(sum(stds) / len(stds), 1) if stds else -1.0
|
||||
per_cell.append({"cell": f"r{r+1}c{c+1}", "edge_std": cell_std})
|
||||
|
||||
valid = [c["edge_std"] for c in per_cell if c["edge_std"] >= 0]
|
||||
srt = sorted(valid)
|
||||
med = srt[len(srt) // 2] if srt else -1.0
|
||||
worst = max(valid) if valid else -1.0
|
||||
ok = med < EDGE_ACCEPT
|
||||
score = int(max(0, min(100, 100 - (med / EDGE_ACCEPT) * 100))) if med >= 0 else 0
|
||||
issues = []
|
||||
if not ok:
|
||||
issues.append(f"多数格的背景不干净（各格中位标准差 {med}，阈值 <{EDGE_ACCEPT}）")
|
||||
return {"ok": ok, "score": score, "grid": grid,
|
||||
"median_cell_std": med, "worst_cell_std": worst, "per_cell": per_cell,
|
||||
"issues": issues}
|
||||
@@ -0,0 +1,351 @@
|
||||
"""
|
||||
变体生成器 (VariantGenerator) — 按变体类型生成角色图像
|
||||
|
||||
使用 Kling API 批量生成各类变体图像：
|
||||
- front_face: 正面高清面部特写
|
||||
- full_body: 全身正面
|
||||
- multi_angle: 多角度全身（front/side/right_three_quarter/right_side/back）
|
||||
- expression: 表情变体（neutral/smile/focused/laugh/serious）
|
||||
- scene: 场景变体
|
||||
- sport_action: 运动姿态
|
||||
- grid: 网格定妆照（标准2×3/运动4×2）
|
||||
|
||||
每张生成结果以 {role_name}_{variant_type}_{variant_option}.png 命名和缓存
|
||||
"""
|
||||
|
||||
import json
|
||||
import hashlib
|
||||
import time
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
path = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, path)
|
||||
|
||||
from lib.config import load_config
|
||||
from lib.logger import get_logger
|
||||
from character_generator.prompt_assembler import PromptAssembler
|
||||
|
||||
logger = get_logger('variant_generator')
|
||||
|
||||
environ = os.environ.get
|
||||
home = os.path.expanduser('~')
|
||||
|
||||
# 缓存目录
|
||||
CACHE_DIR = environ(
|
||||
'AVE_CACHE_DIR',
|
||||
os.path.join(home, 'workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait'),
|
||||
)
|
||||
os.makedirs(CACHE_DIR, exist_ok=True)
|
||||
|
||||
# Kling API 基础 URL
|
||||
KLING_BASE = 'https://api-beijing.klingai.com'
|
||||
|
||||
|
||||
class VariantGenerator:
|
||||
"""使用 Kling API 批量生成角色变体图像。"""
|
||||
|
||||
def __init__(self):
|
||||
self._assembler = PromptAssembler()
|
||||
self._cfg = load_config()
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# 主入口
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def generate(
|
||||
self,
|
||||
attrs: dict,
|
||||
variant_type: str,
|
||||
variant_option: str,
|
||||
lang: str = 'zh',
|
||||
force: bool = False,
|
||||
seed: int = 0,
|
||||
) -> dict:
|
||||
"""生成指定变体类型的图像。
|
||||
|
||||
Args:
|
||||
attrs: 结构化属性字典
|
||||
variant_type: 变体类型 (front_face, full_body, multi_angle, expression, ...)
|
||||
variant_option: 变体选项（如 angle_name, expression_name 等）
|
||||
lang: 语言
|
||||
force: 强制重新生成
|
||||
seed: 固定种子
|
||||
|
||||
Returns:
|
||||
{"path": str, "url": str, "seed": int, "prompt": str, "cached": bool}
|
||||
"""
|
||||
# 组装提示词
|
||||
prompt = self._assembler.build(variant_type, variant_option, attrs)
|
||||
|
||||
# 生成缓存 key
|
||||
opt_suffix = variant_option.replace(' ', '_')
|
||||
cache_key = f'{attrs.get("name", "char")}_{variant_type}_{opt_suffix}'
|
||||
output_path = os.path.join(CACHE_DIR, cache_key)
|
||||
Path(output_path).mkdir(parents=True, exist_ok=True)
|
||||
|
||||
filename = f'{cache_key}_{seed if seed else "rand"}.png'
|
||||
filepath = os.path.join(output_path, filename)
|
||||
|
||||
# 如果缓存命中且不强制，直接返回
|
||||
if not force and os.path.exists(filepath):
|
||||
logger.info(f' 缓存命中: {filepath}')
|
||||
return {'path': filepath, 'url': '', 'seed': seed, 'prompt': prompt, 'cached': True}
|
||||
|
||||
# 调用 Kling API
|
||||
result = self._generate_kling(prompt, filepath, seed)
|
||||
result['prompt'] = prompt
|
||||
return result
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# 批量生成
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def generate_all_angles(
|
||||
self,
|
||||
attrs: dict,
|
||||
lang: str = 'zh',
|
||||
force: bool = False,
|
||||
seed: int = 0,
|
||||
) -> dict:
|
||||
"""生成所有角度的全身图。"""
|
||||
angles = ('front', 'side', 'right_three_quarter', 'right_side', 'back')
|
||||
results = {}
|
||||
for angle in angles:
|
||||
results[angle] = self.generate(attrs, 'multi_angle', angle, lang, force, seed)
|
||||
return results
|
||||
|
||||
def generate_all_expressions(
|
||||
self,
|
||||
attrs: dict,
|
||||
lang: str = 'zh',
|
||||
force: bool = False,
|
||||
seed: int = 0,
|
||||
) -> dict:
|
||||
"""生成所有表情变体。"""
|
||||
expressions = ('neutral', 'smile', 'focused', 'laugh', 'serious')
|
||||
results = {}
|
||||
for expr in expressions:
|
||||
results[expr] = self.generate(attrs, 'expression', expr, lang, force, seed)
|
||||
return results
|
||||
|
||||
def generate_baseline_portrait(
|
||||
self,
|
||||
attrs: dict,
|
||||
lang: str = 'zh',
|
||||
force: bool = False,
|
||||
seed: int = 0,
|
||||
) -> dict:
|
||||
"""生成正面基准肖像（用于 face_anchor）。"""
|
||||
return self.generate(attrs, 'front_face', 'baseline', lang, force, seed)
|
||||
|
||||
def generate_grid_sheet(
|
||||
self,
|
||||
attrs: dict,
|
||||
mode: str = 'standard',
|
||||
lang: str = 'zh',
|
||||
force: bool = False,
|
||||
seed: int = 0,
|
||||
ref_image: str = '',
|
||||
prompt: str = '',
|
||||
) -> dict:
|
||||
"""生成网格定妆照。
|
||||
|
||||
Args:
|
||||
attrs: 结构化属性
|
||||
mode: 网格模式 (standard=2x3, sport=4x2)
|
||||
lang: 语言
|
||||
force: 强制重新生成
|
||||
seed: 固定种子
|
||||
ref_image: 参考图路径（可选）
|
||||
prompt: 手动提示词（可选，为空则自动组装）
|
||||
"""
|
||||
if not prompt:
|
||||
prompt = self._assembler.build('grid', mode, attrs, lang=lang)
|
||||
|
||||
name = attrs.get('name', 'char')
|
||||
output_path = os.path.join(CACHE_DIR, 'grid', name)
|
||||
Path(output_path).mkdir(parents=True, exist_ok=True)
|
||||
|
||||
filename = f'grid_{name}_{mode}_{seed if seed else "rand"}.png'
|
||||
filepath = os.path.join(output_path, filename)
|
||||
|
||||
if not force and os.path.exists(filepath):
|
||||
logger.info(f' 网格缓存命中: {filepath}')
|
||||
return {'path': filepath, 'url': '', 'seed': seed, 'prompt': prompt, 'cached': True}
|
||||
|
||||
result = self._generate_kling(prompt, filepath, seed, ref_image)
|
||||
result['prompt'] = prompt
|
||||
return result
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# 辅助
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def _get_option_key(self, variant_type: str) -> str:
|
||||
"""将变体类型映射到配置中的选项 key。"""
|
||||
mapping = {
|
||||
'multi_angle': 'angles',
|
||||
'expression': 'expressions',
|
||||
'scene': 'scenes',
|
||||
'sport_action': 'actions',
|
||||
'outfit': 'outfits',
|
||||
'grid': 'grid_modes',
|
||||
}
|
||||
return mapping.get(variant_type, variant_type)
|
||||
|
||||
# ══════════════════════════════════════════════════
|
||||
# Kling API 底层
|
||||
# ══════════════════════════════════════════════════
|
||||
|
||||
def _generate_kling(
|
||||
self,
|
||||
prompt: str,
|
||||
output_path: str,
|
||||
seed: int = 0,
|
||||
ref_image: str = '',
|
||||
) -> dict:
|
||||
"""调用 Kling API 生成图像。
|
||||
|
||||
支持 T2I（无 ref_image）和 I2I（有 ref_image）。
|
||||
|
||||
Args:
|
||||
prompt: 生成提示词
|
||||
output_path: 输出文件路径
|
||||
seed: 固定种子（0 = 随机）
|
||||
ref_image: 参考图路径（可选，走 I2I）
|
||||
|
||||
Returns:
|
||||
{"path": str, "seed": int, "filename": str}
|
||||
失败时返回 {"failed": True, "error": str}
|
||||
"""
|
||||
import httpx
|
||||
|
||||
# 加载 Kling 配置
|
||||
kling_cfg = self._cfg.get('kling', {})
|
||||
ak = kling_cfg.get('access_key', environ('KLING_ACCESS_KEY', ''))
|
||||
sk = kling_cfg.get('secret_key', environ('KLING_SECRET_KEY', ''))
|
||||
|
||||
if not ak or not sk:
|
||||
raise ValueError('缺少 Kling API 配置 (kling.access_key / kling.secret_key)')
|
||||
|
||||
# 生成 JWT token
|
||||
import jwt
|
||||
now = int(time.time())
|
||||
token = jwt.encode(
|
||||
{'iss': ak, 'exp': now + 1800, 'nbf': now},
|
||||
sk,
|
||||
algorithm='HS256',
|
||||
)
|
||||
headers = {
|
||||
'Authorization': f'Bearer {token}',
|
||||
'Content-Type': 'application/json',
|
||||
}
|
||||
|
||||
# 判断是否用参考图（I2I）
|
||||
use_ref = bool(ref_image and os.path.exists(ref_image))
|
||||
|
||||
if use_ref:
|
||||
with open(ref_image, 'rb') as f:
|
||||
import base64
|
||||
img_b64 = base64.b64encode(f.read()).decode('utf-8')
|
||||
|
||||
payload = {
|
||||
'model': 'kling-v1',
|
||||
'prompt': prompt,
|
||||
'image': img_b64,
|
||||
'strength': 0.65,
|
||||
'aspect_ratio': '9:16',
|
||||
'n': 1,
|
||||
}
|
||||
else:
|
||||
payload = {
|
||||
'model': 'kling-v3',
|
||||
'prompt': prompt,
|
||||
'aspect_ratio': '9:16',
|
||||
'n': 1,
|
||||
}
|
||||
|
||||
if seed:
|
||||
payload['seed'] = seed
|
||||
|
||||
# 提交任务
|
||||
for attempt in range(3):
|
||||
try:
|
||||
resp = httpx.post(
|
||||
f'{KLING_BASE}/v1/images/generations',
|
||||
headers=headers,
|
||||
json=payload,
|
||||
timeout=30,
|
||||
)
|
||||
if resp.status_code == 429:
|
||||
wait = 15 * (attempt + 1)
|
||||
logger.warning(f' 429 限流，等待 {wait}s...')
|
||||
time.sleep(wait)
|
||||
continue
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
task_id = data.get('data', {}).get('task_id', '')
|
||||
if not task_id:
|
||||
raise RuntimeError(f'Kling 提交失败: {data}')
|
||||
break
|
||||
except Exception as e:
|
||||
if attempt == 2:
|
||||
raise
|
||||
time.sleep(5)
|
||||
else:
|
||||
raise RuntimeError('Kling API 提交失败（重试耗尽）')
|
||||
|
||||
# 轮询任务状态
|
||||
for i in range(40):
|
||||
time.sleep(5)
|
||||
try:
|
||||
q = httpx.get(
|
||||
f'{KLING_BASE}/v1/images/generations/{task_id}',
|
||||
headers=headers,
|
||||
timeout=30,
|
||||
)
|
||||
q.raise_for_status()
|
||||
status_data = q.json()['data']
|
||||
task_status = status_data.get('task_status', '')
|
||||
|
||||
if task_status == 'succeed':
|
||||
images = status_data.get('task_result', {}).get('images', [])
|
||||
if not images:
|
||||
raise RuntimeError('Kling 生成成功但无图像')
|
||||
img_url = images[0].get('url', '')
|
||||
returned_seed = images[0].get('seed', seed)
|
||||
|
||||
# 下载图像
|
||||
img_resp = httpx.get(img_url, timeout=30)
|
||||
img_resp.raise_for_status()
|
||||
|
||||
Path(os.path.dirname(output_path)).mkdir(parents=True, exist_ok=True)
|
||||
with open(output_path, 'wb') as f:
|
||||
f.write(img_resp.content)
|
||||
|
||||
logger.info(f' ✅ Kling 生成完成: {os.path.basename(output_path)} [seed={returned_seed}]')
|
||||
return {
|
||||
'path': output_path,
|
||||
'seed': returned_seed,
|
||||
'filename': os.path.basename(output_path),
|
||||
}
|
||||
|
||||
elif task_status == 'failed':
|
||||
reason = status_data.get('fail_reason', '未知错误')
|
||||
raise RuntimeError(f'Kling 生成失败: {reason}')
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f' 轮询异常: {e}')
|
||||
time.sleep(5)
|
||||
|
||||
raise TimeoutError('Kling 生成超时（200s）')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
gen = VariantGenerator()
|
||||
print('变体生成器初始化完成')
|
||||
available = gen.get_available_variants()
|
||||
print(f'可用变体: {list(available.keys())}')
|
||||
@@ -0,0 +1,790 @@
|
||||
"""
|
||||
AVE character_portrait — 逐帧参考链定妆照生成（v2.0 重写）
|
||||
|
||||
核心原则:
|
||||
⚠️ 前一轮的问题：所有角度共用同一张基准图作 I2I 参考 → 角色漂移 + 方向混乱
|
||||
✅ 本方案：逐帧参考链（Sequential Reference Chain）
|
||||
图1(正面T2I) → 作为图2参考 → 图2(右半侧I2I) → 作为图3参考 → 图3(右侧面I2I) → 作为图4参考 → 图4(背面I2I)
|
||||
每一步只做"小幅姿态变化"，最大限度保持角色一致性
|
||||
|
||||
流程:
|
||||
Phase 1: Kling T2I (kling-v3, 高质量) 生成正面全身基准图
|
||||
Phase 2: 逐帧 I2I 参考链，小步幅递增
|
||||
图1(正面) → 图2(右侧45°半侧面, I2I参考图1)
|
||||
图2(右侧45°) → 图3(纯右侧面, I2I参考图2)
|
||||
图3(右侧面) → 图4(背面, I2I参考图3)
|
||||
Phase 3: 注册到 CharacterRegistry（face_anchor + 各角度图）
|
||||
|
||||
参考强度逐级递增：0.70 → 0.75 → 0.70（近端参考 = 更高一致性）
|
||||
|
||||
用法:
|
||||
python character_portrait.py --character "陆晨晖" --desc "..."
|
||||
|
||||
API调用:
|
||||
POST /api/characters/generate-portrait
|
||||
{"character_name": "陆晨晖", "description": "...", "force": false}
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import json
|
||||
import time
|
||||
import base64
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
|
||||
path = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, path)
|
||||
|
||||
from lib.config import load_config
|
||||
from lib.logger import get_logger
|
||||
|
||||
logger = get_logger('character_portrait')
|
||||
|
||||
environ = os.environ.get
|
||||
home = os.path.expanduser('~')
|
||||
|
||||
# ── 缓存目录 ──
|
||||
CACHE_DIR = environ(
|
||||
'AVE_CACHE_DIR',
|
||||
os.path.join(home, 'workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait'),
|
||||
)
|
||||
os.makedirs(CACHE_DIR, exist_ok=True)
|
||||
|
||||
# ── Kling API ──
|
||||
KLING_BASE = 'https://api-beijing.klingai.com'
|
||||
BASELINE_MODEL = 'kling-v3'
|
||||
ANGLE_MODEL = 'kling-v1'
|
||||
DEFAULT_SEED = 42
|
||||
|
||||
# Phase 2 参考链强度（逐级递增保持一致性）
|
||||
REF_STRENGTH_CHAIN = (0.70, 0.75, 0.70)
|
||||
|
||||
# Phase 1: 基准图提示词模板
|
||||
BASELINE_PROMPT_TEMPLATE = (
|
||||
'一张超写实全身人像定妆照，必须正面朝向镜头，如证件照构图。'
|
||||
'双脚自然分开与肩同宽，双手自然垂于身体两侧，面部正对相机，双耳左右对称可见。'
|
||||
'严禁身体转向侧面或半侧面，不得有角度偏移。'
|
||||
'{desc}'
|
||||
'干净的浅灰色无缝影棚背景，左右双柔光箱对称布光，面部无阴影，'
|
||||
'超高清8k，真实皮肤纹理，织物细节，发丝清晰可见，专业人像摄影质感。'
|
||||
)
|
||||
|
||||
# Phase 2: 逐帧参考链定义
|
||||
# 每帧: key(存储key), label(显示名), ref_source(参考图来源), strength(I2I强度), prompt(生成提示词)
|
||||
ANGLE_CHAIN = [
|
||||
{
|
||||
'key': 'right_three_quarter',
|
||||
'label': '右侧45°半侧面',
|
||||
'ref_source': 'baseline',
|
||||
'strength': 0.70,
|
||||
'prompt': (
|
||||
'与参考图完全相同的人物。全身镜头，四分之三右侧面，身体转向右侧约45度。'
|
||||
'面部朝向镜头右前方，展现右侧面部特征。站姿放松自然。'
|
||||
'保持完全一致的着装、配饰、发型和面部特征。'
|
||||
'浅灰色影棚背景，超写实，专业人像布光，真实皮肤纹理，高清细节。'
|
||||
),
|
||||
},
|
||||
{
|
||||
'key': 'right_side',
|
||||
'label': '纯右侧面',
|
||||
'ref_source': 'right_three_quarter',
|
||||
'strength': 0.75,
|
||||
'prompt': (
|
||||
'与参考图完全相同的人物。全身镜头，纯右侧面视角，人物面向右侧90度。'
|
||||
'镜头从人物正右侧拍摄，展现鼻梁、嘴唇和下颌线的侧面轮廓。'
|
||||
'着装与参考图完全一致，发型侧面层次清晰可见。'
|
||||
'浅灰色影棚背景，超写实，轮廓清晰，高清，侧光勾勒面部轮廓。'
|
||||
),
|
||||
},
|
||||
{
|
||||
'key': 'back',
|
||||
'label': '背面',
|
||||
'ref_source': 'right_side',
|
||||
'strength': 0.70,
|
||||
'prompt': (
|
||||
'与参考图完全相同的人物。全身镜头，人物背对镜头，从正后方拍摄。'
|
||||
'展现后脑勺的发型层次感，肩背和腰线的服装贴合线条。'
|
||||
'从背面可见服装背面的褶皱纹理和鞋子的后跟细节。'
|
||||
'浅灰色影棚背景，超写实，真实皮肤纹理，高清8k。'
|
||||
),
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════
|
||||
# Kling 辅助工具
|
||||
# ══════════════════════════════════════════════════════
|
||||
|
||||
def _get_kling_credentials() -> dict:
|
||||
"""从 AVE 配置加载 Kling API 凭据。
|
||||
|
||||
Returns:
|
||||
{"access_key": str, "secret_key": str}
|
||||
|
||||
Raises:
|
||||
ValueError: 缺少配置
|
||||
"""
|
||||
cfg = load_config()
|
||||
kling_cfg = cfg.get('kling', {})
|
||||
ak = kling_cfg.get('access_key', '')
|
||||
sk = kling_cfg.get('secret_key', '')
|
||||
if not ak or not sk:
|
||||
raise ValueError('缺少 Kling API 配置 (kling.access_key / kling.secret_key)')
|
||||
return {'access_key': ak, 'secret_key': sk}
|
||||
|
||||
|
||||
def _jwt_token(ak: str, sk: str) -> str:
|
||||
"""生成 Kling JWT 认证 token。
|
||||
|
||||
Args:
|
||||
ak: Access Key
|
||||
sk: Secret Key
|
||||
|
||||
Returns:
|
||||
JWT token 字符串
|
||||
"""
|
||||
try:
|
||||
import jwt as pyjwt
|
||||
except ImportError:
|
||||
raise ImportError('需要 PyJWT 库: pip install pyjwt')
|
||||
now = int(time.time())
|
||||
return pyjwt.encode(
|
||||
{'iss': ak, 'exp': now + 1800, 'nbf': now},
|
||||
sk,
|
||||
algorithm='HS256',
|
||||
)
|
||||
|
||||
|
||||
def _download(url: str, path: str):
|
||||
"""下载文件。
|
||||
|
||||
Args:
|
||||
url: 下载 URL
|
||||
path: 保存路径
|
||||
"""
|
||||
import httpx
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with httpx.stream('GET', url, timeout=600) as r:
|
||||
r.raise_for_status()
|
||||
with open(path, 'wb') as f:
|
||||
for chunk in r.iter_bytes(8192):
|
||||
f.write(chunk)
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════
|
||||
# Kling T2I / I2I 核心
|
||||
# ══════════════════════════════════════════════════════
|
||||
|
||||
def _t2i(
|
||||
image_path: str,
|
||||
prompt: str,
|
||||
seed: int = 0,
|
||||
model: str = 'kling-v3',
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""Kling 文生图 (T2I)。
|
||||
|
||||
参数:
|
||||
model: 默认 BASELINE_MODEL (kling-v3)，可覆盖
|
||||
force: 强制重新生成，跳过缓存
|
||||
|
||||
返回: {"path": str, "seed": int}
|
||||
"""
|
||||
# 检查缓存
|
||||
if not force and os.path.exists(image_path):
|
||||
seed_file = f'{image_path}.seed'
|
||||
used_seed = seed
|
||||
if os.path.exists(seed_file):
|
||||
used_seed = int(open(seed_file).read().strip())
|
||||
logger.info(f'T2I 缓存命中: {image_path}')
|
||||
return {'path': image_path, 'seed': used_seed}
|
||||
|
||||
from lib.config import load_config
|
||||
import httpx
|
||||
import jwt
|
||||
|
||||
# 鉴权
|
||||
creds = _get_kling_credentials()
|
||||
ak = creds['access_key']
|
||||
sk = creds['secret_key']
|
||||
token = _jwt_token(ak, sk)
|
||||
headers = {'Authorization': f'Bearer {token}', 'Content-Type': 'application/json'}
|
||||
|
||||
# 请求
|
||||
model_name = model or BASELINE_MODEL
|
||||
payload = {
|
||||
'model': model_name,
|
||||
'prompt': prompt,
|
||||
'aspect_ratio': '9:16',
|
||||
'n': 1,
|
||||
}
|
||||
if seed:
|
||||
payload['seed'] = seed
|
||||
|
||||
# 提交
|
||||
for attempt in range(3):
|
||||
try:
|
||||
resp = httpx.post(
|
||||
f'{KLING_BASE}/v1/images/generations',
|
||||
headers=headers,
|
||||
json=payload,
|
||||
timeout=30,
|
||||
)
|
||||
if resp.status_code == 429:
|
||||
wait = 15 * (attempt + 1)
|
||||
logger.warning(f' 429 限流，等待 {wait}s...')
|
||||
time.sleep(wait)
|
||||
continue
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
task_id = data.get('data', {}).get('task_id', '')
|
||||
if not task_id:
|
||||
raise RuntimeError(f'Kling T2I 提交失败: {data}')
|
||||
break
|
||||
except Exception as e:
|
||||
if attempt == 2:
|
||||
raise
|
||||
time.sleep(5)
|
||||
else:
|
||||
raise RuntimeError('Kling T2I 提交失败（重试耗尽）')
|
||||
|
||||
# 轮询
|
||||
for i in range(40):
|
||||
time.sleep(5)
|
||||
try:
|
||||
q = httpx.get(
|
||||
f'{KLING_BASE}/v1/images/generations/{task_id}',
|
||||
headers=headers,
|
||||
timeout=30,
|
||||
)
|
||||
q.raise_for_status()
|
||||
status_data = q.json()['data']
|
||||
task_status = status_data.get('task_status', '')
|
||||
|
||||
if task_status == 'succeed':
|
||||
images = status_data.get('task_result', {}).get('images', [])
|
||||
if not images:
|
||||
raise RuntimeError('Kling T2I 成功但无图像')
|
||||
img_url = images[0].get('url', '')
|
||||
returned_seed = images[0].get('seed', seed)
|
||||
|
||||
# 下载
|
||||
logger.info(f' T2I 完成，下载中...')
|
||||
_download(img_url, image_path)
|
||||
|
||||
# 写种子文件
|
||||
with open(f'{image_path}.seed', 'w') as f:
|
||||
f.write(str(returned_seed))
|
||||
|
||||
logger.info(f' ✅ 基准图: {image_path} [seed={returned_seed}]')
|
||||
return {'path': image_path, 'seed': returned_seed}
|
||||
|
||||
elif task_status == 'failed':
|
||||
reason = status_data.get('fail_reason', '?')
|
||||
raise RuntimeError(f'T2I 生成失败: {reason}')
|
||||
|
||||
except Exception as e:
|
||||
if '轮询' in str(e):
|
||||
logger.warning(f' {e}')
|
||||
break
|
||||
logger.warning(f' 轮询异常: {e}')
|
||||
time.sleep(5)
|
||||
|
||||
raise TimeoutError('T2I 生成超时（200s）')
|
||||
|
||||
|
||||
def _i2i(
|
||||
output_path: str,
|
||||
ref_image: str,
|
||||
prompt: str,
|
||||
seed: int = 0,
|
||||
strength: float = 0.65,
|
||||
model: str = 'kling-v1',
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""Kling 图生图 (I2I)，基于参考图生成。
|
||||
|
||||
参数:
|
||||
strength: 参考强度，默认 0.65
|
||||
model: 模型，默认 ANGLE_MODEL (kling-v1)
|
||||
force: 强制重新生成
|
||||
|
||||
返回: {"path": str, "seed": int}
|
||||
"""
|
||||
# 检查缓存
|
||||
if not force and os.path.exists(output_path):
|
||||
seed_file = f'{output_path}.seed'
|
||||
used_seed = seed
|
||||
if os.path.exists(seed_file):
|
||||
used_seed = int(open(seed_file).read().strip())
|
||||
logger.info(f'I2I 缓存命中: {output_path}')
|
||||
return {'path': output_path, 'seed': used_seed}
|
||||
|
||||
import httpx
|
||||
import jwt
|
||||
|
||||
# 鉴权
|
||||
creds = _get_kling_credentials()
|
||||
ak = creds['access_key']
|
||||
sk = creds['secret_key']
|
||||
token = _jwt_token(ak, sk)
|
||||
headers = {'Authorization': f'Bearer {token}', 'Content-Type': 'application/json'}
|
||||
|
||||
# 读取参考图
|
||||
if not os.path.exists(ref_image):
|
||||
raise FileNotFoundError(f'I2I 参考图不存在: {ref_image}')
|
||||
with open(ref_image, 'rb') as f:
|
||||
img_b64 = base64.b64encode(f.read()).decode('utf-8')
|
||||
|
||||
model_name = model or ANGLE_MODEL
|
||||
ref_strength = strength or 0.7
|
||||
payload = {
|
||||
'model': model_name,
|
||||
'prompt': prompt,
|
||||
'image': img_b64,
|
||||
'strength': ref_strength,
|
||||
'aspect_ratio': '9:16',
|
||||
'n': 1,
|
||||
}
|
||||
if seed:
|
||||
payload['seed'] = seed
|
||||
|
||||
# 提交
|
||||
for attempt in range(3):
|
||||
try:
|
||||
resp = httpx.post(
|
||||
f'{KLING_BASE}/v1/images/generations',
|
||||
headers=headers,
|
||||
json=payload,
|
||||
timeout=30,
|
||||
)
|
||||
if resp.status_code == 429:
|
||||
wait = 15 * (attempt + 1)
|
||||
logger.warning(f' 429 限流，等待 {wait}s...')
|
||||
time.sleep(wait)
|
||||
continue
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
task_id = data.get('data', {}).get('task_id', '')
|
||||
if not task_id:
|
||||
raise RuntimeError(f'Kling I2I 提交失败: {data}')
|
||||
break
|
||||
except Exception as e:
|
||||
if attempt == 2:
|
||||
raise
|
||||
time.sleep(5)
|
||||
else:
|
||||
raise RuntimeError('Kling I2I 提交失败（重试耗尽）')
|
||||
|
||||
# 轮询
|
||||
for i in range(40):
|
||||
time.sleep(5)
|
||||
try:
|
||||
q = httpx.get(
|
||||
f'{KLING_BASE}/v1/images/generations/{task_id}',
|
||||
headers=headers,
|
||||
timeout=30,
|
||||
)
|
||||
q.raise_for_status()
|
||||
status_data = q.json()['data']
|
||||
task_status = status_data.get('task_status', '')
|
||||
|
||||
if task_status == 'succeed':
|
||||
images = status_data.get('task_result', {}).get('images', [])
|
||||
if not images:
|
||||
raise RuntimeError('Kling I2I 成功但无图像')
|
||||
img_url = images[0].get('url', '')
|
||||
returned_seed = images[0].get('seed', seed)
|
||||
|
||||
# 下载
|
||||
logger.info(f' I2I 完成，下载中...')
|
||||
_download(img_url, output_path)
|
||||
|
||||
# 写种子文件
|
||||
with open(f'{output_path}.seed', 'w') as f:
|
||||
f.write(str(returned_seed))
|
||||
|
||||
logger.info(f' ✅ 角度图: {output_path} [seed={returned_seed}]')
|
||||
return {'path': output_path, 'seed': returned_seed}
|
||||
|
||||
elif task_status == 'failed':
|
||||
reason = status_data.get('fail_reason', '?')
|
||||
raise RuntimeError(f'I2I 生成失败: {reason}')
|
||||
|
||||
except Exception as e:
|
||||
if '轮询' in str(e):
|
||||
logger.warning(f' {e}')
|
||||
break
|
||||
logger.warning(f' 轮询异常: {e}')
|
||||
time.sleep(5)
|
||||
|
||||
raise TimeoutError('I2I 生成超时（200s）')
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════
|
||||
# 主流程
|
||||
# ══════════════════════════════════════════════════════
|
||||
|
||||
def generate_baseline(
|
||||
description: str,
|
||||
char_name: str,
|
||||
force: bool = False,
|
||||
seed: int = 0,
|
||||
ref_image: str = '',
|
||||
) -> dict:
|
||||
"""Phase 1: 生成正面全身基准图。
|
||||
|
||||
策略（v2.2）:
|
||||
1. 优先用真实照片 ref_image 做 I2I（真实照片→影棚风格，强度 0.85 保留面部）
|
||||
2. 无参考照时回退 T2I (kling-v3)
|
||||
|
||||
参数:
|
||||
ref_image: 真实参考照路径。有则走 I2I，无则走 T2I
|
||||
|
||||
返回:
|
||||
{"baseline": "path/to/baseline.png", "seed": int, "method": "i2i|t2i"}
|
||||
"""
|
||||
output_dir = os.path.join(CACHE_DIR, char_name)
|
||||
Path(output_dir).mkdir(parents=True, exist_ok=True)
|
||||
|
||||
baseline_path = os.path.join(output_dir, 'baseline.png')
|
||||
seed_file = f'{baseline_path}.seed'
|
||||
|
||||
# 缓存检查
|
||||
if not force and os.path.exists(baseline_path):
|
||||
cached_seed = seed
|
||||
if os.path.exists(seed_file):
|
||||
cached_seed = int(open(seed_file).read().strip())
|
||||
logger.info(f'基准图缓存命中: {baseline_path}')
|
||||
return {'baseline': baseline_path, 'seed': cached_seed, 'method': 'cache'}
|
||||
|
||||
if ref_image and os.path.exists(ref_image):
|
||||
# 策略 1: 真实照片 I2I → 影棚风格基准图
|
||||
logger.info(f'=== Phase 1: 参考照 I2I 生成基准图 ({char_name}) ===')
|
||||
logger.info(f' 参考照: {ref_image}')
|
||||
|
||||
prompt = (
|
||||
'与参考照片完全相同的人物。全身镜头，正面站立直视镜头，如证件照构图。'
|
||||
'浅灰色无缝影棚背景，左右双柔光箱对称布光，面部无阴影。'
|
||||
'超高清8k，真实皮肤纹理，织物细节。'
|
||||
)
|
||||
|
||||
result = _i2i(
|
||||
baseline_path,
|
||||
ref_image,
|
||||
prompt,
|
||||
seed=seed,
|
||||
strength=0.85,
|
||||
model=ANGLE_MODEL,
|
||||
force=force,
|
||||
)
|
||||
return {
|
||||
'baseline': result['path'],
|
||||
'seed': result['seed'],
|
||||
'method': 'i2i',
|
||||
}
|
||||
else:
|
||||
# 策略 2: 纯 T2I
|
||||
logger.info(f'=== Phase 1: T2I 生成基准图 ({char_name}) [kling-v3] ===')
|
||||
prompt = BASELINE_PROMPT_TEMPLATE.format(desc=description)
|
||||
|
||||
result = _t2i(baseline_path, prompt, seed=seed, model=BASELINE_MODEL, force=force)
|
||||
return {
|
||||
'baseline': result['path'],
|
||||
'seed': result['seed'],
|
||||
'method': 't2i',
|
||||
}
|
||||
|
||||
|
||||
def generate_sequential_chain(
|
||||
baseline_path: str,
|
||||
char_name: str,
|
||||
force: bool = False,
|
||||
seed: int = 0,
|
||||
) -> dict:
|
||||
"""Phase 2: 逐帧参考链生成多角度定妆照。
|
||||
|
||||
v2.0 核心变更（对比 v1.0）:
|
||||
❌ v1.0 问题: 所有角度都用同一张 baseline 作 I2I 参考
|
||||
→ 从正面直接跳到90°侧面，姿态变化太大，角色容易漂移
|
||||
✅ v2.0 方案: 逐帧参考链
|
||||
图1(正面) → 图2(右侧45°) → 图3(纯右侧面) → 图4(背面)
|
||||
每张图都用上一张图作 I2I 参考，只做小幅姿态变化
|
||||
|
||||
返回:
|
||||
{
|
||||
"right_three_quarter": "path/to/angle_right_three_quarter.png",
|
||||
"right_side": "path/to/angle_right_side.png",
|
||||
"back": "path/to/angle_back.png",
|
||||
"chain": [步骤详情],
|
||||
"seed": int
|
||||
}
|
||||
"""
|
||||
output_dir = os.path.join(CACHE_DIR, char_name, 'angles')
|
||||
Path(output_dir).mkdir(parents=True, exist_ok=True)
|
||||
|
||||
results = {}
|
||||
chain_log = []
|
||||
final_seed = seed
|
||||
|
||||
# 构建参考映射: 每个步骤的参考图来源
|
||||
ref_map = {'baseline': baseline_path}
|
||||
for step in ANGLE_CHAIN:
|
||||
ref_map[step['key']] = os.path.join(output_dir, f'angle_{step["key"]}.png')
|
||||
|
||||
for i, step in enumerate(ANGLE_CHAIN):
|
||||
output_path = ref_map[step['key']]
|
||||
ref_image = ref_map[step['ref_source']]
|
||||
step_strength = step['strength']
|
||||
step_prompt = step['prompt']
|
||||
|
||||
logger.info(f'==================================================')
|
||||
logger.info(f'Phase 2 Step {i+1}/{len(ANGLE_CHAIN)}: {step["label"]}')
|
||||
logger.info(f' 参考图: {step["ref_source"]} → {step["key"]}')
|
||||
logger.info(f' 强度: {step_strength}')
|
||||
|
||||
try:
|
||||
result = _i2i(
|
||||
output_path,
|
||||
ref_image,
|
||||
step_prompt,
|
||||
seed=seed,
|
||||
strength=step_strength,
|
||||
model=ANGLE_MODEL,
|
||||
force=force,
|
||||
)
|
||||
results[step['key']] = result['path']
|
||||
final_seed = result.get('seed', seed)
|
||||
logger.info(f' ✅ {step["label"]} [seed={final_seed}]')
|
||||
|
||||
chain_log.append({
|
||||
'step': i + 1,
|
||||
'label': step['label'],
|
||||
'ref_source': step['ref_source'],
|
||||
'strength': step_strength,
|
||||
'output': result['path'],
|
||||
'seed': final_seed,
|
||||
})
|
||||
except Exception as e:
|
||||
logger.warning(f' ❌ {step["label"]} 生成失败: {e}')
|
||||
chain_log.append({
|
||||
'step': i + 1,
|
||||
'label': step['label'],
|
||||
'ref_source': step['ref_source'],
|
||||
'strength': step_strength,
|
||||
'output': None,
|
||||
'error': str(e),
|
||||
})
|
||||
|
||||
# 写 chain meta
|
||||
meta_path = os.path.join(output_dir, '_chain_meta.json')
|
||||
meta = {
|
||||
'character': char_name,
|
||||
'method': 'sequential_reference_chain_v2',
|
||||
'baseline': baseline_path,
|
||||
'seed': final_seed,
|
||||
'chain': chain_log,
|
||||
}
|
||||
Path(meta_path).write_text(json.dumps(meta, ensure_ascii=False, indent=2), encoding='utf-8')
|
||||
|
||||
return {
|
||||
'right_three_quarter': results.get('right_three_quarter', ''),
|
||||
'right_side': results.get('right_side', ''),
|
||||
'back': results.get('back', ''),
|
||||
'chain': chain_log,
|
||||
'seed': final_seed,
|
||||
}
|
||||
|
||||
|
||||
def register_portraits(
|
||||
char_name: str,
|
||||
baseline_path: str,
|
||||
angles: dict,
|
||||
) -> bool:
|
||||
"""Phase 3: 将定妆照注册到 CharacterRegistry。
|
||||
|
||||
参数:
|
||||
char_name: 角色名
|
||||
baseline_path: 基准图路径
|
||||
angles: {"right_three_quarter": ..., "right_side": ..., "back": ...}
|
||||
|
||||
返回: 是否成功
|
||||
"""
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from character_registry import CharacterRegistry
|
||||
|
||||
try:
|
||||
registry_path = os.path.join(
|
||||
os.path.dirname(os.path.abspath(__file__)),
|
||||
'character_registry',
|
||||
'registry.yaml',
|
||||
)
|
||||
registry = CharacterRegistry()
|
||||
|
||||
# 注册 face_anchor
|
||||
ref_images = {'face_anchor': [baseline_path], 'body': []}
|
||||
|
||||
# 注册 body 各角度
|
||||
for key in ('right_three_quarter', 'right_side', 'back'):
|
||||
if key in angles and angles[key]:
|
||||
ref_images['body'].append({'path': angles[key], 'option': key, 'type': 'body'})
|
||||
|
||||
registry.update_reference_images(char_name, ref_images)
|
||||
logger.info(f'✅ 已注册定妆照到 CharacterRegistry: {char_name}')
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f'注册到 CharacterRegistry 失败: {e}')
|
||||
return False
|
||||
|
||||
|
||||
def generate_and_register_portraits(
|
||||
character_name: str,
|
||||
description: str,
|
||||
force: bool = False,
|
||||
seed: int = 0,
|
||||
ref_image: str = '',
|
||||
) -> dict:
|
||||
"""全流程: 基准图 → 多角度 → 注册。
|
||||
|
||||
参数:
|
||||
character_name: 角色名（与 registry.yaml 一致）
|
||||
description: 角色描述（自然语言）
|
||||
force: 强制重新生成
|
||||
seed: 固定种子
|
||||
ref_image: 真实参考照路径。有则 I2I 生成基准图，无则 T2I
|
||||
|
||||
返回:
|
||||
{
|
||||
"character": "角色名",
|
||||
"baseline": "path/to/baseline.png",
|
||||
"angles": {"right_three_quarter": "...", "right_side": "...", "back": "..."},
|
||||
"registry_updated": True/False,
|
||||
"seed": 42,
|
||||
}
|
||||
"""
|
||||
result = {
|
||||
'character': character_name,
|
||||
'baseline': '',
|
||||
'angles': {},
|
||||
'registry_updated': False,
|
||||
'seed': seed,
|
||||
}
|
||||
|
||||
logger.info(f'==================================================')
|
||||
logger.info(f'🎯 定妆照全流程: {character_name}')
|
||||
|
||||
# Phase 1: 基准图
|
||||
try:
|
||||
logger.info('📷 Phase 1/2: 生成基准图（正面全身定妆照）...')
|
||||
baseline_result = generate_baseline(
|
||||
description=description,
|
||||
char_name=character_name,
|
||||
force=force,
|
||||
seed=seed,
|
||||
ref_image=ref_image,
|
||||
)
|
||||
baseline_path = baseline_result.get('baseline', '')
|
||||
used_seed = baseline_result.get('seed', seed)
|
||||
result['baseline'] = baseline_path
|
||||
result['seed'] = used_seed
|
||||
logger.info(f' 基准图: {baseline_path}')
|
||||
logger.info(f' seed: {used_seed}')
|
||||
except Exception as e:
|
||||
logger.error(f' ❌ Phase 1 失败: {e}')
|
||||
result['error'] = f'Phase 1 (基准图) 失败: {str(e)}'
|
||||
return result
|
||||
|
||||
# Phase 2: 逐帧参考链
|
||||
try:
|
||||
logger.info('📸 Phase 2/2: 逐帧参考链生成多角度定妆照...')
|
||||
logger.info(' 参考链: 正面 → 右侧45°半侧面 → 纯右侧面 → 背面')
|
||||
logger.info(' 每张以上一张为 I2I 参考，强度 0.70→0.75→0.70')
|
||||
angles = generate_sequential_chain(
|
||||
baseline_path=baseline_path,
|
||||
char_name=character_name,
|
||||
force=force,
|
||||
seed=used_seed,
|
||||
)
|
||||
result['angles'] = {
|
||||
'right_three_quarter': angles.get('right_three_quarter', ''),
|
||||
'right_side': angles.get('right_side', ''),
|
||||
'back': angles.get('back', ''),
|
||||
}
|
||||
logger.info(f' 右侧45°半侧面: {result["angles"]["right_three_quarter"][:50] if result["angles"]["right_three_quarter"] else "失败"}')
|
||||
logger.info(f' 纯右侧面: {result["angles"]["right_side"][:50] if result["angles"]["right_side"] else "失败"}')
|
||||
logger.info(f' 背面: {result["angles"]["back"][:50] if result["angles"]["back"] else "失败"}')
|
||||
except Exception as e:
|
||||
logger.error(f' ❌ Phase 2 失败: {e}')
|
||||
result['warning'] = f'Phase 2 (多角度) 部分失败: {str(e)}'
|
||||
|
||||
# Phase 3: 注册
|
||||
try:
|
||||
logger.info('📋 Phase 3/2: 注册到 CharacterRegistry...')
|
||||
reg_ok = register_portraits(
|
||||
char_name=character_name,
|
||||
baseline_path=baseline_path,
|
||||
angles=result['angles'],
|
||||
)
|
||||
result['registry_updated'] = reg_ok
|
||||
logger.info(f' {"✅ 注册成功" if reg_ok else "⚠️ 注册失败"}')
|
||||
except Exception as e:
|
||||
logger.warning(f' 注册异常: {e}')
|
||||
result['registry_updated'] = False
|
||||
|
||||
logger.info(f'✅ 定妆照生成完成: {character_name}')
|
||||
return result
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════
|
||||
# CLI 入口
|
||||
# ══════════════════════════════════════════════════════
|
||||
|
||||
def cli(args: list):
|
||||
"""CLI 入口。"""
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(description='两阶段定妆照生成（观鱼方案）')
|
||||
parser.add_argument('--character', required=True, help='角色名')
|
||||
parser.add_argument('--desc', type=str, default='', help='角色描述（自然语言）')
|
||||
parser.add_argument('--force', action='store_true', default=False, help='强制重新生成')
|
||||
parser.add_argument('--seed', type=int, default=0, help='固定种子')
|
||||
parser.add_argument('--desc-file', type=str, default='', help='从文件读取角色描述')
|
||||
parser.add_argument('--ref', type=str, default='', help='真实参考照路径（推荐，代替T2I）')
|
||||
|
||||
parsed = parser.parse_args(args)
|
||||
|
||||
desc = parsed.desc
|
||||
if parsed.desc_file:
|
||||
desc_file = Path(parsed.desc_file)
|
||||
if desc_file.exists():
|
||||
desc = desc_file.read_text(encoding='utf-8').strip()
|
||||
|
||||
result = generate_and_register_portraits(
|
||||
character_name=parsed.character,
|
||||
description=desc,
|
||||
force=parsed.force,
|
||||
seed=parsed.seed,
|
||||
ref_image=parsed.ref,
|
||||
)
|
||||
|
||||
print()
|
||||
print('==================================================')
|
||||
print('✅ 定妆照生成结果:')
|
||||
print(f' 基准图: {result.get("baseline", "失败")}')
|
||||
if result.get('angles'):
|
||||
for key, label in [('right_three_quarter', '右侧45°半侧面'), ('right_side', '纯右侧面'), ('back', '背面')]:
|
||||
print(f' {label}: {result["angles"].get(key, "失败")}')
|
||||
print(f' seed: {result.get("seed", "N/A")}')
|
||||
print(f' 已注册: {"✅" if result.get("registry_updated") else "❌"}')
|
||||
|
||||
if result.get('error'):
|
||||
print(f' 错误: {result["error"]}')
|
||||
if result.get('warning'):
|
||||
print(f' 警告: {result["warning"]}')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
import sys
|
||||
cli(sys.argv[1:])
|
||||
@@ -0,0 +1,318 @@
|
||||
"""
|
||||
角色注册中心 (Character Registry) — 漫剧视频工厂
|
||||
|
||||
功能:
|
||||
- 注册/管理/切换主角
|
||||
- 存储角色描述、外观、性格、音色、画风
|
||||
- 生成角色描述块 (character block)，供角色适配器和视觉生成模块注入
|
||||
- 支持多角色切换
|
||||
|
||||
用法:
|
||||
from character_registry import CharacterRegistry
|
||||
registry = CharacterRegistry()
|
||||
char = registry.get_active_character()
|
||||
block = char.build_prompt_block(style="manhua", expression="微笑", scene="咖啡馆")
|
||||
"""
|
||||
|
||||
import os
|
||||
import json
|
||||
import yaml
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
# ── 默认路径 ──
|
||||
SCRIPTS_DIR = Path(__file__).resolve().parent
|
||||
AVE_ROOT = SCRIPTS_DIR.parent
|
||||
CACHE_DIR = Path(os.environ.get(
|
||||
"AVE_CACHE_DIR",
|
||||
str(Path.home() / "workbuddy-agent-os/agent-local/tools/ave/cache/character_registry")
|
||||
))
|
||||
CACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
REGISTRY_FILE = SCRIPTS_DIR / "registry.yaml"
|
||||
CHARACTERS_DIR = CACHE_DIR / "characters"
|
||||
CHARACTERS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 角色类
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
class Character:
|
||||
"""单个角色的完整定义"""
|
||||
|
||||
def __init__(self, data: dict):
|
||||
self.name: str = data.get("name", "未命名")
|
||||
self.description: str = data.get("description", "")
|
||||
self.appearance: list[str] = data.get("appearance", [])
|
||||
self.personality: str = data.get("personality", "")
|
||||
self.voice_style: str = data.get("voice_style", "默认")
|
||||
self.art_style: str = data.get("art_style", "")
|
||||
self.default_bgm_style: str = data.get("default_bgm_style", "治愈")
|
||||
self.preferred_bgm_tags: list[str] = data.get("preferred_bgm_tags", [])
|
||||
self.reference_images: dict = data.get("reference_images", {})
|
||||
self.prompt_template: dict = data.get("prompt_template", {})
|
||||
self._raw = data
|
||||
|
||||
# ── 角色描述块生成 ──
|
||||
|
||||
def build_prompt_block(self, style: str = "manhua", expression: str = "中性",
|
||||
scene: str = "", camera_angle: str = "中景") -> str:
|
||||
"""
|
||||
构建角色描述块，用于注入视觉生成 prompt
|
||||
|
||||
参数:
|
||||
style: "manhua" | "realistic"
|
||||
expression: 角色当前表情/情绪
|
||||
scene: 场景描述
|
||||
camera_angle: 镜头角度
|
||||
|
||||
返回:
|
||||
格式化的角色描述块字符串
|
||||
"""
|
||||
app_desc = "，".join(self.appearance[:3]) # 取前3个外观特征
|
||||
char_desc = f"{self.name}，{app_desc}"
|
||||
|
||||
template = self.prompt_template.get(style, "")
|
||||
if template:
|
||||
return template.format(
|
||||
character_desc=char_desc,
|
||||
scene_desc=scene,
|
||||
expression=expression,
|
||||
camera_angle=camera_angle,
|
||||
)
|
||||
|
||||
# 默认模板
|
||||
if style == "manhua":
|
||||
return (
|
||||
f"[角色] {char_desc}\n"
|
||||
f"[场景] {scene}\n"
|
||||
f"[表情] {expression}\n"
|
||||
f"[画风] {self.art_style}\n"
|
||||
f"[镜头] {camera_angle}"
|
||||
)
|
||||
else:
|
||||
return (
|
||||
f"[角色] {char_desc}\n"
|
||||
f"[场景] {scene}\n"
|
||||
f"[表情] {expression}\n"
|
||||
f"[画风] 电影级摄影，写实风格\n"
|
||||
f"[镜头] {camera_angle}"
|
||||
)
|
||||
|
||||
def to_storyboard_block(self) -> dict:
|
||||
"""生成角色描述块 JSON（供分镜引用）"""
|
||||
return {
|
||||
"character_name": self.name,
|
||||
"appearance": self.appearance,
|
||||
"personality": self.personality,
|
||||
"voice_style": self.voice_style,
|
||||
"art_style": self.art_style,
|
||||
"reference_images": self.reference_images,
|
||||
"default_bgm_style": self.default_bgm_style,
|
||||
}
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return self._raw
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"<Character: {self.name}>"
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 角色注册中心
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
class CharacterRegistry:
|
||||
"""角色注册中心 — 管理所有已注册角色"""
|
||||
|
||||
def __init__(self, registry_path: Optional[str] = None):
|
||||
self._registry_path = Path(registry_path or REGISTRY_FILE)
|
||||
self._characters: dict[str, Character] = {}
|
||||
self._active_name: str = ""
|
||||
self._load()
|
||||
|
||||
# ── 加载/刷新 ──
|
||||
|
||||
def _load(self):
|
||||
"""从 registry.yaml 加载角色"""
|
||||
if not self._registry_path.exists():
|
||||
raise FileNotFoundError(f"角色注册文件不存在: {self._registry_path}")
|
||||
|
||||
with open(self._registry_path, encoding="utf-8") as f:
|
||||
data = yaml.safe_load(f) or {}
|
||||
|
||||
self._active_name = data.get("active_character", "")
|
||||
chars = data.get("characters", {})
|
||||
|
||||
for name, char_data in chars.items():
|
||||
char_data["name"] = name
|
||||
self._characters[name] = Character(char_data)
|
||||
|
||||
if not self._active_name and self._characters:
|
||||
self._active_name = list(self._characters.keys())[0]
|
||||
|
||||
def reload(self):
|
||||
"""重新加载注册文件"""
|
||||
self._load()
|
||||
|
||||
# ── 查询 ──
|
||||
|
||||
def get_active_character(self) -> Character:
|
||||
"""获取当前激活的角色"""
|
||||
if self._active_name in self._characters:
|
||||
return self._characters[self._active_name]
|
||||
raise KeyError(f"未找到激活角色: {self._active_name}")
|
||||
|
||||
def get_character(self, name: str) -> Character:
|
||||
"""按名字获取角色"""
|
||||
if name in self._characters:
|
||||
return self._characters[name]
|
||||
raise KeyError(f"未找到角色: {name}")
|
||||
|
||||
def list_characters(self) -> list[str]:
|
||||
"""列出所有已注册角色名"""
|
||||
return list(self._characters.keys())
|
||||
|
||||
def get_active_name(self) -> str:
|
||||
return self._active_name
|
||||
|
||||
# ── 参考图管理 ──
|
||||
|
||||
def set_reference_image(self, name: str, key: str, path: str) -> bool:
|
||||
"""
|
||||
设置角色的单张参考图路径
|
||||
|
||||
参数:
|
||||
name: 角色名
|
||||
key: 参考图类型 (portrait / grid / full_body / expressions.happy 等)
|
||||
path: 图片文件路径
|
||||
|
||||
使用嵌套 key 的示例: set_reference_image("小漫", "expressions.happy", "...")
|
||||
"""
|
||||
char = self.get_character(name)
|
||||
if "." in key:
|
||||
# 嵌套 key: "expressions.happy" → reference_images["expressions"]["happy"]
|
||||
parts = key.split(".", 1)
|
||||
if parts[0] not in char.reference_images:
|
||||
char.reference_images[parts[0]] = {}
|
||||
char.reference_images[parts[0]][parts[1]] = path
|
||||
else:
|
||||
char.reference_images[key] = path
|
||||
|
||||
# 同步到 _raw 以持久化
|
||||
char._raw["reference_images"] = char.reference_images
|
||||
self._save()
|
||||
return True
|
||||
|
||||
def update_reference_images(self, name: str, images: dict) -> bool:
|
||||
"""
|
||||
批量更新角色的参考图路径
|
||||
|
||||
参数:
|
||||
name: 角色名
|
||||
images: {
|
||||
"portrait": "path/to/portrait.png",
|
||||
"grid": "path/to/grid.png",
|
||||
"expressions": {"happy": "...", "sad": "...", "angry": "..."}
|
||||
}
|
||||
"""
|
||||
char = self.get_character(name)
|
||||
# 深度合并
|
||||
for key, value in images.items():
|
||||
if isinstance(value, dict) and key in char.reference_images and isinstance(char.reference_images[key], dict):
|
||||
char.reference_images[key].update(value)
|
||||
else:
|
||||
char.reference_images[key] = value
|
||||
|
||||
char._raw["reference_images"] = char.reference_images
|
||||
self._save()
|
||||
return True
|
||||
|
||||
def get_reference_images(self, name: str) -> dict:
|
||||
"""获取角色的所有参考图路径"""
|
||||
char = self.get_character(name)
|
||||
return dict(char.reference_images)
|
||||
|
||||
def clear_reference_images(self, name: str) -> bool:
|
||||
"""清空角色的参考图路径"""
|
||||
char = self.get_character(name)
|
||||
char.reference_images = {}
|
||||
char._raw["reference_images"] = {}
|
||||
self._save()
|
||||
return True
|
||||
|
||||
# ── 切换/注册 ──
|
||||
|
||||
def switch_to(self, name: str) -> bool:
|
||||
"""切换到指定角色"""
|
||||
if name not in self._characters:
|
||||
raise KeyError(f"未找到角色: {name}")
|
||||
self._active_name = name
|
||||
self._save()
|
||||
return True
|
||||
|
||||
def register(self, character_data: dict) -> Character:
|
||||
"""注册新角色"""
|
||||
name = character_data.get("name", "")
|
||||
if not name:
|
||||
raise ValueError("角色必须包含 name 字段")
|
||||
|
||||
self._characters[name] = Character(character_data)
|
||||
if not self._active_name:
|
||||
self._active_name = name
|
||||
self._save()
|
||||
return self._characters[name]
|
||||
|
||||
def delete(self, name: str) -> bool:
|
||||
"""删除角色"""
|
||||
if name not in self._characters:
|
||||
return False
|
||||
del self._characters[name]
|
||||
if self._active_name == name:
|
||||
self._active_name = list(self._characters.keys())[0] if self._characters else ""
|
||||
self._save()
|
||||
return True
|
||||
|
||||
# ── 持久化 ──
|
||||
|
||||
def _save(self):
|
||||
"""保存当前状态到 registry.yaml"""
|
||||
chars_dict = {}
|
||||
for name, char in self._characters.items():
|
||||
chars_dict[name] = char.to_dict()
|
||||
|
||||
data = {
|
||||
"active_character": self._active_name,
|
||||
"characters": chars_dict,
|
||||
}
|
||||
with open(self._registry_path, "w", encoding="utf-8") as f:
|
||||
yaml.dump(data, f, allow_unicode=True, default_flow_style=False, sort_keys=False)
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self._characters)
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"<CharacterRegistry: {self._active_name} active, {len(self)} total>"
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 独立使用入口
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
if __name__ == "__main__":
|
||||
registry = CharacterRegistry()
|
||||
print(f"角色注册中心: {registry}")
|
||||
print(f"活跃角色: {registry.get_active_name()}")
|
||||
|
||||
char = registry.get_active_character()
|
||||
print(f"\n角色详情:")
|
||||
print(f" 名字: {char.name}")
|
||||
print(f" 描述: {char.description}")
|
||||
print(f" 外观: {char.appearance}")
|
||||
print(f" 性格: {char.personality}")
|
||||
print(f" 音色: {char.voice_style}")
|
||||
print(f" 画风: {char.art_style}")
|
||||
|
||||
print(f"\n角色描述块 (漫剧):")
|
||||
print(char.build_prompt_block(style="manhua", expression="微笑", scene="清晨的咖啡馆"))
|
||||
@@ -0,0 +1,700 @@
|
||||
active_character: 阿远
|
||||
characters:
|
||||
小漫:
|
||||
description: 活泼可爱的少女漫画家，短发，大眼睛，总是带着画板和笔
|
||||
appearance:
|
||||
- 短发，齐刘海
|
||||
- 大眼睛，瞳孔呈深棕色
|
||||
- 常穿白色连衣裙配红色蝴蝶结
|
||||
- 身高约160cm，偏瘦
|
||||
personality: 开朗、好奇、偶尔冒失
|
||||
voice_style: 活泼少女音
|
||||
art_style: 日系漫画风格，明亮色调，线条简洁
|
||||
default_bgm_style: 治愈
|
||||
preferred_bgm_tags:
|
||||
- 治愈
|
||||
- 温暖
|
||||
- 轻快
|
||||
reference_images:
|
||||
full_body: ''
|
||||
portrait: ''
|
||||
expressions:
|
||||
happy: ''
|
||||
sad: ''
|
||||
angry: ''
|
||||
prompt_template:
|
||||
manhua: 日系漫画风格，{character_desc}，{scene_desc}，表情{expression}，{camera_angle}
|
||||
realistic: '{character_desc}，{scene_desc}，表情{expression}，电影级摄影，{camera_angle}'
|
||||
name: 小漫
|
||||
阿远:
|
||||
name: 阿远
|
||||
description: 一位25岁的城市程序员，黑短发，戴细框金属眼镜，身高178cm偏瘦，常穿浅灰色卫衣和深蓝色牛仔裤，性格温和内敛有点宅，但在熟悉的人面前也会露出温暖的笑容
|
||||
appearance:
|
||||
- 黑短发，三七分
|
||||
- 细框金属眼镜，单眼皮
|
||||
- 身高约178cm，偏瘦体型
|
||||
- 常穿浅灰色连帽卫衣
|
||||
- 戴一块简约的黑色电子手表
|
||||
personality: 温和、内敛、偶尔倔强、专注、在熟悉的人面前温暖
|
||||
voice_style: 温柔青年音
|
||||
art_style: 日系清新漫画风格，柔和色调，线条干净
|
||||
default_bgm_style: 温暖
|
||||
preferred_bgm_tags:
|
||||
- 温暖
|
||||
- 安静
|
||||
- 治愈
|
||||
- 轻快
|
||||
prompt_template:
|
||||
manhua: 日系清新漫画风格，{character_desc}，{scene_desc}，表情{expression}，{camera_angle}
|
||||
realistic: '{character_desc}，{scene_desc}，表情{expression}，电影级摄影风格，柔和自然光线，{camera_angle}'
|
||||
reference_images:
|
||||
grid: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/阿远/阿远_grid.png
|
||||
portrait: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/阿远/阿远_grid_face_neutral.png
|
||||
expressions:
|
||||
neutral: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/阿远/阿远_grid_face_neutral.png
|
||||
smile: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/阿远/阿远_grid_face_smile.png
|
||||
angry: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/阿远/阿远_grid_face_angry.png
|
||||
ghai_sports:
|
||||
name: ghai_sports
|
||||
base: 一位中国男性，约42岁，体型健壮肩宽，黑色短发利落，国字脸，单眼皮眼尾微垂，鼻梁挺直鼻翼略宽，嘴唇厚度中等嘴角微收，神态沉稳内敛，肤色健康自然
|
||||
description: 一位中国男性，约42岁，体型健壮肩宽，黑色短发利落，国字脸单眼皮，神态沉稳内敛，肤色健康自然，身穿深灰色运动夹克配白色速干内衬，右耳戴着蓝色无线运动耳机。
|
||||
appearance:
|
||||
- 黑色短发，略长顶部，两侧收短，整洁干练
|
||||
- 圆润饱满的国字脸，单眼皮
|
||||
- 体型健壮，肩宽，稳重感强
|
||||
- 深灰色拉链运动夹克，白色圆领速干内衬
|
||||
- 右耳蓝色无线运动耳机
|
||||
personality: 沉稳、专注、坚韧
|
||||
voice_style: 成熟稳重男声
|
||||
art_style: 写实摄影风格，运动场景，自然光线
|
||||
default_bgm_style: 激励
|
||||
preferred_bgm_tags:
|
||||
- 激励
|
||||
- 热血
|
||||
- 节奏感
|
||||
- 运动
|
||||
sheet_mode: sport
|
||||
seed: 42
|
||||
reference_images:
|
||||
real_photo_portrait: /Users/7kecheng/workbuddy-agent-os/agent-sync/05_tools/09_ave/scripts/character_registry/assets/ghai_sports/reference_portrait.jpg
|
||||
real_photo_fullbody: /Users/7kecheng/workbuddy-agent-os/agent-sync/05_tools/09_ave/scripts/character_registry/assets/ghai_sports/reference_fullbody.png
|
||||
face_anchor: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/ghai_sports/face_anchor.png
|
||||
grid: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/ghai_sports/ghai_sports_grid.png
|
||||
portrait: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/ghai_sports/ghai_sports_grid_face_front.png
|
||||
expressions:
|
||||
front: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/ghai_sports/ghai_sports_grid_face_front.png
|
||||
3qL: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/ghai_sports/ghai_sports_grid_face_3qL.png
|
||||
profile: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/ghai_sports/ghai_sports_grid_face_profile.png
|
||||
3qR: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/ghai_sports/ghai_sports_grid_face_3qR.png
|
||||
focus: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/ghai_sports/ghai_sports_grid_face_focus.png
|
||||
intense: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/ghai_sports/ghai_sports_grid_face_intense.png
|
||||
body:
|
||||
front: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/ghai_sports/ghai_sports_grid_body_front.png
|
||||
side: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/ghai_sports/ghai_sports_grid_body_side.png
|
||||
seed: null
|
||||
outfit_presets:
|
||||
running:
|
||||
outfit: 透气速干T恤和跑步短裤
|
||||
accessories: 右耳戴着蓝色无线运动耳机
|
||||
prompt_suffix: 晨跑场景，动态模糊，清晨自然光线
|
||||
gym:
|
||||
outfit: 深色运动背心和训练短裤
|
||||
accessories: ''
|
||||
prompt_suffix: 健身房场景，力量训练，金属器械背景，侧光
|
||||
basketball:
|
||||
outfit: 篮球运动背心和短裤
|
||||
accessories: 运动护腕
|
||||
prompt_suffix: 街头篮球场，水泥地，傍晚光线
|
||||
outdoor_fitness:
|
||||
outfit: 白色健身T恤和运动长裤
|
||||
accessories: ''
|
||||
prompt_suffix: 户外健身，单杠器械，城市公园，午后阳光
|
||||
prompt_template:
|
||||
realistic: '{base}，{outfit}，{scene_desc}，{camera_angle}，写实摄影风格，自然光线'
|
||||
sport: '{base}，{outfit}，{scene_desc}，运动摄影风格，高速快门，{camera_angle}'
|
||||
裔年轻:
|
||||
name: 裔年轻
|
||||
description: 中景镜头，一位气质干净清澈的亚裔年轻男性，长相温润正派，五官端正立体，眼神真诚温暖带笑。身着燕麦色羊绒衫（袖口微微挽起）外搭深藏蓝亚麻外套，直身微侧，姿态挺拔舒展。背景是雨后湿润的江南古巷，斑驳青砖墙，脚下石板路反着水光。他一手拎着一袋用褐色纸袋包着、微微冒着热气的桂花糕，低头看向纸袋，嘴角带着温柔的笑意。
|
||||
高清细节，暖色调，湿漉漉的地面映照天色，整体呈现胶片般柔和细腻的质感。
|
||||
base: 中景镜头，一位气质干净清澈的亚裔年轻男性，长相温润正派，五官端正立体，眼神真诚温暖带笑。身着燕麦色羊绒衫外搭深藏蓝亚麻外套，直身微侧，姿态挺拔舒展。背景是雨后湿润的江南古巷，斑驳青砖墙，脚下石板路反着水光。他一手拎着一袋用褐色纸袋包着、微微冒着热气的桂花糕，低头看向纸袋，嘴角带着温柔的笑意。
|
||||
高清细节，暖色调，湿漉漉的地面映照天色，整体呈现胶片般柔和细腻的质感
|
||||
personality: 气质干净清澈的亚裔年轻男性
|
||||
voice_style: 默认
|
||||
art_style: 写实摄影风格
|
||||
appearance:
|
||||
- 中景镜头
|
||||
- 一位气质干净清澈的亚裔年轻男性
|
||||
- 长相温润正派
|
||||
- 五官端正立体
|
||||
- 眼神真诚温暖带笑
|
||||
face:
|
||||
shape: ''
|
||||
eyes: ''
|
||||
nose: ''
|
||||
lips: 嘴角
|
||||
hair: ''
|
||||
skin: ''
|
||||
facial_hair: ''
|
||||
glasses: ''
|
||||
distinctive: ''
|
||||
body:
|
||||
build: ''
|
||||
height: ''
|
||||
posture: 挺拔
|
||||
age_group: ''
|
||||
clothing:
|
||||
top: 身着燕麦色羊绒衫（袖口微微挽起）外搭深藏蓝亚麻外套
|
||||
inner: 背景是雨后湿润的江南古巷
|
||||
bottom: ''
|
||||
shoes: ''
|
||||
style: ''
|
||||
prompt_template:
|
||||
realistic: '{base}，{scene_desc}，{camera_angle}，写实摄影风格，自然光线'
|
||||
preferred_bgm_tags:
|
||||
- 温暖
|
||||
reference_images:
|
||||
face_anchor: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait/裔年轻/front_face.png
|
||||
baseline: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait/裔年轻/front_face.png
|
||||
body:
|
||||
front: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait/裔年轻/multi_angle_front.png
|
||||
side: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait/裔年轻/multi_angle_side.png
|
||||
right_three_quarter: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait/裔年轻/multi_angle_right_three_quarter.png
|
||||
right_side: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait/裔年轻/multi_angle_right_side.png
|
||||
back: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait/裔年轻/multi_angle_back.png
|
||||
expressions:
|
||||
neutral: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait/裔年轻/expression_neutral.png
|
||||
smile: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait/裔年轻/expression_smile.png
|
||||
focused: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait/裔年轻/expression_focused.png
|
||||
laugh: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait/裔年轻/expression_laugh.png
|
||||
serious: /Users/7kecheng/workbuddy-agent-os/agent-local/tools/ave/cache/character_portrait/裔年轻/expression_serious.png
|
||||
夺:
|
||||
name: 夺
|
||||
description: 奇才
|
||||
base: 奇才
|
||||
personality: 默认
|
||||
voice_style: 默认
|
||||
art_style: 写实摄影风格
|
||||
face:
|
||||
shape: ''
|
||||
eyes: ''
|
||||
nose: ''
|
||||
lips: ''
|
||||
hair: ''
|
||||
skin: ''
|
||||
facial_hair: ''
|
||||
glasses: ''
|
||||
distinctive: ''
|
||||
body:
|
||||
build: ''
|
||||
height: ''
|
||||
posture: ''
|
||||
age_group: ''
|
||||
clothing:
|
||||
top: ''
|
||||
inner: ''
|
||||
bottom: ''
|
||||
shoes: ''
|
||||
style: ''
|
||||
中国:
|
||||
name: 中国
|
||||
description: '【身份】女性、青年、中国、姐三
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
base: '【身份】女性、青年、中国、姐三
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
personality: 默认
|
||||
voice_style: 默认
|
||||
art_style: 写实摄影风格
|
||||
appearance:
|
||||
- 体型:标准
|
||||
face:
|
||||
shape: ''
|
||||
eyes: ''
|
||||
nose: ''
|
||||
lips: ''
|
||||
hair: '【身份】女性、青年、中国、姐三
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
skin: '健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
facial_hair: ''
|
||||
glasses: ''
|
||||
distinctive: ''
|
||||
body:
|
||||
build: 标准
|
||||
height: ''
|
||||
posture: 自然
|
||||
age_group: ''
|
||||
clothing:
|
||||
top: ''
|
||||
inner: '【身份】女性、青年、中国、姐三
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
bottom: '【身份】女性、青年、中国、姐三
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
shoes: ''
|
||||
style: ''
|
||||
测试:
|
||||
name: 测试
|
||||
description: 测试角色
|
||||
base: 测试角色
|
||||
personality: 默认
|
||||
voice_style: 默认
|
||||
art_style: 写实摄影风格
|
||||
face:
|
||||
shape: ''
|
||||
eyes: ''
|
||||
nose: ''
|
||||
lips: ''
|
||||
hair: ''
|
||||
skin: ''
|
||||
facial_hair: ''
|
||||
glasses: ''
|
||||
distinctive: ''
|
||||
body:
|
||||
build: ''
|
||||
height: ''
|
||||
posture: ''
|
||||
age_group: ''
|
||||
clothing:
|
||||
top: ''
|
||||
inner: ''
|
||||
bottom: ''
|
||||
shoes: ''
|
||||
style: ''
|
||||
测试复杂:
|
||||
name: 测试复杂
|
||||
description: '【身份】男性、青年、中国
|
||||
|
||||
【面部轮廓】鹅蛋脸、清晰分明
|
||||
|
||||
【五官细节】丹凤眼、深褐、高挺
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦'
|
||||
base: '【身份】男性、青年、中国
|
||||
|
||||
【面部轮廓】鹅蛋脸、清晰分明
|
||||
|
||||
【五官细节】丹凤眼、深褐、高挺
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦'
|
||||
personality: 默认
|
||||
voice_style: 默认
|
||||
art_style: 写实摄影风格
|
||||
face:
|
||||
shape: ''
|
||||
eyes: ''
|
||||
nose: ''
|
||||
lips: ''
|
||||
hair: '【身份】男性、青年、中国
|
||||
|
||||
【面部轮廓】鹅蛋脸、清晰分明
|
||||
|
||||
【五官细节】丹凤眼、深褐、高挺
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦'
|
||||
skin: 健康小麦
|
||||
facial_hair: ''
|
||||
glasses: ''
|
||||
distinctive: ''
|
||||
body:
|
||||
build: ''
|
||||
height: ''
|
||||
posture: ''
|
||||
age_group: ''
|
||||
clothing:
|
||||
top: ''
|
||||
inner: ''
|
||||
bottom: '【身份】男性、青年、中国
|
||||
|
||||
【面部轮廓】鹅蛋脸、清晰分明
|
||||
|
||||
【五官细节】丹凤眼、深褐、高挺
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦'
|
||||
shoes: ''
|
||||
style: ''
|
||||
测试角色:
|
||||
name: 测试角色
|
||||
description: '【身份】男性、青年、中国
|
||||
|
||||
【面部轮廓】鹅蛋脸
|
||||
|
||||
【五官细节】丹凤眼、深褐
|
||||
|
||||
【头发&皮肤】短发、黑色
|
||||
|
||||
【体型姿态】中等、标准匀称
|
||||
|
||||
【服装配饰】深灰色夹克'
|
||||
base: '【身份】男性、青年、中国
|
||||
|
||||
【面部轮廓】鹅蛋脸
|
||||
|
||||
【五官细节】丹凤眼、深褐
|
||||
|
||||
【头发&皮肤】短发、黑色
|
||||
|
||||
【体型姿态】中等、标准匀称
|
||||
|
||||
【服装配饰】深灰色夹克'
|
||||
personality: 默认
|
||||
voice_style: 默认
|
||||
art_style: 写实摄影风格
|
||||
face:
|
||||
shape: ''
|
||||
eyes: ''
|
||||
nose: ''
|
||||
lips: ''
|
||||
hair: '【身份】男性、青年、中国
|
||||
|
||||
【面部轮廓】鹅蛋脸
|
||||
|
||||
【五官细节】丹凤眼、深褐
|
||||
|
||||
【头发&皮肤】短发、黑色
|
||||
|
||||
【体型姿态】中等、标准匀称
|
||||
|
||||
【服装配饰】深灰色夹克'
|
||||
skin: ''
|
||||
facial_hair: ''
|
||||
glasses: ''
|
||||
distinctive: ''
|
||||
body:
|
||||
build: 标准
|
||||
height: ''
|
||||
posture: ''
|
||||
age_group: ''
|
||||
clothing:
|
||||
top: '【身份】男性、青年、中国
|
||||
|
||||
【面部轮廓】鹅蛋脸
|
||||
|
||||
【五官细节】丹凤眼、深褐
|
||||
|
||||
【头发&皮肤】短发、黑色
|
||||
|
||||
【体型姿态】中等、标准匀称
|
||||
|
||||
【服装配饰】深灰色夹克'
|
||||
inner: ''
|
||||
bottom: '【身份】男性、青年、中国
|
||||
|
||||
【面部轮廓】鹅蛋脸
|
||||
|
||||
【五官细节】丹凤眼、深褐
|
||||
|
||||
【头发&皮肤】短发、黑色
|
||||
|
||||
【体型姿态】中等、标准匀称
|
||||
|
||||
【服装配饰】深灰色夹克'
|
||||
shoes: ''
|
||||
style: ''
|
||||
appearance:
|
||||
- 体型:标准
|
||||
- 服装:深灰色夹克
|
||||
测试角色2:
|
||||
name: 测试角色2
|
||||
description: 测试方向描述
|
||||
base: 测试方向描述
|
||||
personality: 默认
|
||||
voice_style: 默认
|
||||
art_style: 写实摄影风格
|
||||
face:
|
||||
shape: ''
|
||||
eyes: ''
|
||||
nose: ''
|
||||
lips: ''
|
||||
hair: ''
|
||||
skin: ''
|
||||
facial_hair: ''
|
||||
glasses: ''
|
||||
distinctive: ''
|
||||
body:
|
||||
build: ''
|
||||
height: ''
|
||||
posture: ''
|
||||
age_group: ''
|
||||
clothing:
|
||||
top: ''
|
||||
inner: ''
|
||||
bottom: ''
|
||||
shoes: ''
|
||||
style: ''
|
||||
台湾:
|
||||
name: 台湾
|
||||
description: '【身份】青年、中国、少女
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
base: '【身份】青年、中国、少女
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
personality: 默认
|
||||
voice_style: 默认
|
||||
art_style: 写实摄影风格
|
||||
appearance:
|
||||
- 体型:标准
|
||||
face:
|
||||
shape: ''
|
||||
eyes: ''
|
||||
nose: ''
|
||||
lips: ''
|
||||
hair: '【身份】青年、中国、少女
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
skin: '健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
facial_hair: ''
|
||||
glasses: ''
|
||||
distinctive: ''
|
||||
body:
|
||||
build: 标准
|
||||
height: ''
|
||||
posture: 自然
|
||||
age_group: ''
|
||||
clothing:
|
||||
top: ''
|
||||
inner: '【身份】青年、中国、少女
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
bottom: '【身份】青年、中国、少女
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
shoes: ''
|
||||
style: ''
|
||||
test:
|
||||
name: test
|
||||
description: '【身份】男性
|
||||
|
||||
【面部】标准'
|
||||
base: '【身份】男性
|
||||
|
||||
【面部】标准'
|
||||
personality: 默认
|
||||
voice_style: 默认
|
||||
art_style: 写实摄影风格
|
||||
appearance:
|
||||
- 体型:标准
|
||||
face:
|
||||
shape: ''
|
||||
eyes: ''
|
||||
nose: ''
|
||||
lips: ''
|
||||
hair: ''
|
||||
skin: ''
|
||||
facial_hair: ''
|
||||
glasses: ''
|
||||
distinctive: ''
|
||||
body:
|
||||
build: 标准
|
||||
height: ''
|
||||
posture: ''
|
||||
age_group: ''
|
||||
clothing:
|
||||
top: ''
|
||||
inner: ''
|
||||
bottom: ''
|
||||
shoes: ''
|
||||
style: ''
|
||||
顺:
|
||||
name: 顺
|
||||
description: '【身份】女性、青年、中国、止
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
base: '【身份】女性、青年、中国、止
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
personality: 默认
|
||||
voice_style: 默认
|
||||
art_style: 写实摄影风格
|
||||
appearance:
|
||||
- 体型:标准
|
||||
face:
|
||||
shape: ''
|
||||
eyes: ''
|
||||
nose: ''
|
||||
lips: ''
|
||||
hair: '【身份】女性、青年、中国、止
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
skin: '健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
facial_hair: ''
|
||||
glasses: ''
|
||||
distinctive: ''
|
||||
body:
|
||||
build: 标准
|
||||
height: ''
|
||||
posture: 自然
|
||||
age_group: ''
|
||||
clothing:
|
||||
top: ''
|
||||
inner: '【身份】女性、青年、中国、止
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
bottom: '【身份】女性、青年、中国、止
|
||||
|
||||
【面部轮廓】适中自然、适中
|
||||
|
||||
【五官细节】深褐、高挺、适中、自然闭合
|
||||
|
||||
【头发&皮肤】短发、黑色、健康小麦、自然肌理、红润健康、素颜
|
||||
|
||||
【体型姿态】标准、挺拔
|
||||
|
||||
【神态&场景】柔光箱均匀布光、中焦85mm、纯灰色专业背景'
|
||||
shoes: ''
|
||||
style: ''
|
||||
t:
|
||||
name: t
|
||||
description: 测试
|
||||
base: 测试
|
||||
personality: 默认
|
||||
voice_style: 默认
|
||||
art_style: 写实摄影风格
|
||||
face:
|
||||
shape: ''
|
||||
eyes: ''
|
||||
nose: ''
|
||||
lips: ''
|
||||
hair: ''
|
||||
skin: ''
|
||||
facial_hair: ''
|
||||
glasses: ''
|
||||
distinctive: ''
|
||||
body:
|
||||
build: ''
|
||||
height: ''
|
||||
posture: ''
|
||||
age_group: ''
|
||||
clothing:
|
||||
top: ''
|
||||
inner: ''
|
||||
bottom: ''
|
||||
shoes: ''
|
||||
style: ''
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,136 @@
|
||||
"""
|
||||
composer/align.py — 画面跟随音频对齐（"剪画面"策略）
|
||||
|
||||
用户洞察（2026-10-03）：
|
||||
音画不同步时，**不该动声音**（变速会变调、裁剪会丢字），
|
||||
而应该**动画面** —— 画面变速几乎没有质量损失（atempo 保音高、setpts 改时长）。
|
||||
|
||||
适用范围（严格遵守）：
|
||||
· 有对口型的镜头 **不动**（口型已与音频绑定，变速会破坏）
|
||||
· 无音频的镜头不动
|
||||
· 变速比超出安全范围（默认 0.67~1.5 倍）→ 不动并告警（避免明显快进/慢放感）
|
||||
|
||||
用法：
|
||||
from composer.align import align_clips_to_audio
|
||||
new_clips = align_clips_to_audio(work_dir, sb, clips, lipsync_indexes={2,3})
|
||||
"""
|
||||
from pathlib import Path
|
||||
|
||||
from lib.logger import get_logger
|
||||
|
||||
log = get_logger("composer.align")
|
||||
|
||||
|
||||
def _dur(p: str) -> float:
|
||||
try:
|
||||
from lib.ffmpeg import probe_duration
|
||||
return float(probe_duration(str(p)) or 0)
|
||||
except Exception:
|
||||
return 0.0
|
||||
|
||||
|
||||
def _has_audio(p: str) -> bool:
|
||||
try:
|
||||
from lib.ffmpeg import has_audio_stream
|
||||
return bool(has_audio_stream(str(p)))
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def align_clips_to_audio(work_dir, sb, clips: list,
|
||||
lipsync_indexes: set | None = None,
|
||||
max_speed: float = 1.5,
|
||||
min_delta: float = 0.15,
|
||||
progress_cb=None) -> list:
|
||||
"""把「有台词音频、但没做对口型」的镜头**变速**到音频长度
|
||||
|
||||
work_dir : 工作目录（音频在 {work_dir}/pre_audio/{shot_id}.mp3）
|
||||
clips : 当前镜头片段路径列表（与 sb.shots 同序）
|
||||
lipsync_indexes: 已做对口型的镜头序号（1-based）——这些不动
|
||||
max_speed : 允许的最大变速倍率（默认 1.5，atempo 安全区 0.5~2.0）
|
||||
min_delta : 小于该时长差（秒）视为已对齐，不动
|
||||
|
||||
返回：新的片段路径列表（失败的镜头保持原样）
|
||||
"""
|
||||
import subprocess
|
||||
from lib.ffmpeg import FFMPEG
|
||||
|
||||
work = Path(work_dir)
|
||||
pre_dir = work / "pre_audio"
|
||||
skip = set(lipsync_indexes or ())
|
||||
out = list(clips)
|
||||
done = 0
|
||||
|
||||
for idx, shot in enumerate(list(getattr(sb, "shots", []) or []), 1):
|
||||
if idx - 1 >= len(out):
|
||||
break
|
||||
if idx in skip:
|
||||
continue # 对口型镜头不动
|
||||
sid = getattr(shot, "shot_id", f"shot_{idx:02d}")
|
||||
audio = pre_dir / f"{sid}.mp3"
|
||||
if not audio.exists():
|
||||
continue # 无音频 → 不动
|
||||
adur = _dur(str(audio))
|
||||
clip = Path(str(out[idx - 1]))
|
||||
if not clip.exists() or adur <= 0:
|
||||
continue
|
||||
vdur = _dur(str(clip))
|
||||
if vdur <= 0:
|
||||
continue
|
||||
delta = vdur - adur
|
||||
if abs(delta) < min_delta:
|
||||
continue # 已对齐
|
||||
ratio = vdur / adur # >1 = 画面偏长，需加速
|
||||
# 超出安全区**不再跳过**（跳过 = 音画依然不同步）——
|
||||
# 改为：变速到安全极限 + 裁剪剩余（画面比音频长时裁掉尾部，内容损失最小）
|
||||
if ratio > max_speed:
|
||||
applied = max_speed
|
||||
need_trim = True
|
||||
elif ratio < 1.0 / max_speed:
|
||||
applied = 1.0 / max_speed
|
||||
need_trim = False # 画面太短：变速到极限即可（不要补黑）
|
||||
else:
|
||||
applied = ratio
|
||||
need_trim = abs(delta) > min_delta
|
||||
|
||||
dst = work / f"aligned_{idx:02d}.mp4"
|
||||
if progress_cb:
|
||||
progress_cb("align", sid, "running", f"画面变速 {applied:.2f}x")
|
||||
|
||||
# 视频变速用 setpts；音频（若存在）用 atempo 同步变速
|
||||
vf = f"[0:v]setpts=PTS/{applied:.4f}[v]"
|
||||
if _has_audio(str(clip)):
|
||||
ap = min(max(applied, 0.5), 2.0) # atempo 安全区 0.5~2.0
|
||||
fc = f"{vf};[0:a]atempo={ap:.4f}[a]"
|
||||
maps = ["-map", "[v]", "-map", "[a]", "-c:a", "aac", "-b:a", "192k"]
|
||||
else:
|
||||
fc = vf
|
||||
maps = ["-map", "[v]"]
|
||||
|
||||
cmd = [FFMPEG, "-y", "-i", str(clip),
|
||||
"-filter_complex", fc, *maps,
|
||||
"-c:v", "libx264", "-preset", "fast", "-crf", "20",
|
||||
"-pix_fmt", "yuv420p"]
|
||||
if need_trim and applied > 1.0:
|
||||
# 变速后仍偏长 → 裁到音频长度（保留开头，丢尾部多余部分）
|
||||
cmd += ["-t", f"{adur:.3f}"]
|
||||
cmd.append(str(dst))
|
||||
|
||||
r = subprocess.run(cmd, capture_output=True, text=True, timeout=600)
|
||||
|
||||
if r.returncode != 0 or not dst.exists() or dst.stat().st_size < 1_000:
|
||||
log.warning(" ⚠️ %s 变速失败，保留原片段: %s", sid, (r.stderr or "")[-160:])
|
||||
continue
|
||||
|
||||
new_dur = _dur(str(dst))
|
||||
out[idx - 1] = str(dst)
|
||||
done += 1
|
||||
_note = f"{applied:.2f}x" + ("+裁尾" if (need_trim and applied > 1.0) else "")
|
||||
log.info(" ✂️ %s 画面对齐：%.2fs → %.2fs（%s，音频 %.2fs）",
|
||||
sid, vdur, new_dur, _note, adur)
|
||||
if progress_cb:
|
||||
progress_cb("align", sid, "done", f"{vdur:.1f}s → {new_dur:.1f}s（{_note}）")
|
||||
|
||||
if done:
|
||||
log.info(" ✂️ 画面跟随音频对齐完成：%d 个镜头", done)
|
||||
return out
|
||||
@@ -0,0 +1,678 @@
|
||||
"""
|
||||
AVE 06_composer/beat_sync — BGM 节拍驱动的卡点视频合成
|
||||
|
||||
流程:
|
||||
1. BGM 节拍检测 (librosa beat_track)
|
||||
2. 按 N拍 分组 (默认 4拍/组)
|
||||
3. 每组分配素材 (Pexels搜索 或 用户提供)
|
||||
4. 精确裁剪到组时长
|
||||
5. xfade 过渡拼接 (≤8组避免 FFmpeg 崩溃)
|
||||
6. 文字/字幕叠加
|
||||
|
||||
用法:
|
||||
from composer.beat_sync import compose_beat_sync
|
||||
compose_beat_sync(bgm_path="bgm.wav", output="beat_sync.mp4")
|
||||
"""
|
||||
import sys, os; sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from lib.logger import get_logger
|
||||
from lib.ffmpeg import FFMPEG # noqa: E402 (FFmpeg 绝对路径)
|
||||
|
||||
logger = get_logger("beat_sync")
|
||||
|
||||
CACHE_DIR = Path(os.environ.get("AVE_CACHE_DIR",
|
||||
str(Path.home() / "workbuddy-agent-os/agent-local/tools/ave/cache")))
|
||||
BEAT_CACHE = CACHE_DIR / "beat_sync"
|
||||
BEAT_CACHE.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def detect_beats(
|
||||
audio_path: str,
|
||||
group_size: int = 4,
|
||||
enable_energy: bool = False,
|
||||
detect_structure: bool = False,
|
||||
) -> dict:
|
||||
"""
|
||||
检测 BGM 节拍
|
||||
|
||||
参数:
|
||||
audio_path: BGM 音频路径
|
||||
group_size: 每组节拍数 (默认4拍/组)
|
||||
enable_energy: 是否提取 RMS/频谱能量特征 (用于变速卡点)
|
||||
detect_structure: 是否检测歌曲结构 (Intro/Verse/Chorus/Bridge/Outro)
|
||||
|
||||
返回:
|
||||
{
|
||||
"bpm": 120.0,
|
||||
"beat_times": [0.0, 0.5, 1.0, ...],
|
||||
"segments": [{"start": 0.0, "end": 2.0, "beats": 4, "energy": 0.8, ...}, ...],
|
||||
"structure": [{"label": "Intro", "start": 0.0, "end": 8.0, ...}],
|
||||
"total_duration": 30.0,
|
||||
}
|
||||
"""
|
||||
import librosa
|
||||
import numpy as np
|
||||
|
||||
y, sr = librosa.load(audio_path, sr=22050, mono=True)
|
||||
duration = librosa.get_duration(y=y, sr=sr)
|
||||
|
||||
# 节拍检测
|
||||
tempo_arr, beats = librosa.beat.beat_track(y=y, sr=sr)
|
||||
tempo_val = float(tempo_arr.item() if hasattr(tempo_arr, 'item') else tempo_arr)
|
||||
beat_times = librosa.frames_to_time(beats, sr=sr).tolist()
|
||||
|
||||
# 如果没检测到拍点, 按 BPM 估算
|
||||
if len(beat_times) < 2:
|
||||
bpm = tempo_val if tempo_val > 0 else 120.0
|
||||
interval = 60.0 / bpm
|
||||
beat_times = [i * interval for i in range(int(duration / interval))]
|
||||
logger.info(f" 无拍点, 按 BPM={bpm:.0f} 估算 {len(beat_times)} 拍")
|
||||
else:
|
||||
logger.info(f" 检测到 BPM={tempo_val:.1f}, {len(beat_times)} 拍")
|
||||
|
||||
# 提取 RMS 和频谱能量
|
||||
rms_energy = None
|
||||
spectral_features = None
|
||||
if enable_energy:
|
||||
# 用短时傅立叶提取帧级能量
|
||||
hop_length = 512
|
||||
frame_dur = hop_length / sr
|
||||
rms = librosa.feature.rms(y=y, hop_length=hop_length)[0]
|
||||
rms_energy = rms.tolist()
|
||||
logger.debug(f" 能量帧: {len(rms_energy)} 帧, 帧时长={frame_dur:.3f}s")
|
||||
|
||||
# 频谱 flux
|
||||
spec = np.abs(librosa.stft(y, hop_length=hop_length))
|
||||
flux = librosa.onset.onset_strength(y=y, sr=sr, hop_length=hop_length)
|
||||
spectral_features = {
|
||||
"rms": rms_energy,
|
||||
"flux": flux.tolist() if hasattr(flux, 'tolist') else [],
|
||||
"hop_length": hop_length,
|
||||
"frame_duration": frame_dur,
|
||||
}
|
||||
|
||||
# 歌曲结构检测
|
||||
song_structure = None
|
||||
if detect_structure:
|
||||
song_structure = _detect_song_structure(y, sr, duration)
|
||||
if song_structure:
|
||||
logger.info(f" 歌曲结构: {len(song_structure)} 段")
|
||||
for s in song_structure:
|
||||
logger.info(f" {s['label']:8s} {s['start']:.0f}s→{s['end']:.0f}s")
|
||||
|
||||
# 按 N拍 分组
|
||||
segments = []
|
||||
max_beats_per_seg = 0
|
||||
for i in range(0, len(beat_times), group_size):
|
||||
start = beat_times[i]
|
||||
end = beat_times[min(i + group_size, len(beat_times) - 1)]
|
||||
actual_beats = min(group_size, len(beat_times) - i)
|
||||
if end - start > 0.3:
|
||||
seg = {
|
||||
"start": round(start, 2),
|
||||
"end": round(end, 2),
|
||||
"duration": round(end - start, 2),
|
||||
"beats": actual_beats,
|
||||
}
|
||||
# 计算该段平均能量
|
||||
if enable_energy and rms_energy is not None and spectral_features:
|
||||
seg["energy"] = _segment_energy(start, end, spectral_features)
|
||||
segments.append(seg)
|
||||
max_beats_per_seg = max(max_beats_per_seg, actual_beats)
|
||||
|
||||
# 最后一段: 最后一个拍点到音频结束
|
||||
if segments and segments[-1]["end"] < duration - 0.3:
|
||||
seg = {
|
||||
"start": segments[-1]["end"],
|
||||
"end": round(duration, 2),
|
||||
"duration": round(duration - segments[-1]["end"], 2),
|
||||
"beats": 0,
|
||||
}
|
||||
if enable_energy and spectral_features:
|
||||
seg["energy"] = _segment_energy(seg["start"], seg["end"], spectral_features)
|
||||
segments.append(seg)
|
||||
|
||||
# 限制 ≤8 组 (xfade 链限制)
|
||||
if len(segments) > 8:
|
||||
keep = segments[:7]
|
||||
remaining = segments[7:]
|
||||
merged = {
|
||||
"start": keep[-1]["end"],
|
||||
"end": round(duration, 2),
|
||||
"duration": round(duration - keep[-1]["end"], 2),
|
||||
"beats": sum(s["beats"] for s in remaining),
|
||||
}
|
||||
if enable_energy and "energy" in remaining[0]:
|
||||
merged["energy"] = sum(
|
||||
s.get("energy", 0) * s["duration"] for s in remaining
|
||||
) / max(merged["duration"], 0.1)
|
||||
keep.append(merged)
|
||||
segments = keep
|
||||
logger.info(f" 合并为 {len(segments)} 组 (≤8)")
|
||||
|
||||
result = {
|
||||
"bpm": tempo_val,
|
||||
"beat_count": len(beat_times),
|
||||
"beat_times": [round(float(t), 3) for t in beat_times], # 文档承诺的字段（此前漏返回）
|
||||
"segments": segments,
|
||||
"total_duration": round(duration, 2),
|
||||
}
|
||||
if song_structure:
|
||||
result["structure"] = song_structure
|
||||
if spectral_features:
|
||||
result["spectral"] = spectral_features
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def _segment_energy(start_sec: float, end_sec: float,
|
||||
spectral: dict) -> float:
|
||||
"""计算某时间段的平均归一化能量 (基于 RMS + Flux)"""
|
||||
frame_dur = spectral["frame_duration"]
|
||||
rms = spectral.get("rms", [])
|
||||
flux = spectral.get("flux", [])
|
||||
|
||||
if not rms:
|
||||
return 0.5
|
||||
|
||||
start_frame = int(start_sec / max(frame_dur, 0.001))
|
||||
end_frame = int(end_sec / max(frame_dur, 0.001))
|
||||
start_frame = max(0, min(start_frame, len(rms) - 1))
|
||||
end_frame = max(start_frame + 1, min(end_frame, len(rms)))
|
||||
|
||||
seg_rms = rms[start_frame:end_frame]
|
||||
if not seg_rms:
|
||||
return 0.5
|
||||
|
||||
avg_rms = sum(seg_rms) / len(seg_rms) if seg_rms else 0
|
||||
avg_flux = sum(flux[start_frame:end_frame]) / len(seg_rms) if flux and len(seg_rms) > 0 else 0
|
||||
|
||||
# 归一化: 用全局均值和 std
|
||||
rms_mean = sum(rms) / len(rms) if rms else 0.1
|
||||
rms_std = (sum((v - rms_mean)**2 for v in rms) / len(rms)) ** 0.5 if rms and len(rms) > 1 else rms_mean
|
||||
rms_std = max(rms_std, rms_mean * 0.01)
|
||||
|
||||
energy = (avg_rms - rms_mean) / rms_std * 0.3 + 0.5
|
||||
return max(0.0, min(1.0, energy))
|
||||
|
||||
|
||||
def _detect_song_structure(y, sr, duration) -> list[dict]:
|
||||
"""
|
||||
检测歌曲结构段落
|
||||
|
||||
基于 chroma 特征 + 自相似矩阵 + 时序聚类,
|
||||
输出: Intro / Verse / Chorus / Bridge / Outro
|
||||
"""
|
||||
import numpy as np
|
||||
import librosa
|
||||
|
||||
try:
|
||||
# 提取 chroma 特征
|
||||
hop_length = 2048
|
||||
chroma = librosa.feature.chroma_cqt(y=y, sr=sr, hop_length=hop_length)
|
||||
frame_dur = hop_length / sr
|
||||
|
||||
# 自相似矩阵 + 递归分割
|
||||
ssm = librosa.segment.recurrence_matrix(chroma, axis=0)
|
||||
bounds = librosa.segment.agglomerative(ssm, k=min(8, ssm.shape[0]))
|
||||
bounds_frames = librosa.segment.subsegment(ssm, bounds, n_segments=2)
|
||||
|
||||
# 转换为时间
|
||||
boundaries = [0.0]
|
||||
for bf in bounds_frames:
|
||||
t = float(bf) * frame_dur
|
||||
if t < duration - 1:
|
||||
boundaries.append(round(t, 1))
|
||||
boundaries.append(round(duration, 1))
|
||||
|
||||
# 按时间比例分配标签
|
||||
labels = _label_structure(boundaries, duration)
|
||||
|
||||
result = []
|
||||
for i in range(len(boundaries) - 1):
|
||||
result.append({
|
||||
"label": labels[i] if i < len(labels) else "Section",
|
||||
"start": boundaries[i],
|
||||
"end": boundaries[i + 1],
|
||||
"duration": round(boundaries[i + 1] - boundaries[i], 1),
|
||||
})
|
||||
|
||||
return result
|
||||
except Exception as e:
|
||||
logger.warning(f" 歌曲结构检测失败: {e}")
|
||||
return []
|
||||
|
||||
|
||||
def _label_structure(boundaries: list[float], total_duration: float) -> list[str]:
|
||||
"""根据段落在歌曲中的位置分配典型结构标签"""
|
||||
n = len(boundaries) - 1
|
||||
labels = []
|
||||
|
||||
# 按位置比例分配
|
||||
positions = []
|
||||
for i in range(n):
|
||||
mid = (boundaries[i] + boundaries[i + 1]) / 2 / max(total_duration, 1)
|
||||
positions.append(mid)
|
||||
|
||||
for i, pos in enumerate(positions):
|
||||
if i == 0 and pos < 0.05:
|
||||
labels.append("Intro")
|
||||
elif i >= n - 2 and pos > 0.85:
|
||||
labels.append("Outro")
|
||||
elif 0.15 <= pos <= 0.65:
|
||||
# 很可能在 Chorus 区
|
||||
labels.append("Chorus" if i % 2 == 1 else "Verse")
|
||||
elif pos < 0.15:
|
||||
labels.append("Verse")
|
||||
elif pos < 0.4:
|
||||
labels.append("Pre-Chorus" if i % 2 == 0 else "Chorus")
|
||||
else:
|
||||
labels.append("Bridge" if pos < 0.75 else "Outro")
|
||||
|
||||
return labels
|
||||
|
||||
|
||||
def compose_beat_sync(
|
||||
bgm_path: str,
|
||||
output_path: str = "beat_sync.mp4",
|
||||
material_clips: list[str] | None = None,
|
||||
group_size: int = 4,
|
||||
resolution: str = "1080x1920",
|
||||
texts: list[str] | None = None,
|
||||
pexels_api_key: str = "",
|
||||
pexels_search: str = "",
|
||||
enable_speed_ramp: bool = False,
|
||||
frame_locked: bool = True,
|
||||
base_speed: float = 1.0,
|
||||
high_speed: float = 1.5,
|
||||
low_speed: float = 0.7,
|
||||
target_bpm: float = 0.0,
|
||||
) -> str:
|
||||
"""
|
||||
BGM 节拍驱动的卡点视频合成 (V2: 能量感知 + 帧锁定)
|
||||
|
||||
参数:
|
||||
bgm_path: BGM 音频路径
|
||||
output_path: 输出视频路径
|
||||
material_clips: 素材视频/图片列表 (可选, 不提供则 Pexels 搜索)
|
||||
group_size: 每组节拍数
|
||||
resolution: 输出分辨率
|
||||
texts: 每段叠加的文字 (可选)
|
||||
pexels_api_key: Pexels API Key
|
||||
pexels_search: Pexels 搜索关键词
|
||||
enable_speed_ramp: 是否启用能量驱动变速 (高能加速, 低能减速)
|
||||
frame_locked: 是否启用帧锁定时间线 (消除 xfade 漂移)
|
||||
base_speed: 基础速度 (默认 1.0)
|
||||
high_speed: 高能段速度 (默认 1.5)
|
||||
low_speed: 低能段速度 (默认 0.7)
|
||||
target_bpm: 目标 BPM (>0 时用 AudioLine StemAligner 预处理 BGM)
|
||||
|
||||
返回: output_path
|
||||
"""
|
||||
if not os.path.exists(bgm_path):
|
||||
raise FileNotFoundError(f"BGM 文件不存在: {bgm_path}")
|
||||
|
||||
# 0. BPM 标准化预处理 (使用 AudioLine StemAligner)
|
||||
if target_bpm > 0:
|
||||
try:
|
||||
from audio_line.layer4_alignment.stem_aligner import StemAligner
|
||||
import librosa
|
||||
y_detect, sr_detect = librosa.load(bgm_path, sr=None, mono=True)
|
||||
tempo_arr, _ = librosa.beat.beat_track(y=y_detect, sr=sr_detect)
|
||||
source_bpm = float(tempo_arr.item() if hasattr(tempo_arr, 'item') else tempo_arr)
|
||||
logger.info(f" BPM 标准化: {source_bpm:.0f} → {target_bpm:.0f}")
|
||||
|
||||
if source_bpm > 0 and abs(source_bpm - target_bpm) > 1.0:
|
||||
aligner = StemAligner(method="phase_vocoder")
|
||||
result = aligner.align_to_bpm(
|
||||
stems={"bgm": bgm_path},
|
||||
source_bpm=source_bpm,
|
||||
target_bpm=target_bpm,
|
||||
output_dir=str(BEAT_CACHE / "aligned"),
|
||||
)
|
||||
if result.aligned_paths:
|
||||
bgm_path = result.aligned_paths["bgm"]
|
||||
logger.info(f" ✅ 标准化完成: {bgm_path}")
|
||||
else:
|
||||
logger.info(f" ⏭️ 无需标准化 (相近 BPM)")
|
||||
except ImportError:
|
||||
logger.info(f" ⏭️ AudioLine 不可用, 跳过 BPM 标准化")
|
||||
except Exception as e:
|
||||
logger.warning(f" ⚠ BPM 标准化失败: {e} (继续使用原 BGM)")
|
||||
|
||||
# 1. 节拍检测 (启用能量)
|
||||
logger.info("=== Beat-Sync 卡点模式 ===")
|
||||
logger.info(f"BGM: {bgm_path}")
|
||||
beat_info = detect_beats(bgm_path, group_size, enable_energy=enable_speed_ramp)
|
||||
segments = beat_info["segments"]
|
||||
total_dur = beat_info["total_duration"]
|
||||
logger.info(f"BPM: {beat_info['bpm']}, {beat_info['beat_count']}拍, {len(segments)}组")
|
||||
|
||||
if enable_speed_ramp and "energy" in segments[0]:
|
||||
energies = [s.get("energy", 0.5) for s in segments]
|
||||
avg_e = sum(energies) / len(energies) if energies else 0.5
|
||||
logger.info(f" 能量感知变速: ON (均值 {avg_e:.2f}, 范围 {min(energies):.2f}~{max(energies):.2f})")
|
||||
|
||||
# 2. 准备素材
|
||||
clips = material_clips or []
|
||||
if len(clips) < len(segments) and pexels_api_key:
|
||||
from material_producer.pexels.search import search_videos
|
||||
needed = len(segments) - len(clips)
|
||||
logger.info(f"素材不足, 从 Pexels 搜索 {needed} 个...")
|
||||
kw = pexels_search or "background video"
|
||||
found = search_videos(kw, count=needed, api_key=pexels_api_key, orientation="portrait")
|
||||
for f in found:
|
||||
clips.append(f["path"])
|
||||
logger.info(f" 共 {len(clips)} 个素材")
|
||||
|
||||
if not clips:
|
||||
raise ValueError("无可用素材")
|
||||
|
||||
# 3. 为每组分配素材 (循环使用)
|
||||
seg_materials = [] # [(clip_path, adjusted_start, adjusted_end, adjusted_duration, speed), ...]
|
||||
for i, seg in enumerate(segments):
|
||||
clip = clips[i % len(clips)]
|
||||
|
||||
# 计算本段速度 (能量驱动变速)
|
||||
speed = base_speed
|
||||
if enable_speed_ramp:
|
||||
energy = seg.get("energy", 0.5)
|
||||
if energy > 0.6:
|
||||
speed = base_speed + (high_speed - base_speed) * min((energy - 0.6) / 0.4, 1.0)
|
||||
elif energy < 0.3:
|
||||
speed = low_speed + (base_speed - low_speed) * (energy / 0.3)
|
||||
speed = max(0.5, min(2.0, speed))
|
||||
|
||||
# 调整时长 (变速后实际画面时长)
|
||||
orig_dur = seg["duration"]
|
||||
if frame_locked:
|
||||
# 帧锁定: 以 BGM 节拍为准, 用 fps 对齐到整帧
|
||||
fps = 30
|
||||
orig_frames = round(orig_dur * fps)
|
||||
adjusted_dur = orig_frames / fps # 帧对齐后的时长
|
||||
else:
|
||||
adjusted_dur = orig_dur
|
||||
|
||||
# 变速后的素材播放时长 = adjusted_dur / speed
|
||||
# 即: 如果加速 (speed>1), 需要更短的素材
|
||||
material_time_needed = adjusted_dur / speed if speed > 0 else adjusted_dur
|
||||
|
||||
seg_materials.append((clip, seg["start"], seg["end"],
|
||||
adjusted_dur, speed, material_time_needed))
|
||||
|
||||
logger.info(f"分配素材: {len(seg_materials)} 段, {len(clips)} 素材循环"
|
||||
+ (f", 变速范围 {low_speed:.1f}~{high_speed:.1f}x" if enable_speed_ramp else ""))
|
||||
|
||||
# 4. 帧锁定: 预计算精确的 xfade 偏移
|
||||
if frame_locked:
|
||||
fps = 30
|
||||
# 精确计算每段的累计偏移
|
||||
cumulative_frames = 0
|
||||
for idx, (clip, start, end, dur, speed, mt) in enumerate(seg_materials):
|
||||
seg_frames = round(dur * fps)
|
||||
seg_materials[idx] = (clip, start, dur, speed, mt, seg_frames)
|
||||
logger.info(f" 帧锁定: ON (30fps)")
|
||||
|
||||
# 5. 渲染每组为独立视频片段 (带变速)
|
||||
temp_dir = tempfile.mkdtemp(prefix="ave_beat_")
|
||||
seg_files = []
|
||||
for idx in range(len(seg_materials)):
|
||||
clip, start_orig, dur, speed, mt, *rest = seg_materials[idx]
|
||||
if frame_locked:
|
||||
seg_frames = rest[0] if rest else round(dur * 30)
|
||||
# 帧对齐时长
|
||||
dur_aligned = seg_frames / 30.0
|
||||
else:
|
||||
dur_aligned = dur
|
||||
seg_frames = 0
|
||||
|
||||
seg_file = os.path.join(temp_dir, f"seg_{idx:03d}.mp4")
|
||||
|
||||
# 从素材中取一段 (考虑变速需要)
|
||||
clip_dur = _get_duration(clip)
|
||||
offset = (start_orig * 17) % max(clip_dur - mt, 0.1) if clip_dur > mt else 0
|
||||
|
||||
# 先截取素材, 再变速
|
||||
raw_seg = os.path.join(temp_dir, f"seg_{idx:03d}_raw.mp4")
|
||||
subprocess.run([
|
||||
FFMPEG, "-y",
|
||||
"-ss", str(offset),
|
||||
"-i", clip,
|
||||
"-t", str(mt),
|
||||
"-vf", f"scale={resolution.replace('x',':')},setsar=1,fps=30",
|
||||
"-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-an", raw_seg,
|
||||
], capture_output=True, timeout=120, check=True)
|
||||
|
||||
if speed != 1.0 and speed >= 0.5 and speed <= 2.0:
|
||||
# 变速处理
|
||||
subprocess.run([
|
||||
FFMPEG, "-y", "-i", raw_seg,
|
||||
"-filter_complex",
|
||||
f"setpts={1/speed}*PTS[v]",
|
||||
"-map", "[v]",
|
||||
"-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-pix_fmt", "yuv420p",
|
||||
seg_file,
|
||||
], capture_output=True, timeout=120, check=True)
|
||||
else:
|
||||
os.replace(raw_seg, seg_file)
|
||||
|
||||
seg_files.append(seg_file)
|
||||
|
||||
# 6. 使用帧锁定时间线拼接
|
||||
if len(seg_files) == 1:
|
||||
subprocess.run([FFMPEG, "-y", "-i", seg_files[0], "-i", bgm_path,
|
||||
"-shortest", "-c:v", "copy", "-c:a", "aac", output_path],
|
||||
capture_output=True, check=True, timeout=60)
|
||||
return _finalize(output_path, texts, seg_materials, resolution, temp_dir)
|
||||
|
||||
if frame_locked:
|
||||
return _concat_frame_locked(seg_files, bgm_path, seg_materials,
|
||||
output_path, texts, resolution, temp_dir)
|
||||
else:
|
||||
return _concat_xfade(seg_files, bgm_path, seg_materials,
|
||||
output_path, texts, resolution, temp_dir)
|
||||
|
||||
|
||||
def _concat_frame_locked(
|
||||
seg_files: list[str],
|
||||
bgm_path: str,
|
||||
seg_materials: list,
|
||||
output_path: str,
|
||||
texts: list[str] | None,
|
||||
resolution: str,
|
||||
temp_dir: str,
|
||||
) -> str:
|
||||
"""帧锁定拼接: 用 concat demuxer 精确拼接, 无 xfade 漂移"""
|
||||
if len(seg_files) == 1:
|
||||
# 只有一段, 直接合并 BGM
|
||||
cmd = [FFMPEG, "-y", "-i", seg_files[0], "-i", bgm_path,
|
||||
"-shortest", "-c:v", "copy", "-c:a", "aac", "-b:a", "192k", output_path]
|
||||
subprocess.run(cmd, capture_output=True, timeout=120, check=True)
|
||||
return _finalize(output_path, texts, seg_materials, resolution, temp_dir)
|
||||
|
||||
# 用 concat demuxer 拼接 (帧精确, 无过渡)
|
||||
concat_file = tempfile.NamedTemporaryFile(suffix=".txt", delete=False, mode="w").name
|
||||
with open(concat_file, "w") as f:
|
||||
for sf in seg_files:
|
||||
f.write(f"file '{os.path.abspath(sf)}'\n")
|
||||
|
||||
# 先拼视频
|
||||
video_only = output_path.replace(".mp4", "_novideo.mp4")
|
||||
cmd = [FFMPEG, "-y", "-f", "concat", "-safe", "0", "-i", concat_file,
|
||||
"-c:v", "libx264", "-preset", "fast", "-crf", "23", "-an", video_only]
|
||||
subprocess.run(cmd, capture_output=True, timeout=300, check=True)
|
||||
|
||||
# 叠加 BGM
|
||||
cmd2 = [FFMPEG, "-y", "-i", video_only, "-i", bgm_path,
|
||||
"-shortest", "-c:v", "copy", "-c:a", "aac", "-b:a", "192k", output_path]
|
||||
subprocess.run(cmd2, capture_output=True, timeout=120, check=True)
|
||||
|
||||
os.unlink(concat_file)
|
||||
os.unlink(video_only)
|
||||
|
||||
logger.info(f" ✅ 帧锁定拼接完成: {len(seg_files)} 段")
|
||||
return _finalize(output_path, texts, seg_materials, resolution, temp_dir)
|
||||
|
||||
|
||||
def _concat_xfade(
|
||||
seg_files: list[str],
|
||||
bgm_path: str,
|
||||
seg_materials: list,
|
||||
output_path: str,
|
||||
texts: list[str] | None,
|
||||
resolution: str,
|
||||
temp_dir: str,
|
||||
) -> str:
|
||||
"""xfade 过渡拼接: ≤8 段, 逐段交叉淡入淡出"""
|
||||
if len(seg_files) == 1:
|
||||
cmd = [FFMPEG, "-y", "-i", seg_files[0], "-i", bgm_path,
|
||||
"-shortest", "-c:v", "copy", "-c:a", "aac", "-b:a", "192k", output_path]
|
||||
subprocess.run(cmd, capture_output=True, timeout=120, check=True)
|
||||
return _finalize(output_path, texts, seg_materials, resolution, temp_dir)
|
||||
|
||||
# 构建 xfade 链
|
||||
# xfade=transition=fade:duration=0.5:offset=offset1
|
||||
# 每段重叠 0.5s, 累计偏移
|
||||
filter_parts = []
|
||||
inputs = []
|
||||
offset = 0.0
|
||||
prev_label = "v0"
|
||||
|
||||
for i in range(len(seg_files)):
|
||||
inputs += ["-i", seg_files[i]]
|
||||
if i == 0:
|
||||
continue
|
||||
# 前一段的时长
|
||||
seg_dur = seg_materials[i - 1][2] if i - 1 < len(seg_materials) else 1.0
|
||||
offset += float(seg_dur) - 0.5 # 减去过渡重叠
|
||||
label = f"v{i}"
|
||||
filter_parts.append(
|
||||
f"[{prev_label}][{label}]xfade=transition=fade:duration=0.5:offset={offset:.2f}[v{i}]"
|
||||
)
|
||||
prev_label = f"v{i}"
|
||||
|
||||
# 最终输出标签是 v{len(seg_files)-1}
|
||||
last_label = f"v{len(seg_files) - 1}"
|
||||
full_filter = ";".join(filter_parts)
|
||||
|
||||
video_xfade = output_path.replace(".mp4", "_xfade.mp4")
|
||||
cmd = [FFMPEG, "-y"] + inputs + [
|
||||
"-filter_complex", full_filter,
|
||||
"-map", f"[{last_label}]",
|
||||
"-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-pix_fmt", "yuv420p",
|
||||
video_xfade,
|
||||
]
|
||||
try:
|
||||
subprocess.run(cmd, capture_output=True, timeout=300, check=True)
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.warning(f" xfade 失败, 回落 concat: {e.stderr[:200]}")
|
||||
# 回落: concat demuxer
|
||||
concat_file = tempfile.NamedTemporaryFile(suffix=".txt", delete=False, mode="w").name
|
||||
with open(concat_file, "w") as f:
|
||||
for sf in seg_files:
|
||||
f.write(f"file '{os.path.abspath(sf)}'\n")
|
||||
cmd_fb = [FFMPEG, "-y", "-f", "concat", "-safe", "0", "-i", concat_file,
|
||||
"-c:v", "libx264", "-preset", "fast", "-crf", "23", video_xfade]
|
||||
subprocess.run(cmd_fb, capture_output=True, timeout=300, check=True)
|
||||
os.unlink(concat_file)
|
||||
|
||||
# 叠加 BGM
|
||||
cmd2 = [FFMPEG, "-y", "-i", video_xfade, "-i", bgm_path,
|
||||
"-shortest", "-c:v", "copy", "-c:a", "aac", "-b:a", "192k", output_path]
|
||||
subprocess.run(cmd2, capture_output=True, timeout=120, check=True)
|
||||
os.unlink(video_xfade)
|
||||
|
||||
logger.info(f" ✅ xfade 拼接完成: {len(seg_files)} 段")
|
||||
return _finalize(output_path, texts, seg_materials, resolution, temp_dir)
|
||||
|
||||
|
||||
def _finalize(
|
||||
output_path: str,
|
||||
texts: list[str] | None,
|
||||
seg_materials: list,
|
||||
resolution: str,
|
||||
temp_dir: str,
|
||||
) -> str:
|
||||
"""最终处理: 叠加文字字幕 + 清理临时目录"""
|
||||
import shutil
|
||||
|
||||
# 叠加文字
|
||||
if texts and len(texts) > 0 and seg_materials:
|
||||
_add_text_subtitles(output_path, texts, seg_materials, resolution)
|
||||
logger.info(f" ✅ 文字叠加: {len(texts)} 段")
|
||||
|
||||
# 清理临时目录
|
||||
if temp_dir and os.path.exists(temp_dir):
|
||||
shutil.rmtree(temp_dir, ignore_errors=True)
|
||||
logger.debug(f" 临时目录已清理: {temp_dir}")
|
||||
|
||||
return output_path
|
||||
|
||||
|
||||
def _get_duration(path: str) -> float:
|
||||
"""获取媒体文件时长（秒）
|
||||
|
||||
用 lib.ffmpeg.probe_duration（ffmpeg -i 解析）——
|
||||
本机 ffprobe 实为 ffmpeg 的符号链接，不支持 -show_entries（会静默失败）
|
||||
"""
|
||||
try:
|
||||
from lib.ffmpeg import probe_duration
|
||||
d = probe_duration(path)
|
||||
return d if d > 0 else 30.0
|
||||
except Exception:
|
||||
return 30.0
|
||||
|
||||
|
||||
def _add_text_subtitles(video_path: str, texts: list[str],
|
||||
seg_materials: list, resolution: str):
|
||||
"""为每段叠加文字 (ASS 字幕)"""
|
||||
width, height = (int(x) for x in resolution.split("x"))
|
||||
font_size = max(48, int(height * 0.05))
|
||||
margin_v = int(height * 0.3)
|
||||
|
||||
lines = [
|
||||
"[Script Info]", "ScriptType: v4.00+",
|
||||
f"PlayResX: {width}", f"PlayResY: {height}",
|
||||
"ScaledBorderAndShadow: yes", "",
|
||||
"[V4+ Styles]",
|
||||
"Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding",
|
||||
f"Style: Default,Noto Sans SC,{font_size},&H00FFFFFF,&H0000FFFF,&H00000000,&H80000000,-1,0,0,0,100,100,0,0,1,3,1,2,100,100,{margin_v},1",
|
||||
"", "[Events]",
|
||||
"Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text",
|
||||
]
|
||||
|
||||
for i, text in enumerate(texts):
|
||||
if i >= len(seg_materials):
|
||||
break
|
||||
start = seg_materials[i][1]
|
||||
end = seg_materials[i][2]
|
||||
h = int(start); m = int((start - h) * 60); s = int(start - h * 3600 - m * 60)
|
||||
he = int(end); me = int((end - he) * 60); se = int(end - he * 3600 - me * 60)
|
||||
lines.append(
|
||||
f"Dialogue: 0,{h}:{m:02d}:{s:02d}.00,{he}:{me:02d}:{se:02d}.00,Default,,0,0,0,,{text}"
|
||||
)
|
||||
|
||||
ass_path = video_path.replace(".mp4", "_text.ass")
|
||||
with open(ass_path, "w", encoding="utf-8") as f:
|
||||
f.write("\n".join(lines))
|
||||
|
||||
# 叠加字幕
|
||||
tmp = video_path.replace(".mp4", "_tmp.mp4")
|
||||
subprocess.run([
|
||||
FFMPEG, "-y", "-i", video_path,
|
||||
"-vf", f"subtitles={ass_path}",
|
||||
"-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-c:a", "copy", "-pix_fmt", "yuv420p", tmp,
|
||||
], capture_output=True, timeout=120, check=True)
|
||||
os.replace(tmp, video_path)
|
||||
os.remove(ass_path)
|
||||
@@ -0,0 +1,68 @@
|
||||
"""
|
||||
AVE character_locker — 角色描述块锁定器
|
||||
|
||||
职责:
|
||||
在全片 Prompt 链中自动注入固定角色描述块，
|
||||
确保 Kling 文生视频时角色一致性。
|
||||
|
||||
用法:
|
||||
from composer.character_locker import inject_character_block
|
||||
prompt = inject_character_block(segments, character_desc)
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
# ── 角色库路径 ──
|
||||
CHARACTER_DB = Path(os.environ.get("AVE_CACHE_DIR",
|
||||
str(Path.home() / "workbuddy-agent-os/agent-local/tools/ave/cache/character_sheet/characters")))
|
||||
CHARACTER_DB.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def load_character_block(char_name: str) -> str:
|
||||
"""从角色库加载描述块"""
|
||||
safe = char_name.replace(" ", "_").replace("/", "_")
|
||||
path = CHARACTER_DB / f"{safe}.json"
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"角色 '{char_name}' 不存在于角色库: {path}")
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
return data["description"]
|
||||
|
||||
|
||||
def inject_character_block(segments: list[dict], character_desc: str) -> list[dict]:
|
||||
"""
|
||||
在每段素材描述前注入角色描述块
|
||||
|
||||
参数:
|
||||
segments: 导演脚本的 segments 列表
|
||||
character_desc: 角色描述块（从定妆照生成）
|
||||
|
||||
返回:
|
||||
注入后的 segments（直接修改原列表）
|
||||
"""
|
||||
for seg in segments:
|
||||
mat = seg.get("material", {})
|
||||
search = mat.get("search", "")
|
||||
# 如果已包含角色描述, 跳过
|
||||
if character_desc in search:
|
||||
continue
|
||||
# 注入: 角色描述 + 场景描述
|
||||
if search:
|
||||
mat["search_locked"] = search # 保留原搜索词
|
||||
mat["search"] = f"{character_desc}, {search}"
|
||||
else:
|
||||
mat["search"] = character_desc
|
||||
|
||||
return segments
|
||||
|
||||
|
||||
def lock_prompt(prompt: str, character_desc: str) -> str:
|
||||
"""锁定单个 Prompt: 注入角色描述块（如果尚未包含）"""
|
||||
if character_desc in prompt:
|
||||
return prompt
|
||||
return f"{character_desc}, {prompt}"
|
||||
|
||||
|
||||
def batch_lock_prompts(prompts: list[str], character_desc: str) -> list[str]:
|
||||
"""批量锁定多个 Prompt"""
|
||||
return [lock_prompt(p, character_desc) for p in prompts]
|
||||
@@ -0,0 +1,181 @@
|
||||
"""
|
||||
composer.de_ai — 去 AI 化处理链（DCS 第八节）
|
||||
================================================
|
||||
|
||||
目的：消除 AI 生成视频的"塑料感/锐利边缘/过饱和"，做出接近真人实拍（伪纪录片）的质感。
|
||||
|
||||
处理链（按 DCS 文档顺序）：
|
||||
① gblur 轻微模糊 —— 消除 AI 边缘过度锐利
|
||||
② noise 胶片颗粒 —— 添加真实传感器噪点
|
||||
③ eq saturation 降饱和 —— 消 AI 味的艳丽
|
||||
④ curves S 曲线 —— 拉反差，更像实拍
|
||||
⑤ colortemperature 色温 —— 偏移到 5300K（日光偏暖）
|
||||
|
||||
默认参数（DCS 默认，可被分镜的 post_process / color 覆盖）：
|
||||
模糊 sigma 0.4 | 颗粒 12 | 饱和 -0.12 | 色温 5300K
|
||||
|
||||
用法：
|
||||
from composer.de_ai import DeAIProcessor
|
||||
DeAIProcessor().process("in.mp4", "out.mp4") # 默认参数
|
||||
DeAIProcessor().process("in.mp4", "out.mp4", blur_sigma=0.6, grain=20)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
log = logging.getLogger("composer.de_ai")
|
||||
|
||||
# FFmpeg 绝对路径（本机装在 ~/.local/bin，不在默认 PATH）
|
||||
_FFMPEG_CANDIDATES = [
|
||||
os.path.expanduser("~/.local/bin/ffmpeg"),
|
||||
"/usr/local/bin/ffmpeg",
|
||||
"/opt/homebrew/bin/ffmpeg",
|
||||
"ffmpeg",
|
||||
]
|
||||
|
||||
|
||||
def find_ffmpeg() -> str:
|
||||
for c in _FFMPEG_CANDIDATES:
|
||||
if c == "ffmpeg" or Path(c).exists():
|
||||
if c == "ffmpeg" and not shutil.which("ffmpeg"):
|
||||
continue
|
||||
return c
|
||||
raise FileNotFoundError("未找到 ffmpeg（请在 ~/.local/bin 或 PATH 中安装）")
|
||||
|
||||
|
||||
class DeAIProcessor:
|
||||
"""去 AI 化处理器"""
|
||||
|
||||
# DCS 默认参数
|
||||
DEFAULT_BLUR_SIGMA = 0.4
|
||||
DEFAULT_GRAIN = 12
|
||||
DEFAULT_SATURATION = -0.12 # 偏移量（实际 saturation = 1 + offset = 0.88）
|
||||
DEFAULT_TEMPERATURE = 5300 # 色温 K
|
||||
DEFAULT_CRF = 23 # 画质（未显式指定时按**源码率自适应**，见 _adaptive_crf）
|
||||
DEFAULT_PRESET = "medium"
|
||||
|
||||
def __init__(self, ffmpeg: str | None = None):
|
||||
self.ffmpeg = ffmpeg or find_ffmpeg()
|
||||
|
||||
def _adaptive_crf(self, video_path: str) -> int:
|
||||
"""按**源码率**自适应 CRF —— 避免对已压缩素材重编码后体积膨胀
|
||||
|
||||
实测：规范化后的素材（约 1.5 Mbps）用 CRF 23 重编码 → 体积膨胀到 2.7 倍；
|
||||
改用 CRF 27 后回到合理区间（视觉几乎无差）。
|
||||
"""
|
||||
import re
|
||||
import subprocess
|
||||
kbps = 0.0
|
||||
try:
|
||||
# 用 `ffmpeg -i` 解析（本机 ffprobe 不支持 -select_streams / -show_entries，
|
||||
# 用它取码率会静默失败 → 一律落兜底值）
|
||||
r = subprocess.run([self.ffmpeg, "-i", str(video_path)],
|
||||
capture_output=True, text=True, timeout=20)
|
||||
info = (r.stderr or "") + (r.stdout or "")
|
||||
m = re.search(r"bitrate:\s*(\d+)\s*kb/s", info)
|
||||
if m:
|
||||
kbps = float(m.group(1))
|
||||
if not kbps:
|
||||
# 兜底：用「文件大小 / 时长」估算
|
||||
dm = re.search(r"Duration:\s*(\d+):(\d+):([\d.]+)", info)
|
||||
if dm:
|
||||
dur = int(dm.group(1)) * 3600 + int(dm.group(2)) * 60 + float(dm.group(3))
|
||||
if dur > 0:
|
||||
kbps = Path(video_path).stat().st_size * 8 / dur / 1000
|
||||
except Exception:
|
||||
kbps = 0.0
|
||||
if 0 < kbps < 3000:
|
||||
return 27 # 已压缩素材（含规范化后的）→ 不再加码率
|
||||
if kbps and kbps < 6000:
|
||||
return 25
|
||||
return self.DEFAULT_CRF
|
||||
|
||||
def build_filter(self, blur_sigma: float = DEFAULT_BLUR_SIGMA,
|
||||
grain: int = DEFAULT_GRAIN,
|
||||
saturation_offset: float = DEFAULT_SATURATION,
|
||||
temperature: int = DEFAULT_TEMPERATURE,
|
||||
enable_curves: bool = True) -> str:
|
||||
"""构造 FFmpeg 滤镜链（可单独测试）"""
|
||||
parts = []
|
||||
if blur_sigma and blur_sigma > 0:
|
||||
parts.append(f"gblur=sigma={blur_sigma:.2f}")
|
||||
if grain and grain > 0:
|
||||
parts.append(f"noise=alls={int(grain)}:allf=t+u")
|
||||
sat = max(0.0, 1.0 + float(saturation_offset))
|
||||
if abs(saturation_offset) > 1e-6:
|
||||
parts.append(f"eq=saturation={sat:.3f}")
|
||||
if enable_curves:
|
||||
parts.append("curves=all='0/0 0.5/0.55 1/1'")
|
||||
if temperature and int(temperature) != 6500:
|
||||
parts.append(f"colortemperature=temperature={int(temperature)}")
|
||||
return ",".join(parts)
|
||||
|
||||
def process(self, input_path: str, output_path: str,
|
||||
blur_sigma: float | None = None,
|
||||
grain: int | None = None,
|
||||
saturation_offset: float | None = None,
|
||||
temperature: int | None = None,
|
||||
crf: int | None = None,
|
||||
preset: str | None = None,
|
||||
keep_audio: bool = True) -> str:
|
||||
"""执行去 AI 化处理 → 返回输出路径（失败抛异常）"""
|
||||
inp = Path(input_path)
|
||||
if not inp.exists():
|
||||
raise FileNotFoundError(f"输入视频不存在: {input_path}")
|
||||
out = Path(output_path)
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
vf = self.build_filter(
|
||||
blur_sigma=self.DEFAULT_BLUR_SIGMA if blur_sigma is None else blur_sigma,
|
||||
grain=self.DEFAULT_GRAIN if grain is None else grain,
|
||||
saturation_offset=self.DEFAULT_SATURATION if saturation_offset is None else saturation_offset,
|
||||
temperature=self.DEFAULT_TEMPERATURE if temperature is None else temperature,
|
||||
)
|
||||
cmd = [
|
||||
self.ffmpeg, "-y", "-i", str(inp),
|
||||
"-vf", vf,
|
||||
"-c:v", "libx264",
|
||||
"-preset", preset or self.DEFAULT_PRESET,
|
||||
"-crf", str(crf if crf is not None else self._adaptive_crf(input_path)),
|
||||
# ⚠️ 必须显式压回 yuv420p：滤镜链中的 curves / colortemperature 在 RGB 域
|
||||
# 运算后会把像素格式提升为 yuv444p，libx264 无此参数时会保持 444p 输出
|
||||
# （High 4:4:4 Predictive profile）→ 成片在微信/抖音/iOS/多数安卓上无法播放。
|
||||
"-pix_fmt", "yuv420p",
|
||||
]
|
||||
cmd += ["-c:a", "copy"] if keep_audio else ["-an"]
|
||||
cmd.append(str(out))
|
||||
|
||||
log.info(" 🎞️ 去 AI 化: %s → %s", inp.name, out.name)
|
||||
log.debug(" 滤镜: %s", vf)
|
||||
r = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if r.returncode != 0:
|
||||
raise RuntimeError(f"ffmpeg 失败: {(r.stderr or '')[-300:]}")
|
||||
if not out.exists() or out.stat().st_size == 0:
|
||||
raise RuntimeError("ffmpeg 输出为空")
|
||||
log.info(" ✅ 去 AI 化完成: %.1f MB", out.stat().st_size / 1024 / 1024)
|
||||
return str(out)
|
||||
|
||||
def apply_to_storyboard(self, video_path: str, output_path: str,
|
||||
storyboard=None) -> str:
|
||||
"""按分镜的 post_process / color 配置处理（第一期分镜的设置）"""
|
||||
kw: dict = {}
|
||||
if storyboard is not None and getattr(storyboard, "shots", None):
|
||||
pp = storyboard.shots[0].post_process
|
||||
cc = storyboard.shots[0].color
|
||||
kw = {
|
||||
"blur_sigma": pp.blur_sigma,
|
||||
"grain": pp.grain_intensity,
|
||||
"saturation_offset": cc.saturation_offset,
|
||||
"temperature": 6500 + int(cc.temperature_shift * 1000), # 偏移 → 色温
|
||||
}
|
||||
return self.process(video_path, output_path, **kw)
|
||||
|
||||
|
||||
def deai_filter_preview() -> str:
|
||||
"""给前端展示用的默认滤镜链"""
|
||||
return DeAIProcessor().build_filter()
|
||||
@@ -0,0 +1,233 @@
|
||||
"""
|
||||
composer/director_styles.py — 导演风格库（20 位）
|
||||
|
||||
来源：开源项目 DirectorSKILL（wuwangzhang1216/DirectorSKILL）的
|
||||
references/director_styles/*.md，经结构化提取（一句话镜头 / 关键词 /
|
||||
叙事方法 / 镜头语言 / 灯光色彩），保留可执行要点。
|
||||
|
||||
用途：用户选择某位导演 → 注入导演提示词 → 分镜产出该导演的镜头语言。
|
||||
|
||||
⚠️ 使用原则（原仓库要求）：**用于原创创作**，不复制其具体镜头/台词/角色。
|
||||
"""
|
||||
|
||||
DIRECTOR_STYLES: dict = {
|
||||
"spielberg": {
|
||||
"name": "斯皮尔伯格 · 奇观与亲密",
|
||||
"one_line": "先拍看见的人，再拍被看见的东西；奇观永远排在一张脸后面。",
|
||||
"keywords": "情感动员、经典覆盖、奇观与亲密并置、儿童视线高度、家庭裂痕、三层纵深、先反应后揭示、代价之后的修复。",
|
||||
"narrative": "外部目标必须一句话说清，并在前 90 秒说清：救人、找东西、逃出去、回家、揭真相。；- 情感缺口先于奇观：第一次奇观出现前，观众已经知道主角缺什么（一句对白 + 一件道具即可）。；- 高潮由 3–5 个小目标串成，每完成一个立刻出现新障碍；不要用一个长障碍撑满。；- 亚类型可换（战争、科幻、传记），情感核心不换：亲情、失落、恐惧、勇气。",
|
||||
"camera": "主力 35–50mm 建立可信的日常；反应镜头换到 135mm，机位不必逼近人物——135mm 在 3 米处就给到约 0.8 米宽的画幅（胸口以上），景别跳一大级而人不受打扰，背景同时被压平成一片色。；- 三层纵深调度：前景道具或次要人物 / 中景主角 / 背景事件或出口。观众任何一帧都能指认危险与出口。；- 运动必须有目的：推轨速度与人物步速一致（约 1.2–1.4 m/s）；升降镜头只在需要交",
|
||||
"light": "基础是柔和环境底光 + 一束可见硬光（窗、门缝、车灯、探照灯、投影机），硬光必须穿过薄雾或尘才看得见。；- 温暖场景比 3:1；威胁场景拉到 8:1，做法是撤掉补光让暗面自己掉下去，不是关小主光——整体照度不变，观众始终看得清空间。饱和度是另一条轴，属于调色，不参与比值。；- 每个光源在剧作里都有名字：不是\"神秘光芒\"，是\"走廊尽头那盏没关的顶灯\"。写进提示",
|
||||
"scene": "强情绪牵引的商业叙事：冒险、灾难、家庭创伤、童年视角、人道主义题材。；- 需要观众在开场 30 秒内认同主角的短片、预告片情绪段、品牌与公益片。；- 需要\"大事件 + 小人物\"结构，并且观众必须始终知道危险与出口在哪的动作段落。；- 不适用：反英雄叙事、道德暧昧的结局、要求观众保",
|
||||
},
|
||||
"hitchcock": {
|
||||
"name": "希区柯克 · 悬念与信息差",
|
||||
"one_line": "把秘密先交给观众，再让角色慢慢走进去；等待本身就是段落。",
|
||||
"keywords": "悬念优先于惊吓、信息不对称、窥视与共谋、日常空间变危险、关键物件、延迟、几何遮挡、优雅的反派、黑色幽默。",
|
||||
"narrative": "段落开始前先写一行\"观众知道 X，角色不知道 X\"。这一行写不出来，就不要用这个透镜。；- 危险的抵达时间可视化：把 20–60 秒的等待切成 3–5 个可数节拍（脚步近了、门把动了、影子长了）。；- 普通人被卷入犯罪或误认，让安全的日常生活不再可靠；主角越想解释，处境越糟。；- 另一种变体：角色知道真相却无法证明。此时观众与角色同盟，压力来自\"没有人相信\"",
|
||||
"camera": "三段式主观：客观镜头看人（50mm）→ 主观所见（100–135mm 压缩，模拟凝视只挑一样东西看）→ 反应特写（50mm，比第一镜近两级——不是比中间那镜近，中间那镜拍的根本不是他）。三镜必须共享同一条视线轴。；- 广角 28mm 只用来一次性交代\"这个房间是一张地图\"：门在哪、楼梯在哪、可藏身处在哪。之后不再重复给。；- 遮挡构图是默认：前景占画面 20–35%（门框、栏杆、栏杆间隙、窗帘、肩",
|
||||
"light": "主光硬、单侧、有明确来源，比 4:1；阴影是几何形状（百叶、楼梯扶手、门缝），不是氛围雾。；- 明亮空间里的恐怖尤其有效：把照度维持在日常水平，让威胁没有黑暗可以躲，观众只能靠自己找。；- 服装与道具的颜色承担身份与欲望的转变：一个人物在段落里只允许换一次色彩状态，这次换色必须与信息转折同步。；- 主色中性冷灰与米白，重点色饱和度高（正红、深绿、金）；对比中",
|
||||
"scene": "悬疑、惊悚、犯罪、心理恐怖；跟踪戏、误认与身份错置、密室段落。；- 需要在日常空间（楼梯、厨房、办公室、车厢、洗手间）里制造威胁的段落。；- 短片与广告里需要用 20–60 秒把观众按在座位上的单场戏。；- 不适用：需要观众与角色同步无知的鬼屋类恐怖，或需要开放式意象的作者电影（",
|
||||
},
|
||||
"kubrick": {
|
||||
"name": "库布里克 · 几何与秩序",
|
||||
"one_line": "让建筑比人更有权力：广角、单点透视、匀速推进，人被压进几何里。",
|
||||
"keywords": "单点透视、匀速推轨、几何秩序、高照度恐惧、黑色讽刺、理性失控、仪式化重复、分析距离、开放冷结局。",
|
||||
"narrative": "先确定场景背后的制度、机器或哲学命题，再写人物；人物是命题的样本。；- 不解释心理，用重复行为展示异化：同一动作在段落里出现 2–3 次，每次只改变一个细节。；- 结构可章节化、可循环、可对称（开场与结尾用同一构图，只换内容）。；- 反讽的机制是\"形式的庄严 + 内容的荒谬\"同时在场，两者都不能软化。",
|
||||
"camera": "主力 18–25mm。透视变形由机位距离决定，不由焦段决定：18mm 退到 1.5–3 米只是把人拍小，脸仍是正常比例；要拿到脸部膨胀感必须逼到 0.6–0.8 米——广角的用处正是让你贴到这个距离还能把整条走廊装进画面。两种用法都保留，不要用 35mm 去\"修正\"。；- 构图对称到像素级：消失点落在画幅正中（水平 50%，垂直 45–55%），人物骑在中轴上或严格镜像分布。；- 运动只有两种：完",
|
||||
"light": "高照度、均匀；比 2:1 甚至 1.5:1。手段是成排的硬光源对称排布：投出的短硬阴影方向互相抵消——不是没有阴影，而是没有一块可以藏东西的暗部。恐惧来自\"没有黑暗可以解释这件事\"。；- 光源可以是极端自然（只用实景顶灯、窗光、蜡烛级别的低照度）或极端人工（成排灯管、投射的色块），但两者都必须在画面里可见并对称排布。；- 色彩系统化：一个空间给一个色，冷白 ",
|
||||
"scene": "心理恐怖、科幻寓言、战争讽刺、制度与权力机制、失控的理性、封闭空间。；- 需要观众保持分析距离而非情感认同的段落；需要荒诞感与庄严感同时成立的仪式场面。；- 有长走廊、大厅、舱体、机房、营房这类具备强消失点的实景或生成场景。；- 不适用：需要观众共情主角、需要手持临场感、需要柔和",
|
||||
},
|
||||
"kurosawa": {
|
||||
"name": "黑泽明 · 群像与天气",
|
||||
"one_line": "用长焦把队列压成横向的带，让风雨作用在身体上，等待比爆发更长。",
|
||||
"keywords": "人道主义、群像功能分工、横向运动、长焦压缩、天气作为角色、身体姿态、道德选择、悲剧尊严、代价化胜利。",
|
||||
"narrative": "每个角色先定\"社会位置 + 一个道德选择\"，再定性格。群像里 5–8 人，每人一句话能说清用处。；- 结构常是：混乱 → 组织 → 训练与等待 → 决战 → 空虚。等待段是全片的心脏，不是过场。；- 人物不是纯粹英雄，他们在责任、荣誉、生存与道德之间挣扎，并且会输。；- 群像叙事必须有可数的位置关系：谁在圈内、谁在圈外、谁在高处、谁背对着谁。",
|
||||
"camera": "主力 100–250mm。长焦把远近人物压到同一平面上，让\"人群\"变成一堵可读的墙。方向要分清：朝镜头跑的人在 250mm 下几乎看不出在推进（纵深位移被压掉），所以冲锋要设计成\"跑了很久还没到\"；横穿画面的人反而更快，因为视野窄——想让横向运动在画面里停留久一点，只能退远机位或者加长队列，不能靠焦段。；- 运动以横向为主：横移跟拍、摇摄扫过队列。纵深突入只在一次决定性的冲锋时用一次。；- 三机位",
|
||||
"light": "光的作用是让天气可见：逆光让雨丝、尘埃、呼出的白气变成亮线，比 6:1 以上。；- 黑白思维优先——先确认画面在去色后依然靠明暗分区成立，再考虑颜色。；- 彩色时使用浓重的戏剧化色块表现权力与阵营（正红、靛蓝、土黄、墨黑），色块贴在旗帜、盔甲、外衣上，而不是打在空气里。；- 环境的质感高于美感：泥的反光、湿布的深色、干尘的粉白，这些是画面的主要信息来源。",
|
||||
"scene": "武士片、史诗、群像、战争与社会冲突、阶级压迫、英雄牺牲、权力崩塌。；- 需要在恶劣自然环境（雨、风、雪、尘、泥）中完成情绪的段落。；- 需要 5 人以上同框且每人有明确功能与位置的调度；需要动作方向绝对清楚的战斗。；- 不适用：室内心理剧、亲密两人对话、需要浅景深与柔光的当代都市",
|
||||
},
|
||||
"scorsese": {
|
||||
"name": "斯科塞斯 · 都市与旁白",
|
||||
"one_line": "让摄影机被欲望拖着走，用音乐把上升拍得好看，再用静默让代价落地。",
|
||||
"keywords": "都市脉搏、不可靠旁白、圈层规则、流行乐蒙太奇、稳定器长镜头、急推变焦、突发暴力、罪感与自辩、上升到崩塌。",
|
||||
"narrative": "结构固定为四段：进入 → 上升 → 沉迷 → 坠落。四段的音乐密度依次为高、最高、中、零。；- 用第一人称旁白搭建人物的神话，再用画面证伪：旁白说\"我们都是朋友\"，画面里有人被隔在门外。；- 亚文化世界必须有可被列举的规则（谁先动手、钱怎么分、谁能坐哪张桌子）。规则通过一次违规被观众理解。；- 人物不理解自己的罪，只会不断辩护；因此他不设置转折点，转折由后果",
|
||||
"camera": "主力 18–27mm 用于稳定器长镜头（跟随人物穿过空间，与背景保持接触感）；40–75mm 用于人物在自辩时的中近景。；- 长镜头的功能是\"入场仪式\"：从公共空间连续走到私密空间，路上经过至少 3 个需要被许可的关卡。生成端把它拆成 2–3 段，接点放在有人或有物完全挡住画面的那 4–6 帧（一个横过的背影、一扇被推开的门），观众读不出接缝；工具若提供续写/延长入口，直接用它接。；- 急推变焦（",
|
||||
"light": "夜景为主：霓虹、酒吧红、后厨绿、金色包厢、街灯钠黄。实用光源直接入画并且允许过曝。；- 主光可以很硬且方向混乱（多个实用光源互相打架），比 5:1；这种\"脏\"的照明就是都市的质感。；- 上升段饱和度拉高（红金为主），坠落段把饱和度降低 30–40%、把红换成灰绿，同一空间照明不改而观感改变。；- 烟雾常驻：让每个光源都有可见的光锥，画面才有厚度。",
|
||||
"scene": "犯罪片、都市人物传记、道德坠落、亚文化圈层生态、男性友谊与暴力关系、宗教罪感。；- 需要\"进入一个世界\"的段落：从任何人都能进的公共入口，连续走到只有内部人能进的那个房间。；- 需要用 60–120 秒压缩数年命运的音乐蒙太奇。；- 不适用：需要观众保持冷静分析的题材、极简室内心",
|
||||
},
|
||||
"fellini": {
|
||||
"name": "费里尼 · 狂欢与记忆",
|
||||
"one_line": "怪异全部长在真人身上：脸、身体、服装、姿态；摄影机在人群里游动，不停下。",
|
||||
"keywords": "游行式调度、人脸奇观、马戏团与圆舞曲、记忆逻辑、欲望与羞耻、滑稽与悲伤并置、心理舞台、意象回旋、开放结尾。",
|
||||
"narrative": "用记忆与欲望做结构，不用因果。一个段落的连接理由可以是\"同一首曲子\"或\"同一种羞耻\"。；- 主角常是迷惘的创作者、浪子或观察者；他的功能是被环绕，不是推动。；- 现实滑入幻想没有边界线：让观众在事后才意识到已经越界。；- 全片选 3 个意象反复出现，每次改变一个属性（白天变夜晚、活人变照片、乐队变一个人）。",
|
||||
"camera": "主力 18–32mm。正面近景用 25mm 贴到 0.5–0.7 米拍——夸张来自这个距离而不是这支镜头（同样是 25mm，退到 2 米脸就只是变小）。鼻子略大、耳朵后缩是漫画式效果，不要用长焦修正。；- 摄影机持续游动：跟移 + 摇摄组合，速度 0.4–0.8 m/s，很少完全静止。它像一个在派对里找人的宾客。；- 群体调度像游行：人物从画面左右边缘陆续入画，前景不断有人穿过（前景遮挡占 10–",
|
||||
"light": "光像舞台照明：硬边光区打出看得见的边界，雾再把边界柔化半米左右；人物走进走出光区，明暗切换本身是表演。；- 雾与逆光常驻：让人群有轮廓、有层次，也让边界变得不可靠。比 4:1，但阴影里保留细节。；- 彩色时倾向华丽、人工、肉感：粉、金、酒红、天蓝，饱和度高；服装的颜色比环境更响。；- 黑白思维时靠强反差 + 雾 + 白衣黑衣的对撞分区，不靠灰阶细腻。",
|
||||
"scene": "梦境段落、自传性叙事、艺术家困境、荒诞喜剧、回忆结构、城市漫游、群像仪式（游行、派对、集会、婚礼、葬礼）。；- 需要喜剧与悲伤同时在场的段落；需要主角被一群人包围但更孤独的段落。；- 时尚片、音乐录影、品牌短片里需要\"生命剧场\"感的群像调度。；- 不适用：需要严密因果的类型片情节",
|
||||
},
|
||||
"bergman": {
|
||||
"name": "伯格曼 · 凝视与沉默",
|
||||
"one_line": "把摄影机放在视线正前方，用长焦让整张脸清晰，然后不切。",
|
||||
"keywords": "正面凝视、双脸并置、审讯式侧光、存在主义对白、沉默作为台词、室内即牢笼、剥皮式揭露、不治愈的结尾。",
|
||||
"narrative": "人物 2–3 个，空间 1–2 个，时间跨度不超过一夜。外部事件越简单越好。；- 真正的戏剧来自关系里的谎言、依赖、怨恨与羞耻，不来自情节推进。；- 对白像剥皮：每一轮交换都比上一轮更接近人物最不愿说的那句话。写对白时给每一轮标一个\"被剥掉的第几层\"。；- 全场设一次\"权力翻转\"：谁在问、谁在答，必须在中段互换一次。",
|
||||
"camera": "主力 75–150mm 拍脸。75mm 给中近景，100–150mm 给大特写；长焦不放大鼻子也不缩耳朵，脸被拍成它本来的比例，观众找不到一处变形可以拿来替他解释。；- 正面凝视是核心手段：摄影机在视线轴线上（偏差 <10 度），脸占画幅高度 60–80%。这是这个透镜最贵也最有效的一招，一场戏用 2–4 次就够。；- 双脸构图：一张脸正面在中景平面，另一张脸侧面在近景平面，两张脸的边缘接触或轻微",
|
||||
"light": "单侧硬主光，比 6:1，暗面不补光但保留可辨认的轮廓。光源方向决定谁在审问谁。；- 光源常是窗、台灯、火；把光位设在演员正侧方（90 度）而不是 45 度，这样鼻影会把脸分成两半。；- 黑白思维：先确认画面在去色后靠脸的明暗雕塑成立。彩色时也保持低饱和、室内化、冷偏（灰蓝、灰绿），只留皮肤是暖的。；- 背景压到近乎全黑或全白的单色平面，让脸没有可依靠的环境信",
|
||||
"scene": "室内剧、婚姻与亲密关系冲突、信仰危机、死亡焦虑、家庭创伤、忏悔与审问、心理崩溃。；- 两人对话场景；单人独白；需要在 3–5 分钟内不换景完成情绪全程的段落。；- 需要用最少资产（一个房间、两张脸、一扇窗）做出高强度戏剧的低成本项目。；- 不适用：动作、群像、需要空间奇观或类型快",
|
||||
},
|
||||
"tarkovsky": {
|
||||
"name": "塔可夫斯基 · 雕刻时间",
|
||||
"one_line": "镜头站着不动，让水、火和时间在画面里替人物完成表演。",
|
||||
"keywords": "雕刻时间、长镜头、记忆、信仰、牺牲、水、火、风、泥、废墟、梦境、精神流亡、质地、垂直压力。",
|
||||
"narrative": "先定一个精神问题（我能否被原谅／信仰是否值得代价），再设计事件；不要先设计反转。；- 外部旅程与内部旅程一一对应：走向一个具体地点＝走向一次内在清算。；- 梦、回忆、现实、预感之间不加标记地切换，观众靠情绪而非逻辑跟随。；- 一场戏只推进一个精神刻度，允许\"什么都没发生\"的整场戏存在。",
|
||||
"camera": "主力焦段 28–35mm，f/5.6–f/8 深焦：前景的水面、中景的人、远处的门要同时清晰，用景深而不是切镜建立层次。；- 深焦是有代价的：35mm f/8 的超焦距约 5 米，把焦点放在那里，清晰范围是 2.5 米到无限远。所以前景反光面要摆在 2.5 米以外；贴到 1 米内的前景在这个光圈下一定糊，那是另一种镜头（质地特写），不能和深焦分层混在同一个镜头里。；- 移动只有两种：恒速极慢横移（",
|
||||
"light": "反差比 2:1–3:1，柔阴天窗光为主，几乎不出现硬边阴影。；- 唯一允许的明亮点光源是火：烛、油灯、篝火、燃烧的木头。火只做点光和运动源，永远不做主光——主光始终是柔的阴天窗光，所以即使画面里有明火，也不该出现硬边阴影。；- 主色调是泥棕、湿石灰、苔绿、褐调；高光偏乳白，重点色只有火橙和铁锈红。；- 低饱和低对比，但不是灰蒙蒙——湿表面自己反光，质地承担对",
|
||||
"scene": "诗电影、精神与信仰危机、科幻寓言、记忆叙事、废墟与被自然侵入的空间、慢电影、梦与现实无缝交叠的段落。也适合极低预算但可控环境的项目：一个房间、一堵湿墙、一盆水就能成段。；不适合：需要在 30 秒内交代三条信息的商业片、动作片、任何以剪辑节奏为卖点的内容。",
|
||||
},
|
||||
"wong_kar_wai": {
|
||||
"name": "王家卫 · 都市孤独",
|
||||
"one_line": "摄影机贴得太近、挡得太多、又晚了一步，正好是这段关系的形状。",
|
||||
"keywords": "错过、暧昧、时间标记、记忆重写、孤独、霓虹污染、步进印片、手持、前景遮挡、旁白、都市迷宫、怀旧流行乐。",
|
||||
"narrative": "先设计\"差一点发生\"：两个人共享同一个空间但错开 30 秒，或共享同一个物件但不共享时间。；- 用具体数字承载情感：日期、门牌号、班次、第几次、几分钟、还剩几个。抽象情绪一律换成可数的东西——自己发明一个可数物，不要借用别人已经用过的那一个。；- 旁白是第一人称，真诚但不可靠——它在为自己重写事实，观众要能看出画面与旁白的差额。；- 结构碎片化：同一个动作在",
|
||||
"camera": "焦段两极：18–25mm 贴到 40cm 内拍狭窄空间和变形的近身；85mm 隔街压缩人群，中间焦段少用。；- 手持是默认，但幅度小：像人在犹豫时的微调，不是奔跑时的晃动。；- 步进印片感（step-print）：低帧率拍摄（8–12 fps）配长快门（约 1/8 秒），再把每一帧重复打印 2–3 次填回正常时长。拖影来自那个长快门，不是来自抽帧——只抽帧不开慢门只会得到卡顿。一段只用一次。；- ",
|
||||
"light": "反差比 6:1–10:1，光比全靠实用光（霓虹管、路灯、走廊灯泡、冰箱灯、电视机），不带补光。；- 分清哪些实用光是硬的：裸灯泡、射灯、窗外招牌的直射是硬源，负责硬边和高光比；霓虹管是细长的线光源，本身偏柔，它的活是染色和勾轮廓，别指望它给你硬边阴影。；- 一个空间只用两三种颜色，且互补：绿与品红、金与青、红与墨绿。颜色是情绪滤镜，不必符合物理。；- 高饱和",
|
||||
"scene": "都市爱情与错过、暧昧未成形的关系、记忆叙事、夜景与狭窄室内、人物独白、怀旧年代空间、情绪向短片与音乐段落。空间越小越好：走廊、楼梯、单间、面摊、后座。；不适合：需要清楚交代地理关系的场景、群戏调度、任何观众必须看懂\"谁在哪里\"的动作段落。",
|
||||
},
|
||||
"nolan": {
|
||||
"name": "诺兰 · 高概念谜题",
|
||||
"one_line": "先把规则讲清楚，再让三条时间线同时向同一个倒计时收拢。",
|
||||
"keywords": "高概念规则、倒计时、交叉剪辑、实拍质感、谜题结构、理性人物、执念、牺牲、物理重量、情感压抑。",
|
||||
"narrative": "第一步：用一句话写出规则（不超过 25 字），且这句话必须在前 90 秒内以戏剧方式说出或演示出。；- 规则必须同时是情感陷阱：遵守它会让主角失去他最在乎的东西。；- 拼图式给信息：观众先困惑 20%，再补 60%，最后 20% 留到高潮才补齐；每次补齐都要伴随一次代价。；- 多线并行：2–4 条线各有独立倒计时，且倒计时长度不同（例如 A 线 60 分钟、",
|
||||
"camera": "焦段偏广：14–21mm 拍尺度与空间关系，35mm 拍贴身跟随，50mm 是常规上限。更长的焦段全片只留给一种用途——把分处两地的两条线压进相似的画面形状里做匹配剪辑——用完立刻退回广角。；- 广角＋近距离＝物理存在感：让镜头离演员 60–90cm，用 21mm 拍，观众能感到重量和体积。；- 手持与稳定器混用，但手持是\"跟得上\"的手持，不是失控；动作段落每个镜头只表达一个机制事实。；- 大远景",
|
||||
"light": "反差比 3:1–4:1，自然主义动机；奇观也要像是真实发生并被摄影机偶然拍到。；- 主色调冷：混凝土灰、钢青、夜间钠灯的暖橙作唯一暖点；饱和度中低。；- 不同时间线用可辨识但不夸张的差异区分：色温差 500–800K、颗粒差、天气差、镜头畸变差，而不是换成完全不同的滤镜。；- 室内多用可见的实用光源（工作灯、仪表、车灯）；室外用真实天光和阴天散射。",
|
||||
"scene": "高概念科幻惊悚、时间与记忆结构、心理谜题、身份困境、任务片、多线并行高潮、需要\"观众边看边学规则\"的段落。也适合预算有限但结构可以炫技的短片：一个规则、两个空间、三条线。；不适合：氛围向诗电影、纯人物关系剧、没有可陈述规则的情绪散文。",
|
||||
},
|
||||
"villeneuve": {
|
||||
"name": "维伦纽瓦 · 宏大克制",
|
||||
"one_line": "把人缩到画面高度的百分之五，然后给他二十秒，什么也不解释。",
|
||||
"keywords": "宏大尺度、几何对称、单色相、雾与沙、低频轰鸣、寂静、缓慢丈量、纪念碑式建筑、人对系统、克制悬念、形而上恐惧。",
|
||||
"narrative": "主角是专业人士（调查员、士兵、技术员、研究者），带着一套方法论进入一个方法论失效的地方。；- 不解释：把揭示延迟到观众已经形成错误假设之后，让理解本身成为情节事件。；- 事件规模远超主角的处置能力，所以主角的动作永远是\"观察、记录、承受\"，而不是\"解决\"。；- 哲学问题先行（时间、记忆、宿命、暴力的代价），奇观是它的物证。",
|
||||
"camera": "焦段：21mm 拍纪念碑尺度，40mm 拍程序化中景，75mm 压缩疲惫的脸；人物在广角镜头里占画面高度的 5%–15%。；- 运动只有三种：锁死、恒速横移（像在丈量墙面）、恒速推进（正对灭点）。禁止手持、禁止环绕、禁止变速。；- 强制留出\"空镜时间\"：镜头在人物进入前 2 秒开始，在人物离开后 3 秒结束。；- 构图靠中心对称或单一水平线；把地平线放在画面的 1/8 或 7/8，不要放中间。",
|
||||
"light": "一个段落一个主导色相，绝不混：赭橙沙漠、青雾、铜金室内、墨绿林地、灰白冰面。换色相＝换地点或换时间层。；- 逆光为主，光比 5:1–8:1——这个比值指的是逆光面与正面补光之比，不是常规的主光对补光，所以人物是剪影或半剪影，正面只留到轮廓可读为止。；- 柔散射，几乎不出现硬边阴影；光源常是一整面墙、一整扇窗、一整片天。；- 低饱和但高色相纯度：颜色很淡，但整",
|
||||
"scene": "科幻寓言、第一接触、缓慢恐惧、犯罪程序剧、巨型建筑与荒漠地貌、人在系统性力量前的渺小感、需要\"看不清也听不清\"作为主题的段落。低成本也可成立：一堵大墙、一台雾机、一个逆光窗。；不适合：需要密集信息交付的段落、喜剧、群戏对白、任何要求观众跟上快速因果链的内容。",
|
||||
},
|
||||
"fincher": {
|
||||
"name": "芬奇 · 外科精确",
|
||||
"one_line": "用长焦把人从背景里切下来，再从他不知道的位置上看着他。",
|
||||
"keywords": "外科精确、信息密度、监视视角、青绿-琥珀、低饱和低对比、长焦样本化、实用光唯一、机位可复现、程序节奏、反高潮。",
|
||||
"narrative": "结构由信息流动决定：先画出\"谁在第几分钟知道什么\"的表，再写场景。；- 节拍来自工作行为：数据、笔录、屏幕、会议、走访、归档；人物通过职业动作暴露内心。；- 观众的知情度略高于主角约一个证据单位——不是完全同步，也不是全知。；- 暴力不被浪漫化：冷静、突然、无配乐前摇，且后果延迟到下一场才显现。",
|
||||
"camera": "焦段偏长：85mm 与 135mm 是主力，35mm 只用于一次性交代几何；长焦压缩把背景压成色块，人物被\"抠\"出来。；- 机位可复现：写清高度、距离、角度，同一场戏的同一机位必须能精确回到原位；这是与\"差不多的稳定\"最大的区别。；- 推进以百分比计量：一个 8 秒镜头推进画幅宽度的 3%–6%，慢到观众意识不到，但结束时构图已经变紧。；- 监视机位是签名：从玻璃后、门缝、走廊尽头、货架之间、天花",
|
||||
"light": "青绿阴影与琥珀实用光的两级对位；不是整体偏色，是阴影偏青、光源偏黄。；- 光比 4:1–6:1，但对比度整体压低：黑位抬起，阴影里始终保留可读纹理，绝不压死黑。；- 实用光源常是唯一光源：台灯、显示器、街灯、走廊应急灯、车灯；写提示词时点名光源实体，不写氛围。；- 饱和度低但不去色：皮肤保留一点血色，其余颜色被抽走。",
|
||||
"scene": "犯罪悬疑与连环案、调查与程序叙事、技术行业与媒体操纵、心理冷酷的家庭暴力、职业流程剧、阴冷都市夜景。特别适合\"信息不对称\"驱动的段落：谁知道什么、何时知道、怎么被揭穿。；不适合：需要观众与人物共情到失控的段落、手持纪实、温暖家庭剧、任何靠镜头晃动制造紧张的内容。",
|
||||
},
|
||||
"refn": {
|
||||
"name": "雷弗恩 · 霓虹停滞",
|
||||
"one_line": "机位锁死、人在正中、什么都不动，让颜色替所有人做完这场仪式。",
|
||||
"keywords": "霓虹色块、正面对称、催眠停滞、合成器音墙、仪式动作、缓慢凝视、突发暴力、身体物化、平面化构图、无解释寓言。",
|
||||
"narrative": "情节压到寓言级：欲望、复仇、嫉妒、自毁，四选一，且不加动机说明。；- 用仪式动作推进而不是因果链：穿衣、洗手、走向房间中央、跪下、把一样东西放在正中。；- 人物动机藏在沉默里，只通过行为暗示；观众允许猜错。；- 暴力是审美事件，不是问题的解决方案；它不改变人物，只完成一个形式。",
|
||||
"camera": "焦段 25mm（把房间拍成舞台）／40mm（仪式中景）／75mm（锁死的脸部肖像）。25mm 只用来正面平行地拍整堵墙，且人物离镜头 3 米以上——这样广角的变形不会显形；一旦贴到 1.5 米内拍人，脸就会鼓出来，对称的仪式感立刻变成怪诞，那是另一套语言。；- 机位胸高、与墙面成 90 度、绝对居中；人物的中轴与画面中轴误差为零。；- 运动预算是零：整段不动。若必须动，只允许一次恒速正面推进，速度",
|
||||
"light": "极端色块：纯品红、电光蓝、血红、月光紫；一个镜头最多两种颜色，各占约一半画面，边界硬。分界线可以偏离画幅中线 ±10%，但必须落在一条画面里本来就有的竖直边上（门框边、墙角、柱子），否则观众会读成打光失误而不是设计。；- 光比 8:1–12:1，光源像舞台灯：硬边、有明确的落点、可以直接看见颜色的分界线。；- 单一色温支配整段；换颜色＝换心理状态或换叙事阶段",
|
||||
"scene": "风格化犯罪、寓言式短片、时尚惊悚、缓慢恐怖、宗教或仪式感空间、沉默角色与极端美学并存的段落。这套语言对生成模型格外友好：无位移、无复杂调度、单一光源、单一平面，能出错的地方比其他风格少一个数量级——你几乎只需要守住构图不塌。；不适合：需要交代因果的叙事段落、群戏对白、任何依赖观众",
|
||||
},
|
||||
"bi_gan": {
|
||||
"name": "毕赣 · 梦境长镜",
|
||||
"one_line": "摄影机像一个没睡醒的人，沿一条能画在地图上的路，一直走进梦里。",
|
||||
"keywords": "雾镇、超长长镜头、路径、穿越式转场、梦境拼接、诗歌旁白、时间循环、潮湿反光、霓虹与乡土并置、磁带与老歌、记忆寻人。",
|
||||
"narrative": "故事围绕\"寻找一个人或一段记忆\"，但寻找的过程就是全部叙事，找到与否不重要。；- 现实、梦境、回忆、预感被有意混淆，且不加任何标记；观众跟随情绪而非逻辑。；- 时间可循环、倒流、并置：同一空间在段落后段再次出现，但物件或人已经不同。；- 诗歌旁白承担象征任务，与当下画面错位；它谈论的对象不在画面里。",
|
||||
"camera": "焦段 18–25mm 为主，f/2.8–f/4；广角让狭窄空间可通行，同时把湿反光铺满画面。贴到 1.5 米跟拍时，25mm f/3.5 的景深不到一米，背后的霓虹和灯泡自然化成光斑——这正是要的，不要收小光圈去救背景，收了整条街就变成资料片。注意这层虚化只在近端成立：退到 3 米，同样的焦段光圈景深就回到两米以上，背景重新变清楚，所以\"贴着人走\"不是风格偏好，是拿到这个背景的技术前提。；- 摄影",
|
||||
"light": "主光源是潮湿表面的反射：积水、湿砖、玻璃、雨衣、瓷砖；先设计水，再设计灯。；- 混合色温是常态且不校正：卡拉 OK 招牌的霓虹、台球桌上的荧光管、廉价白炽灯泡、电视机的青光同时存在。；- 色调湿润青绿、暗紫、淡橙，低饱和但层次厚；对比低，暗部有雾散射的抬升。；- 雾天自然光形成弥散柔光，几乎无硬阴影；光比 2.5:1–3.5:1。",
|
||||
"scene": "华语小镇叙事、诗电影、梦与现实交叠、寻人与寻记忆、潮湿西南地貌、低成本高氛围段落、长镜头实验、魔幻现实主义。适合有真实场地但没有预算做复杂分镜的项目：一条路线就是一场戏。；不适合：需要多机位覆盖的对白戏、剪辑驱动的类型片、任何需要观众看懂因果的段落。",
|
||||
},
|
||||
"zhang_yimou": {
|
||||
"name": "张艺谋 · 单色仪式",
|
||||
"one_line": "先决定这一段是什么颜色，再决定谁站在里面；颜色说话，人只是破格点。",
|
||||
"keywords": "单色场、图案与破格、群体几何、民俗仪式、院落对称、材料色、顶俯图形、集体规训、命运的可见形状、色彩宣告。",
|
||||
"narrative": "先写\"颜色分段表\"：全片 2–4 个色段，每段一句话写清这个颜色代表什么；换色必须落在一次剪辑点上，且对应一次不可逆的处境变化。；- 冲突结构是\"个体 vs 规矩\"：一个人想要一件很小的东西，一套集体制度不允许。制度必须有可见形式——队列、称量、报数、开门关门的时刻表。；- 仪式承担说明功能：不解释规矩，演一次规矩，观众自己看懂。一场仪式只演一次完整流程，第",
|
||||
"camera": "主力 35mm 做天井与长廊的单点透视对称；85–135mm 把重复单元（列队、屋檐、灯笼、门洞、梯田）压成平面图案。24mm 以下慎用——广角会破坏单元的等距，图案立刻散掉。；- 机位只用两档高度：眼平 1.6m 锁定，用于正面对称；顶俯 90° 用于把人群变成图形。中间高度会同时失去对称和图形，尽量不用。；- 运动只有三种，每种都恒速无加减速：与一排单元平行的横移（0.3–0.5 m/s）、对",
|
||||
"light": "单场单主色：一个段落只允许一个主色占据 60%–80% 画幅。换色即换情绪刻度，且换色必须发生在剪辑点上，不做渐变。；- 外景用高照度硬光，比 3:1，让材料的织纹和颗粒被看见。3:1 是靠地面和同色墙面的大面积反射把暗部托起来的，不是靠加灯；没有这层反射的空旷外景，硬太阳自然会跑到 6:1 以上，那就不是这个风格了——所以选景时先看有没有能反光的大面积同色",
|
||||
"scene": "民俗与仪式题材：婚丧、节庆、宗族规矩、集体劳作、大型团体操演段落（方阵、齐舞、集体仪典）。；- 需要用视觉宣告情绪的段落：情绪不靠台词、不靠特写，靠整块画面换色完成。；- 院落、天井、长廊、梯田、旷野、城墙等有强几何秩序的空间。；- 群体资源可控的项目：只要有 30 个以上可重复",
|
||||
},
|
||||
"hou_hsiao_hsien": {
|
||||
"name": "侯孝贤 · 固定远观",
|
||||
"one_line": "摄影机提前到场，坐在门外，不走近也不离开，让一屋子人把时间过完。",
|
||||
"keywords": "固定长镜、远观、门框纵深、实时、省略、环境声、日常行为、非表演、家族时间、画外事件。",
|
||||
"narrative": "结构靠省略：把最有戏的事件放在两颗镜头之间。观众看见事前的一顿饭和事后的一顿饭，中间那件事永远不拍。；- 一场戏＝一个连续时间块，不做时间压缩。如果吃饭要 4 分钟，就拍 4 分钟里的一段，不剪成 40 秒。；- 信息由行为携带：谁坐在哪、谁给谁盛饭、谁不说话、谁先离席。人物关系全部靠座次和动作分配交代。；- 大历史只从缝隙里进来——广播、一句转述、一个不再",
|
||||
"camera": "主力 50–100mm，机位距被摄体 6–12 米。用中长焦从远处拍全景，而不是用广角贴近拍——这一条决定了整个风格：观众离得远，但看得清。；- 机位高度是坐姿眼高 1.0–1.2 米（板凳、榻榻米、饭桌的高度），不是站姿眼高。整场不变。；- 三脚架锁死。唯一允许的运动是\"补构图\"：人物快要出画时，10–20 度的极慢摇，速度慢到观众意识不到，摇完就停。禁止推、拉、升降、跟。；- 门框、窗框、廊柱",
|
||||
"light": "只用现场光。主光是门口或窗口的白天户外光，室内比它低 1–2 档；人脸经常处在下半档的暗部，允许看不清。；- 光比 2:1–4:1，柔，没有硬边阴影；夜戏由一盏真实的顶灯或日光灯管完成，比 4:1，灯本身可以入画。；- 不给演员补光。人走到窗边就亮，走进屋里就暗——曝光变化本身是场面调度的一部分。；- 色调来自实景材料：旧木头的褐、灰泥墙的白、门外的植物绿、",
|
||||
"scene": "家庭史、时代变迁下的日常生活、代际沉默、离开与返乡、小城与街屋空间。；- 需要\"事件发生在画外\"的叙事：变故已经发生，我们只看见之后的一顿饭。；- 极低成本高完成度的项目：一个真实房间、一扇门、一个下午的自然光就能拍一场。；- 需要观众自己在长时间里发现信息的段落——他必须扫描画",
|
||||
},
|
||||
"park_chan_wook": {
|
||||
"name": "朴赞郁 · 巴洛克反讽",
|
||||
"one_line": "把画面塞满花纹，让摄影机做一件物理上不可能的事，然后礼貌地伤害一个人。",
|
||||
"keywords": "巴洛克、纹样密度、反讽之美、礼貌残酷、不可能的运动、对称正面、道德账目、形式高潮、精确暴力、后果的重量。",
|
||||
"narrative": "结构是账本：谁欠谁什么，写成一张可核对的表。每一次暴力都必须记账，并在后面某处被结算。；- 复仇者与被复仇者共享同一套礼节。他们互相称呼得体、请对方坐下、倒茶、道歉——恐怖来自礼节没有中断。；- 用形式而不是台词交代秘密：一次镜像、一件重复出现的织物、两个不同时间点在同一位置的相同构图。；- 允许一次结构性的反转，但反转必须让此前所有礼貌行为的含义反过来，而",
|
||||
"camera": "主力 40–50mm 做正面对称；28mm 只在那一次不可能运动里用（需要把整个装饰空间收进来）；85mm 给装饰性插入镜头（手、餐具、绣线、纽扣）。；- 每场戏只允许一次\"不可能运动\"，且写成一条有起点和终点的路径：例如从 90 度顶俯的物件开始，下降并旋转 90 度，落在正面对称的人物中景上；或沿一面纹样墙横移，穿过墙体，继续在隔壁房间横移。多做一次，风格就变成噪音。；- 写这条路径时把后退量",
|
||||
"light": "主光是暖实用光（台灯、壁灯、廊下灯笼、窗户的斜射日光），必须以掠射角打在有纹样的表面上，让图案获得浮雕感——正打光会把花纹拍平，等于白做。；- 常规光比 3:1；暴力发生的那一颗拉到 8:1，方法是关掉补光而不是加硬光，让阴影自己吃掉一半的花纹。；- 冷色只从窗外进来，作为暖室内的对位，占画幅不超过 20%。；- 主色：酒红、橄榄绿、芥末金、胡桃木褐；重点色",
|
||||
"scene": "复仇、家族秘密、阶级伪装、囚禁与交换、道德账目型犯罪叙事。；- 需要\"美\"与\"残酷\"同时在场的段落：一次伤害发生在一个非常好看的房间里。；- 有美术预算或有真实纹样环境的项目：壁纸、地砖、刺绣、木雕、庭院植栽、旧式旅馆。；- 需要一次让观众\"看见摄影机\"的形式高潮：一个镜头做完一",
|
||||
},
|
||||
"malick": {
|
||||
"name": "马力克 · 自然光诗意",
|
||||
"one_line": "镜头贴着一只手，太阳在她背后压着地平线，摄影机以步速漂移，不停。",
|
||||
"keywords": "自然光、魔术时刻、广角贴近、步速漂移、手势、自然插入、耳语旁白、联想蒙太奇、无情节、风景中的身体。",
|
||||
"narrative": "不写情节点，写\"记忆序列\"：一串可以任意排序的片段，每段一句话（谁、在哪、做一个什么手势）。；- 时间是主观的：现在、回忆、想象之间不加标记地切换，观众靠旁白的人称和光的方向分辨。；- 旁白是内心的第二人称或对缺席者的呼告，8–15 个字一句，不描述画面，也不解释情节；它提问，画面不回答。；- 人物之间的关系由身体距离与朝向交代：谁走在谁前面、谁被落下、谁转",
|
||||
"camera": "主力 14–28mm，且必须贴近：拍手时镜头离手 30–50 厘米，拍脸时 60–90 厘米。广角加贴近＝前景巨大、地平线在后面，这是这个风格的全部秘密。35mm 只留给贴身跟走的中近景，再长就跟不上人也压不出前景。；- 摄影机永不锁定。手持或轻稳定器，以人的步速漂移（0.8–1.2 m/s），低频晃动，没有一次急停急起。禁止三脚架，禁止对称正面。；- 机位高度持续变化，0.3–1.8 米之间来回",
|
||||
"light": "只用现有光。允许的器材是一块白卡或一面反光板，不允许打灯。做不到就不拍这场。；- 拍摄窗口：日出后 20–35 分钟或日落前 20–35 分钟，太阳高度 2–10 度。太阳每小时最多爬升约 15 度（纬度越高越慢），所以这个高度带只能对应半小时上下的窗口，别按\"一整个金色小时\"排通告。阴天全天可用，晴天正午不可用。；- 太阳放在人物后方 150–210 度范",
|
||||
"scene": "回忆、成长、失去、信仰、家庭裂痕这类没有外部目标的抒情叙事。；- 品牌与公益片中需要\"感受先于信息\"的段落；婚礼、纪念、季节性影像。；- 有真实外景与可等待的日程的项目：能守到日出后 30 分钟或日落前 30 分钟。；- 素材式拍法：先拍大量碎片，在剪辑台上找出顺序——适合没有完",
|
||||
},
|
||||
"michael_mann": {
|
||||
"name": "迈克尔·曼 · 夜色与专业",
|
||||
"one_line": "夜里的城市是一块发着冷光的平原，一个懂行的人站在它前面，谁也不认识他。",
|
||||
"keywords": "都市夜、市政光源、混色温、地平线、专业能力、程序精确、高感噪点、玻璃反射、职业孤独、城市噪声即配乐。",
|
||||
"narrative": "人物由能力定义：出场前 60 秒内必须让观众看见他把一件专业的事做对，不靠台词说明身份。；- 双主角结构：对立的两方共享同一套职业伦理和作息，用平行剪辑并置两条日常线（同样的准备、同样的等待、同样的收尾），让观众先认同能力，再被迫选立场。；- 私人生活以碎片方式插入，永远被工作打断，而且从不解决；感情戏的功能是标出这个人不能拥有什么。；- 流程即情节：把一次",
|
||||
"camera": "主力 18–25mm 做\"室内也有地平线\"：机位退到房间深处，让整面窗和天际线成为人物背后的背景；35mm 做双人；50mm 只给流程插入（手、工具、接口、时间显示）。；- 机位眼平 1.6m，光轴保持水平。仰俯角一律不用，除非从车里拍或从高处看城市。人被压小是靠距离和构图，不靠角度。；- 人物在画幅中的高度目标是 8%–15%——按高度算，不要按宽度：25mm 镜头下 22 米外的人只占画幅宽度",
|
||||
"light": "不打电影灯。每一个光源都要说得出它在城里的名字：钠灯路灯、汞灯停车场、便利店的荧光管、LED 广告、车头灯、写字楼层的顶灯、施工照明。；- 同一帧里必须有至少两种不同色温对抗（例如左侧钠灯橙、右侧汞灯青绿），不要统一白平衡把它们调成一个颜色——那正是最常见的失败。；- 曝光按实用光源给，不按脸给：脸可以低于主光 1.5–2 档，允许看不清，允许出现可见的高感",
|
||||
"scene": "犯罪、盗窃、监视、执法、走私、网络安全这类\"专业者对专业者\"的题材。；- 需要把流程拍成戏的段落：装配、勘察、等待、交接、撤离——每一步都有自己的镜头。；- 都市夜外景：停车楼、高架下、机场外围、码头、深夜写字楼、住宅塔楼的外挂走廊。；- 需要\"孤独＝距离\"的情绪：人物之间永远隔",
|
||||
},
|
||||
"coen_brothers": {
|
||||
"name": "科恩兄弟 · 冷峻荒诞",
|
||||
"one_line": "把画面摆得一丝不苟，让演员绝对认真地演，然后比该切的时候多留一秒。",
|
||||
"keywords": "冷面、几何、地域具体性、广角低机位、命运是个平板笑话、能力反噬、荒诞记账、多留一秒、正面死中、难看的顶光。",
|
||||
"narrative": "先写\"计划书\"：主角的方案分 5–8 步，每步都合理。然后找出其中恰恰因为执行正确才引发灾难的那一步——那是全片的引擎。；- 引入一个不理解这套计划的外部力量（天气、一个较真的职员、一条法规、一只动物），它不邪恶，它只是不参与。；- 暴力必须记账：每一次伤害之后立刻拍善后——擦、搬、填表、解释、赔钱。观众的不适来自后勤，不来自血。；- 至少一个角色是道德常识",
|
||||
"camera": "主力 14–21mm 做低机位运动与怪诞近景；27mm 做冷面正面对称；40mm 只留给全片唯一一次不开玩笑的单人镜头。；- 机位是二元的：要么完全锁定正面死中（眼平 1.6m），要么一次硬性机械运动——0.4–0.8 米高的低机位推轨跑在行走者前方或后方，恒速直线。没有\"飘\"，没有微调构图。；- 广角低机位跑动镜头的规格：镜头高度 0.4–0.6 米，上仰 10–15 度，速度与被摄体一致；14",
|
||||
"light": "光要难看得有道理：顶部荧光灯管、一只裸灯泡、雪地反光、汽车旅馆的床头灯。顶光在每个人眼窝里留下阴影，让称职的人看起来像傻子——这是这个风格的核心照明策略。；- 室内比 3:1，夜外景 8:1；不用柔光箱美化人脸，不用发丝逆光。；- 每个空间有一个统一的、廉价的色偏：办公室是荧光管的绿白，汽车旅馆是钨丝的橙棕，雪地是无色的高亮，警局是日光灯的青白。；- 主色：",
|
||||
"scene": "黑色喜剧、犯罪失控、绑架与勒索、小人物执行大计划、官僚与流程荒诞。；- 强地域性题材：某个具体的县、某种口音、某种天气、某种连锁店——具体到能说出品类名称。；- 需要观众\"笑得不确定\"的段落：同一颗镜头里既有滑稽也有真实的伤害。；- 低成本可行：一条走廊、一间办公室、一片空地加一",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def get_director_style(key: str) -> dict:
|
||||
"""按 key 取导演风格（未知 key 返回空 dict）"""
|
||||
return DIRECTOR_STYLES.get((key or "").strip()) or {}
|
||||
|
||||
|
||||
def director_style_block(key: str) -> str:
|
||||
"""导演风格 → 提示词块（供导演系统注入）
|
||||
|
||||
只注入**可执行要点**（约 150-250 字），保持提示词简洁：
|
||||
核心视角一句 + 关键词 + 镜头语言前两条 + 灯光首要点
|
||||
"""
|
||||
st = get_director_style(key)
|
||||
if not st:
|
||||
return ""
|
||||
|
||||
def _first(s: str, n: int = 2) -> str:
|
||||
parts = [x.strip(" -") for x in str(s or "").split("；") if x.strip(" -")]
|
||||
return "；".join(parts[:n])[:220]
|
||||
|
||||
out = [f"【导演风格：{st['name']}】"]
|
||||
if st.get("one_line"):
|
||||
out.append(f"· 核心视角：{st['one_line'][:90]}")
|
||||
if st.get("keywords"):
|
||||
out.append(f"· 风格关键词：{st['keywords'][:80]}")
|
||||
cam = _first(st.get("camera"), 2)
|
||||
if cam:
|
||||
out.append(f"· 镜头语言：{cam}")
|
||||
lit = _first(st.get("light"), 1)
|
||||
if lit:
|
||||
out.append(f"· 灯光：{lit}")
|
||||
out.append("→ 用以上镜头语言与光影逻辑设计分镜（原创内容，不照搬具体镜头/台词）。")
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def director_style_options() -> dict:
|
||||
"""供前端下拉：{key: 中文名}"""
|
||||
return {k: v.get("name", k) for k, v in DIRECTOR_STYLES.items()}
|
||||
@@ -0,0 +1,607 @@
|
||||
"""
|
||||
composer/dubbing.py — 配音合成（TTS 人声 + 环境音铺底 + 时间轴对齐）
|
||||
|
||||
背景（2026-10-02 实测确认）：
|
||||
· 可灵生成的镜头**没有音轨**（payload 无音频参数）
|
||||
· 分镜的 Shot.text（台词）此前只拼进提示词，不产生任何声音
|
||||
· 导演台出片链路（pipeline.render）没有音频环节 → 成片无声
|
||||
|
||||
本模块补齐「声音」：
|
||||
按镜头时间轴，把台词逐句合成为一条完整音轨；每个镜头一段，段长严格等于
|
||||
镜头时长（短则补静音、长则加速），因此天然对齐画面。
|
||||
|
||||
音色来源（优先级）：
|
||||
1. 演员表指定的 voice_id（CastMember.voice_id）
|
||||
2. 按角色代号稳定分配的路人音色池（同一角色全片同一音色，不会听起来换人）
|
||||
|
||||
环境音来源：
|
||||
分镜的 Shot.audio_cue（如"远处车流声"）→ 与环境音库资产做关键词匹配 →
|
||||
该镜头期间低音量循环铺底。
|
||||
|
||||
输出：一条 44.1kHz 立体声 wav（可直接交给 StoryboardPipeline.mux_final 的 voice_path）
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Callable, Optional
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
from lib.logger import get_logger # noqa: E402
|
||||
from lib.ffmpeg import FFMPEG # noqa: E402 # 统一 ffmpeg 定位（绝对路径）
|
||||
|
||||
log = get_logger("dubbing")
|
||||
|
||||
API_BASE_SR = 44100
|
||||
DEFAULT_SFX_VOLUME = 0.85 # 音效默认音量（非语言发声为主，比环境音高）
|
||||
DEFAULT_AMBIENCE_VOLUME = 0.15
|
||||
DEFAULT_VOICE_VOLUME = 1.0
|
||||
|
||||
ProgressCB = Optional[Callable[[str, str, str, str], None]]
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 音色分配
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
# 路人音色池（未指定音色的角色按代号稳定分配；来自 config/cosyvoice_voices.yaml 的 pools）
|
||||
FALLBACK_MALE = ["longze_v3", "longzhe_v3", "longanlang_v3"]
|
||||
FALLBACK_FEMALE = ["longxing_v3", "longmiao_v3", "longanqin_v3"]
|
||||
NARRATOR_VOICE = "longxiu_v3"
|
||||
|
||||
|
||||
def _is_female(slot: str, note: str = "") -> bool:
|
||||
"""粗判角色性别（用于选路人音色池）：代号/备注里出现女/她/妈/姐/妹/婆 → 女声"""
|
||||
t = f"{slot} {note}"
|
||||
return bool(re.search(r"女|她|妈|姐|妹|婆|娘|太太|女士|girl|woman", t))
|
||||
|
||||
|
||||
def resolve_voice_map(sb, cast_voice_map: dict | None = None) -> dict:
|
||||
"""角色代号 → voice_id（确定性分配）
|
||||
|
||||
cast_voice_map: 演员表里显式指定的 {slot: voice_id}
|
||||
未指定的角色 → 按代号 hash 从路人池稳定取一个（同名永远同音色）
|
||||
"""
|
||||
cast_voice_map = cast_voice_map or {}
|
||||
out: dict[str, str] = {}
|
||||
# ⚠️ 字段名是 global_anchors（不是 anchors）—— 写错会导致演员表里的 voice_id 读不到，
|
||||
# 角色音色全部退化为"路人池按 hash 分配"（用户实测发现）
|
||||
_ga = getattr(sb, "global_anchors", None)
|
||||
_cast = []
|
||||
if _ga is not None and hasattr(_ga, "effective_cast"):
|
||||
try:
|
||||
_cast = _ga.effective_cast() or []
|
||||
except Exception:
|
||||
_cast = []
|
||||
for m in _cast:
|
||||
slot = getattr(m, "slot", "") or ""
|
||||
if not slot:
|
||||
continue
|
||||
vid = (cast_voice_map.get(slot) or getattr(m, "voice_id", "") or "").strip()
|
||||
if not vid:
|
||||
pool = FALLBACK_FEMALE if _is_female(slot, getattr(m, "note", "")) else FALLBACK_MALE
|
||||
vid = pool[sum(ord(c) for c in slot) % len(pool)]
|
||||
out[slot] = vid
|
||||
# 演员表未覆盖的出场代号（导演写的）也要有音色
|
||||
for shot in getattr(sb, "shots", []) or []:
|
||||
for slot in (getattr(shot, "cast_slots", None) or []):
|
||||
if slot and slot not in out:
|
||||
pool = FALLBACK_FEMALE if _is_female(slot) else FALLBACK_MALE
|
||||
out[slot] = pool[sum(ord(c) for c in slot) % len(pool)]
|
||||
return out
|
||||
|
||||
|
||||
def pick_voice_for_shot(shot, voice_map: dict, narrator: str = NARRATOR_VOICE) -> str:
|
||||
"""该镜头台词用谁的音色：出场人物第一顺位；无出场 → 旁白音色"""
|
||||
slots = getattr(shot, "cast_slots", None) or []
|
||||
for s in slots:
|
||||
if s in voice_map:
|
||||
return voice_map[s]
|
||||
if slots:
|
||||
# 演员表里没有的代号 → 用其自身 hash 分配
|
||||
pool = FALLBACK_FEMALE if _is_female(slots[0]) else FALLBACK_MALE
|
||||
return pool[sum(ord(c) for c in slots[0]) % len(pool)]
|
||||
return narrator
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 音频工具
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
def _run(cmd: list[str], timeout: int = 600) -> subprocess.CompletedProcess:
|
||||
return subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
|
||||
|
||||
|
||||
def probe_audio_duration(path: str) -> float:
|
||||
"""音频/视频时长（秒）；用 ffmpeg -i 解析（本机无真 ffprobe，见 lib/ffmpeg.py）"""
|
||||
from lib.ffmpeg import probe_duration
|
||||
d = probe_duration(path)
|
||||
return float(d or 0.0)
|
||||
|
||||
|
||||
def synthesize_line(text: str, voice_id: str, out_path: str,
|
||||
api_key: str = "", speed: float = 1.0) -> str:
|
||||
"""单句 TTS（阿里 CosyVoice）"""
|
||||
from voice_synthesizer.aliyun import synthesize
|
||||
os.makedirs(os.path.dirname(out_path) or ".", exist_ok=True)
|
||||
synthesize(text, out_path, api_key=api_key, voice_id=voice_id, speed=speed)
|
||||
return out_path
|
||||
|
||||
|
||||
def make_silence(duration_sec: float, dst: str) -> str:
|
||||
"""生成指定时长的静音段（44.1k 立体声 wav）"""
|
||||
r = _run([FFMPEG, "-y", "-f", "lavfi",
|
||||
"-i", f"anullsrc=channel_layout=stereo:sample_rate={API_BASE_SR}",
|
||||
"-t", f"{max(0.05, duration_sec):.3f}",
|
||||
"-c:a", "pcm_s16le", dst], timeout=120)
|
||||
if r.returncode != 0:
|
||||
raise RuntimeError(f"生成静音失败: {r.stderr[-300:]}")
|
||||
return dst
|
||||
|
||||
|
||||
def fit_audio_to_duration(src: str, dst: str, target_sec: float,
|
||||
max_speedup: float = 1.6) -> dict:
|
||||
"""把音频适配到目标时长
|
||||
|
||||
· 短于目标 → 补静音（apad），保持原速
|
||||
· 长于目标 → 加速（atempo，最多 max_speedup 倍）；仍超长则截断
|
||||
返回 {out, src_dur, target_dur, action}
|
||||
"""
|
||||
src_dur = probe_audio_duration(src)
|
||||
target = max(0.3, float(target_sec))
|
||||
action = "keep"
|
||||
afilters: list[str] = []
|
||||
if src_dur <= 0:
|
||||
# 探不到时长 → 直接按目标截断 + 补静音，交给 ffmpeg
|
||||
afilters = ["apad"]
|
||||
action = "unknown-duration"
|
||||
elif src_dur > target * 1.02:
|
||||
ratio = src_dur / target
|
||||
if ratio <= max_speedup:
|
||||
afilters = [f"atempo={ratio:.4f}"]
|
||||
action = f"加速×{ratio:.2f}"
|
||||
else:
|
||||
afilters = [f"atempo={max_speedup:.4f}", "apad"]
|
||||
action = f"加速×{max_speedup:.2f}+截断"
|
||||
else:
|
||||
afilters = ["apad"]
|
||||
action = "补静音"
|
||||
af = ",".join(afilters) if afilters else "anull"
|
||||
r = _run([FFMPEG, "-y", "-i", src, "-af", af,
|
||||
"-t", f"{target:.3f}", "-ar", str(API_BASE_SR), "-ac", "2",
|
||||
"-c:a", "pcm_s16le", dst], timeout=300)
|
||||
if r.returncode != 0:
|
||||
raise RuntimeError(f"音频适配失败: {r.stderr[-300:]}")
|
||||
return {"out": dst, "src_dur": round(src_dur, 2), "target_dur": round(target, 2), "action": action}
|
||||
|
||||
|
||||
def mix_segment(voice_path: str, ambience_path: str, duration_sec: float,
|
||||
dst: str, ambience_volume: float = DEFAULT_AMBIENCE_VOLUME,
|
||||
sfx_items: list | None = None) -> str:
|
||||
"""把「人声 + 环境音（循环）+ 音效（按时点插入）」混成一个镜头段，严格等于目标时长
|
||||
|
||||
sfx_items: [(音效文件路径, 插入秒数, 音量)] —— 非语言发声（惨叫/闷哼/关门声…）
|
||||
"""
|
||||
dur = max(0.3, float(duration_sec))
|
||||
sfx_items = [x for x in (sfx_items or [])
|
||||
if x and x[0] and Path(x[0]).exists()]
|
||||
has_v = bool(voice_path and Path(voice_path).exists())
|
||||
has_a = bool(ambience_path and Path(ambience_path).exists())
|
||||
if not (has_v or has_a or sfx_items):
|
||||
return make_silence(dur, dst)
|
||||
|
||||
inputs: list[str] = []
|
||||
if has_v:
|
||||
inputs += ["-i", voice_path]
|
||||
if has_a:
|
||||
inputs += ["-stream_loop", "-1", "-i", ambience_path]
|
||||
for f, _at, _vol in sfx_items:
|
||||
inputs += ["-i", f]
|
||||
|
||||
parts: list[str] = []
|
||||
labels: list[str] = []
|
||||
idx = 0
|
||||
if has_v:
|
||||
parts.append(f"[{idx}:a]volume={DEFAULT_VOICE_VOLUME},apad[v]")
|
||||
labels.append("[v]"); idx += 1
|
||||
if has_a:
|
||||
parts.append(f"[{idx}:a]volume={ambience_volume},atrim=0:{dur:.3f},asetpts=N/SR/TB[a]")
|
||||
labels.append("[a]"); idx += 1
|
||||
for k, (f, at, vol) in enumerate(sfx_items):
|
||||
ms = int(max(0.0, min(float(at), dur - 0.05)) * 1000)
|
||||
parts.append(f"[{idx}:a]volume={vol},adelay={ms}|{ms}[s{k}]")
|
||||
labels.append(f"[s{k}]"); idx += 1
|
||||
|
||||
if len(labels) == 1:
|
||||
parts.append(f"{labels[0]}anull[out]")
|
||||
else:
|
||||
parts.append(f"{''.join(labels)}amix=inputs={len(labels)}:duration=first:normalize=0[out]")
|
||||
|
||||
r = _run([FFMPEG, "-y", *inputs, "-filter_complex", ";".join(parts),
|
||||
"-map", "[out]", "-t", f"{dur:.3f}",
|
||||
"-ar", str(API_BASE_SR), "-ac", "2", "-c:a", "pcm_s16le", dst], timeout=300)
|
||||
if r.returncode != 0:
|
||||
raise RuntimeError(f"段混音失败: {r.stderr[-300:]}")
|
||||
return dst
|
||||
|
||||
|
||||
def concat_audio(seg_paths: list[str], out_path: str) -> str:
|
||||
"""按顺序拼接音频段（同规格 wav → concat demuxer）"""
|
||||
if not seg_paths:
|
||||
raise ValueError("没有音频段可拼接")
|
||||
lst = Path(out_path).with_suffix(".txt")
|
||||
lst.write_text("".join(f"file '{Path(p).resolve()}'\n" for p in seg_paths), encoding="utf-8")
|
||||
r = _run([FFMPEG, "-y", "-f", "concat", "-safe", "0", "-i", str(lst),
|
||||
"-c:a", "pcm_s16le", "-ar", str(API_BASE_SR), "-ac", "2", out_path], timeout=600)
|
||||
lst.unlink(missing_ok=True)
|
||||
if r.returncode != 0:
|
||||
raise RuntimeError(f"音频拼接失败: {r.stderr[-300:]}")
|
||||
return out_path
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 环境音匹配（分镜 audio_cue ↔ 环境音库）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
def load_ambience_assets() -> list[dict]:
|
||||
"""读取环境音库资产（含本地文件路径）"""
|
||||
try:
|
||||
from assets_manager import AssetStore
|
||||
st = AssetStore("ambience")
|
||||
out = []
|
||||
for a in st.list():
|
||||
f = ""
|
||||
for slot in ("audio", "file", "sample"):
|
||||
v = a.get(slot) or ""
|
||||
if v:
|
||||
try:
|
||||
f = str(Path(st.kind_dir(a["id"])) / Path(v).name)
|
||||
except Exception:
|
||||
f = ""
|
||||
break
|
||||
if not f or not Path(f).exists():
|
||||
# 退一步：目录下第一个音频文件
|
||||
try:
|
||||
d = Path(st.kind_dir(a["id"]))
|
||||
cands = [p for p in d.glob("*") if p.suffix.lower() in (".mp3", ".wav", ".m4a", ".aac", ".ogg")]
|
||||
f = str(cands[0]) if cands else ""
|
||||
except Exception:
|
||||
f = ""
|
||||
a["_file"] = f
|
||||
out.append(a)
|
||||
return out
|
||||
except Exception as e:
|
||||
log.warning("环境音库读取失败: %s", e)
|
||||
return []
|
||||
|
||||
|
||||
def load_sfx_assets() -> list[dict]:
|
||||
"""读取音效库资产（含本地文件路径）—— 结构同环境音库"""
|
||||
try:
|
||||
from assets_manager import AssetStore
|
||||
st = AssetStore("sfx")
|
||||
out = []
|
||||
for a in st.list():
|
||||
f = ""
|
||||
for slot in ("audio", "file", "sample"):
|
||||
v = a.get(slot) or ""
|
||||
if v:
|
||||
try:
|
||||
f = str(Path(st.kind_dir(a["id"])) / Path(v).name)
|
||||
except Exception:
|
||||
f = ""
|
||||
break
|
||||
if not f or not Path(f).exists():
|
||||
try:
|
||||
d = Path(st.kind_dir(a["id"]))
|
||||
cands = [p for p in d.glob("*")
|
||||
if p.suffix.lower() in (".mp3", ".wav", ".m4a", ".aac", ".ogg")]
|
||||
f = str(cands[0]) if cands else ""
|
||||
except Exception:
|
||||
f = ""
|
||||
a["_file"] = f
|
||||
out.append(a)
|
||||
return out
|
||||
except Exception as e:
|
||||
log.warning("音效库读取失败: %s", e)
|
||||
return []
|
||||
|
||||
|
||||
def match_sfx(desc: str, assets: list[dict]) -> str:
|
||||
"""用音效描述匹配音效资产（名称/标签最长子串命中）→ 文件路径
|
||||
|
||||
例：desc「痛苦的闷哼」→ 命中音效「闷哼」/「痛呼」
|
||||
"""
|
||||
if not desc or not assets:
|
||||
return ""
|
||||
d = str(desc)
|
||||
best, best_len = "", 0
|
||||
for a in assets:
|
||||
f = a.get("_file") or ""
|
||||
if not f:
|
||||
continue
|
||||
for key in [a.get("name", "")] + list(a.get("tags") or []):
|
||||
k = str(key or "").strip()
|
||||
if len(k) >= 2 and (k in d or d in k) and len(k) > best_len:
|
||||
best, best_len = f, len(k)
|
||||
return best
|
||||
|
||||
|
||||
def match_ambience(cue: str, assets: list[dict]) -> str:
|
||||
"""用 audio_cue 文本匹配环境音资产（名称/标签最长子串命中）→ 文件路径"""
|
||||
if not cue or not assets:
|
||||
return ""
|
||||
best, best_len = "", 0
|
||||
for a in assets:
|
||||
f = a.get("_file") or ""
|
||||
if not f:
|
||||
continue
|
||||
for key in [a.get("name", "")] + list(a.get("tags") or []):
|
||||
k = str(key or "").strip()
|
||||
if len(k) >= 2 and k in cue and len(k) > best_len:
|
||||
best, best_len = f, len(k)
|
||||
return best
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 主入口
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
def pre_synthesize(sb, work_dir: str, api_key: str = "",
|
||||
cast_voice_map: dict | None = None,
|
||||
min_dur: float = 2.0, max_dur: float = 10.0,
|
||||
pad: float = 0.4, progress_cb: ProgressCB = None) -> dict:
|
||||
"""【音频先行】逐镜合成台词 → 按音频**真实时长**更新镜头时长
|
||||
|
||||
为什么：画面时长若拍脑袋定，台词长了就得加速/截断 → 口型节奏必然对不上。
|
||||
音频先行 = 让画面跟着声音走。
|
||||
|
||||
· 有台词的镜头：TTS → 探测时长 → shot.duration = clamp(音频时长 + 气口, min, max)
|
||||
· 无台词镜头：保持原时长
|
||||
· 音频存到 {work_dir}/{shot_id}.mp3 → 后续混流**直接复用，不重复 TTS**
|
||||
|
||||
返回 {updated, shots:{shot_id:{audio,audio_dur,old_dur,new_dur,voice_id}}, notes}
|
||||
"""
|
||||
work = Path(work_dir)
|
||||
work.mkdir(parents=True, exist_ok=True)
|
||||
from lib.config import load_config
|
||||
cfg = load_config()
|
||||
api_key = api_key or (cfg.get("aliyun", {}) or {}).get("api_key", "")
|
||||
if not api_key:
|
||||
raise RuntimeError("音频先行需要百炼 API Key（TTS）")
|
||||
|
||||
voice_map = resolve_voice_map(sb, cast_voice_map)
|
||||
out: dict = {}
|
||||
notes: list = []
|
||||
updated = 0
|
||||
shots = list(getattr(sb, "shots", []) or [])
|
||||
for idx, shot in enumerate(shots, 1):
|
||||
sid = getattr(shot, "shot_id", f"shot_{idx:02d}")
|
||||
text = (getattr(shot, "text", "") or "").strip()
|
||||
if not text:
|
||||
continue
|
||||
vid = pick_voice_for_shot(shot, voice_map)
|
||||
mp3 = work / f"{sid}.mp3"
|
||||
if not (mp3.exists() and mp3.stat().st_size > 1000): # 已有则复用
|
||||
synthesize_line(text, vid, str(mp3), api_key=api_key)
|
||||
adur = probe_audio_duration(str(mp3))
|
||||
if adur <= 0:
|
||||
notes.append(f"{sid}: 音频时长探测失败，保持原时长")
|
||||
continue
|
||||
old = float(getattr(shot, "duration", 5) or 5)
|
||||
new = max(min_dur, min(max_dur, round(adur + pad, 2)))
|
||||
shot.duration = new
|
||||
if abs(new - old) > 0.01:
|
||||
updated += 1
|
||||
notes.append(f"{sid}: {old}s → {new}s（音频 {adur:.2f}s · {vid}）")
|
||||
out[sid] = {"audio": str(mp3), "audio_dur": round(adur, 2),
|
||||
"old_dur": old, "new_dur": new, "voice_id": vid,
|
||||
"has_cast": bool([s for s in (getattr(shot, "cast_slots", None) or []) if s])}
|
||||
if progress_cb:
|
||||
progress_cb("tts", sid, "done", f"{adur:.1f}s → 镜头 {new}s")
|
||||
log.info(" 🎙 音频先行：%d/%d 个镜头时长按台词音频调整", updated, len(shots))
|
||||
return {"updated": updated, "shots": out, "notes": notes, "voice_map": voice_map}
|
||||
|
||||
|
||||
def build_voice_track(sb, out_path: str, work_dir: str,
|
||||
api_key: str = "",
|
||||
cast_voice_map: dict | None = None,
|
||||
use_ambience: bool = True,
|
||||
ambience_volume: float = DEFAULT_AMBIENCE_VOLUME,
|
||||
progress_cb: ProgressCB = None,
|
||||
reuse_dir: str = "") -> dict:
|
||||
"""按分镜时间轴合成完整音轨
|
||||
|
||||
返回 {output, segments, voice_seconds, ambience_hits, voices:[...], notes:[...]}
|
||||
"""
|
||||
work = Path(work_dir)
|
||||
work.mkdir(parents=True, exist_ok=True)
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
from lib.config import load_config
|
||||
cfg = load_config()
|
||||
api_key = api_key or (cfg.get("aliyun", {}) or {}).get("api_key", "")
|
||||
if not api_key:
|
||||
raise RuntimeError("未配置百炼 API Key（TTS 需要）")
|
||||
|
||||
voice_map = resolve_voice_map(sb, cast_voice_map)
|
||||
amb_assets = load_ambience_assets() if use_ambience else []
|
||||
sfx_assets = load_sfx_assets() # 音效库（非语言发声：惨叫/闷哼/关门…）
|
||||
log.info("配音: 音色映射 %s | 环境音库 %d 条 | 音效库 %d 条",
|
||||
voice_map, len(amb_assets), len(sfx_assets))
|
||||
|
||||
segs: list[str] = []
|
||||
notes: list[str] = []
|
||||
voice_seconds = 0.0
|
||||
amb_hits = 0
|
||||
sfx_hits = 0
|
||||
# ★ 分轨（用户 2026-10-03 提出的声音模型）：
|
||||
# · 台词轨（dialogue）：有出场人物的镜头 → 与画面强绑定（还要用于对口型）
|
||||
# · 旁白轨（narration）：无出场人物 → 独立一条声轨，允许与画面有轻微差异
|
||||
dialogue_segs: list[str] = []
|
||||
narration_segs: list[str] = []
|
||||
|
||||
shots = list(getattr(sb, "shots", []) or [])
|
||||
for idx, shot in enumerate(shots, 1):
|
||||
sid = getattr(shot, "shot_id", f"shot_{idx:02d}")
|
||||
dur = float(getattr(shot, "duration", 3) or 3)
|
||||
text = (getattr(shot, "text", "") or "").strip()
|
||||
if progress_cb:
|
||||
progress_cb("dub", sid, "running", f"{idx}/{len(shots)}")
|
||||
|
||||
voice_path = ""
|
||||
if text:
|
||||
vid = pick_voice_for_shot(shot, voice_map)
|
||||
# ★ 优先复用「音频先行」已合成的音频（不重复 TTS = 不重复计费）
|
||||
_pre = Path(reuse_dir) / f"{sid}.mp3" if reuse_dir else None
|
||||
if _pre and _pre.exists() and _pre.stat().st_size > 1000:
|
||||
raw = _pre
|
||||
notes.append(f"{sid}: 复用先行音频")
|
||||
else:
|
||||
raw = work / f"tts_{idx:02d}.mp3"
|
||||
synthesize_line(text, vid, str(raw), api_key=api_key)
|
||||
try:
|
||||
fitted = work / f"voice_{idx:02d}.wav"
|
||||
info = fit_audio_to_duration(str(raw), str(fitted), dur)
|
||||
voice_path = str(fitted)
|
||||
voice_seconds += min(info["src_dur"], dur)
|
||||
notes.append(f"{sid}: {vid} · {info['action']}（{len(text)}字）")
|
||||
except Exception as e:
|
||||
log.warning(" ⚠️ %s 台词合成失败: %s", sid, e)
|
||||
notes.append(f"{sid}: 台词合成失败({str(e)[:40]})")
|
||||
voice_path = ""
|
||||
|
||||
# 环境音（该镜头的 audio_cue）
|
||||
amb_path = ""
|
||||
if amb_assets:
|
||||
_a = getattr(shot, "audio", None)
|
||||
cue = ""
|
||||
if _a is not None:
|
||||
cue = (_a.get("audio_cue", "") if isinstance(_a, dict)
|
||||
else getattr(_a, "audio_cue", "")) or ""
|
||||
cue = str(cue).strip()
|
||||
if cue:
|
||||
amb_path = match_ambience(cue, amb_assets)
|
||||
if amb_path:
|
||||
amb_hits += 1
|
||||
notes.append(f"{sid}: 环境音「{cue}」匹配 ✓")
|
||||
|
||||
# 音效（非语言发声）：按 desc 匹配素材，按 at_second 插入该镜头内
|
||||
sfx_items: list = []
|
||||
if sfx_assets:
|
||||
_a = getattr(shot, "audio", None)
|
||||
raw_sfx = []
|
||||
if _a is not None:
|
||||
raw_sfx = ((_a.get("sfx", []) if isinstance(_a, dict)
|
||||
else getattr(_a, "sfx", [])) or [])
|
||||
if isinstance(raw_sfx, (str, dict)):
|
||||
raw_sfx = [raw_sfx]
|
||||
for s in raw_sfx:
|
||||
if isinstance(s, dict):
|
||||
desc = str(s.get("desc") or s.get("name") or "").strip()
|
||||
at = float(s.get("at_second") or 0)
|
||||
vol = float(s.get("volume") or DEFAULT_SFX_VOLUME)
|
||||
else:
|
||||
desc, at, vol = str(s or "").strip(), 0.0, DEFAULT_SFX_VOLUME
|
||||
if not desc:
|
||||
continue
|
||||
f = match_sfx(desc, sfx_assets)
|
||||
if f:
|
||||
sfx_items.append((f, at, vol))
|
||||
sfx_hits += 1
|
||||
notes.append(f"{sid}: 音效「{desc}」@{at}s ✓")
|
||||
else:
|
||||
notes.append(f"{sid}: 音效「{desc}」未匹配到素材（音效库为空？）")
|
||||
|
||||
# ★ 分轨：判断这一段是「角色台词」还是「旁白」
|
||||
# 台词（有出场人物）→ 主轨；旁白（无出场人物）→ 单独轨（不重复放进主轨）
|
||||
_slots = [x for x in (getattr(shot, "cast_slots", None) or []) if x]
|
||||
is_narration = bool(text) and not _slots
|
||||
voice_for_main = "" if is_narration else voice_path
|
||||
|
||||
seg = work / f"seg_{idx:02d}.wav"
|
||||
mix_segment(voice_for_main, amb_path, dur, str(seg),
|
||||
ambience_volume=ambience_volume, sfx_items=sfx_items)
|
||||
segs.append(str(seg))
|
||||
|
||||
# 旁白轨：该镜头是旁白 → 只放人声；否则留空（拼接时补等长静音）
|
||||
if is_narration:
|
||||
narr_seg = work / f"narr_{idx:02d}.wav"
|
||||
try:
|
||||
mix_segment(voice_path, "", dur, str(narr_seg))
|
||||
narration_segs.append(str(narr_seg))
|
||||
notes.append(f"{sid}: 旁白（单独轨）")
|
||||
except Exception as e:
|
||||
log.warning(" ⚠️ %s 旁白轨生成失败: %s", sid, e)
|
||||
narration_segs.append("")
|
||||
else:
|
||||
narration_segs.append("")
|
||||
|
||||
if progress_cb:
|
||||
progress_cb("dub", sid, "done",
|
||||
"旁白" if is_narration else ("角色台词" if text else "无台词"))
|
||||
|
||||
concat_audio(segs, out_path)
|
||||
|
||||
# ★ 旁白轨（独立输出）：无旁白的镜头补等长静音，保证与主轨时间轴对齐
|
||||
narr_out = ""
|
||||
if any(narration_segs):
|
||||
narr_parts = []
|
||||
for i, (n, shot) in enumerate(zip(narration_segs, shots), 1):
|
||||
if n:
|
||||
narr_parts.append(n)
|
||||
else:
|
||||
dur_i = float(getattr(shot, "duration", 3) or 3)
|
||||
sil = work / f"narr_sil_{i:02d}.wav"
|
||||
try:
|
||||
if not sil.exists():
|
||||
make_silence(dur_i, str(sil))
|
||||
narr_parts.append(str(sil))
|
||||
except Exception:
|
||||
pass
|
||||
if narr_parts:
|
||||
narr_out = str(Path(out_path).with_name("narration_track.wav"))
|
||||
try:
|
||||
concat_audio(narr_parts, narr_out)
|
||||
log.info(" 🎙 旁白轨单独输出: %s（%d 段）",
|
||||
Path(narr_out).name, len([x for x in narration_segs if x]))
|
||||
except Exception as e:
|
||||
log.warning(" ⚠️ 旁白轨拼接失败: %s", e)
|
||||
narr_out = ""
|
||||
|
||||
total = sum(float(getattr(s, "duration", 3) or 3) for s in shots)
|
||||
log.info(" 🎙 配音完成: %s（%d 段 / %.1fs 音轨）", Path(out_path).name, len(segs), total)
|
||||
return {"output": out_path, "narration": narr_out, "segments": len(segs),
|
||||
"voice_seconds": round(voice_seconds, 2), "ambience_hits": amb_hits,
|
||||
"sfx_hits": sfx_hits,
|
||||
"narration_segments": len([x for x in narration_segs if x]),
|
||||
"total_seconds": round(total, 2), "voices": voice_map, "notes": notes}
|
||||
|
||||
|
||||
# ── CLI 自测：从已保存分镜生成音轨 ──
|
||||
if __name__ == "__main__":
|
||||
import argparse, json
|
||||
ap = argparse.ArgumentParser(description="从分镜生成配音音轨（有 TTS 费用）")
|
||||
ap.add_argument("--storyboard", required=True, help="分镜 JSON 路径或 project_id")
|
||||
ap.add_argument("--output", default="", help="输出 wav 路径")
|
||||
ap.add_argument("--no-ambience", action="store_true", help="不使用环境音")
|
||||
args = ap.parse_args()
|
||||
|
||||
p = Path(args.storyboard)
|
||||
if not p.exists():
|
||||
p = Path.home() / "workbuddy-agent-os/agent-local/runtime/ave/storyboards" / f"{args.storyboard}.json"
|
||||
data = json.loads(p.read_text(encoding="utf-8"))
|
||||
from schemas import Storyboard
|
||||
sb = Storyboard.from_dict(data)
|
||||
out = args.output or str(Path.home() / "workbuddy-agent-os/agent-local/runtime/ave/dub_test.wav")
|
||||
r = build_voice_track(sb, out, str(Path(out).parent / "dub_work"),
|
||||
use_ambience=not args.no_ambience)
|
||||
print(json.dumps({k: v for k, v in r.items() if k != "notes"}, ensure_ascii=False, indent=1))
|
||||
for n in r["notes"]:
|
||||
print(" ·", n)
|
||||
@@ -0,0 +1,699 @@
|
||||
"""
|
||||
|
||||
AVE 06_composer — FFmpeg 合成模块
|
||||
|
||||
职责:
|
||||
1. 音频混音: 人声 + BGM → 混合音频
|
||||
2. 视频合成: 素材片段拼接 + 画面过渡
|
||||
3. 字幕叠加: ASS 格式中文字幕（带重音高亮）
|
||||
4. 最终编码: 输出 final.mp4
|
||||
5. 分段渲染: 3分钟以上视频自动分段
|
||||
|
||||
依赖:
|
||||
- ffmpeg (需安装)
|
||||
- Python: ffmpeg-python
|
||||
"""
|
||||
import sys, os; sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from lib.logger import get_logger
|
||||
from lib.ffmpeg import FFMPEG # noqa: E402 (FFmpeg 绝对路径)
|
||||
|
||||
logger = get_logger("composer")
|
||||
|
||||
# ── 常量 ──────────────────────────────────────────────────
|
||||
DEFAULT_RESOLUTION = "1080x1920" # 竖屏
|
||||
DEFAULT_FPS = 30
|
||||
DEFAULT_BITRATE = "8M"
|
||||
SEGMENT_DURATION_MAX = 180 # 3分钟分段
|
||||
|
||||
|
||||
# ── 公共 API ──────────────────────────────────────────────
|
||||
|
||||
def mix_audio(
|
||||
voice_path: str,
|
||||
bgm_path: Optional[str] = None,
|
||||
output_path: str = "mixed_audio.wav",
|
||||
voice_volume: float = 1.0,
|
||||
bgm_volume: float = 0.3,
|
||||
) -> str:
|
||||
"""
|
||||
混音: 人声 + BGM → 混合音频
|
||||
|
||||
- voice_path: 人声 WAV 路径
|
||||
- bgm_path: BGM 路径 (None 则只输出人声)
|
||||
- voice_volume: 人声音量倍率
|
||||
- bgm_volume: BGM 音量倍率
|
||||
- 返回: 混合音频路径
|
||||
"""
|
||||
if not os.path.exists(voice_path):
|
||||
raise FileNotFoundError(f"人声文件不存在: {voice_path}")
|
||||
|
||||
if bgm_path and not os.path.exists(bgm_path):
|
||||
logger.warning(f"BGM 文件不存在，跳过背景音乐: {bgm_path}")
|
||||
bgm_path = None
|
||||
|
||||
if not bgm_path:
|
||||
# 无人声时，直接复制人声
|
||||
subprocess.run(
|
||||
[FFMPEG, "-y", "-i", voice_path, "-acodec", "pcm_s16le", output_path],
|
||||
capture_output=True, check=True
|
||||
)
|
||||
logger.info(f"音频已输出（无BGM）: {output_path}")
|
||||
return output_path
|
||||
|
||||
# 双轨混音
|
||||
cmd = [
|
||||
FFMPEG, "-y",
|
||||
"-i", voice_path,
|
||||
"-i", bgm_path,
|
||||
"-filter_complex",
|
||||
f"[0:a]volume={voice_volume}[v];[1:a]volume={bgm_volume}[b];[v][b]amix=inputs=2:duration=first[a]",
|
||||
"-map", "[a]",
|
||||
"-acodec", "pcm_s16le",
|
||||
output_path,
|
||||
]
|
||||
subprocess.run(cmd, capture_output=True, check=True)
|
||||
logger.info(f"混音完成: {output_path}")
|
||||
return output_path
|
||||
|
||||
|
||||
def duck_bgm(
|
||||
voice_path: str,
|
||||
bgm_path: str,
|
||||
output_path: str = "ducked_bgm.wav",
|
||||
low_volume: float = 0.15,
|
||||
high_volume: float = 0.50,
|
||||
fade_ms: int = 300,
|
||||
word_timestamps: list | None = None,
|
||||
) -> str:
|
||||
"""
|
||||
BGM 音量避让 (Audio Ducking), 基于字级时间戳
|
||||
|
||||
说话时 BGM 压低 (low_volume), 间隙时恢复 (high_volume)。
|
||||
使用 word_timestamps 精确控制每个字的避让区间,
|
||||
比 silencedetect 更准确 (CosyVoice 连续语音无停顿).
|
||||
|
||||
参数:
|
||||
word_timestamps: [{begin_time, end_time}, ...] 单位毫秒
|
||||
来自 synthesize_with_timestamps()
|
||||
"""
|
||||
if not os.path.exists(voice_path):
|
||||
raise FileNotFoundError(f"人声文件不存在: {voice_path}")
|
||||
if not os.path.exists(bgm_path):
|
||||
raise FileNotFoundError(f"BGM 文件不存在: {bgm_path}")
|
||||
|
||||
# 如果没有时间戳, 回退到固定音量
|
||||
if not word_timestamps:
|
||||
logger.info(" 无时间戳, 使用固定音量 0.25")
|
||||
subprocess.run([
|
||||
FFMPEG, "-y", "-i", bgm_path,
|
||||
"-af", f"volume={high_volume}",
|
||||
"-acodec", "pcm_s16le", output_path,
|
||||
], capture_output=True, check=True, timeout=30)
|
||||
return output_path
|
||||
|
||||
duration = _get_media_duration(voice_path)
|
||||
total_ms = duration * 1000
|
||||
fade_ms_f = fade_ms / 1000.0
|
||||
|
||||
logger.info(f"BGM 避让: {low_volume}→{high_volume}, 基于 {len(word_timestamps)} 个字")
|
||||
|
||||
# 构建音量区间: [(start_sec, end_sec, volume), ...]
|
||||
# 每个字的区间: 字开始前 fade_ms 开始压低, 字结束后 fade_ms 恢复
|
||||
segments = []
|
||||
# 先确定所有活跃区间 (字的时间范围 + 过渡)
|
||||
active_zones = [] # [(start, end), ...] 单位为秒
|
||||
for w in word_timestamps:
|
||||
w_start = max(0, (w["begin_time"] - 50) / 1000.0) # 字前 50ms
|
||||
w_end = min(duration, (w["end_time"] + 50) / 1000.0) # 字后 50ms
|
||||
active_zones.append((w_start, w_end))
|
||||
|
||||
# 合并重叠的活跃区间
|
||||
active_zones.sort()
|
||||
merged = []
|
||||
for start, end in active_zones:
|
||||
if merged and start <= merged[-1][1] + 0.05: # 50ms 内视为连续
|
||||
merged[-1] = (merged[-1][0], max(merged[-1][1], end))
|
||||
else:
|
||||
merged.append((start, end))
|
||||
|
||||
# 构建音量分段: 活跃期低音量, 间隙期高音量
|
||||
prev_end = 0.0
|
||||
for a_start, a_end in merged:
|
||||
# 间隙: prev_end → a_start (高音量)
|
||||
if a_start > prev_end + 0.05:
|
||||
gap_mid = (prev_end + a_start) / 2
|
||||
segments.append((prev_end, gap_mid, high_volume))
|
||||
segments.append((gap_mid, a_start, high_volume))
|
||||
# 说话: a_start → a_end (低音量, 带过渡)
|
||||
segments.append((a_start, a_end, low_volume))
|
||||
prev_end = a_end
|
||||
# 最后间隙
|
||||
if duration > prev_end + 0.05:
|
||||
segments.append((prev_end, duration, high_volume))
|
||||
|
||||
if not segments:
|
||||
logger.info(" 无分段, 使用固定音量")
|
||||
subprocess.run([
|
||||
FFMPEG, "-y", "-i", bgm_path,
|
||||
"-af", f"volume={high_volume}",
|
||||
"-acodec", "pcm_s16le", output_path,
|
||||
], capture_output=True, check=True, timeout=30)
|
||||
return output_path
|
||||
|
||||
# 渲染每段再拼接
|
||||
import tempfile, shutil
|
||||
seg_dir = tempfile.mkdtemp(prefix="ave_duck_")
|
||||
seg_wavs = []
|
||||
for idx, (st, en, vol) in enumerate(segments):
|
||||
dur = en - st
|
||||
if dur < 0.05:
|
||||
continue
|
||||
seg_out = os.path.join(seg_dir, f"seg_{idx:03d}.wav")
|
||||
fade_dur = min(fade_ms_f, dur / 3)
|
||||
subprocess.run([
|
||||
FFMPEG, "-y",
|
||||
"-ss", str(st), "-i", bgm_path,
|
||||
"-t", str(dur),
|
||||
"-af", f"volume={vol},afade=t=in:d={fade_dur},afade=t=out:st={dur-fade_dur}:d={fade_dur}",
|
||||
"-acodec", "pcm_s16le", seg_out,
|
||||
], capture_output=True, timeout=60, check=True)
|
||||
seg_wavs.append(seg_out)
|
||||
|
||||
# 拼接
|
||||
concat_file = _create_concat_list(seg_wavs)
|
||||
if len(seg_wavs) == 1:
|
||||
subprocess.run([FFMPEG, "-y", "-i", seg_wavs[0], "-acodec", "pcm_s16le", output_path],
|
||||
capture_output=True, check=True)
|
||||
else:
|
||||
subprocess.run([
|
||||
FFMPEG, "-y",
|
||||
"-f", "concat", "-safe", "0", "-i", concat_file,
|
||||
"-acodec", "pcm_s16le", output_path,
|
||||
], capture_output=True, check=True, timeout=60)
|
||||
shutil.rmtree(seg_dir, ignore_errors=True)
|
||||
|
||||
logger.info(f"BGM 避让完成: {output_path} ({len(seg_wavs)} 段)")
|
||||
return output_path
|
||||
|
||||
|
||||
def image_to_clip(
|
||||
input_path: str,
|
||||
output_path: str,
|
||||
duration: float = 5.0,
|
||||
resolution: str = DEFAULT_RESOLUTION,
|
||||
ken_burns: bool = True,
|
||||
) -> str:
|
||||
"""
|
||||
将静态图片转换为视频分镜（带 Ken Burns 动效）
|
||||
|
||||
处理横图→竖屏转换（如 1248×832→1080×1920），用于 AVE 流水线中
|
||||
图片生成阶段→合成阶段之间的转换步骤。
|
||||
|
||||
策略：先按目标高度等比缩放填满画面，然后 crop 到目标宽度，
|
||||
再添加 Ken Burns 式平滑平移动效。
|
||||
"""
|
||||
target_w, target_h = [int(x) for x in resolution.split("x")]
|
||||
|
||||
if ken_burns:
|
||||
# 使用 cos 实现平滑左右平移: 从右到左的平滑摆动
|
||||
filter_str = (
|
||||
f"scale='max({target_w},iw*{target_h}/ih)':{target_h}:flags=lanczos,"
|
||||
f"crop={target_w}:{target_h}:"
|
||||
f"'(iw-{target_w})*(0.5-0.45*cos(PI*t/{duration}))':'0',"
|
||||
f"setsar=1,fps={DEFAULT_FPS}"
|
||||
)
|
||||
else:
|
||||
# 无动效：居中裁剪
|
||||
filter_str = (
|
||||
f"scale='max({target_w},iw*{target_h}/ih)':{target_h}:flags=lanczos,"
|
||||
f"crop={target_w}:{target_h}:'(iw-{target_w})/2':'(ih-{target_h})/2',"
|
||||
f"setsar=1,fps={DEFAULT_FPS}"
|
||||
)
|
||||
|
||||
cmd = [
|
||||
FFMPEG, "-y",
|
||||
"-loop", "1", "-t", str(duration),
|
||||
"-i", input_path,
|
||||
"-vf", filter_str,
|
||||
"-c:v", "libx264", "-preset", "medium", "-crf", "23",
|
||||
"-pix_fmt", "yuv420p", "-an",
|
||||
output_path,
|
||||
]
|
||||
subprocess.run(cmd, capture_output=True, timeout=300, check=True)
|
||||
logger.info(f"图片→视频分镜: {os.path.basename(input_path)} → {os.path.basename(output_path)} ({duration}s)")
|
||||
return output_path
|
||||
|
||||
|
||||
def _apply_ken_burns(input_path: str, output_path: str, resolution: str = DEFAULT_RESOLUTION) -> str:
|
||||
"""
|
||||
对视频素材施加随机 Ken Burns 效果（缩放+平移），让画面动起来。
|
||||
|
||||
策略：先获取原始尺寸，判断横竖屏，选择合适的缩放+裁切方案。
|
||||
竖屏素材(720x1280等)：放大后以宽度为基准裁切到1080。
|
||||
"""
|
||||
import random
|
||||
duration = _get_media_duration(input_path)
|
||||
if duration < 1.0:
|
||||
duration = 5.0
|
||||
|
||||
target_w, target_h = [int(x) for x in resolution.split("x")]
|
||||
|
||||
# 获取原视频尺寸
|
||||
try:
|
||||
from lib.ffmpeg import probe_info
|
||||
_info = probe_info(input_path)
|
||||
src_w, src_h = _info.get("width") or target_w, _info.get("height") or target_h
|
||||
except Exception:
|
||||
src_w, src_h = target_w, target_h
|
||||
|
||||
# 判断方向
|
||||
is_portrait = src_h > src_w # 竖屏
|
||||
|
||||
if is_portrait and src_w >= target_w:
|
||||
# 竖屏且宽度够 → 只用缩放覆盖高度，平移只在竖直方向
|
||||
zoom = random.uniform(1.10, 1.25)
|
||||
d = f"max(t,0.001)/{max(duration,1)}"
|
||||
trajectory = random.choice(["center_zoom", "pan_up", "pan_down"])
|
||||
if trajectory == "center_zoom":
|
||||
x, y = f"(iw-{target_w})/2", f"(ih-{target_h})/2"
|
||||
elif trajectory == "pan_up":
|
||||
x, y = f"(iw-{target_w})/2", f"(ih-{target_h})*(1-{d})"
|
||||
else: # pan_down
|
||||
x, y = f"(iw-{target_w})/2", f"(ih-{target_h})*{d}"
|
||||
kb_type = trajectory
|
||||
else:
|
||||
# 横屏或小尺寸 → 仅缩放填充，不做平移（避免负值）
|
||||
zoom = max(target_w / max(src_w, 1), target_h / max(src_h, 1)) * 1.05
|
||||
x, y = f"(iw-{target_w})/2", f"(ih-{target_h})/2"
|
||||
kb_type = "center"
|
||||
|
||||
filter_str = (
|
||||
f"scale=iw*{zoom}:ih*{zoom}:flags=lanczos,"
|
||||
f"crop={target_w}:{target_h}:{x}:{y},"
|
||||
f"setsar=1,fps=30"
|
||||
)
|
||||
|
||||
cmd = [
|
||||
FFMPEG, "-y", "-i", input_path,
|
||||
"-vf", filter_str,
|
||||
"-c:v", "libx264", "-preset", "ultrafast", "-crf", "25",
|
||||
"-an", output_path,
|
||||
]
|
||||
subprocess.run(cmd, capture_output=True, timeout=300, check=True)
|
||||
logger.debug(f"Ken Burns [{kb_type} @ {zoom:.2f}x] → {output_path}")
|
||||
return output_path
|
||||
|
||||
|
||||
def compose_video(
|
||||
material_clips: list[str],
|
||||
audio_path: str,
|
||||
output_path: str = "final.mp4",
|
||||
resolution: str = DEFAULT_RESOLUTION,
|
||||
fps: int = DEFAULT_FPS,
|
||||
transitions: list[str] | None = None,
|
||||
subtitles_path: str | None = None,
|
||||
total_duration: float | None = None,
|
||||
ken_burns: bool = True,
|
||||
) -> str:
|
||||
"""
|
||||
视频合成: 素材片段 + 音频 + 字幕 → final.mp4
|
||||
|
||||
- material_clips: 素材视频文件路径列表
|
||||
- audio_path: 混音音频路径
|
||||
- output_path: 输出文件路径
|
||||
- transitions: 每段之间的过渡效果 (fade/cut/dissolve)，长度=clips数-1
|
||||
- subtitles_path: ASS/SRT 字幕文件路径 (可选)
|
||||
- total_duration: 视频总时长(秒)，用于裁剪素材匹配音频
|
||||
- ken_burns: 是否对每个素材施加随机缩放/平移动效（默认开启）
|
||||
|
||||
**素材时长自动匹配策略**:
|
||||
如果所有素材总时长 < 音频时长 → 末尾慢放/循环最后一段
|
||||
如果素材总时长 > 音频时长 → 裁剪素材到恰好匹配
|
||||
"""
|
||||
if not material_clips:
|
||||
raise ValueError("素材列表为空")
|
||||
if not os.path.exists(audio_path):
|
||||
raise FileNotFoundError(f"音频文件不存在: {audio_path}")
|
||||
|
||||
# 1. 可选择对素材施加 Ken Burns 动效（逐素材预处理）
|
||||
final_clips = material_clips
|
||||
if ken_burns:
|
||||
import tempfile, shutil
|
||||
ken_dir = tempfile.mkdtemp(prefix="ave_kenburns_")
|
||||
processed = []
|
||||
for idx, clip in enumerate(material_clips):
|
||||
out = os.path.join(ken_dir, f"kb_{idx:03d}.mp4")
|
||||
_apply_ken_burns(clip, out, resolution)
|
||||
processed.append(out)
|
||||
final_clips = processed
|
||||
logger.info(f"Ken Burns 已应用于 {len(final_clips)} 个素材")
|
||||
|
||||
# 2. 创建素材拼接文件列表
|
||||
concat_file = _create_concat_list(final_clips)
|
||||
|
||||
# 3. 获取音频总时长
|
||||
audio_duration = _get_media_duration(audio_path)
|
||||
|
||||
# 4. 构建 FFmpeg 命令
|
||||
cmd = [
|
||||
FFMPEG, "-y",
|
||||
"-f", "concat", "-safe", "0", "-i", concat_file,
|
||||
"-i", audio_path,
|
||||
]
|
||||
|
||||
filters = []
|
||||
|
||||
# 5. 如果素材总时长 > 音频时长，裁剪
|
||||
materials_duration = sum(_get_media_duration(c) for c in final_clips)
|
||||
output_duration = audio_duration
|
||||
|
||||
# 6. 视频流处理（Ken Burns 已预处理，这里只做字幕叠加）
|
||||
filters.append(f"[0:v]setpts=PTS-STARTPTS,fps={fps}[v]")
|
||||
filters.append("[1:a]volume=1.0[a]")
|
||||
|
||||
# 7. 字幕叠加
|
||||
if subtitles_path and os.path.exists(subtitles_path):
|
||||
filters.append(f"[v]subtitles={subtitles_path}[vo]")
|
||||
map_v = "[vo]"
|
||||
else:
|
||||
map_v = "[v]"
|
||||
|
||||
filter_complex = ";".join(filters)
|
||||
|
||||
cmd.extend(["-filter_complex", filter_complex])
|
||||
cmd.extend(["-map", map_v, "-map", "[a]"])
|
||||
|
||||
# 8. 时长匹配
|
||||
logger.info(f" 素材总时长: {materials_duration:.1f}s, 音频时长: {audio_duration:.1f}s")
|
||||
if materials_duration < audio_duration:
|
||||
logger.warning(f"素材不足，尾部会黑屏补齐")
|
||||
cmd.extend(["-shortest"])
|
||||
|
||||
# 9. 编码参数
|
||||
cmd.extend([
|
||||
"-c:v", "libx264",
|
||||
"-preset", "medium",
|
||||
"-b:v", DEFAULT_BITRATE,
|
||||
"-c:a", "aac",
|
||||
"-b:a", "192k",
|
||||
"-pix_fmt", "yuv420p",
|
||||
"-movflags", "+faststart",
|
||||
output_path,
|
||||
])
|
||||
|
||||
try:
|
||||
subprocess.run(cmd, capture_output=True, timeout=600)
|
||||
logger.info(f"视频合成完成: {output_path}")
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.error("渲染超时（>600s），建议分段渲染")
|
||||
raise
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.error(f"FFmpeg 执行失败: {e.stderr.decode(errors='replace')[:500]}")
|
||||
raise
|
||||
|
||||
# 清理临时文件
|
||||
if ken_burns:
|
||||
shutil.rmtree(ken_dir, ignore_errors=True)
|
||||
|
||||
return output_path
|
||||
|
||||
|
||||
def create_subtitles(
|
||||
segments: list[dict],
|
||||
output_path: str = "subtitles.ass",
|
||||
resolution: tuple = (1080, 1920),
|
||||
) -> str:
|
||||
"""
|
||||
从分段信息生成 ASS 格式字幕
|
||||
|
||||
segments 格式:
|
||||
[{"text": "...", "start_sec": 0.0, "end_sec": 8.0, "emotion": "..."}]
|
||||
"""
|
||||
width, height = resolution
|
||||
|
||||
# 字幕位置: 由下往上 20% (竖屏 1080x1920)
|
||||
margin_v = int(height * 0.2)
|
||||
font_size = max(48, int(height * 0.044)) # 1920 → ~84px
|
||||
|
||||
lines = [
|
||||
"[Script Info]",
|
||||
"ScriptType: v4.00+",
|
||||
f"PlayResX: {width}",
|
||||
f"PlayResY: {height}",
|
||||
"ScaledBorderAndShadow: yes",
|
||||
"",
|
||||
"[V4+ Styles]",
|
||||
"Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding",
|
||||
f"Style: Default,Noto Sans SC,{font_size},&H00FFFFFF,&H0000FFFF,&H00000000,&H80000000,-1,0,0,0,100,100,0,0,1,3,1,2,100,100,{margin_v},1",
|
||||
"",
|
||||
"[Events]",
|
||||
"Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text",
|
||||
]
|
||||
|
||||
for seg in segments:
|
||||
start = _sec_to_ass_time(seg.get("start_sec", 0))
|
||||
end = _sec_to_ass_time(seg.get("end_sec", 8))
|
||||
text = seg.get("text", "").replace("\n", "\\N")
|
||||
lines.append(f"Dialogue: 0,{start},{end},Default,,0,0,0,,{text}")
|
||||
|
||||
with open(output_path, "w", encoding="utf-8") as f:
|
||||
f.write("\n".join(lines))
|
||||
|
||||
logger.info(f"字幕生成: {output_path} ({len(segments)} 段)")
|
||||
return output_path
|
||||
|
||||
|
||||
def segment_render(
|
||||
material_clips: list[str],
|
||||
audio_path: str,
|
||||
output_dir: str = ".",
|
||||
segment_duration: int = SEGMENT_DURATION_MAX,
|
||||
subtitles_path: str | None = None,
|
||||
resolution: str = DEFAULT_RESOLUTION,
|
||||
) -> list[str]:
|
||||
"""
|
||||
分段渲染: 长视频自动分段 → 分别渲染 → 返回分段文件列表
|
||||
后续由 FFmpeg concat 拼接
|
||||
"""
|
||||
audio_duration = _get_media_duration(audio_path)
|
||||
if audio_duration <= segment_duration:
|
||||
# 无需分段
|
||||
out = os.path.join(output_dir, "segment_000.mp4")
|
||||
compose_video(material_clips, audio_path, out,
|
||||
subtitles_path=subtitles_path,
|
||||
resolution=resolution)
|
||||
return [out]
|
||||
|
||||
# 需要分段
|
||||
segments = []
|
||||
for i in range(0, int(audio_duration), segment_duration):
|
||||
start = i
|
||||
end = min(i + segment_duration, audio_duration)
|
||||
out = os.path.join(output_dir, f"segment_{i:03d}.mp4")
|
||||
_render_segment(material_clips, audio_path, start, end, out,
|
||||
subtitles_path=subtitles_path,
|
||||
resolution=resolution)
|
||||
segments.append(out)
|
||||
|
||||
logger.info(f"分段渲染完成: {len(segments)} 段")
|
||||
return segments
|
||||
|
||||
|
||||
def compose_with_anchors(
|
||||
material_clips: list[str],
|
||||
audio_path: str,
|
||||
output_path: str,
|
||||
silence_periods: list[dict],
|
||||
total_duration: float,
|
||||
subtitles_path: str | None = None,
|
||||
resolution: str = DEFAULT_RESOLUTION,
|
||||
) -> str:
|
||||
"""
|
||||
锚点驱动的画面切换合成
|
||||
|
||||
在每段静音处切换素材片段，添加交叉淡变过渡。
|
||||
素材按顺序循环使用，避免单一素材贯穿全片。
|
||||
|
||||
参数:
|
||||
material_clips: 素材视频列表
|
||||
audio_path: 混合音频路径
|
||||
silence_periods: [{"start": s, "end": e, "duration": d}, ...]
|
||||
total_duration: 音频总时长(秒)
|
||||
"""
|
||||
num_clips = len(material_clips)
|
||||
if num_clips == 0:
|
||||
raise ValueError("素材列表为空")
|
||||
if not silence_periods:
|
||||
logger.info("无锚点，回落普通合成")
|
||||
return compose_video(material_clips, audio_path, output_path,
|
||||
subtitles_path=subtitles_path, resolution=resolution)
|
||||
|
||||
import tempfile
|
||||
|
||||
# 1. 在静音中点附近切割素材段落
|
||||
transitions = [0.0] # 每个切换点的时间(秒)
|
||||
for s in silence_periods:
|
||||
mid = (s["start"] + s["end"]) / 2
|
||||
if mid > transitions[-1] + 0.5: # 至少间隔0.5s
|
||||
transitions.append(mid)
|
||||
transitions.append(total_duration)
|
||||
|
||||
# 2. 为每段分配素材（循环使用）
|
||||
seg_info = [] # [(clip_path, start, end), ...]
|
||||
clip_idx = 0
|
||||
for i in range(len(transitions) - 1):
|
||||
seg_start = transitions[i]
|
||||
seg_end = transitions[i + 1]
|
||||
dur = seg_end - seg_start
|
||||
if dur < 0.3:
|
||||
continue # 跳过过短段落
|
||||
seg_info.append((material_clips[clip_idx % num_clips], seg_start, seg_end))
|
||||
clip_idx += 1
|
||||
|
||||
logger.info(f"锚点分段: {len(seg_info)} 段, 使用 {min(clip_idx, num_clips)}/{num_clips} 个素材")
|
||||
|
||||
# 3. 渲染每段为独立视频
|
||||
temp_dir = tempfile.mkdtemp(prefix="ave_anchor_")
|
||||
seg_files = []
|
||||
for idx, (clip, seg_s, seg_e) in enumerate(seg_info):
|
||||
out = os.path.join(temp_dir, f"seg_{idx:03d}.mp4")
|
||||
dur = seg_e - seg_s
|
||||
# 从该素材中截取一段 (用素材的一部分，避免只用开头)
|
||||
clip_dur = _get_media_duration(clip)
|
||||
offset_in_clip = (seg_s * 15) % max(clip_dur, 1) # 取素材中不同位置
|
||||
offset_in_clip = min(offset_in_clip, max(clip_dur - dur, 0))
|
||||
|
||||
# 用 FFmpeg 截取该素材的一段
|
||||
subprocess.run([
|
||||
FFMPEG, "-y",
|
||||
"-ss", str(offset_in_clip),
|
||||
"-i", clip,
|
||||
"-t", str(dur),
|
||||
"-vf", f"scale={resolution.replace('x',':')},setsar=1,fps=30",
|
||||
"-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
out,
|
||||
], capture_output=True, timeout=120, check=True)
|
||||
seg_files.append(out)
|
||||
|
||||
# 4. 简单 concat 拼接 (含交叉淡变的 xfade 链 >10段容易 FFmpeg 崩溃)
|
||||
# 用 concat demuxer 直接拼接，素材切换本身已提供视觉变化
|
||||
concat_list = _create_concat_list(seg_files)
|
||||
|
||||
cmd = [
|
||||
FFMPEG, "-y",
|
||||
"-f", "concat", "-safe", "0", "-i", concat_list,
|
||||
"-i", audio_path,
|
||||
"-af", "afade=t=in:d=1,afade=t=out:st={}:d=1".format(max(0, total_duration - 1)),
|
||||
]
|
||||
|
||||
if subtitles_path and os.path.exists(subtitles_path):
|
||||
cmd += ["-vf", f"subtitles={subtitles_path}"]
|
||||
|
||||
cmd += ["-shortest", "-c:v", "libx264", "-preset", "medium", "-b:v", DEFAULT_BITRATE,
|
||||
"-c:a", "aac", "-b:a", "192k", "-pix_fmt", "yuv420p", output_path]
|
||||
|
||||
subprocess.run(cmd, capture_output=True, check=True, timeout=600)
|
||||
|
||||
# 清理
|
||||
import shutil
|
||||
shutil.rmtree(temp_dir, ignore_errors=True)
|
||||
|
||||
logger.info(f"锚点驱动合成完成: {output_path}")
|
||||
return output_path
|
||||
|
||||
|
||||
def concat_segments(segment_files: list[str], output_path: str = "final.mp4") -> str:
|
||||
"""拼接分段视频"""
|
||||
concat_file = _create_concat_list(segment_files)
|
||||
subprocess.run([
|
||||
FFMPEG, "-y",
|
||||
"-f", "concat", "-safe", "0", "-i", concat_file,
|
||||
"-c", "copy",
|
||||
output_path,
|
||||
], capture_output=True, check=True)
|
||||
logger.info(f"分段拼接完成: {output_path}")
|
||||
return output_path
|
||||
|
||||
|
||||
# ── 内部工具 ──────────────────────────────────────────────
|
||||
|
||||
def _create_concat_list(file_paths: list[str]) -> str:
|
||||
"""创建 FFmpeg concat 文件列表"""
|
||||
tmp = tempfile.NamedTemporaryFile(
|
||||
mode="w", suffix=".txt", delete=False, encoding="utf-8"
|
||||
)
|
||||
for fp in file_paths:
|
||||
abs_path = os.path.abspath(fp)
|
||||
tmp.write(f"file '{abs_path}'\n")
|
||||
tmp.close()
|
||||
return tmp.name
|
||||
|
||||
|
||||
def _get_media_duration(path: str) -> float:
|
||||
"""获取媒体文件时长(秒) —— 用 lib.ffmpeg（本机 ffprobe 实为 ffmpeg 符号链接）"""
|
||||
try:
|
||||
from lib.ffmpeg import probe_duration
|
||||
d = probe_duration(path)
|
||||
if d > 0:
|
||||
return d
|
||||
except Exception:
|
||||
pass
|
||||
logger.warning(f"无法获取时长，默认 30s: {path}")
|
||||
return 30.0
|
||||
|
||||
|
||||
def _sec_to_ass_time(seconds: float) -> str:
|
||||
"""秒 → ASS 时间格式 H:MM:SS.cc"""
|
||||
h = int(seconds // 3600)
|
||||
m = int((seconds % 3600) // 60)
|
||||
s = int(seconds % 60)
|
||||
cs = int((seconds - int(seconds)) * 100)
|
||||
return f"{h}:{m:02d}:{s:02d}.{cs:02d}"
|
||||
|
||||
|
||||
def _render_segment(
|
||||
material_clips: list[str],
|
||||
audio_path: str,
|
||||
start_sec: float,
|
||||
end_sec: float,
|
||||
output_path: str,
|
||||
subtitles_path: str | None = None,
|
||||
resolution: str = DEFAULT_RESOLUTION,
|
||||
):
|
||||
"""渲染单个分段（裁剪音频+对齐素材）"""
|
||||
duration = end_sec - start_sec
|
||||
# 用 concat 合并所有素材作为视频源
|
||||
concat_file = _create_concat_list(material_clips)
|
||||
|
||||
filters = [
|
||||
f"[0:v]setpts=PTS-STARTPTS,scale={resolution.replace('x',':')},fps=30[v]",
|
||||
f"[1:a]atrim=start={start_sec}:duration={duration},asetpts=PTS-STARTPTS[a]",
|
||||
]
|
||||
|
||||
if subtitles_path and os.path.exists(subtitles_path):
|
||||
filters.append(f"[v]subtitles={subtitles_path}[vo]")
|
||||
map_v = "[vo]"
|
||||
else:
|
||||
map_v = "[v]"
|
||||
|
||||
cmd = [
|
||||
FFMPEG, "-y",
|
||||
"-f", "concat", "-safe", "0", "-i", concat_file,
|
||||
"-i", audio_path,
|
||||
"-filter_complex", ";".join(filters),
|
||||
"-map", map_v, "-map", "[a]",
|
||||
"-shortest",
|
||||
"-c:v", "libx264", "-preset", "medium", "-b:v", DEFAULT_BITRATE,
|
||||
"-c:a", "aac", "-b:a", "192k",
|
||||
"-pix_fmt", "yuv420p",
|
||||
output_path,
|
||||
]
|
||||
subprocess.run(cmd, capture_output=True, check=True, timeout=600)
|
||||
@@ -0,0 +1,62 @@
|
||||
"""
|
||||
composer/frames.py — 视频抽帧工具（首尾帧衔接用）
|
||||
|
||||
用途：`chain_from_prev` 的镜头需要「上一镜的尾帧」作为生成参考，
|
||||
这里提供 ffmpeg 抽帧（零成本、本地处理）。
|
||||
"""
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_SCRIPT_ROOT = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPT_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPT_ROOT))
|
||||
|
||||
from lib.logger import get_logger
|
||||
|
||||
log = get_logger("composer.frames")
|
||||
|
||||
|
||||
def _ffmpeg() -> str:
|
||||
try:
|
||||
from lib.ffmpeg import FFMPEG
|
||||
return FFMPEG
|
||||
except Exception:
|
||||
return "ffmpeg"
|
||||
|
||||
|
||||
def extract_tail_frame(video_path: str, out_path: str, offset: float = 0.1) -> str:
|
||||
"""抽取视频**最后一帧**（默认取结束前 0.1 秒处，避开黑帧/抖动）
|
||||
|
||||
用于首尾帧衔接：本镜的起手画面 = 上一镜的收尾画面 → 视觉连续。
|
||||
"""
|
||||
src = Path(video_path)
|
||||
if not src.exists():
|
||||
raise FileNotFoundError(f"视频不存在: {video_path}")
|
||||
dst = Path(out_path)
|
||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||
r = subprocess.run(
|
||||
[_ffmpeg(), "-y", "-sseof", f"-{offset}", "-i", str(src),
|
||||
"-frames:v", "1", "-q:v", "2", str(dst)],
|
||||
capture_output=True, text=True, timeout=90)
|
||||
if r.returncode != 0 or not dst.exists() or dst.stat().st_size < 500:
|
||||
raise RuntimeError(f"抽尾帧失败: {r.stderr.strip()[-160:]}")
|
||||
log.debug(" 抽尾帧: %s → %s", src.name, dst.name)
|
||||
return str(dst)
|
||||
|
||||
|
||||
def extract_head_frame(video_path: str, out_path: str, offset: float = 0.0) -> str:
|
||||
"""抽取视频**第一帧**（用于把"上一镜的收尾"作为本镜的起手参考）"""
|
||||
src = Path(video_path)
|
||||
if not src.exists():
|
||||
raise FileNotFoundError(f"视频不存在: {video_path}")
|
||||
dst = Path(out_path)
|
||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||
cmd = [_ffmpeg(), "-y"]
|
||||
if offset:
|
||||
cmd += ["-ss", str(offset)]
|
||||
cmd += ["-i", str(src), "-frames:v", "1", "-q:v", "2", str(dst)]
|
||||
r = subprocess.run(cmd, capture_output=True, text=True, timeout=90)
|
||||
if r.returncode != 0 or not dst.exists() or dst.stat().st_size < 500:
|
||||
raise RuntimeError(f"抽首帧失败: {r.stderr.strip()[-160:]}")
|
||||
return str(dst)
|
||||
@@ -0,0 +1,457 @@
|
||||
"""
|
||||
AVE hybrid — 口播+卡点融合管线
|
||||
|
||||
功能:
|
||||
将口播的语音锚点驱动画面切换 + 卡点的BGM能量驱动变速结合起来。
|
||||
人声决定"什么时候切画面", BGM决定"画面怎么动"。
|
||||
|
||||
流程:
|
||||
1. 检测人声锚点 (静音段 → 段边界)
|
||||
2. 检测 BGM 节拍 + 能量特征
|
||||
3. 混音 (人声+BGM, BGM 音量避让)
|
||||
4. 为每段分配素材 + 能量驱动变速
|
||||
5. 渲染每段 + 字幕叠加
|
||||
6. 帧锁定拼接
|
||||
|
||||
用法:
|
||||
from composer.hybrid import compose_hybrid
|
||||
|
||||
compose_hybrid(
|
||||
voice_path="narration.wav",
|
||||
bgm_path="bgm.wav",
|
||||
output_path="hybrid.mp4",
|
||||
texts=["第一段文案", "第二段文案", ...],
|
||||
)
|
||||
"""
|
||||
import sys, os; sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from lib.logger import get_logger
|
||||
from lib.ffmpeg import FFMPEG # noqa: E402 (FFmpeg 绝对路径)
|
||||
|
||||
logger = get_logger("hybrid")
|
||||
|
||||
|
||||
def compose_hybrid(
|
||||
voice_path: str,
|
||||
bgm_path: str,
|
||||
output_path: str = "hybrid.mp4",
|
||||
material_clips: list[str] | None = None,
|
||||
resolution: str = "1080x1920",
|
||||
texts: list[str] | None = None,
|
||||
pexels_api_key: str = "",
|
||||
pexels_search: str = "",
|
||||
# 口播锚点参数
|
||||
min_silence_sec: float = 0.15,
|
||||
voice_volume: float = 1.0,
|
||||
bgm_volume: float = 0.35,
|
||||
# 卡点速度参数
|
||||
enable_speed_ramp: bool = True,
|
||||
base_speed: float = 1.0,
|
||||
high_speed: float = 1.5,
|
||||
low_speed: float = 0.7,
|
||||
# 节拍分组
|
||||
group_size: int = 4,
|
||||
) -> str:
|
||||
"""
|
||||
口播+卡点融合管线
|
||||
|
||||
参数:
|
||||
voice_path: 人声 WAV 路径
|
||||
bgm_path: BGM 音频路径
|
||||
output_path: 输出视频路径
|
||||
material_clips: 素材视频/图片列表 (可选)
|
||||
resolution: 输出分辨率
|
||||
texts: 每段文字 (用于字幕, 可选)
|
||||
pexels_api_key: Pexels API Key (素材不够时补充)
|
||||
pexels_search: Pexels 搜索关键词
|
||||
min_silence_sec: 最小静音时长 (秒), 用于分割段
|
||||
voice_volume: 人声音量 (默认 1.0)
|
||||
bgm_volume: BGM 音量 (默认 0.35)
|
||||
enable_speed_ramp: 是否启用能量驱动变速
|
||||
base_speed: 基础速度
|
||||
high_speed: 高能段速度
|
||||
low_speed: 低能段速度
|
||||
group_size: BGM 节拍分组数
|
||||
|
||||
返回: output_path
|
||||
"""
|
||||
logger.info("=== 口播+卡点融合模式 ===")
|
||||
logger.info(f"人声: {voice_path}")
|
||||
logger.info(f"BGM: {bgm_path}")
|
||||
|
||||
# ── 1. 检测人声锚点 ──
|
||||
from anchor_extractor.extractor import get_silence_periods
|
||||
silences = get_silence_periods(voice_path, min_silence_sec=min_silence_sec)
|
||||
|
||||
# 获取人声总时长
|
||||
voice_duration = _get_duration(voice_path)
|
||||
|
||||
# 从静音段反推活跃段 (说话段)
|
||||
voice_segments = _silences_to_segments(silences, voice_duration)
|
||||
logger.info(f" 锚点检测: {len(silences)} 个静音 → {len(voice_segments)} 个说话段")
|
||||
|
||||
# ── 2. 检测 BGM 节拍 (含能量) ──
|
||||
from composer.beat_sync import detect_beats
|
||||
beat_info = detect_beats(bgm_path, group_size=group_size, enable_energy=enable_speed_ramp)
|
||||
bgm_segments = beat_info["segments"]
|
||||
logger.info(f" BPM: {beat_info['bpm']:.0f}, {beat_info['beat_count']}拍, {len(bgm_segments)}组")
|
||||
|
||||
# ── 3. 混音 (人声+BGM, 避让) ──
|
||||
temp_dir = tempfile.mkdtemp(prefix="ave_hybrid_")
|
||||
mixed_audio = os.path.join(temp_dir, "mixed.wav")
|
||||
|
||||
# BGM 避让: 说话时压低BGM
|
||||
mixed_audio = _duck_and_mix(
|
||||
voice_path, bgm_path, voice_segments, mixed_audio,
|
||||
voice_volume=voice_volume, bgm_volume=bgm_volume,
|
||||
)
|
||||
|
||||
# ── 4. 准备素材 ──
|
||||
clips = list(material_clips) if material_clips else []
|
||||
if len(clips) < len(voice_segments) and pexels_api_key:
|
||||
from material_producer.pexels.search import search_videos
|
||||
needed = len(voice_segments) - len(clips)
|
||||
logger.info(f" 素材不足, 从 Pexels 搜索 {needed} 个...")
|
||||
kw = pexels_search or "background video"
|
||||
found = search_videos(kw, count=needed, api_key=pexels_api_key, orientation="portrait")
|
||||
for f in found:
|
||||
clips.append(f["path"])
|
||||
logger.info(f" 共 {len(clips)} 个素材")
|
||||
|
||||
if not clips:
|
||||
raise ValueError("无可用素材")
|
||||
|
||||
# ── 5. 为每段分配素材 + 计算速度 ──
|
||||
# 将 BGM 节拍能量映射到每个说话段上
|
||||
fps = 30
|
||||
seg_plans = [] # [(clip_path, adjusted_dur, speed), ...]
|
||||
|
||||
for idx, vseg in enumerate(voice_segments):
|
||||
clip = clips[idx % len(clips)]
|
||||
v_start = vseg["start"]
|
||||
v_end = vseg["end"]
|
||||
v_dur = v_end - v_start
|
||||
|
||||
if v_dur < 0.5:
|
||||
continue
|
||||
|
||||
# 在该时间段内计算 BGM 平均能量
|
||||
avg_energy = _average_energy_in_range(bgm_segments, v_start, v_end)
|
||||
speed = base_speed
|
||||
if enable_speed_ramp:
|
||||
if avg_energy > 0.6:
|
||||
speed = base_speed + (high_speed - base_speed) * min((avg_energy - 0.6) / 0.4, 1.0)
|
||||
elif avg_energy < 0.3:
|
||||
speed = low_speed + (base_speed - low_speed) * (avg_energy / 0.3)
|
||||
speed = max(0.5, min(2.0, speed))
|
||||
|
||||
# 帧锁定时长
|
||||
frames = round(v_dur * fps)
|
||||
dur_aligned = frames / fps
|
||||
|
||||
seg_plans.append({
|
||||
"clip": clip,
|
||||
"start": v_start,
|
||||
"end": v_end,
|
||||
"duration": dur_aligned,
|
||||
"speed": speed,
|
||||
"energy": avg_energy,
|
||||
})
|
||||
|
||||
logger.info(f" 分配 {len(seg_plans)} 段素材"
|
||||
+ (f", 变速范围 {low_speed:.1f}~{high_speed:.1f}x" if enable_speed_ramp else ""))
|
||||
|
||||
# ── 6. 渲染每段 ──
|
||||
seg_files = []
|
||||
for idx, plan in enumerate(seg_plans):
|
||||
clip = plan["clip"]
|
||||
dur = plan["duration"]
|
||||
speed = plan["speed"]
|
||||
mt = dur / max(speed, 0.01) # 素材需要的时间长度
|
||||
|
||||
seg_file = os.path.join(temp_dir, f"seg_{idx:03d}.mp4")
|
||||
|
||||
# 从素材中截取一段 (考虑变速)
|
||||
clip_dur = _get_duration(clip)
|
||||
offset = (plan["start"] * 17) % max(clip_dur - mt, 0.1) if clip_dur > mt else 0
|
||||
|
||||
raw_seg = os.path.join(temp_dir, f"seg_{idx:03d}_raw.mp4")
|
||||
subprocess.run([
|
||||
FFMPEG, "-y",
|
||||
"-ss", str(offset),
|
||||
"-i", clip,
|
||||
"-t", str(mt),
|
||||
"-vf", f"scale={resolution.replace('x',':')},setsar=1,fps={fps}",
|
||||
"-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-an", raw_seg,
|
||||
], capture_output=True, timeout=120, check=True)
|
||||
|
||||
if speed != 1.0 and 0.5 <= speed <= 2.0:
|
||||
subprocess.run([
|
||||
FFMPEG, "-y", "-i", raw_seg,
|
||||
"-filter_complex", f"setpts={1/speed}*PTS[v]",
|
||||
"-map", "[v]",
|
||||
"-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-pix_fmt", "yuv420p", seg_file,
|
||||
], capture_output=True, timeout=120, check=True)
|
||||
else:
|
||||
os.replace(raw_seg, seg_file)
|
||||
|
||||
seg_files.append(seg_file)
|
||||
|
||||
if not seg_files:
|
||||
raise RuntimeError("渲染失败: 无有效段")
|
||||
|
||||
# ── 7. 拼接 ──
|
||||
if len(seg_files) == 1:
|
||||
cmd = [FFMPEG, "-y", "-i", seg_files[0], "-i", mixed_audio,
|
||||
"-shortest", "-c:v", "copy", "-c:a", "aac", "-b:a", "192k", output_path]
|
||||
subprocess.run(cmd, capture_output=True, timeout=120, check=True)
|
||||
else:
|
||||
_concat_hybrid_segments(seg_files, mixed_audio, output_path, seg_plans, fps)
|
||||
|
||||
# ── 8. 字幕叠加 ──
|
||||
if texts and len(texts) > 0 and seg_plans:
|
||||
_add_text_subtitles(output_path, texts, seg_plans, resolution)
|
||||
|
||||
# ── 9. 清理 ──
|
||||
shutil.rmtree(temp_dir, ignore_errors=True)
|
||||
|
||||
final_size = os.path.getsize(output_path) / 1024 / 1024
|
||||
logger.info(f"✅ 融合完成: {output_path} ({final_size:.1f}MB, "
|
||||
f"{len(seg_files)} 段{', 能量变速' if enable_speed_ramp else ''})")
|
||||
return output_path
|
||||
|
||||
|
||||
# ── 内部辅助函数 ──────────────────────────────────────────
|
||||
|
||||
|
||||
def _silences_to_segments(silences: list[dict], total_duration: float) -> list[dict]:
|
||||
"""从静音段反推人声活跃段"""
|
||||
if not silences:
|
||||
return [{"start": 0.0, "end": total_duration}]
|
||||
|
||||
segments = []
|
||||
prev_end = 0.0
|
||||
|
||||
for s in silences:
|
||||
s_start = s.get("start", 0)
|
||||
s_end = s.get("end", 0)
|
||||
if s_start > prev_end + 0.1:
|
||||
segments.append({"start": prev_end, "end": s_start})
|
||||
prev_end = s_end
|
||||
|
||||
if prev_end < total_duration - 0.3:
|
||||
segments.append({"start": prev_end, "end": total_duration})
|
||||
|
||||
if not segments:
|
||||
return [{"start": 0.0, "end": total_duration}]
|
||||
|
||||
return segments
|
||||
|
||||
|
||||
def _average_energy_in_range(bgm_segments: list[dict], start: float, end: float) -> float:
|
||||
"""计算时间范围 [start, end] 内 BGM 的平均能量"""
|
||||
energies = []
|
||||
for seg in bgm_segments:
|
||||
seg_start = seg["start"]
|
||||
seg_end = seg["end"]
|
||||
# 检查重叠
|
||||
if seg_end < start or seg_start > end:
|
||||
continue
|
||||
energy = seg.get("energy", 0.5)
|
||||
overlap = min(seg_end, end) - max(seg_start, start)
|
||||
if overlap > 0:
|
||||
energies.append((energy, overlap))
|
||||
|
||||
if not energies:
|
||||
return 0.5
|
||||
|
||||
# 加权平均 (按重叠时长)
|
||||
total_weight = sum(w for _, w in energies) or 1.0
|
||||
avg = sum(e * w for e, w in energies) / total_weight
|
||||
return max(0.0, min(1.0, avg))
|
||||
|
||||
|
||||
def _duck_and_mix(
|
||||
voice_path: str,
|
||||
bgm_path: str,
|
||||
voice_segments: list[dict],
|
||||
output_path: str,
|
||||
voice_volume: float = 1.0,
|
||||
bgm_volume: float = 0.35,
|
||||
) -> str:
|
||||
"""
|
||||
BGM 音量避让混音:
|
||||
- 说话时: BGM 音量降到 bgm_volume * 0.4
|
||||
- 间隙时: BGM 恢复到 bgm_volume
|
||||
"""
|
||||
import numpy as np
|
||||
import soundfile as sf
|
||||
|
||||
# 加载音频
|
||||
voice, sr_v = sf.read(voice_path)
|
||||
bgm, sr_b = sf.read(bgm_path)
|
||||
|
||||
# 重采样 BGM 到相同 sr
|
||||
if sr_b != sr_v:
|
||||
import librosa
|
||||
bgm = librosa.resample(bgm, orig_sr=sr_b, target_sr=sr_v)
|
||||
|
||||
# 补齐 BGM 到人声长度
|
||||
if len(bgm) < len(voice):
|
||||
repeats = int(np.ceil(len(voice) / len(bgm)))
|
||||
bgm = np.tile(bgm, repeats)
|
||||
bgm = bgm[:len(voice)]
|
||||
|
||||
# 创建音量包络: 说话段压低 BGM
|
||||
bgm_envelope = np.ones(len(voice))
|
||||
for seg in voice_segments:
|
||||
s = int(seg["start"] * sr_v)
|
||||
e = int(seg["end"] * sr_v)
|
||||
bgm_envelope[s:e] = 0.4 # 说话时 BGM 降到 40%
|
||||
|
||||
# 平滑过渡 (0.05s fade)
|
||||
fade_samples = int(0.05 * sr_v)
|
||||
for i in range(1, len(bgm_envelope) - fade_samples * 2):
|
||||
if bgm_envelope[i] != bgm_envelope[i - 1]:
|
||||
# 在变化点做淡变
|
||||
start = max(0, i - fade_samples)
|
||||
end = min(len(bgm_envelope), i + fade_samples)
|
||||
for j in range(start, end):
|
||||
frac = (j - start) / max(end - start, 1)
|
||||
bgm_envelope[j] = bgm_envelope[start] + frac * (bgm_envelope[end - 1] - bgm_envelope[start])
|
||||
|
||||
# 混音
|
||||
voice_float = voice.astype(np.float32)
|
||||
bgm_float = bgm.astype(np.float32)
|
||||
|
||||
# 归一化到 [-1, 1]
|
||||
voice_float /= max(np.max(np.abs(voice_float)), 0.001)
|
||||
bgm_float /= max(np.max(np.abs(bgm_float)), 0.001)
|
||||
|
||||
mixed = voice_float * voice_volume + bgm_float * bgm_volume * bgm_envelope
|
||||
|
||||
# 限制防止爆音
|
||||
peak = np.max(np.abs(mixed))
|
||||
if peak > 0.99:
|
||||
mixed /= peak * 1.02
|
||||
|
||||
sf.write(output_path, mixed, sr_v)
|
||||
logger.debug(f" 混音完成: {output_path} ({sr_v}Hz, {len(voice)/sr_v:.1f}s)")
|
||||
return output_path
|
||||
|
||||
|
||||
def _concat_hybrid_segments(
|
||||
seg_files: list[str],
|
||||
mixed_audio: str,
|
||||
output_path: str,
|
||||
seg_plans: list[dict],
|
||||
fps: int = 30,
|
||||
) -> str:
|
||||
"""帧锁定拼接混合音频"""
|
||||
if len(seg_files) == 1:
|
||||
cmd = [FFMPEG, "-y", "-i", seg_files[0], "-i", mixed_audio,
|
||||
"-shortest", "-c:v", "copy", "-c:a", "aac", "-b:a", "192k", output_path]
|
||||
subprocess.run(cmd, capture_output=True, timeout=120, check=True)
|
||||
return output_path
|
||||
|
||||
# concat demuxer 拼接视频
|
||||
concat_file = tempfile.NamedTemporaryFile(suffix=".txt", delete=False, mode="w").name
|
||||
with open(concat_file, "w") as f:
|
||||
for sf in seg_files:
|
||||
f.write(f"file '{os.path.abspath(sf)}'\n")
|
||||
|
||||
video_only = output_path.replace(".mp4", "_novideo.mp4")
|
||||
cmd = [FFMPEG, "-y", "-f", "concat", "-safe", "0", "-i", concat_file,
|
||||
"-c:v", "libx264", "-preset", "fast", "-crf", "23", "-an", video_only]
|
||||
subprocess.run(cmd, capture_output=True, timeout=300, check=True)
|
||||
|
||||
# 叠加混合音频
|
||||
cmd2 = [FFMPEG, "-y", "-i", video_only, "-i", mixed_audio,
|
||||
"-shortest", "-c:v", "copy", "-c:a", "aac", "-b:a", "192k", output_path]
|
||||
subprocess.run(cmd2, capture_output=True, timeout=120, check=True)
|
||||
|
||||
os.unlink(concat_file)
|
||||
if os.path.exists(video_only):
|
||||
os.unlink(video_only)
|
||||
|
||||
logger.info(f" ✅ 拼接完成: {len(seg_files)} 段")
|
||||
return output_path
|
||||
|
||||
|
||||
def _add_text_subtitles(
|
||||
video_path: str,
|
||||
texts: list[str],
|
||||
seg_plans: list[dict],
|
||||
resolution: str,
|
||||
):
|
||||
"""为每段叠加文字字幕 (ASS 格式)"""
|
||||
width, height = (int(x) for x in resolution.split("x"))
|
||||
font_size = max(48, int(height * 0.05))
|
||||
margin_v = int(height * 0.3)
|
||||
|
||||
lines = [
|
||||
"[Script Info]", "ScriptType: v4.00+",
|
||||
f"PlayResX: {width}", f"PlayResY: {height}",
|
||||
"ScaledBorderAndShadow: yes", "",
|
||||
"[V4+ Styles]",
|
||||
"Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding",
|
||||
f"Style: Default,Noto Sans SC,{font_size},&H00FFFFFF,&H0000FFFF,&H00000000,&H80000000,-1,0,0,0,100,100,0,0,1,3,1,2,100,100,{margin_v},1",
|
||||
"", "[Events]",
|
||||
"Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text",
|
||||
]
|
||||
|
||||
for i, text in enumerate(texts):
|
||||
if i >= len(seg_plans):
|
||||
break
|
||||
start = seg_plans[i]["start"]
|
||||
dur = seg_plans[i]["duration"]
|
||||
end = start + dur
|
||||
|
||||
def _fmt_ts(sec: float) -> str:
|
||||
h = int(sec // 3600)
|
||||
m = int((sec % 3600) // 60)
|
||||
s = sec % 60
|
||||
return f"{h}:{m:02d}:{s:05.2f}"
|
||||
|
||||
lines.append(
|
||||
f"Dialogue: 0,{_fmt_ts(start)},{_fmt_ts(end)},Default,,0,0,0,,{text}"
|
||||
)
|
||||
|
||||
ass_path = video_path.replace(".mp4", "_hybrid_text.ass")
|
||||
with open(ass_path, "w", encoding="utf-8") as f:
|
||||
f.write("\n".join(lines))
|
||||
|
||||
tmp = video_path.replace(".mp4", "_tmp.mp4")
|
||||
subprocess.run([
|
||||
FFMPEG, "-y", "-i", video_path,
|
||||
"-vf", f"subtitles={ass_path}",
|
||||
"-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-c:a", "copy", "-pix_fmt", "yuv420p", tmp,
|
||||
], capture_output=True, timeout=120, check=True)
|
||||
os.replace(tmp, video_path)
|
||||
os.remove(ass_path)
|
||||
|
||||
logger.info(f" ✅ 字幕叠加: {len(texts)} 段")
|
||||
|
||||
|
||||
def _get_duration(path: str) -> float:
|
||||
"""获取媒体文件时长（秒）
|
||||
|
||||
用 lib.ffmpeg.probe_duration（ffmpeg -i 解析）——
|
||||
本机 ffprobe 实为 ffmpeg 的符号链接，不支持 -show_entries（会静默失败）
|
||||
"""
|
||||
try:
|
||||
from lib.ffmpeg import probe_duration
|
||||
d = probe_duration(path)
|
||||
return d if d > 0 else 30.0
|
||||
except Exception:
|
||||
return 30.0
|
||||
@@ -0,0 +1,508 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
AVE lipsync — Kling LipSync 集成
|
||||
|
||||
功能:
|
||||
Audio→Video: 输入视频 + 语音音频 → 输出口型对齐的视频
|
||||
Text→Video: 输入视频 + 文本 → 输出口型对齐的视频 (TTS自动)
|
||||
|
||||
API:
|
||||
fal.ai endpoint （Kling LipSync 托管在 fal.ai）
|
||||
成本: $0.014/5s ≈ ¥0.10/秒 ≈ ¥0.50/5秒镜头
|
||||
|
||||
用法:
|
||||
python lipsync.py --video clip.mp4 --audio voice.wav --output synced.mp4
|
||||
python lipsync.py --video clip.mp4 --text "你好世界" --output synced.mp4
|
||||
|
||||
集成:
|
||||
作为口播策略后处理钩子, 在 video_factory.py 中调用
|
||||
"""
|
||||
import sys
|
||||
import os
|
||||
import json
|
||||
import hashlib
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from lib.config import load_config
|
||||
from lib.cost_tracker import get_tracker
|
||||
from lib.logger import get_logger
|
||||
|
||||
logger = get_logger("lipsync")
|
||||
|
||||
# ── 缓存 ──
|
||||
CACHE_DIR = Path(os.environ.get("AVE_CACHE_DIR",
|
||||
str(Path.home() / "workbuddy-agent-os/agent-local/tools/ave/cache/lipsync")))
|
||||
CACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# ── fal.ai 端点 ──
|
||||
# Kling LipSync 通过 fal.ai 提供
|
||||
FAL_BASE = "https://fal.run"
|
||||
|
||||
ENDPOINTS = {
|
||||
"audio_to_video": f"{FAL_BASE}/fal-ai/kling-video/lipsync/audio-to-video",
|
||||
"text_to_video": f"{FAL_BASE}/fal-ai/kling-video/lipsync/text-to-video",
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 核心 API
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
def lipsync_audio_to_video(
|
||||
video_path: str,
|
||||
audio_path: str,
|
||||
output_path: str = "",
|
||||
force: bool = False,
|
||||
api_key: str = "",
|
||||
) -> str:
|
||||
"""
|
||||
Audio→Video 唇形同步
|
||||
|
||||
参数:
|
||||
video_path: 输入视频路径 (2-10s, 720p/1080p, ≤100MB)
|
||||
audio_path: 输入音频路径 (.mp3/.wav/.m4a, ≤5MB)
|
||||
output_path: 输出路径
|
||||
force: 强制重新生成
|
||||
api_key: fal.ai API Key
|
||||
|
||||
返回:
|
||||
本地输出视频路径
|
||||
"""
|
||||
if not output_path:
|
||||
joint = hashlib.md5(
|
||||
(video_path + audio_path).encode()
|
||||
).hexdigest()[:16]
|
||||
output_path = str(CACHE_DIR / f"lipsync_{joint}.mp4")
|
||||
|
||||
# 缓存
|
||||
if os.path.exists(output_path) and not force:
|
||||
logger.info(f"缓存命中: {output_path}")
|
||||
return output_path
|
||||
|
||||
cfg = load_config()
|
||||
api_key = api_key or cfg.get("fal", {}).get("api_key", "")
|
||||
if not api_key:
|
||||
raise ValueError("缺少 fal.ai API Key (config fal.api_key)")
|
||||
|
||||
logger.info(f"LipSync Audio→Video:")
|
||||
logger.info(f" video: {video_path}")
|
||||
logger.info(f" audio: {audio_path}")
|
||||
|
||||
# 上传文件或传URL (fal.ai 支持直接传base64或文件URL)
|
||||
# 方案: 使用 fal.ai 的 upload 端点
|
||||
video_url = _upload_to_fal(video_path, api_key)
|
||||
audio_url = _upload_to_fal(audio_path, api_key)
|
||||
|
||||
if not video_url or not audio_url:
|
||||
raise RuntimeError("上传文件到 fal.ai 失败")
|
||||
|
||||
logger.info(f" video_url: {video_url[:60]}...")
|
||||
logger.info(f" audio_url: {audio_url[:60]}...")
|
||||
|
||||
# 提交任务
|
||||
result = _submit_and_poll(
|
||||
endpoint=ENDPOINTS["audio_to_video"],
|
||||
payload={
|
||||
"video_url": video_url,
|
||||
"audio_url": audio_url,
|
||||
},
|
||||
api_key=api_key,
|
||||
timeout_sec=120,
|
||||
)
|
||||
|
||||
# 下载结果
|
||||
video_url_res = result.get("video", {}).get("url", "") or result.get("output", "")
|
||||
if not video_url_res:
|
||||
raise RuntimeError(f"fal.ai 返回无视频URL: {json.dumps(result, ensure_ascii=False)[:200]}")
|
||||
|
||||
logger.info(f" 下载结果中...")
|
||||
_download(video_url_res, output_path)
|
||||
|
||||
# 费用
|
||||
tracker = get_tracker()
|
||||
try:
|
||||
dur = _get_media_duration(output_path)
|
||||
except Exception:
|
||||
dur = 5 # 默认5秒
|
||||
tracker.log("KlingLipSync", duration=dur, note=f"audio-to-video ¥0.014×{dur}s")
|
||||
|
||||
sz_mb = os.path.getsize(output_path) / 1024 / 1024
|
||||
logger.info(f" ✅ LipSync完成: {output_path} ({sz_mb:.1f}MB, {dur:.1f}s)")
|
||||
return output_path
|
||||
|
||||
|
||||
def lipsync_text_to_video(
|
||||
video_path: str,
|
||||
text: str,
|
||||
voice_id: str = "default",
|
||||
output_path: str = "",
|
||||
force: bool = False,
|
||||
api_key: str = "",
|
||||
) -> str:
|
||||
"""
|
||||
Text→Video 唇形同步 (自动 TTS + 口型对齐, ≤120字)
|
||||
|
||||
参数:
|
||||
video_path: 输入视频路径
|
||||
text: 口播文本 (≤120字)
|
||||
voice_id: 音色ID
|
||||
output_path: 输出路径
|
||||
force: 强制重新生成
|
||||
|
||||
返回:
|
||||
本地输出视频路径
|
||||
"""
|
||||
if len(text) > 120:
|
||||
logger.warning(f"文本过长 ({len(text)}字), 截断至120字")
|
||||
text = text[:120]
|
||||
|
||||
if not output_path:
|
||||
joint = hashlib.md5(
|
||||
(video_path + text).encode()
|
||||
).hexdigest()[:16]
|
||||
output_path = str(CACHE_DIR / f"lipsync_txt_{joint}.mp4")
|
||||
|
||||
if os.path.exists(output_path) and not force:
|
||||
logger.info(f"缓存命中: {output_path}")
|
||||
return output_path
|
||||
|
||||
cfg = load_config()
|
||||
api_key = api_key or cfg.get("fal", {}).get("api_key", "")
|
||||
if not api_key:
|
||||
raise ValueError("缺少 fal.ai API Key")
|
||||
|
||||
logger.info(f"LipSync Text→Video: text={text[:40]}...")
|
||||
|
||||
video_url = _upload_to_fal(video_path, api_key)
|
||||
if not video_url:
|
||||
raise RuntimeError("上传视频到 fal.ai 失败")
|
||||
|
||||
result = _submit_and_poll(
|
||||
endpoint=ENDPOINTS["text_to_video"],
|
||||
payload={
|
||||
"video_url": video_url,
|
||||
"text": text,
|
||||
"voice_id": voice_id,
|
||||
},
|
||||
api_key=api_key,
|
||||
timeout_sec=180,
|
||||
)
|
||||
|
||||
video_url_res = result.get("video", {}).get("url", "") or result.get("output", "")
|
||||
if not video_url_res:
|
||||
raise RuntimeError(f"fal.ai 返回无视频URL")
|
||||
|
||||
_download(video_url_res, output_path)
|
||||
|
||||
sz_mb = os.path.getsize(output_path) / 1024 / 1024
|
||||
logger.info(f" ✅ LipSync Text→Video完成: {output_path} ({sz_mb:.1f}MB)")
|
||||
return output_path
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 内部工具
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
def _upload_to_fal(file_path: str, api_key: str) -> str:
|
||||
"""上传文件到 fal.ai, 返回可访问的 URL"""
|
||||
import httpx
|
||||
if not os.path.exists(file_path):
|
||||
raise FileNotFoundError(f"文件不存在: {file_path}")
|
||||
|
||||
with open(file_path, "rb") as f:
|
||||
resp = httpx.post(
|
||||
f"{FAL_BASE}/fal-ai/upload",
|
||||
headers={
|
||||
"Authorization": f"Key {api_key}",
|
||||
},
|
||||
files={"file": f},
|
||||
timeout=120,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
return data.get("url", "")
|
||||
|
||||
|
||||
def _submit_and_poll(
|
||||
endpoint: str,
|
||||
payload: dict,
|
||||
api_key: str,
|
||||
timeout_sec: int = 120,
|
||||
) -> dict:
|
||||
"""提交 fal.ai 任务并轮询, 返回结果"""
|
||||
import httpx
|
||||
headers = {
|
||||
"Authorization": f"Key {api_key}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
|
||||
# 提交
|
||||
resp = httpx.post(
|
||||
endpoint.replace("/run", ""), # 提交到非 /run 端点
|
||||
headers=headers,
|
||||
json=payload,
|
||||
timeout=30,
|
||||
)
|
||||
# fal.ai 的 submit API
|
||||
submit_url = endpoint
|
||||
resp = httpx.post(submit_url, headers=headers, json=payload, timeout=30)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
|
||||
# 检查是否直接返回结果（fal.ai 同步模式）
|
||||
if data.get("status") == "completed" or data.get("video"):
|
||||
return data
|
||||
|
||||
# 获取请求ID进行轮询
|
||||
request_id = data.get("request_id", "")
|
||||
if not request_id:
|
||||
# 有的接口直接返回结果
|
||||
if data.get("video", {}).get("url"):
|
||||
return data
|
||||
raise RuntimeError(f"fal.ai 无 request_id: {json.dumps(data, ensure_ascii=False)[:200]}")
|
||||
|
||||
status_url = f"{endpoint}/requests/{request_id}"
|
||||
logger.info(f" 轮询: {request_id}")
|
||||
|
||||
start = time.time()
|
||||
for i in range(60):
|
||||
# 指数退避
|
||||
if i < 5:
|
||||
time.sleep(3)
|
||||
elif i < 15:
|
||||
time.sleep(5)
|
||||
else:
|
||||
time.sleep(10)
|
||||
|
||||
elapsed = time.time() - start
|
||||
if elapsed > timeout_sec:
|
||||
raise TimeoutError(f"fal.ai 超时 ({timeout_sec}s)")
|
||||
|
||||
try:
|
||||
q = httpx.get(status_url, headers=headers, timeout=15)
|
||||
q.raise_for_status()
|
||||
status_data = q.json()
|
||||
status = status_data.get("status", "")
|
||||
if status == "COMPLETED":
|
||||
return status_data
|
||||
elif status in ("FAILED", "ERROR"):
|
||||
raise RuntimeError(f"fal.ai 失败: {json.dumps(status_data, ensure_ascii=False)[:200]}")
|
||||
logger.debug(f" [{i}] status={status}")
|
||||
except httpx.HTTPError as e:
|
||||
logger.debug(f" [{i}] 查询失败: {e}")
|
||||
time.sleep(5)
|
||||
|
||||
raise TimeoutError("fal.ai 轮询超时")
|
||||
|
||||
|
||||
def _download(url: str, path: str):
|
||||
"""下载文件"""
|
||||
import httpx
|
||||
os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
|
||||
with httpx.stream("GET", url, timeout=600) as r:
|
||||
r.raise_for_status()
|
||||
with open(path, "wb") as f:
|
||||
for chunk in r.iter_bytes(8192):
|
||||
f.write(chunk)
|
||||
|
||||
|
||||
def _get_media_duration(path: str) -> float:
|
||||
"""获取媒体时长 (秒) —— 用 lib.ffmpeg（本机 ffprobe 实为 ffmpeg 符号链接）"""
|
||||
from lib.ffmpeg import probe_duration
|
||||
return probe_duration(path)
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# CLI
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
def cli(args: list[str] | None = None):
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser(description="Kling LipSync 唇形同步")
|
||||
sub = parser.add_subparsers(dest="mode", required=True)
|
||||
|
||||
# audio → video
|
||||
p_av = sub.add_parser("audio-to-video", help="音频驱动唇形同步")
|
||||
p_av.add_argument("--video", required=True, help="输入视频")
|
||||
p_av.add_argument("--audio", required=True, help="输入音频")
|
||||
p_av.add_argument("--output", default="", help="输出路径")
|
||||
p_av.add_argument("--force", action="store_true", help="强制重新生成")
|
||||
|
||||
# text → video
|
||||
p_tv = sub.add_parser("text-to-video", help="文本驱动唇形同步")
|
||||
p_tv.add_argument("--video", required=True, help="输入视频")
|
||||
p_tv.add_argument("--text", required=True, help="口播文本 (≤120字)")
|
||||
p_tv.add_argument("--voice-id", default="default", help="音色ID")
|
||||
p_tv.add_argument("--output", default="", help="输出路径")
|
||||
p_tv.add_argument("--force", action="store_true", help="强制重新生成")
|
||||
|
||||
parsed = parser.parse_args(args)
|
||||
|
||||
if parsed.mode == "audio-to-video":
|
||||
out = lipsync_audio_to_video(
|
||||
video_path=parsed.video,
|
||||
audio_path=parsed.audio,
|
||||
output_path=parsed.output,
|
||||
force=parsed.force,
|
||||
)
|
||||
print(f"\n✅ LipSync 完成: {out}")
|
||||
|
||||
elif parsed.mode == "text-to-video":
|
||||
out = lipsync_text_to_video(
|
||||
video_path=parsed.video,
|
||||
text=parsed.text,
|
||||
voice_id=parsed.voice_id,
|
||||
output_path=parsed.output,
|
||||
force=parsed.force,
|
||||
)
|
||||
print(f"\n✅ LipSync 完成: {out}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
cli()
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# ★ 百炼 VideoRetalk（推荐实现，2026-10-03 实测探明）
|
||||
# 文档：https://help.aliyun.com/zh/model-studio/videoretalk-api
|
||||
# 优势：用现有百炼 key（无需 fal.ai）· ¥0.08/秒 · 1800 秒免费额度
|
||||
# 上传复用 AVE 已有的两步式（getPolicy → OSS 直传 → oss://）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
DASHSCOPE_BASE = "https://dashscope.aliyuncs.com"
|
||||
VR_SUBMIT_URL = f"{DASHSCOPE_BASE}/api/v1/services/aigc/image2video/video-synthesis"
|
||||
VR_TASK_URL = f"{DASHSCOPE_BASE}/api/v1/tasks/{{tid}}"
|
||||
VIDEORETALK_MODEL = "videoretalk"
|
||||
VR_PRICE_PER_SEC = 0.08 # 元/秒（官方后付费单价）
|
||||
VR_FREE_QUOTA_SEC = 1800 # 官方免费额度（秒）
|
||||
|
||||
|
||||
def vr_estimate_cost(duration_sec: float) -> float:
|
||||
"""对口型成本估算（元）—— 实际优先消耗 1800 秒免费额度"""
|
||||
return round(max(0.0, float(duration_sec)) * VR_PRICE_PER_SEC, 2)
|
||||
|
||||
|
||||
def _vr_upload(path: str, api_key: str) -> str:
|
||||
"""两步式上传 → oss:// URL（与 person_swap 同一套；oss:// 需配 Resolve header）"""
|
||||
import httpx
|
||||
p = Path(path)
|
||||
r = httpx.get(f"{DASHSCOPE_BASE}/api/v1/uploads?action=getPolicy&model={VIDEORETALK_MODEL}",
|
||||
headers={"Authorization": f"Bearer {api_key}"}, timeout=30)
|
||||
if r.status_code != 200:
|
||||
raise RuntimeError(f"获取上传凭证失败 HTTP {r.status_code}: {r.text[:200]}")
|
||||
d = r.json().get("data") or {}
|
||||
key = f"{d.get('upload_dir', '')}/{p.name}"
|
||||
form = {
|
||||
"OSSAccessKeyId": d.get("oss_access_key_id", ""),
|
||||
"policy": d.get("policy", ""),
|
||||
"signature": d.get("signature", ""),
|
||||
"key": key,
|
||||
"x-oss-object-acl": d.get("x_oss_object_acl", "private"),
|
||||
"x-oss-forbid-overwrite": "true",
|
||||
}
|
||||
with open(p, "rb") as f:
|
||||
r2 = httpx.post(d["upload_host"], data=form,
|
||||
files={"file": (p.name, f, "application/octet-stream")}, timeout=300,
|
||||
trust_env=False)
|
||||
if r2.status_code not in (200, 201, 204):
|
||||
raise RuntimeError(f"OSS 直传失败 HTTP {r2.status_code}: {r2.text[:200]}")
|
||||
return f"oss://{key}"
|
||||
|
||||
|
||||
def lipsync_videoretalk(video_path: str, audio_path: str, output_path: str,
|
||||
api_key: str = "", ref_image_path: str = "",
|
||||
video_extension: bool = False,
|
||||
timeout: int = 1800, progress_cb=None) -> str:
|
||||
"""人物视频 + 人声 → **口型匹配**的新视频（百炼 VideoRetalk）
|
||||
|
||||
· 成本 ¥0.08/秒（有 1800 秒免费额度）
|
||||
· 视频：正面近景、2~120s、mp4/mov、15~60fps、边长 640~2048
|
||||
· 音频：干净人声、2~120s（TTS 产物天然满足）
|
||||
· ref_image_path：角色参考图（可选，提升人脸匹配；我们正好有）
|
||||
|
||||
返回输出视频本地路径。
|
||||
"""
|
||||
import httpx
|
||||
api_key = api_key or (load_config().get("aliyun", {}) or {}).get("api_key", "")
|
||||
if not api_key:
|
||||
raise ValueError("VideoRetalk 需要百炼 API Key")
|
||||
_cb = progress_cb or (lambda p, n: None)
|
||||
|
||||
vdur = _get_media_duration(video_path)
|
||||
if vdur and not (2.0 < vdur < 120.0):
|
||||
raise ValueError(f"视频时长 {vdur:.1f}s 超出 VideoRetalk 要求（2~120 秒）")
|
||||
|
||||
_cb(5, "上传视频")
|
||||
vurl = _vr_upload(video_path, api_key)
|
||||
_cb(15, "上传音频")
|
||||
aurl = _vr_upload(audio_path, api_key)
|
||||
rurl = ""
|
||||
if ref_image_path and Path(ref_image_path).exists():
|
||||
try:
|
||||
_cb(20, "上传人脸参考图")
|
||||
rurl = _vr_upload(ref_image_path, api_key)
|
||||
except Exception as e:
|
||||
logger.warning("参考图上传失败（继续）：%s", e)
|
||||
|
||||
inp = {"video_url": vurl, "audio_url": aurl}
|
||||
if rurl:
|
||||
inp["ref_image_url"] = rurl
|
||||
payload = {
|
||||
"model": VIDEORETALK_MODEL,
|
||||
"input": inp,
|
||||
"parameters": {"video_extension": bool(video_extension)},
|
||||
}
|
||||
hdr = {
|
||||
"Authorization": f"Bearer {api_key}",
|
||||
"Content-Type": "application/json",
|
||||
"X-DashScope-Async": "enable",
|
||||
"X-DashScope-OssResourceResolve": "enable", # ★ 用 oss:// 时必须
|
||||
}
|
||||
_cb(25, "提交对口型任务")
|
||||
r = httpx.post(VR_SUBMIT_URL, headers=hdr, json=payload, timeout=60, trust_env=False)
|
||||
if r.status_code != 200:
|
||||
raise RuntimeError(f"提交失败 HTTP {r.status_code}: {r.text[:300]}")
|
||||
tid = (r.json().get("output") or {}).get("task_id", "")
|
||||
if not tid:
|
||||
raise RuntimeError(f"提交返回无 task_id: {r.text[:300]}")
|
||||
logger.info(" VideoRetalk 任务已提交: %s（预估 ¥%.2f）", tid, vr_estimate_cost(vdur))
|
||||
|
||||
# 轮询
|
||||
deadline = time.time() + timeout
|
||||
last = ""
|
||||
while time.time() < deadline:
|
||||
rr = httpx.get(VR_TASK_URL.format(tid=tid),
|
||||
headers={"Authorization": f"Bearer {api_key}"}, timeout=30, trust_env=False)
|
||||
out = (rr.json() or {}).get("output") or {}
|
||||
st = out.get("task_status", "")
|
||||
if st == "SUCCEEDED":
|
||||
url = out.get("video_url") or ""
|
||||
if not url:
|
||||
results = out.get("results") or []
|
||||
url = (results[0].get("video_url") or results[0].get("url")) if results else ""
|
||||
if not url:
|
||||
raise RuntimeError(f"成功但无视频 URL: {str(out)[:200]}")
|
||||
_cb(95, "下载结果")
|
||||
_download(url, output_path)
|
||||
_cb(100, "完成")
|
||||
logger.info(" ✅ 对口型完成: %s", Path(output_path).name)
|
||||
return output_path
|
||||
if st in ("FAILED", "CANCELED", "UNKNOWN"):
|
||||
raise RuntimeError(f"任务 {st}: {(out.get('message') or out.get('code') or '')[:200]}")
|
||||
if st != last:
|
||||
last = st
|
||||
logger.info(" [%s] 对口型任务状态: %s", tid[:8], st)
|
||||
_cb(30, f"处理中({st})")
|
||||
time.sleep(10)
|
||||
raise RuntimeError(f"对口型超时（>{timeout}s）")
|
||||
|
||||
|
||||
# ── CLI 扩展：--provider videoretalk ──
|
||||
def cli_videoretalk(video: str, audio: str, output: str,
|
||||
ref_image: str = "", video_extension: bool = False):
|
||||
def _p(p, n):
|
||||
print(f" {p}% {n}")
|
||||
out = lipsync_videoretalk(video, audio, output, ref_image_path=ref_image,
|
||||
video_extension=video_extension, progress_cb=_p)
|
||||
print(f"\n✅ 输出: {out}")
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,111 @@
|
||||
"""
|
||||
composer/qa.py — 出片质检辅助（接触印相表）
|
||||
|
||||
用途：出片后自动抽若干关键帧拼成一张「接触印相表」(contact sheet)，
|
||||
让人**一眼**核对：
|
||||
· 口型是否与台词对上（有台词的镜头）
|
||||
· 跨镜人物是否同一张脸（一致性）
|
||||
· 表情/构图/光线是否连贯
|
||||
|
||||
纯本地 ffmpeg 处理，零成本。
|
||||
"""
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_SCRIPT_ROOT = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPT_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPT_ROOT))
|
||||
|
||||
from lib.logger import get_logger
|
||||
|
||||
log = get_logger("composer.qa")
|
||||
|
||||
|
||||
def _ffmpeg() -> str:
|
||||
try:
|
||||
from lib.ffmpeg import FFMPEG
|
||||
return FFMPEG
|
||||
except Exception:
|
||||
return "ffmpeg"
|
||||
|
||||
|
||||
def _duration(video_path: str) -> float:
|
||||
try:
|
||||
from lib.ffmpeg import probe_duration
|
||||
return float(probe_duration(video_path) or 0)
|
||||
except Exception:
|
||||
return 0.0
|
||||
|
||||
|
||||
def make_contact_sheet(video_path: str, out_path: str,
|
||||
cols: int = 4, rows: int = 2, tile_w: int = 360) -> str:
|
||||
"""抽 `cols*rows` 帧均匀分布拼成一张图（默认 4x2 = 8 帧）
|
||||
|
||||
· 均匀采样（按视频时长算间隔），覆盖整个片段
|
||||
· 输出 jpg，宽 ≈ cols * tile_w
|
||||
"""
|
||||
src = Path(video_path)
|
||||
if not src.exists():
|
||||
raise FileNotFoundError(f"视频不存在: {video_path}")
|
||||
n = max(1, cols * rows)
|
||||
dur = _duration(str(src))
|
||||
interval = max(0.2, (dur / n)) if dur > 0 else 1.0
|
||||
dst = Path(out_path)
|
||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
vf = f"fps=1/{interval:.3f},scale={tile_w}:-2,tile={cols}x{rows}"
|
||||
r = subprocess.run(
|
||||
[_ffmpeg(), "-y", "-i", str(src), "-vf", vf, "-frames:v", "1", "-q:v", "3", str(dst)],
|
||||
capture_output=True, text=True, timeout=180)
|
||||
if r.returncode != 0 or not dst.exists() or dst.stat().st_size < 1000:
|
||||
raise RuntimeError(f"生成接触印相表失败: {r.stderr.strip()[-200:]}")
|
||||
log.info(" 🖼 质检图: %s（%d 帧 · %dx%d）", dst.name, n, cols, rows)
|
||||
return str(dst)
|
||||
|
||||
|
||||
def make_shot_qa(job_dir: str, sb=None) -> dict:
|
||||
"""为一次出片生成质检图（成片整体 + 每个镜头）
|
||||
|
||||
返回 {"final": path, "shots": [paths...]}；失败不抛（质检是辅助，不应阻塞交付）
|
||||
"""
|
||||
out: dict = {"final": "", "shots": []}
|
||||
d = Path(job_dir)
|
||||
if not d.exists():
|
||||
return out
|
||||
# ① 成片整体
|
||||
final = d / "final_muxed.mp4"
|
||||
if not final.exists():
|
||||
final = d / "final.mp4"
|
||||
if final.exists():
|
||||
try:
|
||||
out["final"] = make_contact_sheet(str(final), str(d / "qa" / "final_sheet.jpg"))
|
||||
except Exception as e:
|
||||
log.warning(" ⚠️ 成片质检图失败: %s", e)
|
||||
# ② 逐镜头（对口型后的片段优先，核对口型最准）
|
||||
qa_dir = d / "qa"
|
||||
shots = []
|
||||
try:
|
||||
cands = sorted(d.glob("lipsync_*.mp4")) or sorted(d.glob("shot_*.mp4"))
|
||||
for i, v in enumerate(cands, 1):
|
||||
try:
|
||||
shots.append(make_contact_sheet(
|
||||
str(v), str(qa_dir / f"{v.stem}_sheet.jpg"), cols=3, rows=2, tile_w=280))
|
||||
except Exception as e:
|
||||
log.debug(" 镜头质检图失败 %s: %s", v.name, e)
|
||||
except Exception as e:
|
||||
log.warning(" ⚠️ 逐镜质检图失败: %s", e)
|
||||
out["shots"] = shots
|
||||
return out
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import argparse
|
||||
ap = argparse.ArgumentParser(description="出片质检辅助（接触印相表）")
|
||||
ap.add_argument("job_dir", help="出片目录（含 final.mp4 或 final_muxed.mp4）")
|
||||
args = ap.parse_args()
|
||||
r = make_shot_qa(args.job_dir)
|
||||
print(f"成片质检图: {r['final'] or '（无）'}")
|
||||
print(f"镜头质检图: {len(r['shots'])} 张")
|
||||
for p in r["shots"][:8]:
|
||||
print(f" {p}")
|
||||
@@ -0,0 +1,801 @@
|
||||
"""
|
||||
composer/realism.py — 写实感增强（去 AI 感）
|
||||
|
||||
来源：外部方法论学习（详见 PLANS/REALISM_GUIDE.md + 09_ave/docs/realism/）
|
||||
作用：为提示词编译提供三件东西——
|
||||
① EMOTION_ACTIONS 情绪外化表（抽象情绪 → 具体身体动作）
|
||||
② REALISM_ANCHORS 写实锚点包（按拍摄风格给"反 AI 感"的具体锚点）
|
||||
③ check_conflicts 矛盾约束自检（写了互相抵消的组合）
|
||||
"""
|
||||
|
||||
import re
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# ① 情绪外化表
|
||||
# 原则：视频模型看不懂抽象形容词，必须外化为身体细节；
|
||||
# 每个镜头只写 1-2 个关键动作（多了稀释焦点）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
EMOTION_ACTIONS: dict = {
|
||||
"悲伤": "低头、肩膀微微颤抖、眼眶泛红，手指无意识地攥紧衣角",
|
||||
"难过": "低头、肩膀微微颤抖、眼眶泛红，手指无意识地攥紧衣角",
|
||||
"喜悦": "嘴角抑制不住地上扬、眉眼舒展，说话时眼睛弯起来",
|
||||
"开心": "嘴角抑制不住地上扬、眉眼舒展，说话时眼睛弯起来",
|
||||
"高兴": "嘴角抑制不住地上扬、眉眼舒展，说话时眼睛弯起来",
|
||||
"紧张": "手指无意识地攥紧衣角、眼神闪躲、呼吸略急",
|
||||
"焦虑": "频繁看表、手指轻敲、眼神游移、下意识抿嘴",
|
||||
"愤怒": "双拳紧握、下颌线绷紧、胸口起伏，从牙缝里挤话",
|
||||
"生气": "双拳紧握、下颌线绷紧、胸口起伏，从牙缝里挤话",
|
||||
"释然": "长舒一口气、紧绷的肩膀放松下来，浮起淡淡的微笑",
|
||||
"放松": "长舒一口气、紧绷的肩膀放松下来，浮起淡淡的微笑",
|
||||
"恐惧": "瞳孔放大、后退半步、手不自觉抓住身边物体、嘴唇发抖",
|
||||
"害怕": "瞳孔放大、后退半步、手不自觉抓住身边物体、嘴唇发抖",
|
||||
"疲惫": "揉太阳穴、眼皮下沉、长呼气、动作迟缓",
|
||||
"疲惫不堪": "揉太阳穴、眼皮下沉、长呼气、动作迟缓",
|
||||
"专注": "眉头微皱、眼神锁定一点、嘴唇抿紧、身体前倾",
|
||||
"认真": "眉头微皱、眼神锁定一点、嘴唇抿紧、身体前倾",
|
||||
"犹豫": "张嘴又闭上、视线游移、手悬在半空、欲言又止",
|
||||
"纠结": "张嘴又闭上、视线游移、手悬在半空、欲言又止",
|
||||
"惊喜": "眼睛瞬间睁大、嘴巴微张、身体向前倾，笑容缓慢绽放",
|
||||
"期待": "身体微微前倾、目光停在一点、呼吸放轻",
|
||||
"尴尬": "视线移向别处、嘴角抽动一下、手指摸了摸鼻尖",
|
||||
"无奈": "轻轻摇头、嘴角扯出一个笑、肩膀垮下来",
|
||||
"委屈": "嘴唇抿成一条线、眼眶发热、目光低垂看向地面",
|
||||
"好奇": "微微歪头、眼睛亮起来、视线跟着对象移动",
|
||||
"警觉": "下颌收紧、眼神快速扫向声源、身体下意识绷住",
|
||||
"疏离": "目光越过对方看向远处、表情平静没有起伏、动作缓慢",
|
||||
"冷漠": "目光越过对方看向远处、表情平静没有起伏、动作缓慢",
|
||||
}
|
||||
|
||||
|
||||
def externalize_emotion(sentiment: str) -> str:
|
||||
"""抽象情绪 → 具体身体动作；未命中则原样返回（调用方可自行处理）"""
|
||||
s = (sentiment or "").strip()
|
||||
if not s:
|
||||
return ""
|
||||
# 直接命中
|
||||
if s in EMOTION_ACTIONS:
|
||||
return EMOTION_ACTIONS[s]
|
||||
# 含"紧张/好奇"这类复合写法（用 / 、 分隔）→ 取第一个能命中的
|
||||
for sep in ("/", "、", ",", "，", " "):
|
||||
if sep in s:
|
||||
for piece in s.split(sep):
|
||||
piece = piece.strip()
|
||||
if piece in EMOTION_ACTIONS:
|
||||
return EMOTION_ACTIONS[piece]
|
||||
# 部分包含
|
||||
for k, v in EMOTION_ACTIONS.items():
|
||||
if k in s:
|
||||
return v
|
||||
return s
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# ② 写实锚点包（按拍摄风格）
|
||||
# 核心：用"设备型号 / 真实光源 / 微瑕疵 / 皮肤细节"替换抽象形容词
|
||||
# ════════════════════════════════════════════════════════════
|
||||
REALISM_ANCHORS: dict = {
|
||||
# 📱 自拍 Vlog（用户核心诉求）
|
||||
"selfie_pov": {
|
||||
"name": "手机自拍 · 第一人称",
|
||||
"device": "iPhone 前置摄像头拍摄，竖屏 9:16，一臂距离（鼻梁与脸颊有轻微广角透视变形）",
|
||||
"hand": "手持自拍，画面随呼吸与手臂疲劳轻微起伏；偶尔抬手微调构图",
|
||||
"light": "窗光从侧面打在脸上、室内顶灯偏暖；背景比人脸暗一档",
|
||||
"skin": "真实皮肤纹理、毛孔与细小绒毛可见、不过度磨皮、没有塑料质感",
|
||||
"flaws": ("前置镜头为**定焦**（不会有对焦搜索）；近距离透视变形；"
|
||||
"动态范围有限——窗外或灯下的高光容易过曝、暗部有可见噪点；"
|
||||
"头发有几缕没梳顺"),
|
||||
"motion": "说话时自然眨眼、吞咽、鼻翼微微起伏，视线偶尔离开镜头再回来",
|
||||
"sound": "房间轻微混响、远处环境声、衣服摩擦声（手机麦克风拾音，人声比正常对话稍远）",
|
||||
"_no": ("不要三脚架式的绝对稳定（自然的手持呼吸感要保留）、不要影棚布光、"
|
||||
"不要滤镜磨皮、不要对焦搜索（手机前置是定焦）"),
|
||||
},
|
||||
# 伪纪录片 · 真实自然（现有默认）
|
||||
"documentary_realism": {
|
||||
"device": "手持摄影机质感，轻微呼吸起伏",
|
||||
"hand": "镜头随拍摄者呼吸轻微移动，构图不是完美的",
|
||||
"light": "现场实际光源（窗光、路灯、顶灯），明暗过渡自然",
|
||||
"skin": "真实皮肤纹理、毛孔可见、不过度磨皮",
|
||||
"flaws": "轻微噪点、暗部细节不完全干净、对焦有呼吸感",
|
||||
"motion": "被拍摄者的动作不追求完美，有停顿与迟疑",
|
||||
"_no": "不要运镜特效痕迹、不要 AI 塑料质感、不要字幕水印",
|
||||
},
|
||||
# 电影感
|
||||
"cinematic": {
|
||||
"device": "35mm 电影镜头，浅景深",
|
||||
"hand": "三脚架或稳定器，运动平滑",
|
||||
"light": "动机光（画面内有光源依据）+ 逆光轮廓",
|
||||
"skin": "电影级肤色还原、保留皮肤质感",
|
||||
"flaws": "轻微的镜头光晕、真实的光斑",
|
||||
"motion": "调度从容，动作有起势与收势",
|
||||
"_no": "不要手持晃动、不要纪实粗糙感",
|
||||
},
|
||||
# 棚拍干净
|
||||
"studio_clean": {
|
||||
"device": "专业摄影机，稳定机位",
|
||||
"hand": "固定机位 / 滑轨",
|
||||
"light": "柔光箱主光 + 轮廓光",
|
||||
"skin": "干净但保留皮肤纹理（不要过度磨皮）",
|
||||
"flaws": "极少的噪点，干净背景",
|
||||
"motion": "表演可预期、动作清晰",
|
||||
"_no": "不要手持晃动、不要环境杂光",
|
||||
},
|
||||
# 手持 Vlog · 第一视角（边走边拍，镜头=拍摄者的眼睛）
|
||||
"vlog_handheld": {
|
||||
"device": "手机或轻便相机手持拍摄，竖屏 9:16",
|
||||
"hand": "第一视角手持，画面随走路上下起伏，转向时略有回摆",
|
||||
"light": "自然光为主（ daylight / 室内灯），明暗随走动变化",
|
||||
"skin": "真实皮肤纹理、毛孔可见、不过度磨皮",
|
||||
"flaws": "走动时的轻微抖动、暗部噪点、偶尔的手部入画",
|
||||
"motion": "被拍对象自然走动、走动时说话有些喘",
|
||||
"sound": "环境音随移动变化、脚步声、风噪",
|
||||
"_no": "不要稳定器般平滑、不要影棚布光",
|
||||
},
|
||||
# 访谈纪实 · 半身对镜
|
||||
"interview_doc": {
|
||||
"device": "中焦镜头，半身构图，略浅景深",
|
||||
"hand": "机位基本固定，允许极轻微的手持呼吸感",
|
||||
"light": "柔和侧光或窗户光，面部立体但不过曝",
|
||||
"skin": "真实皮肤纹理、皱纹与毛孔保留、不过度磨皮",
|
||||
"flaws": "背景略暗、轻微噪点、被访者偶尔的停顿与眼神移动",
|
||||
"motion": "被访者自然对镜说话，有停顿、思考、手势",
|
||||
"sound": "安静室内混响、衣料摩擦声",
|
||||
"_no": "不要花哨运镜、不要强风格化调色",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def get_anchors(shooting_style: str) -> dict:
|
||||
"""按拍摄风格取锚点包；未知名回退 documentary_realism"""
|
||||
s = (shooting_style or "").strip().lower()
|
||||
if s in REALISM_ANCHORS:
|
||||
return REALISM_ANCHORS[s]
|
||||
# 常见别名
|
||||
alias = {
|
||||
"selfie": "selfie_pov", "vlog": "selfie_pov", "自拍": "selfie_pov",
|
||||
"手机自拍": "selfie_pov", "第一人称": "selfie_pov",
|
||||
"documentary": "documentary_realism", "纪实": "documentary_realism",
|
||||
"film": "cinematic", "电影感": "cinematic", "movie": "cinematic",
|
||||
"studio": "studio_clean", "棚拍": "studio_clean",
|
||||
}
|
||||
for k, v in alias.items():
|
||||
if k in s:
|
||||
return REALISM_ANCHORS[v]
|
||||
return REALISM_ANCHORS["documentary_realism"]
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# ③ 矛盾约束自检（写了等于没写）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
CONFLICT_PAIRS: list = [
|
||||
(("画面稳定", "稳定机位", "三脚架"), ("手持", "晃动", "呼吸起伏"),
|
||||
"「画面稳定」与「手持晃动」冲突（稳定=三脚架语言）"),
|
||||
(("电影感", "cinematic"), ("纪实", "documentary", "手持"),
|
||||
"「电影感」与「纪实手持」冲突（风格相互拉扯）"),
|
||||
(("胶片", "film grain", "颗粒感"), ("4K", "超清", "ultra sharp", "高清锐利"),
|
||||
"「胶片感」与「超清」冲突（胶片感本身是模糊+颗粒）"),
|
||||
(("慢动作", "slow motion"), ("高速", "快切", "high speed"),
|
||||
"「慢动作」与「高速动作」冲突"),
|
||||
]
|
||||
|
||||
|
||||
def check_conflicts(text: str) -> list:
|
||||
"""扫描提示词里的矛盾组合，返回告警列表（空 = 无冲突）
|
||||
|
||||
⚠️ 先剔除负向语境：负向约束里写「不要画面稳定」是在**否定**该项，
|
||||
不能被当成"要求画面稳定"（否则与"手持晃动"误报冲突）。
|
||||
"""
|
||||
t = (text or "")
|
||||
# 剔除"不要X / 避免X / 禁止X / 无X"片段（X 最多 12 字），再做正向检测
|
||||
t_positive = re.sub(r"(不要|避免|禁止|无)[^、；。\n]{0,12}", " ", t)
|
||||
warns = []
|
||||
for group_a, group_b, msg in CONFLICT_PAIRS:
|
||||
hit_a = next((a for a in group_a if a.lower() in t_positive.lower()), "")
|
||||
hit_b = next((b for b in group_b if b.lower() in t_positive.lower()), "")
|
||||
if hit_a and hit_b:
|
||||
warns.append(f"{msg}（同时出现「{hit_a}」与「{hit_b}」）")
|
||||
return warns
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 必加约束（护栏）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
BASE_CONSTRAINTS = "不要水印、不要 Logo、不要字幕"
|
||||
FACE_CONSTRAINTS = "人物面部稳定不变形、动作连贯自然不僵硬、无穿模无卡顿无闪烁"
|
||||
ANTI_TWIN = "同一画面中不出现外形着装完全一致的人物（防止分身/双胞胎）"
|
||||
|
||||
|
||||
def compose_constraints(extra_no: str = "", with_anti_twin: bool = True) -> str:
|
||||
"""组装约束串（负向 + 正向要求，自动去重）
|
||||
|
||||
注意：两类语义不同——
|
||||
· 负向约束（"不要 X"）用「负向约束：」引导
|
||||
· 正向要求（"人物面部稳定不变形"这类陈述句）用「画面要求：」引导
|
||||
⚠️ 不能给正向要求硬加"不要"前缀（会把要求变成禁止，语义反转）
|
||||
"""
|
||||
negatives: list = [] # 形如 "不要水印"
|
||||
positives: list = [] # 形如 "人物面部稳定不变形"
|
||||
|
||||
for x in BASE_CONSTRAINTS.split("、"):
|
||||
x = x.strip()
|
||||
if x:
|
||||
negatives.append(x if x.startswith("不要") else "不要" + x)
|
||||
for x in FACE_CONSTRAINTS.split("、"):
|
||||
x = x.strip()
|
||||
if x:
|
||||
positives.append(x)
|
||||
if with_anti_twin:
|
||||
positives.append(ANTI_TWIN)
|
||||
if extra_no:
|
||||
e = extra_no.strip()
|
||||
for x in e.split("、"):
|
||||
x = x.strip()
|
||||
# ⚠️ 每项单独去前缀（锚点包每项通常自带"不要"，若只去整体前缀会出现"不要不要X"）
|
||||
while x.startswith("不要"):
|
||||
x = x[len("不要"):].strip()
|
||||
if x:
|
||||
negatives.append("不要" + x)
|
||||
|
||||
def _dedup(seq):
|
||||
"""**完全去重**（相同项只留一条）
|
||||
|
||||
⚠️ 我曾经想在这里做"子串合并"（"不要字幕水印" 吞掉 "不要字幕"），
|
||||
但实测证明那是**错的**：
|
||||
· `不要字幕` 与 `不要字幕水印` 是两条**不同**的约束（后者更具体），都该保留
|
||||
· 而且按子串匹配会把 `不要水印` 误判成 `字幕水印` 的子集 → 错误替换
|
||||
结论：**只做完全去重，不做子串合并。**
|
||||
"""
|
||||
seen, out = set(), []
|
||||
for it in seq:
|
||||
k = it.strip()
|
||||
if not k or k in seen:
|
||||
continue
|
||||
seen.add(k)
|
||||
out.append(it)
|
||||
return out
|
||||
|
||||
parts = []
|
||||
if negatives:
|
||||
parts.append("负向约束：" + "、".join(_dedup(negatives)))
|
||||
if positives:
|
||||
parts.append("画面要求：" + "、".join(_dedup(positives)))
|
||||
return "；".join(parts)
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# ④ 风格包（Style Pack）—— 把"导演提示词知识"变成可选参数
|
||||
# 用户 2026-10-03 要求：把学到的提示词技巧整理成参数选项，
|
||||
# 选一个风格 = 自动带上一整套「设备/光/机位/瑕疵/情绪/负面词」设定。
|
||||
# ════════════════════════════════════════════════════════════
|
||||
STYLE_PACKS: dict = {
|
||||
"selfie_pov": {
|
||||
"label": "📱 手机自拍 · 第一人称",
|
||||
"device": "iPhone 前置摄像头，手持自拍，手臂伸出一臂距离",
|
||||
"lighting": "室内自然光或窗光为主，光源不均匀，脸上有明显明暗过渡",
|
||||
"camera": "轻微手持晃动，构图略微歪斜不居中，对焦偶尔漂移",
|
||||
"flaws": "轻微手机噪点、真实肤质（可见毛孔与细纹）、有一缕碎发没整理好",
|
||||
"emotion": "外化", "realism_default": "mild",
|
||||
"negatives": ["影棚均匀布光", "三脚架固定机位", "过度磨皮", "商业修图感"],
|
||||
},
|
||||
"vlog_handheld": {
|
||||
"label": "🚶 手持 Vlog · 第一视角",
|
||||
"device": "手持相机或手机，走动跟拍，第一视角",
|
||||
"lighting": "户外环境光/店内混合光源，色温不统一",
|
||||
"camera": "明显手持晃动（模拟步行节奏），偶有快速转向，构图随意",
|
||||
"flaws": "轻微运动模糊、曝光偶尔跳动、背景杂乱真实",
|
||||
"emotion": "外化", "realism_default": "mild",
|
||||
"negatives": ["稳定器丝滑运镜", "影棚打光", "摆拍感"],
|
||||
},
|
||||
"documentary_realism": {
|
||||
"label": "🎥 伪纪录片 · 真实自然",
|
||||
"device": "手持摄像机或长焦，像随手记录",
|
||||
"lighting": "以现场自然光为主，不补光，允许暗部欠曝",
|
||||
"camera": "长镜头、轻微呼吸感晃动、不稳定构图",
|
||||
"flaws": "轻微胶片颗粒、真实环境杂乱、画面有生活痕迹",
|
||||
"emotion": "外化", "realism_default": "mild",
|
||||
"negatives": ["影棚布光", "摆拍摆姿", "商业感"],
|
||||
},
|
||||
"interview_doc": {
|
||||
"label": "📺 访谈纪实 · 半身对镜",
|
||||
"device": "固定机位，中焦镜头，半身构图",
|
||||
"lighting": "柔和主光 + 环境光，背景稍暗",
|
||||
"camera": "机位固定，极轻微晃动，人物可略偏离画面中心",
|
||||
"flaws": "背景真实但不抢眼、肤色自然不修饰",
|
||||
"emotion": "外化", "realism_default": "off",
|
||||
"negatives": ["夸张运镜", "过度磨皮"],
|
||||
},
|
||||
"cinematic": {
|
||||
"label": "🎞️ 电影感 · 考究构图",
|
||||
"device": "专业电影机，变焦或定焦，景深控制",
|
||||
"lighting": "有设计感的光比，侧逆光/轮廓光，暗部有层次",
|
||||
"camera": "稳定运镜或精心设计的推拉摇移，构图讲究",
|
||||
"flaws": "轻微胶片颗粒（不要数码锐化的塑料感）",
|
||||
"emotion": "外化", "realism_default": "off",
|
||||
"negatives": ["手机感", "随手拍", "画面杂乱"],
|
||||
},
|
||||
"studio_clean": {
|
||||
"label": "💡 商业棚拍 · 高清干净",
|
||||
"device": "专业摄影机，三脚架固定，中长焦",
|
||||
"lighting": "专业布光：主光 + 辅光 + 轮廓光，光比均匀",
|
||||
"camera": "稳定固定机位，构图工整居中",
|
||||
"flaws": "（此风格刻意干净 —— 但保留真实肤质，避免塑料感）",
|
||||
"emotion": "外化", "realism_default": "off",
|
||||
"negatives": ["画面杂乱", "手持晃动", "环境噪音"],
|
||||
},
|
||||
"anime_2d": {
|
||||
"label": "🎨 2D 动画 · 漫剧",
|
||||
"device": "动画分镜，2D 手绘或动漫渲染",
|
||||
"lighting": "动画式平光或风格化光影（非物理真实）",
|
||||
"camera": "动画运镜：推拉摇移、定格、分格",
|
||||
"flaws": "（动画无需真实瑕疵；改为保持线条稳定、色块不跳变）",
|
||||
"emotion": "夸张外化（动漫式表情与动作幅度）", "realism_default": "off",
|
||||
"negatives": ["写实人脸照片感", "真实摄影质感", "胶片颗粒", "真人实拍"],
|
||||
},
|
||||
"anime_3d": {
|
||||
"label": "🧸 3D 动画 · 卡通渲染",
|
||||
"device": "3D 渲染，卡通/风格化着色",
|
||||
"lighting": "风格化打光，柔和阴影或卡通描边",
|
||||
"camera": "3D 运镜（环绕/位移/景深）",
|
||||
"flaws": "（保持模型一致，避免穿模与闪烁）",
|
||||
"emotion": "夸张外化", "realism_default": "off",
|
||||
"negatives": ["写实照片", "真人实拍", "皮肤毛孔", "胶片噪点"],
|
||||
},
|
||||
"product_ad": {
|
||||
"label": "🧊 产品广告 · 精致特写",
|
||||
"device": "微距或长焦，产品特写，浅景深",
|
||||
"lighting": "精细打光：轮廓光 + 反光板，突出材质",
|
||||
"camera": "稳定运镜（滑轨/环绕），构图讲究",
|
||||
"flaws": "材质细节清晰（不是缩小版瑕疵）",
|
||||
"emotion": "（产品无情绪；如有模特出镜则外化）", "realism_default": "off",
|
||||
"negatives": ["画面杂乱", "廉价感", "低清晰度"],
|
||||
},
|
||||
}
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# ⑤ 反真实词库（★ 研究核心发现）
|
||||
# "AI 感"的头号来源：提示词里堆砌「4K/8K/超清/无瑕/大师作品」——
|
||||
# 模型会理解成"要无瑕"，于是过度磨皮、抹平纹理、统一光影 → 塑料感。
|
||||
# 真实感增强开启时，自动剔除这些词。
|
||||
# ════════════════════════════════════════════════════════════
|
||||
ANTI_REALISM_WORDS: list = [
|
||||
"4k", "8k", "4K", "8K", "超清", "超高清", "高清", "极致画质", "顶级画质",
|
||||
"电影级", "电影感构图", "大片感", "杂志级",
|
||||
"无瑕", "无瑕疵", "完美", "大师作品", "大师级", "杰作",
|
||||
"masterpiece", "best quality", "ultra detailed", "ultra realistic",
|
||||
"hyperrealistic", "photorealistic", "8k uhd", "highly detailed",
|
||||
"过度锐化", "塑料感", "CG感", "3D渲染感",
|
||||
]
|
||||
|
||||
# 反真实词在"写实类"风格下才剔除（动画类风格里这些词无害甚至有用）
|
||||
_REALISM_STYLES = {"selfie_pov", "vlog_handheld", "documentary_realism",
|
||||
"interview_doc", "cinematic", "studio_clean"}
|
||||
|
||||
|
||||
def strip_anti_realism(text: str) -> tuple:
|
||||
"""剔除会导致「塑料感」的反真实词（写实风格下）
|
||||
|
||||
返回 (清洗后文本, 被剔除的词列表)
|
||||
"""
|
||||
if not text:
|
||||
return "", []
|
||||
out = text
|
||||
removed = []
|
||||
for w in ANTI_REALISM_WORDS:
|
||||
if w in out:
|
||||
out = out.replace(w, "")
|
||||
removed.append(w)
|
||||
# 清理残留下的多余标点/空格
|
||||
out = re.sub(r"[、,，]\s*(?=[、,，])", "", out)
|
||||
out = re.sub(r"\s{2,}", " ", out).strip(" 、,，")
|
||||
return out, removed
|
||||
|
||||
|
||||
def apply_style_pack(prompt: str, shooting_style: str,
|
||||
realism_boost: str = "mild") -> tuple:
|
||||
"""把「风格包」应用到提示词上（去 AI 感的核心动作）
|
||||
|
||||
realism_boost: off（不动）/ mild（加瑕疵锚点）/ strong（瑕疵 + 剔除反真实词）
|
||||
|
||||
返回 (增强后的提示词, 说明列表)
|
||||
"""
|
||||
notes: list = []
|
||||
if not prompt:
|
||||
return prompt, notes
|
||||
pack = STYLE_PACKS.get(shooting_style) or {}
|
||||
boost = (realism_boost or "off").lower()
|
||||
out = prompt
|
||||
|
||||
if boost == "off" or not pack:
|
||||
return out, notes
|
||||
|
||||
# ① strong：剔除反真实词（研究里最大的坑）
|
||||
if boost == "strong" and shooting_style in _REALISM_STYLES:
|
||||
out, removed = strip_anti_realism(out)
|
||||
if removed:
|
||||
notes.append(f"已剔除反真实词：{'、'.join(removed[:6])}")
|
||||
|
||||
# ② 追加瑕疵锚点（如果提示词里还没有类似描述）
|
||||
flaws = pack.get("flaws") or ""
|
||||
if flaws and not any(k in out for k in ("噪点", "颗粒", "毛孔", "瑕疵", "纹理")):
|
||||
out = out.rstrip("。. ") + f"。{flaws}。"
|
||||
notes.append("已追加真实质感锚点")
|
||||
|
||||
# ③ strong：额外补一句"反完美"总纲
|
||||
if boost == "strong":
|
||||
tail = "真实拍摄的不完美感：光线不均匀、构图不刻意、画面有生活痕迹"
|
||||
if "不完美" not in out and "不均匀" not in out:
|
||||
out = out.rstrip("。. ") + f"。{tail}。"
|
||||
notes.append("已追加反完美总纲")
|
||||
|
||||
return out, notes
|
||||
|
||||
|
||||
def style_pack_options() -> dict:
|
||||
"""供前端生成「拍摄风格」下拉（含中文标签）"""
|
||||
return {k: v.get("label", k) for k, v in STYLE_PACKS.items()}
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# ★ 去 AI 感扩展库（2026-10-04 研究落地）
|
||||
# 来源：zhouwei713/seedance-prompt 的 anti-ai-checklist.md
|
||||
# （原文存 09_ave/docs/realism/anti-ai-checklist.md）
|
||||
#
|
||||
# 核心洞察：**"AI 感"主要来自模型的默认审美**（电影化运镜/稳定画面/
|
||||
# 商业调色/完美构图/磨皮皮肤）—— 必须用负面约束显式关闭，
|
||||
# 并主动注入「非完美事件」才像真实拍摄。
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
# ① AI 默认行为 → 负面约束句（10 项，含"造成的问题"）
|
||||
AI_DEFAULT_NEGATIVES = [
|
||||
("电影化平滑运镜（推轨/摇臂感）", "没有电影化的摄像机移动"),
|
||||
("画面稳定如三脚架", "不要三脚架式的绝对稳定（自然手持呼吸感是允许的）"),
|
||||
("现代商业调色（青橙色调、高对比）", "没有现代色彩分级"),
|
||||
("完美居中构图", "不完美的构图，构图偏离中心"),
|
||||
("场景自动加商店/咖啡馆/人群/广告牌", "没有商店、广告、咖啡馆、人群或商业活动"),
|
||||
("自动配背景音乐", "没有音乐。没有音效设计。没有旁白"),
|
||||
("表演感强、直视镜头摆姿势", "真实的即兴行为，无剧本的日常生活片段感"),
|
||||
("皮肤过度平滑/磨皮感", "逼真的皮肤纹理"),
|
||||
("角色跨镜头漂移", "在整个视频中保持一致的身份、服装、发型和外貌"),
|
||||
("灯光过于均匀完美", "自然光，在阳光和阴影间移动时的曝光波动"),
|
||||
]
|
||||
|
||||
# ② 非完美事件库（21 个 · 四类）—— 每镜头选一个，全片不重复
|
||||
IMPERFECT_EVENTS = {
|
||||
"photographer": [
|
||||
"自动对焦在主体和前景/背景之间反复搜索",
|
||||
"构图偏离中心（操作者试图跟上主体）",
|
||||
"摄像机稍晚才捕捉到关键瞬间（人已转身才对准）",
|
||||
"变焦生硬，推过了又拉回来",
|
||||
"走动跟拍时的强烈晃动和运动模糊",
|
||||
"手指或背带短暂遮挡镜头一角",
|
||||
"镜头努力对焦但短暂失败，画面呼吸",
|
||||
],
|
||||
"environment": [
|
||||
"一阵微风吹动散落的发丝或衣角",
|
||||
"云影掠过，曝光随之波动",
|
||||
"小动物（猫/狗/鸽子）闯入画面，主体自然回应",
|
||||
"场外有人打招呼，主体转头回应",
|
||||
"阳光透过树叶在脸上闪烁明灭",
|
||||
"路人从镜头前无意穿过",
|
||||
],
|
||||
"subject": [
|
||||
"注意到镜头，露出小而真诚的微笑后移开视线",
|
||||
"说话被环境音打断，笑场",
|
||||
"整理头发、拨发到耳后等无意识小动作",
|
||||
"动作做到一半改变主意（伸手拿杯子又先擦了下桌子）",
|
||||
],
|
||||
"ending": [
|
||||
"录制在中途突然切到黑屏，仿佛摄像机关闭",
|
||||
"主体走出画面，镜头停留在空景两秒",
|
||||
"镜头垂下拍到地面/脚，然后切断",
|
||||
"有人喊拍摄者的名字，镜头晃动着转向声源时切断",
|
||||
],
|
||||
}
|
||||
|
||||
# ③ 特殊字符规范（信息类型标注）—— 来自 07-特殊字符规范.md
|
||||
SYMBOL_SPEC = {
|
||||
"台词": "{你好，世界}（外语需标语种：用日语说道{こんにちは}）",
|
||||
"音效": "<远处传来狗叫声>",
|
||||
"音乐": "（背景中播放着快节奏的摇滚乐）",
|
||||
"字幕": "【第一章：启程】",
|
||||
}
|
||||
|
||||
|
||||
def ai_negative_block(hard: bool = True, extra: list = ()) -> str:
|
||||
"""组装「关闭 AI 默认审美」负面约束段
|
||||
|
||||
hard=False 时只取前 6 条（精简版，用于短提示词）
|
||||
"""
|
||||
items = [neg for _, neg in (AI_DEFAULT_NEGATIVES if hard else AI_DEFAULT_NEGATIVES[:6])]
|
||||
items.extend([str(x) for x in (extra or []) if x])
|
||||
return "【负面约束（关闭 AI 默认审美）】" + "；".join(items) + "。"
|
||||
|
||||
|
||||
def pick_imperfect_events(n_shots: int, categories: tuple = None) -> list:
|
||||
"""为 N 个镜头分配「非完美事件」（不重复；最后一条自动用收束方式）"""
|
||||
import random
|
||||
cats = list(categories or ("photographer", "environment", "subject"))
|
||||
pool = []
|
||||
for c in cats:
|
||||
pool.extend(IMPERFECT_EVENTS.get(c, []))
|
||||
random.shuffle(pool)
|
||||
out = []
|
||||
for i in range(max(0, n_shots)):
|
||||
if i == n_shots - 1 and n_shots > 1:
|
||||
out.append(random.choice(IMPERFECT_EVENTS["ending"]))
|
||||
else:
|
||||
out.append(pool[i % len(pool)] if pool else "")
|
||||
return out
|
||||
|
||||
|
||||
def imperfect_events_prompt() -> str:
|
||||
"""给导演的「非完美事件」纪律（逐镜头分配、不重复、结尾收束、2 秒节奏）"""
|
||||
return (
|
||||
"【真实感纪律：非完美事件（每个镜头分配一个，全片不重复）】\n"
|
||||
"真实拍摄必然有瑕疵。请为每个镜头挑 1 个（**一镜一个，不要堆**）：\n"
|
||||
"· 摄影师失误：" + "；".join(IMPERFECT_EVENTS["photographer"][:4]) + "\n"
|
||||
"· 环境介入：" + "；".join(IMPERFECT_EVENTS["environment"][:4]) + "\n"
|
||||
"· 主体自然行为：" + "；".join(IMPERFECT_EVENTS["subject"][:3]) + "\n"
|
||||
"· **结尾必须用一种收束方式**：" + "；".join(IMPERFECT_EVENTS["ending"]) + "\n"
|
||||
"另外：镜头约 2 秒一个（超过 3 秒会滑向「精心拍摄」的假感）；"
|
||||
"除非明确设定为手机慢动作，否则不要写慢动作。"
|
||||
)
|
||||
|
||||
|
||||
def check_abstract_words(text: str) -> list:
|
||||
"""自检：扫出会引发 AI 感的抽象词（返回命中列表）"""
|
||||
BAD = ["美丽", "唯美", "高级感", "大片感", "史诗", "震撼", "精致", "梦幻", "氛围感"]
|
||||
return [w for w in BAD if w in str(text or "")]
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# ★ 设备美学缺陷参数包（9 个）—— 2026-10-04 研究落地
|
||||
# 来源：camera-aesthetics.md（原文存 docs/realism/）
|
||||
#
|
||||
# 核心原则：**想要某种年代质感，就完整描述那个设备的物理缺陷，缺陷即真实。**
|
||||
# 与 STYLE_PACKS 的关系：
|
||||
# · STYLE_PACKS = 抽象风格取向（自拍/纪录片/电影感…）
|
||||
# · DEVICE_PACKS = **具体设备的物理缺陷**（DV/VHS/胶片/手机/GoPro…）
|
||||
# 两者互补，可叠加。
|
||||
# ════════════════════════════════════════════════════════════
|
||||
DEVICE_PACKS: dict = {
|
||||
"dv_2000s": {
|
||||
"name": "2000年代初消费级DV摄像机",
|
||||
"fit": "家庭录像、朋友随拍、千禧年怀旧、韩日老社区题材",
|
||||
"visual": ["中等数字压缩伪影", "褪色的色彩，柔和的对比度", "轻微的传感器噪点",
|
||||
"轻微的滚动快门", "480p 到轻微高清之间的分辨率感", "高光容易过曝，暗部发灰"],
|
||||
"operation": ["强烈的手持抖动，不完美的构图", "频繁的自动对焦搜索",
|
||||
"镜头呼吸（对焦时视角轻微变化）", "在阳光和阴影间移动时的曝光波动",
|
||||
"偶尔的运动模糊", "变焦生硬，速度不均匀", "录制中途突然开始或结束"],
|
||||
"negative": "不要三脚架式的绝对稳定。不要电影化的摄像机移动。不要现代色彩分级。",
|
||||
},
|
||||
"vhs_8090s": {
|
||||
"name": "80至90年代VHS家用摄像机",
|
||||
"fit": "童年录像、生日聚会、更旧的怀旧感、复古恐怖",
|
||||
"visual": ["明显的磁带噪点和扫描线", "色彩溢出（红色尤其容易晕开）",
|
||||
"底部时间码或日期戳", "画面边缘轻微的几何畸变",
|
||||
"低分辨率，细节模糊", "磁带损伤造成的偶尔画面撕裂或跳帧"],
|
||||
"operation": ["沉重机身带来的缓慢笨重移动", "手动变焦的顿挫感",
|
||||
"白平衡偏移（室内偏橙，室外偏蓝）", "逆光时人物完全剪影化"],
|
||||
"negative": "不要数字锐利感。不要三脚架式的绝对稳定。不要高动态范围。",
|
||||
},
|
||||
"film_8mm": {
|
||||
"name": "8mm/Super 8 胶片",
|
||||
"fit": "更久远的家庭记忆、梦境感、婚礼老片、艺术短片",
|
||||
"visual": ["明显的胶片颗粒", "温暖偏黄或偏品红的色彩偏移", "画面边缘的暗角",
|
||||
"灰尘、毛发、划痕偶尔闪过", "18fps 带来的轻微卡顿感",
|
||||
"过曝的高光呈现柔和的光晕"],
|
||||
"operation": ["每段镜头很短（胶片昂贵，3到5秒就切）", "硬切，没有转场",
|
||||
"曝光靠估计，时对时错", "无声（胶片本身不录音）或后期配的环境音"],
|
||||
"negative": "没有同期声对白。没有平滑运镜。没有数字清晰度。",
|
||||
},
|
||||
"smartphone_vertical": {
|
||||
"name": "智能手机竖屏（2015年后）",
|
||||
"fit": "Vlog、社交媒体感、街头偶遇、当代日常",
|
||||
"visual": ["9:16 竖构图", "过度锐化和 HDR 处理感",
|
||||
"电子防抖造成的果冻效应", "暗光下的涂抹感降噪",
|
||||
"前置镜头的轻微广角畸变"],
|
||||
"operation": ["单手持机的轻微倾斜", "走路时的上下浮动（电子防抖压制后仍残留）",
|
||||
"对着镜头说话的自拍视角切换", "手指偶尔擦过镜头边缘",
|
||||
"突然抬起或放下手机的开始与结束"],
|
||||
"negative": "没有横屏。没有三脚架稳定。没有专业布光。",
|
||||
},
|
||||
"gopro": {
|
||||
"name": "GoPro / 运动相机",
|
||||
"fit": "第一人称视角、运动、旅行、冒险感",
|
||||
"visual": ["超广角鱼眼畸变，边缘拉伸", "高饱和的默认色彩",
|
||||
"强光下的镜头眩光", "水滴或灰尘沾在镜头上"],
|
||||
"operation": ["胸挂或头戴的第一人称视角", "剧烈但被电子增稳压制的震动",
|
||||
"呼吸和身体晃动传导到画面", "视角随头部转动突然甩动"],
|
||||
"negative": "没有第三人称跟拍。没有电影构图。没有慢动作（除非明确要求）。",
|
||||
},
|
||||
"cctv": {
|
||||
"name": "监控摄像头 / CCTV",
|
||||
"fit": "悬疑、都市传说、客观记录感、伪纪录片",
|
||||
"visual": ["固定高位俯视角", "低帧率（每秒约10到15帧的顿挫感）",
|
||||
"单色或低饱和色彩，夜间红外模式呈灰白", "角落的时间戳持续跳动",
|
||||
"低分辨率，人脸模糊难辨"],
|
||||
"operation": ["完全静止，或极缓慢的预设巡航转动", "无声",
|
||||
"人物不知道被拍摄，行为完全自然"],
|
||||
"negative": "没有摄影师存在。没有构图意识。没有声音。没有变焦跟随主体。",
|
||||
},
|
||||
"doc_handheld": {
|
||||
"name": "专业纪录片（手持电影机）",
|
||||
"fit": "高质感纪实、人物特稿、社会观察",
|
||||
"visual": ["浅景深，主体清晰背景柔和", "自然光为主，宽容度高",
|
||||
"克制的调色，接近肉眼所见"],
|
||||
"operation": ["有经验的手持，微晃但不失控", "跟焦偶尔慢半拍（保留这个不完美）",
|
||||
"长镜头耐心观察，不急于切换", "被摄者偶尔瞥向镜头又移开"],
|
||||
"negative": "没有摆拍感。没有采访灯光。没有戏剧化运镜。没有配乐驱动的剪辑节奏。",
|
||||
},
|
||||
"webcam": {
|
||||
"name": "网络摄像头 / 视频通话",
|
||||
"fit": "疫情记忆、远程对话、屏幕录制感、亲密而低保真的交流",
|
||||
"visual": ["720p 以下的低清晰度", "压缩造成的块状伪影，网络波动时画面糊成马赛克",
|
||||
"顶光或屏幕光打在脸上的青白色", "固定机位，仰视或平视的笔记本视角"],
|
||||
"operation": ["音画偶尔不同步", "人物偶尔卡顿定格再恢复", "背景是真实的房间一角"],
|
||||
"negative": "没有专业收音。没有补光灯质感。没有运镜。",
|
||||
},
|
||||
"flip_phone_2003": {
|
||||
"name": "2003年翻盖手机摄像（CMOS低保真）",
|
||||
"fit": "极度怀旧、千禧年初学生时代、赛博朋克低保真、暗角情绪短片",
|
||||
"visual": ["极低分辨率（176×144 或 320×240）", "严重 JPEG 压缩块状伪影，色块化严重",
|
||||
"色彩饱和度极端且偏色严重（通常偏蓝绿或偏品红）", "暗部几乎完全噪点糊死",
|
||||
"画面中央清晰度尚可，边缘急剧劣化", "固定数字变焦（裁切放大）",
|
||||
"超窄动态范围，高光纯白一片"],
|
||||
"operation": ["单手持机，手臂伸直自拍或随意抓拍", "翻盖打开即拍，几乎没有延迟补偿",
|
||||
"快门时滞极长（按下到实际拍摄有0.5到1秒延迟）",
|
||||
"录像按一下开始，再按一下结束，没有预览确认",
|
||||
"对焦基本靠固定焦段，前后移动自身来「对焦」",
|
||||
"录制过程中镜头容易被手指或翻盖边缘遮挡"],
|
||||
"negative": "没有清晰度。没有平滑运动。没有色彩准确度。没有变焦操作。",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# ★ 氛围词典（抽象氛围 → 具体可拍现象）
|
||||
# 来源：atmosphere-dictionary.md（原文存 docs/realism/）
|
||||
#
|
||||
# 原则：**禁止直接使用抽象氛围词**，必须翻译成具体视觉/听觉现象，
|
||||
# 每种氛围至少选 3~5 个现象组合使用。
|
||||
# ════════════════════════════════════════════════════════════
|
||||
ATMOSPHERE_DICT: dict = {
|
||||
"温暖": {
|
||||
"visual": ["午后阳光透过树叶在地面投下移动的斑驳光影", "晾衣绳上的织物在微风中缓慢摇曳",
|
||||
"陶瓷杯口冒出的细微热气", "猫或狗慵懒地靠近人",
|
||||
"盆栽植物摆在窗台和门口台阶上", "人物把头发拨到耳后的小动作",
|
||||
"不经意的真诚微笑，随即移开视线"],
|
||||
"audio": ["晨间鸟鸣", "远处小孩嬉闹声", "织物摩擦声", "猫叫", "勺子碰杯壁"],
|
||||
},
|
||||
"怀旧": {
|
||||
"visual": ["褪色的色彩与柔和的对比度", "老式电线杆和架空电线切割天空",
|
||||
"墙面斑驳的低层住宅", "停在巷口的旧自行车", "画面上的日期戳",
|
||||
"人物穿着宽松水洗牛仔裤等年代服饰"],
|
||||
"audio": ["远处摩托车驶过", "老风扇转动声", "模拟信号的底噪"],
|
||||
},
|
||||
"孤独": {
|
||||
"visual": ["大面积负空间构图，人物偏居画面一角", "空旷街道上唯一移动的身影",
|
||||
"荧光灯的青白色调", "便利店或地铁站的深夜灯光",
|
||||
"人物长时间静止，只有环境在动（车流、雨、霓虹闪烁）", "玻璃反光中模糊的人影"],
|
||||
"audio": ["空调外机嗡鸣", "脚步回声", "远处地铁进站声", "雨点打在遮阳棚上"],
|
||||
},
|
||||
"紧张": {
|
||||
"visual": ["画面边缘偶尔闯入不明物体或人影", "光源不稳定，灯管闪烁",
|
||||
"门缝、走廊尽头等纵深阴影", "手持镜头呼吸感加重，微微颤抖",
|
||||
"人物频繁回头或停顿倾听"],
|
||||
"audio": ["突然中断的环境音", "远处不明的金属碰撞声", "自己的呼吸声被收进麦克风"],
|
||||
},
|
||||
"平静": {
|
||||
"visual": ["极缓慢的光线变化（云影掠过整个画面）", "水面或窗帘的规律性微动",
|
||||
"长镜头，机位几乎不动", "人物做重复性动作（浇水、叠衣、剥豆）"],
|
||||
"audio": ["风穿过树叶的持续沙沙声", "水滴声", "远处若有若无的生活声"],
|
||||
},
|
||||
"热闹": {
|
||||
"visual": ["前景不断有人或车穿过镜头", "摊贩的动作细节（翻炒、找零、擦桌子）",
|
||||
"蒸汽、油烟、烤炉的热浪扭曲空气",
|
||||
"多层次的动态：近景手部动作、中景人流、远景车流"],
|
||||
"audio": ["叫卖声", "锅铲碰撞", "多人交谈混叠", "自行车铃"],
|
||||
},
|
||||
"梦境": {
|
||||
"visual": ["过曝的窗户白成一片", "胶片颗粒与光晕", "镜头短，硬切，时间线跳跃",
|
||||
"人物对镜头挥手或大笑但无声", "逆光轮廓，面部细节丢失"],
|
||||
"audio": ["无声，或只有一层模糊的环境底噪"],
|
||||
},
|
||||
"田园": {
|
||||
"visual": ["大片稻田或麦田在风中如波浪起伏", "人物赤脚踩在湿润泥土或田埂上",
|
||||
"荷叶、野花、藤蔓等植物特写", "阳光透过树叶在人物脸上流动的光斑",
|
||||
"远处农舍的炊烟和黑瓦白墙", "蜻蜓点水、蝴蝶停在花上",
|
||||
"人物背影渐渐走远，融入田野尽头"],
|
||||
"audio": ["连续的蛙鸣和虫鸣", "风吹稻田的沙沙声", "远处公鸡打鸣", "溪流声",
|
||||
"脚踏湿泥的柔软声音", "竹叶碰撞声"],
|
||||
},
|
||||
"夏日": {
|
||||
"visual": ["树影在地面剧烈晃动，光斑密集而明亮", "人物额头和鬓角有细微汗珠",
|
||||
"手中冰饮或西瓜表面凝结水珠", "风扇叶片或蒲扇的转动",
|
||||
"强烈阳光造成的高光过曝和镜头眩光", "热空气扭曲远处景物",
|
||||
"牵牛花、向日葵、荷叶等盛夏植物", "偶尔的云影掠过造成短暂阴凉"],
|
||||
"audio": ["比任何季节都响亮的蝉鸣", "空调外机或风扇的低沉嗡鸣",
|
||||
"冰块碰杯的清脆声", "远处孩童的嬉笑和泼水声"],
|
||||
},
|
||||
"通勤": {
|
||||
"visual": ["密集的人流与单个主体形成对比", "从室内到室外/地上到地下的曝光剧烈变化",
|
||||
"呼出的白气（秋冬）或额头的汗（盛夏）", "低头看手机、翻包找卡等碎片化动作",
|
||||
"车窗玻璃上反射的模糊面容", "通勤者独特的身体语言（缩脖子、拉围巾）",
|
||||
"地铁隧道中快速掠过的灯光管"],
|
||||
"audio": ["广播模糊的播报声", "列车进站的轰鸣和刹车尖叫", "闸机蜂鸣声",
|
||||
"脚步匆忙的节奏", "车门气动声", "隧道风噪声"],
|
||||
},
|
||||
"青春": {
|
||||
"visual": ["漫无目的的闲逛或长时间坐着发呆", "手部小动作（拨弄衣角、卷头发、转笔）",
|
||||
"靠在墙边、栏杆上，重心随意切换", "低头笑、抬头看天、把脸埋进手臂",
|
||||
"旧电扇转头吹动刘海和衣领", "地面上的影子被拉长，时间缓慢推移",
|
||||
"零星对话，说话间隙被环境填满"],
|
||||
"audio": ["远处收音机或电视的模糊声音", "老旧风扇的转动声", "冰棍棒折断声",
|
||||
"偶尔的笑声然后继续安静"],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def device_pack_block(key: str, include_negative: bool = True) -> str:
|
||||
"""设备缺陷参数包 → 提示词片段（可直接粘进「摄像风格」段）"""
|
||||
p = DEVICE_PACKS.get((key or "").strip())
|
||||
if not p:
|
||||
return ""
|
||||
out = [f"【设备美学：{p['name']}】"]
|
||||
out.append("画面特征：" + "；".join(p["visual"]))
|
||||
out.append("操作特征：" + "；".join(p["operation"]))
|
||||
if include_negative and p.get("negative"):
|
||||
out.append("负面约束：" + p["negative"])
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def atmosphere_block(keys, n_each: int = 4) -> str:
|
||||
"""氛围 → 具体现象片段（每种氛围取 n 个视觉 + 全部听觉的前若干个）
|
||||
|
||||
keys 可以是 str 或 list；未命中则原样返回空串（调用方可自行翻译）。
|
||||
用法原则：**不要直接写抽象氛围词，用这些具体现象替代**。
|
||||
"""
|
||||
if isinstance(keys, str):
|
||||
keys = [keys]
|
||||
out = []
|
||||
for k in (keys or []):
|
||||
a = ATMOSPHERE_DICT.get(str(k).strip())
|
||||
if not a:
|
||||
continue
|
||||
vis = a["visual"][:max(1, n_each)]
|
||||
aud = a["audio"][:max(1, n_each // 2)]
|
||||
out.append(f"【{k}（用具体现象替代）】视觉：" + "；".join(vis)
|
||||
+ ("。听觉：" + "；".join(aud) if aud else ""))
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def assign_device_defects(n_shots: int) -> list:
|
||||
"""按「缺陷不重复原则」给每个镜头分配**不同类型**的摄像机缺陷
|
||||
|
||||
来源：camera-aesthetics.md 进阶用法 —— 同一提示词里每镜的缺陷类型不重复：
|
||||
对焦 → 构图 → 抖动 → 曝光 → 反应延迟 → 呼吸感 → 录制中断
|
||||
"""
|
||||
seq = [
|
||||
"自动对焦在前景和背景之间搜索", # 对焦缺陷
|
||||
"构图偏离中心，操作者跟不上主体", # 构图缺陷
|
||||
"跟拍晃动 + 运动模糊", # 抖动缺陷
|
||||
"进入阴影时曝光波动", # 曝光缺陷
|
||||
"镜头慢了半拍才跟上动作", # 反应延迟
|
||||
"自然的镜头呼吸和焦距漂移", # 呼吸感
|
||||
]
|
||||
out = []
|
||||
for i in range(max(0, n_shots)):
|
||||
if i == n_shots - 1 and n_shots > 1:
|
||||
out.append("画面突然切到黑屏（录制中断）") # 结尾固定收束
|
||||
else:
|
||||
out.append(seq[i % len(seq)])
|
||||
return out
|
||||
|
||||
|
||||
def focus_path_hint() -> str:
|
||||
"""「自动对焦路径指定法」提示（效果翻倍的关键技巧）
|
||||
|
||||
不要笼统写"自动对焦搜索"，要给出**对焦起点和终点**。
|
||||
"""
|
||||
return ("【对焦路径指定（关键技巧）】不要笼统写「自动对焦搜索」——"
|
||||
"要明确写出对焦的起点和终点，模型才知道对焦该在哪两个物体之间走：\n"
|
||||
" ❌ 自动对焦反复搜索\n"
|
||||
" ✅ 自动对焦在她脸部和她手中的西瓜之间来回切换\n"
|
||||
" ✅ 自动对焦从远处山影漂移到她后背再拉回远处")
|
||||
@@ -0,0 +1,376 @@
|
||||
"""
|
||||
AVE speed_ramp — 变速卡点引擎
|
||||
|
||||
功能:
|
||||
1. 对视频/音频施加匀速变速
|
||||
2. 缓变速度曲线: 将视频分段, 每段不同速度, 组合成渐变效果
|
||||
3. 卡点策略集成: 根据能量/节拍密度生成动态速度曲线
|
||||
|
||||
核心: FFmpeg setpts(视频) + atempo(音频) 滤波器链
|
||||
|
||||
用法:
|
||||
from composer.speed_ramp import speed_ramp, speed_curve
|
||||
|
||||
# 匀速 0.8x
|
||||
speed_ramp("input.mp4", "slow.mp4", speed=0.8)
|
||||
|
||||
# 缓变 0.7→1.0→1.3 (6段渐变)
|
||||
speed_ramp("input.mp4", "ramp.mp4", curve=[0.7, 1.0, 1.3], segments=6)
|
||||
|
||||
# 节拍驱动的变速 (高能段加速, 低能段减速)
|
||||
speed_by_energy("input.mp4", beat_info={"segments": [...]}, output="energy_ramp.mp4")
|
||||
"""
|
||||
import sys, os; sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from lib.logger import get_logger
|
||||
from lib.ffmpeg import FFMPEG # noqa: E402 (FFmpeg 绝对路径)
|
||||
|
||||
logger = get_logger("speed_ramp")
|
||||
|
||||
# ── 常量 ──────────────────────────────────────────────────
|
||||
|
||||
# atempo 支持的范围: 0.5~2.0 (单次)
|
||||
# 需要更极端的变速时, 级联多个 atempo
|
||||
ATEMPO_MIN = 0.5
|
||||
ATEMPO_MAX = 2.0
|
||||
|
||||
|
||||
# ── 公共 API ──────────────────────────────────────────────
|
||||
|
||||
def speed_ramp(
|
||||
input_path: str,
|
||||
output_path: str = "speed_ramp.mp4",
|
||||
speed: float = 1.0,
|
||||
curve: list[float] | None = None,
|
||||
segments: int = 6,
|
||||
keep_audio: bool = True,
|
||||
) -> str:
|
||||
"""
|
||||
变速处理
|
||||
|
||||
参数:
|
||||
input_path: 输入视频路径
|
||||
output_path: 输出视频路径
|
||||
speed: 匀速倍率 (curve 为空时使用)
|
||||
curve: 缓变速度曲线 [起点倍率, ..., 终点倍率] (优先于 speed)
|
||||
例: [0.7, 1.0, 1.3] 表示从 0.7x 渐变到 1.3x
|
||||
segments: 缓变分段数 (越多越平滑, 默认 6)
|
||||
keep_audio: 是否保留音频 (默认 True)
|
||||
|
||||
返回: output_path
|
||||
"""
|
||||
if not os.path.exists(input_path):
|
||||
raise FileNotFoundError(f"输入文件不存在: {input_path}")
|
||||
|
||||
duration = _get_duration(input_path)
|
||||
|
||||
if curve and len(curve) >= 2:
|
||||
return _apply_curve(input_path, output_path, curve, segments, duration, keep_audio)
|
||||
else:
|
||||
return _apply_constant(input_path, output_path, speed, duration, keep_audio)
|
||||
|
||||
|
||||
def speed_by_energy(
|
||||
input_path: str,
|
||||
output_path: str = "energy_ramp.mp4",
|
||||
beat_segments: list[dict] | None = None,
|
||||
base_speed: float = 1.0,
|
||||
high_speed: float = 1.5,
|
||||
low_speed: float = 0.7,
|
||||
keep_audio: bool = True,
|
||||
) -> str:
|
||||
"""
|
||||
按节拍能量生成变速曲线
|
||||
|
||||
beat_segments 格式: [{"start": 0.0, "end": 2.0, "beats": 4, "energy": 0.8}, ...]
|
||||
高能量(>0.6) → 加速, 低能量(<0.3) → 减速, 中间 → 基础速度
|
||||
|
||||
返回: output_path
|
||||
"""
|
||||
if not os.path.exists(input_path):
|
||||
raise FileNotFoundError(f"输入文件不存在: {input_path}")
|
||||
|
||||
if not beat_segments:
|
||||
logger.warning("无节拍段落, 回落匀速")
|
||||
return _apply_constant(input_path, output_path, base_speed, _get_duration(input_path), keep_audio)
|
||||
|
||||
# 按能量生成每段速度
|
||||
temp_dir = tempfile.mkdtemp(prefix="ave_spd_")
|
||||
seg_files = []
|
||||
total_duration = _get_duration(input_path)
|
||||
|
||||
for idx, seg in enumerate(beat_segments):
|
||||
start = seg["start"]
|
||||
end = seg["end"]
|
||||
dur = end - start
|
||||
if dur < 0.3:
|
||||
continue
|
||||
|
||||
# 根据能量决定速度 (线性插值)
|
||||
energy = seg.get("energy", 0.5)
|
||||
if energy > 0.6:
|
||||
spd = base_speed + (high_speed - base_speed) * min((energy - 0.6) / 0.4, 1.0)
|
||||
elif energy < 0.3:
|
||||
spd = low_speed + (base_speed - low_speed) * (energy / 0.3)
|
||||
else:
|
||||
spd = base_speed
|
||||
|
||||
spd = max(ATEMPO_MIN, min(ATEMPO_MAX, spd))
|
||||
seg_out = os.path.join(temp_dir, f"seg_{idx:03d}.mp4")
|
||||
_apply_constant(input_path, seg_out, spd, dur, keep_audio, ss=start, t=dur)
|
||||
seg_files.append(seg_out)
|
||||
|
||||
if not seg_files:
|
||||
shutil.rmtree(temp_dir, ignore_errors=True)
|
||||
return _apply_constant(input_path, output_path, base_speed, total_duration, keep_audio)
|
||||
|
||||
# 拼接
|
||||
_concat_segments(seg_files, output_path, keep_audio)
|
||||
import shutil
|
||||
shutil.rmtree(temp_dir, ignore_errors=True)
|
||||
|
||||
logger.info(f"✅ 能量变速完成: {output_path} ({len(seg_files)} 段)")
|
||||
return output_path
|
||||
|
||||
|
||||
def make_energy_curve(
|
||||
beat_info: dict,
|
||||
base_speed: float = 1.0,
|
||||
high_speed: float = 1.5,
|
||||
low_speed: float = 0.7,
|
||||
) -> list[tuple[float, float, float]]:
|
||||
"""
|
||||
从节拍检测结果生成能量驱动速度曲线
|
||||
|
||||
返回: [(start_sec, end_sec, speed), ...]
|
||||
"""
|
||||
segments = beat_info.get("segments", [])
|
||||
if not segments:
|
||||
return [(0, beat_info.get("total_duration", 30), base_speed)]
|
||||
|
||||
# 计算每段的归一化能量 (基于拍点密度)
|
||||
max_beats = max(s.get("beats", 0) for s in segments) or 1
|
||||
curves = []
|
||||
for s in segments:
|
||||
dur = s["end"] - s["start"]
|
||||
if dur <= 0:
|
||||
continue
|
||||
# 拍点密度 → 能量
|
||||
density = s.get("beats", 0) / max(max_beats, 1)
|
||||
energy = max(0.0, min(1.0, density))
|
||||
|
||||
if energy > 0.6:
|
||||
spd = base_speed + (high_speed - base_speed) * min((energy - 0.6) / 0.4, 1.0)
|
||||
elif energy < 0.3:
|
||||
spd = low_speed + (base_speed - low_speed) * (energy / 0.3)
|
||||
else:
|
||||
spd = base_speed
|
||||
spd = max(ATEMPO_MIN, min(ATEMPO_MAX, spd))
|
||||
curves.append((s["start"], s["end"], spd))
|
||||
|
||||
return curves
|
||||
|
||||
|
||||
# ── 内部实现 ──────────────────────────────────────────────
|
||||
|
||||
def _apply_constant(
|
||||
input_path: str,
|
||||
output_path: str,
|
||||
speed: float,
|
||||
duration: float | None = None,
|
||||
keep_audio: bool = True,
|
||||
ss: float = 0.0,
|
||||
t: float | None = None,
|
||||
) -> str:
|
||||
"""匀速变速"""
|
||||
speed = max(ATEMPO_MIN, min(ATEMPO_MAX, speed))
|
||||
vf = f"setpts={1/speed}*PTS"
|
||||
|
||||
cmd = [FFMPEG, "-y"]
|
||||
|
||||
# 如果指定了裁剪区间
|
||||
if ss > 0:
|
||||
cmd += ["-ss", str(ss)]
|
||||
cmd += ["-i", input_path]
|
||||
if t is not None:
|
||||
cmd += ["-t", str(t)]
|
||||
|
||||
if keep_audio:
|
||||
# atempo 不支持超出 [0.5, 2.0], 级联处理
|
||||
af = _build_atempo_filter(speed)
|
||||
# ⚠️ af 自带输入/输出标签（`[0:a]atempo=…[a]`），不能再拼一个 [a]
|
||||
# （否则 filter 图非法 → 音频编码 0 字节，输出只有画面）
|
||||
cmd += ["-filter_complex", f"{vf}[v];{af}"]
|
||||
cmd += ["-map", "[v]", "-map", "[a]"]
|
||||
else:
|
||||
cmd += ["-filter_complex", f"{vf}[v]", "-map", "[v]", "-an"]
|
||||
|
||||
cmd += ["-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-c:a", "aac", "-b:a", "192k", "-pix_fmt", "yuv420p", output_path]
|
||||
|
||||
try:
|
||||
subprocess.run(cmd, capture_output=True, timeout=300, check=True)
|
||||
logger.debug(f" 匀速 {speed:.2f}x → {output_path}")
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.warning(f" 变速失败: {e.stderr[:200]}")
|
||||
# 回落: 不变速
|
||||
subprocess.run([FFMPEG, "-y", "-i", input_path,
|
||||
"-c", "copy", output_path], capture_output=True, timeout=60, check=True)
|
||||
|
||||
return output_path
|
||||
|
||||
|
||||
def _apply_curve(
|
||||
input_path: str,
|
||||
output_path: str,
|
||||
curve: list[float],
|
||||
num_segments: int,
|
||||
total_duration: float,
|
||||
keep_audio: bool,
|
||||
) -> str:
|
||||
"""缓变速度曲线: 将输入切成 N 段, 每段按曲线插值变速, 再拼接"""
|
||||
# 生成每段的速度
|
||||
speeds = _interpolate_curve(curve, num_segments)
|
||||
seg_duration = total_duration / num_segments
|
||||
|
||||
temp_dir = tempfile.mkdtemp(prefix="ave_curve_")
|
||||
seg_files = []
|
||||
|
||||
for idx, spd in enumerate(speeds):
|
||||
seg_start = idx * seg_duration
|
||||
seg_out = os.path.join(temp_dir, f"seg_{idx:03d}.mp4")
|
||||
_apply_constant(input_path, seg_out, spd, seg_duration,
|
||||
keep_audio, ss=seg_start, t=seg_duration)
|
||||
seg_files.append(seg_out)
|
||||
|
||||
# 拼接
|
||||
_concat_segments(seg_files, output_path, keep_audio)
|
||||
import shutil
|
||||
shutil.rmtree(temp_dir, ignore_errors=True)
|
||||
|
||||
logger.info(f"✅ 曲线变速完成: {output_path} ({len(speeds)} 段, "
|
||||
f"速度 {speeds[0]:.2f}→{speeds[-1]:.2f})")
|
||||
return output_path
|
||||
|
||||
|
||||
def _interpolate_curve(curve: list[float], num_segments: int) -> list[float]:
|
||||
"""在曲线控制点之间线性插值"""
|
||||
if len(curve) >= num_segments:
|
||||
return curve[:num_segments]
|
||||
|
||||
# 在 curve 点之间均匀插值到 num_segments 个点
|
||||
result = []
|
||||
n_controls = len(curve) - 1
|
||||
for i in range(num_segments):
|
||||
# 映射到 [0, n_controls] 区间
|
||||
pos = i / (num_segments - 1) * n_controls if num_segments > 1 else 0
|
||||
idx_low = int(pos)
|
||||
idx_high = min(idx_low + 1, n_controls)
|
||||
frac = pos - idx_low
|
||||
# 线性插值
|
||||
val = curve[idx_low] * (1 - frac) + curve[idx_high] * frac
|
||||
result.append(max(ATEMPO_MIN, min(ATEMPO_MAX, val)))
|
||||
return result
|
||||
|
||||
|
||||
def _build_atempo_filter(speed: float) -> str:
|
||||
"""构建 atempo 滤波器链 (支持 [0.5, 2.0], 超范围级联)
|
||||
|
||||
atempo 单次只能 [0.5, 2.0].
|
||||
例如 speed=3.0 → atempo=2.0, atempo=1.5
|
||||
例如 speed=0.4 → atempo=0.5, atempo=0.8
|
||||
"""
|
||||
if ATEMPO_MIN <= speed <= ATEMPO_MAX:
|
||||
return f"[0:a]atempo={speed}[a]"
|
||||
|
||||
# 级联: 分解为多个在范围内的 atempo
|
||||
factors = []
|
||||
remaining = speed
|
||||
if speed > ATEMPO_MAX:
|
||||
while remaining > ATEMPO_MAX:
|
||||
factors.append(ATEMPO_MAX)
|
||||
remaining /= ATEMPO_MAX
|
||||
if remaining >= ATEMPO_MIN:
|
||||
factors.append(remaining)
|
||||
elif speed < ATEMPO_MIN:
|
||||
while remaining < ATEMPO_MIN:
|
||||
factors.append(ATEMPO_MIN)
|
||||
remaining /= ATEMPO_MIN
|
||||
if remaining <= ATEMPO_MAX:
|
||||
factors.append(remaining)
|
||||
|
||||
if not factors:
|
||||
factors = [speed]
|
||||
|
||||
# 构建级联链: [0:a]atempo=f0[a0];[a0]atempo=f1[a1];...
|
||||
chain = []
|
||||
prev = "0:a"
|
||||
for i, f in enumerate(factors):
|
||||
label = f"a{i}"
|
||||
chain.append(f"[{prev}]atempo={f:.4f}[{label}]")
|
||||
prev = label
|
||||
return ";".join(chain)
|
||||
|
||||
|
||||
def _concat_segments(seg_files: list[str], output_path: str, keep_audio: bool):
|
||||
"""拼接变速后的视频段"""
|
||||
if len(seg_files) == 1:
|
||||
subprocess.run([FFMPEG, "-y", "-i", seg_files[0],
|
||||
"-c", "copy", output_path], capture_output=True, timeout=60, check=True)
|
||||
return
|
||||
|
||||
# 用 concat demuxer
|
||||
concat_file = tempfile.NamedTemporaryFile(suffix=".txt", delete=False, mode="w").name
|
||||
with open(concat_file, "w") as f:
|
||||
for sf in seg_files:
|
||||
f.write(f"file '{os.path.abspath(sf)}'\n")
|
||||
|
||||
cmd = [FFMPEG, "-y", "-f", "concat", "-safe", "0", "-i", concat_file]
|
||||
if keep_audio:
|
||||
cmd += ["-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-c:a", "aac", "-b:a", "192k", "-pix_fmt", "yuv420p"]
|
||||
else:
|
||||
cmd += ["-c:v", "libx264", "-preset", "fast", "-crf", "23", "-an"]
|
||||
cmd += [output_path]
|
||||
|
||||
try:
|
||||
subprocess.run(cmd, capture_output=True, timeout=300, check=True)
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.warning(f" 拼接失败, 回落 concat: {e.stderr[:200]}")
|
||||
# 回落: filter concat
|
||||
concat_filter = "".join(f"[{i}:v]" for i in range(len(seg_files)))
|
||||
cmd2 = [FFMPEG, "-y"]
|
||||
for sf in seg_files:
|
||||
cmd2 += ["-i", sf]
|
||||
cmd2 += ["-filter_complex",
|
||||
f"{concat_filter}concat=n={len(seg_files)}:v=1{'a=1' if keep_audio else '':1}[v]"
|
||||
+ (f";{concat_filter.replace(':v',':a')}concat=n={len(seg_files)}:v=0:a=1[a]"
|
||||
if keep_audio else ""),
|
||||
"-map", "[v]"]
|
||||
if keep_audio:
|
||||
cmd2 += ["-map", "[a]"]
|
||||
cmd2 += ["-c:v", "libx264", "-preset", "fast", "-crf", "23",
|
||||
"-c:a", "aac", "-b:a", "192k", "-pix_fmt", "yuv420p", output_path]
|
||||
subprocess.run(cmd2, capture_output=True, timeout=300, check=True)
|
||||
|
||||
os.unlink(concat_file)
|
||||
|
||||
|
||||
def _get_duration(path: str) -> float:
|
||||
"""获取媒体文件时长（秒）
|
||||
|
||||
用 lib.ffmpeg.probe_duration（ffmpeg -i 解析）——
|
||||
本机 ffprobe 实为 ffmpeg 的符号链接，不支持 -show_entries（会静默失败）
|
||||
"""
|
||||
try:
|
||||
from lib.ffmpeg import probe_duration
|
||||
d = probe_duration(path)
|
||||
return d if d > 0 else 30.0
|
||||
except Exception:
|
||||
return 30.0
|
||||
@@ -0,0 +1,114 @@
|
||||
meta:
|
||||
project_id: "daoist_001"
|
||||
style: knowledge_lecture
|
||||
voice:
|
||||
provider: aliyun
|
||||
voice_id: cosyvoice-v3.5-plus-bailian-f92cac2a41ce4fc78153c6097755a60e
|
||||
default_emotion: 专业沉稳
|
||||
bgm:
|
||||
provider: chord_pad
|
||||
genre: cinematic
|
||||
tempo: 70
|
||||
mood: calm
|
||||
output:
|
||||
resolution: 1080x1920
|
||||
fps: 30
|
||||
segments:
|
||||
- id: 1
|
||||
text: "道家思想，源远流长。今天和大家分享几句道家经典金句，一起感受古人的智慧。"
|
||||
duration_sec: 10
|
||||
voice_emotion: 专业沉稳
|
||||
bgm_section: intro
|
||||
material:
|
||||
source: pexels
|
||||
search: "chinese landscape"
|
||||
transition_in: fade_in
|
||||
- id: 2
|
||||
text: "上善若水，水善利万物而不争。——《道德经》最上等的善就像水一样，默默滋养万物，却从不争抢。"
|
||||
duration_sec: 12
|
||||
voice_emotion: 专业沉稳
|
||||
bgm_section: main
|
||||
material:
|
||||
source: pexels
|
||||
search: "water flow nature"
|
||||
- id: 3
|
||||
text: "飘风不终朝，骤雨不终日。——《道德经》越是猛烈的狂风暴雨，越不会持久。那些和风细雨，反而悠远绵长。"
|
||||
duration_sec: 12
|
||||
voice_emotion: 专业沉稳
|
||||
bgm_section: main
|
||||
material:
|
||||
source: pexels
|
||||
search: "rain nature calm"
|
||||
- id: 4
|
||||
text: "祸兮福之所倚，福兮祸之所伏。——《道德经》坏事中藏着好运，好事中也藏着危机。顺境莫得意，逆境莫灰心。"
|
||||
duration_sec: 12
|
||||
voice_emotion: 专业沉稳
|
||||
bgm_section: main
|
||||
material:
|
||||
source: pexels
|
||||
search: "yin yang balance"
|
||||
- id: 5
|
||||
text: "大方无隅，大器晚成。——《道德经》最宏大的方正反而没有棱角，最贵重的器物反而最后完成。真正有价值的事，需要时间沉淀。"
|
||||
duration_sec: 12
|
||||
voice_emotion: 专业沉稳
|
||||
bgm_section: main
|
||||
material:
|
||||
source: pexels
|
||||
search: "sunset time"
|
||||
- id: 6
|
||||
text: "吾生也有涯，而知也无涯。——《庄子》以有限的生命去追求无限的知识，是一件危险的事。不是不学，而是要有取舍。"
|
||||
duration_sec: 12
|
||||
voice_emotion: 专业沉稳
|
||||
bgm_section: main
|
||||
material:
|
||||
source: pexels
|
||||
search: "reading book wisdom"
|
||||
- id: 7
|
||||
text: "举世誉之而不加劝，举世非之而不加沮。——《庄子》全世界都赞美你，不会更努力；全世界都批评你，不会更沮丧。内心坚定，不因外物动摇。"
|
||||
duration_sec: 12
|
||||
voice_emotion: 专业沉稳
|
||||
bgm_section: climax
|
||||
material:
|
||||
source: pexels
|
||||
search: "mountain peak sunrise"
|
||||
- id: 8
|
||||
text: "人生天地之间，若白驹过隙，忽然而已。——《庄子》人生在天地之间，就像白色骏马闪过缝隙，转瞬即逝。珍惜当下。"
|
||||
duration_sec: 12
|
||||
voice_emotion: 专业沉稳
|
||||
bgm_section: main
|
||||
material:
|
||||
source: pexels
|
||||
search: "time lapse nature"
|
||||
- id: 9
|
||||
text: "相濡以沫，不如相忘于江湖。——《庄子》与其在干涸的车辙里用口水互相湿润，不如各自在大江大湖里自由遨游。"
|
||||
duration_sec: 10
|
||||
voice_emotion: 专业沉稳
|
||||
bgm_section: main
|
||||
material:
|
||||
source: pexels
|
||||
search: "lake fish swimming"
|
||||
- id: 10
|
||||
text: "致虚极，守静笃。——《道德经》当我们放下一切杂念，内心达到极致的虚空和宁静时，就能感受到大道的存在了。"
|
||||
duration_sec: 12
|
||||
voice_emotion: 专业沉稳
|
||||
bgm_section: main
|
||||
material:
|
||||
source: pexels
|
||||
search: "meditation peace"
|
||||
- id: 11
|
||||
text: "正身直行，众邪自息。——《淮南子》自身端正，走得正行得直，各种邪门歪道自然就会消失。修身立本。"
|
||||
duration_sec: 10
|
||||
voice_emotion: 专业沉稳
|
||||
bgm_section: main
|
||||
material:
|
||||
source: pexels
|
||||
search: "bamboo forest path"
|
||||
- id: 12
|
||||
text: "道法自然，无为而治。以上几句道家智慧，与君共勉。关注我，一起领悟更多人生智慧。"
|
||||
duration_sec: 10
|
||||
voice_emotion: 专业沉稳
|
||||
bgm_section: outro
|
||||
material:
|
||||
source: pexels
|
||||
search: "chinese calligraphy"
|
||||
transition_out: fade_out
|
||||
@@ -0,0 +1,256 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Demo: 漫剧视频工厂全链路流程演示 v1.0
|
||||
|
||||
模拟一次完整的生产流程:
|
||||
灵感 → 脚本 → Gate 1(审核) → 角色适配 → Gate 2(审核) → 选BGM → Gate 3(费用确认) → 开始生产
|
||||
|
||||
通过此演示可以看到三道审核关卡的交互和各个模块的协同工作。
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from script_schemas import ScriptBuilder, validate_script, generate_gate1_summary, generate_gate2_table, save_script
|
||||
from character_registry import CharacterRegistry
|
||||
from character_adapter import CharacterAdapter
|
||||
from music_selector import BGMSelector
|
||||
from pipeline_controller import PipelineController
|
||||
|
||||
|
||||
def print_separator(title):
|
||||
print()
|
||||
print("=" * 70)
|
||||
print(f" {title}")
|
||||
print("=" * 70)
|
||||
|
||||
|
||||
def main():
|
||||
print_separator("漫剧视频工厂 — 全链路流程演示")
|
||||
print("模拟场景: '小漫的一天' — 治愈系漫剧")
|
||||
print("角色: 小漫 (示例角色，已在角色注册中心注册)")
|
||||
print("路径: 漫剧 (即梦文生图)")
|
||||
print()
|
||||
|
||||
# ── 准备 ──
|
||||
print_separator("📋 第 0 步: 初始化模块")
|
||||
|
||||
registry = CharacterRegistry()
|
||||
adapter = CharacterAdapter(registry)
|
||||
bgm = BGMSelector()
|
||||
pc = PipelineController()
|
||||
|
||||
char = registry.get_active_character()
|
||||
print(f" 角色: {char.name}")
|
||||
print(f" 描述: {char.description[:40]}...")
|
||||
print(f" BGM 库: {len(bgm)} 首可用")
|
||||
print(f" BGM 热门: {[e['name'] for e in bgm.get_trending(3)]}")
|
||||
|
||||
# ── 创建脚本 ──
|
||||
print_separator("📝 第 1 步: 创建脚本 (输入层)")
|
||||
script = ScriptBuilder.empty("小漫的一天", "小漫")
|
||||
script["meta"]["source"] = "文字灵感"
|
||||
|
||||
ScriptBuilder.add_segment(script, {
|
||||
"scene": "清晨的咖啡馆",
|
||||
"description": "推开咖啡馆的门，阳光透过窗户洒进来，空气中飘着咖啡香",
|
||||
"dialogue": "（旁白）每个清晨，小漫都会来到这家街角的咖啡馆。",
|
||||
"emotion": "温暖平静",
|
||||
"duration_sec": 8,
|
||||
"camera": "wide_shot",
|
||||
"transition": "fade_in",
|
||||
})
|
||||
ScriptBuilder.add_segment(script, {
|
||||
"scene": "靠窗的座位",
|
||||
"description": "她坐下来，把画板放在桌上，拿出铅笔开始构思",
|
||||
"dialogue": "小漫: '今天一定要把草稿画完！'",
|
||||
"emotion": "专注",
|
||||
"duration_sec": 6,
|
||||
"camera": "medium_shot",
|
||||
"transition": "cut",
|
||||
})
|
||||
ScriptBuilder.add_segment(script, {
|
||||
"scene": "画板上的世界",
|
||||
"description": "铅笔在纸上飞舞，线条逐渐成形，一个奇妙的世界在画中展开",
|
||||
"dialogue": "（旁白）在她的画笔下，每一个角落都充满了故事。",
|
||||
"emotion": "愉悦",
|
||||
"duration_sec": 8,
|
||||
"camera": "close_up",
|
||||
"transition": "zoom_in",
|
||||
})
|
||||
ScriptBuilder.add_segment(script, {
|
||||
"scene": "傍晚的天台",
|
||||
"description": "夕阳西下，天边的云彩染成金色，她看着刚完成的画",
|
||||
"dialogue": "（旁白）有时候，最简单的快乐就在画笔下。",
|
||||
"emotion": "满足",
|
||||
"duration_sec": 10,
|
||||
"camera": "wide_shot",
|
||||
"transition": "dissolve",
|
||||
})
|
||||
|
||||
errors = validate_script(script)
|
||||
print(f" 脚本标题: {script['meta']['title']}")
|
||||
print(f" 分镜数: {len(script['segments'])} 个")
|
||||
print(f" 总时长: {script['meta']['duration_sec']}s")
|
||||
print(f" 验证结果: {'✅ 通过' if not errors else '❌ 有错误: ' + str(errors)}")
|
||||
|
||||
# ── 创建生产任务 ──
|
||||
prod = pc.create("文字灵感", "小漫的一天", "小漫的一天", "小漫", "manhua")
|
||||
pc.set_script(prod.id, script)
|
||||
print(f" 生产任务: {prod.id}")
|
||||
|
||||
# ── Gate 1: 脚本方向审核 ──
|
||||
print_separator("🅰️ 第 2 步: Gate 1 — 脚本方向审核")
|
||||
result = pc.submit_gate(prod.id, gate=1)
|
||||
s = result.get("summary", {})
|
||||
|
||||
print(f" 状态: {result['state']}")
|
||||
print(f" ─────────────────────────────────")
|
||||
print(f" 标题: {s.get('title', '')}")
|
||||
print(f" 来源: {s.get('source', '')}")
|
||||
print(f" 角色: {s.get('character', '')}")
|
||||
print(f" 总时长: {s.get('total_duration', 0)}s ({s.get('segment_count', 0)} 个分镜)")
|
||||
print(f" 结构:")
|
||||
for sc in s.get("structure", []):
|
||||
print(f" #{sc['id']} {sc['scene']} ({sc['duration']}s, {sc['emotion']})")
|
||||
print(f" 情绪分布: {s.get('emotion_distribution', {})}")
|
||||
print(f" ─────────────────────────────────")
|
||||
print(f" 📌 等待你审核: 脚本方向、风格、节奏")
|
||||
print(f" ✅ approve → ❌ reject → 输入反馈")
|
||||
print()
|
||||
|
||||
# 模拟通过
|
||||
print(" >> 模拟: 审核通过 ✅")
|
||||
pc.approve_gate(prod.id, gate=1)
|
||||
|
||||
# ── 角色适配 ──
|
||||
print_separator("🔧 第 3 步: 角色适配 (核心)")
|
||||
|
||||
# 预览适配效果
|
||||
preview = adapter.preview_adaptation(script)
|
||||
print(f" 适配角色: {preview['character']}")
|
||||
print(f" 角色描述: 短发齐刘海、大眼睛深棕色瞳孔、白连衣裙红蝴蝶结")
|
||||
print()
|
||||
for b, a in zip(preview["before"], preview["after"]):
|
||||
print(f" #{b['id']} {b['scene']}")
|
||||
print(f" 描述: {b['description'][:40]}...")
|
||||
print(f" → {a['description'][:40]}...")
|
||||
print()
|
||||
|
||||
# 执行适配
|
||||
adapted_storyboard = adapter.adapt_to_storyboard(script)
|
||||
pc.set_adapted_storyboard(prod.id, adapted_storyboard)
|
||||
print(f" ✅ 适配完成: {len(adapted_storyboard)} 个分镜已注入角色信息")
|
||||
|
||||
# ── Gate 2: 分镜预览审核 ──
|
||||
print_separator("🅱️ 第 4 步: Gate 2 — 分镜预览审核")
|
||||
result = pc.submit_gate(prod.id, gate=2)
|
||||
|
||||
print(f" 状态: {result['state']}")
|
||||
print(f" ─────────────────────────────────")
|
||||
print(f" {'#':>3} | {'场景':12s} | {'时长':4s} | {'情绪':8s} | {'镜头':12s} | {'视觉风格':8s}")
|
||||
print(f" ─────┼─────────────┼──────┼──────────┼──────────────┼──────────")
|
||||
for seg in result.get("storyboard", []):
|
||||
dur = seg.get('duration', seg.get('duration_sec', 0))
|
||||
print(f" {seg['id']:3d} | {seg['scene']:12s} | {dur:3d}s | {seg['emotion']:8s} | {seg['camera']:12s} | {seg.get('visual_style', 'manhua'):8s}")
|
||||
print(f" ─────────────────────────────────")
|
||||
print(f" 📌 等待你审核: 逐分镜检查场景/情绪/视觉风格")
|
||||
print(f" ✅ 全部通过 → ❌ 打回(说明意见)")
|
||||
print()
|
||||
|
||||
# 模拟通过
|
||||
print(" >> 模拟: 审核通过 ✅")
|
||||
pc.approve_gate(prod.id, gate=2)
|
||||
|
||||
# ── BGM 选择 ──
|
||||
print_separator("🎵 第 5 步: BGM 选择")
|
||||
print(" 🔥 热门推荐:")
|
||||
for bgm_entry in bgm.get_trending(3):
|
||||
print(f" {bgm_entry['id']:15s} | {bgm_entry['name']:10s} | {bgm_entry['artist']:10s} | {bgm_entry['bpm']}bpm | {bgm_entry['style']}")
|
||||
print()
|
||||
print(" 📂 按风格浏览:")
|
||||
for style in bgm.list_styles():
|
||||
entries = bgm.get_by_style(style)
|
||||
print(f" {style}: {', '.join(e['name'] for e in entries)}")
|
||||
|
||||
selected_bgm = "向云端"
|
||||
bgm_config = bgm.generate_config(selected_bgm, beat_sync=False, ducking=True, volume=0.6)
|
||||
pc.set_bgm(prod.id, selected_bgm, bgm_config)
|
||||
print(f"\n 🎯 选择: {selected_bgm}")
|
||||
print(f" 音量: {bgm_config['volume']}, 人声避让: {bgm_config['ducking']}, BPM: {bgm_config['bpm']}")
|
||||
|
||||
# ── Gate 3: 费用确认 ──
|
||||
print_separator("💰 第 6 步: Gate 3 — 费用确认")
|
||||
result = pc.submit_gate(prod.id, gate=3)
|
||||
cost = result.get("cost", {})
|
||||
|
||||
print(f" 状态: {result['state']}")
|
||||
print(f" ─────────────────────────────────")
|
||||
print(f" 路径: {cost.get('path_type', 'manhua')}")
|
||||
print(f" 分镜数: {cost.get('image_count', 0)}")
|
||||
print(f" ─────────────────────────────────")
|
||||
print(f" 图片生成: ¥{cost.get('image_cost', 0):.2f}")
|
||||
print(f" 视频生成: ¥{cost.get('video_cost', 0):.2f}")
|
||||
print(f" TTS 费用: ¥{cost.get('tts_cost', 0):.2f}")
|
||||
print(f" BGM 费用: ¥{cost.get('bgm_cost', 0):.2f}")
|
||||
print(f" ─────────────────────────────────")
|
||||
print(f" ══ 总计: ¥{cost.get('total', 0):.2f} ══")
|
||||
print(f" ─────────────────────────────────")
|
||||
print(f" 📌 等待你确认: 费用明细确认")
|
||||
print(f" ✅ 确认并执行 → ❌ 取消 → 回到分镜调整")
|
||||
print()
|
||||
|
||||
# 模拟确认
|
||||
print(" >> 模拟: 费用确认 ✅")
|
||||
pc.confirm_cost(prod.id)
|
||||
|
||||
# ── 开始生成 ──
|
||||
print_separator("🎬 第 7 步: 开始生产")
|
||||
pc.start_generation(prod.id)
|
||||
print(f" 状态: {pc.get(prod.id).state.value}")
|
||||
|
||||
# 模拟完成
|
||||
pc.complete_step(prod.id, "generation")
|
||||
pc.complete_step(prod.id, "composition")
|
||||
|
||||
# ── 最终状态 ──
|
||||
print_separator("🏁 生产完成")
|
||||
final = pc.get(prod.id)
|
||||
print(f" 任务 ID: {final.id}")
|
||||
print(f" 标题: {final.title}")
|
||||
print(f" 状态: {final.state.value} ✅")
|
||||
print(f" 角色: {final.character}")
|
||||
print(f" BGM: {final.bgm_id}")
|
||||
print(f" 创建: {final.created_at}")
|
||||
print(f" 完成: {final.completed_at}")
|
||||
print(f" 成本: ¥{final.cost.total:.2f}")
|
||||
|
||||
# ── 审核打回演示 ──
|
||||
print_separator("🔄 额外演示: 审核打回流程")
|
||||
prod2 = pc.create("文字灵感", "测试", "测试打回", "小漫", "manhua")
|
||||
pc.set_script(prod2.id, script)
|
||||
pc.submit_gate(prod2.id, gate=1)
|
||||
pc.reject_gate(prod2.id, gate=1, feedback="故事节奏太慢，需要增加冲突桥段")
|
||||
print(f" 任务: {prod2.id}")
|
||||
print(f" 状态: {pc.get(prod2.id).state.value}")
|
||||
print(f" 反馈: {pc.get(prod2.id).gate1_feedback}")
|
||||
print()
|
||||
print(f" 你可以:")
|
||||
print(f" - 修改脚本后 resubmit (重提审核)")
|
||||
print(f" - 取消任务 cancel")
|
||||
|
||||
# ── 清理测试数据 ──
|
||||
for pid in [prod.id, prod2.id]:
|
||||
path = pc._data_dir / f"{pid}.json"
|
||||
if path.exists():
|
||||
path.unlink()
|
||||
print()
|
||||
print("=" * 70)
|
||||
print(" ✅ 全链路演示完成!")
|
||||
print(" 📍 所有模块已通过串行测试")
|
||||
print("=" * 70)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,192 @@
|
||||
"""
|
||||
|
||||
AVE 01_director_parser — 文案 → 导演脚本 YAML
|
||||
|
||||
流程:
|
||||
1. 读取文案文本
|
||||
2. 调用本地 oMLX (Qwen3.5) 解析 → 结构化 segments
|
||||
3. 调用 schemas.py 校验
|
||||
4. 输出 director_script.yaml
|
||||
|
||||
本地 LLM: http://localhost:8000/v1/chat/completions
|
||||
"""
|
||||
import sys, os; sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import json
|
||||
import os
|
||||
import yaml
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from lib.logger import get_logger
|
||||
from .schemas import DirectorScript, validate_script
|
||||
|
||||
logger = get_logger("director_parser")
|
||||
|
||||
# 本地 oMLX 配置
|
||||
OMLX_URL = "http://localhost:8000/v1/chat/completions"
|
||||
OMLX_MODEL = "Qwen3.5-4B-MLX-4bit"
|
||||
OMLX_API_KEY = "omlx"
|
||||
|
||||
|
||||
def parse_script(
|
||||
script_path: str,
|
||||
style: str = "knowledge_lecture",
|
||||
output_path: str = "director_script.yaml",
|
||||
use_llm: bool = True,
|
||||
) -> str:
|
||||
"""
|
||||
文案 → 导演脚本 YAML
|
||||
|
||||
- script_path: 文案文件路径 (.txt)
|
||||
- style: 视频风格 (knowledge_lecture | bedtime_story | funny_talk | tech_review)
|
||||
- output_path: 输出的 YAML 路径
|
||||
- use_llm: 是否使用本地 LLM 解析 (False = 手动模板)
|
||||
|
||||
返回: 生成的 YAML 路径
|
||||
"""
|
||||
if not os.path.exists(script_path):
|
||||
raise FileNotFoundError(f"文案文件不存在: {script_path}")
|
||||
|
||||
with open(script_path, "r", encoding="utf-8") as f:
|
||||
script_text = f.read().strip()
|
||||
|
||||
if not script_text:
|
||||
raise ValueError(f"文案内容为空: {script_path}")
|
||||
|
||||
if use_llm:
|
||||
# 用本地 oMLX 解析
|
||||
script_data = _parse_with_llm(script_text, style)
|
||||
else:
|
||||
# 手动模板: 整段文案作为单 segment
|
||||
script_data = _manual_parse(script_text, style)
|
||||
|
||||
# Pydantic 校验
|
||||
validated = validate_script(script_data)
|
||||
|
||||
# 输出 YAML
|
||||
with open(output_path, "w", encoding="utf-8") as f:
|
||||
yaml.dump(
|
||||
validated.model_dump(mode="json"),
|
||||
f,
|
||||
allow_unicode=True,
|
||||
default_flow_style=False,
|
||||
sort_keys=False,
|
||||
)
|
||||
|
||||
logger.info(f"导演脚本已生成: {output_path} ({len(validated.segments)} 段)")
|
||||
return output_path
|
||||
|
||||
|
||||
def _parse_with_llm(script_text: str, style: str) -> dict:
|
||||
"""调用本地 oMLX 解析文案为结构化数据"""
|
||||
import httpx
|
||||
|
||||
prompt = f"""你是一个视频导演脚本生成器。将以下文案解析为结构化导演脚本。
|
||||
|
||||
风格: {style}
|
||||
|
||||
要求:
|
||||
1. 按自然语义停顿分段 (每段6-30秒)
|
||||
2. 为每段分配情绪描述
|
||||
3. 为每段设计视觉素材搜索词 (中文关键词)
|
||||
4. 为每段指定 BGM 段落 (intro/main/climax/outro)
|
||||
5. BGM结构: intro(0-8s) → main(8-50s) → climax(50-65s) → outro(65-75s)
|
||||
6. 输出纯 JSON，不要 markdown 包裹
|
||||
|
||||
文案:
|
||||
{script_text}
|
||||
|
||||
输出 JSON 格式:
|
||||
{{
|
||||
"meta": {{
|
||||
"style": "{style}",
|
||||
"voice": {{"provider": "volcano", "voice_id": "default", "default_emotion": "专业沉稳"}},
|
||||
"bgm": {{
|
||||
"genre": "cinematic", "tempo": 110, "mood": "inspirational",
|
||||
"structure": [{{"section": "intro"}}, {{"section": "main"}}, {{"section": "climax"}}, {{"section": "outro"}}]
|
||||
}},
|
||||
"output": {{"resolution": "1080x1920", "fps": 30}}
|
||||
}},
|
||||
"segments": [
|
||||
{{
|
||||
"id": 1,
|
||||
"text": "分段文案",
|
||||
"duration_sec": 8,
|
||||
"voice_emotion": "开场悬念，压低声音",
|
||||
"camera": "slow_zoom_in",
|
||||
"bgm_section": "intro",
|
||||
"material": {{"source": "pexels", "search": "中文搜索关键词"}},
|
||||
"subtitles": true
|
||||
}}
|
||||
]
|
||||
}}"""
|
||||
|
||||
try:
|
||||
resp = httpx.post(
|
||||
OMLX_URL,
|
||||
json={
|
||||
"model": OMLX_MODEL,
|
||||
"messages": [{"role": "user", "content": prompt}],
|
||||
"temperature": 0.3,
|
||||
"max_tokens": 4096,
|
||||
},
|
||||
headers={"Authorization": f"Bearer {OMLX_API_KEY}"},
|
||||
timeout=120,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
result = resp.json()
|
||||
content = result["choices"][0]["message"]["content"]
|
||||
|
||||
# 清理可能的 markdown 包裹
|
||||
content = content.strip()
|
||||
if content.startswith("```"):
|
||||
content = content.split("\n", 1)[1]
|
||||
content = content.rsplit("```", 1)[0]
|
||||
if content.startswith("json"):
|
||||
content = content[4:]
|
||||
|
||||
data = json.loads(content)
|
||||
data["meta"]["style"] = style
|
||||
logger.info(f"oMLX 解析完成: {len(data.get('segments', []))} 段")
|
||||
return data
|
||||
|
||||
except httpx.ConnectError:
|
||||
logger.warning("oMLX 未运行，降级到手动模板模式")
|
||||
return _manual_parse(script_text, style)
|
||||
except (json.JSONDecodeError, KeyError) as e:
|
||||
logger.warning(f"LLM 解析失败 ({e})，降级到手动模板")
|
||||
return _manual_parse(script_text, style)
|
||||
|
||||
|
||||
def _manual_parse(script_text: str, style: str) -> dict:
|
||||
"""手动模板: 整段文案作为一个 segment（降级方案）"""
|
||||
return {
|
||||
"meta": {
|
||||
"style": style,
|
||||
"voice": {"provider": "volcano", "voice_id": "default", "default_emotion": "专业沉稳"},
|
||||
"bgm": {
|
||||
"genre": "cinematic" if style != "funny_talk" else "upbeat",
|
||||
"tempo": 110,
|
||||
"mood": "inspirational",
|
||||
"structure": [
|
||||
{"section": "intro"},
|
||||
{"section": "main"},
|
||||
{"section": "climax"},
|
||||
{"section": "outro"},
|
||||
],
|
||||
},
|
||||
"output": {"resolution": "1080x1920", "fps": 30},
|
||||
},
|
||||
"segments": [
|
||||
{
|
||||
"id": 1,
|
||||
"text": script_text,
|
||||
"duration_sec": max(15, min(len(script_text) * 0.2, 120)),
|
||||
"voice_emotion": "正常讲述",
|
||||
"camera": "static",
|
||||
"bgm_section": "main",
|
||||
"material": {"source": "pexels", "search": style},
|
||||
"subtitles": True,
|
||||
}
|
||||
],
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
"""
|
||||
|
||||
AVE 01_director_parser — 导演脚本解析器
|
||||
|
||||
职责:
|
||||
1. schemas.py: YAML 结构定义 + Pydantic 校验
|
||||
2. parser.py: 文案 → director_script.yaml (调用本地 oMLX)
|
||||
|
||||
本地 LLM (oMLX) 路径:
|
||||
http://localhost:8000/v1/chat/completions
|
||||
模型: Qwen3.5-4B-MLX-4bit
|
||||
API Key: omlx
|
||||
"""
|
||||
import sys, os; sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from typing import Optional
|
||||
from enum import Enum
|
||||
|
||||
|
||||
class StyleEnum(str, Enum):
|
||||
knowledge_lecture = "knowledge_lecture"
|
||||
bedtime_story = "bedtime_story"
|
||||
funny_talk = "funny_talk"
|
||||
tech_review = "tech_review"
|
||||
|
||||
|
||||
class VoiceProvider(str, Enum):
|
||||
volcano = "volcano"
|
||||
aliyun = "aliyun"
|
||||
|
||||
|
||||
class BGMSection(BaseModel):
|
||||
section: str
|
||||
description: Optional[str] = None
|
||||
|
||||
|
||||
class BGMMeta(BaseModel):
|
||||
provider: str = "ace_step"
|
||||
genre: str = "cinematic"
|
||||
tempo: int = 110
|
||||
mood: str = "inspirational"
|
||||
structure: list[BGMSection] = []
|
||||
|
||||
@field_validator("tempo")
|
||||
@classmethod
|
||||
def tempo_range(cls, v):
|
||||
if v < 40 or v > 220:
|
||||
raise ValueError("tempo 应在 40-220 之间")
|
||||
return v
|
||||
|
||||
|
||||
class OutputMeta(BaseModel):
|
||||
resolution: str = "1080x1920"
|
||||
fps: int = 30
|
||||
|
||||
@field_validator("resolution")
|
||||
@classmethod
|
||||
def valid_resolution(cls, v):
|
||||
valid = ["1080x1920", "1920x1080", "720x1280", "1280x720"]
|
||||
if v not in valid:
|
||||
raise ValueError(f"分辨率必须是 {valid} 之一")
|
||||
return v
|
||||
|
||||
|
||||
class MaterialSource(str, Enum):
|
||||
pexels = "pexels"
|
||||
ai_generate = "ai_generate"
|
||||
local = "local"
|
||||
|
||||
|
||||
class MaterialConfig(BaseModel):
|
||||
source: MaterialSource = MaterialSource.pexels
|
||||
search: str = ""
|
||||
transition_in: str = "fade_in"
|
||||
transition_out: str = "dissolve"
|
||||
fallback: bool = True
|
||||
|
||||
|
||||
class CharacterRef(BaseModel):
|
||||
"""角色引用（定妆照锁定）"""
|
||||
name: str = Field(description="角色名")
|
||||
description: str = Field(description="角色描述块（跨场景一致性注入）")
|
||||
grid_path: Optional[str] = Field(default="", description="定妆照网格图路径")
|
||||
lip_sync: bool = Field(default=False, description="是否对角色视频做 LipSync")
|
||||
|
||||
|
||||
class AvatarConfig(BaseModel):
|
||||
image_url: Optional[str] = None
|
||||
provider: str = "wan2.2_s2v"
|
||||
default_position: str = "bottom_right"
|
||||
|
||||
|
||||
class Segment(BaseModel):
|
||||
id: int = Field(ge=1)
|
||||
text: str = Field(min_length=1)
|
||||
duration_sec: float = Field(default=10, ge=1, le=300)
|
||||
voice_emotion: str = "正常讲述"
|
||||
camera: str = "static"
|
||||
bgm_section: str = "main"
|
||||
avatar: Optional[dict] = None
|
||||
material: MaterialConfig = MaterialConfig()
|
||||
subtitles: bool = True
|
||||
character_ref: Optional[str] = Field(default=None, description="引用角色名（对应 meta.character_refs）")
|
||||
|
||||
|
||||
class VoiceMeta(BaseModel):
|
||||
provider: VoiceProvider = VoiceProvider.volcano
|
||||
voice_id: str = "default"
|
||||
default_emotion: str = "专业沉稳"
|
||||
|
||||
|
||||
class Meta(BaseModel):
|
||||
project_id: str = ""
|
||||
style: StyleEnum = StyleEnum.knowledge_lecture
|
||||
voice: VoiceMeta = VoiceMeta()
|
||||
bgm: BGMMeta = BGMMeta()
|
||||
output: OutputMeta = OutputMeta()
|
||||
avatar: Optional[AvatarConfig] = None
|
||||
material: dict = {"fallback": True}
|
||||
character_refs: list[CharacterRef] = Field(default=[], description="角色引用列表（定妆照一致性）")
|
||||
|
||||
|
||||
class DirectorScript(BaseModel):
|
||||
"""完整导演脚本"""
|
||||
meta: Meta = Meta()
|
||||
segments: list[Segment] = Field(min_length=1)
|
||||
|
||||
|
||||
def validate_script(data: dict) -> DirectorScript:
|
||||
"""校验并返回结构化的导演脚本"""
|
||||
return DirectorScript.model_validate(data)
|
||||
@@ -0,0 +1,74 @@
|
||||
meta:
|
||||
project_id: ''
|
||||
style: knowledge_lecture
|
||||
voice:
|
||||
provider: volcano
|
||||
voice_id: default
|
||||
default_emotion: 专业沉稳
|
||||
bgm:
|
||||
provider: ace_step
|
||||
genre: cinematic
|
||||
tempo: 110
|
||||
mood: inspirational
|
||||
structure:
|
||||
- section: intro
|
||||
description: null
|
||||
- section: main
|
||||
description: null
|
||||
- section: climax
|
||||
description: null
|
||||
- section: outro
|
||||
description: null
|
||||
output:
|
||||
resolution: 1080x1920
|
||||
fps: 30
|
||||
avatar: null
|
||||
material:
|
||||
fallback: true
|
||||
segments:
|
||||
- id: 1
|
||||
text: '# AVE 演示脚本：一个小笑话
|
||||
|
||||
# 用于验证 MVP 链路 (文案→人声→素材→合成)
|
||||
|
||||
|
||||
标题: "一只会打太极的猫"
|
||||
|
||||
|
||||
今天我要讲一个真实的故事。
|
||||
|
||||
|
||||
我家养了一只橘猫，名字叫"大橘"。
|
||||
|
||||
|
||||
大橘有个奇怪的爱好——每天早上六点，它会准时蹲在阳台的瑜伽垫上，开始打太极。
|
||||
|
||||
|
||||
你没听错，打太极。
|
||||
|
||||
|
||||
一开始我以为它在伸懒腰，后来发现不对劲。它的动作实在太标准了——野马分鬃、白鹤亮翅、搂膝拗步，一套下来行云流水。
|
||||
|
||||
|
||||
更离谱的是，如果你在它练功的时候打扰它，它会瞪你一眼，然后换个地方继续打。
|
||||
|
||||
|
||||
后来我调了监控才发现，它是在偷看我奶奶晨练，看了一年，学会了。
|
||||
|
||||
|
||||
现在大橘在我们小区已经出名了，每天早上都有人专门来看它打太极。
|
||||
|
||||
|
||||
所以下次有人说"猫不如狗聪明"，你就告诉他：有一种猫，比你还会养生。'
|
||||
duration_sec: 68.8
|
||||
voice_emotion: 正常讲述
|
||||
camera: static
|
||||
bgm_section: main
|
||||
avatar: null
|
||||
material:
|
||||
source: pexels
|
||||
search: knowledge_lecture
|
||||
transition_in: fade_in
|
||||
transition_out: dissolve
|
||||
fallback: true
|
||||
subtitles: true
|
||||
@@ -0,0 +1,75 @@
|
||||
"""
|
||||
directors — 导演系统注册表（替换点①）
|
||||
========================================
|
||||
|
||||
统一入口：任何导演系统在此注册，下游通过 `get_director(id)` 取用。
|
||||
|
||||
接入新导演（3 步）：
|
||||
1. directors/<新导演>.py 实现 BaseDirector
|
||||
2. 本文件 import + 加进 _DIRECTORS
|
||||
3. 刷新前端，「🎬 导演创作」页的下拉里就会出现
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
from directors.base import BaseDirector # noqa: E402,F401
|
||||
from directors.legacy_scene_planner import LegacyScenePlanner # noqa: E402
|
||||
|
||||
log = logging.getLogger("directors")
|
||||
|
||||
# ── 已注册的导演（后续接入的在此追加）──────────────────
|
||||
_DIRECTORS: list[BaseDirector] = [LegacyScenePlanner()]
|
||||
|
||||
# v5.2（ai-film-skills）：技能文档就位后自动注册
|
||||
try:
|
||||
from directors.storyboard_v52 import StoryboardV52
|
||||
_DIRECTORS.append(StoryboardV52())
|
||||
except Exception as _e: # 技能文档未下载 / 模块未就绪
|
||||
log.debug(" storyboard_v52 未注册: %s", _e)
|
||||
|
||||
# Seedance 导演方法论（精编移植，skill 文档就位后自动注册）
|
||||
try:
|
||||
from directors.seedance_director import SeedanceDirector
|
||||
_DIRECTORS.append(SeedanceDirector())
|
||||
except Exception as _e:
|
||||
log.debug(" seedance_director 未注册: %s", _e)
|
||||
|
||||
|
||||
def all_directors() -> list[BaseDirector]:
|
||||
return list(_DIRECTORS)
|
||||
|
||||
|
||||
def get_director(director_id: str) -> BaseDirector | None:
|
||||
for d in _DIRECTORS:
|
||||
if d.id == director_id:
|
||||
return d
|
||||
return None
|
||||
|
||||
|
||||
def default_director() -> BaseDirector:
|
||||
"""默认导演（第一个 ready 的）"""
|
||||
for d in _DIRECTORS:
|
||||
if d.status == "ready" and getattr(d, "available", True):
|
||||
return d
|
||||
return _DIRECTORS[0]
|
||||
|
||||
|
||||
def list_info() -> list[dict]:
|
||||
"""给前端：全部导演 + 可用性"""
|
||||
out = []
|
||||
for d in _DIRECTORS:
|
||||
info = d.info()
|
||||
info["available"] = bool(getattr(d, "available", True))
|
||||
out.append(info)
|
||||
return out
|
||||
|
||||
|
||||
__all__ = ["BaseDirector", "LegacyScenePlanner", "all_directors",
|
||||
"get_director", "default_director", "list_info"]
|
||||
@@ -0,0 +1,163 @@
|
||||
"""
|
||||
directors._shot_builder — 公共镜头构建器（各 skill 型导演共用）
|
||||
|
||||
职责：把 LLM 产出的 JSON 数组 → 统一的 Shot 列表（含完整镜头契约/表演/声音/技术需求）。
|
||||
被 storyboard_v52（v5.2 技能）与 seedance_director（Seedance 方法论）共用，
|
||||
保证不同导演产出同一套 Schema，下游无感知。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
from schemas import (GlobalAnchors, Shot, ShotContract, # noqa: E402
|
||||
Transition, Speed, AudioConfig, SFX, ExpressionDetail)
|
||||
|
||||
|
||||
def safe_dict(v) -> dict:
|
||||
return v if isinstance(v, dict) else {}
|
||||
|
||||
|
||||
def norm_type(t: Any) -> str:
|
||||
t = (str(t) if t else "").strip().lower()
|
||||
aliases = {"talk": "talking_head", "b_roll": "background", "broll": "background",
|
||||
"scene": "action", "environment": "background"}
|
||||
v = aliases.get(t, t or "action")
|
||||
return v if v in ("action", "talking_head", "background", "vfx", "comfyui", "3d_character") else "action"
|
||||
|
||||
|
||||
def parse_expression(it: dict) -> ExpressionDetail | None:
|
||||
e = safe_dict(it.get("expression_detail"))
|
||||
if not e:
|
||||
return None
|
||||
eyes = (e.get("eyes") or "").strip()
|
||||
mouth = (e.get("mouth") or "").strip()
|
||||
brow = (e.get("brow") or "").strip()
|
||||
if not (eyes or mouth or brow):
|
||||
return None
|
||||
return ExpressionDetail(eyes=eyes or None, mouth=mouth or None, brow=brow or None)
|
||||
|
||||
|
||||
def parse_audio(it: dict) -> AudioConfig | None:
|
||||
cue = (it.get("audio_cue") or "").strip()
|
||||
sfx_raw = it.get("sfx") or []
|
||||
sfx: list[SFX] = []
|
||||
if isinstance(sfx_raw, list):
|
||||
for i, s in enumerate(sfx_raw, 1):
|
||||
d = safe_dict(s)
|
||||
if not d:
|
||||
continue
|
||||
try:
|
||||
at = float(d.get("at_second") or 0)
|
||||
except Exception:
|
||||
at = 0.0
|
||||
sfx.append(SFX(id=f"sfx_{i}", at_second=at,
|
||||
volume=float(d.get("volume") or 0.4),
|
||||
desc=(d.get("desc") or "").strip()))
|
||||
if not cue and not sfx:
|
||||
return None
|
||||
return AudioConfig(audio_cue=cue or None, sfx=sfx)
|
||||
|
||||
|
||||
def parse_transition(v) -> Transition:
|
||||
d = safe_dict(v)
|
||||
t = (d.get("type") or "cut").strip().lower()
|
||||
if t not in ("cut", "dissolve", "fade", "wipe"):
|
||||
t = "cut"
|
||||
try:
|
||||
dur = float(d.get("duration") or 0)
|
||||
except Exception:
|
||||
dur = 0.0
|
||||
return Transition(type=t, duration=dur)
|
||||
|
||||
|
||||
def parse_speed(v) -> Speed:
|
||||
d = safe_dict(v)
|
||||
try:
|
||||
f = float(d.get("factor") or 1.0)
|
||||
except Exception:
|
||||
f = 1.0
|
||||
ramp = (d.get("ramp") or "").strip()
|
||||
return Speed(factor=f, ramp=ramp or None)
|
||||
|
||||
|
||||
def parse_tech_requirements(it: dict) -> dict | None:
|
||||
"""技术需求声明（透传 + 基本校验），供技术优选层消费"""
|
||||
tr = it.get("tech_requirements")
|
||||
if not isinstance(tr, dict):
|
||||
return None
|
||||
cont = safe_dict(tr.get("continuity"))
|
||||
allowed = ("must", "prefer", "none")
|
||||
out = {
|
||||
"continuity": {k: (str(cont.get(k, "none")).strip().lower()
|
||||
if str(cont.get(k, "none")).strip().lower() in allowed else "none")
|
||||
for k in ("character", "wardrobe", "location", "expression", "props")},
|
||||
"chain_from_prev": bool(tr.get("chain_from_prev")),
|
||||
"motion_level": str(tr.get("motion_level") or "small").strip().lower(),
|
||||
"camera_difficulty": str(tr.get("camera_difficulty") or "fixed").strip().lower(),
|
||||
"needs": [str(n).strip() for n in (tr.get("needs") or []) if str(n).strip()],
|
||||
}
|
||||
if out["motion_level"] not in ("static", "small", "medium", "large"):
|
||||
out["motion_level"] = "small"
|
||||
if out["camera_difficulty"] not in ("fixed", "simple", "complex"):
|
||||
out["camera_difficulty"] = "fixed"
|
||||
return out
|
||||
|
||||
|
||||
def build_shots_from_items(items: list, anchors: GlobalAnchors,
|
||||
shot_id_prefix: str = "shot") -> list[Shot]:
|
||||
"""LLM JSON 数组 → Shot 列表（统一契约）
|
||||
|
||||
各导演只需产出相同字段结构的 JSON，本函数负责解析、归一化与容错。
|
||||
"""
|
||||
from directors.base import norm_duration
|
||||
shots: list[Shot] = []
|
||||
for i, it in enumerate(items, 1):
|
||||
if not isinstance(it, dict):
|
||||
continue
|
||||
text = (it.get("text") or "").strip()
|
||||
tech_req = parse_tech_requirements(it)
|
||||
chain = bool((tech_req or {}).get("chain_from_prev"))
|
||||
# 出场人物（戏内代号）：只保留锚点演员表里存在的，避免 AI 写不存在的代号
|
||||
_valid_slots = {m.slot for m in (anchors.effective_cast() if hasattr(anchors, "effective_cast") else [])}
|
||||
_raw_slots = it.get("cast_in_shot") or it.get("cast_slots") or []
|
||||
if isinstance(_raw_slots, str):
|
||||
_raw_slots = [_raw_slots]
|
||||
cast_slots = [str(x).strip() for x in _raw_slots
|
||||
if str(x).strip() and (not _valid_slots or str(x).strip() in _valid_slots)]
|
||||
|
||||
shots.append(Shot(
|
||||
shot_id=f"{shot_id_prefix}_{i:02d}",
|
||||
scene_id=str(i),
|
||||
duration=norm_duration(it.get("duration"), text),
|
||||
type=norm_type(it.get("type")),
|
||||
cast_slots=cast_slots,
|
||||
character_id=anchors.character_id,
|
||||
location_id=anchors.location_id,
|
||||
continuity_mode="previous_frame_continuation" if chain else "fresh_location",
|
||||
tech_requirements=tech_req,
|
||||
visual_description=(it.get("visual_description") or it.get("description") or "").strip(),
|
||||
shot_contract=ShotContract(
|
||||
shot_size=(it.get("shot_size") or "").strip(),
|
||||
angle=(it.get("angle") or "").strip(),
|
||||
camera_move=(it.get("camera_move") or "").strip(),
|
||||
lens_feel=(it.get("lens_feel") or "").strip(),
|
||||
depth_of_field=(it.get("depth_of_field") or "").strip(),
|
||||
frame_rate_feel=(it.get("frame_rate_feel") or "").strip(),
|
||||
),
|
||||
text=text or None,
|
||||
sentiment=(it.get("sentiment") or "").strip() or None,
|
||||
expression_detail=parse_expression(it),
|
||||
action=(it.get("action") or "").strip() or None,
|
||||
audio=parse_audio(it),
|
||||
transition_in=parse_transition(it.get("transition_in")),
|
||||
transition_out=parse_transition(it.get("transition_out")),
|
||||
speed=parse_speed(it.get("speed")),
|
||||
status="pending",
|
||||
))
|
||||
return shots
|
||||
@@ -0,0 +1,304 @@
|
||||
"""
|
||||
directors.base — 导演系统基类（替换点①）
|
||||
==========================================
|
||||
|
||||
任何导演系统只需实现 `generate(story, anchors, options) -> Storyboard`，
|
||||
即可接入视频工厂，下游（编排/能力/执行）完全不感知用的是哪个导演。
|
||||
|
||||
接入新导演（3 步）：
|
||||
1. 在 directors/ 加一个文件，实现 BaseDirector
|
||||
2. 在 directors/__init__.py 的 DIRECTORS 注册
|
||||
3. （可选）填 options_schema，前端自动生成该导演的选项面板
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
from schemas import GlobalAnchors, Storyboard # noqa: E402
|
||||
|
||||
|
||||
class BaseDirector:
|
||||
"""导演系统基类"""
|
||||
|
||||
id: str = ""
|
||||
name: str = ""
|
||||
kind: str = "skill" # skill（提示词包）| service（HTTP 服务）| legacy（本地既有）
|
||||
description: str = ""
|
||||
features: list[str] = [] # ["visual_concept", "two_pass", "consistency", "auto_pipeline"]
|
||||
status: str = "ready" # ready | partial | planned
|
||||
source: str = "" # 来源说明（如 ai-film-skills v5.2 / 本地既有）
|
||||
options_schema: dict = {} # 该导演的专属选项（自动生成 UI 表单）
|
||||
|
||||
def generate(
|
||||
self,
|
||||
story: str,
|
||||
anchors: GlobalAnchors | None = None,
|
||||
options: dict | None = None,
|
||||
) -> Storyboard:
|
||||
"""故事 → 统一分镜 Schema（**唯一契约**）"""
|
||||
raise NotImplementedError
|
||||
|
||||
def revise_shot(self, storyboard: Storyboard, shot_id: str,
|
||||
instruction: str = "") -> Storyboard:
|
||||
"""单镜头重生成（默认不支持，子类可覆写）"""
|
||||
raise NotImplementedError(f"{self.name} 不支持单镜头重生成")
|
||||
|
||||
def info(self) -> dict:
|
||||
"""给前端的能力描述"""
|
||||
return {
|
||||
"id": self.id, "name": self.name, "kind": self.kind,
|
||||
"description": self.description, "features": self.features,
|
||||
"status": self.status, "source": self.source,
|
||||
"options_schema": self.options_schema,
|
||||
}
|
||||
|
||||
|
||||
# 通用选项（各导演共享的基础参数）
|
||||
COMMON_OPTIONS = {
|
||||
# ★ 推荐用法：给「目标总时长」，镜头数与每镜时长交给导演自行决定
|
||||
"total_duration": {"type": "number", "min": 0, "max": 1800, "default": 0,
|
||||
"label": "目标总时长（秒）",
|
||||
"hint": "推荐用法：60 = 一分钟短片 · 180 = 三分钟 · 300 = 五分钟。"
|
||||
"导演据此自由决定镜头数与每镜时长（充分解放创作）。填 0 = 不限制"},
|
||||
# 以下两项**仅作参考**（导演可采纳可不采纳；填 0 = 不干涉）
|
||||
"shot_count": {"type": "number", "min": 0, "max": 30, "default": 0,
|
||||
"label": "镜头数（仅参考）",
|
||||
"hint": "仅作参考：导演可采纳也可不采纳。填 0 = 让导演按内容自行决定（推荐）"},
|
||||
"shot_duration": {"type": "number", "min": 0, "max": 15, "default": 0,
|
||||
"label": "单镜时长（仅参考）",
|
||||
"hint": "仅作参考：导演按内容与节奏自行决定。填 0 = 不限制"},
|
||||
"shooting_style": {
|
||||
"type": "enum",
|
||||
"options": ["documentary_realism", "cinematic", "vlog_handheld",
|
||||
"studio_clean", "selfie_pov", "interview_doc",
|
||||
"anime_2d", "anime_3d", "product_ad"],
|
||||
# 中文标签（前端显示，值保持英文以兼容后端）
|
||||
"option_labels": {
|
||||
"documentary_realism": "🎥 伪纪录片 · 真实自然（像随手拍的生活记录）",
|
||||
"cinematic": "🎞️ 电影感 · 考究构图（有设计感的画面）",
|
||||
"vlog_handheld": "🚶 手持 Vlog · 第一视角（边走边拍）",
|
||||
"studio_clean": "💡 商业棚拍 · 高清干净（打光讲究、背景简洁）",
|
||||
"selfie_pov": "📱 手机自拍 · 第一人称（举着手机自己拍自己）",
|
||||
"interview_doc": "📺 访谈纪实 · 半身对镜（像接受采访）",
|
||||
"anime_2d": "🎨 2D 动画 · 漫剧（手绘/动漫风格）",
|
||||
"anime_3d": "🧸 3D 动画 · 卡通渲染（三维风格化）",
|
||||
"product_ad": "🧊 产品广告 · 精致特写（微距/质感）",
|
||||
},
|
||||
"default": "documentary_realism", "label": "拍摄风格",
|
||||
},
|
||||
"realism_boost": {
|
||||
"type": "enum",
|
||||
"options": ["off", "mild", "strong"],
|
||||
"option_labels": {
|
||||
"off": "关闭（不做额外处理）",
|
||||
"mild": "轻度（追加真实质感：噪点/毛孔/不匀光）",
|
||||
"strong": "强力（↑ 并自动剔除「4K/超清/无瑕/大师作品」等反真实词）",
|
||||
},
|
||||
"default": "mild", "label": "反 AI 感（真实感增强）",
|
||||
},
|
||||
"director_style": {
|
||||
"type": "enum",
|
||||
"options": ["", "spielberg", "hitchcock", "kubrick", "kurosawa", "scorsese",
|
||||
"fellini", "bergman", "tarkovsky", "wong_kar_wai", "nolan",
|
||||
"villeneuve", "fincher", "refn", "bi_gan", "zhang_yimou",
|
||||
"hou_hsiao_hsien", "park_chan_wook", "malick",
|
||||
"michael_mann", "coen_brothers"],
|
||||
"option_labels": {
|
||||
"": "（不使用导演风格）",
|
||||
"spielberg": "斯皮尔伯格 · 奇观与亲密", "hitchcock": "希区柯克 · 悬念与信息差",
|
||||
"kubrick": "库布里克 · 几何与秩序", "kurosawa": "黑泽明 · 群像与天气",
|
||||
"scorsese": "斯科塞斯 · 都市与旁白", "fellini": "费里尼 · 狂欢与记忆",
|
||||
"bergman": "伯格曼 · 凝视与沉默", "tarkovsky": "塔可夫斯基 · 雕刻时间",
|
||||
"wong_kar_wai": "王家卫 · 都市孤独", "nolan": "诺兰 · 高概念谜题",
|
||||
"villeneuve": "维伦纽瓦 · 宏大克制", "fincher": "芬奇 · 外科精确",
|
||||
"refn": "雷弗恩 · 霓虹停滞", "bi_gan": "毕赣 · 梦境长镜",
|
||||
"zhang_yimou": "张艺谋 · 单色仪式", "hou_hsiao_hsien": "侯孝贤 · 固定远观",
|
||||
"park_chan_wook": "朴赞郁 · 巴洛克反讽", "malick": "马力克 · 自然光诗意",
|
||||
"michael_mann": "迈克尔·曼 · 夜色与专业", "coen_brothers": "科恩兄弟 · 冷峻荒诞",
|
||||
},
|
||||
"default": "", "label": "🎥 导演风格（镜头语言与叙事逻辑）",
|
||||
},
|
||||
"lang": {"type": "enum", "options": ["zh", "en"], "default": "zh", "label": "提示词语言"},
|
||||
}
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 公共工具：时长归一化（各导演共用，保证内容驱动语义一致）
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
def shot_brief(opts: dict) -> str:
|
||||
"""把「目标总时长 / 镜头数 / 单镜时长」组装成一句给导演的说明
|
||||
|
||||
设计原则（用户 2026-10-04 明确）：
|
||||
· **总时长 = 主驱动**（"一分钟短片"/"五分钟短片"）
|
||||
· **镜头数与单镜时长仅作参考** —— 导演可采纳可不采纳
|
||||
· 硬约束会削弱创作有效性 → 一律用"参考"语气，不写"必须"
|
||||
|
||||
让导演按题材/内容/目的自由裁镜头数，短片长篇皆可。
|
||||
"""
|
||||
def _int(v):
|
||||
try:
|
||||
return int(float(v or 0))
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
total = _int((opts or {}).get("total_duration"))
|
||||
n = _int((opts or {}).get("shot_count"))
|
||||
d = _int((opts or {}).get("shot_duration"))
|
||||
|
||||
out = ""
|
||||
if total:
|
||||
out = f"目标总时长：约 {total} 秒（{total / 60:.1f} 分钟）。"
|
||||
refs = []
|
||||
if n:
|
||||
refs.append(f"约 {n} 个镜头")
|
||||
if d:
|
||||
refs.append(f"每镜约 {d} 秒")
|
||||
if refs:
|
||||
out += (f"参考：{'、'.join(refs)} —— 可采纳可不采纳，"
|
||||
f"请按内容与叙事节奏自行决定要分多少个镜头、每个镜头多长。")
|
||||
return out or "镜头数量与每镜时长由你按内容与叙事节奏自由决定。"
|
||||
|
||||
|
||||
def norm_duration(d, text: str = "") -> float:
|
||||
"""时长归一化：内容驱动 + 台词兜底 + 上下限（2~10 秒）
|
||||
|
||||
· 台词兜底：中文口播约 4.5 字/秒 → 时长至少能容纳台词
|
||||
· 上限 10 秒（生成模型单镜约束；超长需接续能力，由技术优选层识别）
|
||||
"""
|
||||
try:
|
||||
v = float(d)
|
||||
except Exception:
|
||||
v = 5.0
|
||||
if text:
|
||||
need = len(str(text).strip()) / 4.5
|
||||
if need > v:
|
||||
v = need
|
||||
return round(max(2.0, min(10.0, v)), 1)
|
||||
|
||||
|
||||
def cast_or_character(anchors) -> str:
|
||||
"""角色/演员表 → 提示词文本（**演员表优先**，含出场标注要求）
|
||||
|
||||
用户 2026-10-03 要求：短片常有多人物（主角/配角/人物一），
|
||||
演员表（cast）比单角色字段更贴合；逐镜头出场由 cast_in_shot 标注。
|
||||
"""
|
||||
cc = getattr(anchors, "cast_context", "") or ""
|
||||
if cc:
|
||||
return (cc + '\n（多人物戏：请在每个镜头的 JSON 里用 "cast_in_shot": ["主角"] '
|
||||
'标注谁出场；代号只能取自上面【】里的）')
|
||||
return (getattr(anchors, "character_context", "") or getattr(anchors, "character_id", "")
|
||||
or "（未指定）")
|
||||
|
||||
|
||||
def style_pack_block(anchors) -> str:
|
||||
"""拍摄风格包 → 导演可执行的「拍摄说明」块
|
||||
|
||||
作用：让导演在写分镜时就知道"这戏按什么风格拍"（设备/光/机位/质感/情绪），
|
||||
从而产出风格一致的镜头语言，而不是每部片都写一样的通用描述。
|
||||
"""
|
||||
try:
|
||||
from composer.realism import STYLE_PACKS
|
||||
except Exception:
|
||||
return ""
|
||||
style = (getattr(anchors, "shooting_style", "") or "").strip()
|
||||
pack = STYLE_PACKS.get(style) or {}
|
||||
if not pack:
|
||||
return ""
|
||||
lines = [
|
||||
f"【本次拍摄风格：{pack.get('label', style)}】",
|
||||
f"· 设备：{pack.get('device', '')}",
|
||||
f"· 光线：{pack.get('lighting', '')}",
|
||||
f"· 机位与运镜：{pack.get('camera', '')}",
|
||||
f"· 质感与瑕疵：{pack.get('flaws', '')}",
|
||||
f"· 情绪表达：{pack.get('emotion', '')}",
|
||||
"→ 每个镜头的画面描述都要符合以上设定；不要写与风格冲突的元素。",
|
||||
]
|
||||
boost = (getattr(anchors, "realism_boost", "") or "").strip()
|
||||
if boost == "strong":
|
||||
lines.append("→ 真实感要求高：避免「完美/无瑕/超清」这类修饰，多写生活化的不完美细节。")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def director_style_hint(anchors) -> str:
|
||||
"""导演风格 → 提示词块（空 key 返回空串）
|
||||
|
||||
与 style_pack_block（拍摄风格）**互补**：
|
||||
· 拍摄风格 = 用什么设备/什么质感拍（自拍/手持/电影感…）
|
||||
· 导演风格 = 用什么镜头语言/叙事逻辑（斯皮尔伯格/王家卫…）
|
||||
"""
|
||||
key = (getattr(anchors, "director_style", "") or "").strip()
|
||||
if not key:
|
||||
return ""
|
||||
try:
|
||||
from composer.director_styles import director_style_block
|
||||
return director_style_block(key)
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
def realism_block() -> str:
|
||||
"""「真实感纪律」提示块（注入导演提示词）
|
||||
|
||||
来源：anti-ai-checklist.md（外部开源研究，原文存 docs/realism/）
|
||||
内容：非完美事件分配 + 2 秒节奏 + 禁止慢动作 + 抽象词黑名单
|
||||
"""
|
||||
try:
|
||||
from composer.realism import imperfect_events_prompt
|
||||
return imperfect_events_prompt()
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
# ── 拍摄风格 → 设备缺陷包映射（让导演按真实设备的物理缺陷描述画面）──
|
||||
_DEVICE_MAP = {
|
||||
"selfie_pov": "smartphone_vertical",
|
||||
"vlog_handheld": "smartphone_vertical",
|
||||
"documentary_realism": "dv_2000s",
|
||||
"interview_doc": "doc_handheld",
|
||||
# cinematic / studio_clean：刻意干净，不套缺陷包
|
||||
}
|
||||
|
||||
|
||||
def device_pack_hint(anchors) -> str:
|
||||
"""拍摄风格 → 设备缺陷参数包（研究落地：缺陷即真实）
|
||||
|
||||
来源：camera-aesthetics.md（9 个设备包，原文存 docs/realism/）
|
||||
"""
|
||||
style = (getattr(anchors, "shooting_style", "") or "").strip()
|
||||
key = _DEVICE_MAP.get(style, "")
|
||||
if not key:
|
||||
return ""
|
||||
try:
|
||||
from composer.realism import device_pack_block
|
||||
return device_pack_block(key)
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
def realism_discipline() -> str:
|
||||
"""去 AI 感的**提示词纪律**（供导演写分镜时遵守）
|
||||
|
||||
来源：anti-ai-checklist.md + atmosphere-dictionary.md（研究落地）
|
||||
"""
|
||||
return (
|
||||
"【去 AI 感纪律（★重要，直接决定像不像真拍的）】\n"
|
||||
"1. **禁用抽象氛围词**：不要写「温馨」「紧张」「孤独」这类词 —— "
|
||||
"把它们翻译成**具体可拍的现象**（例：不写「温馨」，写「午后阳光透过树叶在地面"
|
||||
"投下斑驳光影、杯口冒着细微热气」）。\n"
|
||||
"2. **每镜头一个「非完美事件」**（全片不重复，缺陷类型也要换）：\n"
|
||||
" · 摄影师失误：自动对焦在主体与背景间搜索 / 构图偏离中心 / 慢了半拍才跟上\n"
|
||||
" · 环境介入：微风吹动发丝 / 云影掠过导致曝光波动 / 场外有人打招呼\n"
|
||||
" · 主体自然行为：注意到镜头露出真诚微笑后移开视线 / 说话被环境音打断笑场\n"
|
||||
"3. **对焦要写路径**（不要笼统写「自动对焦搜索」）：\n"
|
||||
" ❌ 自动对焦反复搜索\n"
|
||||
" ✅ 自动对焦在她脸部和她手中的杯子之间来回切换\n"
|
||||
"4. **只用现场真实光源**（窗光/顶灯/手机屏幕光），禁影棚词（柔光箱/主光/轮廓光）。\n"
|
||||
"5. **不要写慢动作**（除非明确设定为手机慢动作模式）——真实拍摄默认实时速度。"
|
||||
)
|
||||
@@ -0,0 +1,274 @@
|
||||
"""
|
||||
directors.legacy_scene_planner — 本地导演（现有能力的包装）
|
||||
==============================================================
|
||||
|
||||
把「故事文本 → 分镜」用 LLM 直接完成（不依赖本地 LLM / 脚本 YAML 中转），
|
||||
输出**统一分镜 Schema**。这是 v5.2 接入前的默认可⽤导演。
|
||||
|
||||
特点：
|
||||
· 支持单镜头重生成（revise_shot）
|
||||
· 输出含镜头契约（景别/机位/运动/景深）—— 用简化版专业摄影语言
|
||||
· 无外部依赖，只调 DeepSeek
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
from schemas import GlobalAnchors, Shot, ShotContract, Storyboard # noqa: E402
|
||||
from directors.base import COMMON_OPTIONS, BaseDirector, norm_duration, cast_or_character, style_pack_block, director_style_hint, shot_brief # noqa: E402
|
||||
from directors.llm import DirectorLLM # noqa: E402
|
||||
|
||||
|
||||
def _sc_val(v) -> int:
|
||||
"""镜头数：0 / 空 = 自动（交给导演决定）"""
|
||||
try:
|
||||
return int(v or 0)
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
|
||||
log = logging.getLogger("directors.legacy")
|
||||
|
||||
_SYSTEM = "你是专业的影视分镜师，擅长把故事拆解成可执行的镜头列表。"
|
||||
|
||||
_PROMPT_TMPL = """请把下面的故事拍成一支短片。
|
||||
|
||||
【故事】
|
||||
{story}
|
||||
|
||||
【整体设定】
|
||||
{style_pack}
|
||||
{director_style}
|
||||
|
||||
【镜头数与时长参考】
|
||||
{shot_brief}
|
||||
|
||||
【资产设定（★必须严格使用这些设定，不要自己另编场景/人物；同一资产在所有镜头中保持一致）】
|
||||
- 角色：{character}
|
||||
- 场地：{location}
|
||||
- 服装：{wardrobe}
|
||||
- 产品 / 道具：{props}
|
||||
|
||||
【要求】
|
||||
1. 每个镜头要具体可拍：写清"谁在哪做什么、画面里有什么、光线如何"
|
||||
2. 镜头语言要专业：景别（特写/近景/中景/全景）+ 机位角度 + 摄影机运动
|
||||
3. 保持整体连贯：镜头之间是同一场景/同一人物，风格统一
|
||||
4. 有台词就写台词（像真人口语），没台词留空
|
||||
5. 镜头类型从这几类里选：action（动作/场景）、talking_head（对镜头说话）、background（环境空镜）
|
||||
|
||||
【输出格式】严格输出 JSON 数组，不要任何解释文字：
|
||||
[
|
||||
{{
|
||||
"duration": 5,
|
||||
"type": "action",
|
||||
"visual_description": "画面描述（谁在哪做什么，光线/环境细节）",
|
||||
"shot_size": "景别（特写/近景/中景/全景）",
|
||||
"angle": "机位角度（平视/俯视/仰视/过肩）",
|
||||
"camera_move": "摄影机运动（固定/推/拉/摇/跟/环绕）",
|
||||
"depth_of_field": "景深（浅/中/深）",
|
||||
"text": "台词或旁白（没有就空字符串）",
|
||||
"sentiment": "情绪（如：平静/焦虑/欣慰）",
|
||||
"action": "人物动作",
|
||||
"cast_slots": ["这一镜**出场**的角色名（必须与上面【资产设定】里的角色名完全一致；
|
||||
空镜/无人物镜头给空数组 []）"],
|
||||
"location_id": "这一镜所在场地（必须与上面【资产设定】的场地名一致）"
|
||||
}}
|
||||
]
|
||||
|
||||
★ cast_slots 非常重要：它决定生成关键帧时**用哪几个角色的参考图做多图融合**
|
||||
（特写要保脸、全景要保环境）。漏填会导致人物不一致，请如实填写。
|
||||
"""
|
||||
|
||||
_REVISE_TMPL = """下面是一个分镜表里的第 {idx} 个镜头，请按用户要求重新生成这一个镜头。
|
||||
|
||||
【整片故事】
|
||||
{story}
|
||||
|
||||
【当前镜头】
|
||||
{current}
|
||||
|
||||
【修改要求】
|
||||
{instruction}
|
||||
|
||||
【要求】保持与整片风格一致，只改这一个镜头。输出格式：单个 JSON 对象（不要数组），字段同上。
|
||||
"""
|
||||
|
||||
|
||||
class LegacyScenePlanner(BaseDirector):
|
||||
"""本地导演（LLM 直出分镜）"""
|
||||
|
||||
id = "legacy_scene_planner"
|
||||
name = "本地分镜（内置）"
|
||||
kind = "legacy"
|
||||
description = "用 LLM 把故事直接拆成专业分镜（含景别/机位/运镜），无需额外依赖"
|
||||
features = ["storyboard", "revise_shot"]
|
||||
status = "ready"
|
||||
source = "09_ave/scripts/directors/legacy_scene_planner.py"
|
||||
options_schema = dict(COMMON_OPTIONS)
|
||||
|
||||
def __init__(self):
|
||||
self.llm = DirectorLLM()
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
return self.llm.available
|
||||
|
||||
# ── 生成 ─────────────────────────────────────────
|
||||
|
||||
def generate(self, story: str, anchors: GlobalAnchors | None = None,
|
||||
options: dict | None = None) -> Storyboard:
|
||||
opts = options or {}
|
||||
anchors = anchors or GlobalAnchors()
|
||||
shot_count = int(opts.get("shot_count") or 6)
|
||||
shot_duration = float(opts.get("shot_duration") or 5)
|
||||
shooting_style = opts.get("shooting_style") or "documentary_realism"
|
||||
realism_boost = opts.get("realism_boost") or "mild"
|
||||
director_style = opts.get("director_style") or ""
|
||||
|
||||
prompt = _PROMPT_TMPL.format(
|
||||
shot_brief=shot_brief({"shot_count": shot_count, "shot_duration": shot_duration}),
|
||||
story=(story or "").strip()[:2000],
|
||||
shooting_style=shooting_style,
|
||||
shot_duration=shot_duration,
|
||||
# ★ 资产注入：有详情用详情，否则退回 ID
|
||||
character=cast_or_character(anchors),
|
||||
style_pack=style_pack_block(anchors),
|
||||
director_style=director_style_hint(anchors),
|
||||
location=anchors.location_context or anchors.location_id or "（未指定）",
|
||||
wardrobe=anchors.wardrobe or "（未指定）",
|
||||
props=anchors.props_context or "（无）",
|
||||
)
|
||||
raw = self.llm.chat(prompt, system=_SYSTEM, temperature=0.7, max_tokens=3000)
|
||||
items = self.llm.parse_json(raw)
|
||||
if isinstance(items, dict): # 容错：AI 返回 {"shots": [...]}
|
||||
items = items.get("shots") or items.get("data") or []
|
||||
if not isinstance(items, list) or not items:
|
||||
log.warning(" ⚠️ legacy 导演未产出有效分镜")
|
||||
return Storyboard(project_id=_pid(), story=story, director_id=self.id,
|
||||
global_anchors=anchors, shots=[])
|
||||
|
||||
shots: list[Shot] = []
|
||||
for i, it in enumerate(items, 1):
|
||||
if not isinstance(it, dict):
|
||||
continue
|
||||
# ★ 出场角色（决定关键帧用哪几张角色参考图做多图融合）
|
||||
# LLM 输出优先（但要过滤掉演员表里不存在的代号，防 AI 乱写）；
|
||||
# 没给就兜底用演员表的全部 slot（单角色场景即主角）
|
||||
# 注意 anchors.cast 里是 CastMember 对象（不是 dict）——
|
||||
# 照 _shot_builder 的写法用 effective_cast()
|
||||
try:
|
||||
_members = anchors.effective_cast() if hasattr(anchors, "effective_cast") else []
|
||||
except Exception:
|
||||
_members = []
|
||||
_valid = {str(getattr(m, "slot", "") or "") for m in _members}
|
||||
_valid.discard("")
|
||||
_raw = it.get("cast_slots") or it.get("cast_in_shot") or []
|
||||
if isinstance(_raw, str):
|
||||
_raw = [_raw]
|
||||
_cs = [str(x).strip() for x in _raw
|
||||
if str(x).strip() and (not _valid or str(x).strip() in _valid)]
|
||||
if not _cs:
|
||||
_cs = sorted(_valid) # 兜底：演员表的全部 slot
|
||||
shots.append(Shot(
|
||||
shot_id=f"shot_{i:02d}",
|
||||
scene_id=str(i),
|
||||
duration=norm_duration(it.get("duration"), (it.get("text") or "").strip()),
|
||||
type=_norm_type(it.get("type")),
|
||||
cast_slots=_cs,
|
||||
character_id=anchors.character_id,
|
||||
location_id=(it.get("location_id") or "").strip() or anchors.location_id,
|
||||
visual_description=(it.get("visual_description") or "").strip(),
|
||||
shot_contract=ShotContract(
|
||||
shot_size=(it.get("shot_size") or "").strip(),
|
||||
angle=(it.get("angle") or "").strip(),
|
||||
camera_move=(it.get("camera_move") or "").strip(),
|
||||
lens_feel=(it.get("lens_feel") or "").strip(),
|
||||
depth_of_field=(it.get("depth_of_field") or "").strip(),
|
||||
),
|
||||
text=(it.get("text") or "").strip() or None,
|
||||
sentiment=(it.get("sentiment") or "").strip() or None,
|
||||
action=(it.get("action") or "").strip() or None,
|
||||
status="pending",
|
||||
))
|
||||
|
||||
sb = Storyboard(
|
||||
project_id=_pid(),
|
||||
title=(story or "")[:40],
|
||||
story=story,
|
||||
content_type=opts.get("content_type") or "short_drama",
|
||||
director_id=self.id,
|
||||
global_anchors=anchors,
|
||||
shots=shots,
|
||||
)
|
||||
log.info(" 🎬 legacy 导演产出 %d 个镜头（%.1f 秒）", len(shots), sb.total_duration())
|
||||
return sb
|
||||
|
||||
# ── 单镜头重生成 ──────────────────────────────────
|
||||
|
||||
def revise_shot(self, sb: Storyboard, shot_id: str,
|
||||
instruction: str = "") -> Storyboard:
|
||||
target = sb.shot_by_id(shot_id)
|
||||
if not target:
|
||||
raise ValueError(f"镜头不存在: {shot_id}")
|
||||
idx = sb.shots.index(target) + 1
|
||||
prompt = _REVISE_TMPL.format(
|
||||
idx=idx,
|
||||
story=(sb.story or "")[:800],
|
||||
current=json.dumps({
|
||||
"duration": target.duration,
|
||||
"type": target.type,
|
||||
"visual_description": target.visual_description,
|
||||
"shot_size": target.shot_contract.shot_size,
|
||||
"camera_move": target.shot_contract.camera_move,
|
||||
"text": target.text,
|
||||
"sentiment": target.sentiment,
|
||||
}, ensure_ascii=False, indent=1),
|
||||
instruction=instruction or "让这一幕更自然、更有真实感",
|
||||
)
|
||||
raw = self.llm.chat(prompt, system=_SYSTEM, temperature=0.8, max_tokens=800)
|
||||
it = self.llm.parse_json(raw)
|
||||
if isinstance(it, list) and it:
|
||||
it = it[0]
|
||||
if not isinstance(it, dict):
|
||||
raise RuntimeError("重生成失败：LLM 未返回有效结果")
|
||||
|
||||
target.duration = float(it.get("duration") or target.duration)
|
||||
target.type = _norm_type(it.get("type") or target.type)
|
||||
target.visual_description = (it.get("visual_description") or target.visual_description).strip()
|
||||
target.shot_contract = ShotContract(
|
||||
shot_size=(it.get("shot_size") or "").strip(),
|
||||
angle=(it.get("angle") or "").strip(),
|
||||
camera_move=(it.get("camera_move") or "").strip(),
|
||||
lens_feel=(it.get("lens_feel") or "").strip(),
|
||||
depth_of_field=(it.get("depth_of_field") or "").strip(),
|
||||
)
|
||||
target.text = (it.get("text") or "").strip() or None
|
||||
target.sentiment = (it.get("sentiment") or "").strip() or None
|
||||
target.action = (it.get("action") or "").strip() or None
|
||||
log.info(" 🔄 镜头重生成: %s", shot_id)
|
||||
return sb
|
||||
|
||||
|
||||
# ── 工具 ──────────────────────────────────────────────
|
||||
|
||||
def _norm_type(t: Any) -> str:
|
||||
t = (t or "").strip().lower()
|
||||
aliases = {
|
||||
"action": "action", "talking_head": "talking_head", "talk": "talking_head",
|
||||
"background": "background", "b_roll": "background", "vfx": "vfx",
|
||||
"comfyui": "comfyui", "3d_character": "3d_character",
|
||||
}
|
||||
return aliases.get(t, "action")
|
||||
|
||||
|
||||
def _pid() -> str:
|
||||
from datetime import datetime
|
||||
return "vid_" + datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
@@ -0,0 +1,141 @@
|
||||
"""
|
||||
directors.llm — 导演层共用的轻量 LLM 客户端
|
||||
=============================================
|
||||
|
||||
配置优先级：环境变量 DEEPSEEK_* > agent-local local.yaml(llm) > 09_ave/config.yaml(llm)
|
||||
（与评论生成 AIGenerator 一致的取配置方式，但独立实现，避免跨模块耦合）
|
||||
|
||||
仅提供最简 chat 调用 + JSON 解析，供各导演实现使用。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
from lib.http import urlopen as _noproxy_urlopen # 禁代理+重试
|
||||
|
||||
log = logging.getLogger("directors.llm")
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent # 09_ave/scripts/
|
||||
_AVE_DIR = _SCRIPTS.parent # 09_ave/
|
||||
_AGENT_LOCAL = Path(os.environ.get("AGENT_LOCAL") or (_AVE_DIR.parents[2] / "agent-local"))
|
||||
_LOCAL_CFG = _AGENT_LOCAL / "tools" / "ave" / "config" / "local.yaml"
|
||||
_AVE_CFG = _AVE_DIR / "config.yaml"
|
||||
|
||||
|
||||
def load_llm_config() -> dict:
|
||||
"""三级优先级读取 LLM 配置"""
|
||||
cfg: dict = {}
|
||||
# ③ 09_ave/config.yaml
|
||||
if _AVE_CFG.exists():
|
||||
try:
|
||||
raw = yaml.safe_load(_AVE_CFG.read_text(encoding="utf-8")) or {}
|
||||
cfg.update(raw.get("llm") or {})
|
||||
except Exception:
|
||||
pass
|
||||
# ② 配置中心 local.yaml → llm
|
||||
if _LOCAL_CFG.exists():
|
||||
try:
|
||||
raw = yaml.safe_load(_LOCAL_CFG.read_text(encoding="utf-8")) or {}
|
||||
cfg.update(raw.get("llm") or {})
|
||||
except Exception:
|
||||
pass
|
||||
# ① 环境变量
|
||||
if os.environ.get("DEEPSEEK_API_KEY"):
|
||||
cfg["api_key"] = os.environ["DEEPSEEK_API_KEY"]
|
||||
if os.environ.get("DEEPSEEK_BASE_URL"):
|
||||
cfg["base_url"] = os.environ["DEEPSEEK_BASE_URL"]
|
||||
if os.environ.get("DEEPSEEK_MODEL"):
|
||||
cfg["model"] = os.environ["DEEPSEEK_MODEL"]
|
||||
cfg.setdefault("base_url", "https://api.deepseek.com/v1")
|
||||
cfg.setdefault("model", "deepseek-v4-flash")
|
||||
return cfg
|
||||
|
||||
|
||||
class DirectorLLM:
|
||||
"""导演层 LLM 客户端（单次同步调用）"""
|
||||
|
||||
def __init__(self, timeout: int = 90):
|
||||
cfg = load_llm_config()
|
||||
self.api_key = (cfg.get("api_key") or "").strip()
|
||||
self.base_url = (cfg.get("base_url") or "https://api.deepseek.com/v1").rstrip("/")
|
||||
self.model = cfg.get("model") or "deepseek-v4-flash"
|
||||
self.timeout = timeout
|
||||
self._fails = 0
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
return bool(self.api_key) and self._fails < 3
|
||||
|
||||
def chat(self, prompt: str, system: str = "", temperature: float = 0.7,
|
||||
max_tokens: int = 3000) -> str:
|
||||
"""单次对话（失败返回空串，连续 3 次失败熔断）"""
|
||||
if not self.available:
|
||||
log.warning(" ⚠️ LLM 不可用（未配 key 或已熔断）")
|
||||
return ""
|
||||
messages = []
|
||||
if system:
|
||||
messages.append({"role": "system", "content": system})
|
||||
messages.append({"role": "user", "content": prompt})
|
||||
body = json.dumps({
|
||||
"model": self.model,
|
||||
"messages": messages,
|
||||
"temperature": temperature,
|
||||
"max_tokens": max_tokens,
|
||||
# 关思考模式：分镜生成不需要长推理，省 token 且更快
|
||||
"thinking": {"type": "disabled"},
|
||||
}).encode("utf-8")
|
||||
req = urllib.request.Request(
|
||||
f"{self.base_url}/chat/completions", data=body,
|
||||
headers={"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {self.api_key}"})
|
||||
try:
|
||||
with _noproxy_urlopen(req, timeout=self.timeout) as r:
|
||||
data = json.loads(r.read().decode("utf-8"))
|
||||
content = (data.get("choices") or [{}])[0].get("message", {}).get("content", "")
|
||||
self._fails = 0
|
||||
return (content or "").strip()
|
||||
except urllib.error.HTTPError as e:
|
||||
detail = e.read().decode("utf-8", errors="ignore")[:200]
|
||||
log.error(" ❌ LLM HTTP %s: %s", e.code, detail)
|
||||
self._fails += 1
|
||||
return ""
|
||||
except Exception as e:
|
||||
log.error(" ❌ LLM 调用失败: %s", e)
|
||||
self._fails += 1
|
||||
return ""
|
||||
|
||||
# ── JSON 解析（容错）──────────────────────────────
|
||||
|
||||
@staticmethod
|
||||
def parse_json(text: str) -> Any:
|
||||
"""从 LLM 输出里提取 JSON（容忍 markdown 围栏 / 前后废话）"""
|
||||
if not text:
|
||||
return None
|
||||
t = text.strip()
|
||||
# 去 markdown 围栏
|
||||
m = re.search(r"```(?:json)?\s*(.+?)\s*```", t, re.S)
|
||||
if m:
|
||||
t = m.group(1).strip()
|
||||
# 直接解析
|
||||
try:
|
||||
return json.loads(t)
|
||||
except Exception:
|
||||
pass
|
||||
# 找第一个 { 或 [ 到最后一个 } 或 ]
|
||||
for open_c, close_c in (("{", "}"), ("[", "]")):
|
||||
i, j = t.find(open_c), t.rfind(close_c)
|
||||
if i >= 0 and j > i:
|
||||
try:
|
||||
return json.loads(t[i:j + 1])
|
||||
except Exception:
|
||||
continue
|
||||
log.warning(" ⚠️ LLM 输出无法解析为 JSON（前 120 字: %s）", t[:120])
|
||||
return None
|
||||
@@ -0,0 +1,385 @@
|
||||
"""
|
||||
directors.seedance_director — Seedance 导演方法论（skill 型导演）
|
||||
|
||||
来源：Seedance-Director-4.0（SCAI）导演方法论，去人设/去平台命令后精编移植，
|
||||
见 09_ave/skills/seedance-director/SKILL.md
|
||||
|
||||
特点（与 v5.2 的差异）：
|
||||
· 强调「分轨 / 场一个职 / 条一个目的 / 镜头必须干活」
|
||||
· 时长跟内容与气口走（容量不是目标；超限则切条并声明接续需求）
|
||||
· 产出后自检「五类病」（矛盾/断链/走样/模型雷/红线）
|
||||
· 输出同一套 Shot Schema（复用 _shot_builder），下游无感知
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
from schemas import GlobalAnchors, Storyboard # noqa: E402
|
||||
from directors.base import COMMON_OPTIONS, BaseDirector, cast_or_character, style_pack_block, realism_block, director_style_hint, device_pack_hint, realism_discipline, shot_brief # noqa: E402
|
||||
from directors._shot_builder import build_shots_from_items, norm_type # noqa: E402
|
||||
from directors.llm import DirectorLLM # noqa: E402
|
||||
|
||||
|
||||
def _sc_val(v) -> int:
|
||||
"""镜头数：0 / 空 = 自动（交给导演按叙事需要决定）"""
|
||||
try:
|
||||
return int(v or 0)
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
|
||||
log = logging.getLogger("directors.seedance")
|
||||
|
||||
_AVE_DIR = _SCRIPTS.parent
|
||||
SKILL_PATHS = [
|
||||
_AVE_DIR / "skills" / "seedance-director" / "SKILL.md",
|
||||
]
|
||||
_SKILL_MAX_CHARS = 13000
|
||||
|
||||
_PROMPT_TMPL = """请按「Seedance 导演方法论」把下面的故事拍成一支短片。
|
||||
{shot_brief}
|
||||
|
||||
【故事】
|
||||
{story}
|
||||
|
||||
【整体设定】
|
||||
- 拍摄风格：{shooting_style}
|
||||
- {shot_brief}
|
||||
|
||||
【资产设定（★严格使用，不要另编；同一资产在所有镜头保持一致）】
|
||||
- 角色：{character}
|
||||
- 场地：{location}
|
||||
- 服装：{wardrobe}
|
||||
- 产品/道具：{props}
|
||||
|
||||
【场景真实感纪律（★重要，直接影响"像不像真拍的"）】
|
||||
**禁止影棚语言**——生活场景里不该出现专业影棚的词汇，写了画面立刻变假：
|
||||
- ❌ 禁止：柔光箱 / 布光（"柔光箱均匀布光""侧面打光"）/ 纯色背景 / 纯灰背景 /
|
||||
中焦 85mm / 压缩空间 / 影棚 / 摄影棚 / 反光板 / 灯架
|
||||
- ✅ 只用**场景里真实存在的光源**，并说清它从哪来：
|
||||
窗户自然光 / 室内顶灯 / 床头灯 / 台灯 / 手机屏幕的光 / 走廊灯 / 电视光
|
||||
- ❌ 禁止把背景写成"纯色/虚化背景"——真实房间的背景就是**杂乱的真实物件**
|
||||
（输液架、床头柜、水杯、被子褶皱、墙上的挂钩）
|
||||
|
||||
【拍摄者的动作逻辑（★自拍/Vlog 必检）】
|
||||
画面里"手机是谁拿的"必须说得通，否则一眼假：
|
||||
- **人物自己举着**：那他的**双手被占用**，不能再做"抓床单/换药/双手撑床"这类动作
|
||||
- **他人帮拍**（护士/家属/路人）：必须写明"他人在拍"，并允许这些真实特征：
|
||||
画面晃动 / 对焦慢半拍 / 构图歪 / 短暂拍到地面或天花板 / 拍摄者在画面外说话
|
||||
- **手机放置**（立在床头柜、靠着水杯、半躺在床上）：允许角度歪、中途滑落调整、
|
||||
只拍到半个身子
|
||||
- ⚠️ **手被占住的场景**（换药、剧痛抓床单、双手抱头）**不能**写成"自己举着手机自拍"
|
||||
|
||||
【慌乱/失控场景的真实特征（★疼、急、乱的时候必带）】
|
||||
真实的手持拍摄在慌乱中一定会露出这些"瑕疵"，它们才是真实感的来源：
|
||||
- 镜头晃动加剧、跟不住主体
|
||||
- 焦点来回漂移、短暂失焦
|
||||
- 构图来不及调整（人物时而偏出画面）
|
||||
- 呼吸声/喘气声进麦克风
|
||||
- 镜头短暂甩到别处（墙、天花板、被子）再拉回来
|
||||
|
||||
【镜头时长纪律（★硬约束）】
|
||||
生成模型单镜头**最长 10 秒**（可灵只有 5 秒 / 10 秒两档），这是硬限制。因此：
|
||||
- 单镜时长写 **2~10 秒**（整数或 .5，按内容与气口定，不要凑数）
|
||||
- 超过 10 秒的连续动作**必须主动拆成多个镜头**，拆法：
|
||||
· **按动作阶段**：起势 → 发力 → 收势（如"慢慢坐下"→"落座瞬间的表情"→"坐定后的长舒一口气"）
|
||||
· **按景别递进**：中景交代 → 特写强调 → 反应镜头（观众看不出断点）
|
||||
· **按台词气口**：一句话说完整，下一句换镜头
|
||||
- ❌ 不要写「一个 15 秒的长镜头」「持续 20 秒的连续动作」—— 生成不出来
|
||||
- ✅ 拆分后在每个镜头里写清它负责哪个阶段，让观众自然接上
|
||||
|
||||
【镜头描述纪律（★ 来自专业导演方法论，直接决定生成质量）】
|
||||
|
||||
**① 禁止「形容词堆砌」** —— 这是外行与专业的分水岭：
|
||||
❌ 「电影感、戏剧性、大师作品、8K、唯美、氛围感」
|
||||
这些词不告诉模型：机位在哪、身体做什么、动作在哪结束、什么不能变
|
||||
✅ 用**可观察的物理描述**替代：机位 / 景别 / 身体具体动作 / 环境反应 / 收尾状态
|
||||
|
||||
**② 一个镜头 = 一个主导运动**（或刻意静止）
|
||||
❌ 「镜头旋转、变焦、平移、跟拍、晃动」
|
||||
✅ 「固定机位，极缓慢地向前推；除此之外没有其他运动」
|
||||
|
||||
**③ 每个动作必须有「起止状态」**（最重要、我们过去最缺）：
|
||||
✅ 「他单膝跪地开始 → 缓缓抬头到湿发滑落露出眼睛 → 结束时半起身、重心在前膝、头抬起」
|
||||
（只说"他挣扎着"没有任何起止 = 模型自由发挥 = 不可控）
|
||||
|
||||
**④ 加**一个**环境反应**（让画面"活"起来）：
|
||||
✅ 「雨点不断打破水洼表面；远处闪电短暂照亮街道尽头」
|
||||
|
||||
**⑤ 不变量逐字重复**（跨镜头锁定）：
|
||||
同一场戏里，人物的脸 / 发型 / 服装 / 场地 / 光线，在每个镜头提示词里
|
||||
**用同样的词原样重复** —— 不要换说法（"齐肩直发" 每次都必须写"齐肩直发"）
|
||||
|
||||
**⑥ 负面词只列本镜头**真实风险**，且必须是**具体实例**：
|
||||
❌ 「不要现代物品」（模型无法解析"类别"）
|
||||
✅ 「不要手表、不要印刷 logo、不要橡胶鞋底」
|
||||
|
||||
**⑦ 竖屏（9:16）不是横屏裁切**：
|
||||
· 用**纵深**调度（前后景分层），而不是横向铺开
|
||||
· **主体放大**（竖屏里人物占比要更大）
|
||||
· **每个全景镜头都要重新设计**（横屏的构图逻辑在竖屏里不成立）
|
||||
|
||||
**⑧ 画面必须提供「配音之外的信息」**：
|
||||
如果画面只是把台词演一遍 = 观众没有看的理由。画面要承载：状态变化 / 关系 / 潜台词。
|
||||
|
||||
【台词口语化纪律（★重要，直接决定"像不像真人说话"）】
|
||||
真实的人开口**不是写作文**。台词必须像真人说话，而不是念稿：
|
||||
|
||||
❌ 书面：「我住院第三天了，真没想象中那么可怕。」
|
||||
✅ 口语：「住院第三天了……说实话，没想象中那么可怕。」
|
||||
|
||||
**七条要求**：
|
||||
1. **口语词**：用「嗯、啊、就、那个、说实话、其实、反正、怎么样」这类真实语气词
|
||||
2. **停顿与迟疑**：用「……」表示停顿，用「（停顿）」「（轻声）」标注语气
|
||||
3. **允许不完美**：可以重复（「我、我是说」）、改口、没说完（被打断）
|
||||
4. **短句为主**：一句尽量 ≤15 字；长句要拆开，像真人一口气说不完
|
||||
5. **禁书面词**：不写「因此 / 然而 / 致使 / 综上所述 / 未免」
|
||||
6. **贴合人物**：老人、孩子、医生、病人的说话方式不同（用词、语速感要区分）
|
||||
7. **一句一个意思**：一句话不做两件事（先说完这个，再接着说下个）
|
||||
|
||||
**额外**：如果这个镜头里有人物开口，必须在画面描述里同时写**说话时的身体状态**
|
||||
（呼吸急促 / 嘴角抽动 / 眼神游移 / 手不自觉地攥紧），台词和身体不能分开设计。
|
||||
|
||||
【画面描述的纪律（★重要，直接影响生成质量）】
|
||||
"visual_description" 只写**镜头里能看到什么**，严禁混入制作意图：
|
||||
❌ 禁止：「方便出结尾字幕」「保持2秒让字幕显示」「便于后期剪辑」「留白给标题」「方便加标签」
|
||||
✅ 只写：人物动作 / 表情 / 镜头运动 / 环境细节
|
||||
（字幕、时长、后期、剪辑都属于制作层，不是画面内容 —— 写进提示词会污染生成）
|
||||
|
||||
【非语言发声必须标 sfx（★重要）】
|
||||
人物发出的**非语言声音**要写进 sfx，**不要塞进台词 text**：
|
||||
· 惨叫 / 闷哼 / 痛呼 / 倒吸一口气 / 叹气 / 干笑 / 清嗓 / 吞咽 / 哽咽
|
||||
写法：{{"at_second": 1.5, "desc": "痛苦的闷哼"}}
|
||||
（text 只放"说出来的话"；发声类走 sfx —— 后期能配真实音效，比 TTS 念字自然得多）
|
||||
例：换药镜头 → text 可以是空，sfx 写 [{{"at_second": 0.8, "desc": "倒吸一口气"}}, {{"at_second": 2.0, "desc": "痛苦的闷哼"}}]
|
||||
|
||||
【空间与走位纪律（★重要，直接决定画面是否"看得懂"）】
|
||||
每个镜头必须说清「**人在空间的哪儿、朝哪、怎么动**」——三者缺一，观众就看不懂空间关系：
|
||||
- **位置**：人物处在场景的哪个锚点（如：病床右侧 / 窗前 / 门口 / 走廊中段）
|
||||
- **朝向**：面朝哪个方向（如：面向窗户 / 背对镜头 / 侧向病床 / 半侧身朝门口）
|
||||
- **移动**：这一步是静止不动 / 小幅位移 / 转身；**并写清方位变化**
|
||||
（如：从床边向窗户走两步 / 缓缓转身背对镜头）
|
||||
|
||||
对照写法：
|
||||
❌ 「他在病房里说话」（观众不知道人和空间的关系 → 换机位就乱）
|
||||
✅ 「他坐在病床右侧，面朝窗户，说到一半慢慢站起来转向门口」
|
||||
|
||||
⚠️ **单镜头内不要写长距离移动**（走 5 米这种，模型画不出来，会穿模/瞬移）：
|
||||
· 3 步以内的小幅位移 → 可以（如起身、上前一步、转身）
|
||||
· 超过 3 步 / 跨越整个空间 → **必须拆成多个镜头**：
|
||||
镜1 在床边起身（中景）→ 镜2 走向窗户（全景，能看出距离）→ 镜3 站在窗前（近景）
|
||||
且**每个镜头用对应机位的场景参考图**，空间才连贯。
|
||||
|
||||
【拍摄风格与镜头视角纪律（★重要）】
|
||||
拍摄风格决定"谁在看"，镜头语言必须与之一致，不要写错视角：
|
||||
- 📱 **手机自拍·第一人称（selfie_pov）**：镜头 = 人物自己举着的手机。
|
||||
· 画面里是"他自己"（近脸、一臂距离、轻微广角畸变）
|
||||
· 允许"看镜头说话"（对镜自述）
|
||||
· ❌ 不要写第三人称观察视角（如"镜头从侧面拍他"）
|
||||
· ⚠️ **全程统一**：包括开场/过场镜，也只能是"人物自己的视角"
|
||||
（如：举着手机照路 / 低头看自己脚下的鞋 / 手机屏幕反射中的自己），
|
||||
不要出现"从身后跟拍脚步"这类旁人视角
|
||||
- 🚶 **手持 Vlog·第一视角（vlog_handheld）**：镜头 = 拍摄者的眼睛。
|
||||
· 画面里是"他看到的"（前方、手中物、同行的人）
|
||||
· 拍摄者本人通常不入画，或只有手/手臂入画
|
||||
- 📺 **访谈纪实（interview_doc）**：机位固定，人物半身对镜说话，可有停顿与思考
|
||||
- 🎬 **电影感 / 伪纪录片**：第三人称观察视角，可用正反打 / 过肩镜头
|
||||
|
||||
{style_pack}
|
||||
{device_pack}
|
||||
{realism_discipline}
|
||||
{realism}
|
||||
{director_style}
|
||||
|
||||
【多角色一致性纪律（★重要，直接影响成片质量）】
|
||||
- **同框限制**：图生视频只能喂「一张」角色参考图 → 多角色同框的近景/特写会让次要角色人脸随机漂移。
|
||||
因此：
|
||||
· 避免「两个以上角色同框且都要清晰露脸（近景/特写）」
|
||||
· 双人戏优先用：**正反打**（单人镜头交替）· **过肩镜头**（一方背对或只露侧影）·
|
||||
**中远景**（脸小、容错高）· **背影 / 手部 / 局部**
|
||||
· 确需同框时：让"主角"占据画面视觉中心，次要角色可侧身、偏离焦平面或虚化处理
|
||||
- **出场标注**：每个镜头用 `cast_in_shot` 精确标注**实际出场**的人物代号（没出场不要写）。
|
||||
单人镜头只写一个人 —— 这是生成时绑定正确参考图（人脸一致）的前提。
|
||||
|
||||
【拆条纪律】
|
||||
1. 先分轨（叙事轨要转 / 表演轨要成立 / 展示轨要让观众看懂一件事），再拆条。
|
||||
2. **条一个目的**：一次生成一件事。一条里要用「并且」才说得全 = 该切条。
|
||||
3. **镜头必须干活**（改情绪/推进动作/加压），否则删；运镜必须回答「什么变了」。
|
||||
4. 时长按内容与气口：台词按中文字数（约 4.5 字/秒）估算，情绪停顿/呼吸/凝视要留时间；
|
||||
单镜 2~10 秒；超过 10 秒的内容 → 切成两条，前一条 transition_out 与后一条 chain_from_prev 呼应。
|
||||
|
||||
【输出格式】严格输出 JSON 数组，不要任何解释：
|
||||
[
|
||||
{{
|
||||
"duration": 4,
|
||||
"type": "action | talking_head | background | vfx",
|
||||
"cast_in_shot": ["主角"],
|
||||
"visual_description": "这一条观众看见什么（具体到可拍）",
|
||||
"shot_size": "景别", "angle": "机位角度", "camera_move": "运动（须回答什么变了）",
|
||||
"depth_of_field": "浅/中/深", "lens_feel": "光学质感", "frame_rate_feel": "常规/升格/降格",
|
||||
"text": "台词（逐字保留，不改写不润色；无则空）",
|
||||
"sentiment": "情绪",
|
||||
"expression_detail": {{"eyes": "眼神", "mouth": "嘴部", "brow": "眉宇"}},
|
||||
"action": "可见行为（不写情绪形容词）",
|
||||
"audio_cue": "环境音（有因果）",
|
||||
"sfx": [{{"at_second": 1.2, "desc": "音效"}}],
|
||||
"transition_in": {{"type": "cut", "duration": 0}},
|
||||
"transition_out": {{"type": "cut", "duration": 0}},
|
||||
"speed": {{"factor": 1.0, "ramp": ""}},
|
||||
"tech_requirements": {{
|
||||
"continuity": {{"character": "must", "wardrobe": "must", "location": "must", "expression": "prefer", "props": "none"}},
|
||||
"chain_from_prev": false,
|
||||
"motion_level": "small",
|
||||
"camera_difficulty": "simple",
|
||||
"needs": []
|
||||
}}
|
||||
}}
|
||||
]
|
||||
|
||||
产出前自检（五类病）：矛盾 / 断链 / 走样 / 模型雷 / 红线。
|
||||
前四类自己改；模型雷与红线写进对应镜头的 tech_requirements.needs，不要硬写做不到的画面。
|
||||
"""
|
||||
|
||||
_REVISE_TMPL = """按 Seedance 导演方法论重生成这一个镜头（保持整片统一，镜头必须干活）。
|
||||
【整片故事】{story}
|
||||
【当前镜头（第 {idx} 个）】{current}
|
||||
【修改要求】{instruction}
|
||||
【要求】只改这一个镜头，输出单个 JSON 对象（字段同上）。
|
||||
"""
|
||||
|
||||
|
||||
def skill_available() -> tuple[bool, str]:
|
||||
for p in SKILL_PATHS:
|
||||
if p.exists() and p.stat().st_size > 500:
|
||||
return True, str(p)
|
||||
return False, "技能文档未就位（需 09_ave/skills/seedance-director/SKILL.md）"
|
||||
|
||||
|
||||
def load_skill_doc() -> str:
|
||||
ok, path = skill_available()
|
||||
if not ok:
|
||||
return ""
|
||||
try:
|
||||
return Path(path).read_text(encoding="utf-8")[:_SKILL_MAX_CHARS]
|
||||
except Exception as e:
|
||||
log.warning(" ⚠️ Seedance 技能文档读取失败: %s", e)
|
||||
return ""
|
||||
|
||||
|
||||
class SeedanceDirector(BaseDirector):
|
||||
"""Seedance 导演方法论（分轨 / 条一个目的 / 五类病自检）"""
|
||||
|
||||
id = "seedance_director"
|
||||
name = "Seedance 导演方法论"
|
||||
kind = "skill"
|
||||
description = "Seedance 方法论移植：分轨拆条、时长跟内容与气口走、镜头必须干活、五类病自检"
|
||||
features = ["storyboard", "track_split", "shot_contract", "self_check", "tech_requirements"]
|
||||
status = "ready"
|
||||
source = "Seedance-Director-4.0（SCAI）· 精编移植（去人设）"
|
||||
options_schema = dict(COMMON_OPTIONS)
|
||||
|
||||
def __init__(self):
|
||||
self.llm = DirectorLLM(timeout=120)
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
ok, _ = skill_available()
|
||||
return ok and self.llm.available
|
||||
|
||||
def generate(self, story: str, anchors: GlobalAnchors | None = None,
|
||||
options: dict | None = None) -> Storyboard:
|
||||
opts = options or {}
|
||||
anchors = anchors or GlobalAnchors()
|
||||
skill = load_skill_doc()
|
||||
if not skill:
|
||||
raise RuntimeError("Seedance 技能文档未就位，无法生成")
|
||||
|
||||
prompt = _PROMPT_TMPL.format(
|
||||
shot_brief=shot_brief(opts),
|
||||
story=(story or "").strip()[:2000],
|
||||
shooting_style=opts.get("shooting_style") or "documentary_realism",
|
||||
character=cast_or_character(anchors),
|
||||
style_pack=style_pack_block(anchors),
|
||||
device_pack=device_pack_hint(anchors),
|
||||
realism_discipline=realism_discipline(),
|
||||
realism=realism_block(),
|
||||
director_style=director_style_hint(anchors),
|
||||
location=anchors.location_context or anchors.location_id or "（未指定）",
|
||||
wardrobe=anchors.wardrobe or "（未指定）",
|
||||
props=anchors.props_context or "（无）",
|
||||
)
|
||||
raw = self.llm.chat(prompt, system=skill, temperature=0.7, max_tokens=3500)
|
||||
items = self.llm.parse_json(raw)
|
||||
if isinstance(items, dict):
|
||||
items = items.get("shots") or items.get("data") or items.get("storyboard") or []
|
||||
if not isinstance(items, list) or not items:
|
||||
log.warning(" ⚠️ Seedance 导演未产出有效分镜")
|
||||
return Storyboard(project_id=_pid(), story=story, director_id=self.id,
|
||||
global_anchors=anchors, shots=[])
|
||||
|
||||
shots = build_shots_from_items(items, anchors)
|
||||
durations = [s.duration for s in shots]
|
||||
if len(set(durations)) <= 1 and len(shots) > 1:
|
||||
log.warning(" ⚠️ Seedance 产出镜头时长全部相同(%ss) — 未按气口差异化", durations[:1])
|
||||
|
||||
sb = Storyboard(
|
||||
project_id=_pid(), title=(story or "")[:40], story=story,
|
||||
content_type=opts.get("content_type") or "short_drama",
|
||||
director_id=self.id, global_anchors=anchors, shots=shots,
|
||||
)
|
||||
log.info(" 🎬 Seedance 导演产出 %d 个镜头（%.1f 秒）", len(shots), sb.total_duration())
|
||||
return sb
|
||||
|
||||
def revise_shot(self, sb: Storyboard, shot_id: str,
|
||||
instruction: str = "") -> Storyboard:
|
||||
target = sb.shot_by_id(shot_id)
|
||||
if not target:
|
||||
raise ValueError(f"镜头不存在: {shot_id}")
|
||||
skill = load_skill_doc()
|
||||
import json as _json
|
||||
prompt = _REVISE_TMPL.format(
|
||||
story=(sb.story or "")[:800],
|
||||
idx=sb.shots.index(target) + 1,
|
||||
current=_json.dumps({
|
||||
"visual_description": target.visual_description,
|
||||
"shot_size": target.shot_contract.shot_size,
|
||||
"camera_move": target.shot_contract.camera_move,
|
||||
"text": target.text, "sentiment": target.sentiment,
|
||||
"tech_requirements": target.tech_requirements,
|
||||
}, ensure_ascii=False, indent=1),
|
||||
instruction=instruction or "更有导演感，镜头必须干活",
|
||||
)
|
||||
raw = self.llm.chat(prompt, system=skill[:2500] if skill else "", temperature=0.8, max_tokens=900)
|
||||
it = self.llm.parse_json(raw)
|
||||
if isinstance(it, list) and it:
|
||||
it = it[0]
|
||||
if not isinstance(it, dict):
|
||||
raise RuntimeError("重生成失败：LLM 未返回有效结果")
|
||||
|
||||
# 复用公共构建器解析单条
|
||||
one = build_shots_from_items([it], sb.global_anchors)
|
||||
if one:
|
||||
n = one[0]
|
||||
target.duration = n.duration
|
||||
target.type = n.type
|
||||
target.visual_description = n.visual_description
|
||||
target.shot_contract = n.shot_contract
|
||||
target.text = n.text
|
||||
target.sentiment = n.sentiment
|
||||
target.expression_detail = n.expression_detail
|
||||
target.action = n.action
|
||||
target.audio = n.audio
|
||||
target.tech_requirements = n.tech_requirements
|
||||
return sb
|
||||
|
||||
|
||||
def _pid() -> str:
|
||||
from datetime import datetime
|
||||
return "vid_" + datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
@@ -0,0 +1,424 @@
|
||||
"""
|
||||
directors.storyboard_v52 — AI Storyboard Director v5.2（技能型导演）
|
||||
=====================================================================
|
||||
|
||||
来源：ai-film-skills（Apache-2.0）
|
||||
`skills/ai-film-skills/docs/skills/zh-CN/ai-storyboard-director.md`
|
||||
|
||||
v5.2 的核心特点（相对本地导演）：
|
||||
① **先锁视觉概念，再拆镜头**（两遍交付：设计稿 → 生产稿）
|
||||
② 镜头契约更专业：景别 / 机位 / 光学 / 运动 / 声音 / 切点
|
||||
③ 强调"可执行"：每个镜头能直接交给生成模型
|
||||
|
||||
实现方式：把技能文档作为**系统提示词**注入，让 LLM 按 v5.2 方法论产出分镜，
|
||||
再转换为统一分镜 Schema（下游无感知）。
|
||||
|
||||
技能文档缺失时本导演自动标记为「不可用」，前端下拉会置灰。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
from schemas import (GlobalAnchors, Shot, ShotContract, Storyboard, # noqa: E402
|
||||
Transition, Speed, AudioConfig, SFX, ExpressionDetail)
|
||||
from directors.base import COMMON_OPTIONS, BaseDirector, norm_duration, cast_or_character, style_pack_block, realism_block, director_style_hint, device_pack_hint, realism_discipline, shot_brief # noqa: E402
|
||||
from directors._shot_builder import (build_shots_from_items, # noqa: E402
|
||||
norm_type as _norm_type)
|
||||
from directors.llm import DirectorLLM # noqa: E402
|
||||
|
||||
|
||||
def _sc_val(v) -> int:
|
||||
"""镜头数：0 / 空 = 自动（交给导演按叙事需要决定）"""
|
||||
try:
|
||||
return int(v or 0)
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
|
||||
log = logging.getLogger("directors.v52")
|
||||
|
||||
# 技能文档位置（ai-film-skills 下载到 09_ave/skills/）
|
||||
_AVE_DIR = _SCRIPTS.parent
|
||||
SKILL_PATHS = [
|
||||
_AVE_DIR / "skills" / "ai-film-skills" / "docs" / "skills" / "zh-CN" / "ai-storyboard-director.md",
|
||||
_AVE_DIR / "skills" / "ai-film-skills" / "docs" / "skills" / "ai-storyboard-director.md",
|
||||
]
|
||||
|
||||
# 技能文档注入上限（避免 prompt 过长）
|
||||
_SKILL_MAX_CHARS = 6000
|
||||
|
||||
_PROMPT_TMPL = """请按「导演级分镜」标准，把下面的故事拍成一支短片。
|
||||
{shot_brief}
|
||||
|
||||
【故事】
|
||||
{story}
|
||||
|
||||
【整体设定】
|
||||
- 拍摄风格：{shooting_style}
|
||||
- {shot_brief}
|
||||
|
||||
★【时长规则（必须遵守，按内容算）】
|
||||
· 短对白(≤12字) / 快切反应 / 插入细节 → 2~3 秒
|
||||
· 中长对白(13~30字) → 4~6 秒（按台词字数估算：中文约 4~5 字/秒）
|
||||
· 动作 / 走位调度 → 3~5 秒
|
||||
· 情绪停顿 / 凝视 / 环境空镜 → 3~5 秒
|
||||
· 需要连续表演的长镜头 → 6~10 秒（同时把 tech_requirements.chain_from_prev 设为 true）
|
||||
· 硬约束：时长必须能容纳台词（台词字数 ÷ 4.5 = 最少秒数）；单镜不超过 10 秒
|
||||
|
||||
【资产设定（★必须严格使用这些设定，不要自己另编场景/人物；同一资产在所有镜头中保持一致）】
|
||||
- 角色：{character}
|
||||
- 场地：{location}
|
||||
- 服装：{wardrobe}
|
||||
- 产品 / 道具：{props}
|
||||
|
||||
【场景真实感纪律（★重要，直接影响"像不像真拍的"）】
|
||||
**禁止影棚语言**——生活场景里不该出现专业影棚的词汇，写了画面立刻变假：
|
||||
- ❌ 禁止：柔光箱 / 布光（"柔光箱均匀布光""侧面打光"）/ 纯色背景 / 纯灰背景 /
|
||||
中焦 85mm / 压缩空间 / 影棚 / 摄影棚 / 反光板 / 灯架
|
||||
- ✅ 只用**场景里真实存在的光源**，并说清它从哪来：
|
||||
窗户自然光 / 室内顶灯 / 床头灯 / 台灯 / 手机屏幕的光 / 走廊灯 / 电视光
|
||||
- ❌ 禁止把背景写成"纯色/虚化背景"——真实房间的背景就是**杂乱的真实物件**
|
||||
（输液架、床头柜、水杯、被子褶皱、墙上的挂钩）
|
||||
|
||||
【拍摄者的动作逻辑（★自拍/Vlog 必检）】
|
||||
画面里"手机是谁拿的"必须说得通，否则一眼假：
|
||||
- **人物自己举着**：那他的**双手被占用**，不能再做"抓床单/换药/双手撑床"这类动作
|
||||
- **他人帮拍**（护士/家属/路人）：必须写明"他人在拍"，并允许这些真实特征：
|
||||
画面晃动 / 对焦慢半拍 / 构图歪 / 短暂拍到地面或天花板 / 拍摄者在画面外说话
|
||||
- **手机放置**（立在床头柜、靠着水杯、半躺在床上）：允许角度歪、中途滑落调整、
|
||||
只拍到半个身子
|
||||
- ⚠️ **手被占住的场景**（换药、剧痛抓床单、双手抱头）**不能**写成"自己举着手机自拍"
|
||||
|
||||
【慌乱/失控场景的真实特征（★疼、急、乱的时候必带）】
|
||||
真实的手持拍摄在慌乱中一定会露出这些"瑕疵"，它们才是真实感的来源：
|
||||
- 镜头晃动加剧、跟不住主体
|
||||
- 焦点来回漂移、短暂失焦
|
||||
- 构图来不及调整（人物时而偏出画面）
|
||||
- 呼吸声/喘气声进麦克风
|
||||
- 镜头短暂甩到别处（墙、天花板、被子）再拉回来
|
||||
|
||||
【镜头时长纪律（★硬约束）】
|
||||
生成模型单镜头**最长 10 秒**（可灵只有 5 秒 / 10 秒两档），这是硬限制。因此：
|
||||
- 单镜时长写 **2~10 秒**（整数或 .5，按内容与气口定，不要凑数）
|
||||
- 超过 10 秒的连续动作**必须主动拆成多个镜头**，拆法：
|
||||
· **按动作阶段**：起势 → 发力 → 收势（如"慢慢坐下"→"落座瞬间的表情"→"坐定后的长舒一口气"）
|
||||
· **按景别递进**：中景交代 → 特写强调 → 反应镜头（观众看不出断点）
|
||||
· **按台词气口**：一句话说完整，下一句换镜头
|
||||
- ❌ 不要写「一个 15 秒的长镜头」「持续 20 秒的连续动作」—— 生成不出来
|
||||
- ✅ 拆分后在每个镜头里写清它负责哪个阶段，让观众自然接上
|
||||
|
||||
【镜头描述纪律（★ 来自专业导演方法论，直接决定生成质量）】
|
||||
|
||||
**① 禁止「形容词堆砌」** —— 这是外行与专业的分水岭：
|
||||
❌ 「电影感、戏剧性、大师作品、8K、唯美、氛围感」
|
||||
这些词不告诉模型：机位在哪、身体做什么、动作在哪结束、什么不能变
|
||||
✅ 用**可观察的物理描述**替代：机位 / 景别 / 身体具体动作 / 环境反应 / 收尾状态
|
||||
|
||||
**② 一个镜头 = 一个主导运动**（或刻意静止）
|
||||
❌ 「镜头旋转、变焦、平移、跟拍、晃动」
|
||||
✅ 「固定机位，极缓慢地向前推；除此之外没有其他运动」
|
||||
|
||||
**③ 每个动作必须有「起止状态」**（最重要、我们过去最缺）：
|
||||
✅ 「他单膝跪地开始 → 缓缓抬头到湿发滑落露出眼睛 → 结束时半起身、重心在前膝、头抬起」
|
||||
（只说"他挣扎着"没有任何起止 = 模型自由发挥 = 不可控）
|
||||
|
||||
**④ 加**一个**环境反应**（让画面"活"起来）：
|
||||
✅ 「雨点不断打破水洼表面；远处闪电短暂照亮街道尽头」
|
||||
|
||||
**⑤ 不变量逐字重复**（跨镜头锁定）：
|
||||
同一场戏里，人物的脸 / 发型 / 服装 / 场地 / 光线，在每个镜头提示词里
|
||||
**用同样的词原样重复** —— 不要换说法（"齐肩直发" 每次都必须写"齐肩直发"）
|
||||
|
||||
**⑥ 负面词只列本镜头**真实风险**，且必须是**具体实例**：
|
||||
❌ 「不要现代物品」（模型无法解析"类别"）
|
||||
✅ 「不要手表、不要印刷 logo、不要橡胶鞋底」
|
||||
|
||||
**⑦ 竖屏（9:16）不是横屏裁切**：
|
||||
· 用**纵深**调度（前后景分层），而不是横向铺开
|
||||
· **主体放大**（竖屏里人物占比要更大）
|
||||
· **每个全景镜头都要重新设计**（横屏的构图逻辑在竖屏里不成立）
|
||||
|
||||
**⑧ 画面必须提供「配音之外的信息」**：
|
||||
如果画面只是把台词演一遍 = 观众没有看的理由。画面要承载：状态变化 / 关系 / 潜台词。
|
||||
|
||||
【台词口语化纪律（★重要，直接决定"像不像真人说话"）】
|
||||
真实的人开口**不是写作文**。台词必须像真人说话，而不是念稿：
|
||||
|
||||
❌ 书面：「我住院第三天了，真没想象中那么可怕。」
|
||||
✅ 口语：「住院第三天了……说实话，没想象中那么可怕。」
|
||||
|
||||
**七条要求**：
|
||||
1. **口语词**：用「嗯、啊、就、那个、说实话、其实、反正、怎么样」这类真实语气词
|
||||
2. **停顿与迟疑**：用「……」表示停顿，用「（停顿）」「（轻声）」标注语气
|
||||
3. **允许不完美**：可以重复（「我、我是说」）、改口、没说完（被打断）
|
||||
4. **短句为主**：一句尽量 ≤15 字；长句要拆开，像真人一口气说不完
|
||||
5. **禁书面词**：不写「因此 / 然而 / 致使 / 综上所述 / 未免」
|
||||
6. **贴合人物**：老人、孩子、医生、病人的说话方式不同（用词、语速感要区分）
|
||||
7. **一句一个意思**：一句话不做两件事（先说完这个，再接着说下个）
|
||||
|
||||
**额外**：如果这个镜头里有人物开口，必须在画面描述里同时写**说话时的身体状态**
|
||||
（呼吸急促 / 嘴角抽动 / 眼神游移 / 手不自觉地攥紧），台词和身体不能分开设计。
|
||||
|
||||
【画面描述的纪律（★重要，直接影响生成质量）】
|
||||
"visual_description" 只写**镜头里能看到什么**，严禁混入制作意图：
|
||||
❌ 禁止：「方便出结尾字幕」「保持2秒让字幕显示」「便于后期剪辑」「留白给标题」「方便加标签」
|
||||
✅ 只写：人物动作 / 表情 / 镜头运动 / 环境细节
|
||||
（字幕、时长、后期、剪辑都属于制作层，不是画面内容 —— 写进提示词会污染生成）
|
||||
|
||||
【非语言发声必须标 sfx（★重要）】
|
||||
人物发出的**非语言声音**要写进 sfx，**不要塞进台词 text**：
|
||||
· 惨叫 / 闷哼 / 痛呼 / 倒吸一口气 / 叹气 / 干笑 / 清嗓 / 吞咽 / 哽咽
|
||||
写法：{{"at_second": 1.5, "desc": "痛苦的闷哼"}}
|
||||
（text 只放"说出来的话"；发声类走 sfx —— 后期能配真实音效，比 TTS 念字自然得多）
|
||||
例：换药镜头 → text 可以是空，sfx 写 [{{"at_second": 0.8, "desc": "倒吸一口气"}}, {{"at_second": 2.0, "desc": "痛苦的闷哼"}}]
|
||||
|
||||
【空间与走位纪律（★重要，直接决定画面是否"看得懂"）】
|
||||
每个镜头必须说清「**人在空间的哪儿、朝哪、怎么动**」——三者缺一，观众就看不懂空间关系：
|
||||
- **位置**：人物处在场景的哪个锚点（如：病床右侧 / 窗前 / 门口 / 走廊中段）
|
||||
- **朝向**：面朝哪个方向（如：面向窗户 / 背对镜头 / 侧向病床 / 半侧身朝门口）
|
||||
- **移动**：这一步是静止不动 / 小幅位移 / 转身；**并写清方位变化**
|
||||
（如：从床边向窗户走两步 / 缓缓转身背对镜头）
|
||||
|
||||
对照写法：
|
||||
❌ 「他在病房里说话」（观众不知道人和空间的关系 → 换机位就乱）
|
||||
✅ 「他坐在病床右侧，面朝窗户，说到一半慢慢站起来转向门口」
|
||||
|
||||
⚠️ **单镜头内不要写长距离移动**（走 5 米这种，模型画不出来，会穿模/瞬移）：
|
||||
· 3 步以内的小幅位移 → 可以（如起身、上前一步、转身）
|
||||
· 超过 3 步 / 跨越整个空间 → **必须拆成多个镜头**：
|
||||
镜1 在床边起身（中景）→ 镜2 走向窗户（全景，能看出距离）→ 镜3 站在窗前（近景）
|
||||
且**每个镜头用对应机位的场景参考图**，空间才连贯。
|
||||
|
||||
【拍摄风格与镜头视角纪律（★重要）】
|
||||
拍摄风格决定"谁在看"，镜头语言必须与之一致，不要写错视角：
|
||||
- 📱 **手机自拍·第一人称（selfie_pov）**：镜头 = 人物自己举着的手机。
|
||||
· 画面里是"他自己"（近脸、一臂距离、轻微广角畸变）
|
||||
· 允许"看镜头说话"（对镜自述）
|
||||
· ❌ 不要写第三人称观察视角（如"镜头从侧面拍他"）
|
||||
· ⚠️ **全程统一**：包括开场/过场镜，也只能是"人物自己的视角"
|
||||
（如：举着手机照路 / 低头看自己脚下的鞋 / 手机屏幕反射中的自己），
|
||||
不要出现"从身后跟拍脚步"这类旁人视角
|
||||
- 🚶 **手持 Vlog·第一视角（vlog_handheld）**：镜头 = 拍摄者的眼睛。
|
||||
· 画面里是"他看到的"（前方、手中物、同行的人）
|
||||
· 拍摄者本人通常不入画，或只有手/手臂入画
|
||||
- 📺 **访谈纪实（interview_doc）**：机位固定，人物半身对镜说话，可有停顿与思考
|
||||
- 🎬 **电影感 / 伪纪录片**：第三人称观察视角，可用正反打 / 过肩镜头
|
||||
|
||||
{style_pack}
|
||||
{device_pack}
|
||||
{realism_discipline}
|
||||
{realism}
|
||||
{director_style}
|
||||
|
||||
【多角色一致性纪律（★重要，直接影响成片质量）】
|
||||
- **同框限制**：图生视频只能喂「一张」角色参考图 → 多角色同框的近景/特写会让次要角色人脸随机漂移。
|
||||
因此：
|
||||
· 避免「两个以上角色同框且都要清晰露脸（近景/特写）」
|
||||
· 双人戏优先用：**正反打**（单人镜头交替）· **过肩镜头**（一方背对或只露侧影）·
|
||||
**中远景**（脸小、容错高）· **背影 / 手部 / 局部**
|
||||
· 确需同框时：让"主角"占据画面视觉中心，次要角色可侧身、偏离焦平面或虚化处理
|
||||
- **出场标注**：每个镜头用 `cast_in_shot` 精确标注**实际出场**的人物代号（没出场不要写）。
|
||||
单人镜头只写一个人 —— 这是生成时绑定正确参考图（人脸一致）的前提。
|
||||
|
||||
【工作方法（先锁概念，再拆镜头）】
|
||||
第一步：先在心里锁定整片的视觉概念（色调、光线基调、镜头语言、质感），保证所有镜头统一。
|
||||
第二步：再按叙事顺序拆镜头，每个镜头要写清：
|
||||
· 画面描述：谁在哪做什么、环境里有什么、光线方向与质感（要具体到可拍）
|
||||
· 镜头契约：景别（特写/近景/中景/全景/远景）、机位角度（平视/俯视/仰视/过肩）、
|
||||
摄影机运动（固定/推/拉/摇/跟/环绕）、景深（浅/中/深）、光学质感、帧率感（常规/升格/降格）
|
||||
· 表演与情绪：情绪 + **微表情**（眼神/嘴部/眉宇各写一句）+ 人物动作
|
||||
· 台词/旁白：像真人口语（没有就空）
|
||||
· 声音设计：环境音提示（audio_cue）；需要强调的音效点（sfx，写清第几秒+什么声音）
|
||||
· 镜头衔接：与前后镜的转场（cut 硬切 / dissolve 溶接 / fade 淡变）
|
||||
· **技术需求声明 tech_requirements**（★重要）：只声明"这个镜头需要什么技术条件"，不要指定用什么模型/工具：
|
||||
- continuity：角色/服装/场景/表情/道具 各自要求（must 必须一致 / prefer 尽量 / none 不要求）
|
||||
- chain_from_prev：是否要求画面承接上一镜的最后一帧（长镜头接续/无缝延续 → true）
|
||||
- motion_level：动作幅度（static 静止 / small 小幅 / medium 中等 / large 大幅）
|
||||
- camera_difficulty：运镜难度（fixed 固定 / simple 推拉摇 / complex 环绕跟拍等复杂调度）
|
||||
- needs：其它特殊需求（自由文本，如"需要子弹时间特效""需要手部特写"）
|
||||
|
||||
第三步：检查镜头之间是否连贯（同一场景/人物/风格），有无跳轴或突兀切换。
|
||||
|
||||
【输出格式】严格输出 JSON 数组，不要任何解释：
|
||||
[
|
||||
{{
|
||||
"duration": 4,
|
||||
"type": "action | talking_head | background | vfx",
|
||||
"cast_in_shot": ["主角"],
|
||||
"visual_description": "画面描述（具体到可拍）",
|
||||
"shot_size": "景别", "angle": "机位角度", "camera_move": "摄影机运动",
|
||||
"depth_of_field": "浅/中/深", "lens_feel": "光学质感", "frame_rate_feel": "常规/升格/降格",
|
||||
"text": "台词或旁白（无则空字符串）",
|
||||
"sentiment": "情绪",
|
||||
"expression_detail": {{"eyes": "眼神", "mouth": "嘴部", "brow": "眉宇"}},
|
||||
"action": "人物动作",
|
||||
"audio_cue": "环境音提示（无则空字符串）",
|
||||
"sfx": [{{"at_second": 1.2, "desc": "关门声"}}],
|
||||
"transition_in": {{"type": "cut", "duration": 0}},
|
||||
"transition_out": {{"type": "cut", "duration": 0}},
|
||||
"speed": {{"factor": 1.0, "ramp": ""}},
|
||||
"tech_requirements": {{
|
||||
"continuity": {{"character": "must", "wardrobe": "must", "location": "must", "expression": "prefer", "props": "none"}},
|
||||
"chain_from_prev": false,
|
||||
"motion_level": "small",
|
||||
"camera_difficulty": "simple",
|
||||
"needs": []
|
||||
}}
|
||||
}}
|
||||
]
|
||||
"""
|
||||
|
||||
_REVISE_TMPL = """按导演级标准重新生成这一个镜头（保持整片视觉概念统一）。
|
||||
|
||||
【整片故事】{story}
|
||||
|
||||
【当前镜头（第 {idx} 个）】
|
||||
{current}
|
||||
|
||||
【修改要求】{instruction}
|
||||
|
||||
【要求】只改这一个镜头，输出单个 JSON 对象（字段同上）。
|
||||
"""
|
||||
|
||||
|
||||
def skill_available() -> tuple[bool, str]:
|
||||
"""技能文档是否就位 → (可用, 路径或原因)"""
|
||||
for p in SKILL_PATHS:
|
||||
if p.exists() and p.stat().st_size > 500:
|
||||
return True, str(p)
|
||||
return False, "技能文档未下载（需 ai-film-skills/docs/skills/zh-CN/ai-storyboard-director.md）"
|
||||
|
||||
|
||||
def load_skill_doc() -> str:
|
||||
ok, path = skill_available()
|
||||
if not ok:
|
||||
return ""
|
||||
try:
|
||||
txt = Path(path).read_text(encoding="utf-8")
|
||||
return txt[:_SKILL_MAX_CHARS]
|
||||
except Exception as e:
|
||||
log.warning(" ⚠️ 技能文档读取失败: %s", e)
|
||||
return ""
|
||||
|
||||
|
||||
class StoryboardV52(BaseDirector):
|
||||
"""AI Storyboard Director v5.2（技能型导演）"""
|
||||
|
||||
id = "storyboard_v52"
|
||||
name = "AI Storyboard Director v5.2"
|
||||
kind = "skill"
|
||||
description = "导演级分镜：先锁视觉概念再拆镜头；镜头契约含景别/机位/光学/运动"
|
||||
features = ["storyboard", "visual_concept", "shot_contract", "revise_shot"]
|
||||
status = "ready"
|
||||
source = "ai-film-skills（Apache-2.0）· docs/skills/zh-CN/ai-storyboard-director.md"
|
||||
options_schema = dict(COMMON_OPTIONS)
|
||||
|
||||
def __init__(self):
|
||||
self.llm = DirectorLLM(timeout=120)
|
||||
|
||||
@property
|
||||
def available(self) -> bool:
|
||||
ok, _ = skill_available()
|
||||
return ok and self.llm.available
|
||||
|
||||
# ── 生成 ─────────────────────────────────────────
|
||||
|
||||
def generate(self, story: str, anchors: GlobalAnchors | None = None,
|
||||
options: dict | None = None) -> Storyboard:
|
||||
opts = options or {}
|
||||
anchors = anchors or GlobalAnchors()
|
||||
skill = load_skill_doc()
|
||||
if not skill:
|
||||
raise RuntimeError("v5.2 技能文档未就位，无法生成")
|
||||
|
||||
prompt = _PROMPT_TMPL.format(
|
||||
shot_brief=shot_brief(opts),
|
||||
story=(story or "").strip()[:2000],
|
||||
shooting_style=opts.get("shooting_style") or "documentary_realism",
|
||||
# ★ 资产注入：有详情用详情，否则退回 ID（让导演能写出"具体"的场景）
|
||||
character=cast_or_character(anchors),
|
||||
style_pack=style_pack_block(anchors),
|
||||
device_pack=device_pack_hint(anchors),
|
||||
realism_discipline=realism_discipline(),
|
||||
realism=realism_block(),
|
||||
director_style=director_style_hint(anchors),
|
||||
location=anchors.location_context or anchors.location_id or "（未指定）",
|
||||
wardrobe=anchors.wardrobe or "（未指定）",
|
||||
props=anchors.props_context or "（无）",
|
||||
)
|
||||
raw = self.llm.chat(prompt, system=skill, temperature=0.7, max_tokens=3500)
|
||||
items = self.llm.parse_json(raw)
|
||||
if isinstance(items, dict):
|
||||
items = items.get("shots") or items.get("data") or items.get("storyboard") or []
|
||||
if not isinstance(items, list) or not items:
|
||||
log.warning(" ⚠️ v5.2 未产出有效分镜")
|
||||
return Storyboard(project_id=_pid(), story=story, director_id=self.id,
|
||||
global_anchors=anchors, shots=[])
|
||||
|
||||
shots = build_shots_from_items(items, anchors)
|
||||
durations = [s.duration for s in shots]
|
||||
if len(set(durations)) <= 1 and len(shots) > 1:
|
||||
log.warning(" ⚠️ 导演产出的镜头时长全部相同(%ss) — 可能未按内容差异化", durations[:1])
|
||||
|
||||
sb = Storyboard(
|
||||
project_id=_pid(), title=(story or "")[:40], story=story,
|
||||
content_type=opts.get("content_type") or "short_drama",
|
||||
director_id=self.id, global_anchors=anchors, shots=shots,
|
||||
)
|
||||
log.info(" 🎬 v5.2 产出 %d 个镜头（%.1f 秒）", len(shots), sb.total_duration())
|
||||
return sb
|
||||
|
||||
# ── 单镜头重生成 ──────────────────────────────────
|
||||
|
||||
def revise_shot(self, sb: Storyboard, shot_id: str,
|
||||
instruction: str = "") -> Storyboard:
|
||||
target = sb.shot_by_id(shot_id)
|
||||
if not target:
|
||||
raise ValueError(f"镜头不存在: {shot_id}")
|
||||
skill = load_skill_doc()
|
||||
import json as _json
|
||||
prompt = _REVISE_TMPL.format(
|
||||
story=(sb.story or "")[:800],
|
||||
idx=sb.shots.index(target) + 1,
|
||||
current=_json.dumps({
|
||||
"visual_description": target.visual_description,
|
||||
"shot_size": target.shot_contract.shot_size,
|
||||
"angle": target.shot_contract.angle,
|
||||
"camera_move": target.shot_contract.camera_move,
|
||||
"text": target.text, "sentiment": target.sentiment,
|
||||
}, ensure_ascii=False, indent=1),
|
||||
instruction=instruction or "更有真实感与导演感",
|
||||
)
|
||||
raw = self.llm.chat(prompt, system=skill[:2000] if skill else "", temperature=0.8, max_tokens=900)
|
||||
it = self.llm.parse_json(raw)
|
||||
if isinstance(it, list) and it:
|
||||
it = it[0]
|
||||
if not isinstance(it, dict):
|
||||
raise RuntimeError("重生成失败：LLM 未返回有效结果")
|
||||
|
||||
target.duration = float(it.get("duration") or target.duration)
|
||||
target.type = _norm_type(it.get("type") or target.type)
|
||||
target.visual_description = (it.get("visual_description") or target.visual_description).strip()
|
||||
target.shot_contract = ShotContract(
|
||||
shot_size=(it.get("shot_size") or "").strip(),
|
||||
angle=(it.get("angle") or "").strip(),
|
||||
camera_move=(it.get("camera_move") or "").strip(),
|
||||
lens_feel=(it.get("lens_feel") or "").strip(),
|
||||
depth_of_field=(it.get("depth_of_field") or "").strip(),
|
||||
)
|
||||
target.text = (it.get("text") or "").strip() or None
|
||||
target.sentiment = (it.get("sentiment") or "").strip() or None
|
||||
target.action = (it.get("action") or "").strip() or None
|
||||
return sb
|
||||
|
||||
|
||||
def _pid() -> str:
|
||||
from datetime import datetime
|
||||
return "vid_" + datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
@@ -0,0 +1,40 @@
|
||||
"""
|
||||
gap_engine — 创意缺口引擎（GapEngine 落地包）
|
||||
|
||||
把"导演 1-2 句话的诉求"变成"一批可筛选的反差创意候选"，再沉淀为「创意卡」资源。
|
||||
|
||||
模块：
|
||||
models.py 数据模型（照原文 7 类 + 候选/创意卡扩展）
|
||||
library.py 参数库 / 矩阵配置读写
|
||||
gap_calculator.py 缺口计算（原文口径）+ 联想卡 + 矩阵过滤加权
|
||||
combo_generator.py LLM 批量生成候选
|
||||
store.py 创意卡存储（创意库）
|
||||
|
||||
数据资产（可直接编辑）：
|
||||
09_ave/config/param_library.yaml 六层参数库（6层/38参数/177值）
|
||||
09_ave/config/creative_matrix.yaml 负面/正向/道具场景矩阵
|
||||
"""
|
||||
from gap_engine.models import ( # noqa: F401
|
||||
CreativeCard, ComboCandidate, CreativeConstraints, GapResult,
|
||||
Layer, Param, ParamLibrary, ParamValue, Selection,
|
||||
)
|
||||
from gap_engine.library import ( # noqa: F401
|
||||
load_library, load_matrix, save_matrix, save_library_text,
|
||||
constraints_from_matrix, generation_cfg,
|
||||
)
|
||||
from gap_engine.gap_calculator import ( # noqa: F401
|
||||
calc_gap, build_scene_card, apply_filters, matrix_snapshot,
|
||||
)
|
||||
from gap_engine.combo_generator import ( # noqa: F401
|
||||
generate_candidates, candidate_to_card, preselect_params,
|
||||
)
|
||||
from gap_engine import store # noqa: F401
|
||||
|
||||
__all__ = [
|
||||
"ParamValue", "Param", "Layer", "ParamLibrary", "Selection", "GapResult",
|
||||
"CreativeConstraints", "ComboCandidate", "CreativeCard",
|
||||
"load_library", "load_matrix", "save_matrix", "save_library_text",
|
||||
"constraints_from_matrix", "generation_cfg",
|
||||
"calc_gap", "build_scene_card", "apply_filters", "matrix_snapshot",
|
||||
"generate_candidates", "candidate_to_card", "preselect_params", "store",
|
||||
]
|
||||
@@ -0,0 +1,453 @@
|
||||
"""
|
||||
gap_engine.combo_generator — LLM 批量生成创意候选
|
||||
|
||||
流程（GapEngine「模块2 创意组合生成器」的落地实现）：
|
||||
1. 把参数库可选值清单 + 矩阵约束喂给 LLM
|
||||
2. LLM 输出 N 条候选（每条 = 六层参数选择 + 一句话联想 + 道具场景 + 敏感项标注）
|
||||
3. 用参数库把每条选择解析成 Selection（带 intensity + cue）
|
||||
4. calc_gap 算缺口分 → apply_filters 硬过滤/软标记/正向加权 → 排序
|
||||
|
||||
输出即前端"密集候选列表"的数据源。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
from lib.logger import get_logger # noqa: E402
|
||||
from gap_engine.models import ( # noqa: E402
|
||||
ComboCandidate, CreativeConstraints, ParamLibrary, Selection, utcnow,
|
||||
)
|
||||
from gap_engine.gap_calculator import calc_gap, apply_filters, build_scene_card # noqa: E402
|
||||
from gap_engine.library import load_library, load_matrix, constraints_from_matrix, generation_cfg # noqa: E402
|
||||
|
||||
log = get_logger("gap_engine")
|
||||
|
||||
_SYSTEM = """你是短剧/广告导演的「反差组合生成器」。
|
||||
|
||||
你的任务：根据导演给出的 1-2 句话场景诉求，从给定的六层参数库里挑选参数组合，
|
||||
生成一批**互不相同**的候选创意。每条候选必须能让人一眼脑补出画面。
|
||||
|
||||
硬性规则：
|
||||
1. 只能从给定参数清单里取值，不得自创值名。
|
||||
2. 每层最多选 2 个参数（不同候选之间要拉开差异）。
|
||||
3. 绝不涉及"硬违禁"类内容。
|
||||
4. 尽量避开"未放开的观众反感项"；如果某条创意非用不可，把它标进 needs_soft。
|
||||
5. 每条候选给一句 one_liner —— 用**人话**描述这个画面（导演据此判断想不想拍），
|
||||
不要堆术语。例：「术后第一天，他扶着输液架走出六亲不认的模特步，背景放着迪斯科」。
|
||||
6. 第七层「表达层」是"怎么表达"：直述是默认；也可以用**暗喻/象征/反讽/留白**制造回味。
|
||||
用隐喻时，喻体从「象征物」素材里挑（地震/潮水/笼中鸟/镜子…），**被比喻的事不要直接说出来**，
|
||||
让观众自己完成那次替代。one_liner 里要能看出这层画面（但不必解释含义）。
|
||||
|
||||
【输出纪律（务必遵守，否则内容会被截断丢弃）】
|
||||
- one_liner 控制在 **40 字以内**，一句话说清画面，不要展开解释。
|
||||
- selections 每层**最多 1 个参数**（优先选最能体现这条差异的那个），总量 ≤ 7 条。
|
||||
- 不要输出任何 JSON 之外的文字、注释或解释。
|
||||
|
||||
输出：严格 JSON 数组，不要解释文字。"""
|
||||
|
||||
_TMPL = """【场景诉求】
|
||||
{brief}
|
||||
|
||||
【六层参数清单（只能从中取值）】
|
||||
{param_list}
|
||||
|
||||
【硬违禁（绝对不可涉及）】
|
||||
{hard_list}
|
||||
|
||||
【观众反感项（当前已放开的用 ✓ 标记；未放开的尽量避开，非用不可则标 needs_soft）】
|
||||
{soft_list}
|
||||
|
||||
【可用道具】{props}
|
||||
【可用场景变量】{scenes}
|
||||
【希望传递的正向价值（命中则计入 positive_hits）】
|
||||
{positive_list}
|
||||
|
||||
请生成 {n} 条候选创意，输出 JSON 数组，每条格式：
|
||||
{{
|
||||
"one_liner": "一句话画面联想（人话，能直接脑补）",
|
||||
"selections": [
|
||||
{{"layer_id": "narrative", "param_id": "info_release", "value": "结果前置"}},
|
||||
... 每层 0~2 个 ...
|
||||
],
|
||||
"props": ["童趣道具"],
|
||||
"scenes": ["空间反差"],
|
||||
"needs_soft": [],
|
||||
"needs_hard": [],
|
||||
"positive_hits": ["warm_twist"]
|
||||
}}"""
|
||||
|
||||
|
||||
def parse_json_lenient(llm, raw: str):
|
||||
"""容错解析：先走标准解析；失败则尝试修复"被 max_tokens 截断的 JSON"
|
||||
|
||||
截断修复思路：取到最后一个完整的 `}` 为止，补上 `]`（数组被截断的情形）。
|
||||
这能救回"前面若干条完整、最后一条断掉"的常见情况。
|
||||
"""
|
||||
r = llm.parse_json(raw)
|
||||
if r is not None:
|
||||
return r
|
||||
t = (raw or "").strip()
|
||||
if not t:
|
||||
return None
|
||||
# 去掉 markdown 围栏（可能只有开头）
|
||||
i0 = t.find("[")
|
||||
if i0 < 0:
|
||||
i0 = t.find("{")
|
||||
if i0 < 0:
|
||||
return None
|
||||
body = t[i0:]
|
||||
cut = body.rfind("}")
|
||||
if cut <= 0:
|
||||
return None
|
||||
fixed = body[:cut + 1]
|
||||
# 逐级补全：可能缺 ] 或 }
|
||||
for tail in ("\n]", "]\n]", "\n]", ""):
|
||||
try:
|
||||
return json.loads(fixed + tail)
|
||||
except Exception:
|
||||
continue
|
||||
# 再退一步：切到倒数第二个 } 再试
|
||||
cut2 = body.rfind("}", 0, cut)
|
||||
if cut2 > 0:
|
||||
for tail in ("\n]", "]"):
|
||||
try:
|
||||
return json.loads(body[:cut2 + 1] + tail)
|
||||
except Exception:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def _param_list_text(lib: ParamLibrary) -> str:
|
||||
"""参数库 → 紧凑清单文本（只给 层/参数/可选值，不给 cue 以免 prompt 过长）"""
|
||||
lines = []
|
||||
for L in lib.layers:
|
||||
lines.append(f"· {L.name}({L.id})")
|
||||
for p in L.params:
|
||||
vals = " / ".join(v.value for v in p.values)
|
||||
lines.append(f" - {p.name}({p.id}): {vals}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _matrix_texts(matrix: dict) -> tuple[str, str, str, str, str]:
|
||||
neg = matrix.get("negative") or {}
|
||||
hard = "、".join(f'{h.get("name")}({h.get("id")})' for h in (neg.get("hard") or [])) or "（无）"
|
||||
soft_items = []
|
||||
for s in (neg.get("soft") or []):
|
||||
mark = "✓已放开" if s.get("relaxed") else "未放开"
|
||||
soft_items.append(f'{s.get("name")}({s.get("id")}) [{mark}]')
|
||||
soft = "\n".join(" - " + x for x in soft_items) or " （无）"
|
||||
props = "、".join(f'{p.get("name")}' for p in (matrix.get("props") or [])) or "（无）"
|
||||
scenes = "、".join(f'{s.get("name")}' for s in (matrix.get("scenes") or [])) or "（无）"
|
||||
positive = "\n".join(f' - {p.get("name")}({p.get("id")})' for p in (matrix.get("positive") or [])) or " （无）"
|
||||
return hard, soft, props, scenes, positive
|
||||
|
||||
|
||||
def _resolve_selections(raw_sels: list[dict], lib: ParamLibrary,
|
||||
max_per_layer: int = 2) -> list[Selection]:
|
||||
"""把 LLM 的原始选择解析为 Selection（带 intensity + cue）；丢弃库里不存在的值"""
|
||||
per_layer: dict[str, int] = {}
|
||||
out: list[Selection] = []
|
||||
for rs in (raw_sels or []):
|
||||
lid = str(rs.get("layer_id", "")).strip()
|
||||
pid = str(rs.get("param_id", "")).strip()
|
||||
val = str(rs.get("value", "")).strip()
|
||||
if not (lid and pid and val):
|
||||
continue
|
||||
if max_per_layer and per_layer.get(lid, 0) >= max_per_layer:
|
||||
continue
|
||||
pv = lib.find_value(lid, pid, val)
|
||||
if pv is None:
|
||||
continue # LLM 编造的值 → 丢弃
|
||||
per_layer[lid] = per_layer.get(lid, 0) + 1
|
||||
out.append(Selection(
|
||||
layer_id=lid, param_id=pid,
|
||||
param_name=lib.param_name(lid, pid),
|
||||
value=pv.value, intensity=pv.intensity, cue=pv.cue,
|
||||
))
|
||||
return out
|
||||
|
||||
|
||||
def generate_candidates(brief: str, n: int = 0,
|
||||
matrix: Optional[dict] = None,
|
||||
lib: Optional[ParamLibrary] = None,
|
||||
llm=None,
|
||||
constraint_selections: Optional[list[dict]] = None) -> dict:
|
||||
"""生成候选创意（完整流水线）
|
||||
|
||||
constraint_selections: 导演在 Tab① 定好的参数（作为硬框架约束每条候选）；
|
||||
为空 = 完全随机模式（AI 自由配参数）
|
||||
返回 {candidates:[...], meta:{...}}；candidates 已排序（未过滤在前、final_score 降序）
|
||||
"""
|
||||
brief = (brief or "").strip()
|
||||
if not brief:
|
||||
raise ValueError("需要输入场景诉求（1-2 句话）")
|
||||
|
||||
matrix = matrix or load_matrix()
|
||||
lib = lib or load_library()
|
||||
gen = generation_cfg(matrix)
|
||||
n = int(n or gen.get("candidates_per_round") or 30)
|
||||
max_per_layer = int(gen.get("max_params_per_layer") or 2)
|
||||
|
||||
if llm is None:
|
||||
from directors.llm import DirectorLLM
|
||||
llm = DirectorLLM(timeout=180)
|
||||
if not getattr(llm, "available", True):
|
||||
raise RuntimeError("LLM 未配置（请先在设置中配置大模型 Key）")
|
||||
|
||||
hard, soft, props, scenes, positive = _matrix_texts(matrix)
|
||||
param_list = _param_list_text(lib)
|
||||
|
||||
# 参数框架块（按当前参数生成时使用）
|
||||
constraint_block = ""
|
||||
if constraint_selections:
|
||||
_lines = []
|
||||
for s in constraint_selections:
|
||||
nm = s.get("param_name") or s.get("param_id", "")
|
||||
_lines.append(f"· {nm}（{s.get('layer_id','')}/{s.get('param_id','')}）= {s.get('value','')}")
|
||||
if _lines:
|
||||
constraint_block = (
|
||||
"\n【必须遵守的参数框架（导演已定，这是「做法」）】\n"
|
||||
+ "\n".join(_lines)
|
||||
+ "\n每条候选都**必须**使用这些参数值，在此框架内编不同的点子；"
|
||||
"未列出的参数可以自由发挥以拉开差异。\n"
|
||||
)
|
||||
|
||||
# 分批生成（单批 ≤ batch_size 条）——避免长输出被 max_tokens 截断导致整批丢弃
|
||||
batch_size = max(5, int(gen.get("batch_size") or 15))
|
||||
items: list = []
|
||||
for start in range(0, n, batch_size):
|
||||
bn = min(batch_size, n - len(items))
|
||||
if bn <= 0:
|
||||
break
|
||||
p = _TMPL.format(
|
||||
brief=brief, param_list=param_list, hard_list=hard,
|
||||
soft_list=soft, props=props, scenes=scenes, positive_list=positive, n=bn,
|
||||
)
|
||||
if constraint_block:
|
||||
p += constraint_block
|
||||
if items:
|
||||
p += f"\n\n（前面已给出 {len(items)} 条，本批 **必须不同**，不要重复前面的创意。）"
|
||||
log.info("生成候选 第 %d 批（%d 条）…", start // batch_size + 1, bn)
|
||||
raw = llm.chat(p, system=_SYSTEM, temperature=0.9, max_tokens=8000)
|
||||
got = parse_json_lenient(llm, raw)
|
||||
if isinstance(got, dict):
|
||||
got = got.get("candidates") or got.get("combos") or got.get("data") or []
|
||||
if not isinstance(got, list):
|
||||
log.warning(" 第 %d 批解析失败（输出 %d 字符），跳过", start // batch_size + 1, len(raw or ""))
|
||||
continue
|
||||
items.extend(got)
|
||||
|
||||
if not items:
|
||||
raise RuntimeError("LLM 未返回有效候选列表（可能是输出被截断，请减少条数后重试）")
|
||||
|
||||
constraints = constraints_from_matrix(matrix)
|
||||
candidates: list[ComboCandidate] = []
|
||||
seen: set[str] = set()
|
||||
for i, it in enumerate(items):
|
||||
if not isinstance(it, dict):
|
||||
continue
|
||||
sels = _resolve_selections(it.get("selections") or [], lib, max_per_layer)
|
||||
if len(sels) < 2:
|
||||
continue # 少于 2 个参数无法计算缺口（照原文校验）
|
||||
one = (it.get("one_liner") or "").strip()
|
||||
key = one or "|".join(f"{s.param_id}:{s.value}" for s in sels)
|
||||
if key in seen:
|
||||
continue # 去重
|
||||
seen.add(key)
|
||||
|
||||
gap = calc_gap(sels)
|
||||
# 关键联想点：取强度最高的 2 个 + 最低的 1 个（两端并置 = 反差所在）
|
||||
_ss = sorted(sels, key=lambda s: -(s.intensity or 0))
|
||||
_kp = []
|
||||
if _ss:
|
||||
_kp.append(_ss[0].cue)
|
||||
if len(_ss) >= 3:
|
||||
_kp.append(_ss[1].cue)
|
||||
_kp.append(_ss[-1].cue)
|
||||
elif len(_ss) == 2:
|
||||
_kp.append(_ss[-1].cue)
|
||||
key_points = [k for k in _kp if k][:3]
|
||||
|
||||
cand = ComboCandidate(
|
||||
cid=f"c{i+1:02d}",
|
||||
one_liner=one,
|
||||
key_points=key_points,
|
||||
selections=sels,
|
||||
gap_score=gap.gap_score,
|
||||
extreme_score=gap.extreme_score,
|
||||
variance=gap.variance,
|
||||
level=gap.level,
|
||||
scene_card=gap.scene_card,
|
||||
needs_soft=[str(x) for x in (it.get("needs_soft") or [])],
|
||||
needs_hard=[str(x) for x in (it.get("needs_hard") or [])],
|
||||
positive_hits=[str(x) for x in (it.get("positive_hits") or [])],
|
||||
props=[str(x) for x in (it.get("props") or [])],
|
||||
scenes=[str(x) for x in (it.get("scenes") or [])],
|
||||
)
|
||||
candidates.append(cand)
|
||||
|
||||
apply_filters(candidates, constraints, rebuild_card=False)
|
||||
kept = [c for c in candidates if not c.filtered]
|
||||
log.info("生成完成：%d 条（硬过滤掉 %d 条）", len(kept), len(candidates) - len(kept))
|
||||
return {
|
||||
"candidates": [c.model_dump() for c in candidates],
|
||||
"meta": {
|
||||
"brief": brief, "requested": n, "generated": len(candidates),
|
||||
"mode": "constrained" if constraint_selections else "random",
|
||||
"constraint_count": len(constraint_selections or []),
|
||||
"kept": len(kept), "filtered": len(candidates) - len(kept),
|
||||
"relaxed_soft": constraints.relaxed_soft_ids(),
|
||||
"at": utcnow(),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def candidate_to_card(candidate: ComboCandidate, brief: str = "",
|
||||
title: str = "", matrix: Optional[dict] = None,
|
||||
inject_mode: str = "reference"):
|
||||
"""候选 → 创意卡（入库用）"""
|
||||
from gap_engine.models import CreativeCard
|
||||
from gap_engine.gap_calculator import matrix_snapshot
|
||||
m = matrix or load_matrix()
|
||||
return CreativeCard(
|
||||
title=title or (candidate.one_liner[:30] or brief[:30]),
|
||||
one_liner=candidate.one_liner,
|
||||
brief=brief,
|
||||
selections=candidate.selections,
|
||||
props=candidate.props, scenes=candidate.scenes,
|
||||
gap_score=candidate.gap_score, variance=candidate.variance,
|
||||
level=candidate.level, scene_card=candidate.scene_card,
|
||||
positive_hits=candidate.positive_hits,
|
||||
matrix_snapshot=matrix_snapshot(m),
|
||||
inject_mode=inject_mode,
|
||||
status="selected",
|
||||
created_at=utcnow(), updated_at=utcnow(),
|
||||
)
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# AI 参数预选（Tab① 用：先给基础搭配，人再微调）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
_PRESELECT_SYSTEM = """你是短剧/广告导演的「参数预选助手」。
|
||||
|
||||
导演不想从上百个参数里一个个挑。你要根据他的场景诉求 + 他设的矩阵边界，
|
||||
直接给出一套**可直接开拍的基础参数搭配**（每层 1-2 个参数），他再在你的基础上微调。
|
||||
|
||||
要求：
|
||||
1. 只能从给定参数清单里取值，不得自创。
|
||||
2. 每层选 1-2 个参数（不必每层都选，但要覆盖叙事、表达、视觉、声音这几个主干）。
|
||||
3. 目标是「反差较大但拍得出来」——不追求安全，也不故意猎奇。
|
||||
4. 必须尊重矩阵边界：硬违禁绝不涉及；未放开的观众反感项避开；向已勾选的正向价值倾斜。
|
||||
5. rationale 用一句人话说清"这套搭配想干嘛"。
|
||||
|
||||
【三套必须拉开截然不同的反差层级（这是重点）】
|
||||
- 方案一「大胆」：整体远离常规——多个参数都取高强度值，让整条片子"质地异样"（如：象征+大特写+长沉默+逆光）。
|
||||
- 方案二「落差」：把常规项与反常规项并置，制造"预期落差"（如：严肃题材 + 儿歌 BGM）。
|
||||
- 方案三「克制」：整体贴近真实，只在一处藏一个小反常，靠细节取胜。
|
||||
三套的**参数选择要明显不同**，让导演一眼能选出方向；不要三套都用同一种思路。
|
||||
|
||||
【输出纪律（务必遵守，否则内容会被截断丢弃）】
|
||||
- rationale 控制在 **40 字以内**。
|
||||
- 每套 selections **每层最多 1 个参数**，总量 5-7 条。
|
||||
- 不要输出任何 JSON 之外的文字、注释、解释。
|
||||
|
||||
输出严格 JSON 数组。"""
|
||||
|
||||
_PRESELECT_TMPL = """【场景诉求】
|
||||
{brief}
|
||||
|
||||
【六层参数清单（只能从中取值）】
|
||||
{param_list}
|
||||
|
||||
【硬违禁（绝不涉及）】
|
||||
{hard_list}
|
||||
|
||||
【观众反感项（✓=已放开，可用来制造反差；未放开=避开）】
|
||||
{soft_list}
|
||||
|
||||
【希望倾斜的正向价值】
|
||||
{positive_list}
|
||||
|
||||
请给出 {n} 套**基础搭配方案**（风格拉开差异，让导演一眼能选出方向），
|
||||
输出 JSON 数组：
|
||||
[
|
||||
{{
|
||||
"name": "方案A · 短标题（如：冷眼旁观式）",
|
||||
"rationale": "一句话说明这套搭配的思路与效果",
|
||||
"selections": [
|
||||
{{"layer_id": "narrative", "param_id": "info_release", "value": "结果前置"}},
|
||||
... 每层 1~2 个 ...
|
||||
]
|
||||
}}
|
||||
]"""
|
||||
|
||||
|
||||
def preselect_params(brief: str, matrix: Optional[dict] = None,
|
||||
lib: Optional[ParamLibrary] = None,
|
||||
llm=None, n_schemes: int = 3) -> dict:
|
||||
"""AI 预选：按场景 + 矩阵给 N 套七层参数基础搭配（含缺口分）
|
||||
|
||||
返回 {schemes:[{name, rationale, selections, gap_score, variance, level, scene_card}], meta}
|
||||
"""
|
||||
brief = (brief or "").strip()
|
||||
if not brief:
|
||||
raise ValueError("需要先填写场景诉求（1-2 句话）")
|
||||
|
||||
matrix = matrix or load_matrix()
|
||||
lib = lib or load_library()
|
||||
|
||||
if llm is None:
|
||||
from directors.llm import DirectorLLM
|
||||
llm = DirectorLLM(timeout=180)
|
||||
if not getattr(llm, "available", True):
|
||||
raise RuntimeError("LLM 未配置（请先在设置中配置大模型 Key）")
|
||||
|
||||
hard, soft, _props, _scenes, positive = _matrix_texts(matrix)
|
||||
prompt = _PRESELECT_TMPL.format(
|
||||
brief=brief, param_list=_param_list_text(lib), hard_list=hard,
|
||||
soft_list=soft, positive_list=positive, n=max(1, int(n_schemes)),
|
||||
)
|
||||
log.info("AI 预选参数…（场景: %s）", brief[:40])
|
||||
raw = llm.chat(prompt, system=_PRESELECT_SYSTEM, temperature=0.8, max_tokens=4000)
|
||||
items = parse_json_lenient(llm, raw)
|
||||
if isinstance(items, dict):
|
||||
items = items.get("schemes") or items.get("plans") or items.get("data") or []
|
||||
if not isinstance(items, list):
|
||||
raise RuntimeError("LLM 未返回有效方案")
|
||||
|
||||
gen = generation_cfg(matrix)
|
||||
max_per_layer = int(gen.get("max_params_per_layer") or 2)
|
||||
|
||||
schemes: list[dict] = []
|
||||
for it in items:
|
||||
if not isinstance(it, dict):
|
||||
continue
|
||||
sels = _resolve_selections(it.get("selections") or [], lib, max_per_layer)
|
||||
if len(sels) < 2:
|
||||
continue
|
||||
g = calc_gap(sels)
|
||||
schemes.append({
|
||||
"name": (it.get("name") or f"方案{len(schemes)+1}")[:40],
|
||||
"rationale": (it.get("rationale") or "").strip(),
|
||||
"selections": [s.model_dump() for s in sels],
|
||||
"gap_score": g.gap_score,
|
||||
"extreme_score": g.extreme_score,
|
||||
"variance": g.variance,
|
||||
"level": g.level,
|
||||
"scene_card": g.scene_card,
|
||||
})
|
||||
|
||||
if not schemes:
|
||||
raise RuntimeError("预选失败：未得到有效搭配")
|
||||
|
||||
schemes.sort(key=lambda s: -(s.get("gap_score") or 0))
|
||||
log.info("预选完成：%d 套（最高缺口 %.1f）", len(schemes), schemes[0]["gap_score"])
|
||||
return {"schemes": schemes, "meta": {"brief": brief, "count": len(schemes), "at": utcnow()}}
|
||||
@@ -0,0 +1,168 @@
|
||||
"""
|
||||
gap_engine.gap_calculator — 缺口计算 + 联想卡 + 矩阵过滤/加权
|
||||
|
||||
缺口算法（照 GapEngine 原文口径，未改）：
|
||||
intensities = 被选中参数的 intensity 列表（每层选 0-2 个，不选=中性，不进计算）
|
||||
variance = statistics.variance(intensities)
|
||||
gap_score = min(100, variance * 400) # 理论最大 0.25 × 400 = 100
|
||||
high_count = intensity >= 0.6 的个数
|
||||
low_count = intensity <= 0.3 的个数
|
||||
scene_card = 按层序拼接各值的 cue + 依方差给「缺口显著/中等/微弱」总结
|
||||
|
||||
过滤/加权（原文设计了模块但无代码，此处实现）：
|
||||
硬过滤 = 候选涉及 🔴 锁定项 → 淘汰
|
||||
软风险 = 候选需要放开某 🟡 项但用户未放开 → 标记（不淘汰）
|
||||
正向加权= 命中正向项 → final_score = gap_score + Σ bonus
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import statistics
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
from lib.logger import get_logger # noqa: E402
|
||||
from gap_engine.models import ( # noqa: E402
|
||||
ComboCandidate, CreativeConstraints, GapResult, Selection,
|
||||
)
|
||||
|
||||
log = get_logger("gap_engine")
|
||||
|
||||
# 层顺序（联想卡按此顺序拼接，照原文）
|
||||
LAYER_ORDER = ["narrative", "rhetoric", "visual", "motion", "lighting", "performance", "sound"]
|
||||
|
||||
# 缺口强弱阈值（照原文：variance > 0.06 显著 / > 0.03 中等 / 否则微弱）
|
||||
LEVEL_HIGH = 0.06
|
||||
LEVEL_MID = 0.03
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 缺口计算（照原文）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
def calc_gap(selections: list[Selection]) -> GapResult:
|
||||
"""一次选择的缺口计算"""
|
||||
intensities = [s.intensity for s in selections]
|
||||
if len(intensities) < 2:
|
||||
variance = 0.0
|
||||
else:
|
||||
variance = statistics.variance(intensities)
|
||||
|
||||
gap_score = min(100.0, variance * 400)
|
||||
|
||||
# 极端度：各选中值偏离"中性预期(0.5)"的平均程度 —— 捕捉"整体异质"型反差
|
||||
# （全取高反常规值时方差小但异质度高，原文方差法会漏掉这类）
|
||||
if intensities:
|
||||
extreme_score = min(100.0, (sum(abs(i - 0.5) for i in intensities) / len(intensities)) * 200)
|
||||
else:
|
||||
extreme_score = 0.0
|
||||
high_count = sum(1 for i in intensities if i >= 0.6)
|
||||
low_count = sum(1 for i in intensities if i <= 0.3)
|
||||
|
||||
card, level = build_scene_card(selections)
|
||||
return GapResult(
|
||||
selections=list(selections),
|
||||
variance=round(variance, 4),
|
||||
gap_score=round(gap_score, 1),
|
||||
extreme_score=round(extreme_score, 1),
|
||||
high_count=high_count,
|
||||
low_count=low_count,
|
||||
scene_card=card,
|
||||
level=level,
|
||||
)
|
||||
|
||||
|
||||
def build_scene_card(selections: list[Selection]) -> tuple[str, str]:
|
||||
"""把选中的参数值翻译成一段连贯的画面描述（照原文），返回 (卡片文本, 强弱级别)"""
|
||||
sorted_sels = sorted(
|
||||
selections,
|
||||
key=lambda s: LAYER_ORDER.index(s.layer_id) if s.layer_id in LAYER_ORDER else 99,
|
||||
)
|
||||
body = "。".join(s.cue for s in sorted_sels if s.cue)
|
||||
if body and not body.endswith("。"):
|
||||
body += "。"
|
||||
|
||||
intensities = [s.intensity for s in selections]
|
||||
variance = statistics.variance(intensities) if len(intensities) >= 2 else 0.0
|
||||
if variance > LEVEL_HIGH:
|
||||
level = "显著"
|
||||
summary = ("\n\n【缺口显著】这个组合的参数预期强度差异很大，"
|
||||
"观众会在「预期」和「实际」之间产生强烈的认知落差。")
|
||||
elif variance > LEVEL_MID:
|
||||
level = "中等"
|
||||
summary = ("\n\n【缺口中等】有反差但不极端，"
|
||||
"适合需要平衡真实感和吸引力的场景。")
|
||||
else:
|
||||
level = "微弱"
|
||||
summary = ("\n\n【缺口微弱】参数预期一致，画面和谐但可能缺乏冲击力。"
|
||||
"如果想制造反差，需要调整某个参数的方向。")
|
||||
return body + summary, level
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 矩阵过滤 + 正向加权（新实现）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
def apply_filters(candidates: list[ComboCandidate],
|
||||
constraints: CreativeConstraints,
|
||||
rebuild_card: bool = True) -> list[ComboCandidate]:
|
||||
"""对候选做硬过滤 + 软风险标记 + 正向加权，并排序（未过滤在前，final_score 降序）"""
|
||||
relaxed = set(constraints.relaxed_soft_ids())
|
||||
locked = set(constraints.locked_hard_ids())
|
||||
hard_name = {h["id"]: h.get("name", h["id"]) for h in constraints.negative_hard}
|
||||
soft_name = {s["id"]: s.get("name", s["id"]) for s in constraints.negative_soft}
|
||||
bonus_map = {p["id"]: float(p.get("bonus", 8) or 8) for p in constraints.positive}
|
||||
pos_name = {p["id"]: p.get("name", p["id"]) for p in constraints.positive}
|
||||
|
||||
for c in candidates:
|
||||
# ① 硬过滤：涉及 🔴 锁定项 → 淘汰
|
||||
hard_hit = [h for h in (c.needs_hard or []) if h in locked]
|
||||
if hard_hit:
|
||||
c.filtered = True
|
||||
c.filter_reason = "涉及硬违禁：" + "、".join(hard_name.get(h, h) for h in hard_hit)
|
||||
c.final_score = 0.0
|
||||
continue
|
||||
|
||||
# ② 软风险：需要放开 🟡 项但未放开 → 标记（不淘汰）
|
||||
c.risks = []
|
||||
for s in (c.needs_soft or []):
|
||||
if s in relaxed:
|
||||
continue
|
||||
c.risks.append({
|
||||
"kind": "soft", "id": s, "name": soft_name.get(s, s),
|
||||
"reason": f'需要放开"{soft_name.get(s, s)}"才能拍（当前未放开）',
|
||||
})
|
||||
|
||||
# ③ 正向加权
|
||||
c.bonus = sum(bonus_map.get(p, 0) for p in (c.positive_hits or []))
|
||||
c.positive_hits = [p for p in (c.positive_hits or []) if p in pos_name]
|
||||
c.final_score = round(max(c.gap_score or 0, c.extreme_score or 0) + c.bonus, 1)
|
||||
|
||||
# ④ 重算联想卡（若调用方未生成）
|
||||
if rebuild_card and not c.scene_card and c.selections:
|
||||
card, level = build_scene_card(c.selections)
|
||||
c.scene_card = card
|
||||
c.level = c.level or level
|
||||
|
||||
candidates.sort(key=lambda x: (x.filtered, -(x.final_score or 0)))
|
||||
return candidates
|
||||
|
||||
|
||||
def matrix_snapshot(matrix: dict) -> dict:
|
||||
"""记录本次使用的矩阵（含放开了哪些 🟡）——存进创意卡供追溯"""
|
||||
neg = matrix.get("negative") or {}
|
||||
soft = neg.get("soft") or []
|
||||
pos = matrix.get("positive") or []
|
||||
return {
|
||||
"generation": matrix.get("generation") or {},
|
||||
"negative_hard": [h.get("id") for h in (neg.get("hard") or [])],
|
||||
"negative_soft_relaxed": [s.get("id") for s in soft if s.get("relaxed")],
|
||||
"negative_soft_excluded": [s.get("id") for s in soft if not s.get("relaxed")],
|
||||
"positive": [p.get("id") for p in pos if p.get("enabled", True)],
|
||||
"preset": matrix.get("active_preset", ""),
|
||||
"at": __import__("datetime").datetime.now().isoformat(timespec="seconds"),
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
"""
|
||||
gap_engine.library — 参数库与矩阵配置的读写
|
||||
|
||||
数据来源（都可直接编辑，改动即时生效）：
|
||||
09_ave/config/param_library.yaml 六层参数库（6层/38参数/177值）
|
||||
09_ave/config/creative_matrix.yaml 矩阵配置（负面/正向/道具场景/生成参数）
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
import yaml
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
from lib.logger import get_logger # noqa: E402
|
||||
from gap_engine.models import ParamLibrary, CreativeConstraints # noqa: E402
|
||||
|
||||
log = get_logger("gap_engine")
|
||||
|
||||
_AVE_DIR = _SCRIPTS.parent
|
||||
LIB_PATH = _AVE_DIR / "config" / "param_library.yaml"
|
||||
MATRIX_PATH = _AVE_DIR / "config" / "creative_matrix.yaml"
|
||||
|
||||
# 简易缓存（按文件 mtime 失效）
|
||||
_cache: dict = {}
|
||||
|
||||
|
||||
def _load_yaml(path: Path) -> dict:
|
||||
try:
|
||||
return yaml.safe_load(path.read_text(encoding="utf-8")) or {}
|
||||
except Exception as e:
|
||||
log.warning("读取 %s 失败: %s", path.name, e)
|
||||
return {}
|
||||
|
||||
|
||||
def load_library(use_cache: bool = True) -> ParamLibrary:
|
||||
"""加载六层参数库（按 mtime 自动失效缓存）"""
|
||||
key = "library"
|
||||
try:
|
||||
mtime = LIB_PATH.stat().st_mtime
|
||||
except Exception:
|
||||
mtime = 0
|
||||
if use_cache and _cache.get(key, {}).get("mtime") == mtime:
|
||||
return _cache[key]["obj"]
|
||||
d = _load_yaml(LIB_PATH)
|
||||
lib = ParamLibrary(**d) if d.get("layers") else ParamLibrary(layers=[])
|
||||
_cache[key] = {"mtime": mtime, "obj": lib}
|
||||
return lib
|
||||
|
||||
|
||||
def load_matrix(use_cache: bool = True) -> dict:
|
||||
"""加载矩阵配置（原始 dict，含 generation / negative / positive / props / scenes / presets）"""
|
||||
key = "matrix"
|
||||
try:
|
||||
mtime = MATRIX_PATH.stat().st_mtime
|
||||
except Exception:
|
||||
mtime = 0
|
||||
if use_cache and _cache.get(key, {}).get("mtime") == mtime:
|
||||
return _cache[key]["obj"]
|
||||
d = _load_yaml(MATRIX_PATH)
|
||||
_cache[key] = {"mtime": mtime, "obj": d}
|
||||
return d
|
||||
|
||||
|
||||
def save_matrix(data: dict) -> dict:
|
||||
"""保存矩阵配置（前端 ②③④ 标签页的勾选结果）"""
|
||||
data = dict(data or {})
|
||||
data["updated"] = __import__("datetime").date.today().isoformat()
|
||||
MATRIX_PATH.write_text(
|
||||
yaml.safe_dump(data, allow_unicode=True, sort_keys=False, width=100),
|
||||
encoding="utf-8")
|
||||
_cache.pop("matrix", None)
|
||||
log.info("矩阵配置已保存")
|
||||
return load_matrix(use_cache=False)
|
||||
|
||||
|
||||
def save_library_text(text: str) -> dict:
|
||||
"""保存参数库（整份 YAML 文本；供高级编辑用）"""
|
||||
parsed = yaml.safe_load(text)
|
||||
if not parsed or not parsed.get("layers"):
|
||||
raise ValueError("参数库格式错误：缺少 layers")
|
||||
LIB_PATH.write_text(text, encoding="utf-8")
|
||||
_cache.pop("library", None)
|
||||
lib = load_library(use_cache=False)
|
||||
n_param = sum(len(L.params) for L in lib.layers)
|
||||
n_val = sum(len(p.values) for L in lib.layers for p in L.params)
|
||||
return {"layers": len(lib.layers), "params": n_param, "values": n_val}
|
||||
|
||||
|
||||
def constraints_from_matrix(matrix: Optional[dict] = None) -> CreativeConstraints:
|
||||
"""矩阵 dict → CreativeConstraints（供过滤/加权使用）"""
|
||||
m = matrix or load_matrix()
|
||||
neg = m.get("negative") or {}
|
||||
return CreativeConstraints(
|
||||
negative_hard=neg.get("hard") or [],
|
||||
negative_soft=neg.get("soft") or [],
|
||||
positive=m.get("positive") or [],
|
||||
props=m.get("props") or [],
|
||||
scenes=m.get("scenes") or [],
|
||||
)
|
||||
|
||||
|
||||
def generation_cfg(matrix: Optional[dict] = None) -> dict:
|
||||
m = matrix or load_matrix()
|
||||
return m.get("generation") or {}
|
||||
@@ -0,0 +1,160 @@
|
||||
"""
|
||||
gap_engine.models — 数据模型（照 GapEngine 原文，含创意卡扩展）
|
||||
|
||||
原文 7 个类：ParamValue / Param / Layer / ParamLibrary / Selection / GapResult / CreativeConstraints
|
||||
扩展：ComboCandidate（AI 生成的候选组合）、CreativeCard（入库的创意卡）
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any, Optional
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 参数库（照原文）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
class ParamValue(BaseModel):
|
||||
"""单个参数值：值名 + 预期强度 + 画面联想描述"""
|
||||
value: str
|
||||
intensity: float = 0.5
|
||||
cue: str = ""
|
||||
|
||||
|
||||
class Param(BaseModel):
|
||||
id: str
|
||||
name: str
|
||||
values: list[ParamValue] = Field(default_factory=list)
|
||||
|
||||
|
||||
class Layer(BaseModel):
|
||||
id: str
|
||||
name: str
|
||||
params: list[Param] = Field(default_factory=list)
|
||||
|
||||
|
||||
class ParamLibrary(BaseModel):
|
||||
layers: list[Layer] = Field(default_factory=list)
|
||||
|
||||
def find_value(self, layer_id: str, param_id: str, value: str) -> Optional[ParamValue]:
|
||||
for L in self.layers:
|
||||
if L.id != layer_id:
|
||||
continue
|
||||
for p in L.params:
|
||||
if p.id != param_id:
|
||||
continue
|
||||
for v in p.values:
|
||||
if v.value == value:
|
||||
return v
|
||||
return None
|
||||
|
||||
def param_name(self, layer_id: str, param_id: str) -> str:
|
||||
for L in self.layers:
|
||||
if L.id == layer_id:
|
||||
for p in L.params:
|
||||
if p.id == param_id:
|
||||
return p.name
|
||||
return param_id
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 选择与缺口（照原文）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
class Selection(BaseModel):
|
||||
"""一次选择：某个参数取某个值"""
|
||||
layer_id: str
|
||||
param_id: str
|
||||
param_name: str = ""
|
||||
value: str
|
||||
intensity: float
|
||||
cue: str = ""
|
||||
|
||||
|
||||
class GapResult(BaseModel):
|
||||
"""缺口计算结果（字段口径照原文）"""
|
||||
selections: list[Selection] = Field(default_factory=list)
|
||||
variance: float = 0.0
|
||||
gap_score: float = 0.0 # 0-100（方差 × 400 归一，原文口径：落差型反差）
|
||||
extreme_score: float = 0.0 # 0-100（平均偏离中性 0.5，补足"整体异质"型反差）
|
||||
high_count: int = 0 # intensity >= 0.6
|
||||
low_count: int = 0 # intensity <= 0.3
|
||||
scene_card: str = "" # 画面联想卡
|
||||
level: str = "" # 显著 / 中等 / 微弱
|
||||
|
||||
|
||||
class CreativeConstraints(BaseModel):
|
||||
"""矩阵配置（负面/正向/道具场景）—— 本文档不含生成参数"""
|
||||
negative_hard: list[dict] = Field(default_factory=list)
|
||||
negative_soft: list[dict] = Field(default_factory=list)
|
||||
positive: list[dict] = Field(default_factory=list)
|
||||
props: list[dict] = Field(default_factory=list)
|
||||
scenes: list[dict] = Field(default_factory=list)
|
||||
|
||||
def relaxed_soft_ids(self) -> list[str]:
|
||||
return [s["id"] for s in self.negative_soft if s.get("relaxed")]
|
||||
|
||||
def locked_hard_ids(self) -> list[str]:
|
||||
return [s["id"] for s in self.negative_hard if s.get("locked")]
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 候选组合（AI 生成，扩展）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
class ComboCandidate(BaseModel):
|
||||
"""一条候选创意（= 一次完整选择 + 缺口分 + 一句话联想 + 风险标记）"""
|
||||
cid: str = "" # 候选 id（批次内唯一）
|
||||
one_liner: str = "" # 核心产物：一句话联想（导演据此脑补）
|
||||
selections: list[Selection] = Field(default_factory=list)
|
||||
key_points: list[str] = Field(default_factory=list) # 关键联想点（不展开就能判断）
|
||||
gap_score: float = 0.0
|
||||
extreme_score: float = 0.0 # 整体异质度（全高强度也可能是强反差）
|
||||
variance: float = 0.0
|
||||
level: str = ""
|
||||
scene_card: str = ""
|
||||
risks: list[dict] = Field(default_factory=list) # [{kind:'soft'|'hard', id, name, reason}]
|
||||
needs_soft: list[str] = Field(default_factory=list) # LLM 标注：本条需要放开哪些 🟡 项（id）
|
||||
needs_hard: list[str] = Field(default_factory=list) # LLM 标注：本条涉及哪些 🔴 项（id）→ 命中即淘汰
|
||||
positive_hits: list[str] = Field(default_factory=list) # 命中的正向项 id
|
||||
bonus: float = 0.0 # 正向加权分
|
||||
final_score: float = 0.0 # gap_score + bonus
|
||||
props: list[str] = Field(default_factory=list) # 道具（名称）
|
||||
scenes: list[str] = Field(default_factory=list) # 场景变量（名称）
|
||||
filtered: bool = False # 是否被硬过滤淘汰
|
||||
filter_reason: str = ""
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════
|
||||
# 创意卡（入库，扩展）
|
||||
# ════════════════════════════════════════════════════════════
|
||||
|
||||
INJECT_MODES = ("strict", "reference", "idea_only")
|
||||
|
||||
|
||||
class CreativeCard(BaseModel):
|
||||
"""创意卡 = 入创意库的一条创意（可被分镜/口播/脚本流程引用）"""
|
||||
id: str = ""
|
||||
title: str = "" # 主题（一句话）
|
||||
one_liner: str = "" # 一句话联想
|
||||
brief: str = "" # 原始输入的 1-2 句话
|
||||
selections: list[Selection] = Field(default_factory=list)
|
||||
props: list[str] = Field(default_factory=list)
|
||||
scenes: list[str] = Field(default_factory=list)
|
||||
gap_score: float = 0.0
|
||||
variance: float = 0.0
|
||||
level: str = ""
|
||||
scene_card: str = ""
|
||||
positive_hits: list[str] = Field(default_factory=list)
|
||||
matrix_snapshot: dict = Field(default_factory=dict) # 本次矩阵（含放开了哪些🟡）
|
||||
inject_mode: str = "reference" # strict 严格约束 / reference 参考 / idea_only 仅创意
|
||||
status: str = "draft" # draft 草稿 / selected 已选 / used 已用于
|
||||
used_by: list[dict] = Field(default_factory=list) # [{flow, run_id, at}]
|
||||
created_at: str = ""
|
||||
updated_at: str = ""
|
||||
|
||||
|
||||
def utcnow() -> str:
|
||||
return datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
@@ -0,0 +1,122 @@
|
||||
"""
|
||||
gap_engine.store — 创意卡存储（创意库的持久化）
|
||||
|
||||
位置：agent-local/tools/ave/creatives/{id}.json（运行时数据，非人工编辑）
|
||||
一张卡一个文件，便于 git 之外的本机私有积累与迁移。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
_SCRIPTS = Path(__file__).resolve().parent.parent
|
||||
if str(_SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(_SCRIPTS))
|
||||
|
||||
from lib.logger import get_logger # noqa: E402
|
||||
from gap_engine.models import CreativeCard, utcnow # noqa: E402
|
||||
|
||||
log = get_logger("gap_engine")
|
||||
|
||||
_AGENT_LOCAL = Path(__import__("os").environ.get(
|
||||
"AGENT_LOCAL", Path.home() / "workbuddy-agent-os" / "agent-local"))
|
||||
_DIR = _AGENT_LOCAL / "tools" / "ave" / "creatives"
|
||||
|
||||
|
||||
def _ensure_dir() -> Path:
|
||||
_DIR.mkdir(parents=True, exist_ok=True)
|
||||
return _DIR
|
||||
|
||||
|
||||
def _path(cid: str) -> Path:
|
||||
return _ensure_dir() / f"{cid}.json"
|
||||
|
||||
|
||||
def create(card: CreativeCard) -> dict:
|
||||
"""新建创意卡（自动生成 id 与时间戳）"""
|
||||
if not card.id:
|
||||
card.id = "cre_" + uuid.uuid4().hex[:10]
|
||||
now = utcnow()
|
||||
card.created_at = card.created_at or now
|
||||
card.updated_at = now
|
||||
_path(card.id).write_text(
|
||||
json.dumps(card.model_dump(), ensure_ascii=False, indent=1), encoding="utf-8")
|
||||
log.info("创意卡入库: %s（%s）", card.id, card.title[:30])
|
||||
return card.model_dump()
|
||||
|
||||
|
||||
def get(cid: str) -> Optional[dict]:
|
||||
p = _path(cid)
|
||||
if not p.exists():
|
||||
return None
|
||||
try:
|
||||
return json.loads(p.read_text(encoding="utf-8"))
|
||||
except Exception as e:
|
||||
log.warning("读取创意卡失败 %s: %s", cid, e)
|
||||
return None
|
||||
|
||||
|
||||
def update(cid: str, patch: dict) -> Optional[dict]:
|
||||
cur = get(cid)
|
||||
if not cur:
|
||||
return None
|
||||
cur.update({k: v for k, v in (patch or {}).items() if k not in ("id", "created_at")})
|
||||
cur["updated_at"] = utcnow()
|
||||
_path(cid).write_text(json.dumps(cur, ensure_ascii=False, indent=1), encoding="utf-8")
|
||||
return cur
|
||||
|
||||
|
||||
def delete(cid: str) -> bool:
|
||||
p = _path(cid)
|
||||
if p.exists():
|
||||
p.unlink()
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def list_cards(q: str = "", status: str = "", inject_mode: str = "",
|
||||
limit: int = 200, sort: str = "updated_desc") -> list[dict]:
|
||||
"""创意卡列表（支持关键词/状态/注入模式筛选）"""
|
||||
out: list[dict] = []
|
||||
for f in _ensure_dir().glob("cre_*.json"):
|
||||
try:
|
||||
d = json.loads(f.read_text(encoding="utf-8"))
|
||||
except Exception:
|
||||
continue
|
||||
if status and d.get("status") != status:
|
||||
continue
|
||||
if inject_mode and d.get("inject_mode") != inject_mode:
|
||||
continue
|
||||
if q:
|
||||
blob = " ".join(str(d.get(k, "")) for k in ("title", "one_liner", "brief", "scene_card"))
|
||||
blob += " " + " ".join(d.get("props") or []) + " " + " ".join(d.get("scenes") or [])
|
||||
if q not in blob:
|
||||
continue
|
||||
out.append(d)
|
||||
key = {
|
||||
"updated_desc": lambda d: d.get("updated_at", ""),
|
||||
"gap_desc": lambda d: -(d.get("gap_score") or 0),
|
||||
"created_desc": lambda d: d.get("created_at", ""),
|
||||
}.get(sort, lambda d: d.get("updated_at", ""))
|
||||
reverse = sort != "gap_desc"
|
||||
out.sort(key=key, reverse=reverse)
|
||||
return out[:limit]
|
||||
|
||||
|
||||
def stats() -> dict:
|
||||
cards = list_cards(limit=10000)
|
||||
by_status: dict[str, int] = {}
|
||||
by_mode: dict[str, int] = {}
|
||||
for c in cards:
|
||||
by_status[c.get("status", "draft")] = by_status.get(c.get("status", "draft"), 0) + 1
|
||||
by_mode[c.get("inject_mode", "reference")] = by_mode.get(c.get("inject_mode", "reference"), 0) + 1
|
||||
gaps = [c.get("gap_score") or 0 for c in cards]
|
||||
return {
|
||||
"total": len(cards),
|
||||
"by_status": by_status,
|
||||
"by_inject_mode": by_mode,
|
||||
"avg_gap": round(sum(gaps) / len(gaps), 1) if gaps else 0,
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
# ============================================
|
||||
# AVE 演示：励志金句合集
|
||||
# ============================================
|
||||
meta:
|
||||
project_id: "vid_inspire_quotes_001"
|
||||
style: "knowledge_lecture"
|
||||
voice:
|
||||
provider: "aliyun"
|
||||
voice_id: "cosyvoice-v3.5-plus-bailian-f92cac2a41ce4fc78153c6097755a60e"
|
||||
default_emotion: "温暖坚定"
|
||||
bgm:
|
||||
provider: "mlx-audiocraft"
|
||||
mood: "inspiring"
|
||||
output:
|
||||
resolution: "1080x1920"
|
||||
fps: 30
|
||||
avatar:
|
||||
image: "~/agent-local/tools/ave/avatar/ghai_avatar.jpg"
|
||||
opening_text: "你好，今天分享几句让我醍醐灌顶的话。"
|
||||
closing_text: "关注我，一起聆听世界，我们下期再见"
|
||||
|
||||
segments:
|
||||
- id: 1
|
||||
text: "种一棵树最好的时间是十年前，其次是现在。"
|
||||
duration_sec: 6
|
||||
voice_emotion: "坚定"
|
||||
camera: "slow_zoom_in"
|
||||
material:
|
||||
source: "pexels"
|
||||
search: "trees sunlight forest nature"
|
||||
subtitles: true
|
||||
|
||||
- id: 2
|
||||
text: "人生没有白走的路，每一步都算数。"
|
||||
duration_sec: 5
|
||||
voice_emotion: "温暖"
|
||||
camera: "static"
|
||||
material:
|
||||
source: "pexels"
|
||||
search: "mountain path hiking trail"
|
||||
subtitles: true
|
||||
|
||||
- id: 3
|
||||
text: "不要等待机会，而要创造机会。"
|
||||
duration_sec: 5
|
||||
voice_emotion: "鼓励"
|
||||
camera: "push_in"
|
||||
material:
|
||||
source: "pexels"
|
||||
search: "sunrise sky hopeful"
|
||||
subtitles: true
|
||||
|
||||
- id: 4
|
||||
text: "你的潜力远比你想象的要大得多。"
|
||||
duration_sec: 5
|
||||
voice_emotion: "深沉有力"
|
||||
camera: "slow_zoom_out"
|
||||
material:
|
||||
source: "pexels"
|
||||
search: "stars night sky universe"
|
||||
subtitles: true
|
||||
|
||||
- id: 5
|
||||
text: "成功的秘诀就是坚持做对的事，哪怕进展缓慢。"
|
||||
duration_sec: 7
|
||||
voice_emotion: "温和而坚定"
|
||||
camera: "slow_pan_right"
|
||||
material:
|
||||
source: "pexels"
|
||||
search: "ocean waves calm beach"
|
||||
subtitles: true
|
||||
|
||||
- id: 6
|
||||
text: "当你觉得为时已晚的时候，恰恰是最早的时候。"
|
||||
duration_sec: 6
|
||||
voice_emotion: "启发"
|
||||
camera: "static"
|
||||
material:
|
||||
source: "pexels"
|
||||
search: "sunset golden clouds hope"
|
||||
subtitles: true
|
||||
|
||||
- id: 7
|
||||
text: "相信自己，你比你想象的更优秀。"
|
||||
duration_sec: 5
|
||||
voice_emotion: "温暖收尾"
|
||||
camera: "slow_zoom_in"
|
||||
material:
|
||||
source: "pexels"
|
||||
search: "mountain top view inspiring"
|
||||
subtitles: true
|
||||
@@ -0,0 +1,283 @@
|
||||
#!/usr/bin/env python3
|
||||
"""关键帧生成 —— 千问AI平台（DashScope）多图融合
|
||||
|
||||
2026-10-04 实测确认可用（¥0.2/张）：
|
||||
· 端点 https://maas.qianwenaiapi.com/api/v1/services/aigc/multimodal-generation/generation
|
||||
· 模型 wan2.7-image-pro（最多 9 张输入图 / 顺序模式一次 12 张）
|
||||
· 格式 input.messages[0].content[] = [{text}, {image}, {image}, ...]
|
||||
· 图片 支持 base64 data URL（无需先上传 OSS）；实测 769×1362 竖屏
|
||||
|
||||
用法（CLI 自测，会花钱）：
|
||||
python3 scripts/keyframe_gen.py --role <角色图> --scene <场景图> \
|
||||
--prompt "把图1的人放进图2的房间里，中景，平视" --out /tmp/kf.png
|
||||
python3 scripts/keyframe_gen.py --text-only --prompt "空旷病房" --out /tmp/kf.png
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
SCRIPTS = Path(__file__).resolve().parent
|
||||
if str(SCRIPTS) not in sys.path:
|
||||
sys.path.insert(0, str(SCRIPTS))
|
||||
from lib.http import _NO_PROXY_OPENER # noqa: E402
|
||||
from lib.logger import get_logger # noqa: E402
|
||||
|
||||
log = get_logger("keyframe_gen")
|
||||
|
||||
API = "https://maas.qianwenaiapi.com/api/v1/services/aigc/multimodal-generation/generation"
|
||||
DEFAULT_MODEL = "wan2.7-image-pro" # 9 图输入 / 顺序 12 张；备选 qwen-image-3.0-pro
|
||||
UA = {"User-Agent": "Mozilla/5.0 (AgentOS-Keyframe)"}
|
||||
|
||||
# ── 画幅预设（官方文档实测：size 格式为 `宽*高`，不是只有 1K/2K/4K 档位）──
|
||||
# qwen-image 系列：宽高自由 512~2048
|
||||
# wan2.5+：总像素 1280² ~ 1440²（推荐 9:16 = 960*1696）
|
||||
# wan2.2 及更早：边长 512~1440 · FLUX：仅几个固定值
|
||||
SIZE_PRESETS = {
|
||||
"竖屏 9:16 · 720P（测试推荐）": "720*1280",
|
||||
"竖屏 9:16 · 标准": "960*1696",
|
||||
"竖屏 9:16 · 1080P": "1080*1920",
|
||||
"方图 1:1": "1024*1024",
|
||||
"横屏 16:9 · 720P": "1280*720",
|
||||
"横屏 16:9 · 标准": "1696*960",
|
||||
"省算力 · 小图": "768*1024",
|
||||
}
|
||||
|
||||
DEFAULT_SIZE = "720*1280" # 竖屏 720P —— 测试期默认，省算力
|
||||
# （正式出片可切 960*1696 或 1080*1920）
|
||||
|
||||
|
||||
# ── 模型注册表（config/image_models.yaml）────────────────────
|
||||
def _models_yaml() -> Path:
|
||||
return Path(os.environ.get("AVE_IMAGE_MODELS")
|
||||
or (SCRIPTS.parent / "config" / "image_models.yaml"))
|
||||
|
||||
|
||||
def image_models() -> list[dict]:
|
||||
"""读取模型注册表（含能力注释）。读不到时回退到内置最小集合。"""
|
||||
try:
|
||||
import yaml
|
||||
d = yaml.safe_load(_models_yaml().read_text(encoding="utf-8")) or {}
|
||||
ms = d.get("models") or []
|
||||
if ms:
|
||||
return ms
|
||||
except Exception as e:
|
||||
log.warning("读取 image_models.yaml 失败: %s", e)
|
||||
return [{"id": DEFAULT_MODEL, "name": DEFAULT_MODEL, "vendor": "百炼（千问AI平台）",
|
||||
"price": "¥0.2/张", "note": "内置回退项", "supports": ["multi", "single", "text2img"],
|
||||
"limits": {}, "key_field": "aliyun"}]
|
||||
|
||||
|
||||
def models_for_mode(mode: str) -> list[dict]:
|
||||
"""某生成方式可用的模型（按 supports 过滤）"""
|
||||
m = (mode or "").strip()
|
||||
return [x for x in image_models() if m in (x.get("supports") or [])]
|
||||
|
||||
|
||||
def model_info(mid: str) -> dict:
|
||||
for x in image_models():
|
||||
if x.get("id") == mid:
|
||||
return x
|
||||
return {}
|
||||
|
||||
# ── 按「生成方式」选模型 ──────────────────────────────────
|
||||
# 多图融合 / 文生图 / 图生图 → wan2.7-image-pro（9 图输入，融合强）
|
||||
# 改造已有图（结构保持重绘）→ qwen-image-edit-plus（指令编辑，保结构）
|
||||
MODEL_BY_MODE = {
|
||||
"multi": "wan2.7-image-pro",
|
||||
"text2img": "wan2.7-image-pro",
|
||||
"single": "wan2.7-image-pro",
|
||||
"edit": "qwen-image-edit-plus",
|
||||
}
|
||||
|
||||
# ⚠️ 官方示例里 qwen-image 系列的 content 顺序是「先 image 后 text」，
|
||||
# 而 wan 系列是「先 text 后 image」。混用会导致指令被忽略。
|
||||
IMAGE_FIRST_PREFIXES = ("qwen-image",)
|
||||
|
||||
|
||||
def model_for_mode(mode: str) -> str:
|
||||
return MODEL_BY_MODE.get((mode or "").strip(), DEFAULT_MODEL)
|
||||
|
||||
|
||||
def _api_key() -> str:
|
||||
"""读取百炼/千问平台 key（agent-local 私有配置）"""
|
||||
import yaml
|
||||
p = os.environ.get("AVE_LOCAL_CONFIG") or os.path.expanduser(
|
||||
"~/workbuddy-agent-os/agent-local/tools/ave/config/local.yaml")
|
||||
try:
|
||||
return ((yaml.safe_load(open(p)) or {}).get("aliyun") or {}).get("api_key", "")
|
||||
except Exception as e:
|
||||
log.warning("读取 key 失败: %s", e)
|
||||
return ""
|
||||
|
||||
|
||||
def _assets_dir() -> Path:
|
||||
"""资产库根目录（与 Dashboard 的 AssetStore 保持一致）"""
|
||||
return Path(os.environ.get("AVE_ASSETS_DIR")
|
||||
or os.path.expanduser("~/workbuddy-agent-os/agent-local/tools/ave/assets"))
|
||||
|
||||
|
||||
def _url_to_local(s: str) -> Path | None:
|
||||
"""把「后端 URL 形式」还原成本地文件路径
|
||||
|
||||
前端资产选择器给出的是可访问 URL：
|
||||
/api/ave/assets/{kind}/{aid}/file/{name}
|
||||
它与磁盘上的
|
||||
{assets}/{kind}/{aid}/{name}
|
||||
是同一个文件。这里做还原（含 URL 解码），并保证不越出资产目录。
|
||||
"""
|
||||
from urllib.parse import unquote
|
||||
if not s.startswith("/api/ave/assets/"):
|
||||
return None
|
||||
parts = [x for x in s.split("/") if x] # api ave assets kind aid file name
|
||||
if len(parts) < 7 or parts[5] != "file":
|
||||
return None
|
||||
kind, aid, name = (unquote(parts[3]), unquote(parts[4]), unquote(parts[6]))
|
||||
base = _assets_dir().resolve()
|
||||
cand = (base / kind / aid / name)
|
||||
try:
|
||||
cand = cand.resolve()
|
||||
if not str(cand).startswith(str(base)): # 防路径穿越
|
||||
return None
|
||||
except Exception:
|
||||
return None
|
||||
return cand if cand.exists() else None
|
||||
|
||||
|
||||
def _as_image_value(path_or_url: str) -> str:
|
||||
"""参考图 → 可提交给模型的值
|
||||
|
||||
支持四种输入：
|
||||
1. http(s) URL → 原样返回
|
||||
2. data: base64 → 原样返回
|
||||
3. /api/ave/assets/… → 还原成本地路径后转 base64（前端选择器给的格式）
|
||||
4. 本地文件路径 → 转 base64
|
||||
"""
|
||||
s = str(path_or_url)
|
||||
if s.startswith(("http://", "https://", "data:")):
|
||||
return s
|
||||
p = _url_to_local(s) or Path(s)
|
||||
if not p.exists():
|
||||
raise FileNotFoundError(
|
||||
f"参考图不存在: {p}"
|
||||
+ ("（未能从 /api/ave/assets/ 形式还原出本地文件）" if s.startswith("/api/") else ""))
|
||||
b64 = base64.b64encode(p.read_bytes()).decode()
|
||||
ext = p.suffix.lower().lstrip(".") or "png"
|
||||
mime = {"jpg": "jpeg", "jpeg": "jpeg", "webp": "webp"}.get(ext, "png")
|
||||
return f"data:image/{mime};base64,{b64}"
|
||||
|
||||
|
||||
def build_keyframe(
|
||||
prompt: str,
|
||||
ref_images: list[str] | None = None,
|
||||
*,
|
||||
negative_prompt: str = "",
|
||||
size: str = DEFAULT_SIZE,
|
||||
n: int = 1,
|
||||
model: str = DEFAULT_MODEL,
|
||||
timeout: int = 180,
|
||||
) -> list[str]:
|
||||
"""生成关键帧，返回图片 URL 列表（1~n 张）
|
||||
|
||||
prompt 构图/光线/动作瞬间的描述（视觉描述，不含"运动"—— 运动属于视频阶段）
|
||||
ref_images 参考图（本地路径或 URL），顺序即提示词里的"图1/图2…"
|
||||
推荐：角色正面 + 角色 45° + 场景对应机位 [+ 道具]
|
||||
"""
|
||||
key = _api_key()
|
||||
if not key:
|
||||
raise RuntimeError("未找到 aliyun.api_key（agent-local/tools/ave/config/local.yaml）")
|
||||
|
||||
# ── 模型能力校验（依据 config/image_models.yaml）──────────
|
||||
info = model_info(model)
|
||||
if info.get("status"):
|
||||
raise RuntimeError(
|
||||
f"模型「{info.get('name', model)}」尚未接线（状态：{info['status']}）。"
|
||||
f"请先选百炼系模型，或告知我接入。")
|
||||
if info and info.get("key_field", "aliyun") != "aliyun":
|
||||
raise RuntimeError(
|
||||
f"模型「{info.get('name', model)}」用的是 {info.get('key_field')} 凭证，"
|
||||
f"当前 keyframe_gen 只实现了百炼通道。")
|
||||
mx = (info.get("limits") or {}).get("max_input_images")
|
||||
n_given = len(ref_images or [])
|
||||
if mx and n_given > int(mx):
|
||||
raise RuntimeError(
|
||||
f"模型「{info.get('name', model)}」最多接受 {mx} 张输入图，"
|
||||
f"当前给了 {n_given} 张 —— 请减少参考图，或换模型。")
|
||||
n_max = (info.get("limits") or {}).get("n_max")
|
||||
if n_max and int(n) > int(n_max):
|
||||
raise RuntimeError(f"模型「{info.get('name', model)}」一次最多出 {n_max} 张（当前 {n}）。")
|
||||
|
||||
full_text = prompt + (f"\n\n【避免】{negative_prompt}" if negative_prompt else "")
|
||||
imgs = [{"image": _as_image_value(i)} for i in (ref_images or [])]
|
||||
# 顺序按模型家族区分（见上方 IMAGE_FIRST_PREFIXES 的注释）
|
||||
if str(model).startswith(IMAGE_FIRST_PREFIXES):
|
||||
content: list[dict] = imgs + [{"text": full_text}]
|
||||
else:
|
||||
content = [{"text": full_text}] + imgs
|
||||
|
||||
payload = {
|
||||
"model": model,
|
||||
"input": {"messages": [{"role": "user", "content": content}]},
|
||||
"parameters": {"size": size, "n": int(n), "watermark": False},
|
||||
}
|
||||
log.info("🖼 关键帧生成：model=%s refs=%d size=%s n=%d",
|
||||
model, len(ref_images or []), size, n)
|
||||
req = urllib.request.Request(API, data=json.dumps(payload).encode(),
|
||||
headers={"Authorization": f"Bearer {key}", "Content-Type": "application/json"},
|
||||
method="POST")
|
||||
with _NO_PROXY_OPENER.open(req, timeout=timeout) as r:
|
||||
d = json.loads(r.read().decode())
|
||||
out: list[str] = []
|
||||
for ch in ((d.get("output") or {}).get("choices") or []):
|
||||
for c in ((ch.get("message") or {}).get("content") or []):
|
||||
if c.get("image"):
|
||||
out.append(c["image"])
|
||||
log.info("✅ 完成，产出 %d 张（usage=%s）", len(out),
|
||||
json.dumps(d.get("usage") or {}, ensure_ascii=False))
|
||||
return out
|
||||
|
||||
|
||||
def download(url: str, dest: str | Path, timeout: int = 90) -> str:
|
||||
dest = Path(dest)
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
with _NO_PROXY_OPENER.open(urllib.request.Request(url, headers=UA), timeout=timeout) as r:
|
||||
dest.write_bytes(r.read())
|
||||
log.info("已保存: %s (%d KB)", dest, dest.stat().st_size // 1024)
|
||||
return str(dest)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description="关键帧生成（多图融合）")
|
||||
ap.add_argument("--prompt", required=True, help="画面描述（视觉，不含运动）")
|
||||
ap.add_argument("--role", action="append", default=[], help="角色参考图（可多次）")
|
||||
ap.add_argument("--scene", action="append", default=[], help="场景参考图（可多次）")
|
||||
ap.add_argument("--extra", action="append", default=[], help="其他参考图（道具等）")
|
||||
ap.add_argument("--negative", default="", help="负向提示词")
|
||||
ap.add_argument("--size", default=DEFAULT_SIZE,
|
||||
help="画幅 宽*高（如 960*1696 竖屏 9:16 / 1280*720 横屏）")
|
||||
ap.add_argument("--n", type=int, default=1)
|
||||
ap.add_argument("--model", default=DEFAULT_MODEL)
|
||||
ap.add_argument("--out", default="/tmp/keyframe.png")
|
||||
ap.add_argument("--text-only", action="store_true", help="不用参考图（纯文生图）")
|
||||
args = ap.parse_args()
|
||||
|
||||
refs = [] if args.text_only else (args.role + args.scene + args.extra)
|
||||
urls = build_keyframe(args.prompt, refs, negative_prompt=args.negative,
|
||||
size=args.size, n=args.n, model=args.model)
|
||||
if not urls:
|
||||
print("❌ 未返回图片")
|
||||
return 1
|
||||
for i, u in enumerate(urls):
|
||||
dest = args.out if len(urls) == 1 else args.out.replace(".png", f"_v{i+1}.png")
|
||||
download(u, dest)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,88 @@
|
||||
#!/usr/bin/env python3
|
||||
"""生成可灵任务对账表（供与账单核对）"""
|
||||
import datetime
|
||||
import json
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
urllib.request.install_opener(opener)
|
||||
|
||||
CFG = Path.home() / "workbuddy-agent-os" / "agent-local" / "tools" / "ave" / "config" / "local.yaml"
|
||||
AVE = Path.home() / "workbuddy-agent-os" / "agent-local" / "runtime" / "ave"
|
||||
|
||||
cfg = yaml.safe_load(CFG.read_text(encoding="utf-8"))
|
||||
k = cfg["kling"]
|
||||
hdr = {"Authorization": f"Bearer {k['api_key']}"}
|
||||
base = k["base_url"]
|
||||
|
||||
with urllib.request.urlopen(
|
||||
urllib.request.Request(base + "/v1/videos/text2video", headers=hdr), timeout=30) as r:
|
||||
vids = json.loads(r.read().decode()).get("data") or []
|
||||
|
||||
local_files = []
|
||||
for p in AVE.rglob("*.mp4"):
|
||||
local_files.append((p.stat().st_mtime, str(p.relative_to(AVE))))
|
||||
local_files.sort(reverse=True)
|
||||
|
||||
NOTES = {
|
||||
"934535832286281787": "首次验证 kling-v3 可用",
|
||||
"934536473163501607": "端到端流程 wf_5d5ffff0fc06 镜头1",
|
||||
"934536909153013802": "端到端流程 wf_5d5ffff0fc06 镜头2",
|
||||
"934537556619169795": "端到端流程 wf_5d5ffff0fc06 镜头3",
|
||||
"934539639007219781": "口播线验证（oral_verify2）",
|
||||
"934540864327012427": "并发生成测试 镜头1",
|
||||
"934540864352174168": "并发生成测试 镜头2",
|
||||
"934540864385880141": "并发生成测试 镜头3",
|
||||
"934542570569990201": "流程测试",
|
||||
"934612690390495243": "P2 编排通电验证",
|
||||
"934618791945191489": "完整链路验证 镜头1",
|
||||
"934618965245431828": "完整链路验证 镜头2",
|
||||
}
|
||||
|
||||
now = datetime.datetime.now().strftime("%Y-%m-%d %H:%M")
|
||||
lines = ["# 可灵任务对账表", "", f"生成时间：{now}", "",
|
||||
"## 全部任务", "",
|
||||
"| 时间 | 任务号 | 状态 | 时长 | 用途 |",
|
||||
"|:--|:--|:--|:--|:--|"]
|
||||
|
||||
today_cnt = today_sec = 0
|
||||
rows = sorted(vids, key=lambda t: -int(t.get("created_at") or 0))
|
||||
for t in rows:
|
||||
ms = int(t.get("created_at") or 0)
|
||||
dts = datetime.datetime.fromtimestamp(ms / 1000).strftime("%Y-%m-%d %H:%M")
|
||||
vs = (t.get("task_result") or {}).get("videos") or []
|
||||
dur = float(vs[0].get("duration", 5) or 5) if vs else 5.0
|
||||
note = NOTES.get(str(t.get("task_id")), "（历史任务）")
|
||||
if dts.startswith("2026-10-01"):
|
||||
today_cnt += 1
|
||||
today_sec += dur
|
||||
lines.append(f"| {dts} | `{t.get('task_id')}` | {t.get('task_status')} | {dur:.1f}s | {note} |")
|
||||
|
||||
total_sec = sum(
|
||||
float(((t.get("task_result") or {}).get("videos") or [{}])[0].get("duration", 5) or 5)
|
||||
for t in rows
|
||||
)
|
||||
lines += ["",
|
||||
f"**今天（2026-10-01）：{today_cnt} 条 / {today_sec:.1f} 秒**",
|
||||
f"**全部：{len(rows)} 条 / {total_sec:.1f} 秒**", "",
|
||||
"## 费用估算（按不同单价档）", "",
|
||||
"| 单价档 | 今天 | 全部 |", "|:--|:--|:--|",
|
||||
f"| ¥0.6/秒（无声 720P） | ¥{today_sec * 0.6:.1f} | ¥{total_sec * 0.6:.1f} |",
|
||||
f"| ¥1.2/秒（有声 / 1080P） | ¥{today_sec * 1.2:.1f} | ¥{total_sec * 1.2:.1f} |",
|
||||
f"| ¥2.4/秒（高配） | ¥{today_sec * 2.4:.1f} | ¥{total_sec * 2.4:.1f} |",
|
||||
"",
|
||||
"> ⚠️ 以上为**估算**。准确金额请以可灵控制台「消费明细」为准。",
|
||||
"> 若单价与我估算不符，请把明细发我，我核对后调整代码里的计费参数。",
|
||||
"",
|
||||
"## 本地产物文件（最新 20 个）", ""]
|
||||
for mt, rel in local_files[:20]:
|
||||
dts = datetime.datetime.fromtimestamp(mt).strftime("%m-%d %H:%M")
|
||||
lines.append(f"- `{dts}` {rel}")
|
||||
|
||||
out = AVE / "可灵任务对账表.md"
|
||||
out.write_text("\n".join(lines), encoding="utf-8")
|
||||
print(f"✅ 已生成: {out}")
|
||||
print(f" 今天 {today_cnt} 条 / {today_sec:.1f} 秒 · 全部 {len(rows)} 条 / {total_sec:.1f} 秒")
|
||||
@@ -0,0 +1,47 @@
|
||||
"""
|
||||
AVE 配置加载器
|
||||
优先级: local.yaml > config.yaml > 默认值
|
||||
"""
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
SYNC_ROOT = Path(__file__).resolve().parents[4] # agent-sync/
|
||||
LOCAL_ROOT = SYNC_ROOT.parent / "agent-local" # agent-local/
|
||||
|
||||
|
||||
def load_config() -> dict[str, Any]:
|
||||
"""加载配置，local.yaml 覆盖 config.yaml 的对应字段"""
|
||||
config: dict[str, Any] = {}
|
||||
|
||||
# 1. 模板配置
|
||||
template_path = SYNC_ROOT / "05_tools" / "09_ave" / "config.yaml"
|
||||
if template_path.exists():
|
||||
with open(template_path) as f:
|
||||
config.update(yaml.safe_load(f) or {})
|
||||
|
||||
# 2. 本地配置（覆盖）
|
||||
local_path = LOCAL_ROOT / "tools" / "ave" / "config" / "local.yaml"
|
||||
if local_path.exists():
|
||||
with open(local_path) as f:
|
||||
local = yaml.safe_load(f) or {}
|
||||
_deep_merge(config, local)
|
||||
|
||||
return config
|
||||
|
||||
|
||||
def _deep_merge(base: dict, override: dict) -> None:
|
||||
"""递归合并字典"""
|
||||
for k, v in override.items():
|
||||
if k in base and isinstance(base[k], dict) and isinstance(v, dict):
|
||||
_deep_merge(base[k], v)
|
||||
else:
|
||||
base[k] = v
|
||||
|
||||
|
||||
def get_api_config(service: str) -> dict:
|
||||
"""获取指定服务的 API 配置"""
|
||||
cfg = load_config()
|
||||
return cfg.get(service, {})
|
||||
@@ -0,0 +1,213 @@
|
||||
"""
|
||||
AVE 成本追踪器
|
||||
|
||||
记录每次 API 调用的费用估算, 防止不知不觉扣费过多。
|
||||
|
||||
用法:
|
||||
from lib.cost_tracker import CostTracker
|
||||
tracker = CostTracker()
|
||||
tracker.log("OmniHuman", duration=4.04, resolution="480P")
|
||||
tracker.summary()
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
# ── 已知价格 (元/秒) ───────────────────────────────────────
|
||||
# 来源: 火山引擎即梦AI官方定价 (截至2026-05-09)
|
||||
PRICING = {
|
||||
# OmniHuman 1.5
|
||||
"OmniHuman_480P": 0.5, # ¥0.5/秒
|
||||
"OmniHuman_720P": 0.9, # ¥0.9/秒
|
||||
# Wan2.2 (备用)
|
||||
"Wan2.2_480P": 0.5,
|
||||
"Wan2.2_720P": 0.9,
|
||||
# DreamActor M1 (估算, 与OmniHuman同级)
|
||||
"DreamActor_480P": 0.5,
|
||||
"DreamActor_720P": 0.9,
|
||||
# CosyVoice TTS (阿里云百炼, 按字符计费约 ¥0.002/字符)
|
||||
"CosyVoice_per_char": 0.002,
|
||||
# Pexels (免费)
|
||||
"Pexels": 0,
|
||||
# Kling LipSync (fal.ai, $0.014/5s ≈ ¥0.10/s)
|
||||
"KlingLipSync_per_s": 0.10, # ¥0.10/秒 ≈ $0.014/5s
|
||||
}
|
||||
|
||||
# ── 日志路径 ───────────────────────────────────────────────
|
||||
LOG_DIR = Path(os.environ.get("AVE_LOCAL_DIR",
|
||||
str(Path.home() / "workbuddy-agent-os/agent-local"))) / "tools" / "ave" / "logs"
|
||||
LOG_DIR.mkdir(parents=True, exist_ok=True)
|
||||
LOG_FILE = LOG_DIR / "cost_log.jsonl"
|
||||
|
||||
|
||||
class CostTracker:
|
||||
"""API 成本追踪器"""
|
||||
|
||||
def __init__(self):
|
||||
self._session_log: list[dict] = []
|
||||
|
||||
def log(self, service: str, duration: float = 0, resolution: str = "480P",
|
||||
chars: int = 0, status: str = "success", note: str = ""):
|
||||
"""
|
||||
记录一次 API 调用
|
||||
|
||||
参数:
|
||||
service: API 服务名 (OmniHuman, DreamActor, CosyVoice, Pexels)
|
||||
duration: 视频/音频时长(秒)
|
||||
resolution: 分辨率 (480P/720P)
|
||||
chars: CosyVoice 字符数
|
||||
status: success/failed
|
||||
note: 备注
|
||||
"""
|
||||
# 计算费用
|
||||
if service == "CosyVoice":
|
||||
unit_price = PRICING.get("CosyVoice_per_char", 0.002)
|
||||
cost = round(chars * unit_price, 4)
|
||||
elif service == "KlingLipSync":
|
||||
unit_price = PRICING.get("KlingLipSync_per_s", 0.10)
|
||||
cost = round(duration * unit_price, 4) if status == "success" else 0.0
|
||||
elif service == "Pexels":
|
||||
cost = 0.0
|
||||
else:
|
||||
key = f"{service}_{resolution}"
|
||||
unit_price = PRICING.get(key, 0.5)
|
||||
cost = round(duration * unit_price, 4) if status == "success" else 0.0
|
||||
|
||||
record = {
|
||||
"time": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
|
||||
"service": service,
|
||||
"duration_s": duration,
|
||||
"resolution": resolution,
|
||||
"chars": chars,
|
||||
"status": status,
|
||||
"cost_yuan": cost,
|
||||
"note": note,
|
||||
}
|
||||
self._session_log.append(record)
|
||||
|
||||
# 追加到日志文件
|
||||
with open(LOG_FILE, "a", encoding="utf-8") as f:
|
||||
f.write(json.dumps(record, ensure_ascii=False) + "\n")
|
||||
|
||||
# 同步写入 Dashboard DB (如果当前有 production)
|
||||
if _current_production_id > 0:
|
||||
try:
|
||||
from lib.dashboard import log_cost as _dash_log_cost
|
||||
_dash_log_cost(
|
||||
production_id=_current_production_id,
|
||||
service=service,
|
||||
step_name=note.split(" ")[0] if " " in note else "",
|
||||
duration_s=duration,
|
||||
resolution=resolution,
|
||||
chars=chars,
|
||||
cost_yuan=cost,
|
||||
status=status,
|
||||
note=note,
|
||||
)
|
||||
except ImportError:
|
||||
pass # dashboard 模块不可用时不报错
|
||||
|
||||
# 实时输出
|
||||
if cost > 0:
|
||||
print(f" 💰 {service}: {duration:.1f}s × ¥{unit_price}/s = ¥{cost:.2f}")
|
||||
elif status == "failed":
|
||||
print(f" 💰 {service}: 失败, 不计费")
|
||||
else:
|
||||
print(f" 💰 {service}: 免费")
|
||||
|
||||
return cost
|
||||
|
||||
def summary(self) -> dict:
|
||||
"""输出本次会话的费用汇总"""
|
||||
total = sum(r["cost_yuan"] for r in self._session_log)
|
||||
success_calls = sum(1 for r in self._session_log if r["status"] == "success")
|
||||
failed_calls = sum(1 for r in self._session_log if r["status"] == "failed")
|
||||
|
||||
print("\n" + "=" * 40)
|
||||
print("💰 本次会话费用汇总")
|
||||
print("=" * 40)
|
||||
print(f" 成功调用: {success_calls} 次")
|
||||
print(f" 失败调用: {failed_calls} 次")
|
||||
print(f" 总费用: ¥{total:.2f}")
|
||||
print(f" 账户余额: 未知 (请登录火山引擎控制台查看)")
|
||||
print("=" * 40)
|
||||
|
||||
# 分服务统计
|
||||
by_service = {}
|
||||
for r in self._session_log:
|
||||
s = r["service"]
|
||||
if s not in by_service:
|
||||
by_service[s] = {"count": 0, "total_s": 0, "cost": 0.0}
|
||||
by_service[s]["count"] += 1
|
||||
by_service[s]["total_s"] += r["duration_s"] if r["status"] == "success" else 0
|
||||
by_service[s]["cost"] += r["cost_yuan"]
|
||||
|
||||
if by_service:
|
||||
print("\n 按服务:")
|
||||
for s, info in sorted(by_service.items()):
|
||||
print(f" {s:15s} {info['count']}次, {info['total_s']:.1f}s, ¥{info['cost']:.2f}")
|
||||
|
||||
return {"total": total, "calls": success_calls + failed_calls}
|
||||
|
||||
def get_session_log(self) -> list[dict]:
|
||||
return self._session_log
|
||||
|
||||
def _total_cost(self) -> float:
|
||||
"""当前会话累计费用"""
|
||||
return sum(r["cost_yuan"] for r in self._session_log)
|
||||
|
||||
|
||||
# ── Dashboard 集成 ─────────────────────────────────────────
|
||||
# 当前 production_id, 由 video_factory / main.py 设置
|
||||
# CostTracker.log() 检测到该值 > 0 时自动写 dashboard DB
|
||||
_current_production_id: int = 0
|
||||
|
||||
|
||||
def set_current_production_id(pid: int):
|
||||
"""设置当前 production ID, 后续 cost log 自动关联"""
|
||||
global _current_production_id
|
||||
_current_production_id = pid
|
||||
|
||||
|
||||
def get_current_production_id() -> int:
|
||||
return _current_production_id
|
||||
|
||||
|
||||
# ── 全局单例 (在 main.py 中使用) ────────────────────────────
|
||||
_tracker: CostTracker | None = None
|
||||
|
||||
|
||||
def get_tracker() -> CostTracker:
|
||||
global _tracker
|
||||
if _tracker is None:
|
||||
_tracker = CostTracker()
|
||||
return _tracker
|
||||
|
||||
|
||||
def print_history(days: int = 7):
|
||||
"""打印最近 N 天的费用历史"""
|
||||
if not LOG_FILE.exists():
|
||||
print("暂无费用记录")
|
||||
return
|
||||
|
||||
from datetime import timedelta
|
||||
cutoff = datetime.now() - timedelta(days=days)
|
||||
total = 0.0
|
||||
count = 0
|
||||
with open(LOG_FILE, encoding="utf-8") as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
r = json.loads(line)
|
||||
t = datetime.strptime(r["time"], "%Y-%m-%d %H:%M:%S")
|
||||
if t >= cutoff:
|
||||
total += r["cost_yuan"]
|
||||
count += 1
|
||||
print(f" [{r['time']}] {r['service']:15s} {r['duration_s']:5.1f}s ¥{r['cost_yuan']:.2f} {r['note']}")
|
||||
except (json.JSONDecodeError, KeyError, ValueError):
|
||||
pass
|
||||
|
||||
print(f"\n 近 {days} 天: {count} 次调用, 总费用 ¥{total:.2f}")
|
||||
@@ -0,0 +1,498 @@
|
||||
from __future__ import annotations
|
||||
|
||||
"""
|
||||
AVE dashboard — 生产监控数据层
|
||||
|
||||
职责:
|
||||
SQLite schema 初始化 + 埋点封装
|
||||
CLI 自动写 DB, FastAPI 后端读 DB
|
||||
|
||||
设计原则:
|
||||
- 所有操作 try/except 包裹, 失败不影响 CLI 主线
|
||||
- DB 路径: agent-local/tools/ave/data/ave.db
|
||||
- 连接池: 每次调用创建新连接 (CLI 场景, 无并发问题)
|
||||
|
||||
用法:
|
||||
from lib.dashboard import init_db, log_production, log_step, log_asset
|
||||
|
||||
init_db()
|
||||
pid = log_production(strategy="oral", script_name="test", config={"key": "val"})
|
||||
log_step(pid, "tts", "completed", cost=0.3, detail="85字")
|
||||
log_step(pid, "compose", "completed", cost=0.0, detail="2 clips")
|
||||
log_asset(pid, "generated_clip", "/tmp/out.mp4", tags=["hermit", "kling"])
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
# ── DB 路径 ───────────────────────────────────────────────
|
||||
LOCAL_ROOT = Path(os.environ.get("AVE_LOCAL_DIR",
|
||||
str(Path.home() / "workbuddy-agent-os/agent-local")))
|
||||
DB_DIR = LOCAL_ROOT / "tools" / "ave" / "data"
|
||||
DB_DIR.mkdir(parents=True, exist_ok=True)
|
||||
DB_PATH = DB_DIR / "ave.db"
|
||||
|
||||
# ── Schema ────────────────────────────────────────────────
|
||||
# 从 PLANS/DASHBOARD_DESIGN.md 导出, 6 表
|
||||
SCHEMA_SQL = """
|
||||
CREATE TABLE IF NOT EXISTS productions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
strategy TEXT NOT NULL,
|
||||
status TEXT DEFAULT 'running',
|
||||
script_path TEXT,
|
||||
script_name TEXT,
|
||||
output_path TEXT,
|
||||
duration_sec REAL,
|
||||
total_cost REAL DEFAULT 0.0,
|
||||
config_json TEXT,
|
||||
error_message TEXT,
|
||||
created_at TEXT DEFAULT (datetime('now','localtime')),
|
||||
completed_at TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS production_steps (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
production_id INTEGER NOT NULL REFERENCES productions(id),
|
||||
step_name TEXT NOT NULL,
|
||||
status TEXT DEFAULT 'pending',
|
||||
started_at TEXT,
|
||||
ended_at TEXT,
|
||||
duration_sec REAL,
|
||||
cost REAL DEFAULT 0.0,
|
||||
detail TEXT,
|
||||
error_message TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS assets (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
asset_type TEXT NOT NULL,
|
||||
name TEXT,
|
||||
file_path TEXT,
|
||||
file_size INTEGER,
|
||||
hash TEXT,
|
||||
source TEXT,
|
||||
meta_json TEXT,
|
||||
production_id INTEGER,
|
||||
created_at TEXT DEFAULT (datetime('now','localtime'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS asset_tags (
|
||||
asset_id INTEGER NOT NULL REFERENCES assets(id),
|
||||
tag TEXT NOT NULL,
|
||||
PRIMARY KEY (asset_id, tag)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS production_assets (
|
||||
production_id INTEGER NOT NULL REFERENCES productions(id),
|
||||
asset_id INTEGER NOT NULL REFERENCES assets(id),
|
||||
role TEXT,
|
||||
PRIMARY KEY (production_id, asset_id)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS cost_log (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
production_id INTEGER REFERENCES productions(id),
|
||||
step_name TEXT,
|
||||
service TEXT NOT NULL,
|
||||
duration_s REAL DEFAULT 0,
|
||||
resolution TEXT,
|
||||
chars INTEGER DEFAULT 0,
|
||||
cost_yuan REAL DEFAULT 0,
|
||||
status TEXT DEFAULT 'success',
|
||||
note TEXT,
|
||||
logged_at TEXT DEFAULT (datetime('now','localtime'))
|
||||
);
|
||||
"""
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 初始化
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
def init_db() -> bool:
|
||||
"""初始化 DB schema, 不存在则创建"""
|
||||
try:
|
||||
conn = sqlite3.connect(str(DB_PATH))
|
||||
conn.executescript(SCHEMA_SQL)
|
||||
conn.commit()
|
||||
conn.close()
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f" [dashboard] DB 初始化失败: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def _get_conn() -> sqlite3.Connection | None:
|
||||
"""获取 DB 连接 (带 try/except)"""
|
||||
try:
|
||||
conn = sqlite3.connect(str(DB_PATH))
|
||||
conn.row_factory = sqlite3.Row
|
||||
return conn
|
||||
except Exception as e:
|
||||
print(f" [dashboard] DB 连接失败: {e}")
|
||||
return None
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 埋点函数
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
def log_production(
|
||||
strategy: str,
|
||||
script_path: str = "",
|
||||
script_name: str = "",
|
||||
config: dict | None = None,
|
||||
) -> int:
|
||||
"""
|
||||
记录一条 production 开始
|
||||
|
||||
返回:
|
||||
production_id (失败返回 -1)
|
||||
"""
|
||||
conn = _get_conn()
|
||||
if not conn:
|
||||
return -1
|
||||
try:
|
||||
cur = conn.execute(
|
||||
"""INSERT INTO productions (strategy, status, script_path, script_name, config_json)
|
||||
VALUES (?, 'running', ?, ?, ?)""",
|
||||
(strategy, script_path, script_name, json.dumps(config or {}, ensure_ascii=False)),
|
||||
)
|
||||
conn.commit()
|
||||
pid = cur.lastrowid
|
||||
return pid
|
||||
except Exception as e:
|
||||
print(f" [dashboard] log_production 失败: {e}")
|
||||
return -1
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def complete_production(
|
||||
production_id: int,
|
||||
status: str = "completed",
|
||||
output_path: str = "",
|
||||
duration_sec: float = 0.0,
|
||||
total_cost: float = 0.0,
|
||||
error_message: str = "",
|
||||
):
|
||||
"""标记 production 完成"""
|
||||
conn = _get_conn()
|
||||
if not conn:
|
||||
return
|
||||
try:
|
||||
conn.execute(
|
||||
"""UPDATE productions SET status=?, output_path=?, duration_sec=?,
|
||||
total_cost=?, error_message=?, completed_at=datetime('now','localtime')
|
||||
WHERE id=?""",
|
||||
(status, output_path, duration_sec, total_cost, error_message, production_id),
|
||||
)
|
||||
conn.commit()
|
||||
except Exception as e:
|
||||
print(f" [dashboard] complete_production 失败: {e}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def log_step(
|
||||
production_id: int,
|
||||
step_name: str,
|
||||
status: str = "completed",
|
||||
cost: float = 0.0,
|
||||
detail: str = "",
|
||||
error_message: str = "",
|
||||
) -> bool:
|
||||
"""
|
||||
记录一步生产步骤
|
||||
|
||||
参数:
|
||||
production_id: 所属 production ID
|
||||
step_name: 步骤名 (tts, search_material, compose, bgm, lipsync, ...)
|
||||
status: completed / failed
|
||||
cost: 该步骤费用
|
||||
detail: 文字详情
|
||||
"""
|
||||
conn = _get_conn()
|
||||
if not conn:
|
||||
return False
|
||||
try:
|
||||
now = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
conn.execute(
|
||||
"""INSERT INTO production_steps
|
||||
(production_id, step_name, status, started_at, ended_at, cost, detail, error_message)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)""",
|
||||
(production_id, step_name, status, now, now, cost, detail, error_message),
|
||||
)
|
||||
conn.commit()
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f" [dashboard] log_step 失败: {e}")
|
||||
return False
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def log_asset(
|
||||
production_id: int,
|
||||
asset_type: str,
|
||||
file_path: str = "",
|
||||
name: str = "",
|
||||
source: str = "",
|
||||
tags: list[str] | None = None,
|
||||
meta: dict | None = None,
|
||||
) -> int:
|
||||
"""
|
||||
记录一个生成/使用的资产
|
||||
|
||||
返回:
|
||||
asset_id (失败返回 -1)
|
||||
"""
|
||||
conn = _get_conn()
|
||||
if not conn:
|
||||
return -1
|
||||
try:
|
||||
file_size = os.path.getsize(file_path) if file_path and os.path.exists(file_path) else 0
|
||||
cur = conn.execute(
|
||||
"""INSERT INTO assets (asset_type, name, file_path, file_size, source, meta_json, production_id)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)""",
|
||||
(asset_type, name, file_path, file_size, source,
|
||||
json.dumps(meta or {}, ensure_ascii=False), production_id),
|
||||
)
|
||||
aid = cur.lastrowid
|
||||
|
||||
# 关联 production ↔ asset
|
||||
conn.execute(
|
||||
"INSERT OR IGNORE INTO production_assets (production_id, asset_id, role) VALUES (?, ?, ?)",
|
||||
(production_id, aid, "output" if source == "generated" else "input"),
|
||||
)
|
||||
|
||||
# 标签
|
||||
if tags:
|
||||
for tag in tags:
|
||||
conn.execute(
|
||||
"INSERT OR IGNORE INTO asset_tags (asset_id, tag) VALUES (?, ?)",
|
||||
(aid, tag.strip().lower()),
|
||||
)
|
||||
|
||||
conn.commit()
|
||||
return aid
|
||||
except Exception as e:
|
||||
print(f" [dashboard] log_asset 失败: {e}")
|
||||
return -1
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def log_cost(
|
||||
production_id: int,
|
||||
service: str,
|
||||
step_name: str = "",
|
||||
duration_s: float = 0.0,
|
||||
resolution: str = "",
|
||||
chars: int = 0,
|
||||
cost_yuan: float = 0.0,
|
||||
status: str = "success",
|
||||
note: str = "",
|
||||
):
|
||||
"""记录费用明细"""
|
||||
conn = _get_conn()
|
||||
if not conn:
|
||||
return
|
||||
try:
|
||||
conn.execute(
|
||||
"""INSERT INTO cost_log
|
||||
(production_id, step_name, service, duration_s, resolution, chars, cost_yuan, status, note)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)""",
|
||||
(production_id, step_name, service, duration_s, resolution, chars, cost_yuan, status, note),
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
# 更新 production 累计费用
|
||||
conn.execute(
|
||||
"UPDATE productions SET total_cost = total_cost + ? WHERE id = ?",
|
||||
(cost_yuan, production_id),
|
||||
)
|
||||
conn.commit()
|
||||
except Exception as e:
|
||||
print(f" [dashboard] log_cost 失败: {e}")
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 查询 API (供 FastAPI 后端使用)
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
def get_productions(
|
||||
limit: int = 50,
|
||||
offset: int = 0,
|
||||
strategy: str | None = None,
|
||||
status: str | None = None,
|
||||
) -> list[dict]:
|
||||
"""查询 production 列表"""
|
||||
conn = _get_conn()
|
||||
if not conn:
|
||||
return []
|
||||
try:
|
||||
where = []
|
||||
params = []
|
||||
if strategy:
|
||||
where.append("strategy = ?")
|
||||
params.append(strategy)
|
||||
if status:
|
||||
where.append("status = ?")
|
||||
params.append(status)
|
||||
where_clause = (" WHERE " + " AND ".join(where)) if where else ""
|
||||
rows = conn.execute(
|
||||
f"SELECT * FROM productions{where_clause} ORDER BY id DESC LIMIT ? OFFSET ?",
|
||||
params + [limit, offset],
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
except Exception as e:
|
||||
print(f" [dashboard] get_productions 失败: {e}")
|
||||
return []
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_production_detail(production_id: int) -> dict | None:
|
||||
"""查询单条 production + steps + assets + costs"""
|
||||
conn = _get_conn()
|
||||
if not conn:
|
||||
return None
|
||||
try:
|
||||
prod = conn.execute(
|
||||
"SELECT * FROM productions WHERE id = ?", (production_id,)
|
||||
).fetchone()
|
||||
if not prod:
|
||||
return None
|
||||
|
||||
steps = conn.execute(
|
||||
"SELECT * FROM production_steps WHERE production_id = ? ORDER BY id",
|
||||
(production_id,),
|
||||
).fetchall()
|
||||
|
||||
assets = conn.execute(
|
||||
"""SELECT a.*, GROUP_CONCAT(at.tag, ',') as tags
|
||||
FROM assets a
|
||||
LEFT JOIN asset_tags at ON at.asset_id = a.id
|
||||
WHERE a.production_id = ?
|
||||
GROUP BY a.id ORDER BY a.id""",
|
||||
(production_id,),
|
||||
).fetchall()
|
||||
|
||||
costs = conn.execute(
|
||||
"SELECT * FROM cost_log WHERE production_id = ? ORDER BY id",
|
||||
(production_id,),
|
||||
).fetchall()
|
||||
|
||||
return {
|
||||
"production": dict(prod),
|
||||
"steps": [dict(s) for s in steps],
|
||||
"assets": [dict(a) for a in assets],
|
||||
"costs": [dict(c) for c in costs],
|
||||
}
|
||||
except Exception as e:
|
||||
print(f" [dashboard] get_production_detail 失败: {e}")
|
||||
return None
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_assets(asset_type: str | None = None, tag: str | None = None,
|
||||
limit: int = 100, offset: int = 0) -> list[dict]:
|
||||
"""查询资产列表"""
|
||||
conn = _get_conn()
|
||||
if not conn:
|
||||
return []
|
||||
try:
|
||||
where = []
|
||||
params = []
|
||||
if asset_type:
|
||||
where.append("a.asset_type = ?")
|
||||
params.append(asset_type)
|
||||
if tag:
|
||||
where.append("EXISTS (SELECT 1 FROM asset_tags WHERE asset_id = a.id AND tag = ?)")
|
||||
params.append(tag.lower())
|
||||
where_clause = (" WHERE " + " AND ".join(where)) if where else ""
|
||||
|
||||
rows = conn.execute(
|
||||
f"""SELECT a.*, GROUP_CONCAT(at.tag, ',') as tags
|
||||
FROM assets a
|
||||
LEFT JOIN asset_tags at ON at.asset_id = a.id
|
||||
{where_clause}
|
||||
GROUP BY a.id
|
||||
ORDER BY a.id DESC LIMIT ? OFFSET ?""",
|
||||
params + [limit, offset],
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
except Exception as e:
|
||||
print(f" [dashboard] get_assets 失败: {e}")
|
||||
return []
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_summary() -> dict:
|
||||
"""获取总览统计"""
|
||||
conn = _get_conn()
|
||||
if not conn:
|
||||
return {}
|
||||
try:
|
||||
total_prods = conn.execute("SELECT COUNT(*) FROM productions").fetchone()[0]
|
||||
completed = conn.execute("SELECT COUNT(*) FROM productions WHERE status='completed'").fetchone()[0]
|
||||
failed = conn.execute("SELECT COUNT(*) FROM productions WHERE status='failed'").fetchone()[0]
|
||||
total_cost = conn.execute("SELECT COALESCE(SUM(total_cost), 0) FROM productions").fetchone()[0]
|
||||
total_assets = conn.execute("SELECT COUNT(*) FROM assets").fetchone()[0]
|
||||
# 本月
|
||||
month_start = datetime.now().strftime("%Y-%m-01")
|
||||
month_prods = conn.execute(
|
||||
"SELECT COUNT(*) FROM productions WHERE created_at >= ?", (month_start,)
|
||||
).fetchone()[0]
|
||||
month_cost = conn.execute(
|
||||
"SELECT COALESCE(SUM(total_cost), 0) FROM productions WHERE created_at >= ?",
|
||||
(month_start,),
|
||||
).fetchone()[0]
|
||||
|
||||
return {
|
||||
"total_productions": total_prods,
|
||||
"completed": completed,
|
||||
"failed": failed,
|
||||
"total_cost": round(total_cost, 2),
|
||||
"total_assets": total_assets,
|
||||
"month_productions": month_prods,
|
||||
"month_cost": round(month_cost, 2),
|
||||
}
|
||||
except Exception as e:
|
||||
print(f" [dashboard] get_summary 失败: {e}")
|
||||
return {}
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_cost_breakdown() -> list[dict]:
|
||||
"""按策略统计费用"""
|
||||
conn = _get_conn()
|
||||
if not conn:
|
||||
return []
|
||||
try:
|
||||
rows = conn.execute(
|
||||
"""SELECT strategy,
|
||||
COUNT(*) as count,
|
||||
COALESCE(SUM(duration_sec), 0) as total_duration,
|
||||
COALESCE(SUM(total_cost), 0) as total_cost,
|
||||
ROUND(AVG(total_cost), 2) as avg_cost
|
||||
FROM productions
|
||||
GROUP BY strategy
|
||||
ORDER BY total_cost DESC"""
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
except Exception:
|
||||
return []
|
||||
finally:
|
||||
conn.close()
|
||||
@@ -0,0 +1,131 @@
|
||||
"""
|
||||
lib.ffmpeg — FFmpeg / FFprobe 路径统一入口
|
||||
=============================================
|
||||
|
||||
本机 FFmpeg 装在 `~/.local/bin`（非 brew、不在默认 PATH），
|
||||
所有模块统一从这里取路径，避免 `subprocess.run(["ffmpeg", ...])` 找不到命令。
|
||||
|
||||
用法：
|
||||
from lib.ffmpeg import FFMPEG, FFPROBE
|
||||
subprocess.run([FFMPEG, "-y", "-i", src, dst])
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
_CANDIDATES = [
|
||||
os.path.expanduser("~/.local/bin/ffmpeg"),
|
||||
"/usr/local/bin/ffmpeg",
|
||||
"/opt/homebrew/bin/ffmpeg",
|
||||
]
|
||||
_CANDIDATES_PROBE = [
|
||||
os.path.expanduser("~/.local/bin/ffprobe-real"),
|
||||
"/usr/local/bin/ffprobe",
|
||||
"/opt/homebrew/bin/ffprobe",
|
||||
]
|
||||
|
||||
|
||||
def _find_real_ffprobe() -> str | None:
|
||||
"""查找真正可用的 ffprobe
|
||||
|
||||
⚠️ 本机 `~/.local/bin/ffprobe` 实为 `ffmpeg` 的**符号链接**（不支持 -show_entries）；
|
||||
真正的 ffprobe 在别处（如 Trae/火星代码自带 6.1.1 完整版）。
|
||||
"""
|
||||
import glob
|
||||
pats = [
|
||||
os.path.expanduser("~/.marscode/**/modules/ai-agent/ffprobe"),
|
||||
os.path.expanduser("~/.trae/**/ffprobe"),
|
||||
os.path.expanduser("~/.vscode/**/ffprobe"),
|
||||
"/Applications/*.app/Contents/**/ffprobe",
|
||||
]
|
||||
for pat in pats:
|
||||
for p in sorted(glob.glob(pat, recursive=True), reverse=True):
|
||||
try:
|
||||
if os.path.isfile(p) and os.access(p, os.X_OK):
|
||||
# 验证确实支持 -show_entries（真正的 ffprobe）
|
||||
import subprocess
|
||||
r = subprocess.run([p, "-h"], capture_output=True, text=True, timeout=10)
|
||||
if "show_entries" in ((r.stdout or "") + (r.stderr or "")):
|
||||
return p
|
||||
except Exception:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
# 媒体信息探测（**用 `ffmpeg -i` 解析**）
|
||||
#
|
||||
# ⚠️ 本机 ffprobe 是精简构建：不支持 `-show_entries` / `-select_streams`
|
||||
# （会报 `Unrecognized option`）→ 相对地，`ffmpeg -i` 完全可用。
|
||||
# 因此统一改用 ffmpeg 输出解析，避免各处 ffprobe 调用**静默失败**。
|
||||
# ═══════════════════════════════════════════════════════════
|
||||
|
||||
def probe_info(media_path) -> dict:
|
||||
"""探测媒体信息 → {duration, bitrate_kbps, size_bytes, has_audio, has_video}"""
|
||||
import re
|
||||
import subprocess
|
||||
|
||||
p = Path(media_path)
|
||||
info = {"duration": 0.0, "bitrate_kbps": 0,
|
||||
"size_bytes": p.stat().st_size if p.exists() else 0,
|
||||
"has_audio": False, "has_video": False}
|
||||
if not p.exists():
|
||||
return info
|
||||
try:
|
||||
r = subprocess.run([FFMPEG, "-i", str(p)], capture_output=True, text=True, timeout=30)
|
||||
txt = (r.stderr or "") + (r.stdout or "")
|
||||
except Exception:
|
||||
return info
|
||||
m = re.search(r"Duration:\s*(\d+):(\d+):([\d.]+)", txt)
|
||||
if m:
|
||||
info["duration"] = int(m.group(1)) * 3600 + int(m.group(2)) * 60 + float(m.group(3))
|
||||
m = re.search(r"bitrate:\s*(\d+)\s*kb/s", txt)
|
||||
if m:
|
||||
info["bitrate_kbps"] = int(m.group(1))
|
||||
elif info["duration"] > 0:
|
||||
info["bitrate_kbps"] = int(info["size_bytes"] * 8 / info["duration"] / 1000)
|
||||
info["has_audio"] = "Audio:" in txt
|
||||
info["has_video"] = "Video:" in txt
|
||||
# 视频尺寸（从 "Video: h264 ..., 1080x1920" 解析）
|
||||
m = re.search(r"Video:.*?(\d{2,5})x(\d{2,5})", txt)
|
||||
if m:
|
||||
info["width"], info["height"] = int(m.group(1)), int(m.group(2))
|
||||
else:
|
||||
info["width"], info["height"] = 0, 0
|
||||
return info
|
||||
|
||||
|
||||
def probe_duration(media_path) -> float:
|
||||
"""媒体时长（秒）—— 兼容实现（ffmpeg -i 解析）"""
|
||||
return probe_info(media_path)["duration"]
|
||||
|
||||
|
||||
def has_audio_stream(media_path) -> bool:
|
||||
"""是否含音轨 —— 兼容实现"""
|
||||
return probe_info(media_path)["has_audio"]
|
||||
|
||||
|
||||
def _resolve(candidates: list[str], name: str) -> str:
|
||||
for c in candidates:
|
||||
if Path(c).exists():
|
||||
return c
|
||||
found = shutil.which(name)
|
||||
if found:
|
||||
return found
|
||||
# 找不到也返回名字（让调用时报清晰的错），但记录警告
|
||||
return name
|
||||
|
||||
|
||||
FFMPEG = _resolve(_CANDIDATES, "ffmpeg")
|
||||
# ⚠️ 本机 `~/.local/bin/ffprobe` 实为 ffmpeg 的**符号链接**（不支持 -show_entries）
|
||||
# → 优先查找**真正的** ffprobe；找不到就退回 FFMPEG（配合 probe_info 使用）
|
||||
FFPROBE = _find_real_ffprobe() or FFMPEG
|
||||
|
||||
__all__ = ["FFMPEG", "FFPROBE", "ffmpeg_available"]
|
||||
|
||||
|
||||
def ffmpeg_available() -> bool:
|
||||
"""ffmpeg 是否真的可用"""
|
||||
return Path(FFMPEG).exists() or bool(shutil.which("ffmpeg"))
|
||||
@@ -0,0 +1,135 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
AVE 文件上传助手 — 上传文件到 ghvideo Gitee 仓库获取公网直链
|
||||
|
||||
用途:
|
||||
即梦/火山引擎 API 需要图片/视频/音频的公网 HTTP URL。
|
||||
通过 Gitee raw 服务提供直链: https://gitee.com/babycalf/ghvideo/raw/main/{filename}
|
||||
|
||||
用法:
|
||||
python ghvideo_upload.py /path/to/file.jpg
|
||||
python ghvideo_upload.py --dir /path/to/dir/ # 批量上传目录下所有文件
|
||||
|
||||
返回:
|
||||
https://gitee.com/babycalf/ghvideo/raw/main/filename.ext
|
||||
"""
|
||||
import argparse
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO_DIR = Path.home() / "workbuddy-agent-os" / "agent-local" / "tools" / "ave" / "ghvideo"
|
||||
REPO_URL = "https://gitee.com/babycalf/ghvideo.git"
|
||||
RAW_BASE = "https://gitee.com/babycalf/ghvideo/raw/main"
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="上传文件到 ghvideo 中转仓库")
|
||||
parser.add_argument("path", nargs="*", help="文件或目录路径")
|
||||
parser.add_argument("--dir", help="批量上传目录")
|
||||
args = parser.parse_args()
|
||||
|
||||
files = []
|
||||
if args.dir:
|
||||
for f in sorted(Path(args.dir).iterdir()):
|
||||
if f.is_file():
|
||||
files.append(str(f))
|
||||
elif args.path:
|
||||
for p in args.path:
|
||||
if os.path.isdir(p):
|
||||
for f in sorted(Path(p).iterdir()):
|
||||
if f.is_file():
|
||||
files.append(str(f))
|
||||
else:
|
||||
files.append(p)
|
||||
else:
|
||||
parser.print_help()
|
||||
return
|
||||
|
||||
if not files:
|
||||
print("没有文件")
|
||||
return
|
||||
|
||||
# 确保仓库存在
|
||||
if not (REPO_DIR / ".git").exists():
|
||||
print(f"初始化仓库: {REPO_DIR}")
|
||||
subprocess.run(["git", "init"], cwd=REPO_DIR, capture_output=True)
|
||||
subprocess.run(["git", "remote", "add", "origin", REPO_URL], cwd=REPO_DIR, capture_output=True)
|
||||
|
||||
# 复制文件到仓库目录
|
||||
for f in files:
|
||||
src = Path(f)
|
||||
if not src.exists():
|
||||
print(f" 跳过: {f} (不存在)")
|
||||
continue
|
||||
dest = REPO_DIR / src.name
|
||||
import shutil
|
||||
shutil.copy2(str(src), str(dest))
|
||||
print(f" 复制: {src.name}")
|
||||
|
||||
# Git 提交推送
|
||||
subprocess.run(["git", "add", "-A"], cwd=REPO_DIR, capture_output=True)
|
||||
subprocess.run(["git", "commit", "-m", f"upload: {len(files)} files"], cwd=REPO_DIR, capture_output=True)
|
||||
r = subprocess.run(["git", "push", "-u", "origin", "master"], cwd=REPO_DIR,
|
||||
capture_output=True, text=True, timeout=60)
|
||||
if r.returncode == 0:
|
||||
print(f"\n✅ 推送成功! {len(files)} 个文件")
|
||||
for f in files:
|
||||
name = Path(f).name
|
||||
url = f"{RAW_BASE}/{name}"
|
||||
print(f" {url}")
|
||||
else:
|
||||
# 可能已经有内容, 尝试 pull 再 push
|
||||
print(f" 推送失败, 尝试 pull...")
|
||||
subprocess.run(["git", "pull", "--rebase", "origin", "master"], cwd=REPO_DIR,
|
||||
capture_output=True, timeout=30)
|
||||
r2 = subprocess.run(["git", "push", "-u", "origin", "master"], cwd=REPO_DIR,
|
||||
capture_output=True, text=True, timeout=60)
|
||||
if r2.returncode == 0:
|
||||
print(f"✅ 推送成功!")
|
||||
for f in files:
|
||||
name = Path(f).name
|
||||
url = f"{RAW_BASE}/{name}"
|
||||
print(f" {url}")
|
||||
else:
|
||||
print(f"❌ 推送失败: {r2.stderr[:200]}")
|
||||
|
||||
|
||||
def upload_and_get_url(file_path: str) -> str:
|
||||
"""
|
||||
上传文件到 ghvideo 仓库并返回公网直链
|
||||
|
||||
参数:
|
||||
file_path: 本地文件路径
|
||||
|
||||
返回:
|
||||
公网直链 URL
|
||||
"""
|
||||
import shutil
|
||||
name = Path(file_path).name
|
||||
dest = REPO_DIR / name
|
||||
|
||||
# 复制文件到仓库
|
||||
shutil.copy2(file_path, str(dest))
|
||||
|
||||
# 提交推送
|
||||
subprocess.run(["git", "-C", str(REPO_DIR), "add", name], capture_output=True)
|
||||
subprocess.run(["git", "-C", str(REPO_DIR), "commit", "-m", f"upload {name}"],
|
||||
capture_output=True)
|
||||
r = subprocess.run(["git", "-C", str(REPO_DIR), "push", "origin", "master"],
|
||||
capture_output=True, text=True, timeout=60)
|
||||
if r.returncode != 0:
|
||||
# pull 再试
|
||||
subprocess.run(["git", "-C", str(REPO_DIR), "pull", "--rebase", "origin", "master"],
|
||||
capture_output=True, timeout=30)
|
||||
subprocess.run(["git", "-C", str(REPO_DIR), "push", "origin", "master"],
|
||||
capture_output=True, timeout=60)
|
||||
|
||||
url = f"{RAW_BASE}/{name}"
|
||||
print(f" 上传完成: {url}")
|
||||
return url
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,93 @@
|
||||
"""
|
||||
lib/http.py — 统一 HTTP 入口（禁用系统代理 + 自动重试）
|
||||
|
||||
## 为什么需要（2026-10-04 定位的坑）
|
||||
macOS 开启系统代理（如 127.0.0.1:7892）后，Python 的 `urllib.request.urlopen`
|
||||
会**自动读取系统代理设置**。而系统代理对国内 API（可灵 / 百炼 / 火山）经常
|
||||
时通时不通，表现为：
|
||||
|
||||
[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol
|
||||
|
||||
→ 用户看到的是「生成随机失败」「分镜生成失败」这类难以复现的问题。
|
||||
|
||||
实测：直连（禁代理）3/3 次 HTTP 200、0.2~0.3 秒；走代理则间歇性 SSL 错误。
|
||||
|
||||
## 用法
|
||||
from lib.http import urlopen, request_json
|
||||
|
||||
req = urllib.request.Request(url, headers={...})
|
||||
with urlopen(req, timeout=60) as r: # 已禁代理 + 自动重试
|
||||
data = r.read()
|
||||
|
||||
# 或一步到位：
|
||||
data = request_json(url, method="POST", body={...}, headers={...})
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from typing import Any, Optional
|
||||
|
||||
# 禁代理 opener（进程内复用）
|
||||
_NO_PROXY_OPENER = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
|
||||
DEFAULT_RETRIES = 3
|
||||
DEFAULT_BACKOFF = 1.5 # 秒；第 n 次失败后等待 backoff * n
|
||||
|
||||
# 值得重试的 HTTP 状态码：
|
||||
# · 429 限流（等一会儿可能就通）
|
||||
# · 5xx 服务端临时故障（可灵/百炼偶尔返回 502/503）
|
||||
# 不重试：其余 4xx（参数错/鉴权失败/余额不足）—— 重试只是浪费时间
|
||||
_RETRYABLE_HTTP = frozenset({429, 500, 502, 503, 504})
|
||||
|
||||
|
||||
def urlopen(req, timeout: int = 60, retries: int = DEFAULT_RETRIES):
|
||||
"""禁用代理地发起请求，失败自动重试（指数退避）
|
||||
|
||||
返回的是可作上下文管理器使用的 response 对象。
|
||||
重试覆盖：URLError / SSLError / OSError / 超时 / 连接中断。
|
||||
⚠️ HTTP 4xx/5xx 不重试（那是业务错误，重试无意义）——
|
||||
本函数只对**网络层异常**重试。
|
||||
"""
|
||||
last_exc: Optional[BaseException] = None
|
||||
for attempt in range(max(1, retries)):
|
||||
try:
|
||||
return _NO_PROXY_OPENER.open(req, timeout=timeout)
|
||||
except urllib.error.HTTPError as e:
|
||||
# 5xx（服务端临时故障）与 429（限流）值得重试；
|
||||
# 其余 4xx（参数错/鉴权失败/余额不足）重试无意义 → 直接抛
|
||||
if e.code in _RETRYABLE_HTTP:
|
||||
last_exc = e
|
||||
if attempt < retries - 1:
|
||||
time.sleep(DEFAULT_BACKOFF * (attempt + 1))
|
||||
continue
|
||||
raise
|
||||
except Exception as e:
|
||||
last_exc = e
|
||||
if attempt < retries - 1:
|
||||
time.sleep(DEFAULT_BACKOFF * (attempt + 1))
|
||||
raise last_exc # type: ignore[misc]
|
||||
|
||||
|
||||
def request_json(url: str, method: str = "GET", body: Any = None,
|
||||
headers: Optional[dict] = None, timeout: int = 60,
|
||||
retries: int = DEFAULT_RETRIES) -> dict:
|
||||
"""发起 JSON 请求并解析响应（禁用代理 + 自动重试）
|
||||
|
||||
· body 非 None 时自动 JSON 序列化并加 Content-Type
|
||||
· 返回值：解析后的 dict（解析失败返回 {}）
|
||||
"""
|
||||
hdr = dict(headers or {})
|
||||
data = None
|
||||
if body is not None:
|
||||
data = json.dumps(body).encode("utf-8")
|
||||
hdr.setdefault("Content-Type", "application/json")
|
||||
req = urllib.request.Request(url, data=data, headers=hdr, method=method)
|
||||
with urlopen(req, timeout=timeout, retries=retries) as r:
|
||||
raw = r.read()
|
||||
try:
|
||||
return json.loads(raw.decode("utf-8"))
|
||||
except Exception:
|
||||
return {}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user