Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8072f4a380 | ||
|
|
91b2785c6b | ||
|
|
104964aa53 |
@@ -136,6 +136,7 @@
|
||||
## 6. 里程碑
|
||||
- **P0(本设计主体)**:designer 模式 → 自探(UI 树定位)→ 直接撰写 generic_steps draft → 服务端 schema 校验 → 前端步骤编辑器预填确认保存。验收:一句话在真实设备上生成一条可调度的 generic_steps,步骤全部来自 UI 树且编辑器可打开。
|
||||
- **P1**:`douyin_nurture` 参数预设生成(照 default_params 结构);模板沉淀:把 draft+需求写入 `agent_experience`(新列存结构化 steps 或 JSON),相似需求注入参考;整链「演示试跑」(把 steps 在设备上以受控方式跑一遍并截图回报,需新增端点,复刻 `device_busy` 拒绝语义)。
|
||||
- **进展(2026-09-10)**:动作级沉淀已落地——`agent_action` 表 + 从**成功步骤**蒸馏"命名动作"(steps 用编辑器 schema、带元素定位、禁坐标)+ 执行前按名/别名召回注入(`web/agent_api.py`)。AI 建任务可直接把这些动作当作 generic_steps 的**预制件**复用。
|
||||
- **P2**:自定义动作支持(内联展开成 group,或新增 `action_ref` 节点 + 执行器/编辑器同步);多设备并行;成本与 token 控制。
|
||||
|
||||
## 7. 实现时需新增/改动文件(规划)
|
||||
|
||||
+15
-2
@@ -1193,7 +1193,7 @@ AI 可用设备列表(在线状态 + 是否有任务运行,前端据此把 b
|
||||
|
||||
订阅事件流(SSE,EventSource)。事件:
|
||||
- `event: delta` `{text, kind: content|reasoning}` — 流式文本增量
|
||||
- `event: step` `{tool, args, image?}` — 工具调用完成(MCP 步骤,image 为缩略截图)。另有两类伪卡片:`tool="🧠 经验记忆"` 表示命中历史经验(args 形如「命中 N 条同类历史经验,已注入参考:<配方摘要>」,N 为实际条数)或本轮已写入经验库
|
||||
- `event: step` `{tool, args, image?}` — 工具调用完成(MCP 步骤,image 为缩略截图)。另有三类伪卡片:`tool="🧠 经验记忆"` 表示命中任务级经验(args 形如「命中 N 条同类历史经验,已注入参考:<配方摘要>」)或本轮已写入经验库;`tool="🧠 动作经验"` 表示命中**可复用动作**(「命中 N 个可复用动作,已注入参考:<动作名>」,执行前注入)或本轮已沉淀动作(「已沉淀 N 个可复用动作」,含元素定位、禁坐标)
|
||||
- `event: done` `{answer}` — 完成
|
||||
- `event: error` `{message}` — 失败(若因 MCP Server 未启动/不可达,message 为明确文案「MCP server(8033) 不可达 …」,不再是 SDK 原始的 `Server returned an error response`)
|
||||
- 空闲时每 15s 发一行 `: keepalive` 注释防超时;`done`/`error` 后关流
|
||||
@@ -1252,7 +1252,7 @@ AI 可用设备列表(在线状态 + 是否有任务运行,前端据此把 b
|
||||
|
||||
### GET /api/agent/experience
|
||||
|
||||
经验记忆库列表(自进化,含最近一次巡检结论)。
|
||||
经验记忆库列表(自进化,含最近一次巡检结论)。另有一张**动作经验库**表 `agent_action`(命名动作 + 编辑器 schema 步骤 + 元素定位、禁坐标):任务成功后自动从**成功步骤**蒸馏沉淀,执行前按动作名/别名召回并注入;当前无独立查询接口(命中/沉淀在 AI 控制台的 🧠 卡片可见)。
|
||||
|
||||
**响应**:
|
||||
```json
|
||||
@@ -1275,6 +1275,19 @@ AI 可用设备列表(在线状态 + 是否有任务运行,前端据此把 b
|
||||
|
||||
**响应**:`{"ok": true, "msg": "巡检已启动,完成后刷新列表查看建议"}`
|
||||
|
||||
### GET /api/agent/actions
|
||||
动作经验库列表(命名动作 + 编辑器 schema 步骤 + 元素定位、禁坐标)。仅管理员。
|
||||
响应:`{"ok":true,"actions":[{id,name,app,aliases,params,steps,preconditions,hits,updated_at}]}`。
|
||||
由任务成功后的**成功步骤**自动蒸馏沉淀;执行前按动作名/别名召回并注入 system prompt。
|
||||
|
||||
### POST /api/agent/actions/delete
|
||||
删除动作:`{id}`。仅管理员。
|
||||
|
||||
### POST /api/agent/actions/save
|
||||
新增/编辑动作:`{id?, name, app?, aliases?, params?, steps, preconditions?}`。仅管理员。
|
||||
`steps` 可为数组或 JSON 字符串,经服务端校验(白名单 type + 必填;**拒绝坐标 click_xy**)。
|
||||
校验失败返回 400 并附原因。
|
||||
|
||||
### POST /api/agent/experience/keep
|
||||
|
||||
人工保留经验(撤销"建议删除",后续巡检不再重复建议)。**请求**(JSON):`{"id": 1}`
|
||||
|
||||
+10
-3
@@ -185,7 +185,8 @@ SQLAlchemy 模型,存于 `data/users.db`:
|
||||
|
||||
> 表名默认取类名小写(models.py 未写 `__tablename__`)。另有三张非模型表,由原生 SQL 幂等创建、**不走 SCHEMA_MIGRATIONS**:
|
||||
> - `app_meta`(KV):`_migrate_schema()` 内建表,存 `schema_version`、`discovery_*`、agent 配置 `agent_*` 等;
|
||||
> - `agent_conversation` / `agent_experience` / `experience_audit`:AI 控制台会话 / 经验库 / 经验巡检(`web/agent_api.py` 顶部 `CREATE TABLE IF NOT EXISTS`)。
|
||||
> - `agent_conversation` / `agent_experience` / `experience_audit`:AI 控制台会话 / 任务级经验(配方)/ 经验巡检(`web/agent_api.py` 顶部 `CREATE TABLE IF NOT EXISTS`)。
|
||||
> - `agent_action`:**动作经验库**(命名动作 = 可复用单元,steps 用编辑器 schema 且带元素定位、禁坐标);由任务成功后从**成功步骤**蒸馏,执行前按名字/别名召回并注入(`web/agent_api.py` `_distill_actions/_find_actions`)。
|
||||
|
||||
**数据库初始化**(`init_db`):
|
||||
- 创建所有表
|
||||
@@ -251,7 +252,7 @@ APK 上传/解析/批量安装。
|
||||
| `devices_api.py` | 设备池管理 + 自动发现(扫描/确认/忽略/手动重连/型号采集) |
|
||||
| `apks_api.py` | 应用管理 |
|
||||
| `tailscale_api.py` | Tailscale 管理 |
|
||||
| `agent_api.py` | AI 控制台:会话 / 运行 / SSE / 停止 / 配置 / 经验库与巡检(读写 `agent_conversation`、`agent_experience`、app_meta `agent_*`) |
|
||||
| `agent_api.py` | AI 控制台:会话 / 运行 / SSE / 停止 / 配置 / 经验库与巡检 / **动作库**(读写 `agent_conversation`、`agent_experience`、`agent_action`、app_meta `agent_*`) |
|
||||
| `system_api.py` | 系统数据备份导出 / 导入恢复(`/api/system/backup/*`,仅 admin) |
|
||||
| `common.py` | 跨模块共享(合并设备列表/屏幕状态) |
|
||||
| `context.py` | 共享对象注入(mgr/apk_mgr/device_pool) |
|
||||
@@ -333,7 +334,13 @@ tasks/douyin/actions/like.py — @register_action(ACTIONS) LikeAction
|
||||
|
||||
(原"分组"顶级 Tab 与"维护/STF 服务"子分栏已不存在——分组已移入工具页子分栏,STF 已摘除。)
|
||||
|
||||
### 5.4 元素抓取模态框
|
||||
### 5.4 AI 控制台的记忆面板
|
||||
|
||||
AI 控制台(顶级 Tab)右上角两个模态框,管理自进化记忆:
|
||||
- **🧠 经验库**:任务级经验(`agent_experience`,整任务配方)+ 每日 AI 巡检建议(删除需人工确认)。
|
||||
- **🎬 动作库**:动作级经验(`agent_action`)——命名动作(可含 1~N 步)+ 编辑器 schema 步骤 + **元素定位(禁坐标)**;由任务成功后从**成功步骤**自动蒸馏,执行前按名/别名召回注入;面板支持查看/编辑/删除/手动新建(保存经服务端校验,坐标步骤被拒)。
|
||||
|
||||
### 5.5 元素抓取模态框
|
||||
|
||||
独立的第二层模态框(`el-picker-overlay`,z-index 1100),不影响任务编辑窗口:
|
||||
1. 选择设备 → 2. 加载截图 + 元素树 → 3. 点击元素/边界框 → 4. 回填选择器
|
||||
|
||||
+1
-1
@@ -214,7 +214,7 @@ tasks/<app>/
|
||||
- 模型改 `core/models.py`,首次建表用 `create_all()`;SQLAlchemy 模型未写 `__tablename__` 时默认表名 = 小写类名
|
||||
- **已有数据的老库**:在 `core/models.py` 的 `SCHEMA_MIGRATIONS` 里加迁移(版本号递增 + SQL)
|
||||
- `app_meta`(KV 配置表)由 `core/models.py` 的 `_migrate_schema()` 建表并维护 `schema_version`
|
||||
- `agent_conversation` / `agent_experience` / `experience_audit` 由 `web/agent_api.py` 顶部的原生 `CREATE TABLE IF NOT EXISTS` 幂等创建(模块内首次用时执行),**不经 `SCHEMA_MIGRATIONS`,无版本管理**
|
||||
- `agent_conversation` / `agent_experience` / `experience_audit` / `agent_action`(动作经验库)由 `web/agent_api.py` 内的原生 `CREATE TABLE IF NOT EXISTS` 幂等创建(模块内首次用时执行),**不经 `SCHEMA_MIGRATIONS`,无版本管理**
|
||||
|
||||
### 5.6 改动必须同步文档
|
||||
|
||||
|
||||
@@ -512,6 +512,105 @@ function endRun(){
|
||||
scrollChat();
|
||||
}
|
||||
|
||||
// ================== 动作库(可复用动作:查看/编辑/删除/新建) ==================
|
||||
let _actCache = [];
|
||||
function openActLib(){
|
||||
document.getElementById('agent-act-overlay').style.display = 'flex';
|
||||
loadActLib();
|
||||
}
|
||||
function closeActLib(){
|
||||
document.getElementById('agent-act-overlay').style.display = 'none';
|
||||
}
|
||||
function _actStepBrief(a){
|
||||
return (a.steps||[]).map(s=>{
|
||||
const p = s.params || {};
|
||||
const loc = p.selector_value || p.package || p.fixed_text || '';
|
||||
return (s.type || '?') + (loc ? (':' + loc) : '');
|
||||
}).join(' → ');
|
||||
}
|
||||
function loadActLib(){
|
||||
const list = document.getElementById('act-lib-list');
|
||||
apiGet('/api/agent/actions').then(r=>{
|
||||
if(!r||!r.ok){ showToast((r&&r.error)||'加载失败','error'); return; }
|
||||
_actCache = r.actions || [];
|
||||
document.getElementById('act-lib-hint').textContent = '共 ' + _actCache.length + ' 个动作';
|
||||
if(!_actCache.length){
|
||||
list.innerHTML = '<div class="text-muted" style="padding:20px;text-align:center">暂无动作——跑一轮以元素/文字点击为主的任务后会自动沉淀</div>';
|
||||
return;
|
||||
}
|
||||
list.innerHTML = '';
|
||||
_actCache.forEach(a=>{
|
||||
const card = document.createElement('div');
|
||||
card.style.cssText = 'border:1px solid var(--card-line);border-radius:10px;padding:9px 13px;background:#0d1117';
|
||||
card.innerHTML = '<div style="display:flex;justify-content:space-between;gap:10px;align-items:flex-start">'
|
||||
+ '<div style="flex:1;min-width:0">'
|
||||
+ '<div style="font-weight:600">' + esc(a.name||'')
|
||||
+ (a.app ? '<span class="text-muted" style="font-size:11px;font-weight:400;margin-left:6px">' + esc(a.app) + '</span>' : '')
|
||||
+ '</div>'
|
||||
+ '<div style="font-size:11.5px;color:#9fb3d1;margin-top:2px;word-break:break-all">' + esc(_actStepBrief(a) || '—') + '</div>'
|
||||
+ '<div class="text-muted" style="font-size:11px;margin-top:2px">别名: ' + esc((a.aliases||[]).join('、')||'—')
|
||||
+ ' | 命中 ' + (a.hits||0) + ' 次 | ' + esc(a.updated_at||'') + '</div>'
|
||||
+ '</div>'
|
||||
+ '<div style="display:flex;flex-direction:column;gap:4px;flex:none">'
|
||||
+ '<button class="btn btn-xs" onclick="actEdit(' + a.id + ')">编辑</button>'
|
||||
+ '<button class="btn btn-xs btn-danger" onclick="actDelete(' + a.id + ')">删除</button>'
|
||||
+ '</div></div>';
|
||||
list.appendChild(card);
|
||||
});
|
||||
});
|
||||
}
|
||||
function actDelete(id){
|
||||
if(!confirm('确认删除该动作?删除后同类任务不再自动复用它。')) return;
|
||||
apiPost('/api/agent/actions/delete',{id:id}).then(r=>{
|
||||
showToast((r&&(r.msg||r.error))||'删除失败', (r&&r.ok)?'success':'error');
|
||||
loadActLib();
|
||||
});
|
||||
}
|
||||
function actNew(){ _actForm({name:'', app:'', aliases:[], params:[], steps:[]}); }
|
||||
function actEdit(id){
|
||||
const a = _actCache.find(x=>x.id===id);
|
||||
if(a) _actForm(a);
|
||||
}
|
||||
function _actForm(a){
|
||||
const list = document.getElementById('act-lib-list');
|
||||
const isNew = !a.id;
|
||||
list.innerHTML = '';
|
||||
const box = document.createElement('div');
|
||||
box.style.cssText = 'border:1px solid var(--card-line);border-radius:10px;padding:12px 14px;background:#0d1117';
|
||||
box.innerHTML =
|
||||
'<div style="font-weight:600;margin-bottom:8px">' + (isNew?'新建动作':'编辑动作 #'+a.id) + '</div>'
|
||||
+ '<div style="display:flex;gap:8px;flex-wrap:wrap;margin-bottom:6px">'
|
||||
+ '<input id="act-f-name" class="form-control" style="flex:1;min-width:180px" placeholder="动作名(如 打开抖音)" value="' + esc(a.name||'') + '">'
|
||||
+ '<input id="act-f-app" class="form-control" style="flex:1;min-width:180px" placeholder="包名(可空,如 com.ss.android.ugc.aweme)" value="' + esc(a.app||'') + '">'
|
||||
+ '</div>'
|
||||
+ '<div style="display:flex;gap:8px;flex-wrap:wrap;margin-bottom:6px">'
|
||||
+ '<input id="act-f-alias" class="form-control" style="flex:1;min-width:180px" placeholder="别名(逗号分隔)" value="' + esc((a.aliases||[]).join(',')) + '">'
|
||||
+ '<input id="act-f-params" class="form-control" style="flex:1;min-width:180px" placeholder="参数名(逗号分隔,可空)" value="' + esc((a.params||[]).join(',')) + '">'
|
||||
+ '</div>'
|
||||
+ '<textarea id="act-f-steps" class="form-control" rows="9" style="font-family:monospace;font-size:12px" '
|
||||
+ 'placeholder=\'步骤 JSON 数组,例:[{"type":"click","params":{"selector_type":"text","selector_value":"搜索"}}]\'>'
|
||||
+ esc(JSON.stringify(a.steps||[], null, 2)) + '</textarea>'
|
||||
+ '<div class="text-muted" style="font-size:11px;margin:6px 0">可用 type:open_app/click/input_text/swipe/wait/key_event/group…;click 必须有 selector_value(xpath/text/resourceId/description…);<b>不接受坐标(click_xy)</b>。</div>'
|
||||
+ '<div style="display:flex;gap:8px"><button class="btn btn-primary btn-sm" onclick="actSave(' + (a.id||0) + ')">保存</button>'
|
||||
+ '<button class="btn btn-sm" onclick="loadActLib()">取消</button></div>';
|
||||
list.appendChild(box);
|
||||
}
|
||||
function actSave(id){
|
||||
const g = (i)=>document.getElementById(i).value;
|
||||
const split = (s)=>String(s||'').split(/[,,]/).map(x=>x.trim()).filter(x=>x);
|
||||
let steps;
|
||||
try{ steps = JSON.parse(g('act-f-steps')||'[]'); }
|
||||
catch(e){ showToast('steps 不是合法 JSON: '+e.message,'error'); return; }
|
||||
apiPost('/api/agent/actions/save', {
|
||||
id: id||0, name: g('act-f-name'), app: g('act-f-app'),
|
||||
aliases: split(g('act-f-alias')), params: split(g('act-f-params')), steps: steps
|
||||
}).then(r=>{
|
||||
if(!r||!r.ok){ showToast((r&&r.error)||'保存失败','error'); return; }
|
||||
showToast(r.msg||'已保存','success');
|
||||
loadActLib();
|
||||
});
|
||||
}
|
||||
|
||||
// ================== 经验库(自进化记忆管理) ==================
|
||||
function openExpLib(){
|
||||
document.getElementById('agent-exp-overlay').style.display = 'flex';
|
||||
|
||||
@@ -480,6 +480,7 @@ body{background:var(--bg);font-family:var(--body);color:var(--text);font-size:14
|
||||
<div class="agent-top-actions">
|
||||
<span id="agent-running-tag" class="agent-running-tag" style="display:none">● 运行中</span>
|
||||
<button class="btn btn-xs" onclick="openExpLib()" title="查看/管理自进化经验库(每日 AI 巡检建议,删除需确认)">🧠 经验库</button>
|
||||
<button class="btn btn-xs" onclick="openActLib()" title="查看/管理可复用动作(命名动作 + 元素定位步骤,AI 自动沉淀,可编辑删除)">🎬 动作库</button>
|
||||
<button class="btn btn-xs" onclick="openAgentConfig()">⚙ 配置</button>
|
||||
</div>
|
||||
</div>
|
||||
@@ -563,6 +564,24 @@ body{background:var(--bg);font-family:var(--body);color:var(--text);font-size:14
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ========== 动作库模态框(可复用动作:命名动作 + 元素定位步骤) ========== -->
|
||||
<div id="agent-act-overlay" style="display:none;position:fixed;inset:0;background:rgba(0,0,0,.6);z-index:1320;align-items:center;justify-content:center">
|
||||
<div style="background:#11151c;border:1px solid var(--card-line);border-radius:14px;padding:20px 22px;width:860px;max-width:94vw;max-height:86vh;display:flex;flex-direction:column">
|
||||
<div style="display:flex;align-items:center;justify-content:space-between;margin-bottom:4px">
|
||||
<div style="font-size:15px;font-weight:700">🎬 动作库<span class="text-muted" style="font-size:11.5px;font-weight:400;margin-left:8px" id="act-lib-hint"></span></div>
|
||||
<div style="display:flex;gap:6px;align-items:center">
|
||||
<button class="btn btn-xs" onclick="actNew()">+ 新建动作</button>
|
||||
<button class="btn btn-xs" onclick="loadActLib()">刷新</button>
|
||||
<button class="btn btn-xs" onclick="closeActLib()">✕ 关闭</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="text-muted" style="font-size:11.5px;padding:0 2px 8px;border-bottom:1px solid var(--card-line);margin-bottom:10px">
|
||||
任务成功后自动把<b>成功步骤</b>沉淀为可复用动作(命名动作 + 编辑器 schema 步骤 + 元素定位,<b>不含坐标</b>);执行前按动作名/别名召回注入,AI 可直接复用、跳过重新探索。可在此查看、编辑、删除或手动新建。
|
||||
</div>
|
||||
<div id="act-lib-list" style="flex:1;overflow-y:auto;min-height:200px;display:flex;flex-direction:column;gap:8px"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ========== Tab 4: 日志 ========== -->
|
||||
<div id="tab-logs" class="tab-panel">
|
||||
<div class="page-title">日志查看</div>
|
||||
|
||||
+476
-3
@@ -289,6 +289,360 @@ def _save_experience(prompt, recipe, tool_seq):
|
||||
return False
|
||||
|
||||
|
||||
# ---------- 动作经验库(agent_action)----------
|
||||
# 与「任务级配方」(agent_experience) 互补:动作 = 有语义名的可复用单元,可含 1~N 步,
|
||||
# steps 直接用编辑器 schema(open_app/click/input_text…),带**元素定位**
|
||||
# (selector_type/selector_value),**不含坐标**(分辨率/旋转/改版即失效)。
|
||||
# 复用:执行前按 name/别名/App 召回并注入 system prompt,模型可跳过重新探索。
|
||||
_ACTION_TABLE = (
|
||||
"CREATE TABLE IF NOT EXISTS agent_action ("
|
||||
"id INTEGER PRIMARY KEY AUTOINCREMENT,"
|
||||
"name TEXT NOT NULL,"
|
||||
"app TEXT DEFAULT '',"
|
||||
"aliases TEXT DEFAULT '[]',"
|
||||
"params TEXT DEFAULT '[]',"
|
||||
"steps TEXT NOT NULL,"
|
||||
"preconditions TEXT DEFAULT '',"
|
||||
"hits INTEGER DEFAULT 0,"
|
||||
"source_prompt TEXT DEFAULT '',"
|
||||
"created_at TEXT DEFAULT '',"
|
||||
"updated_at TEXT DEFAULT '')")
|
||||
|
||||
# 允许沉淀的步骤类型(编辑器 STEP_TYPES 子集;显式排除 click_xy 等坐标类)
|
||||
_ACTION_STEP_TYPES = {
|
||||
"open_app", "stop_app", "screen_on", "screen_off", "key_event",
|
||||
"swipe", "swipe_until", "click", "long_click", "wait_el",
|
||||
"input_text", "clipboard", "wait", "loop", "group", "if_el"}
|
||||
_ACTION_REQUIRED = { # 类型 → 必需的 params 键(缺则丢弃该动作)
|
||||
"open_app": ("package",), "stop_app": ("package",),
|
||||
"click": ("selector_value",), "long_click": ("selector_value",),
|
||||
"wait_el": ("selector_value",), "swipe_until": ("selector_value",),
|
||||
"if_el": ("selector_value", "then"),
|
||||
"group": ("children",), "loop": ("children",)}
|
||||
# 可写入动作的关键工具(探索类 de_screenshot/de_ui_tree/de_ocr 不沉淀)
|
||||
_ACTION_TOOLS = {
|
||||
"de_open_app", "de_stop_app", "de_tap_text", "de_tap_element", "de_type_text",
|
||||
"de_set_clipboard", "de_swipe", "de_press_key", "de_wake", "de_sleep"}
|
||||
|
||||
|
||||
def _ensure_action_table():
|
||||
try:
|
||||
db.session.execute(db.text(_ACTION_TABLE))
|
||||
db.session.commit()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _brief_result(result):
|
||||
"""工具结果的精简摘要(判成败 + 供提炼模型参考)。"""
|
||||
try:
|
||||
s = json.dumps(result, ensure_ascii=False) if not isinstance(result, str) else result
|
||||
except Exception:
|
||||
s = str(result)
|
||||
return (s or "")[:160]
|
||||
|
||||
|
||||
def _tool_ok(tool, result):
|
||||
"""启发式判断一次工具调用是否成功(只沉淀成功动作,避免把误点当经验)。"""
|
||||
if result is None:
|
||||
return False
|
||||
if isinstance(result, dict):
|
||||
if result.get("error") or result.get("ok") is False or result.get("success") is False:
|
||||
return False
|
||||
if tool == "de_tap_text":
|
||||
return bool(result.get("matched")) or result.get("method") in ("ui", "ocr")
|
||||
if result.get("matched") is False:
|
||||
return False
|
||||
s = str(result).lower()
|
||||
return not any(k in s for k in ("not_found", "error", "failed", "occupied"))
|
||||
|
||||
|
||||
def _normalize_action_item(a):
|
||||
"""把模型可能的「单动作」形状({action/tool/type, params})归一成标准动作。
|
||||
|
||||
实测蒸馏模型常不按提示词的 name/steps 输出,而是照抄输入行给出
|
||||
{"action":"de_open_app","params":{...}}——这里做兼容映射,避免全被丢弃。
|
||||
"""
|
||||
if not isinstance(a, dict):
|
||||
return None
|
||||
if isinstance(a.get("steps"), list): # 已是标准形状
|
||||
return a
|
||||
tool = str(a.get("action") or a.get("tool") or a.get("type") or "").strip()
|
||||
p = a.get("params") if isinstance(a.get("params"), dict) else {}
|
||||
if not tool:
|
||||
return None
|
||||
name = str(a.get("name") or "").strip()
|
||||
t, sp = None, {}
|
||||
if tool in ("de_open_app", "open_app"):
|
||||
t, sp = "open_app", {"package": p.get("package", "")}
|
||||
name = name or f"打开应用 {p.get('package', '')}"
|
||||
elif tool in ("de_stop_app", "stop_app"):
|
||||
t, sp = "stop_app", {"package": p.get("package", "")}
|
||||
name = name or f"关闭应用 {p.get('package', '')}"
|
||||
elif tool == "de_tap_text":
|
||||
t, sp = "click", {"selector_type": "text", "selector_value": p.get("text", "")}
|
||||
name = name or f"点击「{p.get('text', '')}」"
|
||||
elif tool == "de_tap_element":
|
||||
by = {"text": "text", "id": "resourceId", "desc": "description",
|
||||
"text_contains": "text", "desc_contains": "descriptionContains"}.get(
|
||||
p.get("by"), "text")
|
||||
t, sp = "click", {"selector_type": by, "selector_value": p.get("value", "")}
|
||||
name = name or f"点击元素 {p.get('value', '')}"
|
||||
elif tool == "de_type_text":
|
||||
t, sp = "input_text", {"mode": "fixed", "fixed_text": p.get("text", "")}
|
||||
name = name or f"输入「{p.get('text', '')}」"
|
||||
elif tool == "de_set_clipboard":
|
||||
t, sp, name = "clipboard", {"text": p.get("text", "")}, name or "写入剪贴板"
|
||||
elif tool == "de_swipe":
|
||||
t, sp = "swipe", {"direction": p.get("direction") or "up"}
|
||||
name = name or "滑动"
|
||||
elif tool == "de_press_key":
|
||||
t, sp = "key_event", {"key": p.get("key") or "back"}
|
||||
name = name or f"按键 {p.get('key', '')}"
|
||||
elif tool == "de_wake":
|
||||
t, name = "screen_on", name or "亮屏解锁"
|
||||
elif tool == "de_sleep":
|
||||
t, name = "screen_off", name or "息屏"
|
||||
if not t:
|
||||
return None
|
||||
return {"name": name, "app": a.get("app", ""), "aliases": a.get("aliases") or [],
|
||||
"params": [], "preconditions": a.get("preconditions", ""),
|
||||
"steps": [{"type": t, "params": sp}]}
|
||||
|
||||
|
||||
def _sanitize_actions(actions):
|
||||
"""校验/清洗模型产出的动作:名字非空、步骤白名单+必填、显式禁坐标。"""
|
||||
out = []
|
||||
for a in (actions or []):
|
||||
a = _normalize_action_item(a)
|
||||
if not isinstance(a, dict):
|
||||
continue
|
||||
name = str(a.get("name") or "").strip()[:40]
|
||||
steps = a.get("steps")
|
||||
if not name or not isinstance(steps, list) or not steps:
|
||||
continue
|
||||
good = []
|
||||
for st in steps:
|
||||
if not isinstance(st, dict):
|
||||
continue
|
||||
t = st.get("type")
|
||||
if t not in _ACTION_STEP_TYPES: # click_xy 等坐标类在此被剔除
|
||||
continue
|
||||
p = st.get("params") if isinstance(st.get("params"), dict) else {}
|
||||
if any(not p.get(k) for k in _ACTION_REQUIRED.get(t, ())):
|
||||
continue
|
||||
good.append({"type": t, "label": str(st.get("label") or "")[:20], "params": p})
|
||||
if not good:
|
||||
continue
|
||||
out.append({
|
||||
"name": name,
|
||||
"app": str(a.get("app") or "")[:80],
|
||||
"aliases": [str(x)[:20] for x in (a.get("aliases") or []) if str(x).strip()][:5],
|
||||
"params": [str(x)[:20] for x in (a.get("params") or []) if str(x).strip()][:5],
|
||||
"preconditions": str(a.get("preconditions") or "")[:100],
|
||||
"steps": good})
|
||||
return out
|
||||
|
||||
|
||||
def _iter_json_objects(raw):
|
||||
"""从文本里按大括号配对切出顶层 JSON 对象(容忍坏片段,逐条抢救)。"""
|
||||
depth = 0
|
||||
start = None
|
||||
in_str = esc = False
|
||||
for i, c in enumerate(raw):
|
||||
if in_str:
|
||||
if esc:
|
||||
esc = False
|
||||
elif c == "\\":
|
||||
esc = True
|
||||
elif c == '"':
|
||||
in_str = False
|
||||
continue
|
||||
if c == '"':
|
||||
in_str = True
|
||||
elif c == "{":
|
||||
if depth == 0:
|
||||
start = i
|
||||
depth += 1
|
||||
elif c == "}":
|
||||
depth -= 1
|
||||
if depth == 0 and start is not None:
|
||||
yield raw[start:i + 1]
|
||||
start = None
|
||||
|
||||
|
||||
def _loads_lenient(text):
|
||||
"""尽量解析模型输出的 JSON 数组:容忍 markdown 围栏、尾逗号、中文引号、坏对象。"""
|
||||
s = (text or "").strip()
|
||||
s = re.sub(r"^```[a-zA-Z]*\s*|\s*```$", "", s).strip()
|
||||
m = re.search(r"\[[\s\S]*\]", s)
|
||||
raw = m.group(0) if m else s
|
||||
candidates = [raw,
|
||||
re.sub(r",\s*([\]}])", r"\1", raw), # 去尾逗号
|
||||
re.sub(r"[“”]", '"', raw), # 中文引号 → 英文
|
||||
re.sub(r"[“”]", '"', re.sub(r",\s*([\]}])", r"\1", raw))]
|
||||
for cand in candidates:
|
||||
try:
|
||||
v = json.loads(cand)
|
||||
if isinstance(v, list):
|
||||
return v
|
||||
except Exception:
|
||||
continue
|
||||
# 逐对象抢救(顶层大括号配对),坏的跳过
|
||||
out = []
|
||||
for obj in _iter_json_objects(raw):
|
||||
try:
|
||||
out.append(json.loads(obj))
|
||||
except Exception:
|
||||
continue
|
||||
return out
|
||||
|
||||
|
||||
def _distill_actions(cfg, prompt, trace):
|
||||
"""从**成功**的工具轨迹提炼命名动作(JSON 数组)。失败返回 []。"""
|
||||
ok_ops = [t for t in (trace or [])
|
||||
if t.get("tool") in _ACTION_TOOLS and _tool_ok(t.get("tool"), t.get("result"))]
|
||||
_log.info(f"动作提炼: 轨迹 {len(trace or [])} 步, 成功可沉淀 {len(ok_ops)} 步")
|
||||
if not ok_ops:
|
||||
return []
|
||||
lines = [f"- {t['tool']} 参数={t['args']} 结果={_brief_result(t['result'])}" for t in ok_ops]
|
||||
instruction = (
|
||||
"以下是一次成功的手机自动化操作的**成功步骤**。请把它们提炼为若干「动作」"
|
||||
"(每个动作 = 一个有语义名的可复用单元,可含 1~N 步)。只输出 JSON 数组,不要解释。\n"
|
||||
"字段:name(动作名,如「打开抖音」「搜索关键词」);app(包名,未知则空串);"
|
||||
"aliases(别名数组);params(参数名数组,如[\"关键词\"]);steps(步骤数组)。\n"
|
||||
"steps 每步:type + params,type 取值:open_app{package} / click{selector_type,"
|
||||
"selector_value,wait_timeout?} / input_text{mode,fixed_text} / swipe{direction} /"
|
||||
"wait{min,max} / key_event{key} / group{children}。\n"
|
||||
"**定位必须用元素定位**:selector_type 取 xpath/text/resourceId/description/"
|
||||
"descriptionContains,值用上面步骤里出现的真实文字或 id;**禁止坐标**。"
|
||||
"若某步只能用坐标定位,就不要产出该动作。\n"
|
||||
"输出示例(**顶层字段必须是 name/steps,禁止用 action/tool 当顶层字段**):\n"
|
||||
'[{"name":"打开抖音","app":"com.ss.android.ugc.aweme","aliases":["启动抖音"],'
|
||||
'"params":[],"steps":[{"type":"open_app","params":{"package":"com.ss.android.ugc.aweme"}}]},'
|
||||
'{"name":"搜索关键词","app":"","aliases":["点搜索"],"params":["关键词"],'
|
||||
'"steps":[{"type":"click","params":{"selector_type":"text","selector_value":"搜索"}}]}]\n'
|
||||
f"任务:{prompt[:200]}\n成功步骤:\n" + "\n".join(lines)[:1500])
|
||||
try:
|
||||
import httpx
|
||||
body = {"model": cfg.get("model") or "deepseek-v4-flash-vision-exp",
|
||||
"messages": [{"role": "user", "content": instruction}],
|
||||
"max_tokens": 900}
|
||||
headers = {"Authorization": f"Bearer {cfg.get('api_key', '')}",
|
||||
"Content-Type": "application/json"}
|
||||
r = httpx.post(
|
||||
f"{(cfg.get('api_base') or 'https://api.deepseek.com').rstrip('/')}/chat/completions",
|
||||
json=body, headers=headers, timeout=30)
|
||||
if r.status_code != 200:
|
||||
_log.warning(f"动作提炼: 模型返回 HTTP {r.status_code}")
|
||||
return []
|
||||
content = (((r.json().get("choices") or [{}])[0].get("message") or {}).get("content") or "")
|
||||
acts = _sanitize_actions(_loads_lenient(content))
|
||||
if not acts:
|
||||
_log.info(f"动作提炼: 解析后无有效动作(原始输出 {len(content)} 字符)")
|
||||
return acts
|
||||
except Exception as e:
|
||||
_log.warning(f"动作提炼失败: {e}")
|
||||
return []
|
||||
|
||||
|
||||
def _save_actions(prompt, actions):
|
||||
"""按 (name, app) upsert 保存动作。返回保存条数。"""
|
||||
if not actions or _flask_app is None:
|
||||
return 0
|
||||
n = 0
|
||||
try:
|
||||
from datetime import datetime
|
||||
now = datetime.now().strftime("%Y-%m-%d %H:%M")
|
||||
with _flask_app.app_context():
|
||||
_ensure_action_table()
|
||||
for a in actions:
|
||||
row = db.session.execute(db.text(
|
||||
"SELECT id FROM agent_action WHERE name=:n AND app=:a"),
|
||||
{"n": a["name"], "a": a["app"]}).fetchone()
|
||||
if row:
|
||||
db.session.execute(db.text(
|
||||
"UPDATE agent_action SET aliases=:al, params=:p, steps=:s, "
|
||||
"preconditions=:pc, updated_at=:t WHERE id=:i"),
|
||||
{"al": json.dumps(a["aliases"], ensure_ascii=False),
|
||||
"p": json.dumps(a["params"], ensure_ascii=False),
|
||||
"s": json.dumps(a["steps"], ensure_ascii=False),
|
||||
"pc": a["preconditions"], "t": now, "i": row[0]})
|
||||
else:
|
||||
db.session.execute(db.text(
|
||||
"INSERT INTO agent_action(name, app, aliases, params, steps, "
|
||||
"preconditions, hits, source_prompt, created_at, updated_at) "
|
||||
"VALUES(:n,:a,:al,:p,:s,:pc,0,:sp,:t,:t)"),
|
||||
{"n": a["name"], "a": a["app"],
|
||||
"al": json.dumps(a["aliases"], ensure_ascii=False),
|
||||
"p": json.dumps(a["params"], ensure_ascii=False),
|
||||
"s": json.dumps(a["steps"], ensure_ascii=False),
|
||||
"pc": a["preconditions"], "sp": prompt[:200], "t": now})
|
||||
n += 1
|
||||
db.session.commit()
|
||||
_log.info(f"动作经验已保存 {n} 条")
|
||||
except Exception as e:
|
||||
_log.warning(f"动作保存失败: {e}")
|
||||
return n
|
||||
|
||||
|
||||
def _find_actions(prompt, limit=3):
|
||||
"""召回可复用动作:动作名/别名命中 prompt,或与来源提示够相似。返回 (文本, 名字列表)。"""
|
||||
try:
|
||||
if _flask_app is None:
|
||||
return "", []
|
||||
with _flask_app.app_context():
|
||||
_ensure_action_table()
|
||||
rows = db.session.execute(db.text(
|
||||
"SELECT id, name, app, aliases, params, steps, hits FROM agent_action "
|
||||
"ORDER BY hits DESC, id DESC LIMIT 100")).fetchall()
|
||||
except Exception:
|
||||
return "", []
|
||||
if not rows:
|
||||
return "", []
|
||||
p_norm = "".join(c for c in (prompt or "").lower()
|
||||
if c.isalnum() or "一" <= c <= "鿿")
|
||||
cur = _bigrams(prompt)
|
||||
scored = []
|
||||
for rid, name, app, aliases, params, steps, hits in rows:
|
||||
try:
|
||||
keys = [name] + [str(x) for x in (json.loads(aliases) if aliases else [])]
|
||||
except Exception:
|
||||
keys = [name]
|
||||
hit = any(k and k.lower() in p_norm for k in keys if k)
|
||||
sim = (len(cur & _bigrams(name)) / len(cur)) if cur else 0.0
|
||||
if hit or sim >= 0.34:
|
||||
scored.append((1 if hit else 0, sim, hits or 0, rid, name, app, params, steps))
|
||||
if not scored:
|
||||
return "", []
|
||||
scored.sort(key=lambda x: (-x[0], -x[1], -x[2]))
|
||||
picked = scored[:limit]
|
||||
try:
|
||||
with _flask_app.app_context():
|
||||
for item in picked:
|
||||
db.session.execute(db.text("UPDATE agent_action SET hits=hits+1 WHERE id=:i"),
|
||||
{"i": item[3]})
|
||||
db.session.commit()
|
||||
except Exception:
|
||||
pass
|
||||
items, parts = [], []
|
||||
for item in picked:
|
||||
_hit, _sim, _hts, _rid, name, app, params, steps = item
|
||||
try:
|
||||
st = json.loads(steps) if steps else []
|
||||
except Exception:
|
||||
st = []
|
||||
items.append(name)
|
||||
brief = " → ".join(
|
||||
str(s.get("params", {}).get("selector_value")
|
||||
or s.get("params", {}).get("package")
|
||||
or s.get("params", {}).get("fixed_text")
|
||||
or s.get("type")) for s in st[:8])
|
||||
parts.append(f"- 「{name}」" + (f"(app={app})" if app else "")
|
||||
+ (f" 参数:{params}" if params else "") + f":{brief}")
|
||||
return "\n".join(parts), items
|
||||
|
||||
|
||||
# ================== 经验巡检(AI 质检,删除需人工确认) ==================
|
||||
_audit_state = {"running": False, "last": "", "last_summary": ""}
|
||||
|
||||
@@ -475,6 +829,94 @@ def agent_experience_keep():
|
||||
db.session.commit()
|
||||
return jsonify({"ok": True, "msg": f"经验 #{exp_id} 已保留"})
|
||||
|
||||
# ================== 动作经验库(agent_action:查看/删除/编辑) ==================
|
||||
def _action_row_to_dict(r):
|
||||
def _j(s, d):
|
||||
try:
|
||||
return json.loads(s) if s else d
|
||||
except Exception:
|
||||
return d
|
||||
return {"id": r[0], "name": r[1], "app": r[2], "aliases": _j(r[3], []),
|
||||
"params": _j(r[4], []), "steps": _j(r[5], []), "preconditions": r[6],
|
||||
"hits": r[7] or 0, "updated_at": r[8] or ""}
|
||||
|
||||
|
||||
@bp.route("/api/agent/actions")
|
||||
@admin_required
|
||||
def agent_actions_list():
|
||||
"""动作经验库列表(命名动作 + 编辑器 schema 步骤 + 元素定位,禁坐标)。"""
|
||||
with _flask_app.app_context():
|
||||
_ensure_action_table()
|
||||
rows = db.session.execute(db.text(
|
||||
"SELECT id, name, app, aliases, params, steps, preconditions, hits, updated_at "
|
||||
"FROM agent_action ORDER BY hits DESC, id DESC")).fetchall()
|
||||
return jsonify({"ok": True, "actions": [_action_row_to_dict(r) for r in rows]})
|
||||
|
||||
|
||||
@bp.route("/api/agent/actions/delete", methods=["POST"])
|
||||
@admin_required
|
||||
def agent_actions_delete():
|
||||
aid = int((request.json or {}).get("id") or 0)
|
||||
if aid <= 0:
|
||||
return jsonify({"ok": False, "error": "缺少 id"}), 400
|
||||
with _flask_app.app_context():
|
||||
_ensure_action_table()
|
||||
db.session.execute(db.text("DELETE FROM agent_action WHERE id=:i"), {"i": aid})
|
||||
db.session.commit()
|
||||
return jsonify({"ok": True, "msg": f"动作 #{aid} 已删除"})
|
||||
|
||||
|
||||
@bp.route("/api/agent/actions/save", methods=["POST"])
|
||||
@admin_required
|
||||
def agent_actions_save():
|
||||
"""新增/编辑动作:{id?, name, app?, aliases?, params?, steps, preconditions?}。
|
||||
|
||||
steps 可为数组或 JSON 字符串;经 _sanitize_actions 校验(白名单/必填/禁坐标)。
|
||||
"""
|
||||
from datetime import datetime
|
||||
d = request.json or {}
|
||||
steps = d.get("steps")
|
||||
if isinstance(steps, str):
|
||||
try:
|
||||
steps = json.loads(steps)
|
||||
except Exception:
|
||||
return jsonify({"ok": False, "error": "steps 不是合法 JSON"}), 400
|
||||
item = _sanitize_actions([{"name": d.get("name"), "app": d.get("app", ""),
|
||||
"aliases": d.get("aliases") or [],
|
||||
"params": d.get("params") or [],
|
||||
"preconditions": d.get("preconditions", ""),
|
||||
"steps": steps if isinstance(steps, list) else []}])
|
||||
if not item:
|
||||
return jsonify({"ok": False, "error": "动作无效:需要 name,且 steps 需为受支持的步骤"
|
||||
"(click/open_app/input_text… 且带元素定位;不接受坐标)"}), 400
|
||||
a = item[0]
|
||||
now = datetime.now().strftime("%Y-%m-%d %H:%M")
|
||||
with _flask_app.app_context():
|
||||
_ensure_action_table()
|
||||
aid = int(d.get("id") or 0)
|
||||
if aid > 0:
|
||||
db.session.execute(db.text(
|
||||
"UPDATE agent_action SET name=:n, app=:a, aliases=:al, params=:p, steps=:s, "
|
||||
"preconditions=:pc, updated_at=:t WHERE id=:i"),
|
||||
{"n": a["name"], "a": a["app"],
|
||||
"al": json.dumps(a["aliases"], ensure_ascii=False),
|
||||
"p": json.dumps(a["params"], ensure_ascii=False),
|
||||
"s": json.dumps(a["steps"], ensure_ascii=False),
|
||||
"pc": a["preconditions"], "t": now, "i": aid})
|
||||
else:
|
||||
db.session.execute(db.text(
|
||||
"INSERT INTO agent_action(name, app, aliases, params, steps, preconditions, "
|
||||
"hits, source_prompt, created_at, updated_at) "
|
||||
"VALUES(:n,:a,:al,:p,:s,:pc,0,'(人工新增)',:t,:t)"),
|
||||
{"n": a["name"], "a": a["app"],
|
||||
"al": json.dumps(a["aliases"], ensure_ascii=False),
|
||||
"p": json.dumps(a["params"], ensure_ascii=False),
|
||||
"s": json.dumps(a["steps"], ensure_ascii=False),
|
||||
"pc": a["preconditions"], "t": now})
|
||||
db.session.commit()
|
||||
return jsonify({"ok": True, "msg": "动作已保存"})
|
||||
|
||||
|
||||
# ================== 历史会话(DeepSeek 式:多会话持久化) ==================
|
||||
# 会话 = 消息序列(JSON),一轮 run 的 user/assistant 文本完成后追加落库。
|
||||
# 单表 JSON 存储(每会话几十 KB 内,无需独立消息表)。
|
||||
@@ -829,7 +1271,8 @@ def _agent_thread(run_id, prompt, serial, cfg):
|
||||
def on_delta(text, kind):
|
||||
q.put(("delta", {"text": text, "kind": kind}))
|
||||
|
||||
tool_seq = [] # 本轮工具序列(经验提炼用)
|
||||
tool_seq = [] # 本轮工具序列(任务级配方提炼用)
|
||||
tool_trace = [] # 结构化轨迹:{tool, args, result}(动作经验提炼用,判成败)
|
||||
|
||||
def on_tool(step):
|
||||
# args 保留对象(json.dumps 序列化)——前端要解析 serial 做画面跟随;
|
||||
@@ -839,11 +1282,16 @@ def _agent_thread(run_id, prompt, serial, cfg):
|
||||
if step.get("image"):
|
||||
rec["image"] = _shrink_image(step["image"])
|
||||
q.put(("step", rec))
|
||||
# 记录精简工具序列
|
||||
# 记录精简工具序列 + 结构化轨迹(去 serial、结果截断)
|
||||
try:
|
||||
args = step.get("args") or {}
|
||||
brief = {k: v for k, v in args.items() if k != "serial"}
|
||||
tool_seq.append(f"{step.get('tool')}({str(brief)[:60]})")
|
||||
res = step.get("result")
|
||||
# 可沉淀工具保留原始 result(_tool_ok 需按字段判成败);其余只存摘要
|
||||
tool_trace.append({"tool": step.get("tool"), "args": brief,
|
||||
"result": res if step.get("tool") in _ACTION_TOOLS
|
||||
else _brief_result(res)})
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@@ -885,6 +1333,18 @@ def _agent_thread(run_id, prompt, serial, cfg):
|
||||
"args": f"命中 {len(exp_items)} 条同类历史经验,已注入参考"
|
||||
+ (f":{brief}" if brief else ""),
|
||||
"image": None}))
|
||||
# 动作经验:可复用的命名动作(带元素定位),优先复用可跳过重新探索
|
||||
act_ctx, act_items = _find_actions(prompt)
|
||||
if act_ctx:
|
||||
_log.info(f"命中可复用动作 {len(act_items)} 个,注入参考")
|
||||
q.put(("step", {"tool": "🧠 动作经验",
|
||||
"args": f"命中 {len(act_items)} 个可复用动作,已注入参考"
|
||||
+ (f":{'、'.join(act_items[:3])}" if act_items else ""),
|
||||
"image": None}))
|
||||
recall_ctx = exp_ctx
|
||||
if act_ctx:
|
||||
recall_ctx += ("\n\n## 可复用动作(优先按其中的元素定位操作;"
|
||||
"若与当前界面不符,再自行截图确认)\n" + act_ctx)
|
||||
|
||||
mcp_url = getattr(getattr(agent, "s", None), "mcp_url", "")
|
||||
|
||||
@@ -914,7 +1374,7 @@ def _agent_thread(run_id, prompt, serial, cfg):
|
||||
on_delta=on_delta, on_tool=on_tool,
|
||||
should_stop=lambda: bool(
|
||||
stop_evt and stop_evt.is_set()),
|
||||
extra_context=exp_ctx)
|
||||
extra_context=recall_ctx)
|
||||
except Exception as e:
|
||||
if _mcp_unreachable(e):
|
||||
raise RuntimeError(
|
||||
@@ -961,6 +1421,19 @@ def _agent_thread(run_id, prompt, serial, cfg):
|
||||
"args": "本轮操作已提炼为经验并写入记忆库"
|
||||
"(下次相似任务会自动参考)",
|
||||
"image": None}))
|
||||
# 动作经验:把本轮**成功**步骤沉淀为命名动作(带元素定位,禁坐标)
|
||||
acts = _distill_actions(cfg, prompt, tool_trace)
|
||||
n_act = _save_actions(prompt, acts)
|
||||
if n_act:
|
||||
q.put(("step", {"tool": "🧠 动作经验",
|
||||
"args": f"已沉淀 {n_act} 个可复用动作"
|
||||
"(含元素定位,下次同类任务可直接复用)",
|
||||
"image": None}))
|
||||
else:
|
||||
q.put(("step", {"tool": "🧠 动作经验",
|
||||
"args": "本轮未沉淀动作(步骤以坐标定位为主,"
|
||||
"缺少可复用的元素定位信息)",
|
||||
"image": None}))
|
||||
except Exception as e:
|
||||
_log.warning(f"经验保存异常: {e}")
|
||||
q.put(("done", {"answer": answer}))
|
||||
|
||||
Reference in New Issue
Block a user