feat: 监控面板 / 登录鉴权 / 多平台切换 / MySQL
在上游 MediaCrawler 之上新增一层： - 监控层 api/monitor/ —— 多博主/多笔记的定时采集、指标快照差分、报表、 企业微信通知。每轮采集写入独立目录，差分才成立。 - WebUI 登录鉴权 api/auth.py —— PBKDF2 口令 + 服务端会话，/api 全接口防护。 WebSocket 单独加依赖：BaseHTTPMiddleware 对 ws 作用域直接放行，覆盖不到。 - 全局平台切换 + 能力矩阵 —— 如实区分「爬虫模块支持」与「监控层已接线」， 未接通的平台直接拒绝建任务，而不是静默跑空。 - 监控库改用 MySQL 5.7（可回退 SQLite 供测试）：逐表强制 utf8mb4 （服务端与库默认都是 latin1），启动校验所连 schema 以防写错库， 连接池 recycle + pre_ping 应对 MySQL 的 8 小时空闲断连。 修复上游缺陷： - xhs/core.py: 主页抓取失败会跳掉整个博主，导致一条作品都抓不到， 而那份资料只喂给一个空函数。改为尽力而为，失败不中断。 - xhs/login.py: cookie 登录只注入 web_session，冷启动签名会失败。 新增 INJECT_ALL_COOKIES 开关（默认关闭，原有行为不变）。 - requirements.txt: 补上 websockets。它在上游 pyproject.toml 里有声明、 这里漏了，导致 uvicorn 没有 WebSocket 能力，实时日志流从未工作。 改动过的上游文件清单及合并方式见 UPSTREAM.md。 测试：492 passed（另有 1 个既有的 Windows/gbk 上游测试失败，与本改动无关）
This commit is contained in:
@@ -8,7 +8,7 @@
|
||||
"error": "ERROR"
|
||||
},
|
||||
"sidebar": {
|
||||
"title": "MediaCrawler",
|
||||
"title": "Integrated Collection Platform",
|
||||
"api": "API",
|
||||
"local": "LOCAL",
|
||||
"disclaimer": "For personal learning only, commercial use prohibited",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"title": "ENV_DIAGNOSTICS",
|
||||
"scanning": "[SCAN] Analyzing MediaCrawler environment...",
|
||||
"scanning": "[SCAN] Analyzing Integrated Collection Platform environment...",
|
||||
"success": "[OK] {{message}}",
|
||||
"error": "[ERR] {{message}}",
|
||||
"defaultError": "Cannot connect to API server",
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"title": "Usage Notice",
|
||||
"warning": "Please read the following terms carefully",
|
||||
"content": {
|
||||
"line1": "This project is for personal learning and research purposes only",
|
||||
"line2": "Commercial use or profit-making activities are strictly prohibited",
|
||||
"line3": "By using this project, you agree to comply with all applicable laws",
|
||||
"line4": "Users bear full responsibility for any consequences of misuse"
|
||||
},
|
||||
"license": "View Full License",
|
||||
"github": "Visit GitHub Repository",
|
||||
"confirm": "I understand and agree to the above terms",
|
||||
"decline": "Disagree, Exit",
|
||||
"author": {
|
||||
"name": "Relakkes (阿江)",
|
||||
"tagline": "40K+ Star Open Source Author",
|
||||
"description": "Author of MediaCrawler (40K stars), focused on crawler tech and AI Agent knowledge sharing",
|
||||
"support": "If you find this project helpful, please follow the author on social media",
|
||||
"slogan": "Open source is hard, please star ⭐"
|
||||
}
|
||||
}
|
||||
@@ -8,7 +8,7 @@
|
||||
"error": "错误"
|
||||
},
|
||||
"sidebar": {
|
||||
"title": "MediaCrawler",
|
||||
"title": "综合采集平台",
|
||||
"api": "API",
|
||||
"local": "本地",
|
||||
"disclaimer": "仅供个人学习研究使用，禁止商业用途",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"title": "环境检测",
|
||||
"scanning": "[扫描] 正在分析 MediaCrawler 环境...",
|
||||
"scanning": "[扫描] 正在分析综合采集平台环境...",
|
||||
"success": "[成功] {{message}}",
|
||||
"error": "[错误] {{message}}",
|
||||
"defaultError": "无法连接到 API 服务器",
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"title": "使用须知",
|
||||
"warning": "请仔细阅读以下条款",
|
||||
"content": {
|
||||
"line1": "本项目仅供个人学习和研究爬虫技术使用",
|
||||
"line2": "严禁将本项目用于任何商业用途或盈利活动",
|
||||
"line3": "使用本项目即表示您同意遵守相关法律法规",
|
||||
"line4": "任何因违规使用造成的后果由使用者自行承担"
|
||||
},
|
||||
"license": "查看完整协议",
|
||||
"github": "访问 GitHub 仓库",
|
||||
"confirm": "我已知晓并同意上述条款",
|
||||
"decline": "不同意，退出",
|
||||
"author": {
|
||||
"name": "程序员阿江-Relakkes",
|
||||
"tagline": "40K+ Star 开源项目作者",
|
||||
"description": "40K开源项目MediaCrawler作者，专注爬虫技术和AI Agent知识分享",
|
||||
"support": "如果觉得项目有帮助，欢迎关注作者的社交媒体账号",
|
||||
"slogan": "开源不易，求个关注 ⭐"
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user