feat: 监控面板 / 登录鉴权 / 多平台切换 / MySQL
Deploy VitePress site to Pages / build (push) Canceled after 0s
Deploy VitePress site to Pages / Deploy (push) Canceled after 0s

在上游 MediaCrawler 之上新增一层:

- 监控层 api/monitor/ —— 多博主/多笔记的定时采集、指标快照差分、报表、
  企业微信通知。每轮采集写入独立目录,差分才成立。
- WebUI 登录鉴权 api/auth.py —— PBKDF2 口令 + 服务端会话,/api 全接口防护。
  WebSocket 单独加依赖:BaseHTTPMiddleware 对 ws 作用域直接放行,覆盖不到。
- 全局平台切换 + 能力矩阵 —— 如实区分「爬虫模块支持」与「监控层已接线」,
  未接通的平台直接拒绝建任务,而不是静默跑空。
- 监控库改用 MySQL 5.7(可回退 SQLite 供测试):逐表强制 utf8mb4
  (服务端与库默认都是 latin1),启动校验所连 schema 以防写错库,
  连接池 recycle + pre_ping 应对 MySQL 的 8 小时空闲断连。

修复上游缺陷:

- xhs/core.py: 主页抓取失败会跳掉整个博主,导致一条作品都抓不到,
  而那份资料只喂给一个空函数。改为尽力而为,失败不中断。
- xhs/login.py: cookie 登录只注入 web_session,冷启动签名会失败。
  新增 INJECT_ALL_COOKIES 开关(默认关闭,原有行为不变)。
- requirements.txt: 补上 websockets。它在上游 pyproject.toml 里有声明、
  这里漏了,导致 uvicorn 没有 WebSocket 能力,实时日志流从未工作。

改动过的上游文件清单及合并方式见 UPSTREAM.md。

测试:492 passed(另有 1 个既有的 Windows/gbk 上游测试失败,与本改动无关)
This commit is contained in:
2026-10-07 09:58:40 +08:00
parent 5d547f4586
commit 4e60524f37
88 changed files with 13224 additions and 436 deletions
+72 -32
View File
@@ -1,19 +1,32 @@
import { Bug, Wifi, AlertTriangle, Github } from 'lucide-react'
import { useState } from 'react'
import { Bug, Wifi, BarChart3, Cog, LogOut, Radar, Settings, Terminal } from 'lucide-react'
import { useTranslation } from 'react-i18next'
import { Badge } from '@/components/ui/badge'
import { SystemSettingsDialog } from '@/components/settings/SystemSettingsDialog'
import { useCrawlerStore } from '@/store/crawlerStore'
import { useCrawlerStatus } from '@/hooks/useCrawler'
import type { AppView } from '@/App'
import { LanguageSwitch } from './LanguageSwitch'
import { PlatformSwitcher } from './PlatformSwitcher'
import { ThemeToggle } from './ThemeToggle'
interface SidebarProps {
onShowDisclaimer?: () => void
currentView: AppView
onViewChange: (view: AppView) => void
onLogout: () => void
}
export function Sidebar({ onShowDisclaimer }: SidebarProps) {
const NAV_ITEMS: Array<{ value: AppView; label: string; icon: typeof Terminal }> = [
{ value: 'crawler', label: '采集', icon: Terminal },
{ value: 'monitor', label: '监控', icon: Radar },
{ value: 'report', label: '报表', icon: BarChart3 },
{ value: 'settings', label: '设置', icon: Settings },
]
export function Sidebar({ currentView, onViewChange, onLogout }: SidebarProps) {
const { t } = useTranslation()
const { t: tLicense } = useTranslation('license')
const status = useCrawlerStore((state) => state.status)
const [systemSettingsOpen, setSystemSettingsOpen] = useState(false)
// Poll status
useCrawlerStatus()
@@ -23,21 +36,12 @@ export function Sidebar({ onShowDisclaimer }: SidebarProps) {
return (
<header className="h-14 flex-shrink-0 glass-panel border-b border-cyber-border-subtle relative z-10">
<div className="h-full px-4 flex items-center justify-between">
{/* Left: Logo and GitHub Star */}
{/* Left: Logo, run state, workspace switch */}
<div className="flex items-center gap-3">
<Bug className="w-5 h-5 text-cyber-neon-cyan" />
<span className="font-mono font-bold text-cyber-text-primary tracking-wider text-sm">
MediaCrawler
综合采集平台
</span>
<a
href="https://github.com/NanmiCoder/MediaCrawler"
target="_blank"
rel="noopener noreferrer"
className="flex items-center gap-1.5 px-2 py-1 rounded-md border border-cyber-border-subtle hover:border-cyber-neon-cyan hover:shadow-glow-cyan-sm transition-all bg-cyber-bg-tertiary"
>
<Github className="w-4 h-4 text-cyber-text-secondary" />
<span className="text-xs font-mono text-cyber-text-secondary">Star</span>
</a>
{isRunning && (
<Badge variant="running" className="text-[10px]">
{t('status.active')}
@@ -46,26 +50,46 @@ export function Sidebar({ onShowDisclaimer }: SidebarProps) {
{isRunning && (
<span className="w-2 h-2 bg-cyber-neon-green rounded-full shadow-glow-green-sm animate-pulse-fast" />
)}
{/* Workspace switch */}
<div className="flex items-center gap-1 ml-2 p-0.5 rounded-md border border-cyber-border-subtle bg-cyber-bg-tertiary">
{NAV_ITEMS.map((item) => {
const Icon = item.icon
const active = currentView === item.value
return (
<button
key={item.value}
onClick={() => onViewChange(item.value)}
className={`flex items-center gap-1.5 px-2.5 py-1 rounded text-[11px] font-mono transition-all ${
active
? 'bg-cyber-neon-cyan/20 text-cyber-neon-cyan border border-cyber-neon-cyan/40'
: 'text-cyber-text-muted border border-transparent hover:text-cyber-text-secondary'
}`}
>
<Icon className="w-3 h-3" />
{item.label}
</button>
)
})}
</div>
</div>
{/* Center: Warning Text */}
<button
onClick={onShowDisclaimer}
className="flex items-center gap-3 px-4 py-1.5 rounded-lg border border-cyber-neon-orange/50 bg-cyber-neon-orange/10 hover:bg-cyber-neon-orange/20 transition-all cursor-pointer"
>
<AlertTriangle className="w-4 h-4 text-cyber-neon-orange flex-shrink-0" />
<div className="flex items-center gap-4 text-xs font-mono">
<span className="text-cyber-neon-orange">
<span className="text-cyber-neon-pink font-bold">1.</span> {tLicense('content.line1')}
</span>
<span className="text-cyber-neon-orange">
<span className="text-cyber-neon-pink font-bold">2.</span> {tLicense('content.line2')}
</span>
</div>
</button>
{/* Right: Actions and Status */}
{/* Right: platform scope, then actions and status */}
<div className="flex items-center gap-3">
{/* Scopes every workspace at once. */}
<PlatformSwitcher />
{/* System-wide settings: deliberately separate from the platform-scoped
「设置」 page, because these values are not per platform. */}
<button
onClick={() => setSystemSettingsOpen(true)}
title="系统设置(对所有平台生效)"
className="flex items-center gap-1.5 px-2 py-1 rounded text-[11px] font-mono text-cyber-text-muted border border-transparent hover:text-cyber-neon-cyan hover:border-cyber-neon-cyan/40 transition-all"
>
<Cog className="w-3.5 h-3.5" />
系统设置
</button>
{/* Theme Toggle */}
<ThemeToggle />
{/* Language Switch */}
@@ -81,8 +105,24 @@ export function Sidebar({ onShowDisclaimer }: SidebarProps) {
<span className="status-dot status-dot-online" />
</div>
</div>
{/* Logging out unmounts the whole tree, which closes the log
WebSocket through the existing connection-count cleanup. */}
<button
onClick={onLogout}
title="退出登录"
className="flex items-center gap-1.5 px-2 py-1 rounded text-[11px] font-mono text-cyber-text-muted border border-transparent hover:text-cyber-neon-pink hover:border-cyber-neon-pink/40 transition-all"
>
<LogOut className="w-3 h-3" />
退出
</button>
</div>
</div>
<SystemSettingsDialog
open={systemSettingsOpen}
onOpenChange={setSystemSettingsOpen}
/>
</header>
)
}