Files
MediaCrawler/api/monitor/schedule.py
T
butubb 8f4e5586e9
Deploy VitePress site to Pages / build (push) Canceled after 0s
Deploy VitePress site to Pages / Deploy (push) Canceled after 0s
feat(schedule): 任务支持「每天定时 / 每周定时」,用选择器而不是手写 cron
- schedule.py: 新增调度计算模块(纯函数,便于单测)。三种模式:interval(每 N 分钟)/ daily(选钟点)/ weekly(选星期 + 钟点)
- 钟点模式是「固定时刻」而非「固定延迟」——从日历重算,所以某轮跑晚了不会把之后每一轮都拖晚。interval 保持原语义:从上一轮开始计时
- 抖动只加给 interval。给「每天 9:00」也加抖动就成了 9:00–9:01 随机触发,操作者选的时间被悄悄改掉,只会像 bug
- 模型 / schema / service: 新增 schedule_mode / schedule_hours / schedule_days / schedule_minute。时钟字段存逗号分隔文本——几个小整数、永远整体读写,单开一张表只会换来 join。interval_minutes 保留且仍是默认值,已有任务不受影响
- service: 改动任何调度字段都按合并后的状态重算 next_run_at。重新启用也算改动,否则停用一个月再打开会带着一个月前的 next_run_at,一保存就立即触发
- 前端: 运行方式三选一 + 小时/星期胶囊多选 + 分钟下拉,并实时预览结果句子。任务卡片改显示后端拼好的 schedule_label,避免列表和编辑器对同一计划给出两种说法
- 校验: 钟点模式至少选一个时间,按周至少选一个星期

tests/test_schedule.py 新增 24 个用例,含「恰好等于当前时刻的档位归属下一天」这个会让调度器自循环的边界。
2026-10-07 15:33:17 +08:00

168 lines
5.8 KiB
Python

# -*- coding: utf-8 -*-
# Copyright (c) 2025 [email protected]
#
# This file is part of MediaCrawler project.
# Repository: https://github.com/NanmiCoder/MediaCrawler/blob/main/api/monitor/schedule.py
# GitHub: https://github.com/NanmiCoder
# Licensed under NON-COMMERCIAL LEARNING LICENSE 1.1
#
# 声明:本代码仅供学习和研究目的使用。使用者应遵守以下原则:
# 1. 不得用于任何商业用途。
# 2. 使用时应遵守目标平台的使用条款和robots.txt规则。
# 3. 不得进行大规模爬取或对平台造成运营干扰。
# 4. 应合理控制请求频率,避免给目标平台带来不必要的负担。
# 5. 不得用于任何非法或不当的用途。
#
# 详细许可条款请参阅项目根目录下的LICENSE文件。
# 使用本代码即表示您同意遵守上述原则和LICENSE中的所有条款。
"""Monitor task schedule arithmetic.
Three modes, all expressible by a picker. A raw cron string was ruled out on
purpose -- it is a small language, and the operator should not have to write one
to say "every day at nine":
* ``interval`` -- every N minutes.
* ``daily`` -- at chosen clock times, e.g. 09:00 and 18:30.
* ``weekly`` -- at chosen clock times on chosen weekdays, e.g. Mon-Fri 10:00.
The two clock modes are **fixed-time**, unlike ``interval``, which is fixed-delay.
The distinction matters: for an interval, measuring the next slot from when the
run starts is what stops a slow run from firing back-to-back. For a clock
schedule it would be wrong, because a run that starts at 09:07 would drag every
later run seven minutes late, compounding all day.
Fixed-time also means **no jitter is applied** to clock schedules. The operator
picked a time; quietly running at 09:04 instead of 09:00 is not a feature, it just
looks like a bug. ``interval`` keeps its jitter, where there is no stated time to
contradict.
All arithmetic is in the server's local timezone -- naive datetimes on purpose,
because the container is pinned to the operator's zone via TZ and pretending
otherwise would add a timezone concept nobody asked for.
"""
from datetime import datetime, time, timedelta
from typing import Optional, Sequence
MODE_INTERVAL = "interval"
MODE_DAILY = "daily"
MODE_WEEKLY = "weekly"
SCHEDULE_MODES = (MODE_INTERVAL, MODE_DAILY, MODE_WEEKLY)
CLOCK_MODES = (MODE_DAILY, MODE_WEEKLY)
_MS_PER_MINUTE = 60_000
_WEEKDAY_NAMES = "一二三四五六日"
def parse_hours(raw: Optional[str]) -> list[int]:
"""``"9,18"`` -> ``[9, 18]``. Sorted, de-duplicated, junk dropped."""
return _parse_int_list(raw, 0, 23)
def parse_days(raw: Optional[str]) -> list[int]:
"""``"0,2,4"`` -> ``[0, 2, 4]``. **0 is Monday**, matching ``date.weekday()``."""
return _parse_int_list(raw, 0, 6)
def _parse_int_list(raw: Optional[str], low: int, high: int) -> list[int]:
values: set[int] = set()
for chunk in (raw or "").split(","):
chunk = chunk.strip()
if not chunk:
continue
try:
number = int(chunk)
except ValueError:
# Stored values come from our own UI, but a hand-edited row must not
# be able to crash the scheduler loop.
continue
if low <= number <= high:
values.add(number)
return sorted(values)
def format_hours(hours: Sequence[int]) -> str:
return ",".join(str(hour) for hour in sorted(set(hours)))
def format_days(days: Sequence[int]) -> str:
return ",".join(str(day) for day in sorted(set(days)))
def describe(
*,
mode: str,
interval_minutes: int,
hours: Sequence[int],
days: Sequence[int],
minute: int,
) -> str:
"""One human sentence for the task card.
Lives here rather than in the frontend so the list view and the editor cannot
drift apart on what a schedule means.
"""
if mode == MODE_INTERVAL:
if interval_minutes % 1440 == 0:
return f"每 {interval_minutes // 1440} 天"
if interval_minutes % 60 == 0:
return f"每 {interval_minutes // 60} 小时"
return f"每 {interval_minutes} 分钟"
if not hours:
return "未设置时间"
clock = "、".join(f"{hour:02d}:{minute:02d}" for hour in sorted(set(hours)))
if mode == MODE_DAILY:
return f"每天 {clock}"
if not days:
return f"每天 {clock}"
labels = "、".join(f"周{_WEEKDAY_NAMES[day]}" for day in sorted(set(days)))
return f"{labels} {clock}"
def next_occurrence(
*,
mode: str,
interval_minutes: int,
hours: Sequence[int],
days: Sequence[int],
minute: int,
after_ms: int,
) -> Optional[int]:
"""The next fire time strictly after ``after_ms``, as epoch milliseconds.
``None`` means the schedule can never fire -- a clock mode with no hours
chosen. Callers store that as "no next run" rather than something in the past,
which would otherwise leave the task permanently due and re-running on every
tick.
"""
if mode == MODE_INTERVAL:
return after_ms + max(1, interval_minutes) * _MS_PER_MINUTE
if not hours:
return None
now = datetime.fromtimestamp(after_ms / 1000)
# No weekdays chosen means every day, matching describe(). Without the `days`
# guard an empty selection would produce an empty allowed set, no matching day,
# and a task that silently never runs.
allowed_days = set(days) if (mode == MODE_WEEKLY and days) else set(range(7))
# Eight days of lookahead covers today's remaining slots plus a full week,
# which is more than any weekday selection can need.
for offset in range(8):
day = (now + timedelta(days=offset)).date()
if day.weekday() not in allowed_days:
continue
for hour in sorted(set(hours)):
candidate = datetime.combine(day, time(hour=hour, minute=minute))
if candidate > now:
return int(candidate.timestamp() * 1000)
return None