feat: run the scheduled jobs one after another from 10:00 on weekdays - #67
Merged
Merged
Conversation
A [queue] in jobs.toml replaces the per-job timers of github-prs-digest, zendesk-digest, crowdin-sync, snode-list and crowdin-duplicates. One session-ops-queue.timer, Mon..Fri 10:00 Australia/Melbourne, starts an anchor unit that Wants= each ready queued job; each job's drop-in orders it After= the one before, so they run in turn and a failure does not hold up the rest. crowdin-sync moves from weekly to every weekday, snode-list and crowdin-duplicates from daily to every weekday; their max_age_hours become the weekend gap plus the queue's timeouts. session-ops-silence keeps its own hourly timer.
install.sh links only the ready queued jobs into the queue, and After= orders nothing against a unit that is not being started. With only the previous job named, an empty env file in the middle of the queue let the job after it run alongside the ones before. Naming every job ahead keeps them one at a time whichever subset is ready. Also drops the last weekly/daily wording for crowdin-sync and crowdin-duplicates, and says in the deploy README to re-run one job by its own service, since starting the queue again re-runs the jobs already done.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves github-prs-digest, zendesk-digest, crowdin-sync, snode-list and crowdin-duplicates onto one weekday queue, run one at a time from 10:00 Australia/Melbourne. session-ops-silence keeps its hourly timer.
How
[queue]injobs.toml: the schedule and the order. A queued job takes noscheduleof its own; the registry refuses one, an unknown name or a duplicate.session-ops-queue.timerstartssession-ops-queue.service, a oneshot thatWants=each ready queued job.install.shrebuildssession-ops-queue.service.wants/from the jobs whose env files have content, disables the old per-job timers of queued jobs, and enables the queue timer.After=the job before it. For oneshot units that serialises them, and since it is ordering only, a failed job does not stop the next. Checked on systemd with throwaway user units: strict a → b → c,afailing, and a second start of the anchor re-runs all three.session-ops-queue.timerfor them. Theirmax_age_hoursis 80: the 72 h weekend (73 h in April) plus every job ahead running to its timeout (5 h 15 in all).session-ops list --queuedprints the order.Behaviour changes
Deploying
git -C /opt/session-ops pull && /opt/session-ops/deploy/install.sh, thensystemctl list-timers 'session-ops*'should list only the queue and silence timers, andls /etc/systemd/system/session-ops-queue.service.wants/the five jobs.A newly enabled timer has no last run to catch up, so a deploy after a day's 10:00 skips that day's queue; start any job that matters by hand.