Repository navigation
feat(engines): engine and image changes for v2.6.0 - #20
Merged
Merged
Conversation
The rsync image carries docker/aws.config at /etc/aws.config and sets AWS_CONFIG_FILE to it: a 4 GiB multipart threshold, 512 MiB parts, and the standard retry mode with ten attempts. The four mirror copies differ (tlnet's threshold is 200MB); the image's value wins over a Taskfile env:, so they go dead once this image is live, and each mirror deletes its copy in its own PR. README and the publish comment no longer say the mirror's Taskfile names the file.
The proton image has no keyring, so it sets PROTON_DRIVE_CREDENTIALS_STORE=unsafe_file as an ENV line beside AWS_REGION. The engine's PD drops the setting and keeps the cache dir, which stays with each caller. Every [pd] line of a render changes.
Each engine names its image in vars: as {{.IMAGE | default ...}}, so the
IMAGE on the toolbox include, the examples' -dev say, still wins; a
literal there would beat the include. A mirror passes IMAGE only when
its engine does not name it.
The toolbox drops its report-engine no-op and the rule that every engine
consumer excludes it: an engine defines report-engine, and a mirror with
no engine defines its own (dropbox does). An excludes: entry that names
no task is accepted, so every mirror's Taskfile works as it is. Both
examples drop the exclude, and the README's examples follow.
STAGING is defined once, beside RUN in the toolbox, and clean uses it.
The proton engine's SESSION and UPLOAD keep ROOT_DIR/.run: read from a
root task, a global that reads another include's RUN can render empty,
because go-task orders sibling includes' vars differently from run to
run.
batches counts the batch files once and writes .run/chain when more than MAX_BATCHES remain. delete's and reconcile's status, smoke's tlpdb read-back and smoke's never-landed check counted again; each runs after batches in the same pipeline run, so each now tests the chain file. The report's Delta row keeps its count, because it prints it.
publish uploaded a bucket-root timestamp last, label-trees held it back,
and smoke sampled it, all by that literal name. gnu's
mirror-updated-timestamp.txt and nongnu's 00_TIME.txt could land before
the tree they date. All four places now key on FRESH_KEY, under
{{if .FRESH_KEY}}: ctan's render is unchanged, gnu's and nongnu's name
their own file, and tlnet, which sets none, drops the upload-last step.
The rsync example sets FRESH_KEY: timestamp, so smoke's offline check
now runs fresh over run-smoke. offline writes a current Unix clock into
that copy first, and the fixture lists timestamp at its 11 bytes.
Exit 23 can hide a whole directory, and LIST_FLOOR is the only cap on a deletion list, so a floor at half the usual size lets one run delete half a mirror. The engine's comment and the README's vars table say about 90%.
Only github's render covered the two verbs, so lib's CI could not catch a break in them. The example excludes the engine's prune and defines a two-command one: list the folder stage fills, then trash nothing, since stage always writes the same file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR A of lib v2.6.0, from the consistency review. Every change works with each mirror's Taskfile as it is on main, because the tag moves
v2before any mirror PR merges.docker/aws.configships in the rsync image at/etc/aws.config, and the image setsAWS_CONFIG_FILE. An image env var beats a Taskfileenv:entry (tested on 3.53.1), so the four mirror copies go dead; they are deleted in phase 3.IMAGE: '{{.IMAGE | default "ghcr.io/katoptra/toolbox:<variant>-v2"}}'. The default form keeps the include's IMAGE (the examples'-dev) winning on go-task 3.53.1, where a literal engine var can beat it.FRESH_KEY, not a hard-codedtimestamp. The rsync example setsFRESH_KEY: timestamp..run/chain, whichbatcheswrites earlier in the same run.report-engineno-op is gone. Every engine defines it, and dropbox defines its own (dropbox#15).PROTON_DRIVE_CREDENTIALS_STORE=unsafe_file;pdno longer sets it.STAGINGis defined once, in the toolbox. proton.yml's SESSION and UPLOAD keepROOT_DIR/.run: go-task evaluates sibling includes' globals in random order for a mirror's own tasks, so an engine global that reads the toolbox'sRUNcan see it unset (lib's proton offline failed 6 of 8 runs with{{.RUN}}).list-folderandtrashthrough a two-commandprune.LIST_FLOORguidance is about 90% of the usual line count.Test plan
task image-build,task run -- task tools,task check,task run -- task offline(every commit passestask checkon its own)validate-vars.sh --check,chain-file.sh --check,gpg-gate-check.sh[pd]lines; dropbox unchanged.Rollout notes
mkdir(STAGING unset), before any upload.task image-cleanonce after the tag.task imagedoes not pull while a local image exists.