Sync: dev to main

.env.example

@@ -2,7 +2,7 @@
 # sites/mainweb/.env.local, which is where Next.js looks. Never commit either.
 #
 # Every default below points at the docker-compose.local.yml services, so a
-# fresh checkout touches nothing outside this machine. scripts/guard-local.mjs
+# fresh checkout touches nothing outside this machine. tooling/scripts/src/guard-local.ts
 # refuses to migrate, seed or test against a non-local DATABASE_URL, and refuses
 # Stripe live keys outside production.
 
@@ -28,6 +28,9 @@ GOOGLE_CLIENT_ID=
 GOOGLE_CLIENT_SECRET=
 GITHUB_CLIENT_ID=
 GITHUB_CLIENT_SECRET=
+# Optional: raises the judging repo check GitHub limit from 60 to 5000 requests
+# an hour. A token with no scopes is enough for public repos.
+GITHUB_TOKEN=
 
 # --- Email ------------------------------------------------------------------
 # Mailpit from docker-compose.local.yml. The 6-digit sign-in code and every other

.github/workflows/uptime.yml

@@ -63,7 +63,7 @@ jobs:
       - name: Probe
         env:
           UPTIME_FILE: data/uptime.json
-        run: node scripts/uptime-probe.mjs
+        run: node tooling/scripts/src/uptime-probe.ts
 
       # Always one commit: a fresh orphan, force-pushed over the old one.
       - name: Publish

apphosting.yaml

@@ -47,6 +47,14 @@ env:
     secret: AUTH_GITHUB_ID
   - variable: GITHUB_CLIENT_SECRET
     secret: AUTH_GITHUB_SECRET
+  # Optional: raises the judging repo check from 60 to 5000 GitHub requests an
+  # hour. Left commented out because a reference to a secret that does not exist
+  # breaks the rollout. Create it first (a token with no scopes is enough), then
+  # uncomment:
+  #   firebase apphosting:secrets:set GITHUB_REPO_CHECK_TOKEN
+  #   firebase apphosting:secrets:grantaccess GITHUB_REPO_CHECK_TOKEN
+  # - variable: GITHUB_TOKEN
+  #   secret: GITHUB_REPO_CHECK_TOKEN
   - variable: AUTH_URL
     value: https://datasciencegt.org
   - variable: NEXTAUTH_URL

docs/features/status.md

@@ -25,7 +25,7 @@ Consequences, stated plainly:
 
 The sampled numbers have a blind spot: no traffic means no data, and the app cannot see itself being unreachable. A GitHub Actions workflow, `.github/workflows/uptime.yml`, covers that from outside. The repo is public, so the minutes are free.
 
-- Every 5 minutes (`cron: "*/5 * * * *"`, also `workflow_dispatch`) it runs `scripts/uptime-probe.mjs`: plain Node, no install step. The script requests `https://datasciencegt.org/api/ping` up to 3 times with a 10 second timeout each. A non-2xx answer or a timeout is a failed attempt. It records the status code and latency of the first success, or of the last failure (`code` 0 means no answer).
+- Every 5 minutes (`cron: "*/5 * * * *"`, also `workflow_dispatch`) it runs `tooling/scripts/src/uptime-probe.ts`: plain Node, no install step. The script requests `https://datasciencegt.org/api/ping` up to 3 times with a 10 second timeout each. A non-2xx answer or a timeout is a failed attempt. It records the status code and latency of the first success, or of the last failure (`code` 0 means no answer).
 - `/api/ping` (`app/api/ping/route.ts`) returns `{ ok: true }` with `cache-control: no-store` and touches nothing else. It deliberately never reaches the database: Neon bills compute while awake and suspends after about 5 idle minutes, so a probe that ran a query would keep it awake 24 hours a day. It is not a tRPC route, so it does not count as traffic in `status_sample`, and `proxy.ts` only sets headers on it.
 - The result goes to `uptime.json` on the `status-data` branch. Every run builds a new orphan commit and force-pushes it, so the branch is always exactly one commit and the repository does not grow. The workflow uses the `github-actions[bot]` identity and only needs `contents: write`.
 - The same run records the latest `main` commit (SHA, commit time) and the conclusion of its `App Hosting - Rollout` check run as `deploy`, using the workflow token. The site never calls the GitHub API; a failed lookup keeps the previous value.
@@ -40,7 +40,7 @@ The sampled numbers have a blind spot: no traffic means no data, and the app can
 | `days` | Up to 90 UTC days of probes older than 24 hours: `{ day, probes, up, worstMs }` |
 | `deploy` | `{ sha, committedAt, rollout }` for the newest `main` commit |
 
-Rolling `recent` into `days` is the pure function `applyProbe` in the script, tested in `scripts/uptime-probe.test.mjs`. The read side (windows, stale and down rules) is `sites/mainweb/lib/uptime.ts`, tested in `lib/uptime.test.ts`.
+Rolling `recent` into `days` is the pure function `applyProbe` in the script, tested in `tooling/scripts/src/uptime-probe.test.ts`. The read side (windows, stale and down rules) is `sites/mainweb/lib/uptime.ts`, tested in `lib/uptime.test.ts`.
 
 How the headline uses it:
 
@@ -192,7 +192,7 @@ See [Admin tools](./admin.md).
 | Timing middleware | `packages/api/src/trpc.ts` (`recordDuration`) |
 | Endpoint | `sites/mainweb/app/(portal)/api/metrics/route.ts` |
 | Status page | `sites/mainweb/app/status/` (`page.tsx`, `load.ts`, `uptime.ts`, `AutoRefresh.tsx`), rules in `sites/mainweb/lib/status.ts` and `lib/uptime.ts` |
-| External probe | `.github/workflows/uptime.yml`, `scripts/uptime-probe.mjs`, `sites/mainweb/app/api/ping/route.ts` |
+| External probe | `.github/workflows/uptime.yml`, `tooling/scripts/src/uptime-probe.ts`, `sites/mainweb/app/api/ping/route.ts` |
 | Status samples | `packages/api/src/services/status-samples.ts`, table `packages/db/src/schemas/status.ts` |
 | Stack and rules | `monitoring/` (`prometheus.yml`, `rules/payments.yml`, `rules/hackathon.yml`, `grafana/dashboards/dsgt-portal.json`, `README.md`) |
-| Tests | `services/latency.test.ts`, `services/hackathon-metrics.test.ts`, `services/status-samples.test.ts`, `lib/status.test.ts`, `lib/uptime.test.ts`, `app/status/load.test.ts`, `app/status/uptime.test.ts`, `scripts/uptime-probe.test.mjs` |
+| Tests | `services/latency.test.ts`, `services/hackathon-metrics.test.ts`, `services/status-samples.test.ts`, `lib/status.test.ts`, `lib/uptime.test.ts`, `app/status/load.test.ts`, `app/status/uptime.test.ts`, `tooling/scripts/src/uptime-probe.test.ts` |

docs/getting-started.md

@@ -39,7 +39,7 @@ pnpm dev
 
 **Payments.** `STRIPE_MOCK_MODE=true` (the default in `.env.example`) records a paid membership with no network call. To try real Checkout, switch it off and set **test-mode** keys (`sk_test_`, `pk_test_`); see [Stripe test cards](https://docs.stripe.com/testing), e.g. `4242 4242 4242 4242`.
 
-**Guardrails.** `scripts/guard-local.mjs` runs before `migrate:push`, `db:seed`, `db:seed:club-projects`, `pnpm test`, `pnpm local:up` and the mainweb `dev` script. It refuses a `DATABASE_URL` that is not `localhost`, `127.0.0.1`, `::1`, `db` or `postgres`, and an SMTP host that is not local, unless `ALLOW_NON_LOCAL_SERVICES=1` is set. It refuses Stripe live keys (`sk_live_`, `rk_live_`, `pk_live_`) whenever `NODE_ENV` is not `production`, with no override. An unset `DATABASE_URL` passes, which is how CI runs. Deploys call `drizzle-kit push` directly and are unaffected. The resume and bootcamp buckets are unset locally, so uploads return 503 rather than touching Cloud Storage.
+**Guardrails.** `tooling/scripts/src/guard-local.ts` runs before `migrate:push`, `db:seed`, `db:seed:club-projects`, `pnpm test`, `pnpm local:up` and the mainweb `dev` script. It refuses a `DATABASE_URL` that is not `localhost`, `127.0.0.1`, `::1`, `db` or `postgres`, and an SMTP host that is not local, unless `ALLOW_NON_LOCAL_SERVICES=1` is set. It refuses Stripe live keys (`sk_live_`, `rk_live_`, `pk_live_`) whenever `NODE_ENV` is not `production`, with no override. An unset `DATABASE_URL` passes, which is how CI runs. Deploys call `drizzle-kit push` directly and are unaffected. The resume and bootcamp buckets are unset locally, so uploads return 503 rather than touching Cloud Storage.
 
 ## Local database
 

docs/judging/README.md

@@ -12,6 +12,7 @@ The code is one tRPC router, `judging`, with five sub-routers (`packages/api/src
 | [Judging day](./judging-day.md) | The judge desk, presence, appointment status, takeover, grading hold, leaving a room, announcements, hacker itinerary |
 | [Scoring and results](./scoring-and-results.md) | Evaluations, scoring, calibration, the judge's private shortlist, placements, publishing, reopening a grade |
 | [Scheduler](./scheduler.md) | How room presentations are placed on a timetable and judges assigned to them |
+| [Repo check](./repo-check.md) | Flagging projects pushed to on GitHub after the submission deadline, and the organiser decisions |
 | [Adaptive features](./adaptive.md) | Setup advice, suggested timing, archiving rooms, live re-balance of judges |
 | [Simulations](./simulations.md) | Load simulations and what they found |
 
@@ -74,7 +75,7 @@ The order is a checklist, not a hard gate. `judging.admin.checklist` reports sev
 
 | Table | Holds |
 | --- | --- |
-| `judging_config` | One row per edition: state, switches (`showAllScores`, `itineraryPublished`, `emergencyMode`, `autoRebalance`), lock timestamps |
+| `judging_config` | One row per edition: state, switches (`showAllScores`, `itineraryPublished`, `emergencyMode`, `autoRebalance`, `repoCheckEnabled` and its grace), lock timestamps |
 | `judging_entry`, `judging_entry_member`, `judging_claim_link` | Projects, their teammates, one-time claim links |
 | `judging_challenge`, `judging_entry_challenge` | The challenge tree and which projects are in which challenge |
 | `judging_building`, `judging_room`, `judging_room_link`, `judging_guest_session` | Halls, rooms and tables, QR links, guest cookie sessions |
@@ -83,6 +84,7 @@ The order is a checklist, not a hard gate. `judging.admin.checklist` reports sev
 | `judging_rubric_item`, `judging_evaluation`, `judging_evaluation_draft`, `judging_evaluation_rating`, `judging_evaluation_response`, `judging_evaluation_revision` | The rubric and judges' answers |
 | `judging_deliberation_section`, `judging_deliberation_entry` | A judge's private shortlist |
 | `judging_placement` | Computed and published placings |
+| `judging_repo_check` | Per project: GitHub last-push check against the submission deadline and the organiser decision, see [repo check](./repo-check.md) |
 | `judging_announcement` | The pinned message for all judges or one room |
 
 The older `judging_project`, `judge_vote`, `judge_queue` and `hackathon_result` tables are retired and kept for past editions. `hackathon.getResults` still merges their published rows with new placements.

docs/judging/repo-check.md

@@ -0,0 +1,86 @@
+# Repo check
+
+A late-push check for judging. When an organiser turns it on for an edition, the app looks at each project's GitHub repo and flags the ones that were pushed to after the submission deadline. An organiser then confirms (disqualifies) or clears each flag. Nothing is disqualified automatically. Back to the [judging overview](./README.md).
+
+The code is `services/judging/repo-check.ts` (the check), `routers/judging/admin/repo-check.ts` (the switch and the decisions) and `RepoCheckSection.tsx` in `sites/mainweb/components/admin/judging/` (the console's Repo check section).
+
+## The signal
+
+The check goes by GitHub's own record of the last push, the `pushed_at` field of `GET /repos/{owner}/{repo}`. That is the server's clock and covers a push to any branch. Commit dates are never used to decide: they are set by the hacker's computer and can be faked in either direction.
+
+When `pushed_at` is later than the deadline plus the edition's grace, the project is flagged `late` and the check also fetches evidence for the organiser:
+
+- Commits since the deadline (`GET /repos/{owner}/{repo}/commits?since=...`), the latest 20: sha, first line of the message, author, commit date, link. The commit date is shown for context only. This lists the default branch.
+- Pushes since the deadline from the repo's events (`GET /repos/{owner}/{repo}/events`), the latest 20 PushEvents: time, ref and who pushed. GitHub keeps only recent events.
+
+The deadline is the submission close time, the same one the team submission window uses (`computeSubmissionWindow`: 36 hours after hacking starts). A push exactly on the deadline plus grace is on time.
+
+## Which repo
+
+Every distinct github.com repo the entry links to, at most three: the portal project's GitHub link, then the repo links among the Devpost demo links. The entry is flagged if any of them is late, and evidence is kept per repo. A link is read strictly: a trailing `.git`, `/tree/...` or any other path after the repo is ignored, and anything that is not `github.com/owner/repo` (a profile, another host) counts as no repo.
+
+| Outcome | Meaning |
+| --- | --- |
+| `ok` | Last push at or before the deadline plus grace |
+| `late` | Pushed after the deadline plus grace |
+| `no_repo` | No usable repo link |
+| `unreadable` | GitHub answered 401, 403, 404, 410 or 451: private, deleted or blocked (a 403 that mentions a rate limit is `error` instead) |
+| `error` | Rate limit, timeout or network failure. Retried on the next run. |
+
+Everything except `ok` appears in the flagged list. A flag that is still pending is kept, evidence and all, when a later check cannot look (a repo made private) or finds nothing, since `pushed_at` only moves forward; the latest attempt is shown beside it. Changing the grace re-works pending flags from scratch.
+
+## Organiser flow
+
+Open the Repo check section of the judging console.
+
+1. Optionally set the grace in minutes (default 0). Only pushes after the deadline plus the grace are flagged.
+2. Turn the check on. While judging is open and someone has the console open, it runs every 15 minutes. Check now runs it at once, any time, whatever the switch, and waits up to 25 seconds; whatever finished is saved and the run carries on for the rest.
+3. For each flagged project, read the repo link, the last push against the deadline (Eastern time) and the evidence, add an optional note, then:
+   - **Disqualify** takes the project out of judging, the same way removing it does (see below).
+   - **Clear** closes the flag and changes nothing else.
+   - **Reinstate** undoes a disqualification: the project comes back through the same path as restoring a removed project, and the flag is cleared.
+
+Every decision writes an audit log entry. The note is kept with the decision for organisers only; hackers and the public results never see a reason.
+
+A project that was decided is not looked at again unless its repo was pushed to after the decision. Then the flag reopens as pending with the new evidence. A disqualified project that reopens is still out until an organiser reinstates it.
+
+## What disqualifying does
+
+It sets `deletedAt` on the judging entry, which is what removing a project does. Everything that already leaves a removed project out follows from that: placements and public results, the unscheduled queue, the judge itinerary, and judge assignment and re-balancing (its saved appointments count as removed, so no judge is assigned and nothing is moved for them). Its evaluations stay, pointing at it. A hacker on that team sees what they see for any removed project and no reason. A portal submission removed this way cannot be resubmitted by its team.
+
+Unlike Remove project, it does not require the saved schedule to be dropped first: the board already treats a removed project's appointments as removed.
+
+Reinstating restores only if the project is still deleted with the `deletedAt` the disqualification set (`removed_deleted_at`), so a manual restore followed by another removal is not undone. It clears `deletedAt` through `restoreLockedEntry`, which is also what Restore uses, and so refuses a portal submission whose team withdrew it. It only restores a project the disqualification itself removed (`removed_entry` on the check row); one an organiser had already removed stays removed.
+
+## How it runs
+
+There is no background worker, matching the lazy re-balance. `judging.admin.overview`, which the console polls every 10 seconds, calls `repoCheckIfDue` without waiting for it:
+
+- An in-process throttle allows one attempt per edition every 15 minutes.
+- The attempt claims the run in the database: one update of `judging_config.repo_check_last_run_at`, only if the switch is on, judging is `open` and the last run is older than 15 minutes. Only the instance that wins the claim starts a run, so several instances do not all call GitHub.
+- The run happens in the background; the poll returns at once and a later poll shows the results.
+- Projects checked longest ago (or never) go first, so a low rate limit still reaches the tail over several runs.
+- Each result is saved under the same edition lock the decisions take, against the row as it is then, so a decision made during a run is never overwritten.
+- A run in this process older than 10 minutes no longer blocks the next one.
+- At most 5 GitHub requests are in flight, each with a 10 second timeout and a `User-Agent`.
+- Turning the switch on or changing the grace makes the next poll run it.
+
+## GitHub token and rate limits
+
+Set `GITHUB_TOKEN` (optional, any token that can read public repos) for 5000 requests an hour. The console warns while it is missing, and `apphosting.yaml` has the entry commented out until the secret exists. Without it GitHub allows 60 an hour, enough for a small edition. When GitHub answers with its rate limit, or the last response says no requests remain, the run stops: projects not yet checked are recorded as `error` (a decided project keeps its decision) and the next run, 15 minutes later, picks them up.
+
+## Limits
+
+- Pushes to any branch count, including a branch the team never submitted.
+- A force-push can move a branch back, but `pushed_at` still moves forward, so it is still flagged. The commit list may not show what was pushed.
+- A repo made private or deleted after the deadline reads as `unreadable`. The check cannot tell that from one that was private all along.
+- A team can push from a fork, or keep its real work elsewhere and link another repo. The check does not see either.
+- `pushed_at` can move for reasons that are not code, so the evidence is there for a person to read.
+- The in-process throttle is per instance. A switch changed on one instance can take up to 15 minutes to be picked up by the poll on another; Check now is immediate.
+
+## Data
+
+| Table | Holds |
+| --- | --- |
+| `judging_repo_check` | One row per project: repo, outcome, `pushed_at`, evidence (JSON, at most 20 commits and 20 pushes), when it was checked, and the decision (`pending`, `disqualified`, `cleared`) with who, when and the note |
+| `judging_config` | `repo_check_enabled`, `repo_check_grace_minutes`, `repo_check_last_run_at` |

docs/operations/environment.md

@@ -22,6 +22,7 @@ Without `DATABASE_URL`, `db` is null, sessions fall back to JWT, and tRPC proced
 | `GOOGLE_CLIENT_SECRET` | Google (`AUTH_GOOGLE_SECRET`) |
 | `GITHUB_CLIENT_ID` | GitHub (`AUTH_GITHUB_ID`) — optional; both GitHub vars required to register the provider |
 | `GITHUB_CLIENT_SECRET` | GitHub (`AUTH_GITHUB_SECRET`) |
+| `GITHUB_TOKEN` | Optional. Lets the judging [repo check](../judging/repo-check.md) read repos at 5000 requests an hour instead of 60. Needs no scopes for public repos. |
 
 ## Email
 

docs/operations/testing.md

@@ -12,7 +12,7 @@ That is:
 vitest run packages/api packages/db sites/mainweb/lib
 ```
 
-`pnpm test` first runs `scripts/guard-local.mjs`, which refuses a non-local `DATABASE_URL` or a Stripe live key. The suites mock the database, so CI sets neither and passes.
+`pnpm test` first runs `tooling/scripts/src/guard-local.ts`, which refuses a non-local `DATABASE_URL` or a Stripe live key. The suites mock the database, so CI sets neither and passes.
 
 CI: the `verify` job in `.github/workflows/ci.yml` on pushes to `main`/`dev` and on pull requests.