Skip to content

Simplify team storage and use direct Ethernet for NFS - #122

Merged
anshag01 merged 3 commits into
mainfrom
feat/simple-storage-ethernet-20261021
Oct 22, 2026
Merged

anshag01 merged 3 commits into
mainfrom
feat/simple-storage-ethernet-20261021

Conversation

@utmist-tt

Copy link
Copy Markdown
Collaborator

Changes

  • Replace nested per-team filesystem images with ordinary dataset/result folders on QuietBox under /srv/mist-data. Retain the previous storage unchanged for rollback; no migration.
  • Persist a dedicated 1 Gbps Ethernet link (192.168.240.1/30 ↔ 192.168.240.2/30) without changing internet or cluster node routes. New NFS 4.1 mounts use this link.
  • Make team dataset/model allowances advisory, preserve API fields, show actual shared disk free space and sampled team usage, and retain upload headroom checks.
  • Preserve membership, common-write permissions, cross-team grants, selected read-only dataset mounts and scoped job outputs. Refuse stale volume bindings and unsafe folder layouts.
  • Replace image provisioning/growth tooling with a small folder provisioning/usage service and current operations/rollback documentation.

Latest-main prerequisites

The first commit fixes issues encountered deploying main's new Go auth against the existing Better Auth database: require genuine unexpired sessions (remove user-ID cookie fallbacks), verify existing scrypt hashes, retain inactive-account state, and supply required legacy schema values. Fixtures now use actual test sessions. Existing administrator credentials were verified privately against a database snapshot and the deployed portal. No auth redesign.

Also remove an existing unused frontend variable that blocked type checking and update Vite/launch-editor patch versions; npm audit reports zero vulnerabilities.

Verification

  • 43 API + 6 CLI Go tests with race detection; go vet.
  • 16 React tests, TypeScript, lint and production build.
  • 6 small Python folder tests; shell syntax and documentation links.
  • Live regular-member upload/checksum/download, unsafe archive rejection, team isolation, common-write restrictions and active-dataset deletion protection.
  • Actual NVIDIA training on one RTX A4000 and Tenstorrent training on one n300 board (two chips), reading the uploaded dataset and saving/reloading/downloading checkpoints. Both succeeded with convergence checks.
  • Real browser login, upload and completed-job history; zero JavaScript errors.
  • Final deployed API binary checksum matches the candidate; both cluster nodes Ready.
  • Small 16 MiB direct-I/O NFS sample: 43 MB/s write, 101 MB/s read. This is not a sustained benchmark. No large storage tests or quota allocations.

Deployment and evidence

Already running on the two physical machines. Separate old auth deployment remains scaled to zero for rollback. Private credentials, DB backups and rollback manifests are outside Git.

Boundaries

Shared physical capacity still depends on one storage host. Advisory allowances do not guarantee equal space. No automatic backups, local dataset cache or resumable upload added. Older human-authored operating snapshots remain historical; the repository documentation policy restricts agent edits there, and the current README points to the replacement guide.

Main's independent-review requirement applies; this branch does not bypass it.

@github-actions

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

@anshag01
anshag01 merged commit f920e84 into main Oct 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants