Skip to content

Store image-index embeddings as fp16 - #9696

Merged
joshistoast merged 5 commits into
invoke-ai:mainfrom
JPPhoto:fp16-embedding-storage
Oct 8, 2026
Merged

joshistoast merged 5 commits into
invoke-ai:mainfrom
JPPhoto:fp16-embedding-storage

Conversation

@JPPhoto

@JPPhoto JPPhoto commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Persist image and video embeddings as little-endian float16 with explicit per-row encoding. Search and projection continue to use float32. A SQLite migration converts existing embeddings without rerunning the embedding model, and legacy float32 rows remain readable.

Related Issues / Discussions

QA Instructions

  • Focused image-index and migration suites: 251 passed.
  • Image Map router suite: 93 passed.
  • Chromium component tests for Gallery search and Image Map: 41 passed with a mocked API.
  • Docs package test: 1 passed. Ruff 0.11.13 lint and format checks passed.
  • One-off numerical QC ran against a SQLite backup; the source database was not modified. Across 100 queries against 163,479 images and 23 videos, top-10 overlap was 100%; maximum score drift was 2.0401e-4. There were no ranking changes among 114,421 candidate pairs with an original score gap above 0.002.
  • A 163,000-vector migration fixture took 13.31 seconds including backup. Raw embedding BLOB storage halved; a separate VACUUM reduced the fixture database from about 698.9 MB to 361.4 MB.
  • Exact Ruff 0.11.2 and the full docs build were unavailable because of DNS failures. The docs build could not fetch the Starlight changelog from GitHub.

Review

No material review findings. A reviewer suggested adding a regression test for replacing existing legacy float32 rows and that can be left as a follow-up if deemed important.

Compatibility / Rollout

The migration runs on first startup after upgrade, in a database transaction, and creates a database backup before conversion. Restore the pre-upgrade backup before running an older application version. SQLite may retain freed pages, so the database file may not shrink immediately.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Meaningful regression coverage added / updated where needed; obsolete tests/code removed
  • Persisted-state and API changes include required migrations / compatibility validation
  • Relevant performance/efficiency opportunities considered; material claims have evidence
  • Material review findings resolved and relevant checks rerun
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

Persist image and video embeddings with explicit fp16 encoding and a transactional migration.

Keep search and projection computation in float32; document backup and rollback expectations.
Compute expected score from independently rounded and renormalized fp16 vector.
@github-actions github-actions Bot added python PRs that change python files services PRs that change app services python-tests PRs that change python tests docs PRs that change docs labels Oct 7, 2026
@JPPhoto JPPhoto added the 7.0.0 label Oct 7, 2026
Decode each encoding group in one buffer with chunked float64 renormalization instead of per row.
Drop the migration's own BEGIN; the migrator already runs it in a transaction.
@joshistoast
joshistoast merged commit 0830f0d into invoke-ai:main Oct 8, 2026
14 checks passed
@JPPhoto
JPPhoto deleted the fp16-embedding-storage branch October 8, 2026 03:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

7.0.0 docs PRs that change docs python PRs that change python files python-tests PRs that change python tests services PRs that change app services

Projects

Status: 7.0 Theme: Modular Design

Development

Successfully merging this pull request may close these issues.

4 participants