- 
+ 
+ 
+ 
@@ -19,6 +21,12 @@
+
+
+
@@ -19,6 +21,12 @@
Build with Firefly agents, tools, memory and typed model options — gain lifecycle hooks, delegation, reasoning patterns, validation loops, RAG, and DAG pipelines, all protocol-driven and swappable.
@@ -416,7 +424,9 @@ The diagram distinguishes structural protocols from inheritance-based extension `parse_scope_namespace` helpers) namespaces documents per tenant or scope. Auto-embedding upserts documents without pre-computed vectors. `search_text` embeds a query string and searches in one call, and `SearchFilter` narrows results - by metadata. Namespace scoping isolates document collections. `VectorStoreRegistry` + by metadata. PostgreSQL document IDs remain unique across the table; a conflicting + namespace rejects the whole upsert batch. Applications bind scopes to authenticated + callers and manage database authorization. `VectorStoreRegistry` manages named instances. `EmbeddingStep` and `RetrievalStep` integrate directly into DAG pipelines for retrieval-augmented workflows. diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index ed1e3ce4..70976429 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -40,6 +40,12 @@ Releases use **YY.MM.Patch**, starting with patch `0` each month. Python normali assets; it does not publish to PyPI. Pin the framework and lock application dependencies for reproducible deployments. +The release also provides [SHA256SUMS](https://github.com/fireflyframework/fireflyframework-agentic/releases/download/v26.10.0/SHA256SUMS) +for the wheel and source archive. Verify downloaded files against those hashes +before installation. Existing PostgreSQL vector-store users should read the +[26.10.0 namespace conflict changes](../migration.md#postgresql-vector-store-safety-in-26100). +The fix requires no database migration or PostgreSQL version upgrade. + ## One-line installer The interactive installer detects your platform, checks Python and uv, lets you diff --git a/docs/migration.md b/docs/migration.md index f1964c4f..f73364e9 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -1,8 +1,42 @@ # Migration Guide -This guide covers the Pydantic AI 2.x upgrade and the earlier tool and Dynamic -Workflow changes. It distinguishes dependency changes from changes to Firefly's -public API and model routing. +Copyright 2026 Firefly Software Foundation. SPDX-License-Identifier: Apache-2.0. + +This guide covers PostgreSQL vector-store safety in 26.10.0, the Pydantic AI 2.x +upgrade, and earlier tool and Dynamic Workflow changes. + +## PostgreSQL vector-store safety in 26.10.0 + +Previously, upserting an existing document ID into another namespace could +overwrite and move the original PostgreSQL row. Version 26.10.0 raises +`VectorStoreError` for that conflict and rolls back the whole batch, including +earlier inserts or updates. The check also applies to concurrent writes and +`TenantScopedVectorStore`. + +The table still uses a global `PRIMARY KEY (id)`. Same-namespace updates and +repeated IDs within one same-namespace batch keep their existing behavior. No +database migration, index rebuild, or re-embedding is required for this fix. + +Before upgrading a shared-table deployment, check how the application assigns +document IDs. IDs must be unique across namespaces. New documents in different +scopes need distinct IDs; do not bulk-rename existing IDs without accounting for +their references. If the application used cross-namespace upsert to move a +document, replace that behavior with an explicitly authorized transfer workflow. +Retrying the same conflicting batch will not succeed. + +Install the [26.10.0 release](getting-started/installation.md#install-a-release), +then verify same-namespace updates, rejection of a foreign-namespace ID, and +whole-batch rollback in an isolated application test database. This fix does not +restore content overwritten by earlier versions; any suspected historical loss +requires review of application audit records or backups. + +The wrapper requires scope arguments but does not authenticate them or install +RLS policies. Keep authorization and transaction-local database scope setup in +the application. See [vector-store scoping](vectorstores.md#tenantworkspace-scoping) +and [database authorization](security.md#vector-store-authorization). + +Pgvector/HNSW remains unchanged. PostgreSQL 18 and pg_vectorsearch are not +requirements of this release; [PRISM remains outside the supported backend](pg-vectorsearch-evaluation.md). ## Pydantic AI 2 and OpenAI API selection diff --git a/docs/pg-vectorsearch-evaluation.md b/docs/pg-vectorsearch-evaluation.md new file mode 100644 index 00000000..4ec589e7 --- /dev/null +++ b/docs/pg-vectorsearch-evaluation.md @@ -0,0 +1,103 @@ +# pg_vectorsearch adoption evaluation + +Copyright 2026 Firefly Software Foundation. SPDX-License-Identifier: Apache-2.0. + +The October 14, 2026 evaluation did not qualify pg_vectorsearch for adoption in +Agentic, flycanon, or flyquery. Pgvector/HNSW remains the supported PostgreSQL +backend. Agentic 26.10.0 fixes namespace writes without enabling PRISM or requiring +PostgreSQL 18. No PRISM configuration option, automatic extension installation, +index replacement, or schema migration was added. + +## Release and compatibility + +The evaluated [pg_vectorsearch v0.1.0 release](https://github.com/timescale/pg_vectorsearch/releases/tag/v0.1.0) +is commit `36b75d1cb56c12a380d600ef27217d88c9f419ec`. Its +[changelog](https://github.com/timescale/pg_vectorsearch/blob/36b75d1cb56c12a380d600ef27217d88c9f419ec/CHANGELOG.md) +describes its quality as alpha despite the final release label. It requires +PostgreSQL 18 or later and uses the PostgreSQL License. It can index existing +pgvector columns with its own operator classes, but the native probes rejected +expression indexes, unlogged tables, and `halfvec(3072)` indexes. The reported +maximum was 1,968 dimensions. + +| Product | Existing storage contract | Decision | +| --- | --- | --- | +| Agentic | Fixed-width pgvector column, HNSW bootstrap, namespace and metadata filters | Storage is compatible at supported widths; filtered recall and index lifecycle need further qualification. | +| flycanon | Mixed widths, per-set partial expression indexes, wide halfvec embeddings | Expression indexes and widths above 1,968 are incompatible. A materialized projection would require a separate migration and synchronization design. | +| flyquery | Plain `vector(1536)` columns, dataset joins and RLS; computed ranking for examples | Column shape fits, but filtered retrieval and actual query plans are unqualified. An index change alone cannot convert computed ranking into ANN traversal. | + +The audited source revisions were Agentic `8e56361`, flycanon `2d7dca1`, and +flyquery `be68d344`. The inspected flycanon/flyquery development and integration +stacks used PG16. Their production database upgrades were outside this evaluation. + +## Measured results + +The isolated runtime used PostgreSQL 18.6, pgvector 0.8.7, and pg_vectorsearch +0.1.0 on macOS arm64. PRISM was compiled in release mode with native NEON and +Apple Accelerate. PostgreSQL used 128 MiB shared buffers, 64 MiB work memory, +256 MiB maintenance memory, and no parallel query or maintenance workers for +the measured operations. + +Two synthetic corpora used seed `20261014`: 30,000 vectors at width 384 and +12,000 at width 1,536. Normalized vectors formed 64 clusters across 16 tenants and +four projects. Exact cosine top-10 results came from an unindexed copy of each +table. Each cohort contained 48 distinct queries repeated three times. Opposite +tenant/project scopes deliberately selected rows far from the query's nearest +global clusters; their failure rates are not estimates of production traffic. + +| Corpus | Setting | Global recall@10 | Global p50 latency | Incomplete opposite tenant/project queries | +| --- | --- | ---: | ---: | ---: | +| 30k × 384 | HNSW ef 40 | 99.17% | 0.347 ms | 144/144 | +| 30k × 384 | HNSW strict iterative ef 160 | 100% | 0.395 ms | 0/144 | +| 30k × 384 | PRISM auto | 93.75% | 0.168 ms | 111/144 | +| 30k × 384 | PRISM nprobe 256 | 94.79% | 0.660 ms | 102/144 | +| 12k × 1536 | HNSW ef 40 | 99.79% | 0.252 ms | 144/144 | +| 12k × 1536 | HNSW strict iterative ef 160 | 100% | 0.426 ms | 0/144 | +| 12k × 1536 | PRISM auto | 87.50% | 0.372 ms | 114/144 | +| 12k × 1536 | PRISM nprobe 256 | 95.42% | 1.669 ms | 48/144 | + +Timings forced each ANN index and include the local client round trip. In the +1,536-dimensional run, 40 of 48 sampled natural plans chose a sequential scan +and sort instead. These timings do not describe default application performance. +The lower PRISM latency at width 384 also has lower recall; no global speedup at +matched recall was established. HNSW iterative scans filled the filtered result +slots, but opposite tenant/project recall was only 38.33% and 58.75% in the two +loaded corpora. Result count alone does not establish retrieval quality. + +PRISM index sizes were 5.64 MB versus HNSW's 61.45 MB at width 384, and 5.19 MB +versus 98.31 MB at width 1,536 (decimal MB). Single build times were 0.663 versus +3.971 seconds and 0.518 versus 3.564 seconds. Full vectors remain in the heap. +Build order and background system activity were not controlled across repeated +independent builds. Post-query RSS includes shared mappings and does not establish +peak private memory savings. + +Each corpus then received 5% inserts, 5% vector updates, and 5% deletes, followed +by VACUUM ANALYZE. Under simplified FORCE RLS policies and a restricted reader, +PRISM auto underfilled 37/48 and 38/48 opposite tenant/project queries, with +16.67% and 13.96% recall. No wrong-scope rows were observed. The test did not +reproduce actual service middleware, pooling, grants, or dataset joins. + +One immediate shutdown and WAL recovery per corpus preserved committed ID/vector +fingerprints and six sampled query vectors across four cohorts per configuration. +Uncommitted inserts, deletes, and embedding updates rolled back. This does not +qualify backup/restore, failover, extension upgrades, or interrupted maintenance. +Independent review checked dataset parity, unique churn vectors, recorded plans, +scope results, and recovery comparisons. + +## Conditions for a future pilot + +Reconsider PRISM only after it preserves supported schemas, embedding widths, +IDs, scores, authorization, and source/citation contracts. Installation and index +construction must be operator-owned; PostgreSQL can choose either index when two +exist, so a configuration flag alone cannot guarantee routing. + +A future pilot should achieve at least 95% mean recall@10 in every representative +filtered cohort, return `min(10, eligible rows)` on every query, and expose zero +rows outside the authorized scope. Compare latency at recall within one percentage +point. Require either 20% lower p95 latency or 30% lower measured peak private +memory without more than 20% p95 latency regression, using representative scale +and concurrency. These are proposed acceptance thresholds, not achieved results. + +Qualify concurrent changes, pooled scope reuse, maintenance, recovery, upgrades, +and reversal separately. Keep existing indexes and data intact during evaluation. +For the supported backend's current settings and namespace behavior, see +[vector stores](vectorstores.md) and the [26.10.0 upgrade notes](migration.md#postgresql-vector-store-safety-in-26100). diff --git a/docs/security.md b/docs/security.md index 78aced04..c02df752 100644 --- a/docs/security.md +++ b/docs/security.md @@ -12,6 +12,25 @@ patterns. Blocking or sanitization policy is selected by the application. --- +## Vector-store authorization + +`TenantScopedVectorStore` requires tenant and workspace arguments and maps them +to a namespace. Applications must derive or validate those arguments against the +authenticated caller. Metadata stamps and namespace strings do not grant access. + +Since 26.10.0, `PgVectorVectorStore` rejects an ID conflict with another namespace +and rolls back the entire upsert batch. IDs remain globally unique in its table. +This closes the cross-namespace overwrite path; it does not install PostgreSQL +roles or RLS policies, protect access through other database clients, or recover +previously overwritten records. + +For shared databases, manage privileges and RLS through deployment migrations. +The `_prepare_session` hook runs inside each operation's transaction and can +apply transaction-local scope settings. Test using the actual application role, +including pooled connection reuse and missing or foreign scope. See the +[upgrade notes](migration.md#postgresql-vector-store-safety-in-26100) and +[vector-store guide](vectorstores.md#tenantworkspace-scoping). + ## Architecture This shows an explicitly configured input/output scanning path. Add diff --git a/docs/vectorstores.md b/docs/vectorstores.md index 2add2b8d..a6e05ca8 100644 --- a/docs/vectorstores.md +++ b/docs/vectorstores.md @@ -164,6 +164,17 @@ must bind scopes to authenticated callers and manage database permissions. Install: `pip install fireflyframework-agentic[vectorstores-pgvector]` +For an existing application, follow the [26.10.0 upgrade notes](migration.md#postgresql-vector-store-safety-in-26100) +and [pinned release installation](getting-started/installation.md#install-a-release). +`create_schema=False` leaves schema creation to migrations and checks that the +`vector` extension and table exist. Operators must separately verify the table's +dimensions, indexes, permissions, and any RLS policies. + +There is no supported PRISM selector or `prism.nprobe` setting in this adapter. +The [pg_vectorsearch evaluation](pg-vectorsearch-evaluation.md) records the +compatibility and filtered-recall reasons for retaining pgvector/HNSW. Adding a +second index manually does not guarantee which index PostgreSQL will choose. + ### sqlite-vec A sqlite-vec (`vec0` virtual table) backend that co-resides inside an existing @@ -243,17 +254,22 @@ Note: Filter operator support varies by backend. `eq` is universally supported. ## Namespaces -All operations accept a `namespace` parameter to isolate documents: +All operations accept a `namespace` parameter to scope documents. Use distinct +IDs when writing to different namespaces in a shared PostgreSQL table: ```python -await store.upsert(docs, namespace="project-a") -await store.upsert(docs, namespace="project-b") +from fireflyframework_agentic.vectorstores import VectorDocument + +project_a_docs = [VectorDocument(id="project-a:1", text="Project A notes")] +project_b_docs = [VectorDocument(id="project-b:1", text="Project B notes")] +await store.upsert(project_a_docs, namespace="project-a") +await store.upsert(project_b_docs, namespace="project-b") # Search only within a namespace results = await store.search_text("query", namespace="project-a") # Delete only within a namespace -await store.delete(["id-1"], namespace="project-a") +await store.delete(["project-a:1"], namespace="project-a") ``` --- @@ -305,8 +321,9 @@ every read and write must be confined to a `(tenant_id, workspace_id)` scope, an forgetting the scope must fail loudly rather than silently leak across tenants. `TenantScopedVectorStore` wraps **any** `VectorStoreProtocol` backend and folds the -scope into the canonical `"t/