Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@
<PackageVersion Include="Microsoft.AspNetCore.TestHost" Version="10.0.14" />
<PackageVersion Include="Microsoft.Data.SqlClient" Version="7.1.1" />
<PackageVersion Include="Microsoft.Data.Sqlite" Version="10.0.14" />
<PackageVersion Include="Microsoft.Data.Sqlite.Core" Version="10.0.14" />
<PackageVersion Include="Microsoft.EntityFrameworkCore" Version="10.0.14" />
<PackageVersion Include="Microsoft.EntityFrameworkCore.Relational" Version="10.0.14" />
<PackageVersion Include="Microsoft.EntityFrameworkCore.Sqlite" Version="10.0.14" />
Expand Down
41 changes: 41 additions & 0 deletions docs/CHANGELOG.md

Large diffs are not rendered by default.

105 changes: 104 additions & 1 deletion docs/sharpcoredb-known-issues.md
Original file line number Diff line number Diff line change
Expand Up @@ -421,6 +421,105 @@ attribution before merge.

---

## 8. ✅ FIXED (directory storage) — An UPDATE that moves a PRIMARY KEY onto an existing key lost a row

> ✅ **FIXED (2026-11-17, `fix/pk-update-and-unique-constraints`) on directory storage.** Every UPDATE path —
> `Table.UpdateAffectedCount` (SQL UPDATE), `UpdateMultipleCore` (`ExecuteBatchSQL` and the structured batch API) and
> `UpdateByPrimaryKey` — now validates a new key before any row is written: a key another row holds, or a SET of the key
> that matches more than one row, is rejected and nothing changes. Pinned by `PrimaryKeyUpdateTests` (columnar and
> page-based, in the session and after a reopen). The single-file half — a duplicated key, not a lost row — is the same
> composite-key question as §11 and is tracked there.

Found 2026-11-17. Neither storage mode checks the new key of an UPDATE that changes the primary key:

```sql
CREATE TABLE t (id INTEGER PRIMARY KEY, email TEXT);
INSERT INTO t (id, email) VALUES (1, 'a@x');
INSERT INTO t (id, email) VALUES (3, 'c@x');
UPDATE t SET id = 1 WHERE id = 3; -- SQLite: UNIQUE constraint failed: t.id
SELECT id FROM t;
```

- **Directory storage:** the UPDATE is accepted and the original row `1` is **lost** (silent data loss).
- **Single-file storage:** the UPDATE is accepted and two rows carry id `1`.

**Why it is open:** the directory-mode fix belongs in `Table.Update`, the write path the performance gate in
`docs/performance/AUTONOMOUS_AGENT_BRIEF.md` guards; it needs its own change with a gate run.

---

## 9. ✅ FIXED (directory storage) — A column-level `UNIQUE` constraint was not enforced

> ✅ **FIXED (2026-11-17, `fix/pk-update-and-unique-constraints`) on directory storage.** A single-column UNIQUE
> (column-level or table-level `UNIQUE (col)`) gets an automatic unique hash index, as SQLite's `sqlite_autoindex_…` —
> registered at CREATE TABLE and on every open, so existing tables are covered. INSERT, multi-row INSERT (now atomic on
> tables with a unique index) and every UPDATE path are checked; NULLs never conflict. On the way: a UNIQUE *index* was
> enforced only by a single-row INSERT into a columnar table, and a second NULL was rejected as a duplicate. Pinned by
> `UniqueConstraintTests` (columnar and page-based). **Still open:** single-file storage (its persisted
> `ColumnFlags.Unique` is also written for every inline PRIMARY KEY column, so a reopened file cannot tell a declared
> UNIQUE from an EF Core composite key's column — the format decision of §10/§11; tried and backed out when
> `CompleteExampleIntegrationTests.CompositeKey_*` failed) and a composite `UNIQUE (a, b)` on INSERT.

Found 2026-11-17. A column declared `UNIQUE` accepts duplicates on both storage modes:

```sql
CREATE TABLE t (id INTEGER PRIMARY KEY, email TEXT UNIQUE);
INSERT INTO t (id, email) VALUES (1, 'a@x');
INSERT INTO t (id, email) VALUES (2, 'a@x'); -- SQLite: UNIQUE constraint failed: t.email; SharpCoreDB: accepted
```

A UNIQUE *index* is enforced on directory storage; the column constraint is parsed and, on single-file storage,
persisted as `ColumnFlags.Unique`, but no write path checks it. **Why it is open:** same write-path gate as §8.

---

## 10. 🟡 OPEN (owner decision) — Single-file storage ignores a column's declared `COLLATE`

> ⚠️ **STILL TRUE (verified 2026-11-17), pending an owner decision** on the column-entry format extension below.

Found 2026-11-17. On single-file storage `CREATE TABLE t (name TEXT COLLATE NOCASE)` behaves as BINARY: `WHERE
name = 'X'` does not match `'x'`, and `ORDER BY name` sorts case-sensitively. Directory storage honours it.

Three gaps make it up: the single-file `CREATE TABLE` parser (`DatabaseExtensions`) does not read `COLLATE`; the
single-file column entry (`ColumnDefinitionEntry`, 48 bytes) has no collation field; and the single-file WHERE
evaluator (`SingleFileTable.EvaluateSingleCondition`) is static and has no collation support.

**Decision needed:** persisting the collation means claiming spare bits of `ColumnDefinitionEntry.Flags` (bits 8–11
would hold `CollationType`; a file written before reads as 0 = BINARY, today's behaviour). That is compatible with
`docs/manual/upgrade-and-downgrade.md` — old files keep working, downgrade is already unsupported — but it is an
on-disk format extension. A `Locale(…)` collation would still need a name field. Pinned by the skipped
`SingleTableOrderByTests.OrderBy_ColumnDeclaredNoCase_OnSingleFile_SortsCaseInsensitively`.

---

## 11. 🔴 OPEN (owner decision) — PRIMARY KEY uniqueness and composite keys

> ⚠️ **STILL TRUE (verified 2026-11-17), pending an owner decision** on modelling composite keys. Single-file
> enforcement was implemented and backed out in the same change because of the compatibility hazard below.

Found 2026-11-17 while fixing single-file constraint enforcement.

- **Single-file storage does not enforce PRIMARY KEY uniqueness.** `INSERT … VALUES (2, 'y'), (1, 'dup')` into a
table already holding id 1 leaves two rows with id 1, and an UPDATE that moves a key onto an existing one (§8's
single-file half) duplicates it.
- **Neither mode models a composite key.** EF Core's `EnsureCreated` (`SharpCoreDBDatabaseCreator`) declares one
inline `PRIMARY KEY` per key column; both storage modes keep only the *last* such column as `PrimaryKeyIndex`.
Directory storage then enforces uniqueness on that one column — a false violation as soon as two rows share it
(order lines numbered 1, 2, … per order) — and single-file storage enforces nothing. SQLite rejects the
declaration outright ("table has more than one primary key").

**Why single-file enforcement was not simply switched on:** its format persists `ColumnFlags.PrimaryKey` on that one
column only, so a file cannot tell a single-column key from a composite key that lost its other columns. Enforcing
single-column uniqueness would make existing EF Core single-file databases with composite keys reject legitimate
inserts after an upgrade — exactly what `CompleteExampleIntegrationTests.CompositeKey_*` showed when it was tried.

**Decision needed:** model composite keys (persist the `PrimaryKey` flag on every key column and enforce uniqueness
on the tuple — for files written before, there is no way to recover the lost key columns, so they would stay
unenforced), and only then enforce single-column uniqueness on single-file storage. Pinned by the skipped
single-file cases in `PrimaryKeyUniquenessTests`.

---

## Priority summary

| # | Issue | Severity | Status in ServiceMq.SharpCoreDb | In SharpCoreDB 2.1.0 |
Expand All @@ -431,4 +530,8 @@ attribution before merge.
| 4 | No read-after-write flush | 🟡 Low | Flush calls in provider | **Fixed** — `ExecuteQuery` flushes pending batch writes |
| 5 | SQL validator `@` prefix | 🟡 Low | Keys without `@` prefix | **Fixed** — `NormalizeKey` strips `@` / `:` |
| 6 | INTEGER → Int32 | 🟡 Low | `BIGINT` columns | **Still the default** — opt in with `UseSqliteIntegerAffinity` |
| 7 | Bound text re-parsed as SQL | 🔴 High | Write through `ITable.Insert` instead of SQL text | **Fixed** — one `SqlLiteralScanner` (SQL/SQLite: a doubled quote is one quote, a backslash is data) behind every scanner; `ParseValue` strips one pair and un-collapses doubling; a tuple with too many values is rejected. `ParameterLiteralRoundTripTests` 74/74 |
| 7 | Bound text re-parsed as SQL | 🔴 High | Write through `ITable.Insert` instead of SQL text | **Fixed** — one `SqlLiteralScanner` (SQL/SQLite: a doubled quote is one quote, a backslash is data) behind every scanner; `ParseValue` strips one pair and un-collapses doubling; a tuple with too many values is rejected. `ParameterLiteralRoundTripTests` 74/74 |
| 8 | PK-changing UPDATE loses/duplicates a row | 🔴 High | — | **Fixed on directory storage** — every UPDATE path validates the new key first; single-file part tracked in §11 |
| 9 | Column `UNIQUE` not enforced | 🟠 Medium | — | **Fixed on directory storage** — automatic unique index; single-file (format decision, §10/§11) and composite `UNIQUE (a, b)` still open |
| 10 | Single-file ignores `COLLATE` | 🟡 Low | — | **Open, owner decision** — needs a column-entry format extension (§10) |
| 11 | PK uniqueness / composite keys | 🔴 High | — | **Open, owner decision** — single-file enforces no PK uniqueness; composite keys are modelled as their last column (§11) |
50 changes: 36 additions & 14 deletions src/SharpCoreDB.Functional.Linq2DB/Extensions.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
using LinqToDB;
using LinqToDB.Data;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;

namespace SharpCoreDB.Functional.Linq2DB;

Expand All @@ -21,12 +21,13 @@ public static DataOptions UseSharpCoreDB(this DataOptions options, string connec
ArgumentNullException.ThrowIfNull(options);
ArgumentException.ThrowIfNullOrWhiteSpace(connectionString);

var (provider, connection) = SharpCoreDBDataConnection.CreateProviderAndConnection(connectionString);
return SharpCoreDBDataConnection.Bind(options, provider, connection);
return SharpCoreDBDataConnection.Bind(options, connectionString);
}

/// <summary>
/// Adds SharpCoreDB linq2db DataConnection (via DataOptions) to the service collection.
/// Adds a SharpCoreDB linq2db <see cref="SharpCoreDBDataConnection"/> and a <see cref="FunctionalLinq2DbContext"/>
/// over it to the service collection, both with <paramref name="lifetime"/>. Each resolved connection opens its own
/// SharpCoreDB connection. A <see cref="FunctionalLinq2DbContext"/> registration the caller already made is kept.
/// </summary>
public static IServiceCollection AddSharpCoreDBLinq2Db(
this IServiceCollection services,
Expand All @@ -39,33 +40,54 @@ public static IServiceCollection AddSharpCoreDBLinq2Db(
var options = new DataOptions()
.UseSharpCoreDB(connectionString);

services.Add(new ServiceDescriptor(
typeof(SharpCoreDBDataConnection),
_ => new SharpCoreDBDataConnection(options),
lifetime));

return services;
return Register(services, options, lifetime);
}

/// <summary>
/// Adds SharpCoreDB DataConnection with custom DataOptions configuration.
/// Adds SharpCoreDB DataConnection with custom DataOptions configuration, for example
/// <c>options =&gt; options.UseSharpCoreDB(connectionString).UseAdditionalMappingSchema(mySchema)</c>.
/// <para>
/// <see cref="DataOptions"/> is immutable — every <c>Use…</c> call returns a new instance — so the callback
/// returns the configured options. Up to 2.1.4 this overload took an <c>Action&lt;DataOptions&gt;</c>, discarded
/// what the callback built, and resolving the connection threw "Configuration string is not provided".
/// </para>
/// </summary>
/// <param name="services">The service collection.</param>
/// <param name="configureOptions">Builds the options from an empty <see cref="DataOptions"/>.</param>
/// <param name="lifetime">Lifetime of the registered connection.</param>
/// <returns>The service collection.</returns>
public static IServiceCollection AddSharpCoreDBLinq2Db(
this IServiceCollection services,
Action<DataOptions> configureOptions,
Func<DataOptions, DataOptions> configureOptions,
ServiceLifetime lifetime = ServiceLifetime.Scoped)
{
ArgumentNullException.ThrowIfNull(services);
ArgumentNullException.ThrowIfNull(configureOptions);

var options = new DataOptions();
configureOptions(options);
var options = configureOptions(new DataOptions())
?? throw new InvalidOperationException(
"The configureOptions callback returned null; return the configured DataOptions.");

return Register(services, options, lifetime);
}

/// <summary>
/// Registers the connection and the functional context over it. Up to 2.1.4 only the connection was registered,
/// so every caller wired <see cref="FunctionalLinq2DbContext"/> by hand — often as a singleton over one shared
/// connection.
/// </summary>
private static IServiceCollection Register(IServiceCollection services, DataOptions options, ServiceLifetime lifetime)
{
services.Add(new ServiceDescriptor(
typeof(SharpCoreDBDataConnection),
_ => new SharpCoreDBDataConnection(options),
lifetime));

services.TryAdd(new ServiceDescriptor(
typeof(FunctionalLinq2DbContext),
sp => new FunctionalLinq2DbContext(sp.GetRequiredService<SharpCoreDBDataConnection>()),
lifetime));

return services;
}

Expand Down
Loading
Loading