Add Lyng DB contract and SQLite provider skeleton

notes/db/sqlite_implementation_plan.md

@@ -0,0 +1,261 @@
+# SQLite provider implementation plan
+
+Implementation checklist for the first concrete DB provider:
+- module: `lyng.io.db.sqlite`
+- targets: JVM and Native
+- role: reference implementation for the core `lyng.io.db` contract
+
+## Scope
+
+In scope for the first implementation:
+- provider registration on module import
+- `sqlite:` URL dispatch through `openDatabase(...)`
+- typed `openSqlite(...)` helper
+- `Database` and `SqlTransaction` implementation
+- result-set implementation
+- row and column metadata conversion
+- SQLite savepoint-based nested transactions
+- generated keys for `execute(...)`
+- JVM and Native test coverage for the core portable contract
+
+Out of scope for the first implementation:
+- schema inspection APIs beyond result-set column metadata
+- public prepared statement API
+- batch API
+- JS support
+- heuristic temporal parsing
+- heuristic decimal parsing
+
+## Proposed module layout
+
+Core/public declarations:
+- `.lyng` declaration source for `lyng.io.db`
+- `.lyng` declaration source for `lyng.io.db.sqlite`
+
+Backend/runtime implementation:
+- common transaction/result-set abstractions in `commonMain` where possible
+- JVM SQLite implementation in `jvmMain`
+- Native SQLite implementation in `nativeMain`
+
+## Milestone 1: core DB module skeleton
+
+1. Move or copy the current DB declarations from `notes/db/lyngdb.lyng` into the
+   real module declaration source location.
+2. Add the provider registry runtime for:
+   - normalized lowercase scheme lookup
+   - duplicate registration failure
+   - unknown-scheme failure
+3. Implement generic `openDatabase(connectionUrl, extraParams)` dispatch.
+4. Add basic tests for:
+   - successful provider registration
+   - duplicate scheme registration failure
+   - unknown scheme failure
+   - malformed URL failure
+
+## Milestone 2: SQLite provider skeleton
+
+1. Create `lyng.io.db.sqlite` declaration source with:
+   - typed `openSqlite(...)`
+   - provider-specific documentation
+2. Register `sqlite` scheme at module initialization time.
+3. Parse supported URL forms:
+   - `sqlite::memory:`
+   - `sqlite:relative/path.db`
+   - `sqlite:/absolute/path.db`
+4. Convert typed helper arguments into the same normalized open options used by
+   the generic `openDatabase(...)` path.
+5. Add tests for:
+   - import-time registration
+   - typed helper open
+   - generic URL-based open
+   - invalid URL handling
+
+## Milestone 3: JVM backend
+
+Implementation strategy:
+- use SQLite JDBC under the hood
+- keep JDBC fully internal to the provider
+- preserve the Lyng-facing transaction/result contracts rather than exposing
+  JDBC semantics directly
+
+Steps:
+1. Create JVM `Database` implementation that stores normalized SQLite config.
+2. For each outer `Database.transaction {}`:
+   - open/acquire one JDBC connection
+   - configure connection-level options
+   - begin transaction
+   - commit on normal exit
+   - rollback on uncaught exception
+   - close/release connection
+   - preserve error precedence:
+     - original user exception stays primary on rollback failure
+     - rollback failure becomes primary for intentional `RollbackException`
+     - commit failure is primary on normal-exit commit failure
+3. For nested `SqlTransaction.transaction {}`:
+   - create savepoint
+   - release savepoint on success
+   - rollback to savepoint on failure
+   - preserve the same primary/secondary exception rules for savepoint rollback
+     failures
+4. Implement `select(...)`:
+   - bind positional `?` parameters
+   - expose result rows through `ResultSet`
+   - preserve transaction-scoped lifetime
+   - invalidate both the result set and any rows obtained from it when the
+     owning transaction ends
+5. Implement `execute(...)`:
+   - bind positional parameters
+   - collect affected row count
+   - expose generated keys when supported
+6. Implement column metadata normalization:
+   - output column label
+   - nullable flag where available
+   - portable `SqlType`
+   - backend native type name
+7. Add JVM tests for:
+   - transaction commit
+   - transaction rollback
+   - nested transaction savepoint behavior
+   - rollback failure precedence
+   - commit failure precedence
+   - row lookup by index and name
+   - ambiguous name failure
+   - result-set use-after-transaction failure
+   - row use-after-transaction failure
+   - generated keys
+   - `RETURNING` via `select(...)` if the backend supports it
+
+## Milestone 4: Native backend
+
+Implementation strategy:
+- use direct SQLite C bindings
+- keep semantics aligned with the JVM backend
+
+Steps:
+1. Create Native `Database` implementation that stores normalized SQLite config.
+2. For each outer transaction:
+   - open one SQLite handle if needed
+   - configure pragmas/options
+   - begin transaction
+   - commit or rollback
+3. Implement nested transactions with SQLite savepoints.
+4. Implement prepared statement lifecycle internally for `select(...)` and
+   `execute(...)`.
+5. Implement result-set iteration and statement finalization.
+6. Implement the same value-conversion policy as JVM:
+   - exact integer mapping
+   - `Double`
+   - `String`
+   - `Buffer`
+   - schema-driven `Bool`
+   - schema-driven `Decimal`
+   - schema-driven temporal parsing
+7. Add Native tests matching the JVM behavioral suite as closely as possible.
+
+## Milestone 5: conversion policy
+
+1. Normalize integer reads:
+   - return `Int`
+2. Normalize floating-point reads to `Double`.
+3. Normalize text reads to `String`.
+4. Normalize blob reads to `Buffer`.
+5. Parse `Decimal` only for declared/native `DECIMAL` / `NUMERIC` columns.
+   - bind Lyng `Decimal` as canonical text using existing Decimal formatting
+   - read back with the existing Decimal parser
+6. Parse `Bool` only for declared/native `BOOLEAN` / `BOOL` columns:
+   - prefer integer `0` / `1`
+   - also accept legacy text `true`, `false`, `t`, `f` case-insensitively
+   - always write `Bool` as integer `0` / `1`
+7. Parse temporal values only from an explicit normalized type-name whitelist:
+   - `DATE`
+   - `DATETIME`
+   - `TIMESTAMP`
+   - `TIMESTAMP WITH TIME ZONE`
+   - `TIMESTAMPTZ`
+   - `DATETIME WITH TIME ZONE`
+   - `TIME`
+   - `TIME WITHOUT TIME ZONE`
+   - `TIME WITH TIME ZONE`
+8. Never heuristically parse arbitrary string or numeric values into temporal or
+   decimal values.
+9. If a declared strong type (`Bool`, `Decimal`, `Date`, `DateTime`, `Instant`)
+   cannot be converted from the stored value, fail with `SqlExecutionException`.
+10. Add tests for each conversion rule.
+
+## Milestone 6: result-set contract
+
+1. Ensure `ResultSet.columns` is available before iteration.
+2. Implement name lookup using result-column labels.
+3. Throw `SqlUsageException` for:
+   - invalid row index
+   - missing column name
+   - ambiguous column name
+   - use after transaction end
+4. Implement cheap `isEmpty()` where practical.
+5. Implement `size()` separately, allowing buffering/consumption when required.
+6. Verify resource cleanup on:
+   - full iteration
+   - canceled iteration
+   - transaction rollback
+
+## Milestone 7: provider options
+
+Support these typed helper options first:
+- `readOnly`
+- `createIfMissing`
+- `foreignKeys`
+- `busyTimeoutMillis`
+
+Expected behavior:
+- `foreignKeys` defaults to `true`
+- `busyTimeoutMillis` has a non-zero sensible default
+- `readOnly` is explicit
+- `createIfMissing` is explicit
+
+Add tests for option handling where backend behavior is observable.
+
+## Milestone 8: documentation and examples
+
+1. Add user docs for:
+   - importing `lyng.io.db.sqlite`
+   - generic `openDatabase(...)`
+   - typed `openSqlite(...)`
+   - transaction usage
+   - nested transactions
+2. Add small sample snippets for:
+   - in-memory DB
+   - file-backed DB
+   - schema creation
+   - insert/select/update
+   - rollback on exception
+
+## Testing priorities
+
+Highest priority behavioral tests:
+1. provider registration and scheme dispatch
+2. outer transaction commit/rollback
+3. nested savepoint semantics
+4. row metadata and name lookup behavior
+5. generated keys for inserts
+6. result-set lifetime rules
+7. value conversion rules
+8. helper vs generic-open parity
+
+## Risks
+
+Main implementation risks:
+- keeping JVM and Native value-conversion behavior identical
+- correctly enforcing result-set lifetime across both backends
+- SQLite temporal conversion ambiguity
+- JDBC metadata differences on JVM
+- native statement/finalizer lifecycle bugs
+
+## Suggested implementation order
+
+1. core registry runtime
+2. SQLite `.lyng` provider declarations
+3. JVM SQLite backend
+4. JVM test suite
+5. Native SQLite backend
+6. shared behavioral test suite refinement
+7. documentation/examples