Add Lyng DB contract and SQLite provider skeleton
This commit is contained in:
1 parent
b42ceec686
commit
55ba6113e7
21 files changed
+3860
No files matched your search
@@ -0,0 +1,94 @@
|
||||
# Lyng.io.db
|
||||
|
||||
Core level interface to SQL implementations. Pooling, physical connections, and implementation-specific configuration are hidden behind the opened database handle.
|
||||
|
||||
All providers should support the generic `openDatabase(connectionUrl, extraParams)`
|
||||
entry point for configuration-driven usage. Providers may also expose typed
|
||||
helpers such as `openSqlite(...)` or `openPostgres(...)`.
|
||||
|
||||
Provider modules should register their URL schemes when first imported. The
|
||||
generic `openDatabase(...)` then dispatches by normalized URL scheme to the
|
||||
registered provider.
|
||||
|
||||
See [db definitions](lyngdb.lyng).
|
||||
|
||||
## Platforms support:
|
||||
|
||||
| DB | JVM | Native |
|
||||
|----------|-----|--------|
|
||||
| Postgres | + | ? |
|
||||
| H2 | + | + |
|
||||
| SQLITE | + | + |
|
||||
|
||||
|
||||
Question for the future: what to do on JS platforms? Browsers have their crazy own storages, bit it is not SQL. Probably for this case we need some simpler standard compatible with browsers and with special implementation on JVM.
|
||||
|
||||
## Proposed type mapping:
|
||||
|
||||
| SQL | Lyng | comments |
|
||||
|--------------------------------------------|----------|----------|
|
||||
| SMALLINT | Int | Lyng `Int` is already 64-bit |
|
||||
| INTEGER / INT | Int | |
|
||||
| BIGINT | Int | Lyng `Int` is already 64-bit |
|
||||
| REAL / FLOAT / DOUBLE PRECISION | Double | |
|
||||
| DECIMAL / NUMERIC | Decimal | exact numeric |
|
||||
| BOOLEAN / BOOL | Bool | |
|
||||
| CHAR / VARCHAR / TEXT | String | |
|
||||
| BINARY / VARBINARY / BLOB / BYTEA | Buffer | |
|
||||
| DATE | Date | calendar date |
|
||||
| TIME / TIME WITHOUT TIME ZONE | String | v1: no standalone Lyng `Time` type yet |
|
||||
| TIMESTAMP / TIMESTAMP WITHOUT TIME ZONE | DateTime | local calendar-dependent timestamp |
|
||||
| TIME WITH TIME ZONE | String | v1: preserve exact value textually |
|
||||
| TIMESTAMP WITH TIME ZONE | Instant | absolute point in time |
|
||||
|
||||
Notes:
|
||||
- `TIME` mappings in v1 should stay textual. A SQL time-of-day value has no
|
||||
date component, so mapping it to `DateTime` would require inventing one, and
|
||||
mapping it to `Instant` would incorrectly assign absolute-time semantics.
|
||||
- Because of that, `TIME` and `TIME WITH TIME ZONE` should be exposed as
|
||||
`String` in portable code. The original backend type is still available
|
||||
through `SqlColumn.nativeType`.
|
||||
- Name-based `SqlRow` access uses result-column labels and should fail on
|
||||
missing or ambiguous names.
|
||||
- `SqlColumn` should expose both the normalized portable `SqlType` and the
|
||||
original backend-reported type name.
|
||||
- For any non-null cell, the converted row value should match the column's
|
||||
portable `SqlType`. Providers should not advertise `SqlType.Date`,
|
||||
`SqlType.Decimal`, etc. and then return mismatched raw values for those
|
||||
columns.
|
||||
- If a backend-reported value cannot be converted to the advertised Lyng value
|
||||
type for that column, row production should fail with `SqlExecutionException`
|
||||
rather than silently degrading to some other visible type.
|
||||
- `ResultSet` should stay iterable, but also expose `isEmpty()` for cheap
|
||||
emptiness checks where possible and `size()` as a separate operation.
|
||||
- `ResultSet` and all `SqlRow` instances obtained from it are valid only while
|
||||
the owning transaction is active. After transaction end, any further row or
|
||||
result-set access should fail with `SqlUsageException`, even if the provider
|
||||
had buffered data internally.
|
||||
- Portable SQL parameter values should match the row conversion set: `null`,
|
||||
`Bool`, `Int`, `Double`, `Decimal`, `String`, `Buffer`,
|
||||
`Date`, `DateTime`, and `Instant`.
|
||||
- Lyng has a single integer type, `Int`, with 64-bit range. Portable SQL
|
||||
integer values should therefore normalize to `Int` rather than exposing
|
||||
separate `Short` / `Int` / `Long` categories.
|
||||
- Portable SQL placeholder syntax is positional `?` only.
|
||||
- The core exception model should stay small: `DatabaseException`,
|
||||
`SqlExecutionException`, `SqlConstraintException`, and `SqlUsageException`,
|
||||
plus propagated `RollbackException`.
|
||||
- `openDatabase(...)` should use `IllegalArgumentException` for malformed
|
||||
arguments detected before opening, and `DatabaseException` for runtime open
|
||||
failures such as authentication, connectivity, or provider initialization.
|
||||
- Nested `SqlTransaction.transaction {}` must provide real nested transaction
|
||||
semantics, usually via savepoints. If the backend cannot support this, it
|
||||
should throw `SqlUsageException`.
|
||||
- Transaction failure precedence should be:
|
||||
- if user code throws and rollback succeeds, rethrow the original exception
|
||||
- if user code throws and rollback fails, the original exception stays
|
||||
primary and the rollback failure is secondary/suppressed where possible
|
||||
- if `RollbackException` was used intentionally and rollback itself fails,
|
||||
the rollback failure becomes primary
|
||||
- if commit fails after normal block completion, the commit failure is
|
||||
primary
|
||||
- Provider URL schemes should be matched case-insensitively. Duplicate scheme
|
||||
registration should fail. Unknown schemes or missing providers should fail
|
||||
with `DatabaseException`.
|
||||
@@ -0,0 +1,213 @@
|
||||
/*
|
||||
Portable value categories exposed by the Lyng SQL layer and used in
|
||||
[SqlColumn].
|
||||
*/
|
||||
enum SqlType {
|
||||
Binary, String, Int, Double, Decimal,
|
||||
Bool, Instant, Date, DateTime
|
||||
}
|
||||
|
||||
class SqlColumn(
|
||||
val name: String,
|
||||
val sqlType: SqlType,
|
||||
val nullable: Bool,
|
||||
/*
|
||||
Original database type name as reported by the backend, such as
|
||||
VARCHAR, TEXT, INT8, TIMESTAMPTZ, or BYTEA.
|
||||
*/
|
||||
val nativeType: String
|
||||
)
|
||||
|
||||
class SqlRow(
|
||||
/* Number of columns in the row */
|
||||
val size: Int,
|
||||
val values: ImmutableList<Object?>
|
||||
) {
|
||||
/*
|
||||
Return the already converted Lyng value for a column addressed by
|
||||
index or output column label. SQL NULL is returned as null.
|
||||
|
||||
Name lookup uses result-column labels. If several columns share the same
|
||||
label, name-based access is ambiguous and should fail. Missing column
|
||||
names and invalid indexes should also fail.
|
||||
*/
|
||||
abstract override fun getAt(indexOrName: String|Int): Object?
|
||||
}
|
||||
|
||||
/*
|
||||
A result set is valid only while its owning transaction is active.
|
||||
|
||||
Implementations may stream rows or buffer them internally, but:
|
||||
- rows must be exposed through normal iteration
|
||||
- iteration to the end or canceled iteration should close the underlying
|
||||
resources automatically
|
||||
- using the result set after its transaction ends is invalid
|
||||
- rows obtained from the result set are also invalid after the owning
|
||||
transaction ends, even if the implementation had already buffered them
|
||||
|
||||
If user code wants row data to survive independently, it should copy the
|
||||
values it needs into ordinary Lyng objects while the transaction is active.
|
||||
*/
|
||||
interface ResultSet : Iterable<SqlRow> {
|
||||
/*
|
||||
Column metadata for the result rows, in positional order.
|
||||
*/
|
||||
abstract val columns: ImmutableList<SqlColumn>
|
||||
|
||||
/*
|
||||
Number of rows if the implementation can determine it. Implementations
|
||||
may need to consume or buffer the whole result in order to answer.
|
||||
*/
|
||||
abstract override fun size(): Int
|
||||
|
||||
/*
|
||||
Fast emptiness check when the implementation can provide it without
|
||||
consuming the result.
|
||||
*/
|
||||
abstract override fun isEmpty(): Bool
|
||||
}
|
||||
|
||||
abstract class ExecutionResult(
|
||||
val affectedRowsCount: Int
|
||||
) {
|
||||
/*
|
||||
Return implementation-supported auto-generated values produced by
|
||||
[execute]. This is intentionally stricter than arbitrary SQL result
|
||||
sets: statements such as INSERT/UPDATE/DELETE ... RETURNING should be
|
||||
executed with [select], not exposed here.
|
||||
|
||||
If the statement produced no generated values, the returned result set
|
||||
is empty.
|
||||
*/
|
||||
abstract fun getGeneratedKeys(): ResultSet
|
||||
}
|
||||
|
||||
/*
|
||||
Base exception for the SQL database module.
|
||||
*/
|
||||
open class DatabaseException: Exception
|
||||
|
||||
/*
|
||||
The SQL statement could not be executed successfully by the backend.
|
||||
*/
|
||||
open class SqlExecutionException: DatabaseException
|
||||
|
||||
/*
|
||||
Execution failed because of a constraint violation, such as UNIQUE,
|
||||
FOREIGN KEY, CHECK, or NOT NULL.
|
||||
*/
|
||||
class SqlConstraintException: SqlExecutionException
|
||||
|
||||
/*
|
||||
The DB API was used incorrectly, such as invalid transaction state,
|
||||
ambiguous column-name access, or invalid row indexes.
|
||||
*/
|
||||
class SqlUsageException: DatabaseException
|
||||
|
||||
/*
|
||||
Transaction represents a database transaction (non-transactional operations
|
||||
we intentionally do not support).
|
||||
|
||||
Important: a transaction has __no commit__; instead it commits when
|
||||
leaving the transaction block normally.
|
||||
|
||||
If the transaction block throws any exception not caught inside the calling
|
||||
code, it will be rolled back.
|
||||
*/
|
||||
interface SqlTransaction {
|
||||
/*
|
||||
Execute a SQL statement that returns rows. This includes plain SELECT
|
||||
queries and database-specific DML statements with row-returning clauses
|
||||
such as RETURNING or OUTPUT.
|
||||
|
||||
Portable SQL uses positional ? placeholders only.
|
||||
|
||||
Parameters are already-converted Lyng values bound positionally to the
|
||||
statement. Portable bindable values are:
|
||||
- null
|
||||
- Bool
|
||||
- Int, Double, Decimal
|
||||
- String
|
||||
- Buffer
|
||||
- Date, DateTime, Instant
|
||||
|
||||
Backends may support additional parameter types, but portable code
|
||||
should limit itself to the values above.
|
||||
*/
|
||||
abstract fun select(clause: String,params: Object...): ResultSet
|
||||
|
||||
/*
|
||||
Execute a SQL statement for side effects. Use [select] for any statement
|
||||
whose primary result is a row set.
|
||||
|
||||
Parameters follow the same binding rules as [select].
|
||||
*/
|
||||
abstract fun execute(clause: String,params: Object...): ExecutionResult
|
||||
|
||||
/*
|
||||
Create a nested transaction with real nested semantics, typically using
|
||||
database savepoints.
|
||||
|
||||
If the backend cannot provide real nested transaction semantics, this
|
||||
call should fail with SqlUsageException rather than flattening into the
|
||||
outer transaction.
|
||||
|
||||
Failure inside the nested transaction rolls back only the nested scope;
|
||||
the outer transaction remains active unless the exception is allowed to
|
||||
propagate further.
|
||||
*/
|
||||
abstract fun transaction<T>( block: (SqlTransaction)->T): T
|
||||
}
|
||||
|
||||
/*
|
||||
Special exception to be thrown from SqlTransaction.transaction
|
||||
when nothing else matters/needed (DRY).
|
||||
|
||||
It causes rollback and is propagated to the caller, but should not be
|
||||
treated as a backend/driver failure.
|
||||
|
||||
If rollback itself fails, that rollback failure becomes the primary backend
|
||||
error instead.
|
||||
*/
|
||||
class RollbackException: Exception
|
||||
|
||||
interface Database {
|
||||
/*
|
||||
Open a transaction. Any pooling, physical connection lifecycle, and implementation-specific configuration are owned by the database implementation and hidden from the user.
|
||||
The transaction commits when the block finishes normally,
|
||||
and rolls back if the block exits with an uncaught exception.
|
||||
*/
|
||||
abstract fun transaction<T>(block: (SqlTransaction) -> T): T
|
||||
}
|
||||
|
||||
/*
|
||||
Register a database provider for a URL scheme.
|
||||
|
||||
Provider modules should call this during module initialization when first
|
||||
imported. Scheme matching is case-insensitive and normalized to lowercase.
|
||||
|
||||
Registering the same scheme more than once should fail.
|
||||
*/
|
||||
fun registerDatabaseProvider(
|
||||
scheme: String,
|
||||
opener: (connectionUrl: String, extraParams: Map<String,Object?>) -> Database
|
||||
)
|
||||
|
||||
/*
|
||||
The mandatory generic entry point for all providers. It opens a database
|
||||
handle from a provider-specific connection URL plus extra parameters.
|
||||
|
||||
Providers may expose additional typed constructors such as openSqlite(...)
|
||||
or openPostgres(...), but openDatabase(...) should remain available for
|
||||
configuration-driven usage.
|
||||
|
||||
It should throw IllegalArgumentException for malformed connection URLs or
|
||||
invalid extra parameter shapes detected before opening the backend.
|
||||
Runtime opening failures such as authentication, connectivity, or provider
|
||||
initialization errors should be reported as DatabaseException.
|
||||
|
||||
The matching provider must already be registered, normally because its
|
||||
module was imported and executed. Unknown schemes or missing providers
|
||||
should fail with DatabaseException.
|
||||
*/
|
||||
extern fun openDatabase(connectionUrl: String,extraParams: Map<String,Object?>): Database
|
||||
@@ -0,0 +1,261 @@
|
||||
# SQLite provider implementation plan
|
||||
|
||||
Implementation checklist for the first concrete DB provider:
|
||||
- module: `lyng.io.db.sqlite`
|
||||
- targets: JVM and Native
|
||||
- role: reference implementation for the core `lyng.io.db` contract
|
||||
|
||||
## Scope
|
||||
|
||||
In scope for the first implementation:
|
||||
- provider registration on module import
|
||||
- `sqlite:` URL dispatch through `openDatabase(...)`
|
||||
- typed `openSqlite(...)` helper
|
||||
- `Database` and `SqlTransaction` implementation
|
||||
- result-set implementation
|
||||
- row and column metadata conversion
|
||||
- SQLite savepoint-based nested transactions
|
||||
- generated keys for `execute(...)`
|
||||
- JVM and Native test coverage for the core portable contract
|
||||
|
||||
Out of scope for the first implementation:
|
||||
- schema inspection APIs beyond result-set column metadata
|
||||
- public prepared statement API
|
||||
- batch API
|
||||
- JS support
|
||||
- heuristic temporal parsing
|
||||
- heuristic decimal parsing
|
||||
|
||||
## Proposed module layout
|
||||
|
||||
Core/public declarations:
|
||||
- `.lyng` declaration source for `lyng.io.db`
|
||||
- `.lyng` declaration source for `lyng.io.db.sqlite`
|
||||
|
||||
Backend/runtime implementation:
|
||||
- common transaction/result-set abstractions in `commonMain` where possible
|
||||
- JVM SQLite implementation in `jvmMain`
|
||||
- Native SQLite implementation in `nativeMain`
|
||||
|
||||
## Milestone 1: core DB module skeleton
|
||||
|
||||
1. Move or copy the current DB declarations from `notes/db/lyngdb.lyng` into the
|
||||
real module declaration source location.
|
||||
2. Add the provider registry runtime for:
|
||||
- normalized lowercase scheme lookup
|
||||
- duplicate registration failure
|
||||
- unknown-scheme failure
|
||||
3. Implement generic `openDatabase(connectionUrl, extraParams)` dispatch.
|
||||
4. Add basic tests for:
|
||||
- successful provider registration
|
||||
- duplicate scheme registration failure
|
||||
- unknown scheme failure
|
||||
- malformed URL failure
|
||||
|
||||
## Milestone 2: SQLite provider skeleton
|
||||
|
||||
1. Create `lyng.io.db.sqlite` declaration source with:
|
||||
- typed `openSqlite(...)`
|
||||
- provider-specific documentation
|
||||
2. Register `sqlite` scheme at module initialization time.
|
||||
3. Parse supported URL forms:
|
||||
- `sqlite::memory:`
|
||||
- `sqlite:relative/path.db`
|
||||
- `sqlite:/absolute/path.db`
|
||||
4. Convert typed helper arguments into the same normalized open options used by
|
||||
the generic `openDatabase(...)` path.
|
||||
5. Add tests for:
|
||||
- import-time registration
|
||||
- typed helper open
|
||||
- generic URL-based open
|
||||
- invalid URL handling
|
||||
|
||||
## Milestone 3: JVM backend
|
||||
|
||||
Implementation strategy:
|
||||
- use SQLite JDBC under the hood
|
||||
- keep JDBC fully internal to the provider
|
||||
- preserve the Lyng-facing transaction/result contracts rather than exposing
|
||||
JDBC semantics directly
|
||||
|
||||
Steps:
|
||||
1. Create JVM `Database` implementation that stores normalized SQLite config.
|
||||
2. For each outer `Database.transaction {}`:
|
||||
- open/acquire one JDBC connection
|
||||
- configure connection-level options
|
||||
- begin transaction
|
||||
- commit on normal exit
|
||||
- rollback on uncaught exception
|
||||
- close/release connection
|
||||
- preserve error precedence:
|
||||
- original user exception stays primary on rollback failure
|
||||
- rollback failure becomes primary for intentional `RollbackException`
|
||||
- commit failure is primary on normal-exit commit failure
|
||||
3. For nested `SqlTransaction.transaction {}`:
|
||||
- create savepoint
|
||||
- release savepoint on success
|
||||
- rollback to savepoint on failure
|
||||
- preserve the same primary/secondary exception rules for savepoint rollback
|
||||
failures
|
||||
4. Implement `select(...)`:
|
||||
- bind positional `?` parameters
|
||||
- expose result rows through `ResultSet`
|
||||
- preserve transaction-scoped lifetime
|
||||
- invalidate both the result set and any rows obtained from it when the
|
||||
owning transaction ends
|
||||
5. Implement `execute(...)`:
|
||||
- bind positional parameters
|
||||
- collect affected row count
|
||||
- expose generated keys when supported
|
||||
6. Implement column metadata normalization:
|
||||
- output column label
|
||||
- nullable flag where available
|
||||
- portable `SqlType`
|
||||
- backend native type name
|
||||
7. Add JVM tests for:
|
||||
- transaction commit
|
||||
- transaction rollback
|
||||
- nested transaction savepoint behavior
|
||||
- rollback failure precedence
|
||||
- commit failure precedence
|
||||
- row lookup by index and name
|
||||
- ambiguous name failure
|
||||
- result-set use-after-transaction failure
|
||||
- row use-after-transaction failure
|
||||
- generated keys
|
||||
- `RETURNING` via `select(...)` if the backend supports it
|
||||
|
||||
## Milestone 4: Native backend
|
||||
|
||||
Implementation strategy:
|
||||
- use direct SQLite C bindings
|
||||
- keep semantics aligned with the JVM backend
|
||||
|
||||
Steps:
|
||||
1. Create Native `Database` implementation that stores normalized SQLite config.
|
||||
2. For each outer transaction:
|
||||
- open one SQLite handle if needed
|
||||
- configure pragmas/options
|
||||
- begin transaction
|
||||
- commit or rollback
|
||||
3. Implement nested transactions with SQLite savepoints.
|
||||
4. Implement prepared statement lifecycle internally for `select(...)` and
|
||||
`execute(...)`.
|
||||
5. Implement result-set iteration and statement finalization.
|
||||
6. Implement the same value-conversion policy as JVM:
|
||||
- exact integer mapping
|
||||
- `Double`
|
||||
- `String`
|
||||
- `Buffer`
|
||||
- schema-driven `Bool`
|
||||
- schema-driven `Decimal`
|
||||
- schema-driven temporal parsing
|
||||
7. Add Native tests matching the JVM behavioral suite as closely as possible.
|
||||
|
||||
## Milestone 5: conversion policy
|
||||
|
||||
1. Normalize integer reads:
|
||||
- return `Int`
|
||||
2. Normalize floating-point reads to `Double`.
|
||||
3. Normalize text reads to `String`.
|
||||
4. Normalize blob reads to `Buffer`.
|
||||
5. Parse `Decimal` only for declared/native `DECIMAL` / `NUMERIC` columns.
|
||||
- bind Lyng `Decimal` as canonical text using existing Decimal formatting
|
||||
- read back with the existing Decimal parser
|
||||
6. Parse `Bool` only for declared/native `BOOLEAN` / `BOOL` columns:
|
||||
- prefer integer `0` / `1`
|
||||
- also accept legacy text `true`, `false`, `t`, `f` case-insensitively
|
||||
- always write `Bool` as integer `0` / `1`
|
||||
7. Parse temporal values only from an explicit normalized type-name whitelist:
|
||||
- `DATE`
|
||||
- `DATETIME`
|
||||
- `TIMESTAMP`
|
||||
- `TIMESTAMP WITH TIME ZONE`
|
||||
- `TIMESTAMPTZ`
|
||||
- `DATETIME WITH TIME ZONE`
|
||||
- `TIME`
|
||||
- `TIME WITHOUT TIME ZONE`
|
||||
- `TIME WITH TIME ZONE`
|
||||
8. Never heuristically parse arbitrary string or numeric values into temporal or
|
||||
decimal values.
|
||||
9. If a declared strong type (`Bool`, `Decimal`, `Date`, `DateTime`, `Instant`)
|
||||
cannot be converted from the stored value, fail with `SqlExecutionException`.
|
||||
10. Add tests for each conversion rule.
|
||||
|
||||
## Milestone 6: result-set contract
|
||||
|
||||
1. Ensure `ResultSet.columns` is available before iteration.
|
||||
2. Implement name lookup using result-column labels.
|
||||
3. Throw `SqlUsageException` for:
|
||||
- invalid row index
|
||||
- missing column name
|
||||
- ambiguous column name
|
||||
- use after transaction end
|
||||
4. Implement cheap `isEmpty()` where practical.
|
||||
5. Implement `size()` separately, allowing buffering/consumption when required.
|
||||
6. Verify resource cleanup on:
|
||||
- full iteration
|
||||
- canceled iteration
|
||||
- transaction rollback
|
||||
|
||||
## Milestone 7: provider options
|
||||
|
||||
Support these typed helper options first:
|
||||
- `readOnly`
|
||||
- `createIfMissing`
|
||||
- `foreignKeys`
|
||||
- `busyTimeoutMillis`
|
||||
|
||||
Expected behavior:
|
||||
- `foreignKeys` defaults to `true`
|
||||
- `busyTimeoutMillis` has a non-zero sensible default
|
||||
- `readOnly` is explicit
|
||||
- `createIfMissing` is explicit
|
||||
|
||||
Add tests for option handling where backend behavior is observable.
|
||||
|
||||
## Milestone 8: documentation and examples
|
||||
|
||||
1. Add user docs for:
|
||||
- importing `lyng.io.db.sqlite`
|
||||
- generic `openDatabase(...)`
|
||||
- typed `openSqlite(...)`
|
||||
- transaction usage
|
||||
- nested transactions
|
||||
2. Add small sample snippets for:
|
||||
- in-memory DB
|
||||
- file-backed DB
|
||||
- schema creation
|
||||
- insert/select/update
|
||||
- rollback on exception
|
||||
|
||||
## Testing priorities
|
||||
|
||||
Highest priority behavioral tests:
|
||||
1. provider registration and scheme dispatch
|
||||
2. outer transaction commit/rollback
|
||||
3. nested savepoint semantics
|
||||
4. row metadata and name lookup behavior
|
||||
5. generated keys for inserts
|
||||
6. result-set lifetime rules
|
||||
7. value conversion rules
|
||||
8. helper vs generic-open parity
|
||||
|
||||
## Risks
|
||||
|
||||
Main implementation risks:
|
||||
- keeping JVM and Native value-conversion behavior identical
|
||||
- correctly enforcing result-set lifetime across both backends
|
||||
- SQLite temporal conversion ambiguity
|
||||
- JDBC metadata differences on JVM
|
||||
- native statement/finalizer lifecycle bugs
|
||||
|
||||
## Suggested implementation order
|
||||
|
||||
1. core registry runtime
|
||||
2. SQLite `.lyng` provider declarations
|
||||
3. JVM SQLite backend
|
||||
4. JVM test suite
|
||||
5. Native SQLite backend
|
||||
6. shared behavioral test suite refinement
|
||||
7. documentation/examples
|
||||
@@ -0,0 +1,272 @@
|
||||
# SQLite provider for `lyng.io.db`
|
||||
|
||||
First concrete provider candidate for the DB module.
|
||||
|
||||
Module name:
|
||||
- `lyng.io.db.sqlite`
|
||||
|
||||
Responsibilities:
|
||||
- register SQLite URL schemes on first import
|
||||
- provide the generic `openDatabase(...)` entry point for SQLite URLs
|
||||
- provide typed SQLite-specific helpers for ergonomic opening
|
||||
- implement the core `Database` / `SqlTransaction` API on both JVM and Native
|
||||
|
||||
## Registration and URL schemes
|
||||
|
||||
On first import, the module should register at least:
|
||||
- `sqlite`
|
||||
|
||||
Possible accepted URL forms:
|
||||
- `sqlite::memory:`
|
||||
- `sqlite:./local.db`
|
||||
- `sqlite:/absolute/path/data.db`
|
||||
|
||||
The exact accepted path grammar can be tightened during implementation, but it
|
||||
should stay simple and configuration-friendly.
|
||||
|
||||
## Typed helper
|
||||
|
||||
The provider should also expose a typed helper, e.g.:
|
||||
|
||||
```lyng
|
||||
fun openSqlite(
|
||||
path: String,
|
||||
readOnly: Bool = false,
|
||||
createIfMissing: Bool = true,
|
||||
foreignKeys: Bool = true,
|
||||
busyTimeoutMillis: Int = 5000
|
||||
): Database
|
||||
```
|
||||
|
||||
Possible special values:
|
||||
- `":memory:"` for in-memory DB
|
||||
|
||||
The helper is provider-specific sugar with explicit typed arguments. The generic
|
||||
`openDatabase(...)` path must stay fully supported for configuration-driven
|
||||
usage.
|
||||
|
||||
## Implementation strategy
|
||||
|
||||
SQLite should be the first real provider because it is available on both JVM and
|
||||
Native and exercises almost the whole core API surface.
|
||||
|
||||
### JVM
|
||||
|
||||
Preferred implementation:
|
||||
- JDBC-backed SQLite provider for the JVM-specific backend implementation
|
||||
|
||||
This is acceptable because SQLite itself is local and the JDBC bridge is much
|
||||
simpler here than for network databases.
|
||||
|
||||
### Native
|
||||
|
||||
Preferred implementation:
|
||||
- direct SQLite C library binding
|
||||
|
||||
The provider should present the same Lyng-facing semantics on both backends.
|
||||
|
||||
## Transactions
|
||||
|
||||
Required behavior:
|
||||
- `Database.transaction {}` starts a real SQLite transaction
|
||||
- `SqlTransaction.transaction {}` uses SQLite savepoints
|
||||
- nested transactions must be supported
|
||||
- failures in nested transactions roll back only to the nested savepoint unless
|
||||
the exception escapes further
|
||||
- error precedence follows the core DB contract:
|
||||
- user exception + successful rollback -> user exception escapes unchanged
|
||||
- user exception + rollback failure -> user exception stays primary
|
||||
- intentional `RollbackException` + rollback failure -> rollback failure is
|
||||
primary
|
||||
- commit failure after normal completion -> commit failure is primary
|
||||
|
||||
SQLite is a good fit here because savepoints are well-supported.
|
||||
|
||||
Connection/handle semantics:
|
||||
- one outer `Database.transaction {}` must use exactly one physical SQLite
|
||||
connection/handle for its whole lifetime
|
||||
- nested transactions must stay on that same connection/handle
|
||||
- a transaction must never hop across connections
|
||||
|
||||
This is required because SQLite transaction state, savepoints, and generated
|
||||
row-id behavior are all connection-local.
|
||||
|
||||
## Result sets
|
||||
|
||||
The provider may stream rows or buffer them, but it must preserve the core
|
||||
contract:
|
||||
- result sets are valid only while the owning transaction is active
|
||||
- rows obtained from a result set are also invalid after the owning
|
||||
transaction ends, even if they were already buffered
|
||||
- iteration closes underlying resources when finished or canceled
|
||||
- `isEmpty()` should be cheap where possible
|
||||
- `size()` may consume or buffer the full result
|
||||
|
||||
## SQLite-specific type mapping notes
|
||||
|
||||
SQLite uses dynamic typing and affinity rules, so the provider must normalize
|
||||
returned values into the portable Lyng types.
|
||||
|
||||
Recommended mapping strategy:
|
||||
- integer values -> `Int`
|
||||
- floating-point values -> `Double`
|
||||
- text values -> `String`
|
||||
- blob values -> `Buffer`
|
||||
- declared/native `BOOLEAN` / `BOOL` -> `Bool`
|
||||
- numeric values that are explicitly read/declared as decimal -> `Decimal` when
|
||||
the provider can determine this reliably
|
||||
- date/time values should be parsed only when the declared/native column type
|
||||
indicates temporal intent
|
||||
|
||||
The provider should not heuristically parse arbitrary `TEXT`, `INTEGER`, or
|
||||
`REAL` values into temporal or decimal types just because the stored value looks
|
||||
like one.
|
||||
|
||||
If a column is exposed with a stronger portable `SqlType` such as `Bool`,
|
||||
`Decimal`, `Date`, `DateTime`, or `Instant`, then the produced row value should
|
||||
either be that Lyng value or `null`. Invalid stored representations should fail
|
||||
with `SqlExecutionException`.
|
||||
|
||||
Boolean policy exception:
|
||||
- unlike temporal values, legacy boolean encodings are cheap to recognize and
|
||||
have low ambiguity
|
||||
- therefore SQLite `BOOL` / `BOOLEAN` columns may accept a small tolerant set of
|
||||
legacy boolean encodings on read
|
||||
- writes should still always use integer `0` / `1`
|
||||
- temporal and decimal conversions remain strict/schema-driven
|
||||
|
||||
Declared type-name normalization for SQLite v1:
|
||||
- trim surrounding whitespace
|
||||
- uppercase
|
||||
- collapse internal whitespace runs to a single space
|
||||
- strip a trailing `( ... )` size/precision suffix before matching
|
||||
|
||||
Examples:
|
||||
- ` numeric(10,2) ` -> `NUMERIC`
|
||||
- `timestamp with time zone` -> `TIMESTAMP WITH TIME ZONE`
|
||||
|
||||
SQLite v1 declared type-name whitelist:
|
||||
|
||||
| normalized declared/native type | portable `SqlType` |
|
||||
|---------------------------------|--------------------|
|
||||
| `BOOLEAN` | `Bool` |
|
||||
| `BOOL` | `Bool` |
|
||||
| `DECIMAL` | `Decimal` |
|
||||
| `NUMERIC` | `Decimal` |
|
||||
| `DATE` | `Date` |
|
||||
| `DATETIME` | `DateTime` |
|
||||
| `TIMESTAMP` | `DateTime` |
|
||||
| `TIMESTAMP WITH TIME ZONE` | `Instant` |
|
||||
| `TIMESTAMPTZ` | `Instant` |
|
||||
| `DATETIME WITH TIME ZONE` | `Instant` |
|
||||
| `TIME` | `String` |
|
||||
| `TIME WITHOUT TIME ZONE` | `String` |
|
||||
| `TIME WITH TIME ZONE` | `String` |
|
||||
|
||||
Anything not in this table should not be promoted to a stronger portable type
|
||||
just from its declared name.
|
||||
|
||||
## SQLite temporal policy
|
||||
|
||||
SQLite has no strong built-in temporal storage types, so the provider should use
|
||||
a strict schema-driven conversion policy.
|
||||
|
||||
Binding:
|
||||
- `null` -> SQL `NULL`
|
||||
- `Bool` -> integer `0` / `1`
|
||||
- `Int` -> SQLite integer
|
||||
- `Double` -> SQLite real
|
||||
- `Decimal` -> canonical decimal text representation using the existing Lyng
|
||||
Decimal formatter
|
||||
- `String` -> text
|
||||
- `Buffer` -> blob
|
||||
- `Date` -> ISO text `YYYY-MM-DD`
|
||||
- `DateTime` -> ISO text without timezone
|
||||
- `Instant` -> ISO text in UTC with explicit timezone marker
|
||||
|
||||
Reading:
|
||||
- storage class `NULL` -> `null`
|
||||
- normalized declared/native type `BOOLEAN` or `BOOL` -> parse as `Bool` with
|
||||
this ordered rule:
|
||||
- integer `0` / `1` first
|
||||
- then legacy text forms, case-insensitively: `true`, `false`, `t`, `f`
|
||||
- other stored values are conversion errors
|
||||
- normalized declared/native type `DATE` -> parse as `Date`
|
||||
- normalized declared/native type `DATETIME` or `TIMESTAMP` -> parse as `DateTime`
|
||||
- normalized declared/native type `TIMESTAMP WITH TIME ZONE`,
|
||||
`TIMESTAMPTZ`, or `DATETIME WITH TIME ZONE` -> parse as `Instant`
|
||||
- normalized declared/native type `TIME`, `TIME WITHOUT TIME ZONE`, or
|
||||
`TIME WITH TIME ZONE` -> keep as `String` in v1
|
||||
- normalized declared/native type `DECIMAL` or `NUMERIC` -> parse as `Decimal`
|
||||
- otherwise integer storage -> `Int`
|
||||
- otherwise real storage -> `Double`
|
||||
- otherwise text storage -> `String`
|
||||
- otherwise blob storage -> `Buffer`
|
||||
- otherwise do not guess, and return the raw normalized SQLite value type
|
||||
|
||||
For v1, the provider should not automatically interpret numeric epoch values or
|
||||
Julian date encodings unless this is later added as an explicit provider option.
|
||||
|
||||
## SQLite decimal policy
|
||||
|
||||
Decimal conversion should also be schema-driven:
|
||||
- normalized declared/native type `DECIMAL` or `NUMERIC` -> parse as `Decimal`
|
||||
- otherwise do not guess from text or floating-point storage alone
|
||||
|
||||
Decimal exactness note:
|
||||
- SQLite has no native decimal storage class; values are stored as `INTEGER`,
|
||||
`REAL`, `TEXT`, `BLOB`, or `NULL`
|
||||
- binding Lyng `Decimal` as text is only the provider's chosen encoding, not a
|
||||
native SQLite decimal representation
|
||||
- SQLite v1 should therefore store Lyng `Decimal` values as canonical text and
|
||||
parse them back with the existing Lyng Decimal parser/formatter stack
|
||||
- schemas that care about decimal semantics should still declare
|
||||
`DECIMAL` / `NUMERIC` affinity so the provider knows to expose `Decimal`
|
||||
- exact round-tripping therefore cannot be guaranteed for generic
|
||||
`DECIMAL` / `NUMERIC` columns, because SQLite affinity rules may coerce stored
|
||||
values before they are read back
|
||||
- SQLite values already stored as `REAL` in `DECIMAL` / `NUMERIC` columns may
|
||||
already reflect floating-point precision loss before Lyng sees them
|
||||
- if exact decimal preservation is required, the schema and provider policy must
|
||||
intentionally store decimal values in an exact representation, most simply as
|
||||
canonical text
|
||||
|
||||
This area will need careful implementation rules because SQLite itself does not
|
||||
have a strong native temporal type system.
|
||||
|
||||
## Generated keys
|
||||
|
||||
`ExecutionResult.getGeneratedKeys()` for SQLite should return implementation-
|
||||
supported generated values for `execute(...)`.
|
||||
|
||||
Typical example:
|
||||
- row id generated by an insert into a table with integer primary key
|
||||
|
||||
Statements that explicitly return rows should still go through `select(...)`,
|
||||
for example if the provider eventually supports SQLite `RETURNING`.
|
||||
|
||||
## Options
|
||||
|
||||
Likely provider-specific options:
|
||||
- read-only mode
|
||||
- create-if-missing
|
||||
- busy timeout
|
||||
- foreign keys on/off
|
||||
|
||||
These options can be accepted both through SQLite helper functions and through
|
||||
`openDatabase(..., extraParams)` when the URL scheme is `sqlite`.
|
||||
|
||||
Recommended defaults:
|
||||
- foreign keys enabled by default
|
||||
- busy timeout may be configurable, but should have a sensible default
|
||||
- read-only and create-if-missing should be explicit options rather than hidden
|
||||
URL magic
|
||||
|
||||
## Non-goals for v1
|
||||
|
||||
Not required for the first SQLite provider:
|
||||
- schema metadata beyond result-column metadata
|
||||
- prepared statement API in public surface
|
||||
- batch execution API
|
||||
- provider capability flags
|
||||
- JS/browser support
|
||||
Reference in new issue
Block a user