Compare commits

..
50 Commits
Author SHA1 Message Date
sergeych 9735774efd v1.5.5 release 2026-04-23 15:18:47 +03:00
sergeych 14214e91e1 Fix escaping immutable closure captures 2026-04-22 18:50:49 +03:00
sergeychandClaude Sonnet 4.6 0f5343fa17 Add VPS deployment target and Ansible provisioning for lynglang.com
- deploy_site now defaults to new VPS (94.130.36.94, /var/www/lynglang);
  use --old to deploy to d.lynglang.com as before
- infra/setup_vps.yml: Ansible playbook installs nginx + certbot, obtains
  SSL cert for lynglang.com, fixes Debian buster EOL apt sources
- infra/inventory.ini and infra/templates/nginx_lynglang.conf.j2 included

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 12:31:01 +03:00
sergeych d0f51928de fixed wrong error message in val ++/-- 2026-04-22 08:56:02 +03:00
sergeych 0973a6afeb Fix JVM import caching and class/object bytecode dispatch 2026-04-21 23:25:07 +03:00
sergeych 953f237ca3 Optimize primitive list fill capacity and append 2026-04-21 21:37:04 +03:00
sergeych fbb5688696 Drive higher-order lambda inlining from call metadata 2026-04-21 20:51:16 +03:00
sergeych f4ab2ebab4 Extend lambda inlining to getOrPut and implicit it calls 2026-04-21 19:20:25 +03:00
sergeych 0c3242cbd8 Generalize higher-order lambda inlining 2026-04-21 19:12:16 +03:00
sergeych 1d5caaa836 Broaden lambda method inlining with captures 2026-04-21 19:07:05 +03:00
sergeych 3be2892025 Use fast compiled callbacks in dynamic and flow helpers 2026-04-21 18:06:31 +03:00
sergeych c80900c503 Prevent cross-script module slot aliasing 2026-04-21 17:34:00 +03:00
sergeych ee634c8dff Extend generic fast callable entry points 2026-04-21 17:26:53 +03:00
sergeych 6c91b77a85 Use fast lambda calls in scope facade 2026-04-21 16:43:02 +03:00
sergeych 3721ee8332 Fast-path compiled lambda calls with args 2026-04-21 16:04:04 +03:00
sergeych 97990f00ce Cover when-based exact callable fast paths 2026-04-21 14:37:27 +03:00
sergeych 80693e7690 Inline exact lambdas through wrapper expressions 2026-04-21 14:36:18 +03:00
sergeych 029fe874fa Propagate exact callable refs across expressions 2026-04-21 14:33:23 +03:00
sergeych ffa64d691b Use direct calls for constant constructors 2026-04-21 13:30:09 +03:00
sergeych 30c9a5a565 Propagate exact lambda refs across bytecode slots 2026-04-21 13:20:27 +03:00
sergeych db3a780645 Generalize bytecode fast-call dispatch 2026-04-21 13:09:50 +03:00
sergeych fc7d26ee4b Fast-path bytecode statement execution 2026-04-21 11:39:47 +03:00
sergeych a61b5a31be Add non-suspending fast calls for bytecode callables 2026-04-21 11:35:20 +03:00
sergeych 33d170f525 Optimize fast-path bytecode lambda invocation 2026-04-21 11:29:14 +03:00
sergeych f72cdfdf83 Inline simple lambdas in bytecode fast paths 2026-04-21 09:05:18 +03:00
sergeych 10fa4de4fa regression fix 2026-04-20 18:48:28 +03:00
sergeych 4acee56de8 small optimization of List.fill 2026-04-20 14:09:09 +03:00
sergeych 87060c7eb7 +inference bugs
+List.ensureCapacity
optimized List.fill
2026-04-20 13:56:47 +03:00
sergeych 6ba128f7ba fixed some more inference bugs 2026-04-20 12:23:17 +03:00
sergeych a8f73dc8bd added LaunchPool 2026-04-18 23:37:21 +03:00
sergeychandClaude Sonnet 4.6 145c3ae34a Add completeExceptionally, cancelAndJoin, and refine LaunchPool spec
- CompletableDeferred.completeExceptionally(exception): fails the deferred
  with a Lyng exception; the original exception object is preserved as
  errorObject so assertThrows/catch see the correct class (works for both
  built-in ObjException and user-declared class instances)
- LaunchPool: add maxQueueSize parameter (suspends launch when full),
  add cancelAndJoin() for graceful shutdown that waits for running tasks,
  clarify all semantics in the extern declaration comments
- Two tests for completeExceptionally (built-in and user-defined exception)
- Fix parallelism.md channel doc-test to be illustration-only (avoid
  flakiness from coroutine scheduling in the doc-test runner)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 03:00:40 +03:00
sergeychandClaude Sonnet 4.6 f9a07f176a Add Channel primitive for coroutine inter-task communication
Exposes Kotlin's Channel<Obj> to Lyng scripts as a first-class type
with rendezvous, buffered, and unlimited capacity modes. Supports
suspending send/receive, non-suspending tryReceive, close/drain
semantics, and isClosedForSend/isClosedForReceive properties.

Also fixes a pre-existing typo in BookTest.kt that blocked JVM test
compilation, and adds Channel reference docs and a parallelism.md section.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 02:28:55 +03:00
sergeych 30e56946a0 improved JDBC provider for lyng.io.db 2026-04-18 02:06:29 +03:00
sergeych 49fc700233 Detach materialized SQL rows from transaction lifetime 2026-04-17 20:38:40 +03:00
sergeych 07cb5a519c Add JDBC and websocket/native follow-up changes 2026-04-16 23:35:15 +03:00
sergeych 8f66cd7680 Fix lynglib binding regressions and green test suite 2026-04-16 23:34:45 +03:00
sergeych bb2119b1d1 ignores 2026-04-16 22:12:19 +03:00
sergeych 64273ac60a Add JDBC database provider for JVM 2026-04-16 22:10:36 +03:00
sergeych ee392daa13 another sample 2026-04-16 13:35:12 +03:00
sergeych 0affa92674 better SQL example 2026-04-16 01:08:31 +03:00
sergeych a2e3f80ab6 better sql exaple, some inference bugs fixed 2026-04-16 00:54:00 +03:00
sergeych f8d2533b48 Make SQLite native linker lookup portable 2026-04-15 23:47:38 +03:00
sergeych bb9af2258b Fix SQLite release deployment 2026-04-15 23:35:59 +03:00
sergeych b3be908242 Add Date type and database release docs 2026-04-15 23:20:23 +03:00
sergeych 14dc73db3e Harden SQLite provider release behavior 2026-04-15 23:11:50 +03:00
sergeych f9bbdd56bf Test DB rollback precedence in SQLite 2026-04-15 22:38:11 +03:00
sergeych 04e80c384e Stabilize DB provider registration test 2026-04-15 22:35:19 +03:00
sergeych 6d340824e4 Tighten SQLite transaction failures and tests 2026-04-15 22:20:29 +03:00
sergeych 5f6f6b9ae4 Align SQLite value conversion across JVM and native 2026-04-15 22:03:42 +03:00
sergeych 55ba6113e7 Add Lyng DB contract and SQLite provider skeleton 2026-04-15 20:39:38 +03:00
125 changed files with 14054 additions and 653 deletions

No files matched your search

+2
View File
@@ -28,4 +28,6 @@ debug.log
/compile_metadata_output.txt
test_output*.txt
/site/src/version-template/lyng-version.js
/bugcontents.db
/bugs/
contents.db
+43
View File
@@ -7,6 +7,49 @@ History note:
- Entries below are synchronized and curated for `1.5.x`.
- Earlier history may be incomplete and should be cross-checked with git tags/commits when needed.
## Unreleased
- No unreleased entries yet.
## 1.5.5 (2026-04-23)
### Concurrency and collections
- Added coroutine coordination primitives and helpers for everyday parallel code:
- `Channel` for coroutine-to-coroutine communication
- `LaunchPool` for bounded-concurrency task execution
- `Iterable<Deferred>.joinAll()` to await a whole collection of deferreds in input order
- `CompletableDeferred.completeExceptionally(...)` and `Deferred.cancelAndJoin()`
- Added docs and examples for the new concurrency APIs, including `joinAll()` coverage in iterable and parallelism references.
### Database and time APIs
- Added the portable `lyng.io.db` SQL contract and the first concrete providers:
- `lyng.io.db.sqlite` on JVM and Linux Native
- `lyng.io.db.jdbc` on JVM
- Added SQLite/JDBC release hardening:
- nested transactions via savepoints
- detached materialized rows
- generated-key support through `ExecutionResult.getGeneratedKeys()`
- schema-driven value conversion for `Bool`, `Decimal`, `Date`, `DateTime`, and `Instant`
- portable SQLite linker/deployment fixes and documented runtime options
- Added `Date` to `lyng.time` and the core runtime as a first-class calendar-date type, plus conversions and arithmetic across `Instant`, `DateTime`, and `Date`.
### Language, stdlib, and tooling
- Added extensions on singleton `object` declarations, including object-scoped indexer overrides for bracket syntax.
- Added backtick string literals and formatter support.
- Added `lyng.legacy_digest` for SHA-1 compatibility work, `String.replace`, and `buffer.base64std`.
- Improved CLI/runtime behavior with `atExit` shutdown handlers, native release-binary work, and follow-up CLI packaging/import fixes.
- Expanded docs across the tutorial, stdlib references, database docs, networking docs, and release notes.
### Runtime/compiler stability and performance
- Extended exact-call and higher-order lambda inlining through the bytecode compiler, including compiled fast paths for simple lambdas, wrappers, captures, and common higher-order helpers.
- Fixed import caching and class/object bytecode dispatch on JVM.
- Fixed immutable `val` compound assignments so true mutating `*Assign` operations continue to work while fallback reassignments report the correct read-only error.
- Fixed closure/capture and import regressions across launched loops, singleton/object extensions, aliasing, transitive re-exports, and immutable capture escaping.
- Improved list-fill/list-append fast paths, nullable-let inference, Decimal/Complex interop, and related regression coverage.
### Release notes
- Release metadata, homepage samples, docs, and README now point to `1.5.5`.
## 1.5.4 (2026-04-03)
### Runtime and compiler stability
+5 -3
View File
@@ -48,10 +48,12 @@ assertEquals(A.E.One, A.One)
- [Language home](https://lynglang.com)
- [introduction and tutorial](docs/tutorial.md) - start here please
- [Latest release notes (1.5.4)](docs/whats_new.md)
- [Latest release notes (1.5.5)](docs/whats_new.md)
- [What's New in 1.5](docs/whats_new_1_5.md)
- [Testing and Assertions](docs/Testing.md)
- [Filesystem and Processes (lyngio)](docs/lyngio.md)
- [SQL Databases (lyng.io.db)](docs/lyng.io.db.md)
- [Time and Calendar Types](docs/time.md)
- [Return Statement](docs/return_statement.md)
- [Efficient Iterables in Kotlin Interop](docs/EfficientIterables.md)
- [Samples directory](docs/samples)
@@ -64,7 +66,7 @@ assertEquals(A.E.One, A.One)
### Add dependency to your project
```kotlin
val lyngVersion = "1.5.4"
val lyngVersion = "1.5.5"
repositories {
// ...
@@ -184,7 +186,7 @@ Designed to add scripting to kotlin multiplatform application in easy and effici
# Language Roadmap
The current stable release is **v1.5.4**: the 1.5 cycle is feature-complete, compiler/runtime stabilization work is in, and the language, tooling, and site are aligned around the current release.
The current stable release is **v1.5.5**: the 1.5 cycle now includes the database/date/concurrency additions as well as the latest compiler/runtime stabilization work, and the language, tooling, and site are aligned around this release.
Ready features:
+1 -1
View File
@@ -20,7 +20,7 @@
set -e
echo "publishing all artifacts"
echo
./gradlew publishToMavenLocal site:jsBrowserDistribution publish buildInstallablePlugin :lyng:linkReleaseExecutableLinuxX64 :lyng:installJvmDist --parallel
./gradlew publishToMavenLocal site:jsBrowserDistribution publish buildInstallablePlugin :lyng:linkReleaseExecutableLinuxX64 :lyng:installJvmDist --parallel --no-configuration-cache
#echo
#echo "Creating plugin"
+14 -11
View File
@@ -18,10 +18,12 @@
#
upload_only=false
target=vps # default: new server; use --old for d.lynglang.com
for arg in "$@"; do
if [[ "$arg" == "-u" || "$arg" == "--upload-only" ]]; then
upload_only=true
break
elif [[ "$arg" == "--old" ]]; then
target=com
fi
done
@@ -88,19 +90,20 @@ function updateIdeaPluginDownloadLink() {
fi
}
# default target settings
case "com" in
# target settings (-t com | -t vps)
case "$target" in
com)
SSH_HOST=sergeych@d.lynglang.com # host to deploy to
SSH_PORT=22 # ssh port on it
ROOT=/bigstore/sergeych_pub/lyng # directory to rsync to
SSH_HOST=sergeych@d.lynglang.com
SSH_PORT=22
ROOT=/bigstore/sergeych_pub/lyng
;;
vps)
SSH_HOST=sergeych@94.130.36.94
SSH_PORT=22
ROOT=/var/www/lynglang
;;
# com)
# SSH_HOST=vvk@front-01.neurodatalab.com
# ROOT=/home/vvk
# ;;
*)
echo "*** ERROR: target not specified (use deploy com | dev)"
echo "*** ERROR: unknown target '$target' (use -t com | -t vps)"
echo "*** stop"
exit 101
esac
+211
View File
@@ -0,0 +1,211 @@
# Channel
A `Channel` is a **hot, bidirectional pipe** for passing values between concurrently running coroutines.
Unlike a [Flow], which is cold and replayed on every collection, a `Channel` is stateful: each value
sent is consumed by exactly one receiver.
Channels model the classic _producer / consumer_ pattern and are the right tool when:
- two or more coroutines need to exchange individual values at their own pace;
- you want back-pressure (rendezvous) or explicit buffering control;
- you need a push-based, hot data source (opposite of the pull-based, cold [Flow]).
## Constructors
```
Channel() // rendezvous — sender and receiver must meet
Channel(n: Int) // buffered — sender may run n items ahead of the receiver
Channel(Channel.UNLIMITED) // no limit on buffered items
```
**Rendezvous** (`Channel()`, capacity 0): `send` suspends until a matching `receive` is ready,
and vice-versa. This gives the tightest synchronisation and the smallest memory footprint.
**Buffered** (`Channel(n)`): `send` only suspends when the internal buffer is full. Allows the
producer to get up to _n_ items ahead of the consumer.
**Unlimited** (`Channel(Channel.UNLIMITED)`): `send` never suspends. Useful when the producer is
bursty and you do not want it blocked, but be careful not to grow the buffer without bound.
## Sending and receiving
```lyng
val ch = Channel() // rendezvous channel
val producer = launch {
ch.send("hello") // suspends until the receiver is ready
ch.send("world")
ch.close() // signal: no more values
}
val a = ch.receive() // suspends until "hello" arrives
val b = ch.receive() // suspends until "world" arrives
val c = ch.receive() // channel is closed and drained → null
assertEquals("hello", a)
assertEquals("world", b)
assertEquals(null, c)
```
`receive()` returns `null` when the channel is both **closed** _and_ **fully drained** — that is
the idiomatic loop termination condition:
```lyng
val ch = Channel(4)
launch {
for (i in 1..5) ch.send(i)
ch.close()
}
var item = ch.receive()
while (item != null) {
println(item)
item = ch.receive()
}
```
## Non-suspending poll
`tryReceive()` never suspends. It returns the next buffered value, or `null` if the buffer is
empty or the channel is closed.
```lyng
val ch = Channel(8)
ch.send(42)
println(ch.tryReceive()) // 42
println(ch.tryReceive()) // null — nothing buffered right now
```
Use `tryReceive` for _polling_ patterns where blocking would be unacceptable, for example when
combining channel checks with other work inside a coroutine loop.
## Closing a channel
`close()` marks the channel so that no further `send` calls are accepted. Any items already in the
buffer can still be received. Once the buffer is drained, `receive()` returns `null` and
`isClosedForReceive` becomes `true`.
```lyng
val ch = Channel(2)
ch.send(1)
ch.send(2)
ch.close()
assert(ch.isClosedForSend)
assert(!ch.isClosedForReceive) // still has 2 buffered items
ch.receive() // 1
ch.receive() // 2
assert(ch.isClosedForReceive) // drained
```
Calling `send` after `close()` throws `IllegalStateException`.
## Properties
| property | type | description |
|---------------------|--------|----------------------------------------------------------|
| `isClosedForSend` | `Bool` | `true` after `close()` is called |
| `isClosedForReceive`| `Bool` | `true` when closed _and_ every buffered item is consumed |
## Methods
| method | suspends | description |
|-----------------|----------|----------------------------------------------------------------------------------|
| `send(value)` | yes | send a value; suspends when buffer full (rendezvous: always until partner ready) |
| `receive()` | yes | receive next value; suspends when empty; returns `null` when closed + drained |
| `tryReceive()` | no | return next buffered value or `null`; never suspends |
| `close()` | no | signal end of production; existing buffer items are still receivable |
## Static constants
| constant | value | description |
|---------------------|------------------|-------------------------------------|
| `Channel.UNLIMITED` | `Int.MAX_VALUE` | capacity for an unlimited-buffer channel |
## Common patterns
### Producer / consumer
```lyng
val ch = Channel()
val results = []
val mu = Mutex()
val consumer = launch {
var item = ch.receive()
while (item != null) {
mu.withLock { results += item }
item = ch.receive()
}
}
launch {
for (i in 1..5) ch.send("msg:$i")
ch.close()
}.await()
consumer.await()
println(results)
```
### Fan-out: one channel, many consumers
```lyng
val ch = Channel(16)
// multiple consumers
val workers = (1..4).map { id ->
launch {
var task = ch.receive()
while (task != null) {
println("worker $id handles $task")
task = ch.receive()
}
}
}
// single producer
for (i in 1..20) ch.send(i)
ch.close()
workers.forEach { it.await() }
```
### Ping-pong between two coroutines
```lyng
val ping = Channel()
val pong = Channel()
launch {
repeat(3) {
val msg = ping.receive()
println("got: $msg → sending pong")
pong.send("pong")
}
}
repeat(3) {
ping.send("ping")
println(pong.receive())
}
```
## Channel vs Flow
| | [Flow] | Channel |
|---|---|---|
| **temperature** | cold (lazy) | hot (eager) |
| **replay** | every collector gets a fresh run | each item is consumed once |
| **consumers** | any number; each gets all items | one receiver per item |
| **back-pressure** | built-in via rendezvous | configurable (rendezvous / buffered / unlimited) |
| **typical use** | transform pipelines, sequences | producer–consumer, fan-out |
## See also
- [parallelism] — `launch`, `Deferred`, `Mutex`, `Flow`, and the full concurrency picture
- [Flow] — cold async sequences
[Flow]: parallelism.md#flow
[parallelism]: parallelism.md
+18
View File
@@ -55,6 +55,23 @@ Here is the sample:
assertEquals( (1..3).joinToString { it * 10 }, "10 20 30")
>>> void
## joinAll
`joinAll()` is an `Iterable<Deferred>` helper that awaits every deferred in iteration order and returns a `List`
with the collected results.
val jobs = (1..4).map { n ->
launch { n * n }
}
assertEquals([1, 4, 9, 16], jobs.joinAll())
>>> void
Notes:
- it does not start any task by itself; it only awaits the deferreds already present in the iterable.
- awaiting happens in iteration order, so the result list keeps the same order as the input iterable.
- if any deferred fails or was cancelled, that `await()` error is propagated from `joinAll()`.
## `sum` and `sumOf`
These, again, does the thing:
@@ -184,6 +201,7 @@ Search for the first element that satisfies the given predicate:
| sortedWith(comparator) | sort using a comparator that compares elements (1) |
| sortedBy(predicate) | sort by comparing results of the predicate function |
| joinToString(s,t) | convert iterable to string, see (2) |
| joinAll() | for `Iterable<Deferred>`, await all items in order and collect results to [List] |
| reversed() | create a list containing items from this in reverse order |
| shuffled() | create a list of shuffled elements |
+110
View File
@@ -0,0 +1,110 @@
# LaunchPool
`LaunchPool` is a bounded-concurrency task pool: you submit lambdas with `launch`, and the pool runs them using a fixed number of worker coroutines.
## Constructor
```
LaunchPool(maxWorkers, maxQueueSize = Channel.UNLIMITED)
```
| Parameter | Description |
|-----------|-------------|
| `maxWorkers` | Maximum number of tasks that run in parallel. |
| `maxQueueSize` | Maximum number of tasks that may wait in the queue. When the queue is full, `launch` suspends the caller until space becomes available. Defaults to `Channel.UNLIMITED` (no bound). |
## Methods
### `launch(lambda): Deferred`
Schedules `lambda` for execution and returns a `Deferred` for its result.
- Suspends if the queue is full (`maxQueueSize` reached).
- Throws `IllegalStateException` if the pool is already closed or cancelled.
- Any exception thrown by `lambda` is captured in the returned `Deferred` and **does not escape the pool**.
```lyng
val pool = LaunchPool(4)
val d1 = pool.launch { computeSomething() }
val d2 = pool.launch { computeOther() }
pool.closeAndJoin()
println(d1.await())
println(d2.await())
```
### `closeAndJoin()`
Stops accepting new tasks and suspends until all queued and running tasks complete normally. After this call, any further `launch` throws `IllegalStateException`. Idempotent — safe to call multiple times.
### `cancel()`
Immediately closes the queue and cancels all worker coroutines. Queued but unstarted tasks are discarded. After this call, `launch` throws `IllegalStateException`. Idempotent.
### `cancelAndJoin()`
Like `cancel()`, but also suspends until all worker coroutines have stopped. Useful when you need to be sure no worker code is still running before proceeding. Idempotent.
## Exception handling
Exceptions from submitted lambdas are captured per-task in the returned `Deferred`. The pool itself continues running after a task failure:
```lyng
val pool = LaunchPool(2)
val good = pool.launch { 42 }
val bad = pool.launch { throw IllegalArgumentException("boom") }
pool.closeAndJoin()
assertEquals(42, good.await())
assertThrows(IllegalArgumentException) { bad.await() }
```
## Bounded queue / back-pressure
When `maxQueueSize` is set, the producer suspends if the queue fills up, providing automatic back-pressure:
```lyng
// 1 worker, queue of 2 — producer can be at most 2 tasks ahead of what's running
val pool = LaunchPool(1, 2)
val d1 = pool.launch { delay(10); "a" }
val d2 = pool.launch { delay(10); "b" }
val d3 = pool.launch { delay(10); "c" } // suspends until d1 is picked up by the worker
pool.closeAndJoin()
```
## Collecting all results
`launch` returns a `Deferred`, so you can collect results with `joinAll()`:
```lyng
val pool = LaunchPool(4)
val jobs = (1..10).map { n -> pool.launch { n * n } }
pool.closeAndJoin()
val results = jobs.joinAll()
// results == [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
```
## Concurrency limit in practice
With `maxWorkers = 2`, at most 2 tasks run simultaneously regardless of how many are queued:
```lyng
val mu = Mutex()
var active = 0
var maxSeen = 0
val pool = LaunchPool(2)
(1..8).map {
pool.launch {
mu.withLock { active++; if (active > maxSeen) maxSeen = active }
delay(5)
mu.withLock { active-- }
}
}
pool.closeAndJoin()
assert(maxSeen <= 2)
```
## See also
- [parallelism.md](parallelism.md) — `launch`, `Deferred`, `Mutex`, `Channel`, and coroutine basics
- [Channel.md](Channel.md) — the underlying channel primitive used by `LaunchPool`
+7
View File
@@ -175,6 +175,8 @@ List could be sorted in place, just like [Collection] provide sorted copies, in
| `[Range]` | get slice of the array (copy) | Range |
| `+=` | append element(s) (2) | List or Obj |
| `List.fill(size, block)` | build a new list from indices `0..<size` | Int, Callable |
| `List.fill(size,capacity,block)` | same, pre-allocating capacity slots | Int, Int, Callable |
| `ensureCapacity(count)` | pre-allocate storage for at least `count` elements without reallocation (5) | Int |
| `sort()` | in-place sort, natural order | void |
| `sortBy(predicate)` | in-place sort bu `predicate` call result (3) | void |
| `sortWith(comparator)` | in-place sort using `comarator` function (4) | void |
@@ -197,6 +199,11 @@ order, e.g. is same as `list.sortWith { a,b -> predicate(a) <=> predicate(b) }`
positive if first is greater, and zero if they are equal. For example, the equvalent comparator
for `sort()` will be `sort { a, b -> a <=> b }
(5)
: if the current capacity is already ≥ `count`, this is a no-op. Otherwise the internal storage
is reallocated to hold at least `count` elements. Use this before a bulk `+=` loop to avoid
repeated reallocations. `List.fill(size, capacity, block)` calls this automatically.
It inherits from [Iterable] too and thus all iterable methods are applicable to any list.
## Observable list hooks
+1
View File
@@ -16,6 +16,7 @@ Primary sources used: `lynglib/src/commonMain/kotlin/net/sergeych/lyng/{Parser,T
## 2. Lexical Syntax
- Comments: `// line`, `/* block */`.
- Strings: `"..."` or `` `...` `` (supports escapes). Multiline string content is normalized by indentation logic.
- AI generation preference: use `"..."` by default, including multiline strings; `"` strings are also multiline-capable and should be preferred for ordinary code/doc/SQL text. Use backtick strings mainly when the content contains many double quotes and backticks would make the source clearer.
- Shared escapes: `\n`, `\r`, `\t`, `\\`, `\uXXXX` (4 hex digits).
- Delimiter escapes: `\"` inside `"..."`, ``\` `` inside `` `...` ``.
- Unicode escapes use exactly 4 hex digits (for example: `"\u0416"` -> `Ж`).
+2 -1
View File
@@ -18,6 +18,7 @@ Sources: `lynglib/src/commonMain/kotlin/net/sergeych/lyng/Script.kt`, `lynglib/s
- Async/concurrency: `launch`, `yield`, `flow`, `delay`.
- `Deferred.cancel()` cancels an active task.
- `Deferred.await()` throws `CancellationException` if that task was cancelled.
- `Iterable<Deferred>.joinAll()` awaits every deferred in iteration order and returns a `List` of results.
- Math: `floor`, `ceil`, `round`, `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `sinh`, `cosh`, `tanh`, `asinh`, `acosh`, `atanh`, `exp`, `ln`, `log10`, `log2`, `pow`, `sqrt`, `abs`, `clamp`.
- These helpers also accept `lyng.decimal.Decimal`.
- Exact Decimal path today: `abs`, `floor`, `ceil`, `round`, and `pow` with integral exponent.
@@ -81,7 +82,7 @@ Sources: `lynglib/src/commonMain/kotlin/net/sergeych/lyng/Script.kt`, `lynglib/s
- `import lyng.serialization`
- `Lynon` serialization utilities.
- `import lyng.time`
- `Instant`, `DateTime`, `Duration`, and module `delay`.
- `Instant`, `Date`, `DateTime`, `Duration`, and module `delay`.
## 6. Optional (lyngio) Modules
Requires installing `lyngio` into the import manager from host code.
+401
View File
@@ -0,0 +1,401 @@
### lyng.io.db — SQL database access for Lyng scripts
This module provides the portable SQL database contract for Lyng. The current shipped providers are SQLite via `lyng.io.db.sqlite` and a JVM-only JDBC bridge via `lyng.io.db.jdbc`.
> **Note:** `lyngio` is a separate library module. It must be explicitly added as a dependency to your host application and initialized in your Lyng scopes.
---
#### Install the module into a Lyng session
For SQLite-backed database access, install both the generic DB module and the SQLite provider:
```kotlin
import net.sergeych.lyng.EvalSession
import net.sergeych.lyng.Scope
import net.sergeych.lyng.io.db.createDbModule
import net.sergeych.lyng.io.db.sqlite.createSqliteModule
suspend fun bootstrapDb() {
val session = EvalSession()
val scope: Scope = session.getScope()
createDbModule(scope)
createSqliteModule(scope)
session.eval("""
import lyng.io.db
import lyng.io.db.sqlite
""".trimIndent())
}
```
`createSqliteModule(...)` also registers the `sqlite:` scheme for generic `openDatabase(...)`.
For JVM JDBC-backed access, install the JDBC provider as well:
```kotlin
import net.sergeych.lyng.EvalSession
import net.sergeych.lyng.Scope
import net.sergeych.lyng.io.db.createDbModule
import net.sergeych.lyng.io.db.jdbc.createJdbcModule
suspend fun bootstrapJdbc() {
val session = EvalSession()
val scope: Scope = session.getScope()
createDbModule(scope)
createJdbcModule(scope)
session.eval("""
import lyng.io.db
import lyng.io.db.jdbc
""".trimIndent())
}
```
`createJdbcModule(...)` registers `jdbc:`, `h2:`, `postgres:`, and `postgresql:` for `openDatabase(...)`.
---
#### Using from Lyng scripts
Typed SQLite open helper:
```lyng
import lyng.io.db.sqlite
val db = openSqlite(":memory:")
val userCount = db.transaction { tx ->
tx.execute("create table user(id integer primary key autoincrement, name text not null)")
tx.execute("insert into user(name) values(?)", "Ada")
tx.execute("insert into user(name) values(?)", "Linus")
tx.select("select count(*) as count from user").toList()[0]["count"]
}
assertEquals(2, userCount)
```
Generic provider-based open:
```lyng
import lyng.io.db
import lyng.io.db.sqlite
val db = openDatabase(
"sqlite:./app.db",
Map(
"foreignKeys" => true,
"busyTimeoutMillis" => 5000
)
)
```
JVM JDBC open with H2:
```lyng
import lyng.io.db.jdbc
val db = openH2("mem:demo;DB_CLOSE_DELAY=-1")
val names = db.transaction { tx ->
tx.execute("create table person(id bigint auto_increment primary key, name varchar(120) not null)")
tx.execute("insert into person(name) values(?)", "Ada")
tx.execute("insert into person(name) values(?)", "Linus")
tx.select("select name from person order by id").toList()
}
assertEquals("Ada", names[0]["name"])
assertEquals("Linus", names[1]["name"])
```
Generic JDBC open through `openDatabase(...)`:
```lyng
import lyng.io.db
import lyng.io.db.jdbc
val db = openDatabase(
"jdbc:h2:mem:demo2;DB_CLOSE_DELAY=-1",
Map()
)
val answer = db.transaction { tx ->
tx.select("select 42 as answer").toList()[0]["answer"]
}
assertEquals(42, answer)
```
PostgreSQL typed open:
```lyng
import lyng.io.db.jdbc
val db = openPostgres(
"jdbc:postgresql://127.0.0.1/appdb",
"appuser",
"secret"
)
val titles = db.transaction { tx ->
tx.execute("create table if not exists task(id bigserial primary key, title text not null)")
tx.execute("insert into task(title) values(?)", "Ship JDBC provider")
tx.execute("insert into task(title) values(?)", "Test PostgreSQL path")
tx.select("select title from task order by id").toList()
}
assertEquals("Ship JDBC provider", titles[0]["title"])
```
Nested transactions use real savepoint semantics:
```lyng
import lyng.io.db
import lyng.io.db.sqlite
val db = openSqlite(":memory:")
db.transaction { tx ->
tx.execute("create table item(id integer primary key autoincrement, name text not null)")
tx.execute("insert into item(name) values(?)", "outer")
try {
tx.transaction { inner ->
inner.execute("insert into item(name) values(?)", "inner")
throw IllegalStateException("rollback nested")
}
} catch (_: IllegalStateException) {
}
assertEquals(1, tx.select("select count(*) as count from item").toList()[0]["count"])
}
```
Intentional rollback without treating it as a backend failure:
```lyng
import lyng.io.db
import lyng.io.db.sqlite
val db = openSqlite(":memory:")
assertThrows(RollbackException) {
db.transaction { tx ->
tx.execute("create table item(id integer primary key autoincrement, name text not null)")
tx.execute("insert into item(name) values(?)", "temporary")
throw RollbackException("stop here")
}
}
```
---
#### Portable API
##### `Database`
- `transaction(block)` — opens a transaction, commits on normal exit, rolls back on uncaught failure.
##### `SqlTransaction`
- `select(clause, params...)` — execute a statement whose primary result is a row set.
- `execute(clause, params...)` — execute a side-effect statement and return `ExecutionResult`.
- `transaction(block)` — nested transaction with real savepoint semantics.
##### `ResultSet`
- `columns` — positional `SqlColumn` metadata, available before iteration.
- `size()` — result row count.
- `isEmpty()` — fast emptiness check where possible.
- `iterator()` — normal row iteration while the transaction is active.
- `toList()` — materialize detached `SqlRow` snapshots that may be used after the transaction ends.
##### `SqlRow`
- `row[index]` — zero-based positional access.
- `row["columnName"]` — case-insensitive lookup by output column label.
Name-based access fails with `SqlUsageException` if the name is missing or ambiguous.
##### `ExecutionResult`
- `affectedRowsCount`
- `getGeneratedKeys()`
Statements that return rows directly, such as `... returning ...`, should use `select(...)`, not `execute(...)`.
---
#### Value mapping
Portable bind values:
- `null`
- `Bool`
- `Int`, `Double`, `Decimal`
- `String`
- `Buffer`
- `Date`, `DateTime`, `Instant`
Unsupported parameter values fail with `SqlUsageException`.
Portable result metadata categories:
- `Binary`
- `String`
- `Int`
- `Double`
- `Decimal`
- `Bool`
- `Date`
- `DateTime`
- `Instant`
For temporal types, see [time functions](time.md).
---
#### SQLite provider
`lyng.io.db.sqlite` currently provides the first concrete backend.
Typed helper:
```lyng
openSqlite(
path: String,
readOnly: Bool = false,
createIfMissing: Bool = true,
foreignKeys: Bool = true,
busyTimeoutMillis: Int = 5000
): Database
```
Accepted generic URL forms:
- `sqlite::memory:`
- `sqlite:relative/path.db`
- `sqlite:/absolute/path.db`
Supported `openDatabase(..., extraParams)` keys for SQLite:
- `readOnly: Bool`
- `createIfMissing: Bool`
- `foreignKeys: Bool`
- `busyTimeoutMillis: Int`
SQLite write/read policy in v1:
- `Bool` writes as `0` / `1`
- `Decimal` writes as canonical text
- `Date` writes as `YYYY-MM-DD`
- `DateTime` writes as ISO local timestamp text without timezone
- `Instant` writes as ISO UTC timestamp text with explicit timezone marker
- `TIME*` values stay `String`
- `TIMESTAMP` / `DATETIME` reject timezone-bearing stored text
Open-time validation failures:
- malformed URL or bad option shape -> `IllegalArgumentException`
- runtime open failure -> `DatabaseException`
#### JDBC provider
`lyng.io.db.jdbc` is currently implemented on the JVM target only. The `lyngio-jvm` artifact bundles and explicitly loads these JDBC drivers:
- SQLite
- H2
- PostgreSQL
Typed helpers:
```lyng
openJdbc(
connectionUrl: String,
user: String? = null,
password: String? = null,
driverClass: String? = null,
properties: Map<String, Object?>? = null
): Database
openH2(
connectionUrl: String,
user: String? = null,
password: String? = null,
properties: Map<String, Object?>? = null
): Database
openPostgres(
connectionUrl: String,
user: String? = null,
password: String? = null,
properties: Map<String, Object?>? = null
): Database
```
Accepted generic URL forms:
- `jdbc:h2:mem:test;DB_CLOSE_DELAY=-1`
- `h2:mem:test;DB_CLOSE_DELAY=-1`
- `jdbc:postgresql://localhost/app`
- `postgres://localhost/app`
- `postgresql://localhost/app`
Supported `openDatabase(..., extraParams)` keys for JDBC:
- `driverClass: String`
- `user: String`
- `password: String`
- `properties: Map<String, Object?>`
Behavior notes for the JDBC bridge:
- the portable `Database` / `SqlTransaction` API stays the same as for SQLite
- nested transactions use JDBC savepoints
- JDBC connection properties are built from `user`, `password`, and `properties`
- `properties` values are stringified before being passed to JDBC
- statements with row-returning clauses still must use `select(...)`, not `execute(...)`
Platform support for this provider:
- `lyng.io.db.jdbc` — JVM only
- `openH2(...)` — works out of the box with `lyngio-jvm`
- `openPostgres(...)` — driver included, but an actual PostgreSQL server is still required
PostgreSQL-specific notes:
- `openPostgres(...)` accepts either a full JDBC URL or shorthand forms such as `//localhost/app`
- local peer/trust setups may use an empty password string
- generated keys work with PostgreSQL `bigserial` / identity columns through `ExecutionResult.getGeneratedKeys()`
- for reproducible automated tests, prefer a disposable PostgreSQL instance such as Docker/Testcontainers instead of a long-lived shared server
---
#### Lifetime rules
`ResultSet` is valid only while its owning transaction is active.
`SqlRow` values are detached snapshots once materialized, so this pattern is valid:
```lyng
val rows = db.transaction { tx ->
tx.select("select name from person order by id").toList()
}
assertEquals("Ada", rows[0]["name"])
```
This means:
- do not keep `ResultSet` objects after the transaction block returns
- materialize rows with `toList()` inside the transaction when they must outlive it
The same rule applies to generated keys from `ExecutionResult.getGeneratedKeys()`: the `ResultSet` is transaction-scoped, but rows returned by `toList()` are detached.
---
#### Platform support
- `lyng.io.db` — generic contract, available when host code installs it
- `lyng.io.db.sqlite` — implemented on JVM and Linux Native in the current release tree
- `lyng.io.db.jdbc` — implemented on JVM in the current release tree
For the broader I/O overview, see [lyngio overview](lyngio.md).
+24 -10
View File
@@ -12,6 +12,7 @@
#### Included Modules
- **[lyng.io.db](lyng.io.db.md):** Portable SQL database access. Provides `Database`, `SqlTransaction`, `ResultSet`, SQLite support through `lyng.io.db.sqlite`, and JVM JDBC support through `lyng.io.db.jdbc`.
- **[lyng.io.fs](lyng.io.fs.md):** Async filesystem access. Provides the `Path` class for file/directory operations, streaming, and globbing.
- **[lyng.io.process](lyng.io.process.md):** External process execution and shell commands. Provides `Process`, `RunningProcess`, and `Platform` information.
- **[lyng.io.console](lyng.io.console.md):** Rich console/TTY access. Provides `Console` capability detection, geometry, output, and iterable events.
@@ -43,6 +44,9 @@ To use `lyngio` modules in your scripts, you must install them into your Lyng sc
```kotlin
import net.sergeych.lyng.EvalSession
import net.sergeych.lyng.io.db.createDbModule
import net.sergeych.lyng.io.db.jdbc.createJdbcModule
import net.sergeych.lyng.io.db.sqlite.createSqliteModule
import net.sergeych.lyng.io.fs.createFs
import net.sergeych.lyng.io.process.createProcessModule
import net.sergeych.lyng.io.console.createConsoleModule
@@ -61,6 +65,9 @@ suspend fun runMyScript() {
val scope = session.getScope()
// Install modules with policies
createDbModule(scope)
createJdbcModule(scope)
createSqliteModule(scope)
createFs(PermitAllAccessPolicy, scope)
createProcessModule(PermitAllProcessAccessPolicy, scope)
createConsoleModule(PermitAllConsoleAccessPolicy, scope)
@@ -70,6 +77,9 @@ suspend fun runMyScript() {
// Now scripts can import them
session.eval("""
import lyng.io.db
import lyng.io.db.jdbc
import lyng.io.db.sqlite
import lyng.io.fs
import lyng.io.process
import lyng.io.console
@@ -77,6 +87,8 @@ suspend fun runMyScript() {
import lyng.io.net
import lyng.io.ws
println("H2 JDBC available: " + (openH2("mem:demo;DB_CLOSE_DELAY=-1") != null))
println("SQLite available: " + (openSqlite(":memory:") != null))
println("Working dir: " + Path(".").readUtf8())
println("OS: " + Platform.details().name)
println("TTY: " + Console.isTty())
@@ -94,6 +106,7 @@ suspend fun runMyScript() {
`lyngio` is built with a "Secure by Default" philosophy. Every I/O or process operation is checked against a policy.
- **Filesystem Security:** Implement `FsAccessPolicy` to restrict access to specific paths or operations (e.g., read-only access to a sandbox directory).
- **Database Installation:** Database access is still explicit-capability style. The host must install `lyng.io.db` and at least one provider such as `lyng.io.db.sqlite` or `lyng.io.db.jdbc`; otherwise scripts cannot open databases.
- **Process Security:** Implement `ProcessAccessPolicy` to restrict which executables can be run or to disable shell execution entirely.
- **Console Security:** Implement `ConsoleAccessPolicy` to control output writes, event reads, and raw mode switching.
- **HTTP Security:** Implement `HttpAccessPolicy` to restrict which requests scripts may send.
@@ -101,6 +114,7 @@ suspend fun runMyScript() {
- **WebSocket Security:** Implement `WsAccessPolicy` to restrict websocket connects and message flow.
For more details, see the specific module documentation:
- [Database Module Details](lyng.io.db.md)
- [Filesystem Security Details](lyng.io.fs.md#access-policy-security)
- [Process Security Details](lyng.io.process.md#security-policy)
- [Console Module Details](lyng.io.console.md)
@@ -112,16 +126,16 @@ For more details, see the specific module documentation:
#### Platform Support Overview
| Platform | lyng.io.fs | lyng.io.process | lyng.io.console | lyng.io.http | lyng.io.ws | lyng.io.net |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
| **JVM** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| **Linux Native** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| **Apple Native** | ✅ | ✅ | ✅ | ✅ | ✅ | ⚠️ |
| **Windows Native** | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ |
| **Android** | ✅ | ❌ | ⚠️ | ✅ | ✅ | ✅ |
| **JS / Node** | ✅ | ❌ | ⚠️ | ✅ | ✅ | ✅ |
| **JS / Browser** | ✅ (in-memory) | ❌ | ⚠️ | ✅ | ✅ | ❌ |
| **Wasm** | ✅ (in-memory) | ❌ | ⚠️ | ❌ | ❌ | ❌ |
| Platform | lyng.io.db/sqlite | lyng.io.db/jdbc | lyng.io.fs | lyng.io.process | lyng.io.console | lyng.io.http | lyng.io.ws | lyng.io.net |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **JVM** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| **Linux Native** | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| **Apple Native** | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ⚠️ |
| **Windows Native** | ❌ | ❌ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ |
| **Android** | ⚠️ | ❌ | ✅ | ❌ | ⚠️ | ✅ | ✅ | ✅ |
| **JS / Node** | ❌ | ❌ | ✅ | ❌ | ⚠️ | ✅ | ✅ | ✅ |
| **JS / Browser** | ❌ | ❌ | ✅ (in-memory) | ❌ | ⚠️ | ✅ | ✅ | ❌ |
| **Wasm** | ❌ | ❌ | ✅ (in-memory) | ❌ | ⚠️ | ❌ | ❌ | ❌ |
Legend:
- `✅` supported
+78
View File
@@ -36,6 +36,17 @@ This example shows how to launch a coroutine with `launch` which returns [Deferr
Launch has the only argument which should be a callable (lambda usually) that is run in parallel (or cooperatively in parallel), and return anything as the result.
When you have an iterable of deferreds, use `joinAll()` to await all of them and collect results in input order:
val jobs = (1..4).map { n ->
launch {
delay(1)
n * 10
}
}
assertEquals([10, 20, 30, 40], jobs.joinAll())
>>> void
If you no longer need the result, cancel the deferred. Awaiting a cancelled deferred throws `CancellationException`:
var reached = false
@@ -219,6 +230,73 @@ Flows allow easy transforming of any [Iterable]. See how the standard Lyng libra
}
}
## Channel
A [Channel] is a **hot pipe** between coroutines: values are pushed in by a producer and pulled out by a consumer, with each value consumed exactly once.
Unlike a `Flow` (which is cold and re-runs its generator on every collection), a `Channel` is stateful — the right tool for classic _producer / consumer_ work.
val ch = Channel() // rendezvous: sender waits for receiver
val producer = launch {
for (i in 1..5) ch.send(i)
ch.close() // signal: no more values
}
var item = ch.receive() // suspends until a value is ready
while (item != null) {
println(item)
item = ch.receive()
}
// prints 1 2 3 4 5
`receive()` returns `null` when the channel is both closed and fully drained — that is the idiomatic loop termination condition.
Channels can also be buffered so the producer can run ahead:
val ch = Channel(4) // buffer up to 4 items without blocking
ch.send(10)
ch.send(20)
ch.send(30)
ch.close()
assertEquals(10, ch.receive())
assertEquals(20, ch.receive())
assertEquals(30, ch.receive())
assertEquals(null, ch.receive()) // drained
For the full API — including `tryReceive`, `Channel.UNLIMITED`, and the fan-out / ping-pong patterns — see the [Channel] reference page.
## LaunchPool
When you need **bounded concurrency** — run at most *N* tasks at the same time without spawning a new coroutine per task — use [LaunchPool]:
```lyng
val pool = LaunchPool(4) // at most 4 tasks run in parallel
val jobs = (1..20).map { n ->
pool.launch { expensiveCompute(n) }
}
pool.closeAndJoin() // wait for all tasks to complete
val results = jobs.joinAll()
```
Exceptions thrown inside a submitted lambda are captured in the returned `Deferred` and do not crash the pool, so other tasks continue running normally.
See [LaunchPool] for the full API including bounded queues and cancellation.
[LaunchPool]: LaunchPool.md
| | Flow | Channel |
|---|---|---|
| **temperature** | cold (lazy) | hot (eager) |
| **replay** | every collector gets a fresh run | each item consumed once |
| **consumers** | any number, each gets all items | one receiver per item |
| **typical use** | transform pipelines, sequences | producer–consumer, fan-out |
[Channel]: Channel.md
[Iterable]: Iterable.md
+122 -63
View File
@@ -1,74 +1,135 @@
# Lyng time functions
Lyng date and time support requires importing `lyng.time` packages. Lyng uses simple yet modern time object models:
Lyng date and time support requires importing `lyng.time`. The module provides four related types:
- `Instant` class for absolute time stamps with platform-dependent resolution.
- `DateTime` class for calendar-aware points in time within a specific time zone.
- `Duration` to represent amount of time not depending on the calendar (e.g., milliseconds, seconds).
- `Instant` for absolute timestamps.
- `Date` for calendar dates without time-of-day or timezone.
- `DateTime` for calendar-aware points in time in a specific timezone.
- `Duration` for absolute elapsed time.
## Time instant: `Instant`
Represent some moment of time not depending on the calendar. It is similar to `TIMESTAMP` in SQL or `Instant` in Kotlin.
`Instant` represents some moment of time independently of the calendar. It is similar to SQL `TIMESTAMP`
or Kotlin `Instant`.
### Constructing and converting
import lyng.time
// default constructor returns time now:
val t1 = Instant()
// constructing from a number is treated as seconds since unix epoch:
val t2 = Instant(1704110400) // 2024-01-01T12:00:00Z
// from RFC3339 string:
val t2 = Instant(1704110400)
val t3 = Instant("2024-01-01T12:00:00.123456Z")
// truncation:
val t4 = t3.truncateToMinute
assertEquals(t4.toRFC3339(), "2024-01-01T12:00:00Z")
// to localized DateTime (uses system default TZ if not specified):
val t4 = t3.truncateToMinute()
assertEquals("2024-01-01T12:00:00Z", t4.toRFC3339())
val dt = t3.toDateTime("+02:00")
assertEquals(dt.hour, 14)
assertEquals(14, dt.hour)
val d = t3.toDate("Z")
assertEquals(Date(2024, 1, 1), d)
### Instant members
| member | description |
|--------------------------------|---------------------------------------------------------|
| epochSeconds: Real | positive or negative offset in seconds since Unix epoch |
| epochWholeSeconds: Int | same, but in _whole seconds_. Slightly faster |
| nanosecondsOfSecond: Int | offset from epochWholeSeconds in nanos |
| isDistantFuture: Bool | true if it `Instant.distantFuture` |
| isDistantPast: Bool | true if it `Instant.distantPast` |
| truncateToMinute: Instant | create new instance truncated to minute |
| truncateToSecond: Instant | create new instance truncated to second |
| truncateToMillisecond: Instant | truncate new instance to millisecond |
| truncateToMicrosecond: Instant | truncate new instance to microsecond |
| toRFC3339(): String | format as RFC3339 string (UTC) |
| toDateTime(tz?): DateTime | localize to a TimeZone (ID string or offset seconds) |
| member | description |
|--------------------------------|------------------------------------------------------|
| epochSeconds: Real | offset in seconds since Unix epoch |
| epochWholeSeconds: Int | whole seconds since Unix epoch |
| nanosecondsOfSecond: Int | nanoseconds within the current second |
| isDistantFuture: Bool | true if it is `Instant.distantFuture` |
| isDistantPast: Bool | true if it is `Instant.distantPast` |
| truncateToMinute(): Instant | truncate to minute precision |
| truncateToSecond(): Instant | truncate to second precision |
| truncateToMillisecond(): Instant | truncate to millisecond precision |
| truncateToMicrosecond(): Instant | truncate to microsecond precision |
| toRFC3339(): String | format as RFC3339 string in UTC |
| toDateTime(tz?): DateTime | localize to a timezone |
| toDate(tz?): Date | convert to a calendar date in a timezone |
## Calendar time: `DateTime`
## Calendar date: `Date`
`DateTime` represents a point in time in a specific timezone. It provides access to calendar components like year,
month, and day.
`Date` represents a pure calendar date. It has no time-of-day and no attached timezone. Use it for values
like birthdays, due dates, invoice dates, and SQL `DATE` columns.
### Constructing
import lyng.time
val today = Date()
val d1 = Date(2026, 4, 15)
val d2 = Date("2024-02-29")
val d3 = Date.parseIso("2024-02-29")
val d4 = Date(DateTime(2024, 5, 20, 15, 30, 45, "+02:00"))
val d5 = Date(Instant("2024-01-01T23:30:00Z"), "+02:00")
### Date members
| member | description |
|--------------------------------|------------------------------------------------------------|
| year: Int | year component |
| month: Int | month component (1..12) |
| day: Int | day of month (alias `dayOfMonth`) |
| dayOfWeek: Int | day of week (1=Monday, 7=Sunday) |
| dayOfYear: Int | day of year (1..365/366) |
| isLeapYear: Bool | whether this date is in a leap year |
| lengthOfMonth: Int | number of days in this month |
| lengthOfYear: Int | 365 or 366 |
| toIsoString(): String | ISO `YYYY-MM-DD` string |
| toSortableString(): String | alias to `toIsoString()` |
| toDateTime(tz="Z"): DateTime | start-of-day `DateTime` in the specified timezone |
| atStartOfDay(tz="Z"): DateTime | alias to `toDateTime()` |
| addDays(n): Date | add or subtract calendar days |
| addMonths(n): Date | add or subtract months, normalizing end-of-month |
| addYears(n): Date | add or subtract years |
| daysUntil(other): Int | calendar days until `other` |
| daysSince(other): Int | calendar days since `other` |
| static today(tz?): Date | today in the specified timezone |
| static parseIso(s): Date | parse ISO `YYYY-MM-DD` |
### Date arithmetic
`Date` supports only whole-day arithmetic. This is deliberate: calendar dates should not silently accept
sub-day durations.
import lyng.time
val d1 = Date(2026, 4, 15)
val d2 = d1.addDays(10)
assertEquals(Date(2026, 4, 25), d2)
assertEquals(Date(2026, 4, 18), d1 + 3.days)
assertEquals(Date(2026, 4, 12), d1 - 3.days)
assertEquals(10, d1.daysUntil(d2))
assertEquals(10, d2.daysSince(d1))
assertEquals(10, d2 - d1)
### Date conversions
import lyng.time
val i = Instant("2024-01-01T23:30:00Z")
assertEquals(Date(2024, 1, 1), i.toDate("Z"))
assertEquals(Date(2024, 1, 2), i.toDate("+02:00"))
val dt = DateTime(2024, 5, 20, 15, 30, 45, "+02:00")
assertEquals(Date(2024, 5, 20), dt.date)
assertEquals(Date(2024, 5, 20), dt.toDate())
assertEquals(DateTime(2024, 5, 20, 0, 0, 0, "Z"), Date(2024, 5, 20).toDateTime("Z"))
assertEquals(DateTime(2024, 5, 20, 0, 0, 0, "+02:00"), Date(2024, 5, 20).atStartOfDay("+02:00"))
## Calendar time: `DateTime`
`DateTime` represents a point in time in a specific timezone. It provides access to calendar components
such as year, month, day, and hour.
### Constructing
import lyng.time
// Current time in system default timezone
val now = DateTime.now()
// Specific timezone
val offsetTime = DateTime.now("+02:00")
// From Instant
val dt = Instant().toDateTime("Z")
// By components (year, month, day, hour=0, minute=0, second=0, timeZone="UTC")
val dt2 = DateTime(2024, 1, 1, 12, 0, 0, "Z")
// From RFC3339 string
val dt3 = DateTime.parseRFC3339("2024-01-01T12:00:00+02:00")
### DateTime members
@@ -83,7 +144,9 @@ month, and day.
| second: Int | second component (0..59) |
| dayOfWeek: Int | day of week (1=Monday, 7=Sunday) |
| timeZone: String | timezone ID string |
| date: Date | calendar date component |
| toInstant(): Instant | convert back to absolute Instant |
| toDate(): Date | extract the calendar date in this timezone |
| toUTC(): DateTime | shortcut to convert to UTC |
| toTimeZone(tz): DateTime | convert to another timezone |
| addMonths(n): DateTime | add/subtract months (normalizes end of month) |
@@ -96,28 +159,27 @@ month, and day.
`DateTime` handles calendar arithmetic correctly:
import lyng.time
val leapDay = Instant("2024-02-29T12:00:00Z").toDateTime("Z")
val nextYear = leapDay.addYears(1)
assertEquals(nextYear.day, 28) // Feb 29, 2024 -> Feb 28, 2025
assertEquals(28, nextYear.day)
# `Duration` class
Represent absolute time distance between two `Instant`.
`Duration` represents absolute elapsed time between two instants.
import lyng.time
val t1 = Instant()
// yes we can delay to period, and it is not blocking. is suspends!
delay(1.millisecond)
val t2 = Instant()
// be suspend, so actual time may vary:
assert( t2 - t1 >= 1.millisecond)
assert( t2 - t1 < 100.millisecond)
assert(t2 - t1 >= 1.millisecond)
assert(t2 - t1 < 100.millisecond)
>>> void
Duration can be converted from numbers, like `5.minutes` and so on. Extensions are created for
`Int` and `Real`, so for n as Real or Int it is possible to create durations::
Duration values can be created from numbers using extensions on `Int` and `Real`:
- `n.millisecond`, `n.milliseconds`
- `n.second`, `n.seconds`
@@ -125,10 +187,9 @@ Duration can be converted from numbers, like `5.minutes` and so on. Extensions a
- `n.hour`, `n.hours`
- `n.day`, `n.days`
The bigger time units like months or years are calendar-dependent and can't be used with `Duration`.
Larger units like months or years are calendar-dependent and are intentionally not part of `Duration`.
Each duration instance can be converted to number of any of these time units, as `Real` number, if `d` is a `Duration`
instance:
Each duration instance can be converted to numbers in these units:
- `d.microseconds`
- `d.milliseconds`
@@ -137,18 +198,16 @@ instance:
- `d.hours`
- `d.days`
for example
Example:
import lyng.time
assertEquals( 60, 1.minute.seconds )
assertEquals( 10.milliseconds, 0.01.seconds )
assertEquals(60, 1.minute.seconds)
assertEquals(10.milliseconds, 0.01.seconds)
>>> void
# Utility functions
## delay(duration: Duration)
Suspends current coroutine for at least the specified duration.
## `delay(duration: Duration)`
Suspends the current coroutine for at least the specified duration.
+13 -9
View File
@@ -1,25 +1,29 @@
# What's New in Lyng
This document highlights the current Lyng release, **1.5.4**, and the broader additions from the 1.5 cycle.
This document highlights the current Lyng release, **1.5.5**, and the broader additions from the 1.5 cycle.
It is intentionally user-facing: new language features, new modules, new tools, and the practical things you can build with them.
For a programmer-focused migration summary across 1.5.x, see `docs/whats_new_1_5.md`.
## Release 1.5.4 Highlights
## Release 1.5.5 Highlights
- `1.5.4` is the stabilization release for the 1.5 feature set.
- `1.5.5` extends the 1.5 line with practical database APIs, first-class calendar dates, and better coroutine building blocks.
- The 1.5 line now brings together richer ranges and loops, interpolation, math modules, immutable and observable collections, richer `lyngio`, and much better CLI/IDE support.
- `1.5.4` specifically fixes user-visible issues around decimal arithmetic, mixed numeric flows, list behavior, and observable list hooks.
- `1.5.4` also fixes extension-member registration for named singleton `object` declarations, so `fun X.foo()` and `val X.bar` now work as expected.
- `1.5.4` also lets named singleton `object` declarations use scoped indexer extensions with bracket syntax, so patterns like `Storage["name"]` can be implemented with `override fun Storage.getAt(...)` / `putAt(...)`.
- `1.5.5` adds `Channel`, `LaunchPool`, and `joinAll()` so coroutine-heavy scripts can coordinate work more directly.
- `1.5.5` adds `Date`, the portable `lyng.io.db` layer, SQLite/JDBC providers, and a compatibility `lyng.legacy_digest` module.
- `1.5.5` also continues runtime/compiler hardening with better import dispatch, faster exact lambda calls, and correct `val +=`/`-=` behavior for mutating types versus real reassignment.
- The docs, homepage samples, and release metadata now point at the current stable version.
## User Highlights Across 1.5.x
- Descending ranges and loops with `downTo` / `downUntil`
- String interpolation with `$name` and `${expr}`
- Backtick string literals for raw-ish string text
- Decimal arithmetic, matrices/vectors, and complex numbers
- Calendar `Date` support in `lyng.time`
- `Channel`, `LaunchPool`, and `joinAll()` for coroutine workflows
- Immutable collections and opt-in `ObservableList`
- Rich `lyngio` modules for console, HTTP, WebSocket, TCP, and UDP
- Rich `lyngio` modules for SQLite/JDBC databases, console, HTTP, WebSocket, TCP, and UDP
- Legacy SHA-1 compatibility helpers in `lyng.legacy_digest`
- CLI improvements including the built-in formatter `lyng fmt`
- Better IDE support and stronger docs around the released feature set
@@ -323,7 +327,7 @@ Singleton objects are declared using the `object` keyword. They provide a conven
```lyng
object Config {
val version = "1.5.4"
val version = "1.5.5"
fun show() = println("Config version: " + version)
}
@@ -579,7 +583,7 @@ ws.close()
These modules are capability-gated and host-installed, keeping Lyng safe by default while making networked scripts practical when enabled.
See [lyngio overview](lyngio.md), [lyng.io.http](lyng.io.http.md), [lyng.io.ws](lyng.io.ws.md), and [lyng.io.net](lyng.io.net.md).
See [lyngio overview](lyngio.md), [lyng.io.db](lyng.io.db.md), [lyng.io.http](lyng.io.http.md), [lyng.io.ws](lyng.io.ws.md), and [lyng.io.net](lyng.io.net.md).
### CLI: Formatting Command
A new `fmt` subcommand has been added to the Lyng CLI.
+325
View File
@@ -0,0 +1,325 @@
#!/usr/bin/env lyng
import lyng.io.db
import lyng.io.db.sqlite
import lyng.io.fs
val DB_FILE_NAME = "contents.db"
val ANSI_ESC = "\u001b["
val NEWLINE = "\n"
val WINDOWS_SEPARATOR = "\\"
val SQLITE_JOURNAL_SUFFIXES = ["-wal", "-shm", "-journal"]
val USAGE_TEXT = "
Lyng content index
Scan a directory tree, diff it against a SQLite snapshot, and optionally refresh the snapshot.
usage:
lyng examples/content_index_db.lyng <root> [-u|--update]
options:
-u, --update write the current scan back to $DB_FILE_NAME
notes:
- the database lives inside <root>/$DB_FILE_NAME
- on first run the snapshot is created automatically
- the script ignores its own SQLite sidecar files
"
val CREATE_FILE_INDEX_SQL = "
create table if not exists file_index(
path text primary key not null,
size integer not null,
mtime integer not null
)
"
val CREATE_CURRENT_SCAN_SQL = "
create temp table current_scan(
path text primary key not null,
size integer not null,
mtime integer not null
)
"
val SELECT_ADDED_SQL = "
select
c.path,
c.size,
c.mtime
from current_scan c
left join file_index f on f.path = c.path
where f.path is null
order by c.path
"
val SELECT_REMOVED_SQL = "
select f.path, f.size, f.mtime
from file_index f
left join current_scan c on c.path = f.path
where c.path is null
order by f.path
"
val SELECT_CHANGED_SQL = "
select c.path, f.size as old_size, c.size as new_size, f.mtime as old_mtime,
c.mtime as new_mtime
from current_scan c
join file_index f on f.path = c.path
where c.size != f.size or c.mtime != f.mtime
order by c.path
"
val DELETE_MISSING_SQL = "
delete from file_index
where not exists (
select 1
from current_scan c
where c.path = file_index.path
)
"
val UPSERT_SCAN_SQL = "
insert or replace into file_index(path, size, mtime)
select path, size, mtime
from current_scan
"
val INSERT_SCAN_ROW_SQL = "
insert into current_scan(path, size, mtime)
values(?, ?, ?)
"
val USE_COLOR = true
class CliOptions(val rootText: String, val updateSnapshot: Bool) {}
fun out(text: String? = null): Void {
if (text == null) {
print(NEWLINE)
return
}
print(text + NEWLINE)
}
fun paint(code: String, text: String): String {
if (!USE_COLOR) return text
ANSI_ESC + code + "m" + text + ANSI_ESC + "0m"
}
fun bold(text: String): String = paint("1", text)
fun dim(text: String): String = paint("2", text)
fun cyan(text: String): String = paint("36", text)
fun green(text: String): String = paint("32", text)
fun yellow(text: String): String = paint("33", text)
fun red(text: String): String = paint("31", text)
fun signed(value: Int): String = if (value > 0) "+" + value else value.toString()
fun plural(count: Int, one: String, many: String): String {
if (count == 1) return one
many
}
fun childPath(parent: Path, name: String): Path {
val base = parent.toString()
if (base.endsWith("/") || base.endsWith(WINDOWS_SEPARATOR)) {
return Path(base + name)
}
Path(base + "/" + name)
}
fun relativePath(root: Path, file: Path): String {
val parts: List<String> = []
for (i in root.segments.size..<file.segments.size) {
parts.add(file.segments[i] as String)
}
parts.joinToString("/")
}
fun isDatabaseArtifact(relative: String): Bool {
relative == DB_FILE_NAME || SQLITE_JOURNAL_SUFFIXES.any { relative == DB_FILE_NAME + (it as String) }
}
fun printUsage(message: String? = null): Void {
if (message != null && message.trim().isNotEmpty()) {
out(red("error: ") + message)
out()
}
out(bold(USAGE_TEXT))
}
fun parseArgs(argv: List<String>): CliOptions? {
var rootText: String? = null
var updateSnapshot = false
for (arg in argv) {
when (arg) {
"-u", "--update" -> updateSnapshot = true
"-h", "--help" -> {
printUsage()
return null
}
else -> {
if (arg.startsWith("-")) {
printUsage("unknown option: " + arg)
return null
}
if (rootText != null) {
printUsage("only one root path is allowed")
return null
}
rootText = arg
}
}
}
if (rootText == null) {
printUsage("missing required <root> argument")
return null
}
CliOptions(rootText as String, updateSnapshot)
}
fun printBanner(root: Path, dbFile: Path, dbWasCreated: Bool, updateSnapshot: Bool): Void {
val mode =
if (dbWasCreated) "bootstrap snapshot"
else if (updateSnapshot) "scan + refresh snapshot"
else "scan only"
out(cyan("== Lyng content index =="))
out(dim("root: " + root))
out(dim("db: " + dbFile))
out(dim("mode: " + mode))
out()
}
fun printSection(title: String, accent: (String)->String, rows: List<SqlRow>, render: (SqlRow)->String): Void {
out(accent(title + " (" + rows.size + ")"))
if (rows.isEmpty()) {
out(dim(" none"))
out()
return
}
for (row in rows) {
out(render(row))
}
out()
}
fun renderAdded(row: SqlRow): String {
val path = row["path"] as String
val size = row["size"] as Int
val mtime = row["mtime"] as Int
" " + green("+") + " " + bold(path) + dim(" %12d B mtime %d"(size, mtime))
}
fun renderRemoved(row: SqlRow): String {
val path = row["path"] as String
val size = row["size"] as Int
val mtime = row["mtime"] as Int
" " + red("-") + " " + bold(path) + dim(" %12d B mtime %d"(size, mtime))
}
fun renderChanged(row: SqlRow): String {
val path = row["path"] as String
val oldSize = row["old_size"] as Int
val newSize = row["new_size"] as Int
val oldMtime = row["old_mtime"] as Int
val newMtime = row["new_mtime"] as Int
val sizeDelta = newSize - oldSize
val mtimeDelta = newMtime - oldMtime
" " + yellow("~") + " " + bold(path) +
dim(
" size %d -> %d (%s B), mtime %d -> %d (%s ms)"(
oldSize,
newSize,
signed(sizeDelta),
oldMtime,
newMtime,
signed(mtimeDelta)
)
)
}
fun loadRows(tx: SqlTransaction, query: String): List<SqlRow> = tx.select(query).toList()
fun main() {
val argv: List<String> = []
for (raw in ARGV as List) {
argv.add(raw as String)
}
val options = parseArgs(argv)
if (options == null) {
return
}
val root = Path(options.rootText)
if (!root.exists()) {
printUsage("root does not exist: " + root)
return
}
if (!root.isDirectory()) {
printUsage("root is not a directory: " + root)
return
}
val dbFile = childPath(root, DB_FILE_NAME)
val dbWasCreated = !dbFile.exists()
val shouldUpdateSnapshot = dbWasCreated || options.updateSnapshot
printBanner(root, dbFile, dbWasCreated, shouldUpdateSnapshot)
val db = openSqlite(dbFile.toString())
db.transaction { tx ->
tx.execute(CREATE_FILE_INDEX_SQL)
tx.execute("drop table if exists temp.current_scan")
tx.execute(CREATE_CURRENT_SCAN_SQL)
var scannedFiles = 0
for (rawEntry in root.glob("**")) {
val entry = rawEntry as Path
if (!entry.isFile()) continue
val relative = relativePath(root, entry)
if (isDatabaseArtifact(relative)) continue
val size = entry.size() ?: 0
val mtime = entry.modifiedAtMillis() ?: 0
tx.execute(INSERT_SCAN_ROW_SQL, relative, size, mtime)
scannedFiles++
}
val added = loadRows(tx, SELECT_ADDED_SQL)
val removed = loadRows(tx, SELECT_REMOVED_SQL)
val changed = loadRows(tx, SELECT_CHANGED_SQL)
val totalChanges = added.size + removed.size + changed.size
out(dim("scanned %d %s under %s"(scannedFiles, plural(scannedFiles, "file", "files"), root.toString())))
out(dim("detected %d %s"(totalChanges, plural(totalChanges, "change", "changes"))))
out()
printSection("Added", { green(it) }, added) { renderAdded(it) }
printSection("Removed", { red(it) }, removed) { renderRemoved(it) }
printSection("Changed", { yellow(it) }, changed) { renderChanged(it) }
if (shouldUpdateSnapshot) {
tx.execute(DELETE_MISSING_SQL)
tx.execute(UPSERT_SCAN_SQL)
val action = if (dbWasCreated) "created" else "updated"
out(cyan("snapshot " + action + " in " + dbFile.name))
} else {
out(dim("snapshot unchanged; re-run with -u or --update to persist the scan"))
}
}
}
main()
View File
Whitespace-only changes.
+15
View File
@@ -0,0 +1,15 @@
import lyng.time
val n = 700_000
fun tm<T>(block: ()->T): T {
val t = Instant()
block().also {
println("tm: ${Instant() - t}")
}
}
val x = tm { List.fill(n) { it * 10 + 1 } }
val y = tm { List.fill(n, n + 10) { it * 10 + 1 } }
tm { x.add(-1) }
tm { y.add(-2) }
+43
View File
@@ -0,0 +1,43 @@
import lyng.io.db
import lyng.io.db.jdbc
println("H2 JDBC demo: typed open, generic open, generated keys")
val db = openH2("mem:lyng_h2_demo;DB_CLOSE_DELAY=-1")
db.transaction { tx ->
tx.execute("create table if not exists person(id bigint auto_increment primary key, name varchar(120) not null, active boolean not null)")
tx.execute("delete from person")
val firstInsert = tx.execute(
"insert into person(name, active) values(?, ?)",
"Ada",
true
)
val firstId = firstInsert.getGeneratedKeys().toList()[0][0]
assertEquals(1, firstId)
tx.execute(
"insert into person(name, active) values(?, ?)",
"Linus",
false
)
val rows = tx.select("select id, name, active from person order by id").toList()
assertEquals(2, rows.size)
println("#" + rows[0]["id"] + " " + rows[0]["name"] + " active=" + rows[0]["active"])
println("#" + rows[1]["id"] + " " + rows[1]["name"] + " active=" + rows[1]["active"])
}
val genericDb = openDatabase(
"jdbc:h2:mem:lyng_h2_generic;DB_CLOSE_DELAY=-1",
Map()
)
val answer = genericDb.transaction { tx ->
tx.select("select 42 as answer").toList()[0]["answer"]
}
assertEquals(42, answer)
println("Generic JDBC openDatabase(...) also works: answer=$answer")
println("OK")
+71
View File
@@ -0,0 +1,71 @@
import lyng.io.db.jdbc
/*
PostgreSQL JDBC demo.
Usage:
lyng examples/postgres_basic.lyng [jdbc-url] [user] [password]
Typical local URL:
jdbc:postgresql://127.0.0.1/postgres
*/
fun cliArgs(): List<String> {
val result: List<String> = []
for (raw in ARGV as List) {
result.add(raw as String)
}
return result
}
val argv = cliArgs()
val URL = if (argv.size > 0) argv[0] else "jdbc:postgresql://127.0.0.1/postgres"
val USER = if (argv.size > 1) argv[1] else ""
val PASSWORD = if (argv.size > 2) argv[2] else ""
println("PostgreSQL JDBC demo: typed open, generated keys, nested transaction")
val db = openPostgres(URL, USER, PASSWORD)
db.transaction { tx ->
tx.execute("create table if not exists lyng_pg_demo(id bigserial primary key, title text not null, done boolean not null)")
tx.execute("delete from lyng_pg_demo")
val firstInsert = tx.execute(
"insert into lyng_pg_demo(title, done) values(?, ?)",
"Verify PostgreSQL JDBC support",
false
)
val firstId = firstInsert.getGeneratedKeys().toList()[0][0]
println("First generated id=" + firstId)
tx.execute(
"insert into lyng_pg_demo(title, done) values(?, ?)",
"Review documentation",
true
)
try {
tx.transaction { inner ->
inner.execute(
"insert into lyng_pg_demo(title, done) values(?, ?)",
"This row is rolled back",
false
)
throw IllegalStateException("rollback nested")
}
} catch (_: IllegalStateException) {
println("Nested transaction rolled back as expected")
}
val rows = tx.select("select id, title, done from lyng_pg_demo order by id").toList()
for (row in rows) {
println("#" + row["id"] + " " + row["title"] + " done=" + row["done"])
}
val count = tx.select("select count(*) as count from lyng_pg_demo").toList()[0]["count"]
assertEquals(2, count)
println("Visible rows after nested rollback: " + count)
}
println("OK")
+89
View File
@@ -0,0 +1,89 @@
import lyng.io.db
import lyng.io.db.sqlite
import lyng.time
println("SQLite demo: typed open, generic open, result sets, generated keys, nested rollback")
// The typed helper is the simplest entry point when you know you want SQLite.
val db = openSqlite(":memory:")
db.transaction { tx ->
// Keep schema creation and data changes inside one transaction block.
tx.execute("create table task(id integer primary key autoincrement, title text not null, done integer not null, due_date date not null)")
// execute(...) is for side-effect statements. Generated keys are read from
// ExecutionResult rather than from a synthetic row-returning INSERT.
val firstInsert = tx.execute(
"insert into task(title, done, due_date) values(?, ?, ?)",
"Write a SQLite example",
false,
Date(2026, 4, 15)
)
val firstGeneratedKeys = firstInsert.getGeneratedKeys()
val firstId = firstGeneratedKeys.toList()[0][0]
assertEquals(1, firstId)
tx.execute(
"insert into task(title, done, due_date) values(?, ?, ?)",
"Review the DB API",
true,
Date(2026, 4, 16)
)
// Nested transactions are real savepoints. If the inner block fails,
// only the nested work is rolled back.
try {
tx.transaction { inner ->
inner.execute(
"insert into task(title, done, due_date) values(?, ?, ?)",
"This row is rolled back",
false,
Date(2026, 4, 17)
)
throw IllegalStateException("demonstrate nested rollback")
}
} catch (_: IllegalStateException) {
println("Nested transaction rolled back as expected")
}
// select(...) is for row-producing statements. ResultSet exposes metadata,
// cheap emptiness checks, iteration, and conversion to a plain list.
val tasks = tx.select("select id, title, done, due_date from task order by id")
assertEquals(false, tasks.isEmpty())
assertEquals(2, tasks.size())
println("Columns:")
for (column in tasks.columns) {
println(" " + column.name + " -> " + column.sqlType + " (native " + column.nativeType + ")")
}
val taskRows = tasks.toList()
println("Rows:")
for (row in taskRows) {
// Name lookups are case-insensitive and values are already converted.
println(" #" + row["ID"] + " " + row["title"] + " done=" + row["done"] + " due=" + row["due_date"])
}
// toList() materializes detached rows that stay usable after transaction close.
val snapshot = tx.select("select title, due_date from task order by id").toList()
assertEquals("Write a SQLite example", snapshot[0]["title"])
assertEquals(Date(2026, 4, 16), snapshot[1]["due_date"])
val count = tx.select("select count(*) as count from task").toList()[0]["count"]
assertEquals(2, count)
println("Visible rows after nested rollback: $count")
}
// The generic entry point stays useful for config-driven code.
val genericDb = openDatabase(
"sqlite::memory:",{ foreignKeys: true, busyTimeoutMillis: 1000 }
)
val answer = genericDb.transaction { tx ->
tx.select("select 42 as answer").toList()[0]["answer"]
}
assertEquals(42, answer)
println("Generic openDatabase(...) also works: answer=$answer")
println("OK")
+11
View File
@@ -15,6 +15,11 @@ okioVersion = "3.10.2"
compiler = "3.2.0-alpha11"
ktor = "3.3.1"
slf4j = "2.0.17"
sqlite-jdbc = "3.50.3.0"
h2 = "2.4.240"
postgresql = "42.7.8"
testcontainers = "1.20.6"
hikaricp = "6.2.1"
[libraries]
clikt = { module = "com.github.ajalt.clikt:clikt", version.ref = "clikt" }
@@ -43,6 +48,12 @@ ktor-client-winhttp = { module = "io.ktor:ktor-client-winhttp", version.ref = "k
ktor-client-websockets = { module = "io.ktor:ktor-client-websockets", version.ref = "ktor" }
ktor-network = { module = "io.ktor:ktor-network", version.ref = "ktor" }
slf4j-nop = { module = "org.slf4j:slf4j-nop", version.ref = "slf4j" }
sqlite-jdbc = { module = "org.xerial:sqlite-jdbc", version.ref = "sqlite-jdbc" }
h2 = { module = "com.h2database:h2", version.ref = "h2" }
postgresql = { module = "org.postgresql:postgresql", version.ref = "postgresql" }
testcontainers = { module = "org.testcontainers:testcontainers", version.ref = "testcontainers" }
testcontainers-postgresql = { module = "org.testcontainers:postgresql", version.ref = "testcontainers" }
hikaricp = { module = "com.zaxxer:HikariCP", version.ref = "hikaricp" }
[plugins]
androidLibrary = { id = "com.android.library", version.ref = "agp" }
+2
View File
@@ -0,0 +1,2 @@
[vps]
94.130.36.94 ansible_user=sergeych
+100
View File
@@ -0,0 +1,100 @@
---
- name: Setup lynglang.com static site on VPS
hosts: vps
become: yes
vars:
domain: lynglang.com
web_root: /var/www/lynglang
deploy_user: sergeych
certbot_email: real.sergeych@gmail.com
tasks:
# Debian 10 buster is EOL; security/backports repos moved to archive.debian.org
- name: Fix sources.list for Debian buster EOL
copy:
dest: /etc/apt/sources.list
content: |
deb http://archive.debian.org/debian/ buster main contrib non-free
deb http://archive.debian.org/debian-security/ buster/updates main contrib non-free
deb http://archive.debian.org/debian/ buster-backports main contrib non-free
- name: Remove stale third-party sources (broken for buster EOL)
file:
path: "/etc/apt/sources.list.d/{{ item }}"
state: absent
loop:
- cassandra.list
- icinga.list
- postgres.list
- salt-stack.list
- yarn.list
- name: Install nginx, certbot, and python3-certbot-nginx
apt:
name:
- nginx
- certbot
- python3-certbot-nginx
state: present
update_cache: yes
- name: Create web root directory
file:
path: "{{ web_root }}/release/dist"
state: directory
owner: "{{ deploy_user }}"
group: www-data
mode: "0755"
recurse: yes
- name: Create distributables directory
file:
path: "{{ web_root }}/release/dist/distributables"
state: directory
owner: "{{ deploy_user }}"
group: www-data
mode: "0755"
- name: Deploy nginx site config (HTTP, pre-certbot)
template:
src: templates/nginx_lynglang.conf.j2
dest: /etc/nginx/sites-available/{{ domain }}
notify: reload nginx
- name: Enable nginx site
file:
src: /etc/nginx/sites-available/{{ domain }}
dest: /etc/nginx/sites-enabled/{{ domain }}
state: link
notify: reload nginx
- name: Disable default nginx site
file:
path: /etc/nginx/sites-enabled/default
state: absent
notify: reload nginx
- name: Ensure nginx is started
service:
name: nginx
state: started
enabled: yes
- name: Reload nginx before certbot
meta: flush_handlers
- name: Obtain SSL certificate via certbot (--nginx plugin)
command: >
certbot --nginx
-d {{ domain }} -d www.{{ domain }}
--non-interactive --agree-tos
--email {{ certbot_email }}
--redirect
args:
creates: /etc/letsencrypt/live/{{ domain }}/fullchain.pem
handlers:
- name: reload nginx
service:
name: nginx
state: reloaded
+24
View File
@@ -0,0 +1,24 @@
server {
listen 80;
server_name {{ domain }} www.{{ domain }};
root {{ web_root }}/release/dist;
index index.html;
# SPA fallback
location / {
try_files $uri $uri/ /index.html;
}
# Distributables served directly
location /distributables/ {
try_files $uri =404;
autoindex on;
}
# Long-lived cache for hashed assets
location ~* \.(js|css|woff2?|ttf|eot|svg|png|jpg|ico)$ {
expires 1y;
add_header Cache-Control "public, immutable";
}
}
+43
View File
@@ -19,11 +19,39 @@ plugins {
alias(libs.plugins.kotlinMultiplatform)
}
import org.gradle.api.Project
import org.jetbrains.kotlin.gradle.targets.native.tasks.KotlinNativeTest
group = "net.sergeych"
version = "unspecified"
private fun Project.sqliteLinuxLinkerOpts(vararg defaultDirs: String): List<String> {
val overrideDir = providers.gradleProperty("sqlite3.lib.dir").orNull
?: providers.environmentVariable("SQLITE3_LIB_DIR").orNull
val candidateDirs = buildList {
if (!overrideDir.isNullOrBlank()) {
add(file(overrideDir))
}
defaultDirs.forEach { add(file(it)) }
}.distinctBy { it.absolutePath }
val discoveredLib = sequenceOf("libsqlite3.so", "libsqlite3.so.0")
.mapNotNull { libraryName ->
candidateDirs.firstOrNull { it.resolve(libraryName).isFile }?.let { dir ->
listOf("-L${dir.absolutePath}", "-l:$libraryName")
}
}
.firstOrNull()
?: listOf("-lsqlite3")
return discoveredLib + listOf(
"-ldl",
"-lpthread",
"-lm",
"-Wl,--allow-shlib-undefined"
)
}
repositories {
mavenCentral()
maven("https://maven.universablockchain.com/")
@@ -55,6 +83,16 @@ kotlin {
binaries {
executable()
all {
linkerOpts(
*project.sqliteLinuxLinkerOpts(
"/lib/x86_64-linux-gnu",
"/usr/lib/x86_64-linux-gnu",
"/lib64",
"/usr/lib64",
"/lib",
"/usr/lib"
).toTypedArray()
)
if (buildType == org.jetbrains.kotlin.gradle.plugin.mpp.NativeBuildType.RELEASE) {
debuggable = false
optimized = true
@@ -113,8 +151,13 @@ kotlin {
tasks.named<KotlinNativeTest>("linuxX64Test") {
dependsOn(tasks.named("linkDebugExecutableLinuxX64"))
dependsOn(tasks.named("linkReleaseExecutableLinuxX64"))
environment(
"LYNG_CLI_NATIVE_BIN",
layout.buildDirectory.file("bin/linuxX64/debugExecutable/lyng.kexe").get().asFile.absolutePath
)
environment(
"LYNG_CLI_NATIVE_RELEASE_BIN",
layout.buildDirectory.file("bin/linuxX64/releaseExecutable/lyng.kexe").get().asFile.absolutePath
)
}
+10 -1
View File
@@ -40,6 +40,9 @@ import net.sergeych.lyng.ScriptError
import net.sergeych.lyng.Source
import net.sergeych.lyng.asFacade
import net.sergeych.lyng.io.console.createConsoleModule
import net.sergeych.lyng.io.db.createDbModule
import net.sergeych.lyng.io.db.jdbc.createJdbcModule
import net.sergeych.lyng.io.db.sqlite.createSqliteModule
import net.sergeych.lyng.io.fs.createFs
import net.sergeych.lyng.io.http.createHttpModule
import net.sergeych.lyng.io.net.createNetModule
@@ -116,8 +119,9 @@ internal class CliExecutionRuntime(
facade.call(handler)
}
}
session.cancelAndJoin()
session.cancel()
shutdownSystemNetEngine()
session.join()
}
fun shutdownBlocking() {
@@ -138,6 +142,8 @@ private val baseCliImportManagerDefer = globalDefer {
private fun ImportManager.invalidateCliModuleCaches() {
invalidatePackageCache("lyng.io.fs")
invalidatePackageCache("lyng.io.console")
invalidatePackageCache("lyng.io.db.jdbc")
invalidatePackageCache("lyng.io.db.sqlite")
invalidatePackageCache("lyng.io.http")
invalidatePackageCache("lyng.io.ws")
invalidatePackageCache("lyng.io.net")
@@ -225,6 +231,9 @@ private fun installCliModules(manager: ImportManager) {
// Scripts still need to import the modules they use explicitly.
createFs(PermitAllAccessPolicy, manager)
createConsoleModule(PermitAllConsoleAccessPolicy, manager)
createDbModule(manager)
createJdbcModule(manager)
createSqliteModule(manager)
createHttpModule(PermitAllHttpAccessPolicy, manager)
createWsModule(PermitAllWsAccessPolicy, manager)
createNetModule(PermitAllNetAccessPolicy, manager)
@@ -0,0 +1,132 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng_cli
import kotlinx.cinterop.*
import okio.FileSystem
import okio.Path
import okio.Path.Companion.toPath
import platform.posix.O_CREAT
import platform.posix.O_TRUNC
import platform.posix.O_WRONLY
import platform.posix.SIGKILL
import platform.posix.SIGSEGV
import platform.posix._exit
import platform.posix.close
import platform.posix.dup2
import platform.posix.execvp
import platform.posix.fork
import platform.posix.getenv
import platform.posix.getpid
import platform.posix.kill
import platform.posix.open
import platform.posix.usleep
import platform.posix.waitpid
import kotlin.test.Test
import kotlin.test.assertFalse
import kotlin.test.assertTrue
@OptIn(ExperimentalForeignApi::class)
class CliWebSocketNativeRegressionTest {
@Test
fun releaseCliDoesNotSegfaultOnConcurrentWebSocketClients() {
val executable = getenv("LYNG_CLI_NATIVE_RELEASE_BIN")?.toKString()
?: error("LYNG_CLI_NATIVE_RELEASE_BIN is not set")
val fs = FileSystem.SYSTEM
val repoRoot = ascend(executable.toPath(), 6)
val scriptPath = repoRoot / "bugs" / "ws-segfault.lyng"
check(fs.exists(scriptPath)) { "bug repro script not found at $scriptPath" }
val tempDir = "/tmp/lyng_ws_native_${getpid()}_${kotlin.random.Random.nextInt()}".toPath()
val stdoutPath = tempDir / "stdout.txt"
val stderrPath = tempDir / "stderr.txt"
fs.createDirectories(tempDir)
try {
val pid = launchCli(executable, scriptPath, stdoutPath, stderrPath)
usleep(5_000_000u)
if (kill(pid, 0) == 0) {
kill(pid, SIGKILL)
}
val status = waitForPid(pid)
val termSignal = status and 0x7f
val stdout = readUtf8IfExists(fs, stdoutPath)
val stderr = readUtf8IfExists(fs, stderrPath)
val allOutput = "$stdout\n$stderr"
assertFalse(termSignal == SIGSEGV, "native CLI crashed with SIGSEGV. Output:\n$allOutput")
assertTrue(
stdout.lineSequence().count { it == "test send to ws://127.0.0.1:9998... OK" } == 2,
"expected both websocket clients to finish. Output:\n$allOutput"
)
assertFalse(allOutput.contains("Segmentation fault"), "process output reported a segmentation fault:\n$allOutput")
} finally {
fs.deleteRecursively(tempDir, mustExist = false)
}
}
private fun ascend(path: Path, levels: Int): Path {
var current = path
repeat(levels) {
current = current.parent ?: error("cannot ascend $levels levels from $path")
}
return current
}
private fun readUtf8IfExists(fs: FileSystem, path: Path): String {
return if (fs.exists(path)) fs.read(path) { readUtf8() } else ""
}
private fun waitForPid(pid: Int): Int = memScoped {
val status = alloc<IntVar>()
val waited = waitpid(pid, status.ptr, 0)
check(waited == pid) { "waitpid failed for $pid" }
status.value
}
private fun launchCli(
executable: String,
scriptPath: Path,
stdoutPath: Path,
stderrPath: Path,
): Int = memScoped {
val pid = fork()
check(pid >= 0) { "fork failed" }
if (pid == 0) {
val stdoutFd = open(stdoutPath.toString(), O_WRONLY or O_CREAT or O_TRUNC, 0x1A4)
val stderrFd = open(stderrPath.toString(), O_WRONLY or O_CREAT or O_TRUNC, 0x1A4)
if (stdoutFd < 0 || stderrFd < 0) {
_exit(2)
}
dup2(stdoutFd, 1)
dup2(stderrFd, 2)
close(stdoutFd)
close(stderrFd)
val argv = allocArray<CPointerVar<ByteVar>>(3)
argv[0] = executable.cstr.ptr
argv[1] = scriptPath.toString().cstr.ptr
argv[2] = null
execvp(executable, argv)
_exit(127)
}
pid
}
}
+70
View File
@@ -31,6 +31,33 @@ plugins {
group = "net.sergeych"
version = "0.0.1-SNAPSHOT"
private fun Project.sqliteLinuxLinkerOpts(vararg defaultDirs: String): List<String> {
val overrideDir = providers.gradleProperty("sqlite3.lib.dir").orNull
?: providers.environmentVariable("SQLITE3_LIB_DIR").orNull
val candidateDirs = buildList {
if (!overrideDir.isNullOrBlank()) {
add(file(overrideDir))
}
defaultDirs.forEach { add(file(it)) }
}.distinctBy { it.absolutePath }
val discoveredLib = sequenceOf("libsqlite3.so", "libsqlite3.so.0")
.mapNotNull { libraryName ->
candidateDirs.firstOrNull { it.resolve(libraryName).isFile }?.let { dir ->
listOf("-L${dir.absolutePath}", "-l:$libraryName")
}
}
.firstOrNull()
?: listOf("-lsqlite3")
return discoveredLib + listOf(
"-ldl",
"-lpthread",
"-lm",
"-Wl,--allow-shlib-undefined"
)
}
kotlin {
jvmToolchain(17)
jvm()
@@ -58,6 +85,39 @@ kotlin {
// nodejs()
// }
targets.withType(org.jetbrains.kotlin.gradle.plugin.mpp.KotlinNativeTarget::class.java).configureEach {
compilations.getByName("main").cinterops.create("sqlite3") {
defFile(project.file("src/nativeInterop/cinterop/sqlite/sqlite3.def"))
packageName("net.sergeych.lyng.io.db.sqlite.cinterop")
includeDirs(project.file("src/nativeInterop/cinterop/sqlite"))
}
binaries.all {
when (konanTarget.name) {
"linux_x64" -> linkerOpts(
*project.sqliteLinuxLinkerOpts(
"/lib/x86_64-linux-gnu",
"/usr/lib/x86_64-linux-gnu",
"/lib64",
"/usr/lib64",
"/lib",
"/usr/lib"
).toTypedArray()
)
"linux_arm64" -> linkerOpts(
*project.sqliteLinuxLinkerOpts(
"/lib/aarch64-linux-gnu",
"/usr/lib/aarch64-linux-gnu",
"/lib64",
"/usr/lib64",
"/lib",
"/usr/lib"
).toTypedArray()
)
else -> linkerOpts("-lsqlite3")
}
}
}
// Keep expect/actual warning suppressed consistently with other modules
targets.configureEach {
compilations.configureEach {
@@ -120,6 +180,12 @@ kotlin {
implementation(libs.kotlinx.coroutines.test)
}
}
val jvmTest by getting {
dependencies {
implementation(libs.testcontainers)
implementation(libs.testcontainers.postgresql)
}
}
val linuxTest by creating {
dependsOn(commonTest)
}
@@ -155,6 +221,10 @@ kotlin {
implementation("org.jline:jline-terminal:3.29.0")
implementation(libs.ktor.client.cio)
implementation(libs.ktor.network)
implementation(libs.sqlite.jdbc)
implementation(libs.h2)
implementation(libs.postgresql)
implementation(libs.hikaricp)
}
}
// // For Wasm we use in-memory VFS for now
@@ -0,0 +1,22 @@
package net.sergeych.lyng.io.db.jdbc
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.requireScope
internal actual suspend fun openJdbcBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: JdbcOpenOptions,
): SqlDatabaseBackend {
scope.raiseError(
ObjException(
core.databaseException,
scope.requireScope(),
ObjString("lyng.io.db.jdbc is available only on the JVM target")
)
)
}
@@ -0,0 +1,39 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.requireScope
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjString
internal actual suspend fun openSqliteBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: SqliteOpenOptions,
): SqlDatabaseBackend {
scope.raiseError(
ObjException(
core.databaseException,
scope.requireScope(),
ObjString("SQLite provider is not implemented on this platform yet")
)
)
}
@@ -0,0 +1,142 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.Source
import net.sergeych.lyng.Arguments
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.ObjVoid
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.pacman.ImportManager
import net.sergeych.lyng.requireScope
import net.sergeych.lyngio.stdlib_included.dbLyng
private const val DB_MODULE_NAME = "lyng.io.db"
fun createDbModule(scope: Scope): Boolean = createDbModule(scope.importManager)
fun createDb(scope: Scope): Boolean = createDbModule(scope)
fun createDbModule(manager: ImportManager): Boolean {
if (manager.packageNames.contains(DB_MODULE_NAME)) return false
manager.addPackage(DB_MODULE_NAME) { module ->
buildDbModule(module)
}
return true
}
fun createDb(manager: ImportManager): Boolean = createDbModule(manager)
private suspend fun buildDbModule(module: ModuleScope) {
module.eval(Source(DB_MODULE_NAME, dbLyng))
val exceptions = installDbExceptionClasses(module)
val registry = DbProviderRegistry()
module.addFn("registerDatabaseProvider") {
val scheme = requiredArg<ObjString>(0).value
val opener = args.list.getOrNull(1)
?: raiseError("Expected exactly 2 arguments, got ${args.list.size}")
registry.register(this, scheme, opener)
ObjVoid
}
module.addFn("openDatabase") {
val connectionUrl = requiredArg<ObjString>(0).value
val extraParams = args.list.getOrNull(1)
?: raiseError("Expected exactly 2 arguments, got ${args.list.size}")
if (!extraParams.isInstanceOf("Map")) {
raiseIllegalArgument("extraParams must be Map")
}
val scheme = parseConnectionScheme(connectionUrl)
?: raiseIllegalArgument("Malformed database connection URL: $connectionUrl")
val opener = registry.providers[scheme]
?: raiseDatabaseException(exceptions.database, "No database provider registered for scheme '$scheme'")
call(opener, Arguments(listOf(ObjString(connectionUrl), extraParams)), newThisObj = ObjNull)
}
}
private data class DbExceptionClasses(
val database: ObjException.Companion.ExceptionClass,
val sqlExecution: ObjException.Companion.ExceptionClass,
val sqlConstraint: ObjException.Companion.ExceptionClass,
val sqlUsage: ObjException.Companion.ExceptionClass,
val rollback: ObjException.Companion.ExceptionClass,
)
private fun installDbExceptionClasses(module: ModuleScope): DbExceptionClasses {
val database = ObjException.Companion.ExceptionClass("DatabaseException", ObjException.Root)
val sqlExecution = ObjException.Companion.ExceptionClass("SqlExecutionException", database)
val sqlConstraint = ObjException.Companion.ExceptionClass("SqlConstraintException", sqlExecution)
val sqlUsage = ObjException.Companion.ExceptionClass("SqlUsageException", database)
val rollback = ObjException.Companion.ExceptionClass("RollbackException", ObjException.Root)
module.addConst("DatabaseException", database)
module.addConst("SqlExecutionException", sqlExecution)
module.addConst("SqlConstraintException", sqlConstraint)
module.addConst("SqlUsageException", sqlUsage)
module.addConst("RollbackException", rollback)
return DbExceptionClasses(
database = database,
sqlExecution = sqlExecution,
sqlConstraint = sqlConstraint,
sqlUsage = sqlUsage,
rollback = rollback,
)
}
private class DbProviderRegistry {
val providers: MutableMap<String, Obj> = linkedMapOf()
fun register(scope: ScopeFacade, rawScheme: String, opener: Obj) {
val scheme = normalizeScheme(rawScheme)
?: scope.raiseIllegalArgument("Database provider scheme must not be empty")
if (!opener.isInstanceOf("Callable")) {
scope.raiseIllegalArgument("Database provider opener must be callable")
}
if (providers.containsKey(scheme)) {
scope.raiseIllegalState("Database provider already registered for scheme '$scheme'")
}
providers[scheme] = opener
}
}
private fun normalizeScheme(rawScheme: String): String? {
val trimmed = rawScheme.trim()
if (trimmed.isEmpty()) return null
if (':' in trimmed) return null
return trimmed.lowercase()
}
private fun parseConnectionScheme(connectionUrl: String): String? {
val colonIndex = connectionUrl.indexOf(':')
if (colonIndex <= 0) return null
return normalizeScheme(connectionUrl.substring(0, colonIndex))
}
private fun ScopeFacade.raiseDatabaseException(
exceptionClass: ObjException.Companion.ExceptionClass,
message: String,
): Nothing = raiseError(ObjException(exceptionClass, requireScope(), ObjString(message)))
@@ -0,0 +1,399 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db
import net.sergeych.lyng.Arguments
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Scope
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjBool
import net.sergeych.lyng.obj.ObjClass
import net.sergeych.lyng.obj.ObjEnumClass
import net.sergeych.lyng.obj.ObjEnumEntry
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjImmutableList
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.thisAs
import net.sergeych.lyng.requireScope
import kotlin.collections.List
import kotlin.collections.Map
import kotlin.collections.MutableList
import kotlin.collections.associateWith
import kotlin.collections.drop
import kotlin.collections.first
import kotlin.collections.forEachIndexed
import kotlin.collections.getOrNull
import kotlin.collections.getOrPut
import kotlin.collections.indices
import kotlin.collections.linkedMapOf
import kotlin.collections.listOf
import kotlin.collections.map
import kotlin.collections.mutableListOf
import kotlin.text.lowercase
internal data class SqlColumnMeta(
val name: String,
val sqlType: ObjEnumEntry,
val nullable: Boolean,
val nativeType: String,
)
internal data class SqlResultSetData(
val columns: List<SqlColumnMeta>,
val rows: List<List<Obj>>,
)
internal data class SqlExecutionResultData(
val affectedRowsCount: Int,
val generatedKeys: SqlResultSetData,
)
internal interface SqlDatabaseBackend {
suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T
fun close() {}
}
internal interface SqlTransactionBackend {
suspend fun select(scope: ScopeFacade, clause: String, params: List<Obj>): SqlResultSetData
suspend fun execute(scope: ScopeFacade, clause: String, params: List<Obj>): SqlExecutionResultData
suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T
}
internal class SqlCoreModule private constructor(
val module: ModuleScope,
val databaseClass: ObjClass,
val transactionClass: ObjClass,
val resultSetClass: ObjClass,
val rowClass: ObjClass,
val columnClass: ObjClass,
val executionResultClass: ObjClass,
val databaseException: ObjException.Companion.ExceptionClass,
val sqlExecutionException: ObjException.Companion.ExceptionClass,
val sqlConstraintException: ObjException.Companion.ExceptionClass,
val sqlUsageException: ObjException.Companion.ExceptionClass,
val rollbackException: ObjException.Companion.ExceptionClass,
val sqlTypes: SqlTypeEntries,
) {
companion object {
fun resolve(module: ModuleScope): SqlCoreModule = SqlCoreModule(
module = module,
databaseClass = module.requireClass("Database"),
transactionClass = module.requireClass("SqlTransaction"),
resultSetClass = module.requireClass("ResultSet"),
rowClass = module.requireClass("SqlRow"),
columnClass = module.requireClass("SqlColumn"),
executionResultClass = module.requireClass("ExecutionResult"),
databaseException = module.requireClass("DatabaseException") as ObjException.Companion.ExceptionClass,
sqlExecutionException = module.requireClass("SqlExecutionException") as ObjException.Companion.ExceptionClass,
sqlConstraintException = module.requireClass("SqlConstraintException") as ObjException.Companion.ExceptionClass,
sqlUsageException = module.requireClass("SqlUsageException") as ObjException.Companion.ExceptionClass,
rollbackException = module.requireClass("RollbackException") as ObjException.Companion.ExceptionClass,
sqlTypes = SqlTypeEntries.resolve(module),
)
}
}
internal class SqlTypeEntries private constructor(
private val entries: Map<String, ObjEnumEntry>,
) {
fun require(name: String): ObjEnumEntry = entries[name]
?: error("lyng.io.db.SqlType entry is missing: $name")
companion object {
fun resolve(module: ModuleScope): SqlTypeEntries {
val enumClass = resolveEnum(module, "SqlType")
return SqlTypeEntries(
listOf(
"Binary", "String", "Int", "Double", "Decimal",
"Bool", "Instant", "Date", "DateTime"
).associateWith { name ->
enumClass.byName[ObjString(name)] as? ObjEnumEntry
?: error("lyng.io.db.SqlType.$name is missing")
}
)
}
private fun resolveEnum(module: ModuleScope, enumName: String): ObjEnumClass {
val local = module.get(enumName)?.value as? ObjEnumClass
if (local != null) return local
val root = module.importProvider.rootScope.get(enumName)?.value as? ObjEnumClass
return root ?: error("lyng.io.db declaration enum is missing: $enumName")
}
}
}
internal class SqlRuntimeTypes private constructor(
val core: SqlCoreModule,
val databaseClass: ObjClass,
val transactionClass: ObjClass,
val resultSetClass: ObjClass,
val rowClass: ObjClass,
val columnClass: ObjClass,
val executionResultClass: ObjClass,
) {
companion object {
fun create(prefix: String, core: SqlCoreModule): SqlRuntimeTypes {
val databaseClass = object : ObjClass("${prefix}Database", core.databaseClass) {}
val transactionClass = object : ObjClass("${prefix}Transaction", core.transactionClass) {}
val resultSetClass = object : ObjClass("${prefix}ResultSet", core.resultSetClass) {}
val rowClass = object : ObjClass("${prefix}Row", core.rowClass) {}
val columnClass = object : ObjClass("${prefix}Column", core.columnClass) {}
val executionResultClass = object : ObjClass("${prefix}ExecutionResult", core.executionResultClass) {}
val runtime = SqlRuntimeTypes(
core = core,
databaseClass = databaseClass,
transactionClass = transactionClass,
resultSetClass = resultSetClass,
rowClass = rowClass,
columnClass = columnClass,
executionResultClass = executionResultClass,
)
runtime.bind()
return runtime
}
}
private fun bind() {
databaseClass.addFn("close") {
thisAs<SqlDatabaseObj>().backend.close()
ObjNull
}
databaseClass.addFn("transaction") {
val self = thisAs<SqlDatabaseObj>()
val block = args.list.getOrNull(0) ?: raiseError("Expected exactly 1 argument, got ${args.list.size}")
if (!block.isInstanceOf("Callable")) {
raiseClassCastError("transaction block must be callable")
}
self.backend.transaction(this) { backend ->
val lifetime = SqlTransactionLifetime(this@SqlRuntimeTypes.core)
try {
call(block, Arguments(SqlTransactionObj(this@SqlRuntimeTypes, backend, lifetime)), ObjNull)
} finally {
lifetime.close()
}
}
}
transactionClass.addFn("select") {
val self = thisAs<SqlTransactionObj>()
self.lifetime.ensureActive(this)
val clause = (args.list.getOrNull(0) as? ObjString)?.value
?: raiseClassCastError("query must be String")
val params = args.list.drop(1)
SqlResultSetObj(self.types, self.lifetime, self.backend.select(this, clause, params))
}
transactionClass.addFn("execute") {
val self = thisAs<SqlTransactionObj>()
self.lifetime.ensureActive(this)
val clause = (args.list.getOrNull(0) as? ObjString)?.value
?: raiseClassCastError("query must be String")
val params = args.list.drop(1)
SqlExecutionResultObj(self.types, self.lifetime, self.backend.execute(this, clause, params))
}
transactionClass.addFn("transaction") {
val self = thisAs<SqlTransactionObj>()
self.lifetime.ensureActive(this)
val block = args.list.getOrNull(0) ?: raiseError("Expected exactly 1 argument, got ${args.list.size}")
if (!block.isInstanceOf("Callable")) {
raiseClassCastError("transaction block must be callable")
}
self.backend.transaction(this) { backend ->
val lifetime = SqlTransactionLifetime(this@SqlRuntimeTypes.core)
try {
call(block, Arguments(SqlTransactionObj(self.types, backend, lifetime)), ObjNull)
} finally {
lifetime.close()
}
}
}
resultSetClass.addProperty("columns", getter = {
val self = thisAs<SqlResultSetObj>()
self.lifetime.ensureActive(this)
ObjImmutableList(self.columns)
})
resultSetClass.addFn("size") {
val self = thisAs<SqlResultSetObj>()
self.lifetime.ensureActive(this)
ObjInt.of(self.rows.size.toLong())
}
resultSetClass.addFn("isEmpty") {
val self = thisAs<SqlResultSetObj>()
self.lifetime.ensureActive(this)
ObjBool(self.rows.isEmpty())
}
resultSetClass.addFn("iterator") {
val self = thisAs<SqlResultSetObj>()
self.lifetime.ensureActive(this)
ObjImmutableList(self.rows).invokeInstanceMethod(requireScope(), "iterator")
}
resultSetClass.addFn("toList") {
val self = thisAs<SqlResultSetObj>()
self.lifetime.ensureActive(this)
ObjImmutableList(self.rows)
}
rowClass.addProperty("size", getter = {
val self = thisAs<SqlRowObj>()
ObjInt.of(self.values.size.toLong())
})
rowClass.addProperty("values", getter = {
val self = thisAs<SqlRowObj>()
ObjImmutableList(self.values)
})
columnClass.addProperty("name", getter = { ObjString(thisAs<SqlColumnObj>().meta.name) })
columnClass.addProperty("sqlType", getter = { thisAs<SqlColumnObj>().meta.sqlType })
columnClass.addProperty("nullable", getter = { ObjBool(thisAs<SqlColumnObj>().meta.nullable) })
columnClass.addProperty("nativeType", getter = { ObjString(thisAs<SqlColumnObj>().meta.nativeType) })
executionResultClass.addProperty("affectedRowsCount", getter = {
val self = thisAs<SqlExecutionResultObj>()
self.lifetime.ensureActive(this)
ObjInt.of(self.result.affectedRowsCount.toLong())
})
executionResultClass.addFn("getGeneratedKeys") {
val self = thisAs<SqlExecutionResultObj>()
self.lifetime.ensureActive(this)
SqlResultSetObj(self.types, self.lifetime, self.result.generatedKeys)
}
}
}
internal class SqlTransactionLifetime(
private val core: SqlCoreModule,
) {
private var active = true
fun close() {
active = false
}
fun ensureActive(scope: ScopeFacade) {
if (!active) {
scope.raiseError(
ObjException(core.sqlUsageException, scope.requireScope(), ObjString("SQL result can be used only while its transaction is active"))
)
}
}
}
internal class SqlDatabaseObj(
val types: SqlRuntimeTypes,
val backend: SqlDatabaseBackend,
) : Obj() {
override val objClass: ObjClass
get() = types.databaseClass
}
internal class SqlTransactionObj(
val types: SqlRuntimeTypes,
val backend: SqlTransactionBackend,
val lifetime: SqlTransactionLifetime,
) : Obj() {
override val objClass: ObjClass
get() = types.transactionClass
}
internal class SqlResultSetObj(
val types: SqlRuntimeTypes,
val lifetime: SqlTransactionLifetime,
data: SqlResultSetData,
) : Obj() {
val columns: List<Obj> = data.columns.map { SqlColumnObj(types, it) }
val rows: List<Obj> = buildRows(types, data)
override val objClass: ObjClass
get() = types.resultSetClass
private fun buildRows(
types: SqlRuntimeTypes,
data: SqlResultSetData,
): List<Obj> {
val indexByName = linkedMapOf<String, MutableList<Int>>()
data.columns.forEachIndexed { index, column ->
indexByName.getOrPut(column.name.lowercase()) { mutableListOf() }.add(index)
}
return data.rows.map { rowValues ->
SqlRowObj(types, rowValues, indexByName)
}
}
}
internal class SqlRowObj(
val types: SqlRuntimeTypes,
val values: List<Obj>,
private val indexByName: Map<String, List<Int>>,
) : Obj() {
override val objClass: ObjClass
get() = types.rowClass
override suspend fun getAt(scope: Scope, index: Obj): Obj {
return when (index) {
is ObjInt -> {
val idx = index.value.toInt()
if (idx !in values.indices) {
scope.raiseIndexOutOfBounds("SQL row index $idx is out of bounds")
}
values[idx]
}
is ObjString -> {
val matches = indexByName[index.value.lowercase()]
?: scope.raiseError(
ObjException(
types.core.sqlUsageException,
scope,
ObjString("No such SQL result column: ${index.value}")
)
)
if (matches.size != 1) {
scope.raiseError(
ObjException(
types.core.sqlUsageException,
scope,
ObjString("Ambiguous SQL result column: ${index.value}")
)
)
}
values[matches.first()]
}
else -> scope.raiseClassCastError("SQL row index must be Int or String")
}
}
}
internal class SqlColumnObj(
val types: SqlRuntimeTypes,
val meta: SqlColumnMeta,
) : Obj() {
override val objClass: ObjClass
get() = types.columnClass
}
internal class SqlExecutionResultObj(
val types: SqlRuntimeTypes,
val lifetime: SqlTransactionLifetime,
val result: SqlExecutionResultData,
) : Obj() {
override val objClass: ObjClass
get() = types.executionResultClass
}
@@ -0,0 +1,278 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.jdbc
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.Source
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.io.db.SqlDatabaseObj
import net.sergeych.lyng.io.db.SqlRuntimeTypes
import net.sergeych.lyng.io.db.createDbModule
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjBool
import net.sergeych.lyng.obj.ObjImmutableMap
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjMap
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.ObjVoid
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.pacman.ImportManager
import net.sergeych.lyng.requireScope
import net.sergeych.lyngio.stdlib_included.db_jdbcLyng
private const val JDBC_MODULE_NAME = "lyng.io.db.jdbc"
private const val DB_MODULE_NAME = "lyng.io.db"
private const val JDBC_SCHEME = "jdbc"
private const val H2_DRIVER = "org.h2.Driver"
private const val POSTGRES_DRIVER = "org.postgresql.Driver"
fun createJdbcModule(scope: Scope): Boolean = createJdbcModule(scope.importManager)
fun createJdbc(scope: Scope): Boolean = createJdbcModule(scope)
fun createJdbcModule(manager: ImportManager): Boolean {
createDbModule(manager)
if (manager.packageNames.contains(JDBC_MODULE_NAME)) return false
manager.addPackage(JDBC_MODULE_NAME) { module ->
buildJdbcModule(module)
}
return true
}
fun createJdbc(manager: ImportManager): Boolean = createJdbcModule(manager)
private suspend fun buildJdbcModule(module: ModuleScope) {
module.eval(Source(JDBC_MODULE_NAME, db_jdbcLyng))
val dbModule = module.importProvider.createModuleScope(Pos.builtIn, DB_MODULE_NAME)
val core = SqlCoreModule.resolve(dbModule)
val runtimeTypes = SqlRuntimeTypes.create("Jdbc", core)
module.addFn("openJdbc") {
val options = parseOpenJdbcArgs(this)
SqlDatabaseObj(runtimeTypes, openJdbcBackend(this, core, options))
}
module.addFn("openH2") {
val options = parseOpenShortcutArgs(this, H2_DRIVER, ::normalizeH2Url)
SqlDatabaseObj(runtimeTypes, openJdbcBackend(this, core, options))
}
module.addFn("openPostgres") {
val options = parseOpenShortcutArgs(this, POSTGRES_DRIVER, ::normalizePostgresUrl)
SqlDatabaseObj(runtimeTypes, openJdbcBackend(this, core, options))
}
registerProvider(dbModule, runtimeTypes, core, JDBC_SCHEME, null)
registerProvider(dbModule, runtimeTypes, core, "h2", H2_DRIVER)
registerProvider(dbModule, runtimeTypes, core, "postgres", POSTGRES_DRIVER)
registerProvider(dbModule, runtimeTypes, core, "postgresql", POSTGRES_DRIVER)
}
private suspend fun registerProvider(
dbModule: ModuleScope,
runtimeTypes: SqlRuntimeTypes,
core: SqlCoreModule,
scheme: String,
implicitDriverClass: String?,
) {
dbModule.callFn(
"registerDatabaseProvider",
ObjString(scheme),
net.sergeych.lyng.obj.ObjExternCallable.fromBridge {
val connectionUrl = requiredArg<ObjString>(0).value
val extraParams = args.list.getOrNull(1)
?: raiseError("Expected exactly 2 arguments, got ${args.list.size}")
val options = parseJdbcConnectionUrl(this, scheme, implicitDriverClass, connectionUrl, extraParams)
SqlDatabaseObj(runtimeTypes, openJdbcBackend(this, core, options))
}
)
}
private suspend fun parseOpenJdbcArgs(scope: ScopeFacade): JdbcOpenOptions {
val rawUrl = readArg(scope, "connectionUrl", 0) ?: scope.raiseError("argument 'connectionUrl' is required")
val connectionUrl = (rawUrl as? ObjString)?.value ?: scope.raiseClassCastError("connectionUrl must be String")
val user = readNullableStringArg(scope, "user", 1)
val password = readNullableStringArg(scope, "password", 2)
val driverClass = readNullableStringArg(scope, "driverClass", 3)
val properties = readPropertiesArg(scope, "properties", 4)
return JdbcOpenOptions(
connectionUrl = normalizeJdbcUrl(connectionUrl, scope),
user = user,
password = password,
driverClass = driverClass,
properties = properties,
)
}
private suspend fun parseOpenShortcutArgs(
scope: ScopeFacade,
defaultDriverClass: String,
normalize: (String, ScopeFacade) -> String,
): JdbcOpenOptions {
val rawUrl = readArg(scope, "connectionUrl", 0) ?: scope.raiseError("argument 'connectionUrl' is required")
val connectionUrl = (rawUrl as? ObjString)?.value ?: scope.raiseClassCastError("connectionUrl must be String")
val user = readNullableStringArg(scope, "user", 1)
val password = readNullableStringArg(scope, "password", 2)
val properties = readPropertiesArg(scope, "properties", 3)
return JdbcOpenOptions(
connectionUrl = normalize(connectionUrl, scope),
user = user,
password = password,
driverClass = defaultDriverClass,
properties = properties,
)
}
private suspend fun parseJdbcConnectionUrl(
scope: ScopeFacade,
scheme: String,
implicitDriverClass: String?,
connectionUrl: String,
extraParams: Obj,
): JdbcOpenOptions {
val driverClass = mapNullableString(extraParams, scope, "driverClass") ?: implicitDriverClass
val user = mapNullableString(extraParams, scope, "user")
val password = mapNullableString(extraParams, scope, "password")
val properties = mapProperties(extraParams, scope, "properties")
val normalizedUrl = when (scheme) {
JDBC_SCHEME -> normalizeJdbcUrl(connectionUrl, scope)
"h2" -> normalizeH2Url(connectionUrl, scope)
"postgres", "postgresql" -> normalizePostgresUrl(connectionUrl, scope)
else -> scope.raiseIllegalArgument("Unsupported JDBC provider scheme: $scheme")
}
return JdbcOpenOptions(
connectionUrl = normalizedUrl,
user = user,
password = password,
driverClass = driverClass,
properties = properties,
)
}
private fun normalizeJdbcUrl(rawUrl: String, scope: ScopeFacade): String {
val trimmed = rawUrl.trim()
if (!trimmed.startsWith("jdbc:", ignoreCase = true)) {
scope.raiseIllegalArgument("JDBC connection URL must start with jdbc:")
}
return trimmed
}
private fun normalizeH2Url(rawUrl: String, scope: ScopeFacade): String {
val trimmed = rawUrl.trim()
if (trimmed.isEmpty()) {
scope.raiseIllegalArgument("H2 connection URL must not be empty")
}
return when {
trimmed.startsWith("jdbc:h2:", ignoreCase = true) -> trimmed
trimmed.startsWith("h2:", ignoreCase = true) -> "jdbc:${trimmed}"
else -> "jdbc:h2:$trimmed"
}
}
private fun normalizePostgresUrl(rawUrl: String, scope: ScopeFacade): String {
val trimmed = rawUrl.trim()
if (trimmed.isEmpty()) {
scope.raiseIllegalArgument("PostgreSQL connection URL must not be empty")
}
return when {
trimmed.startsWith("jdbc:postgresql:", ignoreCase = true) -> trimmed
trimmed.startsWith("postgresql:", ignoreCase = true) -> "jdbc:$trimmed"
trimmed.startsWith("postgres:", ignoreCase = true) -> "jdbc:postgresql:${trimmed.substringAfter(':')}"
else -> "jdbc:postgresql:$trimmed"
}
}
private suspend fun readArg(scope: ScopeFacade, name: String, position: Int): Obj? {
val named = scope.args.named[name]
val positional = scope.args.list.getOrNull(position)
if (named != null && positional != null) {
scope.raiseIllegalArgument("argument '$name' is already set")
}
return named ?: positional
}
private suspend fun readNullableStringArg(scope: ScopeFacade, name: String, position: Int): String? {
val value = readArg(scope, name, position) ?: return null
return when (value) {
ObjNull -> null
is ObjString -> value.value
else -> scope.raiseClassCastError("$name must be String?")
}
}
private suspend fun readPropertiesArg(scope: ScopeFacade, name: String, position: Int): Map<String, String> {
val value = readArg(scope, name, position) ?: return emptyMap()
return when (value) {
ObjNull -> emptyMap()
else -> objToStringMap(value, scope, name)
}
}
private suspend fun mapNullableString(map: Obj, scope: ScopeFacade, key: String): String? {
val value = map.getAt(scope.requireScope(), ObjString(key))
return when (value) {
ObjNull -> null
is ObjString -> value.value
else -> scope.raiseClassCastError("extraParams.$key must be String?")
}
}
private suspend fun mapProperties(map: Obj, scope: ScopeFacade, key: String): Map<String, String> {
val value = map.getAt(scope.requireScope(), ObjString(key))
return when (value) {
ObjNull -> emptyMap()
else -> objToStringMap(value, scope, "extraParams.$key")
}
}
private suspend fun objToStringMap(value: Obj, scope: ScopeFacade, label: String): Map<String, String> {
val rawEntries = when (value) {
is ObjMap -> value.map
is ObjImmutableMap -> value.map
else -> scope.raiseClassCastError("$label must be Map<String, Object?>")
}
val properties = linkedMapOf<String, String>()
for ((rawKey, rawValue) in rawEntries) {
val key = (rawKey as? ObjString)?.value ?: scope.raiseClassCastError("$label keys must be String")
if (rawValue == ObjNull) continue
properties[key] = scope.toStringOf(rawValue).value
}
return properties
}
private suspend fun ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
internal data class JdbcOpenOptions(
val connectionUrl: String,
val user: String?,
val password: String?,
val driverClass: String?,
val properties: Map<String, String>,
)
internal expect suspend fun openJdbcBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: JdbcOpenOptions,
): SqlDatabaseBackend
@@ -0,0 +1,194 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import net.sergeych.lyng.Arguments
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.Source
import net.sergeych.lyng.asFacade
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.io.db.SqlDatabaseObj
import net.sergeych.lyng.io.db.SqlRuntimeTypes
import net.sergeych.lyng.io.db.SqlTransactionBackend
import net.sergeych.lyng.io.db.createDbModule
import net.sergeych.lyng.requireScope
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjBool
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.ObjVoid
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.pacman.ImportManager
import net.sergeych.lyngio.stdlib_included.db_sqliteLyng
private const val SQLITE_MODULE_NAME = "lyng.io.db.sqlite"
private const val DB_MODULE_NAME = "lyng.io.db"
fun createSqliteModule(scope: Scope): Boolean = createSqliteModule(scope.importManager)
fun createSqlite(scope: Scope): Boolean = createSqliteModule(scope)
fun createSqliteModule(manager: ImportManager): Boolean {
createDbModule(manager)
if (manager.packageNames.contains(SQLITE_MODULE_NAME)) return false
manager.addPackage(SQLITE_MODULE_NAME) { module ->
buildSqliteModule(module)
}
return true
}
fun createSqlite(manager: ImportManager): Boolean = createSqliteModule(manager)
private suspend fun buildSqliteModule(module: ModuleScope) {
module.eval(Source(SQLITE_MODULE_NAME, db_sqliteLyng))
val dbModule = module.importProvider.createModuleScope(Pos.builtIn, DB_MODULE_NAME)
val core = SqlCoreModule.resolve(dbModule)
val runtimeTypes = SqlRuntimeTypes.create("Sqlite", core)
module.addFn("openSqlite") {
val options = parseOpenSqliteArgs(this)
SqlDatabaseObj(runtimeTypes, openSqliteBackend(this, core, options))
}
dbModule.callFn(
"registerDatabaseProvider",
ObjString("sqlite"),
net.sergeych.lyng.obj.ObjExternCallable.fromBridge {
val connectionUrl = requiredArg<ObjString>(0).value
val extraParams = args.list.getOrNull(1)
?: raiseError("Expected exactly 2 arguments, got ${args.list.size}")
val options = parseSqliteConnectionUrl(this, connectionUrl, extraParams)
SqlDatabaseObj(runtimeTypes, openSqliteBackend(this, core, options))
}
)
}
private suspend fun parseOpenSqliteArgs(scope: ScopeFacade): SqliteOpenOptions {
val pathValue = readArg(scope, "path", 0) ?: scope.raiseError("argument 'path' is required")
val path = (pathValue as? ObjString)?.value ?: scope.raiseClassCastError("path must be String")
val readOnly = readBoolArg(scope, "readOnly", 1, false)
val createIfMissing = readBoolArg(scope, "createIfMissing", 2, true)
val foreignKeys = readBoolArg(scope, "foreignKeys", 3, true)
val busyTimeoutMillis = readIntArg(scope, "busyTimeoutMillis", 4, 5000)
return SqliteOpenOptions(
path = normalizeSqlitePath(path, scope),
readOnly = readOnly,
createIfMissing = createIfMissing,
foreignKeys = foreignKeys,
busyTimeoutMillis = busyTimeoutMillis,
)
}
private suspend fun parseSqliteConnectionUrl(
scope: ScopeFacade,
connectionUrl: String,
extraParams: Obj,
): SqliteOpenOptions {
val prefix = "sqlite:"
if (!connectionUrl.startsWith(prefix, ignoreCase = true)) {
scope.raiseIllegalArgument("Malformed SQLite connection URL: $connectionUrl")
}
val rawPath = connectionUrl.substring(prefix.length)
val path = normalizeSqlitePath(rawPath, scope)
val readOnly = mapBool(extraParams, scope, "readOnly") ?: false
val createIfMissing = mapBool(extraParams, scope, "createIfMissing") ?: true
val foreignKeys = mapBool(extraParams, scope, "foreignKeys") ?: true
val busyTimeoutMillis = mapInt(extraParams, scope, "busyTimeoutMillis") ?: 5000
return SqliteOpenOptions(
path = path,
readOnly = readOnly,
createIfMissing = createIfMissing,
foreignKeys = foreignKeys,
busyTimeoutMillis = busyTimeoutMillis,
)
}
private fun normalizeSqlitePath(rawPath: String, scope: ScopeFacade): String {
val path = rawPath.trim()
if (path.isEmpty()) {
scope.raiseIllegalArgument("SQLite path must not be empty")
}
if (path.startsWith("//")) {
scope.raiseIllegalArgument("Unsupported SQLite URL form: sqlite:$path")
}
return path
}
private suspend fun readArg(scope: ScopeFacade, name: String, position: Int): Obj? {
val named = scope.args.named[name]
val positional = scope.args.list.getOrNull(position)
if (named != null && positional != null) {
scope.raiseIllegalArgument("argument '$name' is already set")
}
return named ?: positional
}
private suspend fun readBoolArg(scope: ScopeFacade, name: String, position: Int, default: Boolean): Boolean {
val value = readArg(scope, name, position) ?: return default
return (value as? ObjBool)?.value ?: scope.raiseClassCastError("$name must be Bool")
}
private suspend fun readIntArg(scope: ScopeFacade, name: String, position: Int, default: Int): Int {
val value = readArg(scope, name, position) ?: return default
return when (value) {
is ObjInt -> value.value.toInt()
else -> scope.raiseClassCastError("$name must be Int")
}
}
private suspend fun mapBool(map: Obj, scope: ScopeFacade, key: String): Boolean? {
val value = map.getAt(scope.requireScope(), ObjString(key))
return when (value) {
ObjNull -> null
is ObjBool -> value.value
else -> scope.raiseClassCastError("extraParams.$key must be Bool")
}
}
private suspend fun mapInt(map: Obj, scope: ScopeFacade, key: String): Int? {
val value = map.getAt(scope.requireScope(), ObjString(key))
return when (value) {
ObjNull -> null
is ObjInt -> value.value.toInt()
else -> scope.raiseClassCastError("extraParams.$key must be Int")
}
}
private suspend fun ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
internal data class SqliteOpenOptions(
val path: String,
val readOnly: Boolean,
val createIfMissing: Boolean,
val foreignKeys: Boolean,
val busyTimeoutMillis: Int,
)
internal expect suspend fun openSqliteBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: SqliteOpenOptions,
): SqlDatabaseBackend
@@ -17,35 +17,46 @@ import kotlinx.coroutines.channels.ClosedReceiveChannelException
internal fun createKtorWsEngine(
engineFactory: HttpClientEngineFactory<HttpClientEngineConfig>,
): LyngWsEngine = KtorLyngWsEngine(engineFactory)
shareClient: Boolean = true,
): LyngWsEngine = KtorLyngWsEngine(engineFactory, shareClient)
private class KtorLyngWsEngine(
engineFactory: HttpClientEngineFactory<HttpClientEngineConfig>,
private val shareClient: Boolean,
) : LyngWsEngine {
private val clientResult = runCatching {
private val clientFactory: () -> HttpClient = {
HttpClient(engineFactory) {
install(WebSockets)
}
}
private val sharedClientResult = if (shareClient) runCatching { clientFactory() } else null
private val supportProbe = if (!shareClient) runCatching { clientFactory().close() } else null
override val isSupported: Boolean
get() = clientResult.isSuccess
get() = sharedClientResult?.isSuccess ?: supportProbe?.isSuccess ?: true
override suspend fun connect(url: String, headers: Map<String, String>): LyngWsSession {
val client = clientResult.getOrElse {
val client = (sharedClientResult?.getOrElse {
throw UnsupportedOperationException(it.message ?: "WebSocket client is not supported")
}
} ?: runCatching { clientFactory() }.getOrElse {
throw UnsupportedOperationException(it.message ?: "WebSocket client is not supported")
})
val session = client.webSocketSession {
url(url)
headers.forEach { (name, value) -> header(name, value) }
}
return KtorLyngWsSession(url, session)
return KtorLyngWsSession(
targetUrl = url,
session = session,
ownedClient = client.takeUnless { shareClient },
)
}
}
private class KtorLyngWsSession(
private val targetUrl: String,
private val session: DefaultWebSocketSession,
private val ownedClient: HttpClient? = null,
) : LyngWsSession {
private var closed = false
@@ -55,12 +66,22 @@ private class KtorLyngWsSession(
override suspend fun sendText(text: String) {
ensureOpen()
session.send(text)
try {
session.send(text)
} catch (e: Throwable) {
release()
throw e
}
}
override suspend fun sendBytes(data: ByteArray) {
ensureOpen()
session.send(data)
try {
session.send(data)
} catch (e: Throwable) {
release()
throw e
}
}
override suspend fun receive(): LyngWsMessage? {
@@ -68,14 +89,17 @@ private class KtorLyngWsSession(
val frame = try {
session.incoming.receive()
} catch (_: ClosedReceiveChannelException) {
closed = true
release()
return null
} catch (e: Throwable) {
release()
throw e
}
return when (frame) {
is Frame.Text -> LyngWsMessage(isText = true, text = frame.readText())
is Frame.Binary -> LyngWsMessage(isText = false, data = frame.data.copyOf())
is Frame.Close -> {
closed = true
release()
null
}
else -> receive()
@@ -84,11 +108,20 @@ private class KtorLyngWsSession(
override suspend fun close(code: Int, reason: String) {
if (closed) return
closed = true
session.close(CloseReason(code.toShort(), reason))
try {
session.close(CloseReason(code.toShort(), reason))
} finally {
release()
}
}
private fun ensureOpen() {
if (closed) throw IllegalStateException("websocket session is closed")
}
private fun release() {
if (closed) return
closed = true
ownedClient?.close()
}
}
@@ -0,0 +1,392 @@
package net.sergeych.lyngio.ws
import io.ktor.http.Url
import io.ktor.http.URLProtocol
import net.sergeych.lyngio.net.LyngNetEngine
import net.sergeych.lyngio.net.LyngTcpSocket
import net.sergeych.lyngio.net.getSystemNetEngine
import net.sergeych.mp_tools.encodeToBase64
import kotlin.random.Random
internal fun createSocketWsEngine(
secureFallback: LyngWsEngine = UnsupportedLyngWsEngine,
): LyngWsEngine = SocketLyngWsEngine(getSystemNetEngine(), secureFallback)
private class SocketLyngWsEngine(
private val netEngine: LyngNetEngine,
private val secureFallback: LyngWsEngine,
) : LyngWsEngine {
override val isSupported: Boolean
get() = (netEngine.isSupported && netEngine.isTcpAvailable) || secureFallback.isSupported
override suspend fun connect(url: String, headers: Map<String, String>): LyngWsSession {
val parsedUrl = Url(url)
return when (parsedUrl.protocol.name.lowercase()) {
URLProtocol.WS.name.lowercase() -> connectPlain(parsedUrl, headers)
URLProtocol.WSS.name.lowercase() -> secureFallback.connect(url, headers)
else -> throw UnsupportedOperationException("Unsupported websocket scheme: ${parsedUrl.protocol.name}")
}
}
private suspend fun connectPlain(url: Url, headers: Map<String, String>): LyngWsSession {
if (!netEngine.isSupported || !netEngine.isTcpAvailable) {
throw UnsupportedOperationException("WebSocket client is not supported on this runtime")
}
val socket = netEngine.tcpConnect(url.host, url.port, timeoutMillis = null, noDelay = true)
try {
val key = randomBytes(16).encodeToBase64()
val requestPath = buildRequestPath(url)
val hostHeader = buildHostHeader(url)
val request = buildString {
append("GET ").append(requestPath).append(" HTTP/1.1\r\n")
append("Host: ").append(hostHeader).append("\r\n")
append("Upgrade: websocket\r\n")
append("Connection: Upgrade\r\n")
append("Sec-WebSocket-Key: ").append(key).append("\r\n")
append("Sec-WebSocket-Version: 13\r\n")
headers.forEach { (name, value) ->
append(name).append(": ").append(value).append("\r\n")
}
append("\r\n")
}
socket.writeUtf8(request)
socket.flush()
validateHandshake(socket, key)
return SocketLyngWsSession(url.toString(), socket)
} catch (e: Throwable) {
socket.close()
throw e
}
}
}
private class SocketLyngWsSession(
private val targetUrl: String,
private val socket: LyngTcpSocket,
) : LyngWsSession {
private var closed = false
private var closeSent = false
private var pending = ByteArray(0)
private var fragmentedOpcode: Int? = null
private var fragmentedPayload = ByteArray(0)
override fun isOpen(): Boolean = !closed && socket.isOpen()
override fun url(): String = targetUrl
override suspend fun sendText(text: String) {
ensureOpen()
sendFrame(OPCODE_TEXT, text.encodeToByteArray())
}
override suspend fun sendBytes(data: ByteArray) {
ensureOpen()
sendFrame(OPCODE_BINARY, data)
}
override suspend fun receive(): LyngWsMessage? {
if (closed) return null
while (true) {
val frame = readFrame() ?: run {
release()
return null
}
when (frame.opcode) {
OPCODE_CONTINUATION -> {
val opcode = fragmentedOpcode ?: throw IllegalStateException("unexpected websocket continuation frame")
fragmentedPayload += frame.payload
if (frame.fin) {
val payload = fragmentedPayload
fragmentedOpcode = null
fragmentedPayload = ByteArray(0)
return payload.toMessage(opcode)
}
}
OPCODE_TEXT, OPCODE_BINARY -> {
if (frame.fin) return frame.payload.toMessage(frame.opcode)
fragmentedOpcode = frame.opcode
fragmentedPayload = frame.payload
}
OPCODE_CLOSE -> {
if (!closeSent) {
sendFrame(OPCODE_CLOSE, frame.payload)
closeSent = true
}
release()
return null
}
OPCODE_PING -> sendFrame(OPCODE_PONG, frame.payload)
OPCODE_PONG -> Unit
else -> Unit
}
}
}
override suspend fun close(code: Int, reason: String) {
if (closed) return
val reasonBytes = reason.encodeToByteArray()
val payload = ByteArray(reasonBytes.size + 2)
payload[0] = (code shr 8).toByte()
payload[1] = code.toByte()
reasonBytes.copyInto(payload, destinationOffset = 2)
try {
if (!closeSent) {
sendFrame(OPCODE_CLOSE, payload)
closeSent = true
}
} finally {
release()
}
}
private suspend fun sendFrame(opcode: Int, payload: ByteArray) {
ensureOpen()
val header = buildFrameHeader(opcode, payload.size, masked = true)
val mask = randomBytes(4)
val maskedPayload = payload.copyOf()
maskedPayload.indices.forEach { index ->
maskedPayload[index] = (maskedPayload[index].toInt() xor mask[index % mask.size].toInt()).toByte()
}
socket.write(header + mask + maskedPayload)
socket.flush()
}
private suspend fun readFrame(): WsFrame? {
val head = readExact(2) ?: return null
val fin = (head[0].toInt() and 0x80) != 0
val opcode = head[0].toInt() and 0x0f
val masked = (head[1].toInt() and 0x80) != 0
val payloadLength = when (val lengthCode = head[1].toInt() and 0x7f) {
126 -> {
val extended = readExact(2) ?: return null
((extended[0].toInt() and 0xff) shl 8) or (extended[1].toInt() and 0xff)
}
127 -> {
val extended = readExact(8) ?: return null
var acc = 0L
extended.forEach { byte ->
acc = (acc shl 8) or (byte.toInt() and 0xff).toLong()
}
require(acc <= Int.MAX_VALUE.toLong()) { "websocket frame is too large" }
acc.toInt()
}
else -> lengthCode
}
val mask = if (masked) readExact(4) ?: return null else null
val payload = if (payloadLength > 0) readExact(payloadLength) ?: return null else ByteArray(0)
if (mask != null) {
payload.indices.forEach { index ->
payload[index] = (payload[index].toInt() xor mask[index % mask.size].toInt()).toByte()
}
}
return WsFrame(fin = fin, opcode = opcode, payload = payload)
}
private suspend fun readExact(byteCount: Int): ByteArray? {
while (pending.size < byteCount) {
val chunk = socket.read(maxOf(4096, byteCount - pending.size)) ?: break
if (chunk.isEmpty()) break
pending += chunk
}
if (pending.size < byteCount) return null
val result = pending.copyOfRange(0, byteCount)
pending = pending.copyOfRange(byteCount, pending.size)
return result
}
private fun release() {
if (closed) return
closed = true
socket.close()
}
private fun ensureOpen() {
if (closed || !socket.isOpen()) throw IllegalStateException("websocket session is closed")
}
}
private data class WsFrame(
val fin: Boolean,
val opcode: Int,
val payload: ByteArray,
)
private suspend fun validateHandshake(socket: LyngTcpSocket, key: String) {
val statusLine = socket.readLine() ?: error("websocket handshake failed: missing response status")
require(statusLine.startsWith("HTTP/1.1 101") || statusLine.startsWith("HTTP/1.0 101")) {
"websocket handshake failed: $statusLine"
}
val headers = linkedMapOf<String, String>()
while (true) {
val line = socket.readLine() ?: error("websocket handshake failed: unexpected EOF")
if (line.isBlank()) break
val colonAt = line.indexOf(':')
require(colonAt > 0) { "invalid websocket header: $line" }
val name = line.substring(0, colonAt).trim().lowercase()
val value = line.substring(colonAt + 1).trim()
headers[name] = value
}
require(headers["upgrade"]?.lowercase() == "websocket") { "websocket handshake failed: missing Upgrade header" }
require(headers["connection"]?.lowercase()?.contains("upgrade") == true) {
"websocket handshake failed: missing Connection header"
}
require(headers["sec-websocket-accept"] == websocketAcceptKey(key)) {
"websocket handshake failed: invalid Sec-WebSocket-Accept"
}
}
private fun websocketAcceptKey(key: String): String =
sha1((key + WS_GUID).encodeToByteArray()).encodeToBase64()
private fun buildRequestPath(url: Url): String {
val path = url.encodedPath.ifEmpty { "/" }
val query = url.encodedQuery
return if (query.isBlank()) path else "$path?$query"
}
private fun buildHostHeader(url: Url): String {
val host = if (':' in url.host && !url.host.startsWith("[")) "[${url.host}]" else url.host
return if (url.port == url.protocol.defaultPort) host else "$host:${url.port}"
}
private fun buildFrameHeader(opcode: Int, payloadSize: Int, masked: Boolean): ByteArray {
require(payloadSize >= 0) { "payload size must be non-negative" }
val firstByte = (0x80 or (opcode and 0x0f)).toByte()
val maskBit = if (masked) 0x80 else 0
return when {
payloadSize <= 125 -> byteArrayOf(firstByte, (maskBit or payloadSize).toByte())
payloadSize <= 0xffff -> byteArrayOf(
firstByte,
(maskBit or 126).toByte(),
((payloadSize ushr 8) and 0xff).toByte(),
(payloadSize and 0xff).toByte(),
)
else -> byteArrayOf(
firstByte,
(maskBit or 127).toByte(),
0,
0,
0,
0,
((payloadSize ushr 24) and 0xff).toByte(),
((payloadSize ushr 16) and 0xff).toByte(),
((payloadSize ushr 8) and 0xff).toByte(),
(payloadSize and 0xff).toByte(),
)
}
}
private fun ByteArray.toMessage(opcode: Int): LyngWsMessage = when (opcode) {
OPCODE_TEXT -> LyngWsMessage(isText = true, text = decodeToString())
OPCODE_BINARY -> LyngWsMessage(isText = false, data = copyOf())
else -> throw IllegalStateException("unsupported websocket opcode: $opcode")
}
private fun randomBytes(size: Int): ByteArray = ByteArray(size).also(Random.Default::nextBytes)
private fun sha1(input: ByteArray): ByteArray {
var h0 = 0x67452301
var h1 = 0xEFCDAB89.toInt()
var h2 = 0x98BADCFE.toInt()
var h3 = 0x10325476
var h4 = 0xC3D2E1F0.toInt()
val msgLen = input.size
val bitLen = msgLen.toLong() * 8L
val totalLen = ((msgLen + 1 + 8 + 63) / 64) * 64
val padded = ByteArray(totalLen).also { buf ->
input.copyInto(buf)
buf[msgLen] = 0x80.toByte()
for (i in 0..7) {
buf[totalLen - 8 + i] = ((bitLen ushr (56 - i * 8)) and 0xff).toByte()
}
}
val words = IntArray(80)
var blockStart = 0
while (blockStart < padded.size) {
for (i in 0..15) {
val off = blockStart + i * 4
words[i] = ((padded[off].toInt() and 0xff) shl 24) or
((padded[off + 1].toInt() and 0xff) shl 16) or
((padded[off + 2].toInt() and 0xff) shl 8) or
(padded[off + 3].toInt() and 0xff)
}
for (i in 16..79) {
val mixed = words[i - 3] xor words[i - 8] xor words[i - 14] xor words[i - 16]
words[i] = (mixed shl 1) or (mixed ushr 31)
}
var a = h0
var b = h1
var c = h2
var d = h3
var e = h4
for (i in 0..19) {
val f = (b and c) or (b.inv() and d)
val temp = ((a shl 5) or (a ushr 27)) + f + e + 0x5A827999 + words[i]
e = d
d = c
c = (b shl 30) or (b ushr 2)
b = a
a = temp
}
for (i in 20..39) {
val f = b xor c xor d
val temp = ((a shl 5) or (a ushr 27)) + f + e + 0x6ED9EBA1 + words[i]
e = d
d = c
c = (b shl 30) or (b ushr 2)
b = a
a = temp
}
for (i in 40..59) {
val f = (b and c) or (b and d) or (c and d)
val temp = ((a shl 5) or (a ushr 27)) + f + e + 0x8F1BBCDC.toInt() + words[i]
e = d
d = c
c = (b shl 30) or (b ushr 2)
b = a
a = temp
}
for (i in 60..79) {
val f = b xor c xor d
val temp = ((a shl 5) or (a ushr 27)) + f + e + 0xCA62C1D6.toInt() + words[i]
e = d
d = c
c = (b shl 30) or (b ushr 2)
b = a
a = temp
}
h0 += a
h1 += b
h2 += c
h3 += d
h4 += e
blockStart += 64
}
return ByteArray(20).also { out ->
fun putInt(offset: Int, value: Int) {
out[offset] = (value ushr 24).toByte()
out[offset + 1] = (value ushr 16).toByte()
out[offset + 2] = (value ushr 8).toByte()
out[offset + 3] = value.toByte()
}
putInt(0, h0)
putInt(4, h1)
putInt(8, h2)
putInt(12, h3)
putInt(16, h4)
}
}
private const val WS_GUID = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
private const val OPCODE_CONTINUATION = 0x0
private const val OPCODE_TEXT = 0x1
private const val OPCODE_BINARY = 0x2
private const val OPCODE_CLOSE = 0x8
private const val OPCODE_PING = 0x9
private const val OPCODE_PONG = 0xA
@@ -0,0 +1,135 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db
import kotlinx.coroutines.test.runTest
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.pacman.ImportManager
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Script
import net.sergeych.lyng.Source
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjExternCallable
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.requireScope
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
import kotlin.test.assertFalse
import kotlin.test.assertTrue
class LyngDbModuleTest {
@Test
fun testModuleRegistrationIsIdempotent() = runTest {
val importManager = ImportManager()
assertTrue(createDbModule(importManager))
assertFalse(createDbModule(importManager))
}
@Test
fun testOpenDatabaseDispatchesByNormalizedScheme() = runTest {
val scope = Script.newScope()
createDbModule(scope.importManager)
val module = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
module.callFn(
"registerDatabaseProvider",
ObjString("TeSt"),
ObjExternCallable.fromBridge {
val url = requiredArg<ObjString>(0).value
val params = requiredArg<Obj>(1)
val size = (params.invokeInstanceMethod(requireScope(), "size") as ObjInt).value
ObjString("$url|$size")
}
)
val code = """
import lyng.io.db
openDatabase("TEST:demo", Map("a" => 1, "b" => 2))
""".trimIndent()
val result = Compiler.compile(Source("<db-test>", code), scope.importManager).execute(scope) as ObjString
assertEquals("TEST:demo|2", result.value)
}
@Test
fun testDuplicateSchemeRegistrationFailsCaseInsensitively() = runTest {
val importManager = ImportManager()
createDbModule(importManager)
val module = importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
val scheme = "provider_test"
module.callFn("registerDatabaseProvider", ObjString(scheme), trivialOpener())
val error = try {
module.callFn("registerDatabaseProvider", ObjString(scheme.uppercase()), trivialOpener())
kotlin.test.fail("expected duplicate registration to fail")
} catch (e: ExecutionError) {
e
}
assertTrue(error.errorMessage.contains("already registered"), error.errorMessage)
}
@Test
fun testMalformedUrlFailsWithIllegalArgument() = runTest {
val scope = Script.newScope()
createDbModule(scope.importManager)
val code = """
import lyng.io.db
openDatabase("not-a-url", Map())
""".trimIndent()
val error = assertFailsWith<ExecutionError> {
Compiler.compile(Source("<db-test>", code), scope.importManager).execute(scope)
}
assertEquals("IllegalArgumentException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("Malformed database connection URL"), error.errorMessage)
}
@Test
fun testUnknownSchemeFailsWithDatabaseException() = runTest {
val scope = Script.newScope()
createDbModule(scope.importManager)
val code = """
import lyng.io.db
openDatabase("unknown:demo", Map())
""".trimIndent()
val error = assertFailsWith<ExecutionError> {
Compiler.compile(Source("<db-test>", code), scope.importManager).execute(scope)
}
assertEquals("DatabaseException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("No database provider registered"), error.errorMessage)
}
private suspend fun net.sergeych.lyng.ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
private fun trivialOpener(): Obj = ObjExternCallable.fromBridge { ObjInt.One }
}
@@ -0,0 +1,22 @@
package net.sergeych.lyng.io.db.jdbc
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.requireScope
internal actual suspend fun openJdbcBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: JdbcOpenOptions,
): SqlDatabaseBackend {
scope.raiseError(
ObjException(
core.databaseException,
scope.requireScope(),
ObjString("lyng.io.db.jdbc is available only on the JVM target")
)
)
}
@@ -0,0 +1,39 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.requireScope
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjString
internal actual suspend fun openSqliteBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: SqliteOpenOptions,
): SqlDatabaseBackend {
scope.raiseError(
ObjException(
core.databaseException,
scope.requireScope(),
ObjString("SQLite provider is not implemented on this platform yet")
)
)
}
@@ -0,0 +1,448 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.jdbc
import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import kotlinx.datetime.LocalDate
import kotlinx.datetime.LocalDateTime
import kotlinx.datetime.TimeZone
import kotlinx.datetime.toInstant
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.*
import net.sergeych.lyng.obj.*
import net.sergeych.lyng.requireScope
import java.math.BigDecimal
import java.sql.*
import kotlin.time.Instant
private val knownJdbcDrivers = listOf(
"org.sqlite.JDBC",
"org.h2.Driver",
"org.postgresql.Driver",
)
internal actual suspend fun openJdbcBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: JdbcOpenOptions,
): SqlDatabaseBackend {
ensureJdbcDriversLoaded(scope, core, options.driverClass)
val dataSource = try {
val config = HikariConfig().apply {
jdbcUrl = options.connectionUrl
options.user?.let { username = it }
options.password?.let { password = it }
options.driverClass?.let { driverClassName = it }
options.properties.forEach { (key, value) -> addDataSourceProperty(key, value) }
// Statement caching: avoids re-parsing identical SQL on every call
addDataSourceProperty("cachePrepStmts", "true")
addDataSourceProperty("prepStmtCacheSize", "250")
addDataSourceProperty("prepStmtCacheSqlLimit", "2048")
// Connections from the pool are already in autoCommit=false territory;
// we manage commits explicitly in transaction().
isAutoCommit = false
}
HikariDataSource(config)
} catch (e: Exception) {
val cause = e.cause as? SQLException ?: (e as? SQLException)
if (cause != null) throw mapOpenException(scope, core, cause)
throw e
}
return JdbcDatabaseBackend(core, dataSource)
}
private class JdbcDatabaseBackend(
private val core: SqlCoreModule,
private val dataSource: HikariDataSource,
) : SqlDatabaseBackend {
override suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T {
val connection = try {
dataSource.connection
} catch (e: SQLException) {
throw mapOpenException(scope, core, e)
}
try {
val tx = JdbcTransactionBackend(core, connection)
val result = try {
block(tx)
} catch (e: Throwable) {
throw finishFailedTransaction(scope, core, e) {
rollbackOrThrow(scope, core, connection)
}
}
try {
connection.commit()
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
return result
} finally {
try {
connection.close() // returns connection to pool
} catch (_: SQLException) {
}
}
}
override fun close() {
dataSource.close()
}
}
private class JdbcTransactionBackend(
private val core: SqlCoreModule,
private val connection: Connection,
) : SqlTransactionBackend {
override suspend fun select(scope: ScopeFacade, clause: String, params: List<Obj>): SqlResultSetData {
try {
connection.prepareStatement(clause).use { statement ->
bindParams(statement, params, scope, core)
statement.executeQuery().use { rs ->
return readResultSet(scope, core, rs)
}
}
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
override suspend fun execute(scope: ScopeFacade, clause: String, params: List<Obj>): SqlExecutionResultData {
if (containsRowReturningClause(clause)) {
scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("execute(...) cannot be used with statements that return rows; use select(...)")
)
)
}
try {
connection.prepareStatement(clause, Statement.RETURN_GENERATED_KEYS).use { statement ->
bindParams(statement, params, scope, core)
val hasResultSet = statement.execute()
if (hasResultSet) {
scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("execute(...) cannot be used with statements that return rows; use select(...)")
)
)
}
val generatedKeys = statement.generatedKeys.use { rs ->
if (rs == null) emptyResultSet() else readResultSet(scope, core, rs)
}
return SqlExecutionResultData(statement.updateCount, generatedKeys)
}
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
override suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T {
val savepoint = try {
connection.setSavepoint()
} catch (e: SQLException) {
throw mapSqlUsage(scope, core, "Nested transactions are not supported by this JDBC backend", e)
}
val nested = JdbcTransactionBackend(core, connection)
val result = try {
block(nested)
} catch (e: Throwable) {
throw finishFailedTransaction(scope, core, e) {
rollbackToSavepointOrThrow(scope, core, connection, savepoint)
releaseSavepointOrThrow(scope, core, connection, savepoint)
}
}
try {
connection.releaseSavepoint(savepoint)
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
return result
}
}
private suspend fun bindParams(statement: PreparedStatement, params: List<Obj>, scope: ScopeFacade, core: SqlCoreModule) {
params.forEachIndexed { index, value ->
val jdbcIndex = index + 1
when (value) {
ObjNull -> statement.setObject(jdbcIndex, null)
is ObjBool -> statement.setBoolean(jdbcIndex, value.value)
is ObjInt -> statement.setLong(jdbcIndex, value.value)
is ObjReal -> statement.setDouble(jdbcIndex, value.value)
is ObjString -> statement.setString(jdbcIndex, value.value)
is ObjBuffer -> statement.setBytes(jdbcIndex, value.byteArray.toByteArray())
is ObjDate -> statement.setObject(jdbcIndex, java.time.LocalDate.parse(value.date.toString()))
is ObjDateTime -> statement.setObject(jdbcIndex, java.time.LocalDateTime.parse(value.localDateTime.toString()))
is ObjInstant -> statement.setObject(jdbcIndex, java.time.Instant.parse(value.instant.toString()))
else -> when (value.objClass.className) {
"Decimal" -> statement.setBigDecimal(jdbcIndex, BigDecimal(scope.toStringOf(value).value))
else -> scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("Unsupported JDBC parameter type: ${value.objClass.className}")
)
)
}
}
}
}
private suspend fun readResultSet(scope: ScopeFacade, core: SqlCoreModule, resultSet: ResultSet): SqlResultSetData {
val meta = resultSet.metaData
val columns = (1..meta.columnCount).map { index ->
val nativeType = meta.getColumnTypeName(index) ?: ""
SqlColumnMeta(
name = meta.getColumnLabel(index),
sqlType = mapSqlType(core, nativeType, meta.getColumnType(index)),
nullable = meta.isNullable(index) != java.sql.ResultSetMetaData.columnNoNulls,
nativeType = nativeType,
)
}
val rows = mutableListOf<List<Obj>>()
while (resultSet.next()) {
rows += columns.mapIndexed { index, column ->
readColumnValue(scope, core, resultSet, index + 1, column.nativeType)
}
}
return SqlResultSetData(columns, rows)
}
private suspend fun readColumnValue(
scope: ScopeFacade,
core: SqlCoreModule,
resultSet: ResultSet,
index: Int,
nativeType: String,
): Obj {
val value = resultSet.getObject(index) ?: return ObjNull
val normalizedNativeType = normalizeDeclaredTypeName(nativeType)
return when (value) {
is Boolean -> ObjBool(value)
is Byte, is Short, is Int -> ObjInt.of((value as Number).toLong())
is Long -> ObjInt.of(value)
is Float, is Double -> ObjReal.of((value as Number).toDouble())
is java.math.BigInteger -> ObjInt.of(value.longValueExact())
is BigDecimal -> decimalFromString(scope, value.toPlainString())
is ByteArray -> ObjBuffer(value.toUByteArray())
is java.sql.Date -> ObjDate(LocalDate.parse(value.toLocalDate().toString()))
is java.sql.Timestamp -> timestampValue(scope, normalizedNativeType, value.toInstant())
is java.time.LocalDate -> ObjDate(LocalDate.parse(value.toString()))
is java.time.LocalDateTime -> ObjDateTime(value.toString().let(LocalDateTime::parse).toInstant(TimeZone.UTC), TimeZone.UTC)
is java.time.OffsetDateTime -> ObjInstant(Instant.parse(value.toInstant().toString()))
is java.time.ZonedDateTime -> ObjInstant(Instant.parse(value.toInstant().toString()))
is java.time.Instant -> ObjInstant(Instant.parse(value.toString()))
is java.sql.Time -> ObjString(value.toLocalTime().toString())
is java.time.LocalTime -> ObjString(value.toString())
is java.time.OffsetTime -> ObjString(value.toString())
is String -> stringValue(scope, normalizedNativeType, value)
else -> ObjString(value.toString())
}
}
private suspend fun stringValue(scope: ScopeFacade, normalizedNativeType: String, value: String): Obj {
return when {
normalizedNativeType == "DATE" -> ObjDate(LocalDate.parse(value.trim()))
normalizedNativeType == "TIMESTAMP WITH TIME ZONE" || normalizedNativeType == "TIMESTAMPTZ" ->
ObjInstant(Instant.parse(value.trim()))
normalizedNativeType == "TIMESTAMP" || normalizedNativeType == "DATETIME" ->
ObjDateTime(LocalDateTime.parse(value.trim()).toInstant(TimeZone.UTC), TimeZone.UTC)
normalizedNativeType == "DECIMAL" || normalizedNativeType == "NUMERIC" -> decimalFromString(scope, value.trim())
else -> ObjString(value)
}
}
private fun timestampValue(scope: ScopeFacade, normalizedNativeType: String, value: java.time.Instant): Obj {
return if (normalizedNativeType == "TIMESTAMP WITH TIME ZONE" || normalizedNativeType == "TIMESTAMPTZ") {
ObjInstant(Instant.parse(value.toString()))
} else {
val local = value.atOffset(java.time.ZoneOffset.UTC).toLocalDateTime().toString()
ObjDateTime(LocalDateTime.parse(local).toInstant(TimeZone.UTC), TimeZone.UTC)
}
}
private suspend fun decimalFromString(scope: ScopeFacade, value: String): Obj {
val decimalModule = scope.requireScope().currentImportProvider.createModuleScope(scope.pos, "lyng.decimal")
val decimalClass = decimalModule.requireClass("Decimal")
return decimalClass.invokeInstanceMethod(scope.requireScope(), "fromString", ObjString(value))
}
private fun normalizeDeclaredTypeName(nativeTypeName: String): String {
val strippedSuffix = nativeTypeName.trim().replace(Regex("""\s*\(.*\)\s*$"""), "")
return strippedSuffix.uppercase().replace(Regex("""\s+"""), " ").trim()
}
private fun mapSqlType(core: SqlCoreModule, nativeTypeName: String, jdbcType: Int): ObjEnumEntry {
val normalized = normalizeDeclaredTypeName(nativeTypeName)
return when {
normalized == "BOOLEAN" || normalized == "BOOL" -> core.sqlTypes.require("Bool")
normalized == "DATE" -> core.sqlTypes.require("Date")
normalized == "TIMESTAMP" || normalized == "DATETIME" -> core.sqlTypes.require("DateTime")
normalized == "TIMESTAMP WITH TIME ZONE" || normalized == "TIMESTAMPTZ" -> core.sqlTypes.require("Instant")
normalized == "TIME" || normalized == "TIME WITHOUT TIME ZONE" || normalized == "TIME WITH TIME ZONE" -> core.sqlTypes.require("String")
normalized == "DECIMAL" || normalized == "NUMERIC" -> core.sqlTypes.require("Decimal")
normalized.contains("BLOB") || normalized.contains("BINARY") || normalized == "BYTEA" -> core.sqlTypes.require("Binary")
normalized.contains("INT") -> core.sqlTypes.require("Int")
normalized.contains("CHAR") || normalized.contains("TEXT") || normalized.contains("CLOB") || normalized == "VARCHAR" -> core.sqlTypes.require("String")
normalized.contains("REAL") || normalized.contains("FLOA") || normalized.contains("DOUB") -> core.sqlTypes.require("Double")
jdbcType == java.sql.Types.BOOLEAN || jdbcType == java.sql.Types.BIT -> core.sqlTypes.require("Bool")
jdbcType == java.sql.Types.DATE -> core.sqlTypes.require("Date")
jdbcType == java.sql.Types.TIMESTAMP -> core.sqlTypes.require("DateTime")
jdbcType == java.sql.Types.TIMESTAMP_WITH_TIMEZONE -> core.sqlTypes.require("Instant")
jdbcType == java.sql.Types.TIME || jdbcType == java.sql.Types.TIME_WITH_TIMEZONE -> core.sqlTypes.require("String")
jdbcType == java.sql.Types.BLOB || jdbcType == java.sql.Types.BINARY || jdbcType == java.sql.Types.VARBINARY || jdbcType == java.sql.Types.LONGVARBINARY -> core.sqlTypes.require("Binary")
jdbcType == java.sql.Types.INTEGER || jdbcType == java.sql.Types.BIGINT || jdbcType == java.sql.Types.SMALLINT || jdbcType == java.sql.Types.TINYINT -> core.sqlTypes.require("Int")
jdbcType == java.sql.Types.DECIMAL || jdbcType == java.sql.Types.NUMERIC -> core.sqlTypes.require("Decimal")
jdbcType == java.sql.Types.FLOAT || jdbcType == java.sql.Types.REAL || jdbcType == java.sql.Types.DOUBLE -> core.sqlTypes.require("Double")
else -> core.sqlTypes.require("String")
}
}
private fun emptyResultSet(): SqlResultSetData = SqlResultSetData(emptyList(), emptyList())
private fun containsRowReturningClause(clause: String): Boolean =
Regex("""\b(returning|output)\b""", RegexOption.IGNORE_CASE).containsMatchIn(clause)
private fun ensureJdbcDriversLoaded(scope: ScopeFacade, core: SqlCoreModule, requestedDriverClass: String?) {
for (driverClass in knownJdbcDrivers) {
try {
Class.forName(driverClass)
} catch (_: Throwable) {
}
}
val explicit = requestedDriverClass ?: return
try {
Class.forName(explicit)
} catch (e: ClassNotFoundException) {
throw ExecutionError(
ObjException(core.databaseException, scope.requireScope(), ObjString("JDBC driver class not found: $explicit")),
scope.pos,
"JDBC driver class not found: $explicit",
e,
)
}
}
private fun rollbackOrThrow(scope: ScopeFacade, core: SqlCoreModule, connection: Connection) {
try {
connection.rollback()
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
private fun rollbackToSavepointOrThrow(
scope: ScopeFacade,
core: SqlCoreModule,
connection: Connection,
savepoint: java.sql.Savepoint,
) {
try {
connection.rollback(savepoint)
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
private fun releaseSavepointOrThrow(
scope: ScopeFacade,
core: SqlCoreModule,
connection: Connection,
savepoint: java.sql.Savepoint,
) {
try {
connection.releaseSavepoint(savepoint)
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
private inline fun finishFailedTransaction(
scope: ScopeFacade,
core: SqlCoreModule,
failure: Throwable,
rollback: () -> Unit,
): Throwable {
return try {
rollback()
failure
} catch (rollbackFailure: Throwable) {
if (isRollbackSignal(failure, core)) {
attachSecondaryFailure(rollbackFailure, failure)
rollbackFailure
} else {
attachSecondaryFailure(failure, rollbackFailure)
failure
}
}
}
private fun isRollbackSignal(failure: Throwable, core: SqlCoreModule): Boolean {
val errorObject = (failure as? ExecutionError)?.errorObject ?: return false
return errorObject.isInstanceOf(core.rollbackException)
}
private fun attachSecondaryFailure(primary: Throwable, secondary: Throwable) {
if (primary === secondary) return
primary.addSuppressed(secondary)
}
private fun mapOpenException(scope: ScopeFacade, core: SqlCoreModule, e: SQLException): Nothing {
val message = e.message ?: "JDBC open failed"
if (e is SQLNonTransientConnectionException) {
throw ExecutionError(
ObjException(core.databaseException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
e,
)
}
throw ExecutionError(
ObjException(core.databaseException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
e,
)
}
private fun mapSqlException(scope: ScopeFacade, core: SqlCoreModule, e: SQLException): ExecutionError {
val exceptionClass = when {
e is SQLIntegrityConstraintViolationException -> core.sqlConstraintException
e.sqlState?.startsWith("23") == true -> core.sqlConstraintException
else -> core.sqlExecutionException
}
return ExecutionError(
ObjException(exceptionClass, scope.requireScope(), ObjString(e.message ?: "JDBC error")),
scope.pos,
e.message ?: "JDBC error",
e,
)
}
private fun mapSqlUsage(scope: ScopeFacade, core: SqlCoreModule, message: String, cause: Throwable? = null): ExecutionError {
return ExecutionError(
ObjException(core.sqlUsageException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
cause,
)
}
@@ -0,0 +1,507 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.Arguments
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.SqlColumnMeta
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.io.db.SqlExecutionResultData
import net.sergeych.lyng.io.db.SqlResultSetData
import net.sergeych.lyng.io.db.SqlTransactionBackend
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjBool
import net.sergeych.lyng.obj.ObjBuffer
import net.sergeych.lyng.obj.ObjDate
import net.sergeych.lyng.obj.ObjDateTime
import net.sergeych.lyng.obj.ObjEnumEntry
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjInstant
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjReal
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.requireScope
import kotlinx.datetime.LocalDate
import kotlinx.datetime.LocalDateTime
import kotlinx.datetime.TimeZone
import kotlinx.datetime.toInstant
import org.sqlite.SQLiteConfig
import org.sqlite.SQLiteErrorCode
import org.sqlite.SQLiteOpenMode
import java.sql.Connection
import java.sql.DriverManager
import java.sql.PreparedStatement
import java.sql.ResultSet
import java.sql.SQLException
import java.sql.Statement
import kotlin.time.Instant
internal actual suspend fun openSqliteBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: SqliteOpenOptions,
): SqlDatabaseBackend {
if (options.busyTimeoutMillis < 0) {
scope.raiseIllegalArgument("busyTimeoutMillis must be >= 0")
}
return JdbcSqlDatabaseBackend(core, options)
}
private class JdbcSqlDatabaseBackend(
private val core: SqlCoreModule,
private val options: SqliteOpenOptions,
) : SqlDatabaseBackend {
override suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T {
val connection = openConnection(scope)
try {
connection.autoCommit = false
val tx = JdbcSqlTransactionBackend(core, connection)
val result = try {
block(tx)
} catch (e: Throwable) {
throw finishFailedTransaction(scope, core, e) {
rollbackOrThrow(scope, core, connection)
}
}
try {
connection.commit()
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
return result
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
} finally {
try {
connection.close()
} catch (_: SQLException) {
}
}
}
private fun openConnection(scope: ScopeFacade): Connection {
try {
val config = SQLiteConfig().apply {
setOpenMode(SQLiteOpenMode.OPEN_URI)
if (options.readOnly) {
setReadOnly(true)
setOpenMode(SQLiteOpenMode.READONLY)
} else {
setReadOnly(false)
setOpenMode(SQLiteOpenMode.READWRITE)
if (options.createIfMissing) {
setOpenMode(SQLiteOpenMode.CREATE)
}
}
enforceForeignKeys(options.foreignKeys)
busyTimeout = options.busyTimeoutMillis
}
return DriverManager.getConnection(jdbcUrl(options.path), config.toProperties())
} catch (e: SQLException) {
throw mapOpenException(scope, core, e)
} catch (e: IllegalArgumentException) {
scope.raiseIllegalArgument(e.message ?: "Invalid SQLite configuration")
}
}
private fun jdbcUrl(path: String): String {
return when (path) {
":memory:" -> "jdbc:sqlite::memory:"
else -> if (path.startsWith("/")) "jdbc:sqlite:$path" else "jdbc:sqlite:$path"
}
}
}
private class JdbcSqlTransactionBackend(
private val core: SqlCoreModule,
private val connection: Connection,
) : SqlTransactionBackend {
override suspend fun select(scope: ScopeFacade, clause: String, params: List<Obj>): SqlResultSetData {
try {
connection.prepareStatement(clause).use { statement ->
bindParams(statement, params, scope, core)
statement.executeQuery().use { rs ->
return readResultSet(scope, core, rs)
}
}
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
override suspend fun execute(scope: ScopeFacade, clause: String, params: List<Obj>): SqlExecutionResultData {
if (containsRowReturningClause(clause)) {
scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("execute(...) cannot be used with statements that return rows; use select(...)")
)
)
}
try {
connection.prepareStatement(clause, Statement.RETURN_GENERATED_KEYS).use { statement ->
bindParams(statement, params, scope, core)
val hasResultSet = statement.execute()
if (hasResultSet) {
scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("execute(...) cannot be used with statements that return rows; use select(...)")
)
)
}
val affected = statement.updateCount
val generatedKeys = statement.generatedKeys.use { rs ->
if (rs == null) {
emptyResultSet(core)
} else {
readResultSet(scope, core, rs)
}
}
return SqlExecutionResultData(affected, generatedKeys)
}
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
override suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T {
val savepoint = try {
connection.setSavepoint()
} catch (e: SQLException) {
throw mapSqlUsage(scope, core, "Nested transactions are not supported by this SQLite backend", e)
}
val nested = JdbcSqlTransactionBackend(core, connection)
val result = try {
block(nested)
} catch (e: Throwable) {
throw finishFailedTransaction(scope, core, e) {
rollbackToSavepointOrThrow(scope, core, connection, savepoint)
releaseSavepointOrThrow(scope, core, connection, savepoint)
}
}
try {
connection.releaseSavepoint(savepoint)
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
return result
}
}
private suspend fun bindParams(statement: PreparedStatement, params: List<Obj>, scope: ScopeFacade, core: SqlCoreModule) {
params.forEachIndexed { index, value ->
val jdbcIndex = index + 1
when (value) {
ObjNull -> statement.setObject(jdbcIndex, null)
is ObjBool -> statement.setLong(jdbcIndex, if (value.value) 1L else 0L)
is ObjInt -> statement.setLong(jdbcIndex, value.value)
is ObjReal -> statement.setDouble(jdbcIndex, value.value)
is ObjString -> statement.setString(jdbcIndex, value.value)
is ObjBuffer -> statement.setBytes(jdbcIndex, value.byteArray.toByteArray())
is ObjInstant -> statement.setString(jdbcIndex, value.instant.toString())
is ObjDateTime -> statement.setString(jdbcIndex, value.localDateTime.toString())
else -> when (value.objClass.className) {
"Date", "Decimal" -> statement.setString(jdbcIndex, scope.toStringOf(value).value)
else -> scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("Unsupported SQLite parameter type: ${value.objClass.className}")
)
)
}
}
}
}
private suspend fun readResultSet(
scope: ScopeFacade,
core: SqlCoreModule,
resultSet: ResultSet,
): SqlResultSetData {
val meta = resultSet.metaData
val columns = (1..meta.columnCount).map { index ->
SqlColumnMeta(
name = meta.getColumnLabel(index),
sqlType = mapSqlType(core, meta.getColumnTypeName(index), meta.getColumnType(index)),
nullable = meta.isNullable(index) != java.sql.ResultSetMetaData.columnNoNulls,
nativeType = meta.getColumnTypeName(index) ?: "",
)
}
val rows = mutableListOf<List<Obj>>()
while (resultSet.next()) {
rows += columns.mapIndexed { index, column ->
readColumnValue(scope, core, resultSet, index + 1, column.nativeType)
}
}
return SqlResultSetData(columns, rows)
}
private suspend fun readColumnValue(
scope: ScopeFacade,
core: SqlCoreModule,
resultSet: ResultSet,
index: Int,
nativeType: String,
): Obj {
val value = resultSet.getObject(index) ?: return ObjNull
val normalizedNativeType = normalizeDeclaredTypeName(nativeType)
if (isDecimalNativeType(normalizedNativeType) && value is Number) {
return decimalFromString(scope, value.toString())
}
return when (value) {
is Boolean -> if (isBooleanNativeType(normalizedNativeType)) ObjBool(value) else ObjInt.of(if (value) 1 else 0)
is Byte, is Short, is Int -> convertIntegerValue(scope, core, normalizedNativeType, (value as Number).toLong())
is Long -> convertIntegerValue(scope, core, normalizedNativeType, value)
is Float, is Double -> ObjReal.of((value as Number).toDouble())
is ByteArray -> ObjBuffer(value.toUByteArray())
is String -> convertStringValue(scope, core, normalizedNativeType, value)
is java.math.BigDecimal -> decimalFromString(scope, value.toPlainString())
else -> ObjString(value.toString())
}
}
private fun convertIntegerValue(
scope: ScopeFacade,
core: SqlCoreModule,
normalizedNativeType: String,
value: Long,
): Obj {
if (!isBooleanNativeType(normalizedNativeType)) {
return ObjInt.of(value)
}
return when (value) {
0L -> ObjBool(false)
1L -> ObjBool(true)
else -> sqlExecutionFailure(scope, core, "Invalid SQLite boolean value: $value")
}
}
private suspend fun convertStringValue(
scope: ScopeFacade,
core: SqlCoreModule,
normalizedNativeType: String,
value: String,
): Obj {
return when {
isBooleanNativeType(normalizedNativeType) -> booleanFromString(scope, core, value)
isDecimalNativeType(normalizedNativeType) -> decimalFromString(scope, value.trim())
normalizedNativeType == "DATE" -> ObjDate(LocalDate.parse(value.trim()))
normalizedNativeType == "DATETIME" || normalizedNativeType == "TIMESTAMP" ->
dateTimeFromString(scope, core, value)
normalizedNativeType == "TIMESTAMP WITH TIME ZONE" ||
normalizedNativeType == "TIMESTAMPTZ" ||
normalizedNativeType == "DATETIME WITH TIME ZONE" -> ObjInstant(Instant.parse(value.trim()))
else -> ObjString(value)
}
}
private fun isDecimalNativeType(normalizedNativeType: String): Boolean =
normalizedNativeType == "DECIMAL" || normalizedNativeType == "NUMERIC"
private fun isBooleanNativeType(normalizedNativeType: String): Boolean =
normalizedNativeType == "BOOLEAN" || normalizedNativeType == "BOOL"
private fun booleanFromString(scope: ScopeFacade, core: SqlCoreModule, value: String): Obj {
return when (value.trim().lowercase()) {
"true", "t" -> ObjBool(true)
"false", "f" -> ObjBool(false)
else -> sqlExecutionFailure(scope, core, "Invalid SQLite boolean value: $value")
}
}
private suspend fun decimalFromString(scope: ScopeFacade, value: String): Obj {
val decimalModule = scope.requireScope().currentImportProvider.createModuleScope(scope.pos, "lyng.decimal")
val decimalClass = decimalModule.requireClass("Decimal")
return decimalClass.invokeInstanceMethod(scope.requireScope(), "fromString", ObjString(value))
}
private fun dateTimeFromString(scope: ScopeFacade, core: SqlCoreModule, value: String): ObjDateTime {
val trimmed = value.trim()
if (hasExplicitTimeZone(trimmed)) {
sqlExecutionFailure(scope, core, "SQLite TIMESTAMP/DATETIME value must not contain a timezone offset: $value")
}
val local = LocalDateTime.parse(trimmed)
return ObjDateTime(local.toInstant(TimeZone.UTC), TimeZone.UTC)
}
private fun hasExplicitTimeZone(value: String): Boolean {
if (value.endsWith("Z", ignoreCase = true)) return true
val tIndex = value.indexOf('T')
if (tIndex < 0) return false
val plus = value.lastIndexOf('+')
val minus = value.lastIndexOf('-')
val offsetStart = maxOf(plus, minus)
return offsetStart > tIndex
}
private fun containsRowReturningClause(clause: String): Boolean =
Regex("""\breturning\b""", RegexOption.IGNORE_CASE).containsMatchIn(clause)
private fun normalizeDeclaredTypeName(nativeTypeName: String): String {
val strippedSuffix = nativeTypeName.trim().replace(Regex("""\s*\(.*\)\s*$"""), "")
return strippedSuffix.uppercase().replace(Regex("""\s+"""), " ").trim()
}
private fun mapSqlType(core: SqlCoreModule, nativeTypeName: String, jdbcType: Int): ObjEnumEntry {
val normalized = normalizeDeclaredTypeName(nativeTypeName)
return when {
normalized == "BOOLEAN" || normalized == "BOOL" -> core.sqlTypes.require("Bool")
normalized == "DATE" -> core.sqlTypes.require("Date")
normalized == "DATETIME" || normalized == "TIMESTAMP" -> core.sqlTypes.require("DateTime")
normalized == "TIMESTAMP WITH TIME ZONE" ||
normalized == "TIMESTAMPTZ" ||
normalized == "DATETIME WITH TIME ZONE" -> core.sqlTypes.require("Instant")
normalized == "TIME" ||
normalized == "TIME WITHOUT TIME ZONE" ||
normalized == "TIME WITH TIME ZONE" -> core.sqlTypes.require("String")
normalized == "DECIMAL" || normalized == "NUMERIC" -> core.sqlTypes.require("Decimal")
normalized.contains("BLOB") -> core.sqlTypes.require("Binary")
normalized.contains("INT") -> core.sqlTypes.require("Int")
normalized.contains("CHAR") || normalized.contains("TEXT") || normalized.contains("CLOB") -> core.sqlTypes.require("String")
normalized.contains("REAL") || normalized.contains("FLOA") || normalized.contains("DOUB") -> core.sqlTypes.require("Double")
jdbcType == java.sql.Types.BOOLEAN -> core.sqlTypes.require("Bool")
jdbcType == java.sql.Types.BLOB || jdbcType == java.sql.Types.BINARY || jdbcType == java.sql.Types.VARBINARY -> core.sqlTypes.require("Binary")
jdbcType == java.sql.Types.INTEGER -> core.sqlTypes.require("Int")
jdbcType == java.sql.Types.BIGINT -> core.sqlTypes.require("Int")
jdbcType == java.sql.Types.DECIMAL || jdbcType == java.sql.Types.NUMERIC -> core.sqlTypes.require("Decimal")
jdbcType == java.sql.Types.FLOAT || jdbcType == java.sql.Types.REAL || jdbcType == java.sql.Types.DOUBLE -> core.sqlTypes.require("Double")
else -> core.sqlTypes.require("String")
}
}
private fun emptyResultSet(core: SqlCoreModule): SqlResultSetData = SqlResultSetData(emptyList(), emptyList())
private fun sqlExecutionFailure(scope: ScopeFacade, core: SqlCoreModule, message: String): Nothing {
throw ExecutionError(
ObjException(core.sqlExecutionException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
)
}
private fun rollbackOrThrow(scope: ScopeFacade, core: SqlCoreModule, connection: Connection) {
try {
connection.rollback()
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
private fun rollbackToSavepointOrThrow(
scope: ScopeFacade,
core: SqlCoreModule,
connection: Connection,
savepoint: java.sql.Savepoint,
) {
try {
connection.rollback(savepoint)
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
private fun releaseSavepointOrThrow(
scope: ScopeFacade,
core: SqlCoreModule,
connection: Connection,
savepoint: java.sql.Savepoint,
) {
try {
connection.releaseSavepoint(savepoint)
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
private inline fun finishFailedTransaction(
scope: ScopeFacade,
core: SqlCoreModule,
failure: Throwable,
rollback: () -> Unit,
): Throwable {
return try {
rollback()
failure
} catch (rollbackFailure: Throwable) {
if (isRollbackSignal(failure, core)) {
attachSecondaryFailure(rollbackFailure, failure)
rollbackFailure
} else {
attachSecondaryFailure(failure, rollbackFailure)
failure
}
}
}
private fun isRollbackSignal(failure: Throwable, core: SqlCoreModule): Boolean {
val errorObject = (failure as? ExecutionError)?.errorObject ?: return false
return errorObject.isInstanceOf(core.rollbackException)
}
private fun attachSecondaryFailure(primary: Throwable, secondary: Throwable) {
if (primary === secondary) return
primary.addSuppressed(secondary)
}
private fun mapOpenException(scope: ScopeFacade, core: SqlCoreModule, e: SQLException): Nothing {
val message = e.message ?: "SQLite open failed"
val lower = message.lowercase()
if ("malformed" in lower || "no such access mode" in lower || "invalid uri" in lower) {
scope.raiseIllegalArgument(message)
}
throw ExecutionError(
ObjException(core.databaseException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
e,
)
}
private fun mapSqlException(scope: ScopeFacade, core: SqlCoreModule, e: SQLException): ExecutionError {
val code = SQLiteErrorCode.getErrorCode(e.errorCode)
val exceptionClass = when (code) {
SQLiteErrorCode.SQLITE_CONSTRAINT,
SQLiteErrorCode.SQLITE_CONSTRAINT_PRIMARYKEY,
SQLiteErrorCode.SQLITE_CONSTRAINT_UNIQUE,
SQLiteErrorCode.SQLITE_CONSTRAINT_FOREIGNKEY,
SQLiteErrorCode.SQLITE_CONSTRAINT_NOTNULL -> core.sqlConstraintException
else -> core.sqlExecutionException
}
return ExecutionError(
ObjException(exceptionClass, scope.requireScope(), ObjString(e.message ?: "SQLite error")),
scope.pos,
e.message ?: "SQLite error",
e,
)
}
private fun mapSqlUsage(scope: ScopeFacade, core: SqlCoreModule, message: String, cause: Throwable? = null): ExecutionError {
return ExecutionError(
ObjException(core.sqlUsageException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
cause,
)
}
@@ -0,0 +1,163 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.jdbc
import kotlinx.coroutines.test.runTest
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Script
import net.sergeych.lyng.Source
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjExternCallable
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjMap
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.requireScope
import kotlin.test.Test
import kotlin.test.assertEquals
class LyngJdbcModuleTest {
@Test
fun testTypedOpenH2ExecutesQueriesAndGeneratedKeys() = runTest {
val scope = Script.newScope()
createJdbcModule(scope.importManager)
val jdbcModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.jdbc")
val db = jdbcModule.callFn("openH2", ObjString("mem:typed_h2_${System.nanoTime()};DB_CLOSE_DELAY=-1"))
val insertedId = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table person(id bigint auto_increment primary key, name varchar(120) not null)")
)
val result = tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into person(name) values(?)"),
ObjString("Ada")
)
val rows = result.invokeInstanceMethod(requireScope(), "getGeneratedKeys")
.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjInt.Zero)
}
) as ObjInt
assertEquals(1L, insertedId.value)
}
@Test
fun testGenericOpenDatabaseUsesJdbcAndH2AliasProviders() = runTest {
val scope = Script.newScope()
createJdbcModule(scope.importManager)
scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.jdbc")
val dbModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
val genericJdbcDb = dbModule.callFn(
"openDatabase",
ObjString("jdbc:h2:mem:generic_jdbc_${System.nanoTime()};DB_CLOSE_DELAY=-1"),
emptyMapObj()
)
val h2AliasDb = dbModule.callFn(
"openDatabase",
ObjString("h2:mem:generic_alias_${System.nanoTime()};DB_CLOSE_DELAY=-1"),
emptyMapObj()
)
assertEquals(42L, scalarSelect(scope, genericJdbcDb, "select 42 as answer"))
assertEquals(7L, scalarSelect(scope, h2AliasDb, "select 7 as answer"))
}
@Test
fun testImportedJdbcOpenersPreserveDeclaredReturnTypesForInference() = runTest {
val scope = Script.newScope()
createJdbcModule(scope.importManager)
val code = """
import lyng.io.db
import lyng.io.db.jdbc
val h2db = openH2("mem:inference_demo;DB_CLOSE_DELAY=-1")
h2db.transaction { 1 }
val jdbcDb = openJdbc("jdbc:h2:mem:inference_demo_2;DB_CLOSE_DELAY=-1")
jdbcDb.transaction { 2 }
val genericDb = openDatabase("jdbc:h2:mem:inference_demo_3;DB_CLOSE_DELAY=-1", Map())
genericDb.transaction { 3 }
""".trimIndent()
val result = Compiler.compile(Source("<jdbc-inference>", code), scope.importManager).execute(scope) as ObjInt
assertEquals(3L, result.value)
}
@Test
fun testMaterializedRowsListCanBeReturnedFromTransaction() = runTest {
val scope = Script.newScope()
createJdbcModule(scope.importManager)
val jdbcModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.jdbc")
val db = jdbcModule.callFn("openH2", ObjString("mem:rows_return_${System.nanoTime()};DB_CLOSE_DELAY=-1"))
val rows = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table person(id bigint auto_increment primary key, name varchar(120) not null)")
)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into person(name) values(?)"), ObjString("Ada"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into person(name) values(?)"), ObjString("Linus"))
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select name from person order by id"))
.invokeInstanceMethod(requireScope(), "toList")
}
)
assertEquals("Ada", (rows.getAt(scope, ObjInt.Zero).getAt(scope, ObjString("name")) as ObjString).value)
assertEquals("Linus", (rows.getAt(scope, ObjInt.of(1)).getAt(scope, ObjString("name")) as ObjString).value)
}
private suspend fun scalarSelect(scope: net.sergeych.lyng.Scope, db: Obj, sql: String): Long {
val result = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
val rows = tx.invokeInstanceMethod(requireScope(), "select", ObjString(sql))
.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjString("answer"))
}
) as ObjInt
return result.value
}
private suspend fun net.sergeych.lyng.ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
private fun emptyMapObj(): Obj = ObjMap()
}
@@ -0,0 +1,123 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.jdbc
import kotlinx.coroutines.test.runTest
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Script
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjExternCallable
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjMap
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.requireScope
import org.testcontainers.containers.PostgreSQLContainer
import kotlin.test.Test
import kotlin.test.assertEquals
class LyngJdbcPostgresContainerTest {
@Test
fun testOpenPostgresAgainstContainer() = runTest {
PostgreSQLContainer("postgres:16-alpine").use { postgres ->
postgres.start()
val scope = Script.newScope()
createJdbcModule(scope.importManager)
val jdbcModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.jdbc")
val db = jdbcModule.callFn(
"openPostgres",
ObjString(postgres.jdbcUrl),
ObjString(postgres.username),
ObjString(postgres.password)
)
val count = rowCount(scope, db, "people", "Ada", "Linus")
assertEquals(2L, count)
}
}
@Test
fun testGenericPostgresAliasAgainstContainer() = runTest {
PostgreSQLContainer("postgres:16-alpine").use { postgres ->
postgres.start()
val scope = Script.newScope()
createJdbcModule(scope.importManager)
scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.jdbc")
val dbModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
val aliasUrl = postgres.jdbcUrl.removePrefix("jdbc:postgresql:")
val db = dbModule.callFn(
"openDatabase",
ObjString("postgres:$aliasUrl"),
mapOfStrings(
"user" to postgres.username,
"password" to postgres.password
)
)
val count = rowCount(scope, db, "pets", "Milo", "Otis", "Pixel")
assertEquals(3L, count)
}
}
private suspend fun rowCount(scope: net.sergeych.lyng.Scope, db: Obj, table: String, vararg names: String): Long {
val result = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table $table(id bigserial primary key, name text not null)")
)
for (name in names) {
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into $table(name) values(?)"),
ObjString(name)
)
}
val rows = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("select count(*) as count from $table")
).invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjString("count"))
}
) as ObjInt
return result.value
}
private suspend fun mapOfStrings(vararg entries: Pair<String, String>): Obj {
val map = ObjMap()
for ((key, value) in entries) {
map.map[ObjString(key)] = ObjString(value)
}
return map
}
private suspend fun net.sergeych.lyng.ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
}
@@ -0,0 +1,949 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import kotlinx.coroutines.test.runTest
import kotlinx.datetime.TimeZone
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.Script
import net.sergeych.lyng.Source
import net.sergeych.lyng.obj.*
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.requireScope
import java.nio.file.Files
import kotlin.io.path.deleteIfExists
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
import kotlin.test.assertTrue
import kotlin.time.Instant
class LyngSqliteModuleTest {
@Test
fun testTypedOpenSqliteExecutesQueriesAndGeneratedKeys() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val db = sqliteModule.callFn("openSqlite", ObjString(":memory:"))
val insertedId = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table person(id integer primary key autoincrement, name text not null)"))
val result = tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into person(name) values(?)"),
ObjString("John Doe")
)
val generatedKeys = result.invokeInstanceMethod(requireScope(), "getGeneratedKeys")
val rows = generatedKeys.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjInt.Zero)
}
) as ObjInt
assertEquals(1L, insertedId.value)
}
@Test
fun testGenericOpenDatabaseUsesRegisteredSqliteProvider() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val dbModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
val db = dbModule.callFn("openDatabase", ObjString("sqlite::memory:"), emptyMapObj())
val count = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table items(id integer primary key autoincrement, name text not null)"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("alpha"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("beta"))
val resultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select count(*) as count from items"))
val rows = resultSet.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjString("count"))
}
) as ObjInt
assertEquals(2L, count.value)
}
@Test
fun testImportedDatabaseOpenersPreserveDeclaredReturnTypesForInference() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val code = """
import lyng.io.db
import lyng.io.db.sqlite
val typedDb = openSqlite(":memory:")
typedDb.transaction { 1 }
val genericDb = openDatabase("sqlite::memory:", Map())
genericDb.transaction { 2 }
""".trimIndent()
Compiler.compile(Source("<sqlite-inference>", code), scope.importManager).execute(scope)
}
@Test
fun testTransactionLambdaParameterTypeIsInferredFromDatabaseSignature() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val code = """
import lyng.io.db
import lyng.io.db.sqlite
import lyng.time
val db = openSqlite(":memory:")
db.transaction { tx ->
tx.execute("create table item(id integer primary key autoincrement, name text not null, due_date date not null)")
tx.execute("insert into item(name, due_date) values(?, ?)", "outer", Date(2026, 4, 16))
tx.transaction { inner ->
inner.execute("insert into item(name, due_date) values(?, ?)", "inner", Date(2026, 4, 17))
1
}
2
}
""".trimIndent()
val result = Compiler.compile(Source("<sqlite-lambda-inference>", code), scope.importManager).execute(scope) as ObjInt
assertEquals(2L, result.value)
}
@Test
fun testNestedTransactionRollbackUsesSavepoint() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val db = sqliteModule.callFn("openSqlite", ObjString(":memory:"))
val count = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table items(id integer primary key autoincrement, name text not null)"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("outer"))
try {
tx.invokeInstanceMethod(
requireScope(),
"transaction",
ObjExternCallable.fromBridge {
val inner = requiredArg<Obj>(0)
inner.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("inner"))
throw IllegalStateException("rollback nested")
}
)
} catch (_: IllegalStateException) {
}
val resultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select count(*) as count from items"))
val rows = resultSet.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjString("count"))
}
) as ObjInt
assertEquals(1L, count.value)
}
@Test
fun testRollbackExceptionRollsBackAndPropagates() = runTest {
val scope = Script.newScope()
withTempDb(scope) { db ->
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table items(id integer primary key autoincrement, name text not null)")
)
}
)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into items(name) values(?)"),
ObjString("rolled-back")
)
rollbackException(requireScope(), "stop here").raiseAsExecutionError(requireScope())
}
)
}
assertEquals("RollbackException", error.errorObject.objClass.className)
val count = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
val resultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select count(*) as count from items"))
rowsOf(requireScope(), resultSet)[0].getAt(requireScope(), ObjString("count"))
}
) as ObjInt
assertEquals(0L, count.value)
}
}
@Test
fun testResultSetFailsAfterTransactionEnds() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val db = sqliteModule.callFn("openSqlite", ObjString(":memory:"))
var leakedResultSet: Obj = ObjNull
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table items(id integer primary key autoincrement, name text not null)"))
leakedResultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select 42 as answer"))
ObjNull
}
)
val error = assertFailsWith<ExecutionError> {
leakedResultSet.invokeInstanceMethod(scope, "size")
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("transaction is active"), error.errorMessage)
}
@Test
fun testMaterializedRowSurvivesAfterTransactionEnds() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
var leakedRow: Obj = ObjNull
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
leakedRow = rowsOf(requireScope(), tx.invokeInstanceMethod(requireScope(), "select", ObjString("select 42 as answer")))[0]
ObjNull
}
)
val answer = leakedRow.getAt(scope, ObjString("answer")) as ObjInt
assertEquals(42L, answer.value)
}
@Test
fun testMaterializedRowsListCanBeReturnedFromTransaction() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val rows = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table items(id integer primary key autoincrement, name text not null)")
)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("alpha"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("beta"))
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select name from items order by id"))
.invokeInstanceMethod(requireScope(), "toList")
}
)
assertEquals("alpha", stringValue(scope, rows.getAt(scope, ObjInt.Zero).getAt(scope, ObjString("name"))))
assertEquals("beta", stringValue(scope, rows.getAt(scope, ObjInt.of(1)).getAt(scope, ObjString("name"))))
}
@Test
fun testInvalidSqliteUrlFailsWithIllegalArgument() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val dbModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
val error = assertFailsWith<ExecutionError> {
dbModule.callFn("openDatabase", ObjString("sqlite://bad"), emptyMapObj())
}
assertEquals("IllegalArgumentException", error.errorObject.objClass.className)
}
@Test
fun testConstraintViolationIsMappedToSqlConstraintException() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table person(id integer primary key autoincrement, email text unique not null)")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into person(email) values(?)"),
ObjString("a@example.com")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into person(email) values(?)"),
ObjString("a@example.com")
)
}
)
}
assertEquals("SqlConstraintException", error.errorObject.objClass.className)
}
@Test
fun testAmbiguousColumnNameAccessFailsWithSqlUsageException() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("select 1 as value, 2 as value")
)
val row = rowsOf(requireScope(), resultSet)[0]
row.getAt(requireScope(), ObjString("value"))
}
)
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("Ambiguous SQL result column"), error.errorMessage)
}
@Test
fun testExecuteRejectsReturningButSelectSupportsIt() = runTest {
val scope = Script.newScope()
withTempDb(scope) { db ->
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table item(id integer primary key autoincrement, name text not null)")
)
}
)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into item(name) values(?) returning id"),
ObjString("bad")
)
}
)
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
val insertedId = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("insert into item(name) values(?) returning id"),
ObjString("good")
)
val row = rowsOf(requireScope(), resultSet)[0]
row.getAt(requireScope(), ObjString("id"))
}
) as ObjInt
assertEquals(1L, insertedId.value)
}
}
@Test
fun testColumnMetadataAndTypedValueConversion() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val summary = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString(
"create table events(" +
"amount NUMERIC not null, " +
"happened TIMESTAMPTZ not null, " +
"scheduled TIMESTAMP not null, " +
"note TEXT not null, " +
"payload BLOB not null)"
)
)
val decimal = decimalOf(requireScope(), "12.50")
val happened = ObjInstant(Instant.parse("2024-05-06T07:08:09Z"))
val scheduled = ObjDateTime(Instant.parse("2024-05-06T10:11:12Z"), TimeZone.UTC)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into events(amount, happened, scheduled, note, payload) values(?, ?, ?, ?, ?)"),
decimal,
happened,
scheduled,
ObjString("hello"),
ObjBuffer(byteArrayOf(1, 2, 3).toUByteArray())
)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("select amount, happened, scheduled, note, payload from events")
)
val columns = field(requireScope(), resultSet, "columns")
val firstColumn = columns.getAt(requireScope(), ObjInt.Zero)
val row = rowsOf(requireScope(), resultSet)[0]
ObjString(
listOf(
stringValue(requireScope(), field(requireScope(), firstColumn, "name")),
enumName(requireScope(), field(requireScope(), firstColumn, "sqlType")),
stringValue(requireScope(), field(requireScope(), firstColumn, "nativeType")),
row.getAt(requireScope(), ObjString("amount")).objClass.className,
row.getAt(requireScope(), ObjString("happened")).objClass.className,
row.getAt(requireScope(), ObjString("scheduled")).objClass.className,
stringValue(requireScope(), row.getAt(requireScope(), ObjString("note"))),
row.getAt(requireScope(), ObjString("payload")).objClass.className,
).joinToString("|")
)
}
) as ObjString
assertEquals(
"amount|Decimal|NUMERIC|Decimal|Instant|DateTime|hello|Buffer",
summary.value
)
}
@Test
fun testDateAndBooleanConversionRules() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val summary = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString(
"create table sample(" +
"flag BOOL not null, " +
"day DATE not null, " +
"clock TIME not null)"
)
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(flag, day, clock) values(?, ?, ?)"),
ObjBool(true),
dateOf(requireScope(), "2026-04-15"),
ObjString("12:34:56")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(flag, day, clock) values(?, ?, ?)"),
ObjString("t"),
ObjString("2026-04-16"),
ObjString("23:59:59")
)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("select flag, day, clock from sample order by day")
)
val rows = rowsOf(requireScope(), resultSet)
ObjString(
listOf(
rows[0].getAt(requireScope(), ObjString("flag")).objClass.className,
rows[0].getAt(requireScope(), ObjString("day")).objClass.className,
stringValue(requireScope(), rows[0].getAt(requireScope(), ObjString("clock"))),
rows[1].getAt(requireScope(), ObjString("flag")).objClass.className,
).joinToString("|")
)
}
) as ObjString
assertEquals("Bool|Date|12:34:56|Bool", summary.value)
}
@Test
fun testUnsupportedParameterTypeFailsWithSqlUsageException() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table sample(value text not null)")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(value) values(?)"),
emptyMapObj()
)
}
)
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("Unsupported SQLite parameter type"), error.errorMessage)
}
@Test
fun testTimestampAndDatetimeRejectTimezoneBearingText() = runTest {
val scope = Script.newScope()
withTempDb(scope) { db ->
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table sample(ts TIMESTAMP not null, dt DATETIME not null)")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(ts, dt) values(?, ?)"),
ObjString("2024-05-06T07:08:09Z"),
ObjString("2024-05-06T10:11:12+03:00")
)
}
)
val timestampError = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select ts from sample"))
}
)
}
assertEquals("SqlExecutionException", timestampError.errorObject.objClass.className)
assertTrue(timestampError.errorMessage.contains("must not contain a timezone offset"), timestampError.errorMessage)
val datetimeError = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select dt from sample"))
}
)
}
assertEquals("SqlExecutionException", datetimeError.errorObject.objClass.className)
assertTrue(datetimeError.errorMessage.contains("must not contain a timezone offset"), datetimeError.errorMessage)
}
}
@Test
fun testReadOnlyOpenPreventsWrites() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val tempFile = Files.createTempFile("lyng-sqlite-", ".db")
try {
val writableDb = sqliteModule.callFn("openSqlite", ObjString(tempFile.toString()))
writableDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table item(id integer primary key autoincrement, name text not null)")
)
}
)
val readOnlyDb = sqliteModule.callFn(
"openSqlite",
ObjString(tempFile.toString()),
net.sergeych.lyng.obj.ObjTrue,
net.sergeych.lyng.obj.ObjFalse
)
val error = assertFailsWith<ExecutionError> {
readOnlyDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into item(name) values(?)"),
ObjString("blocked")
)
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
} finally {
tempFile.deleteIfExists()
}
}
@Test
fun testMissingFileWithCreateIfMissingFalseFailsWithDatabaseException() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val missingDir = Files.createTempDirectory("lyng-sqlite-missing-dir-")
missingDir.deleteIfExists()
val missingFile = missingDir.resolve("missing.db")
try {
val db = sqliteModule.callFn(
"openSqlite",
ObjString(missingFile.toString()),
net.sergeych.lyng.obj.ObjFalse,
net.sergeych.lyng.obj.ObjFalse
)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge { ObjNull }
)
}
assertEquals("DatabaseException", error.errorObject.objClass.className)
} finally {
missingFile.deleteIfExists()
missingDir.deleteIfExists()
}
}
@Test
fun testGenericOpenDatabaseReadOnlyOptionMatchesTypedHelper() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val dbModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
val tempFile = Files.createTempFile("lyng-sqlite-generic-", ".db")
try {
val writableDb = dbModule.callFn("openDatabase", ObjString("sqlite:${tempFile}"), sqliteOptions(scope))
writableDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table item(id integer primary key autoincrement, name text not null)")
)
}
)
val readOnlyDb = dbModule.callFn(
"openDatabase",
ObjString("sqlite:${tempFile}"),
sqliteOptions(
scope,
"readOnly" to net.sergeych.lyng.obj.ObjTrue,
"createIfMissing" to net.sergeych.lyng.obj.ObjFalse
)
)
val error = assertFailsWith<ExecutionError> {
readOnlyDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into item(name) values(?)"),
ObjString("blocked")
)
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
} finally {
tempFile.deleteIfExists()
}
}
@Test
fun testForeignKeysOptionControlsConstraintEnforcement() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val tempFile = Files.createTempFile("lyng-sqlite-fk-", ".db")
try {
val dbNoFk = sqliteModule.callFn(
"openSqlite",
ObjString(tempFile.toString()),
net.sergeych.lyng.obj.ObjFalse,
net.sergeych.lyng.obj.ObjTrue,
net.sergeych.lyng.obj.ObjFalse
)
dbNoFk.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table parent(id integer primary key)"))
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table child(parent_id integer not null references parent(id))")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into child(parent_id) values(?)"),
ObjInt.One
)
}
)
val dbWithFk = sqliteModule.callFn("openSqlite", ObjString(tempFile.toString()))
val error = assertFailsWith<ExecutionError> {
dbWithFk.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into child(parent_id) values(?)"),
ObjInt.of(2)
)
}
)
}
assertEquals("SqlConstraintException", error.errorObject.objClass.className)
} finally {
tempFile.deleteIfExists()
}
}
@Test
fun testCommitFailureBecomesPrimaryAfterNormalCompletion() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("rollback"))
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
}
@Test
fun testUserExceptionStaysPrimaryWhenRollbackFails() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<IllegalStateException> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("rollback"))
throw IllegalStateException("boom")
}
)
}
assertEquals("boom", error.message)
}
@Test
fun testRollbackFailureBecomesPrimaryAfterRollbackException() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("rollback"))
rollbackException(requireScope(), "rollback requested").raiseAsExecutionError(requireScope())
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
}
private suspend fun ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
private suspend fun sqliteOptions(scope: Scope, vararg entries: Pair<String, Obj>): ObjMap {
val result = ObjMap()
for ((key, value) in entries) {
result.putAt(scope, ObjString(key), value)
}
return result
}
private suspend fun openMemoryDb(scope: Scope): Obj {
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
return sqliteModule.callFn("openSqlite", ObjString(":memory:"))
}
private suspend fun withTempDb(scope: Scope, block: suspend (Obj) -> Unit) {
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val tempFile = Files.createTempFile("lyng-sqlite-", ".db")
try {
val db = sqliteModule.callFn("openSqlite", ObjString(tempFile.toString()))
block(db)
} finally {
tempFile.deleteIfExists()
}
}
private suspend fun field(scope: Scope, obj: Obj, name: String): Obj =
obj.readField(scope, name).value
private suspend fun rowsOf(scope: Scope, resultSet: Obj): List<Obj> {
val rows = resultSet.invokeInstanceMethod(scope, "toList")
val size = (field(scope, rows, "size") as ObjInt).value.toInt()
return (0 until size).map { index -> rows.getAt(scope, ObjInt.of(index.toLong())) }
}
private suspend fun stringValue(scope: Scope, obj: Obj): String =
(obj as? ObjString ?: obj.toString(scope)) .value
private suspend fun enumName(scope: Scope, obj: Obj): String =
stringValue(scope, field(scope, obj, "name"))
private suspend fun decimalOf(scope: Scope, value: String): Obj {
val decimalModule = scope.currentImportProvider.createModuleScope(scope.pos, "lyng.decimal")
val decimalClass = decimalModule.requireClass("Decimal")
return decimalClass.invokeInstanceMethod(scope, "fromString", ObjString(value))
}
private suspend fun rollbackException(scope: Scope, message: String): net.sergeych.lyng.obj.ObjException {
val dbModule = scope.currentImportProvider.createModuleScope(scope.pos, "lyng.io.db")
val rollbackClass = dbModule.requireClass("RollbackException")
return rollbackClass.invoke(scope, ObjNull, ObjString(message)) as net.sergeych.lyng.obj.ObjException
}
private suspend fun dateOf(scope: Scope, value: String): Obj {
val timeModule = scope.currentImportProvider.createModuleScope(scope.pos, "lyng.time")
val dateClass = timeModule.requireClass("Date")
return dateClass.invoke(scope, ObjNull, ObjString(value))
}
private fun emptyMapObj(): Obj = ObjMap()
}
@@ -2,4 +2,5 @@ package net.sergeych.lyngio.ws
import io.ktor.client.engine.curl.Curl
actual fun getSystemWsEngine(): LyngWsEngine = createKtorWsEngine(Curl)
actual fun getSystemWsEngine(): LyngWsEngine =
createSocketWsEngine(secureFallback = createKtorWsEngine(Curl, shareClient = false))
@@ -0,0 +1,807 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import kotlinx.coroutines.test.runTest
import kotlinx.datetime.TimeZone
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.Script
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjBool
import net.sergeych.lyng.obj.ObjBuffer
import net.sergeych.lyng.obj.ObjDateTime
import net.sergeych.lyng.obj.ObjExternCallable
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjInstant
import net.sergeych.lyng.obj.ObjMap
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.raiseAsExecutionError
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.requireScope
import okio.FileSystem
import okio.Path
import okio.Path.Companion.toPath
import kotlin.random.Random
import kotlin.time.Instant
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
import kotlin.test.assertTrue
class LyngSqliteModuleNativeTest {
@Test
fun testTypedOpenSqliteExecutesQueriesAndGeneratedKeys() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val db = sqliteModule.callFn("openSqlite", ObjString(":memory:"))
val insertedId = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table person(id integer primary key autoincrement, name text not null)"))
val result = tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into person(name) values(?)"),
ObjString("John Doe")
)
val generatedKeys = result.invokeInstanceMethod(requireScope(), "getGeneratedKeys")
val rows = generatedKeys.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjInt.Zero)
}
) as ObjInt
assertEquals(1L, insertedId.value)
}
@Test
fun testNestedTransactionRollbackUsesSavepoint() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val count = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table items(id integer primary key autoincrement, name text not null)"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("outer"))
try {
tx.invokeInstanceMethod(
requireScope(),
"transaction",
ObjExternCallable.fromBridge {
val inner = requiredArg<Obj>(0)
inner.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("inner"))
throw IllegalStateException("rollback nested")
}
)
} catch (_: IllegalStateException) {
}
val resultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select count(*) as count from items"))
val rows = resultSet.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjString("count"))
}
) as ObjInt
assertEquals(1L, count.value)
}
@Test
fun testRollbackExceptionRollsBackAndPropagates() = runTest {
val scope = Script.newScope()
withTempDb(scope) { db ->
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table items(id integer primary key autoincrement, name text not null)")
)
}
)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into items(name) values(?)"),
ObjString("rolled-back")
)
rollbackException(requireScope(), "stop here").raiseAsExecutionError(requireScope())
}
)
}
assertEquals("RollbackException", error.errorObject.objClass.className)
val count = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
val resultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select count(*) as count from items"))
rowsOf(requireScope(), resultSet)[0].getAt(requireScope(), ObjString("count"))
}
) as ObjInt
assertEquals(0L, count.value)
}
}
@Test
fun testResultSetFailsAfterTransactionEnds() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
var leakedResultSet: Obj = ObjNull
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table items(id integer primary key autoincrement, name text not null)"))
leakedResultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select 42 as answer"))
ObjNull
}
)
val error = assertFailsWith<ExecutionError> {
leakedResultSet.invokeInstanceMethod(scope, "size")
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("transaction is active"), error.errorMessage)
}
@Test
fun testMaterializedRowSurvivesAfterTransactionEnds() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
var leakedRow: Obj = ObjNull
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
leakedRow = rowsOf(requireScope(), tx.invokeInstanceMethod(requireScope(), "select", ObjString("select 42 as answer")))[0]
ObjNull
}
)
val answer = leakedRow.getAt(scope, ObjString("answer")) as ObjInt
assertEquals(42L, answer.value)
}
@Test
fun testMaterializedRowsListCanBeReturnedFromTransaction() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val rows = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table items(id integer primary key autoincrement, name text not null)")
)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("alpha"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("beta"))
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select name from items order by id"))
.invokeInstanceMethod(requireScope(), "toList")
}
)
assertEquals("alpha", stringValue(scope, rows.getAt(scope, ObjInt.Zero).getAt(scope, ObjString("name"))))
assertEquals("beta", stringValue(scope, rows.getAt(scope, ObjInt.of(1)).getAt(scope, ObjString("name"))))
}
@Test
fun testExecuteRejectsReturningButSelectSupportsIt() = runTest {
val scope = Script.newScope()
withTempDb(scope) { db ->
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table item(id integer primary key autoincrement, name text not null)")
)
}
)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into item(name) values(?) returning id"),
ObjString("bad")
)
}
)
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
val insertedId = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("insert into item(name) values(?) returning id"),
ObjString("good")
)
val row = rowsOf(requireScope(), resultSet)[0]
row.getAt(requireScope(), ObjString("id"))
}
) as ObjInt
assertEquals(1L, insertedId.value)
}
}
@Test
fun testColumnMetadataAndTypedValueConversion() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val summary = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString(
"create table events(" +
"amount NUMERIC not null, " +
"happened TIMESTAMPTZ not null, " +
"scheduled TIMESTAMP not null, " +
"note TEXT not null, " +
"payload BLOB not null)"
)
)
val decimal = decimalOf(requireScope(), "12.50")
val happened = ObjInstant(Instant.parse("2024-05-06T07:08:09Z"))
val scheduled = ObjDateTime(Instant.parse("2024-05-06T10:11:12Z"), TimeZone.UTC)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into events(amount, happened, scheduled, note, payload) values(?, ?, ?, ?, ?)"),
decimal,
happened,
scheduled,
ObjString("hello"),
ObjBuffer(byteArrayOf(1, 2, 3).toUByteArray())
)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("select amount, happened, scheduled, note, payload from events")
)
val columns = field(requireScope(), resultSet, "columns")
val firstColumn = columns.getAt(requireScope(), ObjInt.Zero)
val row = rowsOf(requireScope(), resultSet)[0]
ObjString(
listOf(
stringValue(requireScope(), field(requireScope(), firstColumn, "name")),
enumName(requireScope(), field(requireScope(), firstColumn, "sqlType")),
stringValue(requireScope(), field(requireScope(), firstColumn, "nativeType")),
row.getAt(requireScope(), ObjString("amount")).objClass.className,
row.getAt(requireScope(), ObjString("happened")).objClass.className,
row.getAt(requireScope(), ObjString("scheduled")).objClass.className,
stringValue(requireScope(), row.getAt(requireScope(), ObjString("note"))),
row.getAt(requireScope(), ObjString("payload")).objClass.className,
).joinToString("|")
)
}
) as ObjString
assertEquals(
"amount|Decimal|NUMERIC|Decimal|Instant|DateTime|hello|Buffer",
summary.value
)
}
@Test
fun testDateAndBooleanConversionRules() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val summary = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString(
"create table sample(" +
"flag BOOL not null, " +
"day DATE not null, " +
"clock TIME not null)"
)
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(flag, day, clock) values(?, ?, ?)"),
ObjBool(true),
dateOf(requireScope(), "2026-04-15"),
ObjString("12:34:56")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(flag, day, clock) values(?, ?, ?)"),
ObjString("t"),
ObjString("2026-04-16"),
ObjString("23:59:59")
)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("select flag, day, clock from sample order by day")
)
val rows = rowsOf(requireScope(), resultSet)
ObjString(
listOf(
rows[0].getAt(requireScope(), ObjString("flag")).objClass.className,
rows[0].getAt(requireScope(), ObjString("day")).objClass.className,
stringValue(requireScope(), rows[0].getAt(requireScope(), ObjString("clock"))),
rows[1].getAt(requireScope(), ObjString("flag")).objClass.className,
).joinToString("|")
)
}
) as ObjString
assertEquals("Bool|Date|12:34:56|Bool", summary.value)
}
@Test
fun testUnsupportedParameterTypeFailsWithSqlUsageException() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table sample(value text not null)")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(value) values(?)"),
emptyMapObj()
)
}
)
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("Unsupported SQLite parameter type"), error.errorMessage)
}
@Test
fun testTimestampAndDatetimeRejectTimezoneBearingText() = runTest {
val scope = Script.newScope()
withTempDb(scope) { db ->
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table sample(ts TIMESTAMP not null, dt DATETIME not null)")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(ts, dt) values(?, ?)"),
ObjString("2024-05-06T07:08:09Z"),
ObjString("2024-05-06T10:11:12+03:00")
)
}
)
val timestampError = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select ts from sample"))
}
)
}
assertEquals("SqlExecutionException", timestampError.errorObject.objClass.className)
assertTrue(timestampError.errorMessage.contains("must not contain a timezone offset"), timestampError.errorMessage)
val datetimeError = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select dt from sample"))
}
)
}
assertEquals("SqlExecutionException", datetimeError.errorObject.objClass.className)
assertTrue(datetimeError.errorMessage.contains("must not contain a timezone offset"), datetimeError.errorMessage)
}
}
@Test
fun testReadOnlyOpenPreventsWrites() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
withTempPath { tempPath ->
val writableDb = sqliteModule.callFn("openSqlite", ObjString(tempPath.toString()))
writableDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table item(id integer primary key autoincrement, name text not null)")
)
}
)
val readOnlyDb = sqliteModule.callFn(
"openSqlite",
ObjString(tempPath.toString()),
net.sergeych.lyng.obj.ObjTrue,
net.sergeych.lyng.obj.ObjFalse
)
val error = assertFailsWith<ExecutionError> {
readOnlyDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into item(name) values(?)"),
ObjString("blocked")
)
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
}
}
@Test
fun testMissingFileWithCreateIfMissingFalseFailsWithDatabaseException() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val missingDir = "/tmp/lyng-sqlite-missing-${Random.nextInt(Int.MAX_VALUE)}".toPath()
val missingFile = (missingDir.toString() + "/missing.db").toPath()
try {
FileSystem.SYSTEM.delete(missingFile, mustExist = false)
FileSystem.SYSTEM.delete(missingDir, mustExist = false)
val db = sqliteModule.callFn(
"openSqlite",
ObjString(missingFile.toString()),
net.sergeych.lyng.obj.ObjFalse,
net.sergeych.lyng.obj.ObjFalse
)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge { ObjNull }
)
}
assertEquals("DatabaseException", error.errorObject.objClass.className)
} finally {
FileSystem.SYSTEM.delete(missingFile, mustExist = false)
FileSystem.SYSTEM.delete(missingDir, mustExist = false)
}
}
@Test
fun testGenericOpenDatabaseReadOnlyOptionMatchesTypedHelper() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val dbModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
withTempPath { tempPath ->
val writableDb = dbModule.callFn("openDatabase", ObjString("sqlite:${tempPath}"), sqliteOptions(scope))
writableDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table item(id integer primary key autoincrement, name text not null)")
)
}
)
val readOnlyDb = dbModule.callFn(
"openDatabase",
ObjString("sqlite:${tempPath}"),
sqliteOptions(
scope,
"readOnly" to net.sergeych.lyng.obj.ObjTrue,
"createIfMissing" to net.sergeych.lyng.obj.ObjFalse
)
)
val error = assertFailsWith<ExecutionError> {
readOnlyDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into item(name) values(?)"),
ObjString("blocked")
)
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
}
}
@Test
fun testForeignKeysOptionControlsConstraintEnforcement() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
withTempPath { tempPath ->
val dbNoFk = sqliteModule.callFn(
"openSqlite",
ObjString(tempPath.toString()),
net.sergeych.lyng.obj.ObjFalse,
net.sergeych.lyng.obj.ObjTrue,
net.sergeych.lyng.obj.ObjFalse
)
dbNoFk.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table parent(id integer primary key)"))
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table child(parent_id integer not null references parent(id))")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into child(parent_id) values(?)"),
ObjInt.One
)
}
)
val dbWithFk = sqliteModule.callFn("openSqlite", ObjString(tempPath.toString()))
val error = assertFailsWith<ExecutionError> {
dbWithFk.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into child(parent_id) values(?)"),
ObjInt.of(2)
)
}
)
}
assertEquals("SqlConstraintException", error.errorObject.objClass.className)
}
}
@Test
fun testCommitFailureBecomesPrimaryAfterNormalCompletion() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("rollback"))
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
}
@Test
fun testUserExceptionStaysPrimaryWhenRollbackFails() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<IllegalStateException> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("rollback"))
throw IllegalStateException("boom")
}
)
}
assertEquals("boom", error.message)
}
@Test
fun testRollbackFailureBecomesPrimaryAfterRollbackException() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("rollback"))
rollbackException(requireScope(), "rollback requested").raiseAsExecutionError(requireScope())
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
}
private suspend fun ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
private suspend fun sqliteOptions(scope: Scope, vararg entries: Pair<String, Obj>): ObjMap {
val result = ObjMap()
for ((key, value) in entries) {
result.putAt(scope, ObjString(key), value)
}
return result
}
private suspend fun openMemoryDb(scope: Scope): Obj {
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
return sqliteModule.callFn("openSqlite", ObjString(":memory:"))
}
private suspend fun withTempDb(scope: Scope, block: suspend (Obj) -> Unit) {
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
withTempPath { tempPath ->
val db = sqliteModule.callFn("openSqlite", ObjString(tempPath.toString()))
block(db)
}
}
private suspend fun withTempPath(block: suspend (Path) -> Unit) {
val path = "/tmp/lyng-sqlite-${Random.nextInt(Int.MAX_VALUE)}.db".toPath()
try {
block(path)
} finally {
FileSystem.SYSTEM.delete(path, mustExist = false)
}
}
private suspend fun field(scope: Scope, obj: Obj, name: String): Obj =
obj.readField(scope, name).value
private suspend fun rowsOf(scope: Scope, resultSet: Obj): List<Obj> {
val rows = resultSet.invokeInstanceMethod(scope, "toList")
val size = (field(scope, rows, "size") as ObjInt).value.toInt()
return (0 until size).map { index -> rows.getAt(scope, ObjInt.of(index.toLong())) }
}
private suspend fun stringValue(scope: Scope, obj: Obj): String =
(obj as? ObjString ?: obj.toString(scope)).value
private suspend fun enumName(scope: Scope, obj: Obj): String =
stringValue(scope, field(scope, obj, "name"))
private suspend fun decimalOf(scope: Scope, value: String): Obj {
val decimalModule = scope.currentImportProvider.createModuleScope(scope.pos, "lyng.decimal")
val decimalClass = decimalModule.requireClass("Decimal")
return decimalClass.invokeInstanceMethod(scope, "fromString", ObjString(value))
}
private suspend fun rollbackException(scope: Scope, message: String): net.sergeych.lyng.obj.ObjException {
val dbModule = scope.currentImportProvider.createModuleScope(scope.pos, "lyng.io.db")
val rollbackClass = dbModule.requireClass("RollbackException")
return rollbackClass.invoke(scope, ObjNull, ObjString(message)) as net.sergeych.lyng.obj.ObjException
}
private suspend fun dateOf(scope: Scope, value: String): Obj {
val timeModule = scope.currentImportProvider.createModuleScope(scope.pos, "lyng.time")
val dateClass = timeModule.requireClass("Date")
return dateClass.invoke(scope, ObjNull, ObjString(value))
}
private fun emptyMapObj(): Obj = ObjMap()
}
@@ -0,0 +1,2 @@
headers = sqlite3_lyng.h
package = net.sergeych.lyng.io.db.sqlite.cinterop
@@ -0,0 +1,102 @@
#ifndef LYNG_SQLITE3_LYNG_H
#define LYNG_SQLITE3_LYNG_H
#ifdef __cplusplus
extern "C" {
#endif
typedef struct sqlite3 sqlite3;
typedef struct sqlite3_stmt sqlite3_stmt;
typedef long long sqlite3_int64;
typedef void (*sqlite3_destructor_type)(void*);
#define SQLITE_OK 0
#define SQLITE_ERROR 1
#define SQLITE_INTERNAL 2
#define SQLITE_PERM 3
#define SQLITE_ABORT 4
#define SQLITE_BUSY 5
#define SQLITE_LOCKED 6
#define SQLITE_NOMEM 7
#define SQLITE_READONLY 8
#define SQLITE_INTERRUPT 9
#define SQLITE_IOERR 10
#define SQLITE_CORRUPT 11
#define SQLITE_NOTFOUND 12
#define SQLITE_FULL 13
#define SQLITE_CANTOPEN 14
#define SQLITE_PROTOCOL 15
#define SQLITE_EMPTY 16
#define SQLITE_SCHEMA 17
#define SQLITE_TOOBIG 18
#define SQLITE_CONSTRAINT 19
#define SQLITE_MISMATCH 20
#define SQLITE_MISUSE 21
#define SQLITE_NOLFS 22
#define SQLITE_AUTH 23
#define SQLITE_FORMAT 24
#define SQLITE_RANGE 25
#define SQLITE_NOTADB 26
#define SQLITE_NOTICE 27
#define SQLITE_WARNING 28
#define SQLITE_ROW 100
#define SQLITE_DONE 101
#define SQLITE_INTEGER 1
#define SQLITE_FLOAT 2
#define SQLITE_TEXT 3
#define SQLITE_BLOB 4
#define SQLITE_NULL 5
#define SQLITE_OPEN_READONLY 0x00000001
#define SQLITE_OPEN_READWRITE 0x00000002
#define SQLITE_OPEN_CREATE 0x00000004
#define SQLITE_OPEN_URI 0x00000040
int sqlite3_open_v2(const char* filename, sqlite3** ppDb, int flags, const char* zVfs);
int sqlite3_close_v2(sqlite3*);
int sqlite3_extended_result_codes(sqlite3*, int onoff);
int sqlite3_busy_timeout(sqlite3*, int ms);
int sqlite3_prepare_v2(sqlite3* db, const char* zSql, int nByte, sqlite3_stmt** ppStmt, const char** pzTail);
int sqlite3_finalize(sqlite3_stmt* pStmt);
int sqlite3_step(sqlite3_stmt*);
int sqlite3_reset(sqlite3_stmt* pStmt);
int sqlite3_clear_bindings(sqlite3_stmt*);
int sqlite3_bind_parameter_count(sqlite3_stmt*);
int sqlite3_bind_null(sqlite3_stmt*, int);
int sqlite3_bind_int64(sqlite3_stmt*, int, sqlite3_int64);
int sqlite3_bind_double(sqlite3_stmt*, int, double);
int sqlite3_bind_text(sqlite3_stmt*, int, const char*, int n, sqlite3_destructor_type);
int sqlite3_bind_blob(sqlite3_stmt*, int, const void*, int n, sqlite3_destructor_type);
int sqlite3_column_count(sqlite3_stmt* pStmt);
const char* sqlite3_column_name(sqlite3_stmt*, int N);
const char* sqlite3_column_decltype(sqlite3_stmt*, int);
int sqlite3_column_type(sqlite3_stmt*, int iCol);
sqlite3_int64 sqlite3_column_int64(sqlite3_stmt*, int iCol);
double sqlite3_column_double(sqlite3_stmt*, int iCol);
const unsigned char* sqlite3_column_text(sqlite3_stmt*, int iCol);
const void* sqlite3_column_blob(sqlite3_stmt*, int iCol);
int sqlite3_column_bytes(sqlite3_stmt*, int iCol);
const char* sqlite3_errmsg(sqlite3*);
int sqlite3_extended_errcode(sqlite3*);
int sqlite3_changes(sqlite3*);
sqlite3_int64 sqlite3_last_insert_rowid(sqlite3*);
int sqlite3_db_readonly(sqlite3*, const char* zDbName);
static inline sqlite3* lyng_sqlite3_open(const char* filename, int flags) {
sqlite3* db = 0;
sqlite3_open_v2(filename, &db, flags, 0);
return db;
}
static inline sqlite3_stmt* lyng_sqlite3_prepare(sqlite3* db, const char* sql) {
sqlite3_stmt* stmt = 0;
sqlite3_prepare_v2(db, sql, -1, &stmt, 0);
return stmt;
}
#ifdef __cplusplus
}
#endif
#endif
@@ -0,0 +1,22 @@
package net.sergeych.lyng.io.db.jdbc
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.requireScope
internal actual suspend fun openJdbcBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: JdbcOpenOptions,
): SqlDatabaseBackend {
scope.raiseError(
ObjException(
core.databaseException,
scope.requireScope(),
ObjString("lyng.io.db.jdbc is available only on the JVM target")
)
)
}
@@ -0,0 +1,630 @@
@file:OptIn(ExperimentalForeignApi::class)
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import cnames.structs.sqlite3
import cnames.structs.sqlite3_stmt
import kotlinx.cinterop.ByteVar
import kotlinx.cinterop.CFunction
import kotlinx.cinterop.CPointer
import kotlinx.cinterop.COpaquePointer
import kotlinx.cinterop.ExperimentalForeignApi
import kotlinx.cinterop.MemScope
import kotlinx.cinterop.allocArray
import kotlinx.cinterop.addressOf
import kotlinx.cinterop.memScoped
import kotlinx.cinterop.readBytes
import kotlinx.cinterop.reinterpret
import kotlinx.cinterop.toCPointer
import kotlinx.cinterop.toKString
import kotlinx.cinterop.usePinned
import kotlinx.datetime.LocalDate
import kotlinx.datetime.LocalDateTime
import kotlinx.datetime.TimeZone
import kotlinx.datetime.toInstant
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.SqlColumnMeta
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.io.db.SqlExecutionResultData
import net.sergeych.lyng.io.db.SqlResultSetData
import net.sergeych.lyng.io.db.SqlTransactionBackend
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_BLOB
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_CONSTRAINT
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_DONE
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_FLOAT
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_INTEGER
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_NULL
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_OK
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_OPEN_CREATE
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_OPEN_READONLY
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_OPEN_READWRITE
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_OPEN_URI
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_ROW
import net.sergeych.lyng.io.db.sqlite.cinterop.SQLITE_TEXT
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_bind_blob
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_bind_double
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_bind_int64
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_bind_null
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_bind_parameter_count
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_bind_text
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_busy_timeout
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_changes
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_close_v2
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_column_blob
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_column_bytes
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_column_count
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_column_decltype
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_column_double
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_column_int64
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_column_name
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_column_text
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_column_type
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_errmsg
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_extended_errcode
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_extended_result_codes
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_finalize
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_last_insert_rowid
import net.sergeych.lyng.io.db.sqlite.cinterop.lyng_sqlite3_open
import net.sergeych.lyng.io.db.sqlite.cinterop.lyng_sqlite3_prepare
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_open_v2
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_reset
import net.sergeych.lyng.io.db.sqlite.cinterop.sqlite3_step
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjBool
import net.sergeych.lyng.obj.ObjBuffer
import net.sergeych.lyng.obj.ObjDate
import net.sergeych.lyng.obj.ObjDateTime
import net.sergeych.lyng.obj.ObjEnumEntry
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjInstant
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjReal
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.requireScope
import platform.posix.memcpy
import kotlin.time.Instant
internal actual suspend fun openSqliteBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: SqliteOpenOptions,
): SqlDatabaseBackend {
if (options.busyTimeoutMillis < 0) {
scope.raiseIllegalArgument("busyTimeoutMillis must be >= 0")
}
return NativeSqlDatabaseBackend(core, options)
}
private class NativeSqlDatabaseBackend(
private val core: SqlCoreModule,
private val options: SqliteOpenOptions,
) : SqlDatabaseBackend {
override suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T {
val handle = openHandle(scope, core, options)
val savepoints = SavepointCounter()
try {
handle.execUnit(scope, core, "begin")
val tx = NativeSqlTransactionBackend(core, handle, savepoints)
val result = try {
block(tx)
} catch (e: Throwable) {
throw finishFailedTransaction(scope, core, e) {
handle.execUnit(scope, core, "rollback")
}
}
handle.execUnit(scope, core, "commit")
return result
} finally {
handle.close()
}
}
}
private class NativeSqlTransactionBackend(
private val core: SqlCoreModule,
private val handle: NativeSqliteHandle,
private val savepoints: SavepointCounter,
) : SqlTransactionBackend {
override suspend fun select(scope: ScopeFacade, clause: String, params: List<Obj>): SqlResultSetData {
return handle.select(scope, core, clause, params)
}
override suspend fun execute(scope: ScopeFacade, clause: String, params: List<Obj>): SqlExecutionResultData {
return handle.execute(scope, core, clause, params)
}
override suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T {
val savepoint = "lyng_sp_${savepoints.next()}"
handle.execUnit(scope, core, "savepoint $savepoint")
val nested = NativeSqlTransactionBackend(core, handle, savepoints)
val result = try {
block(nested)
} catch (e: Throwable) {
throw finishFailedTransaction(scope, core, e) {
handle.execUnit(scope, core, "rollback to savepoint $savepoint")
handle.execUnit(scope, core, "release savepoint $savepoint")
}
}
handle.execUnit(scope, core, "release savepoint $savepoint")
return result
}
}
private class SavepointCounter {
private var nextValue = 0
fun next(): Int {
nextValue += 1
return nextValue
}
}
private class NativeSqliteHandle(
private val db: CPointer<sqlite3>,
) {
suspend fun select(
scope: ScopeFacade,
core: SqlCoreModule,
clause: String,
params: List<Obj>,
): SqlResultSetData = memScoped {
val stmt = prepare(scope, core, clause)
try {
bindParams(scope, core, stmt, params, this)
readResultSet(scope, core, stmt)
} finally {
sqlite3_finalize(stmt)
}
}
suspend fun execute(
scope: ScopeFacade,
core: SqlCoreModule,
clause: String,
params: List<Obj>,
): SqlExecutionResultData = memScoped {
if (containsRowReturningClause(clause)) {
raiseExecuteReturningUsage(scope, core)
}
val stmt = prepare(scope, core, clause)
try {
bindParams(scope, core, stmt, params, this)
when (val rc = sqlite3_step(stmt)) {
SQLITE_DONE -> {
val affectedRows = sqlite3_changes(db)
val generatedKeys = readGeneratedKeys(core, clause, affectedRows)
SqlExecutionResultData(affectedRows, generatedKeys)
}
SQLITE_ROW -> raiseExecuteReturningUsage(scope, core)
else -> throw sqlError(scope, core, rc)
}
} finally {
sqlite3_reset(stmt)
sqlite3_finalize(stmt)
}
}
fun execUnit(scope: ScopeFacade, core: SqlCoreModule, sql: String) {
memScoped {
val stmt = prepare(scope, core, sql)
try {
when (val rc = sqlite3_step(stmt)) {
SQLITE_DONE, SQLITE_ROW -> Unit
else -> throw sqlError(scope, core, rc)
}
} finally {
sqlite3_finalize(stmt)
}
}
}
fun close() {
sqlite3_close_v2(db)
}
private fun MemScope.prepare(scope: ScopeFacade, core: SqlCoreModule, sql: String): CPointer<sqlite3_stmt> {
return lyng_sqlite3_prepare(db, sql) ?: throw sqlError(scope, core, sqlite3_extended_errcode(db))
}
private suspend fun bindParams(
scope: ScopeFacade,
core: SqlCoreModule,
stmt: CPointer<sqlite3_stmt>,
params: List<Obj>,
memScope: MemScope,
) {
val expectedCount = sqlite3_bind_parameter_count(stmt)
if (expectedCount != params.size) {
throw usageError(
scope,
core,
"SQL parameter count mismatch: statement expects $expectedCount value(s), got ${params.size}"
)
}
params.forEachIndexed { index, value ->
val parameterIndex = index + 1
val rc = when (value) {
ObjNull -> sqlite3_bind_null(stmt, parameterIndex)
is ObjBool -> sqlite3_bind_int64(stmt, parameterIndex, if (value.value) 1L else 0L)
is ObjInt -> sqlite3_bind_int64(stmt, parameterIndex, value.value)
is ObjReal -> sqlite3_bind_double(stmt, parameterIndex, value.value)
is ObjString -> bindText(stmt, parameterIndex, value.value, memScope)
is ObjBuffer -> bindBlob(stmt, parameterIndex, value.byteArray.toByteArray(), memScope)
is ObjInstant -> bindText(stmt, parameterIndex, value.instant.toString(), memScope)
is ObjDateTime -> bindText(stmt, parameterIndex, value.localDateTime.toString(), memScope)
else -> when (value.objClass.className) {
"Date", "Decimal" -> bindText(stmt, parameterIndex, scope.toStringOf(value).value, memScope)
else -> throw usageError(
scope,
core,
"Unsupported SQLite parameter type: ${value.objClass.className}"
)
}
}
if (rc != SQLITE_OK) {
throw sqlError(scope, core, rc)
}
}
}
private fun bindText(
stmt: CPointer<sqlite3_stmt>,
parameterIndex: Int,
value: String,
memScope: MemScope,
): Int {
return sqlite3_bind_text(stmt, parameterIndex, value, -1, SQLITE_TRANSIENT)
}
private fun bindBlob(
stmt: CPointer<sqlite3_stmt>,
parameterIndex: Int,
value: ByteArray,
memScope: MemScope,
): Int {
if (value.isEmpty()) {
return sqlite3_bind_blob(stmt, parameterIndex, null, 0, null)
}
val target = memScope.allocArray<ByteVar>(value.size)
value.usePinned { pinned ->
memcpy(target, pinned.addressOf(0), value.size.toULong())
}
return sqlite3_bind_blob(stmt, parameterIndex, target, value.size, SQLITE_TRANSIENT)
}
private suspend fun readResultSet(
scope: ScopeFacade,
core: SqlCoreModule,
stmt: CPointer<sqlite3_stmt>,
): SqlResultSetData {
val columnCount = sqlite3_column_count(stmt)
val columns = (0 until columnCount).map { index ->
val nativeType = sqlite3_column_decltype(stmt, index)?.toKString().orEmpty()
SqlColumnMeta(
name = sqlite3_column_name(stmt, index)?.toKString().orEmpty(),
sqlType = mapSqlType(core, nativeType, SQLITE_NULL),
nullable = true,
nativeType = nativeType,
)
}.toMutableList()
if (columnCount == 0) {
return emptyResultSet()
}
val rows = mutableListOf<List<Obj>>()
while (true) {
when (val rc = sqlite3_step(stmt)) {
SQLITE_ROW -> {
val row = (0 until columnCount).map { index ->
val dynamicType = sqlite3_column_type(stmt, index)
if (columns[index].nativeType.isBlank()) {
columns[index] = columns[index].copy(sqlType = mapSqlType(core, columns[index].nativeType, dynamicType))
}
readColumnValue(scope, core, stmt, index, columns[index].nativeType)
}
rows += row
}
SQLITE_DONE -> return SqlResultSetData(columns, rows)
else -> throw sqlError(scope, core, rc)
}
}
}
private suspend fun readColumnValue(
scope: ScopeFacade,
core: SqlCoreModule,
stmt: CPointer<sqlite3_stmt>,
index: Int,
nativeType: String,
): Obj {
val normalizedNativeType = normalizeDeclaredTypeName(nativeType)
return when (val type = sqlite3_column_type(stmt, index)) {
SQLITE_NULL -> ObjNull
SQLITE_INTEGER -> {
val value = sqlite3_column_int64(stmt, index)
when {
isBooleanNativeType(normalizedNativeType) -> integerToBool(scope, core, value)
isDecimalNativeType(normalizedNativeType) -> decimalFromString(scope, value.toString())
else -> ObjInt.of(value)
}
}
SQLITE_FLOAT -> {
val value = sqlite3_column_double(stmt, index)
if (isDecimalNativeType(normalizedNativeType)) decimalFromString(scope, value.toString()) else ObjReal.of(value)
}
SQLITE_TEXT -> {
val textPtr = sqlite3_column_text(stmt, index)?.reinterpret<ByteVar>()
val value = textPtr?.toKString() ?: ""
convertStringValue(scope, core, normalizedNativeType, value)
}
SQLITE_BLOB -> {
val size = sqlite3_column_bytes(stmt, index)
val blob = sqlite3_column_blob(stmt, index)
val bytes = if (blob == null || size <= 0) byteArrayOf() else blob.reinterpret<ByteVar>().readBytes(size)
ObjBuffer(bytes.toUByteArray())
}
else -> ObjString(columnText(stmt, index))
}
}
private suspend fun convertStringValue(
scope: ScopeFacade,
core: SqlCoreModule,
normalizedNativeType: String,
value: String,
): Obj {
return when {
isBooleanNativeType(normalizedNativeType) -> stringToBool(scope, core, value)
isDecimalNativeType(normalizedNativeType) -> decimalFromString(scope, value.trim())
normalizedNativeType == "DATE" -> ObjDate(LocalDate.parse(value.trim()))
normalizedNativeType == "DATETIME" || normalizedNativeType == "TIMESTAMP" ->
dateTimeFromString(scope, core, value)
normalizedNativeType == "TIMESTAMP WITH TIME ZONE" ||
normalizedNativeType == "TIMESTAMPTZ" ||
normalizedNativeType == "DATETIME WITH TIME ZONE" -> ObjInstant(Instant.parse(value.trim()))
else -> ObjString(value)
}
}
private fun readGeneratedKeys(
core: SqlCoreModule,
clause: String,
affectedRows: Int,
): SqlResultSetData {
if (affectedRows <= 0 || !looksLikeInsert(clause)) {
return emptyResultSet()
}
return SqlResultSetData(
columns = listOf(
SqlColumnMeta(
name = "generated_key",
sqlType = core.sqlTypes.require("Int"),
nullable = false,
nativeType = "INTEGER",
)
),
rows = listOf(listOf(ObjInt.of(sqlite3_last_insert_rowid(db))))
)
}
private fun columnText(stmt: CPointer<sqlite3_stmt>, index: Int): String {
return sqlite3_column_text(stmt, index)?.reinterpret<ByteVar>()?.toKString().orEmpty()
}
private fun sqlError(scope: ScopeFacade, core: SqlCoreModule, rc: Int): ExecutionError {
val code = sqlite3_extended_errcode(db)
val message = sqlite3_errmsg(db)?.toKString() ?: "SQLite error ($rc)"
val exceptionClass = if ((code and 0xff) == SQLITE_CONSTRAINT) core.sqlConstraintException else core.sqlExecutionException
return ExecutionError(
ObjException(exceptionClass, scope.requireScope(), ObjString(message)),
scope.pos,
message,
)
}
}
private fun openHandle(
scope: ScopeFacade,
core: SqlCoreModule,
options: SqliteOpenOptions,
): NativeSqliteHandle = memScoped {
val flags = buildOpenFlags(options)
val db = lyng_sqlite3_open(options.path, flags)
val rc = db?.let { sqlite3_extended_errcode(it) } ?: SQLITE_OK
if (db == null || rc != SQLITE_OK) {
val message = db?.let { sqlite3_errmsg(it)?.toKString() } ?: "SQLite open failed"
if (db != null) {
sqlite3_close_v2(db)
}
throw databaseError(scope, core, message)
}
sqlite3_extended_result_codes(db, 1)
if (sqlite3_busy_timeout(db, options.busyTimeoutMillis) != SQLITE_OK) {
val message = sqlite3_errmsg(db)?.toKString() ?: "Failed to configure SQLite busy timeout"
sqlite3_close_v2(db)
throw databaseError(scope, core, message)
}
val handle = NativeSqliteHandle(db)
try {
handle.execUnit(scope, core, if (options.foreignKeys) "pragma foreign_keys = on" else "pragma foreign_keys = off")
} catch (e: Throwable) {
handle.close()
throw e
}
handle
}
private fun buildOpenFlags(options: SqliteOpenOptions): Int {
var flags = SQLITE_OPEN_URI
if (options.readOnly) {
flags = flags or SQLITE_OPEN_READONLY
} else {
flags = flags or SQLITE_OPEN_READWRITE
if (options.createIfMissing) {
flags = flags or SQLITE_OPEN_CREATE
}
}
return flags
}
private fun containsRowReturningClause(clause: String): Boolean =
Regex("""\breturning\b""", RegexOption.IGNORE_CASE).containsMatchIn(clause)
private fun looksLikeInsert(clause: String): Boolean = clause.trimStart().startsWith("insert", ignoreCase = true)
private val SQLITE_TRANSIENT = (-1L).toCPointer<CFunction<(COpaquePointer?) -> Unit>>()
private fun emptyResultSet(): SqlResultSetData = SqlResultSetData(emptyList(), emptyList())
private fun mapSqlType(core: SqlCoreModule, nativeType: String, sqliteType: Int): ObjEnumEntry = when (val normalized = normalizeDeclaredTypeName(nativeType)) {
"BOOLEAN", "BOOL" -> core.sqlTypes.require("Bool")
"DATE" -> core.sqlTypes.require("Date")
"DATETIME", "TIMESTAMP" -> core.sqlTypes.require("DateTime")
"TIMESTAMP WITH TIME ZONE", "TIMESTAMPTZ", "DATETIME WITH TIME ZONE" -> core.sqlTypes.require("Instant")
"DECIMAL", "NUMERIC" -> core.sqlTypes.require("Decimal")
"TIME", "TIME WITHOUT TIME ZONE", "TIME WITH TIME ZONE" -> core.sqlTypes.require("String")
else -> when {
normalized.contains("BLOB") -> core.sqlTypes.require("Binary")
normalized.contains("INT") -> core.sqlTypes.require("Int")
normalized.contains("CHAR") || normalized.contains("TEXT") || normalized.contains("CLOB") -> core.sqlTypes.require("String")
normalized.contains("REAL") || normalized.contains("FLOA") || normalized.contains("DOUB") -> core.sqlTypes.require("Double")
sqliteType == SQLITE_INTEGER -> core.sqlTypes.require("Int")
sqliteType == SQLITE_FLOAT -> core.sqlTypes.require("Double")
sqliteType == SQLITE_BLOB -> core.sqlTypes.require("Binary")
else -> core.sqlTypes.require("String")
}
}
private fun isDecimalNativeType(normalizedNativeType: String): Boolean =
normalizedNativeType == "DECIMAL" || normalizedNativeType == "NUMERIC"
private fun isBooleanNativeType(normalizedNativeType: String): Boolean =
normalizedNativeType == "BOOLEAN" || normalizedNativeType == "BOOL"
private suspend fun decimalFromString(scope: ScopeFacade, value: String): Obj {
val decimalModule = scope.requireScope().currentImportProvider.createModuleScope(scope.pos, "lyng.decimal")
val decimalClass = decimalModule.requireClass("Decimal")
return decimalClass.invokeInstanceMethod(scope.requireScope(), "fromString", ObjString(value))
}
private fun dateTimeFromString(scope: ScopeFacade, core: SqlCoreModule, value: String): ObjDateTime {
val trimmed = value.trim()
if (hasExplicitTimeZone(trimmed)) {
throw sqlExecutionError(scope, core, "SQLite TIMESTAMP/DATETIME value must not contain a timezone offset: $value")
}
val local = LocalDateTime.parse(trimmed)
return ObjDateTime(local.toInstant(TimeZone.UTC), TimeZone.UTC)
}
private fun hasExplicitTimeZone(value: String): Boolean {
if (value.endsWith("Z", ignoreCase = true)) return true
val tIndex = value.indexOf('T')
if (tIndex < 0) return false
val plus = value.lastIndexOf('+')
val minus = value.lastIndexOf('-')
val offsetStart = maxOf(plus, minus)
return offsetStart > tIndex
}
private fun raiseExecuteReturningUsage(scope: ScopeFacade, core: SqlCoreModule): Nothing {
scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("execute(...) cannot be used with statements that return rows; use select(...)")
)
)
}
private fun usageError(scope: ScopeFacade, core: SqlCoreModule, message: String): ExecutionError {
return ExecutionError(
ObjException(core.sqlUsageException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
)
}
private fun databaseError(scope: ScopeFacade, core: SqlCoreModule, message: String): ExecutionError {
return ExecutionError(
ObjException(core.databaseException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
)
}
private fun integerToBool(scope: ScopeFacade, core: SqlCoreModule, value: Long): Obj =
when (value) {
0L -> ObjBool(false)
1L -> ObjBool(true)
else -> throw sqlExecutionError(scope, core, "Invalid SQLite boolean value: $value")
}
private fun stringToBool(scope: ScopeFacade, core: SqlCoreModule, value: String): Obj =
when (value.trim().lowercase()) {
"true", "t" -> ObjBool(true)
"false", "f" -> ObjBool(false)
else -> throw sqlExecutionError(scope, core, "Invalid SQLite boolean value: $value")
}
private fun normalizeDeclaredTypeName(nativeTypeName: String): String {
val strippedSuffix = nativeTypeName.trim().replace(Regex("""\s*\(.*\)\s*$"""), "")
return strippedSuffix.uppercase().replace(Regex("""\s+"""), " ").trim()
}
private fun sqlExecutionError(scope: ScopeFacade, core: SqlCoreModule, message: String): ExecutionError {
return ExecutionError(
ObjException(core.sqlExecutionException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
)
}
private inline fun finishFailedTransaction(
scope: ScopeFacade,
core: SqlCoreModule,
failure: Throwable,
rollback: () -> Unit,
): Throwable {
return try {
rollback()
failure
} catch (rollbackFailure: Throwable) {
if (isRollbackSignal(failure, core)) {
attachSecondaryFailure(rollbackFailure, failure)
rollbackFailure
} else {
attachSecondaryFailure(failure, rollbackFailure)
failure
}
}
}
private fun isRollbackSignal(failure: Throwable, core: SqlCoreModule): Boolean {
val errorObject = (failure as? ExecutionError)?.errorObject ?: return false
return errorObject.isInstanceOf(core.rollbackException)
}
private fun attachSecondaryFailure(primary: Throwable, secondary: Throwable) {
if (primary === secondary) return
primary.addSuppressed(secondary)
}
+210
View File
@@ -0,0 +1,210 @@
package lyng.io.db
/*
Portable value categories exposed by the Lyng SQL layer and used in
`SqlColumn`.
*/
enum SqlType {
Binary, String, Int, Double, Decimal,
Bool, Instant, Date, DateTime
}
extern class SqlColumn {
val name: String
val sqlType: SqlType
val nullable: Bool
/*
Original database type name as reported by the backend, such as
VARCHAR, TEXT, INT8, TIMESTAMPTZ, or BYTEA.
*/
val nativeType: String
}
extern class SqlRow {
/* Number of columns in the row */
val size: Int
val values: List<Object?>
/*
Return the already converted Lyng value for a column addressed by
index or output column label. SQL NULL is returned as null.
Name lookup uses result-column labels. If several columns share the same
label, name-based access is ambiguous and should fail. Missing column
names and invalid indexes should also fail.
*/
override fun getAt(indexOrName: String | Int): Object?
}
/*
A result set is valid only while its owning transaction is active.
Implementations may stream rows or buffer them internally, but:
- rows must be exposed through normal iteration
- iteration to the end or canceled iteration should close the underlying
resources automatically
- using the result set after its transaction ends is invalid
- rows obtained from the result set stay usable after the owning
transaction ends once they have been materialized
Calling `toList()` while the transaction is active is the normal way to
detach rows for use after the transaction block returns.
*/
extern class ResultSet : Iterable<SqlRow> {
/*
Column metadata for the result rows, in positional order.
*/
val columns: List<SqlColumn>
/*
Number of rows if the implementation can determine it. Implementations
may need to consume or buffer the whole result in order to answer, but
this must not change visible later iteration behavior.
*/
fun size(): Int
/*
Fast emptiness check when the implementation can provide it without
consuming the result. Implementations may still peek or buffer
internally, but this must not change visible later iteration behavior.
*/
override fun isEmpty(): Bool
}
extern class ExecutionResult {
val affectedRowsCount: Int
/*
Return implementation-supported auto-generated values produced by
`execute`. This is intentionally stricter than arbitrary SQL result
sets: statements such as INSERT/UPDATE/DELETE ... RETURNING should be
executed with `select`, not exposed here.
The returned result set has the same transaction-scoped lifetime as any
other result set.
If the statement produced no generated values, the returned result set
is empty. Call `toList()` if generated-key rows must outlive the
transaction.
*/
fun getGeneratedKeys(): ResultSet
}
extern class DatabaseException: Exception
extern class SqlExecutionException: DatabaseException
extern class SqlConstraintException: SqlExecutionException
extern class SqlUsageException: DatabaseException
/*
Special exception to be thrown from `SqlTransaction.transaction` when an
intentional rollback is requested without treating it as a backend failure.
It causes rollback and is propagated to the caller, but should not be
treated as a backend/driver failure.
If rollback itself fails, that rollback failure becomes the primary backend
error instead.
*/
extern class RollbackException: Exception
/*
Transaction represents a database transaction.
Important: a transaction has no explicit commit; instead it commits when
leaving the transaction block normally.
If the transaction block throws any exception not caught inside the calling
code, it will be rolled back.
*/
extern class SqlTransaction {
/*
Execute a SQL statement that returns rows. This includes plain SELECT
queries and database-specific DML statements with row-returning clauses
such as RETURNING or OUTPUT.
Portable SQL uses positional `?` placeholders only.
Portable bindable values are:
- null
- Bool
- Int, Double, Decimal
- String
- Buffer
- Date, DateTime, Instant
Unsupported parameter values should fail with `SqlUsageException`.
*/
fun select(clause: String, params...): ResultSet
/*
Execute a SQL statement for side effects. Use `select` for any statement
whose primary result is a row set.
Parameters follow the same binding rules as `select`.
*/
fun execute(clause: String, params...): ExecutionResult
/*
Create a nested transaction with real nested semantics, typically using
database savepoints.
If the backend cannot provide real nested transaction semantics, this
call should fail with `SqlUsageException` rather than flattening into
the outer transaction.
Failure inside the nested transaction rolls back only the nested scope;
the outer transaction remains active unless the exception is allowed to
propagate further.
*/
fun transaction<T>(block: (SqlTransaction) -> T): T
}
extern class Database {
/*
Open a transaction. Any pooling, physical connection lifecycle, and
implementation-specific configuration are owned by the database
implementation and hidden from the user.
The transaction commits when the block finishes normally,
and rolls back if the block exits with an uncaught exception.
Failure precedence is:
- user exception + successful rollback -> original exception escapes
- user exception + rollback failure -> original exception stays primary
- RollbackException + rollback failure -> rollback failure is primary
- commit failure after normal completion -> commit failure is primary
*/
fun transaction<T>(block: (SqlTransaction) -> T): T
}
/*
Register a database provider for a URL scheme.
Provider modules should call this during module initialization when first
imported. Scheme matching is case-insensitive and normalized to lowercase.
Registering the same scheme more than once should fail.
*/
extern fun registerDatabaseProvider(
scheme: String,
opener: (String, Map<String, Object?>) -> Database
)
/*
The mandatory generic entry point for all providers. It opens a database
handle from a provider-specific connection URL plus extra parameters.
Providers may expose additional typed constructors such as `openSqlite(...)`
or `openPostgres(...)`, but `openDatabase(...)` should remain available for
configuration-driven usage.
It should throw IllegalArgumentException for malformed connection URLs or
invalid extra parameter shapes detected before opening the backend.
Runtime opening failures such as authentication, connectivity, or provider
initialization errors should be reported as DatabaseException.
The matching provider must already be registered, normally because its
module was imported and executed. Unknown schemes or missing providers
should fail with DatabaseException.
*/
extern fun openDatabase(connectionUrl: String, extraParams: Map<String, Object?>): Database
+57
View File
@@ -0,0 +1,57 @@
package lyng.io.db.jdbc
import lyng.io.db
/*
Generic JDBC provider for `lyng.io.db` on the JVM target.
Importing this module registers:
- `jdbc:` for raw JDBC URLs such as `jdbc:h2:mem:test`
- `h2:` as a shorthand alias for `jdbc:h2:...`
- `postgres:` / `postgresql:` as shorthand aliases for `jdbc:postgresql:...`
The JVM artifact bundles and explicitly loads the SQLite, H2, and
PostgreSQL JDBC drivers, so `openJdbc(...)`, `openH2(...)`, and
`openPostgres(...)` work under the standard `jlyng` launcher without extra
classpath setup.
Supported `openDatabase(..., extraParams)` keys for JDBC:
- `driverClass: String`
- `user: String`
- `password: String`
- `properties: Map<String, Object?>`
The optional `properties` map is converted to JDBC connection properties by
stringifying each non-null value.
*/
extern fun openJdbc(
connectionUrl: String,
user: String? = null,
password: String? = null,
driverClass: String? = null,
properties: Map<String, Object?>? = null
): Database
/*
Open an H2 database. `connectionUrl` may be a full JDBC URL such as
`jdbc:h2:mem:test`, or the shorter H2-specific tail such as
`mem:test;DB_CLOSE_DELAY=-1`.
*/
extern fun openH2(
connectionUrl: String,
user: String? = null,
password: String? = null,
properties: Map<String, Object?>? = null
): Database
/*
Open a PostgreSQL database. `connectionUrl` may be a full JDBC URL such as
`jdbc:postgresql://localhost/app`, or the shorter provider tail such as
`//localhost/app`.
*/
extern fun openPostgres(
connectionUrl: String,
user: String? = null,
password: String? = null,
properties: Map<String, Object?>? = null
): Database
+40
View File
@@ -0,0 +1,40 @@
package lyng.io.db.sqlite
import lyng.io.db
/*
SQLite provider for `lyng.io.db`.
Importing this module registers the `sqlite:` URL scheme for
`openDatabase(...)`.
Accepted generic URL forms:
- `sqlite::memory:`
- `sqlite:relative/path.db`
- `sqlite:/absolute/path.db`
Provider-specific `openDatabase(..., extraParams)` options:
- `readOnly: Bool`
- `createIfMissing: Bool`
- `foreignKeys: Bool`
- `busyTimeoutMillis: Int`
Open-time URL/config validation failures should surface as
`IllegalArgumentException`.
Runtime SQLite open failures such as opening a missing file with
`createIfMissing = false` should surface as `DatabaseException`.
SQLite provider defaults:
- `Bool` is written as `0` / `1`
- `Decimal` is written as canonical text
- `Date` is written as `YYYY-MM-DD`
- `DateTime` is written as an ISO local timestamp without timezone
- `Instant` is written as an ISO UTC timestamp with explicit timezone marker
*/
extern fun openSqlite(
path: String,
readOnly: Bool = false,
createIfMissing: Bool = true,
foreignKeys: Bool = true,
busyTimeoutMillis: Int = 5000
): Database
+1 -4
View File
@@ -21,7 +21,7 @@ import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl
import org.jetbrains.kotlin.gradle.dsl.JvmTarget
group = "net.sergeych"
version = "1.5.5-SNAPSHOT"
version = "1.5.5"
// Removed legacy buildscript classpath declarations; plugins are applied via the plugins DSL below
@@ -302,8 +302,6 @@ android {
}
}
dependencies {
implementation(libs.firebase.crashlytics.buildtools)
implementation(libs.compiler)
}
publishing {
@@ -345,7 +343,6 @@ tasks.withType<org.gradle.api.tasks.testing.Test> {
// Make the flag visible inside tests if they want to branch on it
systemProperty("LYNG_BENCHMARKS", benchmarksEnabled.toString())
if (!benchmarksEnabled) {
// Exclude all JVM tests whose class name ends with or contains BenchmarkTest
// This keeps CI fast and avoids noisy timing logs by default.
@@ -434,6 +434,57 @@ data class ArgsDeclaration(val params: List<Item>, val endTokenType: Token.Type)
}
}
fun supportsFastFrameBinding(arguments: Arguments): Boolean {
if (arguments.named.isNotEmpty() || arguments.tailBlockMode) return false
return params.none { it.isEllipsis || it.defaultValue != null }
}
fun assignToFrameFast(
scope: Scope,
arguments: Arguments = scope.args,
paramSlotPlan: Map<String, Int>,
frame: FrameAccess,
slotOffset: Int = 0
) {
if (!supportsFastFrameBinding(arguments)) {
scope.raiseIllegalState("fast frame binding is not supported for this call shape")
}
fun slotFor(name: String): Int {
val full = paramSlotPlan[name] ?: scope.raiseIllegalState("parameter slot for '$name' is missing")
val slot = full - slotOffset
if (slot < 0) scope.raiseIllegalState("parameter slot for '$name' is out of range")
return slot
}
fun assign(slot: Int, value: Obj) {
when (value) {
is net.sergeych.lyng.obj.ObjInt -> frame.setInt(slot, value.value)
is net.sergeych.lyng.obj.ObjReal -> frame.setReal(slot, value.value)
is net.sergeych.lyng.obj.ObjBool -> frame.setBool(slot, value.value)
else -> frame.setObj(slot, value)
}
}
if (arguments.list.size > params.size) {
scope.raiseIllegalArgument("expected ${params.size} arguments, got ${arguments.list.size}")
}
if (arguments.list.size < params.size) {
for (i in arguments.list.size until params.size) {
val a = params[i]
if (!a.type.isNullable) {
scope.raiseIllegalArgument("expected ${params.size} arguments, got ${arguments.list.size}")
}
}
}
for (i in params.indices) {
val slot = slotFor(params[i].name)
val value = if (i < arguments.list.size) arguments.list[i] else ObjNull
assign(slot, value.byValueCopy())
}
}
/**
* Single argument declaration descriptor.
*
@@ -16,4 +16,12 @@
package net.sergeych.lyng
interface BytecodeCallable
import net.sergeych.lyng.obj.Obj
interface BytecodeCallable {
fun callOnFast(scope: Scope): Obj? = null
}
interface BytecodeArgCallable {
fun callWithArgsFast(scope: Scope, args: Arguments): Obj? = null
}
@@ -28,6 +28,7 @@ internal suspend fun executeBytecodeWithSeed(scope: Scope, stmt: Statement, labe
else -> null
} ?: scope.raiseIllegalState("$label requires bytecode statement")
scope.pos = bytecode.pos
bytecode.callOnFast(scope)?.let { return it }
return CmdVm().execute(bytecode.bytecodeFunction(), scope, scope.args) { frame, _ ->
seedFrameLocalsFromScope(frame, scope)
}
@@ -20,5 +20,30 @@ package net.sergeych.lyng
* Compile-time call metadata for known functions. Used to select lambda receiver semantics.
*/
data class CallSignature(
val tailBlockReceiverType: String? = null
)
val tailBlockReceiverType: String? = null,
val inlineHigherOrder: HigherOrderInline? = null
) {
data class HigherOrderInline(
val kind: Kind,
val result: ResultMode,
val argCount: Int = 1,
val lambdaArgIndex: Int = 0
)
enum class Kind {
UNARY_ARGUMENT,
RECEIVER,
ITERABLE,
MAP_GET_OR_PUT
}
enum class ResultMode {
BLOCK_RESULT,
RETURN_RECEIVER,
FOR_EACH,
MAP,
FILTER,
MAP_NOT_NULL,
ASSOCIATE_BY
}
}
@@ -85,6 +85,15 @@ internal suspend fun executeClassDecl(
}
if (spec.isExtern) {
val parentClasses = spec.baseSpecs.mapNotNull { baseSpec ->
val rec = scope[baseSpec.name]
val cls = rec?.value as? ObjClass
when {
cls != null -> cls
baseSpec.name == "Exception" -> ObjException.Root
else -> null
}
}
val rec = scope[spec.className]
val existing = rec?.value as? ObjClass
val resolved = if (existing != null) {
@@ -94,8 +103,42 @@ internal suspend fun executeClassDecl(
} else {
null
}
val stub = resolved ?: ObjInstanceClass(spec.className).apply { this.isAbstract = true }
spec.declaredName?.let { scope.addItem(it, false, stub) }
val stub = resolved ?: ObjInstanceClass(spec.className, *parentClasses.toTypedArray()).apply {
this.isAbstract = true
constructorMeta = spec.constructorArgs
spec.constructorArgs?.params?.forEach { p ->
if (p.accessType != null) {
createField(
p.name,
ObjNull,
isMutable = p.accessType == AccessType.Var,
visibility = p.visibility ?: Visibility.Public,
declaringClass = this,
pos = Pos.builtIn,
isTransient = p.isTransient,
type = ObjRecord.Type.ConstructorField,
fieldId = spec.constructorFieldIds?.get(p.name)
)
}
}
}
spec.declaredName?.let { name ->
if (scope.getLocalRecordDirect(name)?.value !== stub) {
scope.addItem(name, false, stub)
}
}
if (resolved == null && (spec.bodyInit != null || spec.initScope.isNotEmpty())) {
val classScope = stub.classScope ?: scope.createChildScope(newThisObj = stub).also {
it.currentClassCtx = stub
stub.classScope = it
}
spec.bodyInit?.let { executeBytecodeWithSeed(classScope, it, "extern class body init") }
if (spec.initScope.isNotEmpty()) {
for (s in spec.initScope) {
executeBytecodeWithSeed(classScope, s, "extern class init")
}
}
}
return stub
}
@@ -1,5 +1,5 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -24,7 +24,10 @@ sealed class CodeContext {
val implicitThisMembers: Boolean = false,
val implicitThisTypeName: String? = null,
val typeParams: Set<String> = emptySet(),
val typeParamDecls: List<TypeDecl.TypeParam> = emptyList()
val typeParamDecls: List<TypeDecl.TypeParam> = emptyList(),
/** True for static methods and top-level functions: they have no implicit `this`,
* so class-body field initializers inside them should not inherit the class name. */
val noImplicitThis: Boolean = false
): CodeContext()
class ClassBody(val name: String, val isExtern: Boolean = false): CodeContext() {
var typeParams: Set<String> = emptySet()
@@ -187,10 +187,13 @@ class Compiler(
private val callableReturnTypeByScopeId: MutableMap<Int, MutableMap<Int, ObjClass>> = mutableMapOf()
private val callableReturnTypeByName: MutableMap<String, ObjClass> = mutableMapOf()
private val callableReturnTypeDeclByName: MutableMap<String, TypeDecl> = mutableMapOf()
private val callSignatureByName: MutableMap<String, CallSignature> = mutableMapOf()
private val lambdaReturnTypeByRef: MutableMap<ObjRef, ObjClass> = mutableMapOf()
private val exactLambdaRefByScopeId: MutableMap<Int, MutableMap<Int, LambdaFnRef>> = mutableMapOf()
private val lambdaCaptureEntriesByRef: MutableMap<ValueFnRef, List<net.sergeych.lyng.bytecode.LambdaCaptureEntry>> =
mutableMapOf()
private val classFieldTypesByName: MutableMap<String, MutableMap<String, ObjClass>> = mutableMapOf()
private val classMemberTypeDeclByName: MutableMap<String, MutableMap<String, TypeDecl>> = mutableMapOf()
private val classMethodReturnTypeByName: MutableMap<String, MutableMap<String, ObjClass>> = mutableMapOf()
private val classMethodReturnTypeDeclByName: MutableMap<String, MutableMap<String, TypeDecl>> = mutableMapOf()
private val classScopeMembersByClassName: MutableMap<String, MutableSet<String>> = mutableMapOf()
@@ -263,6 +266,11 @@ class Compiler(
if (plan.slots.containsKey(name)) continue
declareSlotNameIn(plan, name, record.isMutable, record.type == ObjRecord.Type.Delegated)
scopeSeedNames.add(name)
record.callSignature?.let { signature ->
if (!callSignatureByName.containsKey(name)) {
callSignatureByName[name] = signature
}
}
if (record.typeDecl != null && nameTypeDecl[name] == null) {
nameTypeDecl[name] = record.typeDecl
if (nameObjClass[name] == null) {
@@ -289,6 +297,11 @@ class Compiler(
)
scopeSeedNames.add(getterName)
}
record.callSignature?.let { signature ->
if (!callSignatureByName.containsKey(getterName)) {
callSignatureByName[getterName] = signature
}
}
val prop = record.value as? ObjProperty
if (prop?.setter != null) {
val setterName = extensionPropertySetterName(cls.className, name)
@@ -314,6 +327,11 @@ class Compiler(
)
scopeSeedNames.add(callableName)
}
record.callSignature?.let { signature ->
if (!callSignatureByName.containsKey(callableName)) {
callSignatureByName[callableName] = signature
}
}
}
}
}
@@ -345,6 +363,11 @@ class Compiler(
record.type == ObjRecord.Type.Delegated
)
scopeSeedNames.add(name)
record.callSignature?.let { signature ->
if (!callSignatureByName.containsKey(name)) {
callSignatureByName[name] = signature
}
}
if (record.typeDecl != null && nameTypeDecl[name] == null) {
nameTypeDecl[name] = record.typeDecl
if (nameObjClass[name] == null) {
@@ -652,10 +675,31 @@ class Compiler(
val cls = clsFromScope ?: clsFromImports ?: resolveClassByName(name) ?: return null
val fieldIds = cls.instanceFieldIdMap()
val methodIds = cls.instanceMethodIdMap(includeAbstract = true)
val memberTypeDecls = collectClassMemberTypeDecls(cls)
val baseNames = cls.directParents.map { it.className }
val nextFieldId = (fieldIds.values.maxOrNull() ?: -1) + 1
val nextMethodId = (methodIds.values.maxOrNull() ?: -1) + 1
return CompileClassInfo(name, cls.logicalPackageName, fieldIds, methodIds, nextFieldId, nextMethodId, baseNames)
return CompileClassInfo(
name,
cls.logicalPackageName,
fieldIds,
methodIds,
nextFieldId,
nextMethodId,
baseNames,
memberTypeDecls = memberTypeDecls
)
}
private fun collectClassMemberTypeDecls(cls: ObjClass): Map<String, TypeDecl> {
val result = mutableMapOf<String, TypeDecl>()
for (name in cls.instanceFieldIdMap().keys) {
cls.getInstanceMemberOrNull(name, includeAbstract = true)?.typeDecl?.let { result[name] = it }
}
for (name in cls.instanceMethodIdMap(includeAbstract = true).keys) {
cls.getInstanceMemberOrNull(name, includeAbstract = true)?.typeDecl?.let { result[name] = it }
}
return result
}
private data class BaseMemberIds(
@@ -739,6 +783,29 @@ class Compiler(
?: importManager.rootScope.getLocalRecordDirect(name)?.callSignature
}
private fun declaredCallSignature(name: String, extTypeName: String?, actualExtern: Boolean): CallSignature? {
val imported = if (actualExtern) {
importManager.rootScope.getLocalRecordDirect(name)?.callSignature
} else {
null
}
val inferred = when {
extTypeName == "Iterable" && name == "filter" -> CallSignature(
inlineHigherOrder = CallSignature.HigherOrderInline(
kind = CallSignature.Kind.ITERABLE,
result = CallSignature.ResultMode.FILTER
)
)
else -> null
}
return when {
imported == null -> inferred
inferred == null -> imported
imported.inlineHigherOrder != null -> imported
else -> imported.copy(inlineHigherOrder = inferred.inlineHigherOrder)
}
}
internal data class MemberIds(val fieldId: Int?, val methodId: Int?)
private fun resolveMemberIds(name: String, pos: Pos, qualifier: String? = null): MemberIds {
@@ -790,8 +857,18 @@ class Compiler(
private fun currentImplicitThisTypeName(): String? {
for (ctx in codeContexts.asReversed()) {
val fn = ctx as? CodeContext.Function ?: continue
if (fn.implicitThisTypeName != null) return fn.implicitThisTypeName
when (ctx) {
is CodeContext.Function -> {
if (ctx.implicitThisTypeName != null) return ctx.implicitThisTypeName
// A static method or top-level function explicitly has no implicit `this`.
// Stop here — do not fall through to an enclosing ClassBody.
if (ctx.noImplicitThis) return null
}
// Class field initializers are compiled directly under ClassBody with no wrapping
// Function. Lambdas inside those initializers must still see `this` as the class.
is CodeContext.ClassBody -> return ctx.name
else -> {}
}
}
return null
}
@@ -1396,6 +1473,11 @@ class Compiler(
}
private fun seedImportTypeMetadata(name: String, record: ObjRecord) {
record.callSignature?.let { signature ->
if (!callSignatureByName.containsKey(name)) {
callSignatureByName[name] = signature
}
}
if (record.typeDecl != null && nameTypeDecl[name] == null) {
nameTypeDecl[name] = record.typeDecl
}
@@ -1672,7 +1754,8 @@ class Compiler(
val methodIds: Map<String, Int>,
val nextFieldId: Int,
val nextMethodId: Int,
val baseNames: List<String>
val baseNames: List<String>,
val memberTypeDecls: Map<String, TypeDecl> = emptyMap()
)
private val compileClassInfos = mutableMapOf<String, CompileClassInfo>()
@@ -1861,14 +1944,16 @@ class Compiler(
forcedLocalSlotInfo = forcedLocalInfo,
forcedLocalScopeId = forcedLocalScopeId,
slotTypeByScopeId = slotTypeByScopeId,
exactLambdaRefByScopeId = exactLambdaRefByScopeId,
slotTypeDeclByScopeId = slotTypeDeclByScopeId,
knownNameObjClass = knownClassMapForBytecode(),
knownClassNames = knownClassNamesForBytecode(),
knownObjectNames = objectDeclNames,
knownObjectNames = knownObjectNamesForBytecode(),
classFieldTypesByName = classFieldTypesByName,
enumEntriesByName = enumEntriesByName,
callableReturnTypeByScopeId = callableReturnTypeByScopeId,
callableReturnTypeByName = callableReturnTypeByName,
callSignatureByName = callSignatureByName,
externBindingNames = externBindingNames,
preparedModuleBindingNames = importBindings.keys,
scopeRefPosByName = moduleReferencePosByName,
@@ -2063,12 +2148,19 @@ class Compiler(
plan.captureOwners[name] = slotLoc
plan.captures += capture
if (!plan.slotPlan.slots.containsKey(name)) {
val newSlot = plan.slotPlan.nextIndex
plan.slotPlan.slots[name] = SlotEntry(
plan.slotPlan.nextIndex,
newSlot,
isMutable = slotLoc.isMutable,
isDelegated = slotLoc.isDelegated
)
plan.slotPlan.nextIndex += 1
slotTypeByScopeId[slotLoc.scopeId]?.get(slotLoc.slot)?.let { cls ->
slotTypeByScopeId.getOrPut(plan.slotPlan.id) { mutableMapOf() }[newSlot] = cls
}
slotTypeDeclByScopeId[slotLoc.scopeId]?.get(slotLoc.slot)?.let { decl ->
slotTypeDeclByScopeId.getOrPut(plan.slotPlan.id) { mutableMapOf() }[newSlot] = decl
}
}
}
@@ -2185,6 +2277,23 @@ class Compiler(
return result
}
private fun knownObjectNamesForBytecode(): Set<String> {
val result = LinkedHashSet<String>()
fun addScope(scope: Scope?) {
if (scope == null) return
for ((name, rec) in scope.objects) {
if (rec.value is ObjInstance) result.add(name)
}
}
addScope(seedScope)
addScope(importManager.rootScope)
for (module in importedModules) {
addScope(module.scope)
}
result.addAll(objectDeclNames)
return result
}
private fun wrapBytecode(stmt: Statement): Statement {
if (codeContexts.lastOrNull() is CodeContext.Module) return stmt
if (codeContexts.lastOrNull() is CodeContext.ClassBody) return stmt
@@ -2209,14 +2318,16 @@ class Compiler(
scopeSlotNameSet = scopeSlotNameSet,
moduleScopeId = moduleScopeId,
slotTypeByScopeId = slotTypeByScopeId,
exactLambdaRefByScopeId = exactLambdaRefByScopeId,
slotTypeDeclByScopeId = slotTypeDeclByScopeId,
knownNameObjClass = knownClassMapForBytecode(),
knownClassNames = knownClassNamesForBytecode(),
knownObjectNames = objectDeclNames,
knownObjectNames = knownObjectNamesForBytecode(),
classFieldTypesByName = classFieldTypesByName,
enumEntriesByName = enumEntriesByName,
callableReturnTypeByScopeId = callableReturnTypeByScopeId,
callableReturnTypeByName = callableReturnTypeByName,
callSignatureByName = callSignatureByName,
externCallableNames = externCallableNames,
externBindingNames = externBindingNames,
preparedModuleBindingNames = importBindings.keys,
@@ -2242,14 +2353,16 @@ class Compiler(
scopeSlotNameSet = scopeSlotNameSet,
moduleScopeId = moduleScopeId,
slotTypeByScopeId = slotTypeByScopeId,
exactLambdaRefByScopeId = exactLambdaRefByScopeId,
slotTypeDeclByScopeId = slotTypeDeclByScopeId,
knownNameObjClass = knownClassMapForBytecode(),
knownClassNames = knownClassNamesForBytecode(),
knownObjectNames = objectDeclNames,
knownObjectNames = knownObjectNamesForBytecode(),
classFieldTypesByName = classFieldTypesByName,
enumEntriesByName = enumEntriesByName,
callableReturnTypeByScopeId = callableReturnTypeByScopeId,
callableReturnTypeByName = callableReturnTypeByName,
callSignatureByName = callSignatureByName,
externCallableNames = externCallableNames,
externBindingNames = externBindingNames,
preparedModuleBindingNames = importBindings.keys,
@@ -2300,14 +2413,16 @@ class Compiler(
globalSlotInfo = globalSlotInfo,
globalSlotScopeId = globalSlotScopeId,
slotTypeByScopeId = slotTypeByScopeId,
exactLambdaRefByScopeId = exactLambdaRefByScopeId,
slotTypeDeclByScopeId = slotTypeDeclByScopeId,
knownNameObjClass = knownNames,
knownClassNames = knownClassNamesForBytecode(),
knownObjectNames = objectDeclNames,
knownObjectNames = knownObjectNamesForBytecode(),
classFieldTypesByName = classFieldTypesByName,
enumEntriesByName = enumEntriesByName,
callableReturnTypeByScopeId = callableReturnTypeByScopeId,
callableReturnTypeByName = callableReturnTypeByName,
callSignatureByName = callSignatureByName,
externCallableNames = externCallableNames,
externBindingNames = externBindingNames,
preparedModuleBindingNames = importBindings.keys,
@@ -2934,7 +3049,15 @@ class Compiler(
inferReceiverTypeFromRef(left)
} else null
val itType = implicitItTypeForMemberLambda(left, next.value)
val lambda = parseLambdaExpression(receiverType, implicitItType = itType)
val expectedCallableType = expectedCallableArgumentType(
FieldRef(left, next.value, isOptional),
0
)
val lambda = parseLambdaExpression(
receiverType,
implicitItType = itType,
expectedCallableType = expectedCallableType
)
val argPos = next.pos
val args = listOf(ParsedArgument(ExpressionStatement(lambda, argPos), next.pos))
operand = when (left) {
@@ -3341,7 +3464,8 @@ class Compiler(
private suspend fun parseLambdaExpression(
expectedReceiverType: String? = null,
wrapAsExtensionCallable: Boolean = false,
implicitItType: TypeDecl? = null
implicitItType: TypeDecl? = null,
expectedCallableType: TypeDecl.Function? = null
): ObjRef {
// lambda args are different:
val startPos = cc.currentPos()
@@ -3357,16 +3481,40 @@ class Compiler(
val hasImplicitIt = argsDeclaration == null
val slotParamNames = if (hasImplicitIt) paramNames + "it" else paramNames
val paramSlotPlan = buildParamSlotPlan(slotParamNames)
if (implicitItType != null) {
val cls = resolveTypeDeclObjClass(implicitItType)
val itSlot = paramSlotPlan.slots["it"]?.index
if (itSlot != null) {
if (cls != null) {
val paramTypeMap = slotTypeByScopeId.getOrPut(paramSlotPlan.id) { mutableMapOf() }
paramTypeMap[itSlot] = cls
fun seedLambdaParamType(name: String, typeDecl: TypeDecl?) {
if (typeDecl == null) return
val slot = paramSlotPlan.slots[name]?.index ?: return
resolveTypeDeclObjClass(typeDecl)?.let { cls ->
val paramTypeMap = slotTypeByScopeId.getOrPut(paramSlotPlan.id) { mutableMapOf() }
paramTypeMap[slot] = cls
}
val paramTypeDeclMap = slotTypeDeclByScopeId.getOrPut(paramSlotPlan.id) { mutableMapOf() }
paramTypeDeclMap[slot] = typeDecl
}
if (argsDeclaration != null) {
val expectedParams = expectedCallableType?.params.orEmpty()
argsDeclaration.params.forEachIndexed { index, param ->
val effectiveType = if ((param.type == TypeDecl.TypeAny || param.type == TypeDecl.TypeNullableAny) &&
index < expectedParams.size
) {
expectedParams[index]
} else if (param.type == TypeDecl.TypeAny && index == 0 && argsDeclaration.params.size == 1 &&
implicitItType != null
) {
implicitItType
} else {
param.type
}
val paramTypeDeclMap = slotTypeDeclByScopeId.getOrPut(paramSlotPlan.id) { mutableMapOf() }
paramTypeDeclMap[itSlot] = implicitItType
if (effectiveType != TypeDecl.TypeAny && effectiveType != TypeDecl.TypeNullableAny) {
seedLambdaParamType(param.name, effectiveType)
}
}
} else {
val effectiveImplicitItType = implicitItType
?: expectedCallableType?.params?.singleOrNull()
if (effectiveImplicitItType != null) {
seedLambdaParamType("it", effectiveImplicitItType)
}
}
@@ -3466,14 +3614,91 @@ class Compiler(
body
}
val bytecodeFn = (fnStatements as? BytecodeStatement)?.bytecodeFunction()
val inlineBodyRef = argsDeclaration?.let { null } ?: extractInlineLambdaBodyRef(body)
val supportsDirectInvokeFastPath = bytecodeFn != null &&
bytecodeFn.scopeSlotCount == 0 &&
expectedReceiverType == null &&
!wrapAsExtensionCallable &&
!containsDelegatedRefs(body)
val ref = LambdaFnRef(
valueFn = { closureScope ->
val captureRecords = closureScope.captureRecords
val stmt = object : Statement(), BytecodeBodyProvider {
val stmt = object : Statement(), BytecodeBodyProvider, BytecodeCallable {
override val pos: Pos = fnStatements.pos
override fun bytecodeBody(): BytecodeStatement? = fnStatements as? BytecodeStatement
override fun callOnFast(scope: Scope): Obj? {
val context = scope.applyClosureForBytecode(closureScope, preferredThisType = expectedReceiverType).also {
it.args = scope.args
}
if (captureSlots.isNotEmpty()) {
if (captureRecords != null) {
context.captureRecords = captureRecords
context.captureNames = captureSlots.map { it.name }
} else {
val resolvedRecords = ArrayList<ObjRecord>(captureSlots.size)
val resolvedNames = ArrayList<String>(captureSlots.size)
for (capture in captureSlots) {
val rec = resolveStableCaptureRecord(
closureScope,
capture.name,
context.currentClassCtx
) ?: closureScope.raiseSymbolNotFound("symbol ${capture.name} not found")
resolvedRecords.add(freezeImmutableCaptureRecord(rec))
resolvedNames.add(capture.name)
}
context.captureRecords = resolvedRecords
context.captureNames = resolvedNames
}
}
val bytecodeBody = fnStatements as? BytecodeStatement ?: return null
val bytecodeFn = bytecodeBody.bytecodeFunction()
if (!supportsDirectInvokeFastPath || !bytecodeFn.fastOnly) return null
val fastPreboundNames = if (argsDeclaration == null) {
setOf("it")
} else {
argsDeclaration.params.mapTo(LinkedHashSet()) { it.name }
}
val declaredNames = bytecodeFn.constants
.mapNotNull { it as? BytecodeConst.LocalDecl }
.mapTo(mutableSetOf()) { it.name }
if (!canFastSeedUndeclaredLocals(bytecodeFn, declaredNames, fastPreboundNames)) return null
if (argsDeclaration != null && !argsDeclaration.supportsFastFrameBinding(scope.args)) return null
val slotPlan = bytecodeFn.localSlotPlanByName()
val binder: (net.sergeych.lyng.bytecode.CmdFrame, Arguments) -> Unit = { frame, arguments ->
if (argsDeclaration == null) {
val l = arguments.list
val itValue: Obj = when (l.size) {
0 -> ObjVoid
1 -> l[0]
else -> ObjList(l.toMutableList())
}
val itSlot = slotPlan["it"]
if (itSlot != null) {
when (itValue) {
is ObjInt -> frame.frame.setInt(itSlot, itValue.value)
is ObjReal -> frame.frame.setReal(itSlot, itValue.value)
is ObjBool -> frame.frame.setBool(itSlot, itValue.value)
else -> frame.frame.setObj(itSlot, itValue)
}
}
} else {
argsDeclaration.assignToFrameFast(
context,
arguments,
slotPlan,
frame.frame
)
}
}
return try {
net.sergeych.lyng.bytecode.CmdVm().executeFastOnlyNoSuspend(bytecodeFn, context, scope.args, binder)
} catch (e: ReturnException) {
if (e.label == null || returnLabels.contains(e.label)) e.result else throw e
}
}
override suspend fun execute(scope: Scope): Obj {
val context = scope.applyClosureForBytecode(closureScope, preferredThisType = expectedReceiverType).also {
it.args = scope.args
@@ -3548,6 +3773,8 @@ class Compiler(
argsDeclaration = argsDeclaration,
captureEntries = captureEntries,
inferredReturnClass = returnClass,
inlineBodyRef = inlineBodyRef,
supportsDirectInvokeFastPath = supportsDirectInvokeFastPath,
preferredThisType = expectedReceiverType,
wrapAsExtensionCallable = wrapAsExtensionCallable,
returnLabels = returnLabels,
@@ -3562,6 +3789,18 @@ class Compiler(
return ref
}
private fun extractInlineLambdaBodyRef(statement: Statement): ObjRef? {
val target = if (statement is BytecodeStatement) statement.original else statement
return when (target) {
is ExpressionStatement -> target.ref
is BlockStatement -> {
val statements = target.statements()
if (statements.size == 1) extractInlineLambdaBodyRef(statements[0]) else null
}
else -> null
}
}
private suspend fun parseArrayLiteral(): List<ListEntry> {
// it should be called after Token.Type.LBRACKET is consumed
val entries = mutableListOf<ListEntry>()
@@ -4807,6 +5046,23 @@ class Compiler(
return null
}
private fun classMemberTypeDecl(targetClass: ObjClass?, name: String): TypeDecl? {
if (targetClass == null) return null
if (targetClass == ObjDynamic.type) return TypeDecl.TypeAny
val member = targetClass.getInstanceMemberOrNull(name, includeAbstract = true)
val declaringName = member?.declaringClass?.className
val declaringClass = member?.declaringClass
declaringClass?.classScope?.getLocalRecordDirect(name)?.typeDecl?.let { return it }
targetClass.classScope?.getLocalRecordDirect(name)?.typeDecl?.let { return it }
if (declaringName != null) {
classMemberTypeDeclByName[declaringName]?.get(name)?.let { return it }
resolveCompileClassInfo(declaringName)?.memberTypeDecls?.get(name)?.let { return it }
}
classMemberTypeDeclByName[targetClass.className]?.get(name)?.let { return it }
resolveCompileClassInfo(targetClass.className)?.memberTypeDecls?.get(name)?.let { return it }
return member?.typeDecl
}
private fun classMethodReturnClass(targetClass: ObjClass?, name: String): ObjClass? {
if (targetClass == null) return null
if (targetClass == ObjDynamic.type) return ObjDynamic.type
@@ -4878,6 +5134,26 @@ class Compiler(
?: slotTypeDeclByScopeId[slotLoc.scopeId]?.get(slotLoc.slot)?.let { resolveTypeDeclObjClass(it) }
}
private fun lookupExactLambdaRefByName(name: String): LambdaFnRef? {
val slotLoc = lookupSlotLocation(name, includeModule = true) ?: return null
return exactLambdaRefByScopeId[slotLoc.scopeId]?.get(slotLoc.slot)
}
private fun resolveExactLambdaRef(ref: ObjRef?): LambdaFnRef? {
return when (ref) {
is LambdaFnRef -> ref
is LocalVarRef -> lookupExactLambdaRefByName(ref.name)
is FastLocalVarRef -> lookupExactLambdaRefByName(ref.name)
is LocalSlotRef -> {
val ownerScopeId = ref.captureOwnerScopeId ?: ref.scopeId
val ownerSlot = ref.captureOwnerSlot ?: ref.slot
exactLambdaRefByScopeId[ownerScopeId]?.get(ownerSlot)
?: ref.name.takeIf { it.isNotEmpty() }?.let(::lookupExactLambdaRefByName)
}
else -> null
}
}
private fun resolveReceiverTypeDecl(ref: ObjRef): TypeDecl? {
return when (ref) {
is LocalSlotRef -> {
@@ -4896,7 +5172,7 @@ class Compiler(
is ImplicitThisMemberRef -> {
val typeName = ref.preferredThisTypeName() ?: currentImplicitThisTypeName()
val targetClass = typeName?.let { resolveClassByName(it) } ?: return null
targetClass.getInstanceMemberOrNull(ref.name, includeAbstract = true)?.typeDecl?.let { return it }
classMemberTypeDecl(targetClass, ref.name)?.let { return it }
classFieldTypesByName[targetClass.className]?.get(ref.name)
?.let { return TypeDecl.Simple(it.className, false) }
classMethodReturnTypeDeclByName[targetClass.className]?.get(ref.name)?.let { return it }
@@ -4905,7 +5181,7 @@ class Compiler(
is FieldRef -> {
val targetDecl = resolveReceiverTypeDecl(ref.target) ?: return null
val targetClass = resolveTypeDeclObjClass(targetDecl) ?: resolveReceiverClassForMember(ref.target)
targetClass?.getInstanceMemberOrNull(ref.name, includeAbstract = true)?.typeDecl?.let { return it }
classMemberTypeDecl(targetClass, ref.name)?.let { return it }
classFieldTypesByName[targetClass?.className]?.get(ref.name)
?.let { return TypeDecl.Simple(it.className, false) }
when (targetDecl) {
@@ -5003,9 +5279,16 @@ class Compiler(
is QualifiedThisRef -> resolveClassByName(ref.typeName)
is StatementRef -> (ref.statement as? ExpressionStatement)?.let { resolveReceiverClassForMember(it.ref) }
is MethodCallRef -> inferMethodCallReturnClass(ref)
is ImplicitThisMethodCallRef -> inferMethodCallReturnClass(ref.methodName())
is ThisMethodSlotCallRef -> inferMethodCallReturnClass(ref.methodName())
is QualifiedThisMethodSlotCallRef -> inferMethodCallReturnClass(ref.methodName())
is ImplicitThisMethodCallRef -> {
val typeName = ref.preferredThisTypeName() ?: currentImplicitThisTypeName()
inferMethodCallReturnClass(typeName?.let { resolveClassByName(it) }, ref.methodName())
}
is ThisMethodSlotCallRef -> {
val typeName = currentImplicitThisTypeName()
inferMethodCallReturnClass(typeName?.let { resolveClassByName(it) }, ref.methodName())
}
is QualifiedThisMethodSlotCallRef ->
inferMethodCallReturnClass(resolveClassByName(ref.receiverTypeName()), ref.methodName())
is CallRef -> inferCallReturnTypeDecl(ref)?.let { resolveTypeDeclObjClass(it) } ?: inferCallReturnClass(ref)
is BinaryOpRef -> inferBinaryOpReturnClass(ref)
is FieldRef -> {
@@ -5127,6 +5410,8 @@ class Compiler(
leftClass == ObjInstant.type && rightClass == ObjDuration.type -> ObjInstant.type
leftClass == ObjDuration.type && rightClass == ObjInstant.type && ref.op == BinOp.PLUS -> ObjInstant.type
leftClass == ObjDuration.type && rightClass == ObjDuration.type -> ObjDuration.type
leftClass == ObjDate.type && rightClass == ObjDate.type && ref.op == BinOp.MINUS -> ObjInt.type
leftClass == ObjDate.type && rightClass == ObjDuration.type -> ObjDate.type
(leftClass == ObjBuffer.type || leftClass.allParentsSet.contains(ObjBuffer.type)) &&
(rightClass == ObjBuffer.type || rightClass.allParentsSet.contains(ObjBuffer.type)) &&
ref.op == BinOp.PLUS -> ObjBuffer.type
@@ -5198,7 +5483,37 @@ class Compiler(
if (receiverClass != null && isClassScopeCallableMember(receiverClass.className, ref.name)) {
resolveClassByName("${receiverClass.className}.${ref.name}")?.let { return it }
}
return inferMethodCallReturnClass(ref.name)
return inferMethodCallReturnClass(receiverClass, ref.name)
}
private fun inferMethodCallReturnClass(targetClass: ObjClass?, name: String): ObjClass? {
when (targetClass) {
ObjDate.type -> {
return when (name) {
"toIsoString", "toSortableString", "toString" -> ObjString.type
"toDateTime", "atStartOfDay" -> ObjDateTime.type
"addDays", "addMonths", "addYears", "today", "parseIso" -> ObjDate.type
"daysUntil", "daysSince" -> ObjInt.type
else -> inferMethodCallReturnClass(name)
}
}
ObjInstant.type -> {
return when (name) {
"toDateTime" -> ObjDateTime.type
"toDate" -> ObjDate.type
else -> inferMethodCallReturnClass(name)
}
}
ObjDateTime.type -> {
return when (name) {
"toDate" -> ObjDate.type
"toInstant" -> ObjInstant.type
"toUTC", "toTimeZone", "parseRFC3339", "addYears", "addMonths" -> ObjDateTime.type
else -> inferMethodCallReturnClass(name)
}
}
else -> return inferMethodCallReturnClass(name)
}
}
private fun inferMethodCallReturnTypeDecl(ref: MethodCallRef): TypeDecl? {
@@ -5423,13 +5738,13 @@ class Compiler(
"truncateToSecond",
"truncateToMinute",
"truncateToMillisecond" -> ObjInstant.type
"toDateTime",
"today",
"parseIso" -> ObjDate.type
"daysUntil",
"daysSince" -> ObjInt.type
"toTimeZone",
"toUTC",
"parseRFC3339",
"addYears",
"addMonths",
"addDays",
"addHours",
"addMinutes",
"addSeconds" -> ObjDateTime.type
@@ -5486,6 +5801,20 @@ class Compiler(
if (targetClass == ObjInstant.type && (name == "distantFuture" || name == "distantPast")) {
return ObjInstant.type
}
if (targetClass == ObjDate.type) {
return when (name) {
"year",
"month",
"day",
"dayOfMonth",
"dayOfWeek",
"dayOfYear",
"lengthOfMonth",
"lengthOfYear" -> ObjInt.type
"isLeapYear" -> ObjBool.type
else -> null
}
}
if (targetClass == ObjInstant.type && name in listOf(
"truncateToMinute",
"truncateToSecond",
@@ -5538,6 +5867,7 @@ class Compiler(
}
if (targetClass == ObjDateTime.type) {
return when (name) {
"date" -> ObjDate.type
"year",
"month",
"day",
@@ -5794,7 +6124,7 @@ class Compiler(
return
}
val seededCallable = lookupNamedCallableRecord(target)
if (seededCallable != null && seededCallable.type == ObjRecord.Type.Fun && seededCallable.value !is ObjExternCallable) {
if (seededCallable != null && seededCallable.type == ObjRecord.Type.Fun) {
return
}
val decl = (resolveReceiverTypeDecl(target) as? TypeDecl.Function)
@@ -6372,7 +6702,8 @@ class Compiler(
*/
private suspend fun parseArgs(
expectedTailBlockReceiver: String? = null,
implicitItType: TypeDecl? = null
implicitItType: TypeDecl? = null,
expectedTailBlockTarget: ObjRef? = null
): Pair<List<ParsedArgument>, Boolean> {
val args = mutableListOf<ParsedArgument>()
@@ -6430,7 +6761,11 @@ class Compiler(
var lastBlockArgument = false
if (end.type == Token.Type.LBRACE) {
// last argument - callable
val callableAccessor = parseLambdaExpression(expectedTailBlockReceiver, implicitItType = implicitItType)
val callableAccessor = parseLambdaExpression(
expectedTailBlockReceiver,
implicitItType = implicitItType,
expectedCallableType = expectedTailBlockTarget?.let { expectedCallableArgumentType(it, args.size) }
)
args += ParsedArgument(
ExpressionStatement(callableAccessor, end.pos),
end.pos
@@ -6506,6 +6841,7 @@ class Compiler(
): ObjRef {
var detectedBlockArgument = blockArgument
val expectedReceiver = tailBlockReceiverType(left)
val expectedTailCallable = expectedCallableArgumentType(left, 0)
val withReceiver = when (left) {
is LocalVarRef -> if (left.name == "with") left.name else null
is LocalSlotRef -> if (left.name == "with") left.name else null
@@ -6517,7 +6853,10 @@ class Compiler(
// allow any subsequent selectors (like ".last()") to be absorbed
// into the lambda body. This ensures expected order:
// foo { ... }.bar() == (foo { ... }).bar()
val callableAccessor = parseLambdaExpression(expectedReceiver)
val callableAccessor = parseLambdaExpression(
expectedReceiver,
expectedCallableType = expectedTailCallable
)
listOf(ParsedArgument(ExpressionStatement(callableAccessor, cc.currentPos()), cc.currentPos()))
} else {
if (withReceiver != null) {
@@ -6526,7 +6865,11 @@ class Compiler(
val end = cc.next()
if (end.type == Token.Type.LBRACE) {
val receiverType = inferReceiverTypeFromArgs(parsedArgs)
val callableAccessor = parseLambdaExpression(receiverType, wrapAsExtensionCallable = true)
val callableAccessor = parseLambdaExpression(
receiverType,
wrapAsExtensionCallable = true,
expectedCallableType = expectedCallableArgumentType(left, parsedArgs.size)
)
parsedArgs += ParsedArgument(ExpressionStatement(callableAccessor, end.pos), end.pos)
detectedBlockArgument = true
} else {
@@ -6534,7 +6877,7 @@ class Compiler(
}
parsedArgs
} else {
val r = parseArgs(expectedReceiver)
val r = parseArgs(expectedReceiver, expectedTailBlockTarget = left)
detectedBlockArgument = r.second
r.first
}
@@ -6656,6 +6999,26 @@ class Compiler(
return cls?.className
}
private fun expectedCallableArgumentType(target: ObjRef, argIndex: Int): TypeDecl.Function? {
val decl = (resolveReceiverTypeDecl(target) ?: seedTypeDeclFromRef(target)) as? TypeDecl.Function
?: return null
val params = when (target) {
is FieldRef,
is ImplicitThisMemberRef,
is ThisFieldSlotRef,
is ClassScopeMemberRef -> decl.params
else -> buildList {
decl.receiver?.let { add(it) }
addAll(decl.params)
}
}
if (argIndex < params.size) {
return params[argIndex] as? TypeDecl.Function
}
val ellipsis = params.lastOrNull() as? TypeDecl.Ellipsis ?: return null
return ellipsis.elementType as? TypeDecl.Function
}
private fun inferReceiverTypeFromRef(ref: ObjRef): String? {
return when (ref) {
is LocalSlotRef -> {
@@ -7874,7 +8237,8 @@ class Compiler(
methodIds = ctx.memberMethodIds.toMap(),
nextFieldId = ctx.nextFieldId,
nextMethodId = ctx.nextMethodId,
baseNames = baseSpecs.map { it.name }
baseNames = baseSpecs.map { it.name },
memberTypeDecls = classMemberTypeDeclByName[qualifiedName]?.toMap() ?: emptyMap()
)
}
}
@@ -7890,7 +8254,8 @@ class Compiler(
methodIds = ctx.memberMethodIds.toMap(),
nextFieldId = ctx.nextFieldId,
nextMethodId = ctx.nextMethodId,
baseNames = baseSpecs.map { it.name }
baseNames = baseSpecs.map { it.name },
memberTypeDecls = classMemberTypeDeclByName[qualifiedName]?.toMap() ?: emptyMap()
)
}
}
@@ -7977,10 +8342,11 @@ class Compiler(
methodIds = ctx.memberMethodIds.toMap(),
nextFieldId = ctx.nextFieldId,
nextMethodId = ctx.nextMethodId,
baseNames = baseSpecs.map { it.name }
baseNames = baseSpecs.map { it.name },
memberTypeDecls = classMemberTypeDeclByName[qualifiedName]?.toMap() ?: emptyMap()
)
}
}
}
registerClassScopeFieldType(outerClassName, declaredName, qualifiedName)
// restore if no body starts here
cc.restorePos(saved)
@@ -8912,7 +9278,7 @@ class Compiler(
val extensionWrapperName = extTypeName?.let { extensionCallableName(it, name) }
val classCtx = codeContexts.asReversed().firstOrNull { it is CodeContext.ClassBody } as? CodeContext.ClassBody
var memberMethodId = if (extTypeName == null) classCtx?.memberMethodIds?.get(name) else null
val externCallSignature = if (actualExtern) importManager.rootScope.getLocalRecordDirect(name)?.callSignature else null
val externCallSignature = declaredCallSignature(name, extTypeName, actualExtern)
val declKind = if (parentContext is CodeContext.ClassBody) SymbolKind.MEMBER else SymbolKind.FUNCTION
resolutionSink?.declareSymbol(name, declKind, isMutable = false, pos = nameStartPos, isOverride = isOverride)
@@ -8934,7 +9300,9 @@ class Compiler(
}
if (extensionWrapperName != null) {
declareLocalName(extensionWrapperName, isMutable = false)
externCallSignature?.let { callSignatureByName[extensionWrapperName] = it }
}
externCallSignature?.let { callSignatureByName[name] = it }
if (actualExtern && declKind != SymbolKind.MEMBER) {
externCallableNames.add(name)
}
@@ -8982,6 +9350,15 @@ class Compiler(
pendingTypeParamStack.removeLast()
}
if (parentContext is CodeContext.ClassBody && !isStatic && extTypeName == null) {
classMemberTypeDeclByName.getOrPut(parentContext.name) { mutableMapOf() }[name] = TypeDecl.Function(
receiver = receiverTypeDecl,
params = argsDeclaration.params.map { it.type },
returnType = returnTypeDecl ?: TypeDecl.TypeAny,
nullable = false
)
}
var isDelegated = false
var delegateExpression: Statement? = null
if (cc.peekNextNonWhitespace().type == Token.Type.BY) {
@@ -9046,13 +9423,17 @@ class Compiler(
parentContext is CodeContext.ClassBody && !isStatic -> parentContext.name
else -> null
}
// Static methods and top-level functions have no implicit `this`; mark them so that
// currentImplicitThisTypeName() does not fall through to an enclosing ClassBody.
val noImplicitThis = implicitThisTypeName == null && extTypeName == null
return inCodeContext(
CodeContext.Function(
name,
implicitThisMembers = implicitThisMembers,
implicitThisTypeName = implicitThisTypeName,
typeParams = typeParams,
typeParamDecls = typeParamDecls
typeParamDecls = typeParamDecls,
noImplicitThis = noImplicitThis
)
) {
cc.labels.add(name)
@@ -9141,9 +9522,19 @@ class Compiler(
val rawFnStatements = parsedFnStatements?.let { unwrapBytecodeDeep(it) }
val inferredReturnClass = returnTypeDecl?.let { resolveTypeDeclObjClass(it) }
?: inferReturnClassFromStatement(rawFnStatements)
if (declKind == SymbolKind.MEMBER && extTypeName == null) {
val ownerClassName = (parentContext as? CodeContext.ClassBody)?.name
if (ownerClassName != null) {
if (parentContext is CodeContext.ClassBody && !isStatic && extTypeName == null) {
val ownerClassName = parentContext.name
run {
val memberTypeDecl = TypeDecl.Function(
receiver = receiverTypeDecl,
params = argsDeclaration.params.map { it.type },
returnType = returnTypeDecl
?: inferredReturnClass?.let { TypeDecl.Simple(it.className, false) }
?: TypeDecl.TypeAny,
nullable = false
)
classMemberTypeDeclByName
.getOrPut(ownerClassName) { mutableMapOf() }[name] = memberTypeDecl
val returnDecl = returnTypeDecl
?: inferredReturnClass?.let { TypeDecl.Simple(it.className, false) }
if (returnDecl != null) {
@@ -9194,9 +9585,8 @@ class Compiler(
val fnBody = object : Statement(), BytecodeBodyProvider {
override val pos: Pos = start
override fun bytecodeBody(): BytecodeStatement? = fnStatements as? BytecodeStatement
override suspend fun execute(scope: Scope): Obj {
scope.pos = start
override suspend fun execute(scope: Scope): Obj {
// restore closure where the function was defined, and making a copy of it
// for local space. If there is no closure, we are in, say, class context where
// the closure is in the class initialization and we needn't more:
@@ -9205,6 +9595,7 @@ class Compiler(
it.args = scope.args
}
} ?: scope
context.pos = start
// Capacity hint: parameters + declared locals + small overhead
val capacityHint = paramNames.size + fnLocalDecls + 4
@@ -9677,6 +10068,7 @@ class Compiler(
"ChangeRejectionException" -> ObjChangeRejectionExceptionClass
"Exception" -> ObjException.Root
"Instant" -> ObjInstant.type
"Date" -> ObjDate.type
"DateTime" -> ObjDateTime.type
"Duration" -> ObjDuration.type
"Buffer" -> ObjBuffer.type
@@ -9716,7 +10108,8 @@ class Compiler(
pos = Pos.builtIn,
declaringClass = stub,
type = ObjRecord.Type.Field,
fieldId = fieldId
fieldId = fieldId,
typeDecl = info.memberTypeDecls[fieldName]
)
}
}
@@ -9731,7 +10124,8 @@ class Compiler(
declaringClass = stub,
isAbstract = true,
type = ObjRecord.Type.Fun,
methodId = methodId
methodId = methodId,
typeDecl = info.memberTypeDecls[methodName]
)
}
}
@@ -10151,6 +10545,9 @@ class Compiler(
if (initClass != null) {
classFieldTypesByName.getOrPut(declaringClassNameCaptured) { mutableMapOf() }[name] = initClass
}
if (!isDelegate && varTypeDecl is TypeDecl.Generic) {
classMemberTypeDeclByName.getOrPut(declaringClassNameCaptured) { mutableMapOf() }[name] = varTypeDecl
}
}
// Emit MiniValDecl for this declaration (before execution wiring), attach doc if any
@@ -10206,6 +10603,15 @@ class Compiler(
encodedPayloadTypeByName[name] = payloadClass
}
}
if (slotIndex != null && scopeId != null) {
val exactLambdaRef = if (!isMutable) resolveExactLambdaRef(directRef) else null
val scopeMap = exactLambdaRefByScopeId.getOrPut(scopeId) { mutableMapOf() }
if (exactLambdaRef != null) {
scopeMap[slotIndex] = exactLambdaRef
} else {
scopeMap.remove(slotIndex)
}
}
if (initObjClass != null) {
if (slotIndex != null && scopeId != null) {
slotTypeByScopeId.getOrPut(scopeId) { mutableMapOf() }[slotIndex] = initObjClass
@@ -10498,8 +10904,12 @@ class Compiler(
)
}
if (getter != null || setter != null) {
val prop = ObjProperty(name, getter, setter)
val initStmt = if (!isAbstract) {
val prop = if (actualExtern) {
ObjProperty(name, null, null)
} else {
ObjProperty(name, getter, setter)
}
val initStmt = if (!isAbstract && !actualExtern) {
val initStatement = InstancePropertyInitStatement(
storageName = storageName,
isMutable = isMutable,
@@ -73,6 +73,35 @@ internal suspend fun executeFunctionDecl(
return value
}
}
if (spec.actualExtern && spec.extTypeName == null && spec.parentIsClassBody) {
val cls = scope.thisObj as? ObjClass
if (cls != null) {
val existing = cls.members[spec.name]
if (existing != null) {
cls.members[spec.name] = existing.copy(
typeDecl = existing.typeDecl ?: spec.typeDecl
)
val memberValue = cls.members[spec.name]?.value ?: existing.value
val local = scope.getLocalRecordDirect(spec.name)
if (local != null) {
scope.objects[spec.name] = local.copy(
value = memberValue,
typeDecl = local.typeDecl ?: spec.typeDecl
)
} else {
scope.addItem(
spec.name,
false,
memberValue,
spec.visibility,
callSignature = spec.externCallSignature,
typeDecl = spec.typeDecl
)
}
return memberValue
}
}
}
if (spec.isDelegated) {
val delegateExpr = spec.delegateExpression ?: scope.raiseError("delegated function missing delegate")
@@ -89,7 +118,14 @@ internal suspend fun executeFunctionDecl(
scope.addExtension(
type,
spec.name,
ObjRecord(ObjUnset, isMutable = false, visibility = spec.visibility, declaringClass = null, type = ObjRecord.Type.Delegated).apply {
ObjRecord(
ObjUnset,
isMutable = false,
visibility = spec.visibility,
declaringClass = null,
type = ObjRecord.Type.Delegated,
callSignature = spec.externCallSignature
).apply {
delegate = finalDelegate
}
)
@@ -106,7 +142,8 @@ internal suspend fun executeFunctionDecl(
null,
spec.startPos,
isTransient = spec.isTransient,
type = ObjRecord.Type.Delegated
type = ObjRecord.Type.Delegated,
callSignature = spec.externCallSignature
).apply {
delegate = finalDelegate
}
@@ -175,6 +212,7 @@ internal suspend fun executeFunctionDecl(
visibility = spec.visibility,
pos = spec.startPos,
type = ObjRecord.Type.Fun,
callSignature = spec.externCallSignature,
)
} else {
scope.addExtension(
@@ -186,6 +224,7 @@ internal suspend fun executeFunctionDecl(
visibility = spec.visibility,
declaringClass = null,
type = ObjRecord.Type.Fun,
callSignature = spec.externCallSignature,
typeDecl = spec.typeDecl
)
)
@@ -198,6 +237,7 @@ internal suspend fun executeFunctionDecl(
wrapper,
spec.visibility,
recordType = ObjRecord.Type.Fun,
callSignature = spec.externCallSignature,
typeDecl = spec.typeDecl
)
} ?: run {
@@ -216,7 +256,8 @@ internal suspend fun executeFunctionDecl(
isOverride = spec.isOverride,
type = ObjRecord.Type.Fun,
methodId = spec.memberMethodId,
typeDecl = spec.typeDecl
typeDecl = spec.typeDecl,
callSignature = spec.externCallSignature
)
val memberValue = cls.members[spec.name]?.value ?: compiledFnBody
scope.addItem(
@@ -803,12 +803,19 @@ open class Scope(
fun addFn(vararg names: String, callSignature: CallSignature? = null, fn: suspend ScopeFacade.() -> Obj) {
val newFn = net.sergeych.lyng.obj.ObjExternCallable.fromBridge { fn() }
for (name in names) {
val existing = objects[name]
addItem(
name,
false,
existing?.isMutable ?: false,
newFn,
visibility = existing?.visibility ?: Visibility.Public,
writeVisibility = existing?.writeVisibility,
recordType = ObjRecord.Type.Fun,
callSignature = callSignature
isAbstract = false,
isClosed = existing?.isClosed ?: false,
isOverride = existing?.isOverride ?: false,
callSignature = callSignature ?: existing?.callSignature,
typeDecl = existing?.typeDecl
)
}
}
@@ -66,7 +66,11 @@ internal class ScopeBridge(internal val scope: Scope) : ScopeFacade {
override fun raiseIllegalState(message: String): Nothing = scope.raiseIllegalState(message)
override fun raiseNotImplemented(what: String): Nothing = scope.raiseNotImplemented(what)
override suspend fun call(callee: Obj, args: Arguments, newThisObj: Obj?): Obj {
return callee.callOn(scope.createChildScope(scope.pos, args = args, newThisObj = newThisObj))
if (newThisObj == null) {
(callee as? BytecodeArgCallable)?.callWithArgsFast(scope, args)?.let { return it }
}
val child = scope.createChildScope(scope.pos, args = args, newThisObj = newThisObj)
return (callee as? BytecodeCallable)?.callOnFast(child) ?: callee.callOn(child)
}
override suspend fun toStringOf(obj: Obj, forInspect: Boolean): ObjString = obj.toString(scope, forInspect)
override suspend fun inspect(obj: Obj): String = obj.inspect(scope)
@@ -108,11 +108,7 @@ class Script(
seedImportBindings(scope, seedScope)
}
if (moduleSlotPlan.isNotEmpty()) {
scope.applySlotPlan(moduleSlotPlan)
for (name in moduleSlotPlan.keys) {
val record = scope.objects[name] ?: scope.localBindings[name] ?: continue
scope.updateSlotFor(name, record)
}
installModuleSlotPlan(scope)
}
}
@@ -120,12 +116,26 @@ class Script(
if (importBindings.isEmpty() && importedModules.isEmpty()) return
seedImportBindings(scope, seedScope)
if (moduleSlotPlan.isNotEmpty()) {
scope.applySlotPlan(moduleSlotPlan)
for (name in moduleSlotPlan.keys) {
val record = scope.objects[name] ?: scope.localBindings[name] ?: continue
scope.updateSlotFor(name, record)
installModuleSlotPlan(scope)
}
}
private fun installModuleSlotPlan(scope: Scope) {
for ((name, index) in moduleSlotPlan) {
if (scope.getSlotIndexOf(name) != null) continue
if (scope.hasSlotPlanConflict(mapOf(name to index))) {
val record = scope.objects[name]
?: scope.localBindings[name]
?: ObjRecord(ObjUnset, isMutable = true)
scope.allocateSlotFor(name, record)
} else {
scope.applySlotPlan(mapOf(name to index))
}
}
for (name in moduleSlotPlan.keys) {
val record = scope.objects[name] ?: scope.localBindings[name] ?: continue
scope.updateSlotFor(name, record)
}
}
private suspend fun seedModuleLocals(
@@ -634,6 +644,7 @@ class Script(
addConst("Deferred", ObjDeferred.type)
addConst("CompletableDeferred", ObjCompletableDeferred.type)
addConst("Mutex", ObjMutex.type)
addConst("Channel", ObjChannel.type)
addConst("Flow", ObjFlow.type)
addConst("FlowBuilder", ObjFlowBuilder.type)
addConst("Delegate", ObjDynamic.type)
@@ -952,6 +963,12 @@ class Script(
doc = "Point in time (epoch-based).",
type = type("lyng.Class")
)
it.addConstDoc(
name = "Date",
value = ObjDate.type,
doc = "Calendar date without time-of-day or time zone.",
type = type("lyng.Class")
)
it.addConstDoc(
name = "DateTime",
value = ObjDateTime.type,
File diff suppressed because it is too large. Load diff
@@ -40,6 +40,7 @@ sealed class BytecodeConst {
val captureNames: List<String>,
val paramSlotPlan: Map<String, Int>,
val argsDeclaration: ArgsDeclaration?,
val supportsDirectInvokeFastPath: Boolean,
val preferredThisType: String?,
val wrapAsExtensionCallable: Boolean,
val returnLabels: Set<String>,
@@ -18,27 +18,41 @@
package net.sergeych.lyng.bytecode
import net.sergeych.lyng.*
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjClass
import net.sergeych.lyng.obj.ObjRecord
import net.sergeych.lyng.obj.ValueFnRef
import net.sergeych.lyng.obj.*
class BytecodeStatement private constructor(
val original: Statement,
private val function: CmdFunction,
) : Statement(original.isStaticConst, original.isConst, original.returnType) {
) : Statement(original.isStaticConst, original.isConst, original.returnType), BytecodeCallable {
override val pos: Pos = original.pos
private val declaredLocalNames: Set<String> by lazy(LazyThreadSafetyMode.NONE) {
function.constants
.mapNotNull { it as? BytecodeConst.LocalDecl }
.mapTo(mutableSetOf()) { it.name }
}
override fun callOnFast(scope: Scope): Obj? {
scope.pos = pos
if (!function.fastOnly) return null
if (!canFastSeedUndeclaredLocals(function, declaredLocalNames, emptySet())) return null
val binder: (CmdFrame, Arguments) -> Unit = { frame, _ ->
if (!trySeedFrameLocalsFromScopeFast(frame, scope)) {
scope.raiseIllegalState("fast local seeding is not available")
}
}
return CmdVm().executeFastOnlyNoSuspend(function, scope, scope.args, binder)
}
override suspend fun execute(scope: Scope): Obj {
scope.pos = pos
val declaredNames = function.constants
.mapNotNull { it as? BytecodeConst.LocalDecl }
.mapTo(mutableSetOf()) { it.name }
val fastResult = callOnFast(scope)
if (fastResult != null) return fastResult
val binder: suspend (CmdFrame, Arguments) -> Unit = { frame, _ ->
val localNames = frame.fn.localSlotNames
for (i in localNames.indices) {
val name = localNames[i] ?: continue
if (declaredNames.contains(name)) continue
if (declaredLocalNames.contains(name)) continue
val slotType = frame.getLocalSlotTypeCode(i)
if (slotType != SlotType.UNKNOWN.code && slotType != SlotType.OBJ.code) {
continue
@@ -84,6 +98,7 @@ class BytecodeStatement private constructor(
globalSlotInfo: Map<String, ForcedLocalSlotInfo> = emptyMap(),
globalSlotScopeId: Int? = null,
slotTypeByScopeId: Map<Int, Map<Int, ObjClass>> = emptyMap(),
exactLambdaRefByScopeId: Map<Int, Map<Int, LambdaFnRef>> = emptyMap(),
knownNameObjClass: Map<String, ObjClass> = emptyMap(),
knownClassNames: Set<String> = emptySet(),
knownObjectNames: Set<String> = emptySet(),
@@ -91,6 +106,7 @@ class BytecodeStatement private constructor(
enumEntriesByName: Map<String, List<String>> = emptyMap(),
callableReturnTypeByScopeId: Map<Int, Map<Int, ObjClass>> = emptyMap(),
callableReturnTypeByName: Map<String, ObjClass> = emptyMap(),
callSignatureByName: Map<String, CallSignature> = emptyMap(),
externCallableNames: Set<String> = emptySet(),
externBindingNames: Set<String> = emptySet(),
preparedModuleBindingNames: Set<String> = emptySet(),
@@ -122,6 +138,7 @@ class BytecodeStatement private constructor(
globalSlotInfo = globalSlotInfo,
globalSlotScopeId = globalSlotScopeId,
slotTypeByScopeId = slotTypeByScopeId,
exactLambdaRefByScopeId = exactLambdaRefByScopeId,
slotTypeDeclByScopeId = slotTypeDeclByScopeId,
knownNameObjClass = knownNameObjClass,
knownClassNames = knownClassNames,
@@ -130,6 +147,7 @@ class BytecodeStatement private constructor(
enumEntriesByName = enumEntriesByName,
callableReturnTypeByScopeId = callableReturnTypeByScopeId,
callableReturnTypeByName = callableReturnTypeByName,
callSignatureByName = callSignatureByName,
externCallableNames = externCallableNames,
externBindingNames = externBindingNames,
preparedModuleBindingNames = preparedModuleBindingNames,
@@ -112,6 +112,7 @@ class CmdBuilder {
}
cmds.add(createCmd(ins.op, operands, scopeSlotCount, localSlotCaptures))
}
val cmdArray = cmds.toTypedArray()
return CmdFunction(
name = name,
localCount = localCount,
@@ -128,8 +129,9 @@ class CmdBuilder {
localSlotDelegated = localSlotDelegated,
localSlotCaptures = localSlotCaptures,
constants = constPool.toList(),
cmds = cmds.toTypedArray(),
posByIp = posByInstr.toTypedArray()
cmds = cmdArray,
posByIp = posByInstr.toTypedArray(),
fastOnly = computeFastOnlyBytecode(scopeSlotCount, cmdArray)
)
}
@@ -231,8 +233,16 @@ class CmdBuilder {
listOf(OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT)
Opcode.SET_INDEX_INT ->
listOf(OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT)
Opcode.LIST_NEW_INT ->
listOf(OperandKind.SLOT, OperandKind.SLOT)
Opcode.LIST_FILL_INT ->
listOf(OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT)
Opcode.LIST_IOTA_INT ->
listOf(OperandKind.SLOT, OperandKind.SLOT)
Opcode.LIST_NEW_INT_CAP ->
listOf(OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT)
Opcode.LIST_FILL_INT_CAP ->
listOf(OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT)
Opcode.MAKE_RANGE ->
listOf(OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT)
Opcode.LIST_LITERAL ->
@@ -835,7 +845,11 @@ class CmdBuilder {
Opcode.SET_INDEX -> CmdSetIndex(operands[0], operands[1], operands[2])
Opcode.GET_INDEX_INT -> CmdGetIndexInt(operands[0], operands[1], operands[2])
Opcode.SET_INDEX_INT -> CmdSetIndexInt(operands[0], operands[1], operands[2])
Opcode.LIST_NEW_INT -> CmdListNewInt(operands[0], operands[1])
Opcode.LIST_FILL_INT -> CmdListFillInt(operands[0], operands[1], operands[2])
Opcode.LIST_IOTA_INT -> CmdListIotaInt(operands[0], operands[1])
Opcode.LIST_NEW_INT_CAP -> CmdListNewIntCap(operands[0], operands[1], operands[2])
Opcode.LIST_FILL_INT_CAP -> CmdListFillIntCap(operands[0], operands[1], operands[2], operands[3])
Opcode.LIST_LITERAL -> CmdListLiteral(operands[0], operands[1], operands[2], operands[3])
Opcode.GET_MEMBER_SLOT -> CmdGetMemberSlot(operands[0], operands[1], operands[2], operands[3])
Opcode.SET_MEMBER_SLOT -> CmdSetMemberSlot(operands[0], operands[1], operands[2], operands[3])
@@ -495,7 +495,11 @@ object CmdDisassembler {
is CmdSetIndex -> Opcode.SET_INDEX to intArrayOf(cmd.targetSlot, cmd.indexSlot, cmd.valueSlot)
is CmdGetIndexInt -> Opcode.GET_INDEX_INT to intArrayOf(cmd.targetSlot, cmd.indexSlot, cmd.dst)
is CmdSetIndexInt -> Opcode.SET_INDEX_INT to intArrayOf(cmd.targetSlot, cmd.indexSlot, cmd.valueSlot)
is CmdListNewInt -> Opcode.LIST_NEW_INT to intArrayOf(cmd.sizeSlot, cmd.dst)
is CmdListFillInt -> Opcode.LIST_FILL_INT to intArrayOf(cmd.sizeSlot, cmd.callableSlot, cmd.dst)
is CmdListIotaInt -> Opcode.LIST_IOTA_INT to intArrayOf(cmd.sizeSlot, cmd.dst)
is CmdListNewIntCap -> Opcode.LIST_NEW_INT_CAP to intArrayOf(cmd.sizeSlot, cmd.capacitySlot, cmd.dst)
is CmdListFillIntCap -> Opcode.LIST_FILL_INT_CAP to intArrayOf(cmd.sizeSlot, cmd.capacitySlot, cmd.callableSlot, cmd.dst)
is CmdListLiteral -> Opcode.LIST_LITERAL to intArrayOf(cmd.planId, cmd.baseSlot, cmd.count, cmd.dst)
is CmdGetMemberSlot -> Opcode.GET_MEMBER_SLOT to intArrayOf(cmd.recvSlot, cmd.fieldId, cmd.methodId, cmd.dst)
is CmdSetMemberSlot -> Opcode.SET_MEMBER_SLOT to intArrayOf(cmd.recvSlot, cmd.fieldId, cmd.methodId, cmd.valueSlot)
@@ -619,8 +623,16 @@ object CmdDisassembler {
listOf(OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT)
Opcode.SET_INDEX_INT ->
listOf(OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT)
Opcode.LIST_NEW_INT ->
listOf(OperandKind.SLOT, OperandKind.SLOT)
Opcode.LIST_FILL_INT ->
listOf(OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT)
Opcode.LIST_IOTA_INT ->
listOf(OperandKind.SLOT, OperandKind.SLOT)
Opcode.LIST_NEW_INT_CAP ->
listOf(OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT)
Opcode.LIST_FILL_INT_CAP ->
listOf(OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT, OperandKind.SLOT)
Opcode.LIST_LITERAL ->
listOf(OperandKind.CONST, OperandKind.SLOT, OperandKind.COUNT, OperandKind.SLOT)
Opcode.GET_MEMBER_SLOT ->
@@ -35,6 +35,7 @@ data class CmdFunction(
val constants: List<BytecodeConst>,
val cmds: Array<Cmd>,
val posByIp: Array<net.sergeych.lyng.Pos?>,
val fastOnly: Boolean = false,
) {
init {
require(scopeSlotIndices.size == scopeSlotCount) { "scopeSlotIndices size mismatch" }
@@ -71,3 +72,118 @@ data class CmdFunction(
}
}
internal fun computeFastOnlyBytecode(scopeSlotCount: Int, cmds: Array<Cmd>): Boolean {
if (scopeSlotCount != 0) return false
return cmds.all(::supportsFastOnlyExecution)
}
private fun supportsFastOnlyExecution(cmd: Cmd): Boolean {
return when (cmd) {
is CmdMoveIntLocal,
is CmdMoveRealLocal,
is CmdMoveBoolLocal,
is CmdConstObj,
is CmdConstInt,
is CmdConstIntLocal,
is CmdConstReal,
is CmdConstBool,
is CmdConstNull,
is CmdUnboxIntObjLocal,
is CmdUnboxRealObjLocal,
is CmdIntToRealLocal,
is CmdRealToIntLocal,
is CmdBoolToIntLocal,
is CmdIntToBoolLocal,
is CmdAddIntLocal,
is CmdSubIntLocal,
is CmdMulIntLocal,
is CmdDivIntLocal,
is CmdModIntLocal,
is CmdNegIntLocal,
is CmdIncIntLocal,
is CmdDecIntLocal,
is CmdAddRealLocal,
is CmdSubRealLocal,
is CmdMulRealLocal,
is CmdDivRealLocal,
is CmdNegRealLocal,
is CmdAndIntLocal,
is CmdOrIntLocal,
is CmdXorIntLocal,
is CmdShlIntLocal,
is CmdShrIntLocal,
is CmdUshrIntLocal,
is CmdInvIntLocal,
is CmdCmpEqIntLocal,
is CmdCmpNeqIntLocal,
is CmdCmpLtIntLocal,
is CmdCmpLteIntLocal,
is CmdCmpGtIntLocal,
is CmdCmpGteIntLocal,
is CmdCmpEqRealLocal,
is CmdCmpNeqRealLocal,
is CmdCmpLtRealLocal,
is CmdCmpLteRealLocal,
is CmdCmpGtRealLocal,
is CmdCmpGteRealLocal,
is CmdCmpEqBoolLocal,
is CmdCmpNeqBoolLocal,
is CmdCmpEqIntRealLocal,
is CmdCmpEqRealIntLocal,
is CmdCmpLtIntRealLocal,
is CmdCmpLtRealIntLocal,
is CmdCmpLteIntRealLocal,
is CmdCmpLteRealIntLocal,
is CmdCmpGtIntRealLocal,
is CmdCmpGtRealIntLocal,
is CmdCmpGteIntRealLocal,
is CmdCmpGteRealIntLocal,
is CmdCmpNeqIntRealLocal,
is CmdCmpNeqRealIntLocal,
is CmdCmpEqStrLocal,
is CmdCmpNeqStrLocal,
is CmdCmpLtStrLocal,
is CmdCmpLteStrLocal,
is CmdCmpGtStrLocal,
is CmdCmpGteStrLocal,
is CmdCmpEqIntObjLocal,
is CmdCmpNeqIntObjLocal,
is CmdCmpLtIntObjLocal,
is CmdCmpLteIntObjLocal,
is CmdCmpGtIntObjLocal,
is CmdCmpGteIntObjLocal,
is CmdCmpEqRealObjLocal,
is CmdCmpNeqRealObjLocal,
is CmdCmpLtRealObjLocal,
is CmdCmpLteRealObjLocal,
is CmdCmpGtRealObjLocal,
is CmdCmpGteRealObjLocal,
is CmdAddIntObjLocal,
is CmdSubIntObjLocal,
is CmdMulIntObjLocal,
is CmdDivIntObjLocal,
is CmdModIntObjLocal,
is CmdAddRealObjLocal,
is CmdSubRealObjLocal,
is CmdMulRealObjLocal,
is CmdDivRealObjLocal,
is CmdModRealObjLocal,
is CmdNotBoolLocal,
is CmdAndBoolLocal,
is CmdOrBoolLocal,
is CmdJmp,
is CmdJmpIfTrueLocal,
is CmdJmpIfFalseLocal,
is CmdJmpIfEqIntLocal,
is CmdJmpIfNeqIntLocal,
is CmdJmpIfLtIntLocal,
is CmdJmpIfLteIntLocal,
is CmdJmpIfGtIntLocal,
is CmdJmpIfGteIntLocal,
is CmdRet,
is CmdRetVoid -> true
else -> false
}
}
@@ -57,6 +57,64 @@ class CmdVm {
suspend fun execute(fn: CmdFunction, scope0: Scope, args: List<Obj>): Obj {
return execute(fn, scope0, Arguments.from(args))
}
fun executeFastOnlyNoSuspend(
fn: CmdFunction,
scope0: Scope,
args: Arguments,
binder: ((CmdFrame, Arguments) -> Unit)? = null
): Obj {
require(fn.fastOnly) { "fast-only execution requested for non-fast function ${fn.name}" }
result = null
val frame = CmdFrame(this, fn, scope0, args.list)
frame.applyCaptureRecords()
binder?.invoke(frame, args)
val cmds = fn.cmds
try {
while (result == null) {
val cmd = cmds[frame.ip++]
if (!cmd.performFast(frame)) {
error("fast-only command not supported: ${cmd::class.simpleName}")
}
}
} catch (e: Throwable) {
throw frame.normalizeThrowableFast(e)
}
return result ?: ObjVoid
}
suspend fun executeFastOnly(
fn: CmdFunction,
scope0: Scope,
args: Arguments,
binder: ((CmdFrame, Arguments) -> Unit)? = null
): Obj {
require(fn.fastOnly) { "fast-only execution requested for non-fast function ${fn.name}" }
result = null
val frame = CmdFrame(this, fn, scope0, args.list)
frame.applyCaptureRecords()
binder?.invoke(frame, args)
val cmds = fn.cmds
while (true) {
try {
while (result == null) {
val cmd = cmds[frame.ip++]
if (!cmd.performFast(frame)) {
error("fast-only command not supported: ${cmd::class.simpleName}")
}
}
break
} catch (e: Throwable) {
val throwable = frame.normalizeThrowable(e)
if (!frame.handleException(throwable)) {
frame.cancelIterators()
throw throwable
}
}
}
frame.cancelIterators()
return result ?: ObjVoid
}
}
sealed class Cmd {
@@ -280,6 +338,11 @@ class CmdMakeRange(
}
class CmdConstNull(internal val dst: Int) : Cmd() {
override fun performFast(frame: CmdFrame): Boolean {
frame.setObj(dst, ObjNull)
return true
}
override suspend fun perform(frame: CmdFrame) {
frame.setObj(dst, ObjNull)
return
@@ -2141,8 +2204,8 @@ class CmdAssignOpObj(
}
if (result == null) {
val name = (frame.fn.constants.getOrNull(nameId) as? BytecodeConst.StringVal)?.value
if (name != null) frame.ensureScope().raiseIllegalAssignment("symbol is readonly: $name")
frame.ensureScope().raiseIllegalAssignment("symbol is readonly")
if (name != null) frame.ensureScope().raiseIllegalAssignment("can't reassign val $name")
frame.ensureScope().raiseIllegalAssignment("can't reassign val")
}
frame.storeObjResult(dst, result)
return
@@ -2301,6 +2364,11 @@ class CmdJmpIfGteIntLocal(internal val a: Int, internal val b: Int, internal val
}
class CmdRet(internal val slot: Int) : Cmd() {
override fun performFast(frame: CmdFrame): Boolean {
frame.vm.result = frame.storedSlotObj(slot)
return true
}
override suspend fun perform(frame: CmdFrame) {
frame.vm.result = frame.slotToObj(slot)
return
@@ -2322,6 +2390,11 @@ class CmdRetLabel(internal val labelId: Int, internal val slot: Int) : Cmd() {
}
class CmdRetVoid : Cmd() {
override fun performFast(frame: CmdFrame): Boolean {
frame.vm.result = ObjVoid
return true
}
override suspend fun perform(frame: CmdFrame) {
frame.vm.result = ObjVoid
return
@@ -3217,10 +3290,21 @@ class CmdCallDirect(
frame.ensureScope().raiseIllegalState("bytecode runtime cannot call non-bytecode Statement")
}
}
val result = if (PerfFlags.SCOPE_POOL) {
frame.ensureScope().withChildFrame(args) { child -> callee.callOn(child) }
val directFastResult = (callee as? BytecodeArgCallable)?.callWithArgsFast(frame.ensureScope(), args)
val result = if (directFastResult != null) {
directFastResult
} else if (PerfFlags.SCOPE_POOL) {
frame.ensureScope().withChildFrame(args) { child ->
(callee as? BytecodeCallable)?.callOnFast(child) ?: callee.callOn(child)
}
} else {
callee.callOn(frame.ensureScope().createChildScope(frame.ensureScope().pos, args = args))
val scope = frame.ensureScope()
if (callee is BytecodeLambdaCallable && callee.supportsDirectInvokeFastPath()) {
callee.invokeWithArgsFast(scope, args) ?: callee.invokeWithArgs(scope, args)
} else {
val child = scope.createChildScope(scope.pos, args = args)
(callee as? BytecodeCallable)?.callOnFast(child) ?: callee.callOn(child)
}
}
frame.storeObjResult(dst, result)
return
@@ -3247,18 +3331,28 @@ class CmdCallSlot(
frame.ensureScope().raiseUnset(message)
}
val args = frame.buildArguments(argBase, argCount)
val scope = frame.ensureScope()
val directFastResult = (callee as? BytecodeArgCallable)?.callWithArgsFast(scope, args)
val canPool = PerfFlags.SCOPE_POOL && callee !is Statement
val result = if (canPool) {
frame.ensureScope().withChildFrame(args) { child -> callee.callOn(child) }
val result = if (directFastResult != null) {
directFastResult
} else if (canPool) {
frame.ensureScope().withChildFrame(args) { child ->
(callee as? BytecodeCallable)?.callOnFast(child) ?: callee.callOn(child)
}
} else {
val scope = frame.ensureScope()
if (callee is Statement) {
val bytecodeBody = (callee as? BytecodeBodyProvider)?.bytecodeBody()
if (callee !is BytecodeStatement && callee !is BytecodeCallable && bytecodeBody == null) {
scope.raiseIllegalState("bytecode runtime cannot call non-bytecode Statement")
}
}
callee.callOn(scope.createChildScope(scope.pos, args = args))
if (callee is BytecodeLambdaCallable && callee.supportsDirectInvokeFastPath()) {
callee.invokeWithArgsFast(scope, args) ?: callee.invokeWithArgs(scope, args)
} else {
val child = scope.createChildScope(scope.pos, args = args)
(callee as? BytecodeCallable)?.callOnFast(child) ?: callee.callOn(child)
}
}
frame.storeObjResult(dst, result)
return
@@ -3341,10 +3435,53 @@ class CmdListFillInt(
val scope = frame.ensureScope()
val result = ObjList(LongArray(size))
for (i in 0 until size) {
val args = Arguments(ObjInt.of(i.toLong()))
val value = if (callable is BytecodeLambdaCallable && callable.supportsImplicitIntFillFastPath()) {
callable.invokeImplicitIntArg(scope, i.toLong())
callable.invokeImplicitIntArgFast(scope, i.toLong()) ?: callable.invokeImplicitIntArg(scope, i.toLong())
} else if (callable is BytecodeArgCallable) {
callable.callWithArgsFast(scope, args) ?: run {
val child = scope.createChildScope(scope.pos, args = args)
(callable as? BytecodeCallable)?.callOnFast(child) ?: callable.callOn(child)
}
} else {
callable.callOn(scope.createChildScope(scope.pos, args = Arguments(ObjInt.of(i.toLong()))))
val child = scope.createChildScope(scope.pos, args = args)
(callable as? BytecodeCallable)?.callOnFast(child) ?: callable.callOn(child)
}
val intValue = (value as? ObjInt)?.value ?: scope.raiseClassCastError("expected Int fill result")
result.setIntAtFast(i, intValue)
}
frame.storeObjResult(dst, result)
return
}
}
class CmdListFillIntCap(
internal val sizeSlot: Int,
internal val capacitySlot: Int,
internal val callableSlot: Int,
internal val dst: Int,
) : Cmd() {
override suspend fun perform(frame: CmdFrame) {
val size = frame.getInt(sizeSlot).toInt()
if (size < 0) frame.ensureScope().raiseIllegalArgument("list size must be non-negative")
val capacity = frame.getInt(capacitySlot).toInt()
val actualCapacity = maxOf(size, capacity)
if (actualCapacity < 0) frame.ensureScope().raiseIllegalArgument("list capacity must be non-negative")
val callable = frame.storedSlotObj(callableSlot)
val scope = frame.ensureScope()
val result = ObjList(LongArray(actualCapacity), size)
for (i in 0 until size) {
val args = Arguments(ObjInt.of(i.toLong()))
val value = if (callable is BytecodeLambdaCallable && callable.supportsImplicitIntFillFastPath()) {
callable.invokeImplicitIntArgFast(scope, i.toLong()) ?: callable.invokeImplicitIntArg(scope, i.toLong())
} else if (callable is BytecodeArgCallable) {
callable.callWithArgsFast(scope, args) ?: run {
val child = scope.createChildScope(scope.pos, args = args)
(callable as? BytecodeCallable)?.callOnFast(child) ?: callable.callOn(child)
}
} else {
val child = scope.createChildScope(scope.pos, args = args)
(callable as? BytecodeCallable)?.callOnFast(child) ?: callable.callOn(child)
}
val intValue = (value as? ObjInt)?.value ?: scope.raiseClassCastError("expected Int fill result")
result.setIntAtFast(i, intValue)
@@ -3730,6 +3867,50 @@ class CmdCallMemberSlot(
}
}
class CmdListIotaInt(
internal val sizeSlot: Int,
internal val dst: Int,
) : Cmd() {
override suspend fun perform(frame: CmdFrame) {
val size = frame.getInt(sizeSlot).toInt()
if (size < 0) frame.ensureScope().raiseIllegalArgument("list size must be non-negative")
val values = LongArray(size)
for (i in 0 until size) {
values[i] = i.toLong()
}
frame.storeObjResult(dst, ObjList(values))
return
}
}
class CmdListNewInt(
internal val sizeSlot: Int,
internal val dst: Int,
) : Cmd() {
override suspend fun perform(frame: CmdFrame) {
val size = frame.getInt(sizeSlot).toInt()
if (size < 0) frame.ensureScope().raiseIllegalArgument("list size must be non-negative")
frame.storeObjResult(dst, ObjList(LongArray(size)))
return
}
}
class CmdListNewIntCap(
internal val sizeSlot: Int,
internal val capacitySlot: Int,
internal val dst: Int,
) : Cmd() {
override suspend fun perform(frame: CmdFrame) {
val size = frame.getInt(sizeSlot).toInt()
if (size < 0) frame.ensureScope().raiseIllegalArgument("list size must be non-negative")
val capacity = frame.getInt(capacitySlot).toInt()
val actualCapacity = maxOf(size, capacity)
if (actualCapacity < 0) frame.ensureScope().raiseIllegalArgument("list capacity must be non-negative")
frame.storeObjResult(dst, ObjList(LongArray(actualCapacity), size))
return
}
}
class CmdGetIndex(
internal val targetSlot: Int,
internal val indexSlot: Int,
@@ -3834,6 +4015,7 @@ class CmdMakeLambda(internal val id: Int, internal val dst: Int) : Cmd() {
captureNames = lambdaConst.captureNames,
paramSlotPlan = lambdaConst.paramSlotPlan,
argsDeclaration = lambdaConst.argsDeclaration,
supportsDirectInvokeFastPath = lambdaConst.supportsDirectInvokeFastPath,
preferredThisType = lambdaConst.preferredThisType,
returnLabels = lambdaConst.returnLabels,
pos = lambdaConst.pos
@@ -3855,10 +4037,18 @@ class BytecodeLambdaCallable(
private val captureNames: List<String>,
private val paramSlotPlan: Map<String, Int>,
private val argsDeclaration: ArgsDeclaration?,
private val supportsDirectInvokeFastPath: Boolean,
private val preferredThisType: String?,
private val returnLabels: Set<String>,
override val pos: Pos,
) : Statement(), BytecodeCallable {
) : Statement(), BytecodeCallable, BytecodeArgCallable {
private val slotPlanByName: Map<String, Int> by lazy(LazyThreadSafetyMode.NONE) { fn.localSlotPlanByName() }
private val declaredLocalNames: Set<String> by lazy(LazyThreadSafetyMode.NONE) {
fn.constants
.mapNotNull { it as? BytecodeConst.LocalDecl }
.mapTo(mutableSetOf()) { it.name }
}
private fun freezeRecord(record: ObjRecord): ObjRecord {
if (record.isMutable) return record
val raw = record.value as Obj?
@@ -3890,6 +4080,7 @@ class BytecodeLambdaCallable(
captureNames = captureNames,
paramSlotPlan = paramSlotPlan,
argsDeclaration = argsDeclaration,
supportsDirectInvokeFastPath = supportsDirectInvokeFastPath,
preferredThisType = preferredThisType,
returnLabels = returnLabels,
pos = pos
@@ -3906,6 +4097,7 @@ class BytecodeLambdaCallable(
captureNames = captureNames,
paramSlotPlan = paramSlotPlan,
argsDeclaration = argsDeclaration,
supportsDirectInvokeFastPath = supportsDirectInvokeFastPath,
preferredThisType = preferredThisType,
returnLabels = returnLabels,
pos = pos
@@ -3914,9 +4106,29 @@ class BytecodeLambdaCallable(
fun supportsImplicitIntFillFastPath(): Boolean = argsDeclaration == null
suspend fun invokeImplicitIntArg(scope: Scope, arg: Long): Obj {
val context = scope.applyClosureForBytecode(closureScope, preferredThisType).also {
it.args = Arguments.EMPTY
fun supportsDirectInvokeFastPath(): Boolean = supportsDirectInvokeFastPath
private val fastPreboundLocalNames: Set<String> by lazy(LazyThreadSafetyMode.NONE) {
if (argsDeclaration == null) {
setOf("it")
} else {
argsDeclaration.params.mapTo(LinkedHashSet()) { it.name }
}
}
private val supportsFastUndeclaredLocalInit: Boolean by lazy(LazyThreadSafetyMode.NONE) {
canFastSeedUndeclaredLocals(fn, declaredLocalNames, fastPreboundLocalNames)
}
private fun supportsFastOnlyVm(arguments: Arguments): Boolean {
if (!supportsDirectInvokeFastPath || !fn.fastOnly) return false
if (!supportsFastUndeclaredLocalInit) return false
return argsDeclaration == null || argsDeclaration.supportsFastFrameBinding(arguments)
}
private fun buildContext(callScope: Scope, args: Arguments): Scope {
val context = callScope.applyClosureForBytecode(closureScope, preferredThisType).also {
it.args = args
}
if (captureRecords != null) {
context.captureRecords = captureRecords
@@ -3924,98 +4136,161 @@ class BytecodeLambdaCallable(
} else if (captureNames.isNotEmpty()) {
closureScope.raiseIllegalState("bytecode lambda capture records missing")
}
val binder: suspend (CmdFrame, Arguments) -> Unit = { frame, _ ->
paramSlotPlan["it"]?.let { itSlot ->
return context
}
private fun bindArgumentsFast(frame: CmdFrame, context: Scope, arguments: Arguments) {
if (argsDeclaration == null) {
val l = arguments.list
val itValue: Obj = when (l.size) {
0 -> ObjVoid
1 -> l[0]
else -> ObjList(l.toMutableList())
}
val itSlot = slotPlanByName["it"]
if (itSlot != null) {
when (itValue) {
is ObjInt -> frame.frame.setInt(itSlot, itValue.value)
is ObjReal -> frame.frame.setReal(itSlot, itValue.value)
is ObjBool -> frame.frame.setBool(itSlot, itValue.value)
else -> frame.frame.setObj(itSlot, itValue)
}
}
} else {
argsDeclaration.assignToFrameFast(
context,
arguments,
slotPlanByName,
frame.frame
)
}
}
private suspend fun bindArguments(frame: CmdFrame, context: Scope, arguments: Arguments) {
if (argsDeclaration == null) {
bindArgumentsFast(frame, context, arguments)
} else {
argsDeclaration.assignToFrame(
context,
arguments,
slotPlanByName,
frame.frame
)
}
}
private suspend fun seedUndeclaredLocals(frame: CmdFrame, context: Scope) {
val localNames = frame.fn.localSlotNames
for (i in localNames.indices) {
val name = localNames[i] ?: continue
if (declaredLocalNames.contains(name)) continue
val slotType = frame.getLocalSlotTypeCode(i)
if (slotType != SlotType.UNKNOWN.code && slotType != SlotType.OBJ.code) {
continue
}
if (slotType == SlotType.OBJ.code && frame.frame.getRawObj(i) != null) {
continue
}
val record = context.getLocalRecordDirect(name)
?: context.parent?.get(name)
?: context.get(name)
?: continue
val value =
if (record.type == ObjRecord.Type.Delegated || record.type == ObjRecord.Type.Property || record.value is ObjProperty) {
context.resolve(record, name)
} else {
when (val direct = record.value) {
is FrameSlotRef -> direct.read()
is RecordSlotRef -> direct.read(context, name)
is ScopeSlotRef -> direct.read()
else -> direct
}
}
frame.frame.setObj(i, value)
}
}
suspend fun invokeImplicitIntArg(scope: Scope, arg: Long): Obj {
val context = buildContext(scope, Arguments.EMPTY)
val fastBinder: (CmdFrame, Arguments) -> Unit = { frame, _ ->
slotPlanByName["it"]?.let { itSlot ->
frame.frame.setInt(itSlot, arg)
}
}
return try {
CmdVm().execute(fn, context, Arguments.EMPTY, binder)
val vm = CmdVm()
if (supportsFastOnlyVm(Arguments.EMPTY)) {
vm.executeFastOnly(fn, context, Arguments.EMPTY, fastBinder)
} else {
val binder: suspend (CmdFrame, Arguments) -> Unit = { frame, _ ->
slotPlanByName["it"]?.let { itSlot ->
frame.frame.setInt(itSlot, arg)
}
}
vm.execute(fn, context, Arguments.EMPTY, binder)
}
} catch (e: ReturnException) {
if (e.label == null || returnLabels.contains(e.label)) e.result
else throw e
}
}
override suspend fun execute(scope: Scope): Obj {
val context = scope.applyClosureForBytecode(closureScope, preferredThisType).also {
it.args = scope.args
}
if (captureRecords != null) {
context.captureRecords = captureRecords
context.captureNames = captureNames
} else if (captureNames.isNotEmpty()) {
closureScope.raiseIllegalState("bytecode lambda capture records missing")
}
if (argsDeclaration == null) {
// Bound in the bytecode entry binder.
} else {
// args bound into frame slots in the bytecode entry binder
fun invokeImplicitIntArgFast(scope: Scope, arg: Long): Obj? {
if (!supportsFastOnlyVm(Arguments.EMPTY)) return null
val context = buildContext(scope, Arguments.EMPTY)
val binder: (CmdFrame, Arguments) -> Unit = { frame, _ ->
slotPlanByName["it"]?.let { itSlot ->
frame.frame.setInt(itSlot, arg)
}
}
return try {
val declaredNames = fn.constants
.mapNotNull { it as? BytecodeConst.LocalDecl }
.mapTo(mutableSetOf()) { it.name }
val binder: suspend (CmdFrame, Arguments) -> Unit = { frame, arguments ->
val slotPlan = fn.localSlotPlanByName()
if (argsDeclaration == null) {
val l = arguments.list
val itValue: Obj = when (l.size) {
0 -> ObjVoid
1 -> l[0]
else -> ObjList(l.toMutableList())
}
val itSlot = slotPlan["it"]
if (itSlot != null) {
when (itValue) {
is ObjInt -> frame.frame.setInt(itSlot, itValue.value)
is ObjReal -> frame.frame.setReal(itSlot, itValue.value)
is ObjBool -> frame.frame.setBool(itSlot, itValue.value)
else -> frame.frame.setObj(itSlot, itValue)
}
}
} else {
argsDeclaration.assignToFrame(
context,
arguments,
slotPlan,
frame.frame
)
}
val localNames = frame.fn.localSlotNames
for (i in localNames.indices) {
val name = localNames[i] ?: continue
if (declaredNames.contains(name)) continue
val slotType = frame.getLocalSlotTypeCode(i)
if (slotType != SlotType.UNKNOWN.code && slotType != SlotType.OBJ.code) {
continue
}
if (slotType == SlotType.OBJ.code && frame.frame.getRawObj(i) != null) {
continue
}
val record = context.getLocalRecordDirect(name)
?: context.parent?.get(name)
?: context.get(name)
?: continue
val value =
if (record.type == ObjRecord.Type.Delegated || record.type == ObjRecord.Type.Property || record.value is ObjProperty) {
context.resolve(record, name)
} else {
when (val direct = record.value) {
is FrameSlotRef -> direct.read()
is RecordSlotRef -> direct.read(context, name)
is ScopeSlotRef -> direct.read()
else -> direct
}
}
frame.frame.setObj(i, value)
}
}
CmdVm().execute(fn, context, scope.args, binder)
CmdVm().executeFastOnlyNoSuspend(fn, context, Arguments.EMPTY, binder)
} catch (e: ReturnException) {
if (e.label == null || returnLabels.contains(e.label)) e.result else throw e
}
}
suspend fun invokeWithArgs(scope: Scope, args: Arguments): Obj {
val context = buildContext(scope, args)
return try {
val vm = CmdVm()
if (supportsFastOnlyVm(args)) {
val binder: (CmdFrame, Arguments) -> Unit = { frame, arguments ->
bindArgumentsFast(frame, context, arguments)
}
vm.executeFastOnly(fn, context, args, binder)
} else {
val binder: suspend (CmdFrame, Arguments) -> Unit = { frame, arguments ->
bindArguments(frame, context, arguments)
seedUndeclaredLocals(frame, context)
}
vm.execute(fn, context, args, binder)
}
} catch (e: ReturnException) {
if (e.label == null || returnLabels.contains(e.label)) e.result else throw e
}
}
fun invokeWithArgsFast(scope: Scope, args: Arguments): Obj? {
if (!supportsFastOnlyVm(args)) return null
val context = buildContext(scope, args)
val binder: (CmdFrame, Arguments) -> Unit = { frame, arguments ->
bindArgumentsFast(frame, context, arguments)
}
return try {
CmdVm().executeFastOnlyNoSuspend(fn, context, args, binder)
} catch (e: ReturnException) {
if (e.label == null || returnLabels.contains(e.label)) e.result else throw e
}
}
override fun callWithArgsFast(scope: Scope, args: Arguments): Obj? = invokeWithArgsFast(scope, args)
override fun callOnFast(scope: Scope): Obj? = invokeWithArgsFast(scope, scope.args)
override suspend fun execute(scope: Scope): Obj {
return invokeWithArgs(scope, scope.args)
}
}
class CmdIterPush(internal val iterSlot: Int) : Cmd() {
@@ -4174,10 +4449,15 @@ class CmdFrame(
} else {
val value = record.value
if (!record.isMutable && value is FrameSlotRef) {
val resolved = value.peekValue()
val resolved = value.resolvedCaptureValueOrNull()
if (resolved != null) {
if (value.refersTo(frame, localIndex)) continue
frame.setObj(localIndex, value.read())
when (resolved) {
is ObjInt -> frame.setInt(localIndex, resolved.value)
is ObjReal -> frame.setReal(localIndex, resolved.value)
is ObjBool -> frame.setBool(localIndex, resolved.value)
else -> frame.setObj(localIndex, resolved)
}
} else {
frame.setObj(localIndex, value)
}
@@ -4213,9 +4493,14 @@ class CmdFrame(
} else {
val value = record.value
if (!record.isMutable && value is FrameSlotRef) {
val resolved = value.peekValue()
val resolved = value.resolvedCaptureValueOrNull()
if (resolved != null) {
frame.setObj(idx, value.read())
when (resolved) {
is ObjInt -> frame.setInt(idx, resolved.value)
is ObjReal -> frame.setReal(idx, resolved.value)
is ObjBool -> frame.setBool(idx, resolved.value)
else -> frame.setObj(idx, resolved)
}
} else {
frame.setObj(idx, value)
}
@@ -4267,7 +4552,7 @@ class CmdFrame(
type = inherited.type
)
copied.delegate = inherited.delegate
return@mapIndexed copied
return@mapIndexed freezeImmutableCaptureRecord(copied)
}
}
val isMutable = fn.localSlotMutables.getOrNull(localIndex) ?: false
@@ -4279,15 +4564,25 @@ class CmdFrame(
}
} else {
val raw = frame.getRawObj(localIndex)
if (raw == null && name != null) {
// For primitive-typed slots (INT/REAL/BOOL), getRawObj returns null
// but the value exists in the typed slot array. Skip the scope name
// lookup to avoid finding a same-named symbol higher in the scope chain
// (e.g., a List.size property when capturing a `size: Int` parameter).
val isPrimitiveSlot = when (frame.getSlotTypeCode(localIndex)) {
SlotType.INT.code, SlotType.REAL.code, SlotType.BOOL.code -> true
else -> false
}
if (raw == null && name != null && !isPrimitiveSlot) {
val record = findNamedExistingRecord(scope, name)
if (record != null) {
val value = record.value
return@mapIndexed when (value) {
return@mapIndexed freezeImmutableCaptureRecord(
when (value) {
is FrameSlotRef -> ObjRecord(value, isMutable)
is RecordSlotRef -> ObjRecord(value, isMutable)
else -> ObjRecord(value, isMutable)
}
}
)
}
if (hasNamedScopeBinding(scope, name)) {
throw ScriptError(
@@ -4296,11 +4591,13 @@ class CmdFrame(
)
}
}
when (raw) {
is FrameSlotRef -> ObjRecord(raw, isMutable)
is RecordSlotRef -> ObjRecord(raw, isMutable)
else -> ObjRecord(FrameSlotRef(frame, localIndex), isMutable)
}
freezeImmutableCaptureRecord(
when (raw) {
is FrameSlotRef -> ObjRecord(raw, isMutable)
is RecordSlotRef -> ObjRecord(raw, isMutable)
else -> ObjRecord(FrameSlotRef(frame, localIndex), isMutable)
}
)
}
}
@@ -4446,6 +4743,19 @@ class CmdFrame(
return ExecutionError(errorObject, pos, message, t)
}
fun normalizeThrowableFast(t: Throwable): Throwable {
if (t is ExecutionError || t is ReturnException || t is LoopBreakContinueException) return t
val parentScope = ensureScope()
val pos = (t as? ScriptError)?.pos ?: currentErrorPos() ?: parentScope.pos
val throwScope = parentScope.createChildScope(pos = pos)
val message = when (t) {
is ScriptError -> t.errorMessage
else -> t.message ?: t.toString()
}
val errorObject = ObjUnknownException(throwScope, message)
return ExecutionError(errorObject, pos, message, t)
}
suspend fun handleException(t: Throwable): Boolean {
val handler = tryStack.lastOrNull() ?: return false
vmIterDebug {
@@ -5400,6 +5710,14 @@ class CmdFrame(
if (index < target.slotCount) return index
return index
}
if (target.hasSlotPlanConflict(mapOf(name to index))) {
val record = target.getLocalRecordDirect(name)
?: target.localBindings[name]
?: target.parent?.get(name)
?: target.get(name)
?: ObjRecord(ObjUnset, isMutable = true)
return target.allocateSlotFor(name, record)
}
target.applySlotPlan(mapOf(name to index))
val existing = target.getLocalRecordDirect(name) ?: target.localBindings[name]
if (existing != null) {
@@ -175,6 +175,8 @@ enum class Opcode(val code: Int) {
CALL_SLOT(0x93),
CALL_BRIDGE_SLOT(0x94),
LIST_NEW_INT(0xA0),
LIST_IOTA_INT(0xA1),
GET_INDEX(0xA2),
SET_INDEX(0xA3),
GET_INDEX_INT(0xA4),
@@ -188,6 +190,8 @@ enum class Opcode(val code: Int) {
GET_DYNAMIC_MEMBER(0xAC),
SET_DYNAMIC_MEMBER(0xAD),
CALL_DYNAMIC_MEMBER(0xAE),
LIST_NEW_INT_CAP(0xAF),
LIST_FILL_INT_CAP(0xB0),
RESOLVE_SCOPE_SLOT(0xB1),
LOAD_OBJ_ADDR(0xB2),
@@ -17,7 +17,58 @@
package net.sergeych.lyng.bytecode
import net.sergeych.lyng.Scope
import net.sergeych.lyng.FrameSlotRef
import net.sergeych.lyng.RecordSlotRef
import net.sergeych.lyng.ScopeSlotRef
import net.sergeych.lyng.obj.ObjRecord
import net.sergeych.lyng.obj.ObjProperty
internal fun canFastSeedUndeclaredLocals(
fn: CmdFunction,
declaredLocalNames: Set<String>,
preboundLocalNames: Set<String>
): Boolean {
if (fn.localSlotNames.isEmpty()) return true
for (i in fn.localSlotNames.indices) {
val name = fn.localSlotNames[i] ?: continue
if (declaredLocalNames.contains(name)) continue
if (fn.localSlotCaptures.getOrNull(i) == true) continue
if (preboundLocalNames.contains(name)) continue
return false
}
return true
}
internal fun trySeedFrameLocalsFromScopeFast(frame: CmdFrame, scope: Scope): Boolean {
val localNames = frame.fn.localSlotNames
if (localNames.isEmpty()) return true
val base = frame.fn.scopeSlotCount
for (i in localNames.indices) {
val name = localNames[i] ?: continue
val slotType = frame.getLocalSlotTypeCode(i)
if (slotType != SlotType.UNKNOWN.code && slotType != SlotType.OBJ.code) continue
if (slotType == SlotType.OBJ.code && frame.frame.getRawObj(i) != null) continue
val record = scope.getLocalRecordDirect(name)
?: scope.chainLookupIgnoreClosure(name, followClosure = true)
?: continue
val value = when {
record.type == ObjRecord.Type.Delegated -> return false
record.type == ObjRecord.Type.Property -> return false
record.value is ObjProperty -> return false
else -> when (val direct = record.value) {
is FrameSlotRef -> direct.resolvedCaptureValueOrNull() ?: return false
is RecordSlotRef -> direct.resolvedCaptureValueOrNull() ?: return false
is ScopeSlotRef -> direct.resolvedCaptureValueOrNull() ?: return false
else -> direct
}
}
if (value is FrameSlotRef && value.refersTo(frame.frame, i)) {
continue
}
frame.setObjUnchecked(base + i, value)
}
return true
}
internal suspend fun seedFrameLocalsFromScope(frame: CmdFrame, scope: Scope) {
val localNames = frame.fn.localSlotNames
@@ -99,10 +99,11 @@ fun ObjClass.addFnDoc(
visibility: Visibility = Visibility.Public,
tags: Map<String, List<String>> = emptyMap(),
moduleName: String? = null,
callSignature: net.sergeych.lyng.CallSignature? = null,
code: suspend ScopeFacade.() -> Obj
) {
// Register runtime method
addFn(name, isOpen, visibility, code = code)
addFn(name, isOpen, visibility, callSignature = callSignature, code = code)
// Register docs for the member under this class
BuiltinDocRegistry.module(moduleName ?: ownerModuleNameFromClassOrUnknown()) {
classDoc(this@addFnDoc.className, doc = "") {
@@ -137,9 +138,10 @@ fun ObjClass.addClassFnDoc(
isOpen: Boolean = false,
tags: Map<String, List<String>> = emptyMap(),
moduleName: String? = null,
callSignature: net.sergeych.lyng.CallSignature? = null,
code: suspend ScopeFacade.() -> Obj
) {
addClassFn(name, isOpen, code)
addClassFn(name, isOpen, callSignature, code)
BuiltinDocRegistry.module(moduleName ?: ownerModuleNameFromClassOrUnknown()) {
classDoc(this@addClassFnDoc.className, doc = "") {
method(name = name, doc = doc, params = params, returns = returns, isStatic = true, tags = tags)
@@ -64,11 +64,13 @@ object StdlibDocsBootstrap {
val _buffer = net.sergeych.lyng.obj.ObjBuffer.type
// Also touch time module types so their docs (moduleName = "lyng.time") are registered
// This enables completion/quick docs for symbols imported via `import lyng.time` (e.g., Instant, DateTime, Duration)
// This enables completion/quick docs for symbols imported via `import lyng.time` (e.g., Instant, Date, DateTime, Duration)
try {
@Suppress("UNUSED_VARIABLE")
val _instant = net.sergeych.lyng.obj.ObjInstant.type
@Suppress("UNUSED_VARIABLE")
val _date = net.sergeych.lyng.obj.ObjDate.type
@Suppress("UNUSED_VARIABLE")
val _datetime = net.sergeych.lyng.obj.ObjDateTime.type
@Suppress("UNUSED_VARIABLE")
val _duration = net.sergeych.lyng.obj.ObjDuration.type
@@ -30,6 +30,8 @@ class LambdaFnRef(
val argsDeclaration: ArgsDeclaration?,
val captureEntries: List<LambdaCaptureEntry>,
val inferredReturnClass: ObjClass?,
val inlineBodyRef: ObjRef?,
val supportsDirectInvokeFastPath: Boolean,
val preferredThisType: String?,
val wrapAsExtensionCallable: Boolean,
val returnLabels: Set<String>,
@@ -24,6 +24,7 @@ import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.serializer
import net.sergeych.lyng.BytecodeCallable
import net.sergeych.lyng.*
import net.sergeych.lyng.InteropOperator
import net.sergeych.lyng.OperatorInteropRegistry
@@ -723,40 +724,39 @@ open class Obj {
scope.raiseNotImplemented()
}
suspend fun invoke(scope: Scope, thisObj: Obj, args: Arguments, declaringClass: ObjClass? = null): Obj {
val usePool = PerfFlags.SCOPE_POOL && this !is Statement
private suspend fun invokeWithBoundScope(
scope: Scope,
args: Arguments,
thisObj: Obj,
declaringClass: ObjClass? = null,
atPos: Pos = scope.pos
): Obj {
val usePool = PerfFlags.SCOPE_POOL && this !is Statement && atPos == scope.pos
return if (usePool) {
scope.withChildFrame(args, newThisObj = thisObj) { child ->
if (declaringClass != null) child.currentClassCtx = declaringClass
callOn(child)
(this as? BytecodeCallable)?.callOnFast(child) ?: callOn(child)
}
} else {
callOn(scope.createChildScope(scope.pos, args = args, newThisObj = thisObj).also {
val child = scope.createChildScope(atPos, args = args, newThisObj = thisObj).also {
if (declaringClass != null) it.currentClassCtx = declaringClass
})
}
(this as? BytecodeCallable)?.callOnFast(child) ?: callOn(child)
}
}
suspend fun invoke(scope: Scope, thisObj: Obj, args: Arguments, declaringClass: ObjClass? = null): Obj {
return invokeWithBoundScope(scope, args, thisObj, declaringClass)
}
suspend fun invoke(scope: Scope, thisObj: Obj, vararg args: Obj): Obj =
callOn(
scope.createChildScope(
scope.pos,
args = Arguments(args.toList()),
newThisObj = thisObj
)
)
invokeWithBoundScope(scope, Arguments(args.toList()), thisObj)
suspend fun invoke(scope: Scope, thisObj: Obj): Obj =
callOn(
scope.createChildScope(
scope.pos,
args = Arguments.EMPTY,
newThisObj = thisObj
)
)
invokeWithBoundScope(scope, Arguments.EMPTY, thisObj)
suspend fun invoke(scope: Scope, atPos: Pos, thisObj: Obj, args: Arguments): Obj =
callOn(scope.createChildScope(atPos, args = args, newThisObj = thisObj))
invokeWithBoundScope(scope, args, thisObj, atPos = atPos)
val asReadonly: ObjRecord by lazy { ObjRecord(this, false) }
@@ -828,7 +828,13 @@ open class Obj {
name = "let",
doc = "Calls the specified function block with `this` value as its argument and returns its result.",
params = listOf(ParamDoc("block")),
moduleName = "lyng.stdlib"
moduleName = "lyng.stdlib",
callSignature = CallSignature(
inlineHigherOrder = CallSignature.HigherOrderInline(
kind = CallSignature.Kind.UNARY_ARGUMENT,
result = CallSignature.ResultMode.BLOCK_RESULT
)
)
) {
call(args.firstAndOnly(), Arguments(thisObj))
}
@@ -836,7 +842,13 @@ open class Obj {
name = "apply",
doc = "Calls the specified function block with `this` value as its receiver and returns `this` value.",
params = listOf(ParamDoc("block")),
moduleName = "lyng.stdlib"
moduleName = "lyng.stdlib",
callSignature = CallSignature(
inlineHigherOrder = CallSignature.HigherOrderInline(
kind = CallSignature.Kind.RECEIVER,
result = CallSignature.ResultMode.RETURN_RECEIVER
)
)
) {
val body = args.firstAndOnly()
val scope = requireScope()
@@ -852,7 +864,13 @@ open class Obj {
name = "also",
doc = "Calls the specified function block with `this` value as its argument and returns `this` value.",
params = listOf(ParamDoc("block")),
moduleName = "lyng.stdlib"
moduleName = "lyng.stdlib",
callSignature = CallSignature(
inlineHigherOrder = CallSignature.HigherOrderInline(
kind = CallSignature.Kind.UNARY_ARGUMENT,
result = CallSignature.ResultMode.RETURN_RECEIVER
)
)
) {
call(args.firstAndOnly(), Arguments(thisObj))
thisObj
@@ -861,7 +879,13 @@ open class Obj {
name = "run",
doc = "Calls the specified function block with `this` value as its receiver and returns its result.",
params = listOf(ParamDoc("block")),
moduleName = "lyng.stdlib"
moduleName = "lyng.stdlib",
callSignature = CallSignature(
inlineHigherOrder = CallSignature.HigherOrderInline(
kind = CallSignature.Kind.RECEIVER,
result = CallSignature.ResultMode.BLOCK_RESULT
)
)
) {
call(args.firstAndOnly())
}
@@ -0,0 +1,62 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.requiredArg
internal fun ObjClass.bindClassFn(name: String, code: suspend ScopeFacade.() -> Obj) {
val callable = ObjExternCallable.fromBridge { code() }
val memberRecord = members[name]
val classScopeRecord = classScope?.objects?.get(name)
if (memberRecord != null) {
val methodId = ensureMethodIdForBridge(name, memberRecord)
val newRecord = memberRecord.copy(
value = callable,
type = ObjRecord.Type.Fun,
methodId = methodId,
isAbstract = false,
)
replaceMemberForBridge(name, newRecord)
if (classScopeRecord != null) {
replaceClassScopeMemberForBridge(name, newRecord)
}
} else if (classScopeRecord != null) {
val methodId = ensureMethodIdForBridge(name, classScopeRecord)
replaceClassScopeMemberForBridge(
name,
classScopeRecord.copy(
value = callable,
type = ObjRecord.Type.Fun,
methodId = methodId,
isAbstract = false,
)
)
} else {
addClassFn(name, code = code)
}
}
internal fun ObjClass.bindProperty(
name: String,
getter: (suspend ScopeFacade.() -> Obj)? = null,
setter: (suspend ScopeFacade.(Obj) -> Unit)? = null,
) {
val g = getter?.let { ObjExternCallable.fromBridge { it() } }
val s = setter?.let { ObjExternCallable.fromBridge { it(requiredArg(0)); ObjVoid } }
val prop = ObjProperty(name, g, s)
val existing = members[name]
if (existing != null) {
val newRecord = existing.copy(
value = prop,
type = ObjRecord.Type.Property,
methodId = ensureMethodIdForBridge(name, existing),
fieldId = null,
isAbstract = false,
)
replaceMemberForBridge(name, newRecord)
if (classScope?.objects?.containsKey(name) == true) {
replaceClassScopeMemberForBridge(name, newRecord)
}
} else {
addProperty(name, getter = getter, setter = setter)
}
}
@@ -0,0 +1,133 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.obj
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.channels.ClosedReceiveChannelException
import kotlinx.coroutines.channels.ClosedSendChannelException
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.addPropertyDoc
import net.sergeych.lyng.miniast.type
/**
* Lyng-visible wrapper around Kotlin's [Channel].
*
* Construction:
* - `Channel()` – rendezvous channel (capacity 0, sender and receiver synchronise)
* - `Channel(n)` – buffered channel with capacity *n* (n > 0)
* - `Channel(Channel.UNLIMITED)` – unlimited-buffer channel
*
* Methods:
* - `send(value)` – suspend until the value is accepted by a receiver (or buffered)
* - `receive()` – suspend until a value is available; returns `null` when channel is closed and drained
* - `tryReceive()` – return the next value immediately, or `null` if none is available
* - `close()` – signal that no more values will be sent; pending receivers can still drain buffered items
*
* Properties:
* - `isClosedForSend: Bool`
* - `isClosedForReceive: Bool`
*/
class ObjChannel(val channel: Channel<Obj>) : Obj() {
override val objClass get() = type
companion object {
val type = object : ObjClass("Channel") {
override suspend fun callOn(scope: Scope): Obj {
val capacity = scope.args.list.getOrNull(0)
?.let { (it as? ObjInt)?.value?.toInt() ?: scope.raiseIllegalArgument("Channel capacity must be an integer") }
?: Channel.RENDEZVOUS
return ObjChannel(Channel(capacity))
}
}.apply {
// Expose Channel.UNLIMITED as a static constant on the Channel class so scripts can write
// Channel(Channel.UNLIMITED).
addConst("UNLIMITED", ObjInt(Channel.UNLIMITED.toLong()))
addFnDoc(
name = "send",
doc = "Suspend until the value is accepted by a receiver (or placed into the buffer). " +
"Throws if the channel is already closed.",
params = listOf(ParamDoc("value", type("lyng.Any"))),
returns = type("lyng.Void"),
moduleName = "lyng.stdlib"
) {
val value = requiredArg<Obj>(0)
try {
thisAs<ObjChannel>().channel.send(value)
} catch (e: ClosedSendChannelException) {
raiseIllegalState("Channel is closed for send")
}
ObjVoid
}
addFnDoc(
name = "receive",
doc = "Suspend until a value is available and return it, or return `null` when the channel " +
"is closed and all buffered items have been consumed.",
returns = type("lyng.Any"),
moduleName = "lyng.stdlib"
) {
try {
thisAs<ObjChannel>().channel.receive()
} catch (_: ClosedReceiveChannelException) {
ObjNull
}
}
addFnDoc(
name = "tryReceive",
doc = "Return the next buffered value immediately without suspending, or `null` if the " +
"channel is empty or closed.",
returns = type("lyng.Any"),
moduleName = "lyng.stdlib"
) {
val result = thisAs<ObjChannel>().channel.tryReceive()
result.getOrNull() ?: ObjNull
}
addFnDoc(
name = "close",
doc = "Signal that no more values will be sent. Receivers can still drain any buffered items.",
returns = type("lyng.Void"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjChannel>().channel.close()
ObjVoid
}
addPropertyDoc(
name = "isClosedForSend",
doc = "Whether this channel is closed for sending (no more `send` calls are permitted).",
type = type("lyng.Bool"),
moduleName = "lyng.stdlib",
getter = { thisAs<ObjChannel>().channel.isClosedForSend.toObj() }
)
addPropertyDoc(
name = "isClosedForReceive",
doc = "Whether this channel is closed for receiving (closed and fully drained).",
type = type("lyng.Bool"),
moduleName = "lyng.stdlib",
getter = { thisAs<ObjChannel>().channel.isClosedForReceive.toObj() }
)
}
}
}
@@ -809,7 +809,8 @@ open class ObjClass(
if (initStmt is net.sergeych.lyng.Statement) {
executeBytecodeWithSeed(instance.instanceScope, initStmt, "instance init")
} else {
initStmt.callOn(instance.instanceScope)
(initStmt as? net.sergeych.lyng.BytecodeCallable)?.callOnFast(instance.instanceScope)
?: initStmt.callOn(instance.instanceScope)
}
}
} finally {
@@ -821,13 +822,14 @@ open class ObjClass(
c.instanceConstructor?.let { ctor ->
val execScope =
instance.instanceScope.createChildScope(args = argsForThis ?: Arguments.EMPTY, newThisObj = instance)
ctor.callOn(execScope)
(ctor as? net.sergeych.lyng.BytecodeCallable)?.callOnFast(execScope) ?: ctor.callOn(execScope)
}
}
}
suspend fun callWithArgs(scope: Scope, vararg plainArgs: Obj): Obj {
return callOn(scope.createChildScope(Arguments(*plainArgs)))
val child = scope.createChildScope(Arguments(*plainArgs))
return (this as? net.sergeych.lyng.BytecodeCallable)?.callOnFast(child) ?: callOn(child)
}
@@ -847,6 +849,7 @@ open class ObjClass(
fieldId: Int? = null,
methodId: Int? = null,
typeDecl: net.sergeych.lyng.TypeDecl? = null,
callSignature: net.sergeych.lyng.CallSignature? = null,
): ObjRecord {
// Validation of override rules: only for non-system declarations
var existing: ObjRecord? = null
@@ -947,6 +950,7 @@ open class ObjClass(
isOverride = isOverride,
isTransient = isTransient,
type = type,
callSignature = callSignature,
typeDecl = typeDecl,
memberName = name,
fieldId = effectiveFieldId,
@@ -959,7 +963,7 @@ open class ObjClass(
}
private fun initClassScope(): Scope {
if (classScope == null) classScope = Scope()
if (classScope == null) classScope = Scope(parent = null)
return classScope!!
}
@@ -973,7 +977,8 @@ open class ObjClass(
isTransient: Boolean = false,
type: ObjRecord.Type = ObjRecord.Type.Field,
fieldId: Int? = null,
methodId: Int? = null
methodId: Int? = null,
callSignature: net.sergeych.lyng.CallSignature? = null
): ObjRecord {
initClassScope()
val existing = classScope!!.objects[name]
@@ -1014,6 +1019,7 @@ open class ObjClass(
writeVisibility,
recordType = type,
isTransient = isTransient,
callSignature = callSignature,
fieldId = effectiveFieldId,
methodId = effectiveMethodId
)
@@ -1033,6 +1039,7 @@ open class ObjClass(
isOverride: Boolean = false,
pos: Pos = Pos.builtIn,
methodId: Int? = null,
callSignature: net.sergeych.lyng.CallSignature? = null,
code: (suspend net.sergeych.lyng.ScopeFacade.() -> Obj)? = null
) {
val stmt = code?.let { ObjExternCallable.fromBridge { it() } } ?: ObjNull
@@ -1040,7 +1047,8 @@ open class ObjClass(
name, stmt, isMutable, visibility, writeVisibility, pos, declaringClass,
isAbstract = isAbstract, isClosed = isClosed, isOverride = isOverride,
type = ObjRecord.Type.Fun,
methodId = methodId
methodId = methodId,
callSignature = callSignature
)
}
@@ -1072,8 +1080,19 @@ open class ObjClass(
}
fun addClassConst(name: String, value: Obj) = createClassField(name, value)
fun addClassFn(name: String, isOpen: Boolean = false, code: suspend net.sergeych.lyng.ScopeFacade.() -> Obj) {
createClassField(name, ObjExternCallable.fromBridge { code() }, isOpen, type = ObjRecord.Type.Fun)
fun addClassFn(
name: String,
isOpen: Boolean = false,
callSignature: net.sergeych.lyng.CallSignature? = null,
code: suspend net.sergeych.lyng.ScopeFacade.() -> Obj
) {
createClassField(
name,
ObjExternCallable.fromBridge { code() },
isOpen,
type = ObjRecord.Type.Fun,
callSignature = callSignature
)
}
@@ -18,6 +18,7 @@
package net.sergeych.lyng.obj
import kotlinx.coroutines.CompletableDeferred
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.addFnDoc
@@ -43,6 +44,30 @@ class ObjCompletableDeferred(val completableDeferred: CompletableDeferred<Obj>):
thisAs<ObjCompletableDeferred>().completableDeferred.complete(args.firstAndOnly())
ObjVoid
}
addFnDoc(
name = "completeExceptionally",
doc = "Fail this deferred with the given exception. Awaiting it will then throw that exception. " +
"Subsequent calls have no effect. The argument must be an `Exception` instance.",
params = listOf(ParamDoc("exception", type("lyng.Exception"))),
returns = type("lyng.Void"),
moduleName = "lyng.stdlib"
) {
val ex = requiredArg<Obj>(0)
val scope = requireScope()
val msg = when (ex) {
is ObjException -> ex.message.value
else -> ex.toString(scope).value
}
val pos = when (ex) {
is ObjException -> ex.scope.pos
else -> scope.pos
}
// Always carry the original Lyng object as errorObject so that
// assertThrows / catch clauses see the correct exception class.
val cause = ExecutionError(ex, pos, msg)
thisAs<ObjCompletableDeferred>().completableDeferred.completeExceptionally(cause)
ObjVoid
}
}
}
}
@@ -18,13 +18,10 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.*
import net.sergeych.lyng.miniast.addPropertyDoc
import net.sergeych.lyng.miniast.type
import net.sergeych.lyng.requiredArg
object ObjComplexSupport {
private object BoundMarker
private val complexTypeDecl = TypeDecl.Simple("lyng.complex.Complex", false)
suspend fun bindTo(module: ModuleScope) {
val complexClass = module.requireClass("Complex")
@@ -33,36 +30,20 @@ object ObjComplexSupport {
val decimalModule = module.currentImportProvider.createModuleScope(module.pos, "lyng.decimal")
val decimalClass = decimalModule.requireClass("Decimal")
decimalClass.addPropertyDoc(
name = "re",
doc = "Convert this Decimal to a Complex with zero imaginary part.",
type = type("lyng.complex.Complex"),
moduleName = "lyng.complex",
getter = {
newComplex(
complexClass,
decimalToReal(thisObj),
0.0
)
}
)
decimalClass.members["re"] = decimalClass.members.getValue("re").copy(typeDecl = complexTypeDecl)
decimalClass.addPropertyDoc(
name = "i",
doc = "Convert this Decimal to a pure imaginary Complex after rounding to Real.",
type = type("lyng.complex.Complex"),
moduleName = "lyng.complex",
getter = {
newComplex(
complexClass,
0.0,
decimalToReal(thisObj)
)
}
)
decimalClass.members["i"] = decimalClass.members.getValue("i").copy(typeDecl = complexTypeDecl)
decimalClass.bindProperty("re", getter = {
newComplex(
complexClass,
decimalToReal(thisObj),
0.0
)
})
decimalClass.bindProperty("i", getter = {
newComplex(
complexClass,
0.0,
decimalToReal(thisObj)
)
})
OperatorInteropRegistry.register(
leftClass = decimalClass,
@@ -0,0 +1,303 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.obj
import kotlinx.datetime.LocalDate
import kotlinx.datetime.LocalDateTime
import kotlinx.datetime.TimeZone
import kotlinx.datetime.UtcOffset
import kotlinx.datetime.asTimeZone
import kotlinx.datetime.isoDayNumber
import kotlinx.datetime.number
import kotlinx.datetime.toInstant
import kotlinx.datetime.toLocalDateTime
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonPrimitive
import net.sergeych.lyng.Scope
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.asFacade
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.addClassFnDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.addPropertyDoc
import net.sergeych.lyng.miniast.type
import net.sergeych.lynon.LynonDecoder
import net.sergeych.lynon.LynonEncoder
import net.sergeych.lynon.LynonType
import kotlin.math.absoluteValue
import kotlin.time.Clock
import kotlin.time.Duration
import kotlin.time.Duration.Companion.days
class ObjDate(val date: LocalDate) : Obj() {
override val objClass: ObjClass get() = type
override fun toString(): String = date.toString()
override suspend fun plus(scope: Scope, other: Obj): Obj {
return when (other) {
is ObjDuration -> ObjDate(addDays(date, requireWholeDays(scope, other.duration)))
else -> super.plus(scope, other)
}
}
override suspend fun minus(scope: Scope, other: Obj): Obj {
return when (other) {
is ObjDuration -> ObjDate(addDays(date, -requireWholeDays(scope, other.duration)))
is ObjDate -> ObjInt.of(daysBetween(other.date, date).toLong())
else -> super.minus(scope, other)
}
}
override suspend fun compareTo(scope: Scope, other: Obj): Int {
return if (other is ObjDate) {
date.compareTo(other.date)
} else super.compareTo(scope, other)
}
override suspend fun toKotlin(scope: Scope): Any = date
override fun hashCode(): Int = date.hashCode()
override fun equals(other: Any?): Boolean = other is ObjDate && date == other.date
override suspend fun lynonType(): LynonType = LynonType.Date
override suspend fun serialize(scope: Scope, encoder: LynonEncoder, lynonType: LynonType?) {
encoder.encodeSigned(date.year.toLong())
encoder.encodeUnsigned(date.month.number.toULong())
encoder.encodeUnsigned(date.day.toULong())
}
override suspend fun toJson(scope: Scope): JsonElement = JsonPrimitive(date.toString())
companion object {
val type = object : ObjClass("Date") {
override suspend fun callOn(scope: Scope): Obj {
val args = scope.args
return when (val a0 = args.list.getOrNull(0)) {
null -> ObjDate(today(TimeZone.currentSystemDefault()))
is ObjDate -> a0
is ObjInt -> {
val year = a0.value.toInt()
val month = args.list.getOrNull(1)?.toInt() ?: scope.raiseIllegalArgument("month is required")
val day = args.list.getOrNull(2)?.toInt() ?: scope.raiseIllegalArgument("day is required")
ObjDate(createDate(scope, year, month, day))
}
is ObjString -> ObjDate(parseIso(scope, a0.value))
is ObjDateTime -> ObjDate(toDate(a0.instant, a0.timeZone))
is ObjInstant -> {
val tz = parseTimeZoneArg(scope, args.list.getOrNull(1), TimeZone.currentSystemDefault())
ObjDate(toDate(a0.instant, tz))
}
else -> scope.raiseIllegalArgument("can't construct Date from ${args.inspect(scope)}")
}
}
override suspend fun deserialize(scope: Scope, decoder: LynonDecoder, lynonType: LynonType?): Obj {
val year = decoder.unpackSigned().toInt()
val month = decoder.unpackUnsigned().toInt()
val day = decoder.unpackUnsigned().toInt()
return ObjDate(createDate(scope, year, month, day))
}
}.apply {
addPropertyDoc("year", "The year component.", type("lyng.Int"), moduleName = "lyng.time",
getter = { thisAs<ObjDate>().date.year.toObj() })
addPropertyDoc("month", "The month component (1..12).", type("lyng.Int"), moduleName = "lyng.time",
getter = { thisAs<ObjDate>().date.month.number.toObj() })
addPropertyDoc("dayOfMonth", "The day of month component.", type("lyng.Int"), moduleName = "lyng.time",
getter = { thisAs<ObjDate>().date.day.toObj() })
addPropertyDoc("day", "Alias to dayOfMonth.", type("lyng.Int"), moduleName = "lyng.time",
getter = { thisAs<ObjDate>().date.day.toObj() })
addPropertyDoc("dayOfWeek", "The day of week (1=Monday, 7=Sunday).", type("lyng.Int"), moduleName = "lyng.time",
getter = { thisAs<ObjDate>().date.dayOfWeek.isoDayNumber.toObj() })
addPropertyDoc("dayOfYear", "The day of year (1..365/366).", type("lyng.Int"), moduleName = "lyng.time",
getter = { thisAs<ObjDate>().date.dayOfYear.toObj() })
addPropertyDoc("isLeapYear", "Whether this date is in a leap year.", type("lyng.Bool"), moduleName = "lyng.time",
getter = { isLeapYear(thisAs<ObjDate>().date.year).toObj() })
addPropertyDoc("lengthOfMonth", "Number of days in this date's month.", type("lyng.Int"), moduleName = "lyng.time",
getter = { monthLength(thisAs<ObjDate>().date.year, thisAs<ObjDate>().date.month.number).toObj() })
addPropertyDoc("lengthOfYear", "Number of days in this date's year.", type("lyng.Int"), moduleName = "lyng.time",
getter = { (if (isLeapYear(thisAs<ObjDate>().date.year)) 366 else 365).toObj() })
addFnDoc("toIsoString", "Return the canonical ISO date string representation (`YYYY-MM-DD`).",
returns = type("lyng.String"), moduleName = "lyng.time") {
thisAs<ObjDate>().date.toString().toObj()
}
addFnDoc("toSortableString", "Alias to toIsoString.", returns = type("lyng.String"), moduleName = "lyng.time") {
thisAs<ObjDate>().date.toString().toObj()
}
addFnDoc("toDateTime", "Convert this date to a DateTime at the start of day in the specified timezone.",
params = listOf(ParamDoc("tz", type = type("lyng.Any", true))),
returns = type("lyng.DateTime"), moduleName = "lyng.time") {
val tz = parseTimeZoneArg(this, args.list.getOrNull(0), TimeZone.UTC)
toDateTime(thisAs<ObjDate>().date, tz)
}
addFnDoc("atStartOfDay", "Alias to toDateTime.",
params = listOf(ParamDoc("tz", type = type("lyng.Any", true))),
returns = type("lyng.DateTime"), moduleName = "lyng.time") {
val tz = parseTimeZoneArg(this, args.list.getOrNull(0), TimeZone.UTC)
toDateTime(thisAs<ObjDate>().date, tz)
}
addFnDoc("addDays", "Return a new Date with the specified number of days added (or subtracted if negative).",
params = listOf(ParamDoc("days", type = type("lyng.Int"))),
returns = type("lyng.Date"), moduleName = "lyng.time") {
val n = args.list.getOrNull(0)?.toInt() ?: 0
ObjDate(addDays(thisAs<ObjDate>().date, n))
}
addFnDoc("addMonths", "Return a new Date with the specified number of months added (or subtracted if negative). End-of-month values are normalized.",
params = listOf(ParamDoc("months", type = type("lyng.Int"))),
returns = type("lyng.Date"), moduleName = "lyng.time") {
val n = args.list.getOrNull(0)?.toInt() ?: 0
ObjDate(addMonths(thisAs<ObjDate>().date, n))
}
addFnDoc("addYears", "Return a new Date with the specified number of years added (or subtracted if negative).",
params = listOf(ParamDoc("years", type = type("lyng.Int"))),
returns = type("lyng.Date"), moduleName = "lyng.time") {
val n = args.list.getOrNull(0)?.toInt() ?: 0
ObjDate(addYears(thisAs<ObjDate>().date, n))
}
addFnDoc("daysUntil", "Return the number of whole calendar days until the other date.",
params = listOf(ParamDoc("other", type = type("lyng.Date"))),
returns = type("lyng.Int"), moduleName = "lyng.time") {
val other = requiredArg<ObjDate>(0)
daysBetween(thisAs<ObjDate>().date, other.date).toObj()
}
addFnDoc("daysSince", "Return the number of whole calendar days since the other date.",
params = listOf(ParamDoc("other", type = type("lyng.Date"))),
returns = type("lyng.Int"), moduleName = "lyng.time") {
val other = requiredArg<ObjDate>(0)
daysBetween(other.date, thisAs<ObjDate>().date).toObj()
}
addClassFnDoc("today", "Return today's date in the specified timezone, or in the current system timezone if omitted.",
params = listOf(ParamDoc("tz", type = type("lyng.Any", true))),
returns = type("lyng.Date"), moduleName = "lyng.time") {
val tz = parseTimeZoneArg(this, args.list.getOrNull(0), TimeZone.currentSystemDefault())
ObjDate(today(tz))
}
addClassFnDoc("parseIso", "Parse an ISO date string (`YYYY-MM-DD`) into a Date.",
params = listOf(ParamDoc("string", type = type("lyng.String"))),
returns = type("lyng.Date"), moduleName = "lyng.time") {
ObjDate(parseIso(this, requiredArg<ObjString>(0).value))
}
}
}
}
internal fun parseTimeZoneArg(scope: ScopeFacade, value: Obj?, default: TimeZone): TimeZone {
return when (value) {
null -> default
is ObjString -> TimeZone.of(value.value)
is ObjInt -> UtcOffset(seconds = value.value.toInt()).asTimeZone()
else -> scope.raiseIllegalArgument("invalid timezone: $value")
}
}
internal fun parseTimeZoneArg(scope: Scope, value: Obj?, default: TimeZone): TimeZone =
parseTimeZoneArg(scope.asFacade(), value, default)
internal fun toDate(instant: kotlin.time.Instant, tz: TimeZone): LocalDate {
val ldt = instant.toLocalDateTime(tz)
return LocalDate(ldt.year, ldt.month.number, ldt.day)
}
internal fun toDateTime(date: LocalDate, tz: TimeZone): ObjDateTime {
val ldt = LocalDateTime(date.year, date.month.number, date.day, 0, 0, 0)
return ObjDateTime(ldt.toInstant(tz), tz)
}
private fun parseIso(scope: ScopeFacade, value: String): LocalDate {
val match = DATE_REGEX.matchEntire(value.trim()) ?: scope.raiseIllegalArgument("invalid ISO date string: $value")
val year = match.groupValues[1].toInt()
val month = match.groupValues[2].toInt()
val day = match.groupValues[3].toInt()
return createDate(scope, year, month, day)
}
private fun parseIso(scope: Scope, value: String): LocalDate = parseIso(scope.asFacade(), value)
private fun createDate(scope: ScopeFacade, year: Int, month: Int, day: Int): LocalDate {
if (month !in 1..12) scope.raiseIllegalArgument("month must be in 1..12")
val maxDay = monthLength(year, month)
if (day !in 1..maxDay) scope.raiseIllegalArgument("day must be in 1..$maxDay")
return LocalDate(year, month, day)
}
private fun createDate(scope: Scope, year: Int, month: Int, day: Int): LocalDate =
createDate(scope.asFacade(), year, month, day)
private fun today(tz: TimeZone): LocalDate = toDate(Clock.System.now(), tz)
private fun addDays(date: LocalDate, days: Int): LocalDate {
val start = LocalDateTime(date.year, date.month.number, date.day, 0, 0, 0).toInstant(TimeZone.UTC)
return toDate(start + days.days, TimeZone.UTC)
}
private fun daysBetween(start: LocalDate, end: LocalDate): Int {
val startInstant = LocalDateTime(start.year, start.month.number, start.day, 0, 0, 0).toInstant(TimeZone.UTC)
val endInstant = LocalDateTime(end.year, end.month.number, end.day, 0, 0, 0).toInstant(TimeZone.UTC)
return ((endInstant.epochSeconds - startInstant.epochSeconds) / 86_400L).toInt()
}
private fun addMonths(date: LocalDate, months: Int): LocalDate {
if (months == 0) return date
val totalMonths = date.year.toLong() * 12L + (date.month.number - 1).toLong() + months.toLong()
val newYear = floorDiv(totalMonths, 12L).toInt()
val newMonth = floorMod(totalMonths, 12L).toInt() + 1
val newDay = minOf(date.day, monthLength(newYear, newMonth))
return LocalDate(newYear, newMonth, newDay)
}
private fun addYears(date: LocalDate, years: Int): LocalDate {
if (years == 0) return date
val newYear = date.year + years
val newDay = minOf(date.day, monthLength(newYear, date.month.number))
return LocalDate(newYear, date.month.number, newDay)
}
private fun requireWholeDays(scope: Scope, duration: Duration): Int {
val days = duration.inWholeDays
if (days.absoluteValue > Int.MAX_VALUE.toLong()) {
scope.raiseIllegalArgument("date arithmetic day count is too large")
}
if (duration != days.days) {
scope.raiseIllegalArgument("Date arithmetic supports only whole-day durations")
}
return days.toInt()
}
private fun isLeapYear(year: Int): Boolean =
(year % 4 == 0 && year % 100 != 0) || (year % 400 == 0)
private fun monthLength(year: Int, month: Int): Int = when (month) {
1, 3, 5, 7, 8, 10, 12 -> 31
4, 6, 9, 11 -> 30
2 -> if (isLeapYear(year)) 29 else 28
else -> error("invalid month: $month")
}
private fun floorDiv(a: Long, b: Long): Long {
var q = a / b
if ((a xor b) < 0 && q * b != a) q -= 1
return q
}
private fun floorMod(a: Long, b: Long): Long = a - floorDiv(a, b) * b
private val DATE_REGEX = Regex("""([+-]?\d{4,})-(\d{2})-(\d{2})""")
@@ -202,10 +202,15 @@ class ObjDateTime(val instant: Instant, val timeZone: TimeZone) : Obj() {
getter = { thisAs<ObjDateTime>().localDateTime.dayOfWeek.isoDayNumber.toObj() })
addPropertyDoc("timeZone", "The time zone ID (e.g. 'Z', '+02:00', 'Europe/Prague').", type("lyng.String"), moduleName = "lyng.time",
getter = { thisAs<ObjDateTime>().timeZone.id.toObj() })
addPropertyDoc("date", "The calendar date component of this DateTime.", type("lyng.Date"), moduleName = "lyng.time",
getter = { ObjDate(toDate(thisAs<ObjDateTime>().instant, thisAs<ObjDateTime>().timeZone)) })
addFnDoc("toInstant", "Convert this localized date time back to an absolute Instant.", returns = type("lyng.Instant"), moduleName = "lyng.time") {
ObjInstant(thisAs<ObjDateTime>().instant)
}
addFnDoc("toDate", "Return the calendar date component of this DateTime.", returns = type("lyng.Date"), moduleName = "lyng.time") {
ObjDate(toDate(thisAs<ObjDateTime>().instant, thisAs<ObjDateTime>().timeZone))
}
addFnDoc("toEpochSeconds", "Return the number of full seconds since the Unix epoch (UTC).", returns = type("lyng.Int"), moduleName = "lyng.time") {
thisAs<ObjDateTime>().instant.epochSeconds.toObj()
}
@@ -223,11 +228,7 @@ class ObjDateTime(val instant: Instant, val timeZone: TimeZone) : Obj() {
"Accepts a timezone ID string (e.g., 'UTC', '+02:00') or an integer offset in seconds.",
params = listOf(net.sergeych.lyng.miniast.ParamDoc("tz", type = type("lyng.Any"))),
returns = type("lyng.DateTime"), moduleName = "lyng.time") {
val tz = when (val a = args.list.getOrNull(0)) {
is ObjString -> TimeZone.of(a.value)
is ObjInt -> UtcOffset(seconds = a.value.toInt()).asTimeZone()
else -> raiseIllegalArgument("invalid timezone: $a")
}
val tz = parseTimeZoneArg(this, args.list.getOrNull(0), TimeZone.currentSystemDefault())
ObjDateTime(thisAs<ObjDateTime>().instant, tz)
}
addFnDoc("toUTC", "Shortcut to convert this date time to the UTC time zone.", returns = type("lyng.DateTime"), moduleName = "lyng.time") {
@@ -106,15 +106,15 @@ object ObjDecimalSupport {
decimalClass.addFn("toStringExpanded") {
ObjString(valueOf(thisObj).toStringExpanded())
}
decimalClass.addClassFn("fromInt") {
decimalClass.bindClassFn("fromInt") {
val value = requiredArg<ObjInt>(0).value
newInstance(decimalClass, IonBigDecimal.fromLong(value))
}
decimalClass.addClassFn("fromReal") {
decimalClass.bindClassFn("fromReal") {
val value = requiredArg<ObjReal>(0).value
newInstanceFromFiniteReal(decimalClass, value)
}
decimalClass.addClassFn("fromString") {
decimalClass.bindClassFn("fromString") {
val value = requiredArg<ObjString>(0).value
try {
newInstance(decimalClass, IonBigDecimal.parseStringWithMode(value))
@@ -142,7 +142,7 @@ object ObjDecimalSupport {
}
val child = requireScope().createChildScope()
child.addConst(decimalContextVar, context)
block.callOn(child)
(block as? net.sergeych.lyng.BytecodeCallable)?.callOnFast(child) ?: block.callOn(child)
}
registerBuiltinConversions(decimalClass)
registerInterop(decimalClass)
@@ -64,13 +64,19 @@ open class ObjDynamic(var readCallback: Obj? = null, var writeCallback: Obj? = n
return (callback as? BytecodeLambdaCallable)?.rebindClosure(context) ?: callback
}
private suspend fun callCallback(callback: Obj, child: Scope): Obj {
return (callback as? net.sergeych.lyng.BytecodeCallable)?.callOnFast(child) ?: callback.callOn(child)
}
/**
* Use read callback to dynamically resolve the field name. Note that it does not work
* with method invocation which is implemented separately in [invokeInstanceMethod] below.
*/
override suspend fun readField(scope: Scope, name: String): ObjRecord {
val execBase = builderScope?.let { scope.applyClosure(it) } ?: scope
return readCallback?.callOn(execBase.createChildScope(Arguments(ObjString(name))))?.let {
return readCallback?.let { callback ->
callCallback(callback, execBase.createChildScope(Arguments(ObjString(name))))
}?.let {
if (writeCallback != null)
it.asMutable
else
@@ -90,26 +96,34 @@ open class ObjDynamic(var readCallback: Obj? = null, var writeCallback: Obj? = n
onNotFoundResult: (suspend () -> Obj?)?
): Obj {
val execBase = builderScope?.let { scope.applyClosure(it) } ?: scope
val over = readCallback?.callOn(execBase.createChildScope(Arguments(ObjString(name))))
val over = readCallback?.let { callback ->
callCallback(callback, execBase.createChildScope(Arguments(ObjString(name))))
}
return over?.invoke(scope, scope.thisObj, args)
?: super.invokeInstanceMethod(scope, name, args, onNotFoundResult)
}
override suspend fun writeField(scope: Scope, name: String, newValue: Obj) {
val execBase = builderScope?.let { scope.applyClosure(it) } ?: scope
writeCallback?.callOn(execBase.createChildScope(Arguments(ObjString(name), newValue)))
writeCallback?.let { callback ->
callCallback(callback, execBase.createChildScope(Arguments(ObjString(name), newValue)))
}
?: super.writeField(scope, name, newValue)
}
override suspend fun getAt(scope: Scope, index: Obj): Obj {
val execBase = builderScope?.let { scope.applyClosure(it) } ?: scope
return readCallback?.callOn(execBase.createChildScope(Arguments(index)))
return readCallback?.let { callback ->
callCallback(callback, execBase.createChildScope(Arguments(index)))
}
?: super.getAt(scope, index)
}
override suspend fun putAt(scope: Scope, index: Obj, newValue: Obj) {
val execBase = builderScope?.let { scope.applyClosure(it) } ?: scope
writeCallback?.callOn(execBase.createChildScope(Arguments(index, newValue)))
writeCallback?.let { callback ->
callCallback(callback, execBase.createChildScope(Arguments(index, newValue)))
}
?: super.putAt(scope, index, newValue)
}
@@ -124,7 +138,7 @@ open class ObjDynamic(var readCallback: Obj? = null, var writeCallback: Obj? = n
// Snapshot the caller scope to capture locals/args even if the runtime pools/reuses frames.
// Module scope should stay late-bound to allow extern class rebinding and similar updates.
delegate.builderScope = if (scope is net.sergeych.lyng.ModuleScope) null else scope.snapshotForClosure()
builder.callOn(buildScope)
(builder as? net.sergeych.lyng.BytecodeCallable)?.callOnFast(buildScope) ?: builder.callOn(buildScope)
return delegate
}
@@ -395,8 +395,17 @@ suspend fun Obj.getLyngExceptionMessageWithStackTrace(scope: Scope? = null,showD
var at = "unknown"
val stack = if (!trace.list.isEmpty()) {
val first = trace.list[0]
at = (first.readField(s, "at").value as ObjString).value
"\n" + trace.list.map { " at " + it.toString(s).value }.joinToString("\n")
suspend fun formatTraceEntry(entry: Obj): String {
return when (entry) {
is ObjString -> entry.value.removePrefix("#")
else -> entry.toString(s).value
}
}
at = when (first) {
is ObjString -> formatTraceEntry(first)
else -> (first.readField(s, "at").value as ObjString).value
}
"\n" + trace.list.map { " at " + formatTraceEntry(it) }.joinToString("\n")
} else {
val pos = s.pos
if (pos.source.fileName.isNotEmpty() && pos.currentLine.isNotEmpty()) {
@@ -81,7 +81,7 @@ private suspend fun createLyngFlowInput(scope: Scope, producer: Obj, ownerSessio
val runProducer: suspend CoroutineScope.() -> Unit = {
var failure: Throwable? = null
try {
producer.callOn(builderScope)
(producer as? net.sergeych.lyng.BytecodeCallable)?.callOnFast(builderScope) ?: producer.callOn(builderScope)
} catch (x: ScriptFlowIsNoMoreCollected) {
// premature flow closing, OK
} catch (x: Throwable) {
@@ -268,14 +268,19 @@ class ObjInstant(val instant: Instant,val truncateMode: LynonSettings.InstantTru
returns = type("lyng.DateTime"),
moduleName = "lyng.time"
) {
val tz = when (val a = args.list.getOrNull(0)) {
null -> TimeZone.currentSystemDefault()
is ObjString -> TimeZone.of(a.value)
is ObjInt -> UtcOffset(seconds = a.value.toInt()).asTimeZone()
else -> raiseIllegalArgument("invalid timezone: $a")
}
val tz = parseTimeZoneArg(this, args.list.getOrNull(0), TimeZone.currentSystemDefault())
ObjDateTime(thisAs<ObjInstant>().instant, tz)
}
addFnDoc(
name = "toDate",
doc = "Convert this instant to a calendar Date in the specified time zone. If omitted, the system default time zone is used.",
params = listOf(net.sergeych.lyng.miniast.ParamDoc("tz", type = type("lyng.Any", true))),
returns = type("lyng.Date"),
moduleName = "lyng.time"
) {
val tz = parseTimeZoneArg(this, args.list.getOrNull(0), TimeZone.currentSystemDefault())
ObjDate(toDate(thisAs<ObjInstant>().instant, tz))
}
// class members
@@ -18,6 +18,7 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.Arguments
import net.sergeych.lyng.CallSignature
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.addPropertyDoc
@@ -189,7 +190,13 @@ val ObjIterable by lazy {
doc = "Build a map from elements using the lambda result as key.",
params = listOf(ParamDoc("keySelector")),
returns = type("lyng.Map"),
moduleName = "lyng.stdlib"
moduleName = "lyng.stdlib",
callSignature = CallSignature(
inlineHigherOrder = CallSignature.HigherOrderInline(
kind = CallSignature.Kind.ITERABLE,
result = CallSignature.ResultMode.ASSOCIATE_BY
)
)
) {
val association = requireOnlyArg<Obj>()
val result = ObjMap()
@@ -204,7 +211,13 @@ val ObjIterable by lazy {
doc = "Apply the lambda to each element in iteration order.",
params = listOf(ParamDoc("action")),
isOpen = true,
moduleName = "lyng.stdlib"
moduleName = "lyng.stdlib",
callSignature = CallSignature(
inlineHigherOrder = CallSignature.HigherOrderInline(
kind = CallSignature.Kind.ITERABLE,
result = CallSignature.ResultMode.FOR_EACH
)
)
) {
val scope = requireScope()
val it = thisObj.invokeInstanceMethod(scope, "iterator")
@@ -222,7 +235,13 @@ val ObjIterable by lazy {
params = listOf(ParamDoc("transform")),
returns = type("lyng.List"),
isOpen = true,
moduleName = "lyng.stdlib"
moduleName = "lyng.stdlib",
callSignature = CallSignature(
inlineHigherOrder = CallSignature.HigherOrderInline(
kind = CallSignature.Kind.ITERABLE,
result = CallSignature.ResultMode.MAP
)
)
) {
val fn = requiredArg<Obj>(0)
val result = mutableListOf<Obj>()
@@ -238,7 +257,13 @@ val ObjIterable by lazy {
params = listOf(ParamDoc("transform")),
returns = type("lyng.List"),
isOpen = true,
moduleName = "lyng.stdlib"
moduleName = "lyng.stdlib",
callSignature = CallSignature(
inlineHigherOrder = CallSignature.HigherOrderInline(
kind = CallSignature.Kind.ITERABLE,
result = CallSignature.ResultMode.MAP_NOT_NULL
)
)
) {
val fn = requiredArg<Obj>(0)
val result = mutableListOf<Obj>()
@@ -30,19 +30,27 @@ import net.sergeych.lynon.LynonEncoder
import net.sergeych.lynon.LynonType
open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
private var boxedList: MutableList<Obj>? = null
private var primitiveIntList: LongArray? = null
internal var boxedList: MutableList<Obj>? = null
internal var primitiveIntList: LongArray? = null
// Logical size of primitiveIntList; capacity = primitiveIntList!!.size
internal var primitiveIntSize: Int = 0
init {
if (!adoptPrimitiveIntList(initialList)) {
boxedList = initialList
if (initialList.isNotEmpty()) {
if (!adoptPrimitiveIntList(initialList)) {
boxedList = initialList
}
}
// Empty initialList: both null — lazy mode, avoids boxing on first append
}
val list: MutableList<Obj>
get() = ensureBoxedList()
internal fun sizeFast(): Int = primitiveIntList?.size ?: boxedList?.size ?: 0
internal fun sizeFast(): Int = when {
primitiveIntList != null -> primitiveIntSize
else -> boxedList?.size ?: 0
}
internal fun getObjAtFast(index: Int): Obj =
primitiveIntList?.let { ObjInt.of(it[index]) } ?: boxedList!![index]
@@ -80,9 +88,24 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
internal fun appendFast(value: Obj) {
val ints = primitiveIntList
if (ints != null && value is ObjInt) {
primitiveIntList = ints.copyOf(ints.size + 1).also { it[ints.size] = value.value }
return
if (value is ObjInt) {
if (ints != null) {
// Primitive mode: amortized growth (no copy when capacity allows)
if (primitiveIntSize < ints.size) {
ints[primitiveIntSize++] = value.value
} else {
val grown = ints.copyOf(maxOf(ints.size * 2, 16))
grown[primitiveIntSize++] = value.value
primitiveIntList = grown
}
return
}
if (boxedList == null) {
// Lazy empty state: first int element starts primitive mode
primitiveIntList = LongArray(16).also { it[0] = value.value }
primitiveIntSize = 1
return
}
}
ensureBoxedList().add(value)
}
@@ -91,10 +114,12 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
val ints = primitiveIntList
val otherInts = other.primitiveIntList
if (ints != null && otherInts != null) {
primitiveIntList = LongArray(ints.size + otherInts.size).also {
ints.copyInto(it, 0, 0, ints.size)
otherInts.copyInto(it, ints.size, 0, otherInts.size)
}
val otherSize = other.primitiveIntSize
val newSize = primitiveIntSize + otherSize
val dest = if (newSize <= ints.size) ints else ints.copyOf(newSize)
otherInts.copyInto(dest, primitiveIntSize, 0, otherSize)
primitiveIntList = dest
primitiveIntSize = newSize
return
}
ensureBoxedList().addAll(other.list)
@@ -108,6 +133,7 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
ints[i] = value.value
}
primitiveIntList = ints
primitiveIntSize = ints.size
boxedList = null
return true
}
@@ -120,12 +146,13 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
boxedList = empty
return empty
}
val materialized = ArrayList<Obj>(ints.size)
for (value in ints) {
materialized.add(ObjInt.of(value))
val materialized = ArrayList<Obj>(primitiveIntSize)
for (i in 0..<primitiveIntSize) {
materialized.add(ObjInt.of(ints[i]))
}
boxedList = materialized
primitiveIntList = null
primitiveIntSize = 0
return materialized
}
@@ -138,8 +165,9 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
}
}
internal constructor(intValues: LongArray) : this(mutableListOf()) {
internal constructor(intValues: LongArray, size: Int = intValues.size) : this(mutableListOf()) {
primitiveIntList = intValues
primitiveIntSize = size
boxedList = null
}
@@ -253,9 +281,11 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
val ints = primitiveIntList
val otherInts = other.primitiveIntList
if (ints != null && otherInts != null) {
ObjList(LongArray(ints.size + otherInts.size).also {
ints.copyInto(it, 0, 0, ints.size)
otherInts.copyInto(it, ints.size, 0, otherInts.size)
val mySize = primitiveIntSize
val otherSize = other.primitiveIntSize
ObjList(LongArray(mySize + otherSize).also {
ints.copyInto(it, 0, 0, mySize)
otherInts.copyInto(it, mySize, 0, otherSize)
})
} else {
ObjList((list + other.list).toMutableList())
@@ -276,7 +306,9 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
open override suspend fun plusAssign(scope: Scope, other: Obj): Obj {
if (other is ObjList) {
if (other is ObjInt || other is ObjString || other is ObjBool || other is ObjReal || other is ObjNull) {
appendFast(other)
} else if (other is ObjList) {
appendAllFast(other)
} else if (!shouldTreatAsSingleElement(scope, other) && other.isInstanceOf(ObjIterable)) {
val otherList = (other.invokeInstanceMethod(scope, "toList") as ObjList).list
@@ -327,8 +359,8 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
override suspend fun contains(scope: Scope, other: Obj): Boolean {
val ints = primitiveIntList
if (ints != null && other is ObjInt) {
for (value in ints) {
if (value == other.value) return true
for (i in 0..<primitiveIntSize) {
if (ints[i] == other.value) return true
}
return false
}
@@ -351,8 +383,8 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
override suspend fun enumerate(scope: Scope, callback: suspend (Obj) -> Boolean) {
val ints = primitiveIntList
if (ints != null) {
for (value in ints) {
if (!callback(ObjInt.of(value))) break
for (i in 0..<primitiveIntSize) {
if (!callback(ObjInt.of(ints[i]))) break
}
return
}
@@ -366,7 +398,7 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
override suspend fun toKotlin(scope: Scope): Any {
val ints = primitiveIntList
if (ints != null) return ints.map { it }
if (ints != null) return (0..<primitiveIntSize).map { ints[it] }
return list.map { it.toKotlin(scope) }
}
@@ -398,19 +430,24 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
}
override fun hashCode(): Int {
return primitiveIntList?.contentHashCode() ?: list.hashCode()
val ints = primitiveIntList
return if (ints != null) {
var result = 1
for (i in 0..<primitiveIntSize) result = 31 * result + ints[i].hashCode()
result
} else list.hashCode()
}
override fun equals(other: Any?): Boolean {
if (this === other) return true
if (other == null || this::class != other::class) return false
other as ObjList
val ints = primitiveIntList
val otherInts = other.primitiveIntList
return if (ints != null && otherInts != null) {
ints.contentEquals(otherInts)
if (primitiveIntSize != other.primitiveIntSize) return false
for (i in 0..<primitiveIntSize) if (ints[i] != otherInts[i]) return false
true
} else {
list == other.list
}
@@ -419,7 +456,9 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
override suspend fun serialize(scope: Scope, encoder: LynonEncoder, lynonType: LynonType?) {
val ints = primitiveIntList
if (ints != null) {
encoder.encodeAnyList(scope, ints.mapTo(ArrayList(ints.size)) { ObjInt.of(it) })
val boxed = ArrayList<Obj>(primitiveIntSize)
for (i in 0..<primitiveIntSize) boxed.add(ObjInt.of(ints[i]))
encoder.encodeAnyList(scope, boxed)
return
}
encoder.encodeAnyList(scope, list)
@@ -430,7 +469,7 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
override suspend fun toJson(scope: Scope): JsonElement {
val ints = primitiveIntList
if (ints != null) {
return JsonArray(ints.map { ObjInt.of(it).toJson(scope) })
return JsonArray((0..<primitiveIntSize).map { ObjInt.of(ints[it]).toJson(scope) })
}
return JsonArray(list.map { it.toJson(scope) })
}
@@ -441,9 +480,9 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
var first = true
val ints = primitiveIntList
if (ints != null) {
for (v in ints) {
for (i in 0..<primitiveIntSize) {
if (first) first = false else append(",")
append(v)
append(ints[i])
}
} else {
for (v in list) {
@@ -480,8 +519,8 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
doc = "Append one or more elements to the end of this list.",
moduleName = "lyng.stdlib"
) {
val l = thisAs<ObjList>().list
for (a in args) l.add(a)
val l = thisAs<ObjList>()
for (a in args) l.appendFast(a)
ObjVoid
}
addFnDoc(
@@ -528,6 +567,28 @@ open class ObjList(initialList: MutableList<Obj> = mutableListOf()) : Obj() {
} else self.list.removeLast()
self
}
addFnDoc(
name= "ensureCapacity",
doc = """
ensure the list capacity allows storing specified amount if items without reallocation.
If current capacity is greater or equal to `count`, does nothing. Note that possible reallocation
could be a costly operation,
""".trimIndent(),
params = listOf(ParamDoc("count", type("lyng.Int"))),
moduleName = "lyng.stdlib"
) {
val self = thisAs<ObjList>()
val count = requireOnlyArg<ObjInt>().value.toInt()
if (count > 0) {
val ints = self.primitiveIntList
when {
ints != null -> if (ints.size < count) self.primitiveIntList = ints.copyOf(count)
self.boxedList == null -> { self.primitiveIntList = LongArray(count); self.primitiveIntSize = 0 }
else -> (self.boxedList as? ArrayList)?.ensureCapacity(count)
}
}
self
}
addFnDoc(
name = "removeRange",
@@ -19,6 +19,7 @@ package net.sergeych.lyng.obj
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import net.sergeych.lyng.CallSignature
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.*
import net.sergeych.lynon.LynonDecoder
@@ -262,7 +263,15 @@ class ObjMap(val map: MutableMap<Obj, Obj> = mutableMapOf()) : Obj() {
doc = "Get value by key or compute, store, and return the default from a lambda.",
params = listOf(ParamDoc("key"), ParamDoc("default")),
returns = type("lyng.Any"),
moduleName = "lyng.stdlib"
moduleName = "lyng.stdlib",
callSignature = CallSignature(
inlineHigherOrder = CallSignature.HigherOrderInline(
kind = CallSignature.Kind.MAP_GET_OR_PUT,
result = CallSignature.ResultMode.BLOCK_RESULT,
argCount = 2,
lambdaArgIndex = 1
)
)
) {
val key = requiredArg<Obj>(0)
thisAs<ObjMap>().map.getOrPut(key) {
@@ -53,10 +53,10 @@ object ObjMatrixSupport {
}
hooks += { _, instance -> instance.kotlinInstanceData = defaultVector }
vectorClass.addProperty("size", getter = {
vectorClass.bindProperty("size", getter = {
ObjInt.of(vectorOf(thisObj).size.toLong())
})
vectorClass.addProperty("length", getter = {
vectorClass.bindProperty("length", getter = {
ObjInt.of(vectorOf(thisObj).size.toLong())
})
vectorClass.addFn("toList") {
@@ -99,10 +99,10 @@ object ObjMatrixSupport {
ObjInt.of(vectorOf(thisObj).compareTo(coerceVectorArg(requireScope(), args.firstAndOnly())).toLong())
}
vectorClass.addClassFn("fromList") {
vectorClass.bindClassFn("fromList") {
newVector(vectorClass, parseVector(requireScope(), requiredArg(0)))
}
vectorClass.addClassFn("zeros") {
vectorClass.bindClassFn("zeros") {
val size = requiredArg<ObjInt>(0).value.toInt()
if (size <= 0) requireScope().raiseIllegalArgument("vector size must be positive")
newVector(vectorClass, VectorData(DoubleArray(size)))
@@ -119,13 +119,13 @@ object ObjMatrixSupport {
}
hooks += { _, instance -> instance.kotlinInstanceData = defaultMatrix }
matrixClass.addProperty("rows", getter = {
matrixClass.bindProperty("rows", getter = {
ObjInt.of(matrixOf(thisObj).rows.toLong())
})
matrixClass.addProperty("cols", getter = {
matrixClass.bindProperty("cols", getter = {
ObjInt.of(matrixOf(thisObj).cols.toLong())
})
matrixClass.addProperty("shape", getter = {
matrixClass.bindProperty("shape", getter = {
ObjList(
mutableListOf(
ObjInt.of(matrixOf(thisObj).rows.toLong()),
@@ -133,7 +133,7 @@ object ObjMatrixSupport {
)
)
})
matrixClass.addProperty("isSquare", getter = {
matrixClass.bindProperty("isSquare", getter = {
matrixOf(thisObj).isSquare.toObj()
})
@@ -208,17 +208,17 @@ object ObjMatrixSupport {
ObjInt.of(matrixOf(thisObj).compareTo(coerceMatrixArg(requireScope(), args.firstAndOnly())).toLong())
}
matrixClass.addClassFn("fromRows") {
matrixClass.bindClassFn("fromRows") {
newMatrix(matrixClass, parseRows(requireScope(), requiredArg(0)))
}
matrixClass.addClassFn("zeros") {
matrixClass.bindClassFn("zeros") {
val rows = requiredArg<ObjInt>(0).value.toInt()
val cols = requiredArg<ObjInt>(1).value.toInt()
if (rows <= 0) requireScope().raiseIllegalArgument("matrix must have at least one row")
if (cols <= 0) requireScope().raiseIllegalArgument("matrix must have at least one column")
newMatrix(matrixClass, MatrixData(rows, cols, DoubleArray(rows * cols)))
}
matrixClass.addClassFn("identity") {
matrixClass.bindClassFn("identity") {
val size = requiredArg<ObjInt>(0).value.toInt()
if (size <= 0) requireScope().raiseIllegalArgument("identity matrix size must be positive")
val values = DoubleArray(size * size)
@@ -18,6 +18,7 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.Arguments
import net.sergeych.lyng.BytecodeCallable
import net.sergeych.lyng.BytecodeBodyProvider
import net.sergeych.lyng.Scope
import net.sergeych.lyng.Statement
@@ -41,14 +42,16 @@ class ObjProperty(
val instanceScope = (instance as? ObjInstance)?.instanceScope ?: instance.autoInstanceScope(scope)
val execScope = scope.applyClosure(instanceScope).createChildScope(newThisObj = instance)
execScope.currentClassCtx = declaringClass
(g as? BytecodeCallable)?.callOnFast(execScope)?.let { return it }
return when (g) {
is BytecodeStatement -> executeBytecodeWithSeed(execScope, g, "property getter")
is BytecodeBodyProvider -> {
val body = g.bytecodeBody()
if (body != null) executeBytecodeWithSeed(execScope, body, "property getter") else g.callOn(execScope)
if (body != null) executeBytecodeWithSeed(execScope, body, "property getter")
else (g as? BytecodeCallable)?.callOnFast(execScope) ?: g.callOn(execScope)
}
is Statement -> g.callOn(execScope)
else -> g.callOn(execScope)
is Statement -> (g as? BytecodeCallable)?.callOnFast(execScope) ?: g.callOn(execScope)
else -> (g as? BytecodeCallable)?.callOnFast(execScope) ?: g.callOn(execScope)
}
}
@@ -59,14 +62,16 @@ class ObjProperty(
val instanceScope = (instance as? ObjInstance)?.instanceScope ?: instance.autoInstanceScope(scope)
val execScope = scope.applyClosure(instanceScope).createChildScope(args = Arguments(value), newThisObj = instance)
execScope.currentClassCtx = declaringClass
(s as? BytecodeCallable)?.callOnFast(execScope)?.let { return }
when (s) {
is BytecodeStatement -> executeBytecodeWithSeed(execScope, s, "property setter")
is BytecodeBodyProvider -> {
val body = s.bytecodeBody()
if (body != null) executeBytecodeWithSeed(execScope, body, "property setter") else s.callOn(execScope)
if (body != null) executeBytecodeWithSeed(execScope, body, "property setter")
else (s as? BytecodeCallable)?.callOnFast(execScope) ?: s.callOn(execScope)
}
is Statement -> s.callOn(execScope)
else -> s.callOn(execScope)
is Statement -> (s as? BytecodeCallable)?.callOnFast(execScope) ?: s.callOn(execScope)
else -> (s as? BytecodeCallable)?.callOnFast(execScope) ?: s.callOn(execScope)
}
}
@@ -38,17 +38,26 @@ class ImportManager(
val packageNames: List<String> get() = imports.keys.toList()
private class CacheCell(var scope: ModuleScope? = null)
private inner class Entry(
val packageName: String,
val builder: suspend (ModuleScope) -> Unit,
var cachedScope: ModuleScope? = null
val cacheCell: CacheCell = CacheCell()
) {
suspend fun getScope(pos: Pos): ModuleScope {
cachedScope?.let { return it }
return ModuleScope(inner, pos, packageName).apply {
cachedScope = this
builder(this)
cacheCell.scope?.let { return it }
val module = ModuleScope(inner, pos, packageName)
cacheCell.scope = module
return try {
builder(module)
module
} catch (e: Throwable) {
if (cacheCell.scope === module) {
cacheCell.scope = null
}
throw e
}
}
}
@@ -152,14 +161,14 @@ class ImportManager(
op.withLock {
ImportManager(rootScope, securityManager).apply {
for ((name, entry) in this@ImportManager.imports) {
imports[name] = Entry(entry.packageName, entry.builder, entry.cachedScope)
imports[name] = Entry(entry.packageName, entry.builder, entry.cacheCell)
}
}
}
fun invalidatePackageCache(name: String) {
op.withLock {
imports[name]?.cachedScope = null
imports[name]?.cacheCell?.scope = null
}
}
@@ -58,7 +58,10 @@ abstract class Statement(
val type = ObjClass("Callable")
}
suspend fun call(scope: Scope, vararg args: Obj) = execute(scope.createChildScope(args = Arguments(*args)))
suspend fun call(scope: Scope, vararg args: Obj): Obj {
val child = scope.createChildScope(args = Arguments(*args))
return (this as? BytecodeCallable)?.callOnFast(child) ?: execute(child)
}
protected fun bytecodeOnly(scope: Scope, label: String): Nothing {
return scope.raiseIllegalState("bytecode-only execution is required; $label needs compiled bytecode")
@@ -201,14 +201,7 @@ extern class Decimal() {
*
* Contexts are dynamic and block-local. After the block finishes, the previous context is restored.
*/
extern fun withDecimalContext<T>(context: DecimalContext, block: ()->T): T
/**
* Convenience overload for changing only precision.
*
* Equivalent to `withDecimalContext(DecimalContext(precision, DecimalRounding.HalfEven), block)`.
*/
extern fun withDecimalContext<T>(precision: Int, block: ()->T): T
extern fun withDecimalContext<T>(context: Object, block: ()->T): T
/**
* Convenience overload for changing precision and rounding explicitly.
Loaded 100 of 125 files, more files were not shown because too many files have changed in this diff. Show more