Compare commits

...
104 Commits
Author SHA1 Message Date
sergeych 9735774efd v1.5.5 release 2026-04-23 15:18:47 +03:00
sergeych 14214e91e1 Fix escaping immutable closure captures 2026-04-22 18:50:49 +03:00
sergeychandClaude Sonnet 4.6 0f5343fa17 Add VPS deployment target and Ansible provisioning for lynglang.com
- deploy_site now defaults to new VPS (94.130.36.94, /var/www/lynglang);
  use --old to deploy to d.lynglang.com as before
- infra/setup_vps.yml: Ansible playbook installs nginx + certbot, obtains
  SSL cert for lynglang.com, fixes Debian buster EOL apt sources
- infra/inventory.ini and infra/templates/nginx_lynglang.conf.j2 included

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 12:31:01 +03:00
sergeych d0f51928de fixed wrong error message in val ++/-- 2026-04-22 08:56:02 +03:00
sergeych 0973a6afeb Fix JVM import caching and class/object bytecode dispatch 2026-04-21 23:25:07 +03:00
sergeych 953f237ca3 Optimize primitive list fill capacity and append 2026-04-21 21:37:04 +03:00
sergeych fbb5688696 Drive higher-order lambda inlining from call metadata 2026-04-21 20:51:16 +03:00
sergeych f4ab2ebab4 Extend lambda inlining to getOrPut and implicit it calls 2026-04-21 19:20:25 +03:00
sergeych 0c3242cbd8 Generalize higher-order lambda inlining 2026-04-21 19:12:16 +03:00
sergeych 1d5caaa836 Broaden lambda method inlining with captures 2026-04-21 19:07:05 +03:00
sergeych 3be2892025 Use fast compiled callbacks in dynamic and flow helpers 2026-04-21 18:06:31 +03:00
sergeych c80900c503 Prevent cross-script module slot aliasing 2026-04-21 17:34:00 +03:00
sergeych ee634c8dff Extend generic fast callable entry points 2026-04-21 17:26:53 +03:00
sergeych 6c91b77a85 Use fast lambda calls in scope facade 2026-04-21 16:43:02 +03:00
sergeych 3721ee8332 Fast-path compiled lambda calls with args 2026-04-21 16:04:04 +03:00
sergeych 97990f00ce Cover when-based exact callable fast paths 2026-04-21 14:37:27 +03:00
sergeych 80693e7690 Inline exact lambdas through wrapper expressions 2026-04-21 14:36:18 +03:00
sergeych 029fe874fa Propagate exact callable refs across expressions 2026-04-21 14:33:23 +03:00
sergeych ffa64d691b Use direct calls for constant constructors 2026-04-21 13:30:09 +03:00
sergeych 30c9a5a565 Propagate exact lambda refs across bytecode slots 2026-04-21 13:20:27 +03:00
sergeych db3a780645 Generalize bytecode fast-call dispatch 2026-04-21 13:09:50 +03:00
sergeych fc7d26ee4b Fast-path bytecode statement execution 2026-04-21 11:39:47 +03:00
sergeych a61b5a31be Add non-suspending fast calls for bytecode callables 2026-04-21 11:35:20 +03:00
sergeych 33d170f525 Optimize fast-path bytecode lambda invocation 2026-04-21 11:29:14 +03:00
sergeych f72cdfdf83 Inline simple lambdas in bytecode fast paths 2026-04-21 09:05:18 +03:00
sergeych 10fa4de4fa regression fix 2026-04-20 18:48:28 +03:00
sergeych 4acee56de8 small optimization of List.fill 2026-04-20 14:09:09 +03:00
sergeych 87060c7eb7 +inference bugs
+List.ensureCapacity
optimized List.fill
2026-04-20 13:56:47 +03:00
sergeych 6ba128f7ba fixed some more inference bugs 2026-04-20 12:23:17 +03:00
sergeych a8f73dc8bd added LaunchPool 2026-04-18 23:37:21 +03:00
sergeychandClaude Sonnet 4.6 145c3ae34a Add completeExceptionally, cancelAndJoin, and refine LaunchPool spec
- CompletableDeferred.completeExceptionally(exception): fails the deferred
  with a Lyng exception; the original exception object is preserved as
  errorObject so assertThrows/catch see the correct class (works for both
  built-in ObjException and user-declared class instances)
- LaunchPool: add maxQueueSize parameter (suspends launch when full),
  add cancelAndJoin() for graceful shutdown that waits for running tasks,
  clarify all semantics in the extern declaration comments
- Two tests for completeExceptionally (built-in and user-defined exception)
- Fix parallelism.md channel doc-test to be illustration-only (avoid
  flakiness from coroutine scheduling in the doc-test runner)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 03:00:40 +03:00
sergeychandClaude Sonnet 4.6 f9a07f176a Add Channel primitive for coroutine inter-task communication
Exposes Kotlin's Channel<Obj> to Lyng scripts as a first-class type
with rendezvous, buffered, and unlimited capacity modes. Supports
suspending send/receive, non-suspending tryReceive, close/drain
semantics, and isClosedForSend/isClosedForReceive properties.

Also fixes a pre-existing typo in BookTest.kt that blocked JVM test
compilation, and adds Channel reference docs and a parallelism.md section.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 02:28:55 +03:00
sergeych 30e56946a0 improved JDBC provider for lyng.io.db 2026-04-18 02:06:29 +03:00
sergeych 49fc700233 Detach materialized SQL rows from transaction lifetime 2026-04-17 20:38:40 +03:00
sergeych 07cb5a519c Add JDBC and websocket/native follow-up changes 2026-04-16 23:35:15 +03:00
sergeych 8f66cd7680 Fix lynglib binding regressions and green test suite 2026-04-16 23:34:45 +03:00
sergeych bb2119b1d1 ignores 2026-04-16 22:12:19 +03:00
sergeych 64273ac60a Add JDBC database provider for JVM 2026-04-16 22:10:36 +03:00
sergeych ee392daa13 another sample 2026-04-16 13:35:12 +03:00
sergeych 0affa92674 better SQL example 2026-04-16 01:08:31 +03:00
sergeych a2e3f80ab6 better sql exaple, some inference bugs fixed 2026-04-16 00:54:00 +03:00
sergeych f8d2533b48 Make SQLite native linker lookup portable 2026-04-15 23:47:38 +03:00
sergeych bb9af2258b Fix SQLite release deployment 2026-04-15 23:35:59 +03:00
sergeych b3be908242 Add Date type and database release docs 2026-04-15 23:20:23 +03:00
sergeych 14dc73db3e Harden SQLite provider release behavior 2026-04-15 23:11:50 +03:00
sergeych f9bbdd56bf Test DB rollback precedence in SQLite 2026-04-15 22:38:11 +03:00
sergeych 04e80c384e Stabilize DB provider registration test 2026-04-15 22:35:19 +03:00
sergeych 6d340824e4 Tighten SQLite transaction failures and tests 2026-04-15 22:20:29 +03:00
sergeych 5f6f6b9ae4 Align SQLite value conversion across JVM and native 2026-04-15 22:03:42 +03:00
sergeych 55ba6113e7 Add Lyng DB contract and SQLite provider skeleton 2026-04-15 20:39:38 +03:00
sergeych b42ceec686 Preserve VM throw sites in stack traces 2026-04-14 01:17:04 +03:00
sergeych fada848907 Add backtick string literals and formatter support 2026-04-13 23:08:18 +03:00
sergeych e925195495 yet another import fix 2026-04-13 22:49:02 +03:00
sergeych 3b6bdda0a4 another import fix 2026-04-13 20:43:11 +03:00
sergeych ab39110834 Add repeated local import regression coverage 2026-04-12 11:11:32 +03:00
sergeych 47eb2d7d61 Fix transitive import re-exports and stabilize captures 2026-04-12 09:16:58 +03:00
sergeych eefa4c8f1a Fix stdlib import aliasing and launch captures 2026-04-11 20:24:44 +03:00
sergeych 6171b0d95f Fix duplicate stdlib import redefinition 2026-04-11 19:24:08 +03:00
sergeych a1ea09440d Optimize native CLI release binary 2026-04-09 22:30:43 +03:00
sergeych 88ce04102a Add CLI atExit shutdown handlers 2026-04-09 22:17:45 +03:00
sergeych 0e73d80707 plugin fix 2026-04-09 19:37:23 +03:00
sergeych 121d860043 Document native networking limits 2026-04-09 19:17:32 +03:00
sergeych 717758149b cleaned up tcpserver.lyng, again ;) 2026-04-09 19:01:26 +03:00
sergeych 73ec8f9be6 cleaned up tcpserver.lyng/2 2026-04-09 18:28:59 +03:00
sergeych 805c54a408 cleaned up tcpserver.lyng 2026-04-09 18:26:25 +03:00
sergeych 9b90fe370b improved tcpserver.lyng sample 2026-04-09 17:11:30 +03:00
sergeych 840cd32574 Fix captured range bounds in launched for-loops 2026-04-09 17:05:00 +03:00
sergeych fbb1683ba3 Fix concurrent TCP CLI scope and add regressions 2026-04-09 11:55:01 +03:00
sergeych c9eb3df93d Add Deferred.joinAll helper 2026-04-09 11:27:32 +03:00
sergeych 402b8bb1b3 Fix native CLI TCP regression path 2026-04-09 11:00:23 +03:00
sergeych aa1b74620e docs for cli fix; KMP compatibility bug fix 2026-04-08 22:54:22 +03:00
sergeych ef95ed4405 package structure/import for CLI 2026-04-08 22:38:34 +03:00
sergeych b6c6ef021a fixed some exception catching problems on wasm target 2026-04-08 22:05:02 +03:00
sergeych c6cfd52b01 fixed site docs behavior on narrow screens 2026-04-08 10:53:21 +03:00
sergeych 12fb4fe0ba + String.replace 2026-04-08 10:45:01 +03:00
sergeych 368ce2ce8c added buffer.base64std 2026-04-08 09:27:52 +03:00
sergeychandClaude Sonnet 4.6 d0d79d2f07 docs: add LegacyDigest module documentation
- New docs/LegacyDigest.md: full reference for lyng.legacy_digest,
  covering the sha1() API, input types (String / Buffer), FIPS compliance
  note, and explicit guidance on appropriate vs. inappropriate use.

- docs/ai_stdlib_reference.md: entry in section 5 so AI agents know
  LegacyDigest.sha1() exists and is intentionally named as legacy-only.

- docs/whats_new.md: release-note section alongside Complex, Decimal,
  and Matrix, with a minimal runnable example and cross-link.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 19:53:01 +03:00
sergeychandClaude Sonnet 4.6 69013392d3 Add lyng.legacy_digest module with LegacyDigest.sha1()
Provides a pure Kotlin/KMP SHA-1 implementation with no extra dependencies,
exposed as `LegacyDigest.sha1(data)` in the `lyng.legacy_digest` package.

The naming deliberately signals that SHA-1 is cryptographically broken:
the object name `LegacyDigest` and prominent doc-comment warnings steer
users away from security-sensitive use while still enabling interoperability
with legacy protocols and file formats that require SHA-1.

API:
  import lyng.legacy_digest
  val hex: String = LegacyDigest.sha1("some string")   // UTF-8 input
  val hex: String = LegacyDigest.sha1(someBuffer)       // raw-byte input

Tests cover FIPS 180-4 known-answer vectors (empty, "abc", long message,
quick-brown-fox) at both the Kotlin and Lyng integration levels.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 19:51:04 +03:00
sergeychandClaude Sonnet 4.6 f145a90845 Fix method slot ID collision between instance and static methods
When createField() was called with a pre-assigned methodId, the ID was
used but methodIdMap was not updated and nextMethodId was not advanced.
This caused assignMethodId() for static methods to reuse slot IDs
already occupied by instance methods.

In complex.lyng, fromInt/imaginary got IDs 12/14 (same as plus/mul),
causing binary operator dispatch via CALL_MEMBER_SLOT to call the wrong
function (e.g. a*b would invoke imaginary instead of mul).

Fix: after computing effectiveMethodId in createField, always register
it in methodIdMap and advance nextMethodId past it so subsequent
auto-assignments start from a clean range.

Also pre-assigns method IDs for non-static fun/fn declarations during
class body pre-scan so forward references resolve correctly in class
bodies, and adds ComplexModuleTest coverage for operator slot dispatch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 19:41:18 +03:00
sergeych 2f145a0ea7 Fix nullable let member inference 2026-04-07 09:33:40 +03:00
sergeych 15617f6998 more docs 2026-04-07 01:01:26 +03:00
sergeych 583067780f Support object extension indexers 2026-04-07 00:49:51 +03:00
sergeych 8386337c42 missing docs on object extension 2026-04-07 00:05:34 +03:00
sergeych 83099148bd index return value inference fixed 2026-04-06 20:03:55 +03:00
sergeych 1fca2b112f Fix parsing of generic extension functions 2026-04-06 19:29:06 +03:00
sergeych 35628c8453 Support extensions on singleton objects 2026-04-06 19:24:28 +03:00
sergeych 671583638b v1.5.5-SNAPSHOT started. compile if support 2026-04-06 11:09:14 +03:00
sergeych 214f1aec9e v1.5.4 release 2026-04-05 06:34:30 +03:00
sergeych 657c0c5d18 optimize VM: removed cmd.isFast 2026-04-05 06:12:49 +03:00
sergeych 90718c3c17 optimize VM 2026-04-04 10:35:08 +03:00
sergeych d8454a11fc optimize arithmetics 2026-04-04 04:01:43 +03:00
sergeych 161f3f74e2 fixed disabled concurrency in CLI 2026-04-03 23:42:43 +03:00
sergeych 1cead7822a Prepare 1.5.4 release defaults and metadata 2026-04-03 23:28:09 +03:00
sergeych 064b927b1a Stabilize pi benchmark optimizations for release 2026-04-03 23:16:08 +03:00
sergeych a7ab0d3905 added static List.fill(size) { ... } 2026-04-03 21:15:41 +03:00
sergeych f1003f5b95 Add descending ranges and for-loop support 2026-04-03 20:58:08 +03:00
sergeych d8c53c500e Package JVM distribution as a zip, update symlink creation, and document changes in CLI and downloads. 2026-04-03 20:36:30 +03:00
sergeych b3efe019d9 Fix delay function to correctly interpret Int as milliseconds in scripts 2026-04-03 18:52:38 +03:00
sergeych 2414da59a7 fixed terminal problems in lyng CLI with new console support on linux 2026-04-03 12:12:55 +03:00
sergeych fd6d05d568 anotther compiler inference bug andlyng.io.console improvements 2026-04-03 11:55:25 +03:00
sergeych caad7d8ab9 stdlib inference bug fixed 2026-04-03 11:04:16 +03:00
sergeych 88b0bb2147 Add TCP server example and refactor script execution flow 2026-04-02 22:55:29 +03:00
sergeych 38646c6257 Update platform support matrix with detailed statuses and legend 2026-04-02 20:41:54 +03:00
sergeych f61ac35580 added networking to the CLI 2026-04-02 20:27:13 +03:00
232 changed files with 23385 additions and 1773 deletions

No files matched your search

+3
View File
@@ -28,3 +28,6 @@ debug.log
/compile_metadata_output.txt /compile_metadata_output.txt
test_output*.txt test_output*.txt
/site/src/version-template/lyng-version.js /site/src/version-template/lyng-version.js
/bugcontents.db
/bugs/
contents.db
+1
View File
@@ -18,6 +18,7 @@
- Avoid creating suspend lambdas for compiler runtime statements. Prefer explicit `object : Statement()` with `override suspend fun execute(...)`. - Avoid creating suspend lambdas for compiler runtime statements. Prefer explicit `object : Statement()` with `override suspend fun execute(...)`.
- Do not use `statement { ... }` or other inline suspend lambdas in compiler hot paths (e.g., parsing/var declarations, initializer thunks). - Do not use `statement { ... }` or other inline suspend lambdas in compiler hot paths (e.g., parsing/var declarations, initializer thunks).
- If you need a wrapper for delegated properties, check for `getValue` explicitly and return a concrete `Statement` object when missing; avoid `onNotFoundResult` lambdas. - If you need a wrapper for delegated properties, check for `getValue` explicitly and return a concrete `Statement` object when missing; avoid `onNotFoundResult` lambdas.
- For any code in `commonMain`, verify it is Kotlin Multiplatform compatible before finishing. Do not use JVM-only APIs or Java-backed convenience methods such as `Map.putIfAbsent`; prefer stdlib/common equivalents and run at least the relevant compile/test task that exercises the `commonMain` source set.
- If wasmJs browser tests hang, first run `:lynglib:wasmJsNodeTest` and look for wasm compilation errors; hangs usually mean module instantiation failed. - If wasmJs browser tests hang, first run `:lynglib:wasmJsNodeTest` and look for wasm compilation errors; hangs usually mean module instantiation failed.
- Do not increase test timeouts to mask wasm generation errors; fix the invalid IR instead. - Do not increase test timeouts to mask wasm generation errors; fix the invalid IR instead.
+64
View File
@@ -7,6 +7,70 @@ History note:
- Entries below are synchronized and curated for `1.5.x`. - Entries below are synchronized and curated for `1.5.x`.
- Earlier history may be incomplete and should be cross-checked with git tags/commits when needed. - Earlier history may be incomplete and should be cross-checked with git tags/commits when needed.
## Unreleased
- No unreleased entries yet.
## 1.5.5 (2026-04-23)
### Concurrency and collections
- Added coroutine coordination primitives and helpers for everyday parallel code:
- `Channel` for coroutine-to-coroutine communication
- `LaunchPool` for bounded-concurrency task execution
- `Iterable<Deferred>.joinAll()` to await a whole collection of deferreds in input order
- `CompletableDeferred.completeExceptionally(...)` and `Deferred.cancelAndJoin()`
- Added docs and examples for the new concurrency APIs, including `joinAll()` coverage in iterable and parallelism references.
### Database and time APIs
- Added the portable `lyng.io.db` SQL contract and the first concrete providers:
- `lyng.io.db.sqlite` on JVM and Linux Native
- `lyng.io.db.jdbc` on JVM
- Added SQLite/JDBC release hardening:
- nested transactions via savepoints
- detached materialized rows
- generated-key support through `ExecutionResult.getGeneratedKeys()`
- schema-driven value conversion for `Bool`, `Decimal`, `Date`, `DateTime`, and `Instant`
- portable SQLite linker/deployment fixes and documented runtime options
- Added `Date` to `lyng.time` and the core runtime as a first-class calendar-date type, plus conversions and arithmetic across `Instant`, `DateTime`, and `Date`.
### Language, stdlib, and tooling
- Added extensions on singleton `object` declarations, including object-scoped indexer overrides for bracket syntax.
- Added backtick string literals and formatter support.
- Added `lyng.legacy_digest` for SHA-1 compatibility work, `String.replace`, and `buffer.base64std`.
- Improved CLI/runtime behavior with `atExit` shutdown handlers, native release-binary work, and follow-up CLI packaging/import fixes.
- Expanded docs across the tutorial, stdlib references, database docs, networking docs, and release notes.
### Runtime/compiler stability and performance
- Extended exact-call and higher-order lambda inlining through the bytecode compiler, including compiled fast paths for simple lambdas, wrappers, captures, and common higher-order helpers.
- Fixed import caching and class/object bytecode dispatch on JVM.
- Fixed immutable `val` compound assignments so true mutating `*Assign` operations continue to work while fallback reassignments report the correct read-only error.
- Fixed closure/capture and import regressions across launched loops, singleton/object extensions, aliasing, transitive re-exports, and immutable capture escaping.
- Improved list-fill/list-append fast paths, nullable-let inference, Decimal/Complex interop, and related regression coverage.
### Release notes
- Release metadata, homepage samples, docs, and README now point to `1.5.5`.
## 1.5.4 (2026-04-03)
### Runtime and compiler stability
- Stabilized the recent `piSpigot` benchmark/compiler work for release.
- Fixed numeric-mix regressions introduced by overly broad int-coercion in bytecode compilation.
- Restored correct behavior for decimal arithmetic, mixed real/int flows, list literals, list size checks, and national-character script cases.
- Fixed plain-list index fast paths so they no longer bypass subclass behavior such as `ObservableList` hooks and flow notifications.
- Hardened local numeric compare fast paths to correctly handle primitive-coded frame slots.
### Performance and examples
- Added `piSpigot` benchmark/example coverage:
- `examples/pi-test.lyng`
- `examples/pi-bench.lyng`
- JVM benchmark test for release-baseline verification
- Kept the safe list/index/runtime wins that improve the optimized `piSpigot` path without reintroducing type-unsound coercions.
- Changed the default `RVAL_FASTPATH` setting off on JVM/Android and in the benchmark preset after verification that it no longer helps the stabilized `piSpigot` workload.
### Release notes
- Full JVM and wasm test gates pass on the release tree.
- Benchmark findings and remaining post-release optimization targets are documented in `notes/pi_spigot_benchmark_baseline_2026-04-03.md`.
## 1.5.1 (2026-03-25) ## 1.5.1 (2026-03-25)
### Language ### Language
+6 -5
View File
@@ -48,9 +48,12 @@ assertEquals(A.E.One, A.One)
- [Language home](https://lynglang.com) - [Language home](https://lynglang.com)
- [introduction and tutorial](docs/tutorial.md) - start here please - [introduction and tutorial](docs/tutorial.md) - start here please
- [Latest release notes (1.5.5)](docs/whats_new.md)
- [What's New in 1.5](docs/whats_new_1_5.md) - [What's New in 1.5](docs/whats_new_1_5.md)
- [Testing and Assertions](docs/Testing.md) - [Testing and Assertions](docs/Testing.md)
- [Filesystem and Processes (lyngio)](docs/lyngio.md) - [Filesystem and Processes (lyngio)](docs/lyngio.md)
- [SQL Databases (lyng.io.db)](docs/lyng.io.db.md)
- [Time and Calendar Types](docs/time.md)
- [Return Statement](docs/return_statement.md) - [Return Statement](docs/return_statement.md)
- [Efficient Iterables in Kotlin Interop](docs/EfficientIterables.md) - [Efficient Iterables in Kotlin Interop](docs/EfficientIterables.md)
- [Samples directory](docs/samples) - [Samples directory](docs/samples)
@@ -63,8 +66,7 @@ assertEquals(A.E.One, A.One)
### Add dependency to your project ### Add dependency to your project
```kotlin ```kotlin
// update to current please: val lyngVersion = "1.5.5"
val lyngVersion = "1.5.0-SNAPSHOT"
repositories { repositories {
// ... // ...
@@ -184,8 +186,7 @@ Designed to add scripting to kotlin multiplatform application in easy and effici
# Language Roadmap # Language Roadmap
We are now at **v1.5.0-SNAPSHOT** (stable development cycle): basic optimization performed, battery included: standard library is 90% here, initial The current stable release is **v1.5.5**: the 1.5 cycle now includes the database/date/concurrency additions as well as the latest compiler/runtime stabilization work, and the language, tooling, and site are aligned around this release.
support in HTML, popular editors, and IDEA; tools to syntax highlight and format code are ready. It was released closed to schedule.
Ready features: Ready features:
@@ -222,7 +223,7 @@ Ready features:
- [x] assign-if-null operator `?=` - [x] assign-if-null operator `?=`
- [x] user-defined exception classes - [x] user-defined exception classes
All of this is documented in the [language site](https://lynglang.com) and locally [docs/language.md](docs/tutorial.md). the current nightly builds published on the site and in the private maven repository. All of this is documented on the [language site](https://lynglang.com) and locally in [docs/tutorial.md](docs/tutorial.md). The site reflects the current release, while development snapshots continue in the private Maven repository.
## plan: towards v2.0 Next Generation ## plan: towards v2.0 Next Generation
+1 -1
View File
@@ -20,7 +20,7 @@
set -e set -e
echo "publishing all artifacts" echo "publishing all artifacts"
echo echo
./gradlew publishToMavenLocal site:jsBrowserDistribution publish buildInstallablePlugin :lyng:linkReleaseExecutableLinuxX64 :lyng:installJvmDist --parallel ./gradlew publishToMavenLocal site:jsBrowserDistribution publish buildInstallablePlugin :lyng:linkReleaseExecutableLinuxX64 :lyng:installJvmDist --parallel --no-configuration-cache
#echo #echo
#echo "Creating plugin" #echo "Creating plugin"
+14 -11
View File
@@ -18,10 +18,12 @@
# #
upload_only=false upload_only=false
target=vps # default: new server; use --old for d.lynglang.com
for arg in "$@"; do for arg in "$@"; do
if [[ "$arg" == "-u" || "$arg" == "--upload-only" ]]; then if [[ "$arg" == "-u" || "$arg" == "--upload-only" ]]; then
upload_only=true upload_only=true
break elif [[ "$arg" == "--old" ]]; then
target=com
fi fi
done done
@@ -88,19 +90,20 @@ function updateIdeaPluginDownloadLink() {
fi fi
} }
# default target settings # target settings (-t com | -t vps)
case "com" in case "$target" in
com) com)
SSH_HOST=sergeych@d.lynglang.com # host to deploy to SSH_HOST=sergeych@d.lynglang.com
SSH_PORT=22 # ssh port on it SSH_PORT=22
ROOT=/bigstore/sergeych_pub/lyng # directory to rsync to ROOT=/bigstore/sergeych_pub/lyng
;;
vps)
SSH_HOST=sergeych@94.130.36.94
SSH_PORT=22
ROOT=/var/www/lynglang
;; ;;
# com)
# SSH_HOST=vvk@front-01.neurodatalab.com
# ROOT=/home/vvk
# ;;
*) *)
echo "*** ERROR: target not specified (use deploy com | dev)" echo "*** ERROR: unknown target '$target' (use -t com | -t vps)"
echo "*** stop" echo "*** stop"
exit 101 exit 101
esac esac
+12 -10
View File
@@ -1,7 +1,7 @@
#!/bin/bash #!/bin/bash
# #
# Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com # Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
@@ -19,13 +19,15 @@
set -e set -e
root=./lyng/build/install/lyng-jvm/ archive=./lyng/build/distributions/lyng-jvm.zip
install_root="$HOME/bin/jlyng-jvm"
launcher="$install_root/lyng-jvm/bin/lyng"
./gradlew :lyng:installJvmDist ./gradlew :lyng:jvmDistZip
#strip $file mkdir -p ./distributables
#upx $file cp "$archive" ./distributables/lyng-jvm.zip
rm -rf ~/bin/jlyng-jvm || true rm -rf "$install_root" || true
rm ~/bin/jlyng 2>/dev/null || true rm "$HOME/bin/jlyng" 2>/dev/null || true
mkdir -p ~/bin/jlyng-jvm mkdir -p "$install_root"
cp -R $root ~/bin/jlyng-jvm unzip -q ./distributables/lyng-jvm.zip -d "$install_root"
ln -s ~/bin/jlyng-jvm/lyng-jvm/bin/lyng ~/bin/jlyng ln -s "$launcher" "$HOME/bin/jlyng"
+12 -11
View File
@@ -120,17 +120,18 @@ which is used in `toString`) and hex encoding:
## Members ## Members
| name | meaning | type | | name | meaning | type |
|----------------------------|-----------------------------------------|---------------| |----------------------------|------------------------------------------------|---------------|
| `size` | size | Int | | `size` | size | Int |
| `decodeUtf8` | decode to String using UTF8 rules | Any | | `decodeUtf8` | decode to String using UTF8 rules | Any |
| `+` | buffer concatenation | Any | | `+` | buffer concatenation | Any |
| `toMutable()` | create a mutable copy | MutableBuffer | | `toMutable()` | create a mutable copy | MutableBuffer |
| `hex` | encode to hex strign | String | | `hex` | encode to hex strign | String |
| `Buffer.decodeHex(hexStr) | decode hex string | Buffer | | `Buffer.decodeHex(hexStr) | decode hex string | Buffer |
| `base64` | encode to base64 (url flavor) (2) | String | | `base64` | encode to base64 (url flavor) (2) | String |
| `Buffer.decodeBase64(str)` | decode base64 to new Buffer (2) | Buffer | | `base64std` | encode to base64 (default vocabulary, filling) | String |
| `toBitInput()` | create bit input from a byte buffer (3) | | | `Buffer.decodeBase64(str)` | decode base64 to new Buffer (2) | Buffer |
| `toBitInput()` | create bit input from a byte buffer (3) | |
(1) (1)
: optimized implementation that override `Iterable` one : optimized implementation that override `Iterable` one
+211
View File
@@ -0,0 +1,211 @@
# Channel
A `Channel` is a **hot, bidirectional pipe** for passing values between concurrently running coroutines.
Unlike a [Flow], which is cold and replayed on every collection, a `Channel` is stateful: each value
sent is consumed by exactly one receiver.
Channels model the classic _producer / consumer_ pattern and are the right tool when:
- two or more coroutines need to exchange individual values at their own pace;
- you want back-pressure (rendezvous) or explicit buffering control;
- you need a push-based, hot data source (opposite of the pull-based, cold [Flow]).
## Constructors
```
Channel() // rendezvous — sender and receiver must meet
Channel(n: Int) // buffered — sender may run n items ahead of the receiver
Channel(Channel.UNLIMITED) // no limit on buffered items
```
**Rendezvous** (`Channel()`, capacity 0): `send` suspends until a matching `receive` is ready,
and vice-versa. This gives the tightest synchronisation and the smallest memory footprint.
**Buffered** (`Channel(n)`): `send` only suspends when the internal buffer is full. Allows the
producer to get up to _n_ items ahead of the consumer.
**Unlimited** (`Channel(Channel.UNLIMITED)`): `send` never suspends. Useful when the producer is
bursty and you do not want it blocked, but be careful not to grow the buffer without bound.
## Sending and receiving
```lyng
val ch = Channel() // rendezvous channel
val producer = launch {
ch.send("hello") // suspends until the receiver is ready
ch.send("world")
ch.close() // signal: no more values
}
val a = ch.receive() // suspends until "hello" arrives
val b = ch.receive() // suspends until "world" arrives
val c = ch.receive() // channel is closed and drained → null
assertEquals("hello", a)
assertEquals("world", b)
assertEquals(null, c)
```
`receive()` returns `null` when the channel is both **closed** _and_ **fully drained** — that is
the idiomatic loop termination condition:
```lyng
val ch = Channel(4)
launch {
for (i in 1..5) ch.send(i)
ch.close()
}
var item = ch.receive()
while (item != null) {
println(item)
item = ch.receive()
}
```
## Non-suspending poll
`tryReceive()` never suspends. It returns the next buffered value, or `null` if the buffer is
empty or the channel is closed.
```lyng
val ch = Channel(8)
ch.send(42)
println(ch.tryReceive()) // 42
println(ch.tryReceive()) // null — nothing buffered right now
```
Use `tryReceive` for _polling_ patterns where blocking would be unacceptable, for example when
combining channel checks with other work inside a coroutine loop.
## Closing a channel
`close()` marks the channel so that no further `send` calls are accepted. Any items already in the
buffer can still be received. Once the buffer is drained, `receive()` returns `null` and
`isClosedForReceive` becomes `true`.
```lyng
val ch = Channel(2)
ch.send(1)
ch.send(2)
ch.close()
assert(ch.isClosedForSend)
assert(!ch.isClosedForReceive) // still has 2 buffered items
ch.receive() // 1
ch.receive() // 2
assert(ch.isClosedForReceive) // drained
```
Calling `send` after `close()` throws `IllegalStateException`.
## Properties
| property | type | description |
|---------------------|--------|----------------------------------------------------------|
| `isClosedForSend` | `Bool` | `true` after `close()` is called |
| `isClosedForReceive`| `Bool` | `true` when closed _and_ every buffered item is consumed |
## Methods
| method | suspends | description |
|-----------------|----------|----------------------------------------------------------------------------------|
| `send(value)` | yes | send a value; suspends when buffer full (rendezvous: always until partner ready) |
| `receive()` | yes | receive next value; suspends when empty; returns `null` when closed + drained |
| `tryReceive()` | no | return next buffered value or `null`; never suspends |
| `close()` | no | signal end of production; existing buffer items are still receivable |
## Static constants
| constant | value | description |
|---------------------|------------------|-------------------------------------|
| `Channel.UNLIMITED` | `Int.MAX_VALUE` | capacity for an unlimited-buffer channel |
## Common patterns
### Producer / consumer
```lyng
val ch = Channel()
val results = []
val mu = Mutex()
val consumer = launch {
var item = ch.receive()
while (item != null) {
mu.withLock { results += item }
item = ch.receive()
}
}
launch {
for (i in 1..5) ch.send("msg:$i")
ch.close()
}.await()
consumer.await()
println(results)
```
### Fan-out: one channel, many consumers
```lyng
val ch = Channel(16)
// multiple consumers
val workers = (1..4).map { id ->
launch {
var task = ch.receive()
while (task != null) {
println("worker $id handles $task")
task = ch.receive()
}
}
}
// single producer
for (i in 1..20) ch.send(i)
ch.close()
workers.forEach { it.await() }
```
### Ping-pong between two coroutines
```lyng
val ping = Channel()
val pong = Channel()
launch {
repeat(3) {
val msg = ping.receive()
println("got: $msg → sending pong")
pong.send("pong")
}
}
repeat(3) {
ping.send("ping")
println(pong.receive())
}
```
## Channel vs Flow
| | [Flow] | Channel |
|---|---|---|
| **temperature** | cold (lazy) | hot (eager) |
| **replay** | every collector gets a fresh run | each item is consumed once |
| **consumers** | any number; each gets all items | one receiver per item |
| **back-pressure** | built-in via rendezvous | configurable (rendezvous / buffered / unlimited) |
| **typical use** | transform pipelines, sequences | producer–consumer, fan-out |
## See also
- [parallelism] — `launch`, `Deferred`, `Mutex`, `Flow`, and the full concurrency picture
- [Flow] — cold async sequences
[Flow]: parallelism.md#flow
[parallelism]: parallelism.md
+18
View File
@@ -55,6 +55,23 @@ Here is the sample:
assertEquals( (1..3).joinToString { it * 10 }, "10 20 30") assertEquals( (1..3).joinToString { it * 10 }, "10 20 30")
>>> void >>> void
## joinAll
`joinAll()` is an `Iterable<Deferred>` helper that awaits every deferred in iteration order and returns a `List`
with the collected results.
val jobs = (1..4).map { n ->
launch { n * n }
}
assertEquals([1, 4, 9, 16], jobs.joinAll())
>>> void
Notes:
- it does not start any task by itself; it only awaits the deferreds already present in the iterable.
- awaiting happens in iteration order, so the result list keeps the same order as the input iterable.
- if any deferred fails or was cancelled, that `await()` error is propagated from `joinAll()`.
## `sum` and `sumOf` ## `sum` and `sumOf`
These, again, does the thing: These, again, does the thing:
@@ -184,6 +201,7 @@ Search for the first element that satisfies the given predicate:
| sortedWith(comparator) | sort using a comparator that compares elements (1) | | sortedWith(comparator) | sort using a comparator that compares elements (1) |
| sortedBy(predicate) | sort by comparing results of the predicate function | | sortedBy(predicate) | sort by comparing results of the predicate function |
| joinToString(s,t) | convert iterable to string, see (2) | | joinToString(s,t) | convert iterable to string, see (2) |
| joinAll() | for `Iterable<Deferred>`, await all items in order and collect results to [List] |
| reversed() | create a list containing items from this in reverse order | | reversed() | create a list containing items from this in reverse order |
| shuffled() | create a list of shuffled elements | | shuffled() | create a list of shuffled elements |
+110
View File
@@ -0,0 +1,110 @@
# LaunchPool
`LaunchPool` is a bounded-concurrency task pool: you submit lambdas with `launch`, and the pool runs them using a fixed number of worker coroutines.
## Constructor
```
LaunchPool(maxWorkers, maxQueueSize = Channel.UNLIMITED)
```
| Parameter | Description |
|-----------|-------------|
| `maxWorkers` | Maximum number of tasks that run in parallel. |
| `maxQueueSize` | Maximum number of tasks that may wait in the queue. When the queue is full, `launch` suspends the caller until space becomes available. Defaults to `Channel.UNLIMITED` (no bound). |
## Methods
### `launch(lambda): Deferred`
Schedules `lambda` for execution and returns a `Deferred` for its result.
- Suspends if the queue is full (`maxQueueSize` reached).
- Throws `IllegalStateException` if the pool is already closed or cancelled.
- Any exception thrown by `lambda` is captured in the returned `Deferred` and **does not escape the pool**.
```lyng
val pool = LaunchPool(4)
val d1 = pool.launch { computeSomething() }
val d2 = pool.launch { computeOther() }
pool.closeAndJoin()
println(d1.await())
println(d2.await())
```
### `closeAndJoin()`
Stops accepting new tasks and suspends until all queued and running tasks complete normally. After this call, any further `launch` throws `IllegalStateException`. Idempotent — safe to call multiple times.
### `cancel()`
Immediately closes the queue and cancels all worker coroutines. Queued but unstarted tasks are discarded. After this call, `launch` throws `IllegalStateException`. Idempotent.
### `cancelAndJoin()`
Like `cancel()`, but also suspends until all worker coroutines have stopped. Useful when you need to be sure no worker code is still running before proceeding. Idempotent.
## Exception handling
Exceptions from submitted lambdas are captured per-task in the returned `Deferred`. The pool itself continues running after a task failure:
```lyng
val pool = LaunchPool(2)
val good = pool.launch { 42 }
val bad = pool.launch { throw IllegalArgumentException("boom") }
pool.closeAndJoin()
assertEquals(42, good.await())
assertThrows(IllegalArgumentException) { bad.await() }
```
## Bounded queue / back-pressure
When `maxQueueSize` is set, the producer suspends if the queue fills up, providing automatic back-pressure:
```lyng
// 1 worker, queue of 2 — producer can be at most 2 tasks ahead of what's running
val pool = LaunchPool(1, 2)
val d1 = pool.launch { delay(10); "a" }
val d2 = pool.launch { delay(10); "b" }
val d3 = pool.launch { delay(10); "c" } // suspends until d1 is picked up by the worker
pool.closeAndJoin()
```
## Collecting all results
`launch` returns a `Deferred`, so you can collect results with `joinAll()`:
```lyng
val pool = LaunchPool(4)
val jobs = (1..10).map { n -> pool.launch { n * n } }
pool.closeAndJoin()
val results = jobs.joinAll()
// results == [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
```
## Concurrency limit in practice
With `maxWorkers = 2`, at most 2 tasks run simultaneously regardless of how many are queued:
```lyng
val mu = Mutex()
var active = 0
var maxSeen = 0
val pool = LaunchPool(2)
(1..8).map {
pool.launch {
mu.withLock { active++; if (active > maxSeen) maxSeen = active }
delay(5)
mu.withLock { active-- }
}
}
pool.closeAndJoin()
assert(maxSeen <= 2)
```
## See also
- [parallelism.md](parallelism.md) — `launch`, `Deferred`, `Mutex`, `Channel`, and coroutine basics
- [Channel.md](Channel.md) — the underlying channel primitive used by `LaunchPool`
+66
View File
@@ -0,0 +1,66 @@
# Legacy Digest Functions (`lyng.legacy_digest`)
> ⚠️ **Security warning:** The functions in this module use cryptographically broken
> algorithms. Do **not** use them for passwords, digital signatures, integrity
> verification against adversarial tampering, or any other security-sensitive
> purpose. They exist solely for compatibility with legacy protocols and file
> formats that require specific hash values.
Import when you need to produce a SHA-1 digest for an existing protocol or format:
```lyng
import lyng.legacy_digest
```
## `LegacyDigest` Object
### `sha1(data): String`
Computes the SHA-1 digest of `data` and returns it as a 40-character lowercase
hex string.
`data` can be:
| Type | Behaviour |
|----------|----------------------------------------|
| `String` | Encoded as UTF-8, then hashed |
| `Buffer` | Raw bytes hashed directly |
| anything | Falls back to `toString()` then UTF-8 |
```lyng
import lyng.legacy_digest
// String input
val h = LegacyDigest.sha1("abc")
assertEquals("a9993e364706816aba3e25717850c26c9cd0d89d", h)
// Empty string
assertEquals("da39a3ee5e6b4b0d3255bfef95601890afd80709", LegacyDigest.sha1(""))
```
```lyng
import lyng.legacy_digest
import lyng.buffer
// Buffer input (raw bytes)
val buf = Buffer.decodeHex("616263") // 0x61 0x62 0x63 = "abc"
assertEquals("a9993e364706816aba3e25717850c26c9cd0d89d", LegacyDigest.sha1(buf))
```
## Implementation Notes
- Pure Kotlin/KMP — no native libraries or extra dependencies.
- Follows FIPS 180-4.
- The output is always lowercase hex, never uppercase or binary.
## When to Use
Use `lyng.legacy_digest` only when an external system you cannot change requires
a SHA-1 value, for example:
- old git-style content addresses
- some OAuth 1.0 / HMAC-SHA1 signature schemes
- legacy file checksums defined in published specs
For any new design choose a current hash function (SHA-256 or better) once
Lyng adds a `lyng.digest` module.
+18
View File
@@ -45,6 +45,16 @@ You can concatenate lists or iterable objects:
assert( [4,5] + (1..3) == [4, 5, 1, 2, 3]) assert( [4,5] + (1..3) == [4, 5, 1, 2, 3])
>>> void >>> void
## Constructing lists
Besides literals, you can build a list by size using `List.fill`:
val squares = List.fill(5) { i -> i * i }
assertEquals([0, 1, 4, 9, 16], squares)
>>> void
`List.fill(size) { ... }` calls the block once for each index from `0` to `size - 1` and returns a new mutable list.
## Appending ## Appending
To append to lists, use `+=` with elements, lists and any [Iterable] instances, but beware it will To append to lists, use `+=` with elements, lists and any [Iterable] instances, but beware it will
@@ -164,6 +174,9 @@ List could be sorted in place, just like [Collection] provide sorted copies, in
| `[index]` | get or set element at index | Int | | `[index]` | get or set element at index | Int |
| `[Range]` | get slice of the array (copy) | Range | | `[Range]` | get slice of the array (copy) | Range |
| `+=` | append element(s) (2) | List or Obj | | `+=` | append element(s) (2) | List or Obj |
| `List.fill(size, block)` | build a new list from indices `0..<size` | Int, Callable |
| `List.fill(size,capacity,block)` | same, pre-allocating capacity slots | Int, Int, Callable |
| `ensureCapacity(count)` | pre-allocate storage for at least `count` elements without reallocation (5) | Int |
| `sort()` | in-place sort, natural order | void | | `sort()` | in-place sort, natural order | void |
| `sortBy(predicate)` | in-place sort bu `predicate` call result (3) | void | | `sortBy(predicate)` | in-place sort bu `predicate` call result (3) | void |
| `sortWith(comparator)` | in-place sort using `comarator` function (4) | void | | `sortWith(comparator)` | in-place sort using `comarator` function (4) | void |
@@ -186,6 +199,11 @@ order, e.g. is same as `list.sortWith { a,b -> predicate(a) <=> predicate(b) }`
positive if first is greater, and zero if they are equal. For example, the equvalent comparator positive if first is greater, and zero if they are equal. For example, the equvalent comparator
for `sort()` will be `sort { a, b -> a <=> b } for `sort()` will be `sort { a, b -> a <=> b }
(5)
: if the current capacity is already ≥ `count`, this is a no-op. Otherwise the internal storage
is reallocated to hold at least `count` elements. Use this before a bulk `+=` loop to avoid
repeated reallocations. `List.fill(size, capacity, block)` calls this automatically.
It inherits from [Iterable] too and thus all iterable methods are applicable to any list. It inherits from [Iterable] too and thus all iterable methods are applicable to any list.
## Observable list hooks ## Observable list hooks
+129 -2
View File
@@ -1001,9 +1001,9 @@ Static fields can be accessed from static methods via the class qualifier:
assertEquals("bar", Test.getData() ) assertEquals("bar", Test.getData() )
>>> void >>> void
# Extending classes # Extension members
It sometimes happen that the class is missing some particular functionality that can be _added to it_ without rewriting its inner logic and using its private state. In this case _extension members_ could be used. Sometimes an existing type or named singleton object is missing some particular functionality that can be _added to it_ without rewriting its inner logic and without using its private state. In this case, _extension members_ can be used.
## Extension methods ## Extension methods
@@ -1020,6 +1020,133 @@ For example, we want to create an extension method that would test if a value ca
assert( ! "5.2".isInteger() ) assert( ! "5.2".isInteger() )
>>> void >>> void
Extension methods normally act like instance members. If declared as `static`, they are called on the type object itself:
```lyng
static fun List<T>.fill(size: Int, block: (Int)->T): List<T> { ... }
val tens = List.fill(5) { it * 10 }
```
## Extending singleton `object` declarations
Named singleton objects can also be extension receivers. Use the object name as the receiver type:
```lyng
object Config {
fun base() = "cfg"
}
fun Config.describe(value) {
this.base() + ":" + value.toString()
}
val Config.tag get() = this.base() + ":tag"
assertEquals("cfg:42", Config.describe(42))
assertEquals("cfg:tag", Config.tag)
```
This differs from extending a class in one important way:
- `fun Point.foo()` adds a member-like extension for all `Point` instances.
- `fun Config.foo()` adds a member-like extension only for the single named object `Config`.
The same rules still apply:
- Extensions on singleton objects are scope-isolated just like class extensions.
- They cannot access the object's `private` members.
- Inside the extension body, `this` is the singleton object itself.
## Extension indexers
Bracket syntax is just another form of member dispatch. When you write `value[i]` or `value[i] = x`, Lyng lowers it to `getAt(...)` and `putAt(...)`.
That means indexers can be extended in the same way as methods and properties.
### Extending indexers on classes
Use `override fun Type.getAt(...)` and `override fun Type.putAt(...)`:
```lyng
class BoxStore {
val data = {"name": "alice"}
}
override fun BoxStore.getAt(key: String): Object? {
data[key]
}
override fun BoxStore.putAt(key: String, value: Object) {
data[key] = value
}
val store = BoxStore()
assertEquals("alice", store["name"])
store["name"] = "bob"
assertEquals("bob", store["name"])
```
As with other extension members, this does not modify the original class declaration. It adds indexer behavior only in the scope where the extension is visible.
### Extending indexers on singleton `object` declarations
Named singleton objects work the same way:
```lyng
object Storage
var storageData = {}
override fun Storage.getAt(key: String): Object? {
storageData[key]
}
override fun Storage.putAt(key: String, value: Object) {
storageData[key] = value
}
Storage["name"] = "alice"
val name: String? = Storage["name"]
assertEquals("alice", name)
```
This is the indexer equivalent of `fun Config.foo()`: the extension applies to that single named object, not to all instances of some class.
### Selector packing
Index syntax can contain more than one selector:
```lyng
value[i]
value[i, j]
```
The same packing rules still apply for extension indexers:
- `value[i]` calls `getAt(i)` or `putAt(i, value)`
- `value[i, j]` passes `[i, j]` as one list-like index object
- `value[i, j, k]` passes `[i, j, k]`
So if you want multi-selector indexing, define the receiver to accept that packed index object.
### About types and generics
In practice, extension indexers are usually best declared with `Object?` for reads and `Object` for writes:
```lyng
override fun Storage.getAt(key: String): Object? { ... }
override fun Storage.putAt(key: String, value: Object) { ... }
```
Then use the expected type at the call site:
```lyng
val name: String? = Storage["name"]
```
Explicit generic arguments do not fit naturally onto `[]` syntax, so typed assignment on read is usually the clearest approach.
## Extension properties ## Extension properties
Just like methods, you can extend existing classes with properties. These can be defined using simple initialization (for `val` only) or with custom accessors. Just like methods, you can extend existing classes with properties. These can be defined using simple initialization (for `val` only) or with custom accessors.
+45 -1
View File
@@ -25,6 +25,23 @@ Exclusive end ranges are adopted from kotlin either:
assert(4 in r) assert(4 in r)
>>> void >>> void
Descending finite ranges are explicit too:
val r = 5 downTo 1
assert(r.isDescending)
assert(r.toList() == [5,4,3,2,1])
>>> void
Use `downUntil` when the lower bound should be excluded:
val r = 5 downUntil 1
assert(r.toList() == [5,4,3,2])
assert(1 !in r)
>>> void
This is explicit by design: `5..1` is not treated as a reverse range. It is an
ordinary ascending range with no values in it when iterated.
In any case, we can test an object to belong to using `in` and `!in` and In any case, we can test an object to belong to using `in` and `!in` and
access limits: access limits:
@@ -73,6 +90,23 @@ but
>>> 2 >>> 2
>>> void >>> void
Descending ranges work in `for` loops exactly the same way:
for( i in 3 downTo 1 )
println(i)
>>> 3
>>> 2
>>> 1
>>> void
And with an exclusive lower bound:
for( i in 3 downUntil 1 )
println(i)
>>> 3
>>> 2
>>> void
### Stepped ranges ### Stepped ranges
Use `step` to change the iteration increment. The range bounds still define membership, Use `step` to change the iteration increment. The range bounds still define membership,
@@ -80,9 +114,18 @@ so iteration ends when the next value is no longer in the range.
assert( [1,3,5] == (1..5 step 2).toList() ) assert( [1,3,5] == (1..5 step 2).toList() )
assert( [1,3] == (1..<5 step 2).toList() ) assert( [1,3] == (1..<5 step 2).toList() )
assert( [5,3,1] == (5 downTo 1 step 2).toList() )
assert( ['a','c','e'] == ('a'..'e' step 2).toList() ) assert( ['a','c','e'] == ('a'..'e' step 2).toList() )
>>> void >>> void
Descending ranges still use a positive `step`; the direction comes from
`downTo` / `downUntil`:
assert( ['e','c','a'] == ('e' downTo 'a' step 2).toList() )
>>> void
A negative step with `downTo` / `downUntil` is invalid.
Real ranges require an explicit step: Real ranges require an explicit step:
assert( [0,0.25,0.5,0.75,1.0] == (0.0..1.0 step 0.25).toList() ) assert( [0,0.25,0.5,0.75,1.0] == (0.0..1.0 step 0.25).toList() )
@@ -97,7 +140,7 @@ Open-ended ranges require an explicit step to iterate:
You can use Char as both ends of the closed range: You can use Char as both ends of the closed range:
val r = 'a' .. 'c' val r = 'a'..'c'
assert( 'b' in r) assert( 'b' in r)
assert( 'e' !in r) assert( 'e' !in r)
for( ch in r ) for( ch in r )
@@ -119,6 +162,7 @@ Exclusive end char ranges are supported too:
|-----------------|------------------------------|---------------| |-----------------|------------------------------|---------------|
| contains(other) | used in `in` | Range, or Any | | contains(other) | used in `in` | Range, or Any |
| isEndInclusive | true for '..' | Bool | | isEndInclusive | true for '..' | Bool |
| isDescending | true for `downTo`/`downUntil`| Bool |
| isOpen | at any end | Bool | | isOpen | at any end | Bool |
| isIntRange | both start and end are Int | Bool | | isIntRange | both start and end are Int | Bool |
| step | explicit iteration step | Any? | | step | explicit iteration step | Any? |
+12
View File
@@ -64,6 +64,18 @@ Also, string indexing is Regex-aware, and works like `Regex.find` (_not findall!
assert( "cd" == ("abcdef"[ "c.".re ] as RegexMatch).value ) assert( "cd" == ("abcdef"[ "c.".re ] as RegexMatch).value )
>>> void >>> void
Regex replacement is exposed on `String.replace` and `String.replaceFirst`:
assertEquals( "v#.#.#", "v1.2.3".replace( "\d+".re, "#" ) )
assertEquals( "v[1].[2].[3]", "v1.2.3".replace( "(\d+)".re ) { m -> "[" + m[1] + "]" } )
assertEquals( "year-04-08", "2026-04-08".replaceFirst( "\d+".re, "year" ) )
>>> void
When `replace` takes a plain `String`, it is treated literally, not as a regex pattern:
assertEquals( "a-b-c", "a.b.c".replace( ".", "-" ) )
>>> void
# Regex class reference # Regex class reference
+17 -7
View File
@@ -15,15 +15,17 @@ Primary sources used: `lynglib/src/commonMain/kotlin/net/sergeych/lyng/{Parser,T
## 2. Lexical Syntax ## 2. Lexical Syntax
- Comments: `// line`, `/* block */`. - Comments: `// line`, `/* block */`.
- Strings: `"..."` (supports escapes). Multiline string content is normalized by indentation logic. - Strings: `"..."` or `` `...` `` (supports escapes). Multiline string content is normalized by indentation logic.
- Supported escapes: `\n`, `\r`, `\t`, `\"`, `\\`, `\uXXXX` (4 hex digits). - AI generation preference: use `"..."` by default, including multiline strings; `"` strings are also multiline-capable and should be preferred for ordinary code/doc/SQL text. Use backtick strings mainly when the content contains many double quotes and backticks would make the source clearer.
- Shared escapes: `\n`, `\r`, `\t`, `\\`, `\uXXXX` (4 hex digits).
- Delimiter escapes: `\"` inside `"..."`, ``\` `` inside `` `...` ``.
- Unicode escapes use exactly 4 hex digits (for example: `"\u0416"` -> `Ж`). - Unicode escapes use exactly 4 hex digits (for example: `"\u0416"` -> `Ж`).
- Unknown `\x` escapes in strings are preserved literally as two characters (`\` and `x`). - Unknown `\x` escapes in strings are preserved literally as two characters (`\` and `x`).
- String interpolation is supported: - String interpolation is supported:
- identifier form: `"$name"` - identifier form: `"$name"` or `` `$name` ``
- expression form: `"${expr}"` - expression form: `"${expr}"` or `` `${expr}` ``
- escaped dollar: `"\$"` and `"$$"` both produce literal `$`. - escaped dollar: `"\$"`, `"$$"`, `` `\$` ``, and `` `$$` `` all produce literal `$`.
- `\\$x` means backslash + interpolated `x`. - `\\$x` means backslash + interpolated `x` in either delimiter form.
- Per-file opt-out is supported via leading comment directive: - Per-file opt-out is supported via leading comment directive:
- `// feature: interpolation: off` - `// feature: interpolation: off`
- with this directive, `$...` stays literal text. - with this directive, `$...` stays literal text.
@@ -50,8 +52,10 @@ Primary sources used: `lynglib/src/commonMain/kotlin/net/sergeych/lyng/{Parser,T
- Range literals: - Range literals:
- inclusive: `a..b` - inclusive: `a..b`
- exclusive end: `a..<b` - exclusive end: `a..<b`
- descending inclusive: `a downTo b`
- descending exclusive end: `a downUntil b`
- open-ended forms are supported (`a..`, `..b`, `..`). - open-ended forms are supported (`a..`, `..b`, `..`).
- optional step: `a..b step 2` - optional step: `a..b step 2`, `a downTo b step 2`
- Lambda literal: - Lambda literal:
- with params: `{ x, y -> x + y }` - with params: `{ x, y -> x + y }`
- implicit `it`: `{ it + 1 }` - implicit `it`: `{ it + 1 }`
@@ -93,6 +97,7 @@ Primary sources used: `lynglib/src/commonMain/kotlin/net/sergeych/lyng/{Parser,T
- single selector: `a[i]` - single selector: `a[i]`
- multiple selectors: `a[i, j, k]` - multiple selectors: `a[i, j, k]`
- language-level indexing with multiple selectors is passed to `getAt`/`putAt` as one list-like index object, not as multiple method arguments. - language-level indexing with multiple selectors is passed to `getAt`/`putAt` as one list-like index object, not as multiple method arguments.
- indexers can also be supplied by extension members, including named singleton `object` receivers via `override fun Storage.getAt(...)` / `putAt(...)`.
- example: `m[0..2, 2]`. - example: `m[0..2, 2]`.
## 5. Declarations ## 5. Declarations
@@ -114,6 +119,8 @@ Primary sources used: `lynglib/src/commonMain/kotlin/net/sergeych/lyng/{Parser,T
- shorthand: `fun f(x) = expr`. - shorthand: `fun f(x) = expr`.
- generics: `fun f<T>(x: T): T`. - generics: `fun f<T>(x: T): T`.
- extension functions: `fun Type.name(...) { ... }`. - extension functions: `fun Type.name(...) { ... }`.
- named singleton `object` declarations can be extension receivers too: `fun Config.describe(...) { ... }`, `val Config.tag get() = ...`.
- static extension functions are callable on the type object: `static fun List<T>.fill(...)` -> `List.fill(...)`.
- delegated callable: `fun f(...) by delegate`. - delegated callable: `fun f(...) by delegate`.
- Type aliases: - Type aliases:
- `type Name = TypeExpr` - `type Name = TypeExpr`
@@ -129,6 +136,9 @@ Primary sources used: `lynglib/src/commonMain/kotlin/net/sergeych/lyng/{Parser,T
## 6. Control Flow ## 6. Control Flow
- `if` is expression-like. - `if` is expression-like.
- `compile if (cond) { ... } else { ... }` is a compile-time-only conditional.
- current condition grammar is restricted to `defined(NameOr.Package)`, `!`, `&&`, `||`, and parentheses.
- the untaken branch is skipped by the compiler and is not name-resolved or type-checked.
- `when(value) { ... }` supported. - `when(value) { ... }` supported.
- branch conditions support equality, `in`, `!in`, `is`, `!is`, and `nullable` predicate. - branch conditions support equality, `in`, `!in`, `is`, `!is`, and `nullable` predicate.
- `when { ... }` (subject-less) is currently not implemented. - `when { ... }` (subject-less) is currently not implemented.
+7 -2
View File
@@ -18,6 +18,7 @@ Sources: `lynglib/src/commonMain/kotlin/net/sergeych/lyng/Script.kt`, `lynglib/s
- Async/concurrency: `launch`, `yield`, `flow`, `delay`. - Async/concurrency: `launch`, `yield`, `flow`, `delay`.
- `Deferred.cancel()` cancels an active task. - `Deferred.cancel()` cancels an active task.
- `Deferred.await()` throws `CancellationException` if that task was cancelled. - `Deferred.await()` throws `CancellationException` if that task was cancelled.
- `Iterable<Deferred>.joinAll()` awaits every deferred in iteration order and returns a `List` of results.
- Math: `floor`, `ceil`, `round`, `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `sinh`, `cosh`, `tanh`, `asinh`, `acosh`, `atanh`, `exp`, `ln`, `log10`, `log2`, `pow`, `sqrt`, `abs`, `clamp`. - Math: `floor`, `ceil`, `round`, `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `sinh`, `cosh`, `tanh`, `asinh`, `acosh`, `atanh`, `exp`, `ln`, `log10`, `log2`, `pow`, `sqrt`, `abs`, `clamp`.
- These helpers also accept `lyng.decimal.Decimal`. - These helpers also accept `lyng.decimal.Decimal`.
- Exact Decimal path today: `abs`, `floor`, `ceil`, `round`, and `pow` with integral exponent. - Exact Decimal path today: `abs`, `floor`, `ceil`, `round`, and `pow` with integral exponent.
@@ -46,7 +47,8 @@ Sources: `lynglib/src/commonMain/kotlin/net/sergeych/lyng/Script.kt`, `lynglib/s
- Iteration/filtering: `forEach`, `filter`, `filterFlow`, `filterNotNull`, `filterFlowNotNull`, `drop`, `dropLast`, `takeLast`. - Iteration/filtering: `forEach`, `filter`, `filterFlow`, `filterNotNull`, `filterFlowNotNull`, `drop`, `dropLast`, `takeLast`.
- Search/predicates: `findFirst`, `findFirstOrNull`, `any`, `all`, `count`, `first`, `last`. - Search/predicates: `findFirst`, `findFirstOrNull`, `any`, `all`, `count`, `first`, `last`.
- Mapping/aggregation: `map`, `flatMap`, `flatten`, `sum`, `sumOf`, `minOf`, `maxOf`. - Mapping/aggregation: `map`, `flatMap`, `flatten`, `sum`, `sumOf`, `minOf`, `maxOf`.
- Ordering: `sorted`, `sortedBy`, `shuffled`, `List.sort`, `List.sortBy`. - Ordering and list building: `sorted`, `sortedBy`, `shuffled`, `List.sort`, `List.sortBy`, `List.fill`.
- `List.fill(size) { index -> ... }` constructs a new `List<T>` by evaluating the block once per index from `0` to `size - 1`.
- String helper: `joinToString`, `String.re`. - String helper: `joinToString`, `String.re`.
### 4.3 Delegation helpers ### 4.3 Delegation helpers
@@ -74,10 +76,13 @@ Sources: `lynglib/src/commonMain/kotlin/net/sergeych/lyng/Script.kt`, `lynglib/s
- `Matrix`, `Vector`, `matrix(rows)`, `vector(values)`, dense linear algebra, inversion, solving, and matrix slicing with `m[row, col]`. - `Matrix`, `Vector`, `matrix(rows)`, `vector(values)`, dense linear algebra, inversion, solving, and matrix slicing with `m[row, col]`.
- `import lyng.buffer` - `import lyng.buffer`
- `Buffer`, `MutableBuffer`. - `Buffer`, `MutableBuffer`.
- `import lyng.legacy_digest`
- `LegacyDigest.sha1(data): String` — SHA-1 hex digest; `data` may be `String` (UTF-8) or `Buffer` (raw bytes).
- ⚠️ Cryptographically broken. Use only for legacy protocol / file-format compatibility.
- `import lyng.serialization` - `import lyng.serialization`
- `Lynon` serialization utilities. - `Lynon` serialization utilities.
- `import lyng.time` - `import lyng.time`
- `Instant`, `DateTime`, `Duration`, and module `delay`. - `Instant`, `Date`, `DateTime`, `Duration`, and module `delay`.
## 6. Optional (lyngio) Modules ## 6. Optional (lyngio) Modules
Requires installing `lyngio` into the import manager from host code. Requires installing `lyngio` into the import manager from host code.
+1
View File
@@ -3,6 +3,7 @@
## Lync CLI tool ## Lync CLI tool
- [lyng-linuxX64.zip](/distributables/lyng-linuxX64.zip) CLI tool for linuxX64: nodependencies, small monolith executable binary. - [lyng-linuxX64.zip](/distributables/lyng-linuxX64.zip) CLI tool for linuxX64: nodependencies, small monolith executable binary.
- [lyng-jvm.zip](/distributables/lyng-jvm.zip) JVM CLI distribution: download, unpack, and run `lyng-jvm/bin/lyng`.
## IDE plugins ## IDE plugins
+401
View File
@@ -0,0 +1,401 @@
### lyng.io.db — SQL database access for Lyng scripts
This module provides the portable SQL database contract for Lyng. The current shipped providers are SQLite via `lyng.io.db.sqlite` and a JVM-only JDBC bridge via `lyng.io.db.jdbc`.
> **Note:** `lyngio` is a separate library module. It must be explicitly added as a dependency to your host application and initialized in your Lyng scopes.
---
#### Install the module into a Lyng session
For SQLite-backed database access, install both the generic DB module and the SQLite provider:
```kotlin
import net.sergeych.lyng.EvalSession
import net.sergeych.lyng.Scope
import net.sergeych.lyng.io.db.createDbModule
import net.sergeych.lyng.io.db.sqlite.createSqliteModule
suspend fun bootstrapDb() {
val session = EvalSession()
val scope: Scope = session.getScope()
createDbModule(scope)
createSqliteModule(scope)
session.eval("""
import lyng.io.db
import lyng.io.db.sqlite
""".trimIndent())
}
```
`createSqliteModule(...)` also registers the `sqlite:` scheme for generic `openDatabase(...)`.
For JVM JDBC-backed access, install the JDBC provider as well:
```kotlin
import net.sergeych.lyng.EvalSession
import net.sergeych.lyng.Scope
import net.sergeych.lyng.io.db.createDbModule
import net.sergeych.lyng.io.db.jdbc.createJdbcModule
suspend fun bootstrapJdbc() {
val session = EvalSession()
val scope: Scope = session.getScope()
createDbModule(scope)
createJdbcModule(scope)
session.eval("""
import lyng.io.db
import lyng.io.db.jdbc
""".trimIndent())
}
```
`createJdbcModule(...)` registers `jdbc:`, `h2:`, `postgres:`, and `postgresql:` for `openDatabase(...)`.
---
#### Using from Lyng scripts
Typed SQLite open helper:
```lyng
import lyng.io.db.sqlite
val db = openSqlite(":memory:")
val userCount = db.transaction { tx ->
tx.execute("create table user(id integer primary key autoincrement, name text not null)")
tx.execute("insert into user(name) values(?)", "Ada")
tx.execute("insert into user(name) values(?)", "Linus")
tx.select("select count(*) as count from user").toList()[0]["count"]
}
assertEquals(2, userCount)
```
Generic provider-based open:
```lyng
import lyng.io.db
import lyng.io.db.sqlite
val db = openDatabase(
"sqlite:./app.db",
Map(
"foreignKeys" => true,
"busyTimeoutMillis" => 5000
)
)
```
JVM JDBC open with H2:
```lyng
import lyng.io.db.jdbc
val db = openH2("mem:demo;DB_CLOSE_DELAY=-1")
val names = db.transaction { tx ->
tx.execute("create table person(id bigint auto_increment primary key, name varchar(120) not null)")
tx.execute("insert into person(name) values(?)", "Ada")
tx.execute("insert into person(name) values(?)", "Linus")
tx.select("select name from person order by id").toList()
}
assertEquals("Ada", names[0]["name"])
assertEquals("Linus", names[1]["name"])
```
Generic JDBC open through `openDatabase(...)`:
```lyng
import lyng.io.db
import lyng.io.db.jdbc
val db = openDatabase(
"jdbc:h2:mem:demo2;DB_CLOSE_DELAY=-1",
Map()
)
val answer = db.transaction { tx ->
tx.select("select 42 as answer").toList()[0]["answer"]
}
assertEquals(42, answer)
```
PostgreSQL typed open:
```lyng
import lyng.io.db.jdbc
val db = openPostgres(
"jdbc:postgresql://127.0.0.1/appdb",
"appuser",
"secret"
)
val titles = db.transaction { tx ->
tx.execute("create table if not exists task(id bigserial primary key, title text not null)")
tx.execute("insert into task(title) values(?)", "Ship JDBC provider")
tx.execute("insert into task(title) values(?)", "Test PostgreSQL path")
tx.select("select title from task order by id").toList()
}
assertEquals("Ship JDBC provider", titles[0]["title"])
```
Nested transactions use real savepoint semantics:
```lyng
import lyng.io.db
import lyng.io.db.sqlite
val db = openSqlite(":memory:")
db.transaction { tx ->
tx.execute("create table item(id integer primary key autoincrement, name text not null)")
tx.execute("insert into item(name) values(?)", "outer")
try {
tx.transaction { inner ->
inner.execute("insert into item(name) values(?)", "inner")
throw IllegalStateException("rollback nested")
}
} catch (_: IllegalStateException) {
}
assertEquals(1, tx.select("select count(*) as count from item").toList()[0]["count"])
}
```
Intentional rollback without treating it as a backend failure:
```lyng
import lyng.io.db
import lyng.io.db.sqlite
val db = openSqlite(":memory:")
assertThrows(RollbackException) {
db.transaction { tx ->
tx.execute("create table item(id integer primary key autoincrement, name text not null)")
tx.execute("insert into item(name) values(?)", "temporary")
throw RollbackException("stop here")
}
}
```
---
#### Portable API
##### `Database`
- `transaction(block)` — opens a transaction, commits on normal exit, rolls back on uncaught failure.
##### `SqlTransaction`
- `select(clause, params...)` — execute a statement whose primary result is a row set.
- `execute(clause, params...)` — execute a side-effect statement and return `ExecutionResult`.
- `transaction(block)` — nested transaction with real savepoint semantics.
##### `ResultSet`
- `columns` — positional `SqlColumn` metadata, available before iteration.
- `size()` — result row count.
- `isEmpty()` — fast emptiness check where possible.
- `iterator()` — normal row iteration while the transaction is active.
- `toList()` — materialize detached `SqlRow` snapshots that may be used after the transaction ends.
##### `SqlRow`
- `row[index]` — zero-based positional access.
- `row["columnName"]` — case-insensitive lookup by output column label.
Name-based access fails with `SqlUsageException` if the name is missing or ambiguous.
##### `ExecutionResult`
- `affectedRowsCount`
- `getGeneratedKeys()`
Statements that return rows directly, such as `... returning ...`, should use `select(...)`, not `execute(...)`.
---
#### Value mapping
Portable bind values:
- `null`
- `Bool`
- `Int`, `Double`, `Decimal`
- `String`
- `Buffer`
- `Date`, `DateTime`, `Instant`
Unsupported parameter values fail with `SqlUsageException`.
Portable result metadata categories:
- `Binary`
- `String`
- `Int`
- `Double`
- `Decimal`
- `Bool`
- `Date`
- `DateTime`
- `Instant`
For temporal types, see [time functions](time.md).
---
#### SQLite provider
`lyng.io.db.sqlite` currently provides the first concrete backend.
Typed helper:
```lyng
openSqlite(
path: String,
readOnly: Bool = false,
createIfMissing: Bool = true,
foreignKeys: Bool = true,
busyTimeoutMillis: Int = 5000
): Database
```
Accepted generic URL forms:
- `sqlite::memory:`
- `sqlite:relative/path.db`
- `sqlite:/absolute/path.db`
Supported `openDatabase(..., extraParams)` keys for SQLite:
- `readOnly: Bool`
- `createIfMissing: Bool`
- `foreignKeys: Bool`
- `busyTimeoutMillis: Int`
SQLite write/read policy in v1:
- `Bool` writes as `0` / `1`
- `Decimal` writes as canonical text
- `Date` writes as `YYYY-MM-DD`
- `DateTime` writes as ISO local timestamp text without timezone
- `Instant` writes as ISO UTC timestamp text with explicit timezone marker
- `TIME*` values stay `String`
- `TIMESTAMP` / `DATETIME` reject timezone-bearing stored text
Open-time validation failures:
- malformed URL or bad option shape -> `IllegalArgumentException`
- runtime open failure -> `DatabaseException`
#### JDBC provider
`lyng.io.db.jdbc` is currently implemented on the JVM target only. The `lyngio-jvm` artifact bundles and explicitly loads these JDBC drivers:
- SQLite
- H2
- PostgreSQL
Typed helpers:
```lyng
openJdbc(
connectionUrl: String,
user: String? = null,
password: String? = null,
driverClass: String? = null,
properties: Map<String, Object?>? = null
): Database
openH2(
connectionUrl: String,
user: String? = null,
password: String? = null,
properties: Map<String, Object?>? = null
): Database
openPostgres(
connectionUrl: String,
user: String? = null,
password: String? = null,
properties: Map<String, Object?>? = null
): Database
```
Accepted generic URL forms:
- `jdbc:h2:mem:test;DB_CLOSE_DELAY=-1`
- `h2:mem:test;DB_CLOSE_DELAY=-1`
- `jdbc:postgresql://localhost/app`
- `postgres://localhost/app`
- `postgresql://localhost/app`
Supported `openDatabase(..., extraParams)` keys for JDBC:
- `driverClass: String`
- `user: String`
- `password: String`
- `properties: Map<String, Object?>`
Behavior notes for the JDBC bridge:
- the portable `Database` / `SqlTransaction` API stays the same as for SQLite
- nested transactions use JDBC savepoints
- JDBC connection properties are built from `user`, `password`, and `properties`
- `properties` values are stringified before being passed to JDBC
- statements with row-returning clauses still must use `select(...)`, not `execute(...)`
Platform support for this provider:
- `lyng.io.db.jdbc` — JVM only
- `openH2(...)` — works out of the box with `lyngio-jvm`
- `openPostgres(...)` — driver included, but an actual PostgreSQL server is still required
PostgreSQL-specific notes:
- `openPostgres(...)` accepts either a full JDBC URL or shorthand forms such as `//localhost/app`
- local peer/trust setups may use an empty password string
- generated keys work with PostgreSQL `bigserial` / identity columns through `ExecutionResult.getGeneratedKeys()`
- for reproducible automated tests, prefer a disposable PostgreSQL instance such as Docker/Testcontainers instead of a long-lived shared server
---
#### Lifetime rules
`ResultSet` is valid only while its owning transaction is active.
`SqlRow` values are detached snapshots once materialized, so this pattern is valid:
```lyng
val rows = db.transaction { tx ->
tx.select("select name from person order by id").toList()
}
assertEquals("Ada", rows[0]["name"])
```
This means:
- do not keep `ResultSet` objects after the transaction block returns
- materialize rows with `toList()` inside the transaction when they must outlive it
The same rule applies to generated keys from `ExecutionResult.getGeneratedKeys()`: the `ResultSet` is transaction-scoped, but rows returned by `toList()` are detached.
---
#### Platform support
- `lyng.io.db` — generic contract, available when host code installs it
- `lyng.io.db.sqlite` — implemented on JVM and Linux Native in the current release tree
- `lyng.io.db.jdbc` — implemented on JVM in the current release tree
For the broader I/O overview, see [lyngio overview](lyngio.md).
+6
View File
@@ -3,6 +3,12 @@
This module provides minimal raw transport networking for Lyng scripts. It is implemented in `lyngio` and backed by Ktor sockets on the JVM and Linux Native, and by Node networking APIs on JS/Node runtimes. This module provides minimal raw transport networking for Lyng scripts. It is implemented in `lyngio` and backed by Ktor sockets on the JVM and Linux Native, and by Node networking APIs on JS/Node runtimes.
> **Note:** `lyngio` is a separate library module. It must be explicitly added as a dependency to your host application and initialized in your Lyng scopes. > **Note:** `lyngio` is a separate library module. It must be explicitly added as a dependency to your host application and initialized in your Lyng scopes.
>
> **Important native platform limit:** current native TCP/UDP support is backed by a selector with a per-process file descriptor ceiling. On Linux/macOS native targets this makes high-connection-count servers and same-process load tests unsuitable once the process approaches that limit.
>
> **Recommendation:** for serious HTTP/TCP servers, prefer the JVM target today. On native targets, keep concurrency bounded, batch local load tests in waves, and use multiple worker processes behind a reverse proxy if you need more throughput before the backend is reworked.
>
> **Need this fixed?** Please open or upvote an issue at <https://github.com/sergeych/lyng/issues> so native high-concurrency networking can be prioritized.
--- ---
+110 -12
View File
@@ -1,13 +1,15 @@
### Lyng CLI (`lyng`) # Lyng CLI (`lyng`)
The Lyng CLI is the reference command-line tool for the Lyng language. It lets you: The Lyng CLI is the reference command-line tool for the Lyng language. It lets you:
- Run Lyng scripts from files or inline strings (shebangs accepted) - Run Lyng scripts from files or inline strings (shebangs accepted)
- Use standard argument passing (`ARGV`) to your scripts. - Use standard argument passing (`ARGV`) to your scripts.
- Resolve local file imports from the executed script's directory tree.
- Format Lyng source files via the built-in `fmt` subcommand. - Format Lyng source files via the built-in `fmt` subcommand.
- Register synchronous process-exit handlers with `atExit(...)`.
#### Building on Linux ## Building on Linux
Requirements: Requirements:
- JDK 17+ (for Gradle and the JVM distribution) - JDK 17+ (for Gradle and the JVM distribution)
@@ -19,7 +21,7 @@ The repository provides convenience scripts in `bin/` for local builds and insta
Note: In this repository the scripts are named `bin/local_release` and `bin/local_jrelease`. In some environments these may be aliased as `bin/release` and `bin/jrelease`. The steps below use the actual file names present here. Note: In this repository the scripts are named `bin/local_release` and `bin/local_jrelease`. In some environments these may be aliased as `bin/release` and `bin/jrelease`. The steps below use the actual file names present here.
##### Option A: Native linuxX64 executable (`lyng`) ### Option A: Native linuxX64 executable (`lyng`)
1) Build the native binary: 1) Build the native binary:
@@ -38,26 +40,27 @@ What this does:
- Produces `distributables/lyng-linuxX64.zip` containing the `lyng` executable. - Produces `distributables/lyng-linuxX64.zip` containing the `lyng` executable.
##### Option B: JVM distribution (`jlyng` launcher) ### Option B: JVM distribution (`jlyng` launcher)
This creates a JVM distribution with a launcher script and links it to `~/bin/jlyng`. This creates a JVM distribution with a launcher script, packages it as a downloadable zip, and links it to `~/bin/jlyng`.
``` ```
bin/local_jrelease bin/local_jrelease
``` ```
What this does: What this does:
- Runs `./gradlew :lyng:installJvmDist` to build the JVM app distribution to `lyng/build/install/lyng-jvm`. - Runs `./gradlew :lyng:jvmDistZip` to build the JVM app distribution archive at `lyng/build/distributions/lyng-jvm.zip`.
- Copies the distribution under `~/bin/jlyng-jvm`. - Copies the archive to `distributables/lyng-jvm.zip`.
- Unpacks that distribution under `~/bin/jlyng-jvm`.
- Creates a symlink `~/bin/jlyng` pointing to the launcher script. - Creates a symlink `~/bin/jlyng` pointing to the launcher script.
#### Usage ## Usage
Once installed, ensure `~/bin` is on your `PATH`. You can then use either the native `lyng` or the JVM `jlyng` launcher (both have the same CLI surface). Once installed, ensure `~/bin` is on your `PATH`. You can then use either the native `lyng` or the JVM `jlyng` launcher (both have the same CLI surface).
##### Running scripts ### Running scripts
- Run a script by file name and pass arguments to `ARGV`: - Run a script by file name and pass arguments to `ARGV`:
@@ -72,6 +75,7 @@ lyng -- -my-script.lyng arg1 arg2
``` ```
- Execute inline code with `-x/--execute` and pass positional args to `ARGV`: - Execute inline code with `-x/--execute` and pass positional args to `ARGV`:
- Inline execution does not scan the filesystem for local modules; only file-based execution does.
``` ```
lyng -x "println(\"Hello\")" more args lyng -x "println(\"Hello\")" more args
@@ -84,7 +88,101 @@ lyng --version
lyng --help lyng --help
``` ```
### Use in shell scripts ### Exit handlers: `atExit(...)`
The CLI exposes a CLI-only builtin:
```lyng
extern fun atExit(append: Bool=true, handler: ()->Void)
```
Use it to register synchronous cleanup handlers that should run when the CLI process is leaving.
Semantics:
- `append=true` appends the handler to the end of the queue.
- `append=false` inserts the handler at the front of the queue.
- Handlers run one by one.
- Exceptions thrown by a handler are ignored, and the next handler still runs.
- Handlers are best-effort and run on:
- normal script completion
- script failure
- script `exit(code)`
- process shutdown such as `SIGTERM`
Non-goals:
- `SIGKILL`, hard crashes, and power loss cannot be intercepted.
- `atExit` is currently a CLI feature only; it is not part of the general embedding/runtime surface.
Examples:
```lyng
atExit {
println("closing resources")
}
atExit(false) {
println("runs first")
}
```
### Local imports for file execution
When you execute a script file, the CLI builds a temporary local import manager rooted at the directory that contains the entry script.
Formal structure:
- Root directory: the parent directory of the script passed to `lyng`.
- Scan scope: every `.lyng` file under that root directory, recursively.
- Entry script: the executed file itself is not registered as an importable module.
- Module name mapping: `relative/path/to/file.lyng` maps to import name `relative.path.to.file`.
- Package declaration: if a scanned file starts with `package ...` as its first non-blank line, that package name must exactly match the relative path mapping.
- Package omission: if there is no leading `package` declaration, the CLI uses the relative path mapping as the module name.
- Duplicates: if two files resolve to the same module name, CLI execution fails before script execution starts.
- Import visibility: only files inside the entry root subtree are considered. Parent directories and sibling projects are not searched.
Examples:
```
project/
main.lyng
util/answer.lyng
math/add.lyng
```
`util/answer.lyng` is imported as `import util.answer`.
`math/add.lyng` is imported as `import math.add`.
Example contents:
```lyng
// util/answer.lyng
package util.answer
import math.add
fun answer() = plus(40, 2)
```
```lyng
// math/add.lyng
fun plus(a, b) = a + b
```
```lyng
// main.lyng
import util.answer
println(answer())
```
Rationale:
- The module name is deterministic from the filesystem layout.
- Explicit `package` remains available as a consistency check instead of a second, conflicting naming system.
- The import search space stays local to the executed script, which avoids accidental cross-project resolution.
## Use in shell scripts
Standard unix shebangs (`#!`) are supported, so you can make Lyng scripts directly executable on Unix-like systems. For example: Standard unix shebangs (`#!`) are supported, so you can make Lyng scripts directly executable on Unix-like systems. For example:
@@ -92,7 +190,7 @@ Standard unix shebangs (`#!`) are supported, so you can make Lyng scripts direct
println("Hello, world!") println("Hello, world!")
##### Formatting source: `fmt` subcommand ### Formatting source: `fmt` subcommand
Format Lyng files with the built-in formatter. Format Lyng files with the built-in formatter.
@@ -134,7 +232,7 @@ lyng fmt --spacing --wrap src/file.lyng
``` ```
#### Notes ## Notes
- Both native and JVM distributions expose the same CLI interface. Use whichever best fits your environment. - Both native and JVM distributions expose the same CLI interface. Use whichever best fits your environment.
- When executing scripts, all positional arguments after the script name are available in Lyng as `ARGV`. - When executing scripts, all positional arguments after the script name are available in Lyng as `ARGV`.
+31 -8
View File
@@ -2,6 +2,8 @@
`lyngio` is a separate library that extends the Lyng core (`lynglib`) with powerful, multiplatform, and secure I/O capabilities. `lyngio` is a separate library that extends the Lyng core (`lynglib`) with powerful, multiplatform, and secure I/O capabilities.
> **Important native networking limit:** `lyng.io.net` on current native targets is suitable for modest workloads, local tools, and test servers, but not yet for high-connection-count production servers. For serious HTTP/TCP serving, prefer the JVM target for now. If native high-concurrency networking matters for your use case, please open or upvote an issue at <https://github.com/sergeych/lyng/issues>.
#### Why a separate module? #### Why a separate module?
1. **Security:** I/O and process execution are sensitive operations. By keeping them in a separate module, we ensure that the Lyng core remains 100% safe by default. You only enable what you explicitly need. 1. **Security:** I/O and process execution are sensitive operations. By keeping them in a separate module, we ensure that the Lyng core remains 100% safe by default. You only enable what you explicitly need.
@@ -10,6 +12,7 @@
#### Included Modules #### Included Modules
- **[lyng.io.db](lyng.io.db.md):** Portable SQL database access. Provides `Database`, `SqlTransaction`, `ResultSet`, SQLite support through `lyng.io.db.sqlite`, and JVM JDBC support through `lyng.io.db.jdbc`.
- **[lyng.io.fs](lyng.io.fs.md):** Async filesystem access. Provides the `Path` class for file/directory operations, streaming, and globbing. - **[lyng.io.fs](lyng.io.fs.md):** Async filesystem access. Provides the `Path` class for file/directory operations, streaming, and globbing.
- **[lyng.io.process](lyng.io.process.md):** External process execution and shell commands. Provides `Process`, `RunningProcess`, and `Platform` information. - **[lyng.io.process](lyng.io.process.md):** External process execution and shell commands. Provides `Process`, `RunningProcess`, and `Platform` information.
- **[lyng.io.console](lyng.io.console.md):** Rich console/TTY access. Provides `Console` capability detection, geometry, output, and iterable events. - **[lyng.io.console](lyng.io.console.md):** Rich console/TTY access. Provides `Console` capability detection, geometry, output, and iterable events.
@@ -41,6 +44,9 @@ To use `lyngio` modules in your scripts, you must install them into your Lyng sc
```kotlin ```kotlin
import net.sergeych.lyng.EvalSession import net.sergeych.lyng.EvalSession
import net.sergeych.lyng.io.db.createDbModule
import net.sergeych.lyng.io.db.jdbc.createJdbcModule
import net.sergeych.lyng.io.db.sqlite.createSqliteModule
import net.sergeych.lyng.io.fs.createFs import net.sergeych.lyng.io.fs.createFs
import net.sergeych.lyng.io.process.createProcessModule import net.sergeych.lyng.io.process.createProcessModule
import net.sergeych.lyng.io.console.createConsoleModule import net.sergeych.lyng.io.console.createConsoleModule
@@ -59,6 +65,9 @@ suspend fun runMyScript() {
val scope = session.getScope() val scope = session.getScope()
// Install modules with policies // Install modules with policies
createDbModule(scope)
createJdbcModule(scope)
createSqliteModule(scope)
createFs(PermitAllAccessPolicy, scope) createFs(PermitAllAccessPolicy, scope)
createProcessModule(PermitAllProcessAccessPolicy, scope) createProcessModule(PermitAllProcessAccessPolicy, scope)
createConsoleModule(PermitAllConsoleAccessPolicy, scope) createConsoleModule(PermitAllConsoleAccessPolicy, scope)
@@ -68,6 +77,9 @@ suspend fun runMyScript() {
// Now scripts can import them // Now scripts can import them
session.eval(""" session.eval("""
import lyng.io.db
import lyng.io.db.jdbc
import lyng.io.db.sqlite
import lyng.io.fs import lyng.io.fs
import lyng.io.process import lyng.io.process
import lyng.io.console import lyng.io.console
@@ -75,6 +87,8 @@ suspend fun runMyScript() {
import lyng.io.net import lyng.io.net
import lyng.io.ws import lyng.io.ws
println("H2 JDBC available: " + (openH2("mem:demo;DB_CLOSE_DELAY=-1") != null))
println("SQLite available: " + (openSqlite(":memory:") != null))
println("Working dir: " + Path(".").readUtf8()) println("Working dir: " + Path(".").readUtf8())
println("OS: " + Platform.details().name) println("OS: " + Platform.details().name)
println("TTY: " + Console.isTty()) println("TTY: " + Console.isTty())
@@ -92,6 +106,7 @@ suspend fun runMyScript() {
`lyngio` is built with a "Secure by Default" philosophy. Every I/O or process operation is checked against a policy. `lyngio` is built with a "Secure by Default" philosophy. Every I/O or process operation is checked against a policy.
- **Filesystem Security:** Implement `FsAccessPolicy` to restrict access to specific paths or operations (e.g., read-only access to a sandbox directory). - **Filesystem Security:** Implement `FsAccessPolicy` to restrict access to specific paths or operations (e.g., read-only access to a sandbox directory).
- **Database Installation:** Database access is still explicit-capability style. The host must install `lyng.io.db` and at least one provider such as `lyng.io.db.sqlite` or `lyng.io.db.jdbc`; otherwise scripts cannot open databases.
- **Process Security:** Implement `ProcessAccessPolicy` to restrict which executables can be run or to disable shell execution entirely. - **Process Security:** Implement `ProcessAccessPolicy` to restrict which executables can be run or to disable shell execution entirely.
- **Console Security:** Implement `ConsoleAccessPolicy` to control output writes, event reads, and raw mode switching. - **Console Security:** Implement `ConsoleAccessPolicy` to control output writes, event reads, and raw mode switching.
- **HTTP Security:** Implement `HttpAccessPolicy` to restrict which requests scripts may send. - **HTTP Security:** Implement `HttpAccessPolicy` to restrict which requests scripts may send.
@@ -99,6 +114,7 @@ suspend fun runMyScript() {
- **WebSocket Security:** Implement `WsAccessPolicy` to restrict websocket connects and message flow. - **WebSocket Security:** Implement `WsAccessPolicy` to restrict websocket connects and message flow.
For more details, see the specific module documentation: For more details, see the specific module documentation:
- [Database Module Details](lyng.io.db.md)
- [Filesystem Security Details](lyng.io.fs.md#access-policy-security) - [Filesystem Security Details](lyng.io.fs.md#access-policy-security)
- [Process Security Details](lyng.io.process.md#security-policy) - [Process Security Details](lyng.io.process.md#security-policy)
- [Console Module Details](lyng.io.console.md) - [Console Module Details](lyng.io.console.md)
@@ -110,11 +126,18 @@ For more details, see the specific module documentation:
#### Platform Support Overview #### Platform Support Overview
| Platform | lyng.io.fs | lyng.io.process | lyng.io.console | lyng.io.http | lyng.io.ws | lyng.io.net | | Platform | lyng.io.db/sqlite | lyng.io.db/jdbc | lyng.io.fs | lyng.io.process | lyng.io.console | lyng.io.http | lyng.io.ws | lyng.io.net |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **JVM** | ✅ | ✅ | ✅ (baseline) | ✅ | ✅ | ✅ | | **JVM** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| **Native (Linux/macOS)** | ✅ | ✅ | 🚧 | 🚧 | 🚧 | 🚧 | | **Linux Native** | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| **Native (Windows)** | ✅ | 🚧 (Planned) | 🚧 | 🚧 | 🚧 | 🚧 | | **Apple Native** | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ⚠️ |
| **Android** | ✅ | ❌ | ❌ | 🚧 | 🚧 | 🚧 | | **Windows Native** | ❌ | ❌ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ |
| **NodeJS** | ✅ | ❌ | ❌ | 🚧 | 🚧 | 🚧 | | **Android** | ⚠️ | ❌ | ✅ | ❌ | ⚠️ | ✅ | ✅ | ✅ |
| **Browser / Wasm** | ✅ (In-memory) | ❌ | ❌ | 🚧 | 🚧 | 🚧 | | **JS / Node** | ❌ | ❌ | ✅ | ❌ | ⚠️ | ✅ | ✅ | ✅ |
| **JS / Browser** | ❌ | ❌ | ✅ (in-memory) | ❌ | ⚠️ | ✅ | ✅ | ❌ |
| **Wasm** | ❌ | ❌ | ✅ (in-memory) | ❌ | ⚠️ | ❌ | ❌ | ❌ |
Legend:
- `✅` supported
- `⚠️` available but environment-dependent or not fully host-verified yet
- `❌` unsupported
+78
View File
@@ -36,6 +36,17 @@ This example shows how to launch a coroutine with `launch` which returns [Deferr
Launch has the only argument which should be a callable (lambda usually) that is run in parallel (or cooperatively in parallel), and return anything as the result. Launch has the only argument which should be a callable (lambda usually) that is run in parallel (or cooperatively in parallel), and return anything as the result.
When you have an iterable of deferreds, use `joinAll()` to await all of them and collect results in input order:
val jobs = (1..4).map { n ->
launch {
delay(1)
n * 10
}
}
assertEquals([10, 20, 30, 40], jobs.joinAll())
>>> void
If you no longer need the result, cancel the deferred. Awaiting a cancelled deferred throws `CancellationException`: If you no longer need the result, cancel the deferred. Awaiting a cancelled deferred throws `CancellationException`:
var reached = false var reached = false
@@ -219,6 +230,73 @@ Flows allow easy transforming of any [Iterable]. See how the standard Lyng libra
} }
} }
## Channel
A [Channel] is a **hot pipe** between coroutines: values are pushed in by a producer and pulled out by a consumer, with each value consumed exactly once.
Unlike a `Flow` (which is cold and re-runs its generator on every collection), a `Channel` is stateful — the right tool for classic _producer / consumer_ work.
val ch = Channel() // rendezvous: sender waits for receiver
val producer = launch {
for (i in 1..5) ch.send(i)
ch.close() // signal: no more values
}
var item = ch.receive() // suspends until a value is ready
while (item != null) {
println(item)
item = ch.receive()
}
// prints 1 2 3 4 5
`receive()` returns `null` when the channel is both closed and fully drained — that is the idiomatic loop termination condition.
Channels can also be buffered so the producer can run ahead:
val ch = Channel(4) // buffer up to 4 items without blocking
ch.send(10)
ch.send(20)
ch.send(30)
ch.close()
assertEquals(10, ch.receive())
assertEquals(20, ch.receive())
assertEquals(30, ch.receive())
assertEquals(null, ch.receive()) // drained
For the full API — including `tryReceive`, `Channel.UNLIMITED`, and the fan-out / ping-pong patterns — see the [Channel] reference page.
## LaunchPool
When you need **bounded concurrency** — run at most *N* tasks at the same time without spawning a new coroutine per task — use [LaunchPool]:
```lyng
val pool = LaunchPool(4) // at most 4 tasks run in parallel
val jobs = (1..20).map { n ->
pool.launch { expensiveCompute(n) }
}
pool.closeAndJoin() // wait for all tasks to complete
val results = jobs.joinAll()
```
Exceptions thrown inside a submitted lambda are captured in the returned `Deferred` and do not crash the pool, so other tasks continue running normally.
See [LaunchPool] for the full API including bounded queues and cancellation.
[LaunchPool]: LaunchPool.md
| | Flow | Channel |
|---|---|---|
| **temperature** | cold (lazy) | hot (eager) |
| **replay** | every collector gets a fresh run | each item consumed once |
| **consumers** | any number, each gets all items | one receiver per item |
| **typical use** | transform pipelines, sequences | producer–consumer, fan-out |
[Channel]: Channel.md
[Iterable]: Iterable.md [Iterable]: Iterable.md
+62
View File
@@ -0,0 +1,62 @@
## Pi Spigot JVM Baseline
Saved on April 4, 2026 before the `List<Int>` indexed-access follow-up fix.
Benchmark target:
- [examples/pi-bench.py](/home/sergeych/dev/lyng/examples/pi-bench.py)
- [examples/pi-bench.lyng](/home/sergeych/dev/lyng/examples/pi-bench.lyng)
Execution path:
- Python: `python3 examples/pi-bench.py`
- Lyng JVM: `./gradlew :lyng:runJvm --args='/home/sergeych/dev/lyng/examples/pi-bench.lyng'`
- Constraint: do not use Kotlin/Native `lyng` CLI for perf comparisons
Baseline measurements:
- Python full script: `167 ms`
- Lyng JVM full script: `1.287097604 s`
- Python warm function average over 5 runs: `126.126 ms`
- Lyng JVM warm function average over 5 runs: about `1071.6 ms`
Baseline ratio:
- Full script: about `7.7x` slower on Lyng JVM
- Warm function only: about `8.5x` slower on Lyng JVM
Primary finding at baseline:
- The hot `reminders[j]` accesses in `piSpigot` were still lowered through boxed object index ops and boxed arithmetic.
- Newly added `GET_INDEX_INT` and `SET_INDEX_INT` only reached `pi`, not `reminders`.
- Root cause: initializer element inference handled list literals, but not `List.fill(boxes) { 2 }`, so `reminders` did not become known `List<Int>` at compile time.
## After Optimizations 1-4
Follow-up change:
- propagate inferred lambda return class into bytecode compilation
- infer `List.fill(...)` element type from the fill lambda
- lower `reminders[j]` reads and writes to `GET_INDEX_INT` and `SET_INDEX_INT`
- add primitive-backed `ObjList` storage for all-int lists
- lower `List.fill(Int) { Int }` to `LIST_FILL_INT`
- stop boxing the integer index inside `GET_INDEX_INT` / `SET_INDEX_INT`
Verification:
- `piSpigot` disassembly now contains typed ops for `reminders`, for example:
- `GET_INDEX_INT s5(reminders), s10(j), ...`
- `SET_INDEX_INT s5(reminders), s10(j), ...`
Post-change measurements using `jlyng`:
- Full script: `655.819559 ms`
- Warm 5-run total: `1.430945810 s`
- Warm average per run: about `286.2 ms`
Observed improvement vs baseline:
- Full script: about `1.96x` faster (`1.287 s -> 0.656 s`)
- Warm function: about `3.74x` faster (`1071.6 ms -> 286.2 ms`)
Residual gap vs Python baseline:
- Full script: Lyng JVM is still about `3.9x` slower than Python (`655.8 ms` vs `167 ms`)
- Warm function: Lyng JVM is still about `2.3x` slower than Python (`286.2 ms` vs `126.126 ms`)
Current benchmark-test snapshot (`n=200`, JVM test harness):
- `optimized-int-division-rval-off`: `135 ms`
- `optimized-int-division-rval-on`: `125 ms`
- `piSpigot` bytecode now contains:
- `LIST_FILL_INT` for both `pi` and `reminders`
- `GET_INDEX_INT` / `SET_INDEX_INT` for the hot indexed loop
+1
View File
@@ -6,6 +6,7 @@ This page documents the **current** rules: static name resolution, closure captu
## Current rules (bytecode compiler) ## Current rules (bytecode compiler)
- **All names resolve at compile time**: locals, parameters, captures, members, imports, and module globals must be known when compiling. Missing symbols are compile-time errors. - **All names resolve at compile time**: locals, parameters, captures, members, imports, and module globals must be known when compiling. Missing symbols are compile-time errors.
- **Exception: `compile if` can skip dead branches**: inside an untaken `compile if (...)` branch, names are not resolved or type-checked at all. This is the supported way to guard optional classes or packages such as `defined(Udp)` or `defined(lyng.io.net)`.
- **No runtime fallbacks**: there is no dynamic name lookup, no fallback opcodes, and no “search parent scopes” at runtime for missing names. - **No runtime fallbacks**: there is no dynamic name lookup, no fallback opcodes, and no “search parent scopes” at runtime for missing names.
- **Object members on unknown types only**: `toString`, `toInspectString`, `let`, `also`, `apply`, `run` are allowed on unknown types; all other members require a statically known receiver type or an explicit cast. - **Object members on unknown types only**: `toString`, `toInspectString`, `let`, `also`, `apply`, `run` are allowed on unknown types; all other members require a statically known receiver type or an explicit cast.
- **Closures capture slots**: lambdas and nested functions capture **frame slots** directly. Captures are resolved at compile time and compiled to slot references. - **Closures capture slots**: lambdas and nested functions capture **frame slots** directly. Captures are resolved at compile time and compiled to slot references.
+122 -63
View File
@@ -1,74 +1,135 @@
# Lyng time functions # Lyng time functions
Lyng date and time support requires importing `lyng.time` packages. Lyng uses simple yet modern time object models: Lyng date and time support requires importing `lyng.time`. The module provides four related types:
- `Instant` class for absolute time stamps with platform-dependent resolution. - `Instant` for absolute timestamps.
- `DateTime` class for calendar-aware points in time within a specific time zone. - `Date` for calendar dates without time-of-day or timezone.
- `Duration` to represent amount of time not depending on the calendar (e.g., milliseconds, seconds). - `DateTime` for calendar-aware points in time in a specific timezone.
- `Duration` for absolute elapsed time.
## Time instant: `Instant` ## Time instant: `Instant`
Represent some moment of time not depending on the calendar. It is similar to `TIMESTAMP` in SQL or `Instant` in Kotlin. `Instant` represents some moment of time independently of the calendar. It is similar to SQL `TIMESTAMP`
or Kotlin `Instant`.
### Constructing and converting ### Constructing and converting
import lyng.time import lyng.time
// default constructor returns time now:
val t1 = Instant() val t1 = Instant()
val t2 = Instant(1704110400)
// constructing from a number is treated as seconds since unix epoch:
val t2 = Instant(1704110400) // 2024-01-01T12:00:00Z
// from RFC3339 string:
val t3 = Instant("2024-01-01T12:00:00.123456Z") val t3 = Instant("2024-01-01T12:00:00.123456Z")
// truncation: val t4 = t3.truncateToMinute()
val t4 = t3.truncateToMinute assertEquals("2024-01-01T12:00:00Z", t4.toRFC3339())
assertEquals(t4.toRFC3339(), "2024-01-01T12:00:00Z")
// to localized DateTime (uses system default TZ if not specified):
val dt = t3.toDateTime("+02:00") val dt = t3.toDateTime("+02:00")
assertEquals(dt.hour, 14) assertEquals(14, dt.hour)
val d = t3.toDate("Z")
assertEquals(Date(2024, 1, 1), d)
### Instant members ### Instant members
| member | description | | member | description |
|--------------------------------|---------------------------------------------------------| |--------------------------------|------------------------------------------------------|
| epochSeconds: Real | positive or negative offset in seconds since Unix epoch | | epochSeconds: Real | offset in seconds since Unix epoch |
| epochWholeSeconds: Int | same, but in _whole seconds_. Slightly faster | | epochWholeSeconds: Int | whole seconds since Unix epoch |
| nanosecondsOfSecond: Int | offset from epochWholeSeconds in nanos | | nanosecondsOfSecond: Int | nanoseconds within the current second |
| isDistantFuture: Bool | true if it `Instant.distantFuture` | | isDistantFuture: Bool | true if it is `Instant.distantFuture` |
| isDistantPast: Bool | true if it `Instant.distantPast` | | isDistantPast: Bool | true if it is `Instant.distantPast` |
| truncateToMinute: Instant | create new instance truncated to minute | | truncateToMinute(): Instant | truncate to minute precision |
| truncateToSecond: Instant | create new instance truncated to second | | truncateToSecond(): Instant | truncate to second precision |
| truncateToMillisecond: Instant | truncate new instance to millisecond | | truncateToMillisecond(): Instant | truncate to millisecond precision |
| truncateToMicrosecond: Instant | truncate new instance to microsecond | | truncateToMicrosecond(): Instant | truncate to microsecond precision |
| toRFC3339(): String | format as RFC3339 string (UTC) | | toRFC3339(): String | format as RFC3339 string in UTC |
| toDateTime(tz?): DateTime | localize to a TimeZone (ID string or offset seconds) | | toDateTime(tz?): DateTime | localize to a timezone |
| toDate(tz?): Date | convert to a calendar date in a timezone |
## Calendar time: `DateTime` ## Calendar date: `Date`
`DateTime` represents a point in time in a specific timezone. It provides access to calendar components like year, `Date` represents a pure calendar date. It has no time-of-day and no attached timezone. Use it for values
month, and day. like birthdays, due dates, invoice dates, and SQL `DATE` columns.
### Constructing
import lyng.time
val today = Date()
val d1 = Date(2026, 4, 15)
val d2 = Date("2024-02-29")
val d3 = Date.parseIso("2024-02-29")
val d4 = Date(DateTime(2024, 5, 20, 15, 30, 45, "+02:00"))
val d5 = Date(Instant("2024-01-01T23:30:00Z"), "+02:00")
### Date members
| member | description |
|--------------------------------|------------------------------------------------------------|
| year: Int | year component |
| month: Int | month component (1..12) |
| day: Int | day of month (alias `dayOfMonth`) |
| dayOfWeek: Int | day of week (1=Monday, 7=Sunday) |
| dayOfYear: Int | day of year (1..365/366) |
| isLeapYear: Bool | whether this date is in a leap year |
| lengthOfMonth: Int | number of days in this month |
| lengthOfYear: Int | 365 or 366 |
| toIsoString(): String | ISO `YYYY-MM-DD` string |
| toSortableString(): String | alias to `toIsoString()` |
| toDateTime(tz="Z"): DateTime | start-of-day `DateTime` in the specified timezone |
| atStartOfDay(tz="Z"): DateTime | alias to `toDateTime()` |
| addDays(n): Date | add or subtract calendar days |
| addMonths(n): Date | add or subtract months, normalizing end-of-month |
| addYears(n): Date | add or subtract years |
| daysUntil(other): Int | calendar days until `other` |
| daysSince(other): Int | calendar days since `other` |
| static today(tz?): Date | today in the specified timezone |
| static parseIso(s): Date | parse ISO `YYYY-MM-DD` |
### Date arithmetic
`Date` supports only whole-day arithmetic. This is deliberate: calendar dates should not silently accept
sub-day durations.
import lyng.time
val d1 = Date(2026, 4, 15)
val d2 = d1.addDays(10)
assertEquals(Date(2026, 4, 25), d2)
assertEquals(Date(2026, 4, 18), d1 + 3.days)
assertEquals(Date(2026, 4, 12), d1 - 3.days)
assertEquals(10, d1.daysUntil(d2))
assertEquals(10, d2.daysSince(d1))
assertEquals(10, d2 - d1)
### Date conversions
import lyng.time
val i = Instant("2024-01-01T23:30:00Z")
assertEquals(Date(2024, 1, 1), i.toDate("Z"))
assertEquals(Date(2024, 1, 2), i.toDate("+02:00"))
val dt = DateTime(2024, 5, 20, 15, 30, 45, "+02:00")
assertEquals(Date(2024, 5, 20), dt.date)
assertEquals(Date(2024, 5, 20), dt.toDate())
assertEquals(DateTime(2024, 5, 20, 0, 0, 0, "Z"), Date(2024, 5, 20).toDateTime("Z"))
assertEquals(DateTime(2024, 5, 20, 0, 0, 0, "+02:00"), Date(2024, 5, 20).atStartOfDay("+02:00"))
## Calendar time: `DateTime`
`DateTime` represents a point in time in a specific timezone. It provides access to calendar components
such as year, month, day, and hour.
### Constructing ### Constructing
import lyng.time import lyng.time
// Current time in system default timezone
val now = DateTime.now() val now = DateTime.now()
// Specific timezone
val offsetTime = DateTime.now("+02:00") val offsetTime = DateTime.now("+02:00")
// From Instant
val dt = Instant().toDateTime("Z") val dt = Instant().toDateTime("Z")
// By components (year, month, day, hour=0, minute=0, second=0, timeZone="UTC")
val dt2 = DateTime(2024, 1, 1, 12, 0, 0, "Z") val dt2 = DateTime(2024, 1, 1, 12, 0, 0, "Z")
// From RFC3339 string
val dt3 = DateTime.parseRFC3339("2024-01-01T12:00:00+02:00") val dt3 = DateTime.parseRFC3339("2024-01-01T12:00:00+02:00")
### DateTime members ### DateTime members
@@ -83,7 +144,9 @@ month, and day.
| second: Int | second component (0..59) | | second: Int | second component (0..59) |
| dayOfWeek: Int | day of week (1=Monday, 7=Sunday) | | dayOfWeek: Int | day of week (1=Monday, 7=Sunday) |
| timeZone: String | timezone ID string | | timeZone: String | timezone ID string |
| date: Date | calendar date component |
| toInstant(): Instant | convert back to absolute Instant | | toInstant(): Instant | convert back to absolute Instant |
| toDate(): Date | extract the calendar date in this timezone |
| toUTC(): DateTime | shortcut to convert to UTC | | toUTC(): DateTime | shortcut to convert to UTC |
| toTimeZone(tz): DateTime | convert to another timezone | | toTimeZone(tz): DateTime | convert to another timezone |
| addMonths(n): DateTime | add/subtract months (normalizes end of month) | | addMonths(n): DateTime | add/subtract months (normalizes end of month) |
@@ -96,28 +159,27 @@ month, and day.
`DateTime` handles calendar arithmetic correctly: `DateTime` handles calendar arithmetic correctly:
import lyng.time
val leapDay = Instant("2024-02-29T12:00:00Z").toDateTime("Z") val leapDay = Instant("2024-02-29T12:00:00Z").toDateTime("Z")
val nextYear = leapDay.addYears(1) val nextYear = leapDay.addYears(1)
assertEquals(nextYear.day, 28) // Feb 29, 2024 -> Feb 28, 2025 assertEquals(28, nextYear.day)
# `Duration` class # `Duration` class
Represent absolute time distance between two `Instant`. `Duration` represents absolute elapsed time between two instants.
import lyng.time import lyng.time
val t1 = Instant() val t1 = Instant()
// yes we can delay to period, and it is not blocking. is suspends!
delay(1.millisecond) delay(1.millisecond)
val t2 = Instant() val t2 = Instant()
// be suspend, so actual time may vary:
assert( t2 - t1 >= 1.millisecond) assert(t2 - t1 >= 1.millisecond)
assert( t2 - t1 < 100.millisecond) assert(t2 - t1 < 100.millisecond)
>>> void >>> void
Duration can be converted from numbers, like `5.minutes` and so on. Extensions are created for Duration values can be created from numbers using extensions on `Int` and `Real`:
`Int` and `Real`, so for n as Real or Int it is possible to create durations::
- `n.millisecond`, `n.milliseconds` - `n.millisecond`, `n.milliseconds`
- `n.second`, `n.seconds` - `n.second`, `n.seconds`
@@ -125,10 +187,9 @@ Duration can be converted from numbers, like `5.minutes` and so on. Extensions a
- `n.hour`, `n.hours` - `n.hour`, `n.hours`
- `n.day`, `n.days` - `n.day`, `n.days`
The bigger time units like months or years are calendar-dependent and can't be used with `Duration`. Larger units like months or years are calendar-dependent and are intentionally not part of `Duration`.
Each duration instance can be converted to number of any of these time units, as `Real` number, if `d` is a `Duration` Each duration instance can be converted to numbers in these units:
instance:
- `d.microseconds` - `d.microseconds`
- `d.milliseconds` - `d.milliseconds`
@@ -137,18 +198,16 @@ instance:
- `d.hours` - `d.hours`
- `d.days` - `d.days`
for example Example:
import lyng.time import lyng.time
assertEquals( 60, 1.minute.seconds )
assertEquals( 10.milliseconds, 0.01.seconds )
assertEquals(60, 1.minute.seconds)
assertEquals(10.milliseconds, 0.01.seconds)
>>> void >>> void
# Utility functions # Utility functions
## delay(duration: Duration) ## `delay(duration: Duration)`
Suspends current coroutine for at least the specified duration.
Suspends the current coroutine for at least the specified duration.
+139 -2
View File
@@ -811,6 +811,12 @@ Lyng has built-in mutable array class `List` with simple literals:
many collection based methods are implemented there. many collection based methods are implemented there.
For immutable list values, use `list.toImmutable()` and [ImmutableList]. For immutable list values, use `list.toImmutable()` and [ImmutableList].
To construct a list programmatically, use the static helper `List.fill`:
val tens = List.fill(5) { index -> index * 10 }
assertEquals([0, 10, 20, 30, 40], tens)
>>> void
Lists can contain any type of objects, lists too: Lists can contain any type of objects, lists too:
val list = [1, [2, 3], 4] val list = [1, [2, 3], 4]
@@ -1088,6 +1094,37 @@ Or, more neat:
>>> just 3 >>> just 3
>>> void >>> void
## compile if
`compile if` is a compile-time conditional. Unlike normal `if`, the compiler evaluates its condition while compiling
the file and completely skips the untaken branch. This is useful when some class or package may or may not be
available:
compile if (defined(Udp)) {
val socket = Udp()
println("udp is available")
} else {
println("udp is not available")
}
`compile if` also supports single-statement branches:
compile if (defined(lyng.io.net) && !defined(Udp))
println("network module exists, but Udp is not visible here")
else
println("either Udp exists or the module is unavailable")
Current condition syntax is intentionally limited to compile-time symbol checks:
- `defined(Name)`
- `defined(package.name)`
- `!`, `&&`, `||`
- parentheses
Examples:
compile if (defined(Udp) && defined(Tcp))
println("both transports are available")
## When ## When
See also: [Comprehensive guide to `when`](when.md) See also: [Comprehensive guide to `when`](when.md)
@@ -1359,6 +1396,41 @@ size and index access, like lists:
"total letters: "+letters "total letters: "+letters
>>> "total letters: 10" >>> "total letters: 10"
When you need a counting loop that goes backwards, use an explicit descending
range:
var sum = 0
for( i in 5 downTo 1 ) {
sum += i
}
sum
>>> 15
If the lower bound should be excluded, use `downUntil`:
val xs = []
for( i in 5 downUntil 1 ) {
xs.add(i)
}
xs
>>> [5,4,3,2]
This is intentionally explicit: `5..1` is an empty ascending range, not an
implicit reverse loop.
Descending loops also support `step`:
val xs = []
for( i in 10 downTo 1 step 3 ) {
xs.add(i)
}
xs
>>> [10,7,4,1]
For descending ranges, `step` stays positive. The direction comes from
`downTo` / `downUntil`, so `10 downTo 1 step 3` is valid, while
`10 downTo 1 step -3` is an error.
For loop support breaks the same as while loops above: For loop support breaks the same as while loops above:
fun search(haystack, needle) { fun search(haystack, needle) {
@@ -1488,6 +1560,12 @@ It could be open and closed:
assert( 5 !in (1..<5) ) assert( 5 !in (1..<5) )
>>> void >>> void
Descending ranges are explicit too:
assertEquals([5,4,3,2,1], (5 downTo 1).toList())
assertEquals([5,4,3,2], (5 downUntil 1).toList())
>>> void
Ranges could be inside other ranges: Ranges could be inside other ranges:
assert( (2..3) in (1..10) ) assert( (2..3) in (1..10) )
@@ -1500,11 +1578,19 @@ There are character ranges too:
and you can use ranges in for-loops: and you can use ranges in for-loops:
for( x in 'a' ..< 'c' ) println(x) for( x in 'a'..<'c' ) println(x)
>>> a >>> a
>>> b >>> b
>>> void >>> void
Descending character ranges work the same way:
for( ch in 'e' downTo 'a' step 2 ) println(ch)
>>> e
>>> c
>>> a
>>> void
See [Ranges](Range.md) for detailed documentation on it. See [Ranges](Range.md) for detailed documentation on it.
# Time routines # Time routines
@@ -1568,15 +1654,27 @@ The type for the character objects is `Char`.
### String literal escapes ### String literal escapes
Lyng string literals can use either double quotes or backticks:
val a = "hello"
val b = `hello`
assert(a == b)
| escape | ASCII value | | escape | ASCII value |
|--------|-----------------------| |--------|-----------------------|
| \n | 0x10, newline | | \n | 0x10, newline |
| \r | 0x13, carriage return | | \r | 0x13, carriage return |
| \t | 0x07, tabulation | | \t | 0x07, tabulation |
| \\ | \ slash character | | \\ | \ slash character |
| \" | " double quote |
| \uXXXX | unicode code point | | \uXXXX | unicode code point |
Delimiter-specific escapes:
| form | escape | value |
|--------|--------|------------------|
| `"..."` | \" | " double quote |
| `` `...` `` | \` | ` backtick |
Unicode escape form is exactly 4 hex digits, e.g. `"\u263A"` -> `☺`. Unicode escape form is exactly 4 hex digits, e.g. `"\u263A"` -> `☺`.
Other `\c` combinations, where c is any char except mentioned above, are left intact, e.g.: Other `\c` combinations, where c is any char except mentioned above, are left intact, e.g.:
@@ -1609,10 +1707,15 @@ Example:
val name = "Lyng" val name = "Lyng"
assertEquals("hello, Lyng!", "hello, $name!") assertEquals("hello, Lyng!", "hello, $name!")
assertEquals("hello, Lyng!", `hello, $name!`)
assertEquals("sum=3", "sum=${1+2}") assertEquals("sum=3", "sum=${1+2}")
assertEquals("sum=3", `sum=${1+2}`)
assertEquals("\$name", "\$name") assertEquals("\$name", "\$name")
assertEquals("\$name", "$$name") assertEquals("\$name", "$$name")
assertEquals("\$name", `\$name`)
assertEquals("\$name", `$$name`)
assertEquals("\\Lyng", "\\$name") assertEquals("\\Lyng", "\\$name")
assertEquals("\\Lyng", `\\$name`)
>>> void >>> void
Interpolation and `printf`-style formatting can be combined when needed: Interpolation and `printf`-style formatting can be combined when needed:
@@ -1747,6 +1850,14 @@ Part match:
assert( "foo" == ($~ as RegexMatch).value ) assert( "foo" == ($~ as RegexMatch).value )
>>> void >>> void
Replacing text:
assertEquals("bonono", "banana".replace('a', 'o'))
assertEquals("a-b-c", "a.b.c".replace(".", "-")) // string patterns are literal
assertEquals("v#.#.#", "v1.2.3".replace("\d+".re, "#"))
assertEquals("v[1].[2].[3]", "v1.2.3".replace("(\d+)".re) { m -> "[" + m[1] + "]" })
>>> void
Repeating the fragment: Repeating the fragment:
assertEquals("hellohello", "hello"*2) assertEquals("hellohello", "hello"*2)
@@ -1782,6 +1893,8 @@ A typical set of String functions includes:
| characters | create [List] of characters (1) | | characters | create [List] of characters (1) |
| encodeUtf8() | returns [Buffer] with characters encoded to utf8 | | encodeUtf8() | returns [Buffer] with characters encoded to utf8 |
| matches(re) | matches the regular expression (2) | | matches(re) | matches the regular expression (2) |
| replace(old, new) | replace all literal or regex matches; regex needs [Regex] |
| replaceFirst(old,new)| replace the first literal or regex match |
| | | | | |
(1) (1)
@@ -2008,6 +2121,30 @@ Example with custom accessors:
"abc".firstChar "abc".firstChar
>>> 'a' >>> 'a'
### Extension indexers
Indexers can also be extended by overriding `getAt` and `putAt` on the receiver:
```lyng
object Storage
var storageData = {}
override fun Storage.getAt(key: String): Object? {
storageData[key]
}
override fun Storage.putAt(key: String, value: Object) {
storageData[key] = value
}
Storage["answer"] = 42
val answer: Int? = Storage["answer"]
assertEquals(42, answer)
```
This works for classes and named singleton `object` declarations. Bracket syntax is lowered to `getAt` / `putAt`, and multiple selectors are packed into one list-like index object the same way as other custom indexers.
Extension members are **scope-isolated**: they are visible only in the scope where they are defined and its children. This prevents name collisions and improves security. Extension members are **scope-isolated**: they are visible only in the scope where they are defined and its children. This prevents name collisions and improves security.
To get details on OOP in Lyng, see [OOP notes](OOP.md). To get details on OOP in Lyng, see [OOP notes](OOP.md).
+256 -3
View File
@@ -1,10 +1,81 @@
# What's New in Lyng # What's New in Lyng
This document highlights the latest additions and improvements to the Lyng language and its ecosystem. This document highlights the current Lyng release, **1.5.5**, and the broader additions from the 1.5 cycle.
For a programmer-focused migration summary, see `docs/whats_new_1_5.md`. It is intentionally user-facing: new language features, new modules, new tools, and the practical things you can build with them.
For a programmer-focused migration summary across 1.5.x, see `docs/whats_new_1_5.md`.
## Release 1.5.5 Highlights
- `1.5.5` extends the 1.5 line with practical database APIs, first-class calendar dates, and better coroutine building blocks.
- The 1.5 line now brings together richer ranges and loops, interpolation, math modules, immutable and observable collections, richer `lyngio`, and much better CLI/IDE support.
- `1.5.5` adds `Channel`, `LaunchPool`, and `joinAll()` so coroutine-heavy scripts can coordinate work more directly.
- `1.5.5` adds `Date`, the portable `lyng.io.db` layer, SQLite/JDBC providers, and a compatibility `lyng.legacy_digest` module.
- `1.5.5` also continues runtime/compiler hardening with better import dispatch, faster exact lambda calls, and correct `val +=`/`-=` behavior for mutating types versus real reassignment.
- The docs, homepage samples, and release metadata now point at the current stable version.
## User Highlights Across 1.5.x
- Descending ranges and loops with `downTo` / `downUntil`
- String interpolation with `$name` and `${expr}`
- Backtick string literals for raw-ish string text
- Decimal arithmetic, matrices/vectors, and complex numbers
- Calendar `Date` support in `lyng.time`
- `Channel`, `LaunchPool`, and `joinAll()` for coroutine workflows
- Immutable collections and opt-in `ObservableList`
- Rich `lyngio` modules for SQLite/JDBC databases, console, HTTP, WebSocket, TCP, and UDP
- Legacy SHA-1 compatibility helpers in `lyng.legacy_digest`
- CLI improvements including the built-in formatter `lyng fmt`
- Better IDE support and stronger docs around the released feature set
## Language Features ## Language Features
### Descending Ranges and Loops
Lyng ranges are no longer just ascending. You can now write explicit descending ranges with inclusive or exclusive lower bounds.
```lyng
assertEquals([5,4,3,2,1], (5 downTo 1).toList())
assertEquals([5,4,3,2], (5 downUntil 1).toList())
for (i in 10 downTo 1 step 3) {
println(i)
}
```
This also works for characters:
```lyng
assertEquals(['e','c','a'], ('e' downTo 'a' step 2).toList())
```
See [Range](Range.md).
### String Interpolation
Lyng 1.5.1 added built-in string interpolation:
- `$name`
- `${expr}`
Literal dollar forms are explicit too:
- `\$` -> `$`
- `$$` -> `$`
```lyng
val name = "Lyng"
assertEquals("hello, Lyng!", "hello, $name!")
assertEquals("sum=3", "sum=${1+2}")
assertEquals("\$name", "\$name")
assertEquals("\$name", "$$name")
```
If you need legacy literal-dollar behavior in a file, add:
```lyng
// feature: interpolation: off
```
See [Tutorial](tutorial.md).
### Matrix and Vector Module (`lyng.matrix`) ### Matrix and Vector Module (`lyng.matrix`)
Lyng now ships a dense linear algebra module with immutable double-precision `Matrix` and `Vector` types. Lyng now ships a dense linear algebra module with immutable double-precision `Matrix` and `Vector` types.
@@ -82,6 +153,46 @@ The distinction between `Real -> Decimal` and exact decimal parsing is explicit
See [Decimal](Decimal.md). See [Decimal](Decimal.md).
### Complex Numbers (`lyng.complex`)
Lyng also ships a complex-number module for ordinary arithmetic in the complex plane.
```lyng
import lyng.complex
assertEquals(Complex(1.0, 2.0), 1 + 2.i)
assertEquals(Complex(2.0, 2.0), 2.i + 2)
val z = 1 + π.i
println(z.exp())
```
See [Complex](Complex.md).
### Legacy Digest Module (`lyng.legacy_digest`)
For situations where an external protocol or file format requires a SHA-1 value,
Lyng now ships a `lyng.legacy_digest` module backed by a pure Kotlin/KMP
implementation with no extra dependencies.
> ⚠️ SHA-1 is cryptographically broken. Use only for legacy-compatibility work.
```lyng
import lyng.legacy_digest
val hex = LegacyDigest.sha1("abc")
// → "a9993e364706816aba3e25717850c26c9cd0d89d"
// Also accepts raw bytes:
import lyng.buffer
val buf = Buffer.decodeHex("616263")
assertEquals(hex, LegacyDigest.sha1(buf))
```
The name `LegacyDigest` is intentional: it signals that these algorithms belong
to a compatibility layer, not to a current security toolkit.
See [LegacyDigest](LegacyDigest.md).
### Binary Operator Interop Registry ### Binary Operator Interop Registry
Lyng now provides a general mechanism for mixed binary operators through `lyng.operators`. Lyng now provides a general mechanism for mixed binary operators through `lyng.operators`.
@@ -216,13 +327,30 @@ Singleton objects are declared using the `object` keyword. They provide a conven
```lyng ```lyng
object Config { object Config {
val version = "1.5.0-SNAPSHOT" val version = "1.5.5"
fun show() = println("Config version: " + version) fun show() = println("Config version: " + version)
} }
Config.show() Config.show()
``` ```
Named singleton objects can also be used as extension receivers:
```lyng
object X {
fun base() = "base"
}
fun X.decorate(value): String {
this.base() + ":" + value.toString()
}
val X.tag get() = this.base() + ":tag"
assertEquals("base:42", X.decorate(42))
assertEquals("base:tag", X.tag)
```
### Nested Declarations and Lifted Enums ### Nested Declarations and Lifted Enums
You can now declare classes, objects, enums, and type aliases inside another class. These nested declarations live in the class namespace (no outer instance capture) and are accessed with a qualifier. You can now declare classes, objects, enums, and type aliases inside another class. These nested declarations live in the class namespace (no outer instance capture) and are accessed with a qualifier.
@@ -339,8 +467,124 @@ x.clamp(0..10) // returns 10
`clamp()` correctly handles inclusive (`..`) and exclusive (`..<`) ranges. For discrete types like `Int` and `Char`, clamping to an exclusive upper bound returns the previous value. `clamp()` correctly handles inclusive (`..`) and exclusive (`..<`) ranges. For discrete types like `Int` and `Char`, clamping to an exclusive upper bound returns the previous value.
### Immutable Collections
Lyng 1.5 adds immutable collection types for APIs that should not expose mutable state through aliases:
- `ImmutableList`
- `ImmutableSet`
- `ImmutableMap`
```lyng
val a = ImmutableList(1,2,3)
val b = a + 4
assertEquals(ImmutableList(1,2,3), a)
assertEquals(ImmutableList(1,2,3,4), b)
```
See [ImmutableList](ImmutableList.md), [ImmutableSet](ImmutableSet.md), and [ImmutableMap](ImmutableMap.md).
### Observable Mutable Lists
For reactive-style code, `lyng.observable` provides `ObservableList` with hooks and change streams.
```lyng
import lyng.observable
val xs = [1,2].observable()
xs.onChange { println("changed") }
xs += 3
```
You can validate or reject mutations in `beforeChange`, listen in `onChange`, and consume structured change events from `changes()`.
See [ObservableList](ObservableList.md).
### Random API
The standard library now includes a built-in random API plus deterministic seeded generators.
```lyng
val rng = Random.seeded(1234)
assert(rng.next(1..10) in 1..10)
assert(rng.next('a'..<'f') in 'a'..<'f')
```
Use:
- `Random.nextInt()`
- `Random.nextFloat()`
- `Random.next(range)`
- `Random.seeded(seed)`
## Tooling and Infrastructure ## Tooling and Infrastructure
### Rich Console Apps with `lyng.io.console`
`lyngio` now includes a real console module for terminal applications:
- TTY detection
- screen clearing and cursor movement
- alternate screen buffer
- raw input mode
- typed key and resize events
```lyng
import lyng.io.console
Console.enterAltScreen()
Console.clear()
Console.moveTo(1, 1)
Console.write("Hello from Lyng console app")
Console.flush()
Console.leaveAltScreen()
```
The repository includes a full interactive Tetris sample built on this API.
See [lyng.io.console](lyng.io.console.md).
### HTTP, WebSocket, TCP, and UDP in `lyngio`
`lyngio` grew from filesystem/process support into a broader application-facing I/O library. In 1.5.x it includes:
- `lyng.io.http` for HTTP/HTTPS client calls
- `lyng.io.ws` for WebSocket clients
- `lyng.io.net` for raw TCP/UDP transport
HTTP example:
```lyng
import lyng.io.http
val r = Http.get("https://example.com")
println(r.status)
println(r.text())
```
TCP example:
```lyng
import lyng.io.net
val socket = Net.tcpConnect("127.0.0.1", 4040)
socket.writeUtf8("ping")
socket.flush()
println(socket.readLine())
socket.close()
```
WebSocket example:
```lyng
import lyng.io.ws
val ws = Ws.connect("wss://example.com/socket")
ws.sendText("hello")
println(ws.receive())
ws.close()
```
These modules are capability-gated and host-installed, keeping Lyng safe by default while making networked scripts practical when enabled.
See [lyngio overview](lyngio.md), [lyng.io.db](lyng.io.db.md), [lyng.io.http](lyng.io.http.md), [lyng.io.ws](lyng.io.ws.md), and [lyng.io.net](lyng.io.net.md).
### CLI: Formatting Command ### CLI: Formatting Command
A new `fmt` subcommand has been added to the Lyng CLI. A new `fmt` subcommand has been added to the Lyng CLI.
@@ -350,6 +594,15 @@ lyng fmt --in-place MyFile.lyng # Format file in-place
lyng fmt --check MyFile.lyng # Check if file needs formatting lyng fmt --check MyFile.lyng # Check if file needs formatting
``` ```
### CLI: Better Terminal Workflows
The CLI is no longer just a script launcher. In the 1.5 line it also gained:
- built-in formatter support
- integrated `lyng.io.console` support for terminal programs
- downloadable packaged distributions for easier local use
This makes CLI-first scripting and console applications much more practical than in earlier releases.
### IDEA Plugin: Autocompletion ### IDEA Plugin: Autocompletion
Experimental lightweight autocompletion is now available in the IntelliJ plugin. It features type-aware member suggestions and inheritance-aware completion. Experimental lightweight autocompletion is now available in the IntelliJ plugin. It features type-aware member suggestions and inheritance-aware completion.
+325
View File
@@ -0,0 +1,325 @@
#!/usr/bin/env lyng
import lyng.io.db
import lyng.io.db.sqlite
import lyng.io.fs
val DB_FILE_NAME = "contents.db"
val ANSI_ESC = "\u001b["
val NEWLINE = "\n"
val WINDOWS_SEPARATOR = "\\"
val SQLITE_JOURNAL_SUFFIXES = ["-wal", "-shm", "-journal"]
val USAGE_TEXT = "
Lyng content index
Scan a directory tree, diff it against a SQLite snapshot, and optionally refresh the snapshot.
usage:
lyng examples/content_index_db.lyng <root> [-u|--update]
options:
-u, --update write the current scan back to $DB_FILE_NAME
notes:
- the database lives inside <root>/$DB_FILE_NAME
- on first run the snapshot is created automatically
- the script ignores its own SQLite sidecar files
"
val CREATE_FILE_INDEX_SQL = "
create table if not exists file_index(
path text primary key not null,
size integer not null,
mtime integer not null
)
"
val CREATE_CURRENT_SCAN_SQL = "
create temp table current_scan(
path text primary key not null,
size integer not null,
mtime integer not null
)
"
val SELECT_ADDED_SQL = "
select
c.path,
c.size,
c.mtime
from current_scan c
left join file_index f on f.path = c.path
where f.path is null
order by c.path
"
val SELECT_REMOVED_SQL = "
select f.path, f.size, f.mtime
from file_index f
left join current_scan c on c.path = f.path
where c.path is null
order by f.path
"
val SELECT_CHANGED_SQL = "
select c.path, f.size as old_size, c.size as new_size, f.mtime as old_mtime,
c.mtime as new_mtime
from current_scan c
join file_index f on f.path = c.path
where c.size != f.size or c.mtime != f.mtime
order by c.path
"
val DELETE_MISSING_SQL = "
delete from file_index
where not exists (
select 1
from current_scan c
where c.path = file_index.path
)
"
val UPSERT_SCAN_SQL = "
insert or replace into file_index(path, size, mtime)
select path, size, mtime
from current_scan
"
val INSERT_SCAN_ROW_SQL = "
insert into current_scan(path, size, mtime)
values(?, ?, ?)
"
val USE_COLOR = true
class CliOptions(val rootText: String, val updateSnapshot: Bool) {}
fun out(text: String? = null): Void {
if (text == null) {
print(NEWLINE)
return
}
print(text + NEWLINE)
}
fun paint(code: String, text: String): String {
if (!USE_COLOR) return text
ANSI_ESC + code + "m" + text + ANSI_ESC + "0m"
}
fun bold(text: String): String = paint("1", text)
fun dim(text: String): String = paint("2", text)
fun cyan(text: String): String = paint("36", text)
fun green(text: String): String = paint("32", text)
fun yellow(text: String): String = paint("33", text)
fun red(text: String): String = paint("31", text)
fun signed(value: Int): String = if (value > 0) "+" + value else value.toString()
fun plural(count: Int, one: String, many: String): String {
if (count == 1) return one
many
}
fun childPath(parent: Path, name: String): Path {
val base = parent.toString()
if (base.endsWith("/") || base.endsWith(WINDOWS_SEPARATOR)) {
return Path(base + name)
}
Path(base + "/" + name)
}
fun relativePath(root: Path, file: Path): String {
val parts: List<String> = []
for (i in root.segments.size..<file.segments.size) {
parts.add(file.segments[i] as String)
}
parts.joinToString("/")
}
fun isDatabaseArtifact(relative: String): Bool {
relative == DB_FILE_NAME || SQLITE_JOURNAL_SUFFIXES.any { relative == DB_FILE_NAME + (it as String) }
}
fun printUsage(message: String? = null): Void {
if (message != null && message.trim().isNotEmpty()) {
out(red("error: ") + message)
out()
}
out(bold(USAGE_TEXT))
}
fun parseArgs(argv: List<String>): CliOptions? {
var rootText: String? = null
var updateSnapshot = false
for (arg in argv) {
when (arg) {
"-u", "--update" -> updateSnapshot = true
"-h", "--help" -> {
printUsage()
return null
}
else -> {
if (arg.startsWith("-")) {
printUsage("unknown option: " + arg)
return null
}
if (rootText != null) {
printUsage("only one root path is allowed")
return null
}
rootText = arg
}
}
}
if (rootText == null) {
printUsage("missing required <root> argument")
return null
}
CliOptions(rootText as String, updateSnapshot)
}
fun printBanner(root: Path, dbFile: Path, dbWasCreated: Bool, updateSnapshot: Bool): Void {
val mode =
if (dbWasCreated) "bootstrap snapshot"
else if (updateSnapshot) "scan + refresh snapshot"
else "scan only"
out(cyan("== Lyng content index =="))
out(dim("root: " + root))
out(dim("db: " + dbFile))
out(dim("mode: " + mode))
out()
}
fun printSection(title: String, accent: (String)->String, rows: List<SqlRow>, render: (SqlRow)->String): Void {
out(accent(title + " (" + rows.size + ")"))
if (rows.isEmpty()) {
out(dim(" none"))
out()
return
}
for (row in rows) {
out(render(row))
}
out()
}
fun renderAdded(row: SqlRow): String {
val path = row["path"] as String
val size = row["size"] as Int
val mtime = row["mtime"] as Int
" " + green("+") + " " + bold(path) + dim(" %12d B mtime %d"(size, mtime))
}
fun renderRemoved(row: SqlRow): String {
val path = row["path"] as String
val size = row["size"] as Int
val mtime = row["mtime"] as Int
" " + red("-") + " " + bold(path) + dim(" %12d B mtime %d"(size, mtime))
}
fun renderChanged(row: SqlRow): String {
val path = row["path"] as String
val oldSize = row["old_size"] as Int
val newSize = row["new_size"] as Int
val oldMtime = row["old_mtime"] as Int
val newMtime = row["new_mtime"] as Int
val sizeDelta = newSize - oldSize
val mtimeDelta = newMtime - oldMtime
" " + yellow("~") + " " + bold(path) +
dim(
" size %d -> %d (%s B), mtime %d -> %d (%s ms)"(
oldSize,
newSize,
signed(sizeDelta),
oldMtime,
newMtime,
signed(mtimeDelta)
)
)
}
fun loadRows(tx: SqlTransaction, query: String): List<SqlRow> = tx.select(query).toList()
fun main() {
val argv: List<String> = []
for (raw in ARGV as List) {
argv.add(raw as String)
}
val options = parseArgs(argv)
if (options == null) {
return
}
val root = Path(options.rootText)
if (!root.exists()) {
printUsage("root does not exist: " + root)
return
}
if (!root.isDirectory()) {
printUsage("root is not a directory: " + root)
return
}
val dbFile = childPath(root, DB_FILE_NAME)
val dbWasCreated = !dbFile.exists()
val shouldUpdateSnapshot = dbWasCreated || options.updateSnapshot
printBanner(root, dbFile, dbWasCreated, shouldUpdateSnapshot)
val db = openSqlite(dbFile.toString())
db.transaction { tx ->
tx.execute(CREATE_FILE_INDEX_SQL)
tx.execute("drop table if exists temp.current_scan")
tx.execute(CREATE_CURRENT_SCAN_SQL)
var scannedFiles = 0
for (rawEntry in root.glob("**")) {
val entry = rawEntry as Path
if (!entry.isFile()) continue
val relative = relativePath(root, entry)
if (isDatabaseArtifact(relative)) continue
val size = entry.size() ?: 0
val mtime = entry.modifiedAtMillis() ?: 0
tx.execute(INSERT_SCAN_ROW_SQL, relative, size, mtime)
scannedFiles++
}
val added = loadRows(tx, SELECT_ADDED_SQL)
val removed = loadRows(tx, SELECT_REMOVED_SQL)
val changed = loadRows(tx, SELECT_CHANGED_SQL)
val totalChanges = added.size + removed.size + changed.size
out(dim("scanned %d %s under %s"(scannedFiles, plural(scannedFiles, "file", "files"), root.toString())))
out(dim("detected %d %s"(totalChanges, plural(totalChanges, "change", "changes"))))
out()
printSection("Added", { green(it) }, added) { renderAdded(it) }
printSection("Removed", { red(it) }, removed) { renderRemoved(it) }
printSection("Changed", { yellow(it) }, changed) { renderChanged(it) }
if (shouldUpdateSnapshot) {
tx.execute(DELETE_MISSING_SQL)
tx.execute(UPSERT_SCAN_SQL)
val action = if (dbWasCreated) "created" else "updated"
out(cyan("snapshot " + action + " in " + dbFile.name))
} else {
out(dim("snapshot unchanged; re-run with -u or --update to persist the scan"))
}
}
}
main()
View File
Whitespace-only changes.
+23
View File
@@ -0,0 +1,23 @@
#!/env/bin lyng
import lyng.io.http
// Step 1: download the main lynglang.com page.
val home = Http.get("https://lynglang.com").text()
// Step 2: find the version-script reference in the page HTML.
val jsRef = "src=\"([^\"]*lyng-version\\.js)\"".re.find(home)
require(jsRef != null, "lyng-version.js reference not found on the homepage")
// Step 3: extract the referenced script path from the first regex capture.
val versionJsPath = jsRef[1]
// Step 4: download the script that exposes `window.LYNG_VERSION`.
val versionJs = Http.get("https://lynglang.com/" + versionJsPath).text()
// Step 5: pull the actual version string from the JavaScript source.
val versionMatch = "LYNG_VERSION\\s*=\\s*\"([^\"]+)\"".re.find(versionJs)
require(versionMatch != null, "LYNG_VERSION assignment not found")
// Step 6: print the discovered version for the user.
println("Lynglang.com version: " + ((versionMatch as RegexMatch)[1]))
+15
View File
@@ -0,0 +1,15 @@
import lyng.time
val n = 700_000
fun tm<T>(block: ()->T): T {
val t = Instant()
block().also {
println("tm: ${Instant() - t}")
}
}
val x = tm { List.fill(n) { it * 10 + 1 } }
val y = tm { List.fill(n, n + 10) { it * 10 + 1 } }
tm { x.add(-1) }
tm { y.add(-2) }
+43
View File
@@ -0,0 +1,43 @@
import lyng.io.db
import lyng.io.db.jdbc
println("H2 JDBC demo: typed open, generic open, generated keys")
val db = openH2("mem:lyng_h2_demo;DB_CLOSE_DELAY=-1")
db.transaction { tx ->
tx.execute("create table if not exists person(id bigint auto_increment primary key, name varchar(120) not null, active boolean not null)")
tx.execute("delete from person")
val firstInsert = tx.execute(
"insert into person(name, active) values(?, ?)",
"Ada",
true
)
val firstId = firstInsert.getGeneratedKeys().toList()[0][0]
assertEquals(1, firstId)
tx.execute(
"insert into person(name, active) values(?, ?)",
"Linus",
false
)
val rows = tx.select("select id, name, active from person order by id").toList()
assertEquals(2, rows.size)
println("#" + rows[0]["id"] + " " + rows[0]["name"] + " active=" + rows[0]["active"])
println("#" + rows[1]["id"] + " " + rows[1]["name"] + " active=" + rows[1]["active"])
}
val genericDb = openDatabase(
"jdbc:h2:mem:lyng_h2_generic;DB_CLOSE_DELAY=-1",
Map()
)
val answer = genericDb.transaction { tx ->
tx.select("select 42 as answer").toList()[0]["answer"]
}
assertEquals(42, answer)
println("Generic JDBC openDatabase(...) also works: answer=$answer")
println("OK")
+67
View File
@@ -0,0 +1,67 @@
import lyng.time
val WORK_SIZE = 500
val THREADS = 1
fn piSpigot(iThread: Int, n: Int) {
var piIter = 0
var pi = List.fill(n) { 0 }
val boxes = n * 10 / 3
var reminders = List.fill(boxes) { 2 }
var heldDigits = 0
for (i in 0..<n) {
var carriedOver = 0
var sum = 0
for (j in (boxes - 1) downTo 0) {
val denom = j * 2 + 1
reminders[j] *= 10
sum = reminders[j] + carriedOver
val quotient = sum / denom
reminders[j] = sum % denom
carriedOver = quotient * j
}
reminders[0] = sum % 10
var q = sum / 10
if (q == 9) {
++heldDigits
} else if (q == 10) {
q = 0
for (k in 1..heldDigits) {
var replaced = pi[i - k]
if (replaced == 9) {
replaced = 0
} else {
++replaced
}
pi[i - k] = replaced
}
heldDigits = 1
} else {
heldDigits = 1
}
pi[piIter] = q
++piIter
}
var res = ""
for (i in (n - 8)..<n) {
res += pi[i]
}
println(iThread.toString() + ": " + res)
res
}
for( r in 0..100 ) {
val t0 = Instant()
println("piBench (lyng): THREADS = " + THREADS + ", WORK_SIZE = " + WORK_SIZE)
for (i in 0..<THREADS) {
piSpigot(i, WORK_SIZE)
}
val dt = Instant() - t0
println("all done, dt = ", dt)
delay(800)
}
+83
View File
@@ -0,0 +1,83 @@
# Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import time
from multiprocessing import Process
def piSpigot(iThread, nx):
piIter = 0
pi = [None] * nx
boxes = nx * 10 // 3
reminders = [None]*boxes
i = 0
while i < boxes:
reminders[i] = 2
i += 1
heldDigits = 0
i = 0
while i < nx:
carriedOver = 0
sum = 0
j = boxes - 1
while j >= 0:
reminders[j] *= 10
sum = reminders[j] + carriedOver
quotient = sum // (j * 2 + 1)
reminders[j] = sum % (j * 2 + 1)
carriedOver = quotient * j
j -= 1
reminders[0] = sum % 10
q = sum // 10
if q == 9:
heldDigits += 1
elif q == 10:
q = 0
k = 1
while k <= heldDigits:
replaced = pi[i - k]
if replaced == 9:
replaced = 0
else:
replaced += 1
pi[i - k] = replaced
k += 1
heldDigits = 1
else:
heldDigits = 1
pi[piIter] = q
piIter += 1
i += 1
res = ""
for i in range(len(pi)-8, len(pi), 1):
res += str(pi[i])
print(str(iThread) + ": " + res)
def createProcesses():
THREADS = 1
WORK_SIZE = 500
print("piBench (python3): THREADS = " + str(THREADS) + ", WORK_SIZE = " + str(WORK_SIZE))
pa = []
for i in range(THREADS):
p = Process(target=piSpigot, args=(i, WORK_SIZE))
p.start()
pa.append(p)
for p in pa:
p.join()
if __name__ == "__main__":
t1 = time.time()
createProcesses()
dt = time.time() - t1
print("total time: %i ms" % (dt*1000))
+71
View File
@@ -0,0 +1,71 @@
import lyng.io.db.jdbc
/*
PostgreSQL JDBC demo.
Usage:
lyng examples/postgres_basic.lyng [jdbc-url] [user] [password]
Typical local URL:
jdbc:postgresql://127.0.0.1/postgres
*/
fun cliArgs(): List<String> {
val result: List<String> = []
for (raw in ARGV as List) {
result.add(raw as String)
}
return result
}
val argv = cliArgs()
val URL = if (argv.size > 0) argv[0] else "jdbc:postgresql://127.0.0.1/postgres"
val USER = if (argv.size > 1) argv[1] else ""
val PASSWORD = if (argv.size > 2) argv[2] else ""
println("PostgreSQL JDBC demo: typed open, generated keys, nested transaction")
val db = openPostgres(URL, USER, PASSWORD)
db.transaction { tx ->
tx.execute("create table if not exists lyng_pg_demo(id bigserial primary key, title text not null, done boolean not null)")
tx.execute("delete from lyng_pg_demo")
val firstInsert = tx.execute(
"insert into lyng_pg_demo(title, done) values(?, ?)",
"Verify PostgreSQL JDBC support",
false
)
val firstId = firstInsert.getGeneratedKeys().toList()[0][0]
println("First generated id=" + firstId)
tx.execute(
"insert into lyng_pg_demo(title, done) values(?, ?)",
"Review documentation",
true
)
try {
tx.transaction { inner ->
inner.execute(
"insert into lyng_pg_demo(title, done) values(?, ?)",
"This row is rolled back",
false
)
throw IllegalStateException("rollback nested")
}
} catch (_: IllegalStateException) {
println("Nested transaction rolled back as expected")
}
val rows = tx.select("select id, title, done from lyng_pg_demo order by id").toList()
for (row in rows) {
println("#" + row["id"] + " " + row["title"] + " done=" + row["done"])
}
val count = tx.select("select count(*) as count from lyng_pg_demo").toList()[0]["count"]
assertEquals(2, count)
println("Visible rows after nested rollback: " + count)
}
println("OK")
+89
View File
@@ -0,0 +1,89 @@
import lyng.io.db
import lyng.io.db.sqlite
import lyng.time
println("SQLite demo: typed open, generic open, result sets, generated keys, nested rollback")
// The typed helper is the simplest entry point when you know you want SQLite.
val db = openSqlite(":memory:")
db.transaction { tx ->
// Keep schema creation and data changes inside one transaction block.
tx.execute("create table task(id integer primary key autoincrement, title text not null, done integer not null, due_date date not null)")
// execute(...) is for side-effect statements. Generated keys are read from
// ExecutionResult rather than from a synthetic row-returning INSERT.
val firstInsert = tx.execute(
"insert into task(title, done, due_date) values(?, ?, ?)",
"Write a SQLite example",
false,
Date(2026, 4, 15)
)
val firstGeneratedKeys = firstInsert.getGeneratedKeys()
val firstId = firstGeneratedKeys.toList()[0][0]
assertEquals(1, firstId)
tx.execute(
"insert into task(title, done, due_date) values(?, ?, ?)",
"Review the DB API",
true,
Date(2026, 4, 16)
)
// Nested transactions are real savepoints. If the inner block fails,
// only the nested work is rolled back.
try {
tx.transaction { inner ->
inner.execute(
"insert into task(title, done, due_date) values(?, ?, ?)",
"This row is rolled back",
false,
Date(2026, 4, 17)
)
throw IllegalStateException("demonstrate nested rollback")
}
} catch (_: IllegalStateException) {
println("Nested transaction rolled back as expected")
}
// select(...) is for row-producing statements. ResultSet exposes metadata,
// cheap emptiness checks, iteration, and conversion to a plain list.
val tasks = tx.select("select id, title, done, due_date from task order by id")
assertEquals(false, tasks.isEmpty())
assertEquals(2, tasks.size())
println("Columns:")
for (column in tasks.columns) {
println(" " + column.name + " -> " + column.sqlType + " (native " + column.nativeType + ")")
}
val taskRows = tasks.toList()
println("Rows:")
for (row in taskRows) {
// Name lookups are case-insensitive and values are already converted.
println(" #" + row["ID"] + " " + row["title"] + " done=" + row["done"] + " due=" + row["due_date"])
}
// toList() materializes detached rows that stay usable after transaction close.
val snapshot = tx.select("select title, due_date from task order by id").toList()
assertEquals("Write a SQLite example", snapshot[0]["title"])
assertEquals(Date(2026, 4, 16), snapshot[1]["due_date"])
val count = tx.select("select count(*) as count from task").toList()[0]["count"]
assertEquals(2, count)
println("Visible rows after nested rollback: $count")
}
// The generic entry point stays useful for config-driven code.
val genericDb = openDatabase(
"sqlite::memory:",{ foreignKeys: true, busyTimeoutMillis: 1000 }
)
val answer = genericDb.transaction { tx ->
tx.select("select 42 as answer").toList()[0]["answer"]
}
assertEquals(42, answer)
println("Generic openDatabase(...) also works: answer=$answer")
println("OK")
+68
View File
@@ -0,0 +1,68 @@
import lyng.buffer
import lyng.io.net
val host = "127.0.0.1"
val port = 8092
val N = 5
val server = Net.tcpListen(port, host)
println("start tcp server at $host:$port")
fun serveClient(client: TcpSocket) = launch {
try {
while (true) {
val data = client.read()
if (data == null) break
var line = (data as Buffer).decodeUtf8()
line = "[" + client.remoteAddress() + "]> " + line
println(line)
}
} catch (e) {
println("ERROR [reader]: " + e)
}
}
fun serveRequests(server: TcpServer) = launch {
val readers = []
try {
for (i in 0..<5) {
val client = server.accept()
println("accept new connection: " + client.remoteAddress())
readers.add(serveClient(client as TcpSocket))
}
} catch (e) {
println("ERROR [listener]: " + e)
} finally {
server.close()
}
for (i in 0..<readers.size) {
val reader = readers[i]
(reader as Deferred).await()
}
}
val srv = serveRequests(server as TcpServer)
var clients = []
for (i in 0..<N) {
//delay(500)
clients.add(launch {
try{
val socket = Net.tcpConnect(host, port)
socket.writeUtf8("ping1ping2ping3ping4ping5")
socket.flush()
socket.close()
} catch (e) {
println("ERROR [client]: " + e)
}
})
}
for (i in 0..<clients.size) {
val c = clients[i]
(c as Deferred).await()
println("client done")
}
srv.await()
delay(10000)
println("FIN")
+60
View File
@@ -0,0 +1,60 @@
import lyng.io.net
val host = "127.0.0.1"
val clientCount = 1000
val clientWindow = 128
val server = Net.tcpListen(0, host, clientWindow, true)
val port = server.localAddress().port
fun payloadFor(index: Int) = "$index:${Random.nextInt()}:${Random.nextInt()}"
val serverJob = launch {
try {
while (true) {
val client = server.accept()
launch {
try {
client.readLine()?.let { source ->
client.writeUtf8("pong: $source\n")
client.flush()
}
} finally {
client.close()
}
}
}
} catch (e) {
if (server.isOpen()) {
throw e
}
} finally {
if (server.isOpen()) {
server.close()
}
}
}
var completed = 0
for (batchStart in 0..<clientCount step clientWindow) {
val batchEnd = if (batchStart + clientWindow < clientCount) batchStart + clientWindow else clientCount
val replies = (batchStart..<batchEnd).map { index ->
val payload = payloadFor(index)
launch {
val socket = Net.tcpConnect(host, port) as TcpSocket
try {
socket.writeUtf8(payload + "\n")
socket.flush()
val reply = socket.readLine()
assertEquals("pong: $payload", reply)
} finally {
socket.close()
}
}
}.joinAll()
completed += replies.size
}
assertEquals(clientCount, completed)
server.close()
serverJob.await()
println("OK: $clientCount concurrent tcp clients")
+31 -22
View File
@@ -80,6 +80,30 @@ class GameState(
var paused = false var paused = false
} }
class LoopFrame(val resized: Bool, val originRow: Int, val originCol: Int) {} class LoopFrame(val resized: Bool, val originRow: Int, val originCol: Int) {}
class InputBuffer {
private val mutex: Mutex = Mutex()
private val items: List<String> = []
fun push(value: String): Void {
mutex.withLock {
if (items.size >= MAX_PENDING_INPUTS) {
items.removeAt(0)
}
items.add(value)
}
}
fun drain(): List<String> {
val out: List<String> = []
mutex.withLock {
while (items.size > 0) {
out.add(items[0])
items.removeAt(0)
}
}
out
}
}
fun clearAndHome() { fun clearAndHome() {
Console.clear() Console.clear()
@@ -540,9 +564,8 @@ if (!Console.isSupported()) {
) )
var prevFrameLines: List<String> = [] var prevFrameLines: List<String> = []
val gameMutex: Mutex = Mutex()
var forceRedraw = false var forceRedraw = false
val pendingInputs: List<String> = [] val inputBuffer: InputBuffer = InputBuffer()
val rawModeEnabled = Console.setRawMode(true) val rawModeEnabled = Console.setRawMode(true)
if (!rawModeEnabled) { if (!rawModeEnabled) {
@@ -642,13 +665,7 @@ if (!Console.isSupported()) {
continue continue
} }
val mapped = if (ctrl && (key == "c" || key == "C")) "__CTRL_C__" else key val mapped = if (ctrl && (key == "c" || key == "C")) "__CTRL_C__" else key
val mm: Mutex = gameMutex inputBuffer.push(mapped)
mm.withLock {
if (pendingInputs.size >= MAX_PENDING_INPUTS) {
pendingInputs.removeAt(0)
}
pendingInputs.add(mapped)
}
} }
} }
} catch (eventErr: Object) { } catch (eventErr: Object) {
@@ -745,19 +762,11 @@ if (!Console.isSupported()) {
continue continue
} }
val mm: Mutex = gameMutex val toApply = inputBuffer.drain()
mm.withLock { if (toApply.size > 0) {
if (pendingInputs.size > 0) { for (k in toApply) {
val toApply: List<String> = [] applyKeyInput(state, k)
while (pendingInputs.size > 0) { if (!state.running || state.gameOver) break
val k = pendingInputs[0]
pendingInputs.removeAt(0)
toApply.add(k)
}
for (k in toApply) {
applyKeyInput(state, k)
if (!state.running || state.gameOver) break
}
} }
} }
if (!state.running || state.gameOver) { if (!state.running || state.gameOver) {
+14
View File
@@ -14,9 +14,16 @@ firebaseCrashlyticsBuildtools = "3.0.3"
okioVersion = "3.10.2" okioVersion = "3.10.2"
compiler = "3.2.0-alpha11" compiler = "3.2.0-alpha11"
ktor = "3.3.1" ktor = "3.3.1"
slf4j = "2.0.17"
sqlite-jdbc = "3.50.3.0"
h2 = "2.4.240"
postgresql = "42.7.8"
testcontainers = "1.20.6"
hikaricp = "6.2.1"
[libraries] [libraries]
clikt = { module = "com.github.ajalt.clikt:clikt", version.ref = "clikt" } clikt = { module = "com.github.ajalt.clikt:clikt", version.ref = "clikt" }
clikt-core = { module = "com.github.ajalt.clikt:clikt-core", version.ref = "clikt" }
clikt-markdown = { module = "com.github.ajalt.clikt:clikt-markdown", version.ref = "clikt" } clikt-markdown = { module = "com.github.ajalt.clikt:clikt-markdown", version.ref = "clikt" }
mordant-core = { module = "com.github.ajalt.mordant:mordant-core", version.ref = "mordant" } mordant-core = { module = "com.github.ajalt.mordant:mordant-core", version.ref = "mordant" }
mordant-jvm-jna = { module = "com.github.ajalt.mordant:mordant-jvm-jna", version.ref = "mordant" } mordant-jvm-jna = { module = "com.github.ajalt.mordant:mordant-jvm-jna", version.ref = "mordant" }
@@ -40,6 +47,13 @@ ktor-client-js = { module = "io.ktor:ktor-client-js", version.ref = "ktor" }
ktor-client-winhttp = { module = "io.ktor:ktor-client-winhttp", version.ref = "ktor" } ktor-client-winhttp = { module = "io.ktor:ktor-client-winhttp", version.ref = "ktor" }
ktor-client-websockets = { module = "io.ktor:ktor-client-websockets", version.ref = "ktor" } ktor-client-websockets = { module = "io.ktor:ktor-client-websockets", version.ref = "ktor" }
ktor-network = { module = "io.ktor:ktor-network", version.ref = "ktor" } ktor-network = { module = "io.ktor:ktor-network", version.ref = "ktor" }
slf4j-nop = { module = "org.slf4j:slf4j-nop", version.ref = "slf4j" }
sqlite-jdbc = { module = "org.xerial:sqlite-jdbc", version.ref = "sqlite-jdbc" }
h2 = { module = "com.h2database:h2", version.ref = "h2" }
postgresql = { module = "org.postgresql:postgresql", version.ref = "postgresql" }
testcontainers = { module = "org.testcontainers:testcontainers", version.ref = "testcontainers" }
testcontainers-postgresql = { module = "org.testcontainers:postgresql", version.ref = "testcontainers" }
hikaricp = { module = "com.zaxxer:HikariCP", version.ref = "hikaricp" }
[plugins] [plugins]
androidLibrary = { id = "com.android.library", version.ref = "agp" } androidLibrary = { id = "com.android.library", version.ref = "agp" }
+2
View File
@@ -0,0 +1,2 @@
[vps]
94.130.36.94 ansible_user=sergeych
+100
View File
@@ -0,0 +1,100 @@
---
- name: Setup lynglang.com static site on VPS
hosts: vps
become: yes
vars:
domain: lynglang.com
web_root: /var/www/lynglang
deploy_user: sergeych
certbot_email: real.sergeych@gmail.com
tasks:
# Debian 10 buster is EOL; security/backports repos moved to archive.debian.org
- name: Fix sources.list for Debian buster EOL
copy:
dest: /etc/apt/sources.list
content: |
deb http://archive.debian.org/debian/ buster main contrib non-free
deb http://archive.debian.org/debian-security/ buster/updates main contrib non-free
deb http://archive.debian.org/debian/ buster-backports main contrib non-free
- name: Remove stale third-party sources (broken for buster EOL)
file:
path: "/etc/apt/sources.list.d/{{ item }}"
state: absent
loop:
- cassandra.list
- icinga.list
- postgres.list
- salt-stack.list
- yarn.list
- name: Install nginx, certbot, and python3-certbot-nginx
apt:
name:
- nginx
- certbot
- python3-certbot-nginx
state: present
update_cache: yes
- name: Create web root directory
file:
path: "{{ web_root }}/release/dist"
state: directory
owner: "{{ deploy_user }}"
group: www-data
mode: "0755"
recurse: yes
- name: Create distributables directory
file:
path: "{{ web_root }}/release/dist/distributables"
state: directory
owner: "{{ deploy_user }}"
group: www-data
mode: "0755"
- name: Deploy nginx site config (HTTP, pre-certbot)
template:
src: templates/nginx_lynglang.conf.j2
dest: /etc/nginx/sites-available/{{ domain }}
notify: reload nginx
- name: Enable nginx site
file:
src: /etc/nginx/sites-available/{{ domain }}
dest: /etc/nginx/sites-enabled/{{ domain }}
state: link
notify: reload nginx
- name: Disable default nginx site
file:
path: /etc/nginx/sites-enabled/default
state: absent
notify: reload nginx
- name: Ensure nginx is started
service:
name: nginx
state: started
enabled: yes
- name: Reload nginx before certbot
meta: flush_handlers
- name: Obtain SSL certificate via certbot (--nginx plugin)
command: >
certbot --nginx
-d {{ domain }} -d www.{{ domain }}
--non-interactive --agree-tos
--email {{ certbot_email }}
--redirect
args:
creates: /etc/letsencrypt/live/{{ domain }}/fullchain.pem
handlers:
- name: reload nginx
service:
name: nginx
state: reloaded
+24
View File
@@ -0,0 +1,24 @@
server {
listen 80;
server_name {{ domain }} www.{{ domain }};
root {{ web_root }}/release/dist;
index index.html;
# SPA fallback
location / {
try_files $uri $uri/ /index.html;
}
# Distributables served directly
location /distributables/ {
try_files $uri =404;
autoindex on;
}
# Long-lived cache for hashed assets
location ~* \.(js|css|woff2?|ttf|eot|svg|png|jpg|ico)$ {
expires 1y;
add_header Cache-Control "public, immutable";
}
}
@@ -310,7 +310,7 @@ class LyngDocumentationProvider : AbstractDocumentationProvider() {
// Try literal and call-based receiver inference around the dot // Try literal and call-based receiver inference around the dot
val i = TextCtx.prevNonWs(text, dotPos - 1) val i = TextCtx.prevNonWs(text, dotPos - 1)
val className: String? = when { val className: String? = when {
i >= 0 && text[i] == '"' -> "String" i >= 0 && (text[i] == '"' || text[i] == '`') -> "String"
i >= 0 && text[i] == ']' -> "List" i >= 0 && text[i] == ']' -> "List"
i >= 0 && text[i] == '}' -> "Dict" i >= 0 && text[i] == '}' -> "Dict"
i >= 0 && text[i] == ')' -> { i >= 0 && text[i] == ')' -> {
@@ -24,6 +24,7 @@ import com.intellij.psi.codeStyle.CodeStyleManager
import com.intellij.psi.impl.source.codeStyle.PreFormatProcessor import com.intellij.psi.impl.source.codeStyle.PreFormatProcessor
import net.sergeych.lyng.format.LyngFormatConfig import net.sergeych.lyng.format.LyngFormatConfig
import net.sergeych.lyng.format.LyngFormatter import net.sergeych.lyng.format.LyngFormatter
import net.sergeych.lyng.format.LyngStringDelimiterPolicy
import net.sergeych.lyng.idea.LyngLanguage import net.sergeych.lyng.idea.LyngLanguage
/** /**
@@ -170,6 +171,7 @@ class LyngPreFormatProcessor : PreFormatProcessor {
continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)), continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)),
applySpacing = true, applySpacing = true,
applyWrapping = false, applyWrapping = false,
stringDelimiterPolicy = LyngStringDelimiterPolicy.PreferFewerEscapes,
) )
val r = if (runFullFileIndent) currentLocalRange() else workingRangeLocal.intersection(currentLocalRange()) ?: currentLocalRange() val r = if (runFullFileIndent) currentLocalRange() else workingRangeLocal.intersection(currentLocalRange()) ?: currentLocalRange()
val text = doc.getText(r) val text = doc.getText(r)
@@ -189,6 +191,7 @@ class LyngPreFormatProcessor : PreFormatProcessor {
continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)), continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)),
applySpacing = settings.enableSpacing, applySpacing = settings.enableSpacing,
applyWrapping = true, applyWrapping = true,
stringDelimiterPolicy = LyngStringDelimiterPolicy.PreferFewerEscapes,
) )
val r = if (runFullFileIndent) currentLocalRange() else workingRangeLocal.intersection(currentLocalRange()) ?: currentLocalRange() val r = if (runFullFileIndent) currentLocalRange() else workingRangeLocal.intersection(currentLocalRange()) ?: currentLocalRange()
val text = doc.getText(r) val text = doc.getText(r)
@@ -101,8 +101,8 @@ class LyngLexer : LexerBase() {
return return
} }
// String "..." or '...' with simple escape handling // String "...", `...`, or '...' with simple escape handling
if (ch == '"' || ch == '\'') { if (ch == '"' || ch == '\'' || ch == '`') {
val quote = ch val quote = ch
i++ i++
while (i < endOffset) { while (i < endOffset) {
@@ -27,9 +27,9 @@ import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.highlight.offsetOf import net.sergeych.lyng.highlight.offsetOf
import net.sergeych.lyng.idea.LyngFileType import net.sergeych.lyng.idea.LyngFileType
import net.sergeych.lyng.idea.util.LyngAstManager import net.sergeych.lyng.idea.util.LyngAstManager
import net.sergeych.lyng.idea.util.LyngIdeaImportProvider
import net.sergeych.lyng.idea.util.TextCtx import net.sergeych.lyng.idea.util.TextCtx
import net.sergeych.lyng.miniast.* import net.sergeych.lyng.miniast.*
import net.sergeych.lyng.tools.IdeLenientImportProvider
import net.sergeych.lyng.tools.LyngAnalysisRequest import net.sergeych.lyng.tools.LyngAnalysisRequest
import net.sergeych.lyng.tools.LyngLanguageTools import net.sergeych.lyng.tools.LyngLanguageTools
@@ -273,7 +273,7 @@ class LyngPsiReference(element: PsiElement) : PsiPolyVariantReferenceBase<PsiEle
private fun loadMini(file: PsiFile): MiniScript? { private fun loadMini(file: PsiFile): MiniScript? {
LyngAstManager.getMiniAst(file)?.let { return it } LyngAstManager.getMiniAst(file)?.let { return it }
return try { return try {
val provider = IdeLenientImportProvider.create() val provider = LyngIdeaImportProvider.create()
runBlocking { runBlocking {
LyngLanguageTools.analyze( LyngLanguageTools.analyze(
LyngAnalysisRequest(text = file.text, fileName = file.name, importProvider = provider) LyngAnalysisRequest(text = file.text, fileName = file.name, importProvider = provider)
@@ -28,7 +28,10 @@ import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.binding.BindingSnapshot import net.sergeych.lyng.binding.BindingSnapshot
import net.sergeych.lyng.idea.LyngFileType import net.sergeych.lyng.idea.LyngFileType
import net.sergeych.lyng.miniast.* import net.sergeych.lyng.miniast.*
import net.sergeych.lyng.tools.* import net.sergeych.lyng.tools.LyngAnalysisRequest
import net.sergeych.lyng.tools.LyngAnalysisResult
import net.sergeych.lyng.tools.LyngDiagnostic
import net.sergeych.lyng.tools.LyngLanguageTools
object LyngAstManager { object LyngAstManager {
private val MINI_KEY = Key.create<MiniScript>("lyng.mini.cache") private val MINI_KEY = Key.create<MiniScript>("lyng.mini.cache")
@@ -142,7 +145,8 @@ object LyngAstManager {
val text = file.viewProvider.contents.toString() val text = file.viewProvider.contents.toString()
val built = try { val built = try {
val provider = IdeLenientImportProvider.create() DocsBootstrap.ensure()
val provider = LyngIdeaImportProvider.create()
runBlocking { runBlocking {
LyngLanguageTools.analyze( LyngLanguageTools.analyze(
LyngAnalysisRequest(text = text, fileName = file.name, importProvider = provider) LyngAnalysisRequest(text = text, fileName = file.name, importProvider = provider)
@@ -165,7 +169,7 @@ object LyngAstManager {
val dMini = getAnalysis(df)?.mini ?: run { val dMini = getAnalysis(df)?.mini ?: run {
val dText = df.viewProvider.contents.toString() val dText = df.viewProvider.contents.toString()
try { try {
val provider = IdeLenientImportProvider.create() val provider = LyngIdeaImportProvider.create()
runBlocking { runBlocking {
LyngLanguageTools.analyze( LyngLanguageTools.analyze(
LyngAnalysisRequest(text = dText, fileName = df.name, importProvider = provider) LyngAnalysisRequest(text = dText, fileName = df.name, importProvider = provider)
@@ -0,0 +1,68 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.util
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.Script
import net.sergeych.lyng.io.console.createConsoleModule
import net.sergeych.lyng.io.fs.createFs
import net.sergeych.lyng.io.http.createHttpModule
import net.sergeych.lyng.io.net.createNetModule
import net.sergeych.lyng.io.process.createProcessModule
import net.sergeych.lyng.io.ws.createWsModule
import net.sergeych.lyng.pacman.ImportManager
import net.sergeych.lyng.pacman.ImportProvider
import net.sergeych.lyngio.console.security.PermitAllConsoleAccessPolicy
import net.sergeych.lyngio.fs.security.PermitAllAccessPolicy
import net.sergeych.lyngio.http.security.PermitAllHttpAccessPolicy
import net.sergeych.lyngio.net.security.PermitAllNetAccessPolicy
import net.sergeych.lyngio.process.security.PermitAllProcessAccessPolicy
import net.sergeych.lyngio.ws.security.PermitAllWsAccessPolicy
/**
* IDE import provider that knows about optional LyngIO modules used by editor analysis.
*
* The default import manager only exposes core modules; editor features need the pluggable
* `lyng.io.*` packages available as well so imported symbols resolve without false errors.
*/
class LyngIdeaImportProvider private constructor(root: Scope) : ImportProvider(root) {
override suspend fun createModuleScope(pos: Pos, packageName: String): ModuleScope {
return try {
baseImportManager.createModuleScope(pos, packageName)
} catch (_: Throwable) {
ModuleScope(this, pos, packageName)
}
}
companion object {
private val baseImportManager: ImportManager by lazy {
Script.defaultImportManager.copy().apply {
createFs(PermitAllAccessPolicy, this)
createConsoleModule(PermitAllConsoleAccessPolicy, this)
createHttpModule(PermitAllHttpAccessPolicy, this)
createWsModule(PermitAllWsAccessPolicy, this)
createNetModule(PermitAllNetAccessPolicy, this)
createProcessModule(PermitAllProcessAccessPolicy, this)
}
}
fun create(): LyngIdeaImportProvider = LyngIdeaImportProvider(baseImportManager.rootScope)
}
}
@@ -179,4 +179,20 @@ class LyngDefinitionFilesTest : BasePlatformTestCase() {
assertTrue("Should not report unresolved name for PlainDeclared", messages.none { it.contains("unresolved name: PlainDeclared") }) assertTrue("Should not report unresolved name for PlainDeclared", messages.none { it.contains("unresolved name: PlainDeclared") })
assertTrue("Should not report unresolved member for hello", messages.none { it.contains("unresolved member: hello") }) assertTrue("Should not report unresolved member for hello", messages.none { it.contains("unresolved member: hello") })
} }
fun test_DiagnosticsResolveOptionalNetModuleSymbols() {
val code = """
import lyng.io.net
val server = Net.tcpListen(0, "127.0.0.1")
val port = server.localAddress().port
""".trimIndent()
myFixture.configureByText("main.lyng", code)
val analysis = LyngAstManager.getAnalysis(myFixture.file)
val messages = analysis?.diagnostics?.map { it.message } ?: emptyList()
assertTrue("Should not report unresolved name for Net, got=$messages", messages.none { it.contains("unresolved name: Net") })
assertTrue("Should not report unresolved member for tcpListen, got=$messages", messages.none { it.contains("unresolved member: tcpListen") })
assertTrue("Should not report unresolved member for localAddress, got=$messages", messages.none { it.contains("unresolved member: localAddress") })
assertTrue("Should not report unresolved member for port, got=$messages", messages.none { it.contains("unresolved member: port") })
}
} }
@@ -0,0 +1,58 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.highlight
import org.junit.Assert.assertArrayEquals
import org.junit.Assert.assertEquals
import org.junit.Test
class LyngLexerBacktickStringTest {
@Test
fun backtickStringGetsStringTokenAndColor() {
val lexer = LyngLexer()
val source = """val json = `{"name":"lyng","doc":"use \`quotes\`"}`"""
lexer.start(source, 0, source.length, 0)
val tokens = mutableListOf<Pair<String, String>>()
while (lexer.tokenType != null) {
val tokenText = source.substring(lexer.tokenStart, lexer.tokenEnd)
tokens += lexer.tokenType.toString() to tokenText
lexer.advance()
}
assertEquals(
listOf(
"KEYWORD" to "val",
"WHITESPACE" to " ",
"IDENTIFIER" to "json",
"WHITESPACE" to " ",
"PUNCT" to "=",
"WHITESPACE" to " ",
"STRING" to "`{\"name\":\"lyng\",\"doc\":\"use \\`quotes\\`\"}`"
),
tokens
)
val highlighter = LyngSyntaxHighlighter()
assertArrayEquals(
arrayOf(LyngHighlighterColors.STRING),
highlighter.getTokenHighlights(LyngTokenTypes.STRING)
)
}
}
+71 -1
View File
@@ -19,9 +19,39 @@ plugins {
alias(libs.plugins.kotlinMultiplatform) alias(libs.plugins.kotlinMultiplatform)
} }
import org.gradle.api.Project
import org.jetbrains.kotlin.gradle.targets.native.tasks.KotlinNativeTest
group = "net.sergeych" group = "net.sergeych"
version = "unspecified" version = "unspecified"
private fun Project.sqliteLinuxLinkerOpts(vararg defaultDirs: String): List<String> {
val overrideDir = providers.gradleProperty("sqlite3.lib.dir").orNull
?: providers.environmentVariable("SQLITE3_LIB_DIR").orNull
val candidateDirs = buildList {
if (!overrideDir.isNullOrBlank()) {
add(file(overrideDir))
}
defaultDirs.forEach { add(file(it)) }
}.distinctBy { it.absolutePath }
val discoveredLib = sequenceOf("libsqlite3.so", "libsqlite3.so.0")
.mapNotNull { libraryName ->
candidateDirs.firstOrNull { it.resolve(libraryName).isFile }?.let { dir ->
listOf("-L${dir.absolutePath}", "-l:$libraryName")
}
}
.firstOrNull()
?: listOf("-lsqlite3")
return discoveredLib + listOf(
"-ldl",
"-lpthread",
"-lm",
"-Wl,--allow-shlib-undefined"
)
}
repositories { repositories {
mavenCentral() mavenCentral()
maven("https://maven.universablockchain.com/") maven("https://maven.universablockchain.com/")
@@ -52,6 +82,22 @@ kotlin {
linuxX64 { linuxX64 {
binaries { binaries {
executable() executable()
all {
linkerOpts(
*project.sqliteLinuxLinkerOpts(
"/lib/x86_64-linux-gnu",
"/usr/lib/x86_64-linux-gnu",
"/lib64",
"/usr/lib64",
"/lib",
"/usr/lib"
).toTypedArray()
)
if (buildType == org.jetbrains.kotlin.gradle.plugin.mpp.NativeBuildType.RELEASE) {
debuggable = false
optimized = true
}
}
} }
} }
sourceSets { sourceSets {
@@ -63,7 +109,7 @@ kotlin {
// filesystem access into the execution Scope by default. // filesystem access into the execution Scope by default.
implementation(project(":lyngio")) implementation(project(":lyngio"))
implementation(libs.okio) implementation(libs.okio)
implementation(libs.clikt) implementation(libs.clikt.core)
implementation(kotlin("stdlib-common")) implementation(kotlin("stdlib-common"))
// optional support for rendering markdown in help messages // optional support for rendering markdown in help messages
// implementation(libs.clikt.markdown) // implementation(libs.clikt.markdown)
@@ -77,9 +123,17 @@ kotlin {
implementation(libs.okio.fakefilesystem) implementation(libs.okio.fakefilesystem)
} }
} }
val linuxTest by creating {
dependsOn(commonTest)
}
val nativeMain by creating { val nativeMain by creating {
dependsOn(commonMain) dependsOn(commonMain)
} }
val jvmMain by getting {
dependencies {
implementation(libs.slf4j.nop)
}
}
val jvmTest by getting { val jvmTest by getting {
dependencies { dependencies {
implementation(kotlin("test")) implementation(kotlin("test"))
@@ -89,5 +143,21 @@ kotlin {
val linuxX64Main by getting { val linuxX64Main by getting {
dependsOn(nativeMain) dependsOn(nativeMain)
} }
val linuxX64Test by getting {
dependsOn(linuxTest)
}
} }
} }
tasks.named<KotlinNativeTest>("linuxX64Test") {
dependsOn(tasks.named("linkDebugExecutableLinuxX64"))
dependsOn(tasks.named("linkReleaseExecutableLinuxX64"))
environment(
"LYNG_CLI_NATIVE_BIN",
layout.buildDirectory.file("bin/linuxX64/debugExecutable/lyng.kexe").get().asFile.absolutePath
)
environment(
"LYNG_CLI_NATIVE_RELEASE_BIN",
layout.buildDirectory.file("bin/linuxX64/releaseExecutable/lyng.kexe").get().asFile.absolutePath
)
}
+360 -47
View File
@@ -17,7 +17,7 @@
package net.sergeych package net.sergeych
import com.github.ajalt.clikt.core.CliktCommand import com.github.ajalt.clikt.core.CoreCliktCommand
import com.github.ajalt.clikt.core.Context import com.github.ajalt.clikt.core.Context
import com.github.ajalt.clikt.core.main import com.github.ajalt.clikt.core.main
import com.github.ajalt.clikt.core.subcommands import com.github.ajalt.clikt.core.subcommands
@@ -26,28 +26,51 @@ import com.github.ajalt.clikt.parameters.arguments.multiple
import com.github.ajalt.clikt.parameters.arguments.optional import com.github.ajalt.clikt.parameters.arguments.optional
import com.github.ajalt.clikt.parameters.options.flag import com.github.ajalt.clikt.parameters.options.flag
import com.github.ajalt.clikt.parameters.options.option import com.github.ajalt.clikt.parameters.options.option
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.runBlocking import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.Compiler import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withContext
import net.sergeych.lyng.EvalSession
import net.sergeych.lyng.LyngVersion import net.sergeych.lyng.LyngVersion
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.Script import net.sergeych.lyng.Script
import net.sergeych.lyng.ScriptError import net.sergeych.lyng.ScriptError
import net.sergeych.lyng.Source import net.sergeych.lyng.Source
import net.sergeych.lyng.asFacade
import net.sergeych.lyng.io.console.createConsoleModule import net.sergeych.lyng.io.console.createConsoleModule
import net.sergeych.lyng.io.db.createDbModule
import net.sergeych.lyng.io.db.jdbc.createJdbcModule
import net.sergeych.lyng.io.db.sqlite.createSqliteModule
import net.sergeych.lyng.io.fs.createFs import net.sergeych.lyng.io.fs.createFs
import net.sergeych.lyng.io.http.createHttpModule
import net.sergeych.lyng.io.net.createNetModule
import net.sergeych.lyng.io.ws.createWsModule
import net.sergeych.lyng.obj.* import net.sergeych.lyng.obj.*
import net.sergeych.lyng.pacman.ImportManager
import net.sergeych.lyngio.net.shutdownSystemNetEngine
import net.sergeych.lyngio.console.security.PermitAllConsoleAccessPolicy import net.sergeych.lyngio.console.security.PermitAllConsoleAccessPolicy
import net.sergeych.lyngio.fs.security.PermitAllAccessPolicy import net.sergeych.lyngio.fs.security.PermitAllAccessPolicy
import net.sergeych.lyngio.http.security.PermitAllHttpAccessPolicy
import net.sergeych.lyngio.net.security.PermitAllNetAccessPolicy
import net.sergeych.lyngio.ws.security.PermitAllWsAccessPolicy
import net.sergeych.mp_tools.globalDefer import net.sergeych.mp_tools.globalDefer
import okio.FileSystem import okio.*
import okio.Path.Companion.toPath import okio.Path.Companion.toPath
import okio.SYSTEM
import okio.buffer
import okio.use
// common code // common code
expect fun exit(code: Int) expect fun exit(code: Int)
internal expect class CliPlatformShutdownHooks {
fun uninstall()
companion object {
fun install(runtime: CliExecutionRuntime): CliPlatformShutdownHooks
}
}
expect class ShellCommandExecutor { expect class ShellCommandExecutor {
fun executeCommand(command: String): CommandResult fun executeCommand(command: String): CommandResult
@@ -62,19 +85,304 @@ data class CommandResult(
val error: String val error: String
) )
val baseScopeDefer = globalDefer { private const val cliBuiltinsDeclarations = """
Script.newScope().apply { extern fun atExit(append: Bool=true, handler: ()->Void)
addFn("exit") { """
exit(requireOnlyArg<ObjInt>().toInt())
ObjVoid private class CliExitRequested(val code: Int) : RuntimeException("CLI exit requested: $code")
internal class CliExecutionRuntime(
private val session: EvalSession,
private val rootScope: Scope
) {
private val shutdownMutex = Mutex()
private var shutdownStarted = false
private val exitHandlers = mutableListOf<Obj>()
fun registerAtExit(handler: Obj, append: Boolean) {
if (append) {
exitHandlers += handler
} else {
exitHandlers.add(0, handler)
} }
// Install lyng.io.fs module with full access by default for the CLI tool's Scope.
// Scripts still need to `import lyng.io.fs` to use Path API.
createFs(PermitAllAccessPolicy, this)
// Install console access by default for interactive CLI scripts.
// Scripts still need to `import lyng.io.console` to use it.
createConsoleModule(PermitAllConsoleAccessPolicy, this)
} }
suspend fun shutdown() {
shutdownMutex.withLock {
if (shutdownStarted) return
shutdownStarted = true
}
val handlers = exitHandlers.toList()
val facade = rootScope.asFacade()
for (handler in handlers) {
runCatching {
facade.call(handler)
}
}
session.cancel()
shutdownSystemNetEngine()
session.join()
}
fun shutdownBlocking() {
runBlocking {
shutdown()
}
}
}
private val baseCliImportManagerDefer = globalDefer {
val manager = Script.defaultImportManager.copy().apply {
installCliModules(this)
}
manager.newStdScope()
manager
}
private fun ImportManager.invalidateCliModuleCaches() {
invalidatePackageCache("lyng.io.fs")
invalidatePackageCache("lyng.io.console")
invalidatePackageCache("lyng.io.db.jdbc")
invalidatePackageCache("lyng.io.db.sqlite")
invalidatePackageCache("lyng.io.http")
invalidatePackageCache("lyng.io.ws")
invalidatePackageCache("lyng.io.net")
}
val baseScopeDefer = globalDefer {
baseCliImportManagerDefer.await().copy().apply {
invalidateCliModuleCaches()
}.newStdScope().apply {
installCliDeclarations()
installCliBuiltins()
addConst("ARGV", ObjList(mutableListOf()))
}
}
private suspend fun Scope.installCliDeclarations() {
eval(Source("<cli-builtins>", cliBuiltinsDeclarations))
}
private fun Scope.installCliBuiltins(runtime: CliExecutionRuntime? = null) {
addFn("exit") {
val code = requireOnlyArg<ObjInt>().toInt()
if (runtime == null) {
exit(code)
}
throw CliExitRequested(code)
}
addFn("atExit") {
if (runtime == null) {
raiseIllegalState("atExit is only available while running a CLI script")
}
if (args.list.size > 2) {
raiseError("Expected at most 2 positional arguments, got ${args.list.size}")
}
var append = true
var appendSet = false
var handler: Obj? = null
when (args.list.size) {
1 -> {
val only = args.list[0]
if (only.isInstanceOf("Callable")) {
handler = only
} else {
append = only.toBool()
appendSet = true
}
}
2 -> {
append = args.list[0].toBool()
appendSet = true
handler = args.list[1]
}
}
for ((name, value) in args.named) {
when (name) {
"append" -> {
if (appendSet) {
raiseIllegalArgument("argument 'append' is already set")
}
append = value.toBool()
appendSet = true
}
"handler" -> {
if (handler != null) {
raiseIllegalArgument("argument 'handler' is already set")
}
handler = value
}
else -> raiseIllegalArgument("unknown argument '$name'")
}
}
val handlerValue = handler ?: raiseError("argument 'handler' is required")
if (!handlerValue.isInstanceOf("Callable")) {
raiseClassCastError("Expected handler to be callable")
}
runtime.registerAtExit(handlerValue, append)
ObjVoid
}
}
private fun installCliModules(manager: ImportManager) {
// Scripts still need to import the modules they use explicitly.
createFs(PermitAllAccessPolicy, manager)
createConsoleModule(PermitAllConsoleAccessPolicy, manager)
createDbModule(manager)
createJdbcModule(manager)
createSqliteModule(manager)
createHttpModule(PermitAllHttpAccessPolicy, manager)
createWsModule(PermitAllWsAccessPolicy, manager)
createNetModule(PermitAllNetAccessPolicy, manager)
}
private data class LocalCliModule(
val packageName: String,
val source: Source
)
private fun readUtf8(path: Path): String =
FileSystem.SYSTEM.source(path).use { fileSource ->
fileSource.buffer().use { bs ->
bs.readUtf8()
}
}
private fun stripShebang(text: String): String {
if (!text.startsWith("#!")) return text
val pos = text.indexOf('\n')
return if (pos >= 0) text.substring(pos + 1) else ""
}
private fun extractDeclaredPackageNameOrNull(source: Source): String? {
for (line in source.lines) {
if (line.isBlank()) continue
return if (line.startsWith("package ")) {
line.substring(8).trim()
} else {
null
}
}
return null
}
private fun canonicalPath(path: Path): Path = FileSystem.SYSTEM.canonicalize(path)
private fun relativeModuleName(rootDir: Path, file: Path): String {
val rootText = rootDir.toString().trimEnd('/', '\\')
val fileText = file.toString()
val prefix = "$rootText/"
if (!fileText.startsWith(prefix)) {
throw ScriptError(Pos.builtIn, "local import root mismatch: $fileText is not under $rootText")
}
val relative = fileText.removePrefix(prefix)
val modulePath = relative.removeSuffix(".lyng")
return modulePath
.split('/', '\\')
.filter { it.isNotEmpty() }
.joinToString(".")
}
private fun scanLyngFiles(rootDir: Path): List<Path> {
val system = FileSystem.SYSTEM
val pending = ArrayDeque<Path>()
val visited = linkedSetOf<String>()
val files = mutableListOf<Path>()
pending.add(rootDir)
while (pending.isNotEmpty()) {
val dir = pending.removeLast()
val canonicalDir = canonicalPath(dir)
if (!visited.add(canonicalDir.toString())) continue
val children = try {
system.list(canonicalDir)
} catch (_: Exception) {
continue
}
for (child in children) {
val meta = try {
system.metadata(child)
} catch (_: Exception) {
continue
}
when {
meta.isDirectory -> pending.add(child)
child.name.endsWith(".lyng") -> {
val canonicalFile = try {
canonicalPath(child)
} catch (_: Exception) {
continue
}
files += canonicalFile
}
}
}
}
return files
}
private fun discoverLocalCliModules(entryFile: Path): List<LocalCliModule> {
val rootDir = entryFile.parent ?: ".".toPath()
val seenPackages = linkedMapOf<String, Path>()
return scanLyngFiles(rootDir)
.asSequence()
.filter { it != entryFile }
.map { file ->
val text = stripShebang(readUtf8(file))
val source = Source(file.toString(), text)
val expectedPackage = relativeModuleName(rootDir, file)
val declaredPackage = extractDeclaredPackageNameOrNull(source)
if (declaredPackage != null && declaredPackage != expectedPackage) {
throw ScriptError(
source.startPos,
"local module package mismatch: expected '$expectedPackage' for ${file.toString()} but found '$declaredPackage'"
)
}
val packageName = declaredPackage ?: expectedPackage
val previous = seenPackages[packageName]
if (previous != null) {
throw ScriptError(
source.startPos,
"duplicate local module '$packageName': ${previous.toString()} and ${file.toString()}"
)
}
seenPackages[packageName] = file
LocalCliModule(packageName, source)
}
.toList()
}
private fun registerLocalCliModules(manager: ImportManager, modules: List<LocalCliModule>) {
for (module in modules) {
manager.addPackage(module.packageName) { scope ->
scope.eval(module.source)
}
}
}
private suspend fun ImportManager.newCliScope(argv: List<String>): Scope =
newStdScope().apply {
installCliDeclarations()
installCliBuiltins()
addConst("ARGV", ObjList(argv.map { ObjString(it) }.toMutableList()))
}
internal suspend fun newCliScope(argv: List<String>, entryFileName: String? = null): Scope {
val baseManager = baseCliImportManagerDefer.await().copy().apply {
invalidateCliModuleCaches()
}
if (entryFileName == null) {
return baseManager.newCliScope(argv)
}
val entryFile = canonicalPath(entryFileName.toPath())
val localModules = discoverLocalCliModules(entryFile)
if (localModules.isEmpty()) {
return baseManager.newCliScope(argv)
}
registerLocalCliModules(baseManager, localModules)
return baseManager.newCliScope(argv)
} }
fun runMain(args: Array<String>) { fun runMain(args: Array<String>) {
@@ -98,7 +406,7 @@ fun runMain(args: Array<String>) {
.main(args) .main(args)
} }
private class Fmt : CliktCommand(name = "fmt") { private class Fmt : CoreCliktCommand(name = "fmt") {
private val checkOnly by option("--check", help = "Check only; print files that would change").flag() private val checkOnly by option("--check", help = "Check only; print files that would change").flag()
private val inPlace by option("-i", "--in-place", help = "Write changes back to files").flag() private val inPlace by option("-i", "--in-place", help = "Write changes back to files").flag()
private val enableSpacing by option("--spacing", help = "Apply spacing normalization").flag() private val enableSpacing by option("--spacing", help = "Apply spacing normalization").flag()
@@ -121,6 +429,7 @@ private class Fmt : CliktCommand(name = "fmt") {
val cfg = net.sergeych.lyng.format.LyngFormatConfig( val cfg = net.sergeych.lyng.format.LyngFormatConfig(
applySpacing = enableSpacing, applySpacing = enableSpacing,
applyWrapping = enableWrapping, applyWrapping = enableWrapping,
stringDelimiterPolicy = net.sergeych.lyng.format.LyngStringDelimiterPolicy.PreferFewerEscapes,
) )
var anyChanged = false var anyChanged = false
@@ -156,7 +465,7 @@ private class Fmt : CliktCommand(name = "fmt") {
} }
} }
private class Lyng(val launcher: (suspend () -> Unit) -> Unit) : CliktCommand() { private class Lyng(val launcher: (suspend () -> Unit) -> Unit) : CoreCliktCommand() {
override val invokeWithoutSubcommand = true override val invokeWithoutSubcommand = true
override val printHelpOnEmptyArgs = true override val printHelpOnEmptyArgs = true
@@ -187,7 +496,6 @@ private class Lyng(val launcher: (suspend () -> Unit) -> Unit) : CliktCommand()
if (currentContext.invokedSubcommand != null) return if (currentContext.invokedSubcommand != null) return
runBlocking { runBlocking {
val baseScope = baseScopeDefer.await()
when { when {
version -> { version -> {
println("Lyng language version ${LyngVersion}") println("Lyng language version ${LyngVersion}")
@@ -197,20 +505,13 @@ private class Lyng(val launcher: (suspend () -> Unit) -> Unit) : CliktCommand()
val objargs = mutableListOf<String>() val objargs = mutableListOf<String>()
script?.let { objargs += it } script?.let { objargs += it }
objargs += args objargs += args
baseScope.addConst(
"ARGV", ObjList(
objargs.map { ObjString(it) }.toMutableList()
)
)
launcher { launcher {
// there is no script name, it is a first argument instead: // there is no script name, it is a first argument instead:
processErrors { processErrors {
val script = Compiler.compileWithResolution( executeSource(
Source("<eval>", execute!!), Source("<eval>", execute!!),
baseScope.currentImportProvider, newCliScope(objargs)
seedScope = baseScope
) )
script.execute(baseScope)
} }
} }
} }
@@ -220,8 +521,7 @@ private class Lyng(val launcher: (suspend () -> Unit) -> Unit) : CliktCommand()
println("Error: no script specified.\n") println("Error: no script specified.\n")
echoFormattedHelp() echoFormattedHelp()
} else { } else {
baseScope.addConst("ARGV", ObjList(args.map { ObjString(it) }.toMutableList())) launcher { executeFile(script!!, args) }
launcher { executeFile(script!!) }
} }
} }
} }
@@ -231,30 +531,43 @@ private class Lyng(val launcher: (suspend () -> Unit) -> Unit) : CliktCommand()
fun executeFileWithArgs(fileName: String, args: List<String>) { fun executeFileWithArgs(fileName: String, args: List<String>) {
runBlocking { runBlocking {
baseScopeDefer.await().addConst("ARGV", ObjList(args.map { ObjString(it) }.toMutableList())) executeFile(fileName, args)
executeFile(fileName)
} }
} }
suspend fun executeFile(fileName: String) { suspend fun executeSource(source: Source, initialScope: Scope? = null) {
var text = FileSystem.SYSTEM.source(fileName.toPath()).use { fileSource -> val session = EvalSession(initialScope ?: baseScopeDefer.await())
fileSource.buffer().use { bs -> val rootScope = session.getScope()
bs.readUtf8() val runtime = CliExecutionRuntime(session, rootScope)
rootScope.installCliBuiltins(runtime)
val shutdownHooks = CliPlatformShutdownHooks.install(runtime)
var requestedExitCode: Int? = null
try {
try {
evalOnCliDispatcher(session, source)
} catch (e: CliExitRequested) {
requestedExitCode = e.code
} }
} finally {
shutdownHooks.uninstall()
runtime.shutdown()
} }
if( text.startsWith("#!") ) { requestedExitCode?.let { exit(it) }
// skip shebang }
val pos = text.indexOf('\n')
text = text.substring(pos + 1) internal suspend fun evalOnCliDispatcher(session: EvalSession, source: Source): Obj =
withContext(Dispatchers.Default) {
session.eval(source)
} }
suspend fun executeFile(fileName: String, args: List<String> = emptyList()) {
val canonicalFile = canonicalPath(fileName.toPath())
val text = stripShebang(readUtf8(canonicalFile))
processErrors { processErrors {
val scope = baseScopeDefer.await() executeSource(
val script = Compiler.compileWithResolution( Source(canonicalFile.toString(), text),
Source(fileName, text), newCliScope(args, canonicalFile.toString())
scope.currentImportProvider,
seedScope = scope
) )
script.execute(scope)
} }
} }
@@ -0,0 +1,172 @@
package net.sergeych
import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.EvalSession
import net.sergeych.lyng.Source
import net.sergeych.lyng.obj.ObjString
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
class CliTcpServerRegressionTest {
@Test
fun reducedTcpServerExampleRunsWithCopiedCliImportManager() = runBlocking {
val cliScope = newCliScope(emptyList())
val session = EvalSession(cliScope)
try {
val result = evalOnCliDispatcher(
session,
Source(
"<tcp-server-regression>",
"""
import lyng.buffer
import lyng.io.net
val host = "127.0.0.1"
val server = Net.tcpListen(0, host)
val port = server.localAddress().port
val accepted = launch {
val client = server.accept()
val line = (client.read(4) as Buffer).decodeUtf8()
client.close()
server.close()
line
}
val socket = Net.tcpConnect(host, port)
socket.writeUtf8("ping")
socket.flush()
socket.close()
accepted.await()
""".trimIndent()
)
)
assertEquals("ping", (result as ObjString).value)
} finally {
session.cancelAndJoin()
}
}
@Test
fun concurrentTcpExampleRunsInCliScope() = runBlocking {
val cliScope = newCliScope(emptyList())
val session = EvalSession(cliScope)
try {
val result = evalOnCliDispatcher(
session,
Source(
"<tcp-server-concurrency-cli>",
"""
import lyng.io.net
val host = "127.0.0.1"
val clientCount = 32
val server: TcpServer = Net.tcpListen(0, host, 32, true) as TcpServer
val port: Int = server.localAddress().port
fun payloadFor(index: Int): String {
"${'$'}index:${'$'}{Random.nextInt()}:${'$'}{Random.nextInt()}"
}
fun handleClient(client: TcpSocket): String {
try {
val source = client.readLine()
if( source == null ) {
return "server-eof"
}
val reply = "pong: ${'$'}source"
client.writeUtf8(reply + "\n")
client.flush()
reply
} finally {
client.close()
}
}
val serverJob: Deferred = launch {
var handlers: List<Deferred> = List()
try {
for( i in 0..<32 ) {
val client: TcpSocket = server.accept() as TcpSocket
handlers += launch {
handleClient(client)
}
}
handlers.joinAll()
} finally {
server.close()
}
}
val clientJobs = (0..<clientCount).map { index ->
val payload = payloadFor(index)
launch {
val socket: TcpSocket = Net.tcpConnect(host, port) as TcpSocket
try {
socket.writeUtf8(payload + "\n")
socket.flush()
val reply = socket.readLine()
if( reply == null ) {
"client-eof:${'$'}payload"
} else {
assertEquals("pong: ${'$'}payload", reply)
reply
}
} finally {
socket.close()
}
}
}
val replies = clientJobs.joinAll()
val serverReplies = serverJob.await() as List<Object>
assertEquals(clientCount, replies.size)
assertEquals(clientCount, serverReplies.size)
assertEquals(replies.toSet, serverReplies.toSet)
"OK:${'$'}clientCount:${'$'}{replies.toSet}:${'$'}{serverReplies.toSet}"
""".trimIndent()
)
)
val text = (result as ObjString).value
assertTrue(text.startsWith("OK:32:"), text)
} finally {
session.cancelAndJoin()
}
}
@Test
fun mixedModuleAndLocalCapturesWorkInCliScope() = runBlocking {
val cliScope = newCliScope(emptyList())
val session = EvalSession(cliScope)
try {
val result = evalOnCliDispatcher(
session,
Source(
"<cli-capture-regression>",
"""
val prefix = "pong"
val jobs = (0..<32).map { index ->
val payload = "${'$'}index:${'$'}{Random.nextInt()}"
launch {
delay(5)
"${'$'}prefix:${'$'}payload"
}
}
jobs.joinAll()
""".trimIndent()
)
) as net.sergeych.lyng.obj.ObjList
assertEquals(32, result.list.size)
assertEquals(32, result.list.map { (it as ObjString).value }.toSet().size)
} finally {
session.cancelAndJoin()
}
}
}
+29 -1
View File
@@ -24,6 +24,34 @@ import kotlin.system.exitProcess
@PublishedApi @PublishedApi
internal var jvmExitImpl: (Int) -> Nothing = { code -> exitProcess(code) } internal var jvmExitImpl: (Int) -> Nothing = { code -> exitProcess(code) }
internal actual class CliPlatformShutdownHooks private constructor(
private val shutdownHook: Thread?
) {
actual fun uninstall() {
val hook = shutdownHook ?: return
runCatching {
Runtime.getRuntime().removeShutdownHook(hook)
}
}
actual companion object {
actual fun install(runtime: CliExecutionRuntime): CliPlatformShutdownHooks {
val hook = Thread(
{
runtime.shutdownBlocking()
},
"lyng-cli-shutdown"
)
return runCatching {
Runtime.getRuntime().addShutdownHook(hook)
CliPlatformShutdownHooks(hook)
}.getOrElse {
CliPlatformShutdownHooks(null)
}
}
}
}
actual fun exit(code: Int) { actual fun exit(code: Int) {
jvmExitImpl(code) jvmExitImpl(code)
} }
@@ -0,0 +1,104 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych
import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.EvalSession
import net.sergeych.lyng.Script
import net.sergeych.lyng.Source
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjList
import net.sergeych.lyng.obj.ObjString
import org.junit.Test
import kotlin.test.assertEquals
import kotlin.test.assertNotEquals
class CliDispatcherJvmTest {
@Test
fun executeSourceRunsOnDefaultDispatcher() = runBlocking {
val callerThread = Thread.currentThread()
val callerThreadKey = "${System.identityHashCode(callerThread)}:${callerThread.name}"
val scope = Script.newScope().apply {
addFn("threadKey") { ObjString("${System.identityHashCode(Thread.currentThread())}:${Thread.currentThread().name}") }
addFn("threadName") { ObjString(Thread.currentThread().name) }
}
val session = EvalSession(scope)
try {
val result = evalOnCliDispatcher(
session,
Source(
"<test>",
"""
val task = launch { [threadKey(), threadName()] }
val child = task.await()
[threadKey(), threadName(), child]
""".trimIndent()
)
) as ObjList
val topLevelThreadKey = (result.list[0] as ObjString).value
val topLevelThreadName = (result.list[1] as ObjString).value
val child = result.list[2] as ObjList
val childThreadKey = (child.list[0] as ObjString).value
val childThreadName = (child.list[1] as ObjString).value
assertNotEquals(
callerThreadKey,
topLevelThreadKey,
"CLI top-level script body should not run on the runBlocking caller thread: $topLevelThreadName"
)
assertNotEquals(
callerThreadKey,
childThreadKey,
"CLI launch child should not inherit the runBlocking caller thread: $childThreadName"
)
} finally {
session.cancelAndJoin()
}
}
@Test
fun cliEvalInfersDeferredItTypeFromMapLambdaLocal() = runBlocking {
val session = EvalSession(Script.newScope())
try {
val result = evalOnCliDispatcher(
session,
Source(
"<cli-repro>",
"""
var sum = 0
var counter = 0
(1..3).map { n ->
val counterState = counter
val task = launch { counterState + n }
++counter
task
}.forEach { sum += it.await() }
sum
""".trimIndent()
)
)
assertEquals(9, (result as ObjInt).value)
} finally {
session.cancelAndJoin()
}
}
}
@@ -0,0 +1,290 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych
import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.EvalSession
import net.sergeych.lyng.Source
import net.sergeych.lyng.obj.ObjString
import org.junit.After
import org.junit.Before
import java.io.ByteArrayOutputStream
import java.io.PrintStream
import java.nio.file.Files
import kotlin.io.path.writeText
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertTrue
class CliLocalModuleImportRegressionJvmTest {
private val originalOut: PrintStream = System.out
private val originalErr: PrintStream = System.err
private class TestExit(val code: Int) : RuntimeException()
@Before
fun setUp() {
jvmExitImpl = { code -> throw TestExit(code) }
}
@After
fun tearDown() {
System.setOut(originalOut)
System.setErr(originalErr)
jvmExitImpl = { code -> kotlin.system.exitProcess(code) }
}
private data class CliResult(val out: String, val err: String, val exitCode: Int?)
private fun runCli(vararg args: String): CliResult {
val outBuf = ByteArrayOutputStream()
val errBuf = ByteArrayOutputStream()
System.setOut(PrintStream(outBuf, true, Charsets.UTF_8))
System.setErr(PrintStream(errBuf, true, Charsets.UTF_8))
var exitCode: Int? = null
try {
runMain(arrayOf(*args))
} catch (e: TestExit) {
exitCode = e.code
} finally {
System.out.flush()
System.err.flush()
}
return CliResult(outBuf.toString("UTF-8"), errBuf.toString("UTF-8"), exitCode)
}
private fun writeTransitiveImportTree(root: java.nio.file.Path) {
val packageDir = Files.createDirectories(root.resolve("package1"))
val nestedDir = Files.createDirectories(packageDir.resolve("nested"))
packageDir.resolve("alpha.lyng").writeText(
"""
package package1.alpha
import lyng.stdlib
import lyng.io.net
class Alpha {
val headers = Map<String, String>()
fun makeTask(port: Int, host: String): Deferred = launch {
host + ":" + port
}
fun netModule() = Net
}
fun alphaValue() = "alpha"
""".trimIndent()
)
packageDir.resolve("beta.lyng").writeText(
"""
package package1.beta
import lyng.stdlib
import package1.alpha
fun betaValue() = alphaValue() + "|beta"
""".trimIndent()
)
nestedDir.resolve("gamma.lyng").writeText(
"""
package package1.nested.gamma
import lyng.io.net
import package1.alpha
import package1.beta
val String.gammaTag get() = this + "|gamma"
fun gammaValue() = betaValue().gammaTag
fun netModule() = Net
""".trimIndent()
)
packageDir.resolve("entry.lyng").writeText(
"""
package package1.entry
import lyng.stdlib
import lyng.io.net
import package1.alpha
import package1.beta
import package1.nested.gamma
fun report() = gammaValue() + "|entry"
""".trimIndent()
)
}
private fun writeNestedLaunchImportBugTree(root: java.nio.file.Path) {
val packageDir = Files.createDirectories(root.resolve("package1"))
packageDir.resolve("alpha.lyng").writeText(
"""
import lyng.io.net
import package1.bravo
class Alpha {
val tcpServer: TcpServer
val headers = Map<String, String>()
fn startListen(port, host) {
var eager = Bravo()
eager.doSomething()
tcpServer = Net.tcpListen(port, host)
println("tcpServer.isOpen: " + tcpServer.isOpen())
launch {
try {
while (true) {
println("wait for accept...")
val tcpSocket = tcpServer.accept()
println("var bravo = Bravo()")
var bravo = Bravo()
println("bravo.doSomething()...")
bravo.doSomething()
println("bravo.doSomething()... OK")
tcpSocket.close()
break
}
} catch (e) {
println("ERR [Alpha.startListen]: '", e, "'")
} finally {
println("FIN [Alpha.startListen]")
tcpServer.close()
}
}
}
}
""".trimIndent()
)
packageDir.resolve("bravo.lyng").writeText(
"""
class Bravo {
fn doSomething() {
println("Bravo.doSomething")
}
}
""".trimIndent()
)
}
@Test
fun localModuleUsingLaunchAndNetImportsWithoutStdlibRedefinition() = runBlocking {
val root = Files.createTempDirectory("lyng-cli-import-regression")
try {
val mainFile = root.resolve("main.lyng")
writeTransitiveImportTree(root)
mainFile.writeText(
"""
import package1.entry
import package1.beta
import package1.nested.gamma
println(report())
""".trimIndent()
)
executeFile(mainFile.toString(), emptyList())
} finally {
root.toFile().deleteRecursively()
}
}
@Test
fun localModuleImportsAreNoOpsWhenEvaldRepeatedlyOnSameCliContext() = runBlocking {
val root = Files.createTempDirectory("lyng-cli-import-regression-repeat")
try {
val mainFile = root.resolve("main.lyng")
writeTransitiveImportTree(root)
mainFile.writeText("println(\"bootstrap\")")
val session = EvalSession(newCliScope(emptyList(), mainFile.toString()))
try {
repeat(5) { index ->
val result = evalOnCliDispatcher(
session,
Source(
"<repeat-local-import-$index>",
"""
import package1.entry
import package1.nested.gamma
import package1.beta
import package1.alpha
report()
""".trimIndent()
)
) as ObjString
assertEquals(
"alpha|beta|gamma|entry",
result.value
)
}
} finally {
session.cancelAndJoin()
}
} finally {
root.toFile().deleteRecursively()
}
}
@Test
fun localModuleImportUsedOnlyInsideMethodLaunchClosureRemainsPrepared() = runBlocking {
val root = Files.createTempDirectory("lyng-cli-import-regression-launch")
try {
val mainFile = root.resolve("main.lyng")
val port = java.net.ServerSocket(0).let {
val selected = it.localPort
it.close()
selected
}
writeNestedLaunchImportBugTree(root)
mainFile.writeText(
"""
import lyng.io.net
import package1.alpha
val alpha = Alpha()
alpha.startListen($port, "127.0.0.1")
delay(50)
val socket = Net.tcpConnect("127.0.0.1", $port)
println("send ping...")
socket.writeUtf8("ping")
socket.flush()
socket.close()
delay(50)
""".trimIndent()
)
val result = runCli(mainFile.toString())
assertTrue(result.err.isBlank(), result.err)
assertFalse(result.out.contains("ERR [Alpha.startListen]"), result.out)
assertFalse(result.out.contains("module capture 'Bravo'"), result.out)
assertTrue(result.out.contains("bravo.doSomething()... OK"), result.out)
assertEquals(2, Regex("Bravo\\.doSomething").findAll(result.out).count(), result.out)
} finally {
root.toFile().deleteRecursively()
}
}
}
@@ -0,0 +1,119 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng_cli
import net.sergeych.jvmExitImpl
import net.sergeych.runMain
import org.junit.After
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Before
import org.junit.Test
import java.io.ByteArrayOutputStream
import java.io.PrintStream
import java.nio.file.Files
import java.nio.file.Path
class CliAtExitJvmTest {
private val originalOut: PrintStream = System.out
private val originalErr: PrintStream = System.err
private class TestExit(val code: Int) : RuntimeException()
private data class CliResult(val out: String, val err: String, val exitCode: Int?)
@Before
fun setUp() {
jvmExitImpl = { code -> throw TestExit(code) }
}
@After
fun tearDown() {
System.setOut(originalOut)
System.setErr(originalErr)
jvmExitImpl = { code -> kotlin.system.exitProcess(code) }
}
private fun runCli(vararg args: String): CliResult {
val outBuf = ByteArrayOutputStream()
val errBuf = ByteArrayOutputStream()
System.setOut(PrintStream(outBuf, true, Charsets.UTF_8))
System.setErr(PrintStream(errBuf, true, Charsets.UTF_8))
var exitCode: Int? = null
try {
runMain(arrayOf(*args))
} catch (e: TestExit) {
exitCode = e.code
} finally {
System.out.flush()
System.err.flush()
}
return CliResult(outBuf.toString("UTF-8"), errBuf.toString("UTF-8"), exitCode)
}
private fun runScript(scriptText: String): CliResult {
val tmp: Path = Files.createTempFile("lyng_atexit_", ".lyng")
try {
Files.writeString(tmp, scriptText)
return runCli(tmp.toString())
} finally {
Files.deleteIfExists(tmp)
}
}
@Test
fun atExitRunsInRequestedOrderAndIgnoresHandlerExceptions() {
val result = runScript(
"""
atExit {
println("tail")
}
atExit(false) {
println("head")
throw Exception("ignored")
}
println("body")
""".trimIndent()
)
assertNull(result.err.takeIf { it.isNotBlank() })
assertNull(result.exitCode)
val lines = result.out
.lineSequence()
.map { it.trim() }
.filter { it.isNotEmpty() }
.toList()
assertEquals(listOf("body", "head", "tail"), lines)
}
@Test
fun atExitRunsBeforeScriptExitTerminatesProcess() {
val result = runScript(
"""
atExit {
println("cleanup")
}
exit(7)
""".trimIndent()
)
assertEquals(7, result.exitCode)
assertTrue(result.out.lineSequence().any { it.trim() == "cleanup" })
}
}
@@ -0,0 +1,237 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng_cli
import net.sergeych.jvmExitImpl
import net.sergeych.runMain
import org.junit.After
import org.junit.Assert.assertTrue
import org.junit.Before
import org.junit.Test
import java.io.ByteArrayOutputStream
import java.io.PrintStream
import java.nio.file.Files
class CliLocalImportsJvmTest {
private val originalOut: PrintStream = System.out
private val originalErr: PrintStream = System.err
private class TestExit(val code: Int) : RuntimeException()
@Before
fun setUp() {
jvmExitImpl = { code -> throw TestExit(code) }
}
@After
fun tearDown() {
System.setOut(originalOut)
System.setErr(originalErr)
jvmExitImpl = { code -> kotlin.system.exitProcess(code) }
}
private data class CliResult(val out: String, val err: String, val exitCode: Int?)
private fun runCli(vararg args: String): CliResult {
val outBuf = ByteArrayOutputStream()
val errBuf = ByteArrayOutputStream()
System.setOut(PrintStream(outBuf, true, Charsets.UTF_8))
System.setErr(PrintStream(errBuf, true, Charsets.UTF_8))
var exitCode: Int? = null
try {
runMain(arrayOf(*args))
} catch (e: TestExit) {
exitCode = e.code
} finally {
System.out.flush()
System.err.flush()
}
return CliResult(outBuf.toString("UTF-8"), errBuf.toString("UTF-8"), exitCode)
}
private fun writeTransitiveImportTree(root: java.nio.file.Path) {
val packageDir = Files.createDirectories(root.resolve("package1"))
val nestedDir = Files.createDirectories(packageDir.resolve("nested"))
Files.writeString(
packageDir.resolve("alpha.lyng"),
"""
package package1.alpha
import lyng.stdlib
import lyng.io.net
class Alpha {
val headers = Map<String, String>()
fun makeTask(port: Int, host: String): Deferred = launch {
host + ":" + port
}
fun netModule() = Net
}
fun alphaValue() = "alpha"
""".trimIndent()
)
Files.writeString(
packageDir.resolve("beta.lyng"),
"""
package package1.beta
import lyng.stdlib
import package1.alpha
fun betaValue() = alphaValue() + "|beta"
""".trimIndent()
)
Files.writeString(
nestedDir.resolve("gamma.lyng"),
"""
package package1.nested.gamma
import lyng.io.net
import package1.alpha
import package1.beta
val String.gammaTag get() = this + "|gamma"
fun gammaValue() = betaValue().gammaTag
fun netModule() = Net
""".trimIndent()
)
Files.writeString(
packageDir.resolve("entry.lyng"),
"""
package package1.entry
import lyng.stdlib
import lyng.io.net
import package1.alpha
import package1.beta
import package1.nested.gamma
fun report() = gammaValue() + "|entry"
""".trimIndent()
)
}
@Test
fun cliDiscoversSiblingAndNestedLocalImportsFromEntryRoot() {
val dir = Files.createTempDirectory("lyng_cli_local_imports_")
try {
val mathDir = Files.createDirectories(dir.resolve("math"))
val utilDir = Files.createDirectories(dir.resolve("util"))
val mainFile = dir.resolve("main.lyng")
Files.writeString(
mathDir.resolve("add.lyng"),
"""
fun plus(a, b) = a + b
""".trimIndent()
)
Files.writeString(
utilDir.resolve("answer.lyng"),
"""
package util.answer
import math.add
fun answer() = plus(40, 2)
""".trimIndent()
)
Files.writeString(
mainFile,
"""
import util.answer
println(answer())
""".trimIndent()
)
val result = runCli(mainFile.toString())
assertTrue(result.err, result.err.isBlank())
assertTrue(result.out, result.out.contains("42"))
} finally {
dir.toFile().deleteRecursively()
}
}
@Test
fun cliRejectsPackageThatDoesNotMatchRelativePath() {
val dir = Files.createTempDirectory("lyng_cli_local_imports_badpkg_")
try {
val utilDir = Files.createDirectories(dir.resolve("util"))
val mainFile = dir.resolve("main.lyng")
Files.writeString(
utilDir.resolve("answer.lyng"),
"""
package util.wrong
fun answer() = 42
""".trimIndent()
)
Files.writeString(
mainFile,
"""
import util.answer
println(answer())
""".trimIndent()
)
val result = runCli(mainFile.toString())
assertTrue(result.out, result.out.contains("local module package mismatch"))
assertTrue(result.out, result.out.contains("expected 'util.answer'"))
} finally {
dir.toFile().deleteRecursively()
}
}
@Test
fun cliHandlesOverlappingDirectoryImportsWithTransitiveStdlibAndNetSymbols() {
val dir = Files.createTempDirectory("lyng_cli_local_imports_transitive_")
try {
val mainFile = dir.resolve("main.lyng")
writeTransitiveImportTree(dir)
Files.writeString(
mainFile,
"""
import package1.entry
import package1.beta
import package1.nested.gamma
println(report())
println(gammaValue())
""".trimIndent()
)
val result = runCli(mainFile.toString())
assertTrue(result.err, result.err.isBlank())
assertTrue(
result.out,
result.out.contains("alpha|beta|gamma|entry")
)
assertTrue(
result.out,
result.out.contains("alpha|beta|gamma")
)
} finally {
dir.toFile().deleteRecursively()
}
}
}
@@ -0,0 +1,141 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng_cli
import com.sun.net.httpserver.HttpExchange
import com.sun.net.httpserver.HttpServer
import net.sergeych.jvmExitImpl
import net.sergeych.runMain
import org.junit.After
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Before
import org.junit.Test
import java.io.ByteArrayOutputStream
import java.io.PrintStream
import java.net.InetSocketAddress
class CliNetworkJvmTest {
private val originalOut: PrintStream = System.out
private val originalErr: PrintStream = System.err
private class TestExit(val code: Int) : RuntimeException()
@Before
fun setUp() {
jvmExitImpl = { code -> throw TestExit(code) }
}
@After
fun tearDown() {
System.setOut(originalOut)
System.setErr(originalErr)
jvmExitImpl = { code -> kotlin.system.exitProcess(code) }
}
private data class CliResult(val out: String, val err: String, val exitCode: Int?)
private fun runCli(vararg args: String): CliResult {
val outBuf = ByteArrayOutputStream()
val errBuf = ByteArrayOutputStream()
System.setOut(PrintStream(outBuf, true, Charsets.UTF_8))
System.setErr(PrintStream(errBuf, true, Charsets.UTF_8))
var exitCode: Int? = null
try {
runMain(arrayOf(*args))
} catch (e: TestExit) {
exitCode = e.code
} finally {
System.out.flush()
System.err.flush()
}
return CliResult(outBuf.toString("UTF-8"), errBuf.toString("UTF-8"), exitCode)
}
@Test
fun cliHasAllNetworkingModulesInstalled() {
val server = newServer()
try {
val script = """
import lyng.io.http
import lyng.io.ws
import lyng.io.net
assert(Http.isSupported())
println("ws=" + Ws.isSupported())
println("net=" + Net.isSupported())
val home = Http.get("http://127.0.0.1:${server.address.port}/").text()
val jsRef = "src=\"([^\"]*lyng-version\\.js)\"".re.find(home)
require(jsRef != null, "lyng-version.js reference not found")
val versionJsPath = (jsRef as RegexMatch)[1]
val versionJs = Http.get("http://127.0.0.1:${server.address.port}/" + versionJsPath).text()
val versionMatch = "LYNG_VERSION\\s*=\\s*\"([^\"]+)\"".re.find(versionJs)
require(versionMatch != null, "LYNG_VERSION assignment not found")
println("version=" + ((versionMatch as RegexMatch)[1]))
""".trimIndent()
val result = runCli("-x", script)
assertNull(result.exitCode)
assertTrue(result.err, result.err.isBlank())
assertTrue(result.out, result.out.contains("ws="))
assertTrue(result.out, result.out.contains("net="))
assertTrue(result.out, result.out.contains("version=9.9.9-test"))
} finally {
server.stop(0)
}
}
private fun newServer(): HttpServer {
val server = HttpServer.create(InetSocketAddress("127.0.0.1", 0), 0)
server.createContext("/") { exchange ->
when (exchange.requestURI.path) {
"/" -> writeResponse(
exchange,
200,
"""
<!doctype html>
<html>
<head>
<script src="lyng-version.js"></script>
</head>
<body>
<span id="lyng-version-ribbon"></span>
</body>
</html>
""".trimIndent()
)
"/lyng-version.js" -> writeResponse(exchange, 200, """window.LYNG_VERSION = "9.9.9-test";""")
else -> writeResponse(exchange, 404, "missing")
}
}
server.start()
return server
}
private fun writeResponse(exchange: HttpExchange, status: Int, body: String) {
val bytes = body.toByteArray()
exchange.sendResponseHeaders(status, bytes.size.toLong())
exchange.responseBody.use { out ->
out.write(bytes)
}
}
}
@@ -0,0 +1,129 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng_cli
import kotlinx.cinterop.*
import okio.FileSystem
import okio.Path
import okio.Path.Companion.toPath
import platform.posix.O_CREAT
import platform.posix.O_TRUNC
import platform.posix.O_WRONLY
import platform.posix.SIGTERM
import platform.posix._exit
import platform.posix.close
import platform.posix.dup2
import platform.posix.execvp
import platform.posix.fork
import platform.posix.getenv
import platform.posix.getpid
import platform.posix.kill
import platform.posix.open
import platform.posix.usleep
import platform.posix.waitpid
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
@OptIn(ExperimentalForeignApi::class)
class CliAtExitLinuxNativeTest {
@Test
fun atExitRunsOnSigtermForNativeCli() {
val executable = getenv("LYNG_CLI_NATIVE_BIN")?.toKString()
?: error("LYNG_CLI_NATIVE_BIN is not set")
val fs = FileSystem.SYSTEM
val tempDir = "/tmp/lyng_cli_native_${getpid()}_${kotlin.random.Random.nextInt()}".toPath()
val scriptPath = tempDir / "sigterm.lyng"
val stdoutPath = tempDir / "stdout.txt"
val stderrPath = tempDir / "stderr.txt"
fs.createDirectories(tempDir)
try {
fs.write(scriptPath) {
writeUtf8(
"""
atExit {
println("cleanup-native")
}
while(true) {
yield()
}
""".trimIndent()
)
}
val pid = launchCli(executable, scriptPath, stdoutPath, stderrPath)
usleep(300_000u)
assertEquals(0, kill(pid, SIGTERM), "failed to send SIGTERM")
val status = waitForPid(pid)
val exitCode = if ((status and 0x7f) == 0) (status shr 8) and 0xff else -1
val stdout = readUtf8IfExists(fs, stdoutPath)
val stderr = readUtf8IfExists(fs, stderrPath)
assertEquals(143, exitCode, "unexpected native CLI exit status; stderr=$stderr")
assertTrue(stdout.contains("cleanup-native"), "stdout did not contain cleanup marker. stdout=$stdout stderr=$stderr")
} finally {
fs.deleteRecursively(tempDir, mustExist = false)
}
}
private fun readUtf8IfExists(fs: FileSystem, path: Path): String {
return if (fs.exists(path)) {
fs.read(path) { readUtf8() }
} else {
""
}
}
private fun waitForPid(pid: Int): Int = memScoped {
val status = alloc<IntVar>()
val waited = waitpid(pid, status.ptr, 0)
check(waited == pid) { "waitpid failed for $pid" }
status.value
}
private fun launchCli(
executable: String,
scriptPath: Path,
stdoutPath: Path,
stderrPath: Path
): Int = memScoped {
val pid = fork()
check(pid >= 0) { "fork failed" }
if (pid == 0) {
val stdoutFd = open(stdoutPath.toString(), O_WRONLY or O_CREAT or O_TRUNC, 0x1A4)
val stderrFd = open(stderrPath.toString(), O_WRONLY or O_CREAT or O_TRUNC, 0x1A4)
if (stdoutFd < 0 || stderrFd < 0) {
_exit(2)
}
dup2(stdoutFd, 1)
dup2(stderrFd, 2)
close(stdoutFd)
close(stderrFd)
val argv = allocArray<CPointerVar<ByteVar>>(3)
argv[0] = executable.cstr.ptr
argv[1] = scriptPath.toString().cstr.ptr
argv[2] = null
execvp(executable, argv)
_exit(127)
}
pid
}
}
@@ -0,0 +1,132 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng_cli
import kotlinx.cinterop.*
import okio.FileSystem
import okio.Path
import okio.Path.Companion.toPath
import platform.posix.O_CREAT
import platform.posix.O_TRUNC
import platform.posix.O_WRONLY
import platform.posix.SIGKILL
import platform.posix.SIGSEGV
import platform.posix._exit
import platform.posix.close
import platform.posix.dup2
import platform.posix.execvp
import platform.posix.fork
import platform.posix.getenv
import platform.posix.getpid
import platform.posix.kill
import platform.posix.open
import platform.posix.usleep
import platform.posix.waitpid
import kotlin.test.Test
import kotlin.test.assertFalse
import kotlin.test.assertTrue
@OptIn(ExperimentalForeignApi::class)
class CliWebSocketNativeRegressionTest {
@Test
fun releaseCliDoesNotSegfaultOnConcurrentWebSocketClients() {
val executable = getenv("LYNG_CLI_NATIVE_RELEASE_BIN")?.toKString()
?: error("LYNG_CLI_NATIVE_RELEASE_BIN is not set")
val fs = FileSystem.SYSTEM
val repoRoot = ascend(executable.toPath(), 6)
val scriptPath = repoRoot / "bugs" / "ws-segfault.lyng"
check(fs.exists(scriptPath)) { "bug repro script not found at $scriptPath" }
val tempDir = "/tmp/lyng_ws_native_${getpid()}_${kotlin.random.Random.nextInt()}".toPath()
val stdoutPath = tempDir / "stdout.txt"
val stderrPath = tempDir / "stderr.txt"
fs.createDirectories(tempDir)
try {
val pid = launchCli(executable, scriptPath, stdoutPath, stderrPath)
usleep(5_000_000u)
if (kill(pid, 0) == 0) {
kill(pid, SIGKILL)
}
val status = waitForPid(pid)
val termSignal = status and 0x7f
val stdout = readUtf8IfExists(fs, stdoutPath)
val stderr = readUtf8IfExists(fs, stderrPath)
val allOutput = "$stdout\n$stderr"
assertFalse(termSignal == SIGSEGV, "native CLI crashed with SIGSEGV. Output:\n$allOutput")
assertTrue(
stdout.lineSequence().count { it == "test send to ws://127.0.0.1:9998... OK" } == 2,
"expected both websocket clients to finish. Output:\n$allOutput"
)
assertFalse(allOutput.contains("Segmentation fault"), "process output reported a segmentation fault:\n$allOutput")
} finally {
fs.deleteRecursively(tempDir, mustExist = false)
}
}
private fun ascend(path: Path, levels: Int): Path {
var current = path
repeat(levels) {
current = current.parent ?: error("cannot ascend $levels levels from $path")
}
return current
}
private fun readUtf8IfExists(fs: FileSystem, path: Path): String {
return if (fs.exists(path)) fs.read(path) { readUtf8() } else ""
}
private fun waitForPid(pid: Int): Int = memScoped {
val status = alloc<IntVar>()
val waited = waitpid(pid, status.ptr, 0)
check(waited == pid) { "waitpid failed for $pid" }
status.value
}
private fun launchCli(
executable: String,
scriptPath: Path,
stdoutPath: Path,
stderrPath: Path,
): Int = memScoped {
val pid = fork()
check(pid >= 0) { "fork failed" }
if (pid == 0) {
val stdoutFd = open(stdoutPath.toString(), O_WRONLY or O_CREAT or O_TRUNC, 0x1A4)
val stderrFd = open(stderrPath.toString(), O_WRONLY or O_CREAT or O_TRUNC, 0x1A4)
if (stdoutFd < 0 || stderrFd < 0) {
_exit(2)
}
dup2(stdoutFd, 1)
dup2(stderrFd, 2)
close(stdoutFd)
close(stderrFd)
val argv = allocArray<CPointerVar<ByteVar>>(3)
argv[0] = executable.cstr.ptr
argv[1] = scriptPath.toString().cstr.ptr
argv[2] = null
execvp(executable, argv)
_exit(127)
}
pid
}
}
+48 -1
View File
@@ -22,11 +22,40 @@
package net.sergeych package net.sergeych
import kotlinx.cinterop.* import kotlinx.cinterop.*
import kotlin.native.concurrent.ThreadLocal
import platform.posix.fgets import platform.posix.fgets
import platform.posix.pclose import platform.posix.pclose
import platform.posix.popen import platform.posix.popen
import platform.posix.signal
import platform.posix.atexit
import platform.posix.SIGINT
import platform.posix.SIGHUP
import platform.posix.SIGTERM
import kotlin.system.exitProcess import kotlin.system.exitProcess
@ThreadLocal
private var activeCliRuntime: CliExecutionRuntime? = null
@ThreadLocal
private var nativeCliHooksInstalled: Boolean = false
private fun installNativeCliHooksOnce() {
if (nativeCliHooksInstalled) return
nativeCliHooksInstalled = true
atexit(staticCFunction(::nativeCliAtExit))
signal(SIGTERM, staticCFunction(::nativeCliSignalHandler))
signal(SIGINT, staticCFunction(::nativeCliSignalHandler))
signal(SIGHUP, staticCFunction(::nativeCliSignalHandler))
}
private fun nativeCliAtExit() {
activeCliRuntime?.shutdownBlocking()
}
private fun nativeCliSignalHandler(signal: Int) {
exitProcess(128 + signal)
}
actual class ShellCommandExecutor() { actual class ShellCommandExecutor() {
actual fun executeCommand(command: String): CommandResult { actual fun executeCommand(command: String): CommandResult {
val outputBuilder = StringBuilder() val outputBuilder = StringBuilder()
@@ -62,6 +91,24 @@ actual class ShellCommandExecutor() {
} }
} }
internal actual class CliPlatformShutdownHooks private constructor(
private val runtime: CliExecutionRuntime
) {
actual fun uninstall() {
if (activeCliRuntime === runtime) {
activeCliRuntime = null
}
}
actual companion object {
actual fun install(runtime: CliExecutionRuntime): CliPlatformShutdownHooks {
installNativeCliHooksOnce()
activeCliRuntime = runtime
return CliPlatformShutdownHooks(runtime)
}
}
}
actual fun exit(code: Int) { actual fun exit(code: Int) {
exitProcess(code) exitProcess(code)
} }
+70
View File
@@ -31,6 +31,33 @@ plugins {
group = "net.sergeych" group = "net.sergeych"
version = "0.0.1-SNAPSHOT" version = "0.0.1-SNAPSHOT"
private fun Project.sqliteLinuxLinkerOpts(vararg defaultDirs: String): List<String> {
val overrideDir = providers.gradleProperty("sqlite3.lib.dir").orNull
?: providers.environmentVariable("SQLITE3_LIB_DIR").orNull
val candidateDirs = buildList {
if (!overrideDir.isNullOrBlank()) {
add(file(overrideDir))
}
defaultDirs.forEach { add(file(it)) }
}.distinctBy { it.absolutePath }
val discoveredLib = sequenceOf("libsqlite3.so", "libsqlite3.so.0")
.mapNotNull { libraryName ->
candidateDirs.firstOrNull { it.resolve(libraryName).isFile }?.let { dir ->
listOf("-L${dir.absolutePath}", "-l:$libraryName")
}
}
.firstOrNull()
?: listOf("-lsqlite3")
return discoveredLib + listOf(
"-ldl",
"-lpthread",
"-lm",
"-Wl,--allow-shlib-undefined"
)
}
kotlin { kotlin {
jvmToolchain(17) jvmToolchain(17)
jvm() jvm()
@@ -58,6 +85,39 @@ kotlin {
// nodejs() // nodejs()
// } // }
targets.withType(org.jetbrains.kotlin.gradle.plugin.mpp.KotlinNativeTarget::class.java).configureEach {
compilations.getByName("main").cinterops.create("sqlite3") {
defFile(project.file("src/nativeInterop/cinterop/sqlite/sqlite3.def"))
packageName("net.sergeych.lyng.io.db.sqlite.cinterop")
includeDirs(project.file("src/nativeInterop/cinterop/sqlite"))
}
binaries.all {
when (konanTarget.name) {
"linux_x64" -> linkerOpts(
*project.sqliteLinuxLinkerOpts(
"/lib/x86_64-linux-gnu",
"/usr/lib/x86_64-linux-gnu",
"/lib64",
"/usr/lib64",
"/lib",
"/usr/lib"
).toTypedArray()
)
"linux_arm64" -> linkerOpts(
*project.sqliteLinuxLinkerOpts(
"/lib/aarch64-linux-gnu",
"/usr/lib/aarch64-linux-gnu",
"/lib64",
"/usr/lib64",
"/lib",
"/usr/lib"
).toTypedArray()
)
else -> linkerOpts("-lsqlite3")
}
}
}
// Keep expect/actual warning suppressed consistently with other modules // Keep expect/actual warning suppressed consistently with other modules
targets.configureEach { targets.configureEach {
compilations.configureEach { compilations.configureEach {
@@ -120,6 +180,12 @@ kotlin {
implementation(libs.kotlinx.coroutines.test) implementation(libs.kotlinx.coroutines.test)
} }
} }
val jvmTest by getting {
dependencies {
implementation(libs.testcontainers)
implementation(libs.testcontainers.postgresql)
}
}
val linuxTest by creating { val linuxTest by creating {
dependsOn(commonTest) dependsOn(commonTest)
} }
@@ -155,6 +221,10 @@ kotlin {
implementation("org.jline:jline-terminal:3.29.0") implementation("org.jline:jline-terminal:3.29.0")
implementation(libs.ktor.client.cio) implementation(libs.ktor.client.cio)
implementation(libs.ktor.network) implementation(libs.ktor.network)
implementation(libs.sqlite.jdbc)
implementation(libs.h2)
implementation(libs.postgresql)
implementation(libs.hikaricp)
} }
} }
// // For Wasm we use in-memory VFS for now // // For Wasm we use in-memory VFS for now
@@ -0,0 +1,22 @@
package net.sergeych.lyng.io.db.jdbc
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.requireScope
internal actual suspend fun openJdbcBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: JdbcOpenOptions,
): SqlDatabaseBackend {
scope.raiseError(
ObjException(
core.databaseException,
scope.requireScope(),
ObjString("lyng.io.db.jdbc is available only on the JVM target")
)
)
}
@@ -0,0 +1,39 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.requireScope
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjString
internal actual suspend fun openSqliteBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: SqliteOpenOptions,
): SqlDatabaseBackend {
scope.raiseError(
ObjException(
core.databaseException,
scope.requireScope(),
ObjString("SQLite provider is not implemented on this platform yet")
)
)
}
@@ -28,6 +28,8 @@ import java.net.InetAddress
actual fun getSystemNetEngine(): LyngNetEngine = AndroidKtorNetEngine actual fun getSystemNetEngine(): LyngNetEngine = AndroidKtorNetEngine
actual fun shutdownSystemNetEngine() {}
private object AndroidKtorNetEngine : LyngNetEngine { private object AndroidKtorNetEngine : LyngNetEngine {
private val selectorManager: SelectorManager by lazy { ActorSelectorManager(Dispatchers.IO) } private val selectorManager: SelectorManager by lazy { ActorSelectorManager(Dispatchers.IO) }
@@ -17,6 +17,7 @@
package net.sergeych.lyng.io.console package net.sergeych.lyng.io.console
import kotlinx.coroutines.delay
import net.sergeych.lyng.ModuleScope import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Scope import net.sergeych.lyng.Scope
import net.sergeych.lyng.ScopeFacade import net.sergeych.lyng.ScopeFacade
@@ -58,18 +59,31 @@ fun createConsoleModule(policy: ConsoleAccessPolicy, manager: ImportManager): Bo
if (manager.packageNames.contains(CONSOLE_MODULE_NAME)) return false if (manager.packageNames.contains(CONSOLE_MODULE_NAME)) return false
manager.addPackage(CONSOLE_MODULE_NAME) { module -> manager.addPackage(CONSOLE_MODULE_NAME) { module ->
buildConsoleModule(module, policy) buildConsoleModule(module, policy, getSystemConsole())
} }
return true return true
} }
fun createConsole(policy: ConsoleAccessPolicy, manager: ImportManager): Boolean = createConsoleModule(policy, manager) fun createConsole(policy: ConsoleAccessPolicy, manager: ImportManager): Boolean = createConsoleModule(policy, manager)
private suspend fun buildConsoleModule(module: ModuleScope, policy: ConsoleAccessPolicy) { internal fun createConsoleModule(
policy: ConsoleAccessPolicy,
manager: ImportManager,
console: LyngConsole,
): Boolean {
if (manager.packageNames.contains(CONSOLE_MODULE_NAME)) return false
manager.addPackage(CONSOLE_MODULE_NAME) { module ->
buildConsoleModule(module, policy, console)
}
return true
}
private suspend fun buildConsoleModule(module: ModuleScope, policy: ConsoleAccessPolicy, baseConsole: LyngConsole) {
// Load Lyng declarations for console enums/types first (module-local source of truth). // Load Lyng declarations for console enums/types first (module-local source of truth).
module.eval(Source(CONSOLE_MODULE_NAME, consoleLyng)) module.eval(Source(CONSOLE_MODULE_NAME, consoleLyng))
ConsoleEnums.initialize(module) ConsoleEnums.initialize(module)
val console: LyngConsole = LyngConsoleSecured(getSystemConsole(), policy) val console: LyngConsole = LyngConsoleSecured(baseConsole, policy)
val consoleType = object : net.sergeych.lyng.obj.ObjClass("Console") {} val consoleType = object : net.sergeych.lyng.obj.ObjClass("Console") {}
@@ -179,7 +193,7 @@ private suspend fun buildConsoleModule(module: ModuleScope, policy: ConsoleAcces
addClassFn("events") { addClassFn("events") {
consoleGuard { consoleGuard {
console.events().toConsoleEventStream() ObjConsoleEventStream { console.events() }
} }
} }
@@ -210,12 +224,8 @@ private suspend inline fun ScopeFacade.consoleGuard(crossinline block: suspend (
} }
} }
private fun ConsoleEventSource.toConsoleEventStream(): ObjConsoleEventStream {
return ObjConsoleEventStream(this)
}
private class ObjConsoleEventStream( private class ObjConsoleEventStream(
private val source: ConsoleEventSource, private val sourceFactory: () -> ConsoleEventSource,
) : Obj() { ) : Obj() {
override val objClass: net.sergeych.lyng.obj.ObjClass override val objClass: net.sergeych.lyng.obj.ObjClass
get() = type get() = type
@@ -224,35 +234,61 @@ private class ObjConsoleEventStream(
val type = net.sergeych.lyng.obj.ObjClass("ConsoleEventStream", ObjIterable).apply { val type = net.sergeych.lyng.obj.ObjClass("ConsoleEventStream", ObjIterable).apply {
addFn("iterator") { addFn("iterator") {
val stream = thisAs<ObjConsoleEventStream>() val stream = thisAs<ObjConsoleEventStream>()
ObjConsoleEventIterator(stream.source) ObjConsoleEventIterator(stream.sourceFactory)
} }
} }
} }
} }
private class ObjConsoleEventIterator( private class ObjConsoleEventIterator(
private val source: ConsoleEventSource, private val sourceFactory: () -> ConsoleEventSource,
) : Obj() { ) : Obj() {
private var cached: Obj? = null private var cached: Obj? = null
private var closed = false private var closed = false
private var source: ConsoleEventSource? = null
override val objClass: net.sergeych.lyng.obj.ObjClass override val objClass: net.sergeych.lyng.obj.ObjClass
get() = type get() = type
private fun ensureSource(): ConsoleEventSource {
val current = source
if (current != null) return current
return sourceFactory().also { source = it }
}
private suspend fun recycleSource(reason: String, error: Throwable? = null) {
if (error != null) {
consoleFlowDebug(reason, error)
} else {
consoleFlowDebug(reason)
}
val current = source
source = null
runCatching { current?.close() }
.onFailure { consoleFlowDebug("console-bridge: failed to close recycled source", it) }
if (!closed) delay(25)
}
private suspend fun ensureCached(): Boolean { private suspend fun ensureCached(): Boolean {
if (closed) return false if (closed) return false
if (cached != null) return true if (cached != null) return true
while (!closed && cached == null) { while (!closed && cached == null) {
val event = try { val currentSource = try {
source.nextEvent() ensureSource()
} catch (e: Throwable) { } catch (e: Throwable) {
// Consumer loops must survive source/read failures: report and keep polling. recycleSource("console-bridge: source creation failed; retrying", e)
consoleFlowDebug("console-bridge: nextEvent failed; dropping failure and continuing", e) continue
}
val event = try {
currentSource.nextEvent()
} catch (e: Throwable) {
// Consumer loops must survive source/read failures: rebuild the source and keep polling.
recycleSource("console-bridge: nextEvent failed; recycling source", e)
continue continue
} }
if (event == null) { if (event == null) {
closeSource() recycleSource("console-bridge: source ended; recreating")
return false continue
} }
cached = try { cached = try {
event.toObjEvent() event.toObjEvent()
@@ -268,10 +304,10 @@ private class ObjConsoleEventIterator(
private suspend fun closeSource() { private suspend fun closeSource() {
if (closed) return if (closed) return
closed = true closed = true
// Do not close the underlying console source from VM iterator cancellation. val current = source
// CmdFrame.cancelIterators() may call cancelIteration() while user code is still source = null
// expected to keep processing input (e.g. recover from app-level exceptions). runCatching { current?.close() }
// The source lifecycle is managed by the console runtime. .onFailure { consoleFlowDebug("console-bridge: failed to close iterator source", it) }
} }
suspend fun hasNext(): Boolean = ensureCached() suspend fun hasNext(): Boolean = ensureCached()
@@ -0,0 +1,142 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.Source
import net.sergeych.lyng.Arguments
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.ObjVoid
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.pacman.ImportManager
import net.sergeych.lyng.requireScope
import net.sergeych.lyngio.stdlib_included.dbLyng
private const val DB_MODULE_NAME = "lyng.io.db"
fun createDbModule(scope: Scope): Boolean = createDbModule(scope.importManager)
fun createDb(scope: Scope): Boolean = createDbModule(scope)
fun createDbModule(manager: ImportManager): Boolean {
if (manager.packageNames.contains(DB_MODULE_NAME)) return false
manager.addPackage(DB_MODULE_NAME) { module ->
buildDbModule(module)
}
return true
}
fun createDb(manager: ImportManager): Boolean = createDbModule(manager)
private suspend fun buildDbModule(module: ModuleScope) {
module.eval(Source(DB_MODULE_NAME, dbLyng))
val exceptions = installDbExceptionClasses(module)
val registry = DbProviderRegistry()
module.addFn("registerDatabaseProvider") {
val scheme = requiredArg<ObjString>(0).value
val opener = args.list.getOrNull(1)
?: raiseError("Expected exactly 2 arguments, got ${args.list.size}")
registry.register(this, scheme, opener)
ObjVoid
}
module.addFn("openDatabase") {
val connectionUrl = requiredArg<ObjString>(0).value
val extraParams = args.list.getOrNull(1)
?: raiseError("Expected exactly 2 arguments, got ${args.list.size}")
if (!extraParams.isInstanceOf("Map")) {
raiseIllegalArgument("extraParams must be Map")
}
val scheme = parseConnectionScheme(connectionUrl)
?: raiseIllegalArgument("Malformed database connection URL: $connectionUrl")
val opener = registry.providers[scheme]
?: raiseDatabaseException(exceptions.database, "No database provider registered for scheme '$scheme'")
call(opener, Arguments(listOf(ObjString(connectionUrl), extraParams)), newThisObj = ObjNull)
}
}
private data class DbExceptionClasses(
val database: ObjException.Companion.ExceptionClass,
val sqlExecution: ObjException.Companion.ExceptionClass,
val sqlConstraint: ObjException.Companion.ExceptionClass,
val sqlUsage: ObjException.Companion.ExceptionClass,
val rollback: ObjException.Companion.ExceptionClass,
)
private fun installDbExceptionClasses(module: ModuleScope): DbExceptionClasses {
val database = ObjException.Companion.ExceptionClass("DatabaseException", ObjException.Root)
val sqlExecution = ObjException.Companion.ExceptionClass("SqlExecutionException", database)
val sqlConstraint = ObjException.Companion.ExceptionClass("SqlConstraintException", sqlExecution)
val sqlUsage = ObjException.Companion.ExceptionClass("SqlUsageException", database)
val rollback = ObjException.Companion.ExceptionClass("RollbackException", ObjException.Root)
module.addConst("DatabaseException", database)
module.addConst("SqlExecutionException", sqlExecution)
module.addConst("SqlConstraintException", sqlConstraint)
module.addConst("SqlUsageException", sqlUsage)
module.addConst("RollbackException", rollback)
return DbExceptionClasses(
database = database,
sqlExecution = sqlExecution,
sqlConstraint = sqlConstraint,
sqlUsage = sqlUsage,
rollback = rollback,
)
}
private class DbProviderRegistry {
val providers: MutableMap<String, Obj> = linkedMapOf()
fun register(scope: ScopeFacade, rawScheme: String, opener: Obj) {
val scheme = normalizeScheme(rawScheme)
?: scope.raiseIllegalArgument("Database provider scheme must not be empty")
if (!opener.isInstanceOf("Callable")) {
scope.raiseIllegalArgument("Database provider opener must be callable")
}
if (providers.containsKey(scheme)) {
scope.raiseIllegalState("Database provider already registered for scheme '$scheme'")
}
providers[scheme] = opener
}
}
private fun normalizeScheme(rawScheme: String): String? {
val trimmed = rawScheme.trim()
if (trimmed.isEmpty()) return null
if (':' in trimmed) return null
return trimmed.lowercase()
}
private fun parseConnectionScheme(connectionUrl: String): String? {
val colonIndex = connectionUrl.indexOf(':')
if (colonIndex <= 0) return null
return normalizeScheme(connectionUrl.substring(0, colonIndex))
}
private fun ScopeFacade.raiseDatabaseException(
exceptionClass: ObjException.Companion.ExceptionClass,
message: String,
): Nothing = raiseError(ObjException(exceptionClass, requireScope(), ObjString(message)))
@@ -0,0 +1,399 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db
import net.sergeych.lyng.Arguments
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Scope
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjBool
import net.sergeych.lyng.obj.ObjClass
import net.sergeych.lyng.obj.ObjEnumClass
import net.sergeych.lyng.obj.ObjEnumEntry
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjImmutableList
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.thisAs
import net.sergeych.lyng.requireScope
import kotlin.collections.List
import kotlin.collections.Map
import kotlin.collections.MutableList
import kotlin.collections.associateWith
import kotlin.collections.drop
import kotlin.collections.first
import kotlin.collections.forEachIndexed
import kotlin.collections.getOrNull
import kotlin.collections.getOrPut
import kotlin.collections.indices
import kotlin.collections.linkedMapOf
import kotlin.collections.listOf
import kotlin.collections.map
import kotlin.collections.mutableListOf
import kotlin.text.lowercase
internal data class SqlColumnMeta(
val name: String,
val sqlType: ObjEnumEntry,
val nullable: Boolean,
val nativeType: String,
)
internal data class SqlResultSetData(
val columns: List<SqlColumnMeta>,
val rows: List<List<Obj>>,
)
internal data class SqlExecutionResultData(
val affectedRowsCount: Int,
val generatedKeys: SqlResultSetData,
)
internal interface SqlDatabaseBackend {
suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T
fun close() {}
}
internal interface SqlTransactionBackend {
suspend fun select(scope: ScopeFacade, clause: String, params: List<Obj>): SqlResultSetData
suspend fun execute(scope: ScopeFacade, clause: String, params: List<Obj>): SqlExecutionResultData
suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T
}
internal class SqlCoreModule private constructor(
val module: ModuleScope,
val databaseClass: ObjClass,
val transactionClass: ObjClass,
val resultSetClass: ObjClass,
val rowClass: ObjClass,
val columnClass: ObjClass,
val executionResultClass: ObjClass,
val databaseException: ObjException.Companion.ExceptionClass,
val sqlExecutionException: ObjException.Companion.ExceptionClass,
val sqlConstraintException: ObjException.Companion.ExceptionClass,
val sqlUsageException: ObjException.Companion.ExceptionClass,
val rollbackException: ObjException.Companion.ExceptionClass,
val sqlTypes: SqlTypeEntries,
) {
companion object {
fun resolve(module: ModuleScope): SqlCoreModule = SqlCoreModule(
module = module,
databaseClass = module.requireClass("Database"),
transactionClass = module.requireClass("SqlTransaction"),
resultSetClass = module.requireClass("ResultSet"),
rowClass = module.requireClass("SqlRow"),
columnClass = module.requireClass("SqlColumn"),
executionResultClass = module.requireClass("ExecutionResult"),
databaseException = module.requireClass("DatabaseException") as ObjException.Companion.ExceptionClass,
sqlExecutionException = module.requireClass("SqlExecutionException") as ObjException.Companion.ExceptionClass,
sqlConstraintException = module.requireClass("SqlConstraintException") as ObjException.Companion.ExceptionClass,
sqlUsageException = module.requireClass("SqlUsageException") as ObjException.Companion.ExceptionClass,
rollbackException = module.requireClass("RollbackException") as ObjException.Companion.ExceptionClass,
sqlTypes = SqlTypeEntries.resolve(module),
)
}
}
internal class SqlTypeEntries private constructor(
private val entries: Map<String, ObjEnumEntry>,
) {
fun require(name: String): ObjEnumEntry = entries[name]
?: error("lyng.io.db.SqlType entry is missing: $name")
companion object {
fun resolve(module: ModuleScope): SqlTypeEntries {
val enumClass = resolveEnum(module, "SqlType")
return SqlTypeEntries(
listOf(
"Binary", "String", "Int", "Double", "Decimal",
"Bool", "Instant", "Date", "DateTime"
).associateWith { name ->
enumClass.byName[ObjString(name)] as? ObjEnumEntry
?: error("lyng.io.db.SqlType.$name is missing")
}
)
}
private fun resolveEnum(module: ModuleScope, enumName: String): ObjEnumClass {
val local = module.get(enumName)?.value as? ObjEnumClass
if (local != null) return local
val root = module.importProvider.rootScope.get(enumName)?.value as? ObjEnumClass
return root ?: error("lyng.io.db declaration enum is missing: $enumName")
}
}
}
internal class SqlRuntimeTypes private constructor(
val core: SqlCoreModule,
val databaseClass: ObjClass,
val transactionClass: ObjClass,
val resultSetClass: ObjClass,
val rowClass: ObjClass,
val columnClass: ObjClass,
val executionResultClass: ObjClass,
) {
companion object {
fun create(prefix: String, core: SqlCoreModule): SqlRuntimeTypes {
val databaseClass = object : ObjClass("${prefix}Database", core.databaseClass) {}
val transactionClass = object : ObjClass("${prefix}Transaction", core.transactionClass) {}
val resultSetClass = object : ObjClass("${prefix}ResultSet", core.resultSetClass) {}
val rowClass = object : ObjClass("${prefix}Row", core.rowClass) {}
val columnClass = object : ObjClass("${prefix}Column", core.columnClass) {}
val executionResultClass = object : ObjClass("${prefix}ExecutionResult", core.executionResultClass) {}
val runtime = SqlRuntimeTypes(
core = core,
databaseClass = databaseClass,
transactionClass = transactionClass,
resultSetClass = resultSetClass,
rowClass = rowClass,
columnClass = columnClass,
executionResultClass = executionResultClass,
)
runtime.bind()
return runtime
}
}
private fun bind() {
databaseClass.addFn("close") {
thisAs<SqlDatabaseObj>().backend.close()
ObjNull
}
databaseClass.addFn("transaction") {
val self = thisAs<SqlDatabaseObj>()
val block = args.list.getOrNull(0) ?: raiseError("Expected exactly 1 argument, got ${args.list.size}")
if (!block.isInstanceOf("Callable")) {
raiseClassCastError("transaction block must be callable")
}
self.backend.transaction(this) { backend ->
val lifetime = SqlTransactionLifetime(this@SqlRuntimeTypes.core)
try {
call(block, Arguments(SqlTransactionObj(this@SqlRuntimeTypes, backend, lifetime)), ObjNull)
} finally {
lifetime.close()
}
}
}
transactionClass.addFn("select") {
val self = thisAs<SqlTransactionObj>()
self.lifetime.ensureActive(this)
val clause = (args.list.getOrNull(0) as? ObjString)?.value
?: raiseClassCastError("query must be String")
val params = args.list.drop(1)
SqlResultSetObj(self.types, self.lifetime, self.backend.select(this, clause, params))
}
transactionClass.addFn("execute") {
val self = thisAs<SqlTransactionObj>()
self.lifetime.ensureActive(this)
val clause = (args.list.getOrNull(0) as? ObjString)?.value
?: raiseClassCastError("query must be String")
val params = args.list.drop(1)
SqlExecutionResultObj(self.types, self.lifetime, self.backend.execute(this, clause, params))
}
transactionClass.addFn("transaction") {
val self = thisAs<SqlTransactionObj>()
self.lifetime.ensureActive(this)
val block = args.list.getOrNull(0) ?: raiseError("Expected exactly 1 argument, got ${args.list.size}")
if (!block.isInstanceOf("Callable")) {
raiseClassCastError("transaction block must be callable")
}
self.backend.transaction(this) { backend ->
val lifetime = SqlTransactionLifetime(this@SqlRuntimeTypes.core)
try {
call(block, Arguments(SqlTransactionObj(self.types, backend, lifetime)), ObjNull)
} finally {
lifetime.close()
}
}
}
resultSetClass.addProperty("columns", getter = {
val self = thisAs<SqlResultSetObj>()
self.lifetime.ensureActive(this)
ObjImmutableList(self.columns)
})
resultSetClass.addFn("size") {
val self = thisAs<SqlResultSetObj>()
self.lifetime.ensureActive(this)
ObjInt.of(self.rows.size.toLong())
}
resultSetClass.addFn("isEmpty") {
val self = thisAs<SqlResultSetObj>()
self.lifetime.ensureActive(this)
ObjBool(self.rows.isEmpty())
}
resultSetClass.addFn("iterator") {
val self = thisAs<SqlResultSetObj>()
self.lifetime.ensureActive(this)
ObjImmutableList(self.rows).invokeInstanceMethod(requireScope(), "iterator")
}
resultSetClass.addFn("toList") {
val self = thisAs<SqlResultSetObj>()
self.lifetime.ensureActive(this)
ObjImmutableList(self.rows)
}
rowClass.addProperty("size", getter = {
val self = thisAs<SqlRowObj>()
ObjInt.of(self.values.size.toLong())
})
rowClass.addProperty("values", getter = {
val self = thisAs<SqlRowObj>()
ObjImmutableList(self.values)
})
columnClass.addProperty("name", getter = { ObjString(thisAs<SqlColumnObj>().meta.name) })
columnClass.addProperty("sqlType", getter = { thisAs<SqlColumnObj>().meta.sqlType })
columnClass.addProperty("nullable", getter = { ObjBool(thisAs<SqlColumnObj>().meta.nullable) })
columnClass.addProperty("nativeType", getter = { ObjString(thisAs<SqlColumnObj>().meta.nativeType) })
executionResultClass.addProperty("affectedRowsCount", getter = {
val self = thisAs<SqlExecutionResultObj>()
self.lifetime.ensureActive(this)
ObjInt.of(self.result.affectedRowsCount.toLong())
})
executionResultClass.addFn("getGeneratedKeys") {
val self = thisAs<SqlExecutionResultObj>()
self.lifetime.ensureActive(this)
SqlResultSetObj(self.types, self.lifetime, self.result.generatedKeys)
}
}
}
internal class SqlTransactionLifetime(
private val core: SqlCoreModule,
) {
private var active = true
fun close() {
active = false
}
fun ensureActive(scope: ScopeFacade) {
if (!active) {
scope.raiseError(
ObjException(core.sqlUsageException, scope.requireScope(), ObjString("SQL result can be used only while its transaction is active"))
)
}
}
}
internal class SqlDatabaseObj(
val types: SqlRuntimeTypes,
val backend: SqlDatabaseBackend,
) : Obj() {
override val objClass: ObjClass
get() = types.databaseClass
}
internal class SqlTransactionObj(
val types: SqlRuntimeTypes,
val backend: SqlTransactionBackend,
val lifetime: SqlTransactionLifetime,
) : Obj() {
override val objClass: ObjClass
get() = types.transactionClass
}
internal class SqlResultSetObj(
val types: SqlRuntimeTypes,
val lifetime: SqlTransactionLifetime,
data: SqlResultSetData,
) : Obj() {
val columns: List<Obj> = data.columns.map { SqlColumnObj(types, it) }
val rows: List<Obj> = buildRows(types, data)
override val objClass: ObjClass
get() = types.resultSetClass
private fun buildRows(
types: SqlRuntimeTypes,
data: SqlResultSetData,
): List<Obj> {
val indexByName = linkedMapOf<String, MutableList<Int>>()
data.columns.forEachIndexed { index, column ->
indexByName.getOrPut(column.name.lowercase()) { mutableListOf() }.add(index)
}
return data.rows.map { rowValues ->
SqlRowObj(types, rowValues, indexByName)
}
}
}
internal class SqlRowObj(
val types: SqlRuntimeTypes,
val values: List<Obj>,
private val indexByName: Map<String, List<Int>>,
) : Obj() {
override val objClass: ObjClass
get() = types.rowClass
override suspend fun getAt(scope: Scope, index: Obj): Obj {
return when (index) {
is ObjInt -> {
val idx = index.value.toInt()
if (idx !in values.indices) {
scope.raiseIndexOutOfBounds("SQL row index $idx is out of bounds")
}
values[idx]
}
is ObjString -> {
val matches = indexByName[index.value.lowercase()]
?: scope.raiseError(
ObjException(
types.core.sqlUsageException,
scope,
ObjString("No such SQL result column: ${index.value}")
)
)
if (matches.size != 1) {
scope.raiseError(
ObjException(
types.core.sqlUsageException,
scope,
ObjString("Ambiguous SQL result column: ${index.value}")
)
)
}
values[matches.first()]
}
else -> scope.raiseClassCastError("SQL row index must be Int or String")
}
}
}
internal class SqlColumnObj(
val types: SqlRuntimeTypes,
val meta: SqlColumnMeta,
) : Obj() {
override val objClass: ObjClass
get() = types.columnClass
}
internal class SqlExecutionResultObj(
val types: SqlRuntimeTypes,
val lifetime: SqlTransactionLifetime,
val result: SqlExecutionResultData,
) : Obj() {
override val objClass: ObjClass
get() = types.executionResultClass
}
@@ -0,0 +1,278 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.jdbc
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.Source
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.io.db.SqlDatabaseObj
import net.sergeych.lyng.io.db.SqlRuntimeTypes
import net.sergeych.lyng.io.db.createDbModule
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjBool
import net.sergeych.lyng.obj.ObjImmutableMap
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjMap
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.ObjVoid
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.pacman.ImportManager
import net.sergeych.lyng.requireScope
import net.sergeych.lyngio.stdlib_included.db_jdbcLyng
private const val JDBC_MODULE_NAME = "lyng.io.db.jdbc"
private const val DB_MODULE_NAME = "lyng.io.db"
private const val JDBC_SCHEME = "jdbc"
private const val H2_DRIVER = "org.h2.Driver"
private const val POSTGRES_DRIVER = "org.postgresql.Driver"
fun createJdbcModule(scope: Scope): Boolean = createJdbcModule(scope.importManager)
fun createJdbc(scope: Scope): Boolean = createJdbcModule(scope)
fun createJdbcModule(manager: ImportManager): Boolean {
createDbModule(manager)
if (manager.packageNames.contains(JDBC_MODULE_NAME)) return false
manager.addPackage(JDBC_MODULE_NAME) { module ->
buildJdbcModule(module)
}
return true
}
fun createJdbc(manager: ImportManager): Boolean = createJdbcModule(manager)
private suspend fun buildJdbcModule(module: ModuleScope) {
module.eval(Source(JDBC_MODULE_NAME, db_jdbcLyng))
val dbModule = module.importProvider.createModuleScope(Pos.builtIn, DB_MODULE_NAME)
val core = SqlCoreModule.resolve(dbModule)
val runtimeTypes = SqlRuntimeTypes.create("Jdbc", core)
module.addFn("openJdbc") {
val options = parseOpenJdbcArgs(this)
SqlDatabaseObj(runtimeTypes, openJdbcBackend(this, core, options))
}
module.addFn("openH2") {
val options = parseOpenShortcutArgs(this, H2_DRIVER, ::normalizeH2Url)
SqlDatabaseObj(runtimeTypes, openJdbcBackend(this, core, options))
}
module.addFn("openPostgres") {
val options = parseOpenShortcutArgs(this, POSTGRES_DRIVER, ::normalizePostgresUrl)
SqlDatabaseObj(runtimeTypes, openJdbcBackend(this, core, options))
}
registerProvider(dbModule, runtimeTypes, core, JDBC_SCHEME, null)
registerProvider(dbModule, runtimeTypes, core, "h2", H2_DRIVER)
registerProvider(dbModule, runtimeTypes, core, "postgres", POSTGRES_DRIVER)
registerProvider(dbModule, runtimeTypes, core, "postgresql", POSTGRES_DRIVER)
}
private suspend fun registerProvider(
dbModule: ModuleScope,
runtimeTypes: SqlRuntimeTypes,
core: SqlCoreModule,
scheme: String,
implicitDriverClass: String?,
) {
dbModule.callFn(
"registerDatabaseProvider",
ObjString(scheme),
net.sergeych.lyng.obj.ObjExternCallable.fromBridge {
val connectionUrl = requiredArg<ObjString>(0).value
val extraParams = args.list.getOrNull(1)
?: raiseError("Expected exactly 2 arguments, got ${args.list.size}")
val options = parseJdbcConnectionUrl(this, scheme, implicitDriverClass, connectionUrl, extraParams)
SqlDatabaseObj(runtimeTypes, openJdbcBackend(this, core, options))
}
)
}
private suspend fun parseOpenJdbcArgs(scope: ScopeFacade): JdbcOpenOptions {
val rawUrl = readArg(scope, "connectionUrl", 0) ?: scope.raiseError("argument 'connectionUrl' is required")
val connectionUrl = (rawUrl as? ObjString)?.value ?: scope.raiseClassCastError("connectionUrl must be String")
val user = readNullableStringArg(scope, "user", 1)
val password = readNullableStringArg(scope, "password", 2)
val driverClass = readNullableStringArg(scope, "driverClass", 3)
val properties = readPropertiesArg(scope, "properties", 4)
return JdbcOpenOptions(
connectionUrl = normalizeJdbcUrl(connectionUrl, scope),
user = user,
password = password,
driverClass = driverClass,
properties = properties,
)
}
private suspend fun parseOpenShortcutArgs(
scope: ScopeFacade,
defaultDriverClass: String,
normalize: (String, ScopeFacade) -> String,
): JdbcOpenOptions {
val rawUrl = readArg(scope, "connectionUrl", 0) ?: scope.raiseError("argument 'connectionUrl' is required")
val connectionUrl = (rawUrl as? ObjString)?.value ?: scope.raiseClassCastError("connectionUrl must be String")
val user = readNullableStringArg(scope, "user", 1)
val password = readNullableStringArg(scope, "password", 2)
val properties = readPropertiesArg(scope, "properties", 3)
return JdbcOpenOptions(
connectionUrl = normalize(connectionUrl, scope),
user = user,
password = password,
driverClass = defaultDriverClass,
properties = properties,
)
}
private suspend fun parseJdbcConnectionUrl(
scope: ScopeFacade,
scheme: String,
implicitDriverClass: String?,
connectionUrl: String,
extraParams: Obj,
): JdbcOpenOptions {
val driverClass = mapNullableString(extraParams, scope, "driverClass") ?: implicitDriverClass
val user = mapNullableString(extraParams, scope, "user")
val password = mapNullableString(extraParams, scope, "password")
val properties = mapProperties(extraParams, scope, "properties")
val normalizedUrl = when (scheme) {
JDBC_SCHEME -> normalizeJdbcUrl(connectionUrl, scope)
"h2" -> normalizeH2Url(connectionUrl, scope)
"postgres", "postgresql" -> normalizePostgresUrl(connectionUrl, scope)
else -> scope.raiseIllegalArgument("Unsupported JDBC provider scheme: $scheme")
}
return JdbcOpenOptions(
connectionUrl = normalizedUrl,
user = user,
password = password,
driverClass = driverClass,
properties = properties,
)
}
private fun normalizeJdbcUrl(rawUrl: String, scope: ScopeFacade): String {
val trimmed = rawUrl.trim()
if (!trimmed.startsWith("jdbc:", ignoreCase = true)) {
scope.raiseIllegalArgument("JDBC connection URL must start with jdbc:")
}
return trimmed
}
private fun normalizeH2Url(rawUrl: String, scope: ScopeFacade): String {
val trimmed = rawUrl.trim()
if (trimmed.isEmpty()) {
scope.raiseIllegalArgument("H2 connection URL must not be empty")
}
return when {
trimmed.startsWith("jdbc:h2:", ignoreCase = true) -> trimmed
trimmed.startsWith("h2:", ignoreCase = true) -> "jdbc:${trimmed}"
else -> "jdbc:h2:$trimmed"
}
}
private fun normalizePostgresUrl(rawUrl: String, scope: ScopeFacade): String {
val trimmed = rawUrl.trim()
if (trimmed.isEmpty()) {
scope.raiseIllegalArgument("PostgreSQL connection URL must not be empty")
}
return when {
trimmed.startsWith("jdbc:postgresql:", ignoreCase = true) -> trimmed
trimmed.startsWith("postgresql:", ignoreCase = true) -> "jdbc:$trimmed"
trimmed.startsWith("postgres:", ignoreCase = true) -> "jdbc:postgresql:${trimmed.substringAfter(':')}"
else -> "jdbc:postgresql:$trimmed"
}
}
private suspend fun readArg(scope: ScopeFacade, name: String, position: Int): Obj? {
val named = scope.args.named[name]
val positional = scope.args.list.getOrNull(position)
if (named != null && positional != null) {
scope.raiseIllegalArgument("argument '$name' is already set")
}
return named ?: positional
}
private suspend fun readNullableStringArg(scope: ScopeFacade, name: String, position: Int): String? {
val value = readArg(scope, name, position) ?: return null
return when (value) {
ObjNull -> null
is ObjString -> value.value
else -> scope.raiseClassCastError("$name must be String?")
}
}
private suspend fun readPropertiesArg(scope: ScopeFacade, name: String, position: Int): Map<String, String> {
val value = readArg(scope, name, position) ?: return emptyMap()
return when (value) {
ObjNull -> emptyMap()
else -> objToStringMap(value, scope, name)
}
}
private suspend fun mapNullableString(map: Obj, scope: ScopeFacade, key: String): String? {
val value = map.getAt(scope.requireScope(), ObjString(key))
return when (value) {
ObjNull -> null
is ObjString -> value.value
else -> scope.raiseClassCastError("extraParams.$key must be String?")
}
}
private suspend fun mapProperties(map: Obj, scope: ScopeFacade, key: String): Map<String, String> {
val value = map.getAt(scope.requireScope(), ObjString(key))
return when (value) {
ObjNull -> emptyMap()
else -> objToStringMap(value, scope, "extraParams.$key")
}
}
private suspend fun objToStringMap(value: Obj, scope: ScopeFacade, label: String): Map<String, String> {
val rawEntries = when (value) {
is ObjMap -> value.map
is ObjImmutableMap -> value.map
else -> scope.raiseClassCastError("$label must be Map<String, Object?>")
}
val properties = linkedMapOf<String, String>()
for ((rawKey, rawValue) in rawEntries) {
val key = (rawKey as? ObjString)?.value ?: scope.raiseClassCastError("$label keys must be String")
if (rawValue == ObjNull) continue
properties[key] = scope.toStringOf(rawValue).value
}
return properties
}
private suspend fun ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
internal data class JdbcOpenOptions(
val connectionUrl: String,
val user: String?,
val password: String?,
val driverClass: String?,
val properties: Map<String, String>,
)
internal expect suspend fun openJdbcBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: JdbcOpenOptions,
): SqlDatabaseBackend
@@ -0,0 +1,194 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import net.sergeych.lyng.Arguments
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.Source
import net.sergeych.lyng.asFacade
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.io.db.SqlDatabaseObj
import net.sergeych.lyng.io.db.SqlRuntimeTypes
import net.sergeych.lyng.io.db.SqlTransactionBackend
import net.sergeych.lyng.io.db.createDbModule
import net.sergeych.lyng.requireScope
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjBool
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.ObjVoid
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.pacman.ImportManager
import net.sergeych.lyngio.stdlib_included.db_sqliteLyng
private const val SQLITE_MODULE_NAME = "lyng.io.db.sqlite"
private const val DB_MODULE_NAME = "lyng.io.db"
fun createSqliteModule(scope: Scope): Boolean = createSqliteModule(scope.importManager)
fun createSqlite(scope: Scope): Boolean = createSqliteModule(scope)
fun createSqliteModule(manager: ImportManager): Boolean {
createDbModule(manager)
if (manager.packageNames.contains(SQLITE_MODULE_NAME)) return false
manager.addPackage(SQLITE_MODULE_NAME) { module ->
buildSqliteModule(module)
}
return true
}
fun createSqlite(manager: ImportManager): Boolean = createSqliteModule(manager)
private suspend fun buildSqliteModule(module: ModuleScope) {
module.eval(Source(SQLITE_MODULE_NAME, db_sqliteLyng))
val dbModule = module.importProvider.createModuleScope(Pos.builtIn, DB_MODULE_NAME)
val core = SqlCoreModule.resolve(dbModule)
val runtimeTypes = SqlRuntimeTypes.create("Sqlite", core)
module.addFn("openSqlite") {
val options = parseOpenSqliteArgs(this)
SqlDatabaseObj(runtimeTypes, openSqliteBackend(this, core, options))
}
dbModule.callFn(
"registerDatabaseProvider",
ObjString("sqlite"),
net.sergeych.lyng.obj.ObjExternCallable.fromBridge {
val connectionUrl = requiredArg<ObjString>(0).value
val extraParams = args.list.getOrNull(1)
?: raiseError("Expected exactly 2 arguments, got ${args.list.size}")
val options = parseSqliteConnectionUrl(this, connectionUrl, extraParams)
SqlDatabaseObj(runtimeTypes, openSqliteBackend(this, core, options))
}
)
}
private suspend fun parseOpenSqliteArgs(scope: ScopeFacade): SqliteOpenOptions {
val pathValue = readArg(scope, "path", 0) ?: scope.raiseError("argument 'path' is required")
val path = (pathValue as? ObjString)?.value ?: scope.raiseClassCastError("path must be String")
val readOnly = readBoolArg(scope, "readOnly", 1, false)
val createIfMissing = readBoolArg(scope, "createIfMissing", 2, true)
val foreignKeys = readBoolArg(scope, "foreignKeys", 3, true)
val busyTimeoutMillis = readIntArg(scope, "busyTimeoutMillis", 4, 5000)
return SqliteOpenOptions(
path = normalizeSqlitePath(path, scope),
readOnly = readOnly,
createIfMissing = createIfMissing,
foreignKeys = foreignKeys,
busyTimeoutMillis = busyTimeoutMillis,
)
}
private suspend fun parseSqliteConnectionUrl(
scope: ScopeFacade,
connectionUrl: String,
extraParams: Obj,
): SqliteOpenOptions {
val prefix = "sqlite:"
if (!connectionUrl.startsWith(prefix, ignoreCase = true)) {
scope.raiseIllegalArgument("Malformed SQLite connection URL: $connectionUrl")
}
val rawPath = connectionUrl.substring(prefix.length)
val path = normalizeSqlitePath(rawPath, scope)
val readOnly = mapBool(extraParams, scope, "readOnly") ?: false
val createIfMissing = mapBool(extraParams, scope, "createIfMissing") ?: true
val foreignKeys = mapBool(extraParams, scope, "foreignKeys") ?: true
val busyTimeoutMillis = mapInt(extraParams, scope, "busyTimeoutMillis") ?: 5000
return SqliteOpenOptions(
path = path,
readOnly = readOnly,
createIfMissing = createIfMissing,
foreignKeys = foreignKeys,
busyTimeoutMillis = busyTimeoutMillis,
)
}
private fun normalizeSqlitePath(rawPath: String, scope: ScopeFacade): String {
val path = rawPath.trim()
if (path.isEmpty()) {
scope.raiseIllegalArgument("SQLite path must not be empty")
}
if (path.startsWith("//")) {
scope.raiseIllegalArgument("Unsupported SQLite URL form: sqlite:$path")
}
return path
}
private suspend fun readArg(scope: ScopeFacade, name: String, position: Int): Obj? {
val named = scope.args.named[name]
val positional = scope.args.list.getOrNull(position)
if (named != null && positional != null) {
scope.raiseIllegalArgument("argument '$name' is already set")
}
return named ?: positional
}
private suspend fun readBoolArg(scope: ScopeFacade, name: String, position: Int, default: Boolean): Boolean {
val value = readArg(scope, name, position) ?: return default
return (value as? ObjBool)?.value ?: scope.raiseClassCastError("$name must be Bool")
}
private suspend fun readIntArg(scope: ScopeFacade, name: String, position: Int, default: Int): Int {
val value = readArg(scope, name, position) ?: return default
return when (value) {
is ObjInt -> value.value.toInt()
else -> scope.raiseClassCastError("$name must be Int")
}
}
private suspend fun mapBool(map: Obj, scope: ScopeFacade, key: String): Boolean? {
val value = map.getAt(scope.requireScope(), ObjString(key))
return when (value) {
ObjNull -> null
is ObjBool -> value.value
else -> scope.raiseClassCastError("extraParams.$key must be Bool")
}
}
private suspend fun mapInt(map: Obj, scope: ScopeFacade, key: String): Int? {
val value = map.getAt(scope.requireScope(), ObjString(key))
return when (value) {
ObjNull -> null
is ObjInt -> value.value.toInt()
else -> scope.raiseClassCastError("extraParams.$key must be Int")
}
}
private suspend fun ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
internal data class SqliteOpenOptions(
val path: String,
val readOnly: Boolean,
val createIfMissing: Boolean,
val foreignKeys: Boolean,
val busyTimeoutMillis: Int,
)
internal expect suspend fun openSqliteBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: SqliteOpenOptions,
): SqlDatabaseBackend
@@ -97,3 +97,5 @@ internal object UnsupportedLyngNetEngine : LyngNetEngine {
} }
expect fun getSystemNetEngine(): LyngNetEngine expect fun getSystemNetEngine(): LyngNetEngine
expect fun shutdownSystemNetEngine()
@@ -17,35 +17,46 @@ import kotlinx.coroutines.channels.ClosedReceiveChannelException
internal fun createKtorWsEngine( internal fun createKtorWsEngine(
engineFactory: HttpClientEngineFactory<HttpClientEngineConfig>, engineFactory: HttpClientEngineFactory<HttpClientEngineConfig>,
): LyngWsEngine = KtorLyngWsEngine(engineFactory) shareClient: Boolean = true,
): LyngWsEngine = KtorLyngWsEngine(engineFactory, shareClient)
private class KtorLyngWsEngine( private class KtorLyngWsEngine(
engineFactory: HttpClientEngineFactory<HttpClientEngineConfig>, engineFactory: HttpClientEngineFactory<HttpClientEngineConfig>,
private val shareClient: Boolean,
) : LyngWsEngine { ) : LyngWsEngine {
private val clientResult = runCatching { private val clientFactory: () -> HttpClient = {
HttpClient(engineFactory) { HttpClient(engineFactory) {
install(WebSockets) install(WebSockets)
} }
} }
private val sharedClientResult = if (shareClient) runCatching { clientFactory() } else null
private val supportProbe = if (!shareClient) runCatching { clientFactory().close() } else null
override val isSupported: Boolean override val isSupported: Boolean
get() = clientResult.isSuccess get() = sharedClientResult?.isSuccess ?: supportProbe?.isSuccess ?: true
override suspend fun connect(url: String, headers: Map<String, String>): LyngWsSession { override suspend fun connect(url: String, headers: Map<String, String>): LyngWsSession {
val client = clientResult.getOrElse { val client = (sharedClientResult?.getOrElse {
throw UnsupportedOperationException(it.message ?: "WebSocket client is not supported") throw UnsupportedOperationException(it.message ?: "WebSocket client is not supported")
} } ?: runCatching { clientFactory() }.getOrElse {
throw UnsupportedOperationException(it.message ?: "WebSocket client is not supported")
})
val session = client.webSocketSession { val session = client.webSocketSession {
url(url) url(url)
headers.forEach { (name, value) -> header(name, value) } headers.forEach { (name, value) -> header(name, value) }
} }
return KtorLyngWsSession(url, session) return KtorLyngWsSession(
targetUrl = url,
session = session,
ownedClient = client.takeUnless { shareClient },
)
} }
} }
private class KtorLyngWsSession( private class KtorLyngWsSession(
private val targetUrl: String, private val targetUrl: String,
private val session: DefaultWebSocketSession, private val session: DefaultWebSocketSession,
private val ownedClient: HttpClient? = null,
) : LyngWsSession { ) : LyngWsSession {
private var closed = false private var closed = false
@@ -55,12 +66,22 @@ private class KtorLyngWsSession(
override suspend fun sendText(text: String) { override suspend fun sendText(text: String) {
ensureOpen() ensureOpen()
session.send(text) try {
session.send(text)
} catch (e: Throwable) {
release()
throw e
}
} }
override suspend fun sendBytes(data: ByteArray) { override suspend fun sendBytes(data: ByteArray) {
ensureOpen() ensureOpen()
session.send(data) try {
session.send(data)
} catch (e: Throwable) {
release()
throw e
}
} }
override suspend fun receive(): LyngWsMessage? { override suspend fun receive(): LyngWsMessage? {
@@ -68,14 +89,17 @@ private class KtorLyngWsSession(
val frame = try { val frame = try {
session.incoming.receive() session.incoming.receive()
} catch (_: ClosedReceiveChannelException) { } catch (_: ClosedReceiveChannelException) {
closed = true release()
return null return null
} catch (e: Throwable) {
release()
throw e
} }
return when (frame) { return when (frame) {
is Frame.Text -> LyngWsMessage(isText = true, text = frame.readText()) is Frame.Text -> LyngWsMessage(isText = true, text = frame.readText())
is Frame.Binary -> LyngWsMessage(isText = false, data = frame.data.copyOf()) is Frame.Binary -> LyngWsMessage(isText = false, data = frame.data.copyOf())
is Frame.Close -> { is Frame.Close -> {
closed = true release()
null null
} }
else -> receive() else -> receive()
@@ -84,11 +108,20 @@ private class KtorLyngWsSession(
override suspend fun close(code: Int, reason: String) { override suspend fun close(code: Int, reason: String) {
if (closed) return if (closed) return
closed = true try {
session.close(CloseReason(code.toShort(), reason)) session.close(CloseReason(code.toShort(), reason))
} finally {
release()
}
} }
private fun ensureOpen() { private fun ensureOpen() {
if (closed) throw IllegalStateException("websocket session is closed") if (closed) throw IllegalStateException("websocket session is closed")
} }
private fun release() {
if (closed) return
closed = true
ownedClient?.close()
}
} }
@@ -0,0 +1,392 @@
package net.sergeych.lyngio.ws
import io.ktor.http.Url
import io.ktor.http.URLProtocol
import net.sergeych.lyngio.net.LyngNetEngine
import net.sergeych.lyngio.net.LyngTcpSocket
import net.sergeych.lyngio.net.getSystemNetEngine
import net.sergeych.mp_tools.encodeToBase64
import kotlin.random.Random
internal fun createSocketWsEngine(
secureFallback: LyngWsEngine = UnsupportedLyngWsEngine,
): LyngWsEngine = SocketLyngWsEngine(getSystemNetEngine(), secureFallback)
private class SocketLyngWsEngine(
private val netEngine: LyngNetEngine,
private val secureFallback: LyngWsEngine,
) : LyngWsEngine {
override val isSupported: Boolean
get() = (netEngine.isSupported && netEngine.isTcpAvailable) || secureFallback.isSupported
override suspend fun connect(url: String, headers: Map<String, String>): LyngWsSession {
val parsedUrl = Url(url)
return when (parsedUrl.protocol.name.lowercase()) {
URLProtocol.WS.name.lowercase() -> connectPlain(parsedUrl, headers)
URLProtocol.WSS.name.lowercase() -> secureFallback.connect(url, headers)
else -> throw UnsupportedOperationException("Unsupported websocket scheme: ${parsedUrl.protocol.name}")
}
}
private suspend fun connectPlain(url: Url, headers: Map<String, String>): LyngWsSession {
if (!netEngine.isSupported || !netEngine.isTcpAvailable) {
throw UnsupportedOperationException("WebSocket client is not supported on this runtime")
}
val socket = netEngine.tcpConnect(url.host, url.port, timeoutMillis = null, noDelay = true)
try {
val key = randomBytes(16).encodeToBase64()
val requestPath = buildRequestPath(url)
val hostHeader = buildHostHeader(url)
val request = buildString {
append("GET ").append(requestPath).append(" HTTP/1.1\r\n")
append("Host: ").append(hostHeader).append("\r\n")
append("Upgrade: websocket\r\n")
append("Connection: Upgrade\r\n")
append("Sec-WebSocket-Key: ").append(key).append("\r\n")
append("Sec-WebSocket-Version: 13\r\n")
headers.forEach { (name, value) ->
append(name).append(": ").append(value).append("\r\n")
}
append("\r\n")
}
socket.writeUtf8(request)
socket.flush()
validateHandshake(socket, key)
return SocketLyngWsSession(url.toString(), socket)
} catch (e: Throwable) {
socket.close()
throw e
}
}
}
private class SocketLyngWsSession(
private val targetUrl: String,
private val socket: LyngTcpSocket,
) : LyngWsSession {
private var closed = false
private var closeSent = false
private var pending = ByteArray(0)
private var fragmentedOpcode: Int? = null
private var fragmentedPayload = ByteArray(0)
override fun isOpen(): Boolean = !closed && socket.isOpen()
override fun url(): String = targetUrl
override suspend fun sendText(text: String) {
ensureOpen()
sendFrame(OPCODE_TEXT, text.encodeToByteArray())
}
override suspend fun sendBytes(data: ByteArray) {
ensureOpen()
sendFrame(OPCODE_BINARY, data)
}
override suspend fun receive(): LyngWsMessage? {
if (closed) return null
while (true) {
val frame = readFrame() ?: run {
release()
return null
}
when (frame.opcode) {
OPCODE_CONTINUATION -> {
val opcode = fragmentedOpcode ?: throw IllegalStateException("unexpected websocket continuation frame")
fragmentedPayload += frame.payload
if (frame.fin) {
val payload = fragmentedPayload
fragmentedOpcode = null
fragmentedPayload = ByteArray(0)
return payload.toMessage(opcode)
}
}
OPCODE_TEXT, OPCODE_BINARY -> {
if (frame.fin) return frame.payload.toMessage(frame.opcode)
fragmentedOpcode = frame.opcode
fragmentedPayload = frame.payload
}
OPCODE_CLOSE -> {
if (!closeSent) {
sendFrame(OPCODE_CLOSE, frame.payload)
closeSent = true
}
release()
return null
}
OPCODE_PING -> sendFrame(OPCODE_PONG, frame.payload)
OPCODE_PONG -> Unit
else -> Unit
}
}
}
override suspend fun close(code: Int, reason: String) {
if (closed) return
val reasonBytes = reason.encodeToByteArray()
val payload = ByteArray(reasonBytes.size + 2)
payload[0] = (code shr 8).toByte()
payload[1] = code.toByte()
reasonBytes.copyInto(payload, destinationOffset = 2)
try {
if (!closeSent) {
sendFrame(OPCODE_CLOSE, payload)
closeSent = true
}
} finally {
release()
}
}
private suspend fun sendFrame(opcode: Int, payload: ByteArray) {
ensureOpen()
val header = buildFrameHeader(opcode, payload.size, masked = true)
val mask = randomBytes(4)
val maskedPayload = payload.copyOf()
maskedPayload.indices.forEach { index ->
maskedPayload[index] = (maskedPayload[index].toInt() xor mask[index % mask.size].toInt()).toByte()
}
socket.write(header + mask + maskedPayload)
socket.flush()
}
private suspend fun readFrame(): WsFrame? {
val head = readExact(2) ?: return null
val fin = (head[0].toInt() and 0x80) != 0
val opcode = head[0].toInt() and 0x0f
val masked = (head[1].toInt() and 0x80) != 0
val payloadLength = when (val lengthCode = head[1].toInt() and 0x7f) {
126 -> {
val extended = readExact(2) ?: return null
((extended[0].toInt() and 0xff) shl 8) or (extended[1].toInt() and 0xff)
}
127 -> {
val extended = readExact(8) ?: return null
var acc = 0L
extended.forEach { byte ->
acc = (acc shl 8) or (byte.toInt() and 0xff).toLong()
}
require(acc <= Int.MAX_VALUE.toLong()) { "websocket frame is too large" }
acc.toInt()
}
else -> lengthCode
}
val mask = if (masked) readExact(4) ?: return null else null
val payload = if (payloadLength > 0) readExact(payloadLength) ?: return null else ByteArray(0)
if (mask != null) {
payload.indices.forEach { index ->
payload[index] = (payload[index].toInt() xor mask[index % mask.size].toInt()).toByte()
}
}
return WsFrame(fin = fin, opcode = opcode, payload = payload)
}
private suspend fun readExact(byteCount: Int): ByteArray? {
while (pending.size < byteCount) {
val chunk = socket.read(maxOf(4096, byteCount - pending.size)) ?: break
if (chunk.isEmpty()) break
pending += chunk
}
if (pending.size < byteCount) return null
val result = pending.copyOfRange(0, byteCount)
pending = pending.copyOfRange(byteCount, pending.size)
return result
}
private fun release() {
if (closed) return
closed = true
socket.close()
}
private fun ensureOpen() {
if (closed || !socket.isOpen()) throw IllegalStateException("websocket session is closed")
}
}
private data class WsFrame(
val fin: Boolean,
val opcode: Int,
val payload: ByteArray,
)
private suspend fun validateHandshake(socket: LyngTcpSocket, key: String) {
val statusLine = socket.readLine() ?: error("websocket handshake failed: missing response status")
require(statusLine.startsWith("HTTP/1.1 101") || statusLine.startsWith("HTTP/1.0 101")) {
"websocket handshake failed: $statusLine"
}
val headers = linkedMapOf<String, String>()
while (true) {
val line = socket.readLine() ?: error("websocket handshake failed: unexpected EOF")
if (line.isBlank()) break
val colonAt = line.indexOf(':')
require(colonAt > 0) { "invalid websocket header: $line" }
val name = line.substring(0, colonAt).trim().lowercase()
val value = line.substring(colonAt + 1).trim()
headers[name] = value
}
require(headers["upgrade"]?.lowercase() == "websocket") { "websocket handshake failed: missing Upgrade header" }
require(headers["connection"]?.lowercase()?.contains("upgrade") == true) {
"websocket handshake failed: missing Connection header"
}
require(headers["sec-websocket-accept"] == websocketAcceptKey(key)) {
"websocket handshake failed: invalid Sec-WebSocket-Accept"
}
}
private fun websocketAcceptKey(key: String): String =
sha1((key + WS_GUID).encodeToByteArray()).encodeToBase64()
private fun buildRequestPath(url: Url): String {
val path = url.encodedPath.ifEmpty { "/" }
val query = url.encodedQuery
return if (query.isBlank()) path else "$path?$query"
}
private fun buildHostHeader(url: Url): String {
val host = if (':' in url.host && !url.host.startsWith("[")) "[${url.host}]" else url.host
return if (url.port == url.protocol.defaultPort) host else "$host:${url.port}"
}
private fun buildFrameHeader(opcode: Int, payloadSize: Int, masked: Boolean): ByteArray {
require(payloadSize >= 0) { "payload size must be non-negative" }
val firstByte = (0x80 or (opcode and 0x0f)).toByte()
val maskBit = if (masked) 0x80 else 0
return when {
payloadSize <= 125 -> byteArrayOf(firstByte, (maskBit or payloadSize).toByte())
payloadSize <= 0xffff -> byteArrayOf(
firstByte,
(maskBit or 126).toByte(),
((payloadSize ushr 8) and 0xff).toByte(),
(payloadSize and 0xff).toByte(),
)
else -> byteArrayOf(
firstByte,
(maskBit or 127).toByte(),
0,
0,
0,
0,
((payloadSize ushr 24) and 0xff).toByte(),
((payloadSize ushr 16) and 0xff).toByte(),
((payloadSize ushr 8) and 0xff).toByte(),
(payloadSize and 0xff).toByte(),
)
}
}
private fun ByteArray.toMessage(opcode: Int): LyngWsMessage = when (opcode) {
OPCODE_TEXT -> LyngWsMessage(isText = true, text = decodeToString())
OPCODE_BINARY -> LyngWsMessage(isText = false, data = copyOf())
else -> throw IllegalStateException("unsupported websocket opcode: $opcode")
}
private fun randomBytes(size: Int): ByteArray = ByteArray(size).also(Random.Default::nextBytes)
private fun sha1(input: ByteArray): ByteArray {
var h0 = 0x67452301
var h1 = 0xEFCDAB89.toInt()
var h2 = 0x98BADCFE.toInt()
var h3 = 0x10325476
var h4 = 0xC3D2E1F0.toInt()
val msgLen = input.size
val bitLen = msgLen.toLong() * 8L
val totalLen = ((msgLen + 1 + 8 + 63) / 64) * 64
val padded = ByteArray(totalLen).also { buf ->
input.copyInto(buf)
buf[msgLen] = 0x80.toByte()
for (i in 0..7) {
buf[totalLen - 8 + i] = ((bitLen ushr (56 - i * 8)) and 0xff).toByte()
}
}
val words = IntArray(80)
var blockStart = 0
while (blockStart < padded.size) {
for (i in 0..15) {
val off = blockStart + i * 4
words[i] = ((padded[off].toInt() and 0xff) shl 24) or
((padded[off + 1].toInt() and 0xff) shl 16) or
((padded[off + 2].toInt() and 0xff) shl 8) or
(padded[off + 3].toInt() and 0xff)
}
for (i in 16..79) {
val mixed = words[i - 3] xor words[i - 8] xor words[i - 14] xor words[i - 16]
words[i] = (mixed shl 1) or (mixed ushr 31)
}
var a = h0
var b = h1
var c = h2
var d = h3
var e = h4
for (i in 0..19) {
val f = (b and c) or (b.inv() and d)
val temp = ((a shl 5) or (a ushr 27)) + f + e + 0x5A827999 + words[i]
e = d
d = c
c = (b shl 30) or (b ushr 2)
b = a
a = temp
}
for (i in 20..39) {
val f = b xor c xor d
val temp = ((a shl 5) or (a ushr 27)) + f + e + 0x6ED9EBA1 + words[i]
e = d
d = c
c = (b shl 30) or (b ushr 2)
b = a
a = temp
}
for (i in 40..59) {
val f = (b and c) or (b and d) or (c and d)
val temp = ((a shl 5) or (a ushr 27)) + f + e + 0x8F1BBCDC.toInt() + words[i]
e = d
d = c
c = (b shl 30) or (b ushr 2)
b = a
a = temp
}
for (i in 60..79) {
val f = b xor c xor d
val temp = ((a shl 5) or (a ushr 27)) + f + e + 0xCA62C1D6.toInt() + words[i]
e = d
d = c
c = (b shl 30) or (b ushr 2)
b = a
a = temp
}
h0 += a
h1 += b
h2 += c
h3 += d
h4 += e
blockStart += 64
}
return ByteArray(20).also { out ->
fun putInt(offset: Int, value: Int) {
out[offset] = (value ushr 24).toByte()
out[offset + 1] = (value ushr 16).toByte()
out[offset + 2] = (value ushr 8).toByte()
out[offset + 3] = value.toByte()
}
putInt(0, h0)
putInt(4, h1)
putInt(8, h2)
putInt(12, h3)
putInt(16, h4)
}
}
private const val WS_GUID = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
private const val OPCODE_CONTINUATION = 0x0
private const val OPCODE_TEXT = 0x1
private const val OPCODE_BINARY = 0x2
private const val OPCODE_CLOSE = 0x8
private const val OPCODE_PING = 0x9
private const val OPCODE_PONG = 0xA
@@ -0,0 +1,135 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db
import kotlinx.coroutines.test.runTest
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.pacman.ImportManager
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Script
import net.sergeych.lyng.Source
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjExternCallable
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.requireScope
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
import kotlin.test.assertFalse
import kotlin.test.assertTrue
class LyngDbModuleTest {
@Test
fun testModuleRegistrationIsIdempotent() = runTest {
val importManager = ImportManager()
assertTrue(createDbModule(importManager))
assertFalse(createDbModule(importManager))
}
@Test
fun testOpenDatabaseDispatchesByNormalizedScheme() = runTest {
val scope = Script.newScope()
createDbModule(scope.importManager)
val module = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
module.callFn(
"registerDatabaseProvider",
ObjString("TeSt"),
ObjExternCallable.fromBridge {
val url = requiredArg<ObjString>(0).value
val params = requiredArg<Obj>(1)
val size = (params.invokeInstanceMethod(requireScope(), "size") as ObjInt).value
ObjString("$url|$size")
}
)
val code = """
import lyng.io.db
openDatabase("TEST:demo", Map("a" => 1, "b" => 2))
""".trimIndent()
val result = Compiler.compile(Source("<db-test>", code), scope.importManager).execute(scope) as ObjString
assertEquals("TEST:demo|2", result.value)
}
@Test
fun testDuplicateSchemeRegistrationFailsCaseInsensitively() = runTest {
val importManager = ImportManager()
createDbModule(importManager)
val module = importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
val scheme = "provider_test"
module.callFn("registerDatabaseProvider", ObjString(scheme), trivialOpener())
val error = try {
module.callFn("registerDatabaseProvider", ObjString(scheme.uppercase()), trivialOpener())
kotlin.test.fail("expected duplicate registration to fail")
} catch (e: ExecutionError) {
e
}
assertTrue(error.errorMessage.contains("already registered"), error.errorMessage)
}
@Test
fun testMalformedUrlFailsWithIllegalArgument() = runTest {
val scope = Script.newScope()
createDbModule(scope.importManager)
val code = """
import lyng.io.db
openDatabase("not-a-url", Map())
""".trimIndent()
val error = assertFailsWith<ExecutionError> {
Compiler.compile(Source("<db-test>", code), scope.importManager).execute(scope)
}
assertEquals("IllegalArgumentException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("Malformed database connection URL"), error.errorMessage)
}
@Test
fun testUnknownSchemeFailsWithDatabaseException() = runTest {
val scope = Script.newScope()
createDbModule(scope.importManager)
val code = """
import lyng.io.db
openDatabase("unknown:demo", Map())
""".trimIndent()
val error = assertFailsWith<ExecutionError> {
Compiler.compile(Source("<db-test>", code), scope.importManager).execute(scope)
}
assertEquals("DatabaseException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("No database provider registered"), error.errorMessage)
}
private suspend fun net.sergeych.lyng.ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
private fun trivialOpener(): Obj = ObjExternCallable.fromBridge { ObjInt.One }
}
@@ -0,0 +1,126 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.net
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.runBlocking
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeout
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.Script
import net.sergeych.lyngio.net.getSystemNetEngine
import net.sergeych.lyngio.net.security.PermitAllNetAccessPolicy
import kotlin.test.Test
import kotlin.test.assertEquals
class LyngNetTcpServerExampleTest {
private fun concurrentTcpScript(clientCount: Int, clientWindow: Int): String = """
import lyng.io.net
val host = "127.0.0.1"
val clientCount = $clientCount
val clientWindow = $clientWindow
val server: TcpServer = Net.tcpListen(0, host, clientWindow, true) as TcpServer
val port: Int = server.localAddress().port
fun payloadFor(index: Int): String {
"${'$'}index:${'$'}{Random.nextInt()}:${'$'}{Random.nextInt()}"
}
fun handleClient(client: TcpSocket): String {
try {
val source = client.readLine()
if( source == null ) {
return "server-eof"
}
val reply = "pong: ${'$'}source"
client.writeUtf8(reply + "\n")
client.flush()
reply
} finally {
client.close()
}
}
val serverJob: Deferred = launch {
var handlers: List<Deferred> = List()
try {
for( i in 0..<${clientCount} ) {
val client: TcpSocket = server.accept() as TcpSocket
handlers += launch {
handleClient(client)
}
}
handlers.joinAll()
} finally {
server.close()
}
}
var replies: List<Object> = List()
for (batchStart in 0..<clientCount step clientWindow) {
val batchEnd = if (batchStart + clientWindow < clientCount) batchStart + clientWindow else clientCount
val clientJobs: List<Deferred> = (batchStart..<batchEnd).map { index ->
val payload = payloadFor(index)
launch {
val socket: TcpSocket = Net.tcpConnect(host, port) as TcpSocket
try {
socket.writeUtf8(payload + "\n")
socket.flush()
val reply = socket.readLine()
if( reply == null ) {
"client-eof:${'$'}payload"
}
else {
assertEquals("pong: ${'$'}payload", reply)
reply
}
} finally {
socket.close()
}
}
}
replies += clientJobs.joinAll()
}
val serverReplies = serverJob.await() as List<Object>
assertEquals(clientCount, replies.size)
assertEquals(clientCount, serverReplies.size)
assertEquals(replies.toSet, serverReplies.toSet)
"OK:${'$'}clientCount"
""".trimIndent()
@Test
fun tcpServerExampleSurvivesConcurrentLoopbackLoad() = runBlocking {
val engine = getSystemNetEngine()
if (!engine.isSupported || !engine.isTcpAvailable || !engine.isTcpServerAvailable) return@runBlocking
val scope = Script.newScope()
createNetModule(PermitAllNetAccessPolicy, scope)
val result = withContext(Dispatchers.Default) {
withTimeout(20_000) {
Compiler.compile(concurrentTcpScript(clientCount = 1_000, clientWindow = 128)).execute(scope).inspect(scope)
}
}
assertEquals("\"OK:1000\"", result)
}
}
@@ -0,0 +1,68 @@
package net.sergeych.lyngio.net
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import kotlinx.coroutines.runBlocking
import kotlinx.coroutines.withTimeout
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
class NetConcurrentLoopbackTest {
@Test
fun concurrentTcpRoundTripsWorkAtEngineLevel() = runBlocking {
val engine = getSystemNetEngine()
if (!engine.isSupported || !engine.isTcpAvailable || !engine.isTcpServerAvailable) return@runBlocking
withTimeout(10_000) {
val clients = 32
val server = engine.tcpListen(host = "127.0.0.1", port = 0, backlog = 64, reuseAddress = true)
val serverJob = async {
coroutineScope {
val handlers = ArrayList<kotlinx.coroutines.Deferred<String>>(clients)
repeat(clients) {
val client = server.accept()
handlers += async {
try {
val payload = client.readLine()
val reply = "pong:$payload"
client.writeUtf8("$reply\n")
client.flush()
reply
} finally {
client.close()
}
}
}
handlers.awaitAll()
}
}
val clientJobs = coroutineScope {
(0 until clients).map { index ->
async {
val payload = "ping:$index"
val socket = engine.tcpConnect("127.0.0.1", server.localAddress().port, timeoutMillis = 2_000, noDelay = true)
try {
socket.writeUtf8("$payload\n")
socket.flush()
socket.readLine()
} finally {
socket.close()
}
}
}
}
val clientReplies = clientJobs.awaitAll()
val serverReplies = serverJob.await()
server.close()
assertEquals((0 until clients).map { "pong:ping:$it" }, clientReplies)
assertEquals((0 until clients).map { "pong:ping:$it" }, serverReplies)
assertTrue(clientReplies.all { it != null })
}
}
}
@@ -1,8 +1,14 @@
package net.sergeych.lyngio.net package net.sergeych.lyngio.net
actual fun getSystemNetEngine(): LyngNetEngine = createNativeKtorNetEngine( private val systemNetEngine: LyngNetEngine = createNativeKtorNetEngine(
isSupported = true, isSupported = true,
isTcpAvailable = true, isTcpAvailable = true,
isTcpServerAvailable = true, isTcpServerAvailable = true,
isUdpAvailable = true, isUdpAvailable = true,
) )
actual fun getSystemNetEngine(): LyngNetEngine = systemNetEngine
actual fun shutdownSystemNetEngine() {
shutdownNativeKtorNetEngine(systemNetEngine)
}
@@ -0,0 +1,22 @@
package net.sergeych.lyng.io.db.jdbc
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.requireScope
internal actual suspend fun openJdbcBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: JdbcOpenOptions,
): SqlDatabaseBackend {
scope.raiseError(
ObjException(
core.databaseException,
scope.requireScope(),
ObjString("lyng.io.db.jdbc is available only on the JVM target")
)
)
}
@@ -0,0 +1,39 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.requireScope
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjString
internal actual suspend fun openSqliteBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: SqliteOpenOptions,
): SqlDatabaseBackend {
scope.raiseError(
ObjException(
core.databaseException,
scope.requireScope(),
ObjString("SQLite provider is not implemented on this platform yet")
)
)
}
@@ -13,6 +13,8 @@ import org.khronos.webgl.Uint8Array
actual fun getSystemNetEngine(): LyngNetEngine = jsNodeNetEngineOrNull ?: UnsupportedLyngNetEngine actual fun getSystemNetEngine(): LyngNetEngine = jsNodeNetEngineOrNull ?: UnsupportedLyngNetEngine
actual fun shutdownSystemNetEngine() {}
private val jsNodeNetEngineOrNull: LyngNetEngine? by lazy { private val jsNodeNetEngineOrNull: LyngNetEngine? by lazy {
if (!isNodeRuntime()) return@lazy null if (!isNodeRuntime()) return@lazy null
val net = requireNodeModule("net") ?: return@lazy null val net = requireNodeModule("net") ?: return@lazy null
@@ -1,3 +1,20 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.net package net.sergeych.lyng.io.net
import kotlinx.coroutines.DelicateCoroutinesApi import kotlinx.coroutines.DelicateCoroutinesApi
@@ -1,3 +1,20 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio package net.sergeych.lyngio
import kotlinx.coroutines.DelicateCoroutinesApi import kotlinx.coroutines.DelicateCoroutinesApi
@@ -0,0 +1,448 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.jdbc
import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import kotlinx.datetime.LocalDate
import kotlinx.datetime.LocalDateTime
import kotlinx.datetime.TimeZone
import kotlinx.datetime.toInstant
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.*
import net.sergeych.lyng.obj.*
import net.sergeych.lyng.requireScope
import java.math.BigDecimal
import java.sql.*
import kotlin.time.Instant
private val knownJdbcDrivers = listOf(
"org.sqlite.JDBC",
"org.h2.Driver",
"org.postgresql.Driver",
)
internal actual suspend fun openJdbcBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: JdbcOpenOptions,
): SqlDatabaseBackend {
ensureJdbcDriversLoaded(scope, core, options.driverClass)
val dataSource = try {
val config = HikariConfig().apply {
jdbcUrl = options.connectionUrl
options.user?.let { username = it }
options.password?.let { password = it }
options.driverClass?.let { driverClassName = it }
options.properties.forEach { (key, value) -> addDataSourceProperty(key, value) }
// Statement caching: avoids re-parsing identical SQL on every call
addDataSourceProperty("cachePrepStmts", "true")
addDataSourceProperty("prepStmtCacheSize", "250")
addDataSourceProperty("prepStmtCacheSqlLimit", "2048")
// Connections from the pool are already in autoCommit=false territory;
// we manage commits explicitly in transaction().
isAutoCommit = false
}
HikariDataSource(config)
} catch (e: Exception) {
val cause = e.cause as? SQLException ?: (e as? SQLException)
if (cause != null) throw mapOpenException(scope, core, cause)
throw e
}
return JdbcDatabaseBackend(core, dataSource)
}
private class JdbcDatabaseBackend(
private val core: SqlCoreModule,
private val dataSource: HikariDataSource,
) : SqlDatabaseBackend {
override suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T {
val connection = try {
dataSource.connection
} catch (e: SQLException) {
throw mapOpenException(scope, core, e)
}
try {
val tx = JdbcTransactionBackend(core, connection)
val result = try {
block(tx)
} catch (e: Throwable) {
throw finishFailedTransaction(scope, core, e) {
rollbackOrThrow(scope, core, connection)
}
}
try {
connection.commit()
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
return result
} finally {
try {
connection.close() // returns connection to pool
} catch (_: SQLException) {
}
}
}
override fun close() {
dataSource.close()
}
}
private class JdbcTransactionBackend(
private val core: SqlCoreModule,
private val connection: Connection,
) : SqlTransactionBackend {
override suspend fun select(scope: ScopeFacade, clause: String, params: List<Obj>): SqlResultSetData {
try {
connection.prepareStatement(clause).use { statement ->
bindParams(statement, params, scope, core)
statement.executeQuery().use { rs ->
return readResultSet(scope, core, rs)
}
}
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
override suspend fun execute(scope: ScopeFacade, clause: String, params: List<Obj>): SqlExecutionResultData {
if (containsRowReturningClause(clause)) {
scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("execute(...) cannot be used with statements that return rows; use select(...)")
)
)
}
try {
connection.prepareStatement(clause, Statement.RETURN_GENERATED_KEYS).use { statement ->
bindParams(statement, params, scope, core)
val hasResultSet = statement.execute()
if (hasResultSet) {
scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("execute(...) cannot be used with statements that return rows; use select(...)")
)
)
}
val generatedKeys = statement.generatedKeys.use { rs ->
if (rs == null) emptyResultSet() else readResultSet(scope, core, rs)
}
return SqlExecutionResultData(statement.updateCount, generatedKeys)
}
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
override suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T {
val savepoint = try {
connection.setSavepoint()
} catch (e: SQLException) {
throw mapSqlUsage(scope, core, "Nested transactions are not supported by this JDBC backend", e)
}
val nested = JdbcTransactionBackend(core, connection)
val result = try {
block(nested)
} catch (e: Throwable) {
throw finishFailedTransaction(scope, core, e) {
rollbackToSavepointOrThrow(scope, core, connection, savepoint)
releaseSavepointOrThrow(scope, core, connection, savepoint)
}
}
try {
connection.releaseSavepoint(savepoint)
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
return result
}
}
private suspend fun bindParams(statement: PreparedStatement, params: List<Obj>, scope: ScopeFacade, core: SqlCoreModule) {
params.forEachIndexed { index, value ->
val jdbcIndex = index + 1
when (value) {
ObjNull -> statement.setObject(jdbcIndex, null)
is ObjBool -> statement.setBoolean(jdbcIndex, value.value)
is ObjInt -> statement.setLong(jdbcIndex, value.value)
is ObjReal -> statement.setDouble(jdbcIndex, value.value)
is ObjString -> statement.setString(jdbcIndex, value.value)
is ObjBuffer -> statement.setBytes(jdbcIndex, value.byteArray.toByteArray())
is ObjDate -> statement.setObject(jdbcIndex, java.time.LocalDate.parse(value.date.toString()))
is ObjDateTime -> statement.setObject(jdbcIndex, java.time.LocalDateTime.parse(value.localDateTime.toString()))
is ObjInstant -> statement.setObject(jdbcIndex, java.time.Instant.parse(value.instant.toString()))
else -> when (value.objClass.className) {
"Decimal" -> statement.setBigDecimal(jdbcIndex, BigDecimal(scope.toStringOf(value).value))
else -> scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("Unsupported JDBC parameter type: ${value.objClass.className}")
)
)
}
}
}
}
private suspend fun readResultSet(scope: ScopeFacade, core: SqlCoreModule, resultSet: ResultSet): SqlResultSetData {
val meta = resultSet.metaData
val columns = (1..meta.columnCount).map { index ->
val nativeType = meta.getColumnTypeName(index) ?: ""
SqlColumnMeta(
name = meta.getColumnLabel(index),
sqlType = mapSqlType(core, nativeType, meta.getColumnType(index)),
nullable = meta.isNullable(index) != java.sql.ResultSetMetaData.columnNoNulls,
nativeType = nativeType,
)
}
val rows = mutableListOf<List<Obj>>()
while (resultSet.next()) {
rows += columns.mapIndexed { index, column ->
readColumnValue(scope, core, resultSet, index + 1, column.nativeType)
}
}
return SqlResultSetData(columns, rows)
}
private suspend fun readColumnValue(
scope: ScopeFacade,
core: SqlCoreModule,
resultSet: ResultSet,
index: Int,
nativeType: String,
): Obj {
val value = resultSet.getObject(index) ?: return ObjNull
val normalizedNativeType = normalizeDeclaredTypeName(nativeType)
return when (value) {
is Boolean -> ObjBool(value)
is Byte, is Short, is Int -> ObjInt.of((value as Number).toLong())
is Long -> ObjInt.of(value)
is Float, is Double -> ObjReal.of((value as Number).toDouble())
is java.math.BigInteger -> ObjInt.of(value.longValueExact())
is BigDecimal -> decimalFromString(scope, value.toPlainString())
is ByteArray -> ObjBuffer(value.toUByteArray())
is java.sql.Date -> ObjDate(LocalDate.parse(value.toLocalDate().toString()))
is java.sql.Timestamp -> timestampValue(scope, normalizedNativeType, value.toInstant())
is java.time.LocalDate -> ObjDate(LocalDate.parse(value.toString()))
is java.time.LocalDateTime -> ObjDateTime(value.toString().let(LocalDateTime::parse).toInstant(TimeZone.UTC), TimeZone.UTC)
is java.time.OffsetDateTime -> ObjInstant(Instant.parse(value.toInstant().toString()))
is java.time.ZonedDateTime -> ObjInstant(Instant.parse(value.toInstant().toString()))
is java.time.Instant -> ObjInstant(Instant.parse(value.toString()))
is java.sql.Time -> ObjString(value.toLocalTime().toString())
is java.time.LocalTime -> ObjString(value.toString())
is java.time.OffsetTime -> ObjString(value.toString())
is String -> stringValue(scope, normalizedNativeType, value)
else -> ObjString(value.toString())
}
}
private suspend fun stringValue(scope: ScopeFacade, normalizedNativeType: String, value: String): Obj {
return when {
normalizedNativeType == "DATE" -> ObjDate(LocalDate.parse(value.trim()))
normalizedNativeType == "TIMESTAMP WITH TIME ZONE" || normalizedNativeType == "TIMESTAMPTZ" ->
ObjInstant(Instant.parse(value.trim()))
normalizedNativeType == "TIMESTAMP" || normalizedNativeType == "DATETIME" ->
ObjDateTime(LocalDateTime.parse(value.trim()).toInstant(TimeZone.UTC), TimeZone.UTC)
normalizedNativeType == "DECIMAL" || normalizedNativeType == "NUMERIC" -> decimalFromString(scope, value.trim())
else -> ObjString(value)
}
}
private fun timestampValue(scope: ScopeFacade, normalizedNativeType: String, value: java.time.Instant): Obj {
return if (normalizedNativeType == "TIMESTAMP WITH TIME ZONE" || normalizedNativeType == "TIMESTAMPTZ") {
ObjInstant(Instant.parse(value.toString()))
} else {
val local = value.atOffset(java.time.ZoneOffset.UTC).toLocalDateTime().toString()
ObjDateTime(LocalDateTime.parse(local).toInstant(TimeZone.UTC), TimeZone.UTC)
}
}
private suspend fun decimalFromString(scope: ScopeFacade, value: String): Obj {
val decimalModule = scope.requireScope().currentImportProvider.createModuleScope(scope.pos, "lyng.decimal")
val decimalClass = decimalModule.requireClass("Decimal")
return decimalClass.invokeInstanceMethod(scope.requireScope(), "fromString", ObjString(value))
}
private fun normalizeDeclaredTypeName(nativeTypeName: String): String {
val strippedSuffix = nativeTypeName.trim().replace(Regex("""\s*\(.*\)\s*$"""), "")
return strippedSuffix.uppercase().replace(Regex("""\s+"""), " ").trim()
}
private fun mapSqlType(core: SqlCoreModule, nativeTypeName: String, jdbcType: Int): ObjEnumEntry {
val normalized = normalizeDeclaredTypeName(nativeTypeName)
return when {
normalized == "BOOLEAN" || normalized == "BOOL" -> core.sqlTypes.require("Bool")
normalized == "DATE" -> core.sqlTypes.require("Date")
normalized == "TIMESTAMP" || normalized == "DATETIME" -> core.sqlTypes.require("DateTime")
normalized == "TIMESTAMP WITH TIME ZONE" || normalized == "TIMESTAMPTZ" -> core.sqlTypes.require("Instant")
normalized == "TIME" || normalized == "TIME WITHOUT TIME ZONE" || normalized == "TIME WITH TIME ZONE" -> core.sqlTypes.require("String")
normalized == "DECIMAL" || normalized == "NUMERIC" -> core.sqlTypes.require("Decimal")
normalized.contains("BLOB") || normalized.contains("BINARY") || normalized == "BYTEA" -> core.sqlTypes.require("Binary")
normalized.contains("INT") -> core.sqlTypes.require("Int")
normalized.contains("CHAR") || normalized.contains("TEXT") || normalized.contains("CLOB") || normalized == "VARCHAR" -> core.sqlTypes.require("String")
normalized.contains("REAL") || normalized.contains("FLOA") || normalized.contains("DOUB") -> core.sqlTypes.require("Double")
jdbcType == java.sql.Types.BOOLEAN || jdbcType == java.sql.Types.BIT -> core.sqlTypes.require("Bool")
jdbcType == java.sql.Types.DATE -> core.sqlTypes.require("Date")
jdbcType == java.sql.Types.TIMESTAMP -> core.sqlTypes.require("DateTime")
jdbcType == java.sql.Types.TIMESTAMP_WITH_TIMEZONE -> core.sqlTypes.require("Instant")
jdbcType == java.sql.Types.TIME || jdbcType == java.sql.Types.TIME_WITH_TIMEZONE -> core.sqlTypes.require("String")
jdbcType == java.sql.Types.BLOB || jdbcType == java.sql.Types.BINARY || jdbcType == java.sql.Types.VARBINARY || jdbcType == java.sql.Types.LONGVARBINARY -> core.sqlTypes.require("Binary")
jdbcType == java.sql.Types.INTEGER || jdbcType == java.sql.Types.BIGINT || jdbcType == java.sql.Types.SMALLINT || jdbcType == java.sql.Types.TINYINT -> core.sqlTypes.require("Int")
jdbcType == java.sql.Types.DECIMAL || jdbcType == java.sql.Types.NUMERIC -> core.sqlTypes.require("Decimal")
jdbcType == java.sql.Types.FLOAT || jdbcType == java.sql.Types.REAL || jdbcType == java.sql.Types.DOUBLE -> core.sqlTypes.require("Double")
else -> core.sqlTypes.require("String")
}
}
private fun emptyResultSet(): SqlResultSetData = SqlResultSetData(emptyList(), emptyList())
private fun containsRowReturningClause(clause: String): Boolean =
Regex("""\b(returning|output)\b""", RegexOption.IGNORE_CASE).containsMatchIn(clause)
private fun ensureJdbcDriversLoaded(scope: ScopeFacade, core: SqlCoreModule, requestedDriverClass: String?) {
for (driverClass in knownJdbcDrivers) {
try {
Class.forName(driverClass)
} catch (_: Throwable) {
}
}
val explicit = requestedDriverClass ?: return
try {
Class.forName(explicit)
} catch (e: ClassNotFoundException) {
throw ExecutionError(
ObjException(core.databaseException, scope.requireScope(), ObjString("JDBC driver class not found: $explicit")),
scope.pos,
"JDBC driver class not found: $explicit",
e,
)
}
}
private fun rollbackOrThrow(scope: ScopeFacade, core: SqlCoreModule, connection: Connection) {
try {
connection.rollback()
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
private fun rollbackToSavepointOrThrow(
scope: ScopeFacade,
core: SqlCoreModule,
connection: Connection,
savepoint: java.sql.Savepoint,
) {
try {
connection.rollback(savepoint)
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
private fun releaseSavepointOrThrow(
scope: ScopeFacade,
core: SqlCoreModule,
connection: Connection,
savepoint: java.sql.Savepoint,
) {
try {
connection.releaseSavepoint(savepoint)
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
private inline fun finishFailedTransaction(
scope: ScopeFacade,
core: SqlCoreModule,
failure: Throwable,
rollback: () -> Unit,
): Throwable {
return try {
rollback()
failure
} catch (rollbackFailure: Throwable) {
if (isRollbackSignal(failure, core)) {
attachSecondaryFailure(rollbackFailure, failure)
rollbackFailure
} else {
attachSecondaryFailure(failure, rollbackFailure)
failure
}
}
}
private fun isRollbackSignal(failure: Throwable, core: SqlCoreModule): Boolean {
val errorObject = (failure as? ExecutionError)?.errorObject ?: return false
return errorObject.isInstanceOf(core.rollbackException)
}
private fun attachSecondaryFailure(primary: Throwable, secondary: Throwable) {
if (primary === secondary) return
primary.addSuppressed(secondary)
}
private fun mapOpenException(scope: ScopeFacade, core: SqlCoreModule, e: SQLException): Nothing {
val message = e.message ?: "JDBC open failed"
if (e is SQLNonTransientConnectionException) {
throw ExecutionError(
ObjException(core.databaseException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
e,
)
}
throw ExecutionError(
ObjException(core.databaseException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
e,
)
}
private fun mapSqlException(scope: ScopeFacade, core: SqlCoreModule, e: SQLException): ExecutionError {
val exceptionClass = when {
e is SQLIntegrityConstraintViolationException -> core.sqlConstraintException
e.sqlState?.startsWith("23") == true -> core.sqlConstraintException
else -> core.sqlExecutionException
}
return ExecutionError(
ObjException(exceptionClass, scope.requireScope(), ObjString(e.message ?: "JDBC error")),
scope.pos,
e.message ?: "JDBC error",
e,
)
}
private fun mapSqlUsage(scope: ScopeFacade, core: SqlCoreModule, message: String, cause: Throwable? = null): ExecutionError {
return ExecutionError(
ObjException(core.sqlUsageException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
cause,
)
}
@@ -0,0 +1,507 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.Arguments
import net.sergeych.lyng.ScopeFacade
import net.sergeych.lyng.io.db.SqlColumnMeta
import net.sergeych.lyng.io.db.SqlCoreModule
import net.sergeych.lyng.io.db.SqlDatabaseBackend
import net.sergeych.lyng.io.db.SqlExecutionResultData
import net.sergeych.lyng.io.db.SqlResultSetData
import net.sergeych.lyng.io.db.SqlTransactionBackend
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjBool
import net.sergeych.lyng.obj.ObjBuffer
import net.sergeych.lyng.obj.ObjDate
import net.sergeych.lyng.obj.ObjDateTime
import net.sergeych.lyng.obj.ObjEnumEntry
import net.sergeych.lyng.obj.ObjException
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjInstant
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjReal
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.requireScope
import kotlinx.datetime.LocalDate
import kotlinx.datetime.LocalDateTime
import kotlinx.datetime.TimeZone
import kotlinx.datetime.toInstant
import org.sqlite.SQLiteConfig
import org.sqlite.SQLiteErrorCode
import org.sqlite.SQLiteOpenMode
import java.sql.Connection
import java.sql.DriverManager
import java.sql.PreparedStatement
import java.sql.ResultSet
import java.sql.SQLException
import java.sql.Statement
import kotlin.time.Instant
internal actual suspend fun openSqliteBackend(
scope: ScopeFacade,
core: SqlCoreModule,
options: SqliteOpenOptions,
): SqlDatabaseBackend {
if (options.busyTimeoutMillis < 0) {
scope.raiseIllegalArgument("busyTimeoutMillis must be >= 0")
}
return JdbcSqlDatabaseBackend(core, options)
}
private class JdbcSqlDatabaseBackend(
private val core: SqlCoreModule,
private val options: SqliteOpenOptions,
) : SqlDatabaseBackend {
override suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T {
val connection = openConnection(scope)
try {
connection.autoCommit = false
val tx = JdbcSqlTransactionBackend(core, connection)
val result = try {
block(tx)
} catch (e: Throwable) {
throw finishFailedTransaction(scope, core, e) {
rollbackOrThrow(scope, core, connection)
}
}
try {
connection.commit()
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
return result
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
} finally {
try {
connection.close()
} catch (_: SQLException) {
}
}
}
private fun openConnection(scope: ScopeFacade): Connection {
try {
val config = SQLiteConfig().apply {
setOpenMode(SQLiteOpenMode.OPEN_URI)
if (options.readOnly) {
setReadOnly(true)
setOpenMode(SQLiteOpenMode.READONLY)
} else {
setReadOnly(false)
setOpenMode(SQLiteOpenMode.READWRITE)
if (options.createIfMissing) {
setOpenMode(SQLiteOpenMode.CREATE)
}
}
enforceForeignKeys(options.foreignKeys)
busyTimeout = options.busyTimeoutMillis
}
return DriverManager.getConnection(jdbcUrl(options.path), config.toProperties())
} catch (e: SQLException) {
throw mapOpenException(scope, core, e)
} catch (e: IllegalArgumentException) {
scope.raiseIllegalArgument(e.message ?: "Invalid SQLite configuration")
}
}
private fun jdbcUrl(path: String): String {
return when (path) {
":memory:" -> "jdbc:sqlite::memory:"
else -> if (path.startsWith("/")) "jdbc:sqlite:$path" else "jdbc:sqlite:$path"
}
}
}
private class JdbcSqlTransactionBackend(
private val core: SqlCoreModule,
private val connection: Connection,
) : SqlTransactionBackend {
override suspend fun select(scope: ScopeFacade, clause: String, params: List<Obj>): SqlResultSetData {
try {
connection.prepareStatement(clause).use { statement ->
bindParams(statement, params, scope, core)
statement.executeQuery().use { rs ->
return readResultSet(scope, core, rs)
}
}
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
override suspend fun execute(scope: ScopeFacade, clause: String, params: List<Obj>): SqlExecutionResultData {
if (containsRowReturningClause(clause)) {
scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("execute(...) cannot be used with statements that return rows; use select(...)")
)
)
}
try {
connection.prepareStatement(clause, Statement.RETURN_GENERATED_KEYS).use { statement ->
bindParams(statement, params, scope, core)
val hasResultSet = statement.execute()
if (hasResultSet) {
scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("execute(...) cannot be used with statements that return rows; use select(...)")
)
)
}
val affected = statement.updateCount
val generatedKeys = statement.generatedKeys.use { rs ->
if (rs == null) {
emptyResultSet(core)
} else {
readResultSet(scope, core, rs)
}
}
return SqlExecutionResultData(affected, generatedKeys)
}
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
override suspend fun <T> transaction(scope: ScopeFacade, block: suspend (SqlTransactionBackend) -> T): T {
val savepoint = try {
connection.setSavepoint()
} catch (e: SQLException) {
throw mapSqlUsage(scope, core, "Nested transactions are not supported by this SQLite backend", e)
}
val nested = JdbcSqlTransactionBackend(core, connection)
val result = try {
block(nested)
} catch (e: Throwable) {
throw finishFailedTransaction(scope, core, e) {
rollbackToSavepointOrThrow(scope, core, connection, savepoint)
releaseSavepointOrThrow(scope, core, connection, savepoint)
}
}
try {
connection.releaseSavepoint(savepoint)
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
return result
}
}
private suspend fun bindParams(statement: PreparedStatement, params: List<Obj>, scope: ScopeFacade, core: SqlCoreModule) {
params.forEachIndexed { index, value ->
val jdbcIndex = index + 1
when (value) {
ObjNull -> statement.setObject(jdbcIndex, null)
is ObjBool -> statement.setLong(jdbcIndex, if (value.value) 1L else 0L)
is ObjInt -> statement.setLong(jdbcIndex, value.value)
is ObjReal -> statement.setDouble(jdbcIndex, value.value)
is ObjString -> statement.setString(jdbcIndex, value.value)
is ObjBuffer -> statement.setBytes(jdbcIndex, value.byteArray.toByteArray())
is ObjInstant -> statement.setString(jdbcIndex, value.instant.toString())
is ObjDateTime -> statement.setString(jdbcIndex, value.localDateTime.toString())
else -> when (value.objClass.className) {
"Date", "Decimal" -> statement.setString(jdbcIndex, scope.toStringOf(value).value)
else -> scope.raiseError(
ObjException(
core.sqlUsageException,
scope.requireScope(),
ObjString("Unsupported SQLite parameter type: ${value.objClass.className}")
)
)
}
}
}
}
private suspend fun readResultSet(
scope: ScopeFacade,
core: SqlCoreModule,
resultSet: ResultSet,
): SqlResultSetData {
val meta = resultSet.metaData
val columns = (1..meta.columnCount).map { index ->
SqlColumnMeta(
name = meta.getColumnLabel(index),
sqlType = mapSqlType(core, meta.getColumnTypeName(index), meta.getColumnType(index)),
nullable = meta.isNullable(index) != java.sql.ResultSetMetaData.columnNoNulls,
nativeType = meta.getColumnTypeName(index) ?: "",
)
}
val rows = mutableListOf<List<Obj>>()
while (resultSet.next()) {
rows += columns.mapIndexed { index, column ->
readColumnValue(scope, core, resultSet, index + 1, column.nativeType)
}
}
return SqlResultSetData(columns, rows)
}
private suspend fun readColumnValue(
scope: ScopeFacade,
core: SqlCoreModule,
resultSet: ResultSet,
index: Int,
nativeType: String,
): Obj {
val value = resultSet.getObject(index) ?: return ObjNull
val normalizedNativeType = normalizeDeclaredTypeName(nativeType)
if (isDecimalNativeType(normalizedNativeType) && value is Number) {
return decimalFromString(scope, value.toString())
}
return when (value) {
is Boolean -> if (isBooleanNativeType(normalizedNativeType)) ObjBool(value) else ObjInt.of(if (value) 1 else 0)
is Byte, is Short, is Int -> convertIntegerValue(scope, core, normalizedNativeType, (value as Number).toLong())
is Long -> convertIntegerValue(scope, core, normalizedNativeType, value)
is Float, is Double -> ObjReal.of((value as Number).toDouble())
is ByteArray -> ObjBuffer(value.toUByteArray())
is String -> convertStringValue(scope, core, normalizedNativeType, value)
is java.math.BigDecimal -> decimalFromString(scope, value.toPlainString())
else -> ObjString(value.toString())
}
}
private fun convertIntegerValue(
scope: ScopeFacade,
core: SqlCoreModule,
normalizedNativeType: String,
value: Long,
): Obj {
if (!isBooleanNativeType(normalizedNativeType)) {
return ObjInt.of(value)
}
return when (value) {
0L -> ObjBool(false)
1L -> ObjBool(true)
else -> sqlExecutionFailure(scope, core, "Invalid SQLite boolean value: $value")
}
}
private suspend fun convertStringValue(
scope: ScopeFacade,
core: SqlCoreModule,
normalizedNativeType: String,
value: String,
): Obj {
return when {
isBooleanNativeType(normalizedNativeType) -> booleanFromString(scope, core, value)
isDecimalNativeType(normalizedNativeType) -> decimalFromString(scope, value.trim())
normalizedNativeType == "DATE" -> ObjDate(LocalDate.parse(value.trim()))
normalizedNativeType == "DATETIME" || normalizedNativeType == "TIMESTAMP" ->
dateTimeFromString(scope, core, value)
normalizedNativeType == "TIMESTAMP WITH TIME ZONE" ||
normalizedNativeType == "TIMESTAMPTZ" ||
normalizedNativeType == "DATETIME WITH TIME ZONE" -> ObjInstant(Instant.parse(value.trim()))
else -> ObjString(value)
}
}
private fun isDecimalNativeType(normalizedNativeType: String): Boolean =
normalizedNativeType == "DECIMAL" || normalizedNativeType == "NUMERIC"
private fun isBooleanNativeType(normalizedNativeType: String): Boolean =
normalizedNativeType == "BOOLEAN" || normalizedNativeType == "BOOL"
private fun booleanFromString(scope: ScopeFacade, core: SqlCoreModule, value: String): Obj {
return when (value.trim().lowercase()) {
"true", "t" -> ObjBool(true)
"false", "f" -> ObjBool(false)
else -> sqlExecutionFailure(scope, core, "Invalid SQLite boolean value: $value")
}
}
private suspend fun decimalFromString(scope: ScopeFacade, value: String): Obj {
val decimalModule = scope.requireScope().currentImportProvider.createModuleScope(scope.pos, "lyng.decimal")
val decimalClass = decimalModule.requireClass("Decimal")
return decimalClass.invokeInstanceMethod(scope.requireScope(), "fromString", ObjString(value))
}
private fun dateTimeFromString(scope: ScopeFacade, core: SqlCoreModule, value: String): ObjDateTime {
val trimmed = value.trim()
if (hasExplicitTimeZone(trimmed)) {
sqlExecutionFailure(scope, core, "SQLite TIMESTAMP/DATETIME value must not contain a timezone offset: $value")
}
val local = LocalDateTime.parse(trimmed)
return ObjDateTime(local.toInstant(TimeZone.UTC), TimeZone.UTC)
}
private fun hasExplicitTimeZone(value: String): Boolean {
if (value.endsWith("Z", ignoreCase = true)) return true
val tIndex = value.indexOf('T')
if (tIndex < 0) return false
val plus = value.lastIndexOf('+')
val minus = value.lastIndexOf('-')
val offsetStart = maxOf(plus, minus)
return offsetStart > tIndex
}
private fun containsRowReturningClause(clause: String): Boolean =
Regex("""\breturning\b""", RegexOption.IGNORE_CASE).containsMatchIn(clause)
private fun normalizeDeclaredTypeName(nativeTypeName: String): String {
val strippedSuffix = nativeTypeName.trim().replace(Regex("""\s*\(.*\)\s*$"""), "")
return strippedSuffix.uppercase().replace(Regex("""\s+"""), " ").trim()
}
private fun mapSqlType(core: SqlCoreModule, nativeTypeName: String, jdbcType: Int): ObjEnumEntry {
val normalized = normalizeDeclaredTypeName(nativeTypeName)
return when {
normalized == "BOOLEAN" || normalized == "BOOL" -> core.sqlTypes.require("Bool")
normalized == "DATE" -> core.sqlTypes.require("Date")
normalized == "DATETIME" || normalized == "TIMESTAMP" -> core.sqlTypes.require("DateTime")
normalized == "TIMESTAMP WITH TIME ZONE" ||
normalized == "TIMESTAMPTZ" ||
normalized == "DATETIME WITH TIME ZONE" -> core.sqlTypes.require("Instant")
normalized == "TIME" ||
normalized == "TIME WITHOUT TIME ZONE" ||
normalized == "TIME WITH TIME ZONE" -> core.sqlTypes.require("String")
normalized == "DECIMAL" || normalized == "NUMERIC" -> core.sqlTypes.require("Decimal")
normalized.contains("BLOB") -> core.sqlTypes.require("Binary")
normalized.contains("INT") -> core.sqlTypes.require("Int")
normalized.contains("CHAR") || normalized.contains("TEXT") || normalized.contains("CLOB") -> core.sqlTypes.require("String")
normalized.contains("REAL") || normalized.contains("FLOA") || normalized.contains("DOUB") -> core.sqlTypes.require("Double")
jdbcType == java.sql.Types.BOOLEAN -> core.sqlTypes.require("Bool")
jdbcType == java.sql.Types.BLOB || jdbcType == java.sql.Types.BINARY || jdbcType == java.sql.Types.VARBINARY -> core.sqlTypes.require("Binary")
jdbcType == java.sql.Types.INTEGER -> core.sqlTypes.require("Int")
jdbcType == java.sql.Types.BIGINT -> core.sqlTypes.require("Int")
jdbcType == java.sql.Types.DECIMAL || jdbcType == java.sql.Types.NUMERIC -> core.sqlTypes.require("Decimal")
jdbcType == java.sql.Types.FLOAT || jdbcType == java.sql.Types.REAL || jdbcType == java.sql.Types.DOUBLE -> core.sqlTypes.require("Double")
else -> core.sqlTypes.require("String")
}
}
private fun emptyResultSet(core: SqlCoreModule): SqlResultSetData = SqlResultSetData(emptyList(), emptyList())
private fun sqlExecutionFailure(scope: ScopeFacade, core: SqlCoreModule, message: String): Nothing {
throw ExecutionError(
ObjException(core.sqlExecutionException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
)
}
private fun rollbackOrThrow(scope: ScopeFacade, core: SqlCoreModule, connection: Connection) {
try {
connection.rollback()
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
private fun rollbackToSavepointOrThrow(
scope: ScopeFacade,
core: SqlCoreModule,
connection: Connection,
savepoint: java.sql.Savepoint,
) {
try {
connection.rollback(savepoint)
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
private fun releaseSavepointOrThrow(
scope: ScopeFacade,
core: SqlCoreModule,
connection: Connection,
savepoint: java.sql.Savepoint,
) {
try {
connection.releaseSavepoint(savepoint)
} catch (e: SQLException) {
throw mapSqlException(scope, core, e)
}
}
private inline fun finishFailedTransaction(
scope: ScopeFacade,
core: SqlCoreModule,
failure: Throwable,
rollback: () -> Unit,
): Throwable {
return try {
rollback()
failure
} catch (rollbackFailure: Throwable) {
if (isRollbackSignal(failure, core)) {
attachSecondaryFailure(rollbackFailure, failure)
rollbackFailure
} else {
attachSecondaryFailure(failure, rollbackFailure)
failure
}
}
}
private fun isRollbackSignal(failure: Throwable, core: SqlCoreModule): Boolean {
val errorObject = (failure as? ExecutionError)?.errorObject ?: return false
return errorObject.isInstanceOf(core.rollbackException)
}
private fun attachSecondaryFailure(primary: Throwable, secondary: Throwable) {
if (primary === secondary) return
primary.addSuppressed(secondary)
}
private fun mapOpenException(scope: ScopeFacade, core: SqlCoreModule, e: SQLException): Nothing {
val message = e.message ?: "SQLite open failed"
val lower = message.lowercase()
if ("malformed" in lower || "no such access mode" in lower || "invalid uri" in lower) {
scope.raiseIllegalArgument(message)
}
throw ExecutionError(
ObjException(core.databaseException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
e,
)
}
private fun mapSqlException(scope: ScopeFacade, core: SqlCoreModule, e: SQLException): ExecutionError {
val code = SQLiteErrorCode.getErrorCode(e.errorCode)
val exceptionClass = when (code) {
SQLiteErrorCode.SQLITE_CONSTRAINT,
SQLiteErrorCode.SQLITE_CONSTRAINT_PRIMARYKEY,
SQLiteErrorCode.SQLITE_CONSTRAINT_UNIQUE,
SQLiteErrorCode.SQLITE_CONSTRAINT_FOREIGNKEY,
SQLiteErrorCode.SQLITE_CONSTRAINT_NOTNULL -> core.sqlConstraintException
else -> core.sqlExecutionException
}
return ExecutionError(
ObjException(exceptionClass, scope.requireScope(), ObjString(e.message ?: "SQLite error")),
scope.pos,
e.message ?: "SQLite error",
e,
)
}
private fun mapSqlUsage(scope: ScopeFacade, core: SqlCoreModule, message: String, cause: Throwable? = null): ExecutionError {
return ExecutionError(
ObjException(core.sqlUsageException, scope.requireScope(), ObjString(message)),
scope.pos,
message,
cause,
)
}
@@ -36,6 +36,97 @@ import kotlin.time.Duration.Companion.milliseconds
import kotlin.time.Duration.Companion.seconds import kotlin.time.Duration.Companion.seconds
import kotlin.time.TimeSource import kotlin.time.TimeSource
internal object JvmConsoleKeyDecoder {
fun decode(firstCode: Int, nextCode: (Long) -> Int?): ConsoleEvent.KeyDown {
if (firstCode == 27) {
val next = nextCode(25L)
if (next == null || next < 0) {
return key("Escape")
}
if (next == '['.code || next == 'O'.code) {
val sb = StringBuilder()
sb.append(next.toChar())
var i = 0
while (i < 6) {
val c = nextCode(25L) ?: break
if (c < 0) break
sb.append(c.toChar())
if (c.toChar().isLetter() || c == '~'.code) break
i += 1
}
return keyFromAnsiSequence(sb.toString()) ?: key("Escape")
}
val base = decodePlainKey(next)
return ConsoleEvent.KeyDown(
key = base.key,
code = base.code,
ctrl = base.ctrl,
alt = true,
shift = base.shift,
meta = false,
)
}
return decodePlainKey(firstCode)
}
private fun decodePlainKey(code: Int): ConsoleEvent.KeyDown = when (code) {
3 -> key("c", ctrl = true)
9 -> key("Tab")
10, 13 -> key("Enter")
127, 8 -> key("Backspace")
32 -> key(" ")
else -> {
if (code in 1..26) {
val ch = ('a'.code + code - 1).toChar().toString()
key(ch, ctrl = true)
} else {
val ch = code.toChar().toString()
key(ch, shift = ch.length == 1 && ch[0].isLetter() && ch[0].isUpperCase())
}
}
}
private fun keyFromAnsiSequence(seq: String): ConsoleEvent.KeyDown? {
val shift = seq.contains(";2")
val alt = seq.contains(";3")
val ctrl = seq.contains(";5")
val key = when {
seq.endsWith("A") -> "ArrowUp"
seq.endsWith("B") -> "ArrowDown"
seq.endsWith("C") -> "ArrowRight"
seq.endsWith("D") -> "ArrowLeft"
seq.endsWith("H") -> "Home"
seq.endsWith("F") -> "End"
seq.endsWith("~") -> when (seq.substringAfter('[').substringBefore(';').substringBefore('~')) {
"2" -> "Insert"
"3" -> "Delete"
"5" -> "PageUp"
"6" -> "PageDown"
else -> null
}
else -> null
} ?: return null
return key(key, ctrl = ctrl, alt = alt, shift = shift)
}
private fun key(
value: String,
ctrl: Boolean = false,
alt: Boolean = false,
shift: Boolean = false,
): ConsoleEvent.KeyDown {
require(value.isNotEmpty()) { "ConsoleEvent.KeyDown.key must never be empty" }
return ConsoleEvent.KeyDown(
key = value,
code = null,
ctrl = ctrl,
alt = alt,
shift = shift,
meta = false,
)
}
}
/** /**
* JVM console implementation: * JVM console implementation:
* - output/capabilities/input use a single JLine terminal instance * - output/capabilities/input use a single JLine terminal instance
@@ -508,40 +599,7 @@ object JvmLyngConsole : LyngConsole {
val code = reader.read(120L) val code = reader.read(120L)
if (code == NonBlockingReader.READ_EXPIRED) return null if (code == NonBlockingReader.READ_EXPIRED) return null
if (code < 0) throw EOFException("non-blocking reader returned EOF") if (code < 0) throw EOFException("non-blocking reader returned EOF")
return decodeKey(code) { timeout -> readNextCode(reader, timeout) } return JvmConsoleKeyDecoder.decode(code) { timeout -> readNextCode(reader, timeout) }
}
private fun decodeKey(code: Int, nextCode: (Long) -> Int?): ConsoleEvent.KeyDown {
if (code == 27) {
val next = nextCode(25L)
if (next == null || next < 0) {
return key("Escape")
}
if (next == '['.code || next == 'O'.code) {
val sb = StringBuilder()
sb.append(next.toChar())
var i = 0
while (i < 6) {
val c = nextCode(25L) ?: break
if (c < 0) break
sb.append(c.toChar())
if (c.toChar().isLetter() || c == '~'.code) break
i += 1
}
return keyFromAnsiSequence(sb.toString()) ?: key("Escape")
}
// Alt+key
val base = decodePlainKey(next)
return ConsoleEvent.KeyDown(
key = base.key,
code = base.code,
ctrl = base.ctrl,
alt = true,
shift = base.shift,
meta = false
)
}
return decodePlainKey(code)
} }
private fun readNextCode(reader: NonBlockingReader, timeoutMs: Long): Int? { private fun readNextCode(reader: NonBlockingReader, timeoutMs: Long): Int? {
@@ -550,53 +608,4 @@ object JvmLyngConsole : LyngConsole {
if (c < 0) throw EOFException("non-blocking reader returned EOF while decoding key sequence") if (c < 0) throw EOFException("non-blocking reader returned EOF while decoding key sequence")
return c return c
} }
private fun decodePlainKey(code: Int): ConsoleEvent.KeyDown = when (code) {
3 -> key("c", ctrl = true)
9 -> key("Tab")
10, 13 -> key("Enter")
127, 8 -> key("Backspace")
32 -> key(" ")
else -> {
if (code in 1..26) {
val ch = ('a'.code + code - 1).toChar().toString()
key(ch, ctrl = true)
} else {
val ch = code.toChar().toString()
key(ch, shift = ch.length == 1 && ch[0].isLetter() && ch[0].isUpperCase())
}
}
}
private fun keyFromAnsiSequence(seq: String): ConsoleEvent.KeyDown? = when (seq) {
"[A", "OA" -> key("ArrowUp")
"[B", "OB" -> key("ArrowDown")
"[C", "OC" -> key("ArrowRight")
"[D", "OD" -> key("ArrowLeft")
"[H", "OH" -> key("Home")
"[F", "OF" -> key("End")
"[2~" -> key("Insert")
"[3~" -> key("Delete")
"[5~" -> key("PageUp")
"[6~" -> key("PageDown")
else -> null
}
private fun key(
value: String,
ctrl: Boolean = false,
alt: Boolean = false,
shift: Boolean = false,
): ConsoleEvent.KeyDown {
require(value.isNotEmpty()) { "ConsoleEvent.KeyDown.key must never be empty" }
return ConsoleEvent.KeyDown(
key = value,
code = null,
ctrl = ctrl,
alt = alt,
shift = shift,
meta = false
)
}
} }
@@ -45,6 +45,8 @@ import java.net.InetAddress
actual fun getSystemNetEngine(): LyngNetEngine = JvmKtorNetEngine actual fun getSystemNetEngine(): LyngNetEngine = JvmKtorNetEngine
actual fun shutdownSystemNetEngine() {}
private object JvmKtorNetEngine : LyngNetEngine { private object JvmKtorNetEngine : LyngNetEngine {
private val selectorManager: SelectorManager by lazy { ActorSelectorManager(Dispatchers.IO) } private val selectorManager: SelectorManager by lazy { ActorSelectorManager(Dispatchers.IO) }
@@ -22,16 +22,14 @@ import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.Scope import net.sergeych.lyng.Scope
import net.sergeych.lyng.obj.ObjBool import net.sergeych.lyng.obj.ObjBool
import net.sergeych.lyng.obj.ObjIllegalOperationException import net.sergeych.lyng.obj.ObjIllegalOperationException
import net.sergeych.lyngio.console.*
import net.sergeych.lyngio.console.security.ConsoleAccessOp import net.sergeych.lyngio.console.security.ConsoleAccessOp
import net.sergeych.lyngio.console.security.ConsoleAccessPolicy import net.sergeych.lyngio.console.security.ConsoleAccessPolicy
import net.sergeych.lyngio.console.security.PermitAllConsoleAccessPolicy import net.sergeych.lyngio.console.security.PermitAllConsoleAccessPolicy
import net.sergeych.lyngio.fs.security.AccessContext import net.sergeych.lyngio.fs.security.AccessContext
import net.sergeych.lyngio.fs.security.AccessDecision import net.sergeych.lyngio.fs.security.AccessDecision
import net.sergeych.lyngio.fs.security.Decision import net.sergeych.lyngio.fs.security.Decision
import kotlin.test.Test import kotlin.test.*
import kotlin.test.assertFalse
import kotlin.test.assertIs
import kotlin.test.assertTrue
class LyngConsoleModuleTest { class LyngConsoleModuleTest {
@@ -112,4 +110,63 @@ class LyngConsoleModuleTest {
assertIs<ObjIllegalOperationException>(error.errorObject) assertIs<ObjIllegalOperationException>(error.errorObject)
} }
} }
@Test
fun eventsIteratorRecoversAfterSourceFailure() = runBlocking {
val scope = newScope()
var eventsCalls = 0
val console = object : LyngConsole {
override val isSupported: Boolean = true
override suspend fun isTty(): Boolean = true
override suspend fun geometry(): ConsoleGeometry? = null
override suspend fun ansiLevel(): ConsoleAnsiLevel = ConsoleAnsiLevel.NONE
override suspend fun write(text: String) {}
override suspend fun flush() {}
override fun events(): ConsoleEventSource {
eventsCalls += 1
val callNo = eventsCalls
return object : ConsoleEventSource {
private var emitted = false
override suspend fun nextEvent(timeoutMs: Long): ConsoleEvent? {
if (emitted) return null
emitted = true
return when (callNo) {
1 -> throw IllegalStateException("synthetic source failure")
else -> ConsoleEvent.KeyDown(key = "x")
}
}
override suspend fun close() {}
}
}
override suspend fun setRawMode(enabled: Boolean): Boolean = enabled
}
assertTrue(createConsoleModule(PermitAllConsoleAccessPolicy, scope.importManager, console))
val result = scope.eval(
"""
import lyng.io.console
val it = Console.events().iterator()
assert(it.hasNext())
val ev = it.next()
assert(ev is ConsoleKeyEvent)
assert((ev as ConsoleKeyEvent).key == "x")
true
""".trimIndent()
)
assertIs<ObjBool>(result)
assertTrue(result.value)
assertEquals(2, eventsCalls)
}
} }
@@ -0,0 +1,163 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.jdbc
import kotlinx.coroutines.test.runTest
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Script
import net.sergeych.lyng.Source
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjExternCallable
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjMap
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.requireScope
import kotlin.test.Test
import kotlin.test.assertEquals
class LyngJdbcModuleTest {
@Test
fun testTypedOpenH2ExecutesQueriesAndGeneratedKeys() = runTest {
val scope = Script.newScope()
createJdbcModule(scope.importManager)
val jdbcModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.jdbc")
val db = jdbcModule.callFn("openH2", ObjString("mem:typed_h2_${System.nanoTime()};DB_CLOSE_DELAY=-1"))
val insertedId = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table person(id bigint auto_increment primary key, name varchar(120) not null)")
)
val result = tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into person(name) values(?)"),
ObjString("Ada")
)
val rows = result.invokeInstanceMethod(requireScope(), "getGeneratedKeys")
.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjInt.Zero)
}
) as ObjInt
assertEquals(1L, insertedId.value)
}
@Test
fun testGenericOpenDatabaseUsesJdbcAndH2AliasProviders() = runTest {
val scope = Script.newScope()
createJdbcModule(scope.importManager)
scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.jdbc")
val dbModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
val genericJdbcDb = dbModule.callFn(
"openDatabase",
ObjString("jdbc:h2:mem:generic_jdbc_${System.nanoTime()};DB_CLOSE_DELAY=-1"),
emptyMapObj()
)
val h2AliasDb = dbModule.callFn(
"openDatabase",
ObjString("h2:mem:generic_alias_${System.nanoTime()};DB_CLOSE_DELAY=-1"),
emptyMapObj()
)
assertEquals(42L, scalarSelect(scope, genericJdbcDb, "select 42 as answer"))
assertEquals(7L, scalarSelect(scope, h2AliasDb, "select 7 as answer"))
}
@Test
fun testImportedJdbcOpenersPreserveDeclaredReturnTypesForInference() = runTest {
val scope = Script.newScope()
createJdbcModule(scope.importManager)
val code = """
import lyng.io.db
import lyng.io.db.jdbc
val h2db = openH2("mem:inference_demo;DB_CLOSE_DELAY=-1")
h2db.transaction { 1 }
val jdbcDb = openJdbc("jdbc:h2:mem:inference_demo_2;DB_CLOSE_DELAY=-1")
jdbcDb.transaction { 2 }
val genericDb = openDatabase("jdbc:h2:mem:inference_demo_3;DB_CLOSE_DELAY=-1", Map())
genericDb.transaction { 3 }
""".trimIndent()
val result = Compiler.compile(Source("<jdbc-inference>", code), scope.importManager).execute(scope) as ObjInt
assertEquals(3L, result.value)
}
@Test
fun testMaterializedRowsListCanBeReturnedFromTransaction() = runTest {
val scope = Script.newScope()
createJdbcModule(scope.importManager)
val jdbcModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.jdbc")
val db = jdbcModule.callFn("openH2", ObjString("mem:rows_return_${System.nanoTime()};DB_CLOSE_DELAY=-1"))
val rows = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table person(id bigint auto_increment primary key, name varchar(120) not null)")
)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into person(name) values(?)"), ObjString("Ada"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into person(name) values(?)"), ObjString("Linus"))
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select name from person order by id"))
.invokeInstanceMethod(requireScope(), "toList")
}
)
assertEquals("Ada", (rows.getAt(scope, ObjInt.Zero).getAt(scope, ObjString("name")) as ObjString).value)
assertEquals("Linus", (rows.getAt(scope, ObjInt.of(1)).getAt(scope, ObjString("name")) as ObjString).value)
}
private suspend fun scalarSelect(scope: net.sergeych.lyng.Scope, db: Obj, sql: String): Long {
val result = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
val rows = tx.invokeInstanceMethod(requireScope(), "select", ObjString(sql))
.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjString("answer"))
}
) as ObjInt
return result.value
}
private suspend fun net.sergeych.lyng.ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
private fun emptyMapObj(): Obj = ObjMap()
}
@@ -0,0 +1,123 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.jdbc
import kotlinx.coroutines.test.runTest
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Script
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjExternCallable
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjMap
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.requireScope
import org.testcontainers.containers.PostgreSQLContainer
import kotlin.test.Test
import kotlin.test.assertEquals
class LyngJdbcPostgresContainerTest {
@Test
fun testOpenPostgresAgainstContainer() = runTest {
PostgreSQLContainer("postgres:16-alpine").use { postgres ->
postgres.start()
val scope = Script.newScope()
createJdbcModule(scope.importManager)
val jdbcModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.jdbc")
val db = jdbcModule.callFn(
"openPostgres",
ObjString(postgres.jdbcUrl),
ObjString(postgres.username),
ObjString(postgres.password)
)
val count = rowCount(scope, db, "people", "Ada", "Linus")
assertEquals(2L, count)
}
}
@Test
fun testGenericPostgresAliasAgainstContainer() = runTest {
PostgreSQLContainer("postgres:16-alpine").use { postgres ->
postgres.start()
val scope = Script.newScope()
createJdbcModule(scope.importManager)
scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.jdbc")
val dbModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
val aliasUrl = postgres.jdbcUrl.removePrefix("jdbc:postgresql:")
val db = dbModule.callFn(
"openDatabase",
ObjString("postgres:$aliasUrl"),
mapOfStrings(
"user" to postgres.username,
"password" to postgres.password
)
)
val count = rowCount(scope, db, "pets", "Milo", "Otis", "Pixel")
assertEquals(3L, count)
}
}
private suspend fun rowCount(scope: net.sergeych.lyng.Scope, db: Obj, table: String, vararg names: String): Long {
val result = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table $table(id bigserial primary key, name text not null)")
)
for (name in names) {
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into $table(name) values(?)"),
ObjString(name)
)
}
val rows = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("select count(*) as count from $table")
).invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjString("count"))
}
) as ObjInt
return result.value
}
private suspend fun mapOfStrings(vararg entries: Pair<String, String>): Obj {
val map = ObjMap()
for ((key, value) in entries) {
map.map[ObjString(key)] = ObjString(value)
}
return map
}
private suspend fun net.sergeych.lyng.ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
}
@@ -0,0 +1,949 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import kotlinx.coroutines.test.runTest
import kotlinx.datetime.TimeZone
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.Script
import net.sergeych.lyng.Source
import net.sergeych.lyng.obj.*
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.requireScope
import java.nio.file.Files
import kotlin.io.path.deleteIfExists
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
import kotlin.test.assertTrue
import kotlin.time.Instant
class LyngSqliteModuleTest {
@Test
fun testTypedOpenSqliteExecutesQueriesAndGeneratedKeys() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val db = sqliteModule.callFn("openSqlite", ObjString(":memory:"))
val insertedId = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table person(id integer primary key autoincrement, name text not null)"))
val result = tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into person(name) values(?)"),
ObjString("John Doe")
)
val generatedKeys = result.invokeInstanceMethod(requireScope(), "getGeneratedKeys")
val rows = generatedKeys.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjInt.Zero)
}
) as ObjInt
assertEquals(1L, insertedId.value)
}
@Test
fun testGenericOpenDatabaseUsesRegisteredSqliteProvider() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val dbModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
val db = dbModule.callFn("openDatabase", ObjString("sqlite::memory:"), emptyMapObj())
val count = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table items(id integer primary key autoincrement, name text not null)"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("alpha"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("beta"))
val resultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select count(*) as count from items"))
val rows = resultSet.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjString("count"))
}
) as ObjInt
assertEquals(2L, count.value)
}
@Test
fun testImportedDatabaseOpenersPreserveDeclaredReturnTypesForInference() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val code = """
import lyng.io.db
import lyng.io.db.sqlite
val typedDb = openSqlite(":memory:")
typedDb.transaction { 1 }
val genericDb = openDatabase("sqlite::memory:", Map())
genericDb.transaction { 2 }
""".trimIndent()
Compiler.compile(Source("<sqlite-inference>", code), scope.importManager).execute(scope)
}
@Test
fun testTransactionLambdaParameterTypeIsInferredFromDatabaseSignature() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val code = """
import lyng.io.db
import lyng.io.db.sqlite
import lyng.time
val db = openSqlite(":memory:")
db.transaction { tx ->
tx.execute("create table item(id integer primary key autoincrement, name text not null, due_date date not null)")
tx.execute("insert into item(name, due_date) values(?, ?)", "outer", Date(2026, 4, 16))
tx.transaction { inner ->
inner.execute("insert into item(name, due_date) values(?, ?)", "inner", Date(2026, 4, 17))
1
}
2
}
""".trimIndent()
val result = Compiler.compile(Source("<sqlite-lambda-inference>", code), scope.importManager).execute(scope) as ObjInt
assertEquals(2L, result.value)
}
@Test
fun testNestedTransactionRollbackUsesSavepoint() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val db = sqliteModule.callFn("openSqlite", ObjString(":memory:"))
val count = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table items(id integer primary key autoincrement, name text not null)"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("outer"))
try {
tx.invokeInstanceMethod(
requireScope(),
"transaction",
ObjExternCallable.fromBridge {
val inner = requiredArg<Obj>(0)
inner.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("inner"))
throw IllegalStateException("rollback nested")
}
)
} catch (_: IllegalStateException) {
}
val resultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select count(*) as count from items"))
val rows = resultSet.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjString("count"))
}
) as ObjInt
assertEquals(1L, count.value)
}
@Test
fun testRollbackExceptionRollsBackAndPropagates() = runTest {
val scope = Script.newScope()
withTempDb(scope) { db ->
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table items(id integer primary key autoincrement, name text not null)")
)
}
)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into items(name) values(?)"),
ObjString("rolled-back")
)
rollbackException(requireScope(), "stop here").raiseAsExecutionError(requireScope())
}
)
}
assertEquals("RollbackException", error.errorObject.objClass.className)
val count = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
val resultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select count(*) as count from items"))
rowsOf(requireScope(), resultSet)[0].getAt(requireScope(), ObjString("count"))
}
) as ObjInt
assertEquals(0L, count.value)
}
}
@Test
fun testResultSetFailsAfterTransactionEnds() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val db = sqliteModule.callFn("openSqlite", ObjString(":memory:"))
var leakedResultSet: Obj = ObjNull
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table items(id integer primary key autoincrement, name text not null)"))
leakedResultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select 42 as answer"))
ObjNull
}
)
val error = assertFailsWith<ExecutionError> {
leakedResultSet.invokeInstanceMethod(scope, "size")
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("transaction is active"), error.errorMessage)
}
@Test
fun testMaterializedRowSurvivesAfterTransactionEnds() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
var leakedRow: Obj = ObjNull
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
leakedRow = rowsOf(requireScope(), tx.invokeInstanceMethod(requireScope(), "select", ObjString("select 42 as answer")))[0]
ObjNull
}
)
val answer = leakedRow.getAt(scope, ObjString("answer")) as ObjInt
assertEquals(42L, answer.value)
}
@Test
fun testMaterializedRowsListCanBeReturnedFromTransaction() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val rows = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table items(id integer primary key autoincrement, name text not null)")
)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("alpha"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("beta"))
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select name from items order by id"))
.invokeInstanceMethod(requireScope(), "toList")
}
)
assertEquals("alpha", stringValue(scope, rows.getAt(scope, ObjInt.Zero).getAt(scope, ObjString("name"))))
assertEquals("beta", stringValue(scope, rows.getAt(scope, ObjInt.of(1)).getAt(scope, ObjString("name"))))
}
@Test
fun testInvalidSqliteUrlFailsWithIllegalArgument() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val dbModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
val error = assertFailsWith<ExecutionError> {
dbModule.callFn("openDatabase", ObjString("sqlite://bad"), emptyMapObj())
}
assertEquals("IllegalArgumentException", error.errorObject.objClass.className)
}
@Test
fun testConstraintViolationIsMappedToSqlConstraintException() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table person(id integer primary key autoincrement, email text unique not null)")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into person(email) values(?)"),
ObjString("a@example.com")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into person(email) values(?)"),
ObjString("a@example.com")
)
}
)
}
assertEquals("SqlConstraintException", error.errorObject.objClass.className)
}
@Test
fun testAmbiguousColumnNameAccessFailsWithSqlUsageException() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("select 1 as value, 2 as value")
)
val row = rowsOf(requireScope(), resultSet)[0]
row.getAt(requireScope(), ObjString("value"))
}
)
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("Ambiguous SQL result column"), error.errorMessage)
}
@Test
fun testExecuteRejectsReturningButSelectSupportsIt() = runTest {
val scope = Script.newScope()
withTempDb(scope) { db ->
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table item(id integer primary key autoincrement, name text not null)")
)
}
)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into item(name) values(?) returning id"),
ObjString("bad")
)
}
)
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
val insertedId = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("insert into item(name) values(?) returning id"),
ObjString("good")
)
val row = rowsOf(requireScope(), resultSet)[0]
row.getAt(requireScope(), ObjString("id"))
}
) as ObjInt
assertEquals(1L, insertedId.value)
}
}
@Test
fun testColumnMetadataAndTypedValueConversion() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val summary = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString(
"create table events(" +
"amount NUMERIC not null, " +
"happened TIMESTAMPTZ not null, " +
"scheduled TIMESTAMP not null, " +
"note TEXT not null, " +
"payload BLOB not null)"
)
)
val decimal = decimalOf(requireScope(), "12.50")
val happened = ObjInstant(Instant.parse("2024-05-06T07:08:09Z"))
val scheduled = ObjDateTime(Instant.parse("2024-05-06T10:11:12Z"), TimeZone.UTC)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into events(amount, happened, scheduled, note, payload) values(?, ?, ?, ?, ?)"),
decimal,
happened,
scheduled,
ObjString("hello"),
ObjBuffer(byteArrayOf(1, 2, 3).toUByteArray())
)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("select amount, happened, scheduled, note, payload from events")
)
val columns = field(requireScope(), resultSet, "columns")
val firstColumn = columns.getAt(requireScope(), ObjInt.Zero)
val row = rowsOf(requireScope(), resultSet)[0]
ObjString(
listOf(
stringValue(requireScope(), field(requireScope(), firstColumn, "name")),
enumName(requireScope(), field(requireScope(), firstColumn, "sqlType")),
stringValue(requireScope(), field(requireScope(), firstColumn, "nativeType")),
row.getAt(requireScope(), ObjString("amount")).objClass.className,
row.getAt(requireScope(), ObjString("happened")).objClass.className,
row.getAt(requireScope(), ObjString("scheduled")).objClass.className,
stringValue(requireScope(), row.getAt(requireScope(), ObjString("note"))),
row.getAt(requireScope(), ObjString("payload")).objClass.className,
).joinToString("|")
)
}
) as ObjString
assertEquals(
"amount|Decimal|NUMERIC|Decimal|Instant|DateTime|hello|Buffer",
summary.value
)
}
@Test
fun testDateAndBooleanConversionRules() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val summary = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString(
"create table sample(" +
"flag BOOL not null, " +
"day DATE not null, " +
"clock TIME not null)"
)
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(flag, day, clock) values(?, ?, ?)"),
ObjBool(true),
dateOf(requireScope(), "2026-04-15"),
ObjString("12:34:56")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(flag, day, clock) values(?, ?, ?)"),
ObjString("t"),
ObjString("2026-04-16"),
ObjString("23:59:59")
)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("select flag, day, clock from sample order by day")
)
val rows = rowsOf(requireScope(), resultSet)
ObjString(
listOf(
rows[0].getAt(requireScope(), ObjString("flag")).objClass.className,
rows[0].getAt(requireScope(), ObjString("day")).objClass.className,
stringValue(requireScope(), rows[0].getAt(requireScope(), ObjString("clock"))),
rows[1].getAt(requireScope(), ObjString("flag")).objClass.className,
).joinToString("|")
)
}
) as ObjString
assertEquals("Bool|Date|12:34:56|Bool", summary.value)
}
@Test
fun testUnsupportedParameterTypeFailsWithSqlUsageException() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table sample(value text not null)")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(value) values(?)"),
emptyMapObj()
)
}
)
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("Unsupported SQLite parameter type"), error.errorMessage)
}
@Test
fun testTimestampAndDatetimeRejectTimezoneBearingText() = runTest {
val scope = Script.newScope()
withTempDb(scope) { db ->
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table sample(ts TIMESTAMP not null, dt DATETIME not null)")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(ts, dt) values(?, ?)"),
ObjString("2024-05-06T07:08:09Z"),
ObjString("2024-05-06T10:11:12+03:00")
)
}
)
val timestampError = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select ts from sample"))
}
)
}
assertEquals("SqlExecutionException", timestampError.errorObject.objClass.className)
assertTrue(timestampError.errorMessage.contains("must not contain a timezone offset"), timestampError.errorMessage)
val datetimeError = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select dt from sample"))
}
)
}
assertEquals("SqlExecutionException", datetimeError.errorObject.objClass.className)
assertTrue(datetimeError.errorMessage.contains("must not contain a timezone offset"), datetimeError.errorMessage)
}
}
@Test
fun testReadOnlyOpenPreventsWrites() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val tempFile = Files.createTempFile("lyng-sqlite-", ".db")
try {
val writableDb = sqliteModule.callFn("openSqlite", ObjString(tempFile.toString()))
writableDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table item(id integer primary key autoincrement, name text not null)")
)
}
)
val readOnlyDb = sqliteModule.callFn(
"openSqlite",
ObjString(tempFile.toString()),
net.sergeych.lyng.obj.ObjTrue,
net.sergeych.lyng.obj.ObjFalse
)
val error = assertFailsWith<ExecutionError> {
readOnlyDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into item(name) values(?)"),
ObjString("blocked")
)
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
} finally {
tempFile.deleteIfExists()
}
}
@Test
fun testMissingFileWithCreateIfMissingFalseFailsWithDatabaseException() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val missingDir = Files.createTempDirectory("lyng-sqlite-missing-dir-")
missingDir.deleteIfExists()
val missingFile = missingDir.resolve("missing.db")
try {
val db = sqliteModule.callFn(
"openSqlite",
ObjString(missingFile.toString()),
net.sergeych.lyng.obj.ObjFalse,
net.sergeych.lyng.obj.ObjFalse
)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge { ObjNull }
)
}
assertEquals("DatabaseException", error.errorObject.objClass.className)
} finally {
missingFile.deleteIfExists()
missingDir.deleteIfExists()
}
}
@Test
fun testGenericOpenDatabaseReadOnlyOptionMatchesTypedHelper() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val dbModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
val tempFile = Files.createTempFile("lyng-sqlite-generic-", ".db")
try {
val writableDb = dbModule.callFn("openDatabase", ObjString("sqlite:${tempFile}"), sqliteOptions(scope))
writableDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table item(id integer primary key autoincrement, name text not null)")
)
}
)
val readOnlyDb = dbModule.callFn(
"openDatabase",
ObjString("sqlite:${tempFile}"),
sqliteOptions(
scope,
"readOnly" to net.sergeych.lyng.obj.ObjTrue,
"createIfMissing" to net.sergeych.lyng.obj.ObjFalse
)
)
val error = assertFailsWith<ExecutionError> {
readOnlyDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into item(name) values(?)"),
ObjString("blocked")
)
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
} finally {
tempFile.deleteIfExists()
}
}
@Test
fun testForeignKeysOptionControlsConstraintEnforcement() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val tempFile = Files.createTempFile("lyng-sqlite-fk-", ".db")
try {
val dbNoFk = sqliteModule.callFn(
"openSqlite",
ObjString(tempFile.toString()),
net.sergeych.lyng.obj.ObjFalse,
net.sergeych.lyng.obj.ObjTrue,
net.sergeych.lyng.obj.ObjFalse
)
dbNoFk.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table parent(id integer primary key)"))
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table child(parent_id integer not null references parent(id))")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into child(parent_id) values(?)"),
ObjInt.One
)
}
)
val dbWithFk = sqliteModule.callFn("openSqlite", ObjString(tempFile.toString()))
val error = assertFailsWith<ExecutionError> {
dbWithFk.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into child(parent_id) values(?)"),
ObjInt.of(2)
)
}
)
}
assertEquals("SqlConstraintException", error.errorObject.objClass.className)
} finally {
tempFile.deleteIfExists()
}
}
@Test
fun testCommitFailureBecomesPrimaryAfterNormalCompletion() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("rollback"))
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
}
@Test
fun testUserExceptionStaysPrimaryWhenRollbackFails() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<IllegalStateException> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("rollback"))
throw IllegalStateException("boom")
}
)
}
assertEquals("boom", error.message)
}
@Test
fun testRollbackFailureBecomesPrimaryAfterRollbackException() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("rollback"))
rollbackException(requireScope(), "rollback requested").raiseAsExecutionError(requireScope())
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
}
private suspend fun ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
private suspend fun sqliteOptions(scope: Scope, vararg entries: Pair<String, Obj>): ObjMap {
val result = ObjMap()
for ((key, value) in entries) {
result.putAt(scope, ObjString(key), value)
}
return result
}
private suspend fun openMemoryDb(scope: Scope): Obj {
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
return sqliteModule.callFn("openSqlite", ObjString(":memory:"))
}
private suspend fun withTempDb(scope: Scope, block: suspend (Obj) -> Unit) {
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val tempFile = Files.createTempFile("lyng-sqlite-", ".db")
try {
val db = sqliteModule.callFn("openSqlite", ObjString(tempFile.toString()))
block(db)
} finally {
tempFile.deleteIfExists()
}
}
private suspend fun field(scope: Scope, obj: Obj, name: String): Obj =
obj.readField(scope, name).value
private suspend fun rowsOf(scope: Scope, resultSet: Obj): List<Obj> {
val rows = resultSet.invokeInstanceMethod(scope, "toList")
val size = (field(scope, rows, "size") as ObjInt).value.toInt()
return (0 until size).map { index -> rows.getAt(scope, ObjInt.of(index.toLong())) }
}
private suspend fun stringValue(scope: Scope, obj: Obj): String =
(obj as? ObjString ?: obj.toString(scope)) .value
private suspend fun enumName(scope: Scope, obj: Obj): String =
stringValue(scope, field(scope, obj, "name"))
private suspend fun decimalOf(scope: Scope, value: String): Obj {
val decimalModule = scope.currentImportProvider.createModuleScope(scope.pos, "lyng.decimal")
val decimalClass = decimalModule.requireClass("Decimal")
return decimalClass.invokeInstanceMethod(scope, "fromString", ObjString(value))
}
private suspend fun rollbackException(scope: Scope, message: String): net.sergeych.lyng.obj.ObjException {
val dbModule = scope.currentImportProvider.createModuleScope(scope.pos, "lyng.io.db")
val rollbackClass = dbModule.requireClass("RollbackException")
return rollbackClass.invoke(scope, ObjNull, ObjString(message)) as net.sergeych.lyng.obj.ObjException
}
private suspend fun dateOf(scope: Scope, value: String): Obj {
val timeModule = scope.currentImportProvider.createModuleScope(scope.pos, "lyng.time")
val dateClass = timeModule.requireClass("Date")
return dateClass.invoke(scope, ObjNull, ObjString(value))
}
private fun emptyMapObj(): Obj = ObjMap()
}
@@ -0,0 +1,59 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.console
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertTrue
class JvmConsoleKeyDecoderTest {
private fun decode(vararg bytes: Int): ConsoleEvent.KeyDown {
var index = 1
return JvmConsoleKeyDecoder.decode(bytes[0]) { _ ->
if (index >= bytes.size) null else bytes[index++]
}
}
@Test
fun decodesArrowLeft() {
val ev = decode(27, '['.code, 'D'.code)
assertEquals("ArrowLeft", ev.key)
assertFalse(ev.ctrl)
assertFalse(ev.alt)
assertFalse(ev.shift)
}
@Test
fun decodesCtrlArrowRightModifier() {
val ev = decode(27, '['.code, '1'.code, ';'.code, '5'.code, 'C'.code)
assertEquals("ArrowRight", ev.key)
assertTrue(ev.ctrl)
assertFalse(ev.alt)
assertFalse(ev.shift)
}
@Test
fun decodesAltCharacter() {
val ev = decode(27, 'x'.code)
assertEquals("x", ev.key)
assertTrue(ev.alt)
assertFalse(ev.ctrl)
}
}
@@ -22,11 +22,14 @@ import kotlinx.coroutines.delay
import kotlinx.coroutines.sync.Mutex import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock import kotlinx.coroutines.sync.withLock
import platform.posix.* import platform.posix.*
import kotlin.time.Duration.Companion.milliseconds
import kotlin.time.TimeSource import kotlin.time.TimeSource
internal actual fun getNativeSystemConsole(): LyngConsole = LinuxPosixLyngConsole internal actual fun getNativeSystemConsole(): LyngConsole = LinuxPosixLyngConsole
private const val RAW_IDLE_POLL_MS = 10L
private const val NON_RAW_IDLE_POLL_MS = 25L
private const val ESCAPE_FOLLOWUP_TIMEOUT_MS = 25L
internal object LinuxConsoleKeyDecoder { internal object LinuxConsoleKeyDecoder {
fun decode(firstCode: Int, nextCode: (Long) -> Int?): ConsoleEvent.KeyDown { fun decode(firstCode: Int, nextCode: (Long) -> Int?): ConsoleEvent.KeyDown {
if (firstCode == 27) { if (firstCode == 27) {
@@ -132,6 +135,7 @@ object LinuxPosixLyngConsole : LyngConsole {
} }
} }
consoleFlowDebug("linux-events: source created")
return object : ConsoleEventSource { return object : ConsoleEventSource {
var closed = false var closed = false
var lastGeometry: ConsoleGeometry? = null var lastGeometry: ConsoleGeometry? = null
@@ -147,23 +151,27 @@ object LinuxPosixLyngConsole : LyngConsole {
} }
val rawRequested = stateMutex.withLock { rawModeRequested } val rawRequested = stateMutex.withLock { rawModeRequested }
val pollSliceMs = if (timeoutMs <= 0L) 250L else minOf(250L, timeoutMs)
if (rawRequested) { if (rawRequested) {
val ev = readKeyEvent(pollSliceMs) val ev = readKeyEventNonBlocking()
if (ev != null) return ev if (ev != null) return ev
} else {
delay(25)
} }
if (timeoutMs > 0L && started.elapsedNow() >= timeoutMs.milliseconds) { val elapsedMs = started.elapsedNow().inWholeMilliseconds
if (timeoutMs > 0L && elapsedMs >= timeoutMs) {
return null return null
} }
val remainingMs = if (timeoutMs > 0L) timeoutMs - elapsedMs else Long.MAX_VALUE
val idleMs = if (rawRequested) RAW_IDLE_POLL_MS else NON_RAW_IDLE_POLL_MS
val sleepMs = if (timeoutMs > 0L) minOf(idleMs, remainingMs) else idleMs
if (sleepMs > 0L) delay(sleepMs)
} }
return null return null
} }
override suspend fun close() { override suspend fun close() {
closed = true closed = true
consoleFlowDebug("linux-events: source closed")
} }
} }
} }
@@ -183,12 +191,11 @@ object LinuxPosixLyngConsole : LyngConsole {
} }
savedAttrsBlob = saved savedAttrsBlob = saved
attrs.c_lflag = attrs.c_lflag and ICANON.convert<UInt>().inv() and ECHO.convert<UInt>().inv() configureRawInput(attrs)
attrs.c_iflag = attrs.c_iflag and IXON.convert<UInt>().inv() and ISTRIP.convert<UInt>().inv()
attrs.c_oflag = attrs.c_oflag and OPOST.convert<UInt>().inv()
if (tcsetattr(STDIN_FILENO, TCSANOW, attrs.ptr) != 0) return@withLock false if (tcsetattr(STDIN_FILENO, TCSANOW, attrs.ptr) != 0) return@withLock false
} }
rawModeRequested = true rawModeRequested = true
consoleFlowDebug("linux-events: setRawMode(true): enabled")
true true
} else { } else {
val hadRaw = rawModeRequested val hadRaw = rawModeRequested
@@ -203,6 +210,7 @@ object LinuxPosixLyngConsole : LyngConsole {
tcsetattr(STDIN_FILENO, TCSANOW, attrs.ptr) tcsetattr(STDIN_FILENO, TCSANOW, attrs.ptr)
} }
} }
consoleFlowDebug("linux-events: setRawMode(false): disabled hadRaw=$hadRaw")
hadRaw hadRaw
} }
} }
@@ -225,19 +233,55 @@ object LinuxPosixLyngConsole : LyngConsole {
val ready = poll(pfd.ptr, 1.convert(), timeoutMs.toInt()) val ready = poll(pfd.ptr, 1.convert(), timeoutMs.toInt())
if (ready <= 0) return null if (ready <= 0) return null
readReadyByte()
}
private fun readReadyByte(): Int? {
val buf = ByteArray(1) val buf = ByteArray(1)
val count = buf.usePinned { pinned -> val count = buf.usePinned { pinned ->
read(STDIN_FILENO, pinned.addressOf(0), 1.convert()) read(STDIN_FILENO, pinned.addressOf(0), 1.convert())
} }
if (count <= 0) return null if (count <= 0) {
if (count < 0) {
consoleFlowDebug("linux-events: stdin read returned $count errno=$errno")
}
return null
}
val b = buf[0].toInt() val b = buf[0].toInt()
if (b < 0) b + 256 else b return if (b < 0) b + 256 else b
} }
private fun readKeyEvent(timeoutMs: Long): ConsoleEvent.KeyDown? { private fun readByteNow(): Int? = memScoped {
val first = readByte(timeoutMs) ?: return null val pfd = alloc<pollfd>()
pfd.fd = STDIN_FILENO
pfd.events = POLLIN.convert()
pfd.revents = 0
val ready = poll(pfd.ptr, 1.convert(), 0)
if (ready <= 0) return null
readReadyByte()
}
private fun readKeyEventNonBlocking(): ConsoleEvent.KeyDown? {
val first = readByteNow() ?: return null
return LinuxConsoleKeyDecoder.decode(first) { timeout -> return LinuxConsoleKeyDecoder.decode(first) { timeout ->
readByte(timeout) readByte(minOf(timeout, ESCAPE_FOLLOWUP_TIMEOUT_MS))
} }
} }
} }
@OptIn(ExperimentalForeignApi::class)
internal fun configureRawInput(attrs: termios) {
attrs.c_iflag = attrs.c_iflag and BRKINT.convert<UInt>().inv()
attrs.c_iflag = attrs.c_iflag and ICRNL.convert<UInt>().inv()
attrs.c_iflag = attrs.c_iflag and INPCK.convert<UInt>().inv()
attrs.c_iflag = attrs.c_iflag and ISTRIP.convert<UInt>().inv()
attrs.c_iflag = attrs.c_iflag and IXON.convert<UInt>().inv()
attrs.c_oflag = attrs.c_oflag and OPOST.convert<UInt>().inv()
attrs.c_cflag = attrs.c_cflag or CS8.convert<UInt>()
attrs.c_lflag = attrs.c_lflag and ECHO.convert<UInt>().inv()
attrs.c_lflag = attrs.c_lflag and ICANON.convert<UInt>().inv()
attrs.c_lflag = attrs.c_lflag and IEXTEN.convert<UInt>().inv()
attrs.c_lflag = attrs.c_lflag and ISIG.convert<UInt>().inv()
attrs.c_cc[VMIN] = 0u
attrs.c_cc[VTIME] = 1u
}
@@ -1,8 +1,14 @@
package net.sergeych.lyngio.net package net.sergeych.lyngio.net
actual fun getSystemNetEngine(): LyngNetEngine = createNativeKtorNetEngine( private val systemNetEngine: LyngNetEngine = createNativeKtorNetEngine(
isSupported = true, isSupported = true,
isTcpAvailable = true, isTcpAvailable = true,
isTcpServerAvailable = true, isTcpServerAvailable = true,
isUdpAvailable = true, isUdpAvailable = true,
) )
actual fun getSystemNetEngine(): LyngNetEngine = systemNetEngine
actual fun shutdownSystemNetEngine() {
shutdownNativeKtorNetEngine(systemNetEngine)
}
@@ -2,4 +2,5 @@ package net.sergeych.lyngio.ws
import io.ktor.client.engine.curl.Curl import io.ktor.client.engine.curl.Curl
actual fun getSystemWsEngine(): LyngWsEngine = createKtorWsEngine(Curl) actual fun getSystemWsEngine(): LyngWsEngine =
createSocketWsEngine(secureFallback = createKtorWsEngine(Curl, shareClient = false))
@@ -0,0 +1,807 @@
/*
* Copyright 2026 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.io.db.sqlite
import kotlinx.coroutines.test.runTest
import kotlinx.datetime.TimeZone
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.Script
import net.sergeych.lyng.obj.Obj
import net.sergeych.lyng.obj.ObjBool
import net.sergeych.lyng.obj.ObjBuffer
import net.sergeych.lyng.obj.ObjDateTime
import net.sergeych.lyng.obj.ObjExternCallable
import net.sergeych.lyng.obj.ObjInt
import net.sergeych.lyng.obj.ObjInstant
import net.sergeych.lyng.obj.ObjMap
import net.sergeych.lyng.obj.ObjNull
import net.sergeych.lyng.obj.ObjString
import net.sergeych.lyng.obj.raiseAsExecutionError
import net.sergeych.lyng.obj.requiredArg
import net.sergeych.lyng.requireScope
import okio.FileSystem
import okio.Path
import okio.Path.Companion.toPath
import kotlin.random.Random
import kotlin.time.Instant
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
import kotlin.test.assertTrue
class LyngSqliteModuleNativeTest {
@Test
fun testTypedOpenSqliteExecutesQueriesAndGeneratedKeys() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val db = sqliteModule.callFn("openSqlite", ObjString(":memory:"))
val insertedId = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table person(id integer primary key autoincrement, name text not null)"))
val result = tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into person(name) values(?)"),
ObjString("John Doe")
)
val generatedKeys = result.invokeInstanceMethod(requireScope(), "getGeneratedKeys")
val rows = generatedKeys.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjInt.Zero)
}
) as ObjInt
assertEquals(1L, insertedId.value)
}
@Test
fun testNestedTransactionRollbackUsesSavepoint() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val count = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table items(id integer primary key autoincrement, name text not null)"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("outer"))
try {
tx.invokeInstanceMethod(
requireScope(),
"transaction",
ObjExternCallable.fromBridge {
val inner = requiredArg<Obj>(0)
inner.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("inner"))
throw IllegalStateException("rollback nested")
}
)
} catch (_: IllegalStateException) {
}
val resultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select count(*) as count from items"))
val rows = resultSet.invokeInstanceMethod(requireScope(), "toList")
rows.getAt(requireScope(), ObjInt.Zero).getAt(requireScope(), ObjString("count"))
}
) as ObjInt
assertEquals(1L, count.value)
}
@Test
fun testRollbackExceptionRollsBackAndPropagates() = runTest {
val scope = Script.newScope()
withTempDb(scope) { db ->
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table items(id integer primary key autoincrement, name text not null)")
)
}
)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into items(name) values(?)"),
ObjString("rolled-back")
)
rollbackException(requireScope(), "stop here").raiseAsExecutionError(requireScope())
}
)
}
assertEquals("RollbackException", error.errorObject.objClass.className)
val count = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
val resultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select count(*) as count from items"))
rowsOf(requireScope(), resultSet)[0].getAt(requireScope(), ObjString("count"))
}
) as ObjInt
assertEquals(0L, count.value)
}
}
@Test
fun testResultSetFailsAfterTransactionEnds() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
var leakedResultSet: Obj = ObjNull
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table items(id integer primary key autoincrement, name text not null)"))
leakedResultSet = tx.invokeInstanceMethod(requireScope(), "select", ObjString("select 42 as answer"))
ObjNull
}
)
val error = assertFailsWith<ExecutionError> {
leakedResultSet.invokeInstanceMethod(scope, "size")
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("transaction is active"), error.errorMessage)
}
@Test
fun testMaterializedRowSurvivesAfterTransactionEnds() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
var leakedRow: Obj = ObjNull
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
leakedRow = rowsOf(requireScope(), tx.invokeInstanceMethod(requireScope(), "select", ObjString("select 42 as answer")))[0]
ObjNull
}
)
val answer = leakedRow.getAt(scope, ObjString("answer")) as ObjInt
assertEquals(42L, answer.value)
}
@Test
fun testMaterializedRowsListCanBeReturnedFromTransaction() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val rows = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table items(id integer primary key autoincrement, name text not null)")
)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("alpha"))
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("insert into items(name) values(?)"), ObjString("beta"))
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select name from items order by id"))
.invokeInstanceMethod(requireScope(), "toList")
}
)
assertEquals("alpha", stringValue(scope, rows.getAt(scope, ObjInt.Zero).getAt(scope, ObjString("name"))))
assertEquals("beta", stringValue(scope, rows.getAt(scope, ObjInt.of(1)).getAt(scope, ObjString("name"))))
}
@Test
fun testExecuteRejectsReturningButSelectSupportsIt() = runTest {
val scope = Script.newScope()
withTempDb(scope) { db ->
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table item(id integer primary key autoincrement, name text not null)")
)
}
)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into item(name) values(?) returning id"),
ObjString("bad")
)
}
)
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
val insertedId = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("insert into item(name) values(?) returning id"),
ObjString("good")
)
val row = rowsOf(requireScope(), resultSet)[0]
row.getAt(requireScope(), ObjString("id"))
}
) as ObjInt
assertEquals(1L, insertedId.value)
}
}
@Test
fun testColumnMetadataAndTypedValueConversion() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val summary = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString(
"create table events(" +
"amount NUMERIC not null, " +
"happened TIMESTAMPTZ not null, " +
"scheduled TIMESTAMP not null, " +
"note TEXT not null, " +
"payload BLOB not null)"
)
)
val decimal = decimalOf(requireScope(), "12.50")
val happened = ObjInstant(Instant.parse("2024-05-06T07:08:09Z"))
val scheduled = ObjDateTime(Instant.parse("2024-05-06T10:11:12Z"), TimeZone.UTC)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into events(amount, happened, scheduled, note, payload) values(?, ?, ?, ?, ?)"),
decimal,
happened,
scheduled,
ObjString("hello"),
ObjBuffer(byteArrayOf(1, 2, 3).toUByteArray())
)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("select amount, happened, scheduled, note, payload from events")
)
val columns = field(requireScope(), resultSet, "columns")
val firstColumn = columns.getAt(requireScope(), ObjInt.Zero)
val row = rowsOf(requireScope(), resultSet)[0]
ObjString(
listOf(
stringValue(requireScope(), field(requireScope(), firstColumn, "name")),
enumName(requireScope(), field(requireScope(), firstColumn, "sqlType")),
stringValue(requireScope(), field(requireScope(), firstColumn, "nativeType")),
row.getAt(requireScope(), ObjString("amount")).objClass.className,
row.getAt(requireScope(), ObjString("happened")).objClass.className,
row.getAt(requireScope(), ObjString("scheduled")).objClass.className,
stringValue(requireScope(), row.getAt(requireScope(), ObjString("note"))),
row.getAt(requireScope(), ObjString("payload")).objClass.className,
).joinToString("|")
)
}
) as ObjString
assertEquals(
"amount|Decimal|NUMERIC|Decimal|Instant|DateTime|hello|Buffer",
summary.value
)
}
@Test
fun testDateAndBooleanConversionRules() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val summary = db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString(
"create table sample(" +
"flag BOOL not null, " +
"day DATE not null, " +
"clock TIME not null)"
)
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(flag, day, clock) values(?, ?, ?)"),
ObjBool(true),
dateOf(requireScope(), "2026-04-15"),
ObjString("12:34:56")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(flag, day, clock) values(?, ?, ?)"),
ObjString("t"),
ObjString("2026-04-16"),
ObjString("23:59:59")
)
val resultSet = tx.invokeInstanceMethod(
requireScope(),
"select",
ObjString("select flag, day, clock from sample order by day")
)
val rows = rowsOf(requireScope(), resultSet)
ObjString(
listOf(
rows[0].getAt(requireScope(), ObjString("flag")).objClass.className,
rows[0].getAt(requireScope(), ObjString("day")).objClass.className,
stringValue(requireScope(), rows[0].getAt(requireScope(), ObjString("clock"))),
rows[1].getAt(requireScope(), ObjString("flag")).objClass.className,
).joinToString("|")
)
}
) as ObjString
assertEquals("Bool|Date|12:34:56|Bool", summary.value)
}
@Test
fun testUnsupportedParameterTypeFailsWithSqlUsageException() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table sample(value text not null)")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(value) values(?)"),
emptyMapObj()
)
}
)
}
assertEquals("SqlUsageException", error.errorObject.objClass.className)
assertTrue(error.errorMessage.contains("Unsupported SQLite parameter type"), error.errorMessage)
}
@Test
fun testTimestampAndDatetimeRejectTimezoneBearingText() = runTest {
val scope = Script.newScope()
withTempDb(scope) { db ->
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table sample(ts TIMESTAMP not null, dt DATETIME not null)")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into sample(ts, dt) values(?, ?)"),
ObjString("2024-05-06T07:08:09Z"),
ObjString("2024-05-06T10:11:12+03:00")
)
}
)
val timestampError = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select ts from sample"))
}
)
}
assertEquals("SqlExecutionException", timestampError.errorObject.objClass.className)
assertTrue(timestampError.errorMessage.contains("must not contain a timezone offset"), timestampError.errorMessage)
val datetimeError = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "select", ObjString("select dt from sample"))
}
)
}
assertEquals("SqlExecutionException", datetimeError.errorObject.objClass.className)
assertTrue(datetimeError.errorMessage.contains("must not contain a timezone offset"), datetimeError.errorMessage)
}
}
@Test
fun testReadOnlyOpenPreventsWrites() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
withTempPath { tempPath ->
val writableDb = sqliteModule.callFn("openSqlite", ObjString(tempPath.toString()))
writableDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table item(id integer primary key autoincrement, name text not null)")
)
}
)
val readOnlyDb = sqliteModule.callFn(
"openSqlite",
ObjString(tempPath.toString()),
net.sergeych.lyng.obj.ObjTrue,
net.sergeych.lyng.obj.ObjFalse
)
val error = assertFailsWith<ExecutionError> {
readOnlyDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into item(name) values(?)"),
ObjString("blocked")
)
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
}
}
@Test
fun testMissingFileWithCreateIfMissingFalseFailsWithDatabaseException() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val missingDir = "/tmp/lyng-sqlite-missing-${Random.nextInt(Int.MAX_VALUE)}".toPath()
val missingFile = (missingDir.toString() + "/missing.db").toPath()
try {
FileSystem.SYSTEM.delete(missingFile, mustExist = false)
FileSystem.SYSTEM.delete(missingDir, mustExist = false)
val db = sqliteModule.callFn(
"openSqlite",
ObjString(missingFile.toString()),
net.sergeych.lyng.obj.ObjFalse,
net.sergeych.lyng.obj.ObjFalse
)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge { ObjNull }
)
}
assertEquals("DatabaseException", error.errorObject.objClass.className)
} finally {
FileSystem.SYSTEM.delete(missingFile, mustExist = false)
FileSystem.SYSTEM.delete(missingDir, mustExist = false)
}
}
@Test
fun testGenericOpenDatabaseReadOnlyOptionMatchesTypedHelper() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
val dbModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db")
withTempPath { tempPath ->
val writableDb = dbModule.callFn("openDatabase", ObjString("sqlite:${tempPath}"), sqliteOptions(scope))
writableDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table item(id integer primary key autoincrement, name text not null)")
)
}
)
val readOnlyDb = dbModule.callFn(
"openDatabase",
ObjString("sqlite:${tempPath}"),
sqliteOptions(
scope,
"readOnly" to net.sergeych.lyng.obj.ObjTrue,
"createIfMissing" to net.sergeych.lyng.obj.ObjFalse
)
)
val error = assertFailsWith<ExecutionError> {
readOnlyDb.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into item(name) values(?)"),
ObjString("blocked")
)
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
}
}
@Test
fun testForeignKeysOptionControlsConstraintEnforcement() = runTest {
val scope = Script.newScope()
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
withTempPath { tempPath ->
val dbNoFk = sqliteModule.callFn(
"openSqlite",
ObjString(tempPath.toString()),
net.sergeych.lyng.obj.ObjFalse,
net.sergeych.lyng.obj.ObjTrue,
net.sergeych.lyng.obj.ObjFalse
)
dbNoFk.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("create table parent(id integer primary key)"))
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("create table child(parent_id integer not null references parent(id))")
)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into child(parent_id) values(?)"),
ObjInt.One
)
}
)
val dbWithFk = sqliteModule.callFn("openSqlite", ObjString(tempPath.toString()))
val error = assertFailsWith<ExecutionError> {
dbWithFk.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(
requireScope(),
"execute",
ObjString("insert into child(parent_id) values(?)"),
ObjInt.of(2)
)
}
)
}
assertEquals("SqlConstraintException", error.errorObject.objClass.className)
}
}
@Test
fun testCommitFailureBecomesPrimaryAfterNormalCompletion() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("rollback"))
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
}
@Test
fun testUserExceptionStaysPrimaryWhenRollbackFails() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<IllegalStateException> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("rollback"))
throw IllegalStateException("boom")
}
)
}
assertEquals("boom", error.message)
}
@Test
fun testRollbackFailureBecomesPrimaryAfterRollbackException() = runTest {
val scope = Script.newScope()
val db = openMemoryDb(scope)
val error = assertFailsWith<ExecutionError> {
db.invokeInstanceMethod(
scope,
"transaction",
ObjExternCallable.fromBridge {
val tx = requiredArg<Obj>(0)
tx.invokeInstanceMethod(requireScope(), "execute", ObjString("rollback"))
rollbackException(requireScope(), "rollback requested").raiseAsExecutionError(requireScope())
}
)
}
assertEquals("SqlExecutionException", error.errorObject.objClass.className)
}
private suspend fun ModuleScope.callFn(name: String, vararg args: Obj): Obj {
val callee = get(name)?.value ?: error("Missing $name in module")
return callee.invoke(this, ObjNull, *args)
}
private suspend fun sqliteOptions(scope: Scope, vararg entries: Pair<String, Obj>): ObjMap {
val result = ObjMap()
for ((key, value) in entries) {
result.putAt(scope, ObjString(key), value)
}
return result
}
private suspend fun openMemoryDb(scope: Scope): Obj {
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
return sqliteModule.callFn("openSqlite", ObjString(":memory:"))
}
private suspend fun withTempDb(scope: Scope, block: suspend (Obj) -> Unit) {
createSqliteModule(scope.importManager)
val sqliteModule = scope.importManager.createModuleScope(Pos.builtIn, "lyng.io.db.sqlite")
withTempPath { tempPath ->
val db = sqliteModule.callFn("openSqlite", ObjString(tempPath.toString()))
block(db)
}
}
private suspend fun withTempPath(block: suspend (Path) -> Unit) {
val path = "/tmp/lyng-sqlite-${Random.nextInt(Int.MAX_VALUE)}.db".toPath()
try {
block(path)
} finally {
FileSystem.SYSTEM.delete(path, mustExist = false)
}
}
private suspend fun field(scope: Scope, obj: Obj, name: String): Obj =
obj.readField(scope, name).value
private suspend fun rowsOf(scope: Scope, resultSet: Obj): List<Obj> {
val rows = resultSet.invokeInstanceMethod(scope, "toList")
val size = (field(scope, rows, "size") as ObjInt).value.toInt()
return (0 until size).map { index -> rows.getAt(scope, ObjInt.of(index.toLong())) }
}
private suspend fun stringValue(scope: Scope, obj: Obj): String =
(obj as? ObjString ?: obj.toString(scope)).value
private suspend fun enumName(scope: Scope, obj: Obj): String =
stringValue(scope, field(scope, obj, "name"))
private suspend fun decimalOf(scope: Scope, value: String): Obj {
val decimalModule = scope.currentImportProvider.createModuleScope(scope.pos, "lyng.decimal")
val decimalClass = decimalModule.requireClass("Decimal")
return decimalClass.invokeInstanceMethod(scope, "fromString", ObjString(value))
}
private suspend fun rollbackException(scope: Scope, message: String): net.sergeych.lyng.obj.ObjException {
val dbModule = scope.currentImportProvider.createModuleScope(scope.pos, "lyng.io.db")
val rollbackClass = dbModule.requireClass("RollbackException")
return rollbackClass.invoke(scope, ObjNull, ObjString(message)) as net.sergeych.lyng.obj.ObjException
}
private suspend fun dateOf(scope: Scope, value: String): Obj {
val timeModule = scope.currentImportProvider.createModuleScope(scope.pos, "lyng.time")
val dateClass = timeModule.requireClass("Date")
return dateClass.invoke(scope, ObjNull, ObjString(value))
}
private fun emptyMapObj(): Obj = ObjMap()
}
@@ -17,6 +17,8 @@
package net.sergeych.lyngio.console package net.sergeych.lyngio.console
import kotlinx.cinterop.*
import platform.posix.*
import kotlin.test.Test import kotlin.test.Test
import kotlin.test.assertEquals import kotlin.test.assertEquals
import kotlin.test.assertFalse import kotlin.test.assertFalse
@@ -64,4 +66,41 @@ class LinuxPosixLyngConsoleTest {
assertEquals("A", ev.key) assertEquals("A", ev.key)
assertTrue(ev.shift) assertTrue(ev.shift)
} }
@OptIn(ExperimentalForeignApi::class)
@Test
fun configuresRawModeReadSemantics() = memScoped {
val attrs = alloc<termios>()
attrs.c_iflag =
BRKINT.convert<UInt>() or
ICRNL.convert<UInt>() or
INPCK.convert<UInt>() or
ISTRIP.convert<UInt>() or
IXON.convert<UInt>()
attrs.c_oflag = OPOST.convert<UInt>()
attrs.c_cflag = 0u
attrs.c_lflag =
ECHO.convert<UInt>() or
ICANON.convert<UInt>() or
IEXTEN.convert<UInt>() or
ISIG.convert<UInt>()
attrs.c_cc[VMIN] = 9u
attrs.c_cc[VTIME] = 9u
configureRawInput(attrs)
assertEquals(0u, attrs.c_iflag and BRKINT.convert<UInt>())
assertEquals(0u, attrs.c_iflag and ICRNL.convert<UInt>())
assertEquals(0u, attrs.c_iflag and INPCK.convert<UInt>())
assertEquals(0u, attrs.c_iflag and ISTRIP.convert<UInt>())
assertEquals(0u, attrs.c_iflag and IXON.convert<UInt>())
assertEquals(0u, attrs.c_oflag and OPOST.convert<UInt>())
assertEquals(CS8.convert<UInt>(), attrs.c_cflag and CS8.convert<UInt>())
assertEquals(0u, attrs.c_lflag and ECHO.convert<UInt>())
assertEquals(0u, attrs.c_lflag and ICANON.convert<UInt>())
assertEquals(0u, attrs.c_lflag and IEXTEN.convert<UInt>())
assertEquals(0u, attrs.c_lflag and ISIG.convert<UInt>())
assertEquals(0u, attrs.c_cc[VMIN].toUInt())
assertEquals(1u, attrs.c_cc[VTIME].toUInt())
}
} }
Loaded 100 of 232 files, more files were not shown because too many files have changed in this diff. Show more