Compare commits

...
Author SHA1 Message Date
sergeych 9c342c5c72 Make lyng-mark white and regenerate 256x256 PNG 2025-11-29 10:54:27 +01:00
sergeych 59055ace8c Add 256x256 PNG export of lyng-mark.svg at site/src/jsMain/resources/icons/lyng-mark-256.png 2025-11-29 10:51:01 +01:00
sergeych 2005f405e4 site icon 2025-11-29 09:58:53 +01:00
sergeych 062f344676 published lyngio to maven, added to docs 2025-11-29 09:02:28 +01:00
sergeych 438e48959e - fixed bug in compiler (rare)
- added lyng.io.fs (multiplatform)
- CLI tools now have access to the filesystem
2025-11-29 00:51:01 +01:00
sergeych 41746f22e5 some more samples 2025-11-28 12:38:43 +01:00
sergeych e584c7aa63 site/docs improvements 2025-11-28 11:25:47 +01:00
41 changed files with 2252 additions and 21 deletions

No files matched your search

+2 -3
View File
@@ -1,8 +1,6 @@
# Lyng: modern scripting for kotlin multiplatform # Lyng: modern scripting for kotlin multiplatform
Please visit the project homepage: [https://lynglang.com](https://lynglang.com) Please visit the project homepage: [https://lynglang.com](https://lynglang.com) and a [telegram channel](https://t.me/lynglang) for updates.
A KMP library and a standalone interpreter. v1.0.0-SNAPSHOT is now available.
- simple, compact, intuitive and elegant modern code: - simple, compact, intuitive and elegant modern code:
@@ -39,6 +37,7 @@ and it is multithreaded on platforms supporting it (automatically, no code chang
## Resources: ## Resources:
- [Language home](https://lynglang.com)
- [introduction and tutorial](docs/tutorial.md) - start here please - [introduction and tutorial](docs/tutorial.md) - start here please
- [Samples directory](docs/samples) - [Samples directory](docs/samples)
- [Books directory](docs) - [Books directory](docs)
+6 -2
View File
@@ -1,6 +1,10 @@
# OO implementation in Lyng # Object-oriented programming
## Declaration [//]: # (topMenu)
Lyng supports first class OOP constructs, based on classes with multiple inheritance.
## Class Declaration
The class clause looks like The class clause looks like
+2
View File
@@ -1,5 +1,7 @@
# Declaring arguments in Lyng # Declaring arguments in Lyng
[//]: # (topMenu)
It is a common thing that occurs in many places in Lyng, function declarations, It is a common thing that occurs in many places in Lyng, function declarations,
lambdas and class declarations. lambdas and class declarations.
+228
View File
@@ -0,0 +1,228 @@
### lyng.io.fs — async filesystem access for Lyng scripts
This module provides a uniform, suspend-first filesystem API to Lyng scripts, backed by Kotlin Multiplatform implementations.
- JVM/Android/Native: Okio `FileSystem.SYSTEM` (non-blocking via coroutine dispatcher)
- JS/Node: Node filesystem (currently via Okio node backend; a native `fs/promises` backend is planned)
- JS/Browser and Wasm: in-memory virtual filesystem for now
It exposes a Lyng class `Path` with methods for file and directory operations, including streaming readers for large files.
It is a separate library because access to teh filesystem is a security risk we compensate with a separate API that user must explicitly include to the dependency and allow. Together with `FsAceessPolicy` that is required to `createFs()` which actually adds the filesystem to the scope, the security risk is isolated.
Also, it helps keep Lyng core small and focused.
---
#### Add the library to your project (Gradle)
If you use this repository as a multi-module project, add a dependency on `:lyngio`:
```kotlin
dependencies {
implementation("net.sergeych:lyngio:0.0.1-SNAPSHOT")
}
```
Note on maven repository. Lyngio uses ths same maven as Lyng code (`lynglib`) so it is most likely already in your project. If ont, add it to the proper section of your `build.gradle.kts` or settings.gradle.kts:
```kotlin
repositories {
maven("https://gitea.sergeych.net/api/packages/SergeychWorks/maven")
}
```
This brings in:
- `:lynglib` (Lyng engine)
- Okio (`okio`, `okio-fakefilesystem`, and `okio-nodefilesystem` for JS)
- Kotlin coroutines
---
#### Install the module into a Lyng Scope
The filesystem module is not installed automatically. You must explicitly register it in the scope’s `ImportManager` using the installer. You can customize access control via `FsAccessPolicy`.
Kotlin (host) bootstrap example (imports omitted for brevity):
```kotlin
val scope: Scope = Scope.new()
val installed: Boolean = createFs(PermitAllAccessPolicy, scope)
// installed == true on first registration in this ImportManager, false on repeats
// In scripts (or via scope.eval), import the module to use its symbols:
scope.eval("import lyng.io.fs")
```
You can install with a custom policy too (see Access policy below).
---
#### Using from Lyng scripts
```lyng
val p = Path("/tmp/hello.txt")
// Text I/O
p.writeUtf8("Hello Lyng!\n")
println(p.readUtf8())
// Binary I/O
val data = Buffer.fromHex("deadbeef")
p.writeBytes(data)
// Existence and directories
assertTrue(p.exists())
Path("/tmp/work").mkdirs()
// Listing
for (entry in Path("/tmp").list()) {
println(entry)
}
// Globbing
val txts = Path("/tmp").glob("**/*.txt").toList()
// Copy / Move / Delete
Path("/tmp/a.txt").copy("/tmp/b.txt", overwrite=true)
Path("/tmp/b.txt").move("/tmp/c.txt", overwrite=true)
Path("/tmp/c.txt").delete()
// Streaming large files (does not load whole file into memory)
var bytes = 0
val it = Path("/tmp/big.bin").readChunks(1_048_576) // 1MB chunks
val iter = it.iterator()
while (iter.hasNext()) {
val chunk = iter.next()
bytes = bytes + chunk.size()
}
// Text chunks and lines
for (s in Path("/tmp/big.txt").readUtf8Chunks(64_000)) {
// process each string chunk
}
for (ln in Path("/tmp/big.txt").lines()) {
// process line by line
}
```
---
#### API (Lyng class `Path`)
Constructor:
- `Path(path: String)` — creates a Path object
- `Paths(path: String)` — alias
File and directory operations (all suspend under the hood):
- `name`: name, `String`
- `segments`: list of parsed path segments (directories)
- `parent`: parent directory, `Path?`; null if root
- `exists(): Bool`
- `isFile(): Bool` — true if the path points to a regular file (cached metadata)
- `isDirectory(): Bool` — true if the path points to a directory (cached metadata)
- `size(): Int?` — size in bytes or null if unknown (cached metadata)
- `createdAt(): Instant?` — creation time as Lyng `Instant`, or null (cached metadata)
- `createdAtMillis(): Int?` — creation time in epoch milliseconds, or null (cached metadata)
- `modifiedAt(): Instant?` — last modification time as Lyng `Instant`, or null (cached metadata)
- `modifiedAtMillis(): Int?` — last modification time in epoch milliseconds, or null (cached metadata)
- `list(): List<Path>` — children of a directory
- `readBytes(): Buffer`
- `writeBytes(bytes: Buffer)`
- `appendBytes(bytes: Buffer)`
- `readUtf8(): String`
- `writeUtf8(text: String)`
- `appendUtf8(text: String)`
- `metadata(): Map` — keys: `isFile`, `isDirectory`, `size`, `createdAtMillis`, `modifiedAtMillis`, `isSymlink`
- `mkdirs(mustCreate: Bool = false)`
- `move(to: Path|String, overwrite: Bool = false)`
- `delete(mustExist: Bool = false, recursively: Bool = false)`
- `copy(to: Path|String, overwrite: Bool = false)`
- `glob(pattern: String): List<Path>` — supports `**`, `*`, `?` (POSIX-style)
Streaming readers for big files:
- `readChunks(size: Int = 65536): Iterator<Buffer>` — iterate fixed-size byte chunks
- `readUtf8Chunks(size: Int = 65536): Iterator<String>` — iterate text chunks by character count
- `lines(): Iterator<String>` — line iterator built on `readUtf8Chunks`
Notes:
- Iterators implement Lyng iterator protocol. If you break early from a loop, the runtime will attempt to call `cancelIteration()` when available.
- Current implementations chunk in memory. The public API is stable; internals will evolve to true streaming on all platforms.
- Attribute accessors (`isFile`, `isDirectory`, `size`, `createdAt*`, `modifiedAt*`) cache a metadata snapshot inside the `Path` instance to avoid repeated filesystem calls during a sequence of queries. `metadata()` remains available for bulk access.
---
#### Access policy (security)
Access control is enforced by `FsAccessPolicy`. You pass a policy at installation time. The module wraps the filesystem with a secured decorator that consults the policy for each primitive operation.
Main types:
- `FsAccessPolicy` — your policy implementation
- `PermitAllAccessPolicy` — allows all operations (default for testing)
- `AccessOp` (sealed) — operations the policy can decide on:
- `ListDir(path)`
- `CreateFile(path)`
- `OpenRead(path)`
- `OpenWrite(path)`
- `OpenAppend(path)`
- `Delete(path)`
- `Rename(from, to)`
- `UpdateAttributes(path)` — defaults to write-level semantics
Minimal denying policy example (imports omitted for brevity):
```kotlin
val denyWrites = object : FsAccessPolicy {
override suspend fun check(op: AccessOp, ctx: AccessContext): AccessDecision = when (op) {
is AccessOp.OpenRead, is AccessOp.ListDir -> AccessDecision(Decision.Allow)
else -> AccessDecision(Decision.Deny, reason = "read-only policy")
}
}
createFs(denyWrites, scope)
scope.eval("import lyng.io.fs")
```
Composite operations like `copy` and `move` are checked as a set of primitives (e.g., `OpenRead(src)` + `Delete(dst)` if overwriting + `CreateFile(dst)` + `OpenWrite(dst)`).
---
#### Errors and exceptions
Policy denials are surfaced as Lyng runtime errors, not raw Kotlin exceptions:
- Internally, a denial throws `AccessDeniedException`. The module maps it to `ObjIllegalOperationException` wrapped into an `ExecutionError` visible to scripts.
Examples (Lyng):
```lyng
import lyng.io.fs
val p = Path("/protected/file.txt")
try {
p.writeUtf8("x")
fail("expected error")
} catch (e) {
// e is an ExecutionError; message contains the policy reason
}
```
Other I/O failures (e.g., not found, not a directory) are also raised as Lyng errors (`ObjIllegalStateException`, `ObjIllegalArgumentException`, etc.) depending on context.
---
#### Platform notes
- JVM/Android/Native: synchronous Okio calls are executed on `Dispatchers.IO` (JVM/Android) or `Dispatchers.Default` (Native) to avoid blocking the main thread.
- NodeJS: currently uses Okio’s Node backend. For heavy I/O, a native `fs/promises` backend is planned to fully avoid event-loop blocking.
- Browser/Wasm: uses an in-memory filesystem for now. Persistent backends (IndexedDB or File System Access API) are planned.
---
#### Roadmap
- Native NodeJS backend using `fs/promises`
- Browser persistent storage (IndexedDB)
- Streaming readers/writers over real OS streams
- Attribute setters and richer metadata
If you have specific needs (e.g., sandboxing, virtual roots), implement a custom `FsAccessPolicy` or ask us to add a helper.
+3 -1
View File
@@ -1,4 +1,6 @@
# Parallelism in Lyng # Multithreading/parallel execution
[//]: # (topMenu)
Lyng is built to me multithreaded where possible (e.g. all targets byt JS and wasmJS as for now) Lyng is built to me multithreaded where possible (e.g. all targets byt JS and wasmJS as for now)
and cooperatively parallel (coroutine based) everywhere. and cooperatively parallel (coroutine based) everywhere.
+2
View File
@@ -1,6 +1,8 @@
# Map literals — refined proposal # Map literals — refined proposal
[//]: # (excludeFromIndex)
Implement JavaScript-like literals for maps. The syntax and semantics align with named arguments in function calls, but map literals are expressions that construct `Map` values. Implement JavaScript-like literals for maps. The syntax and semantics align with named arguments in function calls, but map literals are expressions that construct `Map` values.
Keys can be either: Keys can be either:
+4 -1
View File
@@ -1,4 +1,7 @@
# Named arguments proposal # Named arguments in calls
[//]: # (excludeFromIndex)
Extend function/method calls to allow setting arguments by name using colon syntax at call sites. This is especially convenient with many parameters and default values. Extend function/method calls to allow setting arguments by name using colon syntax at call sites. This is especially convenient with many parameters and default values.
+14
View File
@@ -0,0 +1,14 @@
#!/bin/env lyng
import lyng.io.fs
import lyng.stdlib
val files = Path("../..").list().toList()
val longestNameLength = files.maxOf { it.name.length }
val format = "%-"+(longestNameLength+1) +"s %d"
for( f in files ) {
var name = f.name
if( f.isDirectory() ) name += "/"
println( format(name, f.size()) )
}
+10
View File
@@ -0,0 +1,10 @@
#!/bin/env lyng
val переменная = "значение"
val data = { greeting: "привет", переменная:, поле: "содержимое" }
assertEquals(data["переменная"], "значение")
assertEquals(data["поле"], "содержимое")
println(data)
+2
View File
@@ -1,5 +1,7 @@
# The `when` statement (expression) # The `when` statement (expression)
[//]: # (topMenu)
Lyng provides a concise multi-branch selection with `when`, heavily inspired by Kotlin. In Lyng, `when` is an expression: it evaluates to a value. If the selected branch contains no value (e.g., it ends with `void` or calls a void function like `println`), the whole `when` expression evaluates to `void`. Lyng provides a concise multi-branch selection with `when`, heavily inspired by Kotlin. In Lyng, `when` is an expression: it evaluates to a value. If the selected branch contains no value (e.g., it ends with `void` or calls a void function like `println`), the whole `when` expression evaluates to `void`.
Currently, Lyng implements the "subject" form: `when(value) { ... }`. The subject-less form `when { condition -> ... }` is not implemented yet. Currently, Lyng implements the "subject" form: `when(value) { ... }`. The subject-less form `when { condition -> ... }` is not implemented yet.
+4 -1
View File
@@ -34,4 +34,7 @@ kotlin.native.cacheKind.linuxX64=none
# On this environment, the system JDK 21 installation lacks `jlink`, causing # On this environment, the system JDK 21 installation lacks `jlink`, causing
# :lynglib:androidJdkImage to fail. Point Gradle to JDK 17 which includes `jlink`. # :lynglib:androidJdkImage to fail. Point Gradle to JDK 17 which includes `jlink`.
# This affects only the JDK Gradle runs with; Kotlin/JVM target remains compatible. # This affects only the JDK Gradle runs with; Kotlin/JVM target remains compatible.
#org.gradle.java.home=/usr/lib/jvm/java-17-openjdk-amd64 #org.gradle.java.home=/usr/lib/jvm/java-17-openjdk-amd64
android.experimental.lint.migrateToK2=false
android.lint.useK2Uast=false
kotlin.mpp.applyDefaultHierarchyTemplate=true
+1
View File
@@ -20,6 +20,7 @@ mp_bintools = { module = "net.sergeych:mp_bintools", version.ref = "mp_bintools"
firebase-crashlytics-buildtools = { group = "com.google.firebase", name = "firebase-crashlytics-buildtools", version.ref = "firebaseCrashlyticsBuildtools" } firebase-crashlytics-buildtools = { group = "com.google.firebase", name = "firebase-crashlytics-buildtools", version.ref = "firebaseCrashlyticsBuildtools" }
okio = { module = "com.squareup.okio:okio", version.ref = "okioVersion" } okio = { module = "com.squareup.okio:okio", version.ref = "okioVersion" }
okio-fakefilesystem = { module = "com.squareup.okio:okio-fakefilesystem", version.ref = "okioVersion" } okio-fakefilesystem = { module = "com.squareup.okio:okio-fakefilesystem", version.ref = "okioVersion" }
okio-nodefilesystem = { module = "com.squareup.okio:okio-nodefilesystem", version.ref = "okioVersion" }
compiler = { group = "androidx.databinding", name = "compiler", version.ref = "compiler" } compiler = { group = "androidx.databinding", name = "compiler", version.ref = "compiler" }
[plugins] [plugins]
+9
View File
@@ -57,6 +57,9 @@ kotlin {
dependencies { dependencies {
implementation(kotlin("stdlib-common")) implementation(kotlin("stdlib-common"))
implementation(project(":lynglib")) implementation(project(":lynglib"))
// Provide Lyng FS module to the CLI tool so it can install
// filesystem access into the execution Scope by default.
implementation(project(":lyngio"))
implementation(libs.okio) implementation(libs.okio)
implementation(libs.clikt) implementation(libs.clikt)
implementation(kotlin("stdlib-common")) implementation(kotlin("stdlib-common"))
@@ -72,6 +75,12 @@ kotlin {
implementation(libs.okio.fakefilesystem) implementation(libs.okio.fakefilesystem)
} }
} }
val jvmTest by getting {
dependencies {
implementation(kotlin("test"))
implementation(kotlin("test-junit"))
}
}
// val nativeMain by getting { // val nativeMain by getting {
// dependencies { // dependencies {
// implementation(kotlin("stdlib-common")) // implementation(kotlin("stdlib-common"))
+5
View File
@@ -30,7 +30,9 @@ import net.sergeych.lyng.LyngVersion
import net.sergeych.lyng.Script import net.sergeych.lyng.Script
import net.sergeych.lyng.ScriptError import net.sergeych.lyng.ScriptError
import net.sergeych.lyng.Source import net.sergeych.lyng.Source
import net.sergeych.lyng.io.fs.createFs
import net.sergeych.lyng.obj.* import net.sergeych.lyng.obj.*
import net.sergeych.lyngio.fs.security.PermitAllAccessPolicy
import net.sergeych.mp_tools.globalDefer import net.sergeych.mp_tools.globalDefer
import okio.FileSystem import okio.FileSystem
import okio.Path.Companion.toPath import okio.Path.Companion.toPath
@@ -62,6 +64,9 @@ val baseScopeDefer = globalDefer {
exit(requireOnlyArg<ObjInt>().toInt()) exit(requireOnlyArg<ObjInt>().toInt())
ObjVoid ObjVoid
} }
// Install lyng.io.fs module with full access by default for the CLI tool's Scope.
// Scripts still need to `import lyng.io.fs` to use Path API.
createFs(PermitAllAccessPolicy, this)
} }
} }
@@ -0,0 +1,64 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng_cli
import kotlinx.coroutines.runBlocking
import net.sergeych.baseScopeDefer
import org.junit.Test
import kotlin.io.path.createTempDirectory
class FsIntegrationJvmTest {
@Test
fun scopeHasFsModuleInstalled() {
runBlocking {
val scope = baseScopeDefer.await()
// Ensure we can import the FS module and use Path bindings from Lyng script
val dir = createTempDirectory("lyng_cli_fs_test_")
try {
val file = dir.resolve("hello.txt")
// Drive the operation via Lyng code to validate bindings end-to-end
scope.eval(
"""
import lyng.io.fs
val p = Path("${'$'}{file}")
p.writeUtf8("hello from cli test")
assertEquals(true, p.exists())
assertEquals("hello from cli test", p.readUtf8())
""".trimIndent()
)
} finally {
dir.toFile().deleteRecursively()
}
}
}
@Test
fun scopeHasFsSeesRealFs() {
runBlocking {
val scope = baseScopeDefer.await()
// Drive the operation via Lyng code to validate bindings end-to-end
scope.eval(
"""
import lyng.io.fs
// list current folder files
println( Path(".").list().toList() )
""".trimIndent()
)
}
}
}
+152
View File
@@ -0,0 +1,152 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
/*
* LyngIO: Compose Multiplatform library module depending on :lynglib
*/
import org.jetbrains.kotlin.gradle.ExperimentalKotlinGradlePluginApi
import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl
import org.jetbrains.kotlin.gradle.dsl.JvmTarget
plugins {
alias(libs.plugins.kotlinMultiplatform)
alias(libs.plugins.androidLibrary)
`maven-publish`
}
group = "net.sergeych"
version = "0.0.1-SNAPSHOT"
kotlin {
jvm()
androidTarget {
publishLibraryVariants("release")
@OptIn(ExperimentalKotlinGradlePluginApi::class)
compilerOptions {
jvmTarget.set(JvmTarget.JVM_11)
}
}
iosX64()
iosArm64()
iosSimulatorArm64()
macosX64()
macosArm64()
mingwX64()
linuxX64()
linuxArm64()
js {
browser()
nodejs()
}
@OptIn(ExperimentalWasmDsl::class)
wasmJs() {
browser()
nodejs()
}
// Keep expect/actual warning suppressed consistently with other modules
targets.configureEach {
compilations.configureEach {
compilerOptions.configure {
freeCompilerArgs.add("-Xexpect-actual-classes")
}
}
}
sourceSets {
all {
languageSettings.optIn("kotlin.ExperimentalUnsignedTypes")
// languageSettings.optIn("kotlinx.coroutines.ExperimentalCoroutinesApi")
// Correct opt-in markers for coroutines
// languageSettings.optIn("kotlinx.coroutines.DelicateCoroutinesApi")
// languageSettings.optIn("kotlin.contracts.ExperimentalContracts")
// languageSettings.optIn("kotlinx.coroutines.FlowPreview")
}
val commonMain by getting {
dependencies {
api(project(":lynglib"))
api(libs.okio)
api(libs.kotlinx.coroutines.core)
}
}
val commonTest by getting {
dependencies {
implementation(libs.kotlin.test)
}
}
// JS: use runtime detection in jsMain to select Node vs Browser implementation
val jsMain by getting {
dependencies {
api(libs.okio)
implementation(libs.okio.fakefilesystem)
implementation("com.squareup.okio:okio-nodefilesystem:${libs.versions.okioVersion.get()}")
}
}
// For Wasm we use in-memory VFS for now
val wasmJsMain by getting {
dependencies {
api(libs.okio)
implementation(libs.okio.fakefilesystem)
}
}
}
}
android {
namespace = "net.sergeych.lyngio"
compileSdk = libs.versions.android.compileSdk.get().toInt()
defaultConfig {
minSdk = libs.versions.android.minSdk.get().toInt()
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_11
targetCompatibility = JavaVersion.VERSION_11
}
lint {
// Prevent Android Lint from failing the build due to Kotlin toolchain
// version mismatches in the environment. This keeps CI green while
// still generating lint reports locally.
abortOnError = false
checkReleaseBuilds = false
}
}
// Disable Android Lint tasks for this module to avoid toolchain incompatibility
// until AGP and Kotlin versions align perfectly in the environment.
tasks.matching { it.name.startsWith("lint", ignoreCase = true) }.configureEach {
this.enabled = false
}
publishing {
val mavenToken by lazy {
File("${System.getProperty("user.home")}/.gitea_token").readText()
}
repositories {
maven {
credentials(HttpHeaderCredentials::class) {
name = "Authorization"
value = mavenToken
}
url = uri("https://gitea.sergeych.net/api/packages/SergeychWorks/maven")
authentication {
create("Authorization", HttpHeaderAuthentication::class)
}
}
}
}
@@ -0,0 +1,25 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.fs
import kotlinx.coroutines.CoroutineDispatcher
import kotlinx.coroutines.Dispatchers
import okio.FileSystem
actual fun platformAsyncFs(): LyngFs = OkioAsyncFs(FileSystem.SYSTEM)
actual val LyngIoDispatcher: CoroutineDispatcher = Dispatchers.IO
@@ -0,0 +1,473 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
/*
* Lyng FS module installer and bindings
*/
package net.sergeych.lyng.io.fs
import net.sergeych.lyng.ModuleScope
import net.sergeych.lyng.Scope
import net.sergeych.lyng.obj.*
import net.sergeych.lyng.pacman.ImportManager
import net.sergeych.lyngio.fs.LyngFS
import net.sergeych.lyngio.fs.LyngFs
import net.sergeych.lyngio.fs.LyngPath
import net.sergeych.lyngio.fs.security.AccessDeniedException
import net.sergeych.lyngio.fs.security.FsAccessPolicy
import net.sergeych.lyngio.fs.security.LyngFsSecured
import okio.Path.Companion.toPath
/**
* Install Lyng module `lyng.io.fs` into the given scope's ImportManager.
* Returns true if installed, false if it was already registered in this manager.
*/
fun createFsModule(policy: FsAccessPolicy, scope: Scope): Boolean =
createFsModule(policy, scope.importManager)
// Alias as requested earlier in discussions
fun createFs(policy: FsAccessPolicy, scope: Scope): Boolean = createFsModule(policy, scope)
/** Same as [createFsModule] but with explicit [ImportManager]. */
fun createFsModule(policy: FsAccessPolicy, manager: ImportManager): Boolean {
val name = "lyng.io.fs"
// Avoid re-registering in this ImportManager
if (manager.packageNames.contains(name)) return false
manager.addPackage(name) { module ->
buildFsModule(module, policy)
}
return true
}
// Alias overload for ImportManager
fun createFs(policy: FsAccessPolicy, manager: ImportManager): Boolean = createFsModule(policy, manager)
// --- Module builder ---
private suspend fun buildFsModule(module: ModuleScope, policy: FsAccessPolicy) {
// Per-module secured FS, captured by all factories and methods
val base: LyngFs = LyngFS.system()
val secured = LyngFsSecured(base, policy)
// Path class bound to this module
val pathType = object : ObjClass("Path") {
override suspend fun callOn(scope: Scope): Obj {
val arg = scope.requireOnlyArg<ObjString>()
val str = arg.value
return ObjPath(this, secured, str.toPath())
}
}.apply {
addFn("name") {
val self = thisAs<ObjPath>()
self.path.name.toObj()
}
addFn("parent") {
val self = thisAs<ObjPath>()
self.path.parent?.let {
ObjPath( this@apply, self.secured, it)
} ?: ObjNull
}
addFn("segments") {
val self = thisAs<ObjPath>()
ObjList(self.path.segments.map { ObjString(it) }.toMutableList())
}
// exists(): Bool
addFn("exists") {
fsGuard {
val self = this.thisObj as ObjPath
(self.secured.exists(self.path)).toObj()
}
}
// isFile(): Bool — cached metadata
addFn("isFile") {
fsGuard {
val self = this.thisObj as ObjPath
self.ensureMetadata().let { ObjBool(it.isRegularFile) }
}
}
// isDirectory(): Bool — cached metadata
addFn("isDirectory") {
fsGuard {
val self = this.thisObj as ObjPath
self.ensureMetadata().let { ObjBool(it.isDirectory) }
}
}
// size(): Int? — null when unavailable
addFn("size") {
fsGuard {
val self = this.thisObj as ObjPath
val m = self.ensureMetadata()
m.size?.let { ObjInt(it) } ?: ObjNull
}
}
// createdAt(): Instant? — Lyng Instant, null when unavailable
addFn("createdAt") {
fsGuard {
val self = this.thisObj as ObjPath
val m = self.ensureMetadata()
m.createdAtMillis?.let { ObjInstant(kotlinx.datetime.Instant.fromEpochMilliseconds(it)) } ?: ObjNull
}
}
// createdAtMillis(): Int? — milliseconds since epoch or null
addFn("createdAtMillis") {
fsGuard {
val self = this.thisObj as ObjPath
val m = self.ensureMetadata()
m.createdAtMillis?.let { ObjInt(it) } ?: ObjNull
}
}
// modifiedAt(): Instant? — Lyng Instant, null when unavailable
addFn("modifiedAt") {
fsGuard {
val self = this.thisObj as ObjPath
val m = self.ensureMetadata()
m.modifiedAtMillis?.let { ObjInstant(kotlinx.datetime.Instant.fromEpochMilliseconds(it)) } ?: ObjNull
}
}
// modifiedAtMillis(): Int? — milliseconds since epoch or null
addFn("modifiedAtMillis") {
fsGuard {
val self = this.thisObj as ObjPath
val m = self.ensureMetadata()
m.modifiedAtMillis?.let { ObjInt(it) } ?: ObjNull
}
}
// list(): List<Path>
addFn("list") {
fsGuard {
val self = this.thisObj as ObjPath
val items = self.secured.list(self.path).map { ObjPath(self.objClass, self.secured, it) }
ObjList(items.toMutableList())
}
}
// readBytes(): Buffer
addFn("readBytes") {
fsGuard {
val self = this.thisObj as ObjPath
val bytes = self.secured.readBytes(self.path)
ObjBuffer(bytes.asUByteArray())
}
}
// writeBytes(bytes: Buffer)
addFn("writeBytes") {
fsGuard {
val self = this.thisObj as ObjPath
val buf = requiredArg<ObjBuffer>(0)
self.secured.writeBytes(self.path, buf.byteArray.asByteArray(), append = false)
ObjVoid
}
}
// appendBytes(bytes: Buffer)
addFn("appendBytes") {
fsGuard {
val self = this.thisObj as ObjPath
val buf = requiredArg<ObjBuffer>(0)
self.secured.writeBytes(self.path, buf.byteArray.asByteArray(), append = true)
ObjVoid
}
}
// readUtf8(): String
addFn("readUtf8") {
fsGuard {
val self = this.thisObj as ObjPath
self.secured.readUtf8(self.path).toObj()
}
}
// writeUtf8(text: String)
addFn("writeUtf8") {
fsGuard {
val self = this.thisObj as ObjPath
val text = requireOnlyArg<ObjString>().value
self.secured.writeUtf8(self.path, text, append = false)
ObjVoid
}
}
// appendUtf8(text: String)
addFn("appendUtf8") {
fsGuard {
val self = this.thisObj as ObjPath
val text = requireOnlyArg<ObjString>().value
self.secured.writeUtf8(self.path, text, append = true)
ObjVoid
}
}
// metadata(): Map
addFn("metadata") {
fsGuard {
val self = this.thisObj as ObjPath
val m = self.secured.metadata(self.path)
ObjMap(mutableMapOf(
ObjString("isFile") to ObjBool(m.isRegularFile),
ObjString("isDirectory") to ObjBool(m.isDirectory),
ObjString("size") to (m.size?.toLong() ?: 0L).toObj(),
ObjString("createdAtMillis") to ((m.createdAtMillis ?: 0L)).toObj(),
ObjString("modifiedAtMillis") to ((m.modifiedAtMillis ?: 0L)).toObj(),
ObjString("isSymlink") to ObjBool(m.isSymlink),
))
}
}
// mkdirs(mustCreate: Bool=false)
addFn("mkdirs") {
fsGuard {
val self = this.thisObj as ObjPath
val mustCreate = args.list.getOrNull(0)?.toBool() ?: false
self.secured.createDirectories(self.path, mustCreate)
ObjVoid
}
}
// move(to: Path|String, overwrite: Bool=false)
addFn("move") {
fsGuard {
val self = this.thisObj as ObjPath
val toPath = parsePathArg(this, self, requiredArg<Obj>(0))
val overwrite = args.list.getOrNull(1)?.toBool() ?: false
self.secured.move(self.path, toPath, overwrite)
ObjVoid
}
}
// delete(mustExist: Bool=false, recursively: Bool=false)
addFn("delete") {
fsGuard {
val self = this.thisObj as ObjPath
val mustExist = args.list.getOrNull(0)?.toBool() ?: false
val recursively = args.list.getOrNull(1)?.toBool() ?: false
self.secured.delete(self.path, mustExist, recursively)
ObjVoid
}
}
// copy(to: Path|String, overwrite: Bool=false)
addFn("copy") {
fsGuard {
val self = this.thisObj as ObjPath
val toPath = parsePathArg(this, self, requiredArg<Obj>(0))
val overwrite = args.list.getOrNull(1)?.toBool() ?: false
self.secured.copy(self.path, toPath, overwrite)
ObjVoid
}
}
// glob(pattern: String): List<Path>
addFn("glob") {
fsGuard {
val self = this.thisObj as ObjPath
val pattern = requireOnlyArg<ObjString>().value
val matches = self.secured.glob(self.path, pattern)
ObjList(matches.map { ObjPath(self.objClass, self.secured, it) }.toMutableList())
}
}
// --- streaming readers (initial version: chunk from whole content, API stable) ---
// readChunks(size: Int = 65536) -> Iterator<Buffer>
addFn("readChunks") {
fsGuard {
val self = this.thisObj as ObjPath
val size = args.list.getOrNull(0)?.toInt() ?: 65536
val bytes = self.secured.readBytes(self.path)
ObjFsBytesIterator(bytes, size)
}
}
// readUtf8Chunks(size: Int = 65536) -> Iterator<String>
addFn("readUtf8Chunks") {
fsGuard {
val self = this.thisObj as ObjPath
val size = args.list.getOrNull(0)?.toInt() ?: 65536
val text = self.secured.readUtf8(self.path)
ObjFsStringChunksIterator(text, size)
}
}
// lines() -> Iterator<String>, implemented via readUtf8Chunks
addFn("lines") {
fsGuard {
val chunkIt = thisObj.invokeInstanceMethod(this, "readUtf8Chunks")
ObjFsLinesIterator(chunkIt)
}
}
}
// Export into the module scope
module.addConst("Path", pathType)
// Alias as requested (Path(s) style)
module.addConst("Paths", pathType)
}
// --- Helper classes and utilities ---
private fun parsePathArg(scope: Scope, self: ObjPath, arg: Obj): LyngPath {
return when (arg) {
is ObjString -> arg.value.toPath()
is ObjPath -> arg.path
else -> scope.raiseIllegalArgument("expected Path or String argument")
}
}
// Map Fs access denials to Lyng runtime exceptions for script-friendly errors
private suspend inline fun Scope.fsGuard(crossinline block: suspend () -> Obj): Obj {
return try {
block()
} catch (e: AccessDeniedException) {
raiseError(ObjIllegalOperationException(this, e.reasonDetail ?: "access denied"))
}
}
/** Kotlin-side instance backing the Lyng class `Path`. */
class ObjPath(
private val klass: ObjClass,
val secured: LyngFs,
val path: LyngPath,
) : Obj() {
// Cache for metadata to avoid repeated FS calls within the same object instance usage
private var _metadata: net.sergeych.lyngio.fs.LyngMetadata? = null
override val objClass: ObjClass get() = klass
override fun toString(): String = path.toString()
suspend fun ensureMetadata(): net.sergeych.lyngio.fs.LyngMetadata {
val cached = _metadata
if (cached != null) return cached
val m = secured.metadata(path)
_metadata = m
return m
}
}
/** Iterator over byte chunks as Buffers. */
class ObjFsBytesIterator(
private val data: ByteArray,
private val chunkSize: Int,
) : Obj() {
private var pos = 0
override val objClass: ObjClass = BytesIteratorType
companion object {
val BytesIteratorType = object : ObjClass("BytesIterator", ObjIterator) {
init {
// make it usable in for-loops
addFn("iterator") { thisObj }
addFn("hasNext") {
val self = thisAs<ObjFsBytesIterator>()
(self.pos < self.data.size).toObj()
}
addFn("next") {
val self = thisAs<ObjFsBytesIterator>()
if (self.pos >= self.data.size) raiseIllegalState("iterator exhausted")
val end = minOf(self.pos + self.chunkSize, self.data.size)
val chunk = self.data.copyOfRange(self.pos, end)
self.pos = end
ObjBuffer(chunk.asUByteArray())
}
addFn("cancelIteration") {
val self = thisAs<ObjFsBytesIterator>()
self.pos = self.data.size
ObjVoid
}
}
}
}
}
/** Iterator over utf-8 text chunks (character-counted chunks). */
class ObjFsStringChunksIterator(
private val text: String,
private val chunkChars: Int,
) : Obj() {
private var pos = 0
override val objClass: ObjClass = StringChunksIteratorType
companion object {
val StringChunksIteratorType = object : ObjClass("StringChunksIterator", ObjIterator) {
init {
// make it usable in for-loops
addFn("iterator") { thisObj }
addFn("hasNext") {
val self = thisAs<ObjFsStringChunksIterator>()
(self.pos < self.text.length).toObj()
}
addFn("next") {
val self = thisAs<ObjFsStringChunksIterator>()
if (self.pos >= self.text.length) raiseIllegalState("iterator exhausted")
val end = minOf(self.pos + self.chunkChars, self.text.length)
val chunk = self.text.substring(self.pos, end)
self.pos = end
ObjString(chunk)
}
addFn("cancelIteration") { ObjVoid }
}
}
}
}
/** Iterator that yields lines using an underlying chunks iterator. */
class ObjFsLinesIterator(
private val chunksIterator: Obj,
) : Obj() {
private var buffer: String = ""
private var exhausted = false
override val objClass: ObjClass = LinesIteratorType
companion object {
val LinesIteratorType = object : ObjClass("LinesIterator", ObjIterator) {
init {
// make it usable in for-loops
addFn("iterator") { thisObj }
addFn("hasNext") {
val self = thisAs<ObjFsLinesIterator>()
self.ensureBufferFilled(this)
(self.buffer.isNotEmpty() || !self.exhausted).toObj()
}
addFn("next") {
val self = thisAs<ObjFsLinesIterator>()
self.ensureBufferFilled(this)
if (self.buffer.isEmpty() && self.exhausted) raiseIllegalState("iterator exhausted")
val idx = self.buffer.indexOf('\n')
val line = if (idx >= 0) {
val l = self.buffer.substring(0, idx)
self.buffer = self.buffer.substring(idx + 1)
l
} else {
// last line without trailing newline
val l = self.buffer
self.buffer = ""
self.exhausted = true
l
}
ObjString(line)
}
addFn("cancelIteration") { ObjVoid }
}
}
}
private suspend fun ensureBufferFilled(scope: Scope) {
if (buffer.contains('\n') || exhausted) return
// Pull next chunk from the underlying iterator
val it = chunksIterator.invokeInstanceMethod(scope, "iterator")
val hasNext = it.invokeInstanceMethod(scope, "hasNext").toBool()
if (!hasNext) {
exhausted = true
return
}
val next = it.invokeInstanceMethod(scope, "next")
buffer += next.toString()
}
}
@@ -0,0 +1,30 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio
/**
* LyngIO: foundation for uniform file access APIs across JVM, Native and JS.
*
* This module depends on `:lynglib` and is configured as a Compose Multiplatform
* library, but it does not require Compose at runtime. Actual file system APIs
* will be added after we agree on the backend choices (Okio, NodeJS FS, browser VFS).
*/
object LyngIoInfo {
val name: String = "LyngIO"
val version: String = "0.0.1-SNAPSHOT"
}
@@ -0,0 +1,201 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.fs
import kotlinx.coroutines.CoroutineDispatcher
import kotlinx.coroutines.withContext
import okio.*
/**
* Async-first FS API (intermediate public surface).
* Note: this is an interim step on the way to final exported entities; names/types may evolve,
* but these APIs will stay public to ease migration.
*/
typealias LyngPath = Path
/** Metadata snapshot for a filesystem node. */
data class LyngMetadata(
val isRegularFile: Boolean,
val isDirectory: Boolean,
val size: Long?,
val createdAtMillis: Long?,
val modifiedAtMillis: Long?,
val isSymlink: Boolean,
)
/**
* Suspend-first, uniform filesystem. Heavy ops must not block the main thread or event loop.
*/
interface LyngFs {
suspend fun exists(path: LyngPath): Boolean
suspend fun list(dir: LyngPath): List<LyngPath>
suspend fun readBytes(path: LyngPath): ByteArray
suspend fun writeBytes(path: LyngPath, data: ByteArray, append: Boolean = false)
suspend fun readUtf8(path: LyngPath): String
suspend fun writeUtf8(path: LyngPath, text: String, append: Boolean = false)
suspend fun metadata(path: LyngPath): LyngMetadata
suspend fun createDirectories(dir: LyngPath, mustCreate: Boolean = false)
suspend fun move(from: LyngPath, to: LyngPath, overwrite: Boolean = false)
suspend fun delete(path: LyngPath, mustExist: Boolean = false, recursively: Boolean = false)
/** Default implementation: naive read-all + write; backends may override for efficiency. */
suspend fun copy(from: LyngPath, to: LyngPath, overwrite: Boolean = false) {
if (!overwrite && exists(to)) error("Target exists: $to")
val data = readBytes(from)
writeBytes(to, data, append = false)
}
/** Glob search starting at [dir], matching [pattern] with `**`, `*`, and `?`. */
suspend fun glob(dir: LyngPath, pattern: String): List<LyngPath> {
val regex = globToRegex(pattern)
val base = dir
val out = mutableListOf<LyngPath>()
suspend fun walk(d: LyngPath, rel: String) {
for (child in list(d)) {
val relPath = if (rel.isEmpty()) child.name else "$rel/${child.name}"
if (regex.matches(relPath)) out += child
if (metadata(child).isDirectory) walk(child, relPath)
}
}
walk(base, "")
return out
}
}
/** Okio-backed async implementation used on JVM/Android/Native and in-memory browser. */
class OkioAsyncFs(private val fs: FileSystem) : LyngFs {
override suspend fun exists(path: LyngPath): Boolean = withContext(LyngIoDispatcher) {
try { fs.metadataOrNull(path) != null } catch (_: Throwable) { false }
}
override suspend fun list(dir: LyngPath): List<LyngPath> = withContext(LyngIoDispatcher) {
fs.list(dir)
}
override suspend fun readBytes(path: LyngPath): ByteArray = withContext(LyngIoDispatcher) {
val buffer = Buffer()
fs.source(path).buffer().use { it.readAll(buffer) }
buffer.readByteArray()
}
override suspend fun writeBytes(path: LyngPath, data: ByteArray, append: Boolean) {
withContext(LyngIoDispatcher) {
val sink = if (append) fs.appendingSink(path) else fs.sink(path)
sink.buffer().use { it.write(data) }
}
}
override suspend fun readUtf8(path: LyngPath): String = withContext(LyngIoDispatcher) {
val buffer = Buffer()
fs.source(path).buffer().use { it.readAll(buffer) }
buffer.readUtf8()
}
override suspend fun writeUtf8(path: LyngPath, text: String, append: Boolean) {
withContext(LyngIoDispatcher) {
val sink = if (append) fs.appendingSink(path) else fs.sink(path)
sink.buffer().use { it.writeUtf8(text) }
}
}
override suspend fun metadata(path: LyngPath): LyngMetadata = withContext(LyngIoDispatcher) {
val m = fs.metadata(path)
LyngMetadata(
isRegularFile = m.isRegularFile,
isDirectory = m.isDirectory,
size = m.size,
createdAtMillis = m.createdAtMillis,
modifiedAtMillis = m.lastModifiedAtMillis,
isSymlink = m.symlinkTarget != null,
)
}
override suspend fun createDirectories(dir: LyngPath, mustCreate: Boolean) = withContext(LyngIoDispatcher) {
fs.createDirectories(dir, mustCreate)
}
override suspend fun move(from: LyngPath, to: LyngPath, overwrite: Boolean) = withContext(LyngIoDispatcher) {
if (overwrite) fs.delete(to, mustExist = false)
fs.atomicMove(from, to)
}
override suspend fun delete(path: LyngPath, mustExist: Boolean, recursively: Boolean) = withContext(LyngIoDispatcher) {
if (!recursively) {
fs.delete(path, mustExist)
return@withContext
}
fun deleteRec(p: Path) {
val meta = fs.metadataOrNull(p)
if (meta == null) {
if (mustExist) throw IllegalStateException("No such file or directory: $p")
return
}
if (meta.isDirectory) for (child in fs.list(p)) deleteRec(child)
fs.delete(p, mustExist = false)
}
deleteRec(path)
if (mustExist && fs.metadataOrNull(path) != null) error("Failed to delete: $path")
}
}
/**
* Default system FS selector per platform.
* - JVM/Android/Native: Okio `FileSystem.SYSTEM`
* - Node: native fs/promises implementation (non-blocking)
* - Browser/Wasm: in-memory Okio `FakeFileSystem`
*/
expect fun platformAsyncFs(): LyngFs
expect val LyngIoDispatcher: CoroutineDispatcher
object LyngFS { // factory holder (interim name)
fun system(): LyngFs = platformAsyncFs()
}
// --- helpers ---
private fun globToRegex(pattern: String): Regex {
// Convert glob with **, *, ? into a Regex that matches relative POSIX paths
val sb = StringBuilder()
var i = 0
sb.append('^')
while (i < pattern.length) {
when (val c = pattern[i]) {
'*' -> {
val isDouble = i + 1 < pattern.length && pattern[i + 1] == '*'
if (isDouble) {
// ** → match across directories
// Consume optional following slash
val nextIsSlash = i + 2 < pattern.length && (pattern[i + 2] == '/' || pattern[i + 2] == '\\')
sb.append(".*")
if (nextIsSlash) i += 1 // skip one extra to consume slash later via i+=2 below
i += 1
} else {
// * within a segment (no slash)
sb.append("[^/]*")
}
}
'?' -> sb.append("[^/]")
'.', '(', ')', '+', '|', '^', '$', '{', '}', '[', ']', '\\' -> sb.append('\\').append(c)
'/' , '\\' -> sb.append('/')
else -> sb.append(c)
}
i += 1
}
sb.append('$')
return Regex(sb.toString())
}
@@ -0,0 +1,105 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.fs.security
import net.sergeych.lyngio.fs.LyngPath
/**
* Primitive filesystem operations for access control decisions.
* Keep this sealed hierarchy minimal and extensible.
*/
sealed interface AccessOp {
data class ListDir(val path: LyngPath) : AccessOp
data class CreateFile(val path: LyngPath) : AccessOp
data class OpenRead(val path: LyngPath) : AccessOp
data class OpenWrite(val path: LyngPath) : AccessOp
data class OpenAppend(val path: LyngPath) : AccessOp
data class Delete(val path: LyngPath) : AccessOp
data class Rename(val from: LyngPath, val to: LyngPath) : AccessOp
/** Update file metadata/attributes (times, permissions, flags, etc.). */
data class UpdateAttributes(val path: LyngPath) : AccessOp
}
/**
* Optional contextual information for access decisions (principal, tags, attributes).
* This is intentionally generic to avoid coupling with specific auth models.
*/
data class AccessContext(
val principal: String? = null,
val attributes: Map<String, Any?> = emptyMap(),
)
enum class Decision { Allow, Deny }
data class AccessDecision(
val decision: Decision,
val reason: String? = null,
) {
fun isAllowed(): Boolean = decision == Decision.Allow
}
class AccessDeniedException(
val op: AccessOp,
val reasonDetail: String? = null,
) : IllegalStateException("Access denied for $op" + (reasonDetail?.let { ": $it" } ?: ""))
/**
* Policy interface that decides whether a specific filesystem operation is allowed.
*
* Note: by convention, updating file attributes should be treated as equivalent to
* write permission unless a stricter policy is desired. I.e., policies may implement
* [AccessOp.UpdateAttributes] by delegating to [AccessOp.OpenWrite] for the same path.
*/
interface FsAccessPolicy {
suspend fun check(op: AccessOp, ctx: AccessContext = AccessContext()): AccessDecision
// Convenience helpers
suspend fun require(op: AccessOp, ctx: AccessContext = AccessContext()) {
val res = check(op, ctx)
if (!res.isAllowed()) throw AccessDeniedException(op, res.reason)
}
suspend fun canList(path: LyngPath, ctx: AccessContext = AccessContext()) =
check(AccessOp.ListDir(path), ctx).isAllowed()
suspend fun canCreateFile(path: LyngPath, ctx: AccessContext = AccessContext()) =
check(AccessOp.CreateFile(path), ctx).isAllowed()
suspend fun canOpenRead(path: LyngPath, ctx: AccessContext = AccessContext()) =
check(AccessOp.OpenRead(path), ctx).isAllowed()
suspend fun canOpenWrite(path: LyngPath, ctx: AccessContext = AccessContext()) =
check(AccessOp.OpenWrite(path), ctx).isAllowed()
suspend fun canOpenAppend(path: LyngPath, ctx: AccessContext = AccessContext()) =
check(AccessOp.OpenAppend(path), ctx).isAllowed()
suspend fun canDelete(path: LyngPath, ctx: AccessContext = AccessContext()) =
check(AccessOp.Delete(path), ctx).isAllowed()
suspend fun canRename(from: LyngPath, to: LyngPath, ctx: AccessContext = AccessContext()) =
check(AccessOp.Rename(from, to), ctx).isAllowed()
/**
* Updating file attributes defaults to the same access level as opening for write
* in typical policies. Policies may override that to be stricter/looser.
*/
suspend fun canUpdateAttributes(path: LyngPath, ctx: AccessContext = AccessContext()) =
check(AccessOp.UpdateAttributes(path), ctx).isAllowed()
}
@@ -0,0 +1,110 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.fs.security
import net.sergeych.lyngio.fs.LyngFs
import net.sergeych.lyngio.fs.LyngMetadata
import net.sergeych.lyngio.fs.LyngPath
/**
* Decorator that applies an [FsAccessPolicy] to a delegate [LyngFs].
* Composite operations are checked as a set of primitive [AccessOp]s.
*
* Note: attribute update operations are covered by [AccessOp.UpdateAttributes] which
* by convention defaults to write permission in policy implementations.
*/
class LyngFsSecured(
private val delegate: LyngFs,
private val policy: FsAccessPolicy,
private val ctx: AccessContext = AccessContext(),
) : LyngFs {
override suspend fun exists(path: LyngPath): Boolean {
// Existence check is safe; do not guard to allow probing. Policies may tighten later.
return delegate.exists(path)
}
override suspend fun list(dir: LyngPath): List<LyngPath> {
policy.require(AccessOp.ListDir(dir), ctx)
return delegate.list(dir)
}
override suspend fun readBytes(path: LyngPath): ByteArray {
policy.require(AccessOp.OpenRead(path), ctx)
return delegate.readBytes(path)
}
override suspend fun writeBytes(path: LyngPath, data: ByteArray, append: Boolean) {
if (append) {
policy.require(AccessOp.OpenAppend(path), ctx)
} else {
policy.require(AccessOp.OpenWrite(path), ctx)
}
// Also require create-file to cover cases when the file does not exist yet.
policy.require(AccessOp.CreateFile(path), ctx)
return delegate.writeBytes(path, data, append)
}
override suspend fun readUtf8(path: LyngPath): String {
policy.require(AccessOp.OpenRead(path), ctx)
return delegate.readUtf8(path)
}
override suspend fun writeUtf8(path: LyngPath, text: String, append: Boolean) {
if (append) {
policy.require(AccessOp.OpenAppend(path), ctx)
} else {
policy.require(AccessOp.OpenWrite(path), ctx)
}
policy.require(AccessOp.CreateFile(path), ctx)
return delegate.writeUtf8(path, text, append)
}
override suspend fun metadata(path: LyngPath): LyngMetadata {
// Not specified in v1; treat as read access for now (can be revised later).
// policy.require(AccessOp.OpenRead(path), ctx)
return delegate.metadata(path)
}
override suspend fun createDirectories(dir: LyngPath, mustCreate: Boolean) {
// Model directory creation using CreateFile on the path to be created.
policy.require(AccessOp.CreateFile(dir), ctx)
return delegate.createDirectories(dir, mustCreate)
}
override suspend fun move(from: LyngPath, to: LyngPath, overwrite: Boolean) {
// Prefer Rename primitive; also check target deletion if overwrite.
policy.require(AccessOp.Rename(from, to), ctx)
if (overwrite) policy.require(AccessOp.Delete(to), ctx)
return delegate.move(from, to, overwrite)
}
override suspend fun delete(path: LyngPath, mustExist: Boolean, recursively: Boolean) {
policy.require(AccessOp.Delete(path), ctx)
return delegate.delete(path, mustExist, recursively)
}
override suspend fun copy(from: LyngPath, to: LyngPath, overwrite: Boolean) {
// Composite checks: read from source, create+write to dest, optional delete target first.
policy.require(AccessOp.OpenRead(from), ctx)
if (overwrite) policy.require(AccessOp.Delete(to), ctx)
policy.require(AccessOp.CreateFile(to), ctx)
policy.require(AccessOp.OpenWrite(to), ctx)
return delegate.copy(from, to, overwrite)
}
}
@@ -0,0 +1,27 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.fs.security
/**
* Minimal policy that allows every operation. Useful as a default or for tests.
*/
object PermitAllAccessPolicy : FsAccessPolicy {
private val allow = AccessDecision(Decision.Allow)
override suspend fun check(op: AccessOp, ctx: AccessContext): AccessDecision = allow
}
@@ -0,0 +1,20 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.fs
// Browser-specific actuals are provided in jsMain to avoid duplicate actuals per JS target.
@@ -0,0 +1,33 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.fs
import kotlinx.coroutines.CoroutineDispatcher
import kotlinx.coroutines.Dispatchers
import okio.FileSystem
import okio.fakefilesystem.FakeFileSystem
@Suppress("UnsafeCastFromDynamic")
private fun systemFsOrNull(): FileSystem? = try {
FileSystem.asDynamic().SYSTEM.unsafeCast<FileSystem>()
} catch (e: Throwable) {
null
}
actual fun platformAsyncFs(): LyngFs = OkioAsyncFs(systemFsOrNull() ?: FakeFileSystem())
actual val LyngIoDispatcher: CoroutineDispatcher = Dispatchers.Default
@@ -0,0 +1,20 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.fs
// Node-specific implementations are provided dynamically in jsMain via FileSystem.SYSTEM detection.
@@ -0,0 +1,25 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.fs
import kotlinx.coroutines.CoroutineDispatcher
import kotlinx.coroutines.Dispatchers
import okio.FileSystem
actual fun platformAsyncFs(): LyngFs = OkioAsyncFs(FileSystem.SYSTEM)
actual val LyngIoDispatcher: CoroutineDispatcher = Dispatchers.IO
@@ -0,0 +1,219 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.fs
import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.Scope
import net.sergeych.lyng.io.fs.createFs
import net.sergeych.lyng.obj.ObjIllegalOperationException
import net.sergeych.lyngio.fs.security.*
import kotlin.io.path.createTempDirectory
import kotlin.io.path.writeText
import kotlin.test.Test
import kotlin.test.assertFailsWith
import kotlin.test.assertTrue
class FsModuleJvmScriptStyleTest {
private fun newScope(): Scope = Scope.new()
private suspend fun installModule(scope: Scope) {
createFs(PermitAllAccessPolicy, scope)
scope.eval("import lyng.io.fs")
}
@Test
fun basicTextIoScript() {
runBlocking {
val scope = newScope()
installModule(scope)
val dir = createTempDirectory("lyngio_script_basic_")
try {
val file = dir.resolve("hello.txt")
scope.eval(
"""
import lyng.io.fs
import lyng.stdlib
val p = Path("${file}")
assertEquals(false, p.exists())
p.writeUtf8("Hello from Lyng")
assertEquals(true, p.exists())
assertEquals("Hello from Lyng", p.readUtf8())
println(p.name)
assertEquals("hello.txt", p.name)
assert(p.segments is List )
assert(p.parent is Path)
""".trimIndent()
)
} finally {
dir.toFile().deleteRecursively()
}
}
}
@Test
fun listGlobAndMetadataScript() {
runBlocking {
val scope = newScope()
installModule(scope)
val dir = createTempDirectory("lyngio_script_glob_")
try {
val a = dir.resolve("a").toFile().apply { mkdirs() }
dir.resolve("a/b").toFile().apply { mkdirs() }
dir.resolve("a/one.txt").writeText("1")
dir.resolve("a/b/two.txt").writeText("2")
dir.resolve("a/b/three.md").writeText("3")
scope.eval(
"""
import lyng.io.fs
val root = Path("${a}")
val list = root.list().toList()
val names = list.map { it.toString() }
var hasOne = false
var hasB = false
val it1 = names.iterator()
while (it1.hasNext()) {
val n = it1.next()
if (n.endsWith("one.txt")) hasOne = true
if (n.endsWith("/b") || n.endsWith("\\b") || n.endsWith("b")) hasB = true
}
assertEquals(true, hasOne)
assertEquals(true, hasB)
val matches = root.glob("**/*.txt").toList()
val mnames = matches.map { it.toString() }
var hasOneTxt = false
var hasTwoTxt = false
val it2 = mnames.iterator()
while (it2.hasNext()) {
val n = it2.next()
if (n.endsWith("one.txt")) hasOneTxt = true
if (n.endsWith("two.txt")) hasTwoTxt = true
}
assertEquals(true, hasOneTxt)
assertEquals(true, hasTwoTxt)
val f = Path("${dir.resolve("a/one.txt")}")
val m = f.metadata()
assertEquals(true, m["isFile"])
""".trimIndent()
)
} finally {
dir.toFile().deleteRecursively()
}
}
}
@Test
fun streamingUtf8ChunksAndLinesScript() {
runBlocking {
val scope = newScope()
installModule(scope)
val dir = createTempDirectory("lyngio_script_stream_")
try {
val text = buildString {
repeat(5000) { append("строка-").append(it).append('\n') }
}
val tf = dir.resolve("big.txt"); tf.writeText(text)
scope.eval(
"""
import lyng.io.fs
val p = Path("${tf}")
var total = 0
val it = p.readUtf8Chunks(4096)
val iter = it.iterator()
while (iter.hasNext()) {
val chunk = iter.next()
total = total + chunk.size()
}
assertEquals(${text.length}, total)
var n = 0
var first = ""
var last = ""
val lit = p.lines()
val liter = lit.iterator()
while (liter.hasNext()) {
val ln = liter.next()
if (n == 0) first = ln
last = ln
n = n + 1
}
assertEquals(5000, n)
assertEquals("строка-0", first)
assertEquals("строка-4999", last)
""".trimIndent()
)
} finally {
dir.toFile().deleteRecursively()
}
}
}
@Test
fun denyingPolicyMappingScript() {
runBlocking {
val denyWrites = object : FsAccessPolicy {
override suspend fun check(op: AccessOp, ctx: AccessContext): AccessDecision = when (op) {
is AccessOp.OpenRead, is AccessOp.ListDir -> AccessDecision(Decision.Allow)
else -> AccessDecision(Decision.Deny, reason = "denied by test policy")
}
}
val prep = newScope()
installModule(prep)
val dir = createTempDirectory("lyngio_script_deny_")
try {
val file = dir.resolve("ro.txt"); file.writeText("ro")
val scope = newScope()
createFs(denyWrites, scope)
scope.eval("import lyng.io.fs")
// reading should succeed
scope.eval(
"""
import lyng.io.fs
val p = Path("${file}")
assertEquals("ro", p.readUtf8())
""".trimIndent()
)
// writing should throw ExecutionError(ObjIllegalOperationException)
val err = assertFailsWith<ExecutionError> {
scope.eval(
"""
import lyng.io.fs
// don't redeclare `p` to avoid same-scope conflicts across eval calls
Path("${file}").writeUtf8("x")
""".trimIndent()
)
}
assertTrue((err as ExecutionError).errorObject is ObjIllegalOperationException)
} finally {
dir.toFile().deleteRecursively()
}
}
}
}
@@ -0,0 +1,254 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
/*
* JVM tests for lyng.io.fs module bindings.
*/
package net.sergeych.lyngio.fs
import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.ExecutionError
import net.sergeych.lyng.Scope
import net.sergeych.lyng.io.fs.createFs
import net.sergeych.lyng.obj.*
import net.sergeych.lyngio.fs.security.*
import kotlin.io.path.*
import kotlin.test.*
class FsModuleJvmTest {
private fun newScope(): Scope = Scope.new()
private suspend fun importFs(scope: Scope) {
// Ensure module is installed and imported into the scope
val installed = createFs(PermitAllAccessPolicy, scope)
// ok to be false if already installed for this manager, but in tests we create a fresh scope
scope.eval("import lyng.io.fs")
}
private suspend fun pathObj(scope: Scope, path: String): Obj {
val pathClass = scope.get("Path")!!.value as ObjClass
return pathClass.callWithArgs(scope, ObjString(path))
}
@Test
fun installerIdempotence() = runBlocking {
val scope = newScope()
val m = scope.importManager
assertTrue(createFs(PermitAllAccessPolicy, m))
assertFalse(createFs(PermitAllAccessPolicy, m))
}
@Test
fun basicReadWriteUtf8AndExists() = runBlocking {
val scope = newScope()
importFs(scope)
val tmpDir = kotlin.io.path.createTempDirectory("lyngio_test_")
try {
val file = tmpDir.resolve("hello.txt")
val p = pathObj(scope, file.toString())
// Initially doesn't exist
assertFalse(p.invokeInstanceMethod(scope, "exists").toBool())
// Write and read
p.invokeInstanceMethod(scope, "writeUtf8", ObjString("Hello Lyng!"))
assertTrue(p.invokeInstanceMethod(scope, "exists").toBool())
val read = p.invokeInstanceMethod(scope, "readUtf8") as ObjString
assertEquals("Hello Lyng!", read.value)
} finally {
tmpDir.toFile().deleteRecursively()
}
}
@Test
fun bytesReadWriteAndMetadata() = runBlocking {
val scope = newScope()
importFs(scope)
val tmpDir = kotlin.io.path.createTempDirectory("lyngio_test_bytes_")
try {
val file = tmpDir.resolve("data.bin")
val dirPath = pathObj(scope, tmpDir.toString())
val p = pathObj(scope, file.toString())
// mkdirs on parent dir is no-op but should succeed
dirPath.invokeInstanceMethod(scope, "mkdirs")
val bytes = (0 until 256).map { it.toByte() }.toByteArray().asUByteArray()
p.invokeInstanceMethod(scope, "writeBytes", ObjBuffer(bytes))
val readBuf = p.invokeInstanceMethod(scope, "readBytes") as ObjBuffer
assertContentEquals(bytes.asByteArray(), readBuf.byteArray.asByteArray())
val meta = p.invokeInstanceMethod(scope, "metadata") as ObjMap
assertEquals(ObjBool(true), meta.map[ObjString("isFile")])
assertEquals(ObjBool(false), meta.map[ObjString("isDirectory")])
} finally {
tmpDir.toFile().deleteRecursively()
}
}
@Test
fun listAndGlob() = runBlocking {
val scope = newScope()
importFs(scope)
val tmpDir = kotlin.io.path.createTempDirectory("lyngio_test_glob_")
try {
// prepare dirs/files
val sub1 = tmpDir.resolve("a"); sub1.createDirectories()
val sub2 = tmpDir.resolve("a/b"); sub2.createDirectories()
val f1 = tmpDir.resolve("a/one.txt"); f1.writeText("1")
val f2 = tmpDir.resolve("a/b/two.txt"); f2.writeText("2")
val f3 = tmpDir.resolve("a/b/three.md"); f3.writeText("3")
val root = pathObj(scope, sub1.toString())
// list should contain children: one.txt and b
val list = root.invokeInstanceMethod(scope, "list") as ObjList
val names = list.list.map { it.toString() }
assertTrue(names.any { it.endsWith("one.txt") })
assertTrue(names.any { it.endsWith("b") })
// glob **/*.txt
val matches = root.invokeInstanceMethod(scope, "glob", ObjString("**/*.txt")) as ObjList
val mnames = matches.list.map { it.toString() }
assertTrue(mnames.any { it.endsWith("one.txt") })
assertTrue(mnames.any { it.endsWith("two.txt") })
assertFalse(mnames.any { it.endsWith("three.md") })
} finally {
tmpDir.toFile().deleteRecursively()
}
}
@Test
fun copyMoveDelete() = runBlocking {
val scope = newScope()
importFs(scope)
val tmpDir = kotlin.io.path.createTempDirectory("lyngio_test_cmd_")
try {
val f1 = tmpDir.resolve("src.txt"); f1.writeText("abc")
val f2 = tmpDir.resolve("dst.txt")
val p1 = pathObj(scope, f1.toString())
val p2 = pathObj(scope, f2.toString())
// copy
p1.invokeInstanceMethod(scope, "copy", p2)
assertTrue(f2.exists())
assertEquals("abc", f2.readText())
// move with overwrite
val f3 = tmpDir.resolve("moved.txt"); f3.writeText("x")
val p3 = pathObj(scope, f3.toString())
p2.invokeInstanceMethod(scope, "move", p3, ObjBool(true))
assertFalse(f2.exists())
assertEquals("abc", f3.readText())
// delete
p3.invokeInstanceMethod(scope, "delete")
assertFalse(f3.exists())
} finally {
tmpDir.toFile().deleteRecursively()
}
}
@Test
fun streamingReaders() = runBlocking {
val scope = newScope()
importFs(scope)
val tmpDir = kotlin.io.path.createTempDirectory("lyngio_test_stream_")
try {
val large = ByteArray(2 * 1024 * 1024) { (it % 251).toByte() }
val file = tmpDir.resolve("big.bin"); file.writeBytes(large)
val p = pathObj(scope, file.toString())
// readChunks
val it = p.invokeInstanceMethod(scope, "readChunks", ObjInt(131072)) // 128KB
var total = 0
val iter = it.invokeInstanceMethod(scope, "iterator")
while (iter.invokeInstanceMethod(scope, "hasNext").toBool()) {
val chunk = iter.invokeInstanceMethod(scope, "next") as ObjBuffer
total += chunk.byteArray.size
if (total >= 512 * 1024) {
// test cancellation early
iter.invokeInstanceMethod(scope, "cancelIteration")
break
}
}
assertTrue(total >= 512 * 1024)
// readUtf8Chunks + lines
val text = buildString {
repeat(10000) { append("line-").append(it).append('\n') }
}
val tf = tmpDir.resolve("big.txt"); tf.writeText(text)
val tp = pathObj(scope, tf.toString())
val lit = tp.invokeInstanceMethod(scope, "lines")
val lines = mutableListOf<String>()
val lIter = lit.invokeInstanceMethod(scope, "iterator")
while (lIter.invokeInstanceMethod(scope, "hasNext").toBool()) {
val s = lIter.invokeInstanceMethod(scope, "next") as ObjString
lines += s.value
}
assertEquals(10000, lines.size)
assertEquals("line-0", lines.first())
assertEquals("line-9999", lines.last())
} finally {
tmpDir.toFile().deleteRecursively()
}
}
@Test
fun forbiddenOperationsRaiseLyngErrors() = runBlocking {
// Policy that denies everything except read and list
val denyWrites = object : FsAccessPolicy {
override suspend fun check(op: AccessOp, ctx: AccessContext): AccessDecision = when (op) {
is AccessOp.OpenRead, is AccessOp.ListDir -> AccessDecision(Decision.Allow)
else -> AccessDecision(Decision.Deny, reason = "denied by test policy")
}
}
// Prepare a file with PermitAll first (separate manager/scope)
val prepScope = newScope()
importFs(prepScope)
val tmpDir = kotlin.io.path.createTempDirectory("lyngio_test_deny_")
try {
val file = tmpDir.resolve("ro.txt"); file.writeText("ro")
// New scope with denying policy
val scope = newScope()
assertTrue(net.sergeych.lyng.io.fs.createFs(denyWrites, scope))
scope.eval("import lyng.io.fs")
val p = pathObj(scope, file.toString())
// Read should work
val text = p.invokeInstanceMethod(scope, "readUtf8") as ObjString
assertEquals("ro", text.value)
// Write should throw ExecutionError with ObjIllegalOperationException
val err = assertFailsWith<ExecutionError> {
p.invokeInstanceMethod(scope, "writeUtf8", ObjString("x"))
}
assertTrue(err.errorObject is ObjIllegalOperationException)
} finally {
tmpDir.toFile().deleteRecursively()
}
}
}
@@ -0,0 +1,25 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.fs
import kotlinx.coroutines.CoroutineDispatcher
import kotlinx.coroutines.Dispatchers
import okio.FileSystem
actual fun platformAsyncFs(): LyngFs = OkioAsyncFs(FileSystem.SYSTEM)
actual val LyngIoDispatcher: CoroutineDispatcher = Dispatchers.Default
@@ -0,0 +1,25 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyngio.fs
import kotlinx.coroutines.CoroutineDispatcher
import kotlinx.coroutines.Dispatchers
import okio.fakefilesystem.FakeFileSystem
actual fun platformAsyncFs(): LyngFs = OkioAsyncFs(FakeFileSystem())
actual val LyngIoDispatcher: CoroutineDispatcher = Dispatchers.Default
@@ -494,9 +494,25 @@ class Compiler(
when (t.value) { when (t.value) {
in stopKeywords -> { in stopKeywords -> {
if (operand != null) throw ScriptError(t.pos, "unexpected keyword") if (operand != null) throw ScriptError(t.pos, "unexpected keyword")
cc.previous() // Allow certain statement-like constructs to act as expressions
val s = parseStatement() ?: throw ScriptError(t.pos, "Expecting valid statement") // when they appear in expression position (e.g., `if (...) ... else ...`).
operand = StatementRef(s) // Other keywords should be handled by the outer statement parser.
when (t.value) {
"if" -> {
val s = parseIfStatement()
operand = StatementRef(s)
}
"when" -> {
val s = parseWhenStatement()
operand = StatementRef(s)
}
else -> {
// Do not consume the keyword as part of a term; backtrack
// and return null so outer parser handles it.
cc.previous()
return null
}
}
} }
"else", "break", "continue" -> { "else", "break", "continue" -> {
@@ -2922,7 +2938,11 @@ class Compiler(
// assigner.generate(context.pos, left, right) // assigner.generate(context.pos, left, right)
// } // }
val lastLevel = lastPriority + 1 // Compute levels from the actual operator table rather than relying on
// the mutable construction counter. This prevents accidental inflation
// of precedence depth that could lead to deep recursive descent and
// StackOverflowError during parsing.
val lastLevel = (allOps.maxOf { it.priority }) + 1
val byLevel: List<Map<Token.Type, Operator>> = (0..<lastLevel).map { l -> val byLevel: List<Map<Token.Type, Operator>> = (0..<lastLevel).map { l ->
allOps.filter { it.priority == l }.associateBy { it.tokenType } allOps.filter { it.priority == l }.associateBy { it.tokenType }
@@ -14,8 +14,8 @@
* limitations under the License. * limitations under the License.
* *
*/ */
package net.sergeych.lyng.stdlib_included package net.sergeych.lyng.stdlib_included
internal val rootLyng = """ internal val rootLyng = """
package lyng.stdlib package lyng.stdlib
@@ -123,6 +123,26 @@ fun Iterable.sumOf(f) {
else null else null
} }
fun Iterable.minOf( lambda ) {
val i = iterator()
var minimum = lambda( i.next() )
while( i.hasNext() ) {
val x = lambda(i.next())
if( x < minimum ) minimum = x
}
minimum
}
fun Iterable.maxOf( lambda ) {
val i = iterator()
var maximum = lambda( i.next() )
while( i.hasNext() ) {
val x = lambda(i.next())
if( x > maximum ) maximum = x
}
maximum
}
fun Iterable.sorted() { fun Iterable.sorted() {
sortedWith { a, b -> a <=> b } sortedWith { a, b -> a <=> b }
} }
@@ -3667,6 +3667,48 @@ class ScriptTest {
} }
@Test
fun testIterableMinMax() = runTest {
eval("""
import lyng.stdlib
assertEquals( -100, (1..100).toList().minOf { -it } )
assertEquals( -1, (1..100).toList().maxOf { -it } )
""".trimIndent()
)
}
@Test
fun testParserOverflow() = runTest {
try {
eval(
"""
fun Iterable.minByWithIndex( lambda ) {
val i = iterator()
var index = 0
if( !i.hasNext() ) return null
var value = i.next()
var n = 1
while( i.hasNext() ) {
val x = lambda(i.next())
if( x < value ) {
index = n
value = x
}
n++
}
index => value
}
""".trimIndent()
)
// If it compiles fine, great. The test passes implicitly.
} catch (e: ScriptError) {
// The important part: no StackOverflowError anymore, but a meaningful ScriptError
// is thrown. Accept any ScriptError as a valid outcome.
println(e.message)
}
}
// @Test // @Test
// fun namedArgsProposal() = runTest { // fun namedArgsProposal() = runTest {
// eval(""" // eval("""
+1
View File
@@ -38,3 +38,4 @@ include(":lynglib")
include(":lyng") include(":lyng")
include(":site") include(":site")
include(":lyngweb") include(":lyngweb")
include(":lyngio")
+9 -4
View File
@@ -379,8 +379,9 @@ suspend fun initDocsDropdown() {
val arr = JSON.parse(text) as Array<String> val arr = JSON.parse(text) as Array<String>
val all = arr.toList().sorted() val all = arr.toList().sorted()
dlog("docs-dd", "index entries=${all.size}") dlog("docs-dd", "index entries=${all.size}")
// Filter excluded by reading each markdown and looking for the marker // Filter excluded by reading each markdown and looking for the marker, and collect titles
val filtered = mutableListOf<String>() val filtered = mutableListOf<String>()
val titles = mutableMapOf<String, String>()
var excluded = 0 var excluded = 0
var failed = 0 var failed = 0
for (path in all) { for (path in all) {
@@ -393,8 +394,12 @@ suspend fun initDocsDropdown() {
continue continue
} }
val body = r.text().await() val body = r.text().await()
if (!body.contains("[//]: # (excludeFromIndex)")) { if (!body.contains("[//]: # (excludeFromIndex)") &&
body.contains("[//]: # (topMenu)") ) {
filtered.add(path) filtered.add(path)
// Reuse shared title extractor: first H1 or fallback to file name
val title = extractTitleFromMarkdown(body) ?: path.substringAfterLast('/')
titles[path] = title
} else excluded++ } else excluded++
} catch (t: Throwable) { } catch (t: Throwable) {
failed++ failed++
@@ -404,10 +409,10 @@ suspend fun initDocsDropdown() {
dlog("docs-dd", "filtered=${filtered.size} excluded=$excluded failed=$failed") dlog("docs-dd", "filtered=${filtered.size} excluded=$excluded failed=$failed")
// Sort entries by display name (file name) case-insensitively // Sort entries by display name (file name) case-insensitively
val sortedFiltered = filtered.sortedWith(compareBy(String.CASE_INSENSITIVE_ORDER) { it.substringAfterLast('/') }) val sortedFiltered = filtered.sortedWith(compareBy(String.CASE_INSENSITIVE_ORDER) { it.substringAfterLast('/') })
// Build items after the static first two existing children (tutorial + divider) // Build items after the static first items (tutorial, optional static entries, divider)
var appended = 0 var appended = 0
sortedFiltered.forEach { path -> sortedFiltered.forEach { path ->
val name = path.substringAfterLast('/') val name = titles[path] ?: path.substringAfterLast('/')
val li = document.createElement("li") val li = document.createElement("li")
val a = document.createElement("a") as HTMLAnchorElement val a = document.createElement("a") as HTMLAnchorElement
a.className = "dropdown-item" a.className = "dropdown-item"
@@ -0,0 +1,15 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- Lyng favicon: scalable SVG that adapts to light/dark themes -->
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" role="img" aria-label="Lyng favicon">
<style>
:root { color-scheme: light dark; color: #000; }
@media (prefers-color-scheme: dark) { :root { color: #fff; } }
.mark { fill: currentColor; }
.math { font-family: 'STIX Two Math', 'Cambria Math', 'Times New Roman', serif; }
</style>
<g class="mark math">
<!-- Keep favicon legible: lambda with superscript y only -->
<text x="3" y="17" font-size="16" font-weight="700">λ</text>
<text x="11.2" y="8" font-size="9">y</text>
</g>
</svg>

After

Width:  |  Height:  |  Size: 685 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.7 KiB

@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" role="img" aria-label="Lyng mark: lambda with superscript y">
<style>
:root { color-scheme: light dark; color: #000; }
@media (prefers-color-scheme: dark) { :root { color: #fff; } }
.mark { fill: #fff; }
.math { font-family: 'STIX Two Math', 'Cambria Math', 'Times New Roman', serif; }
</style>
<g class="mark math">
<text x="3" y="17" font-size="16" font-weight="700">λ</text>
<text x="11.2" y="8" font-size="9">y</text>
</g>
</svg>

After

Width:  |  Height:  |  Size: 566 B

+16 -4
View File
@@ -21,6 +21,8 @@
<meta charset="utf-8" /> <meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Lyng language</title> <title>Lyng language</title>
<!-- Site favicon (SVG, adapts to light/dark) -->
<link rel="icon" type="image/svg+xml" href="icons/lyng-favicon.svg" />
<!-- GitHub Markdown CSS (light and dark). We toggle these from the app. --> <!-- GitHub Markdown CSS (light and dark). We toggle these from the app. -->
<link <link
id="md-light" id="md-light"
@@ -189,7 +191,15 @@
<nav class="navbar navbar-expand-lg bg-body-tertiary fixed-top border-bottom" role="navigation" aria-label="Primary"> <nav class="navbar navbar-expand-lg bg-body-tertiary fixed-top border-bottom" role="navigation" aria-label="Primary">
<div class="container-fluid"> <div class="container-fluid">
<a class="navbar-brand d-flex align-items-center gap-2" href="#"> <a class="navbar-brand d-flex align-items-center gap-2" href="#">
<i class="bi bi-braces-asterisk"></i> <!-- Inline SVG brand icon: λ with superscript y and subscript ng. Uses currentColor to match theme. -->
<svg xmlns="http://www.w3.org/2000/svg" width="30" height="30" viewBox="0 0 24 24" role="img" aria-hidden="true">
<g fill="currentColor" color="inherit" font-family="'STIX Two Math', 'Cambria Math', 'Times New Roman', serif">
<!-- Main lambda -->
<text x="3" y="17" font-size="16" font-weight="700">λ</text>
<!-- Superscript y -->
<text x="11.2" y="8" font-size="10">y</text>
</g>
</svg>
Lyng Lyng
</a> </a>
<button <button
@@ -212,12 +222,14 @@
</a> </a>
<ul id="docsDropdownMenu" class="dropdown-menu"> <ul id="docsDropdownMenu" class="dropdown-menu">
<!-- Will be populated at runtime --> <!-- Will be populated at runtime -->
<li><a class="dropdown-item" href="#/docs/tutorial.md" data-route="docs">tutorial</a></li> <li><a class="dropdown-item" href="#/docs/tutorial.md" data-route="docs">Tutorial</a></li>
<li><hr class="dropdown-divider"></li> <li><a class="dropdown-item" href="#/reference" data-route="reference">Reference</a></li>
<li><hr class="dropdown-divider"></li>
</ul> </ul>
</li> </li>
<li class="nav-item"> <li class="nav-item">
<a class="nav-link" href="#/reference" data-route="reference">Reference</a> <a class="nav-link" href="#/tryling" data-route="tryling">Try in browser</a>
</li> </li>
</ul> </ul>
+11
View File
@@ -0,0 +1,11 @@
# robots.txt for the Lyng site
# Basic, safe policy: allow all well-behaved crawlers to access the site.
User-agent: *
Allow: /
# Optional crawl delay for polite bots (Google ignores this directive)
Crawl-delay: 5
# Add your sitemap URL once available, e.g.:
# Sitemap: https://your-domain.example/sitemap.xml