Compare commits

...
Author SHA1 Message Date
sergeych 9e11519608 revert to wirking ugly fix for decodeClassObj 2025-12-13 23:11:28 +01:00
sergeych fba44622e5 Refactor toString implementations to support Scope context, add inspect, and improve assertions readability. 2025-12-13 13:48:57 +01:00
sergeych 2737aaa14e add mapNotNull to ObjIterable with documentation 2025-12-12 13:48:02 +01:00
sergeych bce88ced43 Merge pull request 'fix/scope-parent-cycle' (#92) from fix/scope-parent-cycle into master
Reviewed-on: #92
2025-12-11 03:09:45 +03:00
sergeych fd473a32d8 refine some tests 2025-12-11 00:55:05 +01:00
sergeych d15dfb6087 core: prevent scope parent-chain cycles when reusing pooled frames
- Scope.resetForReuse: fully detach before re-parenting (clear state, parent=null, new frameId), then validate with ensureNoCycle and assign parent/args/pos/thisObj
- ScopePool.borrow (JVM/Android/JS/Native/Wasm): defensive fallback to fresh Scope allocation if resetForReuse detects a cycle
- docs: add docs/fix-scope-parent-cycle.md describing the change and expectations
- test: add ScopeCycleRegressionTest to ensure instance method call pattern does not crash and returns "ok"
2025-12-11 00:50:46 +01:00
sergeych b953282251 docs on updated scopes 2025-12-10 00:04:07 +01:00
sergeych bcabfc8962 fix endless recursion in scope resolution in some specific cases 2025-12-09 23:55:50 +01:00
sergeych c0fab3d60e bump version to 1.0.7-SNAPSHOT; fix potential infinite loops in Scope traversal 2025-12-09 07:33:05 +01:00
sergeych 55caa65f97 fixed dependencies for plugin building 2025-12-08 18:59:16 +01:00
sergeych b73891d19b ref $91 trace removed 2025-12-08 18:52:38 +01:00
sergeych 8750040926 fix #91 syntax highlighting of kotlin code blocks on the site 2025-12-08 18:51:55 +01:00
sergeych 708b908415 fix #88 fix #90 plugin with autocompletion 2025-12-08 03:28:27 +01:00
sergeych c35d684df1 started autocompletion in the plugin 2025-12-07 23:06:05 +01:00
sergeych 678cfbf45e fix #85 lynon empty list encoding 2025-12-07 21:50:47 +01:00
sergeych bfffea7e69 fix #83 import-aware quickdocs 2025-12-06 22:25:21 +01:00
sergeych 40f11b6f29 fix #82 refactored and added builtin docs to all symbols 2025-12-06 21:10:40 +01:00
sergeych e25fc95cbf fix #81 site search improved 2025-12-06 18:03:15 +01:00
75 changed files with 5060 additions and 600 deletions

No files matched your search

@@ -0,0 +1,28 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Tests in 'lyng.lynglib.jvmTest'" type="GradleRunConfiguration" factoryName="Gradle">
<ExternalSystemSettings>
<option name="executionName" />
<option name="externalProjectPath" value="$PROJECT_DIR$" />
<option name="externalSystemIdString" value="GRADLE" />
<option name="scriptParameters" value="" />
<option name="taskDescriptions">
<list />
</option>
<option name="taskNames">
<list>
<option value=":lynglib:cleanJvmTest" />
<option value=":lynglib:jvmTest" />
</list>
</option>
<option name="vmOptions" />
</ExternalSystemSettings>
<ExternalSystemDebugServerProcess>false</ExternalSystemDebugServerProcess>
<ExternalSystemReattachDebugProcess>true</ExternalSystemReattachDebugProcess>
<ExternalSystemDebugDisabled>false</ExternalSystemDebugDisabled>
<DebugAllEnabled>false</DebugAllEnabled>
<RunAsTest>true</RunAsTest>
<GradleProfilingDisabled>false</GradleProfilingDisabled>
<GradleCoverageDisabled>false</GradleCoverageDisabled>
<method v="2" />
</configuration>
</component>
+14
View File
@@ -2,6 +2,20 @@
### Unreleased
- Docs: Scopes and Closures guidance
- New page: `docs/scopes_and_closures.md` detailing `ClosureScope` resolution order, recursion‑safe helpers (`chainLookupIgnoreClosure`, `chainLookupWithMembers`, `baseGetIgnoreClosure`), cycle prevention, and capturing lexical environments for callbacks (`snapshotForClosure`).
- Updated: `docs/advanced_topics.md` (link to the new page), `docs/parallelism.md` (closures in `launch`/`flow`), `docs/OOP.md` (visibility from closures with preserved `currentClassCtx`), `docs/exceptions_handling.md` (compatibility alias `SymbolNotFound`).
- Tutorial: added quick link to Scopes and Closures.
- IDEA plugin: Lightweight autocompletion (experimental)
- Global completion: local declarations, in‑scope parameters, imported modules, and stdlib symbols.
- Member completion: after a dot, suggests only members of the inferred receiver type (incl. chained calls like `Path(".." ).lines().` → `Iterator` methods). No global identifiers appear after a dot.
- Inheritance-aware: direct class members first, then inherited (e.g., `List` includes `Collection`/`Iterable` methods).
- Heuristics: handles literals (`"…"` → `String`, numbers → `Int/Real`, `[...]` → `List`, `{...}` → `Dict`) and static `Namespace.` members.
- Performance: capped results, early prefix filtering, per‑document MiniAst cache, cancellation checks.
- Toggle: Settings | Lyng Formatter → "Enable Lyng autocompletion (experimental)" (default ON).
- Stabilization: DEBUG completion/Quick Doc logs are OFF by default; behavior aligned between IDE and isolated engine tests.
- Language: Named arguments and named splats
- New call-site syntax for named arguments using colon: `name: value`.
- Positional arguments must come before named; positionals after a named argument inside parentheses are rejected.
+22
View File
@@ -117,6 +117,28 @@ scope.eval("sumOf(1,2,3)") // <- 6
```
Note that the scope stores all changes in it so you can make calls on a single scope to preserve state between calls.
## IntelliJ IDEA plugin: Lightweight autocompletion (experimental)
The IDEA plugin provides a fast, lightweight BASIC completion for Lyng code (IntelliJ IDEA 2024.3+).
What it does:
- Global suggestions: in-scope parameters, same-file declarations (functions/classes/vals), imported modules, and stdlib symbols.
- Member completion after dot: offers only members of the inferred receiver type. It works for chained calls like `Path(".." ).lines().` (suggests `Iterator` methods), and for literals like `"abc".` (String methods) or `[1,2,3].` (List/Iterable methods).
- Inheritance-aware: shows direct class members first, then inherited. For example, `List` also exposes common `Collection`/`Iterable` methods.
- Static/namespace members: `Name.` lists only static members when `Name` is a known class or container (e.g., `Math`).
- Performance: suggestions are capped; prefix filtering is early; parsing is cached; computation is cancellation-friendly.
What it does NOT do (yet):
- No heavy resolve or project-wide indexing. It’s best-effort, driven by a tiny MiniAst + built-in docs registry.
- No control/data-flow type inference.
Enable/disable:
- Settings | Lyng Formatter → "Enable Lyng autocompletion (experimental)" (default: ON).
Tips:
- After a dot, globals are intentionally suppressed (e.g., `lines().Path` is not valid), only the receiver’s members are suggested.
- If completion seems sparse, make sure related modules are imported (e.g., `import lyng.io.fs` so that `Path` and its methods are known).
## Why?
Designed to add scripting to kotlin multiplatform application in easy and efficient way. This is attempt to achieve what Lua is for C/++.
+9
View File
@@ -20,3 +20,12 @@ plugins {
alias(libs.plugins.kotlinMultiplatform) apply false
alias(libs.plugins.vanniktech.mavenPublish) apply false
}
// Convenience alias to run the IntelliJ IDE with the Lyng plugin from the project root.
// Usage: ./gradlew runIde
// It simply delegates to :lyng-idea:runIde provided by the Gradle IntelliJ Plugin.
tasks.register<org.gradle.api.DefaultTask>("runIde") {
group = "intellij"
description = "Run IntelliJ IDEA with the Lyng plugin (:lyng-idea)"
dependsOn(":lyng-idea:runIde")
}
+14
View File
@@ -68,6 +68,20 @@ These, again, does the thing:
>>> void
## map and mapNotNull
Used to transform either the whole iterable stream or also skipping som elements from it:
val source = [1,2,3,4]
// transform every element to string or null:
assertEquals(["n1", "n2", null, "n4"], source.map { if( it == 3 ) null else "n"+it } )
// transform every element to stirng, skipping 3:
assertEquals(["n1", "n2", "n4"], source.mapNotNull { if( it == 3 ) null else "n"+it } )
>>> void
## Instance methods:
| fun/method | description |
+5
View File
@@ -604,3 +604,8 @@ Regular methods are called on instances as usual `instance.method()`. The method
TBD
[argument list](declaring_arguments.md)
### Visibility from within closures and instance scopes
When a closure executes within a method, the closure retains the lexical class context of its creation site. This means private/protected members of that class remain accessible where expected (subject to usual visibility rules). Field resolution checks the declaring class and validates access using the preserved `currentClassCtx`.
See also: [Scopes and Closures: resolution and safety](scopes_and_closures.md)
+6
View File
@@ -158,3 +158,9 @@ Function annotation can have more args specified at call time. There arguments m
>>> void
[parallelism]: parallelism.md
## Scopes and Closures: resolution and safety
Closures and dynamic scope graphs require care to avoid accidental recursion and to keep name resolution predictable. See the dedicated page for detailed rules, helper APIs, and best practices:
- Scopes and Closures: resolution and safety → [scopes_and_closures.md](scopes_and_closures.md)
+15
View File
@@ -169,3 +169,18 @@ _this functionality is not yet released_
| UnknownException | unexpected kotlin exception caught |
| | |
### Symbol resolution errors
For compatibility, `SymbolNotFound` is an alias of `SymbolNotDefinedException`. You can catch either name in examples and tests.
Example:
```lyng
try {
nonExistingMethod()
}
catch(e: SymbolNotFound) {
// handle
}
```
+34
View File
@@ -0,0 +1,34 @@
## Fix: prevent cycles in scope parent chain during pooled frame reuse
### What changed
- Scope.resetForReuse now fully detaches a reused scope from its previous chain/state before re-parenting:
- sets `parent = null` and regenerates `frameId`
- clears locals/slots/bindings caches
- only after that, validates the new parent with `ensureNoCycle` and assigns it
- ScopePool.borrow on all targets (JVM, Android, JS, Native, Wasm) now has a defensive fallback:
- if `resetForReuse` throws `IllegalStateException` indicating a parent-chain cycle, the pool allocates a fresh `Scope` instead of failing.
### Why
In some nested call patterns (notably instance method calls where an instance is produced by another function and immediately used), the same pooled `Scope` object can be rebound into a chain that already (transitively) contains it. Reassigning `parent` in that case forms a structural cycle, which `ensureNoCycle` correctly detects and throws. This could surface as:
```
IllegalStateException: cycle detected in scope parent chain assignment
at net.sergeych.lyng.Scope.ensureNoCycle(...)
at net.sergeych.lyng.Scope.resetForReuse(...)
at net.sergeych.lyng.ScopePool.borrow(...)
... during instance method invocation
```
The fix removes the failure mode by:
1) Detaching the reused frame from its prior chain/state before validating and assigning the new parent.
2) Falling back to a new frame allocation if a cycle is still detected (extremely rare and cheap vs. a crash).
### Expected effects
- Eliminates sporadic `cycle detected in scope parent chain` crashes during instance method invocation.
- No change to public API or normal semantics.
- Pooling remains enabled by default; the fallback only triggers on the detected cycle edge case.
- Negligible performance impact: fresh allocation is used only when a cycle would have crashed the VM previously.
### Notes
- The fix is platform-wide (all ScopePool actuals are covered).
- We recommend adding/keeping a regression test that exercises: a class with a method, a function returning an instance, and an exported function calling the instance method. The test should pass without exceptions.
+19
View File
@@ -221,3 +221,22 @@ Lyng includes an optional optimization for function/method calls on JVM: scope f
- Expected effect (from our JVM micro‑benchmarks): in deep call loops, enabling pooling reduced total time by about 1.38× in a dedicated pooling benchmark; mileage may vary depending on workload.
Future work: introduce thread‑safe pooling (e.g., per‑thread pools or confinement strategies) before considering enabling it by default in multi‑threaded environments.
### Closures inside coroutine helpers (launch/flow)
Closures executed by `launch { ... }` and `flow { ... }` resolve names using the `ClosureScope` rules:
1. Closure frame locals/arguments
2. Captured receiver instance/class members
3. Closure ancestry locals + each frame’s `this` members (cycle‑safe)
4. Caller `this` members
5. Caller ancestry locals + each frame’s `this` members (cycle‑safe)
6. Module pseudo‑symbols (e.g., `__PACKAGE__`)
7. Direct module/global fallback (nearest `ModuleScope` and its parent/root)
Implications:
- Outer locals (e.g., `counter`) stay visible across suspension points.
- Global helpers like `delay(ms)` and `yield()` are available from inside closures.
- If you write your own async helpers, execute user lambdas under `ClosureScope(callScope, capturedCreatorScope)` and avoid manual ancestry walking.
See also: [Scopes and Closures: resolution and safety](scopes_and_closures.md)
+5
View File
@@ -7,6 +7,11 @@ val files = Path("../..").list().toList()
// most long is longest?
val longestNameLength = files.maxOf { it.name.length }
// testdoc
fun test() {
22
}
val format = "%"+(longestNameLength+1) +"s %d"
for( f in files )
+75
View File
@@ -0,0 +1,75 @@
# Scopes and Closures: resolution and safety
This page documents how name resolution works with `ClosureScope`, how to avoid recursion pitfalls, and how to safely capture and execute callbacks that need access to outer locals.
## Why this matters
Name lookup across nested scopes and closures can accidentally form recursive resolution paths or hide expected symbols (outer locals, module/global functions). The rules below ensure predictable resolution and prevent infinite recursion.
## Resolution order in ClosureScope
When evaluating an identifier `name` inside a closure, `ClosureScope.get(name)` resolves in this order:
1. Closure frame locals and arguments
2. Captured receiver (`closureScope.thisObj`) instance/class members
3. Closure ancestry locals + each frame’s `thisObj` members (cycle‑safe)
4. Caller `this` members
5. Caller ancestry locals + each frame’s `thisObj` members (cycle‑safe)
6. Module pseudo‑symbols (e.g., `__PACKAGE__`) from the nearest `ModuleScope`
7. Direct module/global fallback (nearest `ModuleScope` and its parent/root scope)
8. Final fallback: base local/parent lookup for the current frame
This preserves intuitive visibility (locals → captured receiver → closure chain → caller members → caller chain → module/root) while preventing infinite recursion between scope types.
## Use raw‑chain helpers for ancestry walks
When authoring new scope types or advanced lookups, avoid calling virtual `get` while walking parents. Instead, use the non‑dispatch helpers on `Scope`:
- `chainLookupIgnoreClosure(name)`
- Walk raw `parent` chain and check only per‑frame locals/bindings/slots.
- Ignores overridden `get` (e.g., in `ClosureScope`). Cycle‑safe.
- `chainLookupWithMembers(name)`
- Like above, but after locals/bindings it also checks each frame’s `thisObj` members.
- Ignores overridden `get`. Cycle‑safe.
- `baseGetIgnoreClosure(name)`
- For the current frame only: check locals/bindings, then walk raw parents (locals/bindings), then fallback to this frame’s `thisObj` members.
These helpers avoid ping‑pong recursion and make structural cycles harmless (lookups terminate).
## Preventing structural cycles
- Don’t construct parent chains that can point back to a descendant.
- A debug‑time guard throws if assigning a parent would create a cycle; keep it enabled for development builds.
- Even with a cycle, chain helpers break out via a small `visited` set keyed by `frameId`.
## Capturing lexical environments for callbacks
For dynamic objects or custom builders, capture the creator’s lexical scope so callbacks can see outer locals/parameters:
1. Use `snapshotForClosure()` on the caller scope to capture locals/bindings/slots and parent.
2. Store this snapshot and run callbacks under `ClosureScope(callScope, captured)`.
Kotlin sketch:
```kotlin
val captured = scope.snapshotForClosure()
val execScope = ClosureScope(currentCallScope, captured)
callback.execute(execScope)
```
This ensures expressions like `contractName` used inside dynamic `get { name -> ... }` resolve to outer variables defined at the creation site.
## Closures in coroutines (launch/flow)
- The closure frame still prioritizes its own locals/args.
- Outer locals declared before suspension points remain visible through slot‑aware ancestry lookups.
- Global functions like `delay(ms)` and `yield()` are resolved via module/root fallbacks from within closures.
Tip: If a closure unexpectedly cannot see an outer local, check whether an intermediate runtime helper introduced an extra call frame; the built‑in lookup already traverses caller ancestry, so prefer the standard helpers rather than custom dispatch.
## Local variable references and missing symbols
- Unqualified identifier resolution first prefers locals/bindings/slots before falling back to `this` members.
- If neither locals nor members contain the symbol, missing field lookups map to `SymbolNotFound` (compatibility alias for `SymbolNotDefinedException`).
## Performance notes
- The `visited` sets used for cycle detection are tiny and short‑lived; in typical scripts the overhead is negligible.
- If profiling shows hotspots, consider limiting ancestry depth in your custom helpers or using small fixed arrays instead of hash sets—only for extremely hot code paths.
## Dos and Don’ts
- Do use `chainLookupIgnoreClosure` / `chainLookupWithMembers` for ancestry traversals.
- Do maintain the resolution order above for predictable behavior.
- Don’t call virtual `get` while walking parents; it risks recursion across scope types.
- Don’t attach instance scopes to transient/pool frames; bind to a stable parent scope instead.
+1 -1
View File
@@ -6,7 +6,7 @@ In other word, the code usually works as expected when you see it. So, nothing u
__Other documents to read__ maybe after this one:
- [Advanced topics](advanced_topics.md), [declaring arguments](declaring_arguments.md)
- [Advanced topics](advanced_topics.md), [declaring arguments](declaring_arguments.md), [Scopes and Closures](scopes_and_closures.md)
- [OOP notes](OOP.md), [exception handling](exceptions_handling.md)
- [math in Lyng](math.md), [the `when` statement](when.md)
- [time](time.md) and [parallelism](parallelism.md)
+1 -1
View File
@@ -34,7 +34,7 @@ kotlin.native.cacheKind.linuxX64=none
# On this environment, the system JDK 21 installation lacks `jlink`, causing
# :lynglib:androidJdkImage to fail. Point Gradle to a JDK that includes `jlink`.
# This affects only the JDK Gradle runs with; Kotlin/JVM target remains compatible.
org.gradle.java.home=/home/sergeych/.jdks/corretto-21.0.9
#org.gradle.java.home=/home/sergeych/.jdks/corretto-21.0.9
android.experimental.lint.migrateToK2=false
android.lint.useK2Uast=false
kotlin.mpp.applyDefaultHierarchyTemplate=true
+6
View File
@@ -37,8 +37,14 @@ repositories {
dependencies {
implementation(project(":lynglib"))
// Include lyngio so Quick Docs can reflectively load fs docs registrar (FsBuiltinDocs)
implementation(project(":lyngio"))
// Rich Markdown renderer for Quick Docs
implementation("com.vladsch.flexmark:flexmark-all:0.64.8")
// Tests for IntelliJ Platform fixtures rely on JUnit 3/4 API (junit.framework.TestCase)
// Add JUnit 4 which contains the JUnit 3 compatibility classes used by BasePlatformTestCase/UsefulTestCase
testImplementation("junit:junit:4.13.2")
}
intellij {
@@ -0,0 +1,934 @@
/*
* Lightweight BASIC completion for Lyng, MVP version.
* Uses MiniAst (best-effort) + BuiltinDocRegistry to suggest symbols.
*/
package net.sergeych.lyng.idea.completion
import com.intellij.codeInsight.completion.*
import com.intellij.codeInsight.lookup.LookupElementBuilder
import com.intellij.icons.AllIcons
import com.intellij.openapi.diagnostic.Logger
import com.intellij.openapi.editor.Document
import com.intellij.openapi.util.Key
import com.intellij.patterns.PlatformPatterns
import com.intellij.psi.PsiFile
import com.intellij.util.ProcessingContext
import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.Source
import net.sergeych.lyng.highlight.offsetOf
import net.sergeych.lyng.idea.LyngLanguage
import net.sergeych.lyng.idea.settings.LyngFormatterSettings
import net.sergeych.lyng.idea.util.DocsBootstrap
import net.sergeych.lyng.idea.util.IdeLenientImportProvider
import net.sergeych.lyng.idea.util.TextCtx
import net.sergeych.lyng.miniast.*
class LyngCompletionContributor : CompletionContributor() {
init {
extend(
CompletionType.BASIC,
PlatformPatterns.psiElement().withLanguage(LyngLanguage),
Provider
)
}
private object Provider : CompletionProvider<CompletionParameters>() {
private val log = Logger.getInstance(LyngCompletionContributor::class.java)
private const val DEBUG_COMPLETION = false
override fun addCompletions(
parameters: CompletionParameters,
context: ProcessingContext,
result: CompletionResultSet
) {
// Ensure external/bundled docs are registered (e.g., lyng.io.fs with Path)
DocsBootstrap.ensure()
// Ensure stdlib Obj*-defined docs (e.g., String methods via ObjString.addFnDoc) are initialized
StdlibDocsBootstrap.ensure()
val file: PsiFile = parameters.originalFile
if (file.language != LyngLanguage) return
// Feature toggle: allow turning completion off from settings
val settings = LyngFormatterSettings.getInstance(file.project)
if (!settings.enableLyngCompletionExperimental) return
val document: Document = file.viewProvider.document ?: return
val text = document.text
val caret = parameters.offset.coerceIn(0, text.length)
val prefix = TextCtx.prefixAt(text, caret)
val withPrefix = result.withPrefixMatcher(prefix).caseInsensitive()
// Emission with cap
val cap = 200
var added = 0
val emit: (com.intellij.codeInsight.lookup.LookupElement) -> Unit = { le ->
if (added < cap) {
withPrefix.addElement(le)
added++
}
}
// Determine if we are in member context (dot before caret or before word start)
val wordRange = TextCtx.wordRangeAt(text, caret)
val memberDotPos = (wordRange?.let { TextCtx.findDotLeft(text, it.startOffset) })
?: TextCtx.findDotLeft(text, caret)
if (DEBUG_COMPLETION) {
log.info("[LYNG_DEBUG] Completion: caret=$caret prefix='${prefix}' memberDotPos=${memberDotPos} file='${file.name}'")
}
// Build MiniAst (cached) for both global and member contexts to enable local class/val inference
val mini = buildMiniAstCached(file, text)
// Delegate computation to the shared engine to keep behavior in sync with tests
val engineItems = try {
runBlocking { CompletionEngineLight.completeSuspend(text, caret) }
} catch (t: Throwable) {
if (DEBUG_COMPLETION) log.warn("[LYNG_DEBUG] Engine completion failed: ${t.message}")
emptyList()
}
if (DEBUG_COMPLETION) {
val preview = engineItems.take(10).joinToString { it.name }
log.info("[LYNG_DEBUG] Engine items: count=${engineItems.size} preview=[${preview}]")
}
// If we are in member context and the engine produced nothing, try a guarded local fallback
if (memberDotPos != null && engineItems.isEmpty()) {
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Fallback: engine returned 0 in member context; trying local inference")
// Build imported modules from text (lenient) + stdlib; avoid heavy MiniAst here
val fromText = extractImportsFromText(text)
val imported = LinkedHashSet<String>().apply {
fromText.forEach { add(it) }
add("lyng.stdlib")
}.toList()
// Try inferring return/receiver class around the dot
val inferred =
// Prefer MiniAst-based inference (return type from member call or receiver type)
guessReturnClassFromMemberCallBeforeMini(mini, text, memberDotPos, imported)
?: guessReceiverClassViaMini(mini, text, memberDotPos, imported)
?:
guessReturnClassFromMemberCallBefore(text, memberDotPos, imported)
?: guessReturnClassFromTopLevelCallBefore(text, memberDotPos, imported)
?: guessReturnClassAcrossKnownCallees(text, memberDotPos, imported)
?: guessReceiverClass(text, memberDotPos, imported)
if (inferred != null) {
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Fallback inferred receiver/return class='$inferred' — offering its members")
offerMembers(emit, imported, inferred, sourceText = text, mini = mini)
return
} else {
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Fallback could not infer class; keeping list empty (no globals after dot)")
return
}
}
// In global context, add params in scope first (engine does not include them)
if (memberDotPos == null && mini != null) {
offerParamsInScope(emit, mini, text, caret)
}
// Render engine items
for (ci in engineItems) {
val builder = when (ci.kind) {
Kind.Function -> LookupElementBuilder.create(ci.name)
.withIcon(AllIcons.Nodes.Function)
.let { b -> if (!ci.tailText.isNullOrBlank()) b.withTailText(ci.tailText, true) else b }
.let { b -> if (!ci.typeText.isNullOrBlank()) b.withTypeText(ci.typeText, true) else b }
.withInsertHandler(ParenInsertHandler)
Kind.Method -> LookupElementBuilder.create(ci.name)
.withIcon(AllIcons.Nodes.Method)
.let { b -> if (!ci.tailText.isNullOrBlank()) b.withTailText(ci.tailText, true) else b }
.let { b -> if (!ci.typeText.isNullOrBlank()) b.withTypeText(ci.typeText, true) else b }
.withInsertHandler(ParenInsertHandler)
Kind.Class_ -> LookupElementBuilder.create(ci.name)
.withIcon(AllIcons.Nodes.Class)
Kind.Value -> LookupElementBuilder.create(ci.name)
.withIcon(AllIcons.Nodes.Field)
.let { b -> if (!ci.typeText.isNullOrBlank()) b.withTypeText(ci.typeText, true) else b }
Kind.Field -> LookupElementBuilder.create(ci.name)
.withIcon(AllIcons.Nodes.Field)
.let { b -> if (!ci.typeText.isNullOrBlank()) b.withTypeText(ci.typeText, true) else b }
}
emit(builder)
}
// In member context, ensure stdlib extension-like methods (e.g., String.re) are present
if (memberDotPos != null) {
val existing = engineItems.map { it.name }.toMutableSet()
val fromText = extractImportsFromText(text)
val imported = LinkedHashSet<String>().apply {
fromText.forEach { add(it) }
add("lyng.stdlib")
}.toList()
val inferredClass =
guessReturnClassFromMemberCallBeforeMini(mini, text, memberDotPos, imported)
?: guessReceiverClassViaMini(mini, text, memberDotPos, imported)
?: guessReturnClassFromMemberCallBefore(text, memberDotPos, imported)
?: guessReturnClassFromTopLevelCallBefore(text, memberDotPos, imported)
?: guessReturnClassAcrossKnownCallees(text, memberDotPos, imported)
?: guessReceiverClass(text, memberDotPos, imported)
if (!inferredClass.isNullOrBlank()) {
val ext = BuiltinDocRegistry.extensionMethodNamesFor(inferredClass)
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Post-engine extension check for $inferredClass: ${'$'}{ext}")
for (name in ext) {
if (existing.contains(name)) continue
val resolved = DocLookupUtils.resolveMemberWithInheritance(imported, inferredClass, name)
if (resolved != null) {
when (val member = resolved.second) {
is MiniMemberFunDecl -> {
val params = member.params.joinToString(", ") { it.name }
val ret = typeOf(member.returnType)
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("(${ '$' }params)", true)
.withTypeText(ret, true)
.withInsertHandler(ParenInsertHandler)
emit(builder)
existing.add(name)
}
is MiniMemberValDecl -> {
val builder = LookupElementBuilder.create(name)
.withIcon(if (member.mutable) AllIcons.Nodes.Variable else AllIcons.Nodes.Field)
.withTypeText(typeOf(member.type), true)
emit(builder)
existing.add(name)
}
}
} else {
// Fallback: emit simple method name without detailed types
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("()", true)
.withInsertHandler(ParenInsertHandler)
emit(builder)
existing.add(name)
}
}
}
}
// If in member context and engine items are suspiciously sparse, try to enrich via local inference + offerMembers
if (memberDotPos != null && engineItems.size < 3) {
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Engine produced only ${engineItems.size} items in member context — trying enrichment")
val fromText = extractImportsFromText(text)
val imported = LinkedHashSet<String>().apply {
fromText.forEach { add(it) }
add("lyng.stdlib")
}.toList()
val inferred =
guessReturnClassFromMemberCallBeforeMini(mini, text, memberDotPos, imported)
?: guessReceiverClassViaMini(mini, text, memberDotPos, imported)
?: guessReturnClassFromMemberCallBefore(text, memberDotPos, imported)
?: guessReturnClassFromTopLevelCallBefore(text, memberDotPos, imported)
?: guessReturnClassAcrossKnownCallees(text, memberDotPos, imported)
?: guessReceiverClass(text, memberDotPos, imported)
if (inferred != null) {
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Enrichment inferred class='$inferred' — offering its members")
offerMembers(emit, imported, inferred, sourceText = text, mini = mini)
}
}
return
}
private fun offerDecl(emit: (com.intellij.codeInsight.lookup.LookupElement) -> Unit, d: MiniDecl) {
val name = d.name
val builder = when (d) {
is MiniFunDecl -> {
val params = d.params.joinToString(", ") { it.name }
val ret = typeOf(d.returnType)
val tail = "(${params})"
LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Function)
.withTailText(tail, true)
.withTypeText(ret, true)
.withInsertHandler(ParenInsertHandler)
}
is MiniClassDecl -> LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Class)
is MiniValDecl -> {
val kindIcon = if (d.mutable) AllIcons.Nodes.Variable else AllIcons.Nodes.Field
LookupElementBuilder.create(name)
.withIcon(kindIcon)
.withTypeText(typeOf(d.type), true)
}
else -> LookupElementBuilder.create(name)
}
emit(builder)
}
private object ParenInsertHandler : InsertHandler<com.intellij.codeInsight.lookup.LookupElement> {
override fun handleInsert(context: InsertionContext, item: com.intellij.codeInsight.lookup.LookupElement) {
val doc = context.document
val tailOffset = context.tailOffset
val nextChar = doc.charsSequence.getOrNull(tailOffset)
if (nextChar != '(') {
doc.insertString(tailOffset, "()")
context.editor.caretModel.moveToOffset(tailOffset + 1)
}
}
}
// --- Member completion helpers ---
private fun offerMembers(
emit: (com.intellij.codeInsight.lookup.LookupElement) -> Unit,
imported: List<String>,
className: String,
staticOnly: Boolean = false
, sourceText: String,
mini: MiniScript? = null
) {
// Ensure modules are seeded in the registry (triggers lazy stdlib build too)
for (m in imported) BuiltinDocRegistry.docsForModule(m)
val classes = DocLookupUtils.aggregateClasses(imported)
if (DEBUG_COMPLETION) {
val keys = classes.keys.joinToString(", ")
log.info("[LYNG_DEBUG] offerMembers: imported=${imported} classes=[${keys}] target=${className}")
}
val visited = mutableSetOf<String>()
// Collect separated to keep tiers: direct first, then inherited
val directMap = LinkedHashMap<String, MutableList<MiniMemberDecl>>()
val inheritedMap = LinkedHashMap<String, MutableList<MiniMemberDecl>>()
// 0) Prefer locally-declared class members (same-file) when available
val localClass = mini?.declarations?.filterIsInstance<MiniClassDecl>()?.firstOrNull { it.name == className }
if (localClass != null) {
for (m in localClass.members) {
val list = directMap.getOrPut(m.name) { mutableListOf() }
list.add(m)
}
// If MiniAst didn't populate members (empty), try to scan class body text for member signatures
if (localClass.members.isEmpty()) {
val scanned = scanLocalClassMembersFromText(mini, text = sourceText, cls = localClass)
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Local scan for class ${localClass.name}: found ${scanned.size} members -> ${scanned.keys}")
for ((name, sig) in scanned) {
when (sig.kind) {
"fun" -> {
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("(" + (sig.params?.joinToString(", ") ?: "") + ")", true)
.let { b -> sig.typeText?.let { b.withTypeText(": $it", true) } ?: b }
.withInsertHandler(ParenInsertHandler)
emit(builder)
}
"val", "var" -> {
val builder = LookupElementBuilder.create(name)
.withIcon(if (sig.kind == "var") AllIcons.Nodes.Variable else AllIcons.Nodes.Field)
.let { b -> sig.typeText?.let { b.withTypeText(": $it", true) } ?: b }
emit(builder)
}
}
}
}
}
fun addMembersOf(clsName: String, tierDirect: Boolean) {
val cls = classes[clsName] ?: return
val target = if (tierDirect) directMap else inheritedMap
for (m in cls.members) {
if (staticOnly) {
// Filter only static members in namespace/static context
when (m) {
is MiniMemberFunDecl -> if (!m.isStatic) continue
is MiniMemberValDecl -> if (!m.isStatic) continue
}
}
val list = target.getOrPut(m.name) { mutableListOf() }
list.add(m)
}
// Then inherited
for (base in cls.bases) {
if (visited.add(base)) addMembersOf(base, false)
}
}
visited.add(className)
addMembersOf(className, true)
if (DEBUG_COMPLETION) {
log.info("[LYNG_DEBUG] offerMembers: direct=${directMap.size} inherited=${inheritedMap.size} for ${className}")
}
// If the docs model lacks explicit bases for some core container classes,
// conservatively supplement with preferred parents to expose common ops.
fun supplementPreferredBases(receiver: String) {
// Preference/known lineage map kept tiny and safe
val extras = when (receiver) {
"List" -> listOf("Collection", "Iterable")
"Array" -> listOf("Collection", "Iterable")
// In practice, many high-level ops users expect on iteration live on Iterable.
// For editor assistance, expose Iterable ops for Iterator receivers too.
"Iterator" -> listOf("Iterable")
else -> emptyList()
}
for (base in extras) {
if (visited.add(base)) addMembersOf(base, false)
}
}
supplementPreferredBases(className)
fun emitGroup(map: LinkedHashMap<String, MutableList<MiniMemberDecl>>) {
val keys = map.keys.sortedBy { it.lowercase() }
for (name in keys) {
val list = map[name] ?: continue
// Choose a representative for display:
// 1) Prefer a method with a known return type
// 2) Else any method
// 3) Else the first variant
val rep =
list.asSequence()
.filterIsInstance<MiniMemberFunDecl>()
.firstOrNull { it.returnType != null }
?: list.firstOrNull { it is MiniMemberFunDecl }
?: list.first()
when (rep) {
is MiniMemberFunDecl -> {
val params = rep.params.joinToString(", ") { it.name }
val ret = typeOf(rep.returnType)
val extra = list.count { it is MiniMemberFunDecl } - 1
val overloads = if (extra > 0) " (+$extra overloads)" else ""
val tail = "(${params})$overloads"
val icon = AllIcons.Nodes.Method
val builder = LookupElementBuilder.create(name)
.withIcon(icon)
.withTailText(tail, true)
.withTypeText(ret, true)
.withInsertHandler(ParenInsertHandler)
emit(builder)
}
is MiniMemberValDecl -> {
val icon = if (rep.mutable) AllIcons.Nodes.Variable else AllIcons.Nodes.Field
// Prefer a field variant with known type if available
val chosen = list.asSequence()
.filterIsInstance<MiniMemberValDecl>()
.firstOrNull { it.type != null } ?: rep
val builder = LookupElementBuilder.create(name)
.withIcon(icon)
.withTypeText(typeOf((chosen as MiniMemberValDecl).type), true)
emit(builder)
}
}
}
}
// Emit what we have first
emitGroup(directMap)
emitGroup(inheritedMap)
// If suggestions are suspiciously sparse for known container classes,
// try to conservatively supplement using a curated list resolved via docs registry.
val totalSuggested = directMap.size + inheritedMap.size
val isContainer = className in setOf("Iterator", "Iterable", "Collection", "List", "Array")
if (isContainer && totalSuggested < 3) {
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Supplementing members for $className; had=$totalSuggested")
val common = when (className) {
"Iterator" -> listOf(
"hasNext", "next", "forEach", "map", "filter", "take", "drop", "toList", "count", "any", "all"
)
else -> listOf(
// Iterable/Collection/List/Array common ops
"size", "isEmpty", "map", "flatMap", "filter", "first", "last", "contains",
"any", "all", "count", "forEach", "toList", "toSet"
)
}
val already = (directMap.keys + inheritedMap.keys).toMutableSet()
for (name in common) {
if (name in already) continue
// Try resolve across classes first to get types/params; if it fails, emit a synthetic safe suggestion.
val resolved = DocLookupUtils.findMemberAcrossClasses(imported, name)
if (resolved != null) {
val member = resolved.second
when (member) {
is MiniMemberFunDecl -> {
val params = member.params.joinToString(", ") { it.name }
val ret = typeOf(member.returnType)
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("(${params})", true)
.withTypeText(ret, true)
.withInsertHandler(ParenInsertHandler)
emit(builder)
already.add(name)
}
is MiniMemberValDecl -> {
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Field)
.withTypeText(typeOf(member.type), true)
emit(builder)
already.add(name)
}
}
} else {
// Synthetic fallback: method without detailed params/types to improve UX in absence of docs
val isProperty = name in setOf("size", "length")
val builder = if (isProperty) {
LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Field)
} else {
LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("()", true)
.withInsertHandler(ParenInsertHandler)
}
emit(builder)
already.add(name)
}
}
}
// Supplement with stdlib extension-like methods defined in root.lyng (e.g., fun String.trim(...))
run {
val already = (directMap.keys + inheritedMap.keys).toMutableSet()
val ext = BuiltinDocRegistry.extensionMethodNamesFor(className)
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Extensions for $className: count=${ext.size} -> ${ext}")
for (name in ext) {
if (already.contains(name)) continue
// Try to resolve full signature via registry first to get params and return type
val resolved = DocLookupUtils.resolveMemberWithInheritance(imported, className, name)
if (resolved != null) {
when (val member = resolved.second) {
is MiniMemberFunDecl -> {
val params = member.params.joinToString(", ") { it.name }
val ret = typeOf(member.returnType)
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("(${params})", true)
.withTypeText(ret, true)
.withInsertHandler(ParenInsertHandler)
emit(builder)
already.add(name)
continue
}
is MiniMemberValDecl -> {
val builder = LookupElementBuilder.create(name)
.withIcon(if (member.mutable) AllIcons.Nodes.Variable else AllIcons.Nodes.Field)
.withTypeText(typeOf(member.type), true)
emit(builder)
already.add(name)
continue
}
}
}
// Fallback: emit without detailed types if we couldn't resolve
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("()", true)
.withInsertHandler(ParenInsertHandler)
emit(builder)
already.add(name)
}
}
}
// --- MiniAst-based inference helpers ---
private fun previousIdentifierBeforeDot(text: String, dotPos: Int): String? {
var i = dotPos - 1
// skip whitespace
while (i >= 0 && text[i].isWhitespace()) i--
val end = i + 1
while (i >= 0 && TextCtx.isIdentChar(text[i])) i--
val start = i + 1
return if (start < end) text.substring(start, end) else null
}
private fun guessReceiverClassViaMini(mini: MiniScript?, text: String, dotPos: Int, imported: List<String>): String? {
if (mini == null) return null
val ident = previousIdentifierBeforeDot(text, dotPos) ?: return null
// 1) Local val/var in the file
val valDecl = mini.declarations.filterIsInstance<MiniValDecl>().firstOrNull { it.name == ident }
val typeFromVal = valDecl?.type?.let { simpleClassNameOf(it) }
if (!typeFromVal.isNullOrBlank()) return typeFromVal
// If initializer exists, try to sniff ClassName(
val initR = valDecl?.initRange
if (initR != null) {
val src = mini.range.start.source
val s = src.offsetOf(initR.start)
val e = src.offsetOf(initR.end).coerceAtMost(text.length)
if (s in 0..e && e <= text.length) {
val init = text.substring(s, e)
Regex("([A-Za-z_][A-Za-z0-9_]*)\\s*\\(").find(init)?.let { m ->
val cls = m.groupValues[1]
return cls
}
}
}
// 2) Parameters in any function (best-effort without scope mapping)
val paramType = mini.declarations.filterIsInstance<MiniFunDecl>()
.asSequence()
.flatMap { it.params.asSequence() }
.firstOrNull { it.name == ident }?.type
val typeFromParam = simpleClassNameOf(paramType)
if (!typeFromParam.isNullOrBlank()) return typeFromParam
return null
}
private fun guessReturnClassFromMemberCallBeforeMini(mini: MiniScript?, text: String, dotPos: Int, imported: List<String>): String? {
if (mini == null) return null
var i = TextCtx.prevNonWs(text, dotPos - 1)
if (i < 0 || text[i] != ')') return null
// back to matching '('
i--
var depth = 0
while (i >= 0) {
when (text[i]) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
i--
}
if (i < 0 || text[i] != '(') return null
var j = i - 1
while (j >= 0 && text[j].isWhitespace()) j--
val end = j + 1
while (j >= 0 && TextCtx.isIdentChar(text[j])) j--
val start = j + 1
if (start >= end) return null
val callee = text.substring(start, end)
// Ensure member call: dot before callee
var k = start - 1
while (k >= 0 && text[k].isWhitespace()) k--
if (k < 0 || text[k] != '.') return null
val prevDot = k
// Resolve receiver class via MiniAst (ident like `x`)
val receiverClass = guessReceiverClassViaMini(mini, text, prevDot, imported) ?: return null
// If receiver class is a locally declared class, resolve member on it
val localClass = mini.declarations.filterIsInstance<MiniClassDecl>().firstOrNull { it.name == receiverClass }
if (localClass != null) {
val mm = localClass.members.firstOrNull { it.name == callee }
if (mm != null) {
val rt = when (mm) {
is MiniMemberFunDecl -> mm.returnType
is MiniMemberValDecl -> mm.type
else -> null
}
return simpleClassNameOf(rt)
} else {
// Try to scan class body text for method signature and extract return type
val sigs = scanLocalClassMembersFromText(mini, text, localClass)
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Local scan for return type in ${receiverClass}.${callee}: candidates=${sigs.keys}")
val sig = sigs[callee]
if (sig != null && sig.typeText != null) return sig.typeText
}
}
// Else fallback to registry-based resolution (covers imported classes)
return DocLookupUtils.resolveMemberWithInheritance(imported, receiverClass, callee)?.second?.let { m ->
val rt = when (m) {
is MiniMemberFunDecl -> m.returnType
is MiniMemberValDecl -> m.type
}
simpleClassNameOf(rt)
}
}
private data class ScannedSig(val kind: String, val params: List<String>?, val typeText: String?)
private fun scanLocalClassMembersFromText(mini: MiniScript, text: String, cls: MiniClassDecl): Map<String, ScannedSig> {
val src = mini.range.start.source
val start = src.offsetOf(cls.bodyRange?.start ?: cls.range.start)
val end = src.offsetOf(cls.bodyRange?.end ?: cls.range.end).coerceAtMost(text.length)
if (start !in 0..end) return emptyMap()
val body = text.substring(start, end)
val map = LinkedHashMap<String, ScannedSig>()
// fun name(params): Type
val funRe = Regex("(?m)^\\s*fun\\s+([A-Za-z_][A-Za-z0-9_]*)\\s*\\(([^)]*)\\)\\s*(?::\\s*([A-Za-z_][A-Za-z0-9_]*))?")
for (m in funRe.findAll(body)) {
val name = m.groupValues.getOrNull(1) ?: continue
val params = m.groupValues.getOrNull(2)?.split(',')?.mapNotNull { it.trim().takeIf { it.isNotEmpty() } } ?: emptyList()
val type = m.groupValues.getOrNull(3)?.takeIf { it.isNotBlank() }
map[name] = ScannedSig("fun", params, type)
}
// val/var name: Type
val valRe = Regex("(?m)^\\s*(val|var)\\s+([A-Za-z_][A-Za-z0-9_]*)\\s*(?::\\s*([A-Za-z_][A-Za-z0-9_]*))?")
for (m in valRe.findAll(body)) {
val kind = m.groupValues.getOrNull(1) ?: continue
val name = m.groupValues.getOrNull(2) ?: continue
val type = m.groupValues.getOrNull(3)?.takeIf { it.isNotBlank() }
map.putIfAbsent(name, ScannedSig(kind, null, type))
}
return map
}
private fun guessReceiverClass(text: String, dotPos: Int, imported: List<String>): String? {
// 1) Try call-based: ClassName(...).
DocLookupUtils.guessClassFromCallBefore(text, dotPos, imported)?.let { return it }
// 2) Literal heuristics based on the immediate char before '.'
var i = TextCtx.prevNonWs(text, dotPos - 1)
if (i >= 0) {
when (text[i]) {
'"' -> {
// Either regular or triple-quoted string; both map to String
return "String"
}
']' -> return "List" // very rough heuristic
'}' -> return "Dict" // map/dictionary literal heuristic
')' -> {
// Parenthesized expression: walk back to matching '(' and inspect inner expression
var j = i - 1
var depth = 0
while (j >= 0) {
when (text[j]) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
j--
}
if (j >= 0 && text[j] == '(') {
val innerS = (j + 1).coerceAtLeast(0)
val innerE = i.coerceAtMost(text.length)
if (innerS < innerE) {
val inner = text.substring(innerS, innerE).trim()
if (inner.startsWith('"') && inner.endsWith('"')) return "String"
if (inner.startsWith('[') && inner.endsWith(']')) return "List"
if (inner.startsWith('{') && inner.endsWith('}')) return "Dict"
}
}
}
}
// Numeric literal: support decimal, hex (0x..), and scientific notation (1e-3)
var j = i
var hasDigits = false
var hasDot = false
var hasExp = false
// Walk over digits, letters for hex, dots, and exponent markers
while (j >= 0) {
val ch = text[j]
if (ch.isDigit()) { hasDigits = true; j-- ; continue }
if (ch == '.') { hasDot = true; j-- ; continue }
if (ch == 'e' || ch == 'E') { hasExp = true; j-- ; // optional sign directly before digits
if (j >= 0 && (text[j] == '+' || text[j] == '-')) j--
continue
}
if (ch in listOf('x','X')) { // part of 0x prefix
j--
continue
}
if (ch == 'a' || ch == 'b' || ch == 'c' || ch == 'd' || ch == 'f' ||
ch == 'A' || ch == 'B' || ch == 'C' || ch == 'D' || ch == 'F') {
// hex digit in 0x...
j--
continue
}
break
}
// Now check for 0x/0X prefix
val k = j
val isHex = k >= 1 && text[k] == '0' && (text[k+1] == 'x' || text[k+1] == 'X')
if (hasDigits) {
return if (isHex) "Int" else if (hasDot || hasExp) "Real" else "Int"
}
}
return null
}
/**
* Try to infer the class of the return value of the member call immediately before the dot.
* Example: `Path(".." ).lines().<caret>` → detects `lines()` on receiver class `Path` and returns `Iterator`.
*/
private fun guessReturnClassFromMemberCallBefore(text: String, dotPos: Int, imported: List<String>): String? {
var i = TextCtx.prevNonWs(text, dotPos - 1)
if (i < 0) return null
// We expect a call just before the dot, i.e., ')' ... '.'
if (text[i] != ')') return null
// Walk back to matching '('
i--
var depth = 0
while (i >= 0) {
val ch = text[i]
when (ch) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
i--
}
if (i < 0 || text[i] != '(') return null
// Identify callee identifier just before '('
var j = i - 1
while (j >= 0 && text[j].isWhitespace()) j--
val end = j + 1
while (j >= 0 && TextCtx.isIdentChar(text[j])) j--
val start = j + 1
if (start >= end) return null
val callee = text.substring(start, end)
// Ensure it's a member call (there must be a dot immediately before the callee, ignoring spaces)
var k = start - 1
while (k >= 0 && text[k].isWhitespace()) k--
if (k < 0 || text[k] != '.') return null
val prevDot = k
// Infer receiver class at the previous dot
val receiverClass = guessReceiverClass(text, prevDot, imported) ?: return null
// Resolve the callee as a member of receiver class, including inheritance
val resolved = DocLookupUtils.resolveMemberWithInheritance(imported, receiverClass, callee) ?: return null
val member = resolved.second
val returnType = when (member) {
is MiniMemberFunDecl -> member.returnType
is MiniMemberValDecl -> member.type
}
return simpleClassNameOf(returnType)
}
/**
* Infer return class of a top-level call right before the dot: e.g., `files().<caret>`.
* We extract callee name and resolve it among imported modules' top-level functions.
*/
private fun guessReturnClassFromTopLevelCallBefore(text: String, dotPos: Int, imported: List<String>): String? {
var i = TextCtx.prevNonWs(text, dotPos - 1)
if (i < 0 || text[i] != ')') return null
// Walk back to matching '('
i--
var depth = 0
while (i >= 0) {
val ch = text[i]
when (ch) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
i--
}
if (i < 0 || text[i] != '(') return null
// Extract callee ident before '('
var j = i - 1
while (j >= 0 && text[j].isWhitespace()) j--
val end = j + 1
while (j >= 0 && TextCtx.isIdentChar(text[j])) j--
val start = j + 1
if (start >= end) return null
val callee = text.substring(start, end)
// If it's a member call, bail out (handled in member-call inference)
var k = start - 1
while (k >= 0 && text[k].isWhitespace()) k--
if (k >= 0 && text[k] == '.') return null
// Resolve top-level function in imported modules
for (mod in imported) {
val decls = BuiltinDocRegistry.docsForModule(mod)
val fn = decls.asSequence().filterIsInstance<MiniFunDecl>().firstOrNull { it.name == callee }
if (fn != null) return simpleClassNameOf(fn.returnType)
}
return null
}
/**
* Fallback: if we can at least extract a callee name before the dot and it exists across common classes,
* derive its return type using cross-class lookup (Iterable/Iterator/List preference). This ignores the receiver.
* Example: `something.lines().<caret>` where `something` type is unknown, but `lines()` commonly returns Iterator<String>.
*/
private fun guessReturnClassAcrossKnownCallees(text: String, dotPos: Int, imported: List<String>): String? {
var i = TextCtx.prevNonWs(text, dotPos - 1)
if (i < 0 || text[i] != ')') return null
// Walk back to matching '('
i--
var depth = 0
while (i >= 0) {
val ch = text[i]
when (ch) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
i--
}
if (i < 0 || text[i] != '(') return null
// Extract callee ident before '('
var j = i - 1
while (j >= 0 && text[j].isWhitespace()) j--
val end = j + 1
while (j >= 0 && TextCtx.isIdentChar(text[j])) j--
val start = j + 1
if (start >= end) return null
val callee = text.substring(start, end)
// Try cross-class resolution
val resolved = DocLookupUtils.findMemberAcrossClasses(imported, callee) ?: return null
val member = resolved.second
val returnType = when (member) {
is MiniMemberFunDecl -> member.returnType
is MiniMemberValDecl -> member.type
}
return simpleClassNameOf(returnType)
}
/** Convert a MiniTypeRef to a simple class name as used by docs (e.g., Iterator from Iterator<String>). */
private fun simpleClassNameOf(t: MiniTypeRef?): String? = when (t) {
null -> null
is MiniTypeName -> t.segments.lastOrNull()?.name
is MiniGenericType -> simpleClassNameOf(t.base)
is MiniFunctionType -> null
is MiniTypeVar -> null
}
private fun buildMiniAst(text: String): MiniScript? {
return try {
val sink = MiniAstBuilder()
val provider = IdeLenientImportProvider.create()
val src = Source("<ide>", text)
runBlocking { Compiler.compileWithMini(src, provider, sink) }
sink.build()
} catch (_: Throwable) {
null
}
}
// Cached per PsiFile by document modification stamp
private val MINI_KEY = Key.create<MiniScript>("lyng.mini.cache")
private val STAMP_KEY = Key.create<Long>("lyng.mini.cache.stamp")
private fun buildMiniAstCached(file: PsiFile, text: String): MiniScript? {
val doc = file.viewProvider.document ?: return null
val stamp = doc.modificationStamp
val prevStamp = file.getUserData(STAMP_KEY)
val cached = file.getUserData(MINI_KEY)
if (cached != null && prevStamp != null && prevStamp == stamp) return cached
val built = buildMiniAst(text)
// Cache even null? avoid caching failures; only cache non-null
if (built != null) {
file.putUserData(MINI_KEY, built)
file.putUserData(STAMP_KEY, stamp)
}
return built
}
private fun offerParamsInScope(emit: (com.intellij.codeInsight.lookup.LookupElement) -> Unit, mini: MiniScript, text: String, caret: Int) {
val src = mini.range.start.source
// Find function whose body contains caret or whose whole range contains caret
val fns = mini.declarations.filterIsInstance<MiniFunDecl>()
for (fn in fns) {
val start = src.offsetOf(fn.range.start)
val end = src.offsetOf(fn.range.end).coerceAtMost(text.length)
if (caret in start..end) {
for (p in fn.params) {
val builder = LookupElementBuilder.create(p.name)
.withIcon(AllIcons.Nodes.Variable)
.withTypeText(typeOf(p.type), true)
emit(builder)
}
return
}
}
}
// Lenient textual import extractor (duplicated from QuickDoc privately)
private fun extractImportsFromText(text: String): List<String> {
val result = LinkedHashSet<String>()
val re = Regex("(?m)^\\s*import\\s+([a-zA-Z_][a-zA-Z0-9_]*(?:\\.[a-zA-Z_][a-zA-Z0-9_]*)*)")
re.findAll(text).forEach { m ->
val raw = m.groupValues.getOrNull(1)?.trim().orEmpty()
if (raw.isNotEmpty()) {
val canon = if (raw.startsWith("lyng.")) raw else "lyng.$raw"
result.add(canon)
}
}
return result.toList()
}
private fun typeOf(t: MiniTypeRef?): String {
return when (t) {
null -> ""
is MiniTypeName -> t.segments.lastOrNull()?.name?.let { ": $it" } ?: ""
is MiniGenericType -> {
val base = typeOf(t.base).removePrefix(": ")
val args = t.args.joinToString(",") { typeOf(it).removePrefix(": ") }
": ${base}<${args}>"
}
is MiniFunctionType -> ": (fn)"
is MiniTypeVar -> ": ${t.name}"
}
}
}
}
@@ -0,0 +1,55 @@
/*
* Minimal fallback docs seeding for `lyng.io.fs` used only inside the IDEA plugin
* when external docs module (lyngio) is not present on the classpath.
*
* We keep it tiny and plugin-local to avoid coupling core library to external packages.
*/
package net.sergeych.lyng.idea.docs
import net.sergeych.lyng.miniast.BuiltinDocRegistry
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.type
internal object FsDocsFallback {
@Volatile
private var seeded = false
fun ensureOnce(): Boolean {
if (seeded) return true
synchronized(this) {
if (seeded) return true
BuiltinDocRegistry.module("lyng.io.fs") {
// Class Path summary and a few commonly used methods
classDoc(name = "Path", doc = "Filesystem path class. Construct with a string: `Path(\"/tmp\")`.") {
method(name = "exists", doc = "Whether the path exists on the filesystem.", returns = type("lyng.Bool"))
method(name = "isFile", doc = "Whether the path exists and is a file.", returns = type("lyng.Bool"))
method(name = "isDir", doc = "Whether the path exists and is a directory.", returns = type("lyng.Bool"))
method(name = "readUtf8", doc = "Read the entire file as UTF-8 string.", returns = type("lyng.String"))
method(
name = "writeUtf8",
doc = "Write UTF-8 string to the file (overwrite).",
params = listOf(ParamDoc("text", type("lyng.String")))
)
method(
name = "bytes",
doc = "Iterate file content as `Buffer` chunks.",
params = listOf(ParamDoc("size", type("lyng.Int"))),
returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.Buffer")))
)
method(
name = "lines",
doc = "Iterate file as lines of text.",
returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.String")))
)
}
// Top-level exported constants
valDoc(name = "Path", doc = "Filesystem path class. Construct with a string: `Path(\"/tmp\")`.", type = type("Path"))
valDoc(name = "Paths", doc = "Alias of `Path` for those who prefer plural form.", type = type("Path"))
}
seeded = true
return true
}
}
}
@@ -26,10 +26,12 @@ import com.intellij.psi.PsiElement
import com.intellij.psi.PsiFile
import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Source
import net.sergeych.lyng.highlight.offsetOf
import net.sergeych.lyng.idea.LyngLanguage
import net.sergeych.lyng.idea.util.IdeLenientImportProvider
import net.sergeych.lyng.idea.util.TextCtx
import net.sergeych.lyng.miniast.*
/**
@@ -41,7 +43,17 @@ class LyngDocumentationProvider : AbstractDocumentationProvider() {
private val log = Logger.getInstance(LyngDocumentationProvider::class.java)
// Toggle to trace inheritance-based resolutions in Quick Docs. Keep false for normal use.
private val DEBUG_INHERITANCE = false
// Global Quick Doc debug toggle (OFF by default). When false, [LYNG_DEBUG] logs are suppressed.
private val DEBUG_LOG = false
override fun generateDoc(element: PsiElement?, originalElement: PsiElement?): String? {
// Try load external docs registrars (e.g., lyngio) if present on classpath
ensureExternalDocsRegistered()
// Ensure stdlib Obj*-defined docs (e.g., String methods via ObjString.addFnDoc) are initialized
try {
net.sergeych.lyng.miniast.StdlibDocsBootstrap.ensure()
} catch (_: Throwable) {
// best-effort; absence must not break Quick Doc
}
if (element == null) return null
val file: PsiFile = element.containingFile ?: return null
val document: Document = file.viewProvider.document ?: return null
@@ -49,61 +61,135 @@ class LyngDocumentationProvider : AbstractDocumentationProvider() {
// Determine caret/lookup offset from the element range
val offset = originalElement?.textRange?.startOffset ?: element.textRange.startOffset
val idRange = wordRangeAt(text, offset) ?: run {
log.info("[LYNG_DEBUG] QuickDoc: no word at offset=$offset in ${file.name}")
val idRange = TextCtx.wordRangeAt(text, offset) ?: run {
if (DEBUG_LOG) log.info("[LYNG_DEBUG] QuickDoc: no word at offset=$offset in ${file.name}")
return null
}
if (idRange.isEmpty) return null
val ident = text.substring(idRange.startOffset, idRange.endOffset)
log.info("[LYNG_DEBUG] QuickDoc: ident='$ident' at ${idRange.startOffset}..${idRange.endOffset} in ${file.name}")
if (DEBUG_LOG) log.info("[LYNG_DEBUG] QuickDoc: ident='$ident' at ${idRange.startOffset}..${idRange.endOffset} in ${file.name}")
// Build MiniAst for this file (fast and resilient). Best-effort; on failure return null.
// Build MiniAst for this file (fast and resilient). Best-effort; on failure continue with registry lookup only.
val sink = MiniAstBuilder()
// Use lenient import provider so unresolved imports (e.g., lyng.io.fs) don't break docs
val provider = IdeLenientImportProvider.create()
try {
val src = Source("<ide>", text)
var mini: MiniScript? = try {
runBlocking { Compiler.compileWithMini(src, provider, sink) }
sink.build()
} catch (t: Throwable) {
log.warn("[LYNG_DEBUG] QuickDoc: compileWithMini failed: ${t.message}")
return null
// Do not bail out completely: we still can resolve built-in and imported docs (e.g., println)
if (DEBUG_LOG) log.warn("[LYNG_DEBUG] QuickDoc: compileWithMini failed: ${t.message}")
null
}
val mini = sink.build() ?: return null
val source = Source("<ide>", text)
val haveMini = mini != null
if (mini == null) {
// Ensure we have a dummy script object to avoid NPE in downstream helpers that expect a MiniScript
mini = MiniScript(MiniRange(Pos(src, 1, 1), Pos(src, 1, 1)))
}
val source = src
// Try resolve to: function param at position, function/class/val declaration at position
// 1) Check declarations whose name range contains offset
for (d in mini.declarations) {
if (haveMini) for (d in mini.declarations) {
val s = source.offsetOf(d.nameStart)
val e = (s + d.name.length).coerceAtMost(text.length)
if (offset in s until e) {
log.info("[LYNG_DEBUG] QuickDoc: matched decl '${d.name}' kind=${d::class.simpleName}")
if (DEBUG_LOG) log.info("[LYNG_DEBUG] QuickDoc: matched decl '${d.name}' kind=${d::class.simpleName}")
return renderDeclDoc(d)
}
}
// 2) Check parameters of functions
for (fn in mini.declarations.filterIsInstance<MiniFunDecl>()) {
if (haveMini) for (fn in mini.declarations.filterIsInstance<MiniFunDecl>()) {
for (p in fn.params) {
val s = source.offsetOf(p.nameStart)
val e = (s + p.name.length).coerceAtMost(text.length)
if (offset in s until e) {
log.info("[LYNG_DEBUG] QuickDoc: matched param '${p.name}' in fun '${fn.name}'")
if (DEBUG_LOG) log.info("[LYNG_DEBUG] QuickDoc: matched param '${p.name}' in fun '${fn.name}'")
return renderParamDoc(fn, p)
}
}
}
// 3) As a fallback, if the caret is on an identifier text that matches any declaration name, show that
mini.declarations.firstOrNull { it.name == ident }?.let {
// 3) Member-context resolution first (dot immediately before identifier): handle literals and calls
run {
val dotPos = TextCtx.findDotLeft(text, idRange.startOffset)
?: TextCtx.findDotLeft(text, offset)
if (dotPos != null) {
// Build imported modules (MiniAst-derived if available, else lenient from text) and ensure stdlib is present
var importedModules = if (haveMini) DocLookupUtils.canonicalImportedModules(mini) else emptyList()
if (importedModules.isEmpty()) {
val fromText = extractImportsFromText(text)
importedModules = if (fromText.isEmpty()) listOf("lyng.stdlib") else fromText
}
if (!importedModules.contains("lyng.stdlib")) importedModules = importedModules + "lyng.stdlib"
// Try literal and call-based receiver inference around the dot
val i = TextCtx.prevNonWs(text, dotPos - 1)
val className: String? = when {
i >= 0 && text[i] == '"' -> "String"
i >= 0 && text[i] == ']' -> "List"
i >= 0 && text[i] == '}' -> "Dict"
i >= 0 && text[i] == ')' -> {
// Parenthesized expression: walk back to matching '(' and inspect the inner expression
var j = i - 1
var depth = 0
while (j >= 0) {
when (text[j]) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
j--
}
if (j >= 0 && text[j] == '(') {
val innerS = (j + 1).coerceAtLeast(0)
val innerE = i.coerceAtMost(text.length)
if (innerS < innerE) {
val inner = text.substring(innerS, innerE).trim()
when {
inner.startsWith('"') && inner.endsWith('"') -> "String"
inner.startsWith('[') && inner.endsWith(']') -> "List"
inner.startsWith('{') && inner.endsWith('}') -> "Dict"
else -> null
}
} else null
} else null
}
else -> DocLookupUtils.guessClassFromCallBefore(text, dotPos, importedModules)
}
if (DEBUG_LOG) log.info("[LYNG_DEBUG] QuickDoc: memberCtx dotPos=${dotPos} chBeforeDot='${if (dotPos>0) text[dotPos-1] else ' '}' classGuess=${className} imports=${importedModules}")
if (className != null) {
DocLookupUtils.resolveMemberWithInheritance(importedModules, className, ident)?.let { (owner, member) ->
if (DEBUG_INHERITANCE) log.info("[LYNG_DEBUG] QuickDoc: literal/call '$ident' resolved to $owner.${member.name}")
return when (member) {
is MiniMemberFunDecl -> renderMemberFunDoc(owner, member)
is MiniMemberValDecl -> renderMemberValDoc(owner, member)
}
}
log.info("[LYNG_DEBUG] QuickDoc: resolve failed for ${className}.${ident}")
}
}
}
// 4) As a fallback, if the caret is on an identifier text that matches any declaration name, show that
if (haveMini) mini.declarations.firstOrNull { it.name == ident }?.let {
log.info("[LYNG_DEBUG] QuickDoc: fallback by name '${it.name}' kind=${it::class.simpleName}")
return renderDeclDoc(it)
}
// 4) Consult BuiltinDocRegistry for imported modules (top-level and class members)
var importedModules = mini.imports.map { it.segments.joinToString(".") { s -> s.name } }
// Core-module fallback: in scratch/repl-like files without imports, consult stdlib by default
if (importedModules.isEmpty()) importedModules = listOf("lyng.stdlib")
// Canonicalize import names using ImportManager, as users may write shortened names (e.g., "io.fs")
var importedModules = if (haveMini) DocLookupUtils.canonicalImportedModules(mini) else emptyList()
// If MiniAst failed or captured no imports, try a lightweight textual import scan
if (importedModules.isEmpty()) {
val fromText = extractImportsFromText(text)
if (fromText.isNotEmpty()) {
importedModules = fromText
}
}
// Always include stdlib as a fallback context
if (!importedModules.contains("lyng.stdlib")) importedModules = importedModules + "lyng.stdlib"
// 4a) try top-level decls
for (mod in importedModules) {
importedModules.forEach { mod ->
val docs = BuiltinDocRegistry.docsForModule(mod)
val matches = docs.filterIsInstance<MiniFunDecl>().filter { it.name == ident }
if (matches.isNotEmpty()) {
@@ -121,11 +207,19 @@ class LyngDocumentationProvider : AbstractDocumentationProvider() {
// And classes
docs.filterIsInstance<MiniClassDecl>().firstOrNull { it.name == ident }?.let { return renderDeclDoc(it) }
}
// Defensive fallback: if nothing found and it's a well-known stdlib function, render minimal inline docs
if (ident == "println" || ident == "print") {
val fallback = if (ident == "println")
"Print values to the standard output and append a newline. Accepts any number of arguments." else
"Print values to the standard output without a trailing newline. Accepts any number of arguments."
val title = "function $ident(values)"
return "<div class='doc-title'>${htmlEscape(title)}</div>" + styledMarkdown(htmlEscape(fallback))
}
// 4b) try class members like ClassName.member with inheritance fallback
val lhs = previousWordBefore(text, idRange.startOffset)
if (lhs != null && hasDotBetween(text, lhs.endOffset, idRange.startOffset)) {
val className = text.substring(lhs.startOffset, lhs.endOffset)
resolveMemberWithInheritance(importedModules, className, ident)?.let { (owner, member) ->
DocLookupUtils.resolveMemberWithInheritance(importedModules, className, ident)?.let { (owner, member) ->
if (DEBUG_INHERITANCE) log.info("[LYNG_DEBUG] Inheritance resolved $className.$ident to $owner.${member.name}")
return when (member) {
is MiniMemberFunDecl -> renderMemberFunDoc(owner, member)
@@ -140,10 +234,10 @@ class LyngDocumentationProvider : AbstractDocumentationProvider() {
if (dotPos != null) {
val guessed = when {
looksLikeListLiteralBefore(text, dotPos) -> "List"
else -> null
else -> DocLookupUtils.guessClassFromCallBefore(text, dotPos, importedModules)
}
if (guessed != null) {
resolveMemberWithInheritance(importedModules, guessed, ident)?.let { (owner, member) ->
DocLookupUtils.resolveMemberWithInheritance(importedModules, guessed, ident)?.let { (owner, member) ->
if (DEBUG_INHERITANCE) log.info("[LYNG_DEBUG] Heuristic '$guessed.$ident' resolved via inheritance to $owner.${member.name}")
return when (member) {
is MiniMemberFunDecl -> renderMemberFunDoc(owner, member)
@@ -151,8 +245,34 @@ class LyngDocumentationProvider : AbstractDocumentationProvider() {
}
}
} else {
// Extra fallback: try a small set of known receiver classes (covers literals when guess failed)
run {
val candidates = listOf("String", "Iterable", "Iterator", "List", "Collection", "Array", "Dict", "Regex")
for (c in candidates) {
DocLookupUtils.resolveMemberWithInheritance(importedModules, c, ident)?.let { (owner, member) ->
if (DEBUG_INHERITANCE) log.info("[LYNG_DEBUG] Candidate '$c.$ident' resolved via inheritance to $owner.${member.name}")
return when (member) {
is MiniMemberFunDecl -> renderMemberFunDoc(owner, member)
is MiniMemberValDecl -> renderMemberValDoc(owner, member)
}
}
}
}
// As a last resort try aggregated String members (extensions from stdlib text)
run {
val classes = DocLookupUtils.aggregateClasses(importedModules)
val stringCls = classes["String"]
val m = stringCls?.members?.firstOrNull { it.name == ident }
if (m != null) {
if (DEBUG_INHERITANCE) log.info("[LYNG_DEBUG] Aggregated fallback resolved String.$ident")
return when (m) {
is MiniMemberFunDecl -> renderMemberFunDoc("String", m)
is MiniMemberValDecl -> renderMemberValDoc("String", m)
}
}
}
// Search across classes; prefer Iterable, then Iterator, then List for common ops
findMemberAcrossClasses(importedModules, ident)?.let { (owner, member) ->
DocLookupUtils.findMemberAcrossClasses(importedModules, ident)?.let { (owner, member) ->
if (DEBUG_INHERITANCE) log.info("[LYNG_DEBUG] Cross-class '$ident' resolved to $owner.${member.name}")
return when (member) {
is MiniMemberFunDecl -> renderMemberFunDoc(owner, member)
@@ -163,10 +283,55 @@ class LyngDocumentationProvider : AbstractDocumentationProvider() {
}
}
log.info("[LYNG_DEBUG] QuickDoc: nothing found for ident='$ident'")
if (DEBUG_LOG) log.info("[LYNG_DEBUG] QuickDoc: nothing found for ident='$ident'")
return null
}
/**
* Very lenient import extractor for cases when MiniAst is unavailable.
* Looks for lines like `import xxx.yyy` and returns canonical module names
* (prefixing with `lyng.` if missing).
*/
private fun extractImportsFromText(text: String): List<String> {
val result = LinkedHashSet<String>()
val re = Regex("(?m)^\\s*import\\s+([a-zA-Z_][a-zA-Z0-9_]*(?:\\.[a-zA-Z_][a-zA-Z0-9_]*)*)")
re.findAll(text).forEach { m ->
val raw = m.groupValues.getOrNull(1)?.trim().orEmpty()
if (raw.isNotEmpty()) {
val canon = if (raw.startsWith("lyng.")) raw else "lyng.$raw"
result.add(canon)
}
}
return result.toList()
}
// External docs registrars discovery via reflection to avoid hard dependencies on optional modules
private val externalDocsLoaded: Boolean by lazy { tryLoadExternalDocs() }
private fun ensureExternalDocsRegistered() { @Suppress("UNUSED_EXPRESSION") externalDocsLoaded }
private fun tryLoadExternalDocs(): Boolean {
return try {
// Try known registrars; ignore failures if module is absent
val cls = Class.forName("net.sergeych.lyngio.docs.FsBuiltinDocs")
val m = cls.getMethod("ensure")
m.invoke(null)
log.info("[LYNG_DEBUG] QuickDoc: external docs loaded: net.sergeych.lyngio.docs.FsBuiltinDocs.ensure() OK")
true
} catch (_: Throwable) {
// Seed a minimal plugin-local fallback so Path docs still work without lyngio
val seeded = try {
FsDocsFallback.ensureOnce()
} catch (_: Throwable) { false }
if (seeded) {
log.info("[LYNG_DEBUG] QuickDoc: external docs NOT found; seeded plugin fallback for lyng.io.fs")
} else {
log.info("[LYNG_DEBUG] QuickDoc: external docs NOT found (lyngio absent on classpath)")
}
seeded
}
}
override fun getCustomDocumentationElement(
editor: Editor,
file: PsiFile,
@@ -362,46 +527,7 @@ class LyngDocumentationProvider : AbstractDocumentationProvider() {
// --- Helpers for inheritance-aware and heuristic member lookup ---
private fun aggregateClasses(importedModules: List<String>): Map<String, MiniClassDecl> {
val map = LinkedHashMap<String, MiniClassDecl>()
for (mod in importedModules) {
val docs = BuiltinDocRegistry.docsForModule(mod)
docs.filterIsInstance<MiniClassDecl>().forEach { cls ->
// Prefer the first occurrence; allow later duplicates to be ignored
map.putIfAbsent(cls.name, cls)
}
}
return map
}
private fun resolveMemberWithInheritance(importedModules: List<String>, className: String, member: String): Pair<String, MiniMemberDecl>? {
val classes = aggregateClasses(importedModules)
fun dfs(name: String, visited: MutableSet<String>): Pair<String, MiniMemberDecl>? {
val cls = classes[name] ?: return null
cls.members.firstOrNull { it.name == member }?.let { return name to it }
if (!visited.add(name)) return null
for (baseName in cls.bases) {
dfs(baseName, visited)?.let { return it }
}
return null
}
return dfs(className, mutableSetOf())
}
private fun findMemberAcrossClasses(importedModules: List<String>, member: String): Pair<String, MiniMemberDecl>? {
val classes = aggregateClasses(importedModules)
// Preferred order for ambiguous common ops
val preference = listOf("Iterable", "Iterator", "List")
// First, try preference order
for (name in preference) {
resolveMemberWithInheritance(importedModules, name, member)?.let { return it }
}
// Then, scan all
for ((name, cls) in classes) {
cls.members.firstOrNull { it.name == member }?.let { return name to it }
}
return null
}
// Removed: member/class resolution helpers moved to lynglib DocLookupUtils for reuse
private fun findDotLeft(text: String, rightStart: Int): Int? {
var i = (rightStart - 1).coerceAtLeast(0)
@@ -433,4 +559,6 @@ class LyngDocumentationProvider : AbstractDocumentationProvider() {
}
return false
}
// Removed: guessClassFromCallBefore moved to DocLookupUtils
}
@@ -50,6 +50,8 @@ class LyngFormatterSettings(private val project: Project) : PersistentStateCompo
var offerLyngTypoQuickFixes: Boolean = true,
// Per-project learned words (do not flag again)
var learnedWords: MutableSet<String> = mutableSetOf(),
// Experimental: enable Lyng autocompletion (can be disabled if needed)
var enableLyngCompletionExperimental: Boolean = true,
)
private var myState: State = State()
@@ -116,6 +118,10 @@ class LyngFormatterSettings(private val project: Project) : PersistentStateCompo
get() = myState.learnedWords
set(value) { myState.learnedWords = value }
var enableLyngCompletionExperimental: Boolean
get() = myState.enableLyngCompletionExperimental
set(value) { myState.enableLyngCompletionExperimental = value }
companion object {
@JvmStatic
fun getInstance(project: Project): LyngFormatterSettings = project.getService(LyngFormatterSettings::class.java)
@@ -38,6 +38,7 @@ class LyngFormatterSettingsConfigurable(private val project: Project) : Configur
private var debugShowSpellFeedCb: JCheckBox? = null
private var showTyposGreenCb: JCheckBox? = null
private var offerQuickFixesCb: JCheckBox? = null
private var enableCompletionCb: JCheckBox? = null
override fun getDisplayName(): String = "Lyng Formatter"
@@ -57,6 +58,7 @@ class LyngFormatterSettingsConfigurable(private val project: Project) : Configur
debugShowSpellFeedCb = JCheckBox("Debug: show spell-feed ranges (weak warnings)")
showTyposGreenCb = JCheckBox("Show Lyng typos with green underline (TYPO styling)")
offerQuickFixesCb = JCheckBox("Offer Lyng typo quick fixes (Replace…, Add to dictionary) without Spell Checker")
enableCompletionCb = JCheckBox("Enable Lyng autocompletion (experimental)")
// Tooltips / short help
spacingCb?.toolTipText = "Applies minimal, safe spacing (e.g., around commas/operators, control-flow parens)."
@@ -71,6 +73,7 @@ class LyngFormatterSettingsConfigurable(private val project: Project) : Configur
debugShowSpellFeedCb?.toolTipText = "Show the exact ranges we feed to spellcheckers (ids/comments/strings) as weak warnings."
showTyposGreenCb?.toolTipText = "Render Lyng typos using the platform's green TYPO underline instead of generic warnings."
offerQuickFixesCb?.toolTipText = "Provide lightweight Replace… and Add to dictionary quick-fixes without requiring the legacy Spell Checker."
enableCompletionCb?.toolTipText = "Turn on/off the lightweight Lyng code completion (BASIC)."
p.add(spacingCb)
p.add(wrappingCb)
p.add(reindentClosedBlockCb)
@@ -84,6 +87,7 @@ class LyngFormatterSettingsConfigurable(private val project: Project) : Configur
p.add(debugShowSpellFeedCb)
p.add(showTyposGreenCb)
p.add(offerQuickFixesCb)
p.add(enableCompletionCb)
panel = p
reset()
return p
@@ -103,7 +107,8 @@ class LyngFormatterSettingsConfigurable(private val project: Project) : Configur
grazieLiteralsAsCommentsCb?.isSelected != s.grazieTreatLiteralsAsComments ||
debugShowSpellFeedCb?.isSelected != s.debugShowSpellFeed ||
showTyposGreenCb?.isSelected != s.showTyposWithGreenUnderline ||
offerQuickFixesCb?.isSelected != s.offerLyngTypoQuickFixes
offerQuickFixesCb?.isSelected != s.offerLyngTypoQuickFixes ||
enableCompletionCb?.isSelected != s.enableLyngCompletionExperimental
}
override fun apply() {
@@ -121,6 +126,7 @@ class LyngFormatterSettingsConfigurable(private val project: Project) : Configur
s.debugShowSpellFeed = debugShowSpellFeedCb?.isSelected == true
s.showTyposWithGreenUnderline = showTyposGreenCb?.isSelected == true
s.offerLyngTypoQuickFixes = offerQuickFixesCb?.isSelected == true
s.enableLyngCompletionExperimental = enableCompletionCb?.isSelected == true
}
override fun reset() {
@@ -138,5 +144,6 @@ class LyngFormatterSettingsConfigurable(private val project: Project) : Configur
debugShowSpellFeedCb?.isSelected = s.debugShowSpellFeed
showTyposGreenCb?.isSelected = s.showTyposWithGreenUnderline
offerQuickFixesCb?.isSelected = s.offerLyngTypoQuickFixes
enableCompletionCb?.isSelected = s.enableLyngCompletionExperimental
}
}
@@ -0,0 +1,44 @@
/*
* Ensure external/bundled docs are registered in BuiltinDocRegistry
* so completion/quickdoc can resolve things like lyng.io.fs.Path.
*/
package net.sergeych.lyng.idea.util
import com.intellij.openapi.diagnostic.Logger
import net.sergeych.lyng.idea.docs.FsDocsFallback
object DocsBootstrap {
private val log = Logger.getInstance(DocsBootstrap::class.java)
@Volatile private var ensured = false
fun ensure() {
if (ensured) return
synchronized(this) {
if (ensured) return
val loaded = tryLoadExternal() || trySeedFallback()
if (loaded) ensured = true else ensured = true // mark done to avoid repeated attempts
}
}
private fun tryLoadExternal(): Boolean = try {
val cls = Class.forName("net.sergeych.lyngio.docs.FsBuiltinDocs")
val m = cls.getMethod("ensure")
m.invoke(null)
log.info("[LYNG_DEBUG] DocsBootstrap: external docs loaded: net.sergeych.lyngio.docs.FsBuiltinDocs.ensure() OK")
true
} catch (_: Throwable) {
false
}
private fun trySeedFallback(): Boolean = try {
val seeded = FsDocsFallback.ensureOnce()
if (seeded) {
log.info("[LYNG_DEBUG] DocsBootstrap: external docs not found; seeded plugin fallback for lyng.io.fs")
} else {
log.info("[LYNG_DEBUG] DocsBootstrap: external docs not found; no fallback seeded")
}
seeded
} catch (_: Throwable) {
false
}
}
@@ -0,0 +1,60 @@
/*
* Shared tiny, PSI-free text helpers for Lyng editor features (Quick Doc, Completion).
*/
package net.sergeych.lyng.idea.util
import com.intellij.openapi.util.TextRange
object TextCtx {
fun prefixAt(text: String, offset: Int): String {
val off = offset.coerceIn(0, text.length)
var i = (off - 1).coerceAtLeast(0)
while (i >= 0 && isIdentChar(text[i])) i--
val start = i + 1
return if (start in 0..text.length && start <= off) text.substring(start, off) else ""
}
fun wordRangeAt(text: String, offset: Int): TextRange? {
if (text.isEmpty()) return null
val off = offset.coerceIn(0, text.length)
var s = off
var e = off
while (s > 0 && isIdentChar(text[s - 1])) s--
while (e < text.length && isIdentChar(text[e])) e++
return if (s < e) TextRange(s, e) else null
}
fun findDotLeft(text: String, offset: Int): Int? {
var i = (offset - 1).coerceAtLeast(0)
while (i >= 0 && text[i].isWhitespace()) i--
return if (i >= 0 && text[i] == '.') i else null
}
fun previousWordBefore(text: String, offset: Int): String? {
var i = prevNonWs(text, (offset - 1).coerceAtLeast(0))
// Skip trailing identifier at caret if inside word
while (i >= 0 && isIdentChar(text[i])) i--
i = prevNonWs(text, i)
if (i < 0) return null
val end = i + 1
while (i >= 0 && isIdentChar(text[i])) i--
val start = i + 1
return if (start < end) text.substring(start, end) else null
}
fun hasDotBetween(text: String, start: Int, end: Int): Boolean {
if (start >= end) return false
val s = start.coerceAtLeast(0)
val e = end.coerceAtMost(text.length)
for (i in s until e) if (text[i] == '.') return true
return false
}
fun prevNonWs(text: String, start: Int): Int {
var i = start.coerceAtMost(text.length - 1)
while (i >= 0 && text[i].isWhitespace()) i--
return i
}
fun isIdentChar(c: Char): Boolean = c == '_' || c.isLetterOrDigit()
}
@@ -62,6 +62,9 @@
<!-- Quick documentation provider bound to Lyng language -->
<lang.documentationProvider language="Lyng" implementationClass="net.sergeych.lyng.idea.docs.LyngDocumentationProvider"/>
<!-- Basic code completion (MVP) -->
<completion.contributor language="Lyng" implementationClass="net.sergeych.lyng.idea.completion.LyngCompletionContributor"/>
<!-- Comment toggling support -->
<lang.commenter language="Lyng" implementationClass="net.sergeych.lyng.idea.comment.LyngCommenter"/>
@@ -0,0 +1,121 @@
package net.sergeych.lyng.idea.completion
import com.intellij.testFramework.fixtures.BasePlatformTestCase
import net.sergeych.lyng.idea.util.DocsBootstrap
import net.sergeych.lyng.miniast.BuiltinDocRegistry
import net.sergeych.lyng.miniast.DocLookupUtils
import net.sergeych.lyng.miniast.MiniClassDecl
class LyngCompletionMemberTest : BasePlatformTestCase() {
override fun getTestDataPath(): String = ""
private fun complete(code: String): List<String> {
myFixture.configureByText("test.lyng", code)
val items = myFixture.completeBasic()
return myFixture.lookupElementStrings ?: emptyList()
}
private fun ensureDocs(imports: List<String>) {
// Make sure external/bundled docs like lyng.io.fs are registered
DocsBootstrap.ensure()
// Touch modules to force stdlib lazy load and optional modules
for (m in imports) BuiltinDocRegistry.docsForModule(m)
}
private fun aggregateMemberNames(className: String, imported: List<String>): Set<String> {
val classes = DocLookupUtils.aggregateClasses(imported)
val visited = mutableSetOf<String>()
val result = linkedSetOf<String>()
fun dfs(name: String) {
val cls: MiniClassDecl = classes[name] ?: return
for (m in cls.members) result.add(m.name)
if (!visited.add(name)) return
for (b in cls.bases) dfs(b)
}
dfs(className)
// Conservative supplementation mirroring contributor behavior
when (className) {
"List" -> listOf("Collection", "Iterable").forEach { dfs(it) }
"Array" -> listOf("Collection", "Iterable").forEach { dfs(it) }
}
return result
}
fun test_NoGlobalsAfterDot_IteratorFromLines() {
val code = """
import lyng.io.fs
import lyng.stdlib
val files = Path("../..").lines().<caret>
""".trimIndent()
val imported = listOf("lyng.io.fs", "lyng.stdlib")
ensureDocs(imported)
val items = complete(code)
// Must not propose globals after dot
assertFalse(items.contains("Path"))
assertFalse(items.contains("Array"))
assertFalse(items.contains("String"))
// Should contain a reasonable subset of Iterator members
val expected = aggregateMemberNames("Iterator", imported)
// At least one expected member must appear
val intersection = expected.intersect(items.toSet())
assertTrue("Expected Iterator members, but got: $items", intersection.isNotEmpty())
}
fun test_IteratorAfterLines_WithPrefix() {
val code = """
import lyng.io.fs
import lyng.stdlib
val files = Path("../..").lines().t<caret>
""".trimIndent()
val imported = listOf("lyng.io.fs", "lyng.stdlib")
ensureDocs(imported)
val items = complete(code)
// Must not propose globals after dot even with prefix
assertFalse(items.contains("Path"))
assertFalse(items.contains("Array"))
assertFalse(items.contains("String"))
// All suggestions should start with the typed prefix (case-insensitive)
assertTrue(items.all { it.startsWith("t", ignoreCase = true) })
// Some Iterator member starting with 't' should be present (e.g., toList)
val expected = aggregateMemberNames("Iterator", imported).filter { it.startsWith("t", true) }.toSet()
if (expected.isNotEmpty()) {
val intersection = expected.intersect(items.toSet())
assertTrue("Expected Iterator members with prefix 't', got: $items", intersection.isNotEmpty())
} else {
// If registry has no 't*' members, at least suggestions should not be empty
assertTrue(items.isNotEmpty())
}
}
fun test_ListLiteral_MembersWithInherited() {
val code = """
import lyng.stdlib
val x = [1,2,3].<caret>
""".trimIndent()
val imported = listOf("lyng.stdlib")
ensureDocs(imported)
val items = complete(code)
// Must not propose globals after dot
assertFalse(items.contains("Array"))
assertFalse(items.contains("String"))
assertFalse(items.contains("Path"))
// Expect members from List plus parents (Collection/Iterable)
val expected = aggregateMemberNames("List", imported)
val intersection = expected.intersect(items.toSet())
assertTrue("Expected List/Collection/Iterable members, got: $items", intersection.isNotEmpty())
// Heuristic: we expect more than a couple of items (not just size/toList)
assertTrue("Too few member suggestions after list literal: $items", items.size >= 3)
}
}
@@ -322,7 +322,7 @@ private suspend fun buildFsModule(module: ModuleScope, policy: FsAccessPolicy) {
// mkdirs(mustCreate: Bool=false)
addFnDoc(
name = "mkdirs",
doc = "Create directories (like `mkdir -p`). Optional `mustCreate` enforces error if target exists.",
doc = "Create directories (like `mkdir -p`). If `mustCreate` is true and the path already exists, the call fails. Otherwise it is a no‑op when the directory exists.",
params = listOf(ParamDoc("mustCreate", type("lyng.Bool"))),
moduleName = module.packageName
) {
@@ -336,7 +336,7 @@ private suspend fun buildFsModule(module: ModuleScope, policy: FsAccessPolicy) {
// move(to: Path|String, overwrite: Bool=false)
addFnDoc(
name = "move",
doc = "Move this path to a new location. `to` may be a `Path` or `String`. Use `overwrite` to replace existing target.",
doc = "Move this path to a new location. `to` may be a `Path` or `String`. When `overwrite` is false and the target exists, the operation fails (provider may throw `AccessDeniedException`).",
// types vary; keep generic description in doc
params = listOf(ParamDoc("to"), ParamDoc("overwrite", type("lyng.Bool"))),
moduleName = module.packageName
@@ -352,7 +352,7 @@ private suspend fun buildFsModule(module: ModuleScope, policy: FsAccessPolicy) {
// delete(mustExist: Bool=false, recursively: Bool=false)
addFnDoc(
name = "delete",
doc = "Delete this path. Optional flags: `mustExist` and `recursively`.",
doc = "Delete this path. `mustExist=true` causes failure if the path does not exist. `recursively=true` removes directories with their contents. Providers can throw `AccessDeniedException` on policy violations.",
params = listOf(ParamDoc("mustExist", type("lyng.Bool")), ParamDoc("recursively", type("lyng.Bool"))),
moduleName = module.packageName
) {
@@ -367,7 +367,7 @@ private suspend fun buildFsModule(module: ModuleScope, policy: FsAccessPolicy) {
// copy(to: Path|String, overwrite: Bool=false)
addFnDoc(
name = "copy",
doc = "Copy this path to a new location. `to` may be a `Path` or `String`. Use `overwrite` to replace existing target.",
doc = "Copy this path to a new location. `to` may be a `Path` or `String`. When `overwrite` is false and the target exists, the operation fails (provider may throw `AccessDeniedException`).",
params = listOf(ParamDoc("to"), ParamDoc("overwrite", type("lyng.Bool"))),
moduleName = module.packageName
) {
@@ -0,0 +1,78 @@
/*
* Filesystem module builtin docs registration, located in lyngio so core library
* does not depend on external packages. The IDEA plugin (and any other tooling)
* may reflectively call FsBuiltinDocs.ensure() to make sure docs are registered.
*/
package net.sergeych.lyngio.docs
import net.sergeych.lyng.miniast.BuiltinDocRegistry
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.type
object FsBuiltinDocs {
private var registered = false
fun ensure() {
if (registered) return
// Register docs immediately (not lazy) so tooling can see them without executing module builders
BuiltinDocRegistry.module("lyng.io.fs") {
// Class Path with a short summary
classDoc(
name = "Path",
doc = "Filesystem path class. Construct with a string: `Path(\"/tmp\")`."
) {
// Common instance methods (subset sufficient for Quick Docs)
method(
name = "exists",
doc = "Whether the path exists on the filesystem.",
returns = type("lyng.Bool")
)
method(
name = "isFile",
doc = "Whether the path exists and is a file.",
returns = type("lyng.Bool")
)
method(
name = "isDir",
doc = "Whether the path exists and is a directory.",
returns = type("lyng.Bool")
)
method(
name = "readUtf8",
doc = "Read the entire file as UTF-8 string.",
returns = type("lyng.String")
)
method(
name = "writeUtf8",
doc = "Write UTF-8 string to the file (overwrite).",
params = listOf(ParamDoc("text", type("lyng.String")))
)
method(
name = "bytes",
doc = "Iterate file content as `Buffer` chunks.",
params = listOf(ParamDoc("size", type("lyng.Int"))),
returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.Buffer")))
)
method(
name = "lines",
doc = "Iterate file as lines of text.",
returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.String")))
)
}
// Top-level exported constants
valDoc(
name = "Path",
doc = "Filesystem path class. Construct with a string: `Path(\"/tmp\")`.",
type = type("Path")
)
valDoc(
name = "Paths",
doc = "Alias of `Path` for those who prefer plural form.",
type = type("Path")
)
}
registered = true
}
}
+39 -17
View File
@@ -21,7 +21,7 @@ import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl
import org.jetbrains.kotlin.gradle.dsl.JvmTarget
group = "net.sergeych"
version = "1.0.6-SNAPSHOT"
version = "1.0.7-SNAPSHOT"
// Removed legacy buildscript classpath declarations; plugins are applied via the plugins DSL below
@@ -107,29 +107,40 @@ kotlin {
implementation(libs.kotlinx.coroutines.test)
}
}
val jvmTest by getting {
dependencies {
// Allow tests to load external docs like lyng.io.fs via registrar
implementation(project(":lyngio"))
}
}
}
}
// ---- Build-time generation of stdlib text from .lyng files into a Kotlin constant ----
// The .lyng source of the stdlib lives here (module-relative path):
val lyngStdlibDir = layout.projectDirectory.dir("stdlib/lyng")
// The generated Kotlin source will be placed here and added to commonMain sources:
val generatedLyngStdlibDir = layout.buildDirectory.dir("generated/source/lyngStdlib/commonMain/kotlin")
// Implemented as a proper task type compatible with Gradle Configuration Cache
val generateLyngStdlib by tasks.registering {
group = "build"
description = "Generate Kotlin source with embedded lyng stdlib text"
inputs.dir(lyngStdlibDir)
outputs.dir(generatedLyngStdlibDir)
// Simpler: opt out of configuration cache for this ad-hoc generator task
notCompatibleWithConfigurationCache("Uses dynamic file IO in doLast; trivial generator")
abstract class GenerateLyngStdlib : DefaultTask() {
@get:InputDirectory
@get:PathSensitive(PathSensitivity.RELATIVE)
abstract val sourceDir: DirectoryProperty
doLast {
@get:OutputDirectory
abstract val outputDir: DirectoryProperty
@TaskAction
fun generate() {
val targetPkg = "net.sergeych.lyng.stdlib_included"
val targetDir = generatedLyngStdlibDir.get().asFile.resolve(targetPkg.replace('.', '/'))
val pkgPath = targetPkg.replace('.', '/')
val outBase = outputDir.get().asFile
val targetDir = outBase.resolve(pkgPath)
targetDir.mkdirs()
val files = lyngStdlibDir.asFileTree.matching { include("**/*.lyng") }.files.sortedBy { it.name }
val srcDir = sourceDir.get().asFile
val files = srcDir.walkTopDown()
.filter { it.isFile && it.extension == "lyng" }
.sortedBy { it.name }
.toList()
val content = if (files.isEmpty()) "" else buildString {
files.forEachIndexed { idx, f ->
val text = f.readText()
@@ -138,13 +149,12 @@ val generateLyngStdlib by tasks.registering {
}
}
// Emit as a regular quoted Kotlin string to avoid triple-quote edge cases
fun escapeForQuoted(s: String): String = buildString {
for (ch in s) when (ch) {
'\\' -> append("\\\\")
'"' -> append("\\\"")
'\n' -> append("\\n")
'\r' -> {} // drop CR
'\r' -> {}
'\t' -> append("\\t")
else -> append(ch)
}
@@ -162,6 +172,18 @@ val generateLyngStdlib by tasks.registering {
}
}
// The .lyng source of the stdlib lives here (module-relative path):
val lyngStdlibDir = layout.projectDirectory.dir("stdlib/lyng")
// The generated Kotlin source will be placed here and added to commonMain sources:
val generatedLyngStdlibDir = layout.buildDirectory.dir("generated/source/lyngStdlib/commonMain/kotlin")
val generateLyngStdlib by tasks.registering(GenerateLyngStdlib::class) {
group = "build"
description = "Generate Kotlin source with embedded lyng stdlib text"
sourceDir.set(lyngStdlibDir)
outputDir.set(generatedLyngStdlibDir)
}
// Add the generated directory to commonMain sources
kotlin.sourceSets.named("commonMain") {
kotlin.srcDir(generatedLyngStdlibDir)
@@ -39,12 +39,18 @@ actual object ScopePool {
actual fun borrow(parent: Scope, args: Arguments, pos: Pos, thisObj: Obj): Scope {
val pool = pool()
val s = if (pool.isNotEmpty()) pool.removeLast() else Scope(parent, args, pos, thisObj)
return try {
if (s.parent !== parent || s.args !== args || s.pos !== pos || s.thisObj !== thisObj) {
s.resetForReuse(parent, args, pos, thisObj)
} else {
s.frameId = nextFrameId()
}
return s
s
} catch (e: IllegalStateException) {
if (e.message?.contains("cycle") == true && e.message?.contains("scope parent chain") == true) {
Scope(parent, args, pos, thisObj)
} else throw e
}
}
actual fun release(scope: Scope) {
@@ -38,35 +38,80 @@ class ClosureScope(val callScope: Scope, val closureScope: Scope) :
}
override fun get(name: String): ObjRecord? {
// Fast-path built-ins
if (name == "this") return thisObj.asReadonly
// Priority:
// 1) Locals and arguments declared in this lambda frame (including values defined before suspension)
// 2) Instance/class members of the captured receiver (`closureScope.thisObj`), e.g., fields like `coll`, `factor`
// 3) Symbols from the captured closure scope (its locals and parents)
// 2) Instance/class members of the captured receiver (`closureScope.thisObj`)
// 3) Symbols from the captured closure scope chain (locals/parents), ignoring nested ClosureScope overrides
// 4) Instance members of the caller's `this` (e.g., FlowBuilder.emit)
// 5) Fallback to the standard chain (this frame -> parent (callScope) -> class members)
// 5) Symbols from the caller chain (locals/parents), ignoring nested ClosureScope overrides
// 6) Special fallback for module pseudo-symbols (e.g., __PACKAGE__)
// First, prefer locals/arguments bound in this frame
// 1) Locals/arguments in this closure frame
super.objects[name]?.let { return it }
super.localBindings[name]?.let { return it }
// Prefer instance fields/methods declared on the captured receiver:
// First, resolve real instance fields stored in the instance scope (constructor vars like `coll`, `factor`)
// 2) Members on the captured receiver instance
(closureScope.thisObj as? net.sergeych.lyng.obj.ObjInstance)
?.instanceScope
?.objects
?.get(name)
?.let { return it }
// Then, try class-declared members (methods/properties declared in the class body)
closureScope.thisObj.objClass.getInstanceMemberOrNull(name)?.let { return it }
// Then delegate to the full closure scope chain (locals, parents, etc.)
closureScope.get(name)?.let { return it }
// 3) Closure scope chain (locals/parents + members), ignore ClosureScope overrides to prevent recursion
closureScope.chainLookupWithMembers(name)?.let { return it }
// Allow resolving instance members of the caller's `this` (e.g., FlowBuilder.emit)
// 4) Caller `this` members
callScope.thisObj.objClass.getInstanceMemberOrNull(name)?.let { return it }
// Fallback to the standard lookup chain: this frame -> parent (callScope) -> class members
return super.get(name)
// 5) Caller chain (locals/parents + members)
callScope.chainLookupWithMembers(name)?.let { return it }
// 6) Module pseudo-symbols (e.g., __PACKAGE__) — walk caller ancestry and query ModuleScope directly
if (name.startsWith("__")) {
var s: Scope? = callScope
val visited = HashSet<Long>(4)
while (s != null) {
if (!visited.add(s.frameId)) break
if (s is ModuleScope) return s.get(name)
s = s.parent
}
}
// 7) Direct module/global fallback: try to locate nearest ModuleScope and check its own locals
fun lookupInModuleAncestry(from: Scope): ObjRecord? {
var s: Scope? = from
val visited = HashSet<Long>(4)
while (s != null) {
if (!visited.add(s.frameId)) break
if (s is ModuleScope) {
s.objects[name]?.let { return it }
s.localBindings[name]?.let { return it }
// check immediate parent (root scope) locals/constants for globals like `delay`
val p = s.parent
if (p != null) {
p.objects[name]?.let { return it }
p.localBindings[name]?.let { return it }
p.thisObj.objClass.getInstanceMemberOrNull(name)?.let { return it }
}
return null
}
s = s.parent
}
return null
}
lookupInModuleAncestry(closureScope)?.let { return it }
lookupInModuleAncestry(callScope)?.let { return it }
// 8) Global root scope constants/functions (e.g., delay, yield) via current import provider
runCatching { this.currentImportProvider.rootScope.objects[name] }.getOrNull()?.let { return it }
// Final safe fallback: base scope lookup from this frame walking raw parents
return baseGetIgnoreClosure(name)
}
}
@@ -62,6 +62,108 @@ open class Scope(
*/
internal val localBindings: MutableMap<String, ObjRecord> = mutableMapOf()
/** Debug helper: ensure assigning [candidateParent] does not create a structural cycle. */
private fun ensureNoCycle(candidateParent: Scope?) {
if (candidateParent == null) return
var s: Scope? = candidateParent
var hops = 0
while (s != null && hops++ < 1024) {
if (s === this) {
// In production we silently ignore; for debugging throw an error to signal misuse
throw IllegalStateException("cycle detected in scope parent chain assignment")
}
s = s.parent
}
}
/**
* Internal lookup helpers that deliberately avoid invoking overridden `get` implementations
* (notably in ClosureScope) to prevent accidental ping-pong and infinite recursion across
* intertwined closure frames. They traverse the plain parent chain and consult only locals
* and bindings of each frame. Instance/class member fallback must be decided by the caller.
*/
private fun tryGetLocalRecord(s: Scope, name: String): ObjRecord? {
s.objects[name]?.let { return it }
s.localBindings[name]?.let { return it }
s.getSlotIndexOf(name)?.let { return s.getSlotRecord(it) }
return null
}
internal fun chainLookupIgnoreClosure(name: String): ObjRecord? {
var s: Scope? = this
// use frameId to detect unexpected structural cycles in the parent chain
val visited = HashSet<Long>(4)
while (s != null) {
if (!visited.add(s.frameId)) return null
tryGetLocalRecord(s, name)?.let { return it }
s = s.parent
}
return null
}
/**
* Perform base Scope.get semantics for this frame without delegating into parent.get
* virtual dispatch. This checks:
* - locals/bindings in this frame
* - walks raw parent chain for locals/bindings (ignoring ClosureScope-specific overrides)
* - finally falls back to this frame's `thisObj` instance/class members
*/
internal fun baseGetIgnoreClosure(name: String): ObjRecord? {
// 1) locals/bindings in this frame
tryGetLocalRecord(this, name)?.let { return it }
// 2) walk parents for plain locals/bindings only
var s = parent
val visited = HashSet<Long>(4)
while (s != null) {
if (!visited.add(s.frameId)) return null
tryGetLocalRecord(s, name)?.let { return it }
s = s.parent
}
// 3) fallback to instance/class members of this frame's thisObj
return thisObj.objClass.getInstanceMemberOrNull(name)
}
/**
* Walk the ancestry starting from this scope and try to resolve [name] against:
* - locals/bindings of each frame
* - then instance/class members of each frame's `thisObj`.
* This completely avoids invoking overridden `get` implementations, preventing
* ping-pong recursion between `ClosureScope` frames.
*/
internal fun chainLookupWithMembers(name: String): ObjRecord? {
var s: Scope? = this
val visited = HashSet<Long>(4)
while (s != null) {
if (!visited.add(s.frameId)) return null
tryGetLocalRecord(s, name)?.let { return it }
s.thisObj.objClass.getInstanceMemberOrNull(name)?.let { return it }
s = s.parent
}
return null
}
/**
* Create a non-pooled snapshot of this scope suitable for capturing as a closure environment.
* Copies locals, slots, and localBindings; preserves parent chain and class context.
*/
fun snapshotForClosure(): Scope {
val snap = Scope(parent, args, pos, thisObj)
snap.currentClassCtx = this.currentClassCtx
// copy locals and bindings
snap.objects.putAll(this.objects)
snap.localBindings.putAll(this.localBindings)
// copy slots map preserving indices
if (this.slotCount() > 0) {
var i = 0
while (i < this.slotCount()) {
val rec = this.getSlotRecord(i)
snap.allocateSlotFor(this.nameToSlot.entries.firstOrNull { it.value == i }?.key ?: "slot${'$'}i", rec)
i++
}
}
return snap
}
/**
* Hint internal collections to reduce reallocations for upcoming parameter/local assignments.
* Only effective for ArrayList-backed slots; maps are left as-is (rehashed lazily by JVM).
@@ -200,10 +302,9 @@ open class Scope(
* Clears locals and slots, assigns new frameId, and sets parent/args/pos/thisObj.
*/
fun resetForReuse(parent: Scope?, args: Arguments, pos: Pos, thisObj: Obj) {
this.parent = parent
this.args = args
this.pos = pos
this.thisObj = thisObj
// Fully detach from any previous chain/state first to avoid residual ancestry
// that could interact badly with the new parent and produce a cycle.
this.parent = null
this.skipScopeCreation = false
// fresh identity for PIC caches
this.frameId = nextFrameId()
@@ -212,6 +313,12 @@ open class Scope(
slots.clear()
nameToSlot.clear()
localBindings.clear()
// Now safe to validate and re-parent
ensureNoCycle(parent)
this.parent = parent
this.args = args
this.pos = pos
this.thisObj = thisObj
// Pre-size local slots for upcoming parameter assignment where possible
reserveLocalCapacity(args.list.size + 4)
}
@@ -220,7 +327,10 @@ open class Scope(
* Creates a new child scope using the provided arguments and optional `thisObj`.
*/
fun createChildScope(pos: Pos, args: Arguments = Arguments.EMPTY, newThisObj: Obj? = null): Scope =
Scope(this, args, pos, newThisObj ?: thisObj).also { it.reserveLocalCapacity(args.list.size + 4) }
Scope(this, args, pos, newThisObj ?: thisObj).also {
it.ensureNoCycle(it.parent)
it.reserveLocalCapacity(args.list.size + 4)
}
/**
* Execute a block inside a child frame. Guarded for future pooling via [PerfFlags.SCOPE_POOL].
@@ -249,7 +359,7 @@ open class Scope(
* @return A new instance of [Scope] initialized with the specified arguments and `thisObj`.
*/
fun createChildScope(args: Arguments = Arguments.EMPTY, newThisObj: Obj? = null): Scope =
Scope(this, args, pos, newThisObj ?: thisObj)
Scope(this, args, pos, newThisObj ?: thisObj).also { it.ensureNoCycle(it.parent) }
/**
* @return A child scope with the same arguments, position and [thisObj]
@@ -20,6 +20,9 @@ package net.sergeych.lyng
import kotlinx.coroutines.delay
import kotlinx.coroutines.yield
import net.sergeych.lyng.Script.Companion.defaultImportManager
import net.sergeych.lyng.miniast.addConstDoc
import net.sergeych.lyng.miniast.addVoidFnDoc
import net.sergeych.lyng.miniast.type
import net.sergeych.lyng.obj.*
import net.sergeych.lyng.pacman.ImportManager
import net.sergeych.lyng.stdlib_included.rootLyng
@@ -170,31 +173,44 @@ class Script(
val a = requiredArg<Obj>(0)
val b = requiredArg<Obj>(1)
if (a.compareTo(this, b) != 0)
raiseError(ObjAssertionFailedException(this,"Assertion failed: ${a.inspect(this)} == ${b.inspect(this)}"))
raiseError(
ObjAssertionFailedException(
this,
"Assertion failed: ${a.inspect(this)} == ${b.inspect(this)}"
)
)
}
// alias used in tests
addVoidFn("assertEqual") {
val a = requiredArg<Obj>(0)
val b = requiredArg<Obj>(1)
if (a.compareTo(this, b) != 0)
raiseError(ObjAssertionFailedException(this,"Assertion failed: ${a.inspect(this)} == ${b.inspect(this)}"))
raiseError(
ObjAssertionFailedException(
this,
"Assertion failed: ${a.inspect(this)} == ${b.inspect(this)}"
)
)
}
addVoidFn("assertNotEquals") {
val a = requiredArg<Obj>(0)
val b = requiredArg<Obj>(1)
if (a.compareTo(this, b) == 0)
raiseError(ObjAssertionFailedException(this,"Assertion failed: ${a.inspect(this)} != ${b.inspect(this)}"))
raiseError(
ObjAssertionFailedException(
this,
"Assertion failed: ${a.inspect(this)} != ${b.inspect(this)}"
)
)
}
addFn("assertThrows") {
val code = requireOnlyArg<Statement>()
val result = try {
code.execute(this)
null
}
catch( e: ExecutionError ) {
} catch (e: ExecutionError) {
e.errorObject
}
catch (_: ScriptError) {
} catch (_: ScriptError) {
ObjNull
}
result ?: raiseError(ObjAssertionFailedException(this, "Expected exception but nothing was thrown"))
@@ -276,9 +292,19 @@ class Script(
}
val pi = ObjReal(PI)
addConst("π", pi)
addConstDoc(
name = "π",
value = pi,
doc = "The mathematical constant pi (π).",
type = type("lyng.Real")
)
getOrCreateNamespace("Math").apply {
addConst("PI", pi)
addConstDoc(
name = "PI",
value = pi,
doc = "The mathematical constant pi (π) in the Math namespace.",
type = type("lyng.Real")
)
}
}
@@ -288,16 +314,44 @@ class Script(
rootLyng
)
addPackage("lyng.buffer") {
it.addConst("Buffer", ObjBuffer.type)
it.addConst("MutableBuffer", ObjMutableBuffer.type)
it.addConstDoc(
name = "Buffer",
value = ObjBuffer.type,
doc = "Immutable sequence of bytes. Use for binary data and IO.",
type = type("lyng.Class")
)
it.addConstDoc(
name = "MutableBuffer",
value = ObjMutableBuffer.type,
doc = "Mutable byte buffer. Supports in-place modifications.",
type = type("lyng.Class")
)
}
addPackage("lyng.serialization") {
it.addConst("Lynon", ObjLynonClass)
it.addConstDoc(
name = "Lynon",
value = ObjLynonClass,
doc = "Lynon serialization utilities: encode/decode data structures to a portable binary/text form.",
type = type("lyng.Class")
)
}
addPackage("lyng.time") {
it.addConst("Instant", ObjInstant.type)
it.addConst("Duration", ObjDuration.type)
it.addFn("delay") {
it.addConstDoc(
name = "Instant",
value = ObjInstant.type,
doc = "Point in time (epoch-based).",
type = type("lyng.Class")
)
it.addConstDoc(
name = "Duration",
value = ObjDuration.type,
doc = "Time duration with millisecond precision.",
type = type("lyng.Class")
)
it.addVoidFnDoc(
"delay",
doc = "Suspend for the given time. Accepts Duration, Int seconds, or Real seconds."
) {
val a = args.firstAndOnly()
when (a) {
is ObjInt -> delay(a.value * 1000)
@@ -305,7 +359,6 @@ class Script(
is ObjDuration -> delay(a.duration)
else -> raiseIllegalArgument("Expected Duration, Int or Real, got ${a.inspect(this)}")
}
ObjVoid
}
}
}
@@ -22,6 +22,7 @@
package net.sergeych.lyng.miniast
import net.sergeych.lyng.Pos
import net.sergeych.lyng.stdlib_included.rootLyng
// ---------------- Types DSL ----------------
@@ -67,8 +68,14 @@ object BuiltinDocRegistry : BuiltinDocSource {
}
override fun docsForModule(moduleName: String): List<MiniDecl> {
modules[moduleName]?.let { return it }
// Try lazy supplier once
// If module already present but we also have a lazy supplier for it, merge supplier once
modules[moduleName]?.let { existing ->
lazySuppliers.remove(moduleName)?.invoke()?.let { built ->
existing += built
}
return existing
}
// Try lazy supplier once when module is not present
val built = lazySuppliers.remove(moduleName)?.invoke()
if (built != null) {
val list = modules.getOrPut(moduleName) { mutableListOf() }
@@ -96,6 +103,22 @@ object BuiltinDocRegistry : BuiltinDocSource {
init {
registerLazy("lyng.stdlib") { buildStdlibDocs() }
}
/**
* List names of extension-like methods defined for [className] in the stdlib text (`root.lyng`).
* We do a lightweight regex scan like: `fun ClassName.methodName(` and collect distinct names.
*/
fun extensionMethodNamesFor(className: String): List<String> {
val src = try { rootLyng } catch (_: Throwable) { null } ?: return emptyList()
val out = LinkedHashSet<String>()
// Match lines like: fun String.trim(...)
val re = Regex("(?m)^\\s*fun\\s+${className}\\.([A-Za-z_][A-Za-z0-9_]*)\\s*\\(")
re.findAll(src).forEach { m ->
val name = m.groupValues.getOrNull(1)?.trim()
if (!name.isNullOrEmpty()) out.add(name)
}
return out.toList()
}
}
// ---------------- Builders ----------------
@@ -262,7 +285,125 @@ internal fun TypeDoc.toMiniTypeRef(): MiniTypeRef = when (this) {
// ---------------- Built-in module doc seeds ----------------
// Seed docs for lyng.stdlib lazily to avoid init-order coupling.
// ---------------- Inline docs support (.lyng source) ----------------
/**
* Lightweight, single-pass scanner that extracts inline doc comments from the stdlib .lyng source
* and associates them with declarations (top-level functions, classes, and class methods).
* It is intentionally conservative and only recognizes simple patterns present in stdlib sources.
*
* The scan is cached; performed at most once per process.
*/
private object StdlibInlineDocIndex {
private var built = false
// Keys for matching declaration docs
private sealed interface Key {
data class TopFun(val name: String) : Key
data class Clazz(val name: String) : Key
data class Method(val className: String, val name: String) : Key
}
private val docs: MutableMap<Key, String> = mutableMapOf()
private fun putIfAbsent(k: Key, doc: String) {
if (doc.isBlank()) return
if (!docs.containsKey(k)) docs[k] = doc.trim()
}
private fun buildOnce() {
if (built) return
built = true
val text = try { rootLyng } catch (_: Throwable) { null } ?: return
// Simple line-based scan. Collect a doc block immediately preceding a declaration.
val lines = text.lines()
val buf = mutableListOf<String>()
var inBlock = false
var prevWasComment = false
fun flushTo(key: Key) {
if (buf.isNotEmpty()) {
val raw = buf.joinToString("\n").trimEnd()
putIfAbsent(key, raw)
buf.clear()
}
}
for (rawLine in lines) {
val line = rawLine.trim()
when {
// Multiline block comment begin/end
line.startsWith("/*") && !inBlock -> {
inBlock = true
val inner = line.removePrefix("/*").let { l -> if (l.endsWith("*/")) l.removeSuffix("*/") else l }
buf += inner
prevWasComment = true
if (line.endsWith("*/")) inBlock = false
continue
}
inBlock -> {
val content = if (line.endsWith("*/")) {
inBlock = false
line.removeSuffix("*/")
} else line
// Trim leading '*' like Javadoc style
val t = content.trimStart()
buf += if (t.startsWith("*")) t.removePrefix("*").trimStart() else content
prevWasComment = true
continue
}
line.startsWith("//") -> {
buf += line.removePrefix("//")
prevWasComment = true
continue
}
line.isBlank() -> {
// Blank line breaks doc association unless it immediately follows a comment
if (!prevWasComment) buf.clear()
prevWasComment = false
continue
}
else -> {
// Non-comment, non-blank: try to match a declaration just after comments
if (buf.isNotEmpty()) {
// fun Class.name( ... )
val mExt = Regex("^fun\\s+([A-Za-z_][A-Za-z0-9_]*)\\.([A-Za-z_][A-Za-z0-9_]*)\\s*\\(").find(line)
if (mExt != null) {
val (cls, name) = mExt.destructured
flushTo(Key.Method(cls, name))
} else {
// fun name( ... )
val mTop = Regex("^fun\\s+([A-Za-z_][A-Za-z0-9_]*)\\s*\\(").find(line)
if (mTop != null) {
val (name) = mTop.destructured
flushTo(Key.TopFun(name))
} else {
// class Name
val mClass = Regex("^class\\s+([A-Za-z_][A-Za-z0-9_]*)\\b").find(line)
if (mClass != null) {
val (name) = mClass.destructured
flushTo(Key.Clazz(name))
} else {
// Unrecognized line – drop buffer to avoid leaking to unrelated code
buf.clear()
}
}
}
}
prevWasComment = false
}
}
}
}
// Public API: fetch doc text if present
fun topFunDoc(name: String): String? { buildOnce(); return docs[Key.TopFun(name)] }
fun classDoc(name: String): String? { buildOnce(); return docs[Key.Clazz(name)] }
fun methodDoc(className: String, name: String): String? { buildOnce(); return docs[Key.Method(className, name)] }
}
// Seed docs for lyng.stdlib lazily to avoid init-order coupling and prefer inline docs where present.
private fun buildStdlibDocs(): List<MiniDecl> {
val decls = mutableListOf<MiniDecl>()
// Use the same DSL builders to construct decls
@@ -270,7 +411,7 @@ private fun buildStdlibDocs(): List<MiniDecl> {
// Printing
mod.funDoc(
name = "print",
doc = """
doc = StdlibInlineDocIndex.topFunDoc("print") ?: """
Print values to the standard output without a trailing newline.
Accepts any number of arguments and prints them separated by a space.
""".trimIndent(),
@@ -279,7 +420,7 @@ private fun buildStdlibDocs(): List<MiniDecl> {
)
mod.funDoc(
name = "println",
doc = """
doc = StdlibInlineDocIndex.topFunDoc("println") ?: """
Print values to the standard output and append a newline.
Accepts any number of arguments and prints them separated by a space.
""".trimIndent(),
@@ -288,7 +429,7 @@ private fun buildStdlibDocs(): List<MiniDecl> {
// Caching helper
mod.funDoc(
name = "cached",
doc = """
doc = StdlibInlineDocIndex.topFunDoc("cached") ?: """
Wrap a `builder` into a zero-argument thunk that computes once and caches the result.
The first call invokes `builder()` and stores the value; subsequent calls return the cached value.
""".trimIndent(),
@@ -298,44 +439,44 @@ private fun buildStdlibDocs(): List<MiniDecl> {
// Math helpers (scalar versions)
fun math1(name: String) = mod.funDoc(
name = name,
doc = "Compute $name(x).",
doc = StdlibInlineDocIndex.topFunDoc(name) ?: "Compute $name(x).",
params = listOf(ParamDoc("x", type("lyng.Number")))
)
math1("sin"); math1("cos"); math1("tan"); math1("asin"); math1("acos"); math1("atan")
mod.funDoc(name = "floor", doc = "Round down the number to the nearest integer.", params = listOf(ParamDoc("x", type("lyng.Number"))))
mod.funDoc(name = "ceil", doc = "Round up the number to the nearest integer.", params = listOf(ParamDoc("x", type("lyng.Number"))))
mod.funDoc(name = "round", doc = "Round the number to the nearest integer.", params = listOf(ParamDoc("x", type("lyng.Number"))))
mod.funDoc(name = "floor", doc = StdlibInlineDocIndex.topFunDoc("floor") ?: "Round down the number to the nearest integer.", params = listOf(ParamDoc("x", type("lyng.Number"))))
mod.funDoc(name = "ceil", doc = StdlibInlineDocIndex.topFunDoc("ceil") ?: "Round up the number to the nearest integer.", params = listOf(ParamDoc("x", type("lyng.Number"))))
mod.funDoc(name = "round", doc = StdlibInlineDocIndex.topFunDoc("round") ?: "Round the number to the nearest integer.", params = listOf(ParamDoc("x", type("lyng.Number"))))
// Hyperbolic and inverse hyperbolic
math1("sinh"); math1("cosh"); math1("tanh"); math1("asinh"); math1("acosh"); math1("atanh")
// Exponentials and logarithms
mod.funDoc(name = "exp", doc = "Euler's exponential e^x.", params = listOf(ParamDoc("x", type("lyng.Number"))))
mod.funDoc(name = "ln", doc = "Natural logarithm (base e).", params = listOf(ParamDoc("x", type("lyng.Number"))))
mod.funDoc(name = "log10", doc = "Logarithm base 10.", params = listOf(ParamDoc("x", type("lyng.Number"))))
mod.funDoc(name = "log2", doc = "Logarithm base 2.", params = listOf(ParamDoc("x", type("lyng.Number"))))
mod.funDoc(name = "exp", doc = StdlibInlineDocIndex.topFunDoc("exp") ?: "Euler's exponential e^x.", params = listOf(ParamDoc("x", type("lyng.Number"))))
mod.funDoc(name = "ln", doc = StdlibInlineDocIndex.topFunDoc("ln") ?: "Natural logarithm (base e).", params = listOf(ParamDoc("x", type("lyng.Number"))))
mod.funDoc(name = "log10", doc = StdlibInlineDocIndex.topFunDoc("log10") ?: "Logarithm base 10.", params = listOf(ParamDoc("x", type("lyng.Number"))))
mod.funDoc(name = "log2", doc = StdlibInlineDocIndex.topFunDoc("log2") ?: "Logarithm base 2.", params = listOf(ParamDoc("x", type("lyng.Number"))))
// Power/roots and absolute value
mod.funDoc(
name = "pow",
doc = "Raise `x` to the power `y`.",
doc = StdlibInlineDocIndex.topFunDoc("pow") ?: "Raise `x` to the power `y`.",
params = listOf(ParamDoc("x", type("lyng.Number")), ParamDoc("y", type("lyng.Number")))
)
mod.funDoc(
name = "sqrt",
doc = "Square root of `x`.",
doc = StdlibInlineDocIndex.topFunDoc("sqrt") ?: "Square root of `x`.",
params = listOf(ParamDoc("x", type("lyng.Number")))
)
mod.funDoc(
name = "abs",
doc = "Absolute value of a number (works for Int and Real).",
doc = StdlibInlineDocIndex.topFunDoc("abs") ?: "Absolute value of a number (works for Int and Real).",
params = listOf(ParamDoc("x", type("lyng.Number")))
)
// Assertions and checks
mod.funDoc(
name = "assert",
doc = """
doc = StdlibInlineDocIndex.topFunDoc("assert") ?: """
Assert that `cond` is true, otherwise throw an `AssertionFailedException`.
Optionally provide a `message`.
""".trimIndent(),
@@ -343,17 +484,17 @@ private fun buildStdlibDocs(): List<MiniDecl> {
)
mod.funDoc(
name = "assertEquals",
doc = "Assert that `a == b`, otherwise throw an assertion error.",
doc = StdlibInlineDocIndex.topFunDoc("assertEquals") ?: "Assert that `a == b`, otherwise throw an assertion error.",
params = listOf(ParamDoc("a"), ParamDoc("b"))
)
mod.funDoc(
name = "assertNotEquals",
doc = "Assert that `a != b`, otherwise throw an assertion error.",
doc = StdlibInlineDocIndex.topFunDoc("assertNotEquals") ?: "Assert that `a != b`, otherwise throw an assertion error.",
params = listOf(ParamDoc("a"), ParamDoc("b"))
)
mod.funDoc(
name = "assertThrows",
doc = """
doc = StdlibInlineDocIndex.topFunDoc("assertThrows") ?: """
Execute `code` and return the thrown `Exception` object.
If nothing is thrown, an assertion error is raised.
""".trimIndent(),
@@ -364,123 +505,129 @@ private fun buildStdlibDocs(): List<MiniDecl> {
// Utilities
mod.funDoc(
name = "dynamic",
doc = "Wrap a value into a dynamic object that defers resolution to runtime.",
doc = StdlibInlineDocIndex.topFunDoc("dynamic") ?: "Wrap a value into a dynamic object that defers resolution to runtime.",
params = listOf(ParamDoc("value"))
)
mod.funDoc(
name = "require",
doc = "Require `cond` to be true, else throw `IllegalArgumentException` with optional `message`.",
doc = StdlibInlineDocIndex.topFunDoc("require") ?: "Require `cond` to be true, else throw `IllegalArgumentException` with optional `message`.",
params = listOf(ParamDoc("cond", type("lyng.Bool")), ParamDoc("message"))
)
mod.funDoc(
name = "check",
doc = "Check `cond` is true, else throw `IllegalStateException` with optional `message`.",
doc = StdlibInlineDocIndex.topFunDoc("check") ?: "Check `cond` is true, else throw `IllegalStateException` with optional `message`.",
params = listOf(ParamDoc("cond", type("lyng.Bool")), ParamDoc("message"))
)
mod.funDoc(
name = "traceScope",
doc = "Print a debug trace of the current scope chain with an optional label.",
doc = StdlibInlineDocIndex.topFunDoc("traceScope") ?: "Print a debug trace of the current scope chain with an optional label.",
params = listOf(ParamDoc("label", type("lyng.String")))
)
mod.funDoc(
name = "delay",
doc = "Suspend for the specified number of milliseconds.",
doc = StdlibInlineDocIndex.topFunDoc("delay") ?: "Suspend for the specified number of milliseconds.",
params = listOf(ParamDoc("ms", type("lyng.Number")))
)
// Concurrency helpers
mod.funDoc(
name = "launch",
doc = "Launch an asynchronous task and return a `Deferred`.",
doc = StdlibInlineDocIndex.topFunDoc("launch") ?: "Launch an asynchronous task and return a `Deferred`.",
params = listOf(ParamDoc("code")),
returns = type("lyng.Deferred")
)
mod.funDoc(
name = "yield",
doc = "Yield to the scheduler, allowing other tasks to run."
doc = StdlibInlineDocIndex.topFunDoc("yield") ?: "Yield to the scheduler, allowing other tasks to run."
)
mod.funDoc(
name = "flow",
doc = "Create a lazy iterable stream using the provided `builder`.",
doc = StdlibInlineDocIndex.topFunDoc("flow") ?: "Create a lazy iterable stream using the provided `builder`.",
params = listOf(ParamDoc("builder")),
returns = type("lyng.Iterable")
)
// Common Iterable helpers (document top-level extension-like APIs as class members)
mod.classDoc(name = "Iterable", doc = "Helper operations for iterable collections.") {
method(name = "filter", doc = "Filter elements by predicate.", params = listOf(ParamDoc("predicate")), returns = type("lyng.Iterable"))
method(name = "drop", doc = "Skip the first N elements.", params = listOf(ParamDoc("n", type("lyng.Int"))), returns = type("lyng.Iterable"))
method(name = "first", doc = "Return the first element or throw if empty.")
method(name = "last", doc = "Return the last element or throw if empty.")
method(name = "dropLast", doc = "Drop the last N elements.", params = listOf(ParamDoc("n", type("lyng.Int"))), returns = type("lyng.Iterable"))
method(name = "takeLast", doc = "Take the last N elements.", params = listOf(ParamDoc("n", type("lyng.Int"))), returns = type("lyng.List"))
method(name = "joinToString", doc = "Join elements into a string with an optional separator and transformer.", params = listOf(ParamDoc("prefix", type("lyng.String")), ParamDoc("transformer")), returns = type("lyng.String"))
method(name = "any", doc = "Return true if any element matches the predicate.", params = listOf(ParamDoc("predicate")), returns = type("lyng.Bool"))
method(name = "all", doc = "Return true if all elements match the predicate.", params = listOf(ParamDoc("predicate")), returns = type("lyng.Bool"))
method(name = "sum", doc = "Sum all elements; returns null for empty collections.", returns = type("lyng.Number", nullable = true))
method(name = "sumOf", doc = "Sum mapped values of elements; returns null for empty collections.", params = listOf(ParamDoc("f")))
method(name = "minOf", doc = "Minimum of mapped values.", params = listOf(ParamDoc("lambda")))
method(name = "maxOf", doc = "Maximum of mapped values.", params = listOf(ParamDoc("lambda")))
method(name = "sorted", doc = "Return elements sorted by natural order.", returns = type("lyng.Iterable"))
method(name = "sortedBy", doc = "Return elements sorted by the key selector.", params = listOf(ParamDoc("predicate")), returns = type("lyng.Iterable"))
method(name = "shuffled", doc = "Return a shuffled copy as a list.", returns = type("lyng.List"))
method(name = "map", doc = "Transform elements by applying `transform`.", params = listOf(ParamDoc("transform")), returns = type("lyng.Iterable"))
method(name = "toList", doc = "Collect elements of this iterable into a new list.", returns = type("lyng.List"))
mod.classDoc(name = "Iterable", doc = StdlibInlineDocIndex.classDoc("Iterable") ?: "Helper operations for iterable collections.") {
fun md(name: String, fallback: String) = StdlibInlineDocIndex.methodDoc("Iterable", name) ?: fallback
method(name = "filter", doc = md("filter", "Filter elements by predicate."), params = listOf(ParamDoc("predicate")), returns = type("lyng.Iterable"))
method(name = "drop", doc = md("drop", "Skip the first N elements."), params = listOf(ParamDoc("n", type("lyng.Int"))), returns = type("lyng.Iterable"))
method(name = "first", doc = md("first", "Return the first element or throw if empty."))
method(name = "last", doc = md("last", "Return the last element or throw if empty."))
method(name = "dropLast", doc = md("dropLast", "Drop the last N elements."), params = listOf(ParamDoc("n", type("lyng.Int"))), returns = type("lyng.Iterable"))
method(name = "takeLast", doc = md("takeLast", "Take the last N elements."), params = listOf(ParamDoc("n", type("lyng.Int"))), returns = type("lyng.List"))
method(name = "joinToString", doc = md("joinToString", "Join elements into a string with an optional separator and transformer."), params = listOf(ParamDoc("prefix", type("lyng.String")), ParamDoc("transformer")), returns = type("lyng.String"))
method(name = "any", doc = md("any", "Return true if any element matches the predicate."), params = listOf(ParamDoc("predicate")), returns = type("lyng.Bool"))
method(name = "all", doc = md("all", "Return true if all elements match the predicate."), params = listOf(ParamDoc("predicate")), returns = type("lyng.Bool"))
method(name = "sum", doc = md("sum", "Sum all elements; returns null for empty collections."), returns = type("lyng.Number", nullable = true))
method(name = "sumOf", doc = md("sumOf", "Sum mapped values of elements; returns null for empty collections."), params = listOf(ParamDoc("f")))
method(name = "minOf", doc = md("minOf", "Minimum of mapped values."), params = listOf(ParamDoc("lambda")))
method(name = "maxOf", doc = md("maxOf", "Maximum of mapped values."), params = listOf(ParamDoc("lambda")))
method(name = "sorted", doc = md("sorted", "Return elements sorted by natural order."), returns = type("lyng.Iterable"))
method(name = "sortedBy", doc = md("sortedBy", "Return elements sorted by the key selector."), params = listOf(ParamDoc("predicate")), returns = type("lyng.Iterable"))
method(name = "shuffled", doc = md("shuffled", "Return a shuffled copy as a list."), returns = type("lyng.List"))
method(name = "map", doc = md("map", "Transform elements by applying `transform`."), params = listOf(ParamDoc("transform")), returns = type("lyng.Iterable"))
method(name = "toList", doc = md("toList", "Collect elements of this iterable into a new list."), returns = type("lyng.List"))
}
// List helpers
mod.classDoc(name = "List", doc = "List-specific operations.", bases = listOf(type("Collection"), type("Iterable"))) {
method(name = "toString", doc = "Return string representation like [a,b,c].", returns = type("lyng.String"))
method(name = "sortBy", doc = "Sort list in-place by key selector.", params = listOf(ParamDoc("predicate")))
method(name = "sort", doc = "Sort list in-place by natural order.")
method(name = "toList", doc = "Return a shallow copy of this list (new list with the same elements).", returns = type("lyng.List"))
mod.classDoc(name = "List", doc = StdlibInlineDocIndex.classDoc("List") ?: "List-specific operations.", bases = listOf(type("Collection"), type("Iterable"))) {
fun md(name: String, fallback: String) = StdlibInlineDocIndex.methodDoc("List", name) ?: fallback
method(name = "toString", doc = md("toString", "Return string representation like [a,b,c]."), returns = type("lyng.String"))
method(name = "sortBy", doc = md("sortBy", "Sort list in-place by key selector."), params = listOf(ParamDoc("predicate")))
method(name = "sort", doc = md("sort", "Sort list in-place by natural order."))
method(name = "toList", doc = md("toList", "Return a shallow copy of this list (new list with the same elements)."), returns = type("lyng.List"))
}
// Collection helpers (supertype for sized collections)
mod.classDoc(name = "Collection", doc = "Collection operations common to sized collections.", bases = listOf(type("Iterable"))) {
method(name = "size", doc = "Number of elements in the collection.", returns = type("lyng.Int"))
method(name = "toList", doc = "Collect elements into a new list.", returns = type("lyng.List"))
mod.classDoc(name = "Collection", doc = StdlibInlineDocIndex.classDoc("Collection") ?: "Collection operations common to sized collections.", bases = listOf(type("Iterable"))) {
fun md(name: String, fallback: String) = StdlibInlineDocIndex.methodDoc("Collection", name) ?: fallback
method(name = "size", doc = md("size", "Number of elements in the collection."), returns = type("lyng.Int"))
method(name = "toList", doc = md("toList", "Collect elements into a new list."), returns = type("lyng.List"))
}
// Iterator helpers
mod.classDoc(name = "Iterator", doc = "Iterator protocol for sequential access.") {
method(name = "hasNext", doc = "Whether another element is available.", returns = type("lyng.Bool"))
method(name = "next", doc = "Return the next element.")
method(name = "cancelIteration", doc = "Stop the iteration early.")
method(name = "toList", doc = "Consume this iterator and collect elements into a list.", returns = type("lyng.List"))
mod.classDoc(name = "Iterator", doc = StdlibInlineDocIndex.classDoc("Iterator") ?: "Iterator protocol for sequential access.") {
fun md(name: String, fallback: String) = StdlibInlineDocIndex.methodDoc("Iterator", name) ?: fallback
method(name = "hasNext", doc = md("hasNext", "Whether another element is available."), returns = type("lyng.Bool"))
method(name = "next", doc = md("next", "Return the next element."))
method(name = "cancelIteration", doc = md("cancelIteration", "Stop the iteration early."))
method(name = "toList", doc = md("toList", "Consume this iterator and collect elements into a list."), returns = type("lyng.List"))
}
// Exceptions and utilities
mod.classDoc(name = "Exception", doc = "Exception helpers.") {
method(name = "printStackTrace", doc = "Print this exception and its stack trace to standard output.")
mod.classDoc(name = "Exception", doc = StdlibInlineDocIndex.classDoc("Exception") ?: "Exception helpers.") {
method(name = "printStackTrace", doc = StdlibInlineDocIndex.methodDoc("Exception", "printStackTrace") ?: "Print this exception and its stack trace to standard output.")
}
mod.classDoc(name = "String", doc = "String helpers.") {
method(name = "re", doc = "Compile this string into a regular expression.", returns = type("lyng.Regex"))
mod.classDoc(name = "String", doc = StdlibInlineDocIndex.classDoc("String") ?: "String helpers.") {
// Only include inline-source method here; Kotlin-embedded methods are now documented via DocHelpers near definitions.
method(name = "re", doc = StdlibInlineDocIndex.methodDoc("String", "re") ?: "Compile this string into a regular expression.", returns = type("lyng.Regex"))
}
// StackTraceEntry structure
mod.classDoc(name = "StackTraceEntry", doc = "Represents a single stack trace element.") {
mod.classDoc(name = "StackTraceEntry", doc = StdlibInlineDocIndex.classDoc("StackTraceEntry") ?: "Represents a single stack trace element.") {
// Fields are not present as declarations in root.lyng's class header docs. Keep seeded defaults.
field(name = "sourceName", doc = "Source (file) name.", type = type("lyng.String"))
field(name = "line", doc = "Line number (1-based).", type = type("lyng.Int"))
field(name = "column", doc = "Column number (0-based).", type = type("lyng.Int"))
field(name = "sourceString", doc = "The source line text.", type = type("lyng.String"))
method(name = "toString", doc = "Formatted representation: source:line:column: text.", returns = type("lyng.String"))
method(name = "toString", doc = StdlibInlineDocIndex.methodDoc("StackTraceEntry", "toString") ?: "Formatted representation: source:line:column: text.", returns = type("lyng.String"))
}
// Constants and namespaces
mod.valDoc(
name = "π",
doc = "The mathematical constant pi.",
doc = StdlibInlineDocIndex.topFunDoc("π") ?: "The mathematical constant pi.",
type = type("lyng.Real"),
mutable = false
)
mod.classDoc(name = "Math", doc = "Mathematical constants and helpers.") {
field(name = "PI", doc = "The mathematical constant pi.", type = type("lyng.Real"), isStatic = true)
field(name = "PI", doc = StdlibInlineDocIndex.methodDoc("Math", "PI") ?: "The mathematical constant pi.", type = type("lyng.Real"), isStatic = true)
}
decls += mod.build()
return decls
}
// (Registration is triggered from BuiltinDocRegistry.init)
// (Registration for external modules is provided by their own libraries)
@@ -0,0 +1,445 @@
/*
* Pure-Kotlin, PSI-free completion engine used for isolated tests and non-IDE harnesses.
* Mirrors the IntelliJ MVP logic: MiniAst + BuiltinDocRegistry + lenient imports.
*/
package net.sergeych.lyng.miniast
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.Script
import net.sergeych.lyng.Source
import net.sergeych.lyng.pacman.ImportProvider
/** Minimal completion item description (IDE-agnostic). */
data class CompletionItem(
val name: String,
val kind: Kind,
val tailText: String? = null,
val typeText: String? = null,
)
enum class Kind { Function, Class_, Value, Method, Field }
/**
* Platform-free, lenient import provider that never fails on unknown packages.
* Used to allow MiniAst parsing even when external modules are not present at runtime.
*/
class LenientImportProvider private constructor(root: net.sergeych.lyng.Scope) : ImportProvider(root) {
override suspend fun createModuleScope(pos: net.sergeych.lyng.Pos, packageName: String): net.sergeych.lyng.ModuleScope =
net.sergeych.lyng.ModuleScope(this, pos, packageName)
companion object {
fun create(): LenientImportProvider = LenientImportProvider(Script.defaultImportManager.rootScope)
}
}
object CompletionEngineLight {
suspend fun completeAtMarkerSuspend(textWithCaret: String, marker: String = "<caret>"): List<CompletionItem> {
val idx = textWithCaret.indexOf(marker)
require(idx >= 0) { "Caret marker '$marker' not found" }
val text = textWithCaret.replace(marker, "")
return completeSuspend(text, idx)
}
suspend fun completeSuspend(text: String, caret: Int): List<CompletionItem> {
// Ensure stdlib Obj*-defined docs (e.g., String methods) are initialized before registry lookup
StdlibDocsBootstrap.ensure()
val prefix = prefixAt(text, caret)
val mini = buildMiniAst(text)
// Build imported modules as a UNION of MiniAst-derived and textual extraction, always including stdlib
run {
// no-op block to keep local scope tidy
}
val fromMini: List<String> = mini?.let { DocLookupUtils.canonicalImportedModules(it) } ?: emptyList()
val fromText: List<String> = extractImportsFromText(text)
val imported: List<String> = LinkedHashSet<String>().apply {
fromMini.forEach { add(it) }
fromText.forEach { add(it) }
add("lyng.stdlib")
}.toList()
val cap = 200
val out = ArrayList<CompletionItem>(64)
// Member context detection: dot immediately before caret or before current word start
val word = wordRangeAt(text, caret)
val memberDot = findDotLeft(text, word?.first ?: caret)
if (memberDot != null) {
// 0) Try chained member call return type inference
guessReturnClassFromMemberCallBefore(text, memberDot, imported)?.let { cls ->
offerMembersAdd(out, prefix, imported, cls)
return out
}
// 0a) Top-level call before dot
guessReturnClassFromTopLevelCallBefore(text, memberDot, imported)?.let { cls ->
offerMembersAdd(out, prefix, imported, cls)
return out
}
// 0b) Across-known-callees (Iterable/Iterator/List preference)
guessReturnClassAcrossKnownCallees(text, memberDot, imported)?.let { cls ->
offerMembersAdd(out, prefix, imported, cls)
return out
}
// 1) Receiver inference fallback
guessReceiverClass(text, memberDot, imported)?.let { cls ->
offerMembersAdd(out, prefix, imported, cls)
return out
}
// In member context and unknown receiver/return type: show nothing (no globals after dot)
return out
}
// Global identifiers: params > local decls > imported > stdlib; Functions > Classes > Values; alphabetical
mini?.let { m ->
val decls = m.declarations
val funs = decls.filterIsInstance<MiniFunDecl>().sortedBy { it.name.lowercase() }
val classes = decls.filterIsInstance<MiniClassDecl>().sortedBy { it.name.lowercase() }
val vals = decls.filterIsInstance<MiniValDecl>().sortedBy { it.name.lowercase() }
funs.forEach { offerDeclAdd(out, prefix, it) }
classes.forEach { offerDeclAdd(out, prefix, it) }
vals.forEach { offerDeclAdd(out, prefix, it) }
}
// Imported and builtin
val (nonStd, std) = imported.partition { it != "lyng.stdlib" }
val order = nonStd + std
val emptyPrefixThrottle = prefix.isBlank()
var externalAdded = 0
val budget = if (emptyPrefixThrottle) 100 else Int.MAX_VALUE
for (mod in order) {
val decls = BuiltinDocRegistry.docsForModule(mod)
val funs = decls.filterIsInstance<MiniFunDecl>().sortedBy { it.name.lowercase() }
val classes = decls.filterIsInstance<MiniClassDecl>().sortedBy { it.name.lowercase() }
val vals = decls.filterIsInstance<MiniValDecl>().sortedBy { it.name.lowercase() }
funs.forEach { if (externalAdded < budget) { offerDeclAdd(out, prefix, it); externalAdded++ } }
classes.forEach { if (externalAdded < budget) { offerDeclAdd(out, prefix, it); externalAdded++ } }
vals.forEach { if (externalAdded < budget) { offerDeclAdd(out, prefix, it); externalAdded++ } }
if (out.size >= cap || externalAdded >= budget) break
}
return out
}
// --- Emission helpers ---
private fun offerDeclAdd(out: MutableList<CompletionItem>, prefix: String, d: MiniDecl) {
fun add(ci: CompletionItem) { if (ci.name.startsWith(prefix, true)) out += ci }
when (d) {
is MiniFunDecl -> {
val params = d.params.joinToString(", ") { it.name }
val tail = "(${params})"
add(CompletionItem(d.name, Kind.Function, tailText = tail, typeText = typeOf(d.returnType)))
}
is MiniClassDecl -> add(CompletionItem(d.name, Kind.Class_))
is MiniValDecl -> add(CompletionItem(d.name, Kind.Value, typeText = typeOf(d.type)))
// else -> add(CompletionItem(d.name, Kind.Value))
}
}
private fun offerMembersAdd(out: MutableList<CompletionItem>, prefix: String, imported: List<String>, className: String) {
val classes = DocLookupUtils.aggregateClasses(imported)
val visited = mutableSetOf<String>()
val directMap = LinkedHashMap<String, MutableList<MiniMemberDecl>>()
val inheritedMap = LinkedHashMap<String, MutableList<MiniMemberDecl>>()
fun addMembersOf(name: String, direct: Boolean) {
val cls = classes[name] ?: return
val target = if (direct) directMap else inheritedMap
for (m in cls.members) target.getOrPut(m.name) { mutableListOf() }.add(m)
for (b in cls.bases) if (visited.add(b)) addMembersOf(b, false)
}
visited.add(className)
addMembersOf(className, true)
// Conservative supplements for containers
when (className) {
"List" -> listOf("Collection", "Iterable").forEach { if (visited.add(it)) addMembersOf(it, false) }
"Array" -> listOf("Collection", "Iterable").forEach { if (visited.add(it)) addMembersOf(it, false) }
}
fun emitGroup(map: LinkedHashMap<String, MutableList<MiniMemberDecl>>) {
for (name in map.keys.sortedBy { it.lowercase() }) {
val variants = map[name] ?: continue
// Prefer a method with a known return type; else any method; else first variant
val rep =
variants.asSequence()
.filterIsInstance<MiniMemberFunDecl>()
.firstOrNull { it.returnType != null }
?: variants.firstOrNull { it is MiniMemberFunDecl }
?: variants.first()
when (rep) {
is MiniMemberFunDecl -> {
val params = rep.params.joinToString(", ") { it.name }
val extra = variants.count { it is MiniMemberFunDecl } - 1
val ov = if (extra > 0) " (+$extra overloads)" else ""
val ci = CompletionItem(name, Kind.Method, tailText = "(${params})$ov", typeText = typeOf(rep.returnType))
if (ci.name.startsWith(prefix, true)) out += ci
}
is MiniMemberValDecl -> {
// Prefer a field variant with known type if available
val chosen = variants.asSequence()
.filterIsInstance<MiniMemberValDecl>()
.firstOrNull { it.type != null } ?: rep
val ci = CompletionItem(name, Kind.Field, typeText = typeOf((chosen as MiniMemberValDecl).type))
if (ci.name.startsWith(prefix, true)) out += ci
}
}
}
}
emitGroup(directMap)
emitGroup(inheritedMap)
// Supplement with stdlib extension-like methods defined in root.lyng (e.g., fun String.re(...))
run {
val already = (directMap.keys + inheritedMap.keys).toMutableSet()
val ext = BuiltinDocRegistry.extensionMethodNamesFor(className)
for (name in ext) {
if (already.contains(name)) continue
val resolved = DocLookupUtils.resolveMemberWithInheritance(imported, className, name)
if (resolved != null) {
when (val member = resolved.second) {
is MiniMemberFunDecl -> {
val params = member.params.joinToString(", ") { it.name }
val ci = CompletionItem(name, Kind.Method, tailText = "(${params})", typeText = typeOf(member.returnType))
if (ci.name.startsWith(prefix, true)) out += ci
already.add(name)
}
is MiniMemberValDecl -> {
val ci = CompletionItem(name, Kind.Field, typeText = typeOf(member.type))
if (ci.name.startsWith(prefix, true)) out += ci
already.add(name)
}
}
} else {
// Fallback: emit simple method name without detailed types
val ci = CompletionItem(name, Kind.Method, tailText = "()", typeText = null)
if (ci.name.startsWith(prefix, true)) out += ci
already.add(name)
}
}
}
}
// --- Inference helpers (text-only, PSI-free) ---
private fun guessReceiverClass(text: String, dotPos: Int, imported: List<String>): String? {
DocLookupUtils.guessClassFromCallBefore(text, dotPos, imported)?.let { return it }
var i = prevNonWs(text, dotPos - 1)
if (i >= 0) {
when (text[i]) {
'"' -> return "String"
']' -> return "List"
'}' -> return "Dict"
')' -> {
// Parenthesized expression: walk back to matching '(' and inspect the inner expression
var j = i - 1
var depth = 0
while (j >= 0) {
when (text[j]) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
j--
}
if (j >= 0 && text[j] == '(') {
val innerS = (j + 1).coerceAtLeast(0)
val innerE = i.coerceAtMost(text.length)
if (innerS < innerE) {
val inner = text.substring(innerS, innerE).trim()
if (inner.startsWith('"') && inner.endsWith('"')) return "String"
if (inner.startsWith('[') && inner.endsWith(']')) return "List"
if (inner.startsWith('{') && inner.endsWith('}')) return "Dict"
}
}
}
}
// Numeric literal: decimal/int/hex/scientific
var j = i
var hasDigits = false
var hasDot = false
var hasExp = false
while (j >= 0) {
val ch = text[j]
if (ch.isDigit()) { hasDigits = true; j--; continue }
if (ch == '.') { hasDot = true; j--; continue }
if (ch == 'e' || ch == 'E') { hasExp = true; j--; if (j >= 0 && (text[j] == '+' || text[j] == '-')) j--; continue }
if (ch in listOf('x','X','a','b','c','d','f','A','B','C','D','F')) { j--; continue }
break
}
if (hasDigits) return if (hasDot || hasExp) "Real" else "Int"
}
return null
}
private fun guessReturnClassFromMemberCallBefore(text: String, dotPos: Int, imported: List<String>): String? {
var i = prevNonWs(text, dotPos - 1)
if (i < 0 || text[i] != ')') return null
i--
var depth = 0
while (i >= 0) {
when (text[i]) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
i--
}
if (i < 0 || text[i] != '(') return null
var j = i - 1
while (j >= 0 && text[j].isWhitespace()) j--
val end = j + 1
while (j >= 0 && isIdentChar(text[j])) j--
val start = j + 1
if (start >= end) return null
val callee = text.substring(start, end)
var k = start - 1
while (k >= 0 && text[k].isWhitespace()) k--
if (k < 0 || text[k] != '.') return null
val prevDot = k
val receiverClass = guessReceiverClass(text, prevDot, imported) ?: return null
val resolved = DocLookupUtils.resolveMemberWithInheritance(imported, receiverClass, callee) ?: return null
val member = resolved.second
val ret = when (member) {
is MiniMemberFunDecl -> member.returnType
is MiniMemberValDecl -> member.type
}
return simpleClassNameOf(ret)
}
private fun guessReturnClassFromTopLevelCallBefore(text: String, dotPos: Int, imported: List<String>): String? {
var i = prevNonWs(text, dotPos - 1)
if (i < 0 || text[i] != ')') return null
i--
var depth = 0
while (i >= 0) {
when (text[i]) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
i--
}
if (i < 0 || text[i] != '(') return null
var j = i - 1
while (j >= 0 && text[j].isWhitespace()) j--
val end = j + 1
while (j >= 0 && isIdentChar(text[j])) j--
val start = j + 1
if (start >= end) return null
val callee = text.substring(start, end)
var k = start - 1
while (k >= 0 && text[k].isWhitespace()) k--
if (k >= 0 && text[k] == '.') return null // was a member call
for (mod in imported) {
val decls = BuiltinDocRegistry.docsForModule(mod)
val fn = decls.asSequence().filterIsInstance<MiniFunDecl>().firstOrNull { it.name == callee }
if (fn != null) return simpleClassNameOf(fn.returnType)
}
return null
}
private fun guessReturnClassAcrossKnownCallees(text: String, dotPos: Int, imported: List<String>): String? {
var i = prevNonWs(text, dotPos - 1)
if (i < 0 || text[i] != ')') return null
i--
var depth = 0
while (i >= 0) {
when (text[i]) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
i--
}
if (i < 0 || text[i] != '(') return null
var j = i - 1
while (j >= 0 && text[j].isWhitespace()) j--
val end = j + 1
while (j >= 0 && isIdentChar(text[j])) j--
val start = j + 1
if (start >= end) return null
val callee = text.substring(start, end)
val resolved = DocLookupUtils.findMemberAcrossClasses(imported, callee) ?: return null
val member = resolved.second
val ret = when (member) {
is MiniMemberFunDecl -> member.returnType
is MiniMemberValDecl -> member.type
}
return simpleClassNameOf(ret)
}
private fun simpleClassNameOf(t: MiniTypeRef?): String? = when (t) {
null -> null
is MiniTypeName -> t.segments.lastOrNull()?.name
is MiniGenericType -> simpleClassNameOf(t.base)
is MiniFunctionType -> null
is MiniTypeVar -> null
}
// --- MiniAst and small utils ---
private suspend fun buildMiniAst(text: String): MiniScript? = try {
val sink = MiniAstBuilder()
val src = Source("<engine>", text)
val provider = LenientImportProvider.create()
Compiler.compileWithMini(src, provider, sink)
sink.build()
} catch (_: Throwable) {
null
}
private fun typeOf(t: MiniTypeRef?): String = when (t) {
null -> ""
is MiniTypeName -> t.segments.lastOrNull()?.name?.let { ": $it" } ?: ""
is MiniGenericType -> {
val base = typeOf(t.base).removePrefix(": ")
val args = t.args.joinToString(",") { typeOf(it).removePrefix(": ") }
": ${base}<${args}>"
}
is MiniFunctionType -> ": (fn)"
is MiniTypeVar -> ": ${t.name}"
}
// Note: we intentionally skip "params in scope" in the isolated engine to avoid PSI/offset mapping.
// Text helpers
private fun prefixAt(text: String, offset: Int): String {
val off = offset.coerceIn(0, text.length)
var i = (off - 1).coerceAtLeast(0)
while (i >= 0 && isIdentChar(text[i])) i--
val start = i + 1
return if (start in 0..text.length && start <= off) text.substring(start, off) else ""
}
private fun wordRangeAt(text: String, offset: Int): Pair<Int, Int>? {
if (text.isEmpty()) return null
val off = offset.coerceIn(0, text.length)
var s = off
var e = off
while (s > 0 && isIdentChar(text[s - 1])) s--
while (e < text.length && isIdentChar(text[e])) e++
return if (s < e) s to e else null
}
private fun findDotLeft(text: String, offset: Int): Int? {
var i = (offset - 1).coerceAtLeast(0)
while (i >= 0 && text[i].isWhitespace()) i--
return if (i >= 0 && text[i] == '.') i else null
}
private fun prevNonWs(text: String, start: Int): Int {
var i = start.coerceAtMost(text.length - 1)
while (i >= 0 && text[i].isWhitespace()) i--
return i
}
private fun isIdentChar(c: Char): Boolean = c == '_' || c.isLetterOrDigit()
private fun extractImportsFromText(text: String): List<String> {
val result = LinkedHashSet<String>()
val re = Regex("(?m)^\\s*import\\s+([a-zA-Z_][a-zA-Z0-9_]*(?:\\.[a-zA-Z_][a-zA-Z0-9_]*)*)")
re.findAll(text).forEach { m ->
val raw = m.groupValues.getOrNull(1)?.trim().orEmpty()
if (raw.isNotEmpty()) result.add(if (raw.startsWith("lyng.")) raw else "lyng.$raw")
}
return result.toList()
}
}
@@ -0,0 +1,142 @@
/*
* Shared QuickDoc lookup helpers reusable outside the IDEA plugin.
*/
package net.sergeych.lyng.miniast
object DocLookupUtils {
/**
* Convert MiniAst imports to fully-qualified module names expected by BuiltinDocRegistry.
* Heuristics:
* - If an import does not start with "lyng.", prefix it with "lyng." (e.g., "io.fs" -> "lyng.io.fs").
* - Keep original if it already starts with "lyng.".
* - Always include "lyng.stdlib" to make builtins available for docs.
*/
fun canonicalImportedModules(mini: MiniScript): List<String> {
val raw = mini.imports.map { it.segments.joinToString(".") { s -> s.name } }
val result = LinkedHashSet<String>()
for (name in raw) {
val canon = if (name.startsWith("lyng.")) name else "lyng.$name"
result.add(canon)
}
// Always make stdlib available as a fallback context for common types,
// even when there are no explicit imports in the file
result.add("lyng.stdlib")
return result.toList()
}
fun aggregateClasses(importedModules: List<String>): Map<String, MiniClassDecl> {
// Collect all class decls by name across modules, then merge duplicates by unioning members and bases.
val buckets = LinkedHashMap<String, MutableList<MiniClassDecl>>()
for (mod in importedModules) {
val docs = BuiltinDocRegistry.docsForModule(mod)
for (cls in docs.filterIsInstance<MiniClassDecl>()) {
buckets.getOrPut(cls.name) { mutableListOf() }.add(cls)
}
}
fun mergeClassDecls(name: String, list: List<MiniClassDecl>): MiniClassDecl {
if (list.isEmpty()) throw IllegalArgumentException("empty class list for $name")
if (list.size == 1) return list.first()
// Choose a representative for non-merge fields (range/nameStart/bodyRange): take the first
val rep = list.first()
val bases = LinkedHashSet<String>()
val members = LinkedHashMap<String, MutableList<MiniMemberDecl>>()
var doc: MiniDoc? = null
for (c in list) {
bases.addAll(c.bases)
if (doc == null && c.doc != null && c.doc.raw.isNotBlank()) doc = c.doc
for (m in c.members) {
// Group by name to keep overloads together
members.getOrPut(m.name) { mutableListOf() }.add(m)
}
}
// Flatten grouped members back to a list; keep stable name order
val mergedMembers = members.keys.sortedBy { it.lowercase() }.flatMap { members[it] ?: emptyList() }
return MiniClassDecl(
range = rep.range,
name = rep.name,
bases = bases.toList(),
bodyRange = rep.bodyRange,
ctorFields = rep.ctorFields,
classFields = rep.classFields,
doc = doc,
nameStart = rep.nameStart,
members = mergedMembers
)
}
val result = LinkedHashMap<String, MiniClassDecl>()
for ((name, list) in buckets) {
result[name] = mergeClassDecls(name, list)
}
return result
}
fun resolveMemberWithInheritance(importedModules: List<String>, className: String, member: String): Pair<String, MiniMemberDecl>? {
val classes = aggregateClasses(importedModules)
fun dfs(name: String, visited: MutableSet<String>): Pair<String, MiniMemberDecl>? {
val cls = classes[name] ?: return null
cls.members.firstOrNull { it.name == member }?.let { return name to it }
if (!visited.add(name)) return null
for (baseName in cls.bases) {
dfs(baseName, visited)?.let { return it }
}
return null
}
return dfs(className, mutableSetOf())
}
fun findMemberAcrossClasses(importedModules: List<String>, member: String): Pair<String, MiniMemberDecl>? {
val classes = aggregateClasses(importedModules)
// Preferred order for ambiguous common ops
val preference = listOf("Iterable", "Iterator", "List")
for (name in preference) {
resolveMemberWithInheritance(importedModules, name, member)?.let { return it }
}
for ((name, cls) in classes) {
cls.members.firstOrNull { it.name == member }?.let { return name to it }
}
return null
}
/**
* Try to guess a class name of the receiver when the receiver is a call like `ClassName(...)`.
* We walk left from the dot, find a matching `)` and then the identifier immediately before the `(`.
* If that identifier matches a known class in any of the imported modules, return it.
*/
fun guessClassFromCallBefore(text: String, dotPos: Int, importedModules: List<String>): String? {
var i = (dotPos - 1).coerceAtLeast(0)
// Skip spaces
while (i >= 0 && text[i].isWhitespace()) i++
// Note: the previous line is wrong direction; correct implementation below
i = (dotPos - 1).coerceAtLeast(0)
while (i >= 0 && text[i].isWhitespace()) i--
if (i < 0 || text[i] != ')') return null
// Walk back to matching '(' accounting nested parentheses
var depth = 0
i--
while (i >= 0) {
val ch = text[i]
when (ch) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
'\n' -> {}
}
i--
}
if (i < 0 || text[i] != '(') return null
// Now find identifier immediately before '('
var j = i - 1
while (j >= 0 && text[j].isWhitespace()) j--
val end = j + 1
while (j >= 0 && isIdentChar(text[j])) j--
val start = j + 1
if (start >= end) return null
val name = text.substring(start, end)
// Validate against imported classes
val classes = aggregateClasses(importedModules)
return if (classes.containsKey(name)) name else null
}
private fun isIdentChar(c: Char): Boolean = c == '_' || c.isLetterOrDigit()
}
@@ -0,0 +1,26 @@
/*
* Ensure stdlib Obj*-defined docs (like String methods added via ObjString.addFnDoc)
* are initialized before registry lookups for completion/quick docs.
*/
package net.sergeych.lyng.miniast
import net.sergeych.lyng.obj.ObjString
object StdlibDocsBootstrap {
// Simple idempotent guard; races are harmless as initializer side-effects are idempotent
private var ensured = false
fun ensure() {
if (ensured) return
try {
// Touch core Obj* types whose docs are registered via addFnDoc/addConstDoc
// Accessing .type forces their static initializers to run and register docs.
@Suppress("UNUSED_VARIABLE")
val _string = ObjString.type
} catch (_: Throwable) {
// Best-effort; absence should not break consumers
} finally {
ensured = true
}
}
}
@@ -17,6 +17,11 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
val ObjArray by lazy {
/**
@@ -25,19 +30,34 @@ val ObjArray by lazy {
ObjClass("Array", ObjCollection).apply {
// we can create iterators using size/getat:
addFn("iterator") {
ObjArrayIterator(thisObj).also { it.init(this) }
}
addFnDoc(
name = "iterator",
doc = "Iterator over elements of this array using its indexer.",
returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.Any"))),
moduleName = "lyng.stdlib"
) { ObjArrayIterator(thisObj).also { it.init(this) } }
addFn("contains", isOpen = true) {
addFnDoc(
name = "contains",
doc = "Whether the array contains the given element (by equality).",
params = listOf(ParamDoc("element")),
returns = type("lyng.Bool"),
isOpen = true,
moduleName = "lyng.stdlib"
) {
val obj = args.firstAndOnly()
for (i in 0..<thisObj.invokeInstanceMethod(this, "size").toInt()) {
if (thisObj.getAt(this, ObjInt(i.toLong())).compareTo(this, obj) == 0) return@addFn ObjTrue
if (thisObj.getAt(this, ObjInt(i.toLong())).compareTo(this, obj) == 0) return@addFnDoc ObjTrue
}
ObjFalse
}
addFn("last") {
addFnDoc(
name = "last",
doc = "The last element of this array.",
returns = type("lyng.Any"),
moduleName = "lyng.stdlib"
) {
thisObj.invokeInstanceMethod(
this,
"getAt",
@@ -45,13 +65,27 @@ val ObjArray by lazy {
)
}
addFn("lastIndex") { (thisObj.invokeInstanceMethod(this, "size").toInt() - 1).toObj() }
addFnDoc(
name = "lastIndex",
doc = "Index of the last element (size - 1).",
returns = type("lyng.Int"),
moduleName = "lyng.stdlib"
) { (thisObj.invokeInstanceMethod(this, "size").toInt() - 1).toObj() }
addFn("indices") {
ObjRange(0.toObj(), thisObj.invokeInstanceMethod(this, "size"), false)
}
addFnDoc(
name = "indices",
doc = "Range of valid indices for this array.",
returns = type("lyng.Range"),
moduleName = "lyng.stdlib"
) { ObjRange(0.toObj(), thisObj.invokeInstanceMethod(this, "size"), false) }
addFn("binarySearch") {
addFnDoc(
name = "binarySearch",
doc = "Binary search for a target in a sorted array. Returns index or negative insertion point - 1.",
params = listOf(ParamDoc("target")),
returns = type("lyng.Int"),
moduleName = "lyng.stdlib"
) {
val target = args.firstAndOnly()
var low = 0
var high = thisObj.invokeInstanceMethod(this, "size").toInt() - 1
@@ -62,13 +96,12 @@ val ObjArray by lazy {
val cmp = midVal.compareTo(this, target)
when {
cmp == 0 -> return@addFn (mid).toObj()
cmp == 0 -> return@addFnDoc (mid).toObj()
cmp > 0 -> high = mid - 1
else -> low = mid + 1
}
}
// Элемент не найден, возвращаем -(точка вставки) - 1
(-low - 1).toObj()
}
}
@@ -18,6 +18,8 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
class ObjChar(val value: Char): Obj() {
@@ -45,7 +47,12 @@ class ObjChar(val value: Char): Obj() {
companion object {
val type = ObjClass("Char").apply {
addFn("code") { ObjInt(thisAs<ObjChar>().value.code.toLong()) }
addFnDoc(
name = "code",
doc = "Unicode code point (UTF-16 code unit) of this character.",
returns = type("lyng.Int"),
moduleName = "lyng.stdlib"
) { ObjInt(thisAs<ObjChar>().value.code.toLong()) }
}
}
}
@@ -18,13 +18,73 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.*
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
import net.sergeych.lynon.LynonDecoder
import net.sergeych.lynon.LynonType
// Simple id generator for class identities (not thread-safe; fine for scripts)
private object ClassIdGen { var c: Long = 1L; fun nextId(): Long = c++ }
val ObjClassType by lazy { ObjClass("Class") }
val ObjClassType by lazy {
ObjClass("Class").apply {
addFnDoc(
name = "name",
doc = "Simple name of this class (without package).",
returns = type("lyng.String"),
moduleName = "lyng.stdlib"
) { thisAs<ObjClass>().classNameObj }
addFnDoc(
name = "fields",
doc = "Declared instance fields of this class and its ancestors (C3 order), without duplicates.",
returns = TypeGenericDoc(type("lyng.List"), listOf(type("lyng.String"))),
moduleName = "lyng.stdlib"
) {
val cls = thisAs<ObjClass>()
val seen = hashSetOf<String>()
val names = mutableListOf<Obj>()
for (c in cls.mro) {
for ((n, rec) in c.members) {
if (rec.value !is Statement && seen.add(n)) names += ObjString(n)
}
}
ObjList(names.toMutableList())
}
addFnDoc(
name = "methods",
doc = "Declared instance methods of this class and its ancestors (C3 order), without duplicates.",
returns = TypeGenericDoc(type("lyng.List"), listOf(type("lyng.String"))),
moduleName = "lyng.stdlib"
) {
val cls = thisAs<ObjClass>()
val seen = hashSetOf<String>()
val names = mutableListOf<Obj>()
for (c in cls.mro) {
for ((n, rec) in c.members) {
if (rec.value is Statement && seen.add(n)) names += ObjString(n)
}
}
ObjList(names.toMutableList())
}
addFnDoc(
name = "get",
doc = "Lookup a member by name in this class (including ancestors) and return it, or null if absent.",
params = listOf(ParamDoc("name", type("lyng.String"))),
returns = type("lyng.Any", nullable = true),
moduleName = "lyng.stdlib"
) {
val cls = thisAs<ObjClass>()
val name = requiredArg<ObjString>(0).value
val rec = cls.getInstanceMemberOrNull(name)
rec?.value ?: ObjNull
}
}
}
open class ObjClass(
val className: String,
@@ -19,6 +19,9 @@ package net.sergeych.lyng.obj
import kotlinx.coroutines.CompletableDeferred
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
class ObjCompletableDeferred(val completableDeferred: CompletableDeferred<Obj>): ObjDeferred(completableDeferred) {
@@ -30,7 +33,13 @@ class ObjCompletableDeferred(val completableDeferred: CompletableDeferred<Obj>):
return ObjCompletableDeferred(CompletableDeferred())
}
}.apply {
addFn("complete") {
addFnDoc(
name = "complete",
doc = "Complete this deferred with the given value. Subsequent calls have no effect.",
params = listOf(ParamDoc("value")),
returns = type("lyng.Void"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjCompletableDeferred>().completableDeferred.complete(args.firstAndOnly())
ObjVoid
}
@@ -19,6 +19,8 @@ package net.sergeych.lyng.obj
import kotlinx.coroutines.Deferred
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
open class ObjDeferred(val deferred: Deferred<Obj>): Obj() {
@@ -30,20 +32,33 @@ open class ObjDeferred(val deferred: Deferred<Obj>): Obj() {
scope.raiseError("Deferred constructor is not directly callable")
}
}.apply {
addFn("await") {
thisAs<ObjDeferred>().deferred.await()
}
addFn("isCompleted") {
thisAs<ObjDeferred>().deferred.isCompleted.toObj()
}
addFn("isActive") {
addFnDoc(
name = "await",
doc = "Suspend until completion and return the result value (or throw if failed).",
returns = type("lyng.Any"),
moduleName = "lyng.stdlib"
) { thisAs<ObjDeferred>().deferred.await() }
addFnDoc(
name = "isCompleted",
doc = "Whether this deferred has completed (successfully or with an error).",
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) { thisAs<ObjDeferred>().deferred.isCompleted.toObj() }
addFnDoc(
name = "isActive",
doc = "Whether this deferred is currently active (not completed and not cancelled).",
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) {
val d = thisAs<ObjDeferred>().deferred
// Cross-engine tolerant: prefer Deferred.isActive; otherwise treat any not-yet-completed and not-cancelled as active
(d.isActive || (!d.isCompleted && !d.isCancelled)).toObj()
}
addFn("isCancelled") {
thisAs<ObjDeferred>().deferred.isCancelled.toObj()
}
addFnDoc(
name = "isCancelled",
doc = "Whether this deferred was cancelled.",
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) { thisAs<ObjDeferred>().deferred.isCancelled.toObj() }
}
}
@@ -18,6 +18,8 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
import kotlin.time.Duration
import kotlin.time.Duration.Companion.days
import kotlin.time.Duration.Companion.hours
@@ -72,91 +74,169 @@ class ObjDuration(val duration: Duration) : Obj() {
)
}
}.apply {
addFn("days") {
thisAs<ObjDuration>().duration.toDouble(DurationUnit.DAYS).toObj()
}
addFn("hours") {
thisAs<ObjDuration>().duration.toDouble(DurationUnit.HOURS).toObj()
}
addFn("minutes") {
thisAs<ObjDuration>().duration.toDouble(DurationUnit.MINUTES).toObj()
}
addFn("seconds") {
thisAs<ObjDuration>().duration.toDouble(DurationUnit.SECONDS).toObj()
}
addFn("milliseconds") {
thisAs<ObjDuration>().duration.toDouble(DurationUnit.MILLISECONDS).toObj()
}
addFn("microseconds") {
thisAs<ObjDuration>().duration.toDouble(DurationUnit.MICROSECONDS).toObj()
}
addFnDoc(
name = "days",
doc = "Return this duration as a real number of days.",
returns = type("lyng.Real"),
moduleName = "lyng.time"
) { thisAs<ObjDuration>().duration.toDouble(DurationUnit.DAYS).toObj() }
addFnDoc(
name = "hours",
doc = "Return this duration as a real number of hours.",
returns = type("lyng.Real"),
moduleName = "lyng.time"
) { thisAs<ObjDuration>().duration.toDouble(DurationUnit.HOURS).toObj() }
addFnDoc(
name = "minutes",
doc = "Return this duration as a real number of minutes.",
returns = type("lyng.Real"),
moduleName = "lyng.time"
) { thisAs<ObjDuration>().duration.toDouble(DurationUnit.MINUTES).toObj() }
addFnDoc(
name = "seconds",
doc = "Return this duration as a real number of seconds.",
returns = type("lyng.Real"),
moduleName = "lyng.time"
) { thisAs<ObjDuration>().duration.toDouble(DurationUnit.SECONDS).toObj() }
addFnDoc(
name = "milliseconds",
doc = "Return this duration as a real number of milliseconds.",
returns = type("lyng.Real"),
moduleName = "lyng.time"
) { thisAs<ObjDuration>().duration.toDouble(DurationUnit.MILLISECONDS).toObj() }
addFnDoc(
name = "microseconds",
doc = "Return this duration as a real number of microseconds.",
returns = type("lyng.Real"),
moduleName = "lyng.time"
) { thisAs<ObjDuration>().duration.toDouble(DurationUnit.MICROSECONDS).toObj() }
// extensions
ObjInt.type.addFn("seconds") {
ObjDuration(thisAs<ObjInt>().value.seconds)
}
ObjInt.type.addFnDoc(
name = "seconds",
doc = "Construct a `Duration` equal to this integer number of seconds.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjInt>().value.seconds) }
ObjInt.type.addFn("second") {
ObjDuration(thisAs<ObjInt>().value.seconds)
}
ObjInt.type.addFn("milliseconds") {
ObjDuration(thisAs<ObjInt>().value.milliseconds)
}
ObjInt.type.addFnDoc(
name = "second",
doc = "Construct a `Duration` equal to this integer number of seconds.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjInt>().value.seconds) }
ObjInt.type.addFnDoc(
name = "milliseconds",
doc = "Construct a `Duration` equal to this integer number of milliseconds.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjInt>().value.milliseconds) }
ObjInt.type.addFn("millisecond") {
ObjDuration(thisAs<ObjInt>().value.milliseconds)
}
ObjReal.type.addFn("seconds") {
ObjDuration(thisAs<ObjReal>().value.seconds)
}
ObjInt.type.addFnDoc(
name = "millisecond",
doc = "Construct a `Duration` equal to this integer number of milliseconds.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjInt>().value.milliseconds) }
ObjReal.type.addFnDoc(
name = "seconds",
doc = "Construct a `Duration` equal to this real number of seconds.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjReal>().value.seconds) }
ObjReal.type.addFn("second") {
ObjDuration(thisAs<ObjReal>().value.seconds)
}
ObjReal.type.addFnDoc(
name = "second",
doc = "Construct a `Duration` equal to this real number of seconds.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjReal>().value.seconds) }
ObjReal.type.addFn("milliseconds") {
ObjDuration(thisAs<ObjReal>().value.milliseconds)
}
ObjReal.type.addFn("millisecond") {
ObjDuration(thisAs<ObjReal>().value.milliseconds)
}
ObjReal.type.addFnDoc(
name = "milliseconds",
doc = "Construct a `Duration` equal to this real number of milliseconds.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjReal>().value.milliseconds) }
ObjReal.type.addFnDoc(
name = "millisecond",
doc = "Construct a `Duration` equal to this real number of milliseconds.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjReal>().value.milliseconds) }
ObjInt.type.addFn("minutes") {
ObjDuration(thisAs<ObjInt>().value.minutes)
}
ObjReal.type.addFn("minutes") {
ObjDuration(thisAs<ObjReal>().value.minutes)
}
ObjInt.type.addFn("minute") {
ObjDuration(thisAs<ObjInt>().value.minutes)
}
ObjReal.type.addFn("minute") {
ObjDuration(thisAs<ObjReal>().value.minutes)
}
ObjInt.type.addFn("hours") {
ObjDuration(thisAs<ObjInt>().value.hours)
}
ObjReal.type.addFn("hours") {
ObjDuration(thisAs<ObjReal>().value.hours)
}
ObjInt.type.addFn("hour") {
ObjDuration(thisAs<ObjInt>().value.hours)
}
ObjReal.type.addFn("hour") {
ObjDuration(thisAs<ObjReal>().value.hours)
}
ObjInt.type.addFn("days") {
ObjDuration(thisAs<ObjInt>().value.days)
}
ObjReal.type.addFn("days") {
ObjDuration(thisAs<ObjReal>().value.days)
}
ObjInt.type.addFn("day") {
ObjDuration(thisAs<ObjInt>().value.days)
}
ObjReal.type.addFn("day") {
ObjDuration(thisAs<ObjReal>().value.days)
}
ObjInt.type.addFnDoc(
name = "minutes",
doc = "Construct a `Duration` equal to this integer number of minutes.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjInt>().value.minutes) }
ObjReal.type.addFnDoc(
name = "minutes",
doc = "Construct a `Duration` equal to this real number of minutes.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjReal>().value.minutes) }
ObjInt.type.addFnDoc(
name = "minute",
doc = "Construct a `Duration` equal to this integer number of minutes.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjInt>().value.minutes) }
ObjReal.type.addFnDoc(
name = "minute",
doc = "Construct a `Duration` equal to this real number of minutes.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjReal>().value.minutes) }
ObjInt.type.addFnDoc(
name = "hours",
doc = "Construct a `Duration` equal to this integer number of hours.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjInt>().value.hours) }
ObjReal.type.addFnDoc(
name = "hours",
doc = "Construct a `Duration` equal to this real number of hours.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjReal>().value.hours) }
ObjInt.type.addFnDoc(
name = "hour",
doc = "Construct a `Duration` equal to this integer number of hours.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjInt>().value.hours) }
ObjReal.type.addFnDoc(
name = "hour",
doc = "Construct a `Duration` equal to this real number of hours.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjReal>().value.hours) }
ObjInt.type.addFnDoc(
name = "days",
doc = "Construct a `Duration` equal to this integer number of days.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjInt>().value.days) }
ObjReal.type.addFnDoc(
name = "days",
doc = "Construct a `Duration` equal to this real number of days.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjReal>().value.days) }
ObjInt.type.addFnDoc(
name = "day",
doc = "Construct a `Duration` equal to this integer number of days.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjInt>().value.days) }
ObjReal.type.addFnDoc(
name = "day",
doc = "Construct a `Duration` equal to this real number of days.",
returns = type("lyng.Duration"),
moduleName = "lyng.time"
) { ObjDuration(thisAs<ObjReal>().value.days) }
// addFn("epochSeconds") {
@@ -18,6 +18,7 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.Arguments
import net.sergeych.lyng.ClosureScope
import net.sergeych.lyng.Scope
import net.sergeych.lyng.Statement
@@ -54,13 +55,16 @@ class ObjDynamicContext(val delegate: ObjDynamic) : Obj() {
open class ObjDynamic(var readCallback: Statement? = null, var writeCallback: Statement? = null) : Obj() {
override val objClass: ObjClass = type
// Capture the lexical scope used to build this dynamic so callbacks can see outer locals
internal var builderScope: Scope? = null
/**
* Use read callback to dynamically resolve the field name. Note that it does not work
* with method invocation which is implemented separately in [invokeInstanceMethod] below.
*/
override suspend fun readField(scope: Scope, name: String): ObjRecord {
return readCallback?.execute(scope.createChildScope(Arguments(ObjString(name))))?.let {
val execBase = builderScope?.let { ClosureScope(scope, it) } ?: scope
return readCallback?.execute(execBase.createChildScope(Arguments(ObjString(name))))?.let {
if (writeCallback != null)
it.asMutable
else
@@ -79,28 +83,27 @@ open class ObjDynamic(var readCallback: Statement? = null, var writeCallback: St
args: Arguments,
onNotFoundResult: (() -> Obj?)?
): Obj {
val over = readCallback?.execute(
scope.createChildScope(
Arguments(ObjString(name)
)
)
)
val execBase = builderScope?.let { ClosureScope(scope, it) } ?: scope
val over = readCallback?.execute(execBase.createChildScope(Arguments(ObjString(name))))
return over?.invoke(scope, scope.thisObj, args)
?: super.invokeInstanceMethod(scope, name, args, onNotFoundResult)
}
override suspend fun writeField(scope: Scope, name: String, newValue: Obj) {
writeCallback?.execute(scope.createChildScope(Arguments(ObjString(name), newValue)))
val execBase = builderScope?.let { ClosureScope(scope, it) } ?: scope
writeCallback?.execute(execBase.createChildScope(Arguments(ObjString(name), newValue)))
?: super.writeField(scope, name, newValue)
}
override suspend fun getAt(scope: Scope, index: Obj): Obj {
return readCallback?.execute(scope.createChildScope(Arguments(index)))
val execBase = builderScope?.let { ClosureScope(scope, it) } ?: scope
return readCallback?.execute(execBase.createChildScope(Arguments(index)))
?: super.getAt(scope, index)
}
override suspend fun putAt(scope: Scope, index: Obj, newValue: Obj) {
writeCallback?.execute(scope.createChildScope(Arguments(index, newValue)))
val execBase = builderScope?.let { ClosureScope(scope, it) } ?: scope
writeCallback?.execute(execBase.createChildScope(Arguments(index, newValue)))
?: super.putAt(scope, index, newValue)
}
@@ -109,7 +112,12 @@ open class ObjDynamic(var readCallback: Statement? = null, var writeCallback: St
suspend fun create(scope: Scope, builder: Statement): ObjDynamic {
val delegate = ObjDynamic()
val context = ObjDynamicContext(delegate)
builder.execute(scope.createChildScope(newThisObj = context))
// Capture the function's lexical scope (scope) so callbacks can see outer locals like parameters.
// Build the dynamic in a child scope purely to set `this` to context, but keep captured closure at parent.
val buildScope = scope.createChildScope(newThisObj = context)
// Snapshot the caller scope to capture locals/args even if the runtime pools/reuses frames
delegate.builderScope = scope.snapshotForClosure()
builder.execute(buildScope)
return delegate
}
@@ -19,6 +19,10 @@ package net.sergeych.lyng.obj
import net.sergeych.bintools.encodeToHex
import net.sergeych.lyng.*
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.addConstDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
import net.sergeych.lynon.LynonDecoder
import net.sergeych.lynon.LynonEncoder
import net.sergeych.lynon.LynonType
@@ -133,10 +137,21 @@ open class ObjException(
}
val Root = ExceptionClass("Exception").apply {
addConst("message", statement {
addConstDoc(
name = "message",
value = statement {
(thisObj as ObjException).message.toObj()
})
addFn("stackTrace") {
},
doc = "Human‑readable error message.",
type = type("lyng.String"),
moduleName = "lyng.stdlib"
)
addFnDoc(
name = "stackTrace",
doc = "Stack trace captured at throw site as a list of `StackTraceEntry`.",
returns = TypeGenericDoc(type("lyng.List"), listOf(type("lyng.StackTraceEntry"))),
moduleName = "lyng.stdlib"
) {
(thisObj as ObjException).getStackTrace()
}
}
@@ -183,6 +198,10 @@ open class ObjException(
)) {
scope.addConst(name, getOrCreateExceptionClass(name))
}
// Backward compatibility alias used in older tests/docs
val snd = getOrCreateExceptionClass("SymbolNotDefinedException")
scope.addConst("SymbolNotFound", snd)
existingErrorClasses["SymbolNotFound"] = snd
}
}
}
@@ -25,6 +25,10 @@ import kotlinx.coroutines.channels.SendChannel
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import net.sergeych.lyng.*
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
import net.sergeych.mp_tools.globalLaunch
import kotlin.coroutines.cancellation.CancellationException
@@ -36,7 +40,13 @@ class ObjFlowBuilder(val output: SendChannel<Obj>) : Obj() {
companion object {
@OptIn(DelicateCoroutinesApi::class)
val type = object : ObjClass("FlowBuilder") {}.apply {
addFn("emit") {
addFnDoc(
name = "emit",
doc = "Send a value to the flow consumer. Suspends if back‑pressured; no‑ops after consumer stops.",
params = listOf(ParamDoc("value", type("lyng.Any"))),
returns = type("lyng.Void"),
moduleName = "lyng.stdlib"
) {
val data = requireOnlyArg<Obj>()
try {
val channel = thisAs<ObjFlowBuilder>().output
@@ -47,7 +57,6 @@ class ObjFlowBuilder(val output: SendChannel<Obj>) : Obj() {
throw ScriptFlowIsNoMoreCollected()
} catch (x: Exception) {
// Any failure to send (including closed channel) should gracefully stop the producer.
// Do not print stack traces here to keep test output clean on JVM.
if (x is CancellationException) {
// Cancellation is a normal control-flow event
throw ScriptFlowIsNoMoreCollected()
@@ -90,7 +99,12 @@ class ObjFlow(val producer: Statement, val scope: Scope) : Obj() {
scope.raiseError("Flow constructor is not available")
}
}.apply {
addFn("iterator") {
addFnDoc(
name = "iterator",
doc = "Create a pull‑based iterator over this flow. Each step resumes the producer as needed.",
returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.Any"))),
moduleName = "lyng.stdlib"
) {
val objFlow = thisAs<ObjFlow>()
ObjFlowIterator(statement {
objFlow.producer.execute(
@@ -146,14 +160,27 @@ class ObjFlowIterator(val producer: Statement) : Obj() {
val type = object : ObjClass("FlowIterator", ObjIterator) {
}.apply {
addFn("hasNext") {
thisAs<ObjFlowIterator>().hasNext(this).toObj()
}
addFn("next") {
addFnDoc(
name = "hasNext",
doc = "Whether another element is available from the flow.",
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) { thisAs<ObjFlowIterator>().hasNext(this).toObj() }
addFnDoc(
name = "next",
doc = "Receive the next element from the flow or throw if completed.",
returns = type("lyng.Any"),
moduleName = "lyng.stdlib"
) {
val x = thisAs<ObjFlowIterator>()
x.next(this)
}
addFn("cancelIteration") {
addFnDoc(
name = "cancelIteration",
doc = "Stop iteration and cancel the underlying flow producer.",
returns = type("lyng.Void"),
moduleName = "lyng.stdlib"
) {
val x = thisAs<ObjFlowIterator>()
x.cancel()
ObjVoid
@@ -197,9 +197,20 @@ class ObjInstance(override val objClass: ObjClass) : Obj() {
!it.key.contains("::") && it.value.visibility.isPublic && it.value.type.serializable
}
override fun toString(): String {
val fields = publicFields.map { "${it.key}=${it.value.value}" }.joinToString(",")
return "${objClass.className}($fields)"
override suspend fun toString(scope: Scope, calledFromLyng: Boolean): ObjString {
return ObjString(buildString {
append("${objClass.className}(")
var first = true
for ((name, value) in publicFields) {
if (first) first = false else append(",")
append("$name=${value.value.toString(scope)}")
}
append(")")
})
}
override suspend fun inspect(scope: Scope): String {
return toString(scope).value
}
override suspend fun serialize(scope: Scope, encoder: LynonEncoder, lynonType: LynonType?) {
@@ -19,6 +19,9 @@ package net.sergeych.lyng.obj
import net.sergeych.lyng.Arguments
import net.sergeych.lyng.Statement
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
/**
* Abstract class that must provide `iterator` method that returns [ObjIterator] instance.
@@ -26,7 +29,12 @@ import net.sergeych.lyng.Statement
val ObjIterable by lazy {
ObjClass("Iterable").apply {
addFn("toList") {
addFnDoc(
name = "toList",
doc = "Collect elements of this iterable into a new list.",
returns = type("lyng.List"),
moduleName = "lyng.stdlib"
) {
val result = mutableListOf<Obj>()
val iterator = thisObj.invokeInstanceMethod(this, "iterator")
@@ -36,29 +44,48 @@ val ObjIterable by lazy {
}
// it is not effective, but it is open:
addFn("contains", isOpen = true) {
addFnDoc(
name = "contains",
doc = "Whether the iterable contains the given element (by equality).",
params = listOf(ParamDoc("element")),
returns = type("lyng.Bool"),
isOpen = true,
moduleName = "lyng.stdlib"
) {
val obj = args.firstAndOnly()
val it = thisObj.invokeInstanceMethod(this, "iterator")
while (it.invokeInstanceMethod(this, "hasNext").toBool()) {
if (obj.compareTo(this, it.invokeInstanceMethod(this, "next")) == 0)
return@addFn ObjTrue
return@addFnDoc ObjTrue
}
ObjFalse
}
addFn("indexOf", isOpen = true) {
addFnDoc(
name = "indexOf",
doc = "Index of the first occurrence of the given element, or -1 if not found.",
params = listOf(ParamDoc("element")),
returns = type("lyng.Int"),
isOpen = true,
moduleName = "lyng.stdlib"
) {
val obj = args.firstAndOnly()
var index = 0
val it = thisObj.invokeInstanceMethod(this, "iterator")
while (it.invokeInstanceMethod(this, "hasNext").toBool()) {
if (obj.compareTo(this, it.invokeInstanceMethod(this, "next")) == 0)
return@addFn ObjInt(index.toLong())
return@addFnDoc ObjInt(index.toLong())
index++
}
ObjInt(-1L)
}
addFn("toSet") {
addFnDoc(
name = "toSet",
doc = "Collect elements of this iterable into a new set.",
returns = type("lyng.Set"),
moduleName = "lyng.stdlib"
) {
if( thisObj.isInstanceOf(ObjSet.type) )
thisObj
else {
@@ -71,7 +98,12 @@ val ObjIterable by lazy {
}
}
addFn("toMap") {
addFnDoc(
name = "toMap",
doc = "Collect pairs into a map using [0] as key and [1] as value for each element.",
returns = type("lyng.Map"),
moduleName = "lyng.stdlib"
) {
val result = ObjMap()
thisObj.toFlow(this).collect { pair ->
result.map[pair.getAt(this, 0)] = pair.getAt(this, 1)
@@ -79,7 +111,13 @@ val ObjIterable by lazy {
result
}
addFn("associateBy") {
addFnDoc(
name = "associateBy",
doc = "Build a map from elements using the lambda result as key.",
params = listOf(ParamDoc("keySelector")),
returns = type("lyng.Map"),
moduleName = "lyng.stdlib"
) {
val association = requireOnlyArg<Statement>()
val result = ObjMap()
thisObj.toFlow(this).collect {
@@ -88,7 +126,13 @@ val ObjIterable by lazy {
result
}
addFn("forEach", isOpen = true) {
addFnDoc(
name = "forEach",
doc = "Apply the lambda to each element in iteration order.",
params = listOf(ParamDoc("action")),
isOpen = true,
moduleName = "lyng.stdlib"
) {
val it = thisObj.invokeInstanceMethod(this, "iterator")
val fn = requiredArg<Statement>(0)
while (it.invokeInstanceMethod(this, "hasNext").toBool()) {
@@ -98,7 +142,14 @@ val ObjIterable by lazy {
ObjVoid
}
addFn("map", isOpen = true) {
addFnDoc(
name = "map",
doc = "Transform elements by applying the given lambda.",
params = listOf(ParamDoc("transform")),
returns = type("lyng.List"),
isOpen = true,
moduleName = "lyng.stdlib"
) {
val fn = requiredArg<Statement>(0)
val result = mutableListOf<Obj>()
thisObj.toFlow(this).collect {
@@ -107,7 +158,30 @@ val ObjIterable by lazy {
ObjList(result)
}
addFn("take") {
addFnDoc(
name = "mapNotNull",
doc = "Transform elements by applying the given lambda unless it returns null.",
params = listOf(ParamDoc("transform")),
returns = type("lyng.List"),
isOpen = true,
moduleName = "lyng.stdlib"
) {
val fn = requiredArg<Statement>(0)
val result = mutableListOf<Obj>()
thisObj.toFlow(this).collect {
val transformed = fn.call(this, it)
if( transformed != ObjNull) result += transformed
}
ObjList(result)
}
addFnDoc(
name = "take",
doc = "Take the first N elements and return them as a list.",
params = listOf(ParamDoc("n", type("lyng.Int"))),
returns = type("lyng.List"),
moduleName = "lyng.stdlib"
) {
var n = requireOnlyArg<ObjInt>().value.toInt()
val result = mutableListOf<Obj>()
if (n > 0) {
@@ -119,7 +193,12 @@ val ObjIterable by lazy {
ObjList(result)
}
addFn("isEmpty") {
addFnDoc(
name = "isEmpty",
doc = "Whether the iterable has no elements.",
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) {
ObjBool(
thisObj.invokeInstanceMethod(this, "iterator")
.invokeInstanceMethod(this, "hasNext").toBool()
@@ -127,7 +206,13 @@ val ObjIterable by lazy {
)
}
addFn("sortedWith") {
addFnDoc(
name = "sortedWith",
doc = "Return a new list sorted using the provided comparator `(a, b) -> Int`.",
params = listOf(ParamDoc("comparator")),
returns = type("lyng.List"),
moduleName = "lyng.stdlib"
) {
val list = thisObj.callMethod<ObjList>(this, "toList")
val comparator = requireOnlyArg<Statement>()
list.quicksort { a, b ->
@@ -136,7 +221,12 @@ val ObjIterable by lazy {
list
}
addFn("reversed") {
addFnDoc(
name = "reversed",
doc = "Return a new list with elements in reverse order.",
returns = type("lyng.List"),
moduleName = "lyng.stdlib"
) {
val list = thisObj.callMethod<ObjList>(this, "toList")
list.list.reverse()
list
@@ -17,6 +17,10 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
/**
* Iterator should provide lyng-level iterator functions:
*
@@ -28,15 +32,50 @@ package net.sergeych.lyng.obj
*/
val ObjIterator by lazy {
ObjClass("Iterator").apply {
addFn("cancelIteration", true) {
// Base protocol methods; actual iterators override these.
addFnDoc(
name = "cancelIteration",
doc = "Optional hint to stop iteration early and free resources.",
returns = type("lyng.Void"),
isOpen = true,
moduleName = "lyng.stdlib"
) {
ObjVoid
}
addFn("hasNext", true) {
addFnDoc(
name = "hasNext",
doc = "Whether another element is available.",
returns = type("lyng.Bool"),
isOpen = true,
moduleName = "lyng.stdlib"
) {
raiseNotImplemented("hasNext() is not implemented")
}
addFn("next", true) {
addFnDoc(
name = "next",
doc = "Return the next element.",
returns = type("lyng.Any"),
isOpen = true,
moduleName = "lyng.stdlib"
) {
raiseNotImplemented("next() is not implemented")
}
// Helper to consume iterator into a list
addFnDoc(
name = "toList",
doc = "Consume this iterator and collect elements into a list.",
returns = TypeGenericDoc(type("lyng.List"), listOf(type("lyng.Any"))),
moduleName = "lyng.stdlib"
) {
val out = mutableListOf<Obj>()
while (true) {
val has = thisObj.invokeInstanceMethod(this, "hasNext").toBool()
if (!has) break
val v = thisObj.invokeInstanceMethod(this, "next")
out += v
}
ObjList(out.toMutableList())
}
}
}
@@ -21,6 +21,10 @@ import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonElement
import net.sergeych.lyng.Scope
import net.sergeych.lyng.Statement
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.addConstDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
import net.sergeych.lyng.statement
import net.sergeych.lynon.LynonDecoder
import net.sergeych.lynon.LynonEncoder
@@ -194,25 +198,50 @@ class ObjList(val list: MutableList<Obj> = mutableListOf()) : Obj() {
return JsonArray(list.map { it.toJson(scope) })
}
override suspend fun toString(scope: Scope, calledFromLyng: Boolean): ObjString {
return ObjString(buildString {
append("[")
var first = true
for (v in list) {
if (first) first = false else append(",")
append(v.toString(scope).value)
}
append("]")
})
}
companion object {
val type = object : ObjClass("List", ObjArray) {
override suspend fun deserialize(scope: Scope, decoder: LynonDecoder, lynonType: LynonType?): Obj {
return ObjList(decoder.decodeAnyList(scope))
}
}.apply {
createField("size",
statement {
addConstDoc(
name = "size",
value = statement {
(thisObj as ObjList).list.size.toObj()
}
},
doc = "Number of elements in this list.",
type = type("lyng.Int"),
moduleName = "lyng.stdlib"
)
createField("add",
statement {
addConstDoc(
name = "add",
value = statement {
val l = thisAs<ObjList>().list
for (a in args) l.add(a)
ObjVoid
}
},
doc = "Append one or more elements to the end of this list.",
type = type("lyng.Callable"),
moduleName = "lyng.stdlib"
)
addFn("insertAt") {
addFnDoc(
name = "insertAt",
doc = "Insert elements starting at the given index.",
params = listOf(ParamDoc("index", type("lyng.Int"))),
moduleName = "lyng.stdlib"
) {
if (args.size < 2) raiseError("addAt takes 2+ arguments")
val l = thisAs<ObjList>()
var index = requiredArg<ObjInt>(0).value.toInt()
@@ -220,7 +249,12 @@ class ObjList(val list: MutableList<Obj> = mutableListOf()) : Obj() {
ObjVoid
}
addFn("removeAt") {
addFnDoc(
name = "removeAt",
doc = "Remove element at index, or a range [start,end) if two indices are provided. Returns the list.",
params = listOf(ParamDoc("start", type("lyng.Int")), ParamDoc("end", type("lyng.Int"))),
moduleName = "lyng.stdlib"
) {
val self = thisAs<ObjList>()
val start = requiredArg<ObjInt>(0).value.toInt()
if (args.size == 2) {
@@ -231,7 +265,12 @@ class ObjList(val list: MutableList<Obj> = mutableListOf()) : Obj() {
self
}
addFn("removeLast") {
addFnDoc(
name = "removeLast",
doc = "Remove the last element or the last N elements if a count is provided. Returns the list.",
params = listOf(ParamDoc("count", type("lyng.Int"))),
moduleName = "lyng.stdlib"
) {
val self = thisAs<ObjList>()
if (args.isNotEmpty()) {
val count = requireOnlyArg<ObjInt>().value.toInt()
@@ -242,7 +281,12 @@ class ObjList(val list: MutableList<Obj> = mutableListOf()) : Obj() {
self
}
addFn("removeRange") {
addFnDoc(
name = "removeRange",
doc = "Remove a range of elements. Accepts a Range or (start, endInclusive). Returns the list.",
params = listOf(ParamDoc("range")),
moduleName = "lyng.stdlib"
) {
val self = thisAs<ObjList>()
val list = self.list
val range = requiredArg<Obj>(0)
@@ -283,19 +327,32 @@ class ObjList(val list: MutableList<Obj> = mutableListOf()) : Obj() {
self
}
addFn("sortWith") {
addFnDoc(
name = "sortWith",
doc = "Sort this list in-place using a comparator function (a, b) -> Int.",
params = listOf(ParamDoc("comparator")),
moduleName = "lyng.stdlib"
) {
val comparator = requireOnlyArg<Statement>()
thisAs<ObjList>().quicksort { a, b -> comparator.call(this, a, b).toInt() }
ObjVoid
}
addFn("shuffle") {
addFnDoc(
name = "shuffle",
doc = "Shuffle elements of this list in-place.",
moduleName = "lyng.stdlib"
) {
thisAs<ObjList>().list.shuffle()
ObjVoid
}
addFn("sum") {
addFnDoc(
name = "sum",
doc = "Sum elements using dynamic '+' or optimized integer path. Returns null for empty lists.",
moduleName = "lyng.stdlib"
) {
val self = thisAs<ObjList>()
val l = self.list
if (l.isEmpty()) return@addFn ObjNull
if (l.isEmpty()) return@addFnDoc ObjNull
if (net.sergeych.lyng.PerfFlags.PRIMITIVE_FASTOPS) {
// Fast path: all ints → accumulate as long
var i = 0
@@ -312,10 +369,10 @@ class ObjList(val list: MutableList<Obj> = mutableListOf()) : Obj() {
res = res.plus(this, l[i])
i++
}
return@addFn res
return@addFnDoc res
}
}
return@addFn ObjInt(acc)
return@addFnDoc ObjInt(acc)
}
// Generic path: dynamic '+' starting from first element
var res: Obj = l[0]
@@ -326,9 +383,13 @@ class ObjList(val list: MutableList<Obj> = mutableListOf()) : Obj() {
}
res
}
addFn("min") {
addFnDoc(
name = "min",
doc = "Minimum element by natural order. Returns null for empty lists.",
moduleName = "lyng.stdlib"
) {
val l = thisAs<ObjList>().list
if (l.isEmpty()) return@addFn ObjNull
if (l.isEmpty()) return@addFnDoc ObjNull
if (net.sergeych.lyng.PerfFlags.PRIMITIVE_FASTOPS) {
var i = 0
var hasOnlyInts = true
@@ -343,7 +404,7 @@ class ObjList(val list: MutableList<Obj> = mutableListOf()) : Obj() {
}
i++
}
if (hasOnlyInts) return@addFn ObjInt(minVal)
if (hasOnlyInts) return@addFnDoc ObjInt(minVal)
}
var res: Obj = l[0]
var i = 1
@@ -354,9 +415,13 @@ class ObjList(val list: MutableList<Obj> = mutableListOf()) : Obj() {
}
res
}
addFn("max") {
addFnDoc(
name = "max",
doc = "Maximum element by natural order. Returns null for empty lists.",
moduleName = "lyng.stdlib"
) {
val l = thisAs<ObjList>().list
if (l.isEmpty()) return@addFn ObjNull
if (l.isEmpty()) return@addFnDoc ObjNull
if (net.sergeych.lyng.PerfFlags.PRIMITIVE_FASTOPS) {
var i = 0
var hasOnlyInts = true
@@ -371,7 +436,7 @@ class ObjList(val list: MutableList<Obj> = mutableListOf()) : Obj() {
}
i++
}
if (hasOnlyInts) return@addFn ObjInt(maxVal)
if (hasOnlyInts) return@addFnDoc ObjInt(maxVal)
}
var res: Obj = l[0]
var i = 1
@@ -382,21 +447,27 @@ class ObjList(val list: MutableList<Obj> = mutableListOf()) : Obj() {
}
res
}
addFn("indexOf") {
addFnDoc(
name = "indexOf",
doc = "Index of the first occurrence of the given element, or -1 if not found.",
params = listOf(ParamDoc("element")),
returns = type("lyng.Int"),
moduleName = "lyng.stdlib"
) {
val l = thisAs<ObjList>().list
val needle = args.firstAndOnly()
if (net.sergeych.lyng.PerfFlags.PRIMITIVE_FASTOPS && needle is ObjInt) {
var i = 0
while (i < l.size) {
val v = l[i]
if (v is ObjInt && v.value == needle.value) return@addFn ObjInt(i.toLong())
if (v is ObjInt && v.value == needle.value) return@addFnDoc ObjInt(i.toLong())
i++
}
return@addFn ObjInt((-1).toLong())
return@addFnDoc ObjInt((-1).toLong())
}
var i = 0
while (i < l.size) {
if (l[i].compareTo(this, needle) == 0) return@addFn ObjInt(i.toLong())
if (l[i].compareTo(this, needle) == 0) return@addFnDoc ObjInt(i.toLong())
i++
}
ObjInt((-1).toLong())
@@ -21,6 +21,10 @@ import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import net.sergeych.lyng.Scope
import net.sergeych.lyng.Statement
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
import net.sergeych.lynon.LynonDecoder
import net.sergeych.lynon.LynonEncoder
import net.sergeych.lynon.LynonType
@@ -48,8 +52,8 @@ class ObjMapEntry(val key: Obj, val value: Obj) : Obj() {
else -> scope.raiseIndexOutOfBounds()
}
override fun toString(): String {
return "$key=>$value"
override suspend fun toString(scope: Scope, calledFromLyng: Boolean): ObjString {
return ObjString("(${key.toString(scope).value} => ${value.toString(scope).value})")
}
override val objClass = type
@@ -72,9 +76,24 @@ class ObjMapEntry(val key: Obj, val value: Obj) : Obj() {
)
}
}.apply {
addFn("key") { thisAs<ObjMapEntry>().key }
addFn("value") { thisAs<ObjMapEntry>().value }
addFn("size") { 2.toObj() }
addFnDoc(
name = "key",
doc = "Key component of this map entry.",
returns = type("lyng.Any"),
moduleName = "lyng.stdlib"
) { thisAs<ObjMapEntry>().key }
addFnDoc(
name = "value",
doc = "Value component of this map entry.",
returns = type("lyng.Any"),
moduleName = "lyng.stdlib"
) { thisAs<ObjMapEntry>().value }
addFnDoc(
name = "size",
doc = "Number of components in this entry (always 2).",
returns = type("lyng.Int"),
moduleName = "lyng.stdlib"
) { 2.toObj() }
}
}
@@ -184,34 +203,77 @@ class ObjMap(val map: MutableMap<Obj, Obj> = mutableMapOf()) : Obj() {
return ObjMap(keys.zip(values).toMap().toMutableMap())
}
}.apply {
addFn("getOrNull") {
addFnDoc(
name = "getOrNull",
doc = "Get value by key or return null if the key is absent.",
params = listOf(ParamDoc("key")),
returns = type("lyng.Any", nullable = true),
moduleName = "lyng.stdlib"
) {
val key = args.firstAndOnly(pos)
thisAs<ObjMap>().map.getOrElse(key) { ObjNull }
}
addFn("getOrPut") {
addFnDoc(
name = "getOrPut",
doc = "Get value by key or compute, store, and return the default from a lambda.",
params = listOf(ParamDoc("key"), ParamDoc("default")),
returns = type("lyng.Any"),
moduleName = "lyng.stdlib"
) {
val key = requiredArg<Obj>(0)
thisAs<ObjMap>().map.getOrPut(key) {
val lambda = requiredArg<Statement>(1)
lambda.execute(this)
}
}
addFn("size") {
addFnDoc(
name = "size",
doc = "Number of entries in the map.",
returns = type("lyng.Int"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjMap>().map.size.toObj()
}
addFn("remove") {
addFnDoc(
name = "remove",
doc = "Remove the entry by key and return the previous value or null if absent.",
params = listOf(ParamDoc("key")),
returns = type("lyng.Any", nullable = true),
moduleName = "lyng.stdlib"
) {
thisAs<ObjMap>().map.remove(requiredArg<Obj>(0))?.toObj() ?: ObjNull
}
addFn("clear") {
addFnDoc(
name = "clear",
doc = "Remove all entries from this map. Returns the map.",
returns = type("lyng.Map"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjMap>().map.clear()
thisObj
}
addFn("keys") {
addFnDoc(
name = "keys",
doc = "List of keys in this map.",
returns = TypeGenericDoc(type("lyng.List"), listOf(type("lyng.Any"))),
moduleName = "lyng.stdlib"
) {
thisAs<ObjMap>().map.keys.toObj()
}
addFn("values") {
addFnDoc(
name = "values",
doc = "List of values in this map.",
returns = TypeGenericDoc(type("lyng.List"), listOf(type("lyng.Any"))),
moduleName = "lyng.stdlib"
) {
ObjList(thisAs<ObjMap>().map.values.toMutableList())
}
addFn("iterator") {
addFnDoc(
name = "iterator",
doc = "Iterator over map entries as MapEntry objects.",
returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.MapEntry"))),
moduleName = "lyng.stdlib"
) {
ObjKotlinIterator(thisAs<ObjMap>().map.entries.iterator())
}
}
@@ -21,6 +21,9 @@ import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import net.sergeych.lyng.Scope
import net.sergeych.lyng.Statement
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
class ObjMutex(val mutex: Mutex): Obj() {
override val objClass = type
@@ -31,7 +34,13 @@ class ObjMutex(val mutex: Mutex): Obj() {
return ObjMutex(Mutex())
}
}.apply {
addFn("withLock") {
addFnDoc(
name = "withLock",
doc = "Run the given lambda while holding the mutex and return its result.",
params = listOf(ParamDoc("action")),
returns = type("lyng.Any"),
moduleName = "lyng.stdlib"
) {
val f = requiredArg<Statement>(0)
// Execute user lambda directly in the current scope to preserve the active scope
// ancestry across suspension points. The lambda still constructs a ClosureScope
@@ -18,6 +18,9 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
class ObjRange(val start: Obj?, val end: Obj?, val isEndInclusive: Boolean) : Obj() {
@@ -141,25 +144,60 @@ class ObjRange(val start: Obj?, val end: Obj?, val isEndInclusive: Boolean) : Ob
companion object {
val type = ObjClass("Range", ObjIterable).apply {
addFn("start") {
addFnDoc(
name = "start",
doc = "Start bound of the range or null if open.",
returns = type("lyng.Any", nullable = true),
moduleName = "lyng.stdlib"
) {
thisAs<ObjRange>().start ?: ObjNull
}
addFn("end") {
addFnDoc(
name = "end",
doc = "End bound of the range or null if open.",
returns = type("lyng.Any", nullable = true),
moduleName = "lyng.stdlib"
) {
thisAs<ObjRange>().end ?: ObjNull
}
addFn("isOpen") {
addFnDoc(
name = "isOpen",
doc = "Whether the range is open on either side (no start or no end).",
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjRange>().let { it.start == null || it.end == null }.toObj()
}
addFn("isIntRange") {
addFnDoc(
name = "isIntRange",
doc = "True if both bounds are Int values.",
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjRange>().isIntRange.toObj()
}
addFn("isCharRange") {
addFnDoc(
name = "isCharRange",
doc = "True if both bounds are Char values.",
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjRange>().isCharRange.toObj()
}
addFn("isEndInclusive") {
addFnDoc(
name = "isEndInclusive",
doc = "Whether the end bound is inclusive.",
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjRange>().isEndInclusive.toObj()
}
addFn("iterator") {
addFnDoc(
name = "iterator",
doc = "Iterator over elements in this range (optimized for Int ranges).",
returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.Any"))),
moduleName = "lyng.stdlib"
) {
val self = thisAs<ObjRange>()
if (net.sergeych.lyng.PerfFlags.RANGE_FAST_ITER) {
val s = self.start
@@ -169,7 +207,7 @@ class ObjRange(val start: Obj?, val end: Obj?, val isEndInclusive: Boolean) : Ob
val endExclusive = (if (self.isEndInclusive) e.value.toInt() + 1 else e.value.toInt())
// Only for ascending simple ranges; fall back otherwise
if (start <= endExclusive) {
return@addFn ObjFastIntRangeIterator(start, endExclusive)
return@addFnDoc ObjFastIntRangeIterator(start, endExclusive)
}
}
}
@@ -21,6 +21,9 @@ import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonPrimitive
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.addConstDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
import net.sergeych.lyng.statement
import net.sergeych.lynon.LynonDecoder
import net.sergeych.lynon.LynonEncoder
@@ -111,13 +114,22 @@ data class ObjReal(val value: Double) : Obj(), Numeric {
override suspend fun deserialize(scope: Scope, decoder: LynonDecoder, lynonType: LynonType?): Obj =
ObjReal(decoder.unpackDouble())
}.apply {
createField(
"roundToInt",
statement(Pos.builtIn) {
// roundToInt: number rounded to the nearest integer
addConstDoc(
name = "roundToInt",
value = statement(Pos.builtIn) {
(it.thisObj as ObjReal).value.roundToLong().toObj()
},
doc = "This real number rounded to the nearest integer.",
type = type("lyng.Int"),
moduleName = "lyng.stdlib"
)
addFn("toInt") {
addFnDoc(
name = "toInt",
doc = "Truncate this real number toward zero to an integer.",
returns = type("lyng.Int"),
moduleName = "lyng.stdlib"
) {
ObjInt(thisAs<ObjReal>().value.toLong())
}
}
@@ -1240,7 +1240,28 @@ class LocalVarRef(private val name: String, private val atPos: Pos) : ObjRef {
if (PerfFlags.PIC_DEBUG_COUNTERS) PerfStats.localVarPicMiss++
// 2) Fallback to current-scope object or field on `this`
scope[name]?.let { return it }
// 2a) Try nearest ClosureScope's closure ancestry explicitly
run {
var s: Scope? = scope
val visited = HashSet<Long>(4)
while (s != null) {
if (!visited.add(s.frameId)) break
if (s is ClosureScope) {
s.closureScope.chainLookupWithMembers(name)?.let { return it }
}
s = s.parent
}
}
// 2b) Try raw ancestry local/binding lookup (cycle-safe), including slots in parents
scope.chainLookupIgnoreClosure(name)?.let { return it }
try {
return scope.thisObj.readField(scope, name)
} catch (e: ExecutionError) {
// Map missing symbol during unqualified lookup to SymbolNotFound (SymbolNotDefinedException)
// to preserve legacy behavior expected by tests.
if ((e.message ?: "").contains("no such field: $name")) scope.raiseSymbolNotFound(name)
throw e
}
}
val hit = (cachedFrameId == scope.frameId && cachedSlot >= 0 && cachedSlot < scope.slotCount())
val slot = if (hit) cachedSlot else resolveSlot(scope)
@@ -1253,7 +1274,24 @@ class LocalVarRef(private val name: String, private val atPos: Pos) : ObjRef {
if (PerfFlags.PIC_DEBUG_COUNTERS) PerfStats.localVarPicMiss++
// 2) Fallback name in scope or field on `this`
scope[name]?.let { return it }
run {
var s: Scope? = scope
val visited = HashSet<Long>(4)
while (s != null) {
if (!visited.add(s.frameId)) break
if (s is ClosureScope) {
s.closureScope.chainLookupWithMembers(name)?.let { return it }
}
s = s.parent
}
}
scope.chainLookupIgnoreClosure(name)?.let { return it }
try {
return scope.thisObj.readField(scope, name)
} catch (e: ExecutionError) {
if ((e.message ?: "").contains("no such field: $name")) scope.raiseSymbolNotFound(name)
throw e
}
}
override suspend fun evalValue(scope: Scope): Obj {
@@ -1262,14 +1300,48 @@ class LocalVarRef(private val name: String, private val atPos: Pos) : ObjRef {
scope.getSlotIndexOf(name)?.let { return scope.getSlotRecord(it).value }
// fallback to current-scope object or field on `this`
scope[name]?.let { return it.value }
return scope.thisObj.readField(scope, name).value
run {
var s: Scope? = scope
val visited = HashSet<Long>(4)
while (s != null) {
if (!visited.add(s.frameId)) break
if (s is ClosureScope) {
s.closureScope.chainLookupWithMembers(name)?.let { return it.value }
}
s = s.parent
}
}
scope.chainLookupIgnoreClosure(name)?.let { return it.value }
return try {
scope.thisObj.readField(scope, name).value
} catch (e: ExecutionError) {
if ((e.message ?: "").contains("no such field: $name")) scope.raiseSymbolNotFound(name)
throw e
}
}
val hit = (cachedFrameId == scope.frameId && cachedSlot >= 0 && cachedSlot < scope.slotCount())
val slot = if (hit) cachedSlot else resolveSlot(scope)
if (slot >= 0) return scope.getSlotRecord(slot).value
// Fallback name in scope or field on `this`
scope[name]?.let { return it.value }
return scope.thisObj.readField(scope, name).value
run {
var s: Scope? = scope
val visited = HashSet<Long>(4)
while (s != null) {
if (!visited.add(s.frameId)) break
if (s is ClosureScope) {
s.closureScope.chainLookupWithMembers(name)?.let { return it.value }
}
s = s.parent
}
}
scope.chainLookupIgnoreClosure(name)?.let { return it.value }
return try {
scope.thisObj.readField(scope, name).value
} catch (e: ExecutionError) {
if ((e.message ?: "").contains("no such field: $name")) scope.raiseSymbolNotFound(name)
throw e
}
}
override suspend fun setAt(pos: Pos, scope: Scope, newValue: Obj) {
@@ -1286,6 +1358,26 @@ class LocalVarRef(private val name: String, private val atPos: Pos) : ObjRef {
else scope.raiseError("Cannot assign to immutable value")
return
}
run {
var s: Scope? = scope
val visited = HashSet<Long>(4)
while (s != null) {
if (!visited.add(s.frameId)) break
if (s is ClosureScope) {
s.closureScope.chainLookupWithMembers(name)?.let { stored ->
if (stored.isMutable) stored.value = newValue
else scope.raiseError("Cannot assign to immutable value")
return
}
}
s = s.parent
}
}
scope.chainLookupIgnoreClosure(name)?.let { stored ->
if (stored.isMutable) stored.value = newValue
else scope.raiseError("Cannot assign to immutable value")
return
}
// Fallback: write to field on `this`
scope.thisObj.writeField(scope, name, newValue)
return
@@ -1302,6 +1394,26 @@ class LocalVarRef(private val name: String, private val atPos: Pos) : ObjRef {
else scope.raiseError("Cannot assign to immutable value")
return
}
run {
var s: Scope? = scope
val visited = HashSet<Long>(4)
while (s != null) {
if (!visited.add(s.frameId)) break
if (s is ClosureScope) {
s.closureScope.chainLookupWithMembers(name)?.let { stored ->
if (stored.isMutable) stored.value = newValue
else scope.raiseError("Cannot assign to immutable value")
return
}
}
s = s.parent
}
}
scope.chainLookupIgnoreClosure(name)?.let { stored ->
if (stored.isMutable) stored.value = newValue
else scope.raiseError("Cannot assign to immutable value")
return
}
scope.thisObj.writeField(scope, name, newValue)
return
}
@@ -1351,15 +1463,21 @@ class FastLocalVarRef(
if (owner.frameId != cachedOwnerFrameId) return false
// Ensure owner is an ancestor (or same) of current
var s: Scope? = current
var guard = 0
while (s != null) {
if (s === owner) return true
s = s.parent
val next = s.parent
// Defensive: break on self-parent or pathological cycles
if (next === s) return false
s = next
if (++guard > 4096) return false
}
return false
}
private fun resolveSlotInAncestry(scope: Scope): Int {
var s: Scope? = scope
var guard = 0
while (s != null) {
val idx = s.getSlotIndexOf(name)
if (idx != null) {
@@ -1368,7 +1486,10 @@ class FastLocalVarRef(
cachedSlot = idx
return idx
}
s = s.parent
val next = s.parent
if (next === s) return -1
s = next
if (++guard > 4096) return -1
}
return -1
}
@@ -1387,16 +1508,26 @@ class FastLocalVarRef(
// Try per-frame local binding maps in the ancestry first (locals declared in frames)
run {
var s: Scope? = scope
var guard = 0
while (s != null) {
s.localBindings[name]?.let { return it }
s = s.parent
val next = s.parent
if (next === s) break
s = next
if (++guard > 4096) break
}
}
// Try to find a direct local binding in the current ancestry (without invoking name resolution that may prefer fields)
run {
var s: Scope? = scope
var guard = 0
while (s != null) {
s.objects[name]?.let { return it }
s = s.parent
val next = s.parent
if (next === s) break
s = next
if (++guard > 4096) break
}
}
// Fallback to standard name lookup (locals or closure chain) if the slot owner changed across suspension
scope[name]?.let { return it }
@@ -1413,16 +1544,26 @@ class FastLocalVarRef(
// Try per-frame local binding maps in the ancestry first
run {
var s: Scope? = scope
var guard = 0
while (s != null) {
s.localBindings[name]?.let { return it.value }
s = s.parent
val next = s.parent
if (next === s) break
s = next
if (++guard > 4096) break
}
}
// Try to find a direct local binding in the current ancestry first
run {
var s: Scope? = scope
var guard = 0
while (s != null) {
s.objects[name]?.let { return it.value }
s = s.parent
val next = s.parent
if (next === s) break
s = next
if (++guard > 4096) break
}
}
// Fallback to standard name lookup (locals or closure chain)
scope[name]?.let { return it.value }
@@ -1443,6 +1584,7 @@ class FastLocalVarRef(
// Try per-frame local binding maps in the ancestry first
run {
var s: Scope? = scope
var guard = 0
while (s != null) {
val rec = s.localBindings[name]
if (rec != null) {
@@ -1450,7 +1592,10 @@ class FastLocalVarRef(
rec.value = newValue
return
}
s = s.parent
val next = s.parent
if (next === s) break
s = next
if (++guard > 4096) break
}
}
// Fallback to standard name lookup
@@ -1480,7 +1625,7 @@ class ListLiteralRef(private val entries: List<ListEntry>) : ObjRef {
when (elements) {
is ObjList -> {
// Grow underlying array once when possible
if (list is ArrayList) list.ensureCapacity(list.size + elements.list.size)
list.ensureCapacity(list.size + elements.list.size)
list.addAll(elements.list)
}
else -> scope.raiseError("Spread element must be list")
@@ -20,6 +20,10 @@ package net.sergeych.lyng.obj
import net.sergeych.lyng.PerfFlags
import net.sergeych.lyng.RegexCache
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
class ObjRegex(val regex: Regex) : Obj() {
override val objClass = type
@@ -43,13 +47,31 @@ class ObjRegex(val regex: Regex) : Obj() {
return ObjRegex(re)
}
}.apply {
addFn("matches") {
addFnDoc(
name = "matches",
doc = "Whether the entire string matches this regular expression.",
params = listOf(ParamDoc("text", type("lyng.String"))),
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) {
ObjBool(args.firstAndOnly().toString().matches(thisAs<ObjRegex>().regex))
}
addFn("find") {
addFnDoc(
name = "find",
doc = "Find the first match in the given string.",
params = listOf(ParamDoc("text", type("lyng.String"))),
returns = type("lyng.RegexMatch", nullable = true),
moduleName = "lyng.stdlib"
) {
thisAs<ObjRegex>().find(requireOnlyArg<ObjString>())
}
addFn("findAll") {
addFnDoc(
name = "findAll",
doc = "Find all matches in the given string.",
params = listOf(ParamDoc("text", type("lyng.String"))),
returns = TypeGenericDoc(type("lyng.List"), listOf(type("lyng.RegexMatch"))),
moduleName = "lyng.stdlib"
) {
val s = requireOnlyArg<ObjString>().value
ObjList(thisAs<ObjRegex>().regex.findAll(s).map { ObjRegexMatch(it) }.toMutableList())
}
@@ -101,13 +123,28 @@ class ObjRegexMatch(val match: MatchResult) : Obj() {
scope.raiseError("RegexMatch can't be constructed directly")
}
}.apply {
addFn("groups") {
addFnDoc(
name = "groups",
doc = "List of captured groups with index 0 as the whole match.",
returns = TypeGenericDoc(type("lyng.List"), listOf(type("lyng.String"))),
moduleName = "lyng.stdlib"
) {
thisAs<ObjRegexMatch>().objGroups
}
addFn("value") {
addFnDoc(
name = "value",
doc = "The matched substring.",
returns = type("lyng.String"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjRegexMatch>().objValue
}
addFn("range") {
addFnDoc(
name = "range",
doc = "Range of the match in the input (end-exclusive).",
returns = type("lyng.Range"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjRegexMatch>().objRange
}
}
@@ -18,6 +18,10 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
class RingBuffer<T>(val maxSize: Int) : Iterable<T> {
private val data = arrayOfNulls<Any>(maxSize)
@@ -90,19 +94,34 @@ class ObjRingBuffer(val capacity: Int) : Obj() {
return ObjRingBuffer(scope.requireOnlyArg<ObjInt>().toInt())
}
}.apply {
addFn("capacity") {
thisAs<ObjRingBuffer>().capacity.toObj()
}
addFn("size") {
thisAs<ObjRingBuffer>().buffer.size.toObj()
}
addFn("iterator") {
addFnDoc(
name = "capacity",
doc = "Maximum number of elements the buffer can hold.",
returns = type("lyng.Int"),
moduleName = "lyng.stdlib"
) { thisAs<ObjRingBuffer>().capacity.toObj() }
addFnDoc(
name = "size",
doc = "Current number of elements in the buffer.",
returns = type("lyng.Int"),
moduleName = "lyng.stdlib"
) { thisAs<ObjRingBuffer>().buffer.size.toObj() }
addFnDoc(
name = "iterator",
doc = "Iterator over elements in insertion order (oldest to newest).",
returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.Any"))),
moduleName = "lyng.stdlib"
) {
val buffer = thisAs<ObjRingBuffer>().buffer
ObjKotlinObjIterator(buffer.iterator())
}
addFn("add") {
thisAs<ObjRingBuffer>().apply { buffer.add(requireOnlyArg<Obj>()) }
}
addFnDoc(
name = "add",
doc = "Append an element; if full, the oldest element is dropped.",
params = listOf(ParamDoc("value", type("lyng.Any"))),
returns = type("lyng.Void"),
moduleName = "lyng.stdlib"
) { thisAs<ObjRingBuffer>().apply { buffer.add(requireOnlyArg<Obj>()) } }
}
}
}
@@ -18,6 +18,10 @@
package net.sergeych.lyng.obj
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.addFnDoc
import net.sergeych.lyng.miniast.type
import net.sergeych.lynon.LynonDecoder
import net.sergeych.lynon.LynonEncoder
import net.sergeych.lynon.LynonType
@@ -140,22 +144,55 @@ class ObjSet(val set: MutableSet<Obj> = mutableSetOf()) : Obj() {
override suspend fun deserialize(scope: Scope, decoder: LynonDecoder, lynonType: LynonType?): Obj =
ObjSet(decoder.decodeAnyList(scope).toMutableSet())
}.apply {
addFn("size") {
addFnDoc(
name = "size",
doc = "Number of elements in this set.",
returns = type("lyng.Int"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjSet>().set.size.toObj()
}
addFn("intersect") {
addFnDoc(
name = "intersect",
doc = "Intersection with another set. Returns a new set.",
params = listOf(ParamDoc("other", type("lyng.Set"))),
returns = type("lyng.Set"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjSet>().mul(this, args.firstAndOnly())
}
addFn("iterator") {
addFnDoc(
name = "iterator",
doc = "Iterator over elements of this set.",
returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.Any"))),
moduleName = "lyng.stdlib"
) {
thisAs<ObjSet>().set.iterator().toObj()
}
addFn("union") {
addFnDoc(
name = "union",
doc = "Union with another set or iterable. Returns a new set.",
params = listOf(ParamDoc("other")),
returns = type("lyng.Set"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjSet>().plus(this, args.firstAndOnly())
}
addFn("subtract") {
addFnDoc(
name = "subtract",
doc = "Subtract another set or iterable from this set. Returns a new set.",
params = listOf(ParamDoc("other")),
returns = type("lyng.Set"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjSet>().minus(this, args.firstAndOnly())
}
addFn("remove") {
addFnDoc(
name = "remove",
doc = "Remove one or more elements. Returns true if the set changed.",
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) {
val set = thisAs<ObjSet>().set
val n = set.size
for( x in args.list ) set -= x
@@ -24,6 +24,7 @@ import kotlinx.serialization.json.JsonPrimitive
import net.sergeych.lyng.PerfFlags
import net.sergeych.lyng.RegexCache
import net.sergeych.lyng.Scope
import net.sergeych.lyng.miniast.*
import net.sergeych.lyng.statement
import net.sergeych.lynon.LynonDecoder
import net.sergeych.lynon.LynonEncoder
@@ -127,64 +128,155 @@ data class ObjString(val value: String) : Obj() {
override suspend fun deserialize(scope: Scope, decoder: LynonDecoder, lynonType: LynonType?): Obj =
ObjString(decoder.unpackBinaryData().decodeToString())
}.apply {
addFn("toInt") {
addFnDoc(
name = "toInt",
doc = "Parse this string as an integer or throw if it is not a valid integer.",
returns = type("lyng.Int"),
moduleName = "lyng.stdlib"
) {
ObjInt(
thisAs<ObjString>().value.toLongOrNull()
?: raiseIllegalArgument("can't convert to int: $thisObj")
)
}
addFn("startsWith") {
addFnDoc(
name = "startsWith",
doc = "Whether this string starts with the given prefix.",
params = listOf(ParamDoc("prefix", type("lyng.String"))),
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) {
ObjBool(thisAs<ObjString>().value.startsWith(requiredArg<ObjString>(0).value))
}
addFn("endsWith") {
addFnDoc(
name = "endsWith",
doc = "Whether this string ends with the given suffix.",
params = listOf(ParamDoc("suffix", type("lyng.String"))),
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) {
ObjBool(thisAs<ObjString>().value.endsWith(requiredArg<ObjString>(0).value))
}
addConst("length",
statement { ObjInt(thisAs<ObjString>().value.length.toLong()) }
addConstDoc(
name = "length",
value = statement { ObjInt(thisAs<ObjString>().value.length.toLong()) },
doc = "Number of UTF-16 code units in this string.",
type = type("lyng.Int"),
moduleName = "lyng.stdlib"
)
addFn("takeLast") {
addFnDoc(
name = "takeLast",
doc = "Return a string with the last N characters.",
params = listOf(ParamDoc("n", type("lyng.Int"))),
returns = type("lyng.String"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjString>().value.takeLast(
requiredArg<ObjInt>(0).toInt()
).let(::ObjString)
}
addFn("take") {
addFnDoc(
name = "take",
doc = "Return a string with the first N characters.",
params = listOf(ParamDoc("n", type("lyng.Int"))),
returns = type("lyng.String"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjString>().value.take(
requiredArg<ObjInt>(0).toInt()
).let(::ObjString)
}
addFn("drop") {
addFnDoc(
name = "drop",
doc = "Drop the first N characters and return the remainder.",
params = listOf(ParamDoc("n", type("lyng.Int"))),
returns = type("lyng.String"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjString>().value.drop(
requiredArg<ObjInt>(0).toInt()
).let(::ObjString)
}
addFn("dropLast") {
addFnDoc(
name = "dropLast",
doc = "Drop the last N characters and return the remainder.",
params = listOf(ParamDoc("n", type("lyng.Int"))),
returns = type("lyng.String"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjString>().value.dropLast(
requiredArg<ObjInt>(0).toInt()
).let(::ObjString)
}
addFn("lower") {
addFnDoc(
name = "lower",
doc = "Lowercase version of this string (default locale).",
returns = type("lyng.String"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjString>().value.lowercase().let(::ObjString)
}
addFn("upper") {
addFnDoc(
name = "upper",
doc = "Uppercase version of this string (default locale).",
returns = type("lyng.String"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjString>().value.uppercase().let(::ObjString)
}
addFn("characters") {
addFnDoc(
name = "characters",
doc = "List of characters of this string.",
returns = TypeGenericDoc(type("lyng.List"), listOf(type("lyng.Char"))),
moduleName = "lyng.stdlib"
) {
ObjList(
thisAs<ObjString>().value.map { ObjChar(it) }.toMutableList()
)
}
addFn("last") {
addFnDoc(
name = "last",
doc = "The last character of this string or throw if the string is empty.",
returns = type("lyng.Char"),
moduleName = "lyng.stdlib"
) {
ObjChar(thisAs<ObjString>().value.lastOrNull() ?: raiseNoSuchElement("empty string"))
}
addFn("encodeUtf8") { ObjBuffer(thisAs<ObjString>().value.encodeToByteArray().asUByteArray()) }
addFn("size") { ObjInt(thisAs<ObjString>().value.length.toLong()) }
addFn("toReal") {
addFnDoc(
name = "encodeUtf8",
doc = "Encode this string as UTF-8 bytes.",
returns = type("lyng.Buffer"),
moduleName = "lyng.stdlib"
) { ObjBuffer(thisAs<ObjString>().value.encodeToByteArray().asUByteArray()) }
addFnDoc(
name = "size",
doc = "Alias for length: the number of characters (code units) in this string.",
returns = type("lyng.Int"),
moduleName = "lyng.stdlib"
) { ObjInt(thisAs<ObjString>().value.length.toLong()) }
addFnDoc(
name = "toReal",
doc = "Parse this string as a real number (floating point).",
returns = type("lyng.Real"),
moduleName = "lyng.stdlib"
) {
ObjReal(thisAs<ObjString>().value.toDouble())
}
addFn("trim") {
addFnDoc(
name = "trim",
doc = "Return a copy of this string with leading and trailing whitespace removed.",
returns = type("lyng.String"),
moduleName = "lyng.stdlib"
) {
thisAs<ObjString>().value.trim().let(::ObjString)
}
addFn("matches") {
addFnDoc(
name = "matches",
doc = "Whether this string matches the given regular expression or pattern string.",
params = listOf(ParamDoc("pattern")),
returns = type("lyng.Bool"),
moduleName = "lyng.stdlib"
) {
val s = requireOnlyArg<Obj>()
val self = thisAs<ObjString>().value
ObjBool(
@@ -82,7 +82,14 @@ open class LynonDecoder(val bin: BitInput, val settings: LynonSettings = LynonSe
if (it !is ObjClass)
scope.raiseClassCastError("Expected obj class but got ${it::class.simpleName}")
it
} ?: scope.raiseSymbolNotFound("can't deserialize: not found type $className")
} ?: run {
val fallback = runCatching { scope.eval(className.value) }.getOrNull()
if (fallback != null) {
println("Fallback to eval successful")
fallback as ObjClass
}
else scope.raiseSymbolNotFound("can't deserialize: not found type $className")
}
}
suspend fun decodeAnyList(scope: Scope, fixedSize: Int? = null): MutableList<Obj> {
@@ -127,9 +127,11 @@ open class LynonEncoder(val bout: BitOutput, val settings: LynonSettings = Lynon
*/
suspend fun encodeAnyList(scope: Scope, list: List<Obj>,fixedSize: Boolean = false) {
if( list.isEmpty()) {
// any-typed, empty, save space
putBit(0)
encodeUnsigned(0u)
// Empty list: we still need to write the heterogenous/homogenous flag bit
// but must NOT write the size when fixedSize is provided, as decoder will
// use the provided size and won't read it from the stream.
putBit(0) // treat empty lists as heterogeneous
if (!fixedSize) encodeUnsigned(0u)
}
else {
val objClass = list[0].objClass
@@ -0,0 +1,34 @@
/*
* Regression test for scope parent-chain cycle when reusing pooled frames
*/
import kotlinx.coroutines.test.runTest
import net.sergeych.lyng.eval
import kotlin.test.Test
class ScopeCycleRegressionTest {
@Test
fun instanceMethodCallDoesNotCycle() = runTest {
eval(
(
"""
class Whatever {
fun something() {
println
}
}
fun ll() { Whatever() }
fun callTest1() {
val l = ll()
l.something()
"ok"
}
assertEquals("ok", callTest1())
"""
).trimIndent()
)
}
}
+192
View File
@@ -61,6 +61,74 @@ class ScriptTest {
println("version = ${LyngVersion}")
}
@Test
fun testClosureSeesCallerLocalsInLaunch() = runTest {
val scope = Script.newScope()
val res = scope.eval(
"""
var counter = 0
val d = launch {
val c = counter
delay(1)
counter = c + 1
}
d.await()
counter
""".trimIndent()
)
assertEquals(1L, (res as ObjInt).value)
}
@Test
fun testClosureResolvesGlobalsInLaunch() = runTest {
val scope = Script.newScope()
val res = scope.eval(
"""
val d = launch {
delay(1)
yield()
}
d.await()
42
""".trimIndent()
)
assertEquals(42L, (res as ObjInt).value)
}
@Test
fun testClosureSeesModulePseudoSymbol() = runTest {
val scope = Script.newScope()
val res = scope.eval(
"""
val s = { __PACKAGE__ }
s()
""".trimIndent()
)
// __PACKAGE__ is a string; just ensure it's a string and non-empty
assertTrue(res is ObjString && res.value.isNotEmpty())
}
@Test
fun testNoInfiniteRecursionOnUnknownInNestedClosure() = runTest {
val scope = Script.newScope()
withTimeout(1.seconds) {
// Access an unknown symbol inside nested closures; should throw quickly, not hang
try {
scope.eval(
"""
val f = { { unknown_symbol_just_for_test } }
f()()
""".trimIndent()
)
fail("Expected exception not thrown")
} catch (_: ExecutionError) {
// ok
} catch (_: ScriptError) {
// ok
}
}
}
// --- Helpers to test iterator cancellation semantics ---
class ObjTestIterable : Obj() {
@@ -4013,6 +4081,13 @@ class ScriptTest {
assertEquals( TestJson4(TestEnum.One), x)
}
@Test
fun testStringLast() = runTest {
eval("""
assertEquals('t', "assert".last())
""".trimIndent())
}
@Test
fun testLogicalNot() = runTest {
eval("""
@@ -4049,4 +4124,121 @@ class ScriptTest {
""".trimIndent()
)
}
@Test
fun testHangOnPrintlnInMethods() = runTest {
eval("""
class T(someList) {
fun f() {
val x = [...someList]
println(x)
}
}
T([1,2]).f()
""")
}
@Test
fun testHangOnNonexistingMethod() = runTest {
eval("""
class T(someList) {
fun f() {
nonExistingMethod()
}
}
val t = T([1,2])
try {
for( i in 1..10 ) {
t.f()
}
}
catch(t: SymbolNotFound) {
println(t::class)
// ok
}
""")
}
@Test
fun testUsingClassConstructorVars() = runTest {
val r = eval("""
import lyng.time
class Request {
static val id = "rqid"
}
enum Action {
Test
}
class LogEntry(vaultId, action, data=null, createdAt=Instant.now().truncateToSecond()) {
/*
Convert to a map object that can be easily decoded outsude the
contract execution scope.
*/
fun toApi() {
{ createdAt:, requestId: Request.id, vaultId:, action: action.name, data: Map() }
}
}
fun test() {
LogEntry("v1", Action.Test).toApi()
}
test()
""".trimIndent()).toJson()
println(r)
}
@Test
fun testScopeShortCircuit() = runTest() {
val baseScope = Script.newScope()
baseScope.eval("""
val exports = Map()
fun Export(name,f) {
exports[name] = f
f
}
""".trimIndent()
)
val exports: MutableMap<Obj, Obj> = (baseScope.eval("exports") as ObjMap).map
baseScope.eval("""
class A(val a) {
fun methodA() {
a + 1
}
}
val a0 = 100
fun someFunction(x) {
val ia = A(x)
ia.methodA()
}
@Export
fun exportedFunction(x) {
someFunction(x)
}
""".trimIndent())
// Calling from the script is ok:
val instanceScope = baseScope.createChildScope()
instanceScope.eval("""
val a1 = a0 + 1
""".trimIndent())
assertEquals( ObjInt(2), instanceScope.eval("""
exportedFunction(1)
"""))
assertEquals( ObjInt(103), instanceScope.eval("""
exportedFunction(a1 + 1)
"""))
val dummyThis = Obj()
// but we should be able to call it directly
val otherScope = baseScope.createChildScope()
val r = (exports["exportedFunction".toObj()] as Statement).invoke(otherScope, dummyThis,ObjInt(50))
println(r)
assertEquals(51, r.toInt())
}
}
@@ -29,12 +29,18 @@ actual object ScopePool {
actual fun borrow(parent: Scope, args: Arguments, pos: Pos, thisObj: Obj): Scope {
val s = if (pool.isNotEmpty()) pool.removeLast() else Scope(parent, args, pos, thisObj)
return try {
if (s.parent !== parent || s.args !== args || s.pos !== pos || s.thisObj !== thisObj) {
s.resetForReuse(parent, args, pos, thisObj)
} else {
s.frameId = nextFrameId()
}
return s
s
} catch (e: IllegalStateException) {
if (e.message?.contains("cycle") == true && e.message?.contains("scope parent chain") == true) {
Scope(parent, args, pos, thisObj)
} else throw e
}
}
actual fun release(scope: Scope) {
@@ -33,9 +33,19 @@ actual object ScopePool {
actual fun borrow(parent: Scope, args: Arguments, pos: Pos, thisObj: Obj): Scope {
val pool = threadLocalPool.get()
val s = if (pool.isNotEmpty()) pool.removeLast() else Scope(parent, args, pos, thisObj)
return try {
// Always reset state on borrow to guarantee fresh-frame semantics
s.resetForReuse(parent, args, pos, thisObj)
return s
s
} catch (e: IllegalStateException) {
// Defensive fallback: if a cycle in scope parent chain is detected during reuse,
// discard pooled instance for this call frame and allocate a fresh scope instead.
if (e.message?.contains("cycle") == true && e.message?.contains("scope parent chain") == true) {
Scope(parent, args, pos, thisObj)
} else {
throw e
}
}
}
actual fun release(scope: Scope) {
+13
View File
@@ -778,6 +778,19 @@ class Wallet( id, ownerKey, balance=0, createdAt=Instant.now().truncateToSecond(
assert( Lynon.encode(Poin3(1,2)).size <= 110)
""".trimIndent())
}
@Test
fun testLyngDecodeResult() = runTest {
val a = eval("""
[Map(),Map(),Map()]
""".trimIndent())
val b = lynonDecodeAny(Scope(), lynonEncodeAny(Scope(), a))
assertEquals(a, b )
// val c = b.decodeSerializable<JsonElement>()
// println(c)
}
}
@@ -0,0 +1,147 @@
package net.sergeych.lyng.miniast
import kotlinx.coroutines.runBlocking
import kotlin.test.Test
import kotlin.test.assertFalse
import kotlin.test.assertTrue
class CompletionEngineLightTest {
private fun names(items: List<CompletionItem>): List<String> = items.map { it.name }
@Test
fun iteratorAfterLines_noGlobals() = runBlocking {
TestDocsBootstrap.ensure("lyng.stdlib", "lyng.io.fs")
val code = """
import lyng.io.fs
import lyng.stdlib
val files = Path("../..").lines().<caret>
""".trimIndent()
val items = CompletionEngineLight.completeAtMarkerSuspend(code)
val ns = names(items)
// No globals after a dot
assertFalse(ns.contains("Path"), "Should not contain global 'Path' after dot")
assertFalse(ns.contains("Array"), "Should not contain global 'Array' after dot")
assertFalse(ns.contains("String"), "Should not contain global 'String' after dot")
// Should have some iterator members (at least non-empty)
assertTrue(ns.isNotEmpty(), "Iterator members should be suggested")
}
@Test
fun iteratorAfterLines_withPrefix() = runBlocking {
TestDocsBootstrap.ensure("lyng.stdlib", "lyng.io.fs")
val code = """
import lyng.io.fs
import lyng.stdlib
val files = Path("../..").lines().t<caret>
""".trimIndent()
val items = CompletionEngineLight.completeAtMarkerSuspend(code)
val ns = names(items)
// No globals after a dot even with prefix
assertFalse(ns.contains("Path"))
assertFalse(ns.contains("Array"))
assertFalse(ns.contains("String"))
// All start with 't'
assertTrue(ns.all { it.startsWith("t", ignoreCase = true) }, "All suggestions should respect prefix 't'")
}
@Test
fun listLiteral_membersAndInherited() = runBlocking {
TestDocsBootstrap.ensure("lyng.stdlib")
val code = """
import lyng.stdlib
val x = [1,2,3].<caret>
""".trimIndent()
val items = CompletionEngineLight.completeAtMarkerSuspend(code)
val ns = names(items)
// No globals after a dot
assertFalse(ns.contains("Array"))
assertFalse(ns.contains("String"))
assertFalse(ns.contains("Path"))
// Expect more than a couple of items (not just one)
assertTrue(ns.size >= 3, "Too few member suggestions after list literal: $ns")
}
@Test
fun stringLiteral_members() = runBlocking {
TestDocsBootstrap.ensure("lyng.stdlib")
val code = """
import lyng.stdlib
val s = "abc".<caret>
""".trimIndent()
val items = CompletionEngineLight.completeAtMarkerSuspend(code)
val ns = names(items)
assertTrue(ns.isNotEmpty(), "String members should be suggested")
assertFalse(ns.contains("Path"))
}
@Test
fun stringLiteral_re_hasReturnTypeRegex() = runBlocking {
TestDocsBootstrap.ensure("lyng.stdlib")
val code = """
import lyng.stdlib
val s = "abc".<caret>
""".trimIndent()
val items = CompletionEngineLight.completeAtMarkerSuspend(code)
val reItem = items.firstOrNull { it.name == "re" }
assertTrue(reItem != null, "Expected to find 're' in String members, got: ${items.map { it.name }}")
// Type text should contain ": Regex"
assertTrue(reItem!!.typeText?.contains("Regex") == true, "Expected type text to contain 'Regex', was: ${reItem.typeText}")
}
@Test
fun stringLiteral_parenthesized_re_hasReturnTypeRegex() = runBlocking {
TestDocsBootstrap.ensure("lyng.stdlib")
val code = """
// No imports on purpose; stdlib must still be available
val s = ("abc").<caret>
""".trimIndent()
val items = CompletionEngineLight.completeAtMarkerSuspend(code)
val names = items.map { it.name }
assertTrue(names.isNotEmpty(), "Expected String members for parenthesized literal, got empty list")
val reItem = items.firstOrNull { it.name == "re" }
assertTrue(reItem != null, "Expected to find 're' for parenthesized String literal, got: $names")
assertTrue(reItem!!.typeText?.contains("Regex") == true, "Expected ': Regex' for re(), was: ${reItem.typeText}")
}
@Test
fun stringLiteral_noImports_stillHasStringMembers() = runBlocking {
TestDocsBootstrap.ensure("lyng.stdlib")
val code = """
val s = "super".<caret>
""".trimIndent()
val items = CompletionEngineLight.completeAtMarkerSuspend(code)
val names = items.map { it.name }
assertTrue(names.isNotEmpty(), "Expected String members without explicit imports, got empty list")
val reItem = items.firstOrNull { it.name == "re" }
assertTrue(reItem != null, "Expected to find 're' without explicit imports, got: $names")
assertTrue(reItem!!.typeText?.contains("Regex") == true, "Expected ': Regex' for re() without imports, was: ${reItem.typeText}")
}
@Test
fun shebang_and_fs_import_iterator_after_lines() = runBlocking {
TestDocsBootstrap.ensure("lyng.stdlib", "lyng.io.fs")
val code = """
#!/bin/env lyng
import lyng.io.fs
import lyng.stdlib
val files = Path("../..").lines().<caret>
""".trimIndent()
val items = CompletionEngineLight.completeAtMarkerSuspend(code)
val ns = names(items)
// Should not contain globals after dot
assertFalse(ns.contains("Path"))
assertFalse(ns.contains("Array"))
assertFalse(ns.contains("String"))
// Should contain some iterator members
assertTrue(ns.isNotEmpty(), "Iterator members should be suggested after lines() with shebang present")
}
}
@@ -0,0 +1,33 @@
package net.sergeych.lyng.miniast
/** Ensure docs modules are loaded for tests (stdlib + optional lyngio). */
object TestDocsBootstrap {
@Volatile private var ensured = false
fun ensure(vararg modules: String) {
if (ensured) return
synchronized(this) {
if (ensured) return
// Touch stdlib to seed lazy docs
BuiltinDocRegistry.docsForModule("lyng.stdlib")
// Try to load external fs docs registrar reflectively
val ok = try {
val cls = Class.forName("net.sergeych.lyngio.docs.FsBuiltinDocs")
val m = cls.getMethod("ensure")
m.invoke(null)
true
} catch (_: Throwable) { false }
if (!ok) {
// Minimal fallback for lyng.io.fs (Path with lines(): Iterator<String>)
BuiltinDocRegistry.moduleReplace("lyng.io.fs") {
classDoc(name = "Path", doc = "Filesystem path class.") {
method(name = "lines", doc = "Iterate file as lines of text.", returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.String"))))
method(name = "exists", doc = "Whether the path exists.", returns = type("lyng.Bool"))
}
valDoc(name = "Path", doc = "Path class", type = type("Path"))
}
}
ensured = true
}
}
}
@@ -29,12 +29,18 @@ actual object ScopePool {
actual fun borrow(parent: Scope, args: Arguments, pos: Pos, thisObj: Obj): Scope {
val s = if (pool.isNotEmpty()) pool.removeLast() else Scope(parent, args, pos, thisObj)
return try {
if (s.parent !== parent || s.args !== args || s.pos !== pos || s.thisObj !== thisObj) {
s.resetForReuse(parent, args, pos, thisObj)
} else {
s.frameId = nextFrameId()
}
return s
s
} catch (e: IllegalStateException) {
if (e.message?.contains("cycle") == true && e.message?.contains("scope parent chain") == true) {
Scope(parent, args, pos, thisObj)
} else throw e
}
}
actual fun release(scope: Scope) {
@@ -29,12 +29,18 @@ actual object ScopePool {
actual fun borrow(parent: Scope, args: Arguments, pos: Pos, thisObj: Obj): Scope {
val s = if (pool.isNotEmpty()) pool.removeLast() else Scope(parent, args, pos, thisObj)
return try {
if (s.parent !== parent || s.args !== args || s.pos !== pos || s.thisObj !== thisObj) {
s.resetForReuse(parent, args, pos, thisObj)
} else {
s.frameId = nextFrameId()
}
return s
s
} catch (e: IllegalStateException) {
if (e.message?.contains("cycle") == true && e.message?.contains("scope parent chain") == true) {
Scope(parent, args, pos, thisObj)
} else throw e
}
}
actual fun release(scope: Scope) {
+28 -5
View File
@@ -1,5 +1,9 @@
package lyng.stdlib
/*
Wrap a builder into a zero-argument thunk that computes once and caches the result.
The first call invokes builder() and stores the value; subsequent calls return the cached value.
*/
fun cached(builder) {
var calculated = false
var value = null
@@ -11,7 +15,7 @@ fun cached(builder) {
value
}
}
/* Filter elements of this iterable using the provided predicate. */
fun Iterable.filter(predicate) {
val list = this
flow {
@@ -23,17 +27,20 @@ fun Iterable.filter(predicate) {
}
}
/* Skip the first N elements of this iterable. */
fun Iterable.drop(n) {
var cnt = 0
filter { cnt++ >= n }
}
/* Return the first element or throw if the iterable is empty. */
fun Iterable.first() {
val i = iterator()
if( !i.hasNext() ) throw NoSuchElementException()
i.next().also { i.cancelIteration() }
}
/* Return the last element or throw if the iterable is empty. */
fun Iterable.last() {
var found = false
var element = null
@@ -45,6 +52,7 @@ fun Iterable.last() {
element
}
/* Emit all but the last N elements of this iterable. */
fun Iterable.dropLast(n) {
val list = this
val buffer = RingBuffer(n)
@@ -57,12 +65,14 @@ fun Iterable.dropLast(n) {
}
}
/* Return the last N elements of this iterable as a buffer/list. */
fun Iterable.takeLast(n) {
val buffer = RingBuffer(n)
for( item in this ) buffer += item
buffer
}
/* Join elements into a string with a separator (prefix parameter) and optional transformer. */
fun Iterable.joinToString(prefix=" ", transformer=null) {
var result = null
for( part in this ) {
@@ -73,6 +83,7 @@ fun Iterable.joinToString(prefix=" ", transformer=null) {
result ?: ""
}
/* Return true if any element matches the predicate. */
fun Iterable.any(predicate): Bool {
for( i in this ) {
if( predicate(i) )
@@ -80,10 +91,12 @@ fun Iterable.any(predicate): Bool {
} else false
}
/* Return true if all elements match the predicate. */
fun Iterable.all(predicate): Bool {
!any { !predicate(it) }
}
/* Sum all elements; returns null for empty collections. */
fun Iterable.sum() {
val i = iterator()
if( i.hasNext() ) {
@@ -94,6 +107,7 @@ fun Iterable.sum() {
else null
}
/* Sum mapped values of elements; returns null for empty collections. */
fun Iterable.sumOf(f) {
val i = iterator()
if( i.hasNext() ) {
@@ -104,6 +118,7 @@ fun Iterable.sumOf(f) {
else null
}
/* Minimum value of the given function applied to elements of the collection. */
fun Iterable.minOf( lambda ) {
val i = iterator()
var minimum = lambda( i.next() )
@@ -114,9 +129,7 @@ fun Iterable.minOf( lambda ) {
minimum
}
/*
Return maximum value of the given function applied to elements of the collection.
*/
/* Maximum value of the given function applied to elements of the collection. */
fun Iterable.maxOf( lambda ) {
val i = iterator()
var maximum = lambda( i.next() )
@@ -127,41 +140,50 @@ fun Iterable.maxOf( lambda ) {
maximum
}
/* Return elements sorted by natural order. */
fun Iterable.sorted() {
sortedWith { a, b -> a <=> b }
}
/* Return elements sorted by the key selector. */
fun Iterable.sortedBy(predicate) {
sortedWith { a, b -> predicate(a) <=> predicate(b) }
}
/* Return a shuffled copy of the iterable as a list. */
fun Iterable.shuffled() {
toList().apply { shuffle() }
}
/* Return string representation like [a,b,c]. */
fun List.toString() {
"[" + joinToString(",") + "]"
}
/* Sort list in-place by key selector. */
fun List.sortBy(predicate) {
sortWith { a, b -> predicate(a) <=> predicate(b) }
}
/* Sort list in-place by natural order. */
fun List.sort() {
sortWith { a, b -> a <=> b }
}
/* Represents a single stack trace element. */
class StackTraceEntry(
val sourceName: String,
val line: Int,
val column: Int,
val sourceString: String
) {
/* Formatted representation: source:line:column: text. */
fun toString() {
"%s:%d:%d: %s"(sourceName, line, column, sourceString.trim())
}
}
/* Print this exception and its stack trace to standard output. */
fun Exception.printStackTrace() {
println(this)
for( entry in stackTrace() ) {
@@ -169,6 +191,7 @@ fun Exception.printStackTrace() {
}
}
fun String.re() { Regex(this) }
/* Compile this string into a regular expression. */
fun String.re(): Regex { Regex(this) }
+1
View File
@@ -106,6 +106,7 @@ fun App() {
when {
route.isBlank() -> HomePage()
route == "tryling" -> TryLyngPage()
route.startsWith("search") -> SearchPage(route)
!isDocsRoute -> ReferencePage()
else -> DocsPage(
route = route,
+54 -94
View File
@@ -449,13 +449,42 @@ private fun closeNavbarCollapse() {
// ---------------- Site-wide search (client-side) ----------------
private data class DocRecord(val path: String, val title: String, val text: String)
internal data class DocRecord(val path: String, val title: String, val text: String)
private var searchIndex: List<DocRecord>? = null
private var searchBuilding = false
private var searchInitDone = false
private fun norm(s: String): String = s.lowercase()
// ---- Search history (last 7 entries) ----
private const val SEARCH_HISTORY_KEY = "lyng.search.history"
private fun loadSearchHistory(): MutableList<String> {
return try {
val raw = window.localStorage.getItem(SEARCH_HISTORY_KEY) ?: return mutableListOf()
// Stored as newline-separated values to avoid JSON pitfalls across browsers
raw.split('\n').mapNotNull { it.trim() }.filter { it.isNotEmpty() }.toMutableList()
} catch (_: Throwable) {
mutableListOf()
}
}
private fun saveSearchHistory(list: List<String>) {
try {
window.localStorage.setItem(SEARCH_HISTORY_KEY, list.joinToString("\n"))
} catch (_: Throwable) { }
}
internal fun rememberSearchQuery(q: String) {
val query = q.trim()
if (query.isBlank()) return
val list = loadSearchHistory()
list.removeAll { it.equals(query, ignoreCase = true) }
list.add(0, query)
while (list.size > 7) list.removeAt(list.lastIndex)
saveSearchHistory(list)
}
internal fun norm(s: String): String = s.lowercase()
.replace("`", " ")
.replace("*", " ")
.replace("#", " ")
@@ -466,7 +495,7 @@ private fun norm(s: String): String = s.lowercase()
.replace(Regex("\\n+"), " ")
.replace(Regex("\\s+"), " ").trim()
private fun plainFromMarkdown(md: String): String {
internal fun plainFromMarkdown(md: String): String {
// Construct Regex instances at call time inside try/catch to avoid module init crashes
// in browsers that are strict about Unicode RegExp parsing (Safari/Chrome).
// Use non-greedy dot-all equivalents ("[\n\r\s\S]") instead of character classes with ']' where possible.
@@ -597,60 +626,24 @@ private fun scoreQuery(q: String, rec: DocRecord): Int {
return score
}
private fun renderSearchResults(input: HTMLInputElement, menu: HTMLDivElement, q: String, results: List<DocRecord>) {
dlog("search-ui", "renderSearchResults q='$q' results=${results.size} building=$searchBuilding")
if (q.isBlank()) {
dlog("search-ui", "blank query -> hide menu")
menu.classList.remove("show")
menu.innerHTML = ""
return
}
if (results.isEmpty()) {
// If index is building, show a transient indexing indicator instead of hiding everything
if (searchBuilding) {
dlog("search-ui", "indexing in progress -> show placeholder")
menu.innerHTML = "<div class=\"dropdown-item disabled\">Indexing documentation…</div>"
menu.classList.add("show")
private fun renderSearchHistoryDropdown(menu: HTMLDivElement) {
val hist = loadSearchHistory()
if (hist.isEmpty()) {
menu.innerHTML = "<div class=\"dropdown-item disabled\">No recent searches</div>"
} else {
dlog("search-ui", "no results -> show 'No results' item")
val safeQ = q.replace("<", "&lt;").replace(">", "&gt;")
menu.innerHTML = "<div class=\"dropdown-item disabled\">No results for ‘$safeQ’</div>"
menu.classList.add("show")
}
return
}
val top = results.sortedByDescending { scoreQuery(q, it) }.take(8)
val items = buildString {
top.forEach { rec ->
append("<a href=\"#/${rec.path}\" class=\"dropdown-item\" data-path=\"${rec.path}\">")
append("<strong>")
append(rec.title)
append("</strong><br><small class=\"text-muted\">")
append(rec.path.substringAfter("docs/"))
append("</small></a>")
hist.take(7).forEach { hq ->
val safe = hq.replace("<", "&lt;").replace(">", "&gt;")
append("<a href=\"#/search?q=" + encodeURIComponent(hq) + "\" class=\"dropdown-item\" data-q=\"")
append(safe)
append("\">")
append(safe)
append("</a>")
}
}
menu.innerHTML = items
// Position and show
}
menu.classList.add("show")
// Attach click handlers to enforce SPA navigation
val children = menu.getElementsByClassName("dropdown-item")
for (i in 0 until children.length) {
val a = children.item(i) as? HTMLAnchorElement ?: continue
a.onclick = { ev ->
ev.preventDefault()
val path = a.getAttribute("data-path")
if (path != null) {
val inputEl = input
val q = inputEl.value
val suffix = if (q.isNotBlank()) "?q=" + encodeURIComponent(q) else ""
dlog("nav", "search click -> navigate #/$path$suffix")
window.location.hash = "#/$path$suffix"
menu.classList.remove("show")
closeNavbarCollapse()
}
}
}
}
private fun hideSearchResults(menu: HTMLDivElement) {
@@ -709,24 +702,9 @@ fun initTopSearch(attempt: Int = 0) {
dlog("init", "initTopSearch: wiring handlers")
val scope = MainScopeProvider.scope
// Debounced search runner
// Debounced dropdown history refresher
val runSearch = debounce(scope, 120L) {
val q = input.value
dlog("search", "debounced runSearch execute q='$q'")
val results = performSearch(q)
renderSearchResults(input, menu, q, results)
// Also update highlights on the currently visible page content
try {
val root = document.querySelector(".markdown-body") as? HTMLElement
if (root != null) {
val terms = q.trim()
.split(Regex("\\s+"))
.map { it.lowercase() }
.filter { it.isNotEmpty() }
val hits = highlightSearchHits(root, terms)
dlog("search", "live highlight updated, hits=$hits, terms=$terms")
}
} catch (_: Throwable) { }
renderSearchHistoryDropdown(menu)
}
// Keep the input focused when interacting with the dropdown so it doesn't blur/close
@@ -740,15 +718,8 @@ fun initTopSearch(attempt: Int = 0) {
runSearch()
}
input.onfocus = {
// Proactively build the index on first focus for faster first results
dlog("event", "search onfocus")
scope.launch {
if (searchIndex == null && !searchBuilding) {
dlog("search", "onfocus -> buildSearchIndexOnce")
buildSearchIndexOnce()
}
}
runSearch()
dlog("event", "search onfocus: show history")
renderSearchHistoryDropdown(menu)
}
input.onkeydown = { ev ->
val key = ev.asDynamic().key as String
@@ -758,25 +729,14 @@ fun initTopSearch(attempt: Int = 0) {
hideSearchResults(menu)
}
"Enter" -> {
// Navigate to the best match
scope.launch {
val q = input.value
// If index is building and results would be empty, wait for it once
if (searchIndex == null || searchBuilding) {
dlog("search", "Enter -> ensure index")
buildSearchIndexOnce()
}
val results = performSearch(q)
val best = results.maxByOrNull { scoreQuery(q, it) }
if (best != null) {
val suffix = if (q.isNotBlank()) "?q=" + encodeURIComponent(q) else ""
dlog("nav", "Enter -> navigate #/${best.path}$suffix")
window.location.hash = "#/${best.path}$suffix"
val q = input.value.trim()
if (q.isNotBlank()) {
rememberSearchQuery(q)
val url = "#/search?q=" + encodeURIComponent(q)
dlog("nav", "Enter -> navigate $url")
window.location.hash = url
hideSearchResults(menu)
closeNavbarCollapse()
} else {
dlog("search", "Enter -> no results for q='$q'")
}
}
}
}
+176
View File
@@ -0,0 +1,176 @@
/*
* Search results page: async, progressive site-wide search with relevance sorting.
*/
import androidx.compose.runtime.*
import kotlinx.browser.window
import kotlinx.coroutines.await
import kotlinx.coroutines.delay
import org.jetbrains.compose.web.attributes.*
import org.jetbrains.compose.web.dom.*
@Composable
fun SearchPage(route: String) {
val terms = extractSearchTerms(route)
var qRaw by remember { mutableStateOf(terms.joinToString(" ")) }
var total by remember { mutableStateOf(0) }
var checked by remember { mutableStateOf(0) }
var done by remember { mutableStateOf(false) }
data class Item(val rec: DocRecord, val score: Int)
var items by remember { mutableStateOf(listOf<Item>()) }
LaunchedEffect(route) {
// Remember query in history
rememberSearchQuery(qRaw)
// Load index list
try {
checked = 0
done = false
items = emptyList()
val resp = window.fetch("docs-index.json").await()
if (!resp.ok) {
total = 0
done = true
return@LaunchedEffect
}
val arr = (kotlin.js.JSON.parse<dynamic>(resp.text().await()) as Array<String>).toList()
val list = arr.sorted()
total = list.size
// Process sequentially to keep UI responsive
for (path in list) {
try {
val url = "./" + encodeURI(path)
val r = window.fetch(url).await()
if (!r.ok) {
checked++
continue
}
val body = r.text().await()
if (body.contains("[//]: # (excludeFromIndex)")) {
checked++
continue
}
val title = extractTitleFromMarkdown(body) ?: path.substringAfterLast('/')
val plain = plainFromMarkdown(body)
val rec = DocRecord(path, title, plain)
val score = scoreQueryAdvanced(terms, rec)
if (score > 0) {
val newList = (items + Item(rec, score)).sortedByDescending { it.score }.take(200)
items = newList
}
} catch (_: Throwable) { /* ignore one entry errors */ }
checked++
// Yield to UI
delay(0)
}
} finally {
done = true
}
}
H1 { Text("Search") }
if (terms.isEmpty()) {
P { Text("Type something in the search box above.") }
return
}
P {
Text("Query: ")
Code { Text(qRaw) }
}
if (!done) {
Div({ classes("d-flex", "align-items-center", "gap-2", "mb-3") }) {
Div({ classes("spinner-border", "spinner-border-sm") }) {}
Text("Searching… $checked / $total")
}
}
if (items.isEmpty() && done) {
P { Text("No results found.") }
} else {
Ul({ classes("list-unstyled") }) {
items.forEach { item ->
Li({ classes("mb-3") }) {
val href = "#/${item.rec.path}?q=" + encodeURIComponent(qRaw)
A(attrs = { attr("href", href) }) { Text(item.rec.title) }
Br()
Small({ classes("text-muted") }) { Text(item.rec.path.substringAfter("docs/")) }
}
}
}
}
}
// Advanced relevance scoring: combines coverage of distinct terms and proximity (minimal window).
internal fun scoreQueryAdvanced(termsIn: List<String>, rec: DocRecord): Int {
val terms = termsIn.filter { it.isNotBlank() }.map { it.lowercase() }.distinct()
if (terms.isEmpty()) return 0
val title = norm(rec.title)
val text = rec.text
// Title coverage bonus
var titleCoverage = 0
for (t in terms) if (title.contains(t)) titleCoverage++
// Tokenize body and collect positions for each term by first prefix match
val wordRegex = Regex("[A-Za-z0-9_]+")
val posMap = HashMap<String, MutableList<Int>>()
var pos = 0
for (m in wordRegex.findAll(text)) {
val token = m.value.lowercase()
var matched: String? = null
var best = 0
for (t in terms) {
if (token.startsWith(t) && t.length > best) { best = t.length; matched = t }
}
if (matched != null) {
posMap.getOrPut(matched!!) { mutableListOf() }.add(pos)
}
pos++
if (pos > 100000) break // safety for extremely large docs
}
val coverage = terms.count { (posMap[it]?.isNotEmpty()) == true }
if (coverage == 0) return 0
// Proximity: minimal window length covering at least one occurrence of each covered term
var minWindow = Int.MAX_VALUE
if (coverage >= 2) {
// Prepare lists
val lists = terms.mapNotNull { t -> posMap[t]?.let { t to it } }.toList()
// K-way merge style window across sorted lists
val idx = IntArray(lists.size) { 0 }
// Initialize current positions
fun currentWindow(): Int {
var minPos = Int.MAX_VALUE
var maxPos = Int.MIN_VALUE
for (i in lists.indices) {
val p = lists[i].second[idx[i]]
if (p < minPos) minPos = p
if (p > maxPos) maxPos = p
}
return maxPos - minPos + 1
}
// Iterate advancing the pointer at the list with minimal current position
while (true) {
val win = currentWindow()
if (win < minWindow) minWindow = win
// Find list with minimal current pos
var minI = 0
var minVal = lists[0].second[idx[0]]
for (i in 1 until lists.size) {
val v = lists[i].second[idx[i]]
if (v < minVal) { minVal = v; minI = i }
}
idx[minI]++
if (idx[minI] >= lists[minI].second.size) break
// stop if window is already 1 (best possible)
if (minWindow <= 1) break
}
if (minWindow == Int.MAX_VALUE) minWindow = 100000
}
val coverageScore = coverage * 10000
val proximityScore = if (coverage >= 2) (5000.0 / (1 + minWindow)).toInt() else 0
val titleScore = titleCoverage * 3000
// Slight preference for shorter text
val lengthAdj = (200 - kotlin.math.min(200, text.length / 500))
return coverageScore + proximityScore + titleScore + lengthAdj
}
+230 -50
View File
@@ -67,13 +67,17 @@
<!-- Loader is imported in the app bundle; no external script tag required -->
<style>
/* Reserve space for fixed navbar and ensure in-page anchors don't end up hidden under it */
:root { --navbar-offset: 56px; }
:root {
--navbar-offset: 56px;
}
body {
/* Fallback padding; JS will keep this equal to the real navbar height */
/*padding-top: var(--navbar-offset);*/
/* Make native hash jumps account for the fixed header */
/*scroll-padding-top: calc(var(--navbar-offset) + 8px);*/
}
/* Also offset scroll for headings and any element targeted by an id */
.markdown-body h1,
.markdown-body h2,
@@ -84,6 +88,7 @@
[id] {
/*scroll-margin-top: calc(var(--navbar-offset) + 8px);*/
}
/* Unify markdown and page backgrounds with Bootstrap theme variables */
.markdown-body {
box-sizing: border-box;
@@ -92,14 +97,22 @@
background: var(--bs-body-bg) !important;
color: var(--bs-body-color) !important;
}
.markdown-body > :first-child { margin-top: 0 !important; }
.markdown-body table { margin: 1rem 0; }
.markdown-body > :first-child {
margin-top: 0 !important;
}
.markdown-body table {
margin: 1rem 0;
}
.markdown-body pre {
padding: .75rem;
border-radius: .375rem;
background: var(--bs-tertiary-bg);
border: 1px solid var(--bs-border-color);
}
/* Style only inline code, avoid affecting code blocks inside <pre> to prevent first-line extra indent */
.markdown-body :not(pre) > code:not([class]) {
background: var(--bs-tertiary-bg);
@@ -109,47 +122,170 @@
/* Lyng syntax highlighting palette (inspired by default GitHub/VS Code) */
/* Light theme */
.hl-kw { color: #d73a49; font-weight: 600; }
.hl-ty { color: #6f42c1; }
.hl-id { color: #24292e; }
.hl-kw {
color: #d73a49;
font-weight: 600;
}
.hl-ty {
color: #6f42c1;
}
.hl-id {
color: #24292e;
}
/* Declarations (semantic roles) */
.hl-fn { color: #005cc5; font-weight: 600; }
.hl-class { color: #5a32a3; font-weight: 600; }
.hl-val { color: #1b7f5a; }
.hl-var { color: #1b7f5a; text-decoration: underline dotted currentColor; }
.hl-enumc { color: #b08800; font-weight: 600; }
.hl-param { color: #0969da; font-style: italic; }
.hl-num { color: #005cc5; }
.hl-str { color: #032f62; }
.hl-ch { color: #032f62; }
.hl-rx { color: #116329; }
.hl-cmt { color: #6a737d; font-style: italic; }
.hl-op { color: #8250df; }
.hl-punc{ color: #57606a; }
.hl-lbl { color: #e36209; }
.hl-dir { color: #6f42c1; }
.hl-err { color: #b31d28; text-decoration: underline wavy #b31d28; }
.hl-fn {
color: #005cc5;
font-weight: 600;
}
.hl-class {
color: #5a32a3;
font-weight: 600;
}
.hl-val {
color: #1b7f5a;
}
.hl-var {
color: #1b7f5a;
text-decoration: underline dotted currentColor;
}
.hl-enumc {
color: #b08800;
font-weight: 600;
}
.hl-param {
color: #0969da;
font-style: italic;
}
.hl-num {
color: #005cc5;
}
.hl-str {
color: #032f62;
}
.hl-ch {
color: #032f62;
}
.hl-rx {
color: #116329;
}
.hl-cmt {
color: #6a737d;
font-style: italic;
}
.hl-op {
color: #8250df;
}
.hl-punc {
color: #57606a;
}
.hl-lbl {
color: #e36209;
}
.hl-dir {
color: #6f42c1;
}
.hl-err {
color: #b31d28;
text-decoration: underline wavy #b31d28;
}
/* Dark theme overrides (GitHub Dark-like) */
[data-bs-theme="dark"] .hl-id { color: #c9d1d9; }
[data-bs-theme="dark"] .hl-op { color: #d2a8ff; }
[data-bs-theme="dark"] .hl-punc { color: #8b949e; }
[data-bs-theme="dark"] .hl-kw { color: #ff7b72; }
[data-bs-theme="dark"] .hl-ty { color: #d2a8ff; }
[data-bs-theme="dark"] .hl-fn { color: #79c0ff; font-weight: 700; }
[data-bs-theme="dark"] .hl-class{ color: #d2a8ff; font-weight: 700; }
[data-bs-theme="dark"] .hl-val { color: #7ee787; }
[data-bs-theme="dark"] .hl-var { color: #7ee787; text-decoration: underline dotted currentColor; }
[data-bs-theme="dark"] .hl-enumc{ color: #f2cc60; font-weight: 700; }
[data-bs-theme="dark"] .hl-param{ color: #a5d6ff; font-style: italic; }
[data-bs-theme="dark"] .hl-num { color: #79c0ff; }
[data-bs-theme="dark"] .hl-id {
color: #c9d1d9;
}
[data-bs-theme="dark"] .hl-op {
color: #d2a8ff;
}
[data-bs-theme="dark"] .hl-punc {
color: #8b949e;
}
[data-bs-theme="dark"] .hl-kw {
color: #ff7b72;
}
[data-bs-theme="dark"] .hl-ty {
color: #d2a8ff;
}
[data-bs-theme="dark"] .hl-fn {
color: #79c0ff;
font-weight: 700;
}
[data-bs-theme="dark"] .hl-class {
color: #d2a8ff;
font-weight: 700;
}
[data-bs-theme="dark"] .hl-val {
color: #7ee787;
}
[data-bs-theme="dark"] .hl-var {
color: #7ee787;
text-decoration: underline dotted currentColor;
}
[data-bs-theme="dark"] .hl-enumc {
color: #f2cc60;
font-weight: 700;
}
[data-bs-theme="dark"] .hl-param {
color: #a5d6ff;
font-style: italic;
}
[data-bs-theme="dark"] .hl-num {
color: #79c0ff;
}
[data-bs-theme="dark"] .hl-str,
[data-bs-theme="dark"] .hl-ch { color: #a5d6ff; }
[data-bs-theme="dark"] .hl-rx { color: #7ee787; }
[data-bs-theme="dark"] .hl-cmt { color: #8b949e; }
[data-bs-theme="dark"] .hl-lbl { color: #ffa657; }
[data-bs-theme="dark"] .hl-dir { color: #d2a8ff; }
[data-bs-theme="dark"] .hl-err { color: #ffa198; text-decoration-color: #ffa198; }
[data-bs-theme="dark"] .hl-ch {
color: #a5d6ff;
}
[data-bs-theme="dark"] .hl-rx {
color: #7ee787;
}
[data-bs-theme="dark"] .hl-cmt {
color: #8b949e;
}
[data-bs-theme="dark"] .hl-lbl {
color: #ffa657;
}
[data-bs-theme="dark"] .hl-dir {
color: #d2a8ff;
}
[data-bs-theme="dark"] .hl-err {
color: #ffa198;
text-decoration-color: #ffa198;
}
/* Top-left corner ribbon for version label */
.corner-ribbon {
@@ -171,6 +307,7 @@
border: 1px solid rgba(0, 0, 0, .15);
pointer-events: none; /* decorative; don't block clicks underneath */
}
@media (max-width: 576px) {
.corner-ribbon {
width: 195px; /* 10% narrower than 150px */
@@ -180,6 +317,13 @@
}
}
</style>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/styles/dark.min.css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/highlight.min.js"></script>
<!-- and it's easy to individually load additional languages -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/languages/go.min.js"></script>
</head>
<body>
<!-- Top-left version ribbon -->
@@ -189,13 +333,16 @@
</span>
</div>
<!-- Fixed top navbar for the whole site -->
<a href="#root" class="visually-hidden-focusable position-absolute top-0 start-0 m-2 px-2 py-1 bg-body border rounded">Skip to content</a>
<a href="#root" class="visually-hidden-focusable position-absolute top-0 start-0 m-2 px-2 py-1 bg-body border rounded">Skip
to content</a>
<nav class="navbar navbar-expand-lg bg-body-tertiary fixed-top border-bottom" role="navigation" aria-label="Primary">
<div class="container-fluid">
<a class="navbar-brand d-flex align-items-center gap-2" href="#">
<!-- Inline SVG brand icon: λ with superscript y and subscript ng. Uses currentColor to match theme. -->
<svg xmlns="http://www.w3.org/2000/svg" width="30" height="30" viewBox="0 0 24 24" role="img" aria-hidden="true">
<g fill="currentColor" color="inherit" font-family="'STIX Two Math', 'Cambria Math', 'Times New Roman', serif">
<svg xmlns="http://www.w3.org/2000/svg" width="30" height="30" viewBox="0 0 24 24" role="img"
aria-hidden="true">
<g fill="currentColor" color="inherit"
font-family="'STIX Two Math', 'Cambria Math', 'Times New Roman', serif">
<!-- Main lambda -->
<text x="3" y="17" font-size="16" font-weight="700">λ</text>
<!-- Superscript y -->
@@ -219,25 +366,30 @@
<!-- Left-aligned main menu -->
<ul class="navbar-nav me-auto mb-2 mb-lg-0">
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" href="#/docs/Iterator.md" role="button" data-bs-toggle="dropdown" aria-expanded="false">
<a class="nav-link dropdown-toggle" href="#/docs/Iterator.md" role="button"
data-bs-toggle="dropdown" aria-expanded="false">
Documentation
</a>
<ul id="docsDropdownMenu" class="dropdown-menu">
<!-- Will be populated at runtime -->
<li><a class="dropdown-item" href="#/docs/tutorial.md" data-route="docs">Tutorial</a></li>
<li><a class="dropdown-item" href="#/reference" data-route="reference">Reference</a></li>
<li><hr class="dropdown-divider"></li>
<li>
<hr class="dropdown-divider">
</li>
</ul>
</li>
<!-- IDE support dropdown -->
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" href="#/docs/idea_plugin.md" role="button" data-bs-toggle="dropdown" aria-expanded="false">
<a class="nav-link dropdown-toggle" href="#/docs/idea_plugin.md" role="button"
data-bs-toggle="dropdown" aria-expanded="false">
IDE support
</a>
<ul class="dropdown-menu">
<li>
<a class="dropdown-item" href="#/docs/textmate_bundle.md" data-route="docs">Textmate bundle</a>
<a class="dropdown-item" href="#/docs/textmate_bundle.md" data-route="docs">Textmate
bundle</a>
</li>
<li>
<a class="dropdown-item" href="#/docs/idea_plugin.md" data-route="docs">IDEA plugin</a>
@@ -265,11 +417,13 @@
aria-label="Search"
/>
<!-- Results dropdown (controlled by app code) -->
<div id="topSearchMenu" class="dropdown-menu w-100 shadow" style="max-height:60vh; overflow:auto; top: calc(100% + .25rem); left: 0;">
<div id="topSearchMenu" class="dropdown-menu w-100 shadow"
style="max-height:60vh; overflow:auto; top: calc(100% + .25rem); left: 0;">
</div>
</div>
</form>
<a class="btn btn-sm btn-outline-secondary" href="https://gitea.sergeych.net/SergeychWorks/lyng" target="_blank" rel="noopener" aria-label="GitHub">
<a class="btn btn-sm btn-outline-secondary" href="https://gitea.sergeych.net/SergeychWorks/lyng"
target="_blank" rel="noopener" aria-label="GitHub">
<i class="bi bi-git"></i>
</a>
</div>
@@ -302,6 +456,7 @@
// And expose as a CSS variable used by scroll-padding/scroll-margin rules
// document.documentElement.style.setProperty('--navbar-offset', px);
}
window.addEventListener('load', adjustPadding);
window.addEventListener('resize', adjustPadding);
// Also listen to Bootstrap collapse events which may change navbar height on small screens
@@ -331,9 +486,34 @@
activeLink.setAttribute('aria-current', 'page');
}
}
window.addEventListener('hashchange', updateActive);
window.addEventListener('load', updateActive);
})();
}
)();
// Set up mutation observer for Kotlin code blocks
const observer = new MutationObserver((mutations) => {
mutations.forEach((mutation) => {
mutation.addedNodes.forEach((node) => {
if (node.nodeType === Node.ELEMENT_NODE) {
const kotlinBlocks = node.querySelectorAll('code.language-kotlin');
kotlinBlocks.forEach((block) => {
hljs.highlightElement(block);
});
}
});
});
});
// Start observing document body for changes
observer.observe(document.body, {
childList: true,
subtree: true
});
</script>
</body>
</html>
@@ -0,0 +1,31 @@
import kotlinx.browser.window
import kotlin.test.BeforeTest
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
class SearchHistoryTest {
@BeforeTest
fun resetStorage() {
window.localStorage.removeItem("lyng.search.history")
}
@Test
fun keepsOnlySevenMostRecentUnique() {
// Add 9 queries
for (i in 1..9) rememberSearchQuery("query $i")
val raw = window.localStorage.getItem("lyng.search.history") ?: ""
val list = raw.split('\n').filter { it.isNotBlank() }
assertEquals(7, list.size, "history should contain 7 items")
assertEquals("query 9", list[0], "most recent should be first")
assertEquals("query 3", list.last(), "oldest retained should be 3")
// Re-insert existing should move to front without duplicates
rememberSearchQuery("query 5")
val raw2 = window.localStorage.getItem("lyng.search.history") ?: ""
val list2 = raw2.split('\n').filter { it.isNotBlank() }
assertEquals(7, list2.size)
assertEquals("query 5", list2[0])
assertTrue(list2.count { it == "query 5" } == 1)
}
}
@@ -0,0 +1,29 @@
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
class SearchScoringTest {
private fun rec(text: String, title: String = "Doc") = DocRecord("docs/a.md", title, norm(text))
@Test
fun zeroWhenNoTerms() {
assertEquals(0, scoreQueryAdvanced(emptyList(), rec("hello world")))
}
@Test
fun coverageMatters() {
val r = rec("alpha beta gamma alpha beta")
val s1 = scoreQueryAdvanced(listOf("alp"), r)
val s2 = scoreQueryAdvanced(listOf("alp", "bet"), r)
assertTrue(s2 > s1, "two-term coverage should score higher than one-term")
}
@Test
fun proximityImprovesScore() {
val near = rec("alpha beta gamma")
val far = rec(("alpha "+"x ").repeat(50) + "beta")
val sNear = scoreQueryAdvanced(listOf("alp", "bet"), near)
val sFar = scoreQueryAdvanced(listOf("alp", "bet"), far)
assertTrue(sNear > sFar, "closer terms should have higher score")
}
}