From 45f36587429b9b0e10b90027b198f36937154fec Mon Sep 17 00:00:00 2001 From: sergeych Date: Thu, 19 Feb 2026 01:06:08 +0300 Subject: [PATCH] better docs/AI instructions/readme --- AGENTS.md | 1 + LYNG_AI_SPEC.md | 20 +++++-- README.md | 6 +-- docs/parallelism.md | 17 +++--- docs/scopes_and_closures.md | 102 +++++------------------------------- docs/whats_new_1_5.md | 2 +- notes/ai_state.md | 33 +++--------- 7 files changed, 50 insertions(+), 131 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index fa74f8f..7b8554d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,6 +13,7 @@ - Object members are always allowed even on unknown types; non-Object members require explicit casts. Remove `inspect` from Object and use `toInspectString()` instead. - Type expression checks: `x is T` is value instance check; `T1 is T2` is type-subset; `A in T` means `A` is subset of `T`; `==` is structural type equality. - Type aliases: `type Name = TypeExpr` (generic allowed) expand to their underlying type expressions; no nominal distinctness. +- Bounds and variance: `T: A & B` / `T: A | B` for bounds; declaration-site variance with `out` / `in`. - Do not reintroduce bytecode fallback opcodes (e.g., `GET_NAME`, `EVAL_*`, `CALL_FALLBACK`) or runtime name-resolution fallbacks; all symbol resolution must stay compile-time only. ## Bytecode frame-first migration plan diff --git a/LYNG_AI_SPEC.md b/LYNG_AI_SPEC.md index 947e77c..a9de974 100644 --- a/LYNG_AI_SPEC.md +++ b/LYNG_AI_SPEC.md @@ -4,6 +4,7 @@ High-density specification for LLMs. Reference this for all Lyng code generation ## 1. Core Philosophy & Syntax - **Everything is an Expression**: Blocks, `if`, `when`, `for`, `while`, `do-while` return their last expression (or `void`). +- **Static Types + Inference**: Every declaration has a compile-time type (explicit or inferred). Types are Kotlin‑style: non‑null by default, nullable with `?`. - **Loops with `else`**: `for`, `while`, and `do-while` support an optional `else` block. - `else` executes **only if** the loop finishes normally (without a `break`). - `break ` exits the loop and sets its return value. @@ -13,6 +14,7 @@ High-density specification for LLMs. Reference this for all Lyng code generation 3. Result of the last iteration (if loop finished normally and no `else`). 4. `void` (if loop body never executed and no `else`). - **Implicit Coroutines**: All functions are coroutines. No `async/await`. Use `launch { ... }` (returns `Deferred`) or `flow { ... }`. +- **Functions**: Use `fun` or the short form `fn`. Function declarations are expressions returning a callable. - **Variables**: `val` (read-only), `var` (mutable). Supports late-init `val` in classes (must be assigned in `init` or body). - **Serialization**: Use `@Transient` attribute before `val`/`var` or constructor parameters to exclude them from Lynon/JSON serialization. Transient fields are also ignored during `==` structural equality checks. - **Null Safety**: `?` (nullable type), `?.` (safe access), `?( )` (safe invoke), `?{ }` (safe block invoke), `?[ ]` (safe index), `?:` or `??` (elvis), `?=` (assign-if-null). @@ -44,9 +46,21 @@ High-density specification for LLMs. Reference this for all Lyng code generation - **Root Type**: Everything is an `Object` (root of the hierarchy). - **Nullability**: Non-null by default (`T`), nullable with `T?`, `!!` asserts non-null. - **Untyped params**: `fun foo(x)` -> `x: Object`, `fun foo(x?)` -> `x: Object?`. -- **Untyped vars**: `var x` is `Unset` until first assignment locks the type. -- **Inference**: List/map literals infer union element types; empty list is `List`, empty map is `{:}`. -- **Generics**: Bounds with `T: A & B` or `T: A | B`; variance uses `out`/`in`. +- **Untyped vars**: `var x` is `Unset` until first assignment locks the type (including nullability). + - `val x = null` -> type `Null`; `var x = null` -> type `Object?`. +- **Inference**: + - List literals infer union element types; empty list defaults to `List` unless constrained. + - Map literals infer key/value types; empty map defaults to `Map` unless constrained. + - Mixed numeric ops promote `Int` + `Real` to `Real`. +- **Type aliases**: `type Name = TypeExpr` (generic allowed). Aliases expand to their underlying type expressions (no nominal distinctness). +- **Generics**: Bounds with `T: A & B` or `T: A | B`; variance uses `out`/`in` (declaration‑site only). +- **Casts**: `as` is a runtime-checked cast; `as?` is safe-cast returning `null`. If the value is nullable, `as T` implies `!!`. + +## 2.2 Type Expressions and Checks +- **Value checks**: `x is T` (runtime instance check). +- **Type checks**: `T1 is T2` and `A in T` are subset checks between type expressions (compile-time where possible). +- **Type equality**: `T1 == T2` is structural (unions/intersections are order‑insensitive). +- **Compile-time enforcement**: Bounds are checked at call sites; runtime checks only appear when the compile‑time type is too general. ## 3. Delegation (`by`) Unified model for `val`, `var`, and `fun`. diff --git a/README.md b/README.md index 02c5671..67b0006 100644 --- a/README.md +++ b/README.md @@ -32,9 +32,9 @@ class A { enum E* { One, Two } } val ab = A.B() -assertEquals(ab.x, null) -assertEquals(A.Inner.foo, "bar") -assertEquals(A.One, A.E.One) +assertEquals(null, ab.x) +assertEquals("bar", A.Inner.foo) +assertEquals(A.E.One, A.One) ``` - extremely simple Kotlin integration on any platform (JVM, JS, WasmJS, Lunux, MacOS, iOS, Windows) diff --git a/docs/parallelism.md b/docs/parallelism.md index e3fa336..bb90ca1 100644 --- a/docs/parallelism.md +++ b/docs/parallelism.md @@ -223,17 +223,14 @@ Future work: introduce thread‑safe pooling (e.g., per‑thread pools or confin ### Closures inside coroutine helpers (launch/flow) -Closures executed by `launch { ... }` and `flow { ... }` resolve names using the `ClosureScope` rules: +Closures executed by `launch { ... }` and `flow { ... }` use **compile‑time resolution** just like any other Lyng code: -1. **Current frame locals and arguments**: Variables defined within the current closure execution. -2. **Captured lexical ancestry**: Outer local variables captured at the site where the closure was defined (the "lexical environment"). -3. **Captured receiver members**: If the closure was defined within a class or explicitly bound to an object, it checks members of that object (`this`), following MRO and respecting visibility. -4. **Caller environment**: Falls back to the calling context (e.g., the caller's `this` or local variables). -5. **Global/Module fallbacks**: Final check for module-level constants and global functions. +- **Captured locals are slots**: outer locals are resolved at compile time and captured as frame‑slot references, so they remain visible across suspension points. +- **Members are statically resolved**: member access requires a statically known receiver type or an explicit cast (except `Object` members). +- **No runtime fallbacks**: there is no dynamic name lookup or “search parent scopes” at runtime for missing symbols. Implications: -- Outer locals (e.g., `counter`) stay visible across suspension points. -- Global helpers like `delay(ms)` and `yield()` are available from inside closures. -- If you write your own async helpers, execute user lambdas under `ClosureScope(callScope, capturedCreatorScope)` and avoid manual ancestry walking. +- Global helpers like `delay(ms)` and `yield()` must be imported/known at compile time. +- If you need dynamic access, use explicit helpers (e.g., `dynamic { ... }`) rather than relying on scope resolution. -See also: [Scopes and Closures: resolution and safety](scopes_and_closures.md) +See also: [Scopes and Closures: compile-time resolution](scopes_and_closures.md) diff --git a/docs/scopes_and_closures.md b/docs/scopes_and_closures.md index 173ba1a..7bd77a9 100644 --- a/docs/scopes_and_closures.md +++ b/docs/scopes_and_closures.md @@ -1,94 +1,18 @@ -# Scopes and Closures: resolution and safety +# Scopes and Closures: compile-time resolution -Attention to AI: name lookup is ibsolete and must not be used with bytecode compiler +Attention to AI: name lookup in runtime `Scope` is legacy. The bytecode compiler uses **compile-time name/member resolution only**. -This page documents how name resolution works with `ClosureScope`, how to avoid recursion pitfalls, and how to safely capture and execute callbacks that need access to outer locals. +This page documents the **current** rules: static name resolution, closure captures, and the limited role of runtime `Scope` in Kotlin interop and explicit dynamic helpers. -## Why this matters -Name lookup across nested scopes and closures can accidentally form recursive resolution paths or hide expected symbols (outer locals, module/global functions). The rules below ensure predictable resolution and prevent infinite recursion. +## Current rules (bytecode compiler) +- **All names resolve at compile time**: locals, parameters, captures, members, imports, and module globals must be known when compiling. Missing symbols are compile-time errors. +- **No runtime fallbacks**: there is no dynamic name lookup, no fallback opcodes, and no “search parent scopes” at runtime for missing names. +- **Object members on unknown types only**: `toString`, `toInspectString`, `let`, `also`, `apply`, `run` are allowed on unknown types; all other members require a statically known receiver type or an explicit cast. +- **Closures capture slots**: lambdas and nested functions capture **frame slots** directly. Captures are resolved at compile time and compiled to slot references. +- **Scope is a reflection facade**: `Scope` is used only for Kotlin interop or explicit dynamic helpers. It must **not** be used for general symbol resolution in compiled Lyng code. -## Resolution order in ClosureScope -When evaluating an identifier `name` inside a closure, `ClosureScope.get(name)` resolves in this order: +## Explicit dynamic access (opt-in only) +Dynamic name access is available only via explicit helpers (e.g., `dynamic { get { name -> ... } }`). It is **not** a fallback for normal member or variable access. -1. **Current frame locals and arguments**: Variables defined within the current closure execution. -2. **Captured lexical ancestry**: Outer local variables captured at the site where the closure was defined (the "lexical environment"). -3. **Captured receiver members**: If the closure was defined within a class or explicitly bound to an object, it checks members of that object (`this`). This includes both instance fields/methods and class-level static members, following the MRO (C3) and respecting visibility rules (private members are only visible if the closure was defined in their class). -4. **Caller environment**: If not found lexically, it falls back to the calling context (e.g., the DSL's `this` or the caller's local variables). -5. **Global/Module fallbacks**: Final check for module-level constants and global functions. - -This ensures that closures primarily interact with their defining environment (lexical capture) while still being able to participate in DSL-style calling contexts. - -## Use raw‑chain helpers for ancestry walks -When authoring new scope types or advanced lookups, avoid calling virtual `get` while walking parents. Instead, use the non‑dispatch helpers on `Scope`: - -- `chainLookupIgnoreClosure(name)` - - Walk raw `parent` chain and check only per‑frame locals/bindings/slots. - - Ignores overridden `get` (e.g., in `ClosureScope`). Cycle‑safe. -- `chainLookupWithMembers(name)` - - Like above, but after locals/bindings it also checks each frame’s `thisObj` members. - - Ignores overridden `get`. Cycle‑safe. -- `baseGetIgnoreClosure(name)` - - For the current frame only: check locals/bindings, then walk raw parents (locals/bindings), then fallback to this frame’s `thisObj` members. - -These helpers avoid ping‑pong recursion and make structural cycles harmless (lookups terminate). - -## Preventing structural cycles -- Don’t construct parent chains that can point back to a descendant. -- A debug‑time guard throws if assigning a parent would create a cycle; keep it enabled for development builds. -- Even with a cycle, chain helpers break out via a small `visited` set keyed by `frameId`. - -## Capturing lexical environments for callbacks -For dynamic objects or custom builders, capture the creator’s lexical scope so callbacks can see outer locals/parameters: - -1. Use `snapshotForClosure()` on the caller scope to capture locals/bindings/slots and parent. -2. Store this snapshot and run callbacks under `ClosureScope(callScope, captured)`. - -Kotlin sketch: -```kotlin -val captured = scope.snapshotForClosure() -val execScope = ClosureScope(currentCallScope, captured) -callback.execute(execScope) -``` - -This ensures expressions like `contractName` used inside dynamic `get { name -> ... }` resolve to outer variables defined at the creation site. - -## Closures in coroutines (launch/flow) -- The closure frame still prioritizes its own locals/args. -- Outer locals declared before suspension points remain visible through slot‑aware ancestry lookups. -- Global functions like `delay(ms)` and `yield()` are resolved via module/root fallbacks from within closures. - -Tip: If a closure unexpectedly cannot see an outer local, check whether an intermediate runtime helper introduced an extra call frame; the built‑in lookup already traverses caller ancestry, so prefer the standard helpers rather than custom dispatch. - -## Local variable references and missing symbols -- Unqualified identifier resolution first prefers locals/bindings/slots before falling back to `this` members. -- If neither locals nor members contain the symbol, missing field lookups map to `SymbolNotFound` (compatibility alias for `SymbolNotDefinedException`). - -## Performance notes -- The `visited` sets used for cycle detection are tiny and short‑lived; in typical scripts the overhead is negligible. -- If profiling shows hotspots, consider limiting ancestry depth in your custom helpers or using small fixed arrays instead of hash sets—only for extremely hot code paths. - -## Practical Example: `cached` - -The `cached` function (defined in `lyng.stdlib`) is a classic example of using closures to maintain state. It wraps a builder into a zero-argument function that computes once and remembers the result: - -```lyng -fun cached(builder) { - var calculated = false - var value = null - { // This lambda captures `calculated`, `value`, and `builder` - if( !calculated ) { - value = builder() - calculated = true - } - value - } -} -``` - -Because Lyng now correctly isolates closures for each evaluation of a lambda literal, using `cached` inside a class instance works as expected: each instance maintains its own private `calculated` and `value` state, even if they share the same property declaration. - -## Dos and Don’ts -- Do use `chainLookupIgnoreClosure` / `chainLookupWithMembers` for ancestry traversals. -- Do maintain the resolution order above for predictable behavior. -- Don’t call virtual `get` while walking parents; it risks recursion across scope types. -- Don’t attach instance scopes to transient/pool frames; bind to a stable parent scope instead. +## Legacy interpreter behavior (reference only) +The old runtime `Scope`-based resolution order (locals → captured → `this` → caller → globals) is obsolete for bytecode compilation. Keep it only for legacy interpreter paths and tooling that explicitly opts into it. diff --git a/docs/whats_new_1_5.md b/docs/whats_new_1_5.md index 4edfdf7..0e7abb1 100644 --- a/docs/whats_new_1_5.md +++ b/docs/whats_new_1_5.md @@ -22,7 +22,7 @@ The API is fixed and will be kept with further Lyng core changes. It is now the - **Deep inference**: The compiler analyzes types of symbols along the execution path and in many cases eliminates unnecessary casts or type specifications. - **Union and intersection types**: `A & B`, `A | B`. -- **Generics**: Generic types are first-class citizens with support for [bounds and variance](generics.md). No type erasure: in a generic function you can, for example, check `A in T`, where T is the generic type. +- **Generics**: Generic types are first-class citizens with support for [bounds and variance](generics.md). Type params are erased by default and are reified only when needed (e.g., `T::class`, `T is ...`, `as T`, or in extern-facing APIs), which enables checks like `A in T` when `T` is reified. - **Inner classes and enums**: Full support for nested declarations, including [Enums with lifting](OOP.md#lifted-enum-entries). ## Other highlights diff --git a/notes/ai_state.md b/notes/ai_state.md index 2b7996d..90a0048 100644 --- a/notes/ai_state.md +++ b/notes/ai_state.md @@ -1,37 +1,20 @@ AI State (for session restart) -Project: /home/sergeych/dev/ling_lib +Project: /home/sergeych/dev/lyng Module focus: :lynglib Current focus - Enforce compile-time name/member resolution only; no runtime scope lookup or fallback. -- Bytecode uses memberId-based ops (CALL_MEMBER_SLOT/GET_MEMBER_SLOT/SET_MEMBER_SLOT). -- Runtime lookup opcodes (CALL_VIRTUAL/GET_FIELD/SET_FIELD) and fallback callsites are removed. -- Use FrameSlotRef for captures and only materialize Scope for Kotlin interop; use frame.ip -> pos mapping for diagnostics. +- Closures capture frame slots directly; materialize `Scope` only for Kotlin interop or explicit dynamic helpers. +- Object members are allowed on unknown types; other members require a statically known receiver type or explicit cast. +- Type system is Kotlin-style: `T` non-null, `T?` nullable, `!!` asserts non-null; `void` is a singleton of class `Void`. +- Type expressions: unions/intersections with bounds, declaration-site variance (`in`/`out`), and structural equality. Key recent changes -- Removed method callsite PICs and fallback opcodes; bytecode now relies on compile-time member ids only. -- Operator dispatch emits memberId calls when known; falls back to Obj opcodes for allowed built-ins without name lookup. -- Object members are allowed on unknown types; other members still require a statically known receiver type. -- Added frame.ip -> pos mapping; call-site ops restore pos after args to keep stack traces accurate. -- Loop var overrides now take precedence in slot resolution to keep loop locals in frame slots. -- LocalSlotRef now falls back to name lookup when slot plans are missing (closure safety). +- Updated AI helper docs to reflect static typing, type expressions, and compile-time-only name resolution. Known failing tests -- None (jvmTest passing). - -Files touched recently -- notes/type_system_spec.md (spec updated) -- AGENTS.md (type inference reminders) -- lynglib/src/commonMain/kotlin/net/sergeych/lyng/Compiler.kt -- lynglib/src/commonMain/kotlin/net/sergeych/lyng/bytecode/BytecodeCompiler.kt -- lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjRef.kt +- Not checked in this session. Last test run -- ./gradlew :lynglib:jvmTest - -Spec decisions (notes/type_system_spec.md) -- Nullability: Kotlin-style, T non-null, T? nullable, !! asserts non-null. -- void is singleton of class Void (syntax sugar). -- Untyped params default to Object (non-null); syntax sugar: fun foo(x?) and class X(a,b?). -- Object member access requires explicit cast; remove inspect, use toInspectString(). +- Not checked in this session.