Migrate runtime locals to frame slots

notes/ai_state.md

@@ -7,7 +7,6 @@ Current focus
 - Enforce compile-time name/member resolution only; no runtime scope lookup or fallback.
 - Bytecode uses memberId-based ops (CALL_MEMBER_SLOT/GET_MEMBER_SLOT/SET_MEMBER_SLOT).
 - Runtime lookup opcodes (CALL_VIRTUAL/GET_FIELD/SET_FIELD) and fallback callsites are removed.
-- Migrate bytecode to frame-first locals with lazy scope reflection; avoid eager scope slot plans in compiled functions.
 - Use FrameSlotRef for captures and only materialize Scope for Kotlin interop; use frame.ip -> pos mapping for diagnostics.
 
 Key recent changes

notes/archive/bytecode_migration_plan.md

@@ -0,0 +1,175 @@
+# Bytecode Migration Plan
+
+Goal: migrate the compiler so all values live in frames/bytecode, keeping JVM tests green after each step.
+
+## Steps
+
+- [x] Step 1: Module/import slots seeded into module frame; bytecode module resolution works across closures.
+- [x] Step 2: Allow implicit/qualified `this` member refs to compile to bytecode.
+  - [x] Enable bytecode for `ImplicitThisMethodCallRef`, `QualifiedThisMethodSlotCallRef`, `QualifiedThisFieldSlotRef`.
+  - [x] Keep unsupported cases blocked: `ClassScopeMemberRef`, dynamic receivers, delegated members.
+  - [x] JVM tests must be green before commit.
+- [x] Step 3: Bytecode support for `try/catch/finally`.
+  - [x] Implement bytecode emission for try/catch and finally blocks.
+  - [x] Preserve existing error/stack semantics.
+  - [x] JVM tests must be green before commit.
+
+## Remaining Migration (prioritized)
+
+- [x] Step 4: Allow bytecode wrapping for supported declaration statements.
+  - [x] Enable `DestructuringVarDeclStatement` and `ExtensionPropertyDeclStatement` in `containsUnsupportedForBytecode`.
+  - [x] Keep JVM tests green before commit.
+- [x] Step 5: Enable bytecode for delegated var declarations.
+  - [x] Revisit `containsDelegatedRefs` guard for `DelegatedVarDeclStatement`.
+  - [x] Ensure delegate binding uses explicit `Statement` objects (no inline suspend lambdas).
+  - [x] Keep JVM tests green before commit.
+- [x] Step 6: Map literal spread in bytecode.
+  - [x] Replace `MapLiteralEntry.Spread` bytecode exception with runtime `putAll`/merge logic.
+- [x] Step 7: Class-scope member refs in bytecode.
+  - [x] Support `ClassScopeMemberRef` without scope-map fallback.
+- [x] Step 8: ObjDynamic member access in bytecode.
+  - [x] Allow dynamic receiver field/method lookup without falling back to interpreter.
+- [x] Step 9: Module-level bytecode execution.
+  - [x] Compile `Script` bodies to bytecode instead of interpreting at module scope.
+  - [x] Keep import/module slot seeding in frame-only flow.
+- [x] Step 10: Bytecode for declaration statements in module scripts.
+  - [x] Support `ClassDeclStatement`, `FunctionDeclStatement`, `EnumDeclStatement` in bytecode compilation.
+  - [x] Keep a mixed execution path for declarations (module bytecode calls statement bodies via `CALL_SLOT`).
+  - [x] Ensure module object member refs compile as instance access (not class-scope).
+- [x] Step 11: Destructuring assignment bytecode.
+  - [x] Handle `[a, b] = expr` (AssignRef target `ListLiteralRef`) without interpreter fallback.
+- [x] Step 12: Optional member assign-ops and inc/dec in bytecode.
+  - [x] Support `a?.b += 1` and `a?.b++` for `FieldRef` targets.
+  - [x] Fix post-inc return value for object slots stored in scope frames.
+  - [x] Handle optional receivers for member assign-ops and inc/dec without evaluating operands on null.
+  - [x] Support class-scope and index optional inc/dec paths in bytecode.
+- [x] Step 13: Qualified `this` value refs in bytecode.
+  - [x] Compile `QualifiedThisRef` (`this@Type`) via `LOAD_THIS_VARIANT`.
+  - [x] Add a JVM test that evaluates `this@Type` as a value inside nested classes.
+- [x] Step 14: Fast local ref reads in bytecode.
+  - [x] Support `FastLocalVarRef` reads with the same slot resolution as `LocalVarRef`.
+  - [x] If `BoundLocalVarRef` is still emitted, map it to a direct slot read instead of failing.
+  - [x] Add a JVM test that exercises fast-local reads in a bytecode-compiled function.
+- [x] Step 15: Class-scope `?=` in bytecode.
+  - [x] Handle `C.x ?= v` and `C?.x ?= v` for class-scope members without falling back.
+  - [x] Add a JVM test for class-scope `?=` on static vars.
+- [x] Step 16: Remove dead `ToBoolStatement`.
+  - [x] Confirm no parser/compiler paths construct `ToBoolStatement` and delete it plus interpreter hooks.
+  - [x] Keep JVM tests green after removal.
+- [x] Step 17: Callable property calls in bytecode.
+  - [x] Support `CallRef` where the target is a `FieldRef` (e.g., `(obj.fn)()`), keeping compile-time resolution.
+  - [x] Add a JVM test for a callable property call compiled to bytecode.
+- [x] Step 18: Delegated member access in bytecode.
+  - [x] Remove `containsDelegatedRefs` guard once bytecode emits delegated get/set/call correctly.
+  - [x] Add JVM coverage for delegated member get/set/call in bytecode.
+- [x] Step 19: Unknown receiver member access in bytecode.
+  - [x] Reject Object/unknown receiver member calls without explicit cast or Dynamic.
+  - [x] Add union-member dispatch with ordered type checks and runtime mismatch error.
+  - [x] Add JVM tests for unknown receiver and union member access.
+- [x] Step 20: Bytecode support for `NopStatement`.
+  - [x] Allow `NopStatement` in `containsUnsupportedForBytecode`.
+  - [x] Emit `ObjVoid` directly in bytecode for `NopStatement` in statement/value contexts.
+  - [x] Add a JVM test that exercises a code path returning `NopStatement` in bytecode (e.g., static class member decl in class body).
+- [x] Step 21: Union mismatch path in bytecode.
+  - [x] Replace `UnionTypeMismatchStatement` branch with a bytecode-compilable throw path (no custom `StatementRef` that blocks bytecode).
+  - [x] Add a JVM test that forces the union mismatch at runtime and asserts the error message.
+- [x] Step 22: Delegated local slots in bytecode.
+  - [x] Support reads/writes/assign-ops/inc/dec for delegated locals (`LocalSlotRef.isDelegated`) in `BytecodeCompiler`.
+  - [x] Remove `containsDelegatedRefs` guard once delegated locals are bytecode-safe.
+  - [x] Add JVM tests that use delegated locals inside bytecode-compiled functions.
+- [x] Step 23: Refactor delegated locals to keep delegate objects in frame slots.
+  - [x] Add bytecode ops to bind/get/set delegated locals without scope storage.
+  - [x] Store delegated locals in frame slots and compile get/set/assign ops with new ops.
+  - [x] Preserve reflection facade by syncing delegated locals into scope only when needed.
+- [x] Step 24: Remove `ASSIGN_SCOPE_SLOT` now that delegated locals are always frame-backed.
+  - [x] Force delegated locals into local slots (even module) and avoid scope-slot resolution.
+  - [x] Drop opcode/runtime support for `ASSIGN_SCOPE_SLOT`.
+
+## Frame-Only Execution (new, before interpreter removal)
+
+- [x] Step 24A: Bytecode capture tables for lambdas (compiler only).
+  - [x] Emit per-lambda capture tables containing (ownerFrameKind, ownerSlotId).
+  - [x] Create captures only when detected; do not allocate scope slots.
+  - [x] Add disassembler output for capture tables.
+  - [x] JVM tests must be green before commit.
+- [x] Step 24B: Frame-slot captures in bytecode runtime.
+  - [x] Build lambdas from bytecode + capture table (no capture names).
+  - [x] Read captured values via `FrameSlotRef` only.
+  - [x] Forbid `resolveCaptureRecord` in bytecode paths; keep only in interpreter.
+  - [x] JVM tests must be green before commit.
+- [x] Step 24C: Remove scope local mirroring in bytecode execution.
+  - [x] Remove/disable any bytecode runtime code that writes locals into Scope for execution.
+  - [x] Keep Scope creation only for reflection/Kotlin interop paths.
+  - [x] JVM tests must be green before commit.
+- [x] Step 24D: Eliminate `ClosureScope` usage on bytecode execution paths.
+  - [x] Avoid `ClosureScope` in bytecode-related call paths (Block/Lambda/ObjDynamic/ObjProperty).
+  - [x] Keep interpreter path using `ClosureScope` until interpreter removal.
+  - [x] JVM tests must be green before commit.
+- [x] Step 24E: Isolate interpreter-only capture logic.
+  - [x] Mark `resolveCaptureRecord` paths as interpreter-only.
+  - [x] Guard or delete any bytecode path that tries to sync captures into scopes.
+  - [x] JVM tests must be green before commit.
+
+## Interpreter Removal (next)
+
+- [ ] Step 25: Replace Statement-based declaration calls in bytecode.
+  - [x] Add bytecode const/op for enum declarations (no `Statement` objects in constants).
+  - [x] Add bytecode const/op for class declarations (no `Statement` objects in constants).
+  - [x] Add bytecode const/op for function declarations (no `Statement` objects in constants).
+  - [x] Replace `emitStatementCall` usage for `EnumDeclStatement`.
+  - [x] Replace `emitStatementCall` usage for `ClassDeclStatement`.
+  - [x] Replace `emitStatementCall` usage for `FunctionDeclStatement`.
+  - [x] Add JVM disasm coverage to ensure module init has no `CALL_SLOT` to `Callable@...` for declarations.
+- [ ] Step 26: Bytecode-backed lambdas (remove `ValueFnRef` runtime execution).
+  - [x] Compile lambda bodies to bytecode and emit an opcode to create a callable from bytecode + capture plan.
+  - [x] Remove `containsValueFnRef` helper now that lambdas are bytecode-backed.
+  - [x] Remove `forceScopeSlots` branches once no bytecode paths depend on scope slots.
+  - [x] Add JVM tests for captured locals and delegated locals inside lambdas on the bytecode path.
+- [x] Step 27: Remove interpreter opcodes and constants from bytecode runtime.
+  - [x] Delete `BytecodeConst.ValueFn`, `CmdMakeValueFn`, and `MAKE_VALUE_FN`.
+  - [x] Delete `BytecodeConst.StatementVal`, `CmdEvalStmt`, and `EVAL_STMT`.
+  - [x] Add bytecode-backed `::class` via `ClassOperatorRef` + `GET_OBJ_CLASS` to avoid ValueFn for class operator.
+  - [x] Add a bytecode fallback reporter hook for lambdas to locate remaining non-bytecode cases.
+  - [x] Remove `emitStatementCall`/`emitStatementEval` once unused.
+- [ ] Step 28: Scope as facade only.
+  - [x] Audit bytecode execution paths for `Statement.execute` usage and remove remaining calls.
+  - [x] Keep scope sync only for reflection/Kotlin interop, not for execution.
+  - [x] Replace bytecode entry seeding from Scope with frame-only arg/local binding.
+
+## Interpreter Removal v2 (2026-02-12)
+
+Goal: remove interpreter execution paths entirely without regressing semantics; all runtime execution must be bytecode + frame slots.
+
+- [ ] Step A1: Interpreter path audit (map all remaining runtime `Statement.execute` and scope-lookup paths).
+  - [x] Audit results (current hotspots):
+    - `ClassDeclStatement.executeClassDecl`: executes class body/init via `spec.bodyInit?.execute` and `spec.initScope` (Statement list).
+    - `FunctionDeclStatement.executeFunctionDecl`: class-body delegated function init uses `Statement.execute` in initializer thunk.
+    - `EnumDeclStatement.execute`: direct execution path exists (bytecode also emits DECL_ENUM).
+    - `BlockStatement.execute`: creates child scope, applies slot plan/captures, then executes `Script` (which may interpret).
+    - `Script.execute`: interpreter loop when `moduleBytecode` is null or disabled.
+    - `ObjClass.addFn/addProperty` wrappers use `ObjNativeCallable` calling into `Statement.execute` for declared bodies.
+    - Object expressions: class body executed via `executeClassDecl` path (same as above).
+    - Extension wrappers: `ObjExtensionMethodCallable` uses callable that executes `Statement`.
+
+- [ ] Step A2: Bytecode-backed class + init.
+  - Replace class-body and instance init execution with bytecode functions (per-class + per-instance).
+  - Remove all class init `Statement.execute` calls.
+  - [x] Introduce `ClassStaticFieldInitStatement` + bytecode ops `DECL_CLASS_FIELD`/`DECL_CLASS_DELEGATED` for static class field init.
+
+- [ ] Step A3: Bytecode-safe delegated properties/functions and object expressions.
+  - Use explicit `object : Statement()` where needed.
+  - No inline suspend lambdas in hot paths.
+  - Remove interpreter fallbacks.
+
+- [ ] Step A4: Bytecode for all blocks/lambdas (including class bodies).
+  - Compile non-module blocks/lambdas to bytecode; eliminate interpreter gate flags.
+
+- [ ] Step A5: Delete interpreter execution path and dead code.
+  - Remove interpreter ops/constants and any runtime name-lookup fallbacks.
+  - Full test suite green.
+
+## Notes
+
+- Keep imports bound to module frame slots; no scope map writes for imports.
+- Avoid inline suspend lambdas in compiler hot paths; use explicit `object : Statement()`.
+- Do not reintroduce bytecode fallback opcodes; all symbol resolution remains compile-time only.

notes/archive/bytecode_migration_plan_completed.md

@@ -0,0 +1,12 @@
+# Bytecode Frame-First Migration Plan (Completed)
+
+Status: completed (archived for historical reference)
+
+Original goals
+- Migrate bytecode to frame-first locals with lazy scope reflection; avoid eager scope slot plans in compiled functions.
+- Use FrameSlotRef for captures and only materialize Scope for Kotlin interop.
+- Avoid PUSH_SCOPE/POP_SCOPE in bytecode for loops/functions unless dynamic name access or Kotlin reflection is requested.
+
+Notes
+- This plan is now considered done at the architectural level; see current work items in other notes.
+- Scope slots are still present for interop in the current codebase; removal is tracked separately.

notes/plan_scope_slots_elimination.md

@@ -0,0 +1,94 @@
+# Plan: Eliminate Scope Slots for Module + Extensions (Frame-Backed Only)
+
+Goal (1 + 2)
+- Module-level vars must be frame-backed just like locals; no special scope storage.
+- Extension wrappers must use frame slots as well; no scope-slot storage.
+- Scope remains only as a facade for Kotlin extern symbols (reflection/interop), not as primary storage.
+
+Non-Goals (for now)
+- Full reflection redesign (set aside).
+- Removing scope slots entirely (interop/reflection may still need a facade).
+
+Current Findings (high level)
+- BytecodeCompiler still builds scope slots for module symbols and for extension wrappers via pendingScopeNameRefs.
+- CmdRuntime reads/writes scope slots whenever slot < scopeSlotCount.
+- RESOLVE_SCOPE_SLOT + addr slots are emitted for moving between scope and frame slots.
+
+Design Direction
+- Frame slots are the sole storage for module vars and extension wrappers.
+- Scope slots are reserved for explicit extern/interop names only (not part of this change).
+- Module symbols are captured like any other (closure capture via FrameSlotRef), no module slot plan or scope slot mapping.
+- Persist the module frame inside ModuleScope so its lifetime matches the module.
+
+Key Decisions
+- Module scope is just another frame in the parent chain; do not seed or persist module vars through scope slots.
+- Extension wrappers are treated like ordinary locals/globals in frame slots; no pending scope name refs.
+- Compatibility: when `seedScope` is not a `ModuleScope`, module-level declarations still map to scope slots so `Scope.eval(...)` retains cross-call variables.
+
+Work Plan
+
+Phase A: Compiler changes (scope slot creation)
+1) Stop adding module declarations to scope slots
+   - In `BytecodeCompiler.collectScopeSlots`, when `moduleScopeId` matches a declaration, treat it as a local slot, not a scope slot.
+   - Remove `isModuleDecl` special-casing that pushes into `scopeSlotMap`.
+
+2) Remove synthetic scope slots for extension wrappers
+   - Gate `pendingScopeNameRefs` (extension callable/getter/setter queueing) behind explicit scope-slot usage.
+   - Ensure extension wrappers are resolved via local slot indices only (frame-backed).
+
+3) Tighten scope-slot eligibility
+   - `isModuleSlot(...)` should no longer route module names to scope slots.
+   - Only names in `allowedScopeNames` / `scopeSlotNameSet` (extern/interop) should become scope slots.
+
+4) ResolveSlot prefers frame slots for everything
+   - Update `resolveSlot` to pick local slots for module/extension names.
+   - Only return scopeSlotMap entries for explicit extern/interop names.
+
+Phase B: Runtime changes (module frame)
+1) Persist the module frame inside ModuleScope
+   - Add `ModuleScope.ensureModuleFrame(fn)` that creates/reuses a `BytecodeFrame` with `fn.localCount`.
+   - Store local slot metadata on ModuleScope for future interop (names/mutables/delegated).
+
+2) Use the module frame when executing the module bytecode
+   - `CmdFrame` should reuse the module frame when `scope0 is ModuleScope` and args are empty.
+   - Avoid clobbering module locals with args.
+
+3) Remove module seed path that depends on scope slots
+   - Remove `moduleSlotPlan` application in `Script.execute`.
+   - Keep `seedImportBindings` and `seedModuleLocals`, but write into the module frame (scopeSlotCount should be 0).
+
+Phase C: Bytecode + ABI considerations
+1) No new opcodes required for (1) + (2)
+   - If we later need persistence across module reloads, introduce explicit sync opcodes (requires approval).
+
+2) Migration safety
+   - Keep existing scope-slot ops for interop only; do not delete opcodes yet.
+
+Detailed Change List (by file)
+- `lynglib/src/commonMain/kotlin/net/sergeych/lyng/bytecode/BytecodeCompiler.kt`
+  - `collectScopeSlots`: treat module declarations as locals; do not map into `scopeSlotMap`.
+  - Remove/guard `pendingScopeNameRefs` queuing for extension wrappers.
+  - `isModuleSlot`: stop routing by moduleScopeId; only allow for explicit interop names.
+  - `resolveSlot`: prefer local slots; avoid scope slot fallback for module/extension names.
+
+- `lynglib/src/commonMain/kotlin/net/sergeych/lyng/Script.kt`
+  - Remove module slot plan application that assumes scope slots.
+  - Ensure module vars survive via frame capture rather than scope seeding.
+
+- `lynglib/src/commonMain/kotlin/net/sergeych/lyng/ModuleScope.kt`
+  - Add module frame storage + metadata and `ensureModuleFrame(fn)`.
+
+- `lynglib/src/commonMain/kotlin/net/sergeych/lyng/bytecode/CmdRuntime.kt`
+  - Use module frame when `scope0 is ModuleScope` in `CmdFrame`.
+
+Testing / Validation
+- Disassemble compiled modules: verify no `RESOLVE_SCOPE_SLOT` for module vars or extension wrappers.
+- Run `:lynglib:jvmTest` and at least one module script execution that references module vars across calls.
+- Verify extension call wrappers still resolve and invoke correctly without scope-slot backing.
+
+Risks / Open Questions
+- Module persistence: if module vars were previously “persisted” via scope slots, confirm frame-backed storage matches desired lifetime.
+- Extension wrapper lookup: ensure name resolution does not rely on scope slot indexing anywhere else.
+
+Approval Gates
+- Any new opcode or runtime sync step must be proposed separately and approved before implementation.